The Windows->Linux POSIX port left the recompiler and runtime almost
portable; two small platform gaps remained on macOS (AppleClang, arm64),
both guarded so Windows/Linux are unaffected:
- runtime/src/autocompile.c: add #include <stdlib.h> for getenv. glibc and
windows.h leak that declaration transitively; the macOS SDK does not, so
AppleClang errored on the implicit declaration.
- recompiler/src/main_bios.cpp: the C-compiler probe used ">NUL" (Windows
null device). On POSIX that misdirects the redirect and drops a stray
"NUL" file in the cwd; now #ifdef'd to /dev/null.
The ucontext fiber backend already carried the _XOPEN_SOURCE/_DARWIN_C_SOURCE
guards it needs; SDL2 (Homebrew) selects the Metal backend automatically.
Verified end to end on macOS 15.5 / Apple Silicon: BIOS -> KULA WORLD title
-> menu -> 1 Player -> a 3D level that rolls under control.
Adds docs/MACOS_BUILD_KULA.md and a macOS gameplay screenshot; README notes
the port. Ignores the local dist/ .app bundle (recompiled-BIOS derivative).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Step-by-step instructions for an AI agent to reproduce the working Kula
World recomp on Windows: prerequisites (MSYS2 mingw64 + SDL2), clone the
branch, user-supplied BIOS/disc, extract the boot EXE, build the
recompiler, generate BIOS + game C, build the kula-runtime target, run,
controls, and an honest verified/unverified status list.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdjxoRB1f9VQsNzZayCgGS
Adds this fork's Kula World (SCES-01000) end-to-end target to the README with
a title-screen screenshot and a summary of the five runtime bugs fixed during
bring-up (four of them game-agnostic framework improvements): the dirty-RAM
interpreter interrupt-latency fix, the main-RAM mirror, GetID region
derivation, CD status bits, and the SCPH-1001 CD controller version.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdjxoRB1f9VQsNzZayCgGS
ROOT CAUSE of the Kula World black-screen wedge: a guest wait loop (libcd's
CdSync spin) that lives entirely in dirty RAM re-enters the dirty-RAM
interpreter once per short (~7-insn) block. The interpreter's only interrupt
poll was gated by the PER-INVOCATION (insns_executed & 0xFFF) counter, which
short blocks never reach — and when the loop exits to dispatch each iteration
that path isn't reached at all. So the interpreter ran millions of
instructions with psx_check_interrupts never called, while SR (IEc+IM2) and
Cause (IP2) fully satisfied the MIPS interrupt-take condition and i_stat had
CDROM+VBLANK pending. The CD IRQ that sets libcd's wait-flag was never
serviced; the loop spun ~14s per cycle until a rare timeout re-dispatched.
Fix: poll interrupts at the interpreter's per-invocation entry, gated by a
global invocation counter (every 64 entries), cpu->pc set for a coherent
resume — the interpreter analogue of a static block-leader poll.
RESULT: Kula World now boots to its title screen and menu. Diagnosed with the
self-stop trap + gdb sampling of SR/Cause/i_stat + the interrupt gate globals
(in_exception, total_checks) + g_dirty_ram_insns_run, which isolated 'CPU
spinning in the interpreter without polling interrupts'. General fix: any
dirty-RAM spin-loop now yields to interrupts.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdjxoRB1f9VQsNzZayCgGS
With the boot fixed, live state during the game's retry loop shows the CDROM
interrupt is raised (i_stat bit2) AND enabled (i_mask bit2) and the CPU takes
exceptions (VBLANK cleared each frame), yet the CDROM bit is never cleared and
the response FIFO never drained (irq_flag=0x03, response_read=0). The game's
libcd CD interrupt handler in the BIOS IntRP chain is not servicing the CD
IRQ, so CdSync never completes. Distinct from the CD-response-value bugs fixed
today; a BIOS-IRQ-chain / libcd-handler matter. Documents the next step and
the full session tally in docs/kula-world-findings.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdjxoRB1f9VQsNzZayCgGS
The shell CD-init (ROM 0x1DF50) sets kernel flag [0xA000DFFC]=1 when the CD
controller version high byte (Test sub-cmd 0x20) is >= 0x95; the boot CD-open
(0xBFC0D570) then issues a ReadTOC gated on that flag. We returned the
PSone-era 0x97 0x01 0x10 0xC2 (copied from beetle, which hardcodes it) — wrong
for the SCPH-1001 BIOS, whose 1994 sub-CPU reports 0x94 0x09 0x19 0xC0. With
the correct value the flag stays clear, the spurious frame-890 ReadTOC is
gone, and the boot command stream matches the oracle exactly. Found via
self-stop trap + fntrace-all + gdb watchpoint on [0xDFFC] + capstone tracing
the flag setter and its version check.
Also adds a symmetric self-stop trap to psx-beetle's CD-command hook so the
oracle can be frozen at the same boot point for state diffs.
Honest status: this closes the boot-flow divergence but the game still wedges
downstream in its OWN CD-init retry (GetStat/Init loop from game pc 0x63A14);
that is a separate issue, documented in docs/kula-world-findings.md round 6.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdjxoRB1f9VQsNzZayCgGS
Adds PSX_FNTRACE_ALL=1 (records every dispatch from power-on) and documents
the full mechanistic chain of the wedge (docs/kula-world-findings.md round 5):
The frame-890 ReadTOC that breaks the game's CD reads is gated by kernel flag
[0xA000DFFC]: the CD-open function 0xBFC0D570 does 'beqz [0xDFFC], skip;
jal ReadTOC'. On ours [0xDFFC]=1 -> ReadTOC -> wedge; on the oracle it stays
0 -> reads continue. The flag is set to 1 by func_1FC1DF50 (a CD control
routine) called from shell code at ra=0x80036024. The shells run the same
verbatim bytes but diverge early (ours 0x35000-0x39000 issuing Test/ReadTOC,
oracle 0x30D00-0x33200 issuing Init/GetTN), so a branch input differs. Traced
via self-stop trap + fntrace-all + gdb watchpoint on [0xDFFC] + capstone.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdjxoRB1f9VQsNzZayCgGS
Extends psxrecomp_cdcmd_cb_t (and the beetle_cdcmd_trace ring + cdrom_cmd_dump
JSON) with the guest PC at dispatch, read via PS_CPU::GetRegister(GSREG_PC).
Lets the boot-command-stream oracle-diff localize the first divergent
command's call site (kernel vs shell vs game) instead of only the command
byte — the next step in isolating the early-boot branch divergence behind
Kula World bug 3. Builds clean; behavior-inert when no listener is attached.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdjxoRB1f9VQsNzZayCgGS
Bit 2 is ADPBUSY (XA-ADPCM playback in progress) and must idle at 0; we
raised it unconditionally under a mistaken 'ADPCM empty' reading. Bit 7
(BUSYSTS: command written, first response not yet ready) was never raised;
now mapped to the queued-command window, mirroring beetle's
PendingCommandCounter/phase busy window. Verified against beetle cdc.cpp
Read(A==0), which sets bit 2 never and bit 7 during command latency.
Boot-diff verification shows this does NOT change our boot command stream
(the early-boot branch divergence vs the oracle persists — documented in
docs/kula-world-findings.md round 4, with the next sharp step: add guest
PC to the oracle's cdcmd hook and diff the first divergent call site).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdjxoRB1f9VQsNzZayCgGS
New debug tool: PSX_CD_TRAP_CMD=<byte> [PSX_CD_TRAP_NTH=<n>] SIGSTOPs the
process the instant the Nth matching CD command dispatches, so a debugger
attaches to the exact moment with zero run-speed cost (a gdb conditional
breakpoint on exec_command slows the emu ~100x and never reaches the wedge).
Used it to pin Kula bug 3 one level deeper: the pre-wedge ReadTOC at frame
~890 is issued by the BIOS BOOT FLOW itself (guest stack shows only
Main/StartKernel-chain ROM addresses, zero game frames) — the boot's
post-EXE-load disc re-check. And the full command-history dump shows our
boot diverges from the oracle's from the VERY FIRST command: the oracle's
kernel opens with Init/Init/GetTN/GetID and never issues Test or ReadTOC;
ours opens with the shell's Test[20]/GetID/ReadTOC check and re-checks
after the EXE load. Same SCPH1001 bytes, different path — an early-boot
branch divergence (recompiled-BIOS or hardware-sim input), documented in
docs/kula-world-findings.md for the first-divergence session.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdjxoRB1f9VQsNzZayCgGS
Full oracle command trace confirms it NEVER issues ReadTOC; ours does, from
the BIOS (Pause->ReadTOC->GetID) at frame 890, with clean pre-read stat
(0x22, no error/shell bits). Same BIOS bytes on both, so our runtime reaching
the disc-identify path is a state-driven BRANCH divergence, not a CD-sim
error: likely a kernel 'disc already identified' flag left unset, or a
recompiler-level game-logic divergence. Next: instruction-level
first-divergence hunt around frame 890.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdjxoRB1f9VQsNzZayCgGS
Adds a CD-command trace to psx-beetle (beetle_cdcmd_trace_hook.patch:
callback in cdc.cpp's command dispatch, wired through beetle_libretro.cpp
into cdrom_cmd_dump / cdrom_cmd_reset on the oracle debug server). This is
the two-process oracle-diff the setup is built for, now working on Linux.
Live-diff result (docs/kula-world-findings.md, bug 3): the oracle's game
reads level data with a clean Setloc->SeekL->Setmode->ReadN->Pause per
sector (SeekL x18, ReadN x22, GetlocL x89). Our runtime's game, in the
wedge, issues NEITHER SeekL NOR ReadN — only GetStat/Init/Setloc/Pause,
an error/retry branch the oracle never enters.
Crucial: our runtime DID deliver 307 sectors during early boot (ReadN +
data delivery work). Reads only stop at frame ~890 — the BIOS mid-game
region revalidation (ReadTOC+GetID). The region fix made that PASS, but
the post-ReadTOC CD state still stops the game resuming reads. Prime
suspect: our ReadTOC (case 0x1E) is a bare stat+COMPLETE stub that does
not model drive repositioning / the track table the game reads to
recompute its seek. Documented for the next session.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdjxoRB1f9VQsNzZayCgGS
The live trace shows the game's CD-init state machine (GetStat/Init then
Setloc/Pause, ~2.5s cadence) receives CORRECT responses at every step —
stat=0x02 (healthy idle drive), INT3 ACK + INT2 COMPLETE both fire and the
game's ISR processes them — yet it retries forever. So bug 3 is NOT a
wrong-response-value bug like bug 2; the game is waiting on a timing or
event-completion condition its init check never sees satisfied. Documents
two next-session paths: (1) rebuild psx-beetle with a CD-command hook and
live-diff the boot->demo sequence, (2) find the game's init-complete branch
condition and check our INT2-COMPLETE delivery timing against it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdjxoRB1f9VQsNzZayCgGS
Records the two fixed bugs (RAM mirror, GetID region) and characterizes
the open third wedge: the game's own CD driver loops Setloc(PVD)/Pause
without ever reading the directory, so the demo level never loads and
the screen stays black. The oracle loads it, so the divergence is
concrete and localized for the next oracle-diff session.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdjxoRB1f9VQsNzZayCgGS
beetle_libretro.cpp targets the C++ beetle-psx tree; upstream master has
been converted to plain C and no longer compiles against it. Documented
the working recipe: pin beetle-psx at 5759277b (the wtrace patch's base,
located via the patch's own blob hash), apply beetle_wtrace_hook.patch,
and apply the new beetle_sio_trace_hook.patch — FrontIO::SetSIOTraceCallback
(fires once per completed SIO byte exchange), a hook beetle_libretro.cpp
required that was missing from the committed patch set.
Verified on Linux/Xvfb: psx-beetle boots Kula World to demo gameplay and
serves ping/screenshot over the shared TCP debug protocol, alongside
kula-runtime — the two-process oracle setup from CLAUDE.md §16 now works
in this environment. Runtime-created per-game .mcr files are gitignored.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdjxoRB1f9VQsNzZayCgGS
Completeness layer for the Tomba 2 FMV-freeze fix (the discovery/ownership side
of "containment is not ownership, ownership is not entry legality"). In overlay
discovery (main_psx.cpp --overlay), after walking the seed roots, every DIRECT
CALL (jal) target that lands strictly INSIDE an already-discovered function is
registered as an ALIAS ENTRY of that host via the existing alias machinery
(materialize_alias_groups). So when a host's discovered range absorbs a separate
routine (e.g. the crt0 `jal 0x50B08; break` falling through into the init-array
at 0x89788), the host's entry-switch now routes a dispatch of that PC to the
correct absorbed interior block and runs it NATIVE — instead of depending on the
PC having been captured, or falling closed to the interpreter.
Derived purely from static jal evidence (jal encodes a code target, so it never
mints code from data); does not depend on execution capture. Preferred over an
explicit range-clip because the absorbed code IS the right code — it just needed
a routable entry. The static two-phase FunctionDiscovery already enforces
hard_cap = next-entry, so only the overlay path needed this.
VERIFIED: regenerated cache (codegen cg4_42de29a8) emits `case 0x80089788u` in
func_800896E0; cps_probe for 0x89788 drops 1021 -> 0 (now main-chain native, not
the broken CPS find-by-range default); freeze stays gone, 60fps sustained. Pairs
with the fail-closed entry guard (bd60381), which remains the safety net for any
genuinely-unrouted interior PC.
Framework-wide: regression-test Tomba 1 / MMX6 / Ape before any master merge.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Record the implemented fix and verification at the top of the freeze writeup:
overlay CPS entry-switch fails closed on a foreign interior PC -> sanctioned
dirty-RAM interpreter (ABI v6). Verified native passes the freeze (frame
1823 -> 18001+, 60fps, i_mask=0x0D, reval_crc_miss=0). Notes the optional B/C
completeness layer (clip overlay ownership range in the capture-driven overlay
discovery path) and that the static two-phase discovery already enforces
hard_cap=next-entry.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds overlay_cps_probe (debug cmd) to record what the CPS interior-continuation
dispatch decides for a watched PC. Used it to CONFIRM the root cause:
func_80050B08 calls jal 0x89788 (init-array). 0x89788 isn't a registered entry,
so overlay_find_by_range(0x89788) runs -- 6 loaded candidates have a range
containing it -- and returns func_800896E0 (the crt0 bootstrap), whose range
wrongly spans 0x89788. 0x89788 is not a case in its entry-switch, so default
re-runs the bootstrap from the top: InitHeap/A0 spin, VBLANK-masked deadlock.
func_800896E0's range covers 0x89788 because function_discovery.cpp:493-499
treats BREAK as fall-through; the bootstrap's "jal 0x50B08; break" (break guards
the main-returned case) falls through at 0x89784 into the separate function at
0x89788, absorbing it. So the early break-no-op finding WAS the root -- damage is
at discovery/range time, not runtime break execution. Interp is immune (executes
0x89788 bytes directly).
Fix options + recommendation (clip function range at next entry / don't cross a
function boundary on break fall-through) documented in
docs/TOMBA2_NATIVE_FMV_FREEZE.md. Diagnostic only; no fix applied yet (needs
regression on Tomba1/MMX6/Ape). Workaround: overlay_native_block 0x80050B08.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds a phys-keyed blocklist in overlay_loader.c that forces a named overlay
function through the sanctioned dirty-RAM interpreter (it still RUNS — not
skip/stub/HLE), as if s_native_exec were off but for that one function only.
Exposed as the `overlay_native_block` debug command (add/clear/report). This is
the localization tool for native<->interp divergences (e.g. the Tomba 2 FMV
freeze): binary-search which compiled overlay function's native execution flips
the bug, without rebuilding or losing candidate visibility. Runtime/tool-only.
Used it to localize the Tomba 2 end-of-Whoopee-Camp FMV freeze to
func_80050B08's OWN native body (blocking it passes the freeze with callees
still native; all 33 jal targets + continuations verified clean). Narrowed to
the main-loop region with a strong interior-entry suspect at 0x80050CE8.
Findings in docs/TOMBA2_NATIVE_FMV_FREEZE.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Whoopee-Camp-end freeze with the warm overlay cache is a DETERMINISTIC
codegen divergence, not a timing race -- proven via the sandboxed
native<->interp shadow-compare (overlay_diff/shadow_dump): both engines
run from identical CPU+RAM state, no IRQ/timing, and native still diverges.
First divergence (overlay_shadow_detail) is fn 0x800896E0: native t1=0xE4
(=0x39<<2, the A0 dispatcher`s sll t1,t1,2) / t0=0xBFC01E24 (A0_table[0x39],
InitHeap) vs interp t1=0x39 / t0=0x80000000. Native wrongly routes through
the A0:0x39 dispatch path interp doesn`t take -> re-inits every spin-loop
iteration -> endless A0-vector dispatch with VBLANK masked -> deadlock.
Also documents a separate latent CLASS bug: the recompiler does not model
the R3000A load-delay slot (strict_translator.cpp:830) while the dirty
interp does; 50 load-delay hazards found in this game`s overlays (0
control-flow) -- not this freeze, but a real divergence class to class-fix.
Same ~frame-1828 hang a prior session believed fixed by 47f3f15; it was
masked by interp fallback (audit-failed region), the clean warm cache
re-exposes it. Exact diverging instruction + recompiler fix still pending.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
P1+P2 done (c5efd70). Decoupled post-loop fragment pass; validated by probe
(clean mid-entry island for 0x80083094), full run (34/47 executed orphans at
0x38000 incl 0x80083094 as isolated shards, despite that region's normal compile
audit-failing), and a fresh runtime run (20 DLLs / 264 native / crc_miss=0 — no
divergence). Open: clean good-variant warm for end-to-end FMV/title speed (messy
accumulated captures audit-fail the region, losing hot NORMAL funcs); optionally
extend isolated fragments to executed normal funcs; P3 host recovery optional.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Architecture review verdict for the FMV-driver coverage gap: DON'T recover the
host. The regression was mixing speculative recovered-host roots into the trusted
all-roots region DLL (shared failure domain). Fix = three isolated build classes
(REGION_TRUSTED / RECOVERED_HOST_SPECULATIVE / INTERIOR_ENTRY_FRAGMENT); for the
FMV, compile an interior-entry ISLAND that enters at the executed PC 0x80083094
over a worklist-discovered CFG (not linear-to-boundary), its own DLL, keyed by
covered ranges + covered-byte hash. Mid-function start is safe iff codegen makes
no function-entry assumptions (no synthesized frame; out-of-island branches are
dispatcher exits, not audit failures). Phased plan recorded.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Attempted orphan-interior host recovery (walk back to the host start, root it so
the FMV driver 0x80083094 compiles). It REGRESSES: adding the recovered host as a
root to the region's shared seed list (even gated to executed/resident orphans)
makes the recompiler emit C that fails the strict generated-C audit, and because
all roots in a region compile into ONE DLL, one failure skips the whole region ->
the previously-good 0x38000 region stopped compiling. Reverted to clean
checkpoint. Robust path = isolate each recovered host (+ its interior aliases)
into its OWN compile unit / DLL so a bad recovery fails alone (loader already
content-matches per function across DLLs), or audit-validate each host before
including it. Dedicated effort; recorded for next session.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Records the three fixes that made native overlay execution stable through boot
(region_start load fix bb62b20, alias-rollup 47f3f15, timer write8 6004ced) and
the remaining FMV-speed blocker: the hot FMV driver 0x80083094 is a
DISPATCH_INTERIOR orphan (no discovered host -> not compilable -> interp-only).
Plus the next-step plan: recover the host by walking backward from the interior
PC to a callable-start signature, root it, and let the interior become an
alias-entry into it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The slowness root cause (region_start walkback overshoot) is fixed (bb62b20):
native funcs 14 -> 790. Remaining blocker = a native-vs-interp divergence at
0x80051FA4 that hangs the boot when native runs.
0x80051FA4 is a CPS CONTINUATION point inside host 0x80051F80, and the recompiler
emits BOTH a full host func_80051F80 AND a separate epilogue-only alias body
func_80051FA4 -- with the SAME .ranges range (R 80051F80 34) and SAME code_crc
(5CE0B05E). So two candidates cover identical bytes and a continuation to 0x51FA4
can resolve to the epilogue-only alias instead of the full host. Hypothesis:
aliases/interior entries should roll up to (dedup into) the parent host shard.
Recorded for the recompiler fix + ChatGPT architecture review.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Audit of the current overlay-cache subsystem (loader, capture, autocompile,
compile tool) plus the design target. Key finding: the correctness model is
sound (dispatch re-validates live RAM vs per-function code_crc every call, so
no stale execution) -- the problems are efficiency: whole-region-CRC build
keying causes redundant DLLs, a per-dispatch full CRC32 taxes warm code, and
capture/dlopen/JIT run on the main thread. Phased plan P1-P5.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The fixed CD command trace shows single-sector data reads (PVD16->root22->subdir373) all work; the
game stalls AFTER the directory read at the FILE-LOAD DECISION, issuing no further CD command. The
CD-audio/XA streaming is still missing (oracle DMA3-streams, recomp doesn't) but the game never
reaches it, so it's downstream, not the gate. Corrected blocker: a state the game tests after the
dir read diverges (no SetLoc+ReadN for the next file). Supersedes the premature 'CD streaming root
cause' conclusion.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The splash softlock = the CD streaming/XA/CD-DA + DMA-3 audio path being incomplete. Confirmed: the
game's CD irq handler at pc=0x8008B6F4 spins on [0x800AC2C4]=0x1F8010B8 (DMA3/CDROM CHCR) bit
0x01000000, waiting for the CD-streaming DMA to complete; the recomp gets empty reads (no stream to
0x1Axxxxx like the oracle's ~1.1M DMA3 transfers via game routine 0x8008DB88). Directory read is
perfect (full file list) — not a CD-fidelity/game-code bug; the game just never loads the next file
(0x107xxx advance task) because the streaming sequence stalls. cdrom.c Play(0x03) comment confirms
'Red Book sample output is not yet decoded — silent for now'. Tomba 1/MMX6/Ape never streamed CD audio
(SPU-voice/SEQ, single-track data discs) so this path was scaffolded-not-built. FIX = build the
Architecture-A CD streaming hardware-sim: DMA3 delivers requested sectors continuously (data->game buf,
XA->SPU decode, CD-DA Red Book output + position/end), so the irq DMA-wait completes and the splash
advances. STUBS_TO_FIX S3/S4/S5 family, now actually exercised.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Recomp's 0x104368 subdir (LBA 373) decodes to the full file list (A00-A0L.BIN, CRD/DEMO/GAME/OPN/
SOP/START.BIN, valid extents) — dir parse is perfect; recomp just never LOADS the next file. The
LBA-24/0x41800 512KB load holds the splash code; the 0x107xxx advance task is a separate file that
never loads. NEW CLUE: recomp CD froze ~frame 1987 (stuck 4379), cdrom_trace 100% flooded with dma
empty-reads (val=0, func=0x00000E10 exception handler, pc=0x8008B6F4); recomp CD dests only
0x41800/0x104368 while ORACLE hammers DMA3 to 0x1Axxxxx (~1.1M, routine 0x8008DB88). => recomp is NOT
streaming the splash CD-DA/XA audio (Whoopee jingle, multi-track TRACK02/XA) — getting EMPTY reads in
the CD interrupt handler. Hypothesis: splash->attract is gated on the audio stream; empty audio ->
gate never fires -> next-file load (0x107xxx advance task) never issued -> state stuck 0. Ties to the
multi-track CD-DA/XA streaming path in the CD/VBlank handler (same neighborhood as frame-1997). BLOCKING
TOOLING (Rule15): cdrom_trace ring evicted by dma empty-reads — edge-suppress 'D' to see CD commands.
Next: fix the flood; investigate multi-track CD-DA/XA audio read (empty from pc=0x8008B6F4); confirm the
splash gate on audio.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Decoded recomp 0x104368: a well-formed ISO9660 directory (rec0 {len48, extent_LBA 373, size 2048,
flags 0x02 dir, name '.'}, rec1 extent 22 '..', more). So the recomp read+parsed the LBA-373 directory
CORRECTLY — CD fidelity + data are fine. The divergence is AFTER the directory read: the code that
walks the dir to find the target file and issue its load (->0x107xxx) takes a different path in the
recomp and never issues the ReadN. Next: find the routine that issued the LBA-373 read (wtrace CD cmd
reg from boot -> PC/ra), disasm live, compare its post-read branch + tested values vs oracle (likely a
filename compare / dir-walk early-exit / 'already loaded' or event-flag check / differing return value).
DMA3 trace also showed the oracle hammers CD-DA/XA audio DMA (~1.1M MADR writes to 0x1Axxxxx via game
routine 0x8008DB88) during the splash; the recomp's CD dests are only 0x41800/0x104368 (may also not be
streaming the splash audio — possibly related).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Restarted recomp, dumped cd_read_log: total=2651, dest range 0x41800..0x104368, LBAs {16,18,24,373},
covers_0x107xxx=False. Recomp does the directory lookup (LBA16/18/373->0x104368) but never issues the
file load to 0x107xxx, then goes quiet (stuck). NOT a DMA-targeting/CD-fidelity bug (the 0x41800 load
proved raw reads land correct advancing code) — it's a code-path/TASK divergence in the file-load
DECISION. The state-0 handler 0x80050D44 only renders+loops, so the load is issued by another
task/callback during the splash that the recomp doesn't run. Likely candidates: (a) per-frame
scheduler/event chain off the game VBlank handler (same neighborhood as frame-1997 HookEntryInt) so
the scene-loader task never runs; (b) CD-complete-event callback; (c) a thread/TCB never resumed.
Next: trace the oracle's CD cmd stream (@0x1F801801) ~frame 2289-3960 to find the SetLoc+ReadN for
0x107xxx + the issuing PC, then check if the recomp executes it / how its event-callback chain differs.
Tooling debt: beetle lacks cd_read_log + cdrom cmd trace; raw-socket cd_read_log parse fails (use client).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
wtrace 0x80107000-0x80108000 on a fresh oracle: only BIOS zeroing (0xBFC0D864) hits via CPU
stores; the real 0x80107268 code is ZERO at oracle frame 2289 but present by 3960 with ZERO
CPU stores carrying it => installed by CD DMA (ch3 CD->RAM), LATE (mid-splash ~frame 2289-3960),
not at initial boot. So CD is NOT fully ruled out: the early 0x41800 load works but this later
0x80107xxx overlay DMA is the one the recomp never receives (region zero even at frame 78k).
Refined model: the state-0 splash loop initiates/waits on a 2nd CD load that brings the advance
task; on HW the DMA completes and drives state 2<->1; recomp's never lands so state stuck 0.
Next: fix cd_read_log cross-tooling (Rule15), restart recomp w/ from-boot CD logging to see if it
issues the 0x107xxx read at all, and trace the state-0 loop's CD cmd+DeliverEvent right before the
load on both. cd_read_log cross-compare currently blocked (beetle lacks it; recomp dump empty once).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Oracle reverse-writer-hunt (wtrace on the state byte from boot): [0x1F80019C] ping-pongs
every frame on HW — 0x80107268 writes state=2 (advance trigger), 0x80050DA8 writes state=1.
The recomp NEVER advances because 0x80107200-0x8010728C is ALL ZEROS in the recomp vs the
real splash-advance function on the oracle. High-RAM 0x80100000+ is partially-loaded and
divergent (recomp has 0x104368/0x10A000 with different content; 0x107000/0x110000 are zero
vs real code). So the softlock is a LOAD/RELOCATION divergence during boot: the recomp
failed to bring the 0x80107xxx overlay into RAM. JIT/cache ruled out (pure interp:
dispatch_native=0, sljit live=0, autocompile off). Next: loader hunt — wtrace
0x80107000-0x80108000 installs from boot on both + early CD-read log, to find the load
mechanism and why the recomp skips/relocates it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ChatGPT (fresh Recomp thread) gave a reverse-writer-hunt plan: catch the oracle AT the
transition, dump the pre-transition scratchpad write history, find the writer PC + predicate.
Top hypothesis: a GPU/DrawSync/fade-completion or CD/event-callback flag (state stuck while
VBlank pacing + CD load are healthy). Traps: verify committed-PC in the trace + scratchpad
byte/halfword endian.
Execution so far: confirmed the splash state-0 handler (0x80050D44) only renders+loops and
NEVER advances [0x1F80019C]; scanned all loaded code 0x40000-0x92000 and found NO writer that
sets the state byte to 2 (only init->0 and the state-2 handler->1) => the 0->2 advance is
external/event-driven. Confirmed 0x80050CE4 is a common VSync-wait loop both scenes hit
(oracle in attract also samples it). Decisive next step: oracle write-watch on [0x1F80019C]
from boot to catch the transition writer.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Investigated the splash softlock: the attract data loads CORRECTLY (consecutive CD-DMA
dests hold different valid MIPS code; read_msf reached LBA 374). The cd_read_log 'lba'
field misleadingly logs the SetLoc target (24) not the per-sector position — a tool bug
(Rule 15) that cost a wrong 'CD stuck at LBA 24' hypothesis. So the splash->attract
divergence is a true first-divergence in the scene-transition state machine ([0x1F8001A4]
stuck at 0), upstream of the common 0x80050CE8 loop both recomp and oracle reach. Next:
oracle PC-trace from the splash. Secondary: ~frame-17k silent crash (~8e9 dirty insns)
- possible host-stack growth from the frame-1997 fix's re-dispatch on nested RFEs; also
a regression-risk signal for the shared path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Tomba 2 runs past the frame-1997 exit (splash renders, frame 16k+, aborts=0). New
cascade frontier: recomp lingers on splash with CD idle while the Beetle oracle on the
same disc reached the attract/intro gameplay; both at idle PC 0x80050CE8, so the recomp
tracks the oracle's code but diverged in game state (oracle triggered the attract CD
load, recomp did not). Pending gates documented: T1/MMX6/Ape regression + scaffolding
cleanup before master.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implemented + instrumented the consult's Option-A async-resume idea; the data REFUTED
the in_exception=0 async model:
- freeze_check now reports async_rfe_set/fire, reach_dirty/traps, reach_async,
dirty_safe_resume_pc, async_rfe_resume_pc.
- Measured: reach_dirty=19921 (~1/exception), dirty_in_exc0=0, async_rfe_fire=0 ->
the sentinel 0x80000048 ALWAYS arrives at the dirty gate with in_exception=1 and
takes the longjmp. There is no in_exception=0 dispatch of the sentinel.
Corrected mechanism (docs/TOMBA2_WHOOPEE_BAIL.md): the game's ReturnFromException
longjmp lands in psx_check_interrupts (jmp_val==1), restores GPRs and returns with
cpu->pc=0; for ~19920 exceptions it lands at a NESTED dispatch (game continues), but
for the failing one it unwinds to the OUTERMOST trampoline where cpu->pc=0 reads as a
clean exit. It's a longjmp-landing-depth bug in the 'resume via host return' model,
not an async-resume bug.
Latches added (g_dirty_safe_resume_pc around the 3 dirty pumps, g_async_rfe_resume_pc
at dirty-safe exception entry) + the sentinel-gate redirect scaffolding are INERT
under the corrected model (never fire) — kept as tooling + scaffolding for the
candidate fix. Builds clean; behaves identically to pre-fix (exit at frame 1997, no
regression). wt/tomba2 only; not on master.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Turned the prior session's inference into proof via PSX_EXIT_HALT live state:
- dispatch-tail ring shows game VBlank handler 0x80085D8C -> B0:0x17
ReturnFromException -> pc=sentinel 0x80000048 -> exit.
- in_exception=0 at the failing RFE (freeze_check + GATE_COOLDOWN, two ways).
- TCB save_area off128 (EPC)=0x80000048 sentinel, while ra/sp are the correct
interrupted ctx -> the sentinel-in-TCB is the smoking gun.
- handler is HookEntryInt-installed (B0:0x19 @0x80085D3C) and self-drives RFE.
- dirty-interp pump commits cpu->pc before psx_check_interrupts -> a precise
guest resume PC exists but is discarded for the sentinel.
Fix design = scoped EPC de-overload (Option A) in docs; consult package in
docs/TOMBA2_EXC_CONSULT.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
With Tomba 2's overlays now on the local-flow path (prior commit), the
per-4096-insn interrupt pump delivers the exception handler from a NESTED
dispatch (g_psx_dispatch_depth > 0). psx_check_interrupts reset the counter to 0
around the handler but never restored the interrupted code's nesting, and a
ReturnFromException longjmp skips the handler frames' --depth decrements. The
outer frames then unwound below zero (dispatch_depth -> -1), the `outermost`
test misfired, and the top-level dispatch returned to PC=0 ("execution
completed" abnormal exit at frame 1997).
Save the interrupted code's depth before the handler, run the handler as its own
outermost context (0), and restore the saved depth afterward. Depth-transparent
for callers that were already at depth 0 (Tomba 1 / MMX6 / Ape), so no behavior
change there.
Resolves: dispatch_depth stays correct (0, not -1) across delivered interrupts.
NEW failure point (next in cascade, NOT addressed here): a genuine guest-state
divergence at frame ~1997 — overlay chain 0x80089784 -> 0x80080FC4 -> 0x800499E8
-> 0x80050CEC does a jr to a null register (final_ra 0x80050CC8, PC=0). Needs an
oracle first-divergence pass. See docs/TOMBA2_WHOOPEE_BAIL.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The $ra->1 tripwire + first-bail in_exc latch (d965d6e) refuted the
exception/EPC model: in_exc=0 everywhere; $ra->1 is at a normal epilogue
'lw $ra,0x34($sp)' (0x8008B018) loading a corrupt saved-$ra because $sp is
wrong after the first bail's non-local B0 return. Real bug = control-flow/
stack-contract in the bail/flatten of a B0 non-local (tail-call) return
(recomp-debug Class 3), not exception/RFE/EPC. EPC-separation fix marked refuted.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Refined via binary truth + actual code + recomp-debug methodology + ChatGPT.
0x80085900/0x8008AE50 are NOT in the boot EXE (PS-EXE t_size=0x28800) -> disc
OVERLAY run by the dirty-RAM interp. The bug is the dirty-interp/overlay<->
compiled-BIOS boundary when an async interrupt (now delivered by the pump fix
0a6920a) lands mid-call. interrupts.c:355-384 overloads guest EPC as a host
longjmp sentinel (0x80000048), colliding with the BIOS's architectural EPC/TCB
use -> the interrupted continuation's $ra is clobbered to 1 -> wild-return bail
storm. NOT a broken BIOS exception handler (recomp-debug 'platform infra is
correct' trap). Fix design: separate host exception-return metadata from guest
EPC; restore host call-contract on RFE; or defer async IRQs to a safe point.
Confirm-first: $ra->1 tripwire.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Clear writeup of the splash freeze: what's ruled out (all hardware-sim healthy
- interrupts/timer/CD/events), the root cause (wild-call/return-contract
corruption at the exception<->B0 boundary - clobbered to 1 on RFE resume of
an interrupted wild/dirty call -> infinite bail loop = the 68k/frame spin), the
captured evidence (trigger 0xB0->0x8008AE50 @frame 1823, register state), open
questions, and fix direction.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comprehensive 3-way merge of the dev/ape-escape branch into master for
the isolated wt/ape-escape worktree. Brought in every complementary ape
change automatically (dma.c, memory.c interrupt byte/half lanes,
function_analysis discovery, main_bios vector seeds, iso_reader
iso_sector_count, config_loader, crash-trace fields, headers). Resolved
the 13 conflicts by rule:
- CPS dispatch / recompiler (code_generator, full_function_emitter,
dirty_ram_interp, traps, main_psx) -> KEEP MASTER (evolved system).
- CD model (cdrom.c) -> take ape (the model that booted Ape).
- interrupts.c/.h -> reverted to MASTER (ape's interrupt-delivery is
dispatch-coupled; the merged Frankenstein stormed).
- sio/debug_server/freeze_heartbeat/main.cpp -> keep master (evolved).
- crash_trace.c -> master (ape freeze-dump fns omitted to avoid the
debug-ring cascade).
STATUS: does NOT boot yet. ape-cdrom + master-interrupts derails to
PC=0 at frame ~2092 (insta-close); ape-cdrom + ape-interrupts storms
(297M exception re-entries) at frame 2077. Root finding: ape's CD
anti-storm is coupled to ape's interrupt-delivery + call-depth + EPC
model, which master replaced with CPS continuation-passing, so the CD
subsystem does not forward-port as pure hardware simulation. Preserved
as the reference state for a design consult on bridging the two models.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The continuation-passing leak fix (RECURSION_BUG.md §25) is validated (Tomba boots+plays,
ce_profile flat through a 56.9k-frame soak, full overlay CPS). Flip the gate in all three
emitter sites so a normal regen emits CPS; PSX_CPS=0 is the legacy escape hatch. Verified:
default regen -> CPS markers present; PSX_CPS=0 -> none; default-CPS Tomba boots clean, leak flat.
Doc §25.6.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The user-requested async overlay tiering (interp-now / compile-async / native-next, gcc-if-present
else sljit, always async, file-based handoff) is LARGELY ALREADY IMPLEMENTED — overlay_capture.c
(autocapture on interp pressure) + autocompile.c (background compile via a configured command,
toolchain-probed, watcher thread, file-based) + the code_provider abstraction. It just needed
CPS wiring.
This commit:
- overlay_loader_apply_live_policy: when the active backend is gcc, force sljit-live OFF so a gcc
box is gcc>interp (NOT the previous complementary gcc+sljit). Toolchain-less boxes stay
sljit>interp. PSX_OVERLAY_SLJIT_LIVE still overrides. ("mutually exclusive" per user.)
- ASYNC_OVERLAY_COMPILE.md: re-headed to reflect the feature is built+validated under CPS, with
the deltas this session added (compile_overlays --cps from §25.5, the gcc-only policy, the
dev-config overlay_autocompile_cmd --cps).
VALIDATED under CPS: autocapture fires -> background compile_overlays --cps produces CPS overlay
DLLs -> loader loads them -> dispatch_native rises; gcc box shows 0 sljit shards; ce_profile flat.
The native overlay cache auto-builds as you play (each scene compiles on its first sustained
visit). PRODUCTION game.toml overlay_autocompile_cmd still needs --cps appended when CPS ships.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Build plan for the next session: interp-now / compile-async / native-next overlay tiering
(gcc-if-toolchain-present else sljit, mutually exclusive, always async). FILE-BASED handoff —
the worker writes .dll/.sljit artifacts; the dispatch fiber loads them via the existing
try_load_region / deserialize paths, so the candidate table stays single-threaded (no race).
Motivated by the CPS title screen being interp-bound (12.2M interp vs 14.8K native) because its
overlays aren't gcc-compiled + the sync sljit JIT-on-miss hitches the dispatch thread.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
RECURSION_BUG.md §25.5. Brings the overlay native path under continuation-passing so the
idle-freeze leak fix holds with overlay_cache=true (the production gcc>sljit>interp path).
Runtime CPS-mode flag g_psx_cps_mode (overlay_loader.c) set by a __attribute__((constructor))
emitted in the BIOS + game CPS dispatch (full_function_emitter.cpp / main_psx.cpp). Gates all
overlay/sljit CPS behavior; legacy unaffected.
sljit JIT (overlay_sljit.c), scoped:
- jr $ra publishes cpu->pc (was a bare return). jr rX already tail-transferred.
- call-containing fragments DECLINE to the interpreter (CPS-safe, depth-1) — leaf shards still
JIT. (Full sljit call coverage / per-fragment entry-switch is future work.)
gcc-DLL path — FULL native overlay coverage (validated):
- compile_overlays.py --cps sets PSX_CPS for the recompiler -> CPS overlay C (tail-transfer +
per-function entry-switch; --overlay mode only affects widescreen, not CPS). DISPATCH_PREAMBLE
psx_syscall decl fixed void->int.
- PSX_OVERLAY_CODEGEN_VER 3->4: fresh cg4/ cache namespace; legacy unit-model DLLs (cg3) ignored.
- overlay_loader_dispatch: under CPS, when a dispatched addr is not an overlay ENTRY but lies
inside a registered candidate's code range (a tail-transferred call's continuation),
overlay_find_by_range() re-enters that function with cpu->pc=addr so its entry-switch routes
to the block. gcc DLLs are the trusted tier (diff-validated at entry) -> run native directly.
This is the CPS equivalent of the old unit-model that fixed the dwarf->overworld blue screen.
Validated: CPS overlay DLLs compile + load + dispatch_native>0; with continuation routing
dispatch_interp_fallback -> 0 (full native), ce_profile FLAT (6 KB), boots clean. Addresses are
KSEG0-consistent.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Matthew Stanley