Commit Graph

49 Commits

Author SHA1 Message Date
Matthew Stanley 167babab9f Merge Tomba snapshot continuation fix into master
Keep upstream atomic v9 restore and handoff serialization. Add scoped snapshot deferral for host adapters, reset abandoned scopes at scheduler escapes, and rebind enabled mod hooks before restored execution. The backport parent preserves the tested Tomba v0.15 savestate/cache lineage. Focused host-snapshot, restored-handoff, and mod-runtime tests pass.
2026-10-04 17:59:57 -07:00
Matthew Stanley 6426b25f3d Preserve restored handoff and snapshot host-call continuations 2026-10-04 17:51:23 -07:00
Alexandros Mandravillis 8f8325a2fe traps: clear pc after a same-thread changethread
With the HLE scheduler (the default), ChangeThread to the thread that is
already running returns 1 from psx_request_thread_switch. psx_syscall case 3
then returned 0 with cpu->pc still holding the SYSCALL's own address.
Compiled callers fall through on 0, but the dirty-RAM interpreter transfers
whenever cpu->pc is non-zero, so it re-executed the SYSCALL forever.

A game that yields from its main thread this way hung at the kernel's
SYSCALL (EPC 0x650) as soon as that kernel code ran interpreted, for
example when its body did not match the compiled kernel.

Clear cpu->pc on that continuation, as the SYS01/02 continuations do. The
guest-exception syscall path returns before case 3 and is unchanged.

syscall_change_thread_self links the production traps.c into a fixture
with a one-thread TCB table at -O0 and -O2. fixture_link.py stubs every
other runtime seam with abort(), so an unexpected call fails the test.
2026-09-30 22:14:24 +03:00
Matthew Stanley c564ccd1f1 runtime: route every RAM consumer through one live 2/8 MiB geometry
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 06:31:33 -07:00
Alexandros Mandravillis f3229541a1 Preserve registers in ExitCriticalSection 2026-09-05 19:53:03 +03:00
Alex Vanderveen d4ea4ade1f fix crash path from bad splash screen bug, and optimize cross platform pacing + rollback 2026-08-14 02:25:59 -04:00
Alex Vanderveen 9689ad40dd shrink 0's and use explicit portable toolchain python 2026-08-08 22:50:17 -04:00
Alex Vanderveen 94cb3807a9 Merge origin/master into integrate/game-codegen-hooks-into-rollback
Keep rollback/netplay multiplayer routing, dual-raster GL present, and TOC
fingerprints while taking master's CHD support, mod load acceleration,
adaptive widescreen, and geometry/perspective APIs.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-06 22:50:38 -04:00
Alex Vanderveen 652be7ba02 rollback FMV patches and overlay 2026-08-06 00:52:14 -04:00
Matthew Stanley 6e6a0b3ed8 spu: implement reverb, noise, sweeps and SPU IRQ; make CAUSE.IP2 combinational
Closes the four gaps named in issue #103 and lands the separable, verified
parts of PR #102. See docs/internal/SPU_FIDELITY_103.md for the per-piece
disposition and the oracle-verification queue.

SPU DSP (issue #103), clean-room from hardware documentation:

- Reverb: the documented 22050 Hz engine — vLIN/vRIN input, same-side and
  crossed different-side IIR reflections, 4-tap comb early echo, both
  all-pass filters, vLOUT/vROUT output, and the work-area address advance.
  Gated on SPUCNT bit 7 for WRITES only; reads, IRQ checks, address advance
  and output always run, as on hardware. Addressing is confined to
  [mBASE, 0x80000). Per-voice sends from EON, CD send from SPUCNT bits 0+2.
- Noise: 16-bit LFSR with the documented parity feedback, clocked at the
  SPUCNT bits 8-13 rate; NON voices substitute it for their ADPCM sample.
- Volume sweeps: all 24 voice L/R plus main L/R. Bit 15 selects sweep vs
  direct; sweep is a live envelope on the same rate machinery as ADSR, and a
  guest read of a sweeping register returns the live level.
- Capture buffers: CD L/R and voice 1/3 output written to SPU RAM, so a game
  that parks the IRQ address there still gets its interrupt.
- SPU IRQ (I_STAT bit 9): checked at every SPU RAM access class — FIFO, DMA
  both directions, ADPCM block fetch, capture writes, reverb work-area
  accesses — gated on SPUCNT bit 6, mirrored in SPUSTAT bit 6, acked by
  clearing SPUCNT bit 6.

The CD-only fast path in spu_render was removed rather than extended: the
SPU now has per-frame work that must run in every path, and an FMV (XA audio
with zero active voices) is exactly the CD-with-reverb case.

This is a clean-room implementation from the psx-spx/nocash register map and
documented algorithm. Beetle's source was deliberately not consulted: it is
GPL-2.0-or-later and this project is PolyForm Noncommercial. PR #16 was
parked for exactly that reason and PR #13's reverb was separately wrong (it
gated on bit 15 of dAPF1, an address offset, instead of SPUCNT bit 7). The
one place this knowingly deviates from hardware is the 22.05 -> 44.1 kHz
reconstruction filter, which the documentation does not specify; it is
isolated in a single function and flagged for oracle comparison.

CAUSE.IP2 (from PR #102):

IP2 is combinational on R3000A — it mirrors the interrupt line and falls the
instant the guest acks I_STAT or masks the source. It was being OR'd in at
delivery and never cleared, in TWO places (the compiled path and
psx_interpreter.c, the latter missed by PR #102), leaving a phantom pending
interrupt that can spin a kernel dispatcher looping on CAUSE.IP & SR.IM.
psx_irq_refresh_cause_ip2() is now the sole writer of bit 10, refreshed at
raise, I_STAT ack, I_MASK write, HLE context restore and power-on. Verified
against the Beetle oracle's irq.cpp/cpu.cpp rather than asserted, and
covered by test_cause_ip2_combinational.

Mid-dispatch audio pump (from PR #102), with a fix: routed through a gate
that mirrors sdl_audio_update's mute/sink decision. Pumping unconditionally
from the VBlank edge, as submitted, would have pushed real audio during every
turbo-load hard mute and defeated the freeze-in-place mute model.

SPU RAM DMA readback (from PR #102): DMA4 in the SPU->RAM direction wrote
literal zeros; SPU RAM is readable memory and titles carry state through it
across an Exec boundary.

Also fixes an unrelated build race found on the way: tools/embed_spirv.py
wrote its intermediate SPIR-V to <source>.spv, so the two runtime targets
that embed the same shaders raced on one path and one deleted the file the
other was reading.

Co-Authored-By: Alexandros Mandravillis <Alexbeav@live.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 18:16:16 -07:00
Alex Vanderveen fe403e6ce1 rollback patches 2026-07-31 17:14:35 -04:00
Alex Vanderveen 31015ceab1 feat: launcher polish and recomp-net lobby netplay
Wire MotK-style lobby (recomp-net submodule, WS client, match_caps),
netplay delay-sync, and launcher UX so host sim settings stay aligned
across peers.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-21 21:37:57 -07:00
Matthew Stanley 05f6ef3e02 Optimize GTE fallback and isolate interpreter perf runs 2026-07-19 14:23:36 -07:00
Matthew Stanley 44b895e331 Merge remote-tracking branch 'origin/master' into spike/load-time-zero 2026-07-14 22:48:36 -07:00
Matthew Stanley 12934ed2c5 data-shards spike: memoized pure-function replay (capture/verify/replay + cycle-faithful credit)
- runtime/src/data_shards.c + include: read-before-write recorder fed from
  the memory chokepoints; byte-verified replay (kernel-bless pattern);
  recorded-cycle credit in slices through psx_advance_cycles +
  psx_check_interrupts_at; purity poisons (MMIO/DMA/thread-switch/budget);
  .dss persistence under the unified cache dir; TCP data_shards cmd.
- emitter: [data_shards] funcs -> psx_datashard_enter at entry +
  psx_datashard_ret at jr-$ra sites (rolls cg-hash).
- obs: phase_hot set=static keyed on new g_psx_last_fn_entry entry stamp
  (current_func dispatch stamp survives post-IRQ resumption and pooled 55%
  of static samples on kernel 0xF40); TCP starv_ring query for the
  always-on PC-sample ring.
- docs/DATA_SHARDS.md: design, soundness model, gates, FMV exclusion.
2026-07-14 11:08:02 -07:00
Matthew Stanley 7350f2413a traps: enrich misaligned dispatch diagnostics
Adapted from NyperYuhgard's PR #13.

Co-authored-by: NyperYuhgard <93950153+NyperYuhgard@users.noreply.github.com>
2026-07-14 10:20:08 -07:00
Matthew Stanley 3d66850794 fix(shadow-diff): instrument hardening — interior gate + longjmp-escape fixup
Part of the ISSUES #9 shadow-diff work (and the #8 P0 wild-jump hunt it
unblocks). The CPS interior re-entry path in overlay_loader_dispatch ran
native with no diff gate; mirror the entry chain (want_diff + diff_passes
< OVERLAY_DIFF_BUDGET -> interp) and count interior_gated.

Longjmp-escape hardening: a shadow that starts inside an exception
dispatch could be unwound by the guest RFE's longjmp (setjmp frame below
run_shadow_diff), leaving s_in_shadow/s_native_exec/s_suppress_irq stuck
and silently killing the instrument for the rest of the run.
run_shadow_diff now records the pre-shadow globals plus the exception
setjmp epoch at shadow start; overlay_loader_shadow_escape_fixup() (called
from the traps.c deferred-honor longjmp site) restores them iff the target
frame predates the shadow. s_shadow_cand routes only the shadow
candidate's own interiors native; the post-dispatch IRQ pump no-ops in a
shadow so it cannot ack a device out from under the diff.

Refs ISSUES #9. Segment-granular redesign still pending (see ISSUES.md).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 19:38:49 -07:00
Matthew Stanley 4f80d22096 fix(sched): defer in-exception thread switch to a clean block leader (memcard fix #2)
The Ape Escape "Checking MEMORY CARD" thread-switch smear: a genuine in-exception
ChangeThread detected deep in nested host dispatch commits a poisoned TCB context
(stale resume-PC latch ahead of the live GPRs) -> the resumed consumer faults on
a smeared jumptable index (DISPATCH FATAL).

Fix #2 (user-agreed architecture): don't honor the switch at the poisoned nested
point. Resume the outgoing thread transparently at its OWN EPC, record the switch
as pending, and honor it at the next block leader where that thread is running
with a materialized resume PC -- re-saving it cleanly there. Do NOT require the
outermost STATIC dispatch boundary: the memcard consumer lives permanently inside
the nested dirty-RAM interpreter and never reaches it; the scheduler landing
already resets the nested dispatch/call-unit counters on the longjmp, same as the
immediate-switch path.

- interrupts.c: the kind-30 escape splits into immediate (outermost / toggle-off
  / no clean resume PC) vs deferred; new honor hook at the top of
  psx_check_interrupts; always-on thread_trace kinds 31 (defer) / 32 (honor) /
  33 (stale-drop). The deferred branch restores the outgoing thread's clean
  GPRs/HI/LO/SR AND cpu->pc (the in-exception RFE had left cpu->pc at the target).
- traps.c: psx_sched_save_context / set_current_tcb / current_tcb exports over
  the scheduler's private TCB accessors.
- A/B toggle PSX_DEFER_SWITCH (default ON); =0 forces baseline immediate switch.

Validated (LLE + HLE, ring-verified): converts the smear DISPATCH-FATAL into
forward progress -- the card driver now reads 108 blocks. A/B (PSX_DEFER_SWITCH=0)
proves the fix is SOUND: the remaining stall (SIO IRQ bit 7 masked in I_MASK, card
read never completing) reproduces IDENTICALLY with the fix off, so it is a
SEPARATE, pre-existing SIO-mask bug -- the next blocker, not this fix's residual.
Goal (file-select showing File 1-4) NOT yet reached.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 12:29:53 -07:00
Matthew Stanley 5478d8309f fix(sched): defer IRQ inside nested overlay-native call units (partial memcard fix)
Overlay-native callees checked interrupts at every block leader regardless of
nesting; an IRQ + cooperative ChangeThread inside such a nested unit suspended
the interrupted thread with an inconsistent (resume_pc, sp) snapshot -> a leaked
frame that compounded into a smeared jumptable index (Ape memcard). Gate the
overlay CI wrappers on g_call_unit_depth (incremented around the nested unit in
overlay_loader_call_native; scheduler-landing backstop reset). A/B toggle
PSX_OVERLAY_UNIT_DEFER (default on). This deterministically reaches file-select
in some timings but does NOT fully fix it: the residual v1 smear comes from the
kernel async-poll callback clobbering registers across the switch with a stale
resume-PC latch (an exception-save/resume issue), to be fixed against the Beetle
oracle. The broader dirty-pump gating tried here was reverted (all-titles risk,
no benefit for that residual path).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 10:37:45 -07:00
Matthew Stanley 4cc757f0c9 bios: seed all 30 ROM-resident kernel-table targets; unknown dispatch is FATAL by default
Root cause of MMX6 "A game data could not be found" (weeks-old, cross-backend):
the game's save-scan calls memset through the kernel A0 jump table; entry
A0:2B points into BIOS ROM at 0xBFC02B8C (one of the entries the kernel never
patches into RAM). Transitive discovery can never find it (reachable only via
the data table, no jal), so the dispatch missed and psx_unknown_dispatch
silently no-op'd — killing the scan before firstfile ever ran. Every card
layer (SIO, IRQs, SwCARD events, sector data, kernel directory parse) was
verified correct along the way; the sole defect was the undiscovered ROM
function plus the silent miss.

- recompiler/seeds: enumerate the post-init A0/B0/C0 tables (kernel RAM
  0x200/0x874/0x674) and seed every ROM-pointing entry with no dispatch
  coverage — 30 functions (memset, memcmp, memmove, strncat, rand-family,
  card/dev services...). memset was merely the one MMX6 called; the rest were
  identical latent bugs. BIOS regen: 4406 -> 4439 dispatch entries.
  MMX6 CONTINUE now lists and loads saves (user-validated + scripted run:
  full FILE SELECT, 1093 card txns, unknown_dispatch_log total=0).

- traps.c: an unresolvable dispatch target now dumps a crash report and halts
  BY DEFAULT (PSX_FAIL_FAST_UNKNOWN_DISPATCH=0 opts out for diagnostic
  sessions; the miss is ring-recorded either way). The old silent-no-op
  default left stale registers and a severed call chain, and masked this bug
  for weeks. Deliberately NO interp fallback: the dirty-RAM interpreter is
  for runtime-written code (Rule 18); interpreting around a static-discovery
  hole would hide it exactly like the no-op did.

- runtime.cmake: EXE_NAME option — per-game process names (mmx6-runtime.exe)
  so one game's `taskkill /IM psx-runtime.exe` dev sweep no longer kills
  every other game's instance.

- debug rings (the tooling that found this): HLE call ring gains
  consecutive-duplicate collapse w/ repeat counts (a 4-way TestEvent poll at
  175K calls/s no longer flushes the ring in 0.1s), per-entry TCB + in_exc
  (thread attribution), a "since" paging param and 8MB dump buffer;
  card-trace EvCB watch widened to all 7 card event slots (slot 0 =
  SwCARD-IOE, the success event, was previously invisible).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 22:39:25 -07:00
Matthew Stanley 496d1dbbc8 runtime: HLE tier wiring + carried instrumentation from the card-load hunt
HLE tier wiring (completes the previous commit — HEAD builds again):
- main.cpp: [runtime] bios_hle/keep_intro consume + settings.toml layer +
  PSX_BIOS_HLE / PSX_BIOS_HLE_KEEP_INTRO env resolution + backend banner;
  boot-turbo window ORed into the turbo predicate; the fast_boot snapshot
  restore/capture path and s_fast_boot_active are removed (deprecated by
  the HLE boot shell-skip); psx_hle_scheduler_set_default call.
- traps.c: psx_hle_scheduler_enabled default now flows from config
  ([runtime] hle_scheduler) with the PSX_HLE_SCHEDULER env winning.
- debug_server.c: hle_dump command (always-on HLE ring query).

Carried instrumentation/state from the MMX6 card-load sessions (kept
deliberately; documented in memory + handoff):
- dirty_ram_interp.c: XOP_RES interp JAL call-resolution probe + xprobe
  watched dump section (emitted first), watch targets.
- overlay_loader.c: probed wrapper around dispatch_call + psx_sljit_call.
- debug_server.c: card_mgr_trace range widen, TestEvent spin-body
  collapse, 64MB response cap.
- interrupts.c: PSX_SAME_THREAD_RESTORE selector (0/1/2; TCB-equality
  hardening candidate).
- memory.c: PSX_POLL_PROOF temp gate (remove when card work resumes).
- savestate.c/.h (new): user save states (F1-F12), HLE-scheduler-gated
  load via psx_scheduler_resume_at; load-crash known, do not use for
  repros yet.
- psx_cycles/autocompile/cycle_testrom: timing-fastpath + exe-path
  session state.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 16:27:57 -07:00
Matthew Stanley f8aefefec1 tooling(parity): general two-process control-flow parity trace (native producer)
End the "somewhere upstream" blindness with a reusable first-divergence
instrument (PRINCIPLES.md "find the first divergence; is the value WRONG or the
behavior MISSING"). A TCB-scoped, freeze-on-trigger control-flow ring with an
identical struct + JSON wire format on both processes, so a generic sequence-diff
(tools/parity_diff.py) locates the first logical divergence between recomp and
oracle — not a frame diff.

- parity_trace.{h,c}: armed-from-boot ring, gated on current_tcb==watched_tcb,
  latches on a configured trigger target so the pre-divergence window survives
  the wedge loop. Up to 6 watch-word snapshots/row. Self-contained TU shared by
  both binaries; frame source decoupled via parity_host_frame().
- fntrace.c: per-dispatch producer hook (cheap armed gate on the hot path).
- traps.c: YIELD control-event hook in the scheduler switch.
- main.cpp: env-armed from boot (PSX_PARITY_TRACE=1; TCB/trigger/watch tunable),
  defaults target the MMX6 cutscene wedge.
- debug_server.c: parity_dump + parity_ctl commands.
- tools/parity_diff.py: pulls both rings, sequence-diffs by (kind,pc,target).

First use (MMX6 freeze) confirmed MISSING behavior, not a wrong value: native
thread1 is permanently parked at func_8002000C resume (TCB EPC 0x800200F4),
handshake flag 0x8006D9AC never set (0 across 16384 rows), setter func_8001344C
never runs; thread1 re-resumes a dead func_8002000C frame whose saved-ra slot was
reused -> jr 0x800CD3F8 wedge. Upstream context-staleness/scheduling root.
Beetle mirror producer + the cross-process diff are the next step.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 16:38:38 -07:00
Matthew Stanley 0f7d0f1645 spike(scheduler): deterministic TCB scheduler LIVE + HLE/LLE toggle (minimal A+)
Replace the non-deterministic host-fiber thread-switch bridge with a deterministic
TCB scheduler (HLE_SCHEDULER_CARVEOUT_PLAN.md steps 2-8, minimal "A+" scope):

- psx_scheduler_run (traps.c): outer trampoline wrapping the top-level dispatch
  (main.cpp calls it instead of bare psx_dispatch). setjmp(g_scheduler_jmpbuf);
  restore current TCB / dispatch / on-escape reset g_psx_dispatch_depth +
  g_psx_call_bail; boot/pre-TCB runs raw cpu->pc; one-level switch-back safety
  net; fail-closed psx_is_dispatchable on every restored EPC.
- psx_request_thread_switch: save yielder ctx (resume=$ra), repoint
  dword_108->entry, longjmp to the scheduler. No fibers, no cpu->pc=0 signal.
  Fires only with in_exception==0, so it never unwinds a live exception_jmpbuf
  frame (the in-exception RFE path stays LLE/intact).
- sched_escape_ring (step 8) for always-on observability.
- Hidden toggle PSX_HLE_SCHEDULER (default 1=HLE, 0=legacy fibers), read once;
  psx_change_thread branches on it; startup banner prints the mode. The
  trampoline is transparent in LLE mode. Fiber funcs kept compiled-but-dead.

Validated live (HLE): thread1 trace shows ONLY the new TCB-switch events and
ZERO fiber kinds 10/11/13 — the per-frame fiber recreate is eliminated. Builds
clean. Freeze NOT resolved: same 0x800CD3F8 wedge persists (separate upstream
root, per ChatGPT Q5). Keep HLE default until the upstream cause is fixed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 15:52:51 -07:00
Matthew Stanley ab8ed36b47 spike(scheduler): step 1-2 scaffolding — structured-escape types + jmpbuf + fail-closed guard (INERT)
First step of the deterministic TCB cooperative-thread scheduler carve-out
(HLE_SCHEDULER_CARVEOUT_PLAN.md §9, ChatGPT-approved "A+"). Replaces the
non-deterministic host-fiber bridge (confirmed root of the MMX6 cutscene freeze:
thread1's fiber recreate-per-frame, pc=0 silent exit).

- runtime/include/psx_scheduler.h: psx_run_reason_t (CONTINUE / YIELD_TO_TCB /
  RESUME_CURRENT / GUEST_EXIT / FATAL) + psx_sched_escape_t, the design rationale
  (pure-TCB BIOS scheduler, no stack switch; guest state lives only in
  CPUState/RAM/TCB), and the load-bearing invariant.
- traps.c: g_scheduler_jmpbuf + g_sched_escape definitions + psx_is_dispatchable
  (fail-closed on pc==0 and the exception sentinel — the known-bad resume PCs the
  old pc=0 pathology produced).

INERT: nothing calls these yet; psx_change_thread_fiber (host-fiber bridge) is
still authoritative. Builds clean (traps.c only, no game-C regen). No behavior
change. Next: outer scheduler loop + route ChangeThread/RFE through structured
escapes, then remove cpu->pc=0 + fiber recreation.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 14:42:43 -07:00
Matthew Stanley 4ddccccaa9 checkpoint: exact-store-PC tooling fix (verified) + freeze/scheduler diagnostics + faithful-timing WIP
recompiler/code_generator.cpp: stamp g_debug_last_store_pc before EVERY game
store (SB/SH/SWL/SW/SWR, SWC2, + the ws-backdrop / game-option special paths)
and declare the extern in the game-C preamble. Previously only the BIOS emitter
(strict_translator.cpp) stamped it, so game stores inherited the last BIOS store
pc -> wtrace/readtrace mis-attributed game writes to the BIOS. VERIFIED LIVE:
MMX6's producer trace for the 0x800CD3F8 write now reports the real game store
pc 0x8001A708 (matches Beetle), not the stale 0xBFC11BF0.

runtime: g_pc0_reason slice-exit provenance, fail-closed exception-sentinel
asserts, named thread-event kinds, freeze-heartbeat RVA emission (diagnostics,
default-on, additive).

Carries the in-progress faithful-timing / MDEC work (cpu_state.h, memory.c,
dma.c, interrupts.c, dirty_ram_interp.c, full_function_emitter.cpp) as an
uncommitted-WIP checkpoint. tools/: regen_bios.sh, bios_emitter_fingerprint.sh.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 14:20:17 -07:00
Matthew Stanley 2e2795f147 fix(tomba2): route sub-FLOOR boot-text overlays through local-dirty-flow
Tomba 2 discards its boot/init code and loads a gameplay overlay OVER its
own boot-EXE text region (phys 0x1DB8C, below the 0x38800 text-end FLOOR).
The dirty-RAM interp's local-flow gates (is_local_dirty_target,
allow_local_dirty_flow, transfer-chain gate) keyed on OVERLAY_REGION_FLOOR,
so they misclassified that live overlay as immutable static text and routed
the splash loader's `jal 0x8001DB8C` through the bail-prone non-local-call
contract instead of interpreting the overlay locally. That produced a wild
`jr ra` reading a stale saved-ra stack slot (0x801FE994 = "T.BI", bytes 4-7
of START.BIN left by the file-open strcpy), killing the worker thread before
it could ChangeThread to the START.BIN loader -> Whoopee-Camp splash softlock.

Verified against ground truth: EXE + generated C have `move a2,a0` at
0x8001DB8C (mid func_8001DB7C); runtime RAM has a fresh prologue there
(page 0x1D dirty=true). Always-on wtrace_all ring pinned the clobbering
strcpy at pc=0x8008B9A8, frame 2068.

Fix: gate local-dirty-flow on DIRTY_RAM_KERNEL_WINDOW_END (0x10000) instead
of OVERLAY_REGION_FLOOR, via new helper phys_is_overlay_flow_region(). A dirty
page above the kernel window IS an overlay regardless of FLOOR; the inline
dirty_ram_is_dirty() check keeps clean boot text on the compiled fast path,
and the kernel window [0,0x10000) still stays per-block (exception context).
Runtime-only; no regen.

Result: worker no longer wild-jumps; it ChangeThreads to the loader thread
(TCB 0xA000E374); aborts=0; threads switch cooperatively. NEW failure point:
the loader thread does not read START.BIN data sectors (CD idle at directory
sector 373) -> state byte 0x1F80019C stays 0, splash not yet advancing
(next cascade stage).

Also includes the diagnostic tooling that localized this: insn_freeze_target,
insn_freeze_snapshot, ra_load_watch (debug_server.c / dirty_ram_interp.c) and
thread sys3 / dispatch-return traces (traps.c).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 23:16:15 -07:00
Matthew Stanley 9940ab2c0f wip(tomba2 exc): observability + corrected frame-1997 diagnosis (longjmp-to-outermost, NOT async)
Implemented + instrumented the consult's Option-A async-resume idea; the data REFUTED
the in_exception=0 async model:
- freeze_check now reports async_rfe_set/fire, reach_dirty/traps, reach_async,
  dirty_safe_resume_pc, async_rfe_resume_pc.
- Measured: reach_dirty=19921 (~1/exception), dirty_in_exc0=0, async_rfe_fire=0 ->
  the sentinel 0x80000048 ALWAYS arrives at the dirty gate with in_exception=1 and
  takes the longjmp. There is no in_exception=0 dispatch of the sentinel.

Corrected mechanism (docs/TOMBA2_WHOOPEE_BAIL.md): the game's ReturnFromException
longjmp lands in psx_check_interrupts (jmp_val==1), restores GPRs and returns with
cpu->pc=0; for ~19920 exceptions it lands at a NESTED dispatch (game continues), but
for the failing one it unwinds to the OUTERMOST trampoline where cpu->pc=0 reads as a
clean exit. It's a longjmp-landing-depth bug in the 'resume via host return' model,
not an async-resume bug.

Latches added (g_dirty_safe_resume_pc around the 3 dirty pumps, g_async_rfe_resume_pc
at dirty-safe exception entry) + the sentinel-gate redirect scaffolding are INERT
under the corrected model (never fire) — kept as tooling + scaffolding for the
candidate fix. Builds clean; behaves identically to pre-fix (exit at frame 1997, no
regression). wt/tomba2 only; not on master.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 23:16:13 -07:00
Matthew Stanley 35b98086b2 heartbeat: $ra->1 tripwire + first-bail exception context (confirm probe)
Confirm-first probe (recomp-debug "pin before fix" + ChatGPT Q5): latch the
exact first moment guest $ra transitions to 1 (the degenerate $ra=$sp=1 spin
state) with a site tag, plus whether the trigger bail happened in_exception.

RESULT — REFUTES the exception/EPC-overload theory:
  bail_first_in_exc = 0  (the B0->0x8008AE50 trigger bail is NOT during an exc)
  ra_tw: site=INTERP, in_exc=0, pc=0x8008B018, prev_ra=0x8008AF10 -> 1
  0x8008B018 disassembles to `lw $ra, 0x34($sp)` -- a function EPILOGUE.

So the $ra->1 is a normal (non-exception) epilogue restoring a CORRUPTED saved
$ra from the stack -- the stack/$sp is wrong after the first bail's non-local B0
return. It is a control-flow/stack-contract bug in the bail/flatten handling of a
B0 non-local (tail-call) return, NOT an exception/RFE/EPC bug. The tripwire saved
us from implementing the wrong (EPC-separation) core fix.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 23:16:10 -07:00
Matthew Stanley 732e093ab2 heartbeat: IRQ raise/deliver/ack + bail-source telemetry (Tomba 2 spin diag)
Always-on, heartbeat-resident diagnostics for the Whoopee-Camp splash spin:
- VBLANK raise/deliver/ack counters (interrupts.c + memory.c) + irq_deliver.
- Deduped bail-source ledger keyed by (call-site $ra, wild target) (traps.c,
  hooked at psx_call_contract in cpu_state.h) + heartbeat top-key/first-bail.

Result: REFUTES the exception-reentry-storm hypothesis. VBLANK is raised,
delivered AND acked exactly 1/frame (textbook-healthy); irq_deliver is 1/frame,
not a storm. So the freeze is NOT an interrupt/timer/CD problem -- the hardware
sim is healthy. The ~68k bails/frame are the GAME splash loop busy-polling a
kernel event (TestEvent/WaitEvent) that never fires -- the DeliverEvent/WaitEvent
discriminator is the next probe.

(The dominant dispatch-trampoline bails are emitted in generated BIOS code, so
the precise bail call-site needs a recompiler-emit change + BIOS regen; the
direct-call ledger here is partial. Deferred -- not needed to localize to the
event wait.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 23:16:08 -07:00
Matthew Stanley 94a1966444 recursion(fix): universal continuation-passing — FIXES the idle-freeze host-stack leak (BIOS+game)
The complete §25 fix: make BOTH recompiler emitters emit guest calls as
TAIL-TRANSFERS driven by the unified flat trampoline + the guest's own $ra/$sp,
instead of nesting one host C frame per guest call. The host stack can no longer
mirror unbounded guest re-entrancy — the architectural root of the ~14-min idle
freeze. Gated gen-time by PSX_CPS; legacy codegen byte-identical when unset.

VALIDATED (user): full-CPS Tomba boots BIOS -> FMV -> New Game -> village idle and
plays; ce_profile cur_max_kb FLAT at 5 KB from frame ~9.9k through 56,890 (turbo
soak) — blew past the ~50k-frame freeze threshold with the host stack bounded
(legacy climbed +1.17 KB/frame -> 48 MB -> freeze).

KEY CORRECTION to the prior session's §25.1: there are TWO emitters. The BIOS
(full_function_emitter.cpp) was done last session but the §8 leaking chain
(func_8001A954 ...) is GAME code, emitted by code_generator.cpp (via main_psx.cpp)
— never converted. Also, §25.1's "continuations not in the dispatch table"
hypothesis was wrong (they were always added); the real boot regressions were
found by dumping the always-on fntrace ring on the abnormal psx_dispatch return.

Emitter changes (gated PSX_CPS):
- full_function_emitter.cpp (BIOS): BIOS A0/B0/C0 vector runtime-fallback
  tail-transfers instead of nested psx_dispatch_call (was the frame-0 exit).
- code_generator.cpp/.h (game): jal/jalr/jr-$ra/jr-table/branch-split/j-split ->
  tail-transfer (jalr captures rs before the link, alias-safe); each call's
  return point registered as a dispatchable continuation; entry-switch (keyed on
  cpu->pc) at the top of every function AND every overlapping-alias body. 0
  psx_call_contract / call_by_address emitted under CPS.
- main_psx.cpp: psx_dispatch_game_compiled gains continuation cases
  (case cont: cpu->pc=cont; func_owner(cpu)); entries clear cpu->pc.
- strict_translator.cpp: syscall emits `if (psx_syscall(...)) return;` so a void
  syscall falls through to the inline post-syscall jr $ra (real-HW EPC+4).

Runtime changes (the legacy "return via nested dispatch / cpu->pc=0" idioms that
exit the flat CPS trampoline):
- traps.c: psx_syscall returns int (0=void/fall-through, 1=transfer);
  Enter/ExitCriticalSection return 0; case-3 ChangeThread-success returns 0 — the
  fiber scheduler leaves cpu->pc=0 on resume, which was the frame-712 exit.
- cpu_state.h / overlay_api.h: psx_syscall signature void->int; overlay callback
  ABI bumped 4->5 (stale overlay caches rejected).
- dirty_ram_interp.c: gate interp_enter_compiled behind PSX_HAS_GAME_DISPATCH
  (un-breaks the BIOS-only build).
- main.cpp: dump the always-on fntrace ring tail + stack to psx_cps_exit_trace.json
  on an abnormal psx_dispatch return (diagnostic; CLAUDE.md ring-buffer model).

Docs: RECURSION_BUG.md §25.2-§25.4. Remaining: dwarf->overworld playtest;
Stage 3 (sljit/overlay CPS parity); then consider making CPS default.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 12:45:02 -07:00
Matthew Stanley add57f5560 recursion: SOLVE root (gradual per-frame host-stack leak) + instruments + pin exact crossing
The ~14-min Tomba idle freeze is a gradual per-frame host-stack LEAK (not a
within-one-frame spiral, not the gradual-marking theory). Confirmed via a new
compiled-entry depth profile (§19): per-frame MAX guest-stack-used climbs dead-linear
+1.17 KB/frame from gameplay start to 48 MB on the 64 MB main-thread stack -> native
stack guard -> freeze. Once per frame an interpreted overlay (reached via
func_80046264's dirty hop jal 0x80063864) does jalr func_8001A954, nesting a host call
that never unwinds (the per-frame loop never returns). Full writeup RECURSION_BUG.md
§16-§21.

Instruments (always-on rings, runtime-only, no regen):
- §17 stack_profile: per-vblank host-stack sample (main.cpp; stack_profile TCP cmd).
- §18 xprobe / xprobe_arm: interp<->compiled boundary crossing flight recorder
  (dirty_ram_interp.c; per-frame summary + per-crossing detail).
- §19 ce_profile: per-compiled-entry depth profile + raw TEB at the trip
  (debug_server.c; ce_profile TCP cmd) — the trustworthy instrument; the regression
  check for the fix (max_kb must stay FLAT).
- PSX_STACK_GUARD_KB early-trip + PSX_FIBER_STACK_KB (debug_server.c / traps.c).

Fix progress:
- Block-loop tail-transfer surface (dirty_ram_interp.c ~1514): surface compiled
  tail-transfers to the trampoline instead of nesting. Correct for tail-transfers but
  RULED OUT as the leak (§19 climb unchanged) — the leak is the CALL path (jalr/jal).
- Real fix (next): surface compiled-game CALLS to the owner trampoline, gated to
  compiled-game targets only (a naive all-targets surface reintroduces the
  dwarf->overworld lost-continuation bug). Needs a non-nesting membership test
  (generated psx_dispatch_game_has) or §12 CD-DMA dirty-marking precision.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 00:15:15 -07:00
Matthew Stanley 2439d4dd44 dispatch call contract: continuations only run when the guest really returns
Bug D root cause (Tomba chest freeze, also QA issue #1): the game's own
dispatcher func_80056E08 has a dead jumptable case (type 0x21 ->
jal 0x80120B3C) whose target is mid-function in EVERY overlay variant on
the disc (only the boot EXE references it; the real handler entry is
0x80120AC8). Entering mid-function skips the prologue, runs an epilogue
for a frame that was never pushed, and returns through garbage. Our C
model then resumed the suspended generated continuations on the moved
guest stack -> double-executed tails -> infinite zombie loop (~3K wild
calls/frame) -> MMIO fatal. Family root of Bugs A/C/D.

The contract: a generated C continuation may only run if the guest
actually arrived at it ($ra == call site return address, $sp == caller
sp at the call). Violations begin a "bail" unwind: cpu->pc := the wild
jr's true target, every generated frame returns immediately, and the
unwind resolves at the first enclosing site whose contract matches (or
flattens at the outermost dispatch loop, re-dispatching the wild target
on a clean host stack -- matching real-HW one-stack semantics).

- cpu_state.h: bail state + psx_call_contract() helper (+ overlay-DLL
  pointer indirection under PSX_OVERLAY_DLL_BUILD)
- emitters: (ra, sp) checks at every BIOS and game jal/jalr site;
  psx_dispatch_impl validates pc==0 and stop-hits against (stop, sp),
  handles bail resolve/propagate/flatten
- dirty_ram_interp: surfaced calls validate the contract on return;
  bail aborts the interp run; tail transfers into compiled code now run
  with pc cleared (closes a latent double-run on plain C returns)
- overlay cache ABI v2: DLLs export overlay_abi(); loader rejects and
  deletes stale DLLs (autocompile regenerates with contract checks)
- telemetry: bail_first/resolved/flattened/anomaly in heartbeat + dump
  headers; anomaly guard at exception entry; TCP_COMMANDS.md docs

Verified: BIOS+Tomba regen, boot to intro at locked 60.0 fps, all bail
counters 0 through boot/title/attract (no false positives). Chest-area
gameplay verification pending (requires user play).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 22:30:41 -07:00
Matthew Stanley 2d6a75d9a2 tooling: fatal halt-and-serve + always-on GP1 display ring
- psx_fatal_halt: deliberate fatal sites (unhandled MMIO access,
  trap_crash) now write the structured crash report AND a full
  freeze-style ring dump (wedge kind "fatal"), then keep pumping the
  TCP debug server forever in debug-tools builds so every ring stays
  queryable against the exact crash state (ring-buffer-first: crashes
  are interrogated, not snapshotted). Release builds exit(1) as before.
  Re-entry guarded; heartbeat JSON gains a "fatal" field.
- gp1_dump: dedicated always-on GP1 (0x1F801814 display control) ring,
  512K entries (~15 min of gameplay) with frame-window server-side
  filtering. The general MMIO ring rolls over in under a minute of SPU/
  DMA traffic, which kept evicting boot-window display history before a
  probe could ask "who toggled the display during the logo" (bug 1).
- mmio_dump/mmio_clear documented in TCP_COMMANDS.md.

The halt-and-serve path is what made the pig-crash root cause a
one-repro diagnosis.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 12:17:10 -07:00
Matthew Stanley ec4306fbb3 mmx6 framework: CD/DMA/MDEC + codegen fixes (Tomba builds+plays, MMX6 boots to intro)
Framework work on codex/mmx6-support that makes Tomba build and play
(title screen, 0 dispatch misses) and MMX6 boot through BIOS to its
intro. Touches recompiler codegen (code_generator, function_analysis)
and runtime CD-ROM/DMA/MDEC/GPU/SIO timing (cdrom, dma, mdec, gpu,
psx_cycles, traps, fntrace, debug_server) + headers + runtime.cmake.

Milestone snapshot for collapsing onto master. Build dir excluded.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 12:06:02 -07:00
Matthew Stanley b9716154f0 Add native macOS/Linux support (ucontext thread scheduler, fullscreen, build fixes)
The recompiler and runtime now build and run natively on macOS (Apple
Silicon & Intel) and Linux, alongside Windows. The recompiled BIOS boots to
the PlayStation logo, and Tomba boots into gameplay with working memory-card
save/load — the previously Windows-only path.

Core change — the BIOS thread scheduler:
- runtime/src/psx_fiber.{h,c} (new): one cooperative-fiber API, backed by
  Win32 Fibers on Windows and ucontext (makecontext/swapcontext) on POSIX,
  with a GetCurrentFiber analog (psx_fiber_current).
- traps.c: the BIOS thread switcher now uses psx_fiber_* on every platform.
  The old non-Windows setjmp/longjmp fallback is removed — it ran target
  threads nested on the caller's C stack and so couldn't resume an arbitrary
  suspended thread, which hung the BIOS CD-boot WaitEvent handoff. Real
  per-thread stacks (fibers/ucontext) fix it.
- interrupts.c: the exception-deferral path (g_exception_owner_fiber,
  deferred_exception_longjmp) is now fiber-based on all platforms via the
  same abstraction, instead of #ifdef _WIN32.

Cross-platform build fixes:
- recompiler/CMakeLists.txt: vendored fmt 9.1.0 force-enables consteval on
  Apple clang (the __cpp_consteval clause overrides its own Apple exclusion),
  which clang 21 then rejects; force FMT_CONSTEVAL empty on non-MSVC.
- runtime.cmake: add psx_fiber.c; add pkg-config's SDL2_LIBRARY_DIRS so
  -lSDL2 resolves outside default paths (Homebrew /opt/homebrew/lib).
- crash_trace.c: include <stdarg.h> (not pulled in transitively off Windows).
- main.cpp: the forced "opengl" render-driver hint (a Windows-GDI-freeze
  workaround) is now Windows-only so macOS uses Metal; create the renderer
  with PRESENTVSYNC; SDL_RenderSetLogicalSize(640,480) so fullscreen
  letterboxes; add an F11 / Alt+Enter / Cmd+F fullscreen toggle.

Verified: SCPH1001 BIOS boots to the Sony logo on Apple Silicon, and Tomba!
(SCUS-94236) boots into gameplay with memory-card saves persisting across
relaunches. Gamepad (SDL_GameController) was already cross-platform.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-29 18:53:22 -07:00
Matthew Stanley 291e10f00d checkpoint: document Tomba gameplay milestone 2026-05-17 16:49:01 -07:00
Matthew Stanley f6c15de880 checkpoint: options/new-game diagnostics handoff 2026-05-17 11:29:26 -07:00
Matthew Stanley fdbd2b515f runtime: cross-fiber longjmp fix + decouple vblank-present from MMIO read
Two distinct bugs were causing OPTIONS-black hard freezes after 12K-65K
frames:

1. interrupts.c + traps.c: exception_jmpbuf is a single global. setjmp
   captures the running fiber, but longjmp may be requested from a
   different fiber via psx_exception_longjmp / psx_restore_state_escape
   during chain-handler dispatch. Cross-fiber longjmp restores RSP to
   the wrong stack while Win32 still tracks the calling fiber, corrupting
   fiber state until SwitchToFiber eventually hangs.

   Fix: track g_exception_owner_fiber at setjmp time. New
   deferred_exception_longjmp checks the current fiber; if it's not the
   owner, sets a pending flag and SwitchToFiber's back to the owner.
   psx_change_thread_fiber's post-SwitchToFiber path observes the flag
   and executes the longjmp on the correct stack.

2. gpu.c: gpu_read_gpustat fired gpu_vblank_tick() after 1000 polls,
   which invoked sdl_vblank_present (SDL_Delay -> Sleep) from inside
   an MMIO read code path. That meant recompiled MIPS spent wall time
   in Sleep deep inside the fiber call tree -- confusing to reason
   about and a contributor to fiber/longjmp interactions.

   Fix: synthetic VBLANK from a poll-storm only updates emulation state
   (LCF + I_STAT). The host-side render+sleep happens only at the proper
   cycle-paced VBLANK trigger in psx_check_interrupts.

Also adds: dirty_block ring slice to the freeze auto-dump, Win32 main-
thread stack capture for wedge_kind=1 hard freezes (SuspendThread +
StackWalk64 + SymFromAddr), and dbghelp link.

Validated: TombaRecomp psx-runtime ran 30+ minutes / 105K+ frames at
OPTIONS with no hard freeze. Previous freezes consistently hit at
11900, 20493, and 64596 frames.

Diagnostic tooling under tools/_options_session/ that produced this
diagnosis is included.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 12:49:13 -07:00
Matthew Stanley cf05a0ab1f runtime + recompiler: Tomba boot through BIOS into Whoopee Camp FMV
Recompiler — `recompiler/src/function_analysis.cpp` now stops a
function's end_addr at its mapped `jr $ra` + delay slot instead of
stretching through trailing data. Fixes Tomba's 0x80076DB0, a real
136-byte function that was being emitted as a psx_unknown_dispatch
stub because data after it poisoned the boundary heuristic.
`code_generator.cpp` emits `func_<next>(cpu); return;` as a tail
call when a mid-function seed split leaves the physical fall-through
in a different C function, plus a `debug_server_log_call_entry` at
every function entry. `full_function_emitter.cpp` splits dispatch
into `psx_dispatch_impl(cpu, addr, stop_addr)` with public
`psx_dispatch` and new `psx_dispatch_call(cpu, addr, return_addr)`
so JAL/JALR into RAM stubs can return cleanly without growing the
native stack.

Runtime — `runtime/src/dma.c`/`include/dma.h`/`src/memory.c` now
expose `dma_write_masked()`; the 16-bit and 8-bit mmio paths route
DICR/DPCR/CHCR partial writes through it so a partial DICR write no
longer accidentally clears unrelated channels' pending IRQ flags,
and CHCR only triggers DMA when the written mask actually includes
the start bit. Unified `complete_transfer(ch)` across all channels,
removed `prev_master_flag` edge detection.

MDEC (`runtime/src/mdec.c`, `include/mdec.h`) gains a full debug
surface: `MDECDebugState` snapshot, an always-on 4096-entry
`MDECDebugEvent` ring with 11 event kinds (reset, ctrl_write,
cmd_begin/done, decode_done, dma_in/out start/end, output_drained,
read_underflow), and `mdec_state` / `mdec_trace` / `mdec_trace_clear`
debug-server commands. DMA ch0/1 emit start/end events around MDEC
transfers.

CDROM (`runtime/src/cdrom.c`, `include/cdrom.h`) — read stream
separated from one-shot pending state (`read_cmd`, `read_delay`,
`start_read_stream`, `stop_read_stream`, `deliver_read_sector`);
`cdrom_state`/`cdrom_trace` report the new fields.

Thread/fiber tracing — `traps.c`'s Win32-fiber TCB save/restore now
emits 12 thread-event kinds; new `ThreadTraceEntry` ring (64 K) plus
`thread_trace` / `thread_trace_clear` debug-server commands report
both live CPU regs and the target TCB's full saved register block.

wtrace / mmio rings — `WriteTraceEntry` grew (cpu_pc, sp, v0/v1/a0-a3/
t0/t1); capacity 1 M -> 4 M. `MmioTraceEntry` grew (pc, cpu_pc, sp,
a0-a3, sr, epc, i_stat, i_mask). Both dump commands accept
`count`/`newest=1` for reverse iteration. `restore_trace` and
`dirty_block_log` emit extended register fields (a2/a3/sp).

Misc — `interrupts.c` polls debug-server every 16K checks so it
stays responsive without a separate thread. `psx_runtime.h`'s
`call_by_address` routes through `psx_dispatch_call`. Dirty-RAM
interp JAL/JALR consult `psx_dispatch_game_compiled` when the game
is wired in. iso_reader gains the entry points needed by the game
disc path. New `tools/_tomba_probe.py` for port-4470 health checks.

Generated — regenerated `SCPH1001_dispatch.c` against the new
`psx_dispatch_impl` shape.

ISSUES.md — opened Issue #3 (BIOS PS-logo glyph missing on the
disc-detected screen, cosmetic).

.gitignore — added `recompiler/build-asan*/`, `build-checked/`,
`__pycache__/`.

Validated end-to-end: BIOS boots, disc detected, Whoopee Camp FMV
plays with audio. Frame count advances through ~40K frames without
freezes. `unknown_dispatch_log.total = 0`. Remaining FMV cluster
issues (1/4-size render, FMV-2 audio missing, no skip) tracked
under TombaRecomp/ISSUES.md.

Co-Authored-By: Codex (OpenAI) <noreply@openai.com>
2026-05-11 12:34:49 -07:00
Matthew Stanley 632e5dc662 runtime: BIOS thread switching via Win32 fibers + debug_server CDROM/dirty-RAM
traps.c — ThreadWaitFrame setjmp stack plus a Win32-fiber-based
HostThreadFiber pool for BIOS thread context switches. TCB save/restore
maps SR low-4-bits → low-6-bits via the standard PSX exception entry
shift. Backs the BIOS OpenTh/ChangeTh/ReturnFromException paths that
the kernel uses to schedule its own threads on top of the recompiled
exception handler.

debug_server.c — new TCP commands:
- cdrom_state: snapshot of index/stat/irq regs, FIFOs, pending cmd,
  seek/read MSF, i_stat.
- cdrom_trace: paginated dump of the cdrom_trace ring (kind/addr/val/
  func/pc/frame + state snapshot per entry).
- dirty_ram_unsupported: counts and last PC/insn/reason for opcodes
  the dirty-RAM interpreter rejected.
- dirty_ram_stats now also exports the full bitmap as
  dirty_bitmap_words[] (matches the bitmap promotion in this branch).

Both surfaces are query-only against always-on ring buffers, no
log files or stderr emission — consistent with the
no-stdio-debug rule.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 22:51:01 -07:00
Matthew Stanley cf87fd8fc6 crash_trace: structured post-mortem dump (mirrors SMW recomp pattern)
New runtime/src/crash_trace.c writes psx_last_run_report.json with:
  reason / timestamp / frame / last_func_addr / last_store_pc
  full CPU register state (all 32 GPRs + hi/lo/sr/cause/epc)
  dispatch_ring tail (last 256 dispatched func entries)
  unknown_dispatch tail (last 50 entries with addr/phys/ra/a0/a1)
  dirty_block_log tail (last 100 entries)
  Windows SEH info (when triggered from SEH filter)

Triggered from:
  - SIGSEGV / SIGABRT signal handlers
  - Windows SetUnhandledExceptionFilter (catches access violations)
  - atexit
  - psx_unknown_dispatch fail-fast path (set PSX_FAIL_FAST_UNKNOWN_DISPATCH=1)
  - trap_crash (existing fatal trap path)

File overwrites per dump (last-write-wins). Per CLAUDE.md §3 this is not
a log — it's a one-shot final state snapshot for crashes the running
TCP server cannot intercept.

Pattern mirrors F:/Projects/SuperMarioWorldRecomp/src/post_mortem.c.

Used the new infrastructure to capture a controlled crash with
0xBFC0989C re-seeded. Crash dump revealed the chain:
  - JAL at 0xBFC09788 is jalr $s3, NOT jalr $a0
  - $s3 was set in func_1FC09598 prologue: $s3 = $a2 (callee saves arg)
  - Caller at 0xBFC093C4 in func_1FC092EC: $a2 = $s3 - 128 (delay slot)
  - func_1FC092EC at 0xBFC09314: $s3 = $a3 (callee saves arg)
  So real bug is upstream-of-upstream caller passing $a3 = 131 (0x83)
  to func_1FC092EC, propagating down to a misaligned function pointer.
  The 0xA000E1E8 we suspected as a function-pointer table was a
  red herring — $a0 in the loop body is loaded from $fp+s1*4 (the
  ARG to pass to the called fn), not the call target.

0xBFC0989C reverted out of seed list (was diagnostic-only).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 23:45:33 -07:00
Matthew Stanley 6889f5c5d2 recompiler: seed missing modal_stub function (BFC1A1BC), add unknown_dispatch_log ring
Modal close on YES/NO COPY confirm now works on recomp. Cause: function
discovery (Ghidra + transitive walk) missed the function at 0xBFC1A1BC
which the BIOS shell calls indirectly during modal close. The 5 nearby
"missing" PCs the diagnostic identified are all in the same gap from
0xBFC1A1B8 to 0xBFC1A3B8; seeding 0xBFC1A1BC discovers the rest
transitively.

Status: modal closes cleanly on X. Actual copy execution still has
downstream missing-function-discovery issues (3 more PCs hit
psx_unknown_dispatch on copy: 0xBFC09890, 0xBFC0989C, 0xBFC02918)
plus likely additional bugs that softlock when those are seeded.
Those remain for follow-up; this commit is the modal-close milestone.

This is a BANDAID per user permission. The proper fix is to improve
function discovery so missed functions like 0xBFC1A1BC are found
automatically. Ghidra didn't see this function, and our transitive
walk didn't reach it either.

Tooling additions:
  - psx_unknown_dispatch_record() ring buffer + unknown_dispatch_log
    debug command. Replaces the prior file-based miss log per
    CLAUDE.md §3 (no log files, ever). Always-on; queryable summary
    (per-target counts) or tail (last N entries with addr/phys/ra/a0/a1).

  - WTRACE_MAX_RANGES bumped from 16 -> 64 (boot defaults were
    crowding out investigative arms).

  - DirtyRamBlockLogEntry +a0/a1 fields, ring CAP 16K -> 4M
    (previous 28ms eviction made retroactive press-window analysis
    impossible).

Diagnostic scripts (kept):
  tools/_unknown_dispatch.py    — query unknown_dispatch_log
  tools/_dispatch_check_v3.py   — was a PC ever dispatched (RAM->ROM map)
  tools/_dispatch_tail.py       — last N dispatched targets
  tools/_freeze_diag2.py        — softlock investigator

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 22:13:06 -07:00
Matthew Stanley 82aeca98a7 feat: embed Beetle PSX libretro core as golden oracle for RAM comparison
Adds a new psx-beetleoracle build target that embeds beetle-psx-libretro
(mednafen PSX core) alongside the recompiled BIOS. Both execute the same
SCPH1001.BIN and advance in lock-step, enabling byte-level RAM comparison
via the TCP debug server.

New infrastructure:
- psx_oracle_backend.h: abstract oracle backend interface (init, run_frame,
  get_ram, get_vram, get_scratchpad, read_byte/word, get_cpu_regs, sync_to_state)
- beetle_psx_bridge.cpp: libretro bridge with CPU register access via
  PSX_CPU->GetRegister(), VRAM via GPU_get_vram(), scratchpad via ScratchRAM
- psx_oracle_cmds.c: TCP commands — find_first_divergence, emu_read_ram,
  emu_read_vram, emu_read_scratchpad, emu_cpu_regs, emu_step, emu_sync,
  emu_sync_press
- psx_oracle_stub.c: no-op stubs for non-oracle builds
- tools/oracle_diff.py: standalone Python RAM comparison tool
- tools/build_oracle.sh: build script for beetle-psx + oracle target

Also includes: SIO card probe diagnostic counters (mc_probes, mc_acks,
mc_cmds, tx_writes), enhanced dispatch miss logging with register context,
memory_get_ram_ptr() for oracle RAM comparison.

Build: cd beetle-psx && make platform=mingw_x86_64 STATIC_LINKING=1 HAVE_LIGHTREC=0
       cd runtime/build && cmake --build . --target psx-beetleoracle

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-25 00:15:01 -07:00
Matthew Stanley e08bf602c0 fix: raise dispatch miss limit to 100K to prevent false crash on printf label misses
The 10K limit was hit by printf continuation labels (BFC018D0, BFC00F8C)
which fire repeatedly during normal shell operation. These are benign
dispatch misses — the trampoline reader cannot resolve them because
they're mid-function labels, not trampoline patterns. Raising to 100K
prevents false runtime termination while still catching real dispatch
storms.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-24 19:26:49 -07:00
Matthew Stanley 70d39c475e fix: oracle JR/JALR delay slots + RestoreState longjmp-return + VSync mechanism seeds
Oracle interpreter:
- JR and JALR were missing delay slot execution (exec_one recursive call),
  causing the entire boot sequence to malfunction — C0 table never populated,
  oracle stuck at ~65K instructions. Now boots to shell and VSync incrementer
  hit 272 times in 100M instructions (counter = 271).
- Added NULL-PC halt trap (jr to address 0 = corrupt function pointer).
- Reworked oracle main loop: counts VSync hits, steps past breakpoints,
  dumps trace ring correctly (ring-relative indices), reads VSync counter
  and init flags at end of run.

Code generator (longjmp-return):
- RestoreState (A0:0x14) loads $ra from a save buffer via $a0, so jr $ra
  must set cpu->pc for the dispatch loop to tail-call to the restored
  address. Detected by: first instruction is `lw $ra, 0($a0)` (0x8C9F0000).
- Applied in both code_generator.cpp (block-based) and full_function_emitter.cpp
  (linear emitter used by psxrecomp-bios.exe).

Dispatch seeds (+5 new, 832→837):
- 0xBFC421D8: RestoreState longjmp target (mid-function in func_1FC42160)
- 0xBFC4231C: SaveState v0=0 init body (callback table setup)
- 0xBFC42370: func_1FC42160 epilogue (init flag gate branch target)
- 0xBFC42910: post-callback cleanup (was dispatch miss)
- 0xBFC42BF0: shell_register_callback (callback table was empty without it)

Runtime survivability:
- psx_unaligned_access: survivable ADEL with diagnostic dump (was exit(1))
- DMA ch2 linked-list: break on cycle instead of exit(1)
- NULL dispatch (addr=0): silently absorb in psx_unknown_dispatch

Regenerated BIOS: 1056 dispatch entries (was 1051).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-22 00:21:18 -07:00
Matthew Stanley b3833aa253 fix: exception handler chain walk + shell reaches Memory Card/CD-ROM screen
Root cause: the recompiler split the kernel exception handler at the
jalr $s1 (chain handler dispatch) into func_00000C80 and func_00000E10.
The chain-walker continuation at BFC10910 never executed — only the
first handler in chain 0 was ever called.  I_STAT bits 0 (VBlank) and
2 (CDROM) were never acknowledged, causing an interrupt livelock.

Fixes:
- Emitter jalr/jal safety-net: when the return address falls outside
  the current function, emit a continuation psx_dispatch call after
  the jalr dispatch.  This bridges split functions at indirect calls.
- Merged exception handler: removed BFC10910 seed so the chain walker
  continuation is part of func_00000C80 (C fall-through after jalr).
- 0xBFC10910 dispatch special case in psx_unknown_dispatch: chain
  handlers return to $ra=BFC10910 via jr $ra (compiled as `return;`);
  the dispatch loop picks up the $ra value — handled as a no-op since
  the continuation already ran via C fall-through.
- Post-exception adaptive cooldown: if the handler didn't clear any
  I_STAT bits, give main code 500 blocks before re-firing.  Prevents
  VBlank livelock when no handler claims the interrupt.
- SIO byte write handler in mmio_write8 (was missing, caused crash).
- GPU→RAM DMA ch2 stub: fills target with zeros, completes transfer.
- Dispatch trace (debug_server_trace_dispatch) added to emitter
  template so it survives regeneration.

Result: shell device detection completes, StartPad dispatched, chain
E014 populated, GPU actively rendering the Memory Card / CD-ROM
screen.  VSync counter still 0 (handler installed but not yet
incrementing — next investigation).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-21 18:13:51 -07:00
Matthew Stanley 8e81afd111 fix: conditional branch fall-through + Tier 1 reverse debugger
Recompiler bug fix: full_function_emitter treated conditional branch
delay slots at function boundaries as fully resolved control flow,
suppressing the fall-through tail call for the "not taken" path.
This broke 15 cross-function fall-throughs across the BIOS, most
critically the find-free-EvCB-slot function (BFC11800) whose search
loop was silently skipped — causing OpenEvent to return garbage handles
(0xF100E1E8) instead of valid slot indices (0xF1000000..0xF1000005).

Fix: only suppress fall-through for unconditional flow (j/jr/jal/jalr/rfe).
Conditional branches (beq/bne/blez/bgtz/bltz/bgez) now always emit
`cpu->pc = next_addr; return;` for the not-taken path.

Also included:
- Tier 1 reverse debugger: 1M-entry write trace ring (32 MB),
  64K-entry MMIO trace ring (2 MB), 8-slot multi-range address
  filtering, function attribution via g_debug_current_func_addr.
  6 new TCP commands: wtrace_add/del/ranges/stats, mmio_dump/clear.
- 3 new dispatch seeds for functions exposed by fall-through fix
  (SIO epilogue, exception chain walker, GTE init).
- Diagnostic tools: analyze_sio_trace, decode_insn, decode_jt, mmio_tally.
- Pattern 5 trampoline fix + PLAN.md Phase 4 status update.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-21 14:23:09 -07:00
Matthew Stanley fba58d76b3 Phase 4 interrupt fixes: RFE, GPR save/restore, DMA IRQ edge-trigger
Three interrupt delivery bugs fixed that were blocking shell progression:

1. RFE after handler exit: The exception handler's normal exit path
   (jmpbuf longjmp at 0xBFC10944) does not execute RFE. In our model
   the handler returns to psx_check_interrupts, not to the WaitEvent
   caller. Without RFE, IEc stays 0 forever after the first interrupt.
   Fix: apply RFE in psx_check_interrupts if IEc is still 0 after the
   handler returns.

2. GPR save/restore around handler dispatch: The handler's jmpbuf exit
   restores callee-saved registers from the kernel jmpbuf (intended for
   a non-local jump to WaitEvent's caller). In our model this corrupts
   the interrupted code's registers. Fix: save all GPRs before handler
   dispatch and restore them after, matching real hardware where the
   interrupted code's registers are preserved via TCB save/restore.

3. VBlank tick gated on !in_exception: The handler runs thousands of
   block leaders, each calling psx_check_interrupts. Without gating,
   dispatch_count advances inside the handler, firing new VBlanks that
   stack up and starve the main code. Fix: only increment dispatch_count
   and fire VBlank/timer ticks when not in exception handling.

4. DMA IRQ edge-triggered: update_master_irq() was level-triggered,
   re-asserting i_stat bit 3 on every call while master_flag=1. Combined
   with no BIOS callback to acknowledge DMA IRQs, this caused an
   interrupt storm. Fix: edge-detect master_flag 0->1 transition, and
   suppress DMA completion IRQs entirely since transfers are synchronous.

Also adds GPU debug diagnostics: GP0 command counters (nop/fill/draw/
env/copy), draw area/offset in gpu_state, vram_peek debug command.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 20:29:03 -07:00
Matthew Stanley a1165d51b7 Phase 2 complete: BIOS boots to GPUSTAT read
Full recompilation pipeline working:
- Recompiler emits 799 functions (46k instructions) from SCPH1001.BIN
- Tail-call trampoline in dispatch loop prevents stack overflow
- Timer registers (3 counters, mode, target) implemented
- Interrupt check at labels + dispatch boundaries with vblank scheduling
- SYSCALL 1/2/3 (Enter/ExitCriticalSection, ReturnFromException) handled
- EPC correctly set from SYSCALL instruction address

BIOS boot reaches first GPU hardware access:
  UNIMPLEMENTED MMIO READ32 (GPUSTAT) @ 0x1F801814

Phase 3 (Sony logo) starts here: GPU, DMA, display pipeline.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 00:03:46 -07:00