Commit Graph

32 Commits

Author SHA1 Message Date
Matthew Stanley 4c39958226 dispatch: Class-A shell-window fix covers OVERLAY code — Tomba2 Whoopee-logo wedge
Root cause of the fresh-launch logo freeze (present at every commit in the
stack; window-capture-verified 13/13 repros, never a regression): the BIOS
dispatch normalize() maps RAM phys 0x30000-0x5AFFF to the shell ROM alias.
The Class-A fix routed post-game-start shell-window targets to the game/dirty
path first, but its gate also required psx_game_address_in_text() — which
exempts runtime-loaded OVERLAY code above the boot-EXE text end. Tomba2's
text ends at 0x38800; its post-logo main-thread resume PC 0x50B80 fell
through to normalize() and executed func_1FC38B80, a stale shell-image
data-as-code self-loop, starving the main thread forever (one VBlank
delivery per frame at the same EPC, zero main-thread instructions retired;
the game's ISR ran fine, so TCP frame counters kept ticking — the freeze
was invisible to every telemetry-only validation). Even PSX_FORCE_INTERP
missed it: normalize() shadowed the address before the interp path saw it.

Fix: post-game-start, the WHOLE shell-overlap window routes to
dirty_ram_dispatch before normalize() — real hardware executes the RAM
bytes regardless of who loaded them (boot EXE, overlay loader, DMA stream).
Pre-start shell addresses still normalize to the shell as before.

Also:
- interrupts/debug_server: nestgate_* per-guard veto-attribution counters on
  the nested-delivery gate (freeze_check), used to exonerate the IEc/SR
  guards and redirect the hunt from interrupts.c to dispatch.
- tools/window_capture.py: PrintWindow-based SDL-window capture with
  LIVE/FROZEN series verdict — window-liveness ground truth; TCP/VRAM
  screenshots provably lie about presentation (this bug's entire history).

Validated: fresh launch now runs logo -> intro FMV -> title -> attract with
sustained full-frame motion (window-level capture series).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 09:04:17 -07:00
Matthew Stanley 46d7dd4646 Reapply "interrupts: hardware-faithful nested IRQ delivery (SR-gated), not in_exception veto"
This reverts commit 06cab1da46.
2026-07-03 08:44:22 -07:00
Matthew Stanley 06cab1da46 Revert "interrupts: hardware-faithful nested IRQ delivery (SR-gated), not in_exception veto"
This reverts commit d23858b6b1.
2026-07-02 22:55:55 -07:00
Matthew Stanley d23858b6b1 interrupts: hardware-faithful nested IRQ delivery (SR-gated), not in_exception veto
Real R3000A has no 'in exception' delivery gate — SR alone governs. The
handler runs with IEc=0 from the entry bit-shift, so the SR gates already
block normal re-entry; a game that RE-ENABLES IEc inside the handler is
asking for a nested interrupt and hardware delivers it. Tomba2's lava
attract demo parks in the kernel event-poll INSIDE the VBlank handler
waiting for the NEXT VBlank: the unconditional in_exception veto starved
it to ~1fps (VBlank pending+unmasked, ~1100 reentry blocks/s, 1 delivery/s
via a fallback). Nested delivery is allowed when the guest's SR has IEc=1,
with two safety refinements (first attempt wedged boot at f436):
  - never nest while an RFE/escape unwind is in flight (the guest's RFE
    pops SR before the host escape completes);
  - one nesting level (exception_nest_depth cap), counted when blocked.
The synchronous-handler machinery is nesting-safe (saved_gpr/hi/lo and
owner state are host-stack locals); in_exception now restores to its
pre-delivery value at the epilogue so a nested exit leaves the outer
level flagged. Validated: lava demo 1fps -> 60fps, full attract at 60.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 22:42:26 -07:00
Matthew Stanley 496d1dbbc8 runtime: HLE tier wiring + carried instrumentation from the card-load hunt
HLE tier wiring (completes the previous commit — HEAD builds again):
- main.cpp: [runtime] bios_hle/keep_intro consume + settings.toml layer +
  PSX_BIOS_HLE / PSX_BIOS_HLE_KEEP_INTRO env resolution + backend banner;
  boot-turbo window ORed into the turbo predicate; the fast_boot snapshot
  restore/capture path and s_fast_boot_active are removed (deprecated by
  the HLE boot shell-skip); psx_hle_scheduler_set_default call.
- traps.c: psx_hle_scheduler_enabled default now flows from config
  ([runtime] hle_scheduler) with the PSX_HLE_SCHEDULER env winning.
- debug_server.c: hle_dump command (always-on HLE ring query).

Carried instrumentation/state from the MMX6 card-load sessions (kept
deliberately; documented in memory + handoff):
- dirty_ram_interp.c: XOP_RES interp JAL call-resolution probe + xprobe
  watched dump section (emitted first), watch targets.
- overlay_loader.c: probed wrapper around dispatch_call + psx_sljit_call.
- debug_server.c: card_mgr_trace range widen, TestEvent spin-body
  collapse, 64MB response cap.
- interrupts.c: PSX_SAME_THREAD_RESTORE selector (0/1/2; TCB-equality
  hardening candidate).
- memory.c: PSX_POLL_PROOF temp gate (remove when card work resumes).
- savestate.c/.h (new): user save states (F1-F12), HLE-scheduler-gated
  load via psx_scheduler_resume_at; load-crash known, do not use for
  repros yet.
- psx_cycles/autocompile/cycle_testrom: timing-fastpath + exe-path
  session state.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 16:27:57 -07:00
Matthew Stanley 13c5e0c5b5 stale-recompiler shard guard + exception-exit ring; Tomba menu wedge resolved
Root cause of the Tomba pause-menu wedge (loaded saves only): autocompiled
overlay shards were emitted by a recompiler binary OLDER than the last emitter
change, yet stamped with the CURRENT cg tag (compile_overlays derives the tag
from --runtime-include; nothing tied it to the emitting binary). The stale
shards corrupted the display-list task queue during load-game. NOT the
IRQ-resume class the prior handoff claimed - disproven with the new ring.

- recompiler: bake the emitter-source hash (shared canonical list
  runtime/codegen_hash_sources.cmake + hash_codegen.cmake) into psxrecomp-game;
  new --codegen-hash flag prints it (machine-parseable, pre-banner).
- compile_overlays.py: HARD-FAIL when the recompiler binary's baked hash
  differs from the cg tag hash it stamps (or the binary lacks the flag).
  Applies to both cache and --static modes. Closes the read==write-tag /
  stale-content drift class structurally.
- runtime.cmake: codegen-hash source list moved to the shared canonical file.
- interrupts.c: always-on exception-EXIT half added to irqctx_ring (take_pc,
  real_epc, exit_pc, escape reason, same-thread/restore flags, v1/ra
  saved-vs-exit, RestoreState redirect count) - proves per delivery which
  escape path fired and whether the interrupted GPRs were restored. Kept the
  same_thread_resume GPR-restore refinement (ring-verified equal-value in
  practice; faithful: hardware restores all GPRs on every exception return).
- debug_server.c: irqctx_ring dump extended with the exit-half fields.

Validated live: Tomba menu opens on loaded save (Stormy Mountain) with the
cache fully enabled after regenerating shards with a rebuilt recompiler
(build-cosim + build-prod caches). Guard verified positive and negative.
MMX6 gate passed: boot -> title -> opening cutscene -> gameplay handoff OK.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-01 19:50:26 -07:00
Matthew Stanley 553d9933d1 fix mmx6 timing divergences with cosim oracle 2026-07-01 09:28:50 -07:00
Matthew Stanley 18c94a1394 device_trace: general two-process device-event (IRQ) cycle ring
New always-on ring (device_trace.{c,h}, compiles into both binaries like
parity_trace) recording every hardware IRQ-raise edge with its guest cycle,
source (I_STAT bit) and detail. All native device sources funnel their raise
through one choke point psx_irq_raise(bit,detail) in interrupts.c (VBLANK/GPU/
CDROM/DMA/timers/SIO routed; DMA passes DICR channel flags, timers pass index),
so the device-timing audit sees every event from one place. devtrace_dump/ctl
debug commands (cycle-window + source filters) + tools/devtrace_diff.py align
the two device-event timelines by source ordinal and flag where cycles drift.
The Beetle oracle side (mednafen IRQ_Assert rising-edge hook) lives in the main
checkout.

This is the level below parity_trace: parity names the divergent control-flow /
gate input; device_trace names the divergent HARDWARE EVENT behind it.

First result (MMX6 cutscene asset-load over-fast wedge): the load's CD+DMA
activity is compressed into ~232M cycles on native vs ~384M on Beetle (~1.65x),
completing the func_8001CB3C scene setup ~100M cycles / ~176 frames early. The
producer is CD-read / DMA-completion timing (DMA channels 2/3/4 = GPU/CD/SPU
asset streaming) -- confirming the device-completion-latency hypothesis and
ruling in the CDROM/DMA cycle model as the over-fast subsystem. (VBlank verified
correct: native VBlank edges are exactly 564480 cyc apart.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 21:53:27 -07:00
Matthew Stanley 4ddccccaa9 checkpoint: exact-store-PC tooling fix (verified) + freeze/scheduler diagnostics + faithful-timing WIP
recompiler/code_generator.cpp: stamp g_debug_last_store_pc before EVERY game
store (SB/SH/SWL/SW/SWR, SWC2, + the ws-backdrop / game-option special paths)
and declare the extern in the game-C preamble. Previously only the BIOS emitter
(strict_translator.cpp) stamped it, so game stores inherited the last BIOS store
pc -> wtrace/readtrace mis-attributed game writes to the BIOS. VERIFIED LIVE:
MMX6's producer trace for the 0x800CD3F8 write now reports the real game store
pc 0x8001A708 (matches Beetle), not the stale 0xBFC11BF0.

runtime: g_pc0_reason slice-exit provenance, fail-closed exception-sentinel
asserts, named thread-event kinds, freeze-heartbeat RVA emission (diagnostics,
default-on, additive).

Carries the in-progress faithful-timing / MDEC work (cpu_state.h, memory.c,
dma.c, interrupts.c, dirty_ram_interp.c, full_function_emitter.cpp) as an
uncommitted-WIP checkpoint. tools/: regen_bios.sh, bios_emitter_fingerprint.sh.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 14:20:17 -07:00
Matthew Stanley 2d915c3fa9 wip(tomba2): precise-IRQ-slicing groundwork — cycles_to_next_event + design spec
Phase 1 of the cycle-budgeted precise interrupt-take-point fix (root cause:
compiled blocks take IRQs only at block edges; HW + the dirty-RAM interpreter
take them at the exact instruction — oracle-confirmed at the Tomba 2 logo
Timer1 debounce). Design validated with ChatGPT; full spec in PRECISE_IRQ_SLICE.md.

This commit lands the additive, no-behavior-change foundation (Task #2) plus the
prior session's diagnostic take-PC capture, and a build-hygiene dedup:

- cycles_to_next_event() (interrupts.c): single source of truth — min guest-cycle
  distance to the next DELIVERABLE interrupt (source raises its I_STAT bit AND
  that bit is unmasked in i_mask). Aggregates VBLANK + per-peripheral queries.
- timers/cdrom/dma/sio _cycles_to_irq(i_mask): each peripheral owns its own
  next-IRQ-distance query; all conservative under-estimates (smaller => slice
  more => never run past an event).
- debug_server: cycles_to_next_event command for live validation.
- interrupts.c: take-PC capture into the IRQ_DELIVER event-ring aux (prior
  session's diagnostic; records the architectural take-PC for cross-backend diff).
- overlay_loader.c: remove duplicate psx_native_bad_entry definition (HEAD had it
  twice; would not link cleanly).

Live-validated at the logo: returns the VBLANK distance, correctly excludes the
three free-running (unarmed/masked) timers and idle CD/DMA/SIO. Runtime-only,
no regen. The two-tier executor (Task #3) that consumes this is next.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 15:04:25 -07:00
Matthew Stanley 8ab24c5e62 wip(tomba2): ABI v8 check_interrupts_at + emitter charge-cycle work (pre-rebase snapshot)
In-progress investigation work snapshotted before rebasing onto the
tcc-refactored psxrecomp master. Includes overlay ABI v8
(check_interrupts_at resume-PC callback for async RFE recovery) and
code_generator/full_function_emitter delay-slot cycle accounting.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 23:26:04 -07:00
Matthew Stanley 9940ab2c0f wip(tomba2 exc): observability + corrected frame-1997 diagnosis (longjmp-to-outermost, NOT async)
Implemented + instrumented the consult's Option-A async-resume idea; the data REFUTED
the in_exception=0 async model:
- freeze_check now reports async_rfe_set/fire, reach_dirty/traps, reach_async,
  dirty_safe_resume_pc, async_rfe_resume_pc.
- Measured: reach_dirty=19921 (~1/exception), dirty_in_exc0=0, async_rfe_fire=0 ->
  the sentinel 0x80000048 ALWAYS arrives at the dirty gate with in_exception=1 and
  takes the longjmp. There is no in_exception=0 dispatch of the sentinel.

Corrected mechanism (docs/TOMBA2_WHOOPEE_BAIL.md): the game's ReturnFromException
longjmp lands in psx_check_interrupts (jmp_val==1), restores GPRs and returns with
cpu->pc=0; for ~19920 exceptions it lands at a NESTED dispatch (game continues), but
for the failing one it unwinds to the OUTERMOST trampoline where cpu->pc=0 reads as a
clean exit. It's a longjmp-landing-depth bug in the 'resume via host return' model,
not an async-resume bug.

Latches added (g_dirty_safe_resume_pc around the 3 dirty pumps, g_async_rfe_resume_pc
at dirty-safe exception entry) + the sentinel-gate redirect scaffolding are INERT
under the corrected model (never fire) — kept as tooling + scaffolding for the
candidate fix. Builds clean; behaves identically to pre-fix (exit at frame 1997, no
regression). wt/tomba2 only; not on master.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 23:16:13 -07:00
Matthew Stanley 90bb5f95f7 interrupts: preserve dispatch-depth across the async exception handler
With Tomba 2's overlays now on the local-flow path (prior commit), the
per-4096-insn interrupt pump delivers the exception handler from a NESTED
dispatch (g_psx_dispatch_depth > 0). psx_check_interrupts reset the counter to 0
around the handler but never restored the interrupted code's nesting, and a
ReturnFromException longjmp skips the handler frames' --depth decrements. The
outer frames then unwound below zero (dispatch_depth -> -1), the `outermost`
test misfired, and the top-level dispatch returned to PC=0 ("execution
completed" abnormal exit at frame 1997).

Save the interrupted code's depth before the handler, run the handler as its own
outermost context (0), and restore the saved depth afterward. Depth-transparent
for callers that were already at depth 0 (Tomba 1 / MMX6 / Ape), so no behavior
change there.

Resolves: dispatch_depth stays correct (0, not -1) across delivered interrupts.
NEW failure point (next in cascade, NOT addressed here): a genuine guest-state
divergence at frame ~1997 — overlay chain 0x80089784 -> 0x80080FC4 -> 0x800499E8
-> 0x80050CEC does a jr to a null register (final_ra 0x80050CC8, PC=0). Needs an
oracle first-divergence pass. See docs/TOMBA2_WHOOPEE_BAIL.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 23:16:11 -07:00
Matthew Stanley 732e093ab2 heartbeat: IRQ raise/deliver/ack + bail-source telemetry (Tomba 2 spin diag)
Always-on, heartbeat-resident diagnostics for the Whoopee-Camp splash spin:
- VBLANK raise/deliver/ack counters (interrupts.c + memory.c) + irq_deliver.
- Deduped bail-source ledger keyed by (call-site $ra, wild target) (traps.c,
  hooked at psx_call_contract in cpu_state.h) + heartbeat top-key/first-bail.

Result: REFUTES the exception-reentry-storm hypothesis. VBLANK is raised,
delivered AND acked exactly 1/frame (textbook-healthy); irq_deliver is 1/frame,
not a storm. So the freeze is NOT an interrupt/timer/CD problem -- the hardware
sim is healthy. The ~68k bails/frame are the GAME splash loop busy-polling a
kernel event (TestEvent/WaitEvent) that never fires -- the DeliverEvent/WaitEvent
discriminator is the next probe.

(The dominant dispatch-trampoline bails are emitted in generated BIOS code, so
the precise bail call-site needs a recompiler-emit change + BIOS regen; the
direct-call ledger here is partial. Deferred -- not needed to localize to the
event wait.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 23:16:08 -07:00
Matthew Stanley 2439d4dd44 dispatch call contract: continuations only run when the guest really returns
Bug D root cause (Tomba chest freeze, also QA issue #1): the game's own
dispatcher func_80056E08 has a dead jumptable case (type 0x21 ->
jal 0x80120B3C) whose target is mid-function in EVERY overlay variant on
the disc (only the boot EXE references it; the real handler entry is
0x80120AC8). Entering mid-function skips the prologue, runs an epilogue
for a frame that was never pushed, and returns through garbage. Our C
model then resumed the suspended generated continuations on the moved
guest stack -> double-executed tails -> infinite zombie loop (~3K wild
calls/frame) -> MMIO fatal. Family root of Bugs A/C/D.

The contract: a generated C continuation may only run if the guest
actually arrived at it ($ra == call site return address, $sp == caller
sp at the call). Violations begin a "bail" unwind: cpu->pc := the wild
jr's true target, every generated frame returns immediately, and the
unwind resolves at the first enclosing site whose contract matches (or
flattens at the outermost dispatch loop, re-dispatching the wild target
on a clean host stack -- matching real-HW one-stack semantics).

- cpu_state.h: bail state + psx_call_contract() helper (+ overlay-DLL
  pointer indirection under PSX_OVERLAY_DLL_BUILD)
- emitters: (ra, sp) checks at every BIOS and game jal/jalr site;
  psx_dispatch_impl validates pc==0 and stop-hits against (stop, sp),
  handles bail resolve/propagate/flatten
- dirty_ram_interp: surfaced calls validate the contract on return;
  bail aborts the interp run; tail transfers into compiled code now run
  with pc cleared (closes a latent double-run on plain C returns)
- overlay cache ABI v2: DLLs export overlay_abi(); loader rejects and
  deletes stale DLLs (autocompile regenerates with contract checks)
- telemetry: bail_first/resolved/flattened/anomaly in heartbeat + dump
  headers; anomaly guard at exception entry; TCP_COMMANDS.md docs

Verified: BIOS+Tomba regen, boot to intro at locked 60.0 fps, all bail
counters 0 through boot/title/attract (no false positives). Chest-area
gameplay verification pending (requires user play).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 22:30:41 -07:00
Matthew Stanley 5041712a05 runtime: phase-5 checkpoint — event-timeline ring, instant-disc rate fix, overlay diagnostics
Accumulated working-tree state for the overlay-JIT-cache phase, committed as a
checkpoint before the Bug A fix (next commit):

- event_ring.{c,h}: always-on unified event-timeline ring (IRQ deliver/gate/
  raise, I_STAT changes, DMA kick/sched/done, ENQ/DEQ per source, exec-mode
  tagged). Edge-recorded, eviction-bounded, survives a blue screen; dump via
  event_ring_dump/_tail/_clear.
- cdrom.c: disc_speed="instant" no longer collapses every sector read to the
  500cy floor (which flooded ~1100 DMA3 completions/frame and starved main
  code). CDROM_INSTANT_PERIOD bounds sector IRQs to CDROM_INSTANT_MAX_PER_FRAME
  (32)/frame. Known caveat: a transient ~1-2s reentry storm at boot frame ~210
  still trips the watchdog once, then recovers.
- interrupts.c: claimed-IRQ cooldown is now per-source — SIO stays 0 (card
  reads need back-to-back IRQs); DMA/VBLANK/timer get CLAIMED_PROGRESS_QUANTUM
  (8) blocks of guaranteed main-code progress between deliveries. Root cause of
  the instant-disc freeze: cooldown=0 let the block leader re-take the
  exception before its body ran (pinned at the 0x8005A1D8 field loop).
- gpu/dma/memory/sio/timers/psx_cycles: event-ring instrumentation at raise/
  kick/complete/schedule sites; g_event_step_conservative (single-cycle event
  stepping A/B used to refute the event-granularity hypothesis for Bug A).
- overlay_capture.c, overlay_loader.h, crc32: capture/manifest plumbing for
  per-entry validity (Inc3).
- recompiler: overlay-mode emission updates (code_generator, main_psx).
- tools: compile_overlays.py audit/patch updates; tomba_loadtest.py.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-09 12:24:40 -07:00
Matthew Stanley d22c9cec42 runtime: always-on A0/B0/C0 bioscall ring + psx_check_interrupts_at shim
Purely additive diagnostic instrumentation (0 deletions):
- debug_server.c: always-on BIOS-call ring (psx_bioscall_record) hooked at
  the central dispatch chokepoint + unique-call summary table.
- interrupts.{c,h}: psx_check_interrupts_at(cpu, resume_pc) compat shim that
  forwards to psx_check_interrupts, so the baseline runtime can link
  ape-flavored generated code for instrumented good-vs-bad comparison.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-07 11:15:01 -07:00
Matthew Stanley b9716154f0 Add native macOS/Linux support (ucontext thread scheduler, fullscreen, build fixes)
The recompiler and runtime now build and run natively on macOS (Apple
Silicon & Intel) and Linux, alongside Windows. The recompiled BIOS boots to
the PlayStation logo, and Tomba boots into gameplay with working memory-card
save/load — the previously Windows-only path.

Core change — the BIOS thread scheduler:
- runtime/src/psx_fiber.{h,c} (new): one cooperative-fiber API, backed by
  Win32 Fibers on Windows and ucontext (makecontext/swapcontext) on POSIX,
  with a GetCurrentFiber analog (psx_fiber_current).
- traps.c: the BIOS thread switcher now uses psx_fiber_* on every platform.
  The old non-Windows setjmp/longjmp fallback is removed — it ran target
  threads nested on the caller's C stack and so couldn't resume an arbitrary
  suspended thread, which hung the BIOS CD-boot WaitEvent handoff. Real
  per-thread stacks (fibers/ucontext) fix it.
- interrupts.c: the exception-deferral path (g_exception_owner_fiber,
  deferred_exception_longjmp) is now fiber-based on all platforms via the
  same abstraction, instead of #ifdef _WIN32.

Cross-platform build fixes:
- recompiler/CMakeLists.txt: vendored fmt 9.1.0 force-enables consteval on
  Apple clang (the __cpp_consteval clause overrides its own Apple exclusion),
  which clang 21 then rejects; force FMT_CONSTEVAL empty on non-MSVC.
- runtime.cmake: add psx_fiber.c; add pkg-config's SDL2_LIBRARY_DIRS so
  -lSDL2 resolves outside default paths (Homebrew /opt/homebrew/lib).
- crash_trace.c: include <stdarg.h> (not pulled in transitively off Windows).
- main.cpp: the forced "opengl" render-driver hint (a Windows-GDI-freeze
  workaround) is now Windows-only so macOS uses Metal; create the renderer
  with PRESENTVSYNC; SDL_RenderSetLogicalSize(640,480) so fullscreen
  letterboxes; add an F11 / Alt+Enter / Cmd+F fullscreen toggle.

Verified: SCPH1001 BIOS boots to the Sony logo on Apple Silicon, and Tomba!
(SCUS-94236) boots into gameplay with memory-card saves persisting across
relaunches. Gamepad (SDL_GameController) was already cross-platform.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-29 18:53:22 -07:00
Matthew Stanley fdbd2b515f runtime: cross-fiber longjmp fix + decouple vblank-present from MMIO read
Two distinct bugs were causing OPTIONS-black hard freezes after 12K-65K
frames:

1. interrupts.c + traps.c: exception_jmpbuf is a single global. setjmp
   captures the running fiber, but longjmp may be requested from a
   different fiber via psx_exception_longjmp / psx_restore_state_escape
   during chain-handler dispatch. Cross-fiber longjmp restores RSP to
   the wrong stack while Win32 still tracks the calling fiber, corrupting
   fiber state until SwitchToFiber eventually hangs.

   Fix: track g_exception_owner_fiber at setjmp time. New
   deferred_exception_longjmp checks the current fiber; if it's not the
   owner, sets a pending flag and SwitchToFiber's back to the owner.
   psx_change_thread_fiber's post-SwitchToFiber path observes the flag
   and executes the longjmp on the correct stack.

2. gpu.c: gpu_read_gpustat fired gpu_vblank_tick() after 1000 polls,
   which invoked sdl_vblank_present (SDL_Delay -> Sleep) from inside
   an MMIO read code path. That meant recompiled MIPS spent wall time
   in Sleep deep inside the fiber call tree -- confusing to reason
   about and a contributor to fiber/longjmp interactions.

   Fix: synthetic VBLANK from a poll-storm only updates emulation state
   (LCF + I_STAT). The host-side render+sleep happens only at the proper
   cycle-paced VBLANK trigger in psx_check_interrupts.

Also adds: dirty_block ring slice to the freeze auto-dump, Win32 main-
thread stack capture for wedge_kind=1 hard freezes (SuspendThread +
StackWalk64 + SymFromAddr), and dbghelp link.

Validated: TombaRecomp psx-runtime ran 30+ minutes / 105K+ frames at
OPTIONS with no hard freeze. Previous freezes consistently hit at
11900, 20493, and 64596 frames.

Diagnostic tooling under tools/_options_session/ that produced this
diagnosis is included.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 12:49:13 -07:00
Matthew Stanley b486c13c3a runtime: fix VBlank pacing to use guest cycles, not dispatch count
VBlank firing in psx_check_interrupts was gated on dispatch_count
(block-dispatch count, ~5-6 cycles each), not on guest cycles.
50000 dispatches ≈ 289k cycles per VBlank, vs real PSX 564480
cycles per VBlank. Effect: game-internal time ran at ~51% of real
PSX rate, breaking every cycle-paced game flow including FMV.

This was the root cause of TombaRecomp ISSUES.md #4 — FMV played
at 6.7 fps because Tomba's FMV state machine is driven by DMA Ch1
(MDEC-out done) IRQ, which in turn fires once per full FMV-frame
pipeline iteration, which is bounded by cycle-paced steps
(CDROM sector ready, MDEC decode "complete", DMA transfer).
At 51% guest-time rate, a Tomba-designed 15-fps FMV becomes 7-8 fps.

Fix:
- New VBLANK_CYCLES = 564480 (33.8688 MHz / 60 Hz NTSC).
- Add interrupts_advance_cycles(cycles) called from
  psx_advance_cycles() so the cycle counter for VBlank gating
  tracks emitted guest cycles.
- Change the trigger from `dispatch_count >= VBLANK_INTERVAL` to
  `cycles_since_vblank >= VBLANK_CYCLES`.
- On VBlank fire, subtract one period (not reset to 0) so cycle
  overshoot from long blocks carries forward.
- dispatch_count is kept for the SIO card-protocol stale check —
  that uses total_checks, which is the real load-bearing field
  there, dispatch_count itself is now incidental.

Verified:
- Cycles/VBlank: 289k -> 564k (exactly VBLANK_CYCLES).
- Cycles/sec: 20.6 M -> 41 M (121% of real PSX; slight wall-clock
  overshoot to investigate, but game-internal pacing now correct).
- FMV decode rate: 0.111 -> 0.250 decodes/VBlank = 15.0 fps exactly,
  matching Tomba's designed FMV rate.
- Measured via tools/_mdec_pace.py.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 14:09:11 -07:00
Matthew Stanley cf05a0ab1f runtime + recompiler: Tomba boot through BIOS into Whoopee Camp FMV
Recompiler — `recompiler/src/function_analysis.cpp` now stops a
function's end_addr at its mapped `jr $ra` + delay slot instead of
stretching through trailing data. Fixes Tomba's 0x80076DB0, a real
136-byte function that was being emitted as a psx_unknown_dispatch
stub because data after it poisoned the boundary heuristic.
`code_generator.cpp` emits `func_<next>(cpu); return;` as a tail
call when a mid-function seed split leaves the physical fall-through
in a different C function, plus a `debug_server_log_call_entry` at
every function entry. `full_function_emitter.cpp` splits dispatch
into `psx_dispatch_impl(cpu, addr, stop_addr)` with public
`psx_dispatch` and new `psx_dispatch_call(cpu, addr, return_addr)`
so JAL/JALR into RAM stubs can return cleanly without growing the
native stack.

Runtime — `runtime/src/dma.c`/`include/dma.h`/`src/memory.c` now
expose `dma_write_masked()`; the 16-bit and 8-bit mmio paths route
DICR/DPCR/CHCR partial writes through it so a partial DICR write no
longer accidentally clears unrelated channels' pending IRQ flags,
and CHCR only triggers DMA when the written mask actually includes
the start bit. Unified `complete_transfer(ch)` across all channels,
removed `prev_master_flag` edge detection.

MDEC (`runtime/src/mdec.c`, `include/mdec.h`) gains a full debug
surface: `MDECDebugState` snapshot, an always-on 4096-entry
`MDECDebugEvent` ring with 11 event kinds (reset, ctrl_write,
cmd_begin/done, decode_done, dma_in/out start/end, output_drained,
read_underflow), and `mdec_state` / `mdec_trace` / `mdec_trace_clear`
debug-server commands. DMA ch0/1 emit start/end events around MDEC
transfers.

CDROM (`runtime/src/cdrom.c`, `include/cdrom.h`) — read stream
separated from one-shot pending state (`read_cmd`, `read_delay`,
`start_read_stream`, `stop_read_stream`, `deliver_read_sector`);
`cdrom_state`/`cdrom_trace` report the new fields.

Thread/fiber tracing — `traps.c`'s Win32-fiber TCB save/restore now
emits 12 thread-event kinds; new `ThreadTraceEntry` ring (64 K) plus
`thread_trace` / `thread_trace_clear` debug-server commands report
both live CPU regs and the target TCB's full saved register block.

wtrace / mmio rings — `WriteTraceEntry` grew (cpu_pc, sp, v0/v1/a0-a3/
t0/t1); capacity 1 M -> 4 M. `MmioTraceEntry` grew (pc, cpu_pc, sp,
a0-a3, sr, epc, i_stat, i_mask). Both dump commands accept
`count`/`newest=1` for reverse iteration. `restore_trace` and
`dirty_block_log` emit extended register fields (a2/a3/sp).

Misc — `interrupts.c` polls debug-server every 16K checks so it
stays responsive without a separate thread. `psx_runtime.h`'s
`call_by_address` routes through `psx_dispatch_call`. Dirty-RAM
interp JAL/JALR consult `psx_dispatch_game_compiled` when the game
is wired in. iso_reader gains the entry points needed by the game
disc path. New `tools/_tomba_probe.py` for port-4470 health checks.

Generated — regenerated `SCPH1001_dispatch.c` against the new
`psx_dispatch_impl` shape.

ISSUES.md — opened Issue #3 (BIOS PS-logo glyph missing on the
disc-detected screen, cosmetic).

.gitignore — added `recompiler/build-asan*/`, `build-checked/`,
`__pycache__/`.

Validated end-to-end: BIOS boots, disc detected, Whoopee Camp FMV
plays with audio. Frame count advances through ~40K frames without
freezes. `unknown_dispatch_log.total = 0`. Remaining FMV cluster
issues (1/4-size render, FMV-2 audio missing, no skip) tracked
under TombaRecomp/ISSUES.md.

Co-Authored-By: Codex (OpenAI) <noreply@openai.com>
2026-05-11 12:34:49 -07:00
Matthew Stanley b4b5d223e2 runtime: cycle-paced timers + CDROM with sector header + trace ring
timers.c — timers_advance(cycles) replaces per-frame timers_tick().
Per-timer fractional accumulator + proper divisors:
- Timer 0 dotclock approx (cycles/5)
- Timer 1 HBlank clock (cycles/2146, ~263 HBlanks per NTSC frame)
- Timer 2 SYSCLK or SYSCLK/8 per mode register bit 8..9
Target/overflow flags fire IRQ when bit 4/5 of mode is set.

cdrom.c — cdrom_advance(cycles) replaces per-frame cdrom_tick().
Sector pacing from CPU clock: 451584 cycles at 1x, 225792 at 2x
(SetMode bit 7). Whole-sector mode bit 5 prepends a 12-byte BCD-MSF
header to the data buffer; SECTOR_BUFFER_SIZE = 12 + 2048.

cdrom_trace ring: always-on CDROMTraceEntry[1<<16] capturing every
register write, IRQ raise/mask, sector load, and command dispatch
with the current func/PC/frame snapshot. Query via the new
debug_server commands landed in a later commit.

psx_cycles.c — psx_advance_cycles now drives cdrom_advance and
timers_advance in addition to sio_advance.

interrupts.c — legacy per-frame timers_tick + cdrom_tick now gated
behind #ifndef PSX_ENABLE_BLOCK_CYCLES, so block-cycle builds use
the cycle-paced path exclusively.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 22:50:39 -07:00
Matthew Stanley 122dc49f42 Fix BIOS memory card copy writes 2026-05-08 20:50:31 -07:00
Matthew Stanley 0d15e2bd4e phase1.0c-v2: cycle-paced sio_tick + dispatch quantum behind active-guard
Adds the cycle-paced sio_tick body, the sio_tick_quantum() wrapper, and
the pace_state TCP telemetry handler. Crucial change vs. failed 1.0c:
the dispatch-loop quantum tick is gated by g_sio_timing_active (volatile
int), so on the psx_check_interrupts hot path the cost is one load + one
branch when no SIO time is pending — not a full function call into the
cycle-paced loop.

In 1.0c-v2 the TX path is still synchronous and nothing arms shift_active
or pending_ack, so g_sio_timing_active stays 0. Phase 1.0d will set it
when arming the shifter and clear it when the shifter+buffer+ack are all
empty.

Files:
- sio.h: SIO_MODEL_CYCLE_PACED moved to header (default 1); declares
  g_sio_timing_active (extern volatile int) and sio_tick_quantum()
- sio.c: defines g_sio_timing_active=0; adds cycle-paced helpers
  (sio_consume_ack_event, sio_fire_ack_irq, sio_handle_shift_complete);
  sio_tick gains a cycle-paced loop that runs only if shift_active or
  pending_ack is set (currently never), then falls through to the legacy
  IRQ countdown body which remains the live IRQ delivery path; adds
  sio_get_pace_state(uint64_t out[16]) accessor
- interrupts.c: in psx_check_interrupts !in_exception block, calls
  sio_tick_quantum only if g_sio_timing_active is set
- debug_server.c: new pace_state TCP handler exposing the snapshot

Validation against 64e0194 baseline:
- t=20s/30s TCP responsive; pad arm audit 2840/2840 no_ack=0
- 4x CROSS succeeds
- card txn distribution unchanged (10 txns reach bytes=15 ts=10)
- txn 0 still reads "MC" magic at bytes 10-11 (0x4D 0x43)
- pace_state: timing_active=0, all dynamic fields 0 throughout

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 17:04:14 -07:00
Matthew Stanley 5b3c7dd08d phase4: freeze_check diagnostic + accumulated overnight infra
Today's session work
- runtime/src/debug_server.c: freeze_check TCP cmd snapshots all liveness
  counters (PC, exception entries, exception_reentry_blocks, dirty_ram
  blocks/insns, fn_entry total + top-K histogram, SIO IRQ pending +
  countdown, sio_stat/i_stat/i_mask, mc_max_state) for tight-loop
  diagnosis when the runtime hangs
- runtime/src/debug_server.c: wtrace slot 12 covering [0x72F0..0x72F4]
  (data-byte counter); measured peak=1 confirming setup_data_phase resets
  the counter mid-cycle
- runtime/src/interrupts.c: exception_entries_total + exception_reentry_blocks
  counters + psx_get_freeze_diag accessor
- runtime/src/sio.c: sio_get_freeze_diag accessor for IRQ pacing internals
- memory entries: phase4_card_gate_setup_writes_2026_04_28 (refutes the
  prior \"missing [0x7568+slot] clearer\" framing — setup functions write
  LSB=0 directly), phase4_72f0_one_byte_per_vblank_2026_04_28 (peak=1
  measurement)

Overnight session work bundled in
- runtime/src/dirty_ram_interp.{c,h}: small MIPS interpreter for
  install-at-runtime kernel RAM stubs (Rule 18; e.g., SIO data-byte
  handler the BIOS writes to RAM 0xCF0)
- recompiler/src/full_function_emitter.{cpp,h}: 3057 jal/jalr continuation
  return labels added to dispatch table (continuation fix); branch
  fall-through fix for conditional-branch delay slots
- recompiler/src/strict_translator.cpp: matching translator updates
- generated/: regenerated SCPH1001 dispatch + full C output reflecting
  the recompiler changes
- runtime/include/debug_server.h, src/memory.c: card_txn ring + SIO IRQ
  ring + EvCB walk ring + per-PC dirty_ram counters + always-on wtrace
  coverage for [0x755A], [0x7514], [0x7528], [0x7520], [0x74A4],
  [0x75C0..C8], [0x74B8..C0]
- recompiler/tests/, runtime/tests/: SIO/card protocol unit tests
- tools/: 60+ Python diagnostic scripts (chain_trace analysis, SIO trace
  diff, EvCB walker, wtrace dumpers, etc.)
- docs/: psx_bios_disasm reference + dynamic_handler_install rationale
- OVERNIGHT_2026_04_28.md, SUMMARY_20260412_2.md: session logs

Phase 4 status (still blocked, root cause better understood)
- BIOS card chain runs through R-handlers correctly (chain_trace shows
  R1..R11 cycling 26x); shell IS retriggering _card_read
- Install handler at ROM 0xBFC15F1C increments [0x72F0] correctly per
  SIO IRQ; gate at <0x80 / <0x7F is correct
- Bottleneck: SIO IRQ countdown is access-paced, fires ~1× per VBlank
  vs ~128× needed; setup_data_phase (called from outer coord on VBlank)
  zeroes [0x72F0] before counter reaches 128
- Both attempted fixes today caused freezes:
  * VBlank deferral on [0x75C0]==1 — pending VBlank already in i_stat
    delivered anyway; tighter dispatch gate also locked up
  * Cycle-paced sio_tick at rates 50, 200 (and gated on [0x75C0]==1) —
    runtime freezes once data phase activates, root cause unidentified
- delay=5 narrows the threshold (peak=2 vs peak=1, no freeze) but
  doesn't get to 0x80
- Real fix likely requires deeper instrumentation of the freeze (call
  stack / dispatch state at the moment of TCP starvation)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 21:29:19 -07:00
Matthew Stanley 6d2cb65421 fix: reduce post-exception cooldown to 0 for claimed interrupts
Real hardware executes 1 instruction between exceptions. Our previous
cooldown of 1 dispatch block delayed SIO IRQ re-fire during card read
sequences. Setting to 0 allows immediate re-fire when the handler
acknowledged the interrupt (cleared I_STAT bits).

Investigation ongoing: card directory buffer still empty despite
cooldown fix. Root cause traced to _card_status (B0:0x4D) at
BFC15AB8 zeroing buffer pointer at 0x7550 between _card_read
setup and SIO handler data transfer. See memory file
phase4_card_data_loss_root_cause.md for full analysis.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-25 00:26:47 -07:00
Matthew Stanley 2767018b46 fix: SIO pad detection + input override + CARD driver seeds
Root cause: BIOS pad handler polls I_STAT bit 7 waiting for SIO ACK
IRQ. sio_tick() was only called from psx_check_interrupts and SIO
register accesses, but I_STAT reads go through memory.c which never
called sio_tick(). The IRQ countdown was stuck during the polling
loop, causing the BIOS to time out and report "no pad found" (0xFF).

Additionally, sio_tick() in psx_check_interrupts fired the IRQ too
early — during BIOS delay loops BEFORE I_STAT bit 7 was cleared —
so the BIOS cleared it and never saw it re-appear.

Fixes:
- memory.c: Call sio_tick() on I_STAT reads (32/16/8-bit paths)
- interrupts.c: Remove sio_tick() from psx_check_interrupts
- debug_server.c: Don't clear input override on TCP disconnect
  (was wiping override before next VBlank could read it)
- debug_server.c: sio_state reports real pad_buttons via getter
- sio.c/sio.h: Add sio_get_pad_buttons() for debug server
- seeds: Add 20 new seeds for shell input and CARD driver functions

Result: Pad buffer at 0x80083E30 shows status=0x00 (pad found),
type=0x41 (digital), buttons reflect keyboard/override state.
Only 5 COP2/GTE dispatch misses remain (Phase 5).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-23 11:22:39 -07:00
Matthew Stanley 1a0213ec5e fix: RestoreState escape mechanism — VSync counter now increments
The kernel's inline RestoreState (at BFC1094C inside the exception
handler func_00000C80) restored all registers and did `jr $ra`, but
the generated `return;` went back to psx_check_interrupts instead of
dispatching to the restored $ra (0xBFC421D8 — the VSync callback
loop). Then psx_check_interrupts overwrote all GPRs from the
pre-exception save, wiping RestoreState's changes. VSync counter
was permanently stuck at 0.

Fix has three parts:

1. runtime/src/interrupts.c: Add psx_restore_state_escape() which
   longjmps with code 2 (vs code 1 for ReturnFromException).
   psx_check_interrupts now loops: code 2 re-dispatches to cpu->pc
   (still in exception context), code 1 exits, code 0 is normal.

2. recompiler full_function_emitter: Broaden longjmp-return detection
   from first-instruction-only to scanning ALL instructions for the
   exact encoding 0x8C9F0000 (lw $ra, 0($a0)). This catches both
   standalone RestoreState (func_1FC0227C) and the kernel inline
   copy (func_00000C80 at BFC10964).

3. Generated jr $ra terminators in matched functions now emit:
   cpu->pc = cpu->gpr[31]; psx_restore_state_escape(); return;

Result: VSync counter increments steadily (~256/vblank), zero crashes,
shell renders Memory Card / CD Player screen with no ADEL faults.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-22 23:23:25 -07:00
Matthew Stanley b3833aa253 fix: exception handler chain walk + shell reaches Memory Card/CD-ROM screen
Root cause: the recompiler split the kernel exception handler at the
jalr $s1 (chain handler dispatch) into func_00000C80 and func_00000E10.
The chain-walker continuation at BFC10910 never executed — only the
first handler in chain 0 was ever called.  I_STAT bits 0 (VBlank) and
2 (CDROM) were never acknowledged, causing an interrupt livelock.

Fixes:
- Emitter jalr/jal safety-net: when the return address falls outside
  the current function, emit a continuation psx_dispatch call after
  the jalr dispatch.  This bridges split functions at indirect calls.
- Merged exception handler: removed BFC10910 seed so the chain walker
  continuation is part of func_00000C80 (C fall-through after jalr).
- 0xBFC10910 dispatch special case in psx_unknown_dispatch: chain
  handlers return to $ra=BFC10910 via jr $ra (compiled as `return;`);
  the dispatch loop picks up the $ra value — handled as a no-op since
  the continuation already ran via C fall-through.
- Post-exception adaptive cooldown: if the handler didn't clear any
  I_STAT bits, give main code 500 blocks before re-firing.  Prevents
  VBlank livelock when no handler claims the interrupt.
- SIO byte write handler in mmio_write8 (was missing, caused crash).
- GPU→RAM DMA ch2 stub: fills target with zeros, completes transfer.
- Dispatch trace (debug_server_trace_dispatch) added to emitter
  template so it survives regeneration.

Result: shell device detection completes, StartPad dispatched, chain
E014 populated, GPU actively rendering the Memory Card / CD-ROM
screen.  VSync counter still 0 (handler installed but not yet
incrementing — next investigation).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-21 18:13:51 -07:00
Matthew Stanley 6ce012df3e fix: wire cdrom_tick into vblank interval, fire IRQ on command completion
Two CDROM issues preventing BIOS from progressing past Sony logo:

1. cdrom_tick() was never called — delayed CDROM responses (GetStatus,
   Init, Pause) never completed because pending.delay was never
   decremented. Wired into psx_check_interrupts vblank interval.

2. exec_command() called set_irq() but not fire_cdrom_irq() for
   immediate responses (GetStat, SetLoc, Demute, etc.). The CDROM
   IRQ flag was set internally but i_stat bit 2 was never raised,
   so the BIOS exception handler never delivered CDROM events.

BIOS still stalls at same draw count — next bug in chain is EvCB
event delivery (TestEvent returns 0 because VBlank events aren't
being written to the event control block table).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 00:41:02 -07:00
Matthew Stanley fba58d76b3 Phase 4 interrupt fixes: RFE, GPR save/restore, DMA IRQ edge-trigger
Three interrupt delivery bugs fixed that were blocking shell progression:

1. RFE after handler exit: The exception handler's normal exit path
   (jmpbuf longjmp at 0xBFC10944) does not execute RFE. In our model
   the handler returns to psx_check_interrupts, not to the WaitEvent
   caller. Without RFE, IEc stays 0 forever after the first interrupt.
   Fix: apply RFE in psx_check_interrupts if IEc is still 0 after the
   handler returns.

2. GPR save/restore around handler dispatch: The handler's jmpbuf exit
   restores callee-saved registers from the kernel jmpbuf (intended for
   a non-local jump to WaitEvent's caller). In our model this corrupts
   the interrupted code's registers. Fix: save all GPRs before handler
   dispatch and restore them after, matching real hardware where the
   interrupted code's registers are preserved via TCB save/restore.

3. VBlank tick gated on !in_exception: The handler runs thousands of
   block leaders, each calling psx_check_interrupts. Without gating,
   dispatch_count advances inside the handler, firing new VBlanks that
   stack up and starve the main code. Fix: only increment dispatch_count
   and fire VBlank/timer ticks when not in exception handling.

4. DMA IRQ edge-triggered: update_master_irq() was level-triggered,
   re-asserting i_stat bit 3 on every call while master_flag=1. Combined
   with no BIOS callback to acknowledge DMA IRQs, this caused an
   interrupt storm. Fix: edge-detect master_flag 0->1 transition, and
   suppress DMA completion IRQs entirely since transfers are synchronous.

Also adds GPU debug diagnostics: GP0 command counters (nop/fill/draw/
env/copy), draw area/offset in gpu_state, vram_peek debug command.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 20:29:03 -07:00
Matthew Stanley a1165d51b7 Phase 2 complete: BIOS boots to GPUSTAT read
Full recompilation pipeline working:
- Recompiler emits 799 functions (46k instructions) from SCPH1001.BIN
- Tail-call trampoline in dispatch loop prevents stack overflow
- Timer registers (3 counters, mode, target) implemented
- Interrupt check at labels + dispatch boundaries with vblank scheduling
- SYSCALL 1/2/3 (Enter/ExitCriticalSection, ReturnFromException) handled
- EPC correctly set from SYSCALL instruction address

BIOS boot reaches first GPU hardware access:
  UNIMPLEMENTED MMIO READ32 (GPUSTAT) @ 0x1F801814

Phase 3 (Sony logo) starts here: GPU, DMA, display pipeline.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 00:03:46 -07:00