Commit Graph

52 Commits

Author SHA1 Message Date
Matthew Stanley fdbd2b515f runtime: cross-fiber longjmp fix + decouple vblank-present from MMIO read
Two distinct bugs were causing OPTIONS-black hard freezes after 12K-65K
frames:

1. interrupts.c + traps.c: exception_jmpbuf is a single global. setjmp
   captures the running fiber, but longjmp may be requested from a
   different fiber via psx_exception_longjmp / psx_restore_state_escape
   during chain-handler dispatch. Cross-fiber longjmp restores RSP to
   the wrong stack while Win32 still tracks the calling fiber, corrupting
   fiber state until SwitchToFiber eventually hangs.

   Fix: track g_exception_owner_fiber at setjmp time. New
   deferred_exception_longjmp checks the current fiber; if it's not the
   owner, sets a pending flag and SwitchToFiber's back to the owner.
   psx_change_thread_fiber's post-SwitchToFiber path observes the flag
   and executes the longjmp on the correct stack.

2. gpu.c: gpu_read_gpustat fired gpu_vblank_tick() after 1000 polls,
   which invoked sdl_vblank_present (SDL_Delay -> Sleep) from inside
   an MMIO read code path. That meant recompiled MIPS spent wall time
   in Sleep deep inside the fiber call tree -- confusing to reason
   about and a contributor to fiber/longjmp interactions.

   Fix: synthetic VBLANK from a poll-storm only updates emulation state
   (LCF + I_STAT). The host-side render+sleep happens only at the proper
   cycle-paced VBLANK trigger in psx_check_interrupts.

Also adds: dirty_block ring slice to the freeze auto-dump, Win32 main-
thread stack capture for wedge_kind=1 hard freezes (SuspendThread +
StackWalk64 + SymFromAddr), and dbghelp link.

Validated: TombaRecomp psx-runtime ran 30+ minutes / 105K+ frames at
OPTIONS with no hard freeze. Previous freezes consistently hit at
11900, 20493, and 64596 frames.

Diagnostic tooling under tools/_options_session/ that produced this
diagnosis is included.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 12:49:13 -07:00
Matthew Stanley 3dce5e43ce runtime: PSX_DEBUG_TOOLS CMake flag + freeze heartbeat thread
Two observability additions, both gated by the new
-DPSX_DEBUG_TOOLS=OFF flag for production builds.

1. Freeze heartbeat thread (`runtime/src/freeze_heartbeat.c`):
   Background thread spawned at startup that snapshots runtime
   state (frame_count, psx_cycle_count, in_exception, exception
   counters, sio/mc state, i_stat/i_mask, dirty_ram counters) to
   `psx_freeze_heartbeat.json` every 100 ms. Survives main-thread
   freezes — writer is independent of the SDL loop and the TCP
   debug server. Atomic file replace via .tmp + MoveFileEx so a
   reader never catches a half-written JSON.

   Already paid off: captured a freeze pattern (frame_count stuck,
   psx_cycle_count stuck, in_exception=0, current_func in
   install-stub kernel code) that the TCP server couldn't show
   because its accept queue was wedged by the same stalled main
   thread.

2. PSX_DEBUG_TOOLS CMake flag (`runtime/runtime.cmake`):
   - Default ON: builds with the TCP debug server, heartbeat
     thread, and per-block recording (debug_server_log_call_entry
     fires for every recompiled function entry).
   - OFF (`cmake -DPSX_DEBUG_TOOLS=OFF`): production build. The
     hot-path log functions early-return; TCP server doesn't
     start; heartbeat doesn't spawn. The ring buffer arrays still
     exist in the binary (their definitions live in
     debug_server.c) but no writes happen so the demand-paged
     memory stays committed at a much lower level (~77 MB at
     idle vs ~245 MB in debug mode).

   The flag propagates to psx-beetle via the option declared at
   the top of runtime.cmake.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 21:36:37 -07:00
Matthew Stanley 04a5490894 runtime/debug_server: raise listen() backlog to 16
Pure observability change — not a fix claim.

Both psx-runtime (was 1) and psx-beetle (was 4) had small accept-
queue backlogs. When the SDL main thread stalls (the exact state
we want to probe during a freeze investigation), the kernel
queues the first pending connect and rejects subsequent attempts
with RST. From the client side this looks like ConnectionRefused,
making it impossible to diagnose the stalled process from another
shell.

Backlog 16 lets a handful of probes queue up. Main thread will
eventually accept them in order if it ever resumes. If the
main thread is permanently dead the probes still fail, but at
least the failure mode changes from "looks like the server is
gone" to "the queue is genuinely deep + still stalled."

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 20:45:08 -07:00
Matthew Stanley 7faaba24f7 runtime/debug_server: surface dispatch-miss count in ping response
NES recomp template PRINCIPLES.md §13a calls a dispatch miss a
"SILENT GAME-BREAKING BUG" and mandates checking
`dispatch_misses.log` after every game run. We don't write that log
(no log files per CLAUDE.md §3) — we have an in-memory ring +
`unknown_dispatch_log` TCP command instead. But nothing routinely
surfaces the count, so silent misses can accumulate across sessions
unnoticed.

Now `ping` includes `dispatch_miss_total` and `dispatch_miss_unique`
fields. Every routine health check reveals if discovery has gaps.
0 = healthy. Non-zero on any session = stop and add the addresses
to game.toml / regen before continuing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 20:09:25 -07:00
Matthew Stanley f42c597090 runtime/debug_server: remove pause / step / run_to_frame entirely
Per CLAUDE.md global rule #2 (and direct user instruction): pause-and-
inspect is the wrong observation primitive for this codebase. It
synthesizes a snapshot at probe time rather than reading the system's
own continuously-recorded history. It also accidentally became a
freeze vector — a Python client that sent `pause` without `continue`
left s_paused=1 with no client to clear it, so the runtime spun in
debug_server_wait_if_paused forever and looked frozen.

Replacement is the ring buffers that already exist:
  - fn_entry_dump / fn_entry_tail   what code ran
  - wtrace_dump                     what memory was written
  - gpu_frame_dump frame=N          what GP0 commands were issued
  - mdec_trace                      MDEC events
  - sio_trace / sio_pc_trace        SIO history
  - frame_range / get_frame         per-frame state snapshots

Changes:
- handle_pause, handle_continue, handle_step, handle_run_to_frame
  now return send_err with a migration message pointing at the
  appropriate ring. The state variables (s_paused, s_step_count,
  s_run_to) stay declared and zero-valued so freeze_check fields keep
  reporting them as 0 (protocol stability).
- debug_server_wait_if_paused is now a no-op (kept exported so
  main.cpp's vblank callback stays unchanged).
- The post-frame `if (s_step_count > 0)` / `if (s_run_to > 0)` hooks
  are gated behind `if (0)` — dead code, kept as a comment marker.
- debug_server_dirty_break_maybe_pause no longer sets s_paused; it
  still records the hit (target, ra, regs, sp, frame) so
  dirty_break_state can be queried for the latest event.
- Reverted the client-disconnect auto-clear-of-pause logic added in
  40e9b79 — no longer needed, pause is gone.

Verified: pause/step/continue all return the migration error,
frame_count advances steadily (~50 fps) for 30 seconds, no freeze.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 13:47:18 -07:00
Matthew Stanley 40e9b79fb7 runtime/debug_server: fix "frozen" runtime — auto-unpause on client disconnect
Root cause of the apparent "freeze" observed after issuing debug
commands: tools/_dbg.py and similar one-shot Python clients open a
fresh TCP connection per command, send their request, read the reply,
then close. If the command was `pause` or `step`, the runtime is left
with s_paused=1 and no client to issue `continue`. The runtime then
spins in debug_server_wait_if_paused() forever — SDL window stops
updating, frame counter stuck — and the user reads this as "frozen"
even though the process is healthy.

The legitimate "open a persistent connection, pause, investigate,
continue" workflow keeps the socket open across all of that, so the
disconnect-then-clear behavior doesn't affect it.

Fix: when the client disconnects, clear s_paused / s_step_count /
s_run_to. Verified by sending pause via _dbg.py and watching the
frame counter keep advancing on the next freeze_check query.

Visibility: freeze_check now reports frame_count, paused, step_count,
run_to, client_connected. A "frozen" state can now be classified
without guessing — paused=1 = leftover-pause, paused=0 + frame_count
stuck = real game-loop wedge.

Ring cap reductions:
- WRITE_TRACE_CAP   4M -> 256K   (-360 MB worst-case)
- SIO_PC_TRACE_CAP  4M -> 256K
- MMIO_TRACE_CAP    4M -> 256K
- FN_TRACE_CAP      1M -> 256K
- FN_EXIT_TRACE_CAP 4M -> 256K

At 60Hz and ~thousand events/frame each ring still covers ~4 sec —
plenty for the diagnostic flows we actually run. Worst-case runtime
RSS drops from ~1 GB potential to ~100 MB.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 13:43:13 -07:00
Matthew Stanley 3cfcdaba17 runtime: per-frame GP0 command ring buffer
Always-on ring captures every GP0 command (header + first 6 payload
words) stamped with the frame number issued. Hooked at
gp0_execute_command() for fixed-length commands and at the polyline
start path for variable-length ones. ~70 MB at 1 048 576 entries
(34 bytes/entry); lazy-allocated on first record.

New debug-server commands:
- gpu_ring_stats: total/capacity/oldest+newest frame in buffer
- gpu_frame_dump frame=N count=M: per-frame GP0 entries with raw words

Designed per CLAUDE.md ring-buffer rule (CLAUDE.md global #2):
capture is continuous; observers query a window of interest
post-hoc. No arming, no timed attach.

Use case driving this: fuzzy menu text bug on Tomba title screen
(TombaRecomp ISSUES.md #3 'D'). Lifetime opcode counts couldn't
distinguish which 0x64/0x2C calls draw the menu vs the static
text. With per-frame dump we can now show that frame N has 88 GP0
commands and isolate the 6 0x2C entries that render with degenerate
top-right vertices (= the fuzzy text).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 12:25:29 -07:00
Matthew Stanley 2f709ee542 Phase B: config-driven unified runtime + BIOS shell boot restored
- Config loader (recompiler/src/config_loader.{h,cpp}): single source of
  truth for both BIOS and game runtime settings (debug_port, window_title,
  memcard_dir, disc path). psxrecomp-bios and psxrecomp-game both accept
  --config <toml>; runtime takes --game <toml> (no more CLI overrides).
- runtime/src/main.cpp: replaced --disc/--memcard-dir/--game-root CLI
  overrides with --game <toml>; game.toml is the single source of truth.
  Discless launch (no args) goes to BIOS shell.
- runtime/src/dma.c: fixed update_master_irq() re-asserting I_STAT bit 3
  on every DICR read. BIOS shell DMA handler at 0x80058628 only clears
  DICR bit 27 (ch3 flag), so level-sensitive re-assert pinned the kernel
  in the exception loop. Bit 3 is now edge-triggered (one assert per DMA
  completion). Discless boot now reaches MAIN MENU / MEMORY CARD / CD
  PLAYER. With-disc Tomba boots through to Whoopee Camp splash.
- Audit tooling: codegen_audit.py (BIOS), codegen_audit_game.py (game),
  audit_config.py (config loader for tools), verify_audit_schema.py.
  Replaces older tools/_codegen_audit.py probe.
- Docs: CURRENT_STATE.md (project state), UNIFICATION_PLAN.md (Phase C
  translator merge plan), docs/config_schema.md (TOML schema reference),
  docs/audit_inventory.md (audit tier inventory).
- .gitignore: tools/_*.py convention for ad-hoc scratch probes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 01:31:27 -07:00
Matthew Stanley 185162428e runtime + recompiler: boot through main menu, Load Game enter/exit
Recompiler: register jump-table targets with the cross-function
continuation pass before filtering local goto labels — BIOS jump
tables can legally branch into another discovered function's body,
so the prior local-only check dropped valid emit-side edges.

Dirty-RAM interpreter:
  - Dispatch non-local JAL/JALR targets back through psx_dispatch_call
    instead of forcing them into the interpreter when the target sits
    in static-recompiled code (only targets in dirty pages stay local).
  - New always-on flow + per-instruction ring buffers (insn_log,
    flow_log) capturing reg state, current_tcb, and task fields for
    post-hoc analysis. Block log gains t0/t1/t2.
  - guard_yields counter; richer last_unsupported state (entry PC,
    entry RA/SP, insn count) for dead-end diagnosis.

SIO: ACK is now visible across a small read window (2 STAT reads)
matching BIOS poll patterns that re-read STAT after CTRL_ACK clears
IRQ but expect ACK to still latch. Per-channel ack_irq_en captured
at shift start so a CTRL write that disables ACK_IRQ_EN mid-shift
doesn't suppress the firing. sio_init now zeros all paced-model
state. Read/write paths poll the debug server every ~1k accesses
so long pad/card spins don't starve diagnostics.

Memcard: auto-format slot files on first init when missing, so
the BIOS shell sees a present-but-empty card instead of "no card".

Runtime main: SDL nearest-neighbor texture scaling (no bilinear
blur on the integer-scaled framebuffer).

Debug server: large expansion of ring-buffer queries and dump
commands feeding the new insn/flow logs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 10:34:22 -07:00
Matthew Stanley 88a7ecdcaa runtime: fix Tomba FMV presentation audio and skip 2026-05-11 14:34:17 -07:00
Matthew Stanley cf05a0ab1f runtime + recompiler: Tomba boot through BIOS into Whoopee Camp FMV
Recompiler — `recompiler/src/function_analysis.cpp` now stops a
function's end_addr at its mapped `jr $ra` + delay slot instead of
stretching through trailing data. Fixes Tomba's 0x80076DB0, a real
136-byte function that was being emitted as a psx_unknown_dispatch
stub because data after it poisoned the boundary heuristic.
`code_generator.cpp` emits `func_<next>(cpu); return;` as a tail
call when a mid-function seed split leaves the physical fall-through
in a different C function, plus a `debug_server_log_call_entry` at
every function entry. `full_function_emitter.cpp` splits dispatch
into `psx_dispatch_impl(cpu, addr, stop_addr)` with public
`psx_dispatch` and new `psx_dispatch_call(cpu, addr, return_addr)`
so JAL/JALR into RAM stubs can return cleanly without growing the
native stack.

Runtime — `runtime/src/dma.c`/`include/dma.h`/`src/memory.c` now
expose `dma_write_masked()`; the 16-bit and 8-bit mmio paths route
DICR/DPCR/CHCR partial writes through it so a partial DICR write no
longer accidentally clears unrelated channels' pending IRQ flags,
and CHCR only triggers DMA when the written mask actually includes
the start bit. Unified `complete_transfer(ch)` across all channels,
removed `prev_master_flag` edge detection.

MDEC (`runtime/src/mdec.c`, `include/mdec.h`) gains a full debug
surface: `MDECDebugState` snapshot, an always-on 4096-entry
`MDECDebugEvent` ring with 11 event kinds (reset, ctrl_write,
cmd_begin/done, decode_done, dma_in/out start/end, output_drained,
read_underflow), and `mdec_state` / `mdec_trace` / `mdec_trace_clear`
debug-server commands. DMA ch0/1 emit start/end events around MDEC
transfers.

CDROM (`runtime/src/cdrom.c`, `include/cdrom.h`) — read stream
separated from one-shot pending state (`read_cmd`, `read_delay`,
`start_read_stream`, `stop_read_stream`, `deliver_read_sector`);
`cdrom_state`/`cdrom_trace` report the new fields.

Thread/fiber tracing — `traps.c`'s Win32-fiber TCB save/restore now
emits 12 thread-event kinds; new `ThreadTraceEntry` ring (64 K) plus
`thread_trace` / `thread_trace_clear` debug-server commands report
both live CPU regs and the target TCB's full saved register block.

wtrace / mmio rings — `WriteTraceEntry` grew (cpu_pc, sp, v0/v1/a0-a3/
t0/t1); capacity 1 M -> 4 M. `MmioTraceEntry` grew (pc, cpu_pc, sp,
a0-a3, sr, epc, i_stat, i_mask). Both dump commands accept
`count`/`newest=1` for reverse iteration. `restore_trace` and
`dirty_block_log` emit extended register fields (a2/a3/sp).

Misc — `interrupts.c` polls debug-server every 16K checks so it
stays responsive without a separate thread. `psx_runtime.h`'s
`call_by_address` routes through `psx_dispatch_call`. Dirty-RAM
interp JAL/JALR consult `psx_dispatch_game_compiled` when the game
is wired in. iso_reader gains the entry points needed by the game
disc path. New `tools/_tomba_probe.py` for port-4470 health checks.

Generated — regenerated `SCPH1001_dispatch.c` against the new
`psx_dispatch_impl` shape.

ISSUES.md — opened Issue #3 (BIOS PS-logo glyph missing on the
disc-detected screen, cosmetic).

.gitignore — added `recompiler/build-asan*/`, `build-checked/`,
`__pycache__/`.

Validated end-to-end: BIOS boots, disc detected, Whoopee Camp FMV
plays with audio. Frame count advances through ~40K frames without
freezes. `unknown_dispatch_log.total = 0`. Remaining FMV cluster
issues (1/4-size render, FMV-2 audio missing, no skip) tracked
under TombaRecomp/ISSUES.md.

Co-Authored-By: Codex (OpenAI) <noreply@openai.com>
2026-05-11 12:34:49 -07:00
Matthew Stanley 632e5dc662 runtime: BIOS thread switching via Win32 fibers + debug_server CDROM/dirty-RAM
traps.c — ThreadWaitFrame setjmp stack plus a Win32-fiber-based
HostThreadFiber pool for BIOS thread context switches. TCB save/restore
maps SR low-4-bits → low-6-bits via the standard PSX exception entry
shift. Backs the BIOS OpenTh/ChangeTh/ReturnFromException paths that
the kernel uses to schedule its own threads on top of the recompiled
exception handler.

debug_server.c — new TCP commands:
- cdrom_state: snapshot of index/stat/irq regs, FIFOs, pending cmd,
  seek/read MSF, i_stat.
- cdrom_trace: paginated dump of the cdrom_trace ring (kind/addr/val/
  func/pc/frame + state snapshot per entry).
- dirty_ram_unsupported: counts and last PC/insn/reason for opcodes
  the dirty-RAM interpreter rejected.
- dirty_ram_stats now also exports the full bitmap as
  dirty_bitmap_words[] (matches the bitmap promotion in this branch).

Both surfaces are query-only against always-on ring buffers, no
log files or stderr emission — consistent with the
no-stdio-debug rule.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 22:51:01 -07:00
Matthew Stanley 1c189530ba Add reusable runtime target plumbing 2026-05-10 15:54:15 -07:00
Matthew Stanley f75159c8d6 SPU ADSR + free-voice CURVOL fix: chime fans across all 24 voices
Root cause: BIOS music engine reads voice register 6 (byte offset 0x0C,
CURVOL / ADSR_LEVEL) to find a free voice; recomp returned the never-
written spu_regs value (always 0), so the engine saw every voice as
silent and over-recycled voices 0-3 instead of fanning across all 24.
After fix, recomp KEYON-per-voice counts match Beetle exactly (e.g.
v0 2/2, v18 2/2, v22 1/1).

Also:
- Full ADSR envelope (Attack/Decay/Sustain/Release) ported from Beetle's
  CalcVCDelta/RunEnvelope. KEYOFF now triggers Release-phase decay
  rather than immediate silencing.
- End-without-loop triggers Release (matches RunDecoder), not active=0.
- ENDX latch at 0x1F801D9C/9E now reports actual block-end-reached
  state instead of active-mask.
- SPU event ring (KEYON/KEYOFF/END_STOP/END_LOOP) on both backends with
  frame-stamped per-voice payload; 1 M entries each (32 MB) for chime +
  game-scene headroom. Beetle-side env>0 filter so silent-loop noise
  doesn't bury the audible-voice timeline (matches recomp behavior
  where deactivated voices don't fire further events).
- spu_voices / spu_events / spu_events_reset debug-server commands on
  both ports. send_fmt buffer kept single-shot to avoid extra newlines.
- tools/_spu_ring_diff.py: frame-aligned event-ring diff with per-voice
  count summary; chunk-count cap removed (was aborting reads at 98 KB).
- tools/_spu_capture.py, _spu_probe.py, _probe_one.py: targeted probes
  used during diagnosis.

Local-only beetle-psx change (gitignored): same env>0 filter at
mednafen/psx/spu.cpp around line 345/352. Rebuild beetle-psx static
lib if reapplying.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 19:09:48 -07:00
Matthew Stanley 2191f929b4 Smooth boot: gate per-dispatch diagnostics + 59.94 Hz wall-clock pacer
Boot was running at ~17 simulated FPS (12x slower than realtime) and
audio dropouts followed because the SPU pump generated 735 samples
per simulated vblank. Two changes:

* fntrace_record now no-ops by default (record only when armed). The
  legacy "record-all-when-unarmed" path made the runtime burn ~10%
  CPU writing every dispatch into a 4M-entry ring. Opt-in to
  record-all via fntrace_arm(0xFFFFFFFF).

* function_trace_record gated behind s_fn_trace_active flag (default
  off). Previously did a stack walk + tail-call detection + 9-field
  ring write on every dispatch, even when no fn-trace filter was set.
  Flipped on by handle_fn_filter; new fn_disable command turns it
  off again.

Combined: ~12x speedup. Recomp now hits 200+ simulated FPS uncapped,
matching beetle's ~230 FPS.

* Wall-clock pacer in sdl_vblank_present and beetle_main.cpp locks
  both backends to PSX-native 59.94 Hz. Audio queue (SDL device
  drains at 44100 Hz) now matches generation rate (735 samples * 60
  ≈ 44100/sec). TAB still works as turbo override on both windows.
2026-05-09 15:43:28 -07:00
Matthew Stanley 9b332ba5be Persist memcards and add SPU audio output 2026-05-09 06:32:22 -07:00
Matthew Stanley 3ef8b72791 Fix BIOS delete modal delay-slot dispatch 2026-05-08 23:08:29 -07:00
Matthew Stanley 122dc49f42 Fix BIOS memory card copy writes 2026-05-08 20:50:31 -07:00
Matthew Stanley 97f30d2a76 Preserve PSX memory-card debug state 2026-05-07 17:57:34 -07:00
Matthew Stanley cf87fd8fc6 crash_trace: structured post-mortem dump (mirrors SMW recomp pattern)
New runtime/src/crash_trace.c writes psx_last_run_report.json with:
  reason / timestamp / frame / last_func_addr / last_store_pc
  full CPU register state (all 32 GPRs + hi/lo/sr/cause/epc)
  dispatch_ring tail (last 256 dispatched func entries)
  unknown_dispatch tail (last 50 entries with addr/phys/ra/a0/a1)
  dirty_block_log tail (last 100 entries)
  Windows SEH info (when triggered from SEH filter)

Triggered from:
  - SIGSEGV / SIGABRT signal handlers
  - Windows SetUnhandledExceptionFilter (catches access violations)
  - atexit
  - psx_unknown_dispatch fail-fast path (set PSX_FAIL_FAST_UNKNOWN_DISPATCH=1)
  - trap_crash (existing fatal trap path)

File overwrites per dump (last-write-wins). Per CLAUDE.md §3 this is not
a log — it's a one-shot final state snapshot for crashes the running
TCP server cannot intercept.

Pattern mirrors F:/Projects/SuperMarioWorldRecomp/src/post_mortem.c.

Used the new infrastructure to capture a controlled crash with
0xBFC0989C re-seeded. Crash dump revealed the chain:
  - JAL at 0xBFC09788 is jalr $s3, NOT jalr $a0
  - $s3 was set in func_1FC09598 prologue: $s3 = $a2 (callee saves arg)
  - Caller at 0xBFC093C4 in func_1FC092EC: $a2 = $s3 - 128 (delay slot)
  - func_1FC092EC at 0xBFC09314: $s3 = $a3 (callee saves arg)
  So real bug is upstream-of-upstream caller passing $a3 = 131 (0x83)
  to func_1FC092EC, propagating down to a misaligned function pointer.
  The 0xA000E1E8 we suspected as a function-pointer table was a
  red herring — $a0 in the loop body is loaded from $fp+s1*4 (the
  ARG to pass to the called fn), not the call target.

0xBFC0989C reverted out of seed list (was diagnostic-only).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 23:45:33 -07:00
Matthew Stanley 6889f5c5d2 recompiler: seed missing modal_stub function (BFC1A1BC), add unknown_dispatch_log ring
Modal close on YES/NO COPY confirm now works on recomp. Cause: function
discovery (Ghidra + transitive walk) missed the function at 0xBFC1A1BC
which the BIOS shell calls indirectly during modal close. The 5 nearby
"missing" PCs the diagnostic identified are all in the same gap from
0xBFC1A1B8 to 0xBFC1A3B8; seeding 0xBFC1A1BC discovers the rest
transitively.

Status: modal closes cleanly on X. Actual copy execution still has
downstream missing-function-discovery issues (3 more PCs hit
psx_unknown_dispatch on copy: 0xBFC09890, 0xBFC0989C, 0xBFC02918)
plus likely additional bugs that softlock when those are seeded.
Those remain for follow-up; this commit is the modal-close milestone.

This is a BANDAID per user permission. The proper fix is to improve
function discovery so missed functions like 0xBFC1A1BC are found
automatically. Ghidra didn't see this function, and our transitive
walk didn't reach it either.

Tooling additions:
  - psx_unknown_dispatch_record() ring buffer + unknown_dispatch_log
    debug command. Replaces the prior file-based miss log per
    CLAUDE.md §3 (no log files, ever). Always-on; queryable summary
    (per-target counts) or tail (last N entries with addr/phys/ra/a0/a1).

  - WTRACE_MAX_RANGES bumped from 16 -> 64 (boot defaults were
    crowding out investigative arms).

  - DirtyRamBlockLogEntry +a0/a1 fields, ring CAP 16K -> 4M
    (previous 28ms eviction made retroactive press-window analysis
    impossible).

Diagnostic scripts (kept):
  tools/_unknown_dispatch.py    — query unknown_dispatch_log
  tools/_dispatch_check_v3.py   — was a PC ever dispatched (RAM->ROM map)
  tools/_dispatch_tail.py       — last N dispatched targets
  tools/_freeze_diag2.py        — softlock investigator

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 22:13:06 -07:00
Matthew Stanley 7a12a0baac two-process refactor: replace embedded oracle with independent psx-runtime + psx-beetle
Switches diagnostic methodology from a single binary running both backends
in lockstep (psx-beetleoracle.exe) to two independent processes:
  psx-runtime.exe (recomp only, port 4370)
  psx-beetle.exe  (Beetle libretro only, port 4380)

Both expose the same JSON wire protocol for the common debug commands
(read_ram, press, set_input, clear_input, pad_status, screenshot_file,
wtrace_*, fntrace_* on Beetle). Cross-process comparison is done by
querying both ports from a tool, NOT by sharing input or memory.

Why: the embedded-oracle setup required keyboard input to drive both
backends in lockstep. Once their state diverged, the same keypress
drove them to different screens and "compare on the same press" became
unreliable. Two independent processes lets each backend be navigated
to its own state and queried on its own timeline.

Removed:
  - runtime/src/beetle_psx_bridge.cpp (cannibalised into beetle_libretro.cpp)
  - runtime/src/duckstation_bridge.cpp
  - runtime/src/psx_oracle_cmds.c
  - runtime/src/psx_oracle_stub.c
  - runtime/include/psx_oracle_backend.h
  - psx-beetleoracle and psx-dsoracle cmake targets
  - 22 dual-backend tools (emu_read_ram / find_first_divergence / etc)

Added:
  - runtime/src/beetle_main.cpp     (SDL window + libretro driver + keyboard)
  - runtime/src/beetle_libretro.cpp (libretro callbacks + wtrace/fntrace/sio rings)
  - runtime/src/beetle_debug_server.c (TCP server, port 4380, same wire protocol)
  - psx-beetle cmake target

Also bumps tooling caps surfaced during modal investigation:
  - DIRTY_RAM_BLOCK_LOG_CAP: 16K -> 4M (28ms eviction -> ~7s retention)
  - WTRACE_MAX_RANGES:        16  -> 64  (boot defaults filled the prior cap)
  - DirtyRamBlockLogEntry: added a0/a1 fields for richer dispatch context

CLAUDE.md section 16 rewritten for the new methodology. New memory note
locks the direction so future sessions don't reach for the deleted
embedded-oracle commands.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 21:06:44 -07:00
Matthew Stanley bf60f821b1 debug-server: always-on wtrace on shell COPY/DELETE sub-state 0x80066BC0
Adds 0x66BC0..0x66BD0 to the always-on wtrace ranges. Used to find that
the freeze on COPY click in BIOS shell memcard menu happens because:

  - shell state-machine index at mem[0x80078320] advances to 7
  - state-7 struct at 0x80113C80 has handler=0x800321BC (RAM, non-null)
    and input_mask=0x00F000F0 (D-pad bits only)
  - non-D-pad buttons (CROSS/CIRCLE/TRIANGLE) are masked out in this
    state, and the D-pad handler at 0x800321BC never advances state

Memo follows separately. This commit is purely the diagnostic addition
so the wtrace coverage is in master for future repro.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 17:22:27 -07:00
Matthew Stanley 2c3de2acee debug-server: don't clear input override when TCP client disconnects
Python clients open a fresh socket per command — every press call would
return its OK, the client would close, and the server's recv path would
see the close and reset s_input_override = -1 the same frame. Net effect
was that every scheduled press lasted ~1 frame regardless of the
requested frame count, leaving the BIOS to see at most a single edge
event before the override evaporated.

Removing the reset lets s_input_frames count down naturally, so a press
holds for the full requested duration even though the issuing client
isn't connected for the duration. pad_status now reports
override=0xBFFF override_frames=110/99/88/... across half-second polls
during a 120-frame press, instead of override=-1 the moment after the
press is scheduled.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 16:57:39 -07:00
Matthew Stanley e88e9a0233 phase 4: fix mc_flag persistence so multi-sector card reads work
Before: only sector 0 read succeeded; sector ≥1 reads aborted at 2 bytes
because mc_flag was loaded from mc_slots[].flag on every 0x81 select but
never saved back. The clear in MC_CMD lived only in the working static, so
every reselect re-loaded the stale 0x08 ("new card / changed battery") init
value, and the BIOS bailed every multi-sector op as "card was just inserted,
re-init."

After: BIOS shell memcard menu reads the full 15-sector directory area for
both slots. Real save filenames decode (e.g. "BISLPS-01234SAVE00") and per
card icons render in the bottom panes — the Phase 4 success criterion.

Also bundled this session:
- debug_server: B9D0..B9F0 added to always-on wtrace ranges; fn_dump_parse
  default window scoped to 1M most-recent entries (filter scans against the
  full 128M ring used to freeze the debug server thread for many seconds).
- ISSUES.md + diagnostic tools/*.py written during the audit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 16:43:07 -07:00
Matthew Stanley 1b59103b3c runtime: card-read summary + card-data-writes diagnostic rings
Two passive, always-on rings for auditing memory-card reads end-to-end
without arming/timing probes:

card_read_summary (non-evicting, first 32 successful reads):
  Captured at the MC_READ_END transition in sio_process_byte. Records
  slot/cmd/sector/checksum/data_idx/first 16 bytes/cycle/caller func.
  TCP cmds: card_read_summary, card_read_summary_reset.

card_data_writes (4096 entries, rolling tail):
  Arm side: sio_read(SIO_RX_DATA) sets a pending flag while the card
  protocol is in MC_READ_DATA..MC_READ_END.
  Check side: psx_write_byte/half/word capture the next RAM write
  whose new_val matches the armed RX byte, then disarm.
  Records dest addr / value / store_pc / func / mc_idx / slot / cycle.
  TCP cmds: card_data_writes, card_data_writes_reset.

Helpers:
  tools/probe_card_reads.py
  tools/probe_card_writes.py
  tools/probe_card_writes_raw.py
  tools/peek_cards.py

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 22:20:55 -07:00
Matthew Stanley 479c51934a runtime: drive SIO TX via cycle-paced shifter
TX_DATA writes now route through the SIO0 shifter under
SIO_MODEL_CYCLE_PACED. Pad and card share the single-byte shifter and
TX buffer; bus owner is tagged on the first byte of each SELECT cycle.
sio_advance() is driven from psx_advance_cycles() so the shifter and
ACK pipeline progress on the guest CPU clock without touching the
legacy IRQ countdown.

The SELECT-deassert and TX_EN 1->0 paths now restore TX_RDY|TX_EMPTY
when they kill an in-flight shifter. Without this, sio_stat retained
the TX-clear bits set by SHIFT_START and the next handler busy-waited
forever on TX_RDY=0 inside the exception, deadlocking the BIOS.
Watchdog dump captured the deadlock at chain handler 0x445C with
stat=0x0000 across 4 seconds of PC samples.

Diagnostics added (passive/gated):
- starvation_ring records SIO MMIO events + periodic PC samples,
  watchdog dumps to JSONL on TCP-poll starvation
- starvation_walk.py queries the dump (last/last_change/why_idle)

Validation: shell-only sanity stable; one CROSS press no longer
hangs; mc_read_done climbs from 0 -> 30+ with mc_aborts=0.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 21:15:09 -07:00
Matthew Stanley 5d4d4bd30b phase1.0e-a: psx_cycle_count scaffold + cycle-timing arch doc
Scaffold for guest-cycle-driven peripheral timing per
docs/CYCLE_TIMING_ARCH.md. The counter is defined and exposed via
freeze_check, but psx_advance_cycles has no callers yet — counter
stays 0. Validation: zero behavior change.

Files:
- runtime/include/psx_cycles.h: extern uint64_t psx_cycle_count;
  void psx_advance_cycles(uint32_t cycles); uint64_t psx_get_cycle_count(void);
- runtime/src/psx_cycles.c: scaffold body; peripheral hooks empty
  (1.0e-b will add sio_advance, 1.0e-c will add gpu_advance, etc.)
- runtime/CMakeLists.txt: psx_cycles.c added to PSX_SHARED_SOURCES
- runtime/src/debug_server.c: freeze_check exposes psx_cycle_count
- docs/CYCLE_TIMING_ARCH.md: design doc — single cycle clock,
  generic per-block emit (no card-specific paths), MMIO-split codegen
  per explicit user direction, IRQ-pending-vs-CPU-entry separation,
  five-step migration plan (1.0e-a..e)

Validation against 0d15e2b baseline:
- t=20s/30s tx_writes match (1160/5480)
- 4x CROSS succeeds; card txn distribution unchanged (10 reach READ_DATA)
- "MC" bytes still flow at txn 0 bytes 10-11
- psx_cycle_count = 0 throughout (no callers in this slice)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 19:04:11 -07:00
Matthew Stanley 0d15e2bd4e phase1.0c-v2: cycle-paced sio_tick + dispatch quantum behind active-guard
Adds the cycle-paced sio_tick body, the sio_tick_quantum() wrapper, and
the pace_state TCP telemetry handler. Crucial change vs. failed 1.0c:
the dispatch-loop quantum tick is gated by g_sio_timing_active (volatile
int), so on the psx_check_interrupts hot path the cost is one load + one
branch when no SIO time is pending — not a full function call into the
cycle-paced loop.

In 1.0c-v2 the TX path is still synchronous and nothing arms shift_active
or pending_ack, so g_sio_timing_active stays 0. Phase 1.0d will set it
when arming the shifter and clear it when the shifter+buffer+ack are all
empty.

Files:
- sio.h: SIO_MODEL_CYCLE_PACED moved to header (default 1); declares
  g_sio_timing_active (extern volatile int) and sio_tick_quantum()
- sio.c: defines g_sio_timing_active=0; adds cycle-paced helpers
  (sio_consume_ack_event, sio_fire_ack_irq, sio_handle_shift_complete);
  sio_tick gains a cycle-paced loop that runs only if shift_active or
  pending_ack is set (currently never), then falls through to the legacy
  IRQ countdown body which remains the live IRQ delivery path; adds
  sio_get_pace_state(uint64_t out[16]) accessor
- interrupts.c: in psx_check_interrupts !in_exception block, calls
  sio_tick_quantum only if g_sio_timing_active is set
- debug_server.c: new pace_state TCP handler exposing the snapshot

Validation against 64e0194 baseline:
- t=20s/30s TCP responsive; pad arm audit 2840/2840 no_ack=0
- 4x CROSS succeeds
- card txn distribution unchanged (10 txns reach bytes=15 ts=10)
- txn 0 still reads "MC" magic at bytes 10-11 (0x4D 0x43)
- pace_state: timing_active=0, all dynamic fields 0 throughout

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 17:04:14 -07:00
Matthew Stanley 8b0fc34714 phase4: SELECT-deassert protocol reset + slot-state instrumentation
Real card protocol restarts on every fresh 0x81 — SELECT deassert is
a bus-wide reset signal. Earlier model preserved mc_state across
deassert via mc_save_slot, which caused every subsequent 0x81 to land
on a stale non-IDLE saved state and trigger abort_reselect bookkeeping.
Audit confirmed the BIOS-visible RX was still correct (the abort path
itself reset mc_state), but the retained sector/cmd/counter context
across logically-independent transactions was wrong per no$psx.

Clear volatile protocol state on SELECT deassert (state, cmd, sector,
sector_msb, sector_lsb, data_idx, checksum on both slots). Preserve
mc_slots[].flag (persistent "new card" metadata) and mc_slots[].data
(sector buffer, harmless to retain).

Result on a 4-press boot run: 6 transactions reach READ_DATA (ts=10)
and successfully read the "MC" header magic (RX bytes 0x4D 0x43);
remaining txns continue as 4-byte presence pings. Pre-fix: 0 reads
of card data ever; post-fix: card protocol can complete the setup
phase and enter data phase.

The byte-15 abort that follows is a separate blocker — kernel chain
dispatcher interleaves pad polling into the card data read. Tracked
for next session via the chain dispatcher disassembly.

Also includes:
- Slot-state capture in SioTraceEntry + sio_trace JSON emission
- dirty_ram_interp: replace mid-block fprintf+fflush with a counter
  (CLAUDE.md §3 — synchronous Windows console I/O at high rate
  starves the dispatch loop)
- Passive Python TCP probes (raw_tcp, trace_audit, trace_range,
  txn_summary, txn_dist, txn_bytes, drs_filter, readram_test,
  step_audit{,2}, verify_card_buffer, wtrace_filter)
- docs/MEMCARD_AUDIT_2026_04_28.md (prior session's 7-layer audit)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 14:46:26 -07:00
Matthew Stanley 5b3c7dd08d phase4: freeze_check diagnostic + accumulated overnight infra
Today's session work
- runtime/src/debug_server.c: freeze_check TCP cmd snapshots all liveness
  counters (PC, exception entries, exception_reentry_blocks, dirty_ram
  blocks/insns, fn_entry total + top-K histogram, SIO IRQ pending +
  countdown, sio_stat/i_stat/i_mask, mc_max_state) for tight-loop
  diagnosis when the runtime hangs
- runtime/src/debug_server.c: wtrace slot 12 covering [0x72F0..0x72F4]
  (data-byte counter); measured peak=1 confirming setup_data_phase resets
  the counter mid-cycle
- runtime/src/interrupts.c: exception_entries_total + exception_reentry_blocks
  counters + psx_get_freeze_diag accessor
- runtime/src/sio.c: sio_get_freeze_diag accessor for IRQ pacing internals
- memory entries: phase4_card_gate_setup_writes_2026_04_28 (refutes the
  prior \"missing [0x7568+slot] clearer\" framing — setup functions write
  LSB=0 directly), phase4_72f0_one_byte_per_vblank_2026_04_28 (peak=1
  measurement)

Overnight session work bundled in
- runtime/src/dirty_ram_interp.{c,h}: small MIPS interpreter for
  install-at-runtime kernel RAM stubs (Rule 18; e.g., SIO data-byte
  handler the BIOS writes to RAM 0xCF0)
- recompiler/src/full_function_emitter.{cpp,h}: 3057 jal/jalr continuation
  return labels added to dispatch table (continuation fix); branch
  fall-through fix for conditional-branch delay slots
- recompiler/src/strict_translator.cpp: matching translator updates
- generated/: regenerated SCPH1001 dispatch + full C output reflecting
  the recompiler changes
- runtime/include/debug_server.h, src/memory.c: card_txn ring + SIO IRQ
  ring + EvCB walk ring + per-PC dirty_ram counters + always-on wtrace
  coverage for [0x755A], [0x7514], [0x7528], [0x7520], [0x74A4],
  [0x75C0..C8], [0x74B8..C0]
- recompiler/tests/, runtime/tests/: SIO/card protocol unit tests
- tools/: 60+ Python diagnostic scripts (chain_trace analysis, SIO trace
  diff, EvCB walker, wtrace dumpers, etc.)
- docs/: psx_bios_disasm reference + dynamic_handler_install rationale
- OVERNIGHT_2026_04_28.md, SUMMARY_20260412_2.md: session logs

Phase 4 status (still blocked, root cause better understood)
- BIOS card chain runs through R-handlers correctly (chain_trace shows
  R1..R11 cycling 26x); shell IS retriggering _card_read
- Install handler at ROM 0xBFC15F1C increments [0x72F0] correctly per
  SIO IRQ; gate at <0x80 / <0x7F is correct
- Bottleneck: SIO IRQ countdown is access-paced, fires ~1× per VBlank
  vs ~128× needed; setup_data_phase (called from outer coord on VBlank)
  zeroes [0x72F0] before counter reaches 128
- Both attempted fixes today caused freezes:
  * VBlank deferral on [0x75C0]==1 — pending VBlank already in i_stat
    delivered anyway; tighter dispatch gate also locked up
  * Cycle-paced sio_tick at rates 50, 200 (and gated on [0x75C0]==1) —
    runtime freezes once data phase activates, root cause unidentified
- delay=5 narrows the threshold (peak=2 vs peak=1, no freeze) but
  doesn't get to 0x80
- Real fix likely requires deeper instrumentation of the freeze (call
  stack / dispatch state at the moment of TCP starvation)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 21:29:19 -07:00
Matthew Stanley 3971af1cc3 debug: pad_status TCP cmd + sio_card_protocol_active() accessor
pad_status reports both runtimes' current pad word side-by-side so
input dispatch symmetry can be validated empirically. Verified that
the existing `press` cmd already drives Beetle (via
psx_oracle_run_frame in main.cpp's per-frame loop); the bouncing-vs-
stable behavior the operator observed was downstream of input
(shell-state-machine pacing differences), not an input dispatch
asymmetry.

sio_card_protocol_active() accessor returns 1 if any slot has an
in-flight memcard protocol exchange. Building block for a future
fix to defer VBlank-driven pad polling while a card read is in
progress (root cause of the directory-load failure — see memory
phase4_real_root_cause_d3_to_d4 for the byte-level ours-vs-Beetle
divergence proof and the failed fix attempts).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 16:38:59 -07:00
Matthew Stanley 7c9ea5f397 debug: card txn ring + SIO IRQ ring + EvCB walk ring + diff/nav tools
Three new always-on diagnostic rings, plus tools to drive Beetle in
parallel and toggle the SDL window between ours and Beetle for visual
validation. Built to support deterministic root-cause analysis of the
memcard directory-load failure.

Card transaction ring (sio.c, sio.h): one entry per 0x81-rooted card
protocol run with full TX/RX bytes, cmd, sector, ack count, terminal
state, end_reason classification (success/abort_reselect/abort_reset/
abort_slot/abort_bad_cmd/abort_other). 4096-entry cap. Surfaced via
card_txn_dump TCP cmd. Live (in-progress) txn included separately.

SIO IRQ #7 delivery ring (sio.c, sio.h): every IRQ #7 fire with byte
seq linkage, mc_state at fire, active_device, ctrl, counter_7514,
i_stat before/after, source classification (card/pad), slot, applied
delay. 4096-entry cap. Surfaced via sio_irq_dump.

EvCB walk ring (debug_server.c): captures full kernel EvCB table
(class/status/spec/mode/fhandler) on every DeliverEvent entry. Reads
ToT layout per nocash: 0x120=ptr, 0x124=size. Exit detection tracked
via direct RA-watch (shadow stack unreliable due to
psx_restore_state_escape longjmp). Surfaced via evcb_snapshot,
evcb_walk_dump, evcb_walk_stats.

card_txn_diff (psx_oracle_cmds.c): aligns our structured txn ring
against Beetle's 64K SIO ring split into transactions, finds first
byte-level divergence, returns context window. SIO_FFD_MAX_BYTES
bumped 16K -> 64K so historical Beetle reads remain in scope.

emu_press (psx_oracle_cmds.c, beetle_psx_bridge.cpp): drives Beetle's
pad input for navigation parity with our `press` cmd. 180-frame cycles
verified to navigate Beetle to MC screen.

F7 toggle (main.cpp): swaps SDL window display source between ours
and Beetle's framebuffer for visual A/B comparison without restart.

Hygiene: 162 stale probe artifacts moved into probes/. .gitignore
extended for probes/ + root-level *.json/*.png/*.bmp/*.txt scratch.
Runtime data (card*.mcd, dummy.*.mcr) intentionally not covered.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 15:04:37 -07:00
Matthew Stanley 188bdc860d fn-trace: bump shadow stack depth 256 -> 4096
256 was too small under heavy fall-through dispatch (depth always
saturated at 256, exit tracking broken). 4096 keeps the stack
non-saturated for normal call patterns and allows fn_exit to
correctly pair with fn_entry.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 09:41:07 -07:00
Matthew Stanley 84180e2a31 debug: function ENTRY + EXIT trace rings with shadow call stack
Two always-on parallel ring buffers hooked off psx_dispatch via
debug_server_trace_dispatch:

- fn_entry: every dispatch in CALL ORDER with target, $ra, $a0..$a3,
  $t1 (B0/A0/C0 fn index when target is a BIOS vector), shadow-stack
  depth, frame
- fn_exit: every function in FINISH ORDER with $v0, $v1, plus
  entry_seq link back to its paired entry

Shadow call stack classifies each dispatch as:
- RETURN — target matches some open frame's saved RA → pop frames as
  exited, record exit values
- TAIL CALL — $ra unchanged from previous dispatch → replace top
  frame in place (keeps the stack bounded under heavy fall-through
  dispatch, which is how chain handlers tail-jump between states)
- NEW CALL — $ra changed (fresh JAL) → push new frame

When the 256-frame stack overflows, oldest frame is rotated out
(unbounded recursion / longjmp pathology) so tracking continues.

TCP commands:
- fn_filter {lo, hi}    set [lo, hi) phys addr filter
- fn_clear              reset both rings + shadow stack
- fn_stats              entry/exit totals, stack depth, overflows
- fn_entry_dump {count, seq_lo, seq_hi, addr_lo, addr_hi}
- fn_exit_dump  {count, seq_lo, seq_hi, addr_lo, addr_hi}

Validation: with filter set to BIOS card area 0x4000..0x6000, captured
state-1..state-11 each ran 4 times during one press-X cycle, all with
ra=0x4DDC (chain dispatcher's jalr return). Also revealed THREE
intermediate functions (0x59D0, 0x5A08, 0x5A48) called between state
addresses via tail-jump — visibility we did not have before.

Memory: 32 MB per ring × 2 = 64 MB heap-allocated at debug_server_init.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 09:33:37 -07:00
Matthew Stanley 8a694af69c debug: always-on wtrace coverage for card-chain state addrs
Per global rule: use ring buffers, never time/attach. Add static
always-on wtrace coverage at boot for the addresses that determine
card-chain progress, so we can answer "when was X written?" by
querying the ring instead of racing live samples.

Newly always-watched (range 5..9, total ranges 7→10, MAX 8→16):
- 0x7514..18: chain counter (shared by detection + read chains)
- 0x7528..30: per-slot chain handler ptr (=0x5688 read, =0x5B64 detect)
- 0x755A:     chain abort flag (D1/state-1 set, outer coord checks)
- 0x7520:     read-chain success flag (state-3 sets, cascade reads)
- 0x74A4:     chain status flag

This let us deterministically prove (without racing samples) that:
- read chain DOES get installed at 0x7528 briefly on press X (seq 12754)
- counter DOES advance 0→11 successfully during read-chain runs
- but resets at counter=11 via func_00005000's BFC14B90 abort path
- reset triggers because state-1 set 0x755A=1 and chain never
  reaches state-13 to clear it

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 09:00:26 -07:00
Matthew Stanley 3d364091e7 fix(memcard): clear FLAG byte after read — the four-day stall
This is the bug. Single line. Four days lost.

# The fuck-up

`runtime/src/sio.c` returned `mc_flag` (init 0x08) on every 0x52 (read
command) but NEVER cleared it. Only writes cleared the flag.

Per no$psx spec: real PSX cards return FLAG=0x08 ONCE — first read
after fresh-battery / power-cycle / write — then clear it to 0x00 for
all subsequent reads.

# Why this stalled directory loading

BIOS state-3 chain handler (func_0000579C / 0xBFC152DC) tests bit 3:

    andi $t1, $a0, 0x8
    beq  $t1, $0, BFC15650    # bit 3 NOT set: success (v0=0, advance)
                              # bit 3 SET:     "fresh card" (v0=-1)

Our card returned 0x08 forever, so EVERY read attempt landed in the
"fresh card" path: state-3 returned v0=-1, dispatcher's $v0 cascade
(BFC1499C) reset the chain counter, infinite loop. The directory
literally could not be read.

Beetle's card sim returns 0x00 in steady-state. That was the
canonical first divergence per recomp-template/NES/PRINCIPLES.md #3.

# Why we couldn't find it for four days

We didn't have a transaction-aligned SIO byte diff. The existing
sio_trace dumps were timing-dependent and didn't survive the
press-X window. We were chasing symptoms (dispatch misses, missing
state-12/13 seeds, chain counter reset paths, RAM divergences at
0x74F8 / 0x7560) — all real but all DOWNSTREAM of this one byte.

Per principle "BUILD TOOLS, NOT GUESSWORK" we built
`sio_first_divergence` (transaction-aligned ours-vs-Beetle byte diff,
splits each ring at 0x81 select, compares same-position TX/RX),
which immediately reported byte 1 of the read transaction:

    ours:    tx=0x52 rx=0x08
    beetle:  tx=0x52 rx=0x00

Two-line fix in mc_process_byte's MC_CMD case: clear `mc_flag = 0x00`
after returning it.

# Measured impact

Before:
- chain_trace state-3 returned v0=-1 every cycle
- 0x7514 counter cycled 0->1->2->3->4 then reset (BFC1499C path)
- mc_max_state stuck at 10 with mc_aborts ~30
- Both runtimes stuck at MAIN MENU shell selector

After:
- chain_trace states 1..11 all return v0=0, run 14+ times each
- counter no longer resets at state-3
- Both runtimes navigate to MEMORY CARD screen (matching layout)
- byte-1 SIO RX divergence eliminated (both return 0x00)

Memory cards still don't show save data (state-12/13 read-data
handlers don't fire because outer counter never reaches >=5 to enter
the read-chain path at BFC1588C). That's a separate downstream
divergence which the new diff tool will pinpoint next.

# Other artifacts in this commit

- recompiler/seeds/phase2_ghidra_seeds.json: added state-12 (0x5A58)
  and state-13 (0x5AB0) chain entries from no$psx-spec read-data
  handler ranges. Necessary for state-12/13 to dispatch when the
  outer chain finally enters the read-chain path. Dispatch table
  4295 -> 4297 entries, no dispatch misses.
- runtime/src/psx_oracle_cmds.c: new sio_first_divergence TCP cmd —
  splits each side's SIO ring into card-protocol transactions
  (0x81-start, 0x01 or next 0x81 ends), compares the most recent K
  transactions byte-by-byte to find first (matching TX, differing RX)
  or transaction-length mismatch.
- runtime/src/debug_server.c: chain_trace ring buffer + mc_status
  TCP cmd for chain-state visibility.
- runtime/src/memcard.c + memcard.h: memcard_debug_info() accessor.
- runtime/include/sio.h + sio.c: MC_READ_MSB_ECHO/LSB_ECHO states
  per no$psx address-handshake spec.
- generated/SCPH1001_*.c: regen with seed additions.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 00:46:31 -07:00
Matthew Stanley 711d22cff3 fix: per-slot card SIO state + card-specific IRQ delay
The SIO model had a single global mc_state shared between both card slots,
which caused 100% probe abort rate when the BIOS alternates between slot 0
and slot 1: each new 0x81 probe on the other slot would abort the in-flight
state from the previous slot.

Changes:
- Per-slot McSlotState saved/restored on slot switch via mc_save_slot/mc_load_slot
- 0x01 (pad select) preserves card state instead of aborting it
- SIO_IRQ_DELAY split into PAD (4) and CARD (8) — card delay tuned so the
  SIO chain walker can dispatch func_00004D6C before next idle path tick
  clears the protocol counter

Diagnostic tooling:
- I_MASK[7] trace ring buffer with bit-7 set/clear counts and caller info
- Counter (0x7514) field added to SIO trace so card protocol state machine
  progress is visible
- check_card.py inspects card directory contents

Runtime evidence: mc_aborts went from 100% → 0% (when slots don't overlap).
mc_max_state advanced from 1 (only MC_CMD) to 8 (MC_READ_DATA). dispatch_check
confirms _new_card AND its body 0x5688 are dispatched. Detection callback
(func_00005D94 at BFC15894) verified to clear v0=0 before return — continue
path of func_00004D6C IS being taken (133 I_MASK[7] sets observed).

Remaining: 0x7550 stays NULL despite _new_card being dispatched. Either
shell passes NULL buffer or early-return path of _new_card triggers.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-26 00:36:15 -07:00
Matthew Stanley 82aeca98a7 feat: embed Beetle PSX libretro core as golden oracle for RAM comparison
Adds a new psx-beetleoracle build target that embeds beetle-psx-libretro
(mednafen PSX core) alongside the recompiled BIOS. Both execute the same
SCPH1001.BIN and advance in lock-step, enabling byte-level RAM comparison
via the TCP debug server.

New infrastructure:
- psx_oracle_backend.h: abstract oracle backend interface (init, run_frame,
  get_ram, get_vram, get_scratchpad, read_byte/word, get_cpu_regs, sync_to_state)
- beetle_psx_bridge.cpp: libretro bridge with CPU register access via
  PSX_CPU->GetRegister(), VRAM via GPU_get_vram(), scratchpad via ScratchRAM
- psx_oracle_cmds.c: TCP commands — find_first_divergence, emu_read_ram,
  emu_read_vram, emu_read_scratchpad, emu_cpu_regs, emu_step, emu_sync,
  emu_sync_press
- psx_oracle_stub.c: no-op stubs for non-oracle builds
- tools/oracle_diff.py: standalone Python RAM comparison tool
- tools/build_oracle.sh: build script for beetle-psx + oracle target

Also includes: SIO card probe diagnostic counters (mc_probes, mc_acks,
mc_cmds, tx_writes), enhanced dispatch miss logging with register context,
memory_get_ram_ptr() for oracle RAM comparison.

Build: cd beetle-psx && make platform=mingw_x86_64 STATIC_LINKING=1 HAVE_LIGHTREC=0
       cd runtime/build && cmake --build . --target psx-beetleoracle

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-25 00:15:01 -07:00
Matthew Stanley 4054dc1c08 fix: DMA ch2 GPU→RAM now reads GPUREAD instead of writing zeros
The VRAM-to-RAM DMA path (channel 2, direction 0) was stubbed to fill
the destination buffer with zeros. This broke all offscreen VRAM
readback — the BIOS shell draws spiral/splash textures to an offscreen
VRAM region, then DMA-transfers the rendered pixels back to RAM for
later use as texture atlas entries. With the stub, all texture buffers
stayed zero, producing invisible button backgrounds.

Fix: call gpu_read_gpuread() for each DMA word instead of writing 0.

Also includes: CDROM GetID two-phase response fix, GPU raw texture
mode fix (opcode bit 0 instead of color==0x808080), debug server
A0/C0 upload history commands with CPU context capture, and extended
write trace ranges.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-24 14:30:57 -07:00
Matthew Stanley c4398aa35a debug: add DF8/EB4 write trace ranges for Phase 4.5 3D investigation
Track writes to 0x80079EB0-0x80079EC0 (EB4 status byte area) and
0x80079DF0-0x80079E04 (DF8/DFC/DF0 gate variables) at startup.
This traces the root cause chain: CDROM status → EB4 → DF8 → 3D gate.

Key findings from trace:
- DF8 set to 1 at frame 322 (init), NEVER cleared
- EB4 set to 0x10 (CDSTAT_SHELL) at frame 335 by func_1FC413B0
- func_800393FC(55,6) times out during boot (state doesn't reach 55)
- Root cause: events for state machine not delivered during boot init

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-24 01:05:11 -07:00
Matthew Stanley 5c6e00b00f feat: 5 new shell seeds + extended default write trace to 0xE300
Seeds BFC41D18, BFC41DC8, BFC41DFC, BFC41E14, BFC210F0 discovered
via DuckStation oracle TCB comparison. Extended default wtrace range
from 0xE1F0 to 0xE300 to cover EvCB divergence area at boot time.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-23 18:08:37 -07:00
Matthew Stanley 3b9ac6b84b perf: read_ram 16KB cap + 4KB divergence chunks + send_fmt 64KB buffer
- debug_server.c: increase read_ram from 256 bytes to 16KB max,
  use dynamic allocation for hex buffer; increase send_fmt buffer
  from 16KB to 64KB to hold large responses
- find_divergence.py: use 4KB chunks (was 256), add total diff byte
  count, remove run-length cap, remove hardcoded VSync exclusion,
  add data length verification on reads

512KB paused scan: 128s (was 1393s with 256-byte chunks)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-23 17:50:47 -07:00
Matthew Stanley bd6dfa12da fix: memcard absent detection + raw texture bug + GTE exec counter + quad capture
- memcard.c: stop auto-creating blank cards when .mcd files don't
  exist; missing file now correctly means "no card in slot"
- gpu.c: fix setup_textured_draw raw texture flag (was ? 0 : 0,
  now ? 1 : 0 for identity-modulation bypass)
- gpu.c: add shaded quad vertex capture mechanism for Phase 4.5
  debug (capture_quads / get_quads debug commands)
- gte.cpp: add gte_exec_count to track total GTE compute commands
  executed (exposed via gte_state debug command)
- debug_server.c: wire up capture_quads, get_quads commands and
  gte_exec field in gte_state response

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-23 15:33:19 -07:00
Matthew Stanley 00cc73f20e feat: GPU semi-transparency + 85 GTE library seeds + debug commands
GPU fixes:
- Mono tri/quad/rect/dot/8x8/lines: route through sw_renderer with
  semi-transparency support (bit 25 extraction + sw_set_semi_transparency)
- Add 16x16 mono sprite (0x78-0x7B) and 1x1 textured dot (0x6C-0x6F)
- Add per-opcode execution counter (gpu_get_opcode_count)

GTE library: batch-discovered 85 functions in BFC33000-BFC36000
including SetRotMatrix, SetTransMatrix, and full transform pipeline.
Emitted: 1173 (was 1086). Seeds: 955. Zero dispatch misses.

Debug commands: gpu_opcodes, gte_state, screenshot_file (BMP to disk).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-23 14:25:34 -07:00
Matthew Stanley 66c9f21f83 feat: MTC2/MFC2 special register handling + screenshot_file command
MTC2 to registers 15 (SXYP FIFO push), 28 (IRGB decompose), and 30
(LZCS leading-zero count) now routes through gte_write_data() for
correct side effects. MFC2 from registers 28/29 (IRGB/ORGB packed
read) routes through gte_read_data(). All other GTE register
transfers remain plain assignments for performance.

Added screenshot_file debug command: writes psx_screenshot.bmp to
disk (640x478 24-bit BMP), convertible to PNG for visual inspection.

Verified: BIOS shell renders correctly — main menu with Memory Card
and CD Player options, Memory Card management screen with CARD 1/2
labels, EXIT/COPY/COPY ALL/DELETE menu. Missing: 3D card slot
graphics (requires GTE 3D transform pipeline investigation).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-23 14:00:16 -07:00
Matthew Stanley 2767018b46 fix: SIO pad detection + input override + CARD driver seeds
Root cause: BIOS pad handler polls I_STAT bit 7 waiting for SIO ACK
IRQ. sio_tick() was only called from psx_check_interrupts and SIO
register accesses, but I_STAT reads go through memory.c which never
called sio_tick(). The IRQ countdown was stuck during the polling
loop, causing the BIOS to time out and report "no pad found" (0xFF).

Additionally, sio_tick() in psx_check_interrupts fired the IRQ too
early — during BIOS delay loops BEFORE I_STAT bit 7 was cleared —
so the BIOS cleared it and never saw it re-appear.

Fixes:
- memory.c: Call sio_tick() on I_STAT reads (32/16/8-bit paths)
- interrupts.c: Remove sio_tick() from psx_check_interrupts
- debug_server.c: Don't clear input override on TCP disconnect
  (was wiping override before next VBlank could read it)
- debug_server.c: sio_state reports real pad_buttons via getter
- sio.c/sio.h: Add sio_get_pad_buttons() for debug server
- seeds: Add 20 new seeds for shell input and CARD driver functions

Result: Pad buffer at 0x80083E30 shows status=0x00 (pad found),
type=0x41 (digital), buttons reflect keyboard/override state.
Only 5 COP2/GTE dispatch misses remain (Phase 5).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-23 11:22:39 -07:00
Matthew Stanley 70e7e7aea2 fix: enable DMA completion IRQ + DICR byte access + 5 VBlank seeds
Root cause of VSync counter stuck at 0: DMA completion IRQs were
deliberately suppressed in dma.c, preventing the shell's chain-E004
handler (func_1FC40628) from ever seeing I_STAT bit 3.  The handler
gates all VBlank callback processing on DMA interrupt presence.

Changes:
- dma.c: directly assert i_stat bit 3 on DMA ch2/ch6 completion
  (bypass DICR edge detection which gets stuck because the shell
  handler only acknowledges ch3 in DICR, not ch2/ch6)
- dma.c: expose dma_get_dicr()/dma_get_dpcr() for debug server
- memory.c: add DMA register range (0x1F801080-0x1F8010FF) to
  mmio_read8/write8/read16/write16 — fixes crash on DICR byte read
- debug_server.c: add DPCR/DICR to irq_state command
- seeds: add 0xBFC42BA0 (shell_vblank_dispatch), 0xBFC38AF0 (shell
  hot function, 178 misses), 0xBFC137D0 (kernel RAM), 0xBFC42BE0
  (shell_vblank_process), 0xBFC4226C (mid-function callback loop)
- regen: 1051 dispatch entries (was 1046)

Verified: chain-E004 handler + verify now dispatch.  VSync counter
still 0 because the VBlank callback loop inside func_1FC42160 is
gated by an init-done flag and the mid-function entry mechanism is
not yet resolved.  New EvCB crash at BFC12A84 exposed by DMA IRQ —
next session.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-21 20:06:15 -07:00
Matthew Stanley b3833aa253 fix: exception handler chain walk + shell reaches Memory Card/CD-ROM screen
Root cause: the recompiler split the kernel exception handler at the
jalr $s1 (chain handler dispatch) into func_00000C80 and func_00000E10.
The chain-walker continuation at BFC10910 never executed — only the
first handler in chain 0 was ever called.  I_STAT bits 0 (VBlank) and
2 (CDROM) were never acknowledged, causing an interrupt livelock.

Fixes:
- Emitter jalr/jal safety-net: when the return address falls outside
  the current function, emit a continuation psx_dispatch call after
  the jalr dispatch.  This bridges split functions at indirect calls.
- Merged exception handler: removed BFC10910 seed so the chain walker
  continuation is part of func_00000C80 (C fall-through after jalr).
- 0xBFC10910 dispatch special case in psx_unknown_dispatch: chain
  handlers return to $ra=BFC10910 via jr $ra (compiled as `return;`);
  the dispatch loop picks up the $ra value — handled as a no-op since
  the continuation already ran via C fall-through.
- Post-exception adaptive cooldown: if the handler didn't clear any
  I_STAT bits, give main code 500 blocks before re-firing.  Prevents
  VBlank livelock when no handler claims the interrupt.
- SIO byte write handler in mmio_write8 (was missing, caused crash).
- GPU→RAM DMA ch2 stub: fills target with zeros, completes transfer.
- Dispatch trace (debug_server_trace_dispatch) added to emitter
  template so it survives regeneration.

Result: shell device detection completes, StartPad dispatched, chain
E014 populated, GPU actively rendering the Memory Card / CD-ROM
screen.  VSync counter still 0 (handler installed but not yet
incrementing — next investigation).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-21 18:13:51 -07:00
Matthew Stanley 8e81afd111 fix: conditional branch fall-through + Tier 1 reverse debugger
Recompiler bug fix: full_function_emitter treated conditional branch
delay slots at function boundaries as fully resolved control flow,
suppressing the fall-through tail call for the "not taken" path.
This broke 15 cross-function fall-throughs across the BIOS, most
critically the find-free-EvCB-slot function (BFC11800) whose search
loop was silently skipped — causing OpenEvent to return garbage handles
(0xF100E1E8) instead of valid slot indices (0xF1000000..0xF1000005).

Fix: only suppress fall-through for unconditional flow (j/jr/jal/jalr/rfe).
Conditional branches (beq/bne/blez/bgtz/bltz/bgez) now always emit
`cpu->pc = next_addr; return;` for the not-taken path.

Also included:
- Tier 1 reverse debugger: 1M-entry write trace ring (32 MB),
  64K-entry MMIO trace ring (2 MB), 8-slot multi-range address
  filtering, function attribution via g_debug_current_func_addr.
  6 new TCP commands: wtrace_add/del/ranges/stats, mmio_dump/clear.
- 3 new dispatch seeds for functions exposed by fall-through fix
  (SIO epilogue, exception chain walker, GTE init).
- Diagnostic tools: analyze_sio_trace, decode_insn, decode_jt, mmio_tally.
- Pattern 5 trampoline fix + PLAN.md Phase 4 status update.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-21 14:23:09 -07:00