Observability for the 16:9 backdrop-cull investigation (runtime-only,
4:3-identity-safe, no regen):
- gpu.c: always-on per-prim draw-census ring (ws_census). Every drawn GP0
primitive (0x20-0x7F) records {frame, src_addr, camX/camY, first-vertex
screen x/y, opcode} so object spawn/despawn and edge culls are observable
in data instead of screenshots. gpu_ws_census_dump() writes a frame range
to CSV (large dumps must not ride TCP).
- gpu.c: gpu_ws_set_margin_override() forces psx_ws_x_margin() to any value
for a live cull sweep at a fixed camera; gpu_ws_get_debug() snapshots live
widescreen state.
- debug_server.c: gpu_state now reports ws.{configured,active,game_mode,
present_native_43,x_margin,squash,cur_frame,last_tag_frame}; new ws_margin
and ws_census TCP commands.
- gpu.h: GpuWsDebug + census/override API.
- debug_client.py: ws_margin and ws_census subcommands.
Used to prove the shipped [widescreen.cull] sites are a no-op for 8C (their
4:3 margins already cover the 16:9 view) and that the real pop is an
overlay-resident backdrop layer. See TombaRecomp ISSUES.md #8C.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- debug_server.c: dump_ram = alias of read_ram (one request, one response,
up to full 2 MB); screenshot = screenshot_file (24-bit BMP, optional
"path", default psx_screenshot.bmp); vram_peek cap 64->128; read_ram hex
encode via nibble table (was 2M snprintf calls stalling the main thread);
send_all_blocking bounded (1 MB chunks, SO_SNDTIMEO 2s, heartbeats the
starvation watchdog per chunk) and drops the client on send failure
instead of letting the watchdog kill the runtime; handle_quit tags exit
origin.
- beetle_debug_server.c: read_ram cap 64 KB->2 MB via heap+send_raw;
dump_ram/screenshot aliases; screenshot w/h->width/height parity;
set_input now holds until clear_input (frames param optional) matching
psx-runtime semantics — the old default expired after 1 frame and the
off-by-one in the getter cleared the override before it was ever read,
so scripted beetle navigation silently did nothing; wtrace_dump response
heap-built + send_raw (per-entry send_fmt dropped chunks on multi-MB
responses, producing broken JSON).
- crash_trace: new psx_crash_trace_set_exit_origin(); report gains
exit_origin (tcp_quit / sdl_window_close / unknown).
- debug_client.py: O(n) string-aware brace parser (old per-byte loop was
O(n^2) — minutes on a 2 MB response, which starved the server's send and
got the runtime killed); generic key=value passthrough for any server
command without a bespoke CLI mapping.
- TCP_COMMANDS.md updated for all of the above.
Verified: 2 MB read_ram round-trip 0.75 s; scripted beetle navigation
reaches the memcard file-select; wtrace_dump returns valid JSON at 21 MB.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Inline multi-MB dumps wedge the single-threaded debug server (third
occurrence: dirty_block_log count=16384). Block ring joins the insn ring
and event ring in having a file-dump path; client plumbing included.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Root cause of the dwarf-village->overworld native-overlay blue screen
(deterministic, every run, at native call #70):
The dirty-RAM interpreter's jal/jalr handlers preserved the call's
return-address obligation for statically-compiled callees
(psx_dispatch_game_compiled -> *next_pc_out = return_pc) and for non-local
targets (dispatch_nonlocal_call passes return_pc as stop_addr), but a LOCAL
DIRTY target fell through to a bare `cpu->pc = target; return 1`. When that
target was a NATIVE overlay candidate, the dispatch loop ran it as a C
function; its pc==0 return made psx_dispatch_impl unwind to the C caller,
silently dropping the suspended interpreted caller's continuation.
Concrete case: main-EXE field dispatcher F=0x800338A8 (undiscovered entry —
discovery walked it from 0x338B0; reached via data-table jalr, so it runs
under the interpreter) does `jal 0x8011B0D4` per entity. Under native exec
F's epilogue never ran: guest sp leaked 0x18 per entity until the world
corrupted. Interp-only mode is immune (the callee's own jr ra is interpreted).
Evidence chain (full-register fingerprint diff ON vs OFF localized first
divergence to sp at exec #70; frozen dirty-block ring shows F re-entered at
sp-0x18 with no epilogue between) is in docs §9 and the session memory.
Fix: overlay_loader_call_native() — run a validated native candidate as a
unit and resume at return_pc — called from both jal and jalr handlers,
mirroring the psx_dispatch_game_compiled contract exactly. Side effect:
native overlay coverage rose from 86 calls per village->overworld run to
~112,000 (interpreted calls now route to native candidates).
Verified: dwarf->overworld renders and plays under native overlays.
Same-scene A/B: host CPU 7.31 (native) vs 7.83 (interp) CPU-s per 15s wall.
Timing hypotheses refuted by experiment before fixing: IRQ-check cadence
(suppress armed, identical failure) and device-event granularity
(conservative stepping, identical failure).
Also in this commit (the capture tooling that found it):
- fingerprint ring now stores full entry/exit register files + guest cycle
(FP_CAP 2^20 crc-only -> 2^16 full-reg); overlay_fp_dump takes a path.
- insn_freeze <addr> <nth>: one-shot latch freezing insn/block/flow/fntrace
rings immediately BEFORE the Nth dispatch of a watched candidate, so the
pre-divergence window survives until dumped (ring-first, no arm-then-hope).
- dirty_insn_gate <lo> <hi>: runtime-settable extra PC range for the per-insn
interp log; native entry/exit markers (transferred=200/201) in that ring.
- dirty_insn_dump_file: file dump for the insn ring — the 16MB inline TCP
dump wedged the single-threaded debug server.
- freeze gating in the flow-log path gates ONLY the ring write, never flow.
- debug_client plumbing for the new commands + fntrace_arm/fntrace_dump.
- docs/overlay-recompilation-design.md §9: the dispatch call-contract
invariant, the Bug A case study, verification numbers, and known residue
(data-jump-table discovery-miss class; one unreproduced black-screen
one-off; §8.3 tail-position caveat).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Real card protocol restarts on every fresh 0x81 — SELECT deassert is
a bus-wide reset signal. Earlier model preserved mc_state across
deassert via mc_save_slot, which caused every subsequent 0x81 to land
on a stale non-IDLE saved state and trigger abort_reselect bookkeeping.
Audit confirmed the BIOS-visible RX was still correct (the abort path
itself reset mc_state), but the retained sector/cmd/counter context
across logically-independent transactions was wrong per no$psx.
Clear volatile protocol state on SELECT deassert (state, cmd, sector,
sector_msb, sector_lsb, data_idx, checksum on both slots). Preserve
mc_slots[].flag (persistent "new card" metadata) and mc_slots[].data
(sector buffer, harmless to retain).
Result on a 4-press boot run: 6 transactions reach READ_DATA (ts=10)
and successfully read the "MC" header magic (RX bytes 0x4D 0x43);
remaining txns continue as 4-byte presence pings. Pre-fix: 0 reads
of card data ever; post-fix: card protocol can complete the setup
phase and enter data phase.
The byte-15 abort that follows is a separate blocker — kernel chain
dispatcher interleaves pad polling into the card data read. Tracked
for next session via the chain dispatcher disassembly.
Also includes:
- Slot-state capture in SioTraceEntry + sio_trace JSON emission
- dirty_ram_interp: replace mid-block fprintf+fflush with a counter
(CLAUDE.md §3 — synchronous Windows console I/O at high rate
starves the dispatch loop)
- Passive Python TCP probes (raw_tcp, trace_audit, trace_range,
txn_summary, txn_dist, txn_bytes, drs_filter, readram_test,
step_audit{,2}, verify_card_buffer, wtrace_filter)
- docs/MEMCARD_AUDIT_2026_04_28.md (prior session's 7-layer audit)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Root cause: the recompiler split the kernel exception handler at the
jalr $s1 (chain handler dispatch) into func_00000C80 and func_00000E10.
The chain-walker continuation at BFC10910 never executed — only the
first handler in chain 0 was ever called. I_STAT bits 0 (VBlank) and
2 (CDROM) were never acknowledged, causing an interrupt livelock.
Fixes:
- Emitter jalr/jal safety-net: when the return address falls outside
the current function, emit a continuation psx_dispatch call after
the jalr dispatch. This bridges split functions at indirect calls.
- Merged exception handler: removed BFC10910 seed so the chain walker
continuation is part of func_00000C80 (C fall-through after jalr).
- 0xBFC10910 dispatch special case in psx_unknown_dispatch: chain
handlers return to $ra=BFC10910 via jr $ra (compiled as `return;`);
the dispatch loop picks up the $ra value — handled as a no-op since
the continuation already ran via C fall-through.
- Post-exception adaptive cooldown: if the handler didn't clear any
I_STAT bits, give main code 500 blocks before re-firing. Prevents
VBlank livelock when no handler claims the interrupt.
- SIO byte write handler in mmio_write8 (was missing, caused crash).
- GPU→RAM DMA ch2 stub: fills target with zeros, completes transfer.
- Dispatch trace (debug_server_trace_dispatch) added to emitter
template so it survives regeneration.
Result: shell device detection completes, StartPad dispatched, chain
E014 populated, GPU actively rendering the Memory Card / CD-ROM
screen. VSync counter still 0 (handler installed but not yet
incrementing — next investigation).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Recompiler bug fix: full_function_emitter treated conditional branch
delay slots at function boundaries as fully resolved control flow,
suppressing the fall-through tail call for the "not taken" path.
This broke 15 cross-function fall-throughs across the BIOS, most
critically the find-free-EvCB-slot function (BFC11800) whose search
loop was silently skipped — causing OpenEvent to return garbage handles
(0xF100E1E8) instead of valid slot indices (0xF1000000..0xF1000005).
Fix: only suppress fall-through for unconditional flow (j/jr/jal/jalr/rfe).
Conditional branches (beq/bne/blez/bgtz/bltz/bgez) now always emit
`cpu->pc = next_addr; return;` for the not-taken path.
Also included:
- Tier 1 reverse debugger: 1M-entry write trace ring (32 MB),
64K-entry MMIO trace ring (2 MB), 8-slot multi-range address
filtering, function attribution via g_debug_current_func_addr.
6 new TCP commands: wtrace_add/del/ranges/stats, mmio_dump/clear.
- 3 new dispatch seeds for functions exposed by fall-through fix
(SIO epilogue, exception chain walker, GTE init).
- Diagnostic tools: analyze_sio_trace, decode_insn, decode_jt, mmio_tally.
- Pattern 5 trampoline fix + PLAN.md Phase 4 status update.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Recovery + restructure after rm -rf destroyed duckstation/build/ earlier this
session. Adopts the nesrecomp pattern (see F:/Projects/nesrecomp): pristine
upstream as submodule, our changes as a patch file that gets auto-applied by a
setup script.
Submodule:
- duckstation/ now pins stenzek/duckstation@ffb33c281 (release-20260328 era)
- .gitmodules records the pin; .gitignore stops ignoring the submodule path but
still ignores build/ and dep/prebuilt/ (the submodule's own .gitignore also
handles these)
Patch + setup:
- tools/duckstation/psxrecomp_oracle.patch (1187 lines, 7 files) captures our
TCP debug-server on port 4371 plus the unfinished pc_break family of commands
(with the VirtualMemoryAddress compile fix)
- tools/duckstation/setup.sh clones submodule, fetches + verifies + extracts
prebuilt Windows deps, rewrites stale absolute paths baked into the prebuilt
CMake metadata, applies the oracle patch. Idempotent.
- tools/duckstation/build.sh configures CMake (VS 17 2022) + runs MSBuild, then
provisions bin/ with portable marker, BIOS copy, and settings.ini post-fixes
- tools/duckstation/README.md documents the layout, the regeneration flow, and
the known headless-boot configuration issue
TCP documentation (salvaged from v3):
- TCP_COMMANDS.md: full command table with native (4370) / duckstation (4371)
/ both columns, reflecting actual v4 implementation including wtrace_* and
pc_break/pc_hit_last families
- tools/debug_client.py: unified CLI from v3 (ports 4370/4371 match v4)
- DEBUG.md: merged v3's END-TO-END COMPLETION RULE, Stale Hypothesis Rule,
TOOLING RULE, and TOOL LIFECYCLE RULE sections into v4's existing loop
Phase 4 diagnostic artefacts from this session:
- generated/relocation_proofs/vblank_registrar/ — corrected VBlank-stall
diagnosis. Prior handoff claimed the shell VBlank handler was never installed;
live TCP inspection of both runtime and DuckStation oracle proved handler IS
installed at 0x800DFEE0. Real gap is one level up in the kernel IRQ chain at
*(0xA000E014).
- tools/duck_query*.py, duck_search.py, disasm_ram.py, scan_*.py — focused
probes used to establish the above; reusable against either server
CLAUDE.md §2 and new §15 document the submodule-based oracle setup and the
"never edit submodule, regenerate patch instead" rule.
Known issues (tracked in tools/duckstation/README.md):
- Fresh settings.ini doesn't complete BIOS boot under -nogui; one-time launch
with GUI is required to warm up the settings before headless mode works
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Matthew Stanley