The install-slot range work is only trustworthy if a second title can
reproduce it, and the two tools that produced its evidence lived in one
game repo with that game's paths baked in. Moved into the framework and
generalised: every path defaults off --project-root (cwd) and the BIOS
profile's out_stem, and the exe is found by globbing the build tree,
because its name is derived from the window title and so differs per
title. Run either from a game project root.
kernel_patch_diff.py boots headless and diffs live kernel RAM against
the ROM copy it came from, joining each differing word to the compiled
kernel bodies. It reads the emitted PsxKernelPatchRange table, so it
separates a DECLARED patch (the verifier skips it, the body still runs
native) from one that genuinely unblesses a body. Without that split its
verdict line calls every declared range "these interpret", which is
exactly backwards once the ranges work. This is the tool you use to find
ranges for a new image or a new Psy-Q release.
kernel_patch_ab.py answers "what did declaring them buy". It boots the
same binary twice to the same frame, once with the ranges live and once
with PSX_KERNEL_PATCH_RANGES=0, which drops them and restores the
whole-body bless memcmp. Holding the binary constant holds the overlays,
the codegen and the disc constant too, so only the bless verdict moves.
Headless runs uncapped, so it normalises per frame rather than comparing
raw totals at different frame counts.
Two things it gets right that cost real time to learn:
- It does NOT discount the A0/B0/C0 vectors from the headline. An earlier
version did, reasoning that they interpret on both sides and would mask
the result. True on one title, false in general: on Mega Man X6 with
OpenBIOS the B0 vector burns 2,361,470 instructions over 1,180,735
entries with the ranges off and vanishes entirely with them on, because
a clean bless verdict lets the vector body dispatch native. Discounting
it hid a 130 insn/frame win. Kernel and all-interp are the headline;
the bodies/vectors split is printed underneath as diagnosis.
- It fails fast on a wedge, detected as a frame counter that stops
advancing while the debug server still answers, instead of burning the
whole timeout. That is how a dispatch key inside a declared range was
caught spinning the dispatch loop at frame 0.
debug_client.query grew an optional timeout so these can use it instead
of carrying their own transport; the 10 s default is too short for
dirty_ram_stats and a whole-window read_ram.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
screenshot/screenshot_file read native 15-bit VRAM via gpu_display_pixel_rgb.
That is correct for faithfulness work but BLIND to any enhancement that only
exists in the supersampled mirror: at supersampling >= 2 the geometry-correction
sub-pixel vertices, SSAA edges and perspective UVs are resolved in the hi-res
surface and gone once pixels are packed back to native.
This bit hard. Native captures of Ape Escape looked clean while the user was
looking at a visibly cracked frame, so a real regression got reported as fixed
and the argument moved to counters. An instrument that cannot show the defect
is worse than no instrument.
screenshot_hires routes gr_render_display_hires — the same present path the
window uses — and falls back to the native resolve (reporting scale 1) when
there is no hi-res surface, so it always answers truthfully about real output.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both [video] enhancements are silent no-ops on anything they cannot prove is
projected geometry: a vertex whose sub-pixel fraction was never cached, or a
packet whose position words lack full GTE projection provenance, just draws the
faithful way. So the enable flag alone says nothing about whether a given title
benefits, and the existing hit counters had no reader.
geom_correction reports the flag plus the free-running geometry_vertex_hits and
perspective_triangles totals, so per-title bring-up can tell an engaged
correction from an inert one before anyone argues about screenshots. Sample
twice and diff for a per-window rate.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Observability for the 16:9 backdrop-cull investigation (runtime-only,
4:3-identity-safe, no regen):
- gpu.c: always-on per-prim draw-census ring (ws_census). Every drawn GP0
primitive (0x20-0x7F) records {frame, src_addr, camX/camY, first-vertex
screen x/y, opcode} so object spawn/despawn and edge culls are observable
in data instead of screenshots. gpu_ws_census_dump() writes a frame range
to CSV (large dumps must not ride TCP).
- gpu.c: gpu_ws_set_margin_override() forces psx_ws_x_margin() to any value
for a live cull sweep at a fixed camera; gpu_ws_get_debug() snapshots live
widescreen state.
- debug_server.c: gpu_state now reports ws.{configured,active,game_mode,
present_native_43,x_margin,squash,cur_frame,last_tag_frame}; new ws_margin
and ws_census TCP commands.
- gpu.h: GpuWsDebug + census/override API.
- debug_client.py: ws_margin and ws_census subcommands.
Used to prove the shipped [widescreen.cull] sites are a no-op for 8C (their
4:3 margins already cover the 16:9 view) and that the real pop is an
overlay-resident backdrop layer. See TombaRecomp ISSUES.md #8C.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- debug_server.c: dump_ram = alias of read_ram (one request, one response,
up to full 2 MB); screenshot = screenshot_file (24-bit BMP, optional
"path", default psx_screenshot.bmp); vram_peek cap 64->128; read_ram hex
encode via nibble table (was 2M snprintf calls stalling the main thread);
send_all_blocking bounded (1 MB chunks, SO_SNDTIMEO 2s, heartbeats the
starvation watchdog per chunk) and drops the client on send failure
instead of letting the watchdog kill the runtime; handle_quit tags exit
origin.
- beetle_debug_server.c: read_ram cap 64 KB->2 MB via heap+send_raw;
dump_ram/screenshot aliases; screenshot w/h->width/height parity;
set_input now holds until clear_input (frames param optional) matching
psx-runtime semantics — the old default expired after 1 frame and the
off-by-one in the getter cleared the override before it was ever read,
so scripted beetle navigation silently did nothing; wtrace_dump response
heap-built + send_raw (per-entry send_fmt dropped chunks on multi-MB
responses, producing broken JSON).
- crash_trace: new psx_crash_trace_set_exit_origin(); report gains
exit_origin (tcp_quit / sdl_window_close / unknown).
- debug_client.py: O(n) string-aware brace parser (old per-byte loop was
O(n^2) — minutes on a 2 MB response, which starved the server's send and
got the runtime killed); generic key=value passthrough for any server
command without a bespoke CLI mapping.
- TCP_COMMANDS.md updated for all of the above.
Verified: 2 MB read_ram round-trip 0.75 s; scripted beetle navigation
reaches the memcard file-select; wtrace_dump returns valid JSON at 21 MB.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Inline multi-MB dumps wedge the single-threaded debug server (third
occurrence: dirty_block_log count=16384). Block ring joins the insn ring
and event ring in having a file-dump path; client plumbing included.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Root cause of the dwarf-village->overworld native-overlay blue screen
(deterministic, every run, at native call #70):
The dirty-RAM interpreter's jal/jalr handlers preserved the call's
return-address obligation for statically-compiled callees
(psx_dispatch_game_compiled -> *next_pc_out = return_pc) and for non-local
targets (dispatch_nonlocal_call passes return_pc as stop_addr), but a LOCAL
DIRTY target fell through to a bare `cpu->pc = target; return 1`. When that
target was a NATIVE overlay candidate, the dispatch loop ran it as a C
function; its pc==0 return made psx_dispatch_impl unwind to the C caller,
silently dropping the suspended interpreted caller's continuation.
Concrete case: main-EXE field dispatcher F=0x800338A8 (undiscovered entry —
discovery walked it from 0x338B0; reached via data-table jalr, so it runs
under the interpreter) does `jal 0x8011B0D4` per entity. Under native exec
F's epilogue never ran: guest sp leaked 0x18 per entity until the world
corrupted. Interp-only mode is immune (the callee's own jr ra is interpreted).
Evidence chain (full-register fingerprint diff ON vs OFF localized first
divergence to sp at exec #70; frozen dirty-block ring shows F re-entered at
sp-0x18 with no epilogue between) is in docs §9 and the session memory.
Fix: overlay_loader_call_native() — run a validated native candidate as a
unit and resume at return_pc — called from both jal and jalr handlers,
mirroring the psx_dispatch_game_compiled contract exactly. Side effect:
native overlay coverage rose from 86 calls per village->overworld run to
~112,000 (interpreted calls now route to native candidates).
Verified: dwarf->overworld renders and plays under native overlays.
Same-scene A/B: host CPU 7.31 (native) vs 7.83 (interp) CPU-s per 15s wall.
Timing hypotheses refuted by experiment before fixing: IRQ-check cadence
(suppress armed, identical failure) and device-event granularity
(conservative stepping, identical failure).
Also in this commit (the capture tooling that found it):
- fingerprint ring now stores full entry/exit register files + guest cycle
(FP_CAP 2^20 crc-only -> 2^16 full-reg); overlay_fp_dump takes a path.
- insn_freeze <addr> <nth>: one-shot latch freezing insn/block/flow/fntrace
rings immediately BEFORE the Nth dispatch of a watched candidate, so the
pre-divergence window survives until dumped (ring-first, no arm-then-hope).
- dirty_insn_gate <lo> <hi>: runtime-settable extra PC range for the per-insn
interp log; native entry/exit markers (transferred=200/201) in that ring.
- dirty_insn_dump_file: file dump for the insn ring — the 16MB inline TCP
dump wedged the single-threaded debug server.
- freeze gating in the flow-log path gates ONLY the ring write, never flow.
- debug_client plumbing for the new commands + fntrace_arm/fntrace_dump.
- docs/overlay-recompilation-design.md §9: the dispatch call-contract
invariant, the Bug A case study, verification numbers, and known residue
(data-jump-table discovery-miss class; one unreproduced black-screen
one-off; §8.3 tail-position caveat).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Real card protocol restarts on every fresh 0x81 — SELECT deassert is
a bus-wide reset signal. Earlier model preserved mc_state across
deassert via mc_save_slot, which caused every subsequent 0x81 to land
on a stale non-IDLE saved state and trigger abort_reselect bookkeeping.
Audit confirmed the BIOS-visible RX was still correct (the abort path
itself reset mc_state), but the retained sector/cmd/counter context
across logically-independent transactions was wrong per no$psx.
Clear volatile protocol state on SELECT deassert (state, cmd, sector,
sector_msb, sector_lsb, data_idx, checksum on both slots). Preserve
mc_slots[].flag (persistent "new card" metadata) and mc_slots[].data
(sector buffer, harmless to retain).
Result on a 4-press boot run: 6 transactions reach READ_DATA (ts=10)
and successfully read the "MC" header magic (RX bytes 0x4D 0x43);
remaining txns continue as 4-byte presence pings. Pre-fix: 0 reads
of card data ever; post-fix: card protocol can complete the setup
phase and enter data phase.
The byte-15 abort that follows is a separate blocker — kernel chain
dispatcher interleaves pad polling into the card data read. Tracked
for next session via the chain dispatcher disassembly.
Also includes:
- Slot-state capture in SioTraceEntry + sio_trace JSON emission
- dirty_ram_interp: replace mid-block fprintf+fflush with a counter
(CLAUDE.md §3 — synchronous Windows console I/O at high rate
starves the dispatch loop)
- Passive Python TCP probes (raw_tcp, trace_audit, trace_range,
txn_summary, txn_dist, txn_bytes, drs_filter, readram_test,
step_audit{,2}, verify_card_buffer, wtrace_filter)
- docs/MEMCARD_AUDIT_2026_04_28.md (prior session's 7-layer audit)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Root cause: the recompiler split the kernel exception handler at the
jalr $s1 (chain handler dispatch) into func_00000C80 and func_00000E10.
The chain-walker continuation at BFC10910 never executed — only the
first handler in chain 0 was ever called. I_STAT bits 0 (VBlank) and
2 (CDROM) were never acknowledged, causing an interrupt livelock.
Fixes:
- Emitter jalr/jal safety-net: when the return address falls outside
the current function, emit a continuation psx_dispatch call after
the jalr dispatch. This bridges split functions at indirect calls.
- Merged exception handler: removed BFC10910 seed so the chain walker
continuation is part of func_00000C80 (C fall-through after jalr).
- 0xBFC10910 dispatch special case in psx_unknown_dispatch: chain
handlers return to $ra=BFC10910 via jr $ra (compiled as `return;`);
the dispatch loop picks up the $ra value — handled as a no-op since
the continuation already ran via C fall-through.
- Post-exception adaptive cooldown: if the handler didn't clear any
I_STAT bits, give main code 500 blocks before re-firing. Prevents
VBlank livelock when no handler claims the interrupt.
- SIO byte write handler in mmio_write8 (was missing, caused crash).
- GPU→RAM DMA ch2 stub: fills target with zeros, completes transfer.
- Dispatch trace (debug_server_trace_dispatch) added to emitter
template so it survives regeneration.
Result: shell device detection completes, StartPad dispatched, chain
E014 populated, GPU actively rendering the Memory Card / CD-ROM
screen. VSync counter still 0 (handler installed but not yet
incrementing — next investigation).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Recompiler bug fix: full_function_emitter treated conditional branch
delay slots at function boundaries as fully resolved control flow,
suppressing the fall-through tail call for the "not taken" path.
This broke 15 cross-function fall-throughs across the BIOS, most
critically the find-free-EvCB-slot function (BFC11800) whose search
loop was silently skipped — causing OpenEvent to return garbage handles
(0xF100E1E8) instead of valid slot indices (0xF1000000..0xF1000005).
Fix: only suppress fall-through for unconditional flow (j/jr/jal/jalr/rfe).
Conditional branches (beq/bne/blez/bgtz/bltz/bgez) now always emit
`cpu->pc = next_addr; return;` for the not-taken path.
Also included:
- Tier 1 reverse debugger: 1M-entry write trace ring (32 MB),
64K-entry MMIO trace ring (2 MB), 8-slot multi-range address
filtering, function attribution via g_debug_current_func_addr.
6 new TCP commands: wtrace_add/del/ranges/stats, mmio_dump/clear.
- 3 new dispatch seeds for functions exposed by fall-through fix
(SIO epilogue, exception chain walker, GTE init).
- Diagnostic tools: analyze_sio_trace, decode_insn, decode_jt, mmio_tally.
- Pattern 5 trampoline fix + PLAN.md Phase 4 status update.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Recovery + restructure after rm -rf destroyed duckstation/build/ earlier this
session. Adopts the nesrecomp pattern (see F:/Projects/nesrecomp): pristine
upstream as submodule, our changes as a patch file that gets auto-applied by a
setup script.
Submodule:
- duckstation/ now pins stenzek/duckstation@ffb33c281 (release-20260328 era)
- .gitmodules records the pin; .gitignore stops ignoring the submodule path but
still ignores build/ and dep/prebuilt/ (the submodule's own .gitignore also
handles these)
Patch + setup:
- tools/duckstation/psxrecomp_oracle.patch (1187 lines, 7 files) captures our
TCP debug-server on port 4371 plus the unfinished pc_break family of commands
(with the VirtualMemoryAddress compile fix)
- tools/duckstation/setup.sh clones submodule, fetches + verifies + extracts
prebuilt Windows deps, rewrites stale absolute paths baked into the prebuilt
CMake metadata, applies the oracle patch. Idempotent.
- tools/duckstation/build.sh configures CMake (VS 17 2022) + runs MSBuild, then
provisions bin/ with portable marker, BIOS copy, and settings.ini post-fixes
- tools/duckstation/README.md documents the layout, the regeneration flow, and
the known headless-boot configuration issue
TCP documentation (salvaged from v3):
- TCP_COMMANDS.md: full command table with native (4370) / duckstation (4371)
/ both columns, reflecting actual v4 implementation including wtrace_* and
pc_break/pc_hit_last families
- tools/debug_client.py: unified CLI from v3 (ports 4370/4371 match v4)
- DEBUG.md: merged v3's END-TO-END COMPLETION RULE, Stale Hypothesis Rule,
TOOLING RULE, and TOOL LIFECYCLE RULE sections into v4's existing loop
Phase 4 diagnostic artefacts from this session:
- generated/relocation_proofs/vblank_registrar/ — corrected VBlank-stall
diagnosis. Prior handoff claimed the shell VBlank handler was never installed;
live TCP inspection of both runtime and DuckStation oracle proved handler IS
installed at 0x800DFEE0. Real gap is one level up in the kernel IRQ chain at
*(0xA000E014).
- tools/duck_query*.py, duck_search.py, disasm_ram.py, scan_*.py — focused
probes used to establish the above; reusable against either server
CLAUDE.md §2 and new §15 document the submodule-based oracle setup and the
"never edit submodule, regenerate patch instead" rule.
Known issues (tracked in tools/duckstation/README.md):
- Fresh settings.ini doesn't complete BIOS boot under -nogui; one-time launch
with GUI is required to warm up the settings before headless mode works
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Kevin