Commit Graph

367 Commits

Author SHA1 Message Date
Matthew Stanley e8ad8ade3d debug: always-on display ring + full-VRAM aux capture
New always-on ring in the debug server: every vblank captures the display
area's raw 15-bit pixels AND the full 1024x512 VRAM as the ACTIVE RENDERER
sees it (GL: FBO truth via fbo_peek; SW: CPU truth), 64 frames of lookback.
Commands: display_ring_get (frame -> PNG), display_ring_aux (frame -> raw
u16 VRAM blob), display_ring_stats.

Purpose: FRAME-EXACT visual forensics. screenshot_file returns whatever is
on screen when the command lands (1-3 frames late), which cannot support
cross-renderer same-frame comparison; the ring can. Proven use (MMX6 sprite
triage): pulled the SAME deterministic frame from a GL run and a software
run, micro-cosimed the suspect sprite prim from the GP0 ring against the
same-frame captured texels/CLUT, and established GL and SW render the scene
pixel-identically with byte-identical texture content - reclassifying the
reported artifact from renderer bug to (pending Beetle oracle) game content.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 18:26:31 -07:00
Matthew Stanley e86099f46e gpu gl/vk: fix mirrored-2D uv sampling (MMX6 sprite slivers)
Modern-GPU center-sampled interpolation vs the PS1 DDA's top-left-corner
latch: the sample-grid shift (u_shift = 0.5/S - 1/64) makes floor(uv) land
on the exact PS1 texel for FORWARD axis-aligned mappings, but MIRRORED ones
(u or v decreasing along their screen axis - X/Y-flipped 2D sprites, e.g.
right-facing MMX entity quads whose cels are stored facing left) interpolate
1/64 short of each integer and floor one texel LOW. That painted the cel's
never-sampled edge column/row (opaque body texels) as detached 1px slivers
at quad seams: stray sprite-colored lines right of X when facing right, a
pink line under the rolling wheel mechaniloids. The old tri_uv_limits
comment explicitly 'accepted' this.

Fix = the Beetle-PSX / parallel-psx Calc_UVOffsets_Adjust_Verts model:
bump the prim's uv by +1 along the decreasing axis (tri_uv_mirror_offset,
applied in the GL and VK poly paths before limits are derived). This also
puts the max-uv vertex back on the exclusive raster edge, so the existing
unconditional max back-off in tri_uv_limits is correct for both directions.
Forward mappings are untouched (no regression surface for the original
Tomba seam fix). The software renderer's native DDA truncates exactly like
the PS1 and needs no compensation; its limits function only feeds the
bilinear neighbour clamp (comment clarified).

Verified on MMX6 attract demo (deterministic boot): pre-fix GL showed
detached X-palette 1px columns at quad seam x=173/175/179/181 (frames
~2088/2250) while software was clean; post-fix GL matches software and
user-confirmed clean live. Residual detector hits classified via GP0 ring
dumps as legit charge-spark effect tiles.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 15:40:32 -07:00
Matthew Stanley 0b23ea3318 vulkan: native-wide (16:9) compositor - R2 item 5
Mirrors the GL design with the Part-A lessons baked in:
- Per-base_x wide surfaces (RGBA8 color + OWN stencil image + framebuffer on
  the SHARED render pass, so every existing pipeline works unchanged and the
  mask-bit stencil mirror is real on the wide surface from day one - GL
  originally shipped a stencil-less wide FBO).
- The mirror is a second render pass appended to the SAME one-shot command
  buffer as the canonical batch (flush_geometry + flush_tex_batch): no extra
  submits, u_xoff/u_xhalf push constants carry the x-translation/wider clip
  (the VK shaders were pre-plumbed for this).
- Mirror scissor = full-width X but DRAW-AREA Y (the GL band-bleed lesson:
  full-height Y let draws bleed across vertical double-buffer bands and
  flicker the top/bottom edges).
- wide_clear via vkCmdClearAttachments (color + stencil to bit15) over the
  cleared band; full-screen overlay rects suppress the batch mirror and draw
  one full-wide-width rect instead (margins dim/fade correctly).
- GPU-direct present: vk_renderer_present_wide blits the displayed band from
  the wide image to the swapchain, letterboxed at (4*wide_w : 3*native_w);
  returns 0 -> caller falls back (unchanged) when no surface exists.
- vkb_render_wide_display CPU readback backs gr_wide_supported() + the ws
  debug dump; wide entry points no-op when nothing changed (they sit inside
  guest emulation - Part-A lesson).
- vk_perf gains wide/wclr per-frame counters.

First validation: Tomba2 build-vk 16:9 (PSX_WS_FORCE_2D=1) locks 60fps
(frame_period p50 16.68ms) with wide passes active; wide surface allocs are
one-time (vk_perf allocs ~0 steady state).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 20:56:06 -07:00
Matthew Stanley 8b819eb7e0 interp: word-tag the ws site caches - stale verdicts corrupted the guest
MMX6 16:9 attract wedged (misaligned wild dispatch 0x21010001 -> fatal halt,
stack-sampled at psx_unknown_dispatch under dirty_ram_dispatch). Mechanism:
the enlarged site caches persist across overlay reloads, and the generation
counter only sees page-marking hooks (kernel writes + DMA executable-range
marks) - code replaced by plain CPU stores (loader memcpy) keeps the page
dirty and never bumps the generation, so a cached backdrop verdict derived
from the PREVIOUS overlay bytes could fire a GPR rewrite at the wrong
instruction. The old 256-slot cache had the same hole but evicted within
seconds; the 8192-slot cache widened the exposure to a real wedge.

Each entry now also stores the site instruction word and revalidates it on
lookup (one fetch_word) - any code change at the site forces re-derivation.
Cost on the hot path: one load + compare per candidate instruction.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 20:51:17 -07:00
Matthew Stanley dbe7812a6a interp: fix the REAL native-wide CPU collapse - per-instruction ws site scans
Stack-sampled (devkitPro gdb, Tomba2 16:9 attract in the 12fps wedge): the
main thread sat in ws_backdrop_site_kind <- exec_one. Under native-wide,
exec_one classified EVERY interpreted instruction as a potential backdrop
rewrite site; the classifier scans +/-512 bytes of RAM per miss, and its
256-slot direct-mapped cache (any two hot PCs 1KB apart collide) thrashed
on overlay-sized working sets, so hot interpreted loops re-scanned per
executed instruction. Squash mode gates the whole path off, which is why
ws_nw on=0 instantly restored 60fps while every GPU-side theory (mirror
passes, FBO ping-pong, extra prims, present path) failed - the earlier
GL GPU-time numbers were CPU-starvation-inflated timestamp gaps.

Fixes:
- exec_one pre-filters by opcode class first (addu/or/addi/addiu are the
  only rewrite-site forms) - detection no longer runs for ~85% of instrs.
- Both site caches (cull + backdrop) grow to 8192 full-PC-tagged slots.
- NEW g_dirty_ram_code_gen (memory.c): bumped on clean->dirty page
  transitions and every executable-range mark (overlay/EXE loads); cached
  classifications carry the generation, so an overlay reload re-derives
  them. This also fixes the latent staleness bug where both caches could
  serve a kind derived from a PREVIOUS overlay bytes at the same PC.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 20:34:41 -07:00
Matthew Stanley cfa79bbc57 gl: native-wide perf collapse root cause - wide FBO lacked a stencil attachment
Proven by ablation on Tomba2 16:9 attract (12fps): a full mirror pass cost
~616us GPU per pass, but ablate=2 (FBO binds + state churn, no draws) = 3us
and ablate=3 (identical draws into the stencil-equipped hr FBO) = 1.5us.
The only difference between the fast and catastrophic configurations is
drawing with stencil test/write enabled (every batch's mask fixup pass)
into the one FBO with no stencil attachment - the driver handles that
combination with ~0.6ms of per-draw work. This also explains why the
deferred once-per-frame replay experiment (ws branch a8181dd) did not help:
it still drew into the stencil-less wide FBO, so the per-draw cost stayed.

Fix: each wide surface gets a DEPTH24_STENCIL8 renderbuffer like the hr FBO.
This is also a correctness fix: the wide surface now carries a real PSX
mask-bit stencil mirror (previously mask set/check silently no-opped on the
wide pass). wide_clear clears the stencil to bit15 alongside the color, and
surface creation clears both.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 20:26:53 -07:00
Matthew Stanley e9b4ed5bbf gl: frame_perf CPU-side attribution - flush/wide wall time + plumbing counters
Tomba2 16:9 GL reproduced the reported collapse (12-14fps) and the GPU-side
split EXONERATED the GPU: scene_gpu ~10ms, mirror ~0.2ms, while emu_cpu is
~82ms (vs ~15ms in squash). gl_ws_ablate=1 (mirror passes fully off) leaves
it at ~75ms, so the wedge is CPU work outside the mirror draws. The GL
submission paths run INSIDE guest emulation (GP0/DMA processing), so driver
CPU cost lands in emu_cpu where no GPU timer sees it.

frame_perf now reports, per wide/4:3 frame class: CPU wall inside
flush_tex_batch (cpu_flush_ms_avg), CPU wall inside the glb_wide_* entry
points (cpu_wide_ms_avg), batches/frame, wide target sets/frame, and wide
FBO creations/frame (a nonzero steady value = surface recreation churn).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 20:23:06 -07:00
Matthew Stanley a0b58433c2 gl: fix native-wide band bleed - mirror scissor Y clamps to the draw area
The wide-mirror pass scissored the FULL wide surface (both X and Y). The SW
reference (rt_wide) only widens X; its Y clip stays the canonical draw-area
clip. Under a vertical double-buffer (MMX6: draw area alternates y=0/y=240,
both bands in ONE full-height wide surface keyed by base_x) the full-height
Y scissor let draws that canonically clip at the band boundary bleed into
the OTHER band's rows, presented a frame later as user-visible top/bottom
edge flicker (16:9 GL only). Scissor is now full-width X, draw-area Y,
matching rt_wide exactly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 20:04:19 -07:00
Matthew Stanley f5362f4ce9 gl: frame_perf mirror-pass GPU split + gl_ws_ablate ablation toggles
Native-wide perf attribution instrumentation (permanent):
- Each wide-mirror pass (tex-batch mirror, flat-geometry mirror, overlay
  full-width rect, wide_clear) is bracketed with a GL_TIMESTAMP pair
  (glQueryCounter; TIME_ELAPSED cannot nest inside the frame scene query).
  Pairs pool per buffered frame (1024/frame), summed at readback into
  mirror_gpu_ms + mirror_passes per sample; frame_perf now reports
  mirror_gpu_ms_avg/max, canon_gpu_ms_avg (scene minus mirror),
  mirror_passes_avg and mirror_pass_us for wide frames.
- New debug cmd gl_ws_ablate mode=<0..3>: 0 normal, 1 skip mirror passes
  entirely, 2 mirror state churn without draws, 3 mirror draws stay on the
  hr FBO (no per-pass FBO rebind; content-corrupting perf probe).

First MMX6 16:9 measurements with the split: mirror passes cost 20-125us
EACH (FBO ping-pong penalty per pass) but MMX6 2D batches to ~3-15
passes/frame so 16:9 attract holds ~60fps on this branch; the per-pass
cost x ~900 immediate flat prims is exactly the Tomba2 3D 60ms scene.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 19:59:53 -07:00
Matthew Stanley d0cb91313f vulkan: FMV verified + CPU-present resource cache + explicit DS stencil barrier
- FMV on-screen verification (gap 2): Whoopee logo movie + intro CG movie
  render correctly on VK (window captures vs the earlier software runs).
  Finding: Tomba2 streams movies as 15-bit MDEC->VRAM uploads, not depth24 —
  the depth24 compose path is no-regression code pending a 24-bit title.
- cpres cache (gap 3): the depth24 CPU-present path now keeps a persistent
  VkImage + persistently-mapped staging keyed by (w,h) instead of
  creating/destroying both every movie frame; freed on resize + shutdown.
  The existing per-frame queue-idle wait is what makes reuse race-free.
- DS barrier (gap 4): begin_geo_pass now issues an explicit stencil-aspect
  self-barrier (LATE_FRAGMENT_TESTS write -> EARLY|LATE tests read/write) so
  cross-submit mask-bit ordering no longer rides on color-barrier side
  effects (spec-fragile). Layouts verified against the init chain and the
  render pass (DEPTH_STENCIL_ATTACHMENT_OPTIMAL throughout).

Noted for later (ENHANCEMENTS R2 4b): flush_cpu_upload's 2 staging allocs
per flush (~16/frame during MDEC FMV only, 0 in gameplay).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 19:20:35 -07:00
Matthew Stanley 5d8a0f19df vulkan: exact-rect pending uploads with a coalesced two-submit flush
Ports the GL stale-stomp fix (R1: a union of disjoint uploads painted stale
CPU-mirror pixels over live framebuffer content) to the Vulkan backend, with
a VK-specific flush design: the naive per-rect port re-created the boot-wedge
submit churn (2 staging allocs + 3 submits PER RECT; measured 0.7 fps at the
Whoopee logo under MDEC streaming). The coalesced flush packs all rect data
into ONE 1555 staging + ONE RGBA8 staging (VkBufferImageCopy bufferOffset per
rect), then exactly TWO queue submits per flush: one command buffer with both
N-region buffer->image copy sets, one render pass with N scissored 2-draw
blits into hr. UP_RECTS_MAX=64 on VK (row-band merging coalesces MDEC
macroblock streams to ~one rect per block row, so a streamed frame fits in a
single flush).

Verified: Tomba2 attract renders correctly on Vulkan at ~51 fps; vk_perf
shows mostly zero-submit frames with occasional small bursts (was a
sustained per-pixel churn wedge before the original batching fix).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 17:32:18 -07:00
Matthew Stanley f5ea3b0d76 vulkan: bring-up to full-speed gameplay — batched uploads, VB cursors, STP order, FMV path
Three root-caused fixes (verified with guest-frame-aligned VRAM diffs vs the
software reference + window captures; Tomba2 title now pixel-identical to
software, attract within 0.62% of the GL oracle, 60.5 fps sustained):

1. Boot wedge: every GP0 pixel upload did 2 vkAllocateMemory + 2
   vkQueueSubmit (driver churn, minutes-long stall, watchdog abort).
   Deferred batched uploads (GL model): CPU writes land in the CPU array +
   dirty rect; one batched upload flushes at each GPU-op entry point.
2. Shredded 3D: draws raced CPU rewrites of the persistently-mapped vertex
   buffer from offset 0 (69.5% pixel divergence at guest frame 3000).
   Sub-allocation cursors + firstVertex bases, recycled at queue-idle.
3. Semi-transparency order violations (59.5% divergence): VK still had GL's
   retired whole-batch STP split; ported GL's current model (ordered color
   pass + color-masked stencil fixup via a new pipeline dimension; semi
   prims isolated one-per-batch).

main.cpp: Vulkan present block composes 24-bit FMV frames via
gpu_display_pixel_argb + vk_renderer_present_cpu (was NULL -> guaranteed
black video; on-screen verification still owed) + the no-VK-build
downgrade-to-GL guard salvaged from the retired _wt-vulkan worktree.
debug_server screenshot_file now syncs the VK backend (previous commit).

Known follow-ups (ENHANCEMENTS.md R2): port the exact-rect upload fix from
GL (VK carries the union model = same flicker class), verify FMV
on-screen, cache the FMV staging image, explicit DS barrier between
one-shot submits, wide compositor, SSAA validation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 16:27:06 -07:00
Matthew Stanley 2d2f4aebb4 gl: fix black-frame flicker (ISSUES #7) — exact pending-upload rects, not a union
Root cause (proven via the new present ring + gl_coh_ring correlation):
flush_cpu_upload() merged all pending CPU->VRAM writes into ONE union
bounding box. A frame with two disjoint uploads (sprite column + tile row —
routine MMX6 texture streaming) produced a union spanning the display
framebuffers, which the flush painted from the STALE CPU VRAM mirror (the
FBO is authoritative under GL) — stomping live frames with black for
exactly two presents (one per double-buffer parity). Scene-dependence of
the uploads made it look config/timing dependent; the software renderer is
immune because its CPU array is authoritative.

Fix: exact pending-rect list (16 rects; merges only when zero uncovered
pixels are added — containment or same-band extension; wrap-aware GP0(A0)
transfers split into up to 4 exact rects instead of the old "wrapped: take
all" full-VRAM union; overflow -> order-preserving flush-all; pre-context
union retained where the CPU array is authoritative). Merge rule verified
by a 20k-randomized-rect host test (0 stale / 0 missing painted pixels).

Observability (always-on): gl_present_ring records every SwapWindow (path,
src/letterbox rects, drained glGetError, wall-ms, backbuffer + blit-source
pixel samples) — the instrument that gave the 1:1 black-capture
correlation. debug_server_send_fmt now heap-formats responses >64KB (they
previously truncated into unparseable JSON, breaking large ring dumps).
(debug_server.c also carries the Vulkan screenshot_file sync hunk from the
concurrent VK bring-up; see the next commit.)

Validated: ~18-min MMX6 GL attract soak, ~1600 window captures across 3+
attract cycles — zero isolated black frames, zero GL errors (pre-fix:
19/40 captures black at the intro). Tomba1 build-gl boots clean; its full
attract soak follows. ENHANCEMENTS.md added (framework enhancement ledger,
R1 OpenGL / R2 Vulkan state).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 16:26:48 -07:00
Matthew Stanley db159a1ea6 overlay loader: startup ABI batch-purge + cache-index cap 256->4096
Two defects found by the MMX6/Tomba2 pre-merge regression gate:

1. An ABI bump (v9->v10) invalidated every cached DLL, and the lazy path
   migrated them PER-DISPATCH (LoadLibrary -> reject -> DeleteFile -> retry)
   on the emulation thread that owns the window pump — a large cache
   presented as a "(Not Responding)" black window for minutes. Now a
   one-time preflight sweep at cache-scan enumerates the DIRECTORY, checks
   overlay_abi per DLL, purges mismatches (DLL + .ranges + index entry) in
   one pass, and writes a .abi_<tag>.ok marker so healthy caches skip the
   sweep on later boots. The per-load ABI gate stays as defense in depth.

2. CACHE_IDX_CAP was 256 and overflow was SILENT — Tomba2's 399-DLL cache
   truncated, leaving ~140 real DLLs invisible (their regions interpreted
   forever, no diagnostic). Cap raised to 4096 with a loud never-again
   loader_log on overflow.

Negative-tested: planted a v6 DLL under a valid name — swept on boot along
with 25 latent v9 orphans the lazy path had never touched; marker written;
game boots to title. Sweep enumerates the directory rather than the index
precisely because the index is capacity-bounded (bug 2 hid the plant on the
first test iteration).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 15:01:57 -07:00
Matthew Stanley bbe0d2ae8e dirty-interp: bound PC-table probe length; 4x table — Trolley-demo 0.3fps host burn
Tomba2's Trolley attract demo interprets a working set past 64K distinct
PCs. The per-PC tracking tables (g_dirty_ram_pc_table / exec_pc_table) used
an UNBOUNDED linear probe, so once a table filled, every lookup for an
absent PC scanned all 65536 slots before returning NULL — per executed
instruction. Host thread profile showed ~99% of emu-thread time inside
pc_table_get_or_insert_in; guest crawled at 0.3 fps (1M guest cycles per
3s wall) while telemetry showed the window "LIVE" on residual motion.

Fix: hard-bound the probe to 128 slots (miss => NULL => "stop tracking",
the documented degradation) and grow the tables 65536 -> 262144 so the
Trolley working set actually fits — the same table feeds overlay-capture
seed lists (MAX_CAPTURE_PCS aliases it), so the old size also silently
truncated capture manifests for large regions.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 10:09:36 -07:00
Matthew Stanley 1444394242 overlay ABI v10: forward GTE special-register accessors — unblock new overlay coverage
The emitter routes mfc2/cfc2/mtc2/ctc2 on flag/IR/derived GTE registers
through direct gte_read_data/gte_read_ctrl/gte_write_data/gte_write_ctrl
calls (code_generator.cpp COP2 case), but the overlay DLL preamble had no
shims for them — any GTE-heavy overlay failed to LINK (undefined
gte_read_ctrl) and its region ran interpreted forever. That silently
blocked all new native coverage for the lava/Trolley attract-demo regions
after the caches rotated.

GTE state is shared runtime state, so the shims forward unconditionally
through OverlayCallbacks (a local copy would fork the flag/IR timeline).
ABI bumped 9 -> 10; the loader gate rejects+deletes pre-shim DLLs and
autocompile self-heals; plain gte_data[]/gte_ctrl[] accesses stay direct
CPUState reads in emitted code, matching the emitter.

Smoke-validated: previously-failing GTE-heavy captures now compile clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 09:14:40 -07:00
Matthew Stanley 4c39958226 dispatch: Class-A shell-window fix covers OVERLAY code — Tomba2 Whoopee-logo wedge
Root cause of the fresh-launch logo freeze (present at every commit in the
stack; window-capture-verified 13/13 repros, never a regression): the BIOS
dispatch normalize() maps RAM phys 0x30000-0x5AFFF to the shell ROM alias.
The Class-A fix routed post-game-start shell-window targets to the game/dirty
path first, but its gate also required psx_game_address_in_text() — which
exempts runtime-loaded OVERLAY code above the boot-EXE text end. Tomba2's
text ends at 0x38800; its post-logo main-thread resume PC 0x50B80 fell
through to normalize() and executed func_1FC38B80, a stale shell-image
data-as-code self-loop, starving the main thread forever (one VBlank
delivery per frame at the same EPC, zero main-thread instructions retired;
the game's ISR ran fine, so TCP frame counters kept ticking — the freeze
was invisible to every telemetry-only validation). Even PSX_FORCE_INTERP
missed it: normalize() shadowed the address before the interp path saw it.

Fix: post-game-start, the WHOLE shell-overlap window routes to
dirty_ram_dispatch before normalize() — real hardware executes the RAM
bytes regardless of who loaded them (boot EXE, overlay loader, DMA stream).
Pre-start shell addresses still normalize to the shell as before.

Also:
- interrupts/debug_server: nestgate_* per-guard veto-attribution counters on
  the nested-delivery gate (freeze_check), used to exonerate the IEc/SR
  guards and redirect the hunt from interrupts.c to dispatch.
- tools/window_capture.py: PrintWindow-based SDL-window capture with
  LIVE/FROZEN series verdict — window-liveness ground truth; TCP/VRAM
  screenshots provably lie about presentation (this bug's entire history).

Validated: fresh launch now runs logo -> intro FMV -> title -> attract with
sustained full-frame motion (window-level capture series).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 09:04:17 -07:00
Matthew Stanley 46d7dd4646 Reapply "interrupts: hardware-faithful nested IRQ delivery (SR-gated), not in_exception veto"
This reverts commit 06cab1da46.
2026-07-03 08:44:22 -07:00
Matthew Stanley 06cab1da46 Revert "interrupts: hardware-faithful nested IRQ delivery (SR-gated), not in_exception veto"
This reverts commit d23858b6b1.
2026-07-02 22:55:55 -07:00
Matthew Stanley f47310bede memory/dma: faithful I/O decode — KSEG2 no-op, sub-word memctrl, open-bus unmapped I/O
Tomba2's attract loop-back runs a BIOS bzero + read sweep over a wild
0xDF80xxxx (KSEG2) pointer. Three unfaithful behaviors surfaced, each a
fatal-halt in sequence (f9705/f10002):
  1. Flat 'addr & 0x1FFFFFFF' decode routed KSEG2 onto LIVE registers —
     the sweep zeroed the SIO/memctrl I/O block byte-by-byte. Real
     hardware maps nothing in KSEG2 beyond cache control; Beetle
     (cpu.cpp addr_mask[6..7]=0xFFFFFFFF) leaves it unmapped. All six
     CPU accessors now no-op KSEG2 (counted: g_kseg2_ignored_*).
  2. Memory-control block (0x1F801000..0x103C) and RAM_SIZE (0x1F801060)
     had 32-bit-only handlers; byte/halfword lanes now RMW faithfully.
  3. Unmapped registers INSIDE the I/O window (0x1F8010F8, SIO1 bytes,
     etc.) fatal'd on access; real hardware open-buses them and Beetle
     returns 0/ignores. The MMIO window tails and dma_read/write_masked
     unknown offsets are now counted open-bus (g_io_openbus_*); the
     always-on MMIO rings keep every such access observable.
Validated: attract runs 60fps past both fatal points through multiple
demo cycles. Known next: the attract loop-back logo REPLAY soft-locks
(guest-side; boot-time logo plays fine) — first reachable with this fix.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 22:42:27 -07:00
Matthew Stanley d23858b6b1 interrupts: hardware-faithful nested IRQ delivery (SR-gated), not in_exception veto
Real R3000A has no 'in exception' delivery gate — SR alone governs. The
handler runs with IEc=0 from the entry bit-shift, so the SR gates already
block normal re-entry; a game that RE-ENABLES IEc inside the handler is
asking for a nested interrupt and hardware delivers it. Tomba2's lava
attract demo parks in the kernel event-poll INSIDE the VBlank handler
waiting for the NEXT VBlank: the unconditional in_exception veto starved
it to ~1fps (VBlank pending+unmasked, ~1100 reentry blocks/s, 1 delivery/s
via a fallback). Nested delivery is allowed when the guest's SR has IEc=1,
with two safety refinements (first attempt wedged boot at f436):
  - never nest while an RFE/escape unwind is in flight (the guest's RFE
    pops SR before the host escape completes);
  - one nesting level (exception_nest_depth cap), counted when blocked.
The synchronous-handler machinery is nesting-safe (saved_gpr/hi/lo and
owner state are host-stack locals); in_exception now restores to its
pre-delivery value at the epilogue so a nested exit leaves the outer
level flagged. Validated: lava demo 1fps -> 60fps, full attract at 60.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 22:42:26 -07:00
Matthew Stanley 95ca9d80e5 runtime: fmv_skip_no_xa detection key + host_thread_probe tool
- [video] fmv_skip_no_xa: broaden FMV detection to MDEC activity alone.
  Silent RAM-preloaded movies never stream XA, so the MDEC+XA detector
  missed them and no skip mechanism was even attempted. Per-game opt-in.
- tools/host_thread_probe.py: suspend/sample RIP+RSP of every host
  thread of a live or wedged runtime, stack-scan return addresses into
  the main exe, addr2line-symbolize (RelWithDebInfo). Diagnosed the
  fatal-halt-serve loop in minutes; use whenever a runtime 'freezes'.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 22:42:26 -07:00
Matthew Stanley 2dc938816b debug-server: parse bare json numbers as decimal, not saturated hex
json_get_str fed every consumer the raw value text, and address fields
went through hex_to_u32: a client passing a bare int addr (python
json.dumps of 0x8013DDC4 emits '2148784580') got it parsed AS HEX;
strtoul saturates 10-digit decimals to 0xFFFFFFFF, so read_ram/write_ram
silently operated on junk. This fabricated a false 'stream overwrites
overlay code' lead during the Tomba2 attract hunt (2026-07-02).

json_get_str now normalizes bare pure-decimal values to 0x-hex text
(JSON bare numbers are decimal by definition; quoted values keep the
hex wire convention), and json_get_int scans the raw json itself so
integer fields keep decimal semantics.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 22:42:26 -07:00
Matthew Stanley 3c67a593b3 gpu: fix GP0 polyline terminator — masked equality 0x50005000, not any-high-bits
The mono and shaded polyline states ended a polyline on
(val & 0xF000F000) != 0, which fires on ANY negative vertex coordinate
(Y=0xFFxx) and on bright color components at shaded color positions.
Hardware (Beetle gpu.cpp:1030, psx-spx) terminates ONLY on
(val & 0xF000F000) == 0x50005000 (the 0x55555555 terminator).

Early termination re-parsed the polyline's remaining words as new GP0
commands, de-phasing the whole command stream: garbage prims everywhere
(Tomba2 attract 'texture garble') until a legit texcoord word (0xFE65FE58)
hit IDLE state -> 'GP0 unknown command 0xFE' fatal at the village demo,
deterministically at frame 3946 across 5 repros. Validated: attract now
runs f3946 -> f7004+ through village/water/cave scenes artifact-free.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 22:42:26 -07:00
Matthew Stanley 4cc757f0c9 bios: seed all 30 ROM-resident kernel-table targets; unknown dispatch is FATAL by default
Root cause of MMX6 "A game data could not be found" (weeks-old, cross-backend):
the game's save-scan calls memset through the kernel A0 jump table; entry
A0:2B points into BIOS ROM at 0xBFC02B8C (one of the entries the kernel never
patches into RAM). Transitive discovery can never find it (reachable only via
the data table, no jal), so the dispatch missed and psx_unknown_dispatch
silently no-op'd — killing the scan before firstfile ever ran. Every card
layer (SIO, IRQs, SwCARD events, sector data, kernel directory parse) was
verified correct along the way; the sole defect was the undiscovered ROM
function plus the silent miss.

- recompiler/seeds: enumerate the post-init A0/B0/C0 tables (kernel RAM
  0x200/0x874/0x674) and seed every ROM-pointing entry with no dispatch
  coverage — 30 functions (memset, memcmp, memmove, strncat, rand-family,
  card/dev services...). memset was merely the one MMX6 called; the rest were
  identical latent bugs. BIOS regen: 4406 -> 4439 dispatch entries.
  MMX6 CONTINUE now lists and loads saves (user-validated + scripted run:
  full FILE SELECT, 1093 card txns, unknown_dispatch_log total=0).

- traps.c: an unresolvable dispatch target now dumps a crash report and halts
  BY DEFAULT (PSX_FAIL_FAST_UNKNOWN_DISPATCH=0 opts out for diagnostic
  sessions; the miss is ring-recorded either way). The old silent-no-op
  default left stale registers and a severed call chain, and masked this bug
  for weeks. Deliberately NO interp fallback: the dirty-RAM interpreter is
  for runtime-written code (Rule 18); interpreting around a static-discovery
  hole would hide it exactly like the no-op did.

- runtime.cmake: EXE_NAME option — per-game process names (mmx6-runtime.exe)
  so one game's `taskkill /IM psx-runtime.exe` dev sweep no longer kills
  every other game's instance.

- debug rings (the tooling that found this): HLE call ring gains
  consecutive-duplicate collapse w/ repeat counts (a 4-way TestEvent poll at
  175K calls/s no longer flushes the ring in 0.1s), per-entry TCB + in_exc
  (thread attribution), a "since" paging param and 8MB dump buffer;
  card-trace EvCB watch widened to all 7 card event slots (slot 0 =
  SwCARD-IOE, the success event, was previously invisible).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 22:39:25 -07:00
Matthew Stanley c3c43e68ae launcher: drop the "Skip PSX BIOS" toggle — HLE boot-skip is automatic now
Releases ship [runtime] bios_hle = true, so the boot skip is on by
default and no longer a per-user launcher setting (the old toggle also
described the retired snapshot mechanism's "first boot still loads"
behavior, which no longer exists). Power users can still set bios_hle /
fast_boot in settings.toml or game.toml by hand.

Also aligns CLAUDE.md: the session-start mantra now points at the §0
amendments instead of a blanket "No HLE", and the 2026-07-02 amendment
names the real config key ([runtime] bios_hle).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 16:50:41 -07:00
Matthew Stanley 496d1dbbc8 runtime: HLE tier wiring + carried instrumentation from the card-load hunt
HLE tier wiring (completes the previous commit — HEAD builds again):
- main.cpp: [runtime] bios_hle/keep_intro consume + settings.toml layer +
  PSX_BIOS_HLE / PSX_BIOS_HLE_KEEP_INTRO env resolution + backend banner;
  boot-turbo window ORed into the turbo predicate; the fast_boot snapshot
  restore/capture path and s_fast_boot_active are removed (deprecated by
  the HLE boot shell-skip); psx_hle_scheduler_set_default call.
- traps.c: psx_hle_scheduler_enabled default now flows from config
  ([runtime] hle_scheduler) with the PSX_HLE_SCHEDULER env winning.
- debug_server.c: hle_dump command (always-on HLE ring query).

Carried instrumentation/state from the MMX6 card-load sessions (kept
deliberately; documented in memory + handoff):
- dirty_ram_interp.c: XOP_RES interp JAL call-resolution probe + xprobe
  watched dump section (emitted first), watch targets.
- overlay_loader.c: probed wrapper around dispatch_call + psx_sljit_call.
- debug_server.c: card_mgr_trace range widen, TestEvent spin-body
  collapse, 64MB response cap.
- interrupts.c: PSX_SAME_THREAD_RESTORE selector (0/1/2; TCB-equality
  hardening candidate).
- memory.c: PSX_POLL_PROOF temp gate (remove when card work resumes).
- savestate.c/.h (new): user save states (F1-F12), HLE-scheduler-gated
  load via psx_scheduler_resume_at; load-crash known, do not use for
  repros yet.
- psx_cycles/autocompile/cycle_testrom: timing-fastpath + exe-path
  session state.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 16:27:57 -07:00
Matthew Stanley 582aeccdac bios: HLE as a first-class swappable tier (gbarecomp model) + HLE boot shell-skip
USER PIVOT 2026-07-02 (CLAUDE.md §0 amendment included): psxrecomp now
carries an opt-in High-Level Emulation backend alongside LLE. LLE (the
recompiled BIOS) stays the default, the reference implementation, and the
correctness/timing oracle; every service HLE does not implement falls
through to LLE, so HLE is never load-bearing beyond what it covers.

Mechanism: full_function_emitter.cpp emits a null-by-default hook
g_psx_bios_hle_hook consulted at the top of every psx_dispatch_impl
iteration, on the pre-normalize physical address, BEFORE any backend
(game image / dirty-RAM / static table) claims the target. A handled
call resumes at $ra through the trampoline's existing return/tail
contract. NULL (default) = pure LLE, dispatch-identical. Regenerating
BIOS+game images is required (emitter change).

v1 call-HLE (runtime/src/bios_hle.c): the B0 event family — DeliverEvent
(incl. callback dispatch with the kernel-true $ra 0x1720), OpenEvent,
CloseEvent, TestEvent, EnableEvent, DisableEvent — each ground-truthed
instruction-by-instruction against the SCPH1001 kernel disassembly
(0xBFC11644..0xBFC11A84) and operating on the REAL guest EvCB table
([0x120]/[0x124], stride 0x1C). WaitEvent/threads/pads/card/A0/C0 fall
through to LLE. Cycle charges are documented approximations; LLE remains
the timing oracle. No-stubs stands.

Boot HLE: with the tier on (and no keep-intro opt-out) the hook one-shot
intercepts the first dispatch of the BIOS shell entry (RAM 0x30000 —
LoadRunShell's indirect call always routes through the dispatcher), so
BIOS Main() proceeds straight to SYSTEM.CNF + game EXE load with kernel
state built entirely by the real recompiled kernel init, unpaced (boot
turbo) until game handoff. This REPLACES the fast_boot snapshot
capture/restore; fast_boot=true now aliases the boot-skip alone.

Selection: [runtime] bios_hle / bios_hle_keep_intro (+ settings.toml
bios_hle mirror), PSX_BIOS_HLE / PSX_BIOS_HLE_KEEP_INTRO env overrides,
startup banner bios_backend=/bios_boot=. The PSX_HLE_SCHEDULER spike is
folded in as the tier's standing subsystem replacement: default from
[runtime] hle_scheduler via psx_hle_scheduler_set_default, env wins.

Observability: always-on 16K-entry HLE ring (route LLE-fallthrough /
HLE / boot-skip per record) + hle_dump TCP command (TCP_COMMANDS.md).

VALIDATED on Tomba (build-wt): LLE pass byte-identical behavior (real
intro, card check, save/load OK); HLE pass boots shell-less, TestEvent/
DeliverEvent route HLE, ChangeTh/ReturnFromException/OpenTh/CloseTh and
the card stack fall through to LLE, save/load OK (user-driven).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 16:27:33 -07:00
Matthew Stanley 13c5e0c5b5 stale-recompiler shard guard + exception-exit ring; Tomba menu wedge resolved
Root cause of the Tomba pause-menu wedge (loaded saves only): autocompiled
overlay shards were emitted by a recompiler binary OLDER than the last emitter
change, yet stamped with the CURRENT cg tag (compile_overlays derives the tag
from --runtime-include; nothing tied it to the emitting binary). The stale
shards corrupted the display-list task queue during load-game. NOT the
IRQ-resume class the prior handoff claimed - disproven with the new ring.

- recompiler: bake the emitter-source hash (shared canonical list
  runtime/codegen_hash_sources.cmake + hash_codegen.cmake) into psxrecomp-game;
  new --codegen-hash flag prints it (machine-parseable, pre-banner).
- compile_overlays.py: HARD-FAIL when the recompiler binary's baked hash
  differs from the cg tag hash it stamps (or the binary lacks the flag).
  Applies to both cache and --static modes. Closes the read==write-tag /
  stale-content drift class structurally.
- runtime.cmake: codegen-hash source list moved to the shared canonical file.
- interrupts.c: always-on exception-EXIT half added to irqctx_ring (take_pc,
  real_epc, exit_pc, escape reason, same-thread/restore flags, v1/ra
  saved-vs-exit, RestoreState redirect count) - proves per delivery which
  escape path fired and whether the interrupted GPRs were restored. Kept the
  same_thread_resume GPR-restore refinement (ring-verified equal-value in
  practice; faithful: hardware restores all GPRs on every exception return).
- debug_server.c: irqctx_ring dump extended with the exit-half fields.

Validated live: Tomba menu opens on loaded save (Stormy Mountain) with the
cache fully enabled after regenerating shards with a rebuilt recompiler
(build-cosim + build-prod caches). Guard verified positive and negative.
MMX6 gate passed: boot -> title -> opening cutscene -> gameplay handoff OK.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-01 19:50:26 -07:00
Matthew Stanley 0ecf552d6b Merge codex/mmx6-precise-slice: faithful-timing core + MMX6 timing fix
Brings the faithful cycle/IRQ-timing work onto master:
- due-cycle event scheduler (VBLANK raised at the correct guest cycle);
  MMX6 cutscene->gameplay handoff now runs compiled (verified).
- R3000A load-delay interlock, i-cache modeling, MMIO device-region waits.
- overlay ABI v9 (12 timing runtime fns via OverlayCallbacks) so overlays
  link and run native under the new emitter.
- cosim first-divergence oracle + tooling.

KNOWN REGRESSION tracked for immediate follow-up: Tomba pause menu wedges
(interrupt-resume re-enters function entry, re-zeroing a VSync wait counter).
Being patched on fix/tomba-menu-irq-resume before declaring victory.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 14:32:12 -07:00
Matthew Stanley c78eeb72cc overlay ABI v9: route faithful-timing runtime fns through OverlayCallbacks
The cycle-accuracy emitter emits calls to 12 timing runtime functions
(psx_cyc_load_*, psx_icache_fetch, psx_muldiv_*, psx_gte_*, psx_slice_block,
...) + debug_server_cyc_observe + g_debug_last_store_pc. Overlay DLLs never
got these symbols, so every overlay failed to link -> 0 native shards ->
100% interpreter (unplayable). Fix (option A): expose the 12 via the overlay
callback ABI.

- overlay_api.h: +12 timing callbacks in OverlayCallbacks, ABI 8 -> 9.
- overlay_loader.c: populate the 12; add warn_on_cgtag_mismatch() (loud
  warning when shards exist under the wrong cg-hash).
- compile_overlays.py: 12 preamble forwarders + local g_debug_last_store_pc
  + -DPSX_NO_DEBUG_TOOLS.
- cosim.c/main.cpp: input-cosim setpad + held pad (deterministic input).
- cosim.py: env-overridable COSIM_EXE/CWD/GAME.

KNOWN REGRESSION (patched on a follow-up branch): Tomba pause menu (triangle)
wedges. Root-caused: an IRQ taken mid-block resumes at the function entry
leader instead of the interrupted PC, re-zeroing the menu's VSync wait
counter (scratchpad 0x1F8001E8) every frame so it never reaches its target.
Not the scheduler (LLE reproduces) and not timing math -- interrupt-resume
re-entry.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 14:31:59 -07:00
Matthew Stanley 553d9933d1 fix mmx6 timing divergences with cosim oracle 2026-07-01 09:28:50 -07:00
Matthew Stanley 0f34c29c45 diagnostics: add compiled block lockstep comparator 2026-06-30 16:59:58 -07:00
Matthew Stanley ce71f652e6 checkpoint: preserve pre-mmx6-debug state 2026-06-30 16:55:28 -07:00
Matthew Stanley 18c94a1394 device_trace: general two-process device-event (IRQ) cycle ring
New always-on ring (device_trace.{c,h}, compiles into both binaries like
parity_trace) recording every hardware IRQ-raise edge with its guest cycle,
source (I_STAT bit) and detail. All native device sources funnel their raise
through one choke point psx_irq_raise(bit,detail) in interrupts.c (VBLANK/GPU/
CDROM/DMA/timers/SIO routed; DMA passes DICR channel flags, timers pass index),
so the device-timing audit sees every event from one place. devtrace_dump/ctl
debug commands (cycle-window + source filters) + tools/devtrace_diff.py align
the two device-event timelines by source ordinal and flag where cycles drift.
The Beetle oracle side (mednafen IRQ_Assert rising-edge hook) lives in the main
checkout.

This is the level below parity_trace: parity names the divergent control-flow /
gate input; device_trace names the divergent HARDWARE EVENT behind it.

First result (MMX6 cutscene asset-load over-fast wedge): the load's CD+DMA
activity is compressed into ~232M cycles on native vs ~384M on Beetle (~1.65x),
completing the func_8001CB3C scene setup ~100M cycles / ~176 frames early. The
producer is CD-read / DMA-completion timing (DMA channels 2/3/4 = GPU/CD/SPU
asset streaming) -- confirming the device-completion-latency hypothesis and
ruling in the CDROM/DMA cycle model as the over-fast subsystem. (VBlank verified
correct: native VBlank edges are exactly 564480 cyc apart.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 21:53:27 -07:00
Matthew Stanley 5da57aa478 parity_trace: per-watch-word LAST-WRITER provenance (name the early producer)
Extends the two-process parity ring so each row carries, per watch word, the
exact store PC + guest cycle + frame + (approx) writing TCB of the most recent
write to that word. parity_trace_note_write() maintains the table from BOTH
processes' write paths (native: memory.c psx_write_* with g_debug_last_store_pc;
Beetle: beetle_libretro.cpp wtrace_callback with the store pc). Both parity_dump
serializers emit the new wwpc/wwcy/wwf/wwt arrays (identical wire format), and
parity_diff.py prints a side-by-side WATCH PROVENANCE block at the divergence.

This turns "the gate-input value diverged" into "who/when produced it" -- the
instrument ChatGPT recommended for the MMX6 over-fast wedge.

First result (MMX6 cutscene->gameplay, watch 0x80071088/0x800CD3F8/0x8006D9AC/
0x800CD3FC/0x800CD404/0x800200F4): the cutscene-setup func_8001CB3C (writer pc
0x8001CB7C of countdown 0x800CD3FC) runs ~99.7M guest cycles / ~176 frames
EARLIER on native (cyc 998,597,937 frame 1769) than Beetle (cyc 1,098,296,890
frame 1942) -- same writer PC, fired early. On Beetle the state struct
0x800CD3F8 is never touched by game code (stays BIOS-init 0 -> setter path); on
native it advances. The over-fast wait is localized one level above the state
machine, at whatever schedules func_8001CB3C.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 20:43:40 -07:00
Matthew Stanley 774875dc8d runtime bisect gate: PSX_LOAD_DELAY=0 to disable the R3000A load-delay interlock
Mirrors the existing PSX_ICACHE / PSX_MMIO_WAIT runtime bisect gates. When
PSX_LOAD_DELAY=0, psx_cyc_load_timing() skips the entire load-delay interlock
(base/deps/cancel/lds/readmem charge) so psx_cyc_load_{word,half,byte} act as
plain reads, while leaving PSX_ENABLE_BLOCK_CYCLES on for everything else.

Bisect result (MMX6 cutscene->gameplay wedge): PSX_LOAD_DELAY=0 does NOT fix
the wedge -- boot instead hangs earlier at the intro story cutscene ("As for
the battle against Sigma"), gp0_draw frozen at 12692, and the cutscene
sub-state (0x800CD3F9) still advances to 6. Combined with the clock-agreement
evidence (native and Beetle both ~566K cyc/frame, <0.2% apart), the load-delay
interlock is largely exonerated as the cause of the early cutscene advance: the
wedge is a specific event/wait completing at a divergent cycle, not a global
per-instruction rate effect.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 20:04:54 -07:00
Matthew Stanley 245b0eef8c tooling(bisect): runtime PSX_MMIO_WAIT gate for the MMX6 cutscene-timing bisect
Add PSX_MMIO_WAIT=0 to disable the device-region MMIO read waits (9ae534d) at
runtime, mirroring the existing PSX_ICACHE gate, so the cutscene-ordering bisect
can score each timing feature without a rebuild. Result so far: PSX_ICACHE=0 and
PSX_MMIO_WAIT=0 both still wedge — i-cache and MMIO waits are NOT the culprit.
Remaining suspect by elimination: the R3000A load-delay interlock / base
block-cycle costs (the largest remaining piece of the faithful-timing stack).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 19:36:24 -07:00
Matthew Stanley 2ed1dbbec7 tooling(parity): server-side transitions filter (collapse identical-state runs)
parity_dump gains transitions=1: emit only rows where the watched STATE changes
(watch words + epc + tcb_state; pc/ra/sp ignored), collapsing runs of identical-
state dispatch rows into one with a reps count. A 130k-row cutscene window trims
from ~9.9 MB to ~224 KB (44x) and fits with no truncation — fixes the large-dump
transport limit and makes cross-process divergence fall out of a compact state
timeline. Beetle dump handler mirrored in the main checkout.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 19:27:39 -07:00
Matthew Stanley 7224248c82 tooling(parity): stamp guest-cycle into the parity ring (cycle-drift ruler)
Add absolute guest-CPU-cycle (since boot) to every parity row via a per-process
parity_host_cycle() accessor (native psx_get_cycle_count; Beetle
beetle_core_get_guest_cycles — directly comparable). Widen ParityEntry.cycle to
64-bit; both dump handlers emit it. This turns the cutscene divergence from "273
frames early" into a hard cycle delta: Beetle runs 566k cycles/frame (≈ real PSX
VBlank 564480); native reaches the cutscene state-advance ~150-200M guest-cycles
early — an over-fast cycle deficit (DMA-cycle-steal signature) in the faithful-
timing stack. Beetle producer side mirrored in the main checkout.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 19:19:08 -07:00
Matthew Stanley f8aefefec1 tooling(parity): general two-process control-flow parity trace (native producer)
End the "somewhere upstream" blindness with a reusable first-divergence
instrument (PRINCIPLES.md "find the first divergence; is the value WRONG or the
behavior MISSING"). A TCB-scoped, freeze-on-trigger control-flow ring with an
identical struct + JSON wire format on both processes, so a generic sequence-diff
(tools/parity_diff.py) locates the first logical divergence between recomp and
oracle — not a frame diff.

- parity_trace.{h,c}: armed-from-boot ring, gated on current_tcb==watched_tcb,
  latches on a configured trigger target so the pre-divergence window survives
  the wedge loop. Up to 6 watch-word snapshots/row. Self-contained TU shared by
  both binaries; frame source decoupled via parity_host_frame().
- fntrace.c: per-dispatch producer hook (cheap armed gate on the hot path).
- traps.c: YIELD control-event hook in the scheduler switch.
- main.cpp: env-armed from boot (PSX_PARITY_TRACE=1; TCB/trigger/watch tunable),
  defaults target the MMX6 cutscene wedge.
- debug_server.c: parity_dump + parity_ctl commands.
- tools/parity_diff.py: pulls both rings, sequence-diffs by (kind,pc,target).

First use (MMX6 freeze) confirmed MISSING behavior, not a wrong value: native
thread1 is permanently parked at func_8002000C resume (TCB EPC 0x800200F4),
handshake flag 0x8006D9AC never set (0 across 16384 rows), setter func_8001344C
never runs; thread1 re-resumes a dead func_8002000C frame whose saved-ra slot was
reused -> jr 0x800CD3F8 wedge. Upstream context-staleness/scheduling root.
Beetle mirror producer + the cross-process diff are the next step.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 16:38:38 -07:00
Matthew Stanley 0f7d0f1645 spike(scheduler): deterministic TCB scheduler LIVE + HLE/LLE toggle (minimal A+)
Replace the non-deterministic host-fiber thread-switch bridge with a deterministic
TCB scheduler (HLE_SCHEDULER_CARVEOUT_PLAN.md steps 2-8, minimal "A+" scope):

- psx_scheduler_run (traps.c): outer trampoline wrapping the top-level dispatch
  (main.cpp calls it instead of bare psx_dispatch). setjmp(g_scheduler_jmpbuf);
  restore current TCB / dispatch / on-escape reset g_psx_dispatch_depth +
  g_psx_call_bail; boot/pre-TCB runs raw cpu->pc; one-level switch-back safety
  net; fail-closed psx_is_dispatchable on every restored EPC.
- psx_request_thread_switch: save yielder ctx (resume=$ra), repoint
  dword_108->entry, longjmp to the scheduler. No fibers, no cpu->pc=0 signal.
  Fires only with in_exception==0, so it never unwinds a live exception_jmpbuf
  frame (the in-exception RFE path stays LLE/intact).
- sched_escape_ring (step 8) for always-on observability.
- Hidden toggle PSX_HLE_SCHEDULER (default 1=HLE, 0=legacy fibers), read once;
  psx_change_thread branches on it; startup banner prints the mode. The
  trampoline is transparent in LLE mode. Fiber funcs kept compiled-but-dead.

Validated live (HLE): thread1 trace shows ONLY the new TCB-switch events and
ZERO fiber kinds 10/11/13 — the per-frame fiber recreate is eliminated. Builds
clean. Freeze NOT resolved: same 0x800CD3F8 wedge persists (separate upstream
root, per ChatGPT Q5). Keep HLE default until the upstream cause is fixed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 15:52:51 -07:00
Matthew Stanley ab8ed36b47 spike(scheduler): step 1-2 scaffolding — structured-escape types + jmpbuf + fail-closed guard (INERT)
First step of the deterministic TCB cooperative-thread scheduler carve-out
(HLE_SCHEDULER_CARVEOUT_PLAN.md §9, ChatGPT-approved "A+"). Replaces the
non-deterministic host-fiber bridge (confirmed root of the MMX6 cutscene freeze:
thread1's fiber recreate-per-frame, pc=0 silent exit).

- runtime/include/psx_scheduler.h: psx_run_reason_t (CONTINUE / YIELD_TO_TCB /
  RESUME_CURRENT / GUEST_EXIT / FATAL) + psx_sched_escape_t, the design rationale
  (pure-TCB BIOS scheduler, no stack switch; guest state lives only in
  CPUState/RAM/TCB), and the load-bearing invariant.
- traps.c: g_scheduler_jmpbuf + g_sched_escape definitions + psx_is_dispatchable
  (fail-closed on pc==0 and the exception sentinel — the known-bad resume PCs the
  old pc=0 pathology produced).

INERT: nothing calls these yet; psx_change_thread_fiber (host-fiber bridge) is
still authoritative. Builds clean (traps.c only, no game-C regen). No behavior
change. Next: outer scheduler loop + route ChangeThread/RFE through structured
escapes, then remove cpu->pc=0 + fiber recreation.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 14:42:43 -07:00
Matthew Stanley 4ddccccaa9 checkpoint: exact-store-PC tooling fix (verified) + freeze/scheduler diagnostics + faithful-timing WIP
recompiler/code_generator.cpp: stamp g_debug_last_store_pc before EVERY game
store (SB/SH/SWL/SW/SWR, SWC2, + the ws-backdrop / game-option special paths)
and declare the extern in the game-C preamble. Previously only the BIOS emitter
(strict_translator.cpp) stamped it, so game stores inherited the last BIOS store
pc -> wtrace/readtrace mis-attributed game writes to the BIOS. VERIFIED LIVE:
MMX6's producer trace for the 0x800CD3F8 write now reports the real game store
pc 0x8001A708 (matches Beetle), not the stale 0xBFC11BF0.

runtime: g_pc0_reason slice-exit provenance, fail-closed exception-sentinel
asserts, named thread-event kinds, freeze-heartbeat RVA emission (diagnostics,
default-on, additive).

Carries the in-progress faithful-timing / MDEC work (cpu_state.h, memory.c,
dma.c, interrupts.c, dirty_ram_interp.c, full_function_emitter.cpp) as an
uncommitted-WIP checkpoint. tools/: regen_bios.sh, bios_emitter_fingerprint.sh.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 14:20:17 -07:00
Matthew Stanley 08bb5a2728 fix(mdec): truncate 15bpp channel to uint8 before RGB555 (the rainbow bug)
The faithful dequant/IDCT/YUV transcription was correct, but the 15bpp pack did
(c+4)>>3 on `r ^ 0x80` as a SIGNED int. For negative Mask9ClampS8 results that
yields a negative int (r=-1 → -129) → garbage 5-bit channel → B saturated, R/G
doubled = the rainbow fringing on real FMV. Beetle's RGB_to_RGB555 takes uint8
params (truncates to 0..255); the 24bpp path already cast to uint8_t. Fix:
rgb_to_555_chan(uint8_t) so the caller's int truncates on the call.

Caught only by comparing our VRAM to the REAL Beetle process (new vram_peek on
the Beetle side, held-FMV diff) — both standalone pins shared the bug in their
self-transcribed reference and falsely passed. After the fix the per-channel VRAM
delta vs Beetle collapsed from mean (12,16,27) to (0.7,0.9,1.3) and the FMV renders
natural colours. Full-path pin (mdec_e2e, corrected uint8 oracle) is byte-exact.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-27 21:06:04 -07:00
Matthew Stanley 7c307982d3 WIP(mdec): faithful Beetle dequant/IDCT/YUV transcription — UNVALIDATED
Transcribes the MDEC numeric core from Beetle mdec.cpp per accuracy/axis5_mdec.md
(D1/D2 dequant <<4 domain + sign bias + ±0x4000 clamp; D4 two-pass IDCT
(sum+0x4000)>>15 with Mask9ClampS8 + transposed >>3 scale matrix + Beetle ZigZag;
D5 YUV→RGB /256 coeffs + green-truncation mask + RGB555 rounding + signed xor).

!! DO NOT MERGE — UNVALIDATED, has a known visual bug. !!
A live FMV-frame check (the wrong tool, per Rule 15) showed RAINBOW colour
fringing on the Whoopee Camp logo frame where Beetle renders clean pink, so the
transcription has a defect I could not localize by inspection (IDCT transpose,
dequant domain, and YUV all read correct vs the source — which is exactly why
eyeballing is insufficient here).

PROPER COMPLETION = the byte-exact block-level pin the doc mandates (§4.A):
feed an identical synthetic SET_QUANT/SET_SCALE + one macroblock of known RLE
to BOTH cores via a debug command, dump post-dequant Coeff[64] / post-IDCT
block_* / packed output, diff element-wise to isolate D1-D2 (Coeff) vs D4
(block) vs D5 (pixels). Build that tooling, fix the divergent stage, THEN merge.
This branch preserves the transcription so that pin work starts from it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-27 20:22:44 -07:00
Matthew Stanley 82e0523749 fix(sio): faithful DualShock 0x43/0x45 + analog-mode lock (axis 5)
Transcribe the modern-SM pad config semantics from Beetle dualshock.cpp, fixing
the hybrid phantom-input root causes (accuracy/axis5_sio_controller.md):

- 0x43 (enter config) now transmits the LIVE poll frame — id, 0x5A, buttons,
  + 4 stick axes in analog — identical framing to 0x42 (dualshock.cpp:471-490),
  instead of an all-zero frame. The zero frame fed any driver that reads the
  0x43 response as input a phantom "all-pressed"/centered-stick (active-low
  0x00 = all buttons down). In-config 0x43 (exit) keeps the zero frame
  (dualshock.cpp:660-674). [D4]
- 0x45 status byte now reports the LIVE analog state (pad_analog?1:0) rather
  than a hard-coded analog-on, so a driver tracking the mode parses the right
  poll-frame length (dualshock.cpp:743). [D8]
- 0x44 0x..03/0x02 now sets/clears a per-slot analog_mode_locked; a locked slot
  ignores the host hybrid auto-flip (the emulated analog button), matching a
  real pad that ignores the physical button when locked (dualshock.cpp:203,
  714-725). [D6]

Gated to the modern SM; Tomba 1's legacy_pad_config path is untouched. MMX6
(modern SM) gains the phantom-dash fix. Validated no-regression on Tomba 2
(digital pad, 0xFFFF at rest, boots). Retiring g_pad_legacy_cfg (the doc's
fix 4) is gated on a DualShock-game behavioral validation pass — see the
STATUS note in the doc.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-27 19:40:10 -07:00
Matthew Stanley 9ae534dc42 timing: faithful device-region MMIO read waits (SPU +36, CDC +6*size, +1, ...)
Replace the placeholder region cost (RAM +3 / everything-else +0) in
psx_cyc_readmem with the full Beetle MemRW device-region read-wait table
(libretro.cpp), size-aware:

  main RAM (<0x800000)            +3
  SPU   0x1F801C00-1FFF          +36 (32-bit) / +16 (8/16-bit)
  CDC   0x1F801800-180F          +6 * size
  GPU/MDEC/SysCtrl/FIO/SIO/IRQ/DMA/Timers (0x1F801000-113F)  +1
  BIOS ROM / Expansion-PIO / unmatched / scratchpad           +0

Thread the access size (1/2/4) from psx_cyc_load_word/half/byte and
psx_cyc_lwc2_read into psx_cyc_readmem (SPU/CDC waits are width-dependent).
The device wait combines with the existing +2 completion (+1 LWC2) and fudge
exactly as Beetle ReadMemory (LDAbsorb = region + completion). Runtime-only:
psx_cyc_load_* signatures unchanged, no emitter regen.

New ruler #2 loops mmio_timer (Timer0 read -> +3) and mmio_spu (32-bit SPU
read -> +38) added to gen_testrom.py. Validated: cyctest compiled (4600) ==
Beetle (4382) EXACT on all 15 loops incl. mmio_timer +3 / mmio_spu +38; the 13
prior loops unchanged; Tomba 2 boots past the BIOS to its SCEA-Presents intro
splash, no freeze.

Documented residual: DMACycleSteal (dynamic per-read DMA bus-steal) remains
unmodeled — needs the live steal count from the DMA controller; not statically
rulerable.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-27 18:17:07 -07:00
Matthew Stanley 0edb9355bb timing: I-cache fetch Stage 2 — compiled-path emit (both emitters) + default-on
Charge the faithful R3000A I-cache fetch cost in BOTH static emitters at each
cache-line leader, completing the model (interp-only since Stage 1). A leader is
a block leader / mid-block jump-table target (a possibly-cold line entry) OR a
16-byte-line start (addr&0xC==0); intra-line fall-through followers are provable
hits (+0) and emit nothing. Emitted BEFORE the per-instruction interlock/load so
a fetch miss clears the pending load give-back first (Beetle ReadInstruction
order).

- code_generator.cpp (game): emit_pre_icache at the 3 timing sites (normal,
  branch, delay slot); `addr` is already the KSEG0 runtime PC.
- full_function_emitter.cpp (BIOS): emit_icache_fetch at all 5 emit_insn_interlock
  sites (main loop + 4 orphaned delay slots), mapping the ROM/compile addr to the
  RUNTIME guest PC via relocate_ra (BIOS main stays KSEG1 0xBFC.. uncached;
  kernel Part 2 -> 0x500+, shell -> 0x80030000+). Same address value the interp
  passes for the same instruction, so the shared TV array stays consistent.
- psx_icache.c: flip psx_icache_enabled() DEFAULT ON (PSX_ICACHE=0 to disable).

Validated: ruler #2 compiled (4600) == Beetle (4382) EXACT on all 13 loops incl.
icache_miss +14 (was +0); ruler #1 steady delta 0 (56==56) with native cold-refill
spikes (77/84) now matching Beetle's range; Tomba 2 boots past the load wedge to
the intro FMV + PS BIOS logo, no freeze. Default-on path re-validated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-27 17:59:13 -07:00
Matthew Stanley 958a928d14 timing: faithful R3000A I-cache fetch model + interp validation (Stage 1)
New runtime/src/psx_icache.c: direct-mapped (4 KB / 256-line) instruction-
cache fetch cost, transcribed from Beetle PS_CPU::ReadInstruction:
  HIT  (TV == addr)            -> +0, no give-back clear
  MISS, KSEG1/uncached         -> +4 (BIOS ROM)
  MISS, cached (KSEG0/KUSEG)   -> +3, then +1 per word refilled from the
                                  missing word to the line end (earlier words
                                  stay invalid); also clears the load give-back.
Mirrors only the per-word TV tag array (cost + tag state; the value is already
in hand). Wired into the dirty-RAM interpreter (exec_one) per instruction,
charged BEFORE the §1 base — Beetle order.

New ruler #2 loop `icache_miss`: loop top + a victim 0x1000 apart alias the
same direct-mapped line with different tags, forcing a refill miss every
iteration (other loops are all-hits after warm-up).

VALIDATED interp vs Beetle (PSX_FORCE_INTERP=1 PSX_ICACHE=1): icache_miss
native +14 == Beetle +14 (per-iter 17); the other 12 loops unchanged at +0
fetch. So the I-cache hit AND refill-miss costs are MEASURED equal to the
oracle on the interp path.

Opt-in via PSX_ICACHE=1 (default OFF) — charging fetch cost in only one
backend would fork mixed compiled/interp timing. Production activation +
the compiled-path emit (per cache-line leader) + ruler #1 cold-spike
validation are Stage 2. Default-off => no production change (the interp's
fetch call is a no-op without the env).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-27 16:48:56 -07:00