End the "somewhere upstream" blindness with a reusable first-divergence
instrument (PRINCIPLES.md "find the first divergence; is the value WRONG or the
behavior MISSING"). A TCB-scoped, freeze-on-trigger control-flow ring with an
identical struct + JSON wire format on both processes, so a generic sequence-diff
(tools/parity_diff.py) locates the first logical divergence between recomp and
oracle — not a frame diff.
- parity_trace.{h,c}: armed-from-boot ring, gated on current_tcb==watched_tcb,
latches on a configured trigger target so the pre-divergence window survives
the wedge loop. Up to 6 watch-word snapshots/row. Self-contained TU shared by
both binaries; frame source decoupled via parity_host_frame().
- fntrace.c: per-dispatch producer hook (cheap armed gate on the hot path).
- traps.c: YIELD control-event hook in the scheduler switch.
- main.cpp: env-armed from boot (PSX_PARITY_TRACE=1; TCB/trigger/watch tunable),
defaults target the MMX6 cutscene wedge.
- debug_server.c: parity_dump + parity_ctl commands.
- tools/parity_diff.py: pulls both rings, sequence-diffs by (kind,pc,target).
First use (MMX6 freeze) confirmed MISSING behavior, not a wrong value: native
thread1 is permanently parked at func_8002000C resume (TCB EPC 0x800200F4),
handshake flag 0x8006D9AC never set (0 across 16384 rows), setter func_8001344C
never runs; thread1 re-resumes a dead func_8002000C frame whose saved-ra slot was
reused -> jr 0x800CD3F8 wedge. Upstream context-staleness/scheduling root.
Beetle mirror producer + the cross-process diff are the next step.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
recompiler/code_generator.cpp: stamp g_debug_last_store_pc before EVERY game
store (SB/SH/SWL/SW/SWR, SWC2, + the ws-backdrop / game-option special paths)
and declare the extern in the game-C preamble. Previously only the BIOS emitter
(strict_translator.cpp) stamped it, so game stores inherited the last BIOS store
pc -> wtrace/readtrace mis-attributed game writes to the BIOS. VERIFIED LIVE:
MMX6's producer trace for the 0x800CD3F8 write now reports the real game store
pc 0x8001A708 (matches Beetle), not the stale 0xBFC11BF0.
runtime: g_pc0_reason slice-exit provenance, fail-closed exception-sentinel
asserts, named thread-event kinds, freeze-heartbeat RVA emission (diagnostics,
default-on, additive).
Carries the in-progress faithful-timing / MDEC work (cpu_state.h, memory.c,
dma.c, interrupts.c, dirty_ram_interp.c, full_function_emitter.cpp) as an
uncommitted-WIP checkpoint. tools/: regen_bios.sh, bios_emitter_fingerprint.sh.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Build the game-independent cycle-attribution ruler the plan called for, and
fix two real bugs it exposed.
Tooling (debug_server.c, full_function_emitter.cpp):
- cyc_watch double-fire dedupe: observe fired from BOTH the dispatcher and the
function prologue at the same cycle, double-recording every dispatched entry.
Key on (phys,cycle); a genuine re-entry is always >=1 cycle later.
- Per-block-leader cycle observe (debug_server_cyc_observe), emitted by the BIOS
emitter under #ifndef PSX_NO_DEBUG_TOOLS (prod = zero overhead). Native now
samples at EVERY compiled block leader (matching Beetle's before-instruction
sample), so cyc_watch can anchor any block-leader PC, not just fn entries.
Emitted at normalize_address() so relocated-kernel anchors match the RAM space.
Bug fix (psx_instr_cost.h):
- Loads were double-counted. Stage-2 #1a (2ef47bd) returned 3 for loads (+2
data-access) in psx_instr_base_cycles WHILE memory.c charge_main_ram_read
already charged +6 per main-RAM read. The ruler region [0x80001C5C->0x80001CA4]
decomposed native=34 as 16(base) + 2 + 12(mem) with loads counted twice. The
opaque 0x80017FC4 window hid it because the load over-charge masked the entirely
unmodeled divu->mflo stall. Reverted the opcode fn to pure execute base (loads=1)
per its own stated contract; memory.c is the single address-keyed data-access
owner (like Beetle ReadMemory). Ruler: native 34->30 (exact), FMV unaffected.
Docs: CYCLE_MODEL_BEETLE.md (full R3000A model transcribed from in-tree Beetle
cpu.cpp, cited), plan + burndown updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
cyc_watch + cycle_compare.py now take an optional END anchor: each recorded value
is Δcycles of one START->END pass (cost of the known code path between two dispatch
points). [A,A] = Δ between consecutive A hits (call-to-call cost). Mirrored EXACTLY
on both backends (native debug_server.c + Beetle libretro.cpp/beetle_debug_server.c,
the latter uncommitted in main checkout, additive). VALIDATED: region [0x80017FC4,
0x80017FC4] = native Δ 56 vs Beetle Δ 91, gap -35, ZERO variance across 8 passes —
a stable, offset-independent per-region gate to drive Stage-2 components to
convergence (faithful components shrink the gap monotonically; can't overshoot).
Note: [A,B] with distinct B needs B to reliably follow A (a dispatch point on the
deterministic path); B=0x8001EFFC failed (func has multiple exits) — pick wrapper/
single-exit pairs or use [A,A].
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Native cyc_watch (ported from agent branch) + the missing universal hook:
debug_server_log_call_entry now samples cyc_watch_observe at every compiled
function entry (game AND BIOS; the agent's debug_server_trace_dispatch hook only
covered the BIOS trampoline, so game-text anchors fired 0). tools/cycle_compare.py
arms the same anchor on native(4500)+Beetle(4382) and diffs per-hit cycles.
Beetle side (libretro.cpp cyc_watch state/observe + cpu.cpp RunReal loop hook +
beetle_debug_server.c commands) is in the MAIN checkout (additive diagnostic,
uncommitted) and built into psx-beetle.
VALIDATED end-to-end: both backends capture 16 hits at game-text anchor 0x80017FC4.
FINDING: native is ~121M cycles BEHIND Beetle at that anchor = a gross boot-cycle-
path divergence (turbo-loads/fast-CD + overlay/dirty charge far fewer cycles than
Beetle's real-time CD boot), NOT the -8 fine drift. => absolute-cycle comparison
through boot is dominated by load-model differences; the -8 needs a controlled
anchor (early/pre-divergence or an interp-convergence point). Relocated BIOS-shell
funcs (phys 0x30000-0x5AFFF, e.g. 0x80050B0C) dispatch at a different native phys —
anchor on game-text or BIOS-ROM funcs, or the relocated phys.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The multi-week Whoopee-Camp logo->FMV stall was a TIMING-FAITHFULNESS bug, not
game logic: the static recompiler charged cycles block-granular and a branch
whose delay slot is a SEPARATE block leader (branch at end_addr, delay slot
outside the block) undercharged by one cycle on every execution -- the
always-executed delay-slot clone was emitted but counted by neither the branch
block nor the (unentered) delay-slot block. ~8 such sites on the boot init path
= the measured -8 cycle drift that made native run behind the interpreter, so
the frame-1824 elapsed-Timer1 logo-delay loop exited after ~42 iterations
(native) instead of ~1557 (interp/HW) and never reached the FMV.
Fix (code_generator.cpp translate_basic_block, the faithful class fix -- every
dynamically executed instruction charged exactly once):
block_exec_cycles = instruction_count
+ (exit branch at end_addr with delay slot outside ? 1 : 0)
applied to both the block cycle charge and the slice budget. R3000 is MIPS-I so
the delay slot is unconditional; the not-taken path already falls through to
branch_pc+8 (no double-count). Result, screenshot-verified: native passes the
logo-delay loop, screen animates, CDROM+DMA stream active, the jungle intro FMV
plays.
Also in this checkpoint (all on wt/tomba2, not merged/pinned):
- PARKED precise interrupt-slicing (PRECISE_IRQ_SLICE.md): root-caused its pc=0
boot exit to a mid-function clean-text resume PC the dispatcher can't re-enter;
ChatGPT-validated fix is "every block leader = re-enterable CPS continuation"
(psx_game_is_function_entry predicate added; precise_pc_dispatchable +
irq_taken-once restructure in psx_run_precise). Default OFF (PSX_PRECISE_SLICE=1
to resume); inert on the baseline. Slice-trace diagnostics added to freeze_check.
- FAITHFUL_TIMING_PLAN.md: authoritative game plan (P1-P6) + standing directive
(build the faithful core, never hacks; ecosystem is being built, not preserved);
referenced from CLAUDE.md Rule -1.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Phase 1 of the cycle-budgeted precise interrupt-take-point fix (root cause:
compiled blocks take IRQs only at block edges; HW + the dirty-RAM interpreter
take them at the exact instruction — oracle-confirmed at the Tomba 2 logo
Timer1 debounce). Design validated with ChatGPT; full spec in PRECISE_IRQ_SLICE.md.
This commit lands the additive, no-behavior-change foundation (Task #2) plus the
prior session's diagnostic take-PC capture, and a build-hygiene dedup:
- cycles_to_next_event() (interrupts.c): single source of truth — min guest-cycle
distance to the next DELIVERABLE interrupt (source raises its I_STAT bit AND
that bit is unmasked in i_mask). Aggregates VBLANK + per-peripheral queries.
- timers/cdrom/dma/sio _cycles_to_irq(i_mask): each peripheral owns its own
next-IRQ-distance query; all conservative under-estimates (smaller => slice
more => never run past an event).
- debug_server: cycles_to_next_event command for live validation.
- interrupts.c: take-PC capture into the IRQ_DELIVER event-ring aux (prior
session's diagnostic; records the architectural take-PC for cross-backend diff).
- overlay_loader.c: remove duplicate psx_native_bad_entry definition (HEAD had it
twice; would not link cleanly).
Live-validated at the logo: returns the VBLANK distance, correctly excludes the
three free-running (unarmed/masked) timers and idle CD/DMA/SIO. Runtime-only,
no regen. The two-tier executor (Task #3) that consumes this is next.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
In-progress investigation work snapshotted before rebasing onto the
tcc-refactored psxrecomp master. Includes overlay ABI v8
(check_interrupts_at resume-PC callback for async RFE recovery) and
code_generator/full_function_emitter delay-slot cycle accounting.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root-caused the Tomba2 native logo-stuck bug: it is a guest-cycle-timeline
divergence between the compiled-overlay backend and the dirty-RAM interpreter,
surfacing through a Timer1 (0x1F801110) debounce loop at game pc 0x8008592C that
reads 1 (native) vs 2 (interp), forking logo->FMV. The handoff's "jalr target
divergence" was a benign store-PC artifact (read-watch proves a0=0xB870 and
t3=[a0]=0x800896E0 are identical in both backends); the scratchpad blind spot
was ruled out (separate sp fingerprint forks at the same frame).
First-divergence observability (debug_server.c, memory.c):
- frame_fingerprint: added separate sp (scratchpad) cumulative hash column +
per-entry cyc, so a scratchpad or cycle fork is classified, not hidden.
- Unified ordered access recorder: one buffer, kind = ramw/spw/mmiow/mmior/ramr,
array index = true execution order, per-entry cyc (psx_get_cycle_count).
record_frame_dump emits the unified stream; record_reads_dump filters reads.
- Targeted main-RAM read-watch (PSX_READ_WATCH="lo,hi" + g_ram_read_watch_active
guard in psx_read_word) — proves a load's value/address per backend.
- PSX_RECORD_FRAME env seed: arm the recorder from boot (race-free), never
arm-then-time.
Fix A (cycle-model unification, part 1): compiled-overlay code was built WITHOUT
PSX_ENABLE_BLOCK_CYCLES, so every psx_advance_cycles() in overlay code was
compiled out -> a function run as a native overlay charged ~0 cycles while the
same function via the interp charged per-instruction (~32x), the entire drift.
- overlay_api.h: OverlayCallbacks gains advance_cycles; ABI v6->v7 (rejects
stale zero-cycle DLLs).
- compile_overlays.py: build overlays with -DPSX_ENABLE_BLOCK_CYCLES=1 and
emit a psx_advance_cycles shim forwarding to the shared host timeline.
- overlay_loader.c: wire s_callbacks.advance_cycles = psx_advance_cycles.
Result: the first debounce now reads Timer1=2 in native (was 1); overlay cycle
charge went 9.5k -> 380k (interp 306k). Native still logo-stuck: a residual,
perfectly linear ~1 cycle/write over-charge remains (block.instruction_count
counts branch + delay-slot as 2 while the interp charges the pair as 1 step),
which flips a later debounce 3 vs 2. Fix A pt2 (cycle-exact block totals) is the
next step. wt/tomba2 only; no pin bump, no master merge, no other-game regen.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds overlay_cps_probe (debug cmd) to record what the CPS interior-continuation
dispatch decides for a watched PC. Used it to CONFIRM the root cause:
func_80050B08 calls jal 0x89788 (init-array). 0x89788 isn't a registered entry,
so overlay_find_by_range(0x89788) runs -- 6 loaded candidates have a range
containing it -- and returns func_800896E0 (the crt0 bootstrap), whose range
wrongly spans 0x89788. 0x89788 is not a case in its entry-switch, so default
re-runs the bootstrap from the top: InitHeap/A0 spin, VBLANK-masked deadlock.
func_800896E0's range covers 0x89788 because function_discovery.cpp:493-499
treats BREAK as fall-through; the bootstrap's "jal 0x50B08; break" (break guards
the main-returned case) falls through at 0x89784 into the separate function at
0x89788, absorbing it. So the early break-no-op finding WAS the root -- damage is
at discovery/range time, not runtime break execution. Interp is immune (executes
0x89788 bytes directly).
Fix options + recommendation (clip function range at next entry / don't cross a
function boundary on break fall-through) documented in
docs/TOMBA2_NATIVE_FMV_FREEZE.md. Diagnostic only; no fix applied yet (needs
regression on Tomba1/MMX6/Ape). Workaround: overlay_native_block 0x80050B08.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds a phys-keyed blocklist in overlay_loader.c that forces a named overlay
function through the sanctioned dirty-RAM interpreter (it still RUNS — not
skip/stub/HLE), as if s_native_exec were off but for that one function only.
Exposed as the `overlay_native_block` debug command (add/clear/report). This is
the localization tool for native<->interp divergences (e.g. the Tomba 2 FMV
freeze): binary-search which compiled overlay function's native execution flips
the bug, without rebuilding or losing candidate visibility. Runtime/tool-only.
Used it to localize the Tomba 2 end-of-Whoopee-Camp FMV freeze to
func_80050B08's OWN native body (blocking it passes the freeze with callees
still native; all 33 jal targets + continuations verified clean). Narrowed to
the main-loop region with a strong interior-entry suspect at 0x80050CE8.
Findings in docs/TOMBA2_NATIVE_FMV_FREEZE.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The dispatch path re-CRC32'd a native overlay function's ENTIRE code body on
every single call (cand_crc, unconditional) before running it -- a real tax on
hot per-frame code that blunted the native win. The page-generation counter that
could make this a cheap skip existed but was demoted to a diagnostic.
Promote it: a watched-page write is the ONLY way an entry's code can change, and
every CPU/DMA store funnels through the store chokepoint which bumps
overlay_page_gen for watched pages. So if the page-generation sum is unchanged
since this entry was last confirmed VALID, the bytes are provably unchanged ->
run native WITHOUT re-hashing. Fall back to the full crc32 only when the gen
moved or the entry isn't known-valid (this also re-confirms reload-on-return,
since write-away-then-back bumps the gen). Applied to both the main candidate
chain and the CPS continuation re-entry path. Correctness identical to the old
unconditional hash; only redundant hashing is cut. New gen_fastpath counter
exposed in the overlay status.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Tomba 2 discards its boot/init code and loads a gameplay overlay OVER its
own boot-EXE text region (phys 0x1DB8C, below the 0x38800 text-end FLOOR).
The dirty-RAM interp's local-flow gates (is_local_dirty_target,
allow_local_dirty_flow, transfer-chain gate) keyed on OVERLAY_REGION_FLOOR,
so they misclassified that live overlay as immutable static text and routed
the splash loader's `jal 0x8001DB8C` through the bail-prone non-local-call
contract instead of interpreting the overlay locally. That produced a wild
`jr ra` reading a stale saved-ra stack slot (0x801FE994 = "T.BI", bytes 4-7
of START.BIN left by the file-open strcpy), killing the worker thread before
it could ChangeThread to the START.BIN loader -> Whoopee-Camp splash softlock.
Verified against ground truth: EXE + generated C have `move a2,a0` at
0x8001DB8C (mid func_8001DB7C); runtime RAM has a fresh prologue there
(page 0x1D dirty=true). Always-on wtrace_all ring pinned the clobbering
strcpy at pc=0x8008B9A8, frame 2068.
Fix: gate local-dirty-flow on DIRTY_RAM_KERNEL_WINDOW_END (0x10000) instead
of OVERLAY_REGION_FLOOR, via new helper phys_is_overlay_flow_region(). A dirty
page above the kernel window IS an overlay regardless of FLOOR; the inline
dirty_ram_is_dirty() check keeps clean boot text on the compiled fast path,
and the kernel window [0,0x10000) still stays per-block (exception context).
Runtime-only; no regen.
Result: worker no longer wild-jumps; it ChangeThreads to the loader thread
(TCB 0xA000E374); aborts=0; threads switch cooperatively. NEW failure point:
the loader thread does not read START.BIN data sectors (CD idle at directory
sector 373) -> state byte 0x1F80019C stays 0, splash not yet advancing
(next cascade stage).
Also includes the diagnostic tooling that localized this: insn_freeze_target,
insn_freeze_snapshot, ra_load_watch (debug_server.c / dirty_ram_interp.c) and
thread sys3 / dispatch-return traces (traps.c).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implemented + instrumented the consult's Option-A async-resume idea; the data REFUTED
the in_exception=0 async model:
- freeze_check now reports async_rfe_set/fire, reach_dirty/traps, reach_async,
dirty_safe_resume_pc, async_rfe_resume_pc.
- Measured: reach_dirty=19921 (~1/exception), dirty_in_exc0=0, async_rfe_fire=0 ->
the sentinel 0x80000048 ALWAYS arrives at the dirty gate with in_exception=1 and
takes the longjmp. There is no in_exception=0 dispatch of the sentinel.
Corrected mechanism (docs/TOMBA2_WHOOPEE_BAIL.md): the game's ReturnFromException
longjmp lands in psx_check_interrupts (jmp_val==1), restores GPRs and returns with
cpu->pc=0; for ~19920 exceptions it lands at a NESTED dispatch (game continues), but
for the failing one it unwinds to the OUTERMOST trampoline where cpu->pc=0 reads as a
clean exit. It's a longjmp-landing-depth bug in the 'resume via host return' model,
not an async-resume bug.
Latches added (g_dirty_safe_resume_pc around the 3 dirty pumps, g_async_rfe_resume_pc
at dirty-safe exception entry) + the sentinel-gate redirect scaffolding are INERT
under the corrected model (never fire) — kept as tooling + scaffolding for the
candidate fix. Builds clean; behaves identically to pre-fix (exit at frame 1997, no
regression). wt/tomba2 only; not on master.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Compiler-neutral tier-selection policy extracted from the mis-named
overlay_sljit.{c,h} into NEW overlay_backend.{c,h} (OverlayBackend enum +
resolve/name/active). overlay_sljit.{c,h} is now just the deprecated emitter.
New tcc tier. gcc and tcc run the IDENTICAL pipeline (recompiler -> C ->
compiler -> DLL -> loader); only the compiler binary differs, so both map to the
autocompile-spawn provider (code_provider.c) and the COMMAND is chosen by the
resolved backend (main.cpp). compile_overlays.py gains --compiler gcc|tcc: tcc
writes a separate tcc/ cache namespace, BOM-strips the C + a copy of the include
headers (tcc 0.9.27 can't skip a UTF-8 BOM), and uses tcc's built-in linker.
Backends (overlay_backend): auto (gcc if a gcc TOOLCHAIN is present, else tcc --
was sljit), gcc, tcc, auto-no-gcc (dev/test: force tcc even with gcc present, to
exercise the toolchain-less user path), sljit (DEPRECATED, gated off, lowest).
Priority: static > gcc shard > tcc shard > sljit(off) > interp. The loader scans
gcc/ then tcc/ and dedups by region+crc (filename) so a gcc shard always wins a
tie.
Self-contained tcc: with no explicit overlay_autocompile_cmd_tcc, the runtime
CONSTRUCTS the tcc command from a toolchain bundled beside the exe
(<exe>/overlay_toolchain/: embedded python + tcc + recompiler + compile_overlays.py
+ runtime headers) -- no system python or gcc needed. config_loader gains
overlay_autocompile_cmd_tcc (a dev override). autocompile cmd buffers enlarged for
the constructed path-heavy command.
Compilation stays OFF the dispatch/render thread (off-process autocompile
subprocess, same model as gcc); the main thread only spawns it and dlopens the
finished DLL.
Validated: MMX6 with overlay_backend=auto-no-gcc fills the tcc/ namespace live
from the bundled toolchain and plays through save-load + gameplay -- the exact
path the sljit tier softlocked on. The gcc path is untouched.
Runtime-only, no regen.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Off-thread sljit compile worker (overlay_compile_worker.c/.h): on a dispatch
gap-fill miss the dispatch thread snapshots the region bytes and ENQUEUES; a
background worker JITs from the snapshot (never live RAM) and publishes the shard
to the on-disk cache via an atomic temp+rename; dispatch picks it up on a later
miss via an idempotent rescan, so the candidate table stays single-threaded. This
removes the synchronous JIT frame hitch. Validated on Tomba + MMX6: compiles
off-thread (one MMX6 compile was 26ms, paid entirely off the dispatch thread with
no frame-pacing spike), shards persist + reload, queue never backs up.
Observability (ring-buffer doctrine): always-on recent-compile ring + per-compile
timing in the worker, the resolved on-disk cache dir, and a tier-enabled flag, all
surfaced via a new sljit_async TCP debug command (documented in TCP_COMMANDS.md).
DISABLE the sljit tier by default (g_sljit_tier_enabled = 0): the sljit EMITTER
mistranslates some MMX6 overlay instruction(s) -- sljit-only save-load softlocks/
loopbacks while PURE INTERP works (discriminator), and it reproduces with the
worker OFF in the synchronous path (PSX_SLJIT_SYNC=1), so it is the emitter, not
the async worker. Until the emitter bug is found + fixed: no shard READ
(scan_sljit_cache_dir skipped), no GENERATE (worker not started + sljit_live
forced 0 + dispatch gap-fill gated), every overlay miss falls to the interpreter.
gcc (DLL cache + autocompile) and overlay capture are untouched, so priority is
gcc > interp and dispatch misses are still recorded for future gcc coverage. gcc
shards are bundled in release zips, so shipped builds are unaffected.
PSX_SLJIT_ENABLE=1 re-enables the tier for emitter-fix testing.
Runtime-only, no regen.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The write-trace ring proves whether a device-register store happened, but
to diagnose a CD interrupt that was never serviced you need to see what the
IRQ handler READ. Add an MMIO-read trace mirroring the existing MMIO-write
ring:
- debug_server.c: s_mmio_rtrace ring (reuses MmioTraceEntry, snapshots
func/pc/ra + i_stat/i_mask at read time), default-armed to the CD regs
(0x1F801800-1803) + I_STAT/I_MASK so it captures continuously from init;
rtrace_dump/clear/arm/ranges/stats commands.
- memory.c: wrap mmio_read32/16/8 (body -> _impl, thin wrapper records the
loaded value AFTER the single side-effecting read).
Range-filtered + always-on so the decisive device-register reads survive a
livelock that polls other MMIO. Additive diagnostics; no runtime behavior
change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The launcher Hybrid controller mode (built first, for Tomba, to mimic Tomba:
Special Edition) flips the emulated pad's reported type 0x41<->0x73 on
d-pad<->stick. After the real DualShock config state machine was added for MMX6
(98aa688), that flip regressed every game's libpad re-detect into a 1-frame
"pad unplugged": Tomba unpauses its menu / drops input, MMX6 phantom-dashes.
The modern config SM is correct and required by MMX6, so instead of weakening it
globally, restore the pre-98aa688 "config commands always answer 0xF3" behaviour
as a per-game opt-in: [controller] legacy_pad_config (default OFF). Only Tomba
sets it; with the flag off the modern pad path is byte-for-byte unchanged, so no
other title is affected. A/B verified live: legacy = clean Tomba flip / MMX6
input; modern = Tomba disconnect + MMX6 dash.
Explicitly LEGACY: a correct findpad/re-detect refactor would make the flip
benign for every game with no per-game branch (real hardware tolerates unlimited
analog-button presses); when that lands this whole feature set is to be removed.
Full rationale in sio.c (g_pad_legacy_cfg).
Also includes the deferred coherent pad-type apply (pad_type_req, applied only at
an idle/non-config bus boundary so a flip can't split a poll/handshake) and a
`pad_cfg` debug command + tools/padcfg.py + tools/sio_flip_trace.py used to A/B
the config SM live.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 16:9 BG widen draws LEFT extra tile columns each side, but the engine's
streamer (FUN_800273e4 -> FUN_800274a0) only refreshes one column per side per
frame at the leading edge. On scene entry / fast scroll the inner reveal columns
held stale tiles from a prior scrollY -> real-but-misplaced tiles spilling
up-and-left (the "staircase"). Census confirmed this is NOT the BG packet-budget
overflow (674-816 prims/frame, well under the 1000-tile cap).
Fix (render window == refresh window): clone FUN_800274a0's inner metatile->ring
fill into mmx6_fill_column() and re-stream every column of the widened window for
all 3 layers each frame (psx_ws_mmx6_bg_refill_all), triggered once/frame from the
existing count_site hook (mmx6_bg_refill_tick, frame-gated). Uses each layer's own
scroll, omits the engine's scrollX wrap-bookkeeping writes (those drive the engine's
own incremental streamer), skips map-loop seams; off-map columns fill tile 0 (the
renderer skips them). Validated byte-exact against the engine's own streaming
(0 mismatches / 1113 cells x 3 frames; debug cmd mmx6_freshfix {validate:1}). Gated
native-wide <=384w so 4:3 and the 512 hi-res title stay byte-identical.
Also removes the 2-col reveal cap (WS_FULL2D_MAX_REVEAL): it never addressed the
staircase and cost screen real-estate; the full 53px/4-col reveal is restored.
Parked, gated-off experiments ride along (inert, default off): host-side reveal
(Attempt A: gpu_ws_mmx6_emit_reveal + sw_wide_emit_tile) and BG packet-buffer
relocation (Attempt B: [widescreen.bg2d] bufbase/cap recompiler hooks). These target
the separate dense-stage budget-overflow concern, not this fix.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pure-2D sprite games (Mega Man X6) render a true 4:3 field of view with no
overscan to reveal, so 16:9 requires making the engine draw more. Two parts:
- [widescreen] full_2d opts a pure-2D game into the native-wide present path
(it never emits the 3D sprite-tag hook the gameplay detector keys on, so
every in-game frame was misclassified as a 4:3 menu and pillarboxed).
- [widescreen.bg2d] is a gen-time hook on the per-layer background tile
renderer (MMX6 FUN_800270d0) at its column-count / start-tile-column /
start-screen-x instructions, routed through gpu.c psx_ws_mmx6_bg_{cols,
startcol,startx}. It draws LEFT=ceil(nw_offset/16) extra 16px columns on
BOTH sides so the loop fills the 16:9 reveal margins with real adjacent
stage, existing columns staying pixel-aligned with objects. Identity at
4:3 and in the engine's own 512 hi-res mode (so the title is untouched).
Tooling: wide_full debug command + sw_wide_dump_full dump the entire
native-wide compositor surface (both vertical-double-buffer y-bands and both
reveal margins) for diagnosis.
Known follow-ups: object screen-X cull widen (actors in the margins);
top/bottom flicker + stale-margin refresh on the vertical double-buffer.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two CD-controller correctness fixes toward universal fast disc loads
(disc_speed=4x/instant) without per-game tuning.
1. Response arbiter (the core fix). The PS1 CD controller serializes
responses: only one occupies the IRQ slot (irq_flag + response FIFO)
at a time, and a later response is not exposed until the guest ACKs
the current one. set_irq() previously OVERWROTE irq_flag directly,
so when responses arrived faster than the guest acked (fast disc
timing) it silently DROPPED them. Proven: 24 dropped responses at
MMX6 boot under disc_speed=instant (INT2 completion clobbering an
unacked INT3 ACK), which wedged the loader.
set_irq() now snapshots the response bytes and either exposes (slot
free, gap elapsed) or QUEUES behind the visible response. The guest
ACK edge actively pumps the arbiter (cd_arbiter_pump): drain queued
responses in order, then start a command latched while the controller
was busy. cdrom_advance also pumps once the inter-IRQ gap elapses, so
a queued response is delivered even when no further ACK edge arrives.
Commands written while busy are latched and executed on ACK.
Result: 0 dropped responses at 1x/4x/instant. 1x boots to title
identically (no regression). The overwrite counter is retained as a
regression detector (now only counts true queue overflow).
2. Command-completion latency is now fixed, not speed-scaled. Pause/
Init/SeekL/GetID/ReadTOC second responses are controller/firmware
operations whose timing does NOT scale with read speed on real HW.
They used apply_speed() (divides by the disc-speed divisor), so at
4x the Pause completed 4x too fast and desynced the BIOS CD driver.
New cmd_latency() keeps them at authentic timing; only sector DATA
delivery (sector_delay_cycles) scales with disc_speed — which is
where essentially all load time lives, so loads still get fast.
Runtime-only (no regen). 1x verified clean on MMX6. A downstream
load-timing race still wedges 4x/instant at boot (guest spins with
interrupts disabled after a clean Pause) — tracked separately.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Model the R3000A having no usable D-cache (its data cache is repurposed as
the scratchpad): every CPU data read from main DRAM costs +6 wait-state
cycles on top of the 1-cycle opcode (DuckStation RAM_READ_TICKS = 6). The
recompiler previously charged 1 cycle/instruction flat, so BIOS/game busy-
wait poll loops ran ~6x too fast in guest time.
Concretely this fixes Mega Man X6's "This data is invalid" memory-card
error: the game's fixed-iteration TestEvent poll (A0(0xAC) directory load)
exhausted at ~43 frames while the VBlank-paced card directory op needs ~72
frames, so the poll timed out mid-operation; the retry re-called A0(0xAC),
A0(0x4F) returned card-busy, cleared A[port]=0, and orphaned the in-flight
sector completion -> permanent stall. With realistic per-read latency the
poll now outlasts the op (op runs once to completion, broken-sector index
reaches 20, terminal event fires), and the card screen shows the correct
"no save data found" message. Verified: empty card + a populated card both
load correctly; gameplay stable.
- memory.c: psx_guest_read_word/half/byte wrappers add +6 cyc for reads
whose effective physical address is main RAM (keyed on runtime address,
so relocated kernel code is timed correctly; scratchpad/MMIO/ROM excluded).
- main.cpp: wire cpu.read_word/half/byte to the guest wrappers (covers
static-recompiled and dirty-RAM-interpreted loads, incl. LWC2).
- debug_server.c: trace_read_word/half use psx_read_* directly so debug
reads (IO thread) never charge guest cycles or race the emu thread.
Also includes prior uncommitted DualShock SIO config state machine
(sio.c) + dual keyboard/pad P1 input (main.cpp) from the MMX6 bring-up.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per-op counters (allocs/alloc_kb, oneshots, submits, syncs, pack_flushes,
blits, upload_blocks, copy_rects, geo/tex flushes) accumulate per frame and
snapshot into a 256-entry ring at each present(); queryable via the debug
server "vk_perf" command (count=N, default 60). Always on (Release too),
bounded memory. Purpose: pinpoint which op explodes during the VK transition
stall (e.g. make_staging vkAllocateMemory churn) instead of guessing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Vulkan backend alongside software + OpenGL: geometry/textured-prim
pipelines, GPU-authoritative VRAM image, deferred-submit + drain-at-present
perf model, GPU-direct present (vk_renderer_present_vram), and a
full-VRAM-pack readback path (ensure_cpu rect_adds the whole VRAM rect
before pack_flush so screenshots/guest VRAM reads reflect live content,
not just the incremental pack-dirty rect — readback-only, no per-frame cost).
Runtime plumbing: --debug-port / --renderer / --window-title CLI overrides
(multi-instance fleet), latency_ring, SPIR-V shaders + embed_spirv.py,
config_loader renderer selection, debug_server screenshot wiring.
All on feat/vulkan-renderer; snapshot in backup/vulkan-wip-20260617.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Oracle disambiguation (RECURSION_BUG.md §23–§24). Traced one func_8001A954
invocation on real HW (psx-beetle) vs the recomp in the IDENTICAL Tomba idle
state (New Game village dialogue; object 0x801FD950, state/scene/jumptable=0).
VERDICT: real HW's per-frame chain is BALANCED — func_8001A954 is entered once
per frame from main_loop (JAL 0x8001A630, sp=0x801FE3E0) and RETURNS once per
frame (jr ra @0x8001A9E8 -> 0x8001A638, SP fully restored); no re-entry, the
overlay 0x80063864 never calls back (refutes §21/§22's "call-cycle"). The recomp
runs the same guest flow and the guest SP unwinds identically, but the HOST stack
leaks +1.17 KB/frame (ce_profile, live-reconfirmed: 20099->22215 KB / 1807 frames)
because the per-frame compiled chain (func_8001A954->AC00->B2B4->B5A8->func_80046264,
emitted as DIRECT func_X(cpu) host calls) re-enters itself via func_80046264's
CD-DMA-dirty hop and never unwinds.
native_stack early-trip (PSX_STACK_GUARD_KB=6144, frame 7592, 6 MB) pins the
leaking host cycle. Cross-checked: xprobe mixed_depth bounded at 6, dirty-insn/
dirty-block show NO literal func_8001A954 transfer, and a954=1/frame is a SURFACED
guest tail-transfer (j/jr) to func_8001A954 that the oracle proves real HW never
makes — i.e. there is both (A) a correctness divergence (spurious per-frame
re-entry) and (B) the architectural host-mirroring limitation that amplifies it.
Fix scope CORRECTED from §23's "quick contained runtime patch": the UNIVERSAL fix
is §11 — route in-range guest calls through the depth-managed psx_dispatch_impl
trampoline instead of mirroring them as direct host calls, so the host stack stays
bounded regardless of guest re-entrancy (real or fabricated). §12 (CD-DMA dirty
precision) and fixing the specific divergence are surgical mitigations, not
universal. Performance loss accepted for now (static recomp: right before fast).
Instruments added (runtime-only, no regen):
- SP capture in the fntrace control-flow ring on BOTH backends (GPR[29] read live):
psx-beetle (beetle_libretro.cpp + beetle_debug_server.c) and the recomp
(fntrace.h/.c + debug_server.c) — enables the func_8001A954 SP/RA-lifecycle oracle.
- psx-beetle default debug port 4380 -> 4382 (+ --port N override): 4380 collided
with psxref and cdirecomp/CdiRuntime.exe; two LISTENers on one port silently
misroute connections (the phantom "oracle keeps dying" early in the session).
Oracle tooling: the control-flow oracle is psx-beetle (Beetle interprets -> records
every J/JAL/JR/JALR), NOT psxref (no PC trace; the assumed emu_step/emu_trace_addr
never existed). Loads Tomba via: psx-beetle.exe --disc tomba.cue <bios> --port 4382.
Next: implement the §11 universal fix (codegen + full regen).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The ~14-min Tomba idle freeze is a gradual per-frame host-stack LEAK (not a
within-one-frame spiral, not the gradual-marking theory). Confirmed via a new
compiled-entry depth profile (§19): per-frame MAX guest-stack-used climbs dead-linear
+1.17 KB/frame from gameplay start to 48 MB on the 64 MB main-thread stack -> native
stack guard -> freeze. Once per frame an interpreted overlay (reached via
func_80046264's dirty hop jal 0x80063864) does jalr func_8001A954, nesting a host call
that never unwinds (the per-frame loop never returns). Full writeup RECURSION_BUG.md
§16-§21.
Instruments (always-on rings, runtime-only, no regen):
- §17 stack_profile: per-vblank host-stack sample (main.cpp; stack_profile TCP cmd).
- §18 xprobe / xprobe_arm: interp<->compiled boundary crossing flight recorder
(dirty_ram_interp.c; per-frame summary + per-crossing detail).
- §19 ce_profile: per-compiled-entry depth profile + raw TEB at the trip
(debug_server.c; ce_profile TCP cmd) — the trustworthy instrument; the regression
check for the fix (max_kb must stay FLAT).
- PSX_STACK_GUARD_KB early-trip + PSX_FIBER_STACK_KB (debug_server.c / traps.c).
Fix progress:
- Block-loop tail-transfer surface (dirty_ram_interp.c ~1514): surface compiled
tail-transfers to the trampoline instead of nesting. Correct for tail-transfers but
RULED OUT as the leak (§19 climb unchanged) — the leak is the CALL path (jalr/jal).
- Real fix (next): surface compiled-game CALLS to the owner trampoline, gated to
compiled-game targets only (a naive all-targets surface reintroduces the
dwarf->overworld lost-continuation bug). Needs a non-nesting membership test
(generated psx_dispatch_game_has) or §12 CD-DMA dirty-marking precision.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Live-tunable selectable stretch gate to identify which drawn prims form the
native-wide 2D-backdrop void: ws_dbg_stretch modes = OT-range / clut / tpage /
tagged / untagged-tex / all-tex, with per-frame matched + matched-tagged counts
(psx_ws_dbg_gate_frame_snapshot). Routes through the existing GL centre-stretch.
Confirmed live: the void layer is the early-drawn, SPRITE-TAGGED 0x65 background
grid (6 cols x 7, step-64, cluts unique to the backdrop); stretching it about
screen-centre fills the void with no inter-tile gaps (GL scales width too).
Tagged fills it; untagged (3D) does not. The prior content-match plan is
DISPROVEN (template transformed beyond recognition before draw) -- corrected
model recorded in BACKDROP_PRELOAD.md. Fix gate chosen: native-wide + tagged +
background-phase (draw-time), since backdrop X is written by a shared sprite
helper with no clean per-emit source signal.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 16:9 dwarf/flower-field void is native-wide 2D-backdrop POSITIONING, not a
column-loading problem. The earlier auto_backdrop whole-row preload is proven
IRRELEVANT (ws_backdrop_ring: whole 41-col row loads, void persists; margin off
vs whole-row pixel-identical). The flower-field is a fixed-screen 2D tile field
(static 11-bit vertices, ~4:3 width) that bypasses the GTE, so native-wide (which
widens 3D via the GTE) never widens it. Fix = geometric X-stretch about
screen-centre of the 2D backdrop in the GL wide-mirror — USER-CONFIRMED it fills
the void (with the gate disabled). Full record: BACKDROP_PRELOAD.md (2026-06-16).
Remaining: gate the stretch to ONLY the backdrop (currently also stretches Tomba/
HUD/3D). Path forward = draw-time gp0_cmd_source_addr / sprite-tag classification.
Runtime WIP checkpoint (all behind native-wide + runtime toggles, 4:3 untouched):
- gpu_gl_renderer.c: u_xscale/u_xcenter geometric stretch in the geo+tex
wide-mirror passes; per-frame backdrop-phase reset moved to gl_perf_present_enter
(both present paths); textured-only phase mode; per-prim trace + dbg counters.
- gpu.c: ws_backdrop_ring (always-on rewrite ring); ws_backdrop_margin live
column-widen (proven irrelevant); psx_ws_backdrop_value whole-row path.
- dirty_ram_interp.c: ring capture (extent/camx/count/a1/dl) at the interp hook.
- debug_server.c: ws_backdrop_ring / ws_backdrop_margin / ws_backdrop_stretch.
- main.cpp: keyboard ALWAYS drives player 1 (OR'd with any controller).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On macOS, defining _POSIX_C_SOURCE restricts the visible namespace and HIDES
BSD symbols the TCP debug server needs — notably INADDR_LOOPBACK (and
struct timeval). clang then fails with 'use of undeclared identifier
INADDR_LOOPBACK'. Use _DARWIN_C_SOURCE on Apple (which exposes the full
BSD+POSIX namespace incl. clock_gettime), keeping _POSIX_C_SOURCE for glibc.
Mirrors the same __APPLE__/_DARWIN_C_SOURCE guard already used in psx_fiber.c.
handle_synth_recurse (a command handler, compiled even in Release) called
debug_server_synth_recurse_arm, which is defined only under #ifndef
PSX_NO_DEBUG_TOOLS — so a Release build (PSX_DEBUG_TOOLS=OFF) failed to link with
an undefined reference. The self-test is debug-only and the TCP server never
starts in Release, so guard the call; the handler is dead code there.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Persist validated sljit shards across launches (SLJIT_PERSIST_CACHE.md). On JIT,
serialize the (position-independent, Stage 1) LIR and write it to
<cache>/<game_id>/sljit/<arch-abi>/cg<N>/<entry8>_<crc8>.sljit with a small header
(magic, format/helper-order versions, entry/lo/len/crc/size). At init, scan that
dir, deserialize + sljit_generate_code for THIS process, and register each as a
dll<0 candidate carrying the blob's crc; dispatch re-hashes live RAM against it so
a reloaded shard re-earns trust exactly like a fresh JIT. Priority gcc > sljit >
interp is unchanged (a gcc DLL still supersedes a reloaded shard via the obsolete
path). Persist is write-on-generate (measured negligible; crash-safe).
Threads the serialized blob through OverlaySljitResult -> CompiledFragment;
register_sljit_candidate gains a crc_override (reloaded shards use the header crc
since the overlay RAM may not be loaded at init). sljit_status exposes
reloaded / persist / reload-diagnostic counters.
Two gotchas found + fixed in testing (vs the original design notes):
- serialize MUST precede generate_code (generate sets compiler->error =
SLJIT_ERR_COMPILED, after which serialize returns NULL);
- serialize with options=0, not SLJIT_SERIALIZE_IGNORE_DEBUG, so an
argument-checks build can deserialize its own blobs.
Validated: two-launch test reloaded 12 shards (deserfail=0), ran them at a fresh
process base with no crash, gameplay renders correctly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Controller (item 3): present an analog (DualShock) pad by default for
analog-capable games so the left stick gives variable run speed; fold the
physical D-pad / keyboard arrows onto the left analog axes at full deflection so
they still move the character (avoids the Special Edition full-throttle-D-pad
bug). Add a launcher-editable analog-stick deadzone option (settings.toml +
input.ini + game.toml [controller] default), modeled on snesrecomp.
Settings persistence (item 6): deterministic restore-at-init + save-on-exit for a
game's native OPTION-screen settings, declared per game in a dedicated
game_options.toml. The recompiler rewrites each setting's boot-init store (sb/sh)
to route its value through psx_game_option_store(addr,val), applying a value
persisted from a prior session over the game default (identity when none, so a
fresh install is byte-identical). The runtime saves the live config globals to
<memcard_dir>/<game_id>.options on exit (atexit). Adds a game_options debug
command. Requires a regen for the hook to emit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The native stack guard captures a report when a runaway guest recursion is
about to overflow, but the report couldn't reliably NAME the recursing function:
the SEH stack_scan (host offsets, build-specific) is absent on the graceful-halt
path, last_func_addr comes from the dispatcher (blind to direct func_X
recursion), and the existing fn-entry ring is debug-only + fntrace-gated (empty
in release).
Add an always-on (debug AND release) ring of the last 64 recompiled function
entries in GUEST addresses, plus g_psx_recursion_func = the function being
entered when the guard trips. crash_trace dumps recursion_func + recent_fn. For
a runaway the recent_fn tail repeats the recursing func, so the next ORGANIC
crash is root-causable in Ghidra with no symbol map — pair it with cpu.gpr (the
args/corrupt value at the trip).
Validated via the new `synth_recurse` debug self-test (forces a runaway): guard
halts cleanly, report shows recursion_func=0x8000DEAD + recent_fn all 0x8000DEAD.
No measurable overhead (16:9 stays 60fps); no false-trips across village play.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The watchtower force-close (and the matching GitHub issue) is a host C-stack
overflow (SEH 0xC00000FD) from runaway guest recursion — the seesaw/Bug-D
wild-call family, where a corrupt jump-table loops a handler back on itself.
An in-range guest `jal` emits a DIRECT func_X(cpu) call, so this recursion grows
the native stack WITHOUT going through psx_dispatch_impl: g_psx_dispatch_depth
(and the existing depth-counter guard in dirty_ram_dispatch) is blind to it —
the captured report shows STACK_OVERFLOW with dispatch_depth=0.
Add a path-independent guard at debug_server_log_call_entry (emitted at the top
of every recompiled function, so it covers direct-call recursion too, and runs
in release since it precedes the PSX_NO_DEBUG_TOOLS early-return). It reads the
running fiber's real stack bounds from the x64 TEB (StackBase GS:[0x08],
DeallocationStack GS:[0x1478]) and, when headroom drops below 1/4 of the stack,
halts via psx_fatal_halt — captured crash report + halt-and-serve in debug,
clean exit + report in release — BEFORE the stack overflows. Guest fibers are
1 MB (traps.c); the 1/4 headroom only trips on runaway recursion (thousands of
frames), never on legitimate gameplay depth. Verified: no false-trigger across
extended village play (frame 6117); compiles + present in both debug and release.
TENTATIVE containment + diagnostics, NOT a root fix — converts a silent
force-close into a captured, parseable report (dirty_block cycle names the
recursing PCs) so the intermittent crash can finally be root-caused from a
repro. The upstream data corruption is unaddressed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Measured Tomba village = 93% textured prims, so draw batching must target the
textured path (per-prim uv-limits/texpage/clut/depth move to vertex attributes
so consecutive textured prims coalesce into one draw). Flat batching alone
would touch only 7% of prims.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds prims/frame and per_prim_us to the frame_perf aggregate. Proved Tomba's
village is draw-call bound: ~1000 prims/frame at ~14us of GPU time PER
glDrawArrays (a 1-triangle draw should be <1us) — pipeline-flush overhead from
re-binding state between every primitive. 16:9 is 2x slower purely because the
native-wide double-draw issues 2x the draw calls. Points squarely at draw
batching (group consecutive same-state prims) as the fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Always-on per-frame ring in the GL renderer: two GL_TIME_ELAPSED queries
bracket each frame's scene draws vs the present clear+blit (true GPU time,
independent of CPU/GPU overlap), plus SDL perf-counter CPU wall time
(present-to-present total, present call). Exposed via the debug server
"frame_perf" command, aggregated over the recent ring and split by 4:3 vs
native-wide so a 16:9-vs-4:3 A/B reads off one query.
Built to answer where a frame goes without guessing. First use proved
Tomba's 16:9 village lag is the native-wide DOUBLE-DRAW (every primitive
re-issued into the wide FBO), NOT supersampling (SS2==SS1) and NOT the
compositor (present_gpu ~1ms): scene_gpu 16:9=28ms vs 4:3=17ms.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two bugs in the GL native-wide present (both root-caused via the new wide_shot
capture):
1. ZOOM + wide_shot crash: glb_scale() was a stub returning 1 while the GL
renderer actually renders at s_scale (e.g. 2). main.cpp:1801
`g_video_scale = gr_scale()` collapsed g_video_scale to 1 on GL, so the wide
CPU present sized buffers/dims for scale 1 (426x224) while gr_render_wide_
display produced scale-2 data (852x448) -> present uploaded it as 426-wide and
showed a magnified sub-region (the zoom); the wide_shot probe overflowed its
buffer (crash). Fixed glb_scale() to return s_scale (matches the SW backend's
sw_renderer_scale contract; both gr_scale() callers want the real scale; the
GL FBO-direct present is unaffected — it never reads gr_scale()).
2. UPSIDE-DOWN: glb_render_wide_display flipped rows on readback, but the wide
FBO stores PS1-y inverted (vram_y=0 -> NDC -1 -> FBO bottom) and glReadPixels
reads bottom-up, so the two inversions already cancel — the extra flip turned
the frame upside-down. Removed the reversal (copy straight = top-down, matching
sw_render_wide_display + the PRESENT_VS V-flip).
Adds the `wide_shot` debug command (debug_server.c): dumps the composited
native-wide present surface to a PNG (gr_render_wide_display), unlike
screenshot_file which captures only the canonical pre-compositor VRAM. This is
how both bugs were seen + verified.
Verified: wide_shot returns a right-side-up, full-frame 16:9 capture (no zoom,
no flip, no crash). Final visual sign-off pending (human).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root-cause class: load priority is static > gcc shard > sljit shard > interp,
but it was only emergent from chain registration order (LIFO prepend), not
enforced, and a superseded sljit shard was never retired. A shipped/optimized
gcc DLL must outrank — and obsolete — a user machine's locally-JIT'd sljit shard
for the same content.
Fix (overlay_loader.c): when a gcc/DLL candidate registers (cand_register),
retire any same-PC sljit shard JIT'd from IDENTICAL bytes (same crc_code) —
blacklist it so it can never run even if the gcc candidate is later self-mod
blacklisted. Precise by content: a sljit shard for a DIFFERENT overlay variant
at the same address is distinct coverage and is kept. New counter
s_sljit_obsoleted, surfaced via overlay_loader_sljit_obsoleted().
debug_server.c: sljit_status now reports live + obsoleted (observability for
the validation playtest — measure, don't eyeball).
Runtime-only (no regen). Refs SLJIT.md §11 Phase B.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Render the genuine 16:9 FOV at correct proportions instead of squash-then-stretch,
using a separate per-framebuffer wide surface so canonical PSX VRAM stays faithful
(4:3 / non-opted games byte-identical; the wide canvas only exists when opted in).
- Wide-aspect mode descriptor (off / squash / native-wide) in gpu.c; opt-in via
[video] aspect_ratio. GTE fed the 4:3 ratio (identity, no squash) in native-wide;
canonical draw_offset/draw_area left faithful (no VRAM widening, no wrap).
- Separate wide present-surfaces (gpu_sw_renderer.c) keyed by back-buffer base:
framebuffer prims mirrored at local x = vram_x - base_x + OFFSET; margins cleared
via the frame-clear mirror; present reads the displayed buffer's surface 1:1.
Eliminates the A/B bleed + flicker (and the segfault) of the shared-canvas attempt.
gr_* vtable extended; GL backend reports unsupported -> canonical present until
the GL port (task 8).
- psx_ws_x_margin native-wide-aware (reuses the existing [widescreen.cull] emit).
CONFIRMED a no-op for Tomba's edge objects: they spawn via a non-FUN_80022E44
path (the foreground classifier is already 16:9-safe). See handoff.
- Draw-census extended with per-prim screen-X extent + back-buffer base for the
native-wide void diagnosis; ws_nw TCP toggle; gpu_state ws.mode/nw_extra.
- Controller: "auto"/"gamepad"/"controller" device = first available SDL pad.
WIP checkpoint. SW compositor works: flicker fixed, faithful, terrain platform
fills the wide frame. REMAINING: discrete foreground OBJECTS (rock pillar, tree
branch, bushes) + the parallax backdrop are still culled at the 4:3 edge via a
separate generation/spawn path (wtrace the OT display-list writers next); GL
compositor port. Detail in NATIVE_WIDE_PLAN.md + the session handoff.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The server was pumped from the emu thread, so when the emu thread is
buried (e.g. the same-state differential's per-dispatch overhead) the
socket goes unanswered and clients time out.
- A dedicated SDL I/O thread now owns accept/recv/send. Commands still
EXECUTE on the emu thread at the debug_server_poll() safe point, via a
single in-flight request slot (the client is one-command-per-connection)
-- so there are no new state races and the ~200 send sites are unchanged:
send_line now buffers into a growable response buffer (s_in_command),
which the I/O thread sends. Async pushes (watchpoint events) have no
persistent client under this protocol and are dropped, as before.
- `ping` is answered directly on the I/O thread, lock-free, so liveness is
queryable even while the emu thread is buried or frozen.
- The I/O thread bounds its wait for a response (30s) so a wedged emu
doesn't hang the connection; the emu discards a response the I/O thread
already gave up on.
Deadlock-free: the emu thread only ever signals the I/O thread, never
waits on it. Builds clean; exercised live in the next validation playtest.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The MIPS->sljit emitter, paralleling dirty_ram_interp.c's exec_one but
emitting host code. First slice = single-block LEAF functions ending in
`jr $ra`; everything else aborts the whole fragment to the interpreter
(precision over recall).
- overlay_sljit.c: memory-backed GPRs, 32-bit ops (R3000A wraparound),
loads/stores via icall through the cpu read/write callbacks (same
MMIO/page-watch/dirty-tracking path as the interpreter). Opcodes:
SLL/SRL/SRA/SLLV/SRLV/SRAV, ADD(U)/SUB(U), AND/OR/XOR/NOR, SLT(U),
ADDI(U)/SLTI(U)/ANDI/ORI/XORI/LUI, LB/LH/LW/LBU/LHU/SB/SH/SW, SYNC.
- compile_fragment now returns CompiledFragment{fn,code_lo,code_len};
overlay_loader registers an sljit candidate (crc hashed from the live
bytes it JIT'd from, same live-byte validation as gcc candidates).
- Step 5 wiring is VALIDATION-GATED: JIT-on-miss fires only when
backend==sljit AND the same-state differential is armed (never in a
shadow run). Normal play stays byte-identical; live sljit execution
stays off until the differential gate passes.
- Debug: sljit_try <addr> probe; sljit_status reports shards_registered.
Verified: runtime/tests/sljit_emit_test.c (links overlay_sljit.c +
sljitLir.c only) JITs hand-built leaf functions and checks output —
26/26 pass incl. gpr0 invariant, sign/zero-ext, signed/unsigned compare,
and the decline cases. Full psx-runtime build clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Introduce one backend-agnostic code-production seam the overlay
capture/dispatch spine talks to, and make the validated gcc spawn->DLL
path its first implementation. Pure refactor; gcc stays the default and
its behavior is byte-identical.
- New code_provider.h/.c: a CodeProvider struct exposing BOTH production
lifecycles so the interface isn't gcc-shaped -- batch/async
(request/busy/poll_main, the gcc model) and sync-on-miss
(compile_fragment, the sljit model). compile_fragment is defined and
implemented per-provider but not yet called from dispatch (step 5).
gcc provider = direct pass-throughs to autocompile_* (compile_fragment
NULL: a compiler spawn is too slow for the dispatch path); sljit
provider wraps overlay_sljit_* (still declines every fragment).
- Wire up overlay_backend_resolve, which was previously dead code:
code_provider_init() resolves the backend at overlay-cache init and
caches the active provider; code_provider_active() is the accessor.
- Add [runtime] overlay_backend config (auto|gcc|sljit; empty=auto) so
resolution has a config source alongside PSX_OVERLAY_BACKEND.
- Route the 3 direct autocompile_* call sites through the active
provider: overlay_capture.c autocapture tick, main.cpp per-frame poll
+ the init call, debug_server.c autocompile_run (now reports backend).
- runtime.cmake += code_provider.c.
Builds + links clean (TombaRecomp/build-dev, MinGW gcc 15.2).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add sljit as a self-contained, toolchain-less codegen backend for the
overlay self-improvement loop. gcc stays the default (dev: optimized +
inspectable); sljit is the fallback / production producer. Interpreter
remains the correctness floor.
This commit lands the backend-agnostic foundation only; the MIPS->sljit
emitter is not built yet, so every fragment safely declines to the
interpreter (no change to the validated gcc path).
- Vendor sljit (BSD-2-Clause) at lib/sljit/, wired into runtime.cmake
(compile sljitLir.c + include dir; sljit auto-detects the host arch).
- overlay_sljit.{h,c}: backend-selection policy (PSX_OVERLAY_BACKEND=
auto|gcc|sljit), availability, a real codegen smoke test, and
try_compile() honoring the precision-over-recall bail-to-interp contract.
- sljit_status debug-server command (Rule-3 clean: status via getter).
- THIRD_PARTY_ATTRIBUTION.md: sljit entry.
Verified: TombaRecomp/build-dev (MinGW gcc 15.2) compiles + links
psx-runtime.exe; standalone smoke harness JITs f(a)=a+1234 and runs it
correctly (26 bytes x86-64). Design + locked tier decisions in SLJIT.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The far backdrop (ocean/cloud/distant mountain) is GTE-projected 3D drawn
by the main-EXE driver FUN_8004db3c (sole caller of the GTE emitter
FUN_80027600). The global widescreen GTE X-squash compresses it toward
centre, so its geometry falls short of the revealed 16:9 edges -> blue void
that pops as the camera moves.
Fix: bracket the backdrop driver with gte_ws_set_suppress(1)/(0) so its
draws skip the X-squash and fill the stretched frame (skybox treatment),
while the near 3D world keeps the wider FOV.
FUN_8004db3c draws a MIX (far backdrop + nearer props/scenery), so the
suppress is DEPTH-GATED in gte_rtps_internal: only vertices with projected
SZ >= s_ws_far_threshold (default 900, tunable live via the ws_far_threshold
TCP cmd) un-squash; nearer props stay squashed and aligned. Validated:
threshold 900 fills the void, keeps the foreground door + background
scenery (path/tree) aligned. Residual: the mountain grass-top straddles the
depth band (per-vertex limitation) -> flickers when the camera dips; this
is the checkpoint before exploring native-wide rendering to remove the
squash (and this whole artifact class) entirely.
Mechanism:
- recompiler: [widescreen.backdrop] unsquash_funcs emits
gte_ws_set_suppress(1) at entry + (0) before each jr-$ra return
(code_generator + config_loader + main_psx). Main-EXE -> regen-class.
- runtime: gte.cpp depth-gated squash + s_ws_far_threshold; cpu_state.h
decl; debug_server.c ws_aspect (live squash on/off) + ws_far_threshold
(set split + read SZ stats) diagnostics.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Observability for the 16:9 backdrop-cull investigation (runtime-only,
4:3-identity-safe, no regen):
- gpu.c: always-on per-prim draw-census ring (ws_census). Every drawn GP0
primitive (0x20-0x7F) records {frame, src_addr, camX/camY, first-vertex
screen x/y, opcode} so object spawn/despawn and edge culls are observable
in data instead of screenshots. gpu_ws_census_dump() writes a frame range
to CSV (large dumps must not ride TCP).
- gpu.c: gpu_ws_set_margin_override() forces psx_ws_x_margin() to any value
for a live cull sweep at a fixed camera; gpu_ws_get_debug() snapshots live
widescreen state.
- debug_server.c: gpu_state now reports ws.{configured,active,game_mode,
present_native_43,x_margin,squash,cur_frame,last_tag_frame}; new ws_margin
and ws_census TCP commands.
- gpu.h: GpuWsDebug + census/override API.
- debug_client.py: ws_margin and ws_census subcommands.
Used to prove the shipped [widescreen.cull] sites are a no-op for 8C (their
4:3 margins already cover the 16:9 view) and that the real pop is an
overlay-resident backdrop layer. See TombaRecomp ISSUES.md #8C.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Per-player input routing with a real device dropdown (None / Keyboard /
each plugged-in SDL controller) and a digital-vs-DualShock pad-type toggle,
wired end-to-end through the runtime.
- sio.c/.h: per-slot pad state (pad_buttons[2]) + per-slot pad type. Analog
poll emulation (id 0x73 + 4 stick axes) when a slot is in DualShock mode;
digital (0x41) stays the default so existing games are unaffected. New
setters: sio_set_pad_state_slot / sio_set_pad_analog / sio_set_pad_connected,
getters sio_get_pad_{buttons,connected,analog}_slot.
- config_loader: settings.toml [controller] p1_device/p2_device (none/keyboard/
<SDL GUID>) + p1_analog/p2_analog; load + save round-trip.
- main.cpp: replaced the single auto-opened controller with a two-slot
PlayerInput model. Devices opened by GUID (with a first-free fallback so a
re-paired pad still works), keyboard or controller feeds each PSX port, and
analog stick axes drive the DualShock poll. Hot-plug re-resolves both slots.
Defaults: P1 keyboard/digital, P2 none.
- launcher: enumerates SDL controllers (name+GUID); each player card has a
custom click-to-open dropdown (option list injected via data-rml, the
data-for path being unreliable in this RmlUi build) and a DualShock switch
that drives the pad art (digital/analog) + status line. Zero-config default
selects the first plugged-in pad. A saved-but-unplugged controller is kept
as an "(offline)" option so the selection survives. Choices persist on launch.
- debug_server: pad_status now reports both slots' buttons + connected + analog.
UI polish from review:
- Removed the redundant disc-verdict widget (the green checks already convey it);
Change ISO now anchors to the disc panel instead of floating at the page top.
- Added a prominent "Settings" button in the top bar (renderer/SS/AA/etc. were
only reachable via the easily-missed footer link).
- Fixed the player-card asymmetry on toggle: status text no longer wraps and the
player card body is a fixed height, so the pad art renders at a consistent size
(measured 216x110 analog vs 219x110 digital) regardless of pad type.
Verified: device dropdown enumerates a real DualSense; selection + DualShock
toggle update the card; pad_status confirms per-slot connect + analog routing
(slot0 analog / slot1 digital and the reverse); default keyboard P1 unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
New debug server commands for the GL backend: gl_fbo_peek reads a rect
of GPU-side VRAM (via the pack pass) without touching the CPU array,
gl_vram_diff reports full-VRAM FBO-vs-CPU mismatch count/bbox/samples,
and gl_coh_ring dumps the always-on coherency event ring (last n events,
optional frame_min filter).
crash_trace SEH dumps now report the faulting module + base + offset and
scan the stack for executable return addresses, so dumps stay
addr2line-able despite ASLR.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The FBO-present crop shader sampled a sub-region of the 1024x512 VRAM
texture with vertex-interpolated UVs; sub-texel precision over the
fullscreen triangle produced a vertical seam and text smearing on screen
(the GPU render itself was correct — forcing the software present path
showed a clean image, which localized the bug to the present sampling).
Replace the crop shader with glBlitFramebuffer: an exact integer-rect
copy of the display region from the FBO to the window, with the vertical
flip folded into the dst rect. No per-fragment UV math, no sub-texel
error. linear=GL_LINEAR (AA) else GL_NEAREST.
Also:
- screenshot now calls gl_renderer_sync_cpu() first, so captures reflect
the on-screen frame under the FBO-present path (was reading stale CPU
VRAM). Safe: the debug server is pumped on the main GL-context thread.
- PSX_GL_FORCE_CPU_PRESENT=1 forces the software readout present path —
a fallback/diagnostic that also keeps CPU VRAM current every frame.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Matthew Stanley