The dirty-RAM text-image guard (685f38d) only armed when the loose local
boot-EXE file next to game.toml existed - found via find_project_root's
repo-marker walk. End-user release installs have neither a repo checkout
nor the EXE file (it is copyrighted game data), so the guard silently
never armed. Without it, fntrace's alternate game-start latch
(psx_game_address_in_text && dirty_ram_text_native_ok) cannot fire for
titles whose BIOS handoff enters text inside the shell-window shadow
(RAM 0x30000-0x5AFFF -> shell ROM until game-start): the shadow never
lifts, execution runs dead shell ROM bytes over real game text, and the
boot dies in a jump to PC=0 (Ape Escape v0.0.2/v0.0.3 release zips,
frame 14, both HLE and LLE boots; in-repo runs armed invisibly and
masked the bug during validation).
Fix, class-level for every title:
- main.cpp: defer guard arming until the disc path is resolved and add a
second source - extract the boot EXE from the mounted disc image via
ISOReader (basename of game.toml's exe field, SYSTEM.CNF BOOT token
fallback). The disc is the same bytes the BIOS loads, i.e. the true
reference; every install has one. Local EXE file still wins for dev
checkouts. A loud WARNING prints when neither source arms (previously
silent, which is how the broken releases shipped green).
- fntrace.c: the alternate game-start latch now requires a registered
image (dirty_ram_text_image_registered). Its safety argument - "RAM
still holds shell bytes that differ from the game image, so it cannot
fire prematurely" - only holds when an image exists; with no image,
native_ok degrades to !dirty and untouched text pages pass it,
latching game-start before the EXE loads and clearing the dirty
baseline mid-load. Without an image, only the exact entry_pc dispatch
may latch (the pre-685f38d conservative behavior).
Verified: v0.0.3 zip contents + rebuilt exe in an isolated directory
(no repo above it) boots to the Ape Escape intro on both HLE and LLE
paths, guard armed from disc SCUS_944.23; in-repo runs still arm from
the local EXE.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Generalizes the Capcom 2D widescreen machinery for the MMX4 engine revision:
- [widescreen.bg2d] count_site accepts the inline slti/sltiu loop compare
(X4/X5 style) alongside MMX6's li; startx_site accepts subu rd,zero,rt
(X4's -(scrollX & 15)); inline-count sites route through the generic
psx_ws_bg2d_* helpers that keep the emitted ring mask and skip MMX6's
title-specific host-side refill state.
- [widescreen.cull] a1_sites now also accept a move rd,a1 (addu encoding):
classifiers that copy the caller margin with no delay-slot nop widen the
copied value instead.
- [widescreen] nw_hud_source_lo/hi: optional GP0 command-source range filter
for nw_hud_corners, so a pure-2D game re-anchors only its dedicated HUD
packet arena and never world sprites. 0/0 preserves legacy behavior.
ws_nw_hud_shift_vertices re-anchors polygon/line prims as one rigid
composite; mono rect/dot/8x8 sites shift too.
All helpers identity at 4:3; unset config keys are byte-identical.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two hardware-faithfulness fixes found by the always-on tap rings + Beetle
oracle A/B on fingerprint-matched attract music (tools/audio_pair_segments):
the recomp played the identical track at -3.8 dBFS vs the oracle's
-14.1 dBFS — +10.3 dB hot, clipping continuously.
1. spu.c: the voice repeat/loop address register (voice reg 7) is LIVE
state on real hardware — writing it after KEYON retargets where the
next END+REPEAT block jumps (Beetle spu.cpp:1150/333), and the block
auto-latch (flag bit 2) updates the guest-visible register (386/1302).
The recomp cached repeat_addr at KEYON only, so X5's driver idiom —
KEYON, then point the loop register at a silent tail block; no KEYOFF
ever — left voices looping the LOUD sample body forever. Measured: 3
stuck voices re-looping ~1400x/s, 160k SPU events/min vs the oracle's
633, zero game KEYOFF writes (the game never needs one), +11 dB.
2. cdrom.c: implement the CD controller's decode volume matrix
([src][port], 0x80 unity, pending banks + apply latch at 1F801803.3
bit 5 — mapping copied from Beetle cdc.cpp:1483-1501, applied after
the XA resample per PS1 hardware tests). X5 drives it constantly:
0x73-0x7E steady, ramped to 0x00 at scene changes (measured via the
new beetle `cdc_volume` command + PS_CDC::PSXRecomp_GetDecodeVolume
peek). Without it music fades never happened and steady level sat
~0.6 dB hot.
Also: AUDIO_TAP_VOICES (T0 voice-sum tap) for source-by-source mix
decomposition, per-tap WAV rates, bridge stats in audio_stats.
Measured after (MMX5 attract, 20-25 s windows):
spu_out RMS: -3.8 dBFS (clipping) -> -15.5 dBFS [oracle -14.1]
music-only chain already matched the oracle at -14.8 dBFS, confirming
decoder + mixer gain equivalence; the entire excess was stuck voices.
underruns: 2 events / 98 s (delivery fix holding).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replace the push-model SDL_QueueAudio output with the shared band-limited
polyphase resampler + P-only fill servo (recomp_audio_drc.h), fed by an SPSC
ring under SDL_LockAudioDevice and drained by a pull callback. The SPU renders
44100/60 per wall-clock frame while the host consumes on its own crystal; the
unsynchronized clocks drifted the queue to underrun (silence gaps = sustained
crackle). The bridge nudges the resample ratio <=+/-0.5% to hold the ring near
target so the queue never runs dry.
Measured on MMX6: legacy push ~1.5-2.5 underruns/s (queue draining to 0ms) ->
bridge 0 underruns, fill held ~50ms. PSXRECOMP_AUDIO_LEGACY=1 keeps the old
push path for A/B; PSXRECOMP_AUDIO_PROBE=1 prints the fill/underrun trend.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Measurement infrastructure for the audio-quality campaign (MMX5 complaint),
ported from snesrecomp's audio_trace/oracle methodology and the
_audio_round2 tap model. No behavior change to the audio path itself.
- audio_trace.{c,h}: three always-on 95 s PCM rings — T1 spu_out (canonical
spu_render output), T2 cd_in (XA/CD-DA PCM entering the SPU CD bus),
T3 host (exact post-fade bytes handed to SDL_QueueAudio) — plus a 512k
event ring (every SPU reg write stamped with the SPU sample clock;
RENDER/PUMP_SKIP/UNDERRUN/MUTE/UNMUTE/CD_PUSH/SPU-DMA), audible-vs-
silent counters (-42 dBFS gate), queue watermarks, and a server-side
44100 Hz WAV dumper (rate exactness matters: snesrecomp da33c06).
- psx-beetle: the libretro audio callbacks stop discarding Beetle's fully-
mixed reference SPU PCM and record it into the same tap ring. Port 4380
gains audio_stats/audio_wav/audio_events with the identical wire protocol
as the runtime (rule 16 symmetric harness) — the oracle WAV capture.
- debug_server: audio_stats / audio_wav / audio_events on the runtime port.
- tools: audio_ab_diff.py + audio_spectral_ab.py (drift-tolerant offline
A/B: drift ppm, onsets, cents, timbre, click rate) and audio_analyze.py
(first-dirty-tap cross-tap decision tree) ported from snesrecomp/
_audio_round2; audio_capture_ab.py dumps both processes symmetrically.
Probes query history; nothing is armed at probe time.
Detect a side-effect-free poll loop by its repeated same-PC interrupt
checks and, when the window touched no MMIO and issued no stores,
fast-forward guest time to the next internal device event
(vblank/timer/cdrom/dma/sio) in whole loop quanta — bit-exact versus
executing the spins one by one. Off switch: PSX_IDLE_SKIP=0 env or the
idle_skip TCP command; gated off in exception context, precise mode, and
lockstep.
memory.c adds the two always-on gates the skip requires: g_guest_store_count
(bumped at every raw store chokepoint) and g_mmio_access_count (bumped on
every device-page read/write). Any store or MMIO touch in the window
disqualifies it as a pure poll loop.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Games that discard their boot/init code load gameplay overlays OVER the
main-EXE text window (Tomba 2: hot code at 0x1DA88/0x24548/0x3116C, below
its 0x38800 floor). Post-baseline, a DIRTY page in [KERNEL_WINDOW_END,
FLOOR) is by definition live overlay code the compiled image is stale for,
so make overlay_cache_window_contains() admit dirty boot-text pages as a
capture/candidate window instead of forcing ~300ns/insn interpretation.
Auto-capture convergence: replace the fixed trigger gate with
insn-pressure sizing and exponential futility backoff — retry only when
the manifest signature or loader candidate count actually changes, so
coverage stops churning once it has converged. compile_overlays.py moves
to entry-based coverage; autocapture.c escapes >=0x80 cp1252 bytes from
child-compile output so the status JSON stays valid UTF-8.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Ape Escape "Checking MEMORY CARD" thread-switch smear: a genuine in-exception
ChangeThread detected deep in nested host dispatch commits a poisoned TCB context
(stale resume-PC latch ahead of the live GPRs) -> the resumed consumer faults on
a smeared jumptable index (DISPATCH FATAL).
Fix#2 (user-agreed architecture): don't honor the switch at the poisoned nested
point. Resume the outgoing thread transparently at its OWN EPC, record the switch
as pending, and honor it at the next block leader where that thread is running
with a materialized resume PC -- re-saving it cleanly there. Do NOT require the
outermost STATIC dispatch boundary: the memcard consumer lives permanently inside
the nested dirty-RAM interpreter and never reaches it; the scheduler landing
already resets the nested dispatch/call-unit counters on the longjmp, same as the
immediate-switch path.
- interrupts.c: the kind-30 escape splits into immediate (outermost / toggle-off
/ no clean resume PC) vs deferred; new honor hook at the top of
psx_check_interrupts; always-on thread_trace kinds 31 (defer) / 32 (honor) /
33 (stale-drop). The deferred branch restores the outgoing thread's clean
GPRs/HI/LO/SR AND cpu->pc (the in-exception RFE had left cpu->pc at the target).
- traps.c: psx_sched_save_context / set_current_tcb / current_tcb exports over
the scheduler's private TCB accessors.
- A/B toggle PSX_DEFER_SWITCH (default ON); =0 forces baseline immediate switch.
Validated (LLE + HLE, ring-verified): converts the smear DISPATCH-FATAL into
forward progress -- the card driver now reads 108 blocks. A/B (PSX_DEFER_SWITCH=0)
proves the fix is SOUND: the remaining stall (SIO IRQ bit 7 masked in I_MASK, card
read never completing) reproduces IDENTICALLY with the fix off, so it is a
SEPARATE, pre-existing SIO-mask bug -- the next blocker, not this fix's residual.
Goal (file-select showing File 1-4) NOT yet reached.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- wide_blit_center now copies the canonical column into the wide-surface CENTRE
over the FULL VRAM height, not just [disp_y, disp_h]. Ape vertical-double-
buffers (display_y 0<->256, both bands in one wide surface); a scrolling
cutscene (the cityscape intro) exposed centre rows outside the display band,
which the disp_h-only blit left black -> horizontal black-band tearing. Full-
height blit makes the wide centre bit-identical to the full-mirror output for
every band. Fixes the regression; keeps the perf win.
- sticky gte_game_mode threshold/hysteresis retained (cutscene flicker fix).
- New GL wide-surface inspection over TCP: wide_dump_full backend method
(glb_wide_dump_full) so the `wide_full` command dumps the ENTIRE GL compositor
surface (both double-buffer bands + margins), matching the SW path. wide_shot
(displayed band) already worked on GL. Lets native-wide be debugged entirely
over the TCP server, no window focus.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds full 16:9/21:9 native-wide support for fully-3D titles (Ape Escape,
SCUS-94423) as a generic, opt-in, 4:3-byte-identical framework capability, plus
a native-wide GL compositor perf fix and a cutscene present-mode flicker fix.
Widescreen (generic, per-game opt-in via [widescreen*]):
- Shared screen-extent cull detector (runtime/include/ws_cull_detect.h) used by
all three backends (recompiler, dirty-RAM interp, sljit). Per-game signature
immediates ([widescreen.cull] screen_w_imms/screen_h_imms) — Ape's funnel
tests 0x181/0xF1 on its 368px display, not Tomba's 0x140/0xE0.
- Three funnel idioms widened: unsigned per-vertex sltiu (psx_ws_cull_sltiu),
signed min/max slti right edge + bare-bltz left edge (psx_ws_cull_slti/bltz,
new helpers + SLJIT_HLP appends), and billboard center+/-halfwidth. Explicit
[widescreen.cull] slti_sites for X-only funnels the auto-detector can't
qualify.
- gte_game_mode: GTE-activity gameplay detector for 3D titles with no sprite-tag
helper (gte.cpp notes every RTPS/RTPT). STICKY (>=3 verts, 45-frame hysteresis)
so sparse cutscene frames never flip to a 4:3 pillarbox = fixes intro flicker.
- nw_hud_corners: push outer-third screen-space HUD sprites to the true wide
corners in native-wide (else inset by the reveal).
- nw_backdrop: stretch a full-frame axis-aligned 2D backdrop quad to fill the
wide frame (for titles with a flat sky plane).
- Pattern-scanned interp/sljit widen hooks now gated per-game (auto_screen_x /
auto_backdrop): fixes an ungated auto_backdrop false-positive that rewrote a
live GPR = the ~24s native-wide crash on Ape.
Native-wide GL perf (fixes 16:9 frame drops):
- The wide surface centre equals the canonical 4:3 framebuffer, so wide_blit_center
copies canonical->wide-centre at present (both present paths) and the per-prim
mirror SKIPS prims fully inside the 4:3 frame (mirror_x/geo/batch_center_only).
Only reveal-margin prims re-rasterize. Peak frame time 38.8ms -> 17.5ms, no
vsync misses. Toggle gl_wide_fast; 4:3 untouched (all gated on g_wide_cur).
Runtime-only except the recompiler cull emit (regen needed for cull widening).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Master's text-divergence guard (dirty_ram_text_native_ok) blocked native dispatch
of any compiled function whose text page is runtime-dirty (an overlay overwrote
it), falling back to interp and — when the entry can't be resolved there — to
psx_unknown_dispatch (FATAL). Tsumu's translation layer patches string/glyph DATA
tables that share a 4 KB page (and the guard's 256-byte compare window) with real
compiled functions, so a data patch diverged the page and fatally blocked
func_80070248 on boot ("unknown dispatch: 0x80070248").
Two-part faithful fix (general, not per-game):
- dirty_ram_text_native_ok now decides on the dirty/flagged path by BYTE-COMPARING
the entry window against the reference image, not the coarse page-dirty
heuristic. A data-only change no longer blocks a function whose own code bytes
are unchanged; a genuine code overwrite still diverges -> interp (guard intent
preserved). Pristine pages keep the no-compare fast path.
- New dirty_ram_text_bless(phys,bytes,len): text_xlate blesses every in-place RAM
patch (glyph/msg_inplace/msg_sep) into the (runtime-owned, mutable) reference
image, so intentional data patches read as data. Genuine game self-mod writing
DIFFERENT bytes still diverges and is still caught.
Verified: Tsumu boots to the title screen with the English translation applied,
no fatal. Benefits every title (more precise guard).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a general, opt-in language selector to the RmlUi launcher, driven entirely
by game.toml [localization]:
[localization]
default = "en"
languages = [ {code="en", label="English"}, {code="off", label="Default"} ]
- config_loader: parse [localization].default + languages[] into RuntimeConfig;
add UserSettings.language with settings.toml [localization] parse + write.
- launcher: a "LOCALIZATION" panel in the Settings view (cycle toggle, matching
the Renderer/Screen-model idiom) shown only when the game declares languages.
The chosen code persists to settings.toml and rides UserSettings.
- text_xlate: new text_xlate_set_language() re-applies the language after the
launcher (init runs before it); "off"/"jp"/"" disable APPLY (native game).
- main: seed/read-back the language through the launcher and re-apply on every
boot path (so a settings.toml choice sticks even when the launcher is skipped).
- Settings view laid out in two columns (VIDEO | LOCALIZATION+AUDIO+CONTROLLER)
with SYSTEM full-width below, so it uses the horizontal space and fits without
scrolling; #root gains overflow-y:auto as a safety net.
Only games that declare [localization] get the menu (Tsumu today); byte-identical
for every other title.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two interlocking pieces from the Tsumu HUD/menu-label investigation:
- GP0 ring builder attribution (gpu.h/gpu.c/debug_server.c): op-0x80
entries now carry a bounded guest-stack unwind (bld[6]) + copy-time
$sp (csp); gpu_frame_dump emits func/ra per entry and csp+bld[] for
copies. This cracked the HUD compose: whole label strips are
MoveImaged from the glyph sheet (src 834/852,352 -> HUD cache y=32).
- [[vram_patch]] (text_xlate.cpp/.h + gpu.c hook): third translation
layer for text that exists only as pre-rendered pixels in an
uploaded texture asset (Tsumu HUD labels, menus, prompts live in
the glyph-sheet image; the 144-glyph font has no katakana at all).
On every GP0 0xA0 upload completion the configured rects are
verified halfword-for-halfword against expected JP pixels and only
then rewritten with the target-language pixels via the renderer
facade (software/GL/supersample coherent). Re-applies on every
re-upload; verify-fail leaves foreign assets untouched. Debug:
xlate stats gains vram_patches/vram_applied; new "vpatch"
force-apply subcommand for config iteration without rebuild.
Verified on-screen: HUD "STAGE: 1-1 / BATTERY: 100", pause menu
(CONTINUE/RESTART/GIVE UP/CHANGE CONTROLS, TARGET BATT), title-menu
strips. docs/STRING_TRANSLATION.md Appendix Z documents the model,
mechanism, authoring recipe, and remaining strip backlog.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds a third translation path to the framework for UI text drawn
glyph-by-glyph as sprites from a fixed-stride, space-padded EXE table
(Tsumu's 59-slot level/stage-name array at 0x80070300..0x800708FE).
Such labels carry no string pointer and no terminator, so the existing
dispatch/arg-scan message hook can never reach them.
Instead, [[glyph_label]] TOML entries drive a load-time RAM source patch:
once the EXE label region is resident, each confirmed slot is verified
byte-for-byte against its expected JP source bytes, then overwritten with
the target-language text (fullwidth-Latin Shift-JIS, little-endian)
re-padded with fullwidth spaces to exactly the slot width. The game's own
per-glyph routine then draws English. Verify-before-patch means a slot is
never touched unless the exact JP label is present; writing exactly
`width` bytes (== the table stride) means a patch never spills into the
adjacent slot.
- label_transcode_le(): UTF-8 -> fixed-width LE fullwidth-SJIS, truncated
to whole cells, fw-space padded; never exceeds width.
- glyph_labels_patch_locked(): idempotent verify-and-patch scan, run from
the dispatch hook while any label is unpatched (throttled; a single
relaxed load once g_glyph_pending hits 0).
- Loaded/gated exactly like message entries (language column, PSX_LANG,
apply-armed). "xlate glyph" debug subcommand reports per-slot status.
Verified (build-dbg, headless): all 59 slots patched (glyph_patched=59,
pending=0), RAM byte-exact English at TOWER/SANDBOX/JELLYFISH/LOOP THE
WARP, game alive at 26M dispatches. Framework code only; no regen, works
in Release. Table + user visual sign-off in TsumuRecomp/translations.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
RmlUi launcher grows a Settings surface (renderer / supersampling / AA /
texture filter / screen model / skip-FMV / turbo / fullscreen / widescreen /
SPU-HQ / deadzone / BIOS / disc) and a live Controls rebind page. Bindings
persist to keybinds.ini through the new shared runtime module
runtime/include/psx_keybinds.h + runtime/src/psx_keybinds.c; main.cpp routes
SDL input through the rebindable table. Ape Escape is the first consumer;
the feature is game-agnostic framework code.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Implements the framework from docs/STRING_TRANSLATION.md: an always-on
capture inventory + a draw-time apply hook, both hung off the psx_dispatch
chokepoint via fntrace_record (no generated-code edits, no BIOS regen,
works in Release).
- text_xlate.{h,cpp}: EncodingProfile abstraction (shift_jis profile:
SJIS record reader with NUL/0xFFFF terminators + halfwidth-kana + control
framing; UTF-8 -> fullwidth-SJIS transcoder so English renders with the
font's existing fullwidth Latin, zero glyph injection). Capture inventory
(FNV-1a64 keyed) queryable over TCP; apply repoints the string-arg reg at
guest-stack scratch (KV-gated: only known records are swapped, stray
pointers untouched; own scratch is skipped to avoid recapture).
- fntrace.c: call text_xlate_on_dispatch() at the dispatch chokepoint.
- config_loader: [localization].language (+ [runtime].language alias);
env PSX_LANG overrides. Loads translations/*.toml per project.
- main.cpp: text_xlate_init() at game-config load.
- debug_server: "xlate" command (stats|dump|todo|reload) — observability
over TCP, no log files.
Verified: builds (Release), Tsumu boots headless, capture live
(distinct records accumulating), apply disabled until a table is present.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
debug_server_record_frame (every vblank) and the sio_state handler read
SIO_STAT/CTRL/RX via the GUEST-BUS sio_read(), which has hardware
side-effects: sio_tick(0) advances the delayed-IRQ state machine, reading
SIO_STAT decrements sio_ack_visible_reads and clears the ACK bit, reading
SIO_RX_DATA pops the RX FIFO. So the observer silently mutated the SIO/pad
handshake every frame in dev builds (the cardinal observability-alters-state
violation the event_ring header itself warns against).
Add side-effect-free sio_peek_stat/ctrl/rx_data() and use them on the debug
path; the guest bus keeps sio_read() with its real side effects. Also add an
fmv_state TCP command reporting the resolved config path + auto_skip_fmv +
frame-count-table params + live MDEC/XA/pad state, so the FMV-skip + config
state can be observed rather than inferred.
Note: this removes a real per-vblank SIO perturbation but is NOT the full fix
for the dev-build FMV-skip divergence (residual is the cycle-timing-core
class; see precise_irq_slice_state). It is correct observability hygiene.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The committed beetle debug server (h_parity_dump reads=1) and beetle_libretro
read hook already referenced parity_trace_reads_get / parity_trace_reads_total /
parity_trace_note_read, but only a dead per-watch scalar table existed — the two
readback functions were never implemented, so every psx-beetle rebuild failed to
link (blocking the oracle entirely).
Implement the dedicated READ ring as the symmetric counterpart of the write ring:
note_read pushes one ParityEntry per read of a watched word, carrying reader
provenance (pc = load PC, epc = read address, target/watch[slot] = loaded value)
PLUS that slot's current last-WRITER metadata (wpc/wcycle/wframe/wtcb). One read
row therefore answers "who read what value, and who last wrote it." reads_get /
reads_total mirror parity_trace_get / parity_trace_total (newest rows, oldest
first). Reset clears the ring. Removes the unused scalar last-reader table.
Validated: psx-beetle links + boots; PSX_PARITY_TRACE=1 accrues well-formed read
rows (reads_total>0, reader+writer provenance populated).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Packed games can rewrite code over their own loaded EXE text after static recompilation. A dirty-page bit alone is too coarse for dispatch decisions: it prevents stale native execution, but it also cannot distinguish data writes in a text page from instruction bytes that actually diverged from the compiled image.\n\nRegister the boot EXE image as a reference and let memory writes mark text pages whose bytes differ from that image. Static game dispatch now asks whether the target remains native-safe; diverged pages fall through to execute live RAM through the dirty interpreter, while matching code bytes can still use the compiled path.\n\nConstraint: Runtime-installed or rewritten RAM code must execute the program's own instructions, not HLE or stale static code\nRejected: Keep the existing dirty-page-only gate | too coarse for packed/self-modifying main text and data writes sharing code pages\nConfidence: medium\nScope-risk: moderate\nDirective: Do not bypass dirty_ram_text_native_ok() at game compiled dispatch sites without proving RAM still matches the compiled image\nTested: Local Tekken 3 branch using this mechanism reaches playable fights; clean upstream worktree compiles changed main/memory/dirty files before hitting an unrelated existing macOS overlay_loader LONG/InterlockedExchange build error\nNot-tested: Full upstream macOS build completion; non-macOS build matrix
Tomba2 16:9 GL reproduced the reported collapse (12-14fps) and the GPU-side
split EXONERATED the GPU: scene_gpu ~10ms, mirror ~0.2ms, while emu_cpu is
~82ms (vs ~15ms in squash). gl_ws_ablate=1 (mirror passes fully off) leaves
it at ~75ms, so the wedge is CPU work outside the mirror draws. The GL
submission paths run INSIDE guest emulation (GP0/DMA processing), so driver
CPU cost lands in emu_cpu where no GPU timer sees it.
frame_perf now reports, per wide/4:3 frame class: CPU wall inside
flush_tex_batch (cpu_flush_ms_avg), CPU wall inside the glb_wide_* entry
points (cpu_wide_ms_avg), batches/frame, wide target sets/frame, and wide
FBO creations/frame (a nonzero steady value = surface recreation churn).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Native-wide perf attribution instrumentation (permanent):
- Each wide-mirror pass (tex-batch mirror, flat-geometry mirror, overlay
full-width rect, wide_clear) is bracketed with a GL_TIMESTAMP pair
(glQueryCounter; TIME_ELAPSED cannot nest inside the frame scene query).
Pairs pool per buffered frame (1024/frame), summed at readback into
mirror_gpu_ms + mirror_passes per sample; frame_perf now reports
mirror_gpu_ms_avg/max, canon_gpu_ms_avg (scene minus mirror),
mirror_passes_avg and mirror_pass_us for wide frames.
- New debug cmd gl_ws_ablate mode=<0..3>: 0 normal, 1 skip mirror passes
entirely, 2 mirror state churn without draws, 3 mirror draws stay on the
hr FBO (no per-pass FBO rebind; content-corrupting perf probe).
First MMX6 16:9 measurements with the split: mirror passes cost 20-125us
EACH (FBO ping-pong penalty per pass) but MMX6 2D batches to ~3-15
passes/frame so 16:9 attract holds ~60fps on this branch; the per-pass
cost x ~900 immediate flat prims is exactly the Tomba2 3D 60ms scene.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Root cause (proven via the new present ring + gl_coh_ring correlation):
flush_cpu_upload() merged all pending CPU->VRAM writes into ONE union
bounding box. A frame with two disjoint uploads (sprite column + tile row —
routine MMX6 texture streaming) produced a union spanning the display
framebuffers, which the flush painted from the STALE CPU VRAM mirror (the
FBO is authoritative under GL) — stomping live frames with black for
exactly two presents (one per double-buffer parity). Scene-dependence of
the uploads made it look config/timing dependent; the software renderer is
immune because its CPU array is authoritative.
Fix: exact pending-rect list (16 rects; merges only when zero uncovered
pixels are added — containment or same-band extension; wrap-aware GP0(A0)
transfers split into up to 4 exact rects instead of the old "wrapped: take
all" full-VRAM union; overflow -> order-preserving flush-all; pre-context
union retained where the CPU array is authoritative). Merge rule verified
by a 20k-randomized-rect host test (0 stale / 0 missing painted pixels).
Observability (always-on): gl_present_ring records every SwapWindow (path,
src/letterbox rects, drained glGetError, wall-ms, backbuffer + blit-source
pixel samples) — the instrument that gave the 1:1 black-capture
correlation. debug_server_send_fmt now heap-formats responses >64KB (they
previously truncated into unparseable JSON, breaking large ring dumps).
(debug_server.c also carries the Vulkan screenshot_file sync hunk from the
concurrent VK bring-up; see the next commit.)
Validated: ~18-min MMX6 GL attract soak, ~1600 window captures across 3+
attract cycles — zero isolated black frames, zero GL errors (pre-fix:
19/40 captures black at the intro). Tomba1 build-gl boots clean; its full
attract soak follows. ENHANCEMENTS.md added (framework enhancement ledger,
R1 OpenGL / R2 Vulkan state).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Tomba2's Trolley attract demo interprets a working set past 64K distinct
PCs. The per-PC tracking tables (g_dirty_ram_pc_table / exec_pc_table) used
an UNBOUNDED linear probe, so once a table filled, every lookup for an
absent PC scanned all 65536 slots before returning NULL — per executed
instruction. Host thread profile showed ~99% of emu-thread time inside
pc_table_get_or_insert_in; guest crawled at 0.3 fps (1M guest cycles per
3s wall) while telemetry showed the window "LIVE" on residual motion.
Fix: hard-bound the probe to 128 slots (miss => NULL => "stop tracking",
the documented degradation) and grow the tables 65536 -> 262144 so the
Trolley working set actually fits — the same table feeds overlay-capture
seed lists (MAX_CAPTURE_PCS aliases it), so the old size also silently
truncated capture manifests for large regions.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The emitter routes mfc2/cfc2/mtc2/ctc2 on flag/IR/derived GTE registers
through direct gte_read_data/gte_read_ctrl/gte_write_data/gte_write_ctrl
calls (code_generator.cpp COP2 case), but the overlay DLL preamble had no
shims for them — any GTE-heavy overlay failed to LINK (undefined
gte_read_ctrl) and its region ran interpreted forever. That silently
blocked all new native coverage for the lava/Trolley attract-demo regions
after the caches rotated.
GTE state is shared runtime state, so the shims forward unconditionally
through OverlayCallbacks (a local copy would fork the flag/IR timeline).
ABI bumped 9 -> 10; the loader gate rejects+deletes pre-shim DLLs and
autocompile self-heals; plain gte_data[]/gte_ctrl[] accesses stay direct
CPUState reads in emitted code, matching the emitter.
Smoke-validated: previously-failing GTE-heavy captures now compile clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Root cause of MMX6 "A game data could not be found" (weeks-old, cross-backend):
the game's save-scan calls memset through the kernel A0 jump table; entry
A0:2B points into BIOS ROM at 0xBFC02B8C (one of the entries the kernel never
patches into RAM). Transitive discovery can never find it (reachable only via
the data table, no jal), so the dispatch missed and psx_unknown_dispatch
silently no-op'd — killing the scan before firstfile ever ran. Every card
layer (SIO, IRQs, SwCARD events, sector data, kernel directory parse) was
verified correct along the way; the sole defect was the undiscovered ROM
function plus the silent miss.
- recompiler/seeds: enumerate the post-init A0/B0/C0 tables (kernel RAM
0x200/0x874/0x674) and seed every ROM-pointing entry with no dispatch
coverage — 30 functions (memset, memcmp, memmove, strncat, rand-family,
card/dev services...). memset was merely the one MMX6 called; the rest were
identical latent bugs. BIOS regen: 4406 -> 4439 dispatch entries.
MMX6 CONTINUE now lists and loads saves (user-validated + scripted run:
full FILE SELECT, 1093 card txns, unknown_dispatch_log total=0).
- traps.c: an unresolvable dispatch target now dumps a crash report and halts
BY DEFAULT (PSX_FAIL_FAST_UNKNOWN_DISPATCH=0 opts out for diagnostic
sessions; the miss is ring-recorded either way). The old silent-no-op
default left stale registers and a severed call chain, and masked this bug
for weeks. Deliberately NO interp fallback: the dirty-RAM interpreter is
for runtime-written code (Rule 18); interpreting around a static-discovery
hole would hide it exactly like the no-op did.
- runtime.cmake: EXE_NAME option — per-game process names (mmx6-runtime.exe)
so one game's `taskkill /IM psx-runtime.exe` dev sweep no longer kills
every other game's instance.
- debug rings (the tooling that found this): HLE call ring gains
consecutive-duplicate collapse w/ repeat counts (a 4-way TestEvent poll at
175K calls/s no longer flushes the ring in 0.1s), per-entry TCB + in_exc
(thread attribution), a "since" paging param and 8MB dump buffer;
card-trace EvCB watch widened to all 7 card event slots (slot 0 =
SwCARD-IOE, the success event, was previously invisible).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
USER PIVOT 2026-07-02 (CLAUDE.md §0 amendment included): psxrecomp now
carries an opt-in High-Level Emulation backend alongside LLE. LLE (the
recompiled BIOS) stays the default, the reference implementation, and the
correctness/timing oracle; every service HLE does not implement falls
through to LLE, so HLE is never load-bearing beyond what it covers.
Mechanism: full_function_emitter.cpp emits a null-by-default hook
g_psx_bios_hle_hook consulted at the top of every psx_dispatch_impl
iteration, on the pre-normalize physical address, BEFORE any backend
(game image / dirty-RAM / static table) claims the target. A handled
call resumes at $ra through the trampoline's existing return/tail
contract. NULL (default) = pure LLE, dispatch-identical. Regenerating
BIOS+game images is required (emitter change).
v1 call-HLE (runtime/src/bios_hle.c): the B0 event family — DeliverEvent
(incl. callback dispatch with the kernel-true $ra 0x1720), OpenEvent,
CloseEvent, TestEvent, EnableEvent, DisableEvent — each ground-truthed
instruction-by-instruction against the SCPH1001 kernel disassembly
(0xBFC11644..0xBFC11A84) and operating on the REAL guest EvCB table
([0x120]/[0x124], stride 0x1C). WaitEvent/threads/pads/card/A0/C0 fall
through to LLE. Cycle charges are documented approximations; LLE remains
the timing oracle. No-stubs stands.
Boot HLE: with the tier on (and no keep-intro opt-out) the hook one-shot
intercepts the first dispatch of the BIOS shell entry (RAM 0x30000 —
LoadRunShell's indirect call always routes through the dispatcher), so
BIOS Main() proceeds straight to SYSTEM.CNF + game EXE load with kernel
state built entirely by the real recompiled kernel init, unpaced (boot
turbo) until game handoff. This REPLACES the fast_boot snapshot
capture/restore; fast_boot=true now aliases the boot-skip alone.
Selection: [runtime] bios_hle / bios_hle_keep_intro (+ settings.toml
bios_hle mirror), PSX_BIOS_HLE / PSX_BIOS_HLE_KEEP_INTRO env overrides,
startup banner bios_backend=/bios_boot=. The PSX_HLE_SCHEDULER spike is
folded in as the tier's standing subsystem replacement: default from
[runtime] hle_scheduler via psx_hle_scheduler_set_default, env wins.
Observability: always-on 16K-entry HLE ring (route LLE-fallthrough /
HLE / boot-skip per record) + hle_dump TCP command (TCP_COMMANDS.md).
VALIDATED on Tomba (build-wt): LLE pass byte-identical behavior (real
intro, card check, save/load OK); HLE pass boots shell-less, TestEvent/
DeliverEvent route HLE, ChangeTh/ReturnFromException/OpenTh/CloseTh and
the card stack fall through to LLE, save/load OK (user-driven).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
GetID's last four response bytes were hardcoded 'SCEI' (NTSC-J). Real
hardware reports the region of the INSERTED disc. Kula World uses the
BIOS CD API, and the kernel CD driver revalidates the disc (ReadTOC +
GetID, polled with irq_enable=0x18) when it reinitializes after the
game's first file load. Seeing a 'Japanese' disc, it fell into the
endless GetStat/Init retry loop: CD delivery froze at sector 307 (LBA
318, just before the first level directory) and the game waited forever
on a black screen.
Diagnosed from the always-on cdrom trace ring (grabbed via gdb from the
live process): 307 sectors stream cleanly, Pause completes, then the
BIOS's own ROM CD firmware (0xBFC0Dxxx) masks INTC delivery, issues
ReadTOC + GetID, reads the response... and rejects it. Beetle PSX (the
oracle; boots the same dump + BIOS to demo gameplay) derives the SCEx
string from the disc's SYSTEM.CNF boot serial - so we now do the same:
cdrom_set_disc_scex() is fed at launch from the existing disc_identity
module (PAL->SCEE, NTSC-J->SCEI, NTSC-U->SCEA; default SCEA matches the
one supported BIOS). Verified so far: launch log reports 'disc region
PAL (serial SCES-01000)'; the sector-counter wall-break run is in
flight. Framework-wide note: Tomba/MMX6 (NTSC-U, own CD drivers) never
hit this path but now get 'SCEA' - matching real hardware - instead of
'SCEI'; regression-test before any master merge.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdjxoRB1f9VQsNzZayCgGS
The cycle-accuracy emitter emits calls to 12 timing runtime functions
(psx_cyc_load_*, psx_icache_fetch, psx_muldiv_*, psx_gte_*, psx_slice_block,
...) + debug_server_cyc_observe + g_debug_last_store_pc. Overlay DLLs never
got these symbols, so every overlay failed to link -> 0 native shards ->
100% interpreter (unplayable). Fix (option A): expose the 12 via the overlay
callback ABI.
- overlay_api.h: +12 timing callbacks in OverlayCallbacks, ABI 8 -> 9.
- overlay_loader.c: populate the 12; add warn_on_cgtag_mismatch() (loud
warning when shards exist under the wrong cg-hash).
- compile_overlays.py: 12 preamble forwarders + local g_debug_last_store_pc
+ -DPSX_NO_DEBUG_TOOLS.
- cosim.c/main.cpp: input-cosim setpad + held pad (deterministic input).
- cosim.py: env-overridable COSIM_EXE/CWD/GAME.
KNOWN REGRESSION (patched on a follow-up branch): Tomba pause menu (triangle)
wedges. Root-caused: an IRQ taken mid-block resumes at the function entry
leader instead of the interrupted PC, re-zeroing the menu's VSync wait
counter (scratchpad 0x1F8001E8) every frame so it never reaches its target.
Not the scheduler (LLE reproduces) and not timing math -- interrupt-resume
re-entry.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New always-on ring (device_trace.{c,h}, compiles into both binaries like
parity_trace) recording every hardware IRQ-raise edge with its guest cycle,
source (I_STAT bit) and detail. All native device sources funnel their raise
through one choke point psx_irq_raise(bit,detail) in interrupts.c (VBLANK/GPU/
CDROM/DMA/timers/SIO routed; DMA passes DICR channel flags, timers pass index),
so the device-timing audit sees every event from one place. devtrace_dump/ctl
debug commands (cycle-window + source filters) + tools/devtrace_diff.py align
the two device-event timelines by source ordinal and flag where cycles drift.
The Beetle oracle side (mednafen IRQ_Assert rising-edge hook) lives in the main
checkout.
This is the level below parity_trace: parity names the divergent control-flow /
gate input; device_trace names the divergent HARDWARE EVENT behind it.
First result (MMX6 cutscene asset-load over-fast wedge): the load's CD+DMA
activity is compressed into ~232M cycles on native vs ~384M on Beetle (~1.65x),
completing the func_8001CB3C scene setup ~100M cycles / ~176 frames early. The
producer is CD-read / DMA-completion timing (DMA channels 2/3/4 = GPU/CD/SPU
asset streaming) -- confirming the device-completion-latency hypothesis and
ruling in the CDROM/DMA cycle model as the over-fast subsystem. (VBlank verified
correct: native VBlank edges are exactly 564480 cyc apart.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extends the two-process parity ring so each row carries, per watch word, the
exact store PC + guest cycle + frame + (approx) writing TCB of the most recent
write to that word. parity_trace_note_write() maintains the table from BOTH
processes' write paths (native: memory.c psx_write_* with g_debug_last_store_pc;
Beetle: beetle_libretro.cpp wtrace_callback with the store pc). Both parity_dump
serializers emit the new wwpc/wwcy/wwf/wwt arrays (identical wire format), and
parity_diff.py prints a side-by-side WATCH PROVENANCE block at the divergence.
This turns "the gate-input value diverged" into "who/when produced it" -- the
instrument ChatGPT recommended for the MMX6 over-fast wedge.
First result (MMX6 cutscene->gameplay, watch 0x80071088/0x800CD3F8/0x8006D9AC/
0x800CD3FC/0x800CD404/0x800200F4): the cutscene-setup func_8001CB3C (writer pc
0x8001CB7C of countdown 0x800CD3FC) runs ~99.7M guest cycles / ~176 frames
EARLIER on native (cyc 998,597,937 frame 1769) than Beetle (cyc 1,098,296,890
frame 1942) -- same writer PC, fired early. On Beetle the state struct
0x800CD3F8 is never touched by game code (stays BIOS-init 0 -> setter path); on
native it advances. The over-fast wait is localized one level above the state
machine, at whatever schedules func_8001CB3C.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add absolute guest-CPU-cycle (since boot) to every parity row via a per-process
parity_host_cycle() accessor (native psx_get_cycle_count; Beetle
beetle_core_get_guest_cycles — directly comparable). Widen ParityEntry.cycle to
64-bit; both dump handlers emit it. This turns the cutscene divergence from "273
frames early" into a hard cycle delta: Beetle runs 566k cycles/frame (≈ real PSX
VBlank 564480); native reaches the cutscene state-advance ~150-200M guest-cycles
early — an over-fast cycle deficit (DMA-cycle-steal signature) in the faithful-
timing stack. Beetle producer side mirrored in the main checkout.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
End the "somewhere upstream" blindness with a reusable first-divergence
instrument (PRINCIPLES.md "find the first divergence; is the value WRONG or the
behavior MISSING"). A TCB-scoped, freeze-on-trigger control-flow ring with an
identical struct + JSON wire format on both processes, so a generic sequence-diff
(tools/parity_diff.py) locates the first logical divergence between recomp and
oracle — not a frame diff.
- parity_trace.{h,c}: armed-from-boot ring, gated on current_tcb==watched_tcb,
latches on a configured trigger target so the pre-divergence window survives
the wedge loop. Up to 6 watch-word snapshots/row. Self-contained TU shared by
both binaries; frame source decoupled via parity_host_frame().
- fntrace.c: per-dispatch producer hook (cheap armed gate on the hot path).
- traps.c: YIELD control-event hook in the scheduler switch.
- main.cpp: env-armed from boot (PSX_PARITY_TRACE=1; TCB/trigger/watch tunable),
defaults target the MMX6 cutscene wedge.
- debug_server.c: parity_dump + parity_ctl commands.
- tools/parity_diff.py: pulls both rings, sequence-diffs by (kind,pc,target).
First use (MMX6 freeze) confirmed MISSING behavior, not a wrong value: native
thread1 is permanently parked at func_8002000C resume (TCB EPC 0x800200F4),
handshake flag 0x8006D9AC never set (0 across 16384 rows), setter func_8001344C
never runs; thread1 re-resumes a dead func_8002000C frame whose saved-ra slot was
reused -> jr 0x800CD3F8 wedge. Upstream context-staleness/scheduling root.
Beetle mirror producer + the cross-process diff are the next step.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the non-deterministic host-fiber thread-switch bridge with a deterministic
TCB scheduler (HLE_SCHEDULER_CARVEOUT_PLAN.md steps 2-8, minimal "A+" scope):
- psx_scheduler_run (traps.c): outer trampoline wrapping the top-level dispatch
(main.cpp calls it instead of bare psx_dispatch). setjmp(g_scheduler_jmpbuf);
restore current TCB / dispatch / on-escape reset g_psx_dispatch_depth +
g_psx_call_bail; boot/pre-TCB runs raw cpu->pc; one-level switch-back safety
net; fail-closed psx_is_dispatchable on every restored EPC.
- psx_request_thread_switch: save yielder ctx (resume=$ra), repoint
dword_108->entry, longjmp to the scheduler. No fibers, no cpu->pc=0 signal.
Fires only with in_exception==0, so it never unwinds a live exception_jmpbuf
frame (the in-exception RFE path stays LLE/intact).
- sched_escape_ring (step 8) for always-on observability.
- Hidden toggle PSX_HLE_SCHEDULER (default 1=HLE, 0=legacy fibers), read once;
psx_change_thread branches on it; startup banner prints the mode. The
trampoline is transparent in LLE mode. Fiber funcs kept compiled-but-dead.
Validated live (HLE): thread1 trace shows ONLY the new TCB-switch events and
ZERO fiber kinds 10/11/13 — the per-frame fiber recreate is eliminated. Builds
clean. Freeze NOT resolved: same 0x800CD3F8 wedge persists (separate upstream
root, per ChatGPT Q5). Keep HLE default until the upstream cause is fixed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Matthew Stanley