Tomba 16:9: rooms, sky falls, and task-card sequences presented stretched
because native-wide's 2D-backdrop stretch fires wholesale in scenes whose
background phase never ends. Such scenes now present native 4:3 from the
faithful canonical buffer, keyed on DEEP UNTAGGED POLYGON OVERHANG (>=24px
past the canonical window, >=4 polys, sustained 2 frames) — the only world
signal that survived adversarial scenes (shaded prims, GTE volume, any-poly
and shallow overhang all falsified by census evidence; lineage in gpu.c).
Scoped to sprite-tag titles; full_2d (MMX6) and gte_game_mode (Ape) paths
unchanged. Plus: native-wide never stretches a canonical-width game frame
(fallback pillarboxes); always-on present-classification ring (TCP
present_ring) across all backends; debug input injection drives the full
pad model (hybrid d-pad drop + stick axes); savestate TCP honesty; census
tagged column + ws_hud_mode A/B (HUD-corners deferred: ENHANCEMENTS.md W2).
USER-VALIDATED on Tomba (SCUS-94236).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Tomba nw_hud_corners experiment surfaced two blockers, both documented
in ENHANCEMENTS.md W2 with census evidence: composite dialog boxes spanning
zone boundaries tear (end caps pulled to opposite edges), and the AP counter
renders through the TAGGED sprite funnel so the untagged-only gate skips it.
Groundwork kept for the next attempt:
- WsCensusEntry gains a `tagged` column (ws_census CSV) so funnel membership
is measurable per prim.
- TCP ws_hud_mode {"tag_rects":0|1} live-A/Bs the tagged-rect gate.
Tomba's game.toml reverts nw_hud_corners to false (its own repo).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2D-only-scene classifier (fixes room/fall/title-card stretch, Tomba):
- Native-wide's 2D-backdrop stretch fires wholesale in scenes whose
background phase never ends (room interiors, sky-only falls), stretching
the whole frame. Such scenes now present native 4:3 from the faithful
canonical buffer (gpu_ws_present_native_43).
- The world signal is DEEP UNTAGGED POLYGON OVERHANG: polys whose raw SX
extent crosses >=24px past the canonical window, >=4 per frame, sustained
2+ consecutive frames. Lineage of rejected signals (each defeated by a
real scene, census-evidenced): shaded-prim presence (title-card letter
tiles are gouraud), GTE projection volume (task-jingle projects
world-scale counts; title screen idles at 1680 verts), any-poly overhang
(card letters slide in from off-screen), shallow overhang (jingle
sparkles poke <=20px). Outdoor terrain measures 55 deep-overhang
polys/frame vs 0-1 for every observed 2D scene/effect.
- Scoped to sprite-tag titles: full_2d (MMX6) and gte_game_mode (Ape)
classification is unchanged.
Native-wide present invariant:
- A game frame that cannot present wide (compositor fallback) pillarboxes
4:3 instead of stretching the canonical read across the wide window
(all backends; squash mode still stretches by design).
HUD corner re-anchoring for sprite-tag titles ([widescreen] nw_hud_corners):
- Poly/line prims never shift (they are the world / tagged billboards);
rect-family prims shift only when UNTAGGED — the proven hud_sprt_squash
discriminator — so enabling nw_hud_corners is safe without a packet
arena range.
Observability (always-on, institutional):
- present_ring: per-present classification ring (path, present width,
game_mode, native_43, wide fallback, tag age, gte_verts, ovh_prims)
for every backend; TCP "present_ring". Answers "why did this frame
stretch" after the fact.
- gpu_state ws block gains last_3d_frame, gte_verts, last_world3d_frame,
ovh_prims, last_ovh_frame; GTE vert counting is now always-on.
Input injection drives the full pad model:
- set_input/press now resolve pad type like real sampling: injected d-pad
drops a hybrid pad to digital (movement works), optional lx/ly/rx/ry
stick override (0..255), type via the coherent request channel.
Previously buttons-only: menus reacted, walking never did.
Savestate honesty:
- savestate_request_save/load return staged/refused; the TCP handler
reports ok:false instead of silently acking loads it cannot perform
(LLE host-fiber runs). Torn-restore reliability audit remains open.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1) cdrom: continuous disc clock — the read stream froze disc time while
the guest had an unacked INT, starving the XA->SPU ring ~5-6x/s
(6 Hz music dropout flutter). Replaced with the Beetle-mirrored
SetAIP/CheckAIP model: disc never pauses, data-ready INT1 pends one
deep, presented on ack. read_hold_cycles/int1_* tripwires in
cdrom_state.
2) spu: END-without-REPEAT now jumps to the repeat address AND forces
the envelope to zero (Beetle spu.cpp:333,341-352). Previously parked
one-shot SFX voices decoded past the terminator at full envelope —
X4's SFX bank parks them on +28672-DC filler with ~infinite release,
railing the mix at +32767 (38-41% samples clipped = the 'static').
User ear-validated on the MMX4 attract ice-stage demo. Gate: MMX4 0.00%
clip post-fix; MMX5 boots healthy (pend path exercised 12-13x, 0 lost);
Tomba behavior equivalent to master. Runtime-only — no regen needed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
An ADPCM block whose flags carry END (bit 0) but not REPEAT (bit 1)
must (a) jump the decode pointer to the latched repeat address and
(b) force the ADSR envelope to ZERO on entering Release — Beetle
spu.cpp:333 (CurAddr = LoopAddr, unconditional) and :341-352 ("Force
enveloping to 0 if not looping"). Our voice_next_sample did neither:
it entered Release with the envelope intact and kept decoding FORWARD
past the terminator, assuming the release ramp would mask whatever
followed in SPU RAM.
X4's SFX bank breaks that assumption three ways: its one-shot samples
terminate on an END-only block; the next block in RAM is non-silent
filler (0x77 nibbles at shift 0 = constant +28672) carrying a
self-referential LOOPSTART flag, so the walker parks there forever;
and the bank's ADSR release shift is ~infinite, so the envelope never
decays. Every finished SFX voice became a full-scale DC source. In the
X4 attract ice-stage demo (user report), a handful of ice-block-break
SFX stacked ~15 such voices and pinned the whole mix at +32767 for
~35 s — heard as static / the music "cutting out". Measured: 38-41%
of samples hard-clipped in the spu_out AND host taps during the demo.
With the fix (jump + env=0; the env-0 Release path then deactivates
the voice, matching Beetle's silent-voice behavior): 0.00% clipped
samples over the same demo window, peak 32291, zero railed polls in
150 across the full attract cycle; the ice-crack SFX bodies still
play normally.
Found via the new rail-triggered spu_voices capture + the spu_ram
peek: 15 voices sharing start=0x16AD0 all parked at cur_addr=0x17930
looping 0x17920 (flags 0x07) at env 0x7FBF in phase RELEASE, and the
block bytes 00 07 77 77 .. decoding to +28672 DC.
Debug surface added (always-on doctrine):
- spu_voices now reports live env level + ADSR phase per voice.
- new spu_ram command: {"addr","len"} -> hex peek of SPU sample RAM.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The read stream froze read_delay whenever the guest had an unacked
controller INT (and while the sector buffer was undrained). Real drives
never pause: measured on MMX4's attract/title XA music, the freeze
accumulated ~1M guest cycles/second (~3% of disc time), every XA sector
arrived 32-92 samples late, and the XA->SPU cd_ring starved ~147 samples
5-6x per second — audible as a 6 Hz crackle/dropout flutter in all
XA-streamed music (user report: MMX4 Zero ice-stage attract demo).
Faithful model, mirrored from the in-tree Beetle oracle (cdc.cpp
HandleSectorRead / SetAIP / CheckAIP / ClearAIP):
- Disc time NEVER pauses while reading; sectors are read on exact
cycle schedule. XA-ADPCM realtime audio flows to the SPU decoder
unconditionally (this alone removes the crackle).
- A data sector that comes due while the guest still has an unacked
INT overwrites the sector buffer on schedule (hardware clobbers its
FIFO the same way) and its data-ready INT1 pends ONE deep, presented
the moment the guest fully acks (Beetle CheckAIP). A newer sector
replacing an unpresented notification is counted + traced 'P'
(Beetle "Previous notification skipped").
- Drive-state changes (Read/Play/Pause/Stop/Seek paths through
start/stop_read_stream) cancel the pended notification (ClearAIP).
- The active-DMA refill path (deliver without new INT while a
multi-sector CD DMA drains) is unchanged.
Verification on MMX4 (SLUS-00561) title/attract XA:
- CD_PUSH guest-cycle spacing: exactly 1,806,336 cycles (8 sectors at
2x), min == max over 148 pushes (was 1.83-1.88M, systematically late).
- read_hold_cycles == 0 (was ~30.8M per 30 s) — kept as an always-on
regression tripwire in cdrom_state.
- int1_pended == 0 during music: X4 acks fast; the pending path only
arms for genuinely slow acks.
- spu_out/host tap rings: 6 Hz zero-gap storm (86-144 gaps/95 s of
147-265 samples) eliminated; only real content silences remain.
- cd_underflow_frames stays 0 during playback.
Always-on observability added (ring-buffer doctrine):
- XA zero-run scanner in the decode path -> AUDIO_EV_XA_ZERO events
(stage 0 = post-ADPCM, stage 1 = post-resample+volume, with LBA).
- CD_PUSH audio events now stamp the global guest-cycle clock (the
spu_out sample stamp is pump-quantized and cannot measure cadence).
- cdrom_state: read_hold_cycles/read_hold_events (must stay 0),
int1_pended / int1_lost / int1_pending_now.
- Savestate: pending_dataready fields appended to CDROM_SNAP_FIELDS.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The dirty-RAM text-image guard (685f38d) only armed when the loose local
boot-EXE file next to game.toml existed - found via find_project_root's
repo-marker walk. End-user release installs have neither a repo checkout
nor the EXE file (it is copyrighted game data), so the guard silently
never armed. Without it, fntrace's alternate game-start latch
(psx_game_address_in_text && dirty_ram_text_native_ok) cannot fire for
titles whose BIOS handoff enters text inside the shell-window shadow
(RAM 0x30000-0x5AFFF -> shell ROM until game-start): the shadow never
lifts, execution runs dead shell ROM bytes over real game text, and the
boot dies in a jump to PC=0 (Ape Escape v0.0.2/v0.0.3 release zips,
frame 14, both HLE and LLE boots; in-repo runs armed invisibly and
masked the bug during validation).
Fix, class-level for every title:
- main.cpp: defer guard arming until the disc path is resolved and add a
second source - extract the boot EXE from the mounted disc image via
ISOReader (basename of game.toml's exe field, SYSTEM.CNF BOOT token
fallback). The disc is the same bytes the BIOS loads, i.e. the true
reference; every install has one. Local EXE file still wins for dev
checkouts. A loud WARNING prints when neither source arms (previously
silent, which is how the broken releases shipped green).
- fntrace.c: the alternate game-start latch now requires a registered
image (dirty_ram_text_image_registered). Its safety argument - "RAM
still holds shell bytes that differ from the game image, so it cannot
fire prematurely" - only holds when an image exists; with no image,
native_ok degrades to !dirty and untouched text pages pass it,
latching game-start before the EXE loads and clearing the dirty
baseline mid-load. Without an image, only the exact entry_pc dispatch
may latch (the pre-685f38d conservative behavior).
Verified: v0.0.3 zip contents + rebuilt exe in an isolated directory
(no repo above it) boots to the Ape Escape intro on both HLE and LLE
paths, guard armed from disc SCUS_944.23; in-repo runs still arm from
the local EXE.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Generalizes the Capcom 2D widescreen machinery for the MMX4 engine revision:
- [widescreen.bg2d] count_site accepts the inline slti/sltiu loop compare
(X4/X5 style) alongside MMX6's li; startx_site accepts subu rd,zero,rt
(X4's -(scrollX & 15)); inline-count sites route through the generic
psx_ws_bg2d_* helpers that keep the emitted ring mask and skip MMX6's
title-specific host-side refill state.
- [widescreen.cull] a1_sites now also accept a move rd,a1 (addu encoding):
classifiers that copy the caller margin with no delay-slot nop widen the
copied value instead.
- [widescreen] nw_hud_source_lo/hi: optional GP0 command-source range filter
for nw_hud_corners, so a pure-2D game re-anchors only its dedicated HUD
packet arena and never world sprites. 0/0 preserves legacy behavior.
ws_nw_hud_shift_vertices re-anchors polygon/line prims as one rigid
composite; mono rect/dot/8x8 sites shift too.
All helpers identity at 4:3; unset config keys are byte-identical.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two hardware-faithfulness fixes found by the always-on tap rings + Beetle
oracle A/B on fingerprint-matched attract music (tools/audio_pair_segments):
the recomp played the identical track at -3.8 dBFS vs the oracle's
-14.1 dBFS — +10.3 dB hot, clipping continuously.
1. spu.c: the voice repeat/loop address register (voice reg 7) is LIVE
state on real hardware — writing it after KEYON retargets where the
next END+REPEAT block jumps (Beetle spu.cpp:1150/333), and the block
auto-latch (flag bit 2) updates the guest-visible register (386/1302).
The recomp cached repeat_addr at KEYON only, so X5's driver idiom —
KEYON, then point the loop register at a silent tail block; no KEYOFF
ever — left voices looping the LOUD sample body forever. Measured: 3
stuck voices re-looping ~1400x/s, 160k SPU events/min vs the oracle's
633, zero game KEYOFF writes (the game never needs one), +11 dB.
2. cdrom.c: implement the CD controller's decode volume matrix
([src][port], 0x80 unity, pending banks + apply latch at 1F801803.3
bit 5 — mapping copied from Beetle cdc.cpp:1483-1501, applied after
the XA resample per PS1 hardware tests). X5 drives it constantly:
0x73-0x7E steady, ramped to 0x00 at scene changes (measured via the
new beetle `cdc_volume` command + PS_CDC::PSXRecomp_GetDecodeVolume
peek). Without it music fades never happened and steady level sat
~0.6 dB hot.
Also: AUDIO_TAP_VOICES (T0 voice-sum tap) for source-by-source mix
decomposition, per-tap WAV rates, bridge stats in audio_stats.
Measured after (MMX5 attract, 20-25 s windows):
spu_out RMS: -3.8 dBFS (clipping) -> -15.5 dBFS [oracle -14.1]
music-only chain already matched the oracle at -14.8 dBFS, confirming
decoder + mixer gain equivalence; the entire excess was stuck voices.
underruns: 2 events / 98 s (delivery fix holding).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replace the push-model SDL_QueueAudio output with the shared band-limited
polyphase resampler + P-only fill servo (recomp_audio_drc.h), fed by an SPSC
ring under SDL_LockAudioDevice and drained by a pull callback. The SPU renders
44100/60 per wall-clock frame while the host consumes on its own crystal; the
unsynchronized clocks drifted the queue to underrun (silence gaps = sustained
crackle). The bridge nudges the resample ratio <=+/-0.5% to hold the ring near
target so the queue never runs dry.
Measured on MMX6: legacy push ~1.5-2.5 underruns/s (queue draining to 0ms) ->
bridge 0 underruns, fill held ~50ms. PSXRECOMP_AUDIO_LEGACY=1 keeps the old
push path for A/B; PSXRECOMP_AUDIO_PROBE=1 prints the fill/underrun trend.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Measurement infrastructure for the audio-quality campaign (MMX5 complaint),
ported from snesrecomp's audio_trace/oracle methodology and the
_audio_round2 tap model. No behavior change to the audio path itself.
- audio_trace.{c,h}: three always-on 95 s PCM rings — T1 spu_out (canonical
spu_render output), T2 cd_in (XA/CD-DA PCM entering the SPU CD bus),
T3 host (exact post-fade bytes handed to SDL_QueueAudio) — plus a 512k
event ring (every SPU reg write stamped with the SPU sample clock;
RENDER/PUMP_SKIP/UNDERRUN/MUTE/UNMUTE/CD_PUSH/SPU-DMA), audible-vs-
silent counters (-42 dBFS gate), queue watermarks, and a server-side
44100 Hz WAV dumper (rate exactness matters: snesrecomp da33c06).
- psx-beetle: the libretro audio callbacks stop discarding Beetle's fully-
mixed reference SPU PCM and record it into the same tap ring. Port 4380
gains audio_stats/audio_wav/audio_events with the identical wire protocol
as the runtime (rule 16 symmetric harness) — the oracle WAV capture.
- debug_server: audio_stats / audio_wav / audio_events on the runtime port.
- tools: audio_ab_diff.py + audio_spectral_ab.py (drift-tolerant offline
A/B: drift ppm, onsets, cents, timbre, click rate) and audio_analyze.py
(first-dirty-tap cross-tap decision tree) ported from snesrecomp/
_audio_round2; audio_capture_ab.py dumps both processes symmetrically.
Probes query history; nothing is armed at probe time.
Detect a side-effect-free poll loop by its repeated same-PC interrupt
checks and, when the window touched no MMIO and issued no stores,
fast-forward guest time to the next internal device event
(vblank/timer/cdrom/dma/sio) in whole loop quanta — bit-exact versus
executing the spins one by one. Off switch: PSX_IDLE_SKIP=0 env or the
idle_skip TCP command; gated off in exception context, precise mode, and
lockstep.
memory.c adds the two always-on gates the skip requires: g_guest_store_count
(bumped at every raw store chokepoint) and g_mmio_access_count (bumped on
every device-page read/write). Any store or MMIO touch in the window
disqualifies it as a pure poll loop.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Games that discard their boot/init code load gameplay overlays OVER the
main-EXE text window (Tomba 2: hot code at 0x1DA88/0x24548/0x3116C, below
its 0x38800 floor). Post-baseline, a DIRTY page in [KERNEL_WINDOW_END,
FLOOR) is by definition live overlay code the compiled image is stale for,
so make overlay_cache_window_contains() admit dirty boot-text pages as a
capture/candidate window instead of forcing ~300ns/insn interpretation.
Auto-capture convergence: replace the fixed trigger gate with
insn-pressure sizing and exponential futility backoff — retry only when
the manifest signature or loader candidate count actually changes, so
coverage stops churning once it has converged. compile_overlays.py moves
to entry-based coverage; autocapture.c escapes >=0x80 cp1252 bytes from
child-compile output so the status JSON stays valid UTF-8.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Ape Escape "Checking MEMORY CARD" thread-switch smear: a genuine in-exception
ChangeThread detected deep in nested host dispatch commits a poisoned TCB context
(stale resume-PC latch ahead of the live GPRs) -> the resumed consumer faults on
a smeared jumptable index (DISPATCH FATAL).
Fix#2 (user-agreed architecture): don't honor the switch at the poisoned nested
point. Resume the outgoing thread transparently at its OWN EPC, record the switch
as pending, and honor it at the next block leader where that thread is running
with a materialized resume PC -- re-saving it cleanly there. Do NOT require the
outermost STATIC dispatch boundary: the memcard consumer lives permanently inside
the nested dirty-RAM interpreter and never reaches it; the scheduler landing
already resets the nested dispatch/call-unit counters on the longjmp, same as the
immediate-switch path.
- interrupts.c: the kind-30 escape splits into immediate (outermost / toggle-off
/ no clean resume PC) vs deferred; new honor hook at the top of
psx_check_interrupts; always-on thread_trace kinds 31 (defer) / 32 (honor) /
33 (stale-drop). The deferred branch restores the outgoing thread's clean
GPRs/HI/LO/SR AND cpu->pc (the in-exception RFE had left cpu->pc at the target).
- traps.c: psx_sched_save_context / set_current_tcb / current_tcb exports over
the scheduler's private TCB accessors.
- A/B toggle PSX_DEFER_SWITCH (default ON); =0 forces baseline immediate switch.
Validated (LLE + HLE, ring-verified): converts the smear DISPATCH-FATAL into
forward progress -- the card driver now reads 108 blocks. A/B (PSX_DEFER_SWITCH=0)
proves the fix is SOUND: the remaining stall (SIO IRQ bit 7 masked in I_MASK, card
read never completing) reproduces IDENTICALLY with the fix off, so it is a
SEPARATE, pre-existing SIO-mask bug -- the next blocker, not this fix's residual.
Goal (file-select showing File 1-4) NOT yet reached.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- wide_blit_center now copies the canonical column into the wide-surface CENTRE
over the FULL VRAM height, not just [disp_y, disp_h]. Ape vertical-double-
buffers (display_y 0<->256, both bands in one wide surface); a scrolling
cutscene (the cityscape intro) exposed centre rows outside the display band,
which the disp_h-only blit left black -> horizontal black-band tearing. Full-
height blit makes the wide centre bit-identical to the full-mirror output for
every band. Fixes the regression; keeps the perf win.
- sticky gte_game_mode threshold/hysteresis retained (cutscene flicker fix).
- New GL wide-surface inspection over TCP: wide_dump_full backend method
(glb_wide_dump_full) so the `wide_full` command dumps the ENTIRE GL compositor
surface (both double-buffer bands + margins), matching the SW path. wide_shot
(displayed band) already worked on GL. Lets native-wide be debugged entirely
over the TCP server, no window focus.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds full 16:9/21:9 native-wide support for fully-3D titles (Ape Escape,
SCUS-94423) as a generic, opt-in, 4:3-byte-identical framework capability, plus
a native-wide GL compositor perf fix and a cutscene present-mode flicker fix.
Widescreen (generic, per-game opt-in via [widescreen*]):
- Shared screen-extent cull detector (runtime/include/ws_cull_detect.h) used by
all three backends (recompiler, dirty-RAM interp, sljit). Per-game signature
immediates ([widescreen.cull] screen_w_imms/screen_h_imms) — Ape's funnel
tests 0x181/0xF1 on its 368px display, not Tomba's 0x140/0xE0.
- Three funnel idioms widened: unsigned per-vertex sltiu (psx_ws_cull_sltiu),
signed min/max slti right edge + bare-bltz left edge (psx_ws_cull_slti/bltz,
new helpers + SLJIT_HLP appends), and billboard center+/-halfwidth. Explicit
[widescreen.cull] slti_sites for X-only funnels the auto-detector can't
qualify.
- gte_game_mode: GTE-activity gameplay detector for 3D titles with no sprite-tag
helper (gte.cpp notes every RTPS/RTPT). STICKY (>=3 verts, 45-frame hysteresis)
so sparse cutscene frames never flip to a 4:3 pillarbox = fixes intro flicker.
- nw_hud_corners: push outer-third screen-space HUD sprites to the true wide
corners in native-wide (else inset by the reveal).
- nw_backdrop: stretch a full-frame axis-aligned 2D backdrop quad to fill the
wide frame (for titles with a flat sky plane).
- Pattern-scanned interp/sljit widen hooks now gated per-game (auto_screen_x /
auto_backdrop): fixes an ungated auto_backdrop false-positive that rewrote a
live GPR = the ~24s native-wide crash on Ape.
Native-wide GL perf (fixes 16:9 frame drops):
- The wide surface centre equals the canonical 4:3 framebuffer, so wide_blit_center
copies canonical->wide-centre at present (both present paths) and the per-prim
mirror SKIPS prims fully inside the 4:3 frame (mirror_x/geo/batch_center_only).
Only reveal-margin prims re-rasterize. Peak frame time 38.8ms -> 17.5ms, no
vsync misses. Toggle gl_wide_fast; 4:3 untouched (all gated on g_wide_cur).
Runtime-only except the recompiler cull emit (regen needed for cull widening).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Master's text-divergence guard (dirty_ram_text_native_ok) blocked native dispatch
of any compiled function whose text page is runtime-dirty (an overlay overwrote
it), falling back to interp and — when the entry can't be resolved there — to
psx_unknown_dispatch (FATAL). Tsumu's translation layer patches string/glyph DATA
tables that share a 4 KB page (and the guard's 256-byte compare window) with real
compiled functions, so a data patch diverged the page and fatally blocked
func_80070248 on boot ("unknown dispatch: 0x80070248").
Two-part faithful fix (general, not per-game):
- dirty_ram_text_native_ok now decides on the dirty/flagged path by BYTE-COMPARING
the entry window against the reference image, not the coarse page-dirty
heuristic. A data-only change no longer blocks a function whose own code bytes
are unchanged; a genuine code overwrite still diverges -> interp (guard intent
preserved). Pristine pages keep the no-compare fast path.
- New dirty_ram_text_bless(phys,bytes,len): text_xlate blesses every in-place RAM
patch (glyph/msg_inplace/msg_sep) into the (runtime-owned, mutable) reference
image, so intentional data patches read as data. Genuine game self-mod writing
DIFFERENT bytes still diverges and is still caught.
Verified: Tsumu boots to the title screen with the English translation applied,
no fatal. Benefits every title (more precise guard).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a general, opt-in language selector to the RmlUi launcher, driven entirely
by game.toml [localization]:
[localization]
default = "en"
languages = [ {code="en", label="English"}, {code="off", label="Default"} ]
- config_loader: parse [localization].default + languages[] into RuntimeConfig;
add UserSettings.language with settings.toml [localization] parse + write.
- launcher: a "LOCALIZATION" panel in the Settings view (cycle toggle, matching
the Renderer/Screen-model idiom) shown only when the game declares languages.
The chosen code persists to settings.toml and rides UserSettings.
- text_xlate: new text_xlate_set_language() re-applies the language after the
launcher (init runs before it); "off"/"jp"/"" disable APPLY (native game).
- main: seed/read-back the language through the launcher and re-apply on every
boot path (so a settings.toml choice sticks even when the launcher is skipped).
- Settings view laid out in two columns (VIDEO | LOCALIZATION+AUDIO+CONTROLLER)
with SYSTEM full-width below, so it uses the horizontal space and fits without
scrolling; #root gains overflow-y:auto as a safety net.
Only games that declare [localization] get the menu (Tsumu today); byte-identical
for every other title.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two interlocking pieces from the Tsumu HUD/menu-label investigation:
- GP0 ring builder attribution (gpu.h/gpu.c/debug_server.c): op-0x80
entries now carry a bounded guest-stack unwind (bld[6]) + copy-time
$sp (csp); gpu_frame_dump emits func/ra per entry and csp+bld[] for
copies. This cracked the HUD compose: whole label strips are
MoveImaged from the glyph sheet (src 834/852,352 -> HUD cache y=32).
- [[vram_patch]] (text_xlate.cpp/.h + gpu.c hook): third translation
layer for text that exists only as pre-rendered pixels in an
uploaded texture asset (Tsumu HUD labels, menus, prompts live in
the glyph-sheet image; the 144-glyph font has no katakana at all).
On every GP0 0xA0 upload completion the configured rects are
verified halfword-for-halfword against expected JP pixels and only
then rewritten with the target-language pixels via the renderer
facade (software/GL/supersample coherent). Re-applies on every
re-upload; verify-fail leaves foreign assets untouched. Debug:
xlate stats gains vram_patches/vram_applied; new "vpatch"
force-apply subcommand for config iteration without rebuild.
Verified on-screen: HUD "STAGE: 1-1 / BATTERY: 100", pause menu
(CONTINUE/RESTART/GIVE UP/CHANGE CONTROLS, TARGET BATT), title-menu
strips. docs/STRING_TRANSLATION.md Appendix Z documents the model,
mechanism, authoring recipe, and remaining strip backlog.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds a third translation path to the framework for UI text drawn
glyph-by-glyph as sprites from a fixed-stride, space-padded EXE table
(Tsumu's 59-slot level/stage-name array at 0x80070300..0x800708FE).
Such labels carry no string pointer and no terminator, so the existing
dispatch/arg-scan message hook can never reach them.
Instead, [[glyph_label]] TOML entries drive a load-time RAM source patch:
once the EXE label region is resident, each confirmed slot is verified
byte-for-byte against its expected JP source bytes, then overwritten with
the target-language text (fullwidth-Latin Shift-JIS, little-endian)
re-padded with fullwidth spaces to exactly the slot width. The game's own
per-glyph routine then draws English. Verify-before-patch means a slot is
never touched unless the exact JP label is present; writing exactly
`width` bytes (== the table stride) means a patch never spills into the
adjacent slot.
- label_transcode_le(): UTF-8 -> fixed-width LE fullwidth-SJIS, truncated
to whole cells, fw-space padded; never exceeds width.
- glyph_labels_patch_locked(): idempotent verify-and-patch scan, run from
the dispatch hook while any label is unpatched (throttled; a single
relaxed load once g_glyph_pending hits 0).
- Loaded/gated exactly like message entries (language column, PSX_LANG,
apply-armed). "xlate glyph" debug subcommand reports per-slot status.
Verified (build-dbg, headless): all 59 slots patched (glyph_patched=59,
pending=0), RAM byte-exact English at TOWER/SANDBOX/JELLYFISH/LOOP THE
WARP, game alive at 26M dispatches. Framework code only; no regen, works
in Release. Table + user visual sign-off in TsumuRecomp/translations.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
RmlUi launcher grows a Settings surface (renderer / supersampling / AA /
texture filter / screen model / skip-FMV / turbo / fullscreen / widescreen /
SPU-HQ / deadzone / BIOS / disc) and a live Controls rebind page. Bindings
persist to keybinds.ini through the new shared runtime module
runtime/include/psx_keybinds.h + runtime/src/psx_keybinds.c; main.cpp routes
SDL input through the rebindable table. Ape Escape is the first consumer;
the feature is game-agnostic framework code.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Implements the framework from docs/STRING_TRANSLATION.md: an always-on
capture inventory + a draw-time apply hook, both hung off the psx_dispatch
chokepoint via fntrace_record (no generated-code edits, no BIOS regen,
works in Release).
- text_xlate.{h,cpp}: EncodingProfile abstraction (shift_jis profile:
SJIS record reader with NUL/0xFFFF terminators + halfwidth-kana + control
framing; UTF-8 -> fullwidth-SJIS transcoder so English renders with the
font's existing fullwidth Latin, zero glyph injection). Capture inventory
(FNV-1a64 keyed) queryable over TCP; apply repoints the string-arg reg at
guest-stack scratch (KV-gated: only known records are swapped, stray
pointers untouched; own scratch is skipped to avoid recapture).
- fntrace.c: call text_xlate_on_dispatch() at the dispatch chokepoint.
- config_loader: [localization].language (+ [runtime].language alias);
env PSX_LANG overrides. Loads translations/*.toml per project.
- main.cpp: text_xlate_init() at game-config load.
- debug_server: "xlate" command (stats|dump|todo|reload) — observability
over TCP, no log files.
Verified: builds (Release), Tsumu boots headless, capture live
(distinct records accumulating), apply disabled until a table is present.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
debug_server_record_frame (every vblank) and the sio_state handler read
SIO_STAT/CTRL/RX via the GUEST-BUS sio_read(), which has hardware
side-effects: sio_tick(0) advances the delayed-IRQ state machine, reading
SIO_STAT decrements sio_ack_visible_reads and clears the ACK bit, reading
SIO_RX_DATA pops the RX FIFO. So the observer silently mutated the SIO/pad
handshake every frame in dev builds (the cardinal observability-alters-state
violation the event_ring header itself warns against).
Add side-effect-free sio_peek_stat/ctrl/rx_data() and use them on the debug
path; the guest bus keeps sio_read() with its real side effects. Also add an
fmv_state TCP command reporting the resolved config path + auto_skip_fmv +
frame-count-table params + live MDEC/XA/pad state, so the FMV-skip + config
state can be observed rather than inferred.
Note: this removes a real per-vblank SIO perturbation but is NOT the full fix
for the dev-build FMV-skip divergence (residual is the cycle-timing-core
class; see precise_irq_slice_state). It is correct observability hygiene.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The committed beetle debug server (h_parity_dump reads=1) and beetle_libretro
read hook already referenced parity_trace_reads_get / parity_trace_reads_total /
parity_trace_note_read, but only a dead per-watch scalar table existed — the two
readback functions were never implemented, so every psx-beetle rebuild failed to
link (blocking the oracle entirely).
Implement the dedicated READ ring as the symmetric counterpart of the write ring:
note_read pushes one ParityEntry per read of a watched word, carrying reader
provenance (pc = load PC, epc = read address, target/watch[slot] = loaded value)
PLUS that slot's current last-WRITER metadata (wpc/wcycle/wframe/wtcb). One read
row therefore answers "who read what value, and who last wrote it." reads_get /
reads_total mirror parity_trace_get / parity_trace_total (newest rows, oldest
first). Reset clears the ring. Removes the unused scalar last-reader table.
Validated: psx-beetle links + boots; PSX_PARITY_TRACE=1 accrues well-formed read
rows (reads_total>0, reader+writer provenance populated).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Packed games can rewrite code over their own loaded EXE text after static recompilation. A dirty-page bit alone is too coarse for dispatch decisions: it prevents stale native execution, but it also cannot distinguish data writes in a text page from instruction bytes that actually diverged from the compiled image.\n\nRegister the boot EXE image as a reference and let memory writes mark text pages whose bytes differ from that image. Static game dispatch now asks whether the target remains native-safe; diverged pages fall through to execute live RAM through the dirty interpreter, while matching code bytes can still use the compiled path.\n\nConstraint: Runtime-installed or rewritten RAM code must execute the program's own instructions, not HLE or stale static code\nRejected: Keep the existing dirty-page-only gate | too coarse for packed/self-modifying main text and data writes sharing code pages\nConfidence: medium\nScope-risk: moderate\nDirective: Do not bypass dirty_ram_text_native_ok() at game compiled dispatch sites without proving RAM still matches the compiled image\nTested: Local Tekken 3 branch using this mechanism reaches playable fights; clean upstream worktree compiles changed main/memory/dirty files before hitting an unrelated existing macOS overlay_loader LONG/InterlockedExchange build error\nNot-tested: Full upstream macOS build completion; non-macOS build matrix
Tomba2 16:9 GL reproduced the reported collapse (12-14fps) and the GPU-side
split EXONERATED the GPU: scene_gpu ~10ms, mirror ~0.2ms, while emu_cpu is
~82ms (vs ~15ms in squash). gl_ws_ablate=1 (mirror passes fully off) leaves
it at ~75ms, so the wedge is CPU work outside the mirror draws. The GL
submission paths run INSIDE guest emulation (GP0/DMA processing), so driver
CPU cost lands in emu_cpu where no GPU timer sees it.
frame_perf now reports, per wide/4:3 frame class: CPU wall inside
flush_tex_batch (cpu_flush_ms_avg), CPU wall inside the glb_wide_* entry
points (cpu_wide_ms_avg), batches/frame, wide target sets/frame, and wide
FBO creations/frame (a nonzero steady value = surface recreation churn).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Native-wide perf attribution instrumentation (permanent):
- Each wide-mirror pass (tex-batch mirror, flat-geometry mirror, overlay
full-width rect, wide_clear) is bracketed with a GL_TIMESTAMP pair
(glQueryCounter; TIME_ELAPSED cannot nest inside the frame scene query).
Pairs pool per buffered frame (1024/frame), summed at readback into
mirror_gpu_ms + mirror_passes per sample; frame_perf now reports
mirror_gpu_ms_avg/max, canon_gpu_ms_avg (scene minus mirror),
mirror_passes_avg and mirror_pass_us for wide frames.
- New debug cmd gl_ws_ablate mode=<0..3>: 0 normal, 1 skip mirror passes
entirely, 2 mirror state churn without draws, 3 mirror draws stay on the
hr FBO (no per-pass FBO rebind; content-corrupting perf probe).
First MMX6 16:9 measurements with the split: mirror passes cost 20-125us
EACH (FBO ping-pong penalty per pass) but MMX6 2D batches to ~3-15
passes/frame so 16:9 attract holds ~60fps on this branch; the per-pass
cost x ~900 immediate flat prims is exactly the Tomba2 3D 60ms scene.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Root cause (proven via the new present ring + gl_coh_ring correlation):
flush_cpu_upload() merged all pending CPU->VRAM writes into ONE union
bounding box. A frame with two disjoint uploads (sprite column + tile row —
routine MMX6 texture streaming) produced a union spanning the display
framebuffers, which the flush painted from the STALE CPU VRAM mirror (the
FBO is authoritative under GL) — stomping live frames with black for
exactly two presents (one per double-buffer parity). Scene-dependence of
the uploads made it look config/timing dependent; the software renderer is
immune because its CPU array is authoritative.
Fix: exact pending-rect list (16 rects; merges only when zero uncovered
pixels are added — containment or same-band extension; wrap-aware GP0(A0)
transfers split into up to 4 exact rects instead of the old "wrapped: take
all" full-VRAM union; overflow -> order-preserving flush-all; pre-context
union retained where the CPU array is authoritative). Merge rule verified
by a 20k-randomized-rect host test (0 stale / 0 missing painted pixels).
Observability (always-on): gl_present_ring records every SwapWindow (path,
src/letterbox rects, drained glGetError, wall-ms, backbuffer + blit-source
pixel samples) — the instrument that gave the 1:1 black-capture
correlation. debug_server_send_fmt now heap-formats responses >64KB (they
previously truncated into unparseable JSON, breaking large ring dumps).
(debug_server.c also carries the Vulkan screenshot_file sync hunk from the
concurrent VK bring-up; see the next commit.)
Validated: ~18-min MMX6 GL attract soak, ~1600 window captures across 3+
attract cycles — zero isolated black frames, zero GL errors (pre-fix:
19/40 captures black at the intro). Tomba1 build-gl boots clean; its full
attract soak follows. ENHANCEMENTS.md added (framework enhancement ledger,
R1 OpenGL / R2 Vulkan state).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Tomba2's Trolley attract demo interprets a working set past 64K distinct
PCs. The per-PC tracking tables (g_dirty_ram_pc_table / exec_pc_table) used
an UNBOUNDED linear probe, so once a table filled, every lookup for an
absent PC scanned all 65536 slots before returning NULL — per executed
instruction. Host thread profile showed ~99% of emu-thread time inside
pc_table_get_or_insert_in; guest crawled at 0.3 fps (1M guest cycles per
3s wall) while telemetry showed the window "LIVE" on residual motion.
Fix: hard-bound the probe to 128 slots (miss => NULL => "stop tracking",
the documented degradation) and grow the tables 65536 -> 262144 so the
Trolley working set actually fits — the same table feeds overlay-capture
seed lists (MAX_CAPTURE_PCS aliases it), so the old size also silently
truncated capture manifests for large regions.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The emitter routes mfc2/cfc2/mtc2/ctc2 on flag/IR/derived GTE registers
through direct gte_read_data/gte_read_ctrl/gte_write_data/gte_write_ctrl
calls (code_generator.cpp COP2 case), but the overlay DLL preamble had no
shims for them — any GTE-heavy overlay failed to LINK (undefined
gte_read_ctrl) and its region ran interpreted forever. That silently
blocked all new native coverage for the lava/Trolley attract-demo regions
after the caches rotated.
GTE state is shared runtime state, so the shims forward unconditionally
through OverlayCallbacks (a local copy would fork the flag/IR timeline).
ABI bumped 9 -> 10; the loader gate rejects+deletes pre-shim DLLs and
autocompile self-heals; plain gte_data[]/gte_ctrl[] accesses stay direct
CPUState reads in emitted code, matching the emitter.
Smoke-validated: previously-failing GTE-heavy captures now compile clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Root cause of MMX6 "A game data could not be found" (weeks-old, cross-backend):
the game's save-scan calls memset through the kernel A0 jump table; entry
A0:2B points into BIOS ROM at 0xBFC02B8C (one of the entries the kernel never
patches into RAM). Transitive discovery can never find it (reachable only via
the data table, no jal), so the dispatch missed and psx_unknown_dispatch
silently no-op'd — killing the scan before firstfile ever ran. Every card
layer (SIO, IRQs, SwCARD events, sector data, kernel directory parse) was
verified correct along the way; the sole defect was the undiscovered ROM
function plus the silent miss.
- recompiler/seeds: enumerate the post-init A0/B0/C0 tables (kernel RAM
0x200/0x874/0x674) and seed every ROM-pointing entry with no dispatch
coverage — 30 functions (memset, memcmp, memmove, strncat, rand-family,
card/dev services...). memset was merely the one MMX6 called; the rest were
identical latent bugs. BIOS regen: 4406 -> 4439 dispatch entries.
MMX6 CONTINUE now lists and loads saves (user-validated + scripted run:
full FILE SELECT, 1093 card txns, unknown_dispatch_log total=0).
- traps.c: an unresolvable dispatch target now dumps a crash report and halts
BY DEFAULT (PSX_FAIL_FAST_UNKNOWN_DISPATCH=0 opts out for diagnostic
sessions; the miss is ring-recorded either way). The old silent-no-op
default left stale registers and a severed call chain, and masked this bug
for weeks. Deliberately NO interp fallback: the dirty-RAM interpreter is
for runtime-written code (Rule 18); interpreting around a static-discovery
hole would hide it exactly like the no-op did.
- runtime.cmake: EXE_NAME option — per-game process names (mmx6-runtime.exe)
so one game's `taskkill /IM psx-runtime.exe` dev sweep no longer kills
every other game's instance.
- debug rings (the tooling that found this): HLE call ring gains
consecutive-duplicate collapse w/ repeat counts (a 4-way TestEvent poll at
175K calls/s no longer flushes the ring in 0.1s), per-entry TCB + in_exc
(thread attribution), a "since" paging param and 8MB dump buffer;
card-trace EvCB watch widened to all 7 card event slots (slot 0 =
SwCARD-IOE, the success event, was previously invisible).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
USER PIVOT 2026-07-02 (CLAUDE.md §0 amendment included): psxrecomp now
carries an opt-in High-Level Emulation backend alongside LLE. LLE (the
recompiled BIOS) stays the default, the reference implementation, and the
correctness/timing oracle; every service HLE does not implement falls
through to LLE, so HLE is never load-bearing beyond what it covers.
Mechanism: full_function_emitter.cpp emits a null-by-default hook
g_psx_bios_hle_hook consulted at the top of every psx_dispatch_impl
iteration, on the pre-normalize physical address, BEFORE any backend
(game image / dirty-RAM / static table) claims the target. A handled
call resumes at $ra through the trampoline's existing return/tail
contract. NULL (default) = pure LLE, dispatch-identical. Regenerating
BIOS+game images is required (emitter change).
v1 call-HLE (runtime/src/bios_hle.c): the B0 event family — DeliverEvent
(incl. callback dispatch with the kernel-true $ra 0x1720), OpenEvent,
CloseEvent, TestEvent, EnableEvent, DisableEvent — each ground-truthed
instruction-by-instruction against the SCPH1001 kernel disassembly
(0xBFC11644..0xBFC11A84) and operating on the REAL guest EvCB table
([0x120]/[0x124], stride 0x1C). WaitEvent/threads/pads/card/A0/C0 fall
through to LLE. Cycle charges are documented approximations; LLE remains
the timing oracle. No-stubs stands.
Boot HLE: with the tier on (and no keep-intro opt-out) the hook one-shot
intercepts the first dispatch of the BIOS shell entry (RAM 0x30000 —
LoadRunShell's indirect call always routes through the dispatcher), so
BIOS Main() proceeds straight to SYSTEM.CNF + game EXE load with kernel
state built entirely by the real recompiled kernel init, unpaced (boot
turbo) until game handoff. This REPLACES the fast_boot snapshot
capture/restore; fast_boot=true now aliases the boot-skip alone.
Selection: [runtime] bios_hle / bios_hle_keep_intro (+ settings.toml
bios_hle mirror), PSX_BIOS_HLE / PSX_BIOS_HLE_KEEP_INTRO env overrides,
startup banner bios_backend=/bios_boot=. The PSX_HLE_SCHEDULER spike is
folded in as the tier's standing subsystem replacement: default from
[runtime] hle_scheduler via psx_hle_scheduler_set_default, env wins.
Observability: always-on 16K-entry HLE ring (route LLE-fallthrough /
HLE / boot-skip per record) + hle_dump TCP command (TCP_COMMANDS.md).
VALIDATED on Tomba (build-wt): LLE pass byte-identical behavior (real
intro, card check, save/load OK); HLE pass boots shell-less, TestEvent/
DeliverEvent route HLE, ChangeTh/ReturnFromException/OpenTh/CloseTh and
the card stack fall through to LLE, save/load OK (user-driven).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
GetID's last four response bytes were hardcoded 'SCEI' (NTSC-J). Real
hardware reports the region of the INSERTED disc. Kula World uses the
BIOS CD API, and the kernel CD driver revalidates the disc (ReadTOC +
GetID, polled with irq_enable=0x18) when it reinitializes after the
game's first file load. Seeing a 'Japanese' disc, it fell into the
endless GetStat/Init retry loop: CD delivery froze at sector 307 (LBA
318, just before the first level directory) and the game waited forever
on a black screen.
Diagnosed from the always-on cdrom trace ring (grabbed via gdb from the
live process): 307 sectors stream cleanly, Pause completes, then the
BIOS's own ROM CD firmware (0xBFC0Dxxx) masks INTC delivery, issues
ReadTOC + GetID, reads the response... and rejects it. Beetle PSX (the
oracle; boots the same dump + BIOS to demo gameplay) derives the SCEx
string from the disc's SYSTEM.CNF boot serial - so we now do the same:
cdrom_set_disc_scex() is fed at launch from the existing disc_identity
module (PAL->SCEE, NTSC-J->SCEI, NTSC-U->SCEA; default SCEA matches the
one supported BIOS). Verified so far: launch log reports 'disc region
PAL (serial SCES-01000)'; the sector-counter wall-break run is in
flight. Framework-wide note: Tomba/MMX6 (NTSC-U, own CD drivers) never
hit this path but now get 'SCEA' - matching real hardware - instead of
'SCEI'; regression-test before any master merge.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdjxoRB1f9VQsNzZayCgGS
The cycle-accuracy emitter emits calls to 12 timing runtime functions
(psx_cyc_load_*, psx_icache_fetch, psx_muldiv_*, psx_gte_*, psx_slice_block,
...) + debug_server_cyc_observe + g_debug_last_store_pc. Overlay DLLs never
got these symbols, so every overlay failed to link -> 0 native shards ->
100% interpreter (unplayable). Fix (option A): expose the 12 via the overlay
callback ABI.
- overlay_api.h: +12 timing callbacks in OverlayCallbacks, ABI 8 -> 9.
- overlay_loader.c: populate the 12; add warn_on_cgtag_mismatch() (loud
warning when shards exist under the wrong cg-hash).
- compile_overlays.py: 12 preamble forwarders + local g_debug_last_store_pc
+ -DPSX_NO_DEBUG_TOOLS.
- cosim.c/main.cpp: input-cosim setpad + held pad (deterministic input).
- cosim.py: env-overridable COSIM_EXE/CWD/GAME.
KNOWN REGRESSION (patched on a follow-up branch): Tomba pause menu (triangle)
wedges. Root-caused: an IRQ taken mid-block resumes at the function entry
leader instead of the interrupted PC, re-zeroing the menu's VSync wait
counter (scratchpad 0x1F8001E8) every frame so it never reaches its target.
Not the scheduler (LLE reproduces) and not timing math -- interrupt-resume
re-entry.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Matthew Stanley