Tomba 16:9: rooms, sky falls, and task-card sequences presented stretched
because native-wide's 2D-backdrop stretch fires wholesale in scenes whose
background phase never ends. Such scenes now present native 4:3 from the
faithful canonical buffer, keyed on DEEP UNTAGGED POLYGON OVERHANG (>=24px
past the canonical window, >=4 polys, sustained 2 frames) — the only world
signal that survived adversarial scenes (shaded prims, GTE volume, any-poly
and shallow overhang all falsified by census evidence; lineage in gpu.c).
Scoped to sprite-tag titles; full_2d (MMX6) and gte_game_mode (Ape) paths
unchanged. Plus: native-wide never stretches a canonical-width game frame
(fallback pillarboxes); always-on present-classification ring (TCP
present_ring) across all backends; debug input injection drives the full
pad model (hybrid d-pad drop + stick axes); savestate TCP honesty; census
tagged column + ws_hud_mode A/B (HUD-corners deferred: ENHANCEMENTS.md W2).
USER-VALIDATED on Tomba (SCUS-94236).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2D-only-scene classifier (fixes room/fall/title-card stretch, Tomba):
- Native-wide's 2D-backdrop stretch fires wholesale in scenes whose
background phase never ends (room interiors, sky-only falls), stretching
the whole frame. Such scenes now present native 4:3 from the faithful
canonical buffer (gpu_ws_present_native_43).
- The world signal is DEEP UNTAGGED POLYGON OVERHANG: polys whose raw SX
extent crosses >=24px past the canonical window, >=4 per frame, sustained
2+ consecutive frames. Lineage of rejected signals (each defeated by a
real scene, census-evidenced): shaded-prim presence (title-card letter
tiles are gouraud), GTE projection volume (task-jingle projects
world-scale counts; title screen idles at 1680 verts), any-poly overhang
(card letters slide in from off-screen), shallow overhang (jingle
sparkles poke <=20px). Outdoor terrain measures 55 deep-overhang
polys/frame vs 0-1 for every observed 2D scene/effect.
- Scoped to sprite-tag titles: full_2d (MMX6) and gte_game_mode (Ape)
classification is unchanged.
Native-wide present invariant:
- A game frame that cannot present wide (compositor fallback) pillarboxes
4:3 instead of stretching the canonical read across the wide window
(all backends; squash mode still stretches by design).
HUD corner re-anchoring for sprite-tag titles ([widescreen] nw_hud_corners):
- Poly/line prims never shift (they are the world / tagged billboards);
rect-family prims shift only when UNTAGGED — the proven hud_sprt_squash
discriminator — so enabling nw_hud_corners is safe without a packet
arena range.
Observability (always-on, institutional):
- present_ring: per-present classification ring (path, present width,
game_mode, native_43, wide fallback, tag age, gte_verts, ovh_prims)
for every backend; TCP "present_ring". Answers "why did this frame
stretch" after the fact.
- gpu_state ws block gains last_3d_frame, gte_verts, last_world3d_frame,
ovh_prims, last_ovh_frame; GTE vert counting is now always-on.
Input injection drives the full pad model:
- set_input/press now resolve pad type like real sampling: injected d-pad
drops a hybrid pad to digital (movement works), optional lx/ly/rx/ry
stick override (0..255), type via the coherent request channel.
Previously buttons-only: menus reacted, walking never did.
Savestate honesty:
- savestate_request_save/load return staged/refused; the TCP handler
reports ok:false instead of silently acking loads it cannot perform
(LLE host-fiber runs). Torn-restore reliability audit remains open.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The dirty-RAM text-image guard (685f38d) only armed when the loose local
boot-EXE file next to game.toml existed - found via find_project_root's
repo-marker walk. End-user release installs have neither a repo checkout
nor the EXE file (it is copyrighted game data), so the guard silently
never armed. Without it, fntrace's alternate game-start latch
(psx_game_address_in_text && dirty_ram_text_native_ok) cannot fire for
titles whose BIOS handoff enters text inside the shell-window shadow
(RAM 0x30000-0x5AFFF -> shell ROM until game-start): the shadow never
lifts, execution runs dead shell ROM bytes over real game text, and the
boot dies in a jump to PC=0 (Ape Escape v0.0.2/v0.0.3 release zips,
frame 14, both HLE and LLE boots; in-repo runs armed invisibly and
masked the bug during validation).
Fix, class-level for every title:
- main.cpp: defer guard arming until the disc path is resolved and add a
second source - extract the boot EXE from the mounted disc image via
ISOReader (basename of game.toml's exe field, SYSTEM.CNF BOOT token
fallback). The disc is the same bytes the BIOS loads, i.e. the true
reference; every install has one. Local EXE file still wins for dev
checkouts. A loud WARNING prints when neither source arms (previously
silent, which is how the broken releases shipped green).
- fntrace.c: the alternate game-start latch now requires a registered
image (dirty_ram_text_image_registered). Its safety argument - "RAM
still holds shell bytes that differ from the game image, so it cannot
fire prematurely" - only holds when an image exists; with no image,
native_ok degrades to !dirty and untouched text pages pass it,
latching game-start before the EXE loads and clearing the dirty
baseline mid-load. Without an image, only the exact entry_pc dispatch
may latch (the pre-685f38d conservative behavior).
Verified: v0.0.3 zip contents + rebuilt exe in an isolated directory
(no repo above it) boots to the Ape Escape intro on both HLE and LLE
paths, guard armed from disc SCUS_944.23; in-repo runs still arm from
the local EXE.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The controller map had no l3/r3 rows at all, so pad bits 1/2 could never
be produced by a gamepad — DualShock-only games (Ape Escape) were
unplayable. Keyboard slots existed but shipped unbound.
- main.cpp: PAD_L3/PAD_R3 bits; controller_map 14->16 entries; defaults
l3=leftstick, r3=rightstick; input.ini template + doc comment updated.
- psx_keybinds.c: keyboard defaults L3=T, R3=Y (continue the Q/W/E/R
shoulder row). Launcher Controls page picks them up automatically.
- beetle_main.cpp: same T/Y keys on the oracle so psx-beetle stays
drivable for L3/R3 games.
- tools/package_release.ps1: shipped input.ini template gains l3/r3.
- README: keyboard + controller map tables.
Wire is unchanged: sio.c transmits the button word verbatim, matching
Beetle's dualshock.cpp. Validated in-game on Ape Escape (user-confirmed
R3/L3 inputs on a real pad).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
GTE honors per-instruction lm (bit 10) and sf (bit 19) in the depth-cue and
lighting family (INTPL/DPCS/DPCT/DPCL/NCDS/NCDT/CDP/...) with the faithful
hardware formula and always-on GTE observability rings (gte_ring_dump,
gte_intpl_dump, gte_frame_stats, gte_latch_dump). Fixes Crash Bash
menu-character strobe/flicker and in-game flattening (signed INTPL vertex
tweens were clamped to zero by the hardcoded lm=1).
Also: dispatch no longer shadows runtime-relocated game code in the BIOS
shell window (native-ok byte compare routes matching text to the game path);
stale GPUSTAT-poll VBlank over-delivery fix + present pacing tooling.
Cross-title validation (GTE_LM_FIX.md): Crash Bash (fix target), Ape Escape,
Tomba 2, Mega Man X6 - all rebuilt branch-flavor (regen included) and verified.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three producer/delivery fixes, each measured on MMX5 attract (port 4520
audio_stats, always-on rings):
1. Guest-cycle sample budget (main.cpp): the SPU pump produced 44100/60
samples per host present — assuming a 60.00 Hz vblank against the real
NTSC 59.94 Hz and quantizing to presents. Production measured 43950/s
vs the device's 44100/s. Now the budget derives from psx_cycle_count:
exactly 768 guest cycles per output frame (33.8688 MHz / 44100), so
audio tracks guest time like every other device.
2. Bounded pacer catch-up (frame_pacing.c): frame_pacer_wait re-anchored
(forgave all debt) whenever a frame ran >1 period late, permanently
slipping the timeline on every heavy frame — measured 59.80 Hz average
against the 59.94 target. Debt up to 8 periods is now repaid by running
unpaced, so transient stalls no longer starve the audio ring; only
sustained sub-realtime forgives.
3. PI bridge servo (recomp_audio_drc.h): the P-only fill servo needed a
standing fill deficit of corr/kp to hold any steady correction (0.3%
skew = 7.5 ms of permanently eaten margin -> jitter spikes hit the dry
floor). Added an anti-windup integral term that holds steady clock skew
at zero fill error, and raised target/ring 50/150 -> 80/250 ms for
emulated-machine jitter (tens of ms, far beyond host-driver jitter).
Also: bridge dry-pulls during an intentional turbo mute are resynced
past, not reported as gaps (the mute IS the silence).
Measured, MMX5 attract, 150 s steady windows:
legacy push baseline: ~360 underrun events / 150 s (queue to 0 = gap)
bridge as merged: ~90 underrun events-equivalent / 150 s
this commit: 1 underrun event / 150 s, fill ~55-73 ms,
production 43981/s, corr -0.5% (absorbs the
guest's residual ~0.27% sub-realtime pace).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replace the push-model SDL_QueueAudio output with the shared band-limited
polyphase resampler + P-only fill servo (recomp_audio_drc.h), fed by an SPSC
ring under SDL_LockAudioDevice and drained by a pull callback. The SPU renders
44100/60 per wall-clock frame while the host consumes on its own crystal; the
unsynchronized clocks drifted the queue to underrun (silence gaps = sustained
crackle). The bridge nudges the resample ratio <=+/-0.5% to hold the ring near
target so the queue never runs dry.
Measured on MMX6: legacy push ~1.5-2.5 underruns/s (queue draining to 0ms) ->
bridge 0 underruns, fill held ~50ms. PSXRECOMP_AUDIO_LEGACY=1 keeps the old
push path for A/B; PSXRECOMP_AUDIO_PROBE=1 prints the fill/underrun trend.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Measurement infrastructure for the audio-quality campaign (MMX5 complaint),
ported from snesrecomp's audio_trace/oracle methodology and the
_audio_round2 tap model. No behavior change to the audio path itself.
- audio_trace.{c,h}: three always-on 95 s PCM rings — T1 spu_out (canonical
spu_render output), T2 cd_in (XA/CD-DA PCM entering the SPU CD bus),
T3 host (exact post-fade bytes handed to SDL_QueueAudio) — plus a 512k
event ring (every SPU reg write stamped with the SPU sample clock;
RENDER/PUMP_SKIP/UNDERRUN/MUTE/UNMUTE/CD_PUSH/SPU-DMA), audible-vs-
silent counters (-42 dBFS gate), queue watermarks, and a server-side
44100 Hz WAV dumper (rate exactness matters: snesrecomp da33c06).
- psx-beetle: the libretro audio callbacks stop discarding Beetle's fully-
mixed reference SPU PCM and record it into the same tap ring. Port 4380
gains audio_stats/audio_wav/audio_events with the identical wire protocol
as the runtime (rule 16 symmetric harness) — the oracle WAV capture.
- debug_server: audio_stats / audio_wav / audio_events on the runtime port.
- tools: audio_ab_diff.py + audio_spectral_ab.py (drift-tolerant offline
A/B: drift ppm, onsets, cents, timbre, click rate) and audio_analyze.py
(first-dirty-tap cross-tap decision tree) ported from snesrecomp/
_audio_round2; audio_capture_ab.py dumps both processes symmetrically.
Probes query history; nothing is armed at probe time.
Framework-level improvements distilled from an external Crash Bash fork of
psxrecomp, rebased onto current master. Two of the fork's five touched files
(code_provider.c stddef include, overlay_loader.c non-Windows atomic shim) are
already fixed upstream and were dropped; the CrashBash runtime target is
game-side and lives in the game repo, not here.
gpu.c — the core bug. gpu_read_gpustat()'s poll fallback raised IRQ_VBLANK+LCF
every 1000 GPUSTAT reads with no cycle gate. A title that busy-polls GPUSTAT for
GPU/DMA readiness (not just vsync) got ~38 extra "fake" vblanks/s — ~96/s
delivered vs the correct 60 — over-advancing the guest VSync frame counter and
jittering animation (measured on Crash Bash as character strobe). The recompiled
BIOS+game now carry per-block interrupt checks + cycle charging, so the tight
poll loops that once justified this hack already advance guest cycles and let the
one cycle-paced VBlank authority (interrupts.c) fire on schedule. The hack is now
OFF by default; PSX_POLLHACK_VBLANK=1 restores legacy for A/B. Also adds per-frame
GP0(E5) draw-offset-Y range instrumentation for double-buffer analysis.
debug_server.c — two debug commands: `vblank_rate` (cycle-paced raise/deliver
counts, poll-fallback raises, per-frame draw-offset range) and `dump_buffer`
(dumps a raw 512x240 VRAM band at a given display-Y so both double-buffer halves
can be inspected in one call).
main.cpp — two presentation improvements, both generic:
* sync-to-host-refresh: nudge the wall-clock pacer to the exact panel period
when the display is within ~2% of 60 Hz, so the pacer and SDL PRESENTVSYNC
stop fighting (a 59.94 pacer vs a 60.00 panel beats 2/3/1 = judder). ON by
default but only engages on ~60 Hz panels; other refresh rates keep 59.94.
* frame-blend masker: optional two-frame average for 30fps double-buffered
content, guarded to the plain software path. OFF by default (PSX_FRAME_BLEND=1).
Not yet compile-verified in this environment (no toolchain/ISO here); anchors,
symbol linkage (g_vblank_*/g_doff_*/g_pollhack_*, gpu_display_pixel_rgb,
png_write_rgb) and brace balance verified against master by inspection.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
While dev_any_input_enabled() (env PSX_DEV_INPUT, default ON), Player 1's pad
word is the active-low merge of the keyboard AND every plugged-in game
controller, so a tester can navigate P1 from whatever is connected with no
launcher setup. The P1 pad TYPE is unchanged — a launcher-assigned analog
DualShock still presents as analog, preserving its input path / SIO handshake
cadence (critical for the memcard scene repro). A P1 with no assigned device
presents as HYBRID so any controller/keyboard navigates. P2 keeps strict
per-port routing. Set PSX_DEV_INPUT=0 to restore single-device-per-port.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Ape Escape runs its own cooperative scheduler inside the kernel exception
handler (VBlank callback moves PCB[0]); handle the in-exception thread switch,
bump the IRQ-context ring to 4096 with a matching debug_server dump cap, and
carry the scheduler-escape trace state. WIP snapshot on the Ape branch (the
thread-smear investigation), preserved so the shared checkout can return to
master cleanly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
For a title whose controller type is fixed and auto-bound (e.g. Ape Escape =
DualShock analog), exposing a device/mode picker is pointless. New per-game
[controller] lock_device flag hides the Player 1/2 cards in the launcher
entirely (distinct from lock_mode, which only hides the pad-mode segment but
keeps the device dropdown). Plumbed config_loader -> RuntimeConfig ->
GameInfo.lock_device -> launcher model `device_locked` -> rml
`data-if="device_locked == false"` on the players cards-row. Default false, so
every other game is unchanged. The runtime still auto-binds whatever pad/keyboard
is present and presents the game's declared pad type.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds full 16:9/21:9 native-wide support for fully-3D titles (Ape Escape,
SCUS-94423) as a generic, opt-in, 4:3-byte-identical framework capability, plus
a native-wide GL compositor perf fix and a cutscene present-mode flicker fix.
Widescreen (generic, per-game opt-in via [widescreen*]):
- Shared screen-extent cull detector (runtime/include/ws_cull_detect.h) used by
all three backends (recompiler, dirty-RAM interp, sljit). Per-game signature
immediates ([widescreen.cull] screen_w_imms/screen_h_imms) — Ape's funnel
tests 0x181/0xF1 on its 368px display, not Tomba's 0x140/0xE0.
- Three funnel idioms widened: unsigned per-vertex sltiu (psx_ws_cull_sltiu),
signed min/max slti right edge + bare-bltz left edge (psx_ws_cull_slti/bltz,
new helpers + SLJIT_HLP appends), and billboard center+/-halfwidth. Explicit
[widescreen.cull] slti_sites for X-only funnels the auto-detector can't
qualify.
- gte_game_mode: GTE-activity gameplay detector for 3D titles with no sprite-tag
helper (gte.cpp notes every RTPS/RTPT). STICKY (>=3 verts, 45-frame hysteresis)
so sparse cutscene frames never flip to a 4:3 pillarbox = fixes intro flicker.
- nw_hud_corners: push outer-third screen-space HUD sprites to the true wide
corners in native-wide (else inset by the reveal).
- nw_backdrop: stretch a full-frame axis-aligned 2D backdrop quad to fill the
wide frame (for titles with a flat sky plane).
- Pattern-scanned interp/sljit widen hooks now gated per-game (auto_screen_x /
auto_backdrop): fixes an ungated auto_backdrop false-positive that rewrote a
live GPR = the ~24s native-wide crash on Ape.
Native-wide GL perf (fixes 16:9 frame drops):
- The wide surface centre equals the canonical 4:3 framebuffer, so wide_blit_center
copies canonical->wide-centre at present (both present paths) and the per-prim
mirror SKIPS prims fully inside the 4:3 frame (mirror_x/geo/batch_center_only).
Only reveal-margin prims re-rasterize. Peak frame time 38.8ms -> 17.5ms, no
vsync misses. Toggle gl_wide_fast; 4:3 untouched (all gated on g_wide_cur).
Runtime-only except the recompiler cull emit (regen needed for cull widening).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A title whose widescreen is unported/unvalidated (MMX4 at bring-up) had no
way to hide the launcher's EXPERIMENTAL Widescreen toggle — ws_eligible was
hardcoded true. Add game.toml [widescreen] offer (default true, runtime-only,
no regen):
- offer=false hides the launcher's Widescreen row and resets a stale
persisted 16:9 aspect back to 4:3 at launcher-model init;
- the runtime also clamps the display aspect to 4:3 after all config
sources (same completeness treatment as the lock_mode clamp), covering
launcher-less builds and hand-edited settings.toml;
- refresh_labels() no longer stomps ws_eligible every refresh — it is set
once from GameInfo.ws_offered at model init.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
lock_mode declares the game supports exactly ONE pad type (Tomba 2, and now
MMX4 — whose pre-DualShock libpad silently discards input from a pad
answering id 0x73). Hiding the launcher selector alone left two holes:
- launcher-less builds (PSX_LAUNCHER=OFF / skip_launcher) still honoured a
settings.toml p1_mode/p2_mode, booting the game into a pad type it
rejects;
- a settings.toml persisted BEFORE the game declared lock_mode fed the
stale mode back to the launcher as its locked_mode (ginfo.locked_mode was
read from the post-override p1_mode).
Capture the game-declared port modes at game.toml load and clamp
p1_mode/p2_mode to them after all config/settings sources apply. Verified
on MMX4: settings.toml pinning p1_mode/p2_mode=analog now boots with
slot 0 presenting digital (pad_status analog:false).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds a general, opt-in language selector to the RmlUi launcher, driven entirely
by game.toml [localization]:
[localization]
default = "en"
languages = [ {code="en", label="English"}, {code="off", label="Default"} ]
- config_loader: parse [localization].default + languages[] into RuntimeConfig;
add UserSettings.language with settings.toml [localization] parse + write.
- launcher: a "LOCALIZATION" panel in the Settings view (cycle toggle, matching
the Renderer/Screen-model idiom) shown only when the game declares languages.
The chosen code persists to settings.toml and rides UserSettings.
- text_xlate: new text_xlate_set_language() re-applies the language after the
launcher (init runs before it); "off"/"jp"/"" disable APPLY (native game).
- main: seed/read-back the language through the launcher and re-apply on every
boot path (so a settings.toml choice sticks even when the launcher is skipped).
- Settings view laid out in two columns (VIDEO | LOCALIZATION+AUDIO+CONTROLLER)
with SYSTEM full-width below, so it uses the horizontal space and fits without
scrolling; #root gains overflow-y:auto as a safety net.
Only games that declare [localization] get the menu (Tsumu today); byte-identical
for every other title.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
RmlUi launcher grows a Settings surface (renderer / supersampling / AA /
texture filter / screen model / skip-FMV / turbo / fullscreen / widescreen /
SPU-HQ / deadzone / BIOS / disc) and a live Controls rebind page. Bindings
persist to keybinds.ini through the new shared runtime module
runtime/include/psx_keybinds.h + runtime/src/psx_keybinds.c; main.cpp routes
SDL input through the rebindable table. Ape Escape is the first consumer;
the feature is game-agnostic framework code.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Implements the framework from docs/STRING_TRANSLATION.md: an always-on
capture inventory + a draw-time apply hook, both hung off the psx_dispatch
chokepoint via fntrace_record (no generated-code edits, no BIOS regen,
works in Release).
- text_xlate.{h,cpp}: EncodingProfile abstraction (shift_jis profile:
SJIS record reader with NUL/0xFFFF terminators + halfwidth-kana + control
framing; UTF-8 -> fullwidth-SJIS transcoder so English renders with the
font's existing fullwidth Latin, zero glyph injection). Capture inventory
(FNV-1a64 keyed) queryable over TCP; apply repoints the string-arg reg at
guest-stack scratch (KV-gated: only known records are swapped, stray
pointers untouched; own scratch is skipped to avoid recapture).
- fntrace.c: call text_xlate_on_dispatch() at the dispatch chokepoint.
- config_loader: [localization].language (+ [runtime].language alias);
env PSX_LANG overrides. Loads translations/*.toml per project.
- main.cpp: text_xlate_init() at game-config load.
- debug_server: "xlate" command (stats|dump|todo|reload) — observability
over TCP, no log files.
Verified: builds (Release), Tsumu boots headless, capture live
(distinct records accumulating), apply disabled until a table is present.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
debug_server_record_frame (every vblank) and the sio_state handler read
SIO_STAT/CTRL/RX via the GUEST-BUS sio_read(), which has hardware
side-effects: sio_tick(0) advances the delayed-IRQ state machine, reading
SIO_STAT decrements sio_ack_visible_reads and clears the ACK bit, reading
SIO_RX_DATA pops the RX FIFO. So the observer silently mutated the SIO/pad
handshake every frame in dev builds (the cardinal observability-alters-state
violation the event_ring header itself warns against).
Add side-effect-free sio_peek_stat/ctrl/rx_data() and use them on the debug
path; the guest bus keeps sio_read() with its real side effects. Also add an
fmv_state TCP command reporting the resolved config path + auto_skip_fmv +
frame-count-table params + live MDEC/XA/pad state, so the FMV-skip + config
state can be observed rather than inferred.
Note: this removes a real per-vblank SIO perturbation but is NOT the full fix
for the dev-build FMV-skip divergence (residual is the cycle-timing-core
class; see precise_irq_slice_state). It is correct observability hygiene.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Packed games can rewrite code over their own loaded EXE text after static recompilation. A dirty-page bit alone is too coarse for dispatch decisions: it prevents stale native execution, but it also cannot distinguish data writes in a text page from instruction bytes that actually diverged from the compiled image.\n\nRegister the boot EXE image as a reference and let memory writes mark text pages whose bytes differ from that image. Static game dispatch now asks whether the target remains native-safe; diverged pages fall through to execute live RAM through the dirty interpreter, while matching code bytes can still use the compiled path.\n\nConstraint: Runtime-installed or rewritten RAM code must execute the program's own instructions, not HLE or stale static code\nRejected: Keep the existing dirty-page-only gate | too coarse for packed/self-modifying main text and data writes sharing code pages\nConfidence: medium\nScope-risk: moderate\nDirective: Do not bypass dirty_ram_text_native_ok() at game compiled dispatch sites without proving RAM still matches the compiled image\nTested: Local Tekken 3 branch using this mechanism reaches playable fights; clean upstream worktree compiles changed main/memory/dirty files before hitting an unrelated existing macOS overlay_loader LONG/InterlockedExchange build error\nNot-tested: Full upstream macOS build completion; non-macOS build matrix
Three root-caused fixes (verified with guest-frame-aligned VRAM diffs vs the
software reference + window captures; Tomba2 title now pixel-identical to
software, attract within 0.62% of the GL oracle, 60.5 fps sustained):
1. Boot wedge: every GP0 pixel upload did 2 vkAllocateMemory + 2
vkQueueSubmit (driver churn, minutes-long stall, watchdog abort).
Deferred batched uploads (GL model): CPU writes land in the CPU array +
dirty rect; one batched upload flushes at each GPU-op entry point.
2. Shredded 3D: draws raced CPU rewrites of the persistently-mapped vertex
buffer from offset 0 (69.5% pixel divergence at guest frame 3000).
Sub-allocation cursors + firstVertex bases, recycled at queue-idle.
3. Semi-transparency order violations (59.5% divergence): VK still had GL's
retired whole-batch STP split; ported GL's current model (ordered color
pass + color-masked stencil fixup via a new pipeline dimension; semi
prims isolated one-per-batch).
main.cpp: Vulkan present block composes 24-bit FMV frames via
gpu_display_pixel_argb + vk_renderer_present_cpu (was NULL -> guaranteed
black video; on-screen verification still owed) + the no-VK-build
downgrade-to-GL guard salvaged from the retired _wt-vulkan worktree.
debug_server screenshot_file now syncs the VK backend (previous commit).
Known follow-ups (ENHANCEMENTS.md R2): port the exact-rect upload fix from
GL (VK carries the union model = same flicker class), verify FMV
on-screen, cache the FMV staging image, explicit DS barrier between
one-shot submits, wide compositor, SSAA validation.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- [video] fmv_skip_no_xa: broaden FMV detection to MDEC activity alone.
Silent RAM-preloaded movies never stream XA, so the MDEC+XA detector
missed them and no skip mechanism was even attempted. Per-game opt-in.
- tools/host_thread_probe.py: suspend/sample RIP+RSP of every host
thread of a live or wedged runtime, stack-scan return addresses into
the main exe, addr2line-symbolize (RelWithDebInfo). Diagnosed the
fatal-halt-serve loop in minutes; use whenever a runtime 'freezes'.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
GetID's last four response bytes were hardcoded 'SCEI' (NTSC-J). Real
hardware reports the region of the INSERTED disc. Kula World uses the
BIOS CD API, and the kernel CD driver revalidates the disc (ReadTOC +
GetID, polled with irq_enable=0x18) when it reinitializes after the
game's first file load. Seeing a 'Japanese' disc, it fell into the
endless GetStat/Init retry loop: CD delivery froze at sector 307 (LBA
318, just before the first level directory) and the game waited forever
on a black screen.
Diagnosed from the always-on cdrom trace ring (grabbed via gdb from the
live process): 307 sectors stream cleanly, Pause completes, then the
BIOS's own ROM CD firmware (0xBFC0Dxxx) masks INTC delivery, issues
ReadTOC + GetID, reads the response... and rejects it. Beetle PSX (the
oracle; boots the same dump + BIOS to demo gameplay) derives the SCEx
string from the disc's SYSTEM.CNF boot serial - so we now do the same:
cdrom_set_disc_scex() is fed at launch from the existing disc_identity
module (PAL->SCEE, NTSC-J->SCEI, NTSC-U->SCEA; default SCEA matches the
one supported BIOS). Verified so far: launch log reports 'disc region
PAL (serial SCES-01000)'; the sector-counter wall-break run is in
flight. Framework-wide note: Tomba/MMX6 (NTSC-U, own CD drivers) never
hit this path but now get 'SCEA' - matching real hardware - instead of
'SCEI'; regression-test before any master merge.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdjxoRB1f9VQsNzZayCgGS
The cycle-accuracy emitter emits calls to 12 timing runtime functions
(psx_cyc_load_*, psx_icache_fetch, psx_muldiv_*, psx_gte_*, psx_slice_block,
...) + debug_server_cyc_observe + g_debug_last_store_pc. Overlay DLLs never
got these symbols, so every overlay failed to link -> 0 native shards ->
100% interpreter (unplayable). Fix (option A): expose the 12 via the overlay
callback ABI.
- overlay_api.h: +12 timing callbacks in OverlayCallbacks, ABI 8 -> 9.
- overlay_loader.c: populate the 12; add warn_on_cgtag_mismatch() (loud
warning when shards exist under the wrong cg-hash).
- compile_overlays.py: 12 preamble forwarders + local g_debug_last_store_pc
+ -DPSX_NO_DEBUG_TOOLS.
- cosim.c/main.cpp: input-cosim setpad + held pad (deterministic input).
- cosim.py: env-overridable COSIM_EXE/CWD/GAME.
KNOWN REGRESSION (patched on a follow-up branch): Tomba pause menu (triangle)
wedges. Root-caused: an IRQ taken mid-block resumes at the function entry
leader instead of the interrupted PC, re-zeroing the menu's VSync wait
counter (scratchpad 0x1F8001E8) every frame so it never reaches its target.
Not the scheduler (LLE reproduces) and not timing math -- interrupt-resume
re-entry.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New always-on ring (device_trace.{c,h}, compiles into both binaries like
parity_trace) recording every hardware IRQ-raise edge with its guest cycle,
source (I_STAT bit) and detail. All native device sources funnel their raise
through one choke point psx_irq_raise(bit,detail) in interrupts.c (VBLANK/GPU/
CDROM/DMA/timers/SIO routed; DMA passes DICR channel flags, timers pass index),
so the device-timing audit sees every event from one place. devtrace_dump/ctl
debug commands (cycle-window + source filters) + tools/devtrace_diff.py align
the two device-event timelines by source ordinal and flag where cycles drift.
The Beetle oracle side (mednafen IRQ_Assert rising-edge hook) lives in the main
checkout.
This is the level below parity_trace: parity names the divergent control-flow /
gate input; device_trace names the divergent HARDWARE EVENT behind it.
First result (MMX6 cutscene asset-load over-fast wedge): the load's CD+DMA
activity is compressed into ~232M cycles on native vs ~384M on Beetle (~1.65x),
completing the func_8001CB3C scene setup ~100M cycles / ~176 frames early. The
producer is CD-read / DMA-completion timing (DMA channels 2/3/4 = GPU/CD/SPU
asset streaming) -- confirming the device-completion-latency hypothesis and
ruling in the CDROM/DMA cycle model as the over-fast subsystem. (VBlank verified
correct: native VBlank edges are exactly 564480 cyc apart.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
End the "somewhere upstream" blindness with a reusable first-divergence
instrument (PRINCIPLES.md "find the first divergence; is the value WRONG or the
behavior MISSING"). A TCB-scoped, freeze-on-trigger control-flow ring with an
identical struct + JSON wire format on both processes, so a generic sequence-diff
(tools/parity_diff.py) locates the first logical divergence between recomp and
oracle — not a frame diff.
- parity_trace.{h,c}: armed-from-boot ring, gated on current_tcb==watched_tcb,
latches on a configured trigger target so the pre-divergence window survives
the wedge loop. Up to 6 watch-word snapshots/row. Self-contained TU shared by
both binaries; frame source decoupled via parity_host_frame().
- fntrace.c: per-dispatch producer hook (cheap armed gate on the hot path).
- traps.c: YIELD control-event hook in the scheduler switch.
- main.cpp: env-armed from boot (PSX_PARITY_TRACE=1; TCB/trigger/watch tunable),
defaults target the MMX6 cutscene wedge.
- debug_server.c: parity_dump + parity_ctl commands.
- tools/parity_diff.py: pulls both rings, sequence-diffs by (kind,pc,target).
First use (MMX6 freeze) confirmed MISSING behavior, not a wrong value: native
thread1 is permanently parked at func_8002000C resume (TCB EPC 0x800200F4),
handshake flag 0x8006D9AC never set (0 across 16384 rows), setter func_8001344C
never runs; thread1 re-resumes a dead func_8002000C frame whose saved-ra slot was
reused -> jr 0x800CD3F8 wedge. Upstream context-staleness/scheduling root.
Beetle mirror producer + the cross-process diff are the next step.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the non-deterministic host-fiber thread-switch bridge with a deterministic
TCB scheduler (HLE_SCHEDULER_CARVEOUT_PLAN.md steps 2-8, minimal "A+" scope):
- psx_scheduler_run (traps.c): outer trampoline wrapping the top-level dispatch
(main.cpp calls it instead of bare psx_dispatch). setjmp(g_scheduler_jmpbuf);
restore current TCB / dispatch / on-escape reset g_psx_dispatch_depth +
g_psx_call_bail; boot/pre-TCB runs raw cpu->pc; one-level switch-back safety
net; fail-closed psx_is_dispatchable on every restored EPC.
- psx_request_thread_switch: save yielder ctx (resume=$ra), repoint
dword_108->entry, longjmp to the scheduler. No fibers, no cpu->pc=0 signal.
Fires only with in_exception==0, so it never unwinds a live exception_jmpbuf
frame (the in-exception RFE path stays LLE/intact).
- sched_escape_ring (step 8) for always-on observability.
- Hidden toggle PSX_HLE_SCHEDULER (default 1=HLE, 0=legacy fibers), read once;
psx_change_thread branches on it; startup banner prints the mode. The
trampoline is transparent in LLE mode. Fiber funcs kept compiled-but-dead.
Validated live (HLE): thread1 trace shows ONLY the new TCB-switch events and
ZERO fiber kinds 10/11/13 — the per-frame fiber recreate is eliminated. Builds
clean. Freeze NOT resolved: same 0x800CD3F8 wedge persists (separate upstream
root, per ChatGPT Q5). Keep HLE default until the upstream cause is fixed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New runtime/src/psx_icache.c: direct-mapped (4 KB / 256-line) instruction-
cache fetch cost, transcribed from Beetle PS_CPU::ReadInstruction:
HIT (TV == addr) -> +0, no give-back clear
MISS, KSEG1/uncached -> +4 (BIOS ROM)
MISS, cached (KSEG0/KUSEG) -> +3, then +1 per word refilled from the
missing word to the line end (earlier words
stay invalid); also clears the load give-back.
Mirrors only the per-word TV tag array (cost + tag state; the value is already
in hand). Wired into the dirty-RAM interpreter (exec_one) per instruction,
charged BEFORE the §1 base — Beetle order.
New ruler #2 loop `icache_miss`: loop top + a victim 0x1000 apart alias the
same direct-mapped line with different tags, forcing a refill miss every
iteration (other loops are all-hits after warm-up).
VALIDATED interp vs Beetle (PSX_FORCE_INTERP=1 PSX_ICACHE=1): icache_miss
native +14 == Beetle +14 (per-iter 17); the other 12 loops unchanged at +0
fetch. So the I-cache hit AND refill-miss costs are MEASURED equal to the
oracle on the interp path.
Opt-in via PSX_ICACHE=1 (default OFF) — charging fetch cost in only one
backend would fork mixed compiled/interp timing. Production activation +
the compiled-path emit (per cache-line leader) + ruler #1 cold-spike
validation are Stage 2. Default-off => no production change (the interp's
fetch call is a no-op without the env).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implements the last unmodeled load-path cycle component — the R3000A
load-delay pipeline interlock (Beetle ReadAbsorb/ReadFudge/LDAbsorb/
LDWhich) — as ONE shared per-instruction model across the dirty interp
and BOTH static emitters, replacing the flat per-instruction +1 base and
the flat +4 main-RAM read wait-state.
Model (transcribed from beetle-psx cpu.cpp + libretro.cpp, verified Δ):
- §1 base: a pending load's per-register give-back (read_absorb) is
consumed INSTEAD of charging +1 (pipeline write-back overlap).
- GPR_DEPRES: each opcode's src/dst regs end the give-back on use.
- DO_LDS: commits the prior load's give-back + ReadFudge.
- load ReadMemory: +2 fudge iff the predecessor committed no load;
main-RAM region +3 + completion +2 (LWC2 +1) become the LDAbsorb
give-back; scratchpad +0.
New runtime/include/psx_cyc.h (inline §1/deps/DO_LDS + step) over new
CPUState timing fields; psx_cyc_load_*/psx_cyc_lwc2_read in memory.c
(region-aware, own the whole load interlock, value via uncharged
psx_read_*); psx_cyc_dep_res_mask in psx_instr_cost.h (per-opcode
GPR_DEP/RES, gen-time literal). cpu->read_* rewired to uncharged in
main.cpp; charge_main_ram_read removed. Wired before each instruction
body in code_generator.cpp (game) and full_function_emitter.cpp +
strict_translator.cpp (BIOS); loads route through psx_cyc_load_*.
Validated against the Beetle oracle:
- ruler #2 load2: native +10 -> +11 == Beetle (all other components
still exact: alu+1/load+5/div+38/mult+15/gte_rtps+15/gte_nclip+8).
- ruler #1 [0x80001C5C->0x80001CA4]: 54 -> 56 == Beetle steady-state
(residual 84/77 spikes = I-cache cold refill, the separate P2 axis).
- Tomba 2 boots to the intro FMV (pixels on screen, no regression).
Follow-up (separate commit): GTE-read (MFC2/CFC2) + MFC0 LDAbsorb
give-back, and muldiv-stall give-back consumption — needed for mixed
game/FMV faithfulness but do not affect the rulers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The multi-week Whoopee-Camp logo->FMV stall was a TIMING-FAITHFULNESS bug, not
game logic: the static recompiler charged cycles block-granular and a branch
whose delay slot is a SEPARATE block leader (branch at end_addr, delay slot
outside the block) undercharged by one cycle on every execution -- the
always-executed delay-slot clone was emitted but counted by neither the branch
block nor the (unentered) delay-slot block. ~8 such sites on the boot init path
= the measured -8 cycle drift that made native run behind the interpreter, so
the frame-1824 elapsed-Timer1 logo-delay loop exited after ~42 iterations
(native) instead of ~1557 (interp/HW) and never reached the FMV.
Fix (code_generator.cpp translate_basic_block, the faithful class fix -- every
dynamically executed instruction charged exactly once):
block_exec_cycles = instruction_count
+ (exit branch at end_addr with delay slot outside ? 1 : 0)
applied to both the block cycle charge and the slice budget. R3000 is MIPS-I so
the delay slot is unconditional; the not-taken path already falls through to
branch_pc+8 (no double-count). Result, screenshot-verified: native passes the
logo-delay loop, screen animates, CDROM+DMA stream active, the jungle intro FMV
plays.
Also in this checkpoint (all on wt/tomba2, not merged/pinned):
- PARKED precise interrupt-slicing (PRECISE_IRQ_SLICE.md): root-caused its pc=0
boot exit to a mid-function clean-text resume PC the dispatcher can't re-enter;
ChatGPT-validated fix is "every block leader = re-enterable CPS continuation"
(psx_game_is_function_entry predicate added; precise_pc_dispatchable +
irq_taken-once restructure in psx_run_precise). Default OFF (PSX_PRECISE_SLICE=1
to resume); inert on the baseline. Slice-trace diagnostics added to freeze_check.
- FAITHFUL_TIMING_PLAN.md: authoritative game plan (P1-P6) + standing directive
(build the faithful core, never hacks; ecosystem is being built, not preserved);
referenced from CLAUDE.md Rule -1.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
In-progress investigation work snapshotted before rebasing onto the
tcc-refactored psxrecomp master. Includes overlay ABI v8
(check_interrupts_at resume-PC callback for async RFE recovery) and
code_generator/full_function_emitter delay-slot cycle accounting.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
For a game that supports exactly one pad type, the launcher's 3-way pad-mode
selector (Hybrid | Analog | D-Pad) offers modes that don't work. New
game.toml [controller] lock_mode = true hides the whole selector and forces
every port to default_mode, so the player can't pick a broken mode.
Tomba 2 is the first user: its driver only works as a plain digital pad
(the DualShock config-mode handshake emulation is incomplete — see sio.c
pad_supports_config / commit 1141f62), so analog/hybrid re-trigger the
phantom-input bug. lock_mode supersedes allow_hybrid (which only hid Hybrid).
Plumbing mirrors allow_hybrid: config_loader (controller_lock_mode) -> main.cpp
(ginfo.lock_mode/locked_mode) -> launcher (m.mode_selectable + force p1/p2 mode,
overriding stale settings.toml) -> launcher.rml (data-if="mode_selectable" on
both .seg selectors). Verified: compiles in a PSX_LAUNCHER=ON build (build-t2-launcher).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Tomba 2's title menu was unresponsive and the idle attract demo never
started. Root cause (proven against the Beetle oracle): the SIO pad
state machine answered the DualShock config-mode commands (0x43 enter-
config, 0x4D rumble-map, etc.) for EVERY pad, including a plain digital
controller. Tomba 2's pad driver probes the controller with 0x43 every
frame to detect a DualShock; on real hardware a digital pad (SCPH-1080,
id 0x41) ignores 0x43 (hi-z, no ACK), so the driver classifies it as
digital-only and just polls with 0x42. Our SM answered 0x43, so the
driver went down the DualShock config path and read the 0x00 config-
response data bytes as button data -> phantom "all buttons pressed"
(pad word 0x0000). That made the menu's edge-detected navigation see
nothing newly-pressed (unresponsive) AND reset the idle/attract timer
every frame (no attract).
Fix: model a plain digital controller. New per-slot pad_supports_config
flag (sio_set_pad_config_capable), set from the player pad mode
(PAD_MODE_DIGITAL => 0, ANALOG/HYBRID => 1, default 1). When 0, all
config-command branches (0x43 / legacy / modern 0x44-0x4F) fall through
to the hi-z "no response" path, so a digital pad answers ONLY the 0x42
poll -- byte-for-byte matching the oracle.
Validated live: SIO now shows only 0x42 polls (id 0x41, buttons 0xFFFF),
idle pad word is clean (no 0x0000 phantom), every injected button reaches
the menu var, and START navigates the title menu and starts the game.
Gated so only DIGITAL-mode pads change behaviour; HYBRID/ANALOG pads
(MMX6, Tomba 1, Ape) are unaffected (config-capable as before) -- but a
T1/MMX6/Ape regression pass is still required before any master merge.
NEW failure point (next bug in the chain, now reachable): with input
fixed, the idle attract demo triggers and loads a gameplay level, which
FREEZES -- guest spins in compiled kernel/dispatch code, dirty-interp
idle (insns_run flat), vblank stalled, level textures corrupted. This is
the untested gameplay-overlay stage, a separate issue.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
When the top-level psx_dispatch returns with cpu->pc==0 (the guest published a
null PC — abnormal boot/run exit), shutting down loses the live state. With
PSX_EXIT_HALT set, halt-and-serve instead so the still-loaded overlays + full
guest state are inspectable over TCP. Default off; zero impact on normal runs.
Used to pin the Tomba 2 frame-1997 exit: the guest's hooked interrupt handler
(installed via B0:0x19 HookEntryInt) calls B0:0x17 ReturnFromException
(overlay thunks at 0x80086200/0x80086220); the EPC-sentinel + host-longjmp
exception model resolves the RFE to cpu->pc=0. See docs/TOMBA2_WHOOPEE_BAIL.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
OVERLAY_REGION_FLOOR was hardcoded to 0x98000 = Tomba 1's main-EXE text end
(0x10000 + 0x88000). Tomba 2's boot EXE is only 0x28800 (text ends 0x38800),
so its disc overlays load at 0x80085000+ — inside [0x38800, 0x98000), which the
hardcoded floor wrongly classified as statically-recompiled main-EXE text. That
forced every Tomba 2 overlay onto (a) slow block-by-block dispatch and (b) the
bail-prone dispatch_nonlocal_call contract path; when the per-frame interrupt
pump fired mid-contract it cascaded into the $ra=$sp=1 bail storm = the
Whoopee-Camp splash freeze.
Make the floor a runtime value (g_overlay_region_floor), pinned at game load to
(load_address + text_size) & 0x1FFFFFFF. Defaults to 0x98000 for BIOS-only;
evaluates to the same 0x98000 for Tomba 1 (no behavior change), and is
correct-by-construction for every game.
Resolves: bail storm gone (bail_first=0); splash advances at 60 fps past the
old frame-1823 wedge. NEW failure point exposed next: dispatch_depth desync on
the now-local-flow overlay path (fixed in the following commit).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two framework bugs that made overlay variants run interpreted forever (the
"agonizing lag" in overlay-heavy areas like Tomba's dwarf village):
1. Cache read/write divergence. The loader reads <exe>/cache and the runtime
writes captures to <exe>/overlay_captures.json, but the gcc autocompile path
used the game.toml command verbatim, whose hard-coded --out-dir/--captures
pointed elsewhere (build-stable/build-modern). So compiled shards landed where
the loader never looked and captures were read stale -> the variant never went
native. Fix: the runtime now injects PSX_OVERLAY_CACHE_DIR / PSX_OVERLAY_CAPTURES
(the canonical loader paths) into every autocompile spawn; compile_overlays.py
honors them over any CLI flag (--captures now optional). One source of truth for
the cache location, uniform across all games, dev or prod. gcc/tcc stay separated
by the existing <compiler> namespace.
2. Overlay DLL preamble missing psx_ws_cull_sltiu. auto_screen_x emits this helper
at every width-cull site into EVERY build's C (identity at 4:3), and the main EXE
links it from gpu.c, but the overlay DLL preamble never defined it -> any overlay
variant with cull sites failed to link -> no shard -> interpreter. Fix: provide
the self-contained helper (depends only on the already-forwarded psx_ws_x_margin)
in the DLL preamble.
Proven on Tomba dwarf village: dispatch_native 29/s -> 11454/s (was ~97% interp,
now ~97% native).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Matthew Stanley