The runtime's autocompile spawns `cmd.exe /C python tools/compile_overlays.py
...`; whichever python PATH resolves runs it. On a machine where that is an
MSYS/Cygwin-flavored interpreter (hit live 2026-07-15: devkitPro's MSYS2
python), every gcc shard compile died exit-1 with EMPTY stderr, zero shards
built — and before the observability suite this looked identical to a clean
run.
Root cause (proven by A/B test): _toolchain_env prepended the toolchain dir
to the child PATH as a Windows path ('C:/msys64/mingw64/bin') — under an MSYS
python, PATH is a ':'-separated POSIX list, the drive colon reads as a
separator, and the whole child PATH is mangled, so cc1/as/ld/collect2 lose
their DLLs and die before printing anything. POSIX-prepend rc 0,
Windows-prepend rc 1, same compile.
- _toolchain_env: prepend the toolchain dir in the RUNNING interpreter's
path flavor (X:/... -> /x/... when os.sep == '/').
- native_path (new single chokepoint for compiler-bound paths): absolute,
interpreter-flavor. Documents why flavors must NOT be translated here:
the msys runtime translates POSIX args at the exec boundary with its own
mount table (verified via gcc -v), while a PATH-resolved foreign cygpath
(e.g. Git-for-Windows') corrupts them.
- compile_dll: when the compiler produces no output at all, say so
("gcc likely failed to launch", with the resolved toolchain dir) instead
of printing an empty COMPILE ERROR.
Verified under the exact failing interpreter (devkitPro MSYS2 python,
Tomba SCUS-94236): before = 0 ok / all failed, empty errors; after =
17/17 shards ok, exit 0; the live runtime's next autocompile run loaded
cg5 shards ("-> 122 candidates") and the cache-hash-mismatch banner cleared.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The tier mis-compiled and has been hard-gated off since 2026-06-25; overlay
gaps fall to gcc/tcc shards and the dirty-RAM interpreter, unchanged.
Re-do on master of the removal validated on the stale branch (6da8c02),
reconciled with everything master grew since the fork:
- Delete overlay_sljit.{c,h}, overlay_compile_worker.{c,h}, lib/sljit,
runtime/tests/sljit_*.c, and the runtime.cmake sources/include-dirs.
- Strip the sljit provider/backend from code_provider.{c,h},
overlay_backend.{c,h} ("sljit" in overlay_backend/PSX_OVERLAY_BACKEND now
degrades to auto so stale configs still boot), main.cpp, debug_server.c
(sljit_status/sljit_async/sljit_try TCP commands), overlay_loader.{c,h}
(live policy, JIT-on-miss, persisted-shard cache, async worker glue,
psx_sljit_call/cop2/memx host helpers — reachable only via the removed
cpu->sljit_helpers table), and cpu_state.h (SLJIT_HLP_* enum + the
sljit_helpers[] member; CPUState layout change reshards via the cg tag).
- Master-only entanglement resolved before deletion: 28165d6's Einhander WS
player-X bound emit existed in three paths (gcc code_generator, sljit,
dirty_ram_interp). The gcc shard link contract now forwards
psx_ws_player_x_bound (ABI v13, previous commit), so deleting the sljit
emit path loses no widescreen behavior; ffc6688's game-dispatch block
inside the dead psx_sljit_call helper goes with it.
- Reword comments that described sljit as a live backend (gpu.{c,h},
ws_*_detect.h, cosim_state.c, dirty_ram_interp.c, autocompile.h,
boot_state.h, config_loader.h, compile_overlays.py, and the CPS marker
comments the recompiler emits).
recompiler psxrecomp-game builds clean; game runtimes rebuilt + boot-verified
separately.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Re-land of the shard-observability suite from 6da8c02 (stale branch), merged
against current master (keeps master's overlay_ext() platform suffixes,
recompiler_patch hash sources, and the unconditional v12 rfe forwarder).
- compile_overlays.py: ShardStats tallies every shard outcome — per-shard
"SHARD FAIL [class]" lines, a SHARD BUILD SUMMARY, a machine-readable
"PSX_SHARD_RESULT ok=N failed=M skipped=K" line, and a non-zero exit when
any shard that should have built failed (previously a run that built zero
shards exited 0, indistinguishable from a clean run).
- New --check preflight: build every shard into a throwaway dir and report
pass/fail — answers "would this game's shards compile against the current
recompiler + headers?" without a runtime.
- Retain audit-failed interior-fragment C (<phys>_<interior>_fragment_FAILED.c)
for triage, plus a persistent doomed-interior memo (interior_fail_memo.txt)
keyed (phys,interior,region_crc) so autocompile stops re-walking
data-as-code fragments every cycle (~90x off the V8 fragment pass).
- Extract DISPATCH_PREAMBLE to runtime/include/overlay_dispatch_preamble.c.inc
(single source of truth for the shard link contract, loaded by
load_dispatch_preamble(); missing file is a hard error).
- codegen_hash_sources.cmake: fold the shard COMPILE surface (overlay_api.h,
cpu_state.h, the .inc) into the cg tag so header/ABI/link-contract drift
reshards same-tree instead of silently breaking every new shard compile.
- autocompile.c: parse PSX_SHARD_RESULT from the child-output tail, expose
shard_ok/shard_fail/shard_skipped/shard_fail_total over autocompile_status;
ALWAYS rescan on completion so shards that did build load even when the
run also reported failures.
Live proof on the stale branch: Tomba's first autocompile surfaced 9 shard
link failures (undefined psx_rfe_mark_escape — since fixed by v12) and V8
surfaced 91 benign data-as-code fragment failures, all previously silent.
Codegen hash changes => every title reshards once on next build (expected;
loader rejects/regenerates stale shards).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
cdrom_bursts spans lines and the server aborts a second request per
socket; parse by raw_decode accumulation and reconnect per command
(debug_client.py convention).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The emitter emits psx_rfe_mark_escape() at every rfe — including in
overlay-resident exception-context code — but the dispatch preamble
never provided it, so any rfe-containing overlay failed to LINK
(undefined reference) and ran interpreted forever: the gte_read_ctrl
silent-coverage-blocker class again (11 of Tomba's vault shards hit
it). It mutates shared RFE-pending escape state (interrupts.c), so the
shim forwards unconditionally through OverlayCallbacks; ABI bump to 12
rejects pre-shim caches.
Found live during LOAD_TIME_ZERO E0 bring-up: the user's first pig
load triggered autocapture and the compile died on this link error.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Passive ring consumer: samples freeze_check series while the game
free-runs, then locates completed cdrom_bursts inside the span and
reports per-burst guest cycles / effective multiplier vs 1x /
phase_profile shares / phase_hot + dirty_ram per_pc deltas. No arming,
no pause/step.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Triage of Tomba2Recomp GH issue #1 exposed three ways a release exe
behaves differently on the build machine than on every user machine:
- DEFAULT_BIOS_PATH baked an ABSOLUTE build-tree path into the binary,
so promoted/release exes silently load the builder's BIOS wherever
that path exists and the clean-install picker flow is never
exercised where releases are validated. runtime.cmake now refuses
absolute values (loud configure warning, converts to the relative
default); dev checkouts keep prompt-free boots via a new
resolve_bios_path rung that walks exe-dir ancestors for
psxrecomp-v4/<relative>, a layout only dev trees have.
- A settings.toml with a TOML syntax error was silently discarded
wholesale (user's renderer choice, BIOS/disc paths - everything),
and the next launcher save then overwrote the file with defaults.
load_user_settings now reports parse_error; the runtime preserves
the broken file as settings.toml.bad and warns via message box +
stderr.
- package_release.ps1 gains guards: fail packaging if the staged exe
contains a drive-letter absolute SCPH1001.BIN path, if the stage
carries BIOS/disc/save/sidecar files (SCPH*.BIN, *.cue, *.iso,
*.mcd, bios.cfg, disc.cfg, settings.toml, keybinds.ini,
overlay_captures.json), or if saves/ is non-empty.
Verified: rebuilt Tomba2 against this tree - configure warning fires
on the absolute arg, strings scan of the exe finds no drive-letter
BIOS path, and a boot with no [bios] setting and no bios.cfg resolves
the BIOS through the psxrecomp-v4 dev rung.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The offline cache build ran one capture at a time - classify, recompiler
subprocess, patch/audit, gcc - so a full coverage-vault rebuild (Tomba 2:
1293 captures) took hours. The wall clock is subprocess-dominated and the
subprocesses release the GIL, so a ThreadPoolExecutor across captures
fills the machine: measured ~24x on the capture-reprocess phase
(6/min -> ~143/min, 24 concurrent recompiler/gcc children).
Concurrency contract:
- cov_lock guards the per-region dedup coverage sets. The
check->build->update window is deliberately unlocked: concurrent
captures may build overlapping DLLs - redundancy, never corruption
(the loader content-matches every function by (entry, code_crc)
across all DLLs at a region).
- A per-(region, crc32) key lock serializes captures of IDENTICAL bytes,
which share one dll/ranges filename: the second sees the first build
(dll-exists skip / prior-ranges merge) exactly as it would sequentially.
- The interior-fragment pass still runs after all captures, reading the
final on-disk entry coverage (unchanged failure-domain decoupling).
- Per-thread stdout capture prints each capture log as one atomic block.
- --jobs 1 and --static take the exact sequential path.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The controller map had no l3/r3 rows at all, so pad bits 1/2 could never
be produced by a gamepad — DualShock-only games (Ape Escape) were
unplayable. Keyboard slots existed but shipped unbound.
- main.cpp: PAD_L3/PAD_R3 bits; controller_map 14->16 entries; defaults
l3=leftstick, r3=rightstick; input.ini template + doc comment updated.
- psx_keybinds.c: keyboard defaults L3=T, R3=Y (continue the Q/W/E/R
shoulder row). Launcher Controls page picks them up automatically.
- beetle_main.cpp: same T/Y keys on the oracle so psx-beetle stays
drivable for L3/R3 games.
- tools/package_release.ps1: shipped input.ini template gains l3/r3.
- README: keyboard + controller map tables.
Wire is unchanged: sio.c transmits the button word verbatim, matching
Beetle's dualshock.cpp. Validated in-game on Ape Escape (user-confirmed
R3/L3 inputs on a real pad).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two hardware-faithfulness fixes found by the always-on tap rings + Beetle
oracle A/B on fingerprint-matched attract music (tools/audio_pair_segments):
the recomp played the identical track at -3.8 dBFS vs the oracle's
-14.1 dBFS — +10.3 dB hot, clipping continuously.
1. spu.c: the voice repeat/loop address register (voice reg 7) is LIVE
state on real hardware — writing it after KEYON retargets where the
next END+REPEAT block jumps (Beetle spu.cpp:1150/333), and the block
auto-latch (flag bit 2) updates the guest-visible register (386/1302).
The recomp cached repeat_addr at KEYON only, so X5's driver idiom —
KEYON, then point the loop register at a silent tail block; no KEYOFF
ever — left voices looping the LOUD sample body forever. Measured: 3
stuck voices re-looping ~1400x/s, 160k SPU events/min vs the oracle's
633, zero game KEYOFF writes (the game never needs one), +11 dB.
2. cdrom.c: implement the CD controller's decode volume matrix
([src][port], 0x80 unity, pending banks + apply latch at 1F801803.3
bit 5 — mapping copied from Beetle cdc.cpp:1483-1501, applied after
the XA resample per PS1 hardware tests). X5 drives it constantly:
0x73-0x7E steady, ramped to 0x00 at scene changes (measured via the
new beetle `cdc_volume` command + PS_CDC::PSXRecomp_GetDecodeVolume
peek). Without it music fades never happened and steady level sat
~0.6 dB hot.
Also: AUDIO_TAP_VOICES (T0 voice-sum tap) for source-by-source mix
decomposition, per-tap WAV rates, bridge stats in audio_stats.
Measured after (MMX5 attract, 20-25 s windows):
spu_out RMS: -3.8 dBFS (clipping) -> -15.5 dBFS [oracle -14.1]
music-only chain already matched the oracle at -14.8 dBFS, confirming
decoder + mixer gain equivalence; the entire excess was stuck voices.
underruns: 2 events / 98 s (delivery fix holding).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Measurement infrastructure for the audio-quality campaign (MMX5 complaint),
ported from snesrecomp's audio_trace/oracle methodology and the
_audio_round2 tap model. No behavior change to the audio path itself.
- audio_trace.{c,h}: three always-on 95 s PCM rings — T1 spu_out (canonical
spu_render output), T2 cd_in (XA/CD-DA PCM entering the SPU CD bus),
T3 host (exact post-fade bytes handed to SDL_QueueAudio) — plus a 512k
event ring (every SPU reg write stamped with the SPU sample clock;
RENDER/PUMP_SKIP/UNDERRUN/MUTE/UNMUTE/CD_PUSH/SPU-DMA), audible-vs-
silent counters (-42 dBFS gate), queue watermarks, and a server-side
44100 Hz WAV dumper (rate exactness matters: snesrecomp da33c06).
- psx-beetle: the libretro audio callbacks stop discarding Beetle's fully-
mixed reference SPU PCM and record it into the same tap ring. Port 4380
gains audio_stats/audio_wav/audio_events with the identical wire protocol
as the runtime (rule 16 symmetric harness) — the oracle WAV capture.
- debug_server: audio_stats / audio_wav / audio_events on the runtime port.
- tools: audio_ab_diff.py + audio_spectral_ab.py (drift-tolerant offline
A/B: drift ppm, onsets, cents, timbre, click rate) and audio_analyze.py
(first-dirty-tap cross-tap decision tree) ported from snesrecomp/
_audio_round2; audio_capture_ab.py dumps both processes symmetrically.
Probes query history; nothing is armed at probe time.
Games that discard their boot/init code load gameplay overlays OVER the
main-EXE text window (Tomba 2: hot code at 0x1DA88/0x24548/0x3116C, below
its 0x38800 floor). Post-baseline, a DIRTY page in [KERNEL_WINDOW_END,
FLOOR) is by definition live overlay code the compiled image is stale for,
so make overlay_cache_window_contains() admit dirty boot-text pages as a
capture/candidate window instead of forcing ~300ns/insn interpretation.
Auto-capture convergence: replace the fixed trigger gate with
insn-pressure sizing and exponential futility backoff — retry only when
the manifest signature or loader candidate count actually changes, so
coverage stops churning once it has converged. compile_overlays.py moves
to entry-based coverage; autocapture.c escapes >=0x80 cp1252 bytes from
child-compile output so the status JSON stays valid UTF-8.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds full 16:9/21:9 native-wide support for fully-3D titles (Ape Escape,
SCUS-94423) as a generic, opt-in, 4:3-byte-identical framework capability, plus
a native-wide GL compositor perf fix and a cutscene present-mode flicker fix.
Widescreen (generic, per-game opt-in via [widescreen*]):
- Shared screen-extent cull detector (runtime/include/ws_cull_detect.h) used by
all three backends (recompiler, dirty-RAM interp, sljit). Per-game signature
immediates ([widescreen.cull] screen_w_imms/screen_h_imms) — Ape's funnel
tests 0x181/0xF1 on its 368px display, not Tomba's 0x140/0xE0.
- Three funnel idioms widened: unsigned per-vertex sltiu (psx_ws_cull_sltiu),
signed min/max slti right edge + bare-bltz left edge (psx_ws_cull_slti/bltz,
new helpers + SLJIT_HLP appends), and billboard center+/-halfwidth. Explicit
[widescreen.cull] slti_sites for X-only funnels the auto-detector can't
qualify.
- gte_game_mode: GTE-activity gameplay detector for 3D titles with no sprite-tag
helper (gte.cpp notes every RTPS/RTPT). STICKY (>=3 verts, 45-frame hysteresis)
so sparse cutscene frames never flip to a 4:3 pillarbox = fixes intro flicker.
- nw_hud_corners: push outer-third screen-space HUD sprites to the true wide
corners in native-wide (else inset by the reveal).
- nw_backdrop: stretch a full-frame axis-aligned 2D backdrop quad to fill the
wide frame (for titles with a flat sky plane).
- Pattern-scanned interp/sljit widen hooks now gated per-game (auto_screen_x /
auto_backdrop): fixes an ungated auto_backdrop false-positive that rewrote a
live GPR = the ~24s native-wide crash on Ape.
Native-wide GL perf (fixes 16:9 frame drops):
- The wide surface centre equals the canonical 4:3 framebuffer, so wide_blit_center
copies canonical->wide-centre at present (both present paths) and the per-prim
mirror SKIPS prims fully inside the 4:3 frame (mirror_x/geo/batch_center_only).
Only reveal-margin prims re-rasterize. Peak frame time 38.8ms -> 17.5ms, no
vsync misses. Toggle gl_wide_fast; 4:3 untouched (all gated on g_wide_cur).
Runtime-only except the recompiler cull emit (regen needed for cull widening).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replace absolute F:/C: paths with repo-relative (../TombaRecomp, bios/, beetle-psx/)
or placeholders (<desktop>/<psxref>/<nesrecomp>/<psxrecomp-v4>) across accuracy docs,
WIDESCREEN/SHADOW/RECURSION_BUG/config_schema/ISSUES/README, and tools/*.py|.ps1.
Removed a personal C:\Users\Matthew\Desktop path (ISSUES.md). Left CLAUDE.md/PLAN.md
sibling-project + version-history paths intact (intentional layout documentation).
PR #9 makes our CD controller more faithful than Beetle (SCPH-1001 version
94/09/19/C0 vs Beetle's hardcoded 97, ADPBUSY idle=0, disc-derived GetID region).
devtrace_diff.py now LABELS a cdrom-source native/beetle delta as KNOWN intentional
(more faithful) instead of red-flagging it; --strict shows raw. Beetle stays the
independent oracle. Documented in ACCURACY_BURNDOWN.md Axis 5.
merge_cache used a flat listdir() against a nested cache
(<compiler>/<arch-abi>/cg<N>_<hash>/) so it matched nothing — the vault's
DLL mirror has been empty since the tool shipped. Walk the tree and keep
relative paths: the same content-keyed filename exists under different cg
dirs with different compiled bytes, so flattening would mix emitter
generations. stats() walks recursively to match.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The emitter routes mfc2/cfc2/mtc2/ctc2 on flag/IR/derived GTE registers
through direct gte_read_data/gte_read_ctrl/gte_write_data/gte_write_ctrl
calls (code_generator.cpp COP2 case), but the overlay DLL preamble had no
shims for them — any GTE-heavy overlay failed to LINK (undefined
gte_read_ctrl) and its region ran interpreted forever. That silently
blocked all new native coverage for the lava/Trolley attract-demo regions
after the caches rotated.
GTE state is shared runtime state, so the shims forward unconditionally
through OverlayCallbacks (a local copy would fork the flag/IR timeline).
ABI bumped 9 -> 10; the loader gate rejects+deletes pre-shim DLLs and
autocompile self-heals; plain gte_data[]/gte_ctrl[] accesses stay direct
CPUState reads in emitted code, matching the emitter.
Smoke-validated: previously-failing GTE-heavy captures now compile clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Root cause of the fresh-launch logo freeze (present at every commit in the
stack; window-capture-verified 13/13 repros, never a regression): the BIOS
dispatch normalize() maps RAM phys 0x30000-0x5AFFF to the shell ROM alias.
The Class-A fix routed post-game-start shell-window targets to the game/dirty
path first, but its gate also required psx_game_address_in_text() — which
exempts runtime-loaded OVERLAY code above the boot-EXE text end. Tomba2's
text ends at 0x38800; its post-logo main-thread resume PC 0x50B80 fell
through to normalize() and executed func_1FC38B80, a stale shell-image
data-as-code self-loop, starving the main thread forever (one VBlank
delivery per frame at the same EPC, zero main-thread instructions retired;
the game's ISR ran fine, so TCP frame counters kept ticking — the freeze
was invisible to every telemetry-only validation). Even PSX_FORCE_INTERP
missed it: normalize() shadowed the address before the interp path saw it.
Fix: post-game-start, the WHOLE shell-overlap window routes to
dirty_ram_dispatch before normalize() — real hardware executes the RAM
bytes regardless of who loaded them (boot EXE, overlay loader, DMA stream).
Pre-start shell addresses still normalize to the shell as before.
Also:
- interrupts/debug_server: nestgate_* per-guard veto-attribution counters on
the nested-delivery gate (freeze_check), used to exonerate the IEc/SR
guards and redirect the hunt from interrupts.c to dispatch.
- tools/window_capture.py: PrintWindow-based SDL-window capture with
LIVE/FROZEN series verdict — window-liveness ground truth; TCP/VRAM
screenshots provably lie about presentation (this bug's entire history).
Validated: fresh launch now runs logo -> intro FMV -> title -> attract with
sustained full-frame motion (window-level capture series).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
platform.system() under an MSYS/Cygwin python returns 'MSYS_NT-...', not
'Windows', so cache_arch_abi() filed every compiled overlay DLL under
gcc/linux-x64/ while the Windows loader reads gcc/win-x64/. Result: the
loader found no native overlays (interp forever), while autocompile kept
checking its own linux-x64 dir and reporting 'already covered — no new
native code to build'. Measured on Tomba2 attract: ~14M interp insns/s,
dirty dispatches 534-695/s, 55-57fps / 91-95%% guest speed. After fixing
the tag (+ migrating the misfiled DLLs): 58-60fps, 33.8M cycles/s = 100%%
speed, dirty dispatches down 3x. Also fixes the -fPIC-on-Windows flag
picked by the same broken check.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- [video] fmv_skip_no_xa: broaden FMV detection to MDEC activity alone.
Silent RAM-preloaded movies never stream XA, so the MDEC+XA detector
missed them and no skip mechanism was even attempted. Per-game opt-in.
- tools/host_thread_probe.py: suspend/sample RIP+RSP of every host
thread of a live or wedged runtime, stack-scan return addresses into
the main exe, addr2line-symbolize (RelWithDebInfo). Diagnosed the
fatal-halt-serve loop in minutes; use whenever a runtime 'freezes'.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Root cause of the Tomba pause-menu wedge (loaded saves only): autocompiled
overlay shards were emitted by a recompiler binary OLDER than the last emitter
change, yet stamped with the CURRENT cg tag (compile_overlays derives the tag
from --runtime-include; nothing tied it to the emitting binary). The stale
shards corrupted the display-list task queue during load-game. NOT the
IRQ-resume class the prior handoff claimed - disproven with the new ring.
- recompiler: bake the emitter-source hash (shared canonical list
runtime/codegen_hash_sources.cmake + hash_codegen.cmake) into psxrecomp-game;
new --codegen-hash flag prints it (machine-parseable, pre-banner).
- compile_overlays.py: HARD-FAIL when the recompiler binary's baked hash
differs from the cg tag hash it stamps (or the binary lacks the flag).
Applies to both cache and --static modes. Closes the read==write-tag /
stale-content drift class structurally.
- runtime.cmake: codegen-hash source list moved to the shared canonical file.
- interrupts.c: always-on exception-EXIT half added to irqctx_ring (take_pc,
real_epc, exit_pc, escape reason, same-thread/restore flags, v1/ra
saved-vs-exit, RestoreState redirect count) - proves per delivery which
escape path fired and whether the interrupted GPRs were restored. Kept the
same_thread_resume GPR-restore refinement (ring-verified equal-value in
practice; faithful: hardware restores all GPRs on every exception return).
- debug_server.c: irqctx_ring dump extended with the exit-half fields.
Validated live: Tomba menu opens on loaded save (Stormy Mountain) with the
cache fully enabled after regenerating shards with a rebuilt recompiler
(build-cosim + build-prod caches). Guard verified positive and negative.
MMX6 gate passed: boot -> title -> opening cutscene -> gameplay handoff OK.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The cycle-accuracy emitter emits calls to 12 timing runtime functions
(psx_cyc_load_*, psx_icache_fetch, psx_muldiv_*, psx_gte_*, psx_slice_block,
...) + debug_server_cyc_observe + g_debug_last_store_pc. Overlay DLLs never
got these symbols, so every overlay failed to link -> 0 native shards ->
100% interpreter (unplayable). Fix (option A): expose the 12 via the overlay
callback ABI.
- overlay_api.h: +12 timing callbacks in OverlayCallbacks, ABI 8 -> 9.
- overlay_loader.c: populate the 12; add warn_on_cgtag_mismatch() (loud
warning when shards exist under the wrong cg-hash).
- compile_overlays.py: 12 preamble forwarders + local g_debug_last_store_pc
+ -DPSX_NO_DEBUG_TOOLS.
- cosim.c/main.cpp: input-cosim setpad + held pad (deterministic input).
- cosim.py: env-overridable COSIM_EXE/CWD/GAME.
KNOWN REGRESSION (patched on a follow-up branch): Tomba pause menu (triangle)
wedges. Root-caused: an IRQ taken mid-block resumes at the function entry
leader instead of the interrupted PC, re-zeroing the menu's VSync wait
counter (scratchpad 0x1F8001E8) every frame so it never reaches its target.
Not the scheduler (LLE reproduces) and not timing math -- interrupt-resume
re-entry.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New always-on ring (device_trace.{c,h}, compiles into both binaries like
parity_trace) recording every hardware IRQ-raise edge with its guest cycle,
source (I_STAT bit) and detail. All native device sources funnel their raise
through one choke point psx_irq_raise(bit,detail) in interrupts.c (VBLANK/GPU/
CDROM/DMA/timers/SIO routed; DMA passes DICR channel flags, timers pass index),
so the device-timing audit sees every event from one place. devtrace_dump/ctl
debug commands (cycle-window + source filters) + tools/devtrace_diff.py align
the two device-event timelines by source ordinal and flag where cycles drift.
The Beetle oracle side (mednafen IRQ_Assert rising-edge hook) lives in the main
checkout.
This is the level below parity_trace: parity names the divergent control-flow /
gate input; device_trace names the divergent HARDWARE EVENT behind it.
First result (MMX6 cutscene asset-load over-fast wedge): the load's CD+DMA
activity is compressed into ~232M cycles on native vs ~384M on Beetle (~1.65x),
completing the func_8001CB3C scene setup ~100M cycles / ~176 frames early. The
producer is CD-read / DMA-completion timing (DMA channels 2/3/4 = GPU/CD/SPU
asset streaming) -- confirming the device-completion-latency hypothesis and
ruling in the CDROM/DMA cycle model as the over-fast subsystem. (VBlank verified
correct: native VBlank edges are exactly 564480 cyc apart.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extends the two-process parity ring so each row carries, per watch word, the
exact store PC + guest cycle + frame + (approx) writing TCB of the most recent
write to that word. parity_trace_note_write() maintains the table from BOTH
processes' write paths (native: memory.c psx_write_* with g_debug_last_store_pc;
Beetle: beetle_libretro.cpp wtrace_callback with the store pc). Both parity_dump
serializers emit the new wwpc/wwcy/wwf/wwt arrays (identical wire format), and
parity_diff.py prints a side-by-side WATCH PROVENANCE block at the divergence.
This turns "the gate-input value diverged" into "who/when produced it" -- the
instrument ChatGPT recommended for the MMX6 over-fast wedge.
First result (MMX6 cutscene->gameplay, watch 0x80071088/0x800CD3F8/0x8006D9AC/
0x800CD3FC/0x800CD404/0x800200F4): the cutscene-setup func_8001CB3C (writer pc
0x8001CB7C of countdown 0x800CD3FC) runs ~99.7M guest cycles / ~176 frames
EARLIER on native (cyc 998,597,937 frame 1769) than Beetle (cyc 1,098,296,890
frame 1942) -- same writer PC, fired early. On Beetle the state struct
0x800CD3F8 is never touched by game code (stays BIOS-init 0 -> setter path); on
native it advances. The over-fast wait is localized one level above the state
machine, at whatever schedules func_8001CB3C.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
End the "somewhere upstream" blindness with a reusable first-divergence
instrument (PRINCIPLES.md "find the first divergence; is the value WRONG or the
behavior MISSING"). A TCB-scoped, freeze-on-trigger control-flow ring with an
identical struct + JSON wire format on both processes, so a generic sequence-diff
(tools/parity_diff.py) locates the first logical divergence between recomp and
oracle — not a frame diff.
- parity_trace.{h,c}: armed-from-boot ring, gated on current_tcb==watched_tcb,
latches on a configured trigger target so the pre-divergence window survives
the wedge loop. Up to 6 watch-word snapshots/row. Self-contained TU shared by
both binaries; frame source decoupled via parity_host_frame().
- fntrace.c: per-dispatch producer hook (cheap armed gate on the hot path).
- traps.c: YIELD control-event hook in the scheduler switch.
- main.cpp: env-armed from boot (PSX_PARITY_TRACE=1; TCB/trigger/watch tunable),
defaults target the MMX6 cutscene wedge.
- debug_server.c: parity_dump + parity_ctl commands.
- tools/parity_diff.py: pulls both rings, sequence-diffs by (kind,pc,target).
First use (MMX6 freeze) confirmed MISSING behavior, not a wrong value: native
thread1 is permanently parked at func_8002000C resume (TCB EPC 0x800200F4),
handshake flag 0x8006D9AC never set (0 across 16384 rows), setter func_8001344C
never runs; thread1 re-resumes a dead func_8002000C frame whose saved-ra slot was
reused -> jr 0x800CD3F8 wedge. Upstream context-staleness/scheduling root.
Beetle mirror producer + the cross-process diff are the next step.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
recompiler/code_generator.cpp: stamp g_debug_last_store_pc before EVERY game
store (SB/SH/SWL/SW/SWR, SWC2, + the ws-backdrop / game-option special paths)
and declare the extern in the game-C preamble. Previously only the BIOS emitter
(strict_translator.cpp) stamped it, so game stores inherited the last BIOS store
pc -> wtrace/readtrace mis-attributed game writes to the BIOS. VERIFIED LIVE:
MMX6's producer trace for the 0x800CD3F8 write now reports the real game store
pc 0x8001A708 (matches Beetle), not the stale 0xBFC11BF0.
runtime: g_pc0_reason slice-exit provenance, fail-closed exception-sentinel
asserts, named thread-event kinds, freeze-heartbeat RVA emission (diagnostics,
default-on, additive).
Carries the in-progress faithful-timing / MDEC work (cpu_state.h, memory.c,
dma.c, interrupts.c, dirty_ram_interp.c, full_function_emitter.cpp) as an
uncommitted-WIP checkpoint. tools/: regen_bios.sh, bios_emitter_fingerprint.sh.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The faithful dequant/IDCT/YUV transcription was correct, but the 15bpp pack did
(c+4)>>3 on `r ^ 0x80` as a SIGNED int. For negative Mask9ClampS8 results that
yields a negative int (r=-1 → -129) → garbage 5-bit channel → B saturated, R/G
doubled = the rainbow fringing on real FMV. Beetle's RGB_to_RGB555 takes uint8
params (truncates to 0..255); the 24bpp path already cast to uint8_t. Fix:
rgb_to_555_chan(uint8_t) so the caller's int truncates on the call.
Caught only by comparing our VRAM to the REAL Beetle process (new vram_peek on
the Beetle side, held-FMV diff) — both standalone pins shared the bug in their
self-transcribed reference and falsely passed. After the fix the per-channel VRAM
delta vs Beetle collapsed from mean (12,16,27) to (0.7,0.9,1.3) and the FMV renders
natural colours. Full-path pin (mdec_e2e, corrected uint8 oracle) is byte-exact.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The isolated dequant/IDCT/YUV pin matches Beetle byte-exact on all stages, but the
WIP numeric-only change REGRESSES the live FMV (rainbow fringing; baseline is clean)
because the block-assembly (append_color_macroblock) was not transcribed to the new
block layout. Proper fix needs Beetle's EncodeImage assembly + a full-path pin. See
tools/mdec_pin/FINDING.md. DO NOT MERGE the WIP until the full-path pin + live FMV pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the placeholder region cost (RAM +3 / everything-else +0) in
psx_cyc_readmem with the full Beetle MemRW device-region read-wait table
(libretro.cpp), size-aware:
main RAM (<0x800000) +3
SPU 0x1F801C00-1FFF +36 (32-bit) / +16 (8/16-bit)
CDC 0x1F801800-180F +6 * size
GPU/MDEC/SysCtrl/FIO/SIO/IRQ/DMA/Timers (0x1F801000-113F) +1
BIOS ROM / Expansion-PIO / unmatched / scratchpad +0
Thread the access size (1/2/4) from psx_cyc_load_word/half/byte and
psx_cyc_lwc2_read into psx_cyc_readmem (SPU/CDC waits are width-dependent).
The device wait combines with the existing +2 completion (+1 LWC2) and fudge
exactly as Beetle ReadMemory (LDAbsorb = region + completion). Runtime-only:
psx_cyc_load_* signatures unchanged, no emitter regen.
New ruler #2 loops mmio_timer (Timer0 read -> +3) and mmio_spu (32-bit SPU
read -> +38) added to gen_testrom.py. Validated: cyctest compiled (4600) ==
Beetle (4382) EXACT on all 15 loops incl. mmio_timer +3 / mmio_spu +38; the 13
prior loops unchanged; Tomba 2 boots past the BIOS to its SCEA-Presents intro
splash, no freeze.
Documented residual: DMACycleSteal (dynamic per-read DMA bus-steal) remains
unmodeled — needs the live steal count from the DMA controller; not statically
rulerable.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New runtime/src/psx_icache.c: direct-mapped (4 KB / 256-line) instruction-
cache fetch cost, transcribed from Beetle PS_CPU::ReadInstruction:
HIT (TV == addr) -> +0, no give-back clear
MISS, KSEG1/uncached -> +4 (BIOS ROM)
MISS, cached (KSEG0/KUSEG) -> +3, then +1 per word refilled from the
missing word to the line end (earlier words
stay invalid); also clears the load give-back.
Mirrors only the per-word TV tag array (cost + tag state; the value is already
in hand). Wired into the dirty-RAM interpreter (exec_one) per instruction,
charged BEFORE the §1 base — Beetle order.
New ruler #2 loop `icache_miss`: loop top + a victim 0x1000 apart alias the
same direct-mapped line with different tags, forcing a refill miss every
iteration (other loops are all-hits after warm-up).
VALIDATED interp vs Beetle (PSX_FORCE_INTERP=1 PSX_ICACHE=1): icache_miss
native +14 == Beetle +14 (per-iter 17); the other 12 loops unchanged at +0
fetch. So the I-cache hit AND refill-miss costs are MEASURED equal to the
oracle on the interp path.
Opt-in via PSX_ICACHE=1 (default OFF) — charging fetch cost in only one
backend would fork mixed compiled/interp timing. Production activation +
the compiled-path emit (per cache-line leader) + ruler #1 cold-spike
validation are Stage 2. Default-off => no production change (the interp's
fetch call is a no-op without the env).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds a force-interp knob (dirty_ram_is_dirty reports all RAM above the
kernel window dirty when PSX_FORCE_INTERP=1) so the dispatcher routes
clean compiled game text through the dirty-RAM interpreter — the SAME
path overlays take — with no emitter/dispatch change. This is the
interp-path Δ-ruler enabler: it lets the cyctest isolation loops be
measured native-INTERP vs Beetle.
RESULT — the dirty-RAM interpreter == Beetle EXACT on all 12 ruler#2
components (baseline/alu/load/load2/load_use/div/div_spaced/mult/
gte_rtps/gte_nclip/gte_read_use/ld_div). The interpreter's per-instruction
load-delay + GTE/muldiv interlock is now MEASURED equal to the oracle and
to the compiled backend, not merely shared-by-construction.
Process notes (cost real time, now documented in the cyctest README):
- launch psx-cyctest via PowerShell Start-Process; a bash '&' launch
fails to boot (pc=0).
- sample cyc_watch / freeze_check at STEADY STATE: an early query (before
the BIOS boots to the EXE entry) reports dirty_ram_insns=0 / warm-up
values. This premature-sampling artifact was the whole "dispatch
paradox" — the interp engages only after the EXE entry is reached.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds three cycle-isolation loops that exercise the load-delay give-back
TERMINATION and the two not-yet-modeled give-back interactions, each
measured against the Beetle oracle AT STEADY STATE (Beetle's warm-up
window reports the no-give-back value — the source of the handoff's
mis-recorded "gte_rtps +15 EXACT"; true steady Beetle = +11):
- load_use (lw; addu deps the loaded reg): native +5 == Beetle +5 —
Δ-validates the GPR_DEP give-back termination shipped in the prior
commit (load/load2 load into dead regs and never exercise it).
- gte_read_use / gte_rtps / gte_nclip: native OVER-counts by 4-5 — the
MFC2/CFC2 LDAbsorb give-back (gte_ts_done-now) is unmodeled.
- ld_div (lw; divu; mflo): native UNDER-counts by 4 — the muldiv stall
must CONSUME the pending lw give-back (Beetle decrements ReadAbsorb
while stalling MFLO to muldiv_ts_done).
The 8 CPU-load/alu/div/mult components all MATCH at steady state, so the
prior load-interlock commit is solid. The 4 confirmed divergences are
the Δ-gates for the GTE-read/MFC0/muldiv give-back follow-up; spec +
targets in accuracy/load_readfudge_ldabsorb.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Models the GTE command latency + stall-on-COP2-access, the next Stage-2 cycle
component. Mirrors the mult/div completion-stall mechanism:
- cpu->gte_ts_done (new CPUState field, appended): a GTE command sets it to
psx_cycle_count + (cost-1); back-to-back commands serialize (psx_gte_set
stalls to the prior deadline first). Any COP2 register access
(MFC2/CFC2/MTC2/CTC2/LWC2/SWC2) calls psx_gte_stall, advancing guest cycles
to the deadline if the op is still in flight.
- Cost table (psx_cycles.c, psx_gte_cmd_latency): per-op cost-1, transcribed +
verified from beetle-psx gte.cpp op returns (RTPS15 NCLIP8 MVMVA8 NCDS19
NCDT44 RTPT23 NCCT39 ... AVSZ4=5 not 6). cost-1 because the COP2 instruction's
own +1 base is charged separately (Beetle GTE_Instruction returns ret-1).
- Set armed in the shared gte_execute() (one site for both backends); stall
emitted before every COP2 reg access in both emitters (code_generator,
strict_translator) and the dirty interp. The compiled "+1 at top" vs interp
"base after exec_one" offset cancels (same as muldiv).
Validated EXACT on ruler #2 (new gte_rtps + gte_nclip loops): native +15/+8 ==
Beetle +15/+8. All other components unchanged (no regression); Tomba2 boots to
FMV. Requires regen (emitters changed); runtime+BIOS+game regenerated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Stage-2's biggest cost gap closed. MULT/MULTU/DIV/DIVU now set a completion
deadline; MFLO/MFHI stall guest cycles until it — exactly like Beetle's
muldiv_ts_done. Latencies transcribed from cpu.cpp: DIV/DIVU=37; MULT via
MULT_Tab24 (14/10/7) on the first-operand magnitude. Native previously charged
ZERO for this.
- Per-instruction cycle charging is now the DEFAULT on this audit branch
(PSX_CODEGEN_CYCLE_PER_INSN), required so the stall absorbs correctly (the
running cycle count must be accurate mid-block). Validated cycle-neutral for the
identity base model; Tomba 2 still reaches the FMV.
- runtime: CPUState.muldiv_ts_done + psx_muldiv_set/stall + psx_mult_latency_s/u
in psx_cycles.c. Emitted at the mult/div/mflo/mfhi sites in the GAME emitter
(gated PSX_ENABLE_BLOCK_CYCLES). Stall fires even for mflo/mfhi to $zero.
- ruler #2 native half: tools/cycle_testrom/CMakeLists.txt builds psx-cyctest
(links the recompiled test EXE), boots disc/cyctest.cue (serial CYCT-00101 so
disc-identity matches, no warning dialog). measure.py --port 4600.
VALIDATION (native vs Beetle oracle, per-iteration component cost):
div +38 vs +38 EXACT | div_spaced +38 vs +38 EXACT (absorb works) |
mult +15 vs +15 EXACT | baseline/alu match.
Remaining: load native +7 vs Beetle +5 (memory.c flat +6 wait-state needs
calibration to ~4) — a smaller refinement. BIOS emitter still block-mode (ruler
#1 unchanged at 30) — applying per-insn + muldiv there is the next step to close
ruler #1's div-stall gap.
No FMV regression (i_stat 0x8D, screenshot verified).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
NyperYuhgard