merge_cache used a flat listdir() against a nested cache
(<compiler>/<arch-abi>/cg<N>_<hash>/) so it matched nothing — the vault's
DLL mirror has been empty since the tool shipped. Walk the tree and keep
relative paths: the same content-keyed filename exists under different cg
dirs with different compiled bytes, so flattening would mix emitter
generations. stats() walks recursively to match.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The emitter routes mfc2/cfc2/mtc2/ctc2 on flag/IR/derived GTE registers
through direct gte_read_data/gte_read_ctrl/gte_write_data/gte_write_ctrl
calls (code_generator.cpp COP2 case), but the overlay DLL preamble had no
shims for them — any GTE-heavy overlay failed to LINK (undefined
gte_read_ctrl) and its region ran interpreted forever. That silently
blocked all new native coverage for the lava/Trolley attract-demo regions
after the caches rotated.
GTE state is shared runtime state, so the shims forward unconditionally
through OverlayCallbacks (a local copy would fork the flag/IR timeline).
ABI bumped 9 -> 10; the loader gate rejects+deletes pre-shim DLLs and
autocompile self-heals; plain gte_data[]/gte_ctrl[] accesses stay direct
CPUState reads in emitted code, matching the emitter.
Smoke-validated: previously-failing GTE-heavy captures now compile clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Root cause of the fresh-launch logo freeze (present at every commit in the
stack; window-capture-verified 13/13 repros, never a regression): the BIOS
dispatch normalize() maps RAM phys 0x30000-0x5AFFF to the shell ROM alias.
The Class-A fix routed post-game-start shell-window targets to the game/dirty
path first, but its gate also required psx_game_address_in_text() — which
exempts runtime-loaded OVERLAY code above the boot-EXE text end. Tomba2's
text ends at 0x38800; its post-logo main-thread resume PC 0x50B80 fell
through to normalize() and executed func_1FC38B80, a stale shell-image
data-as-code self-loop, starving the main thread forever (one VBlank
delivery per frame at the same EPC, zero main-thread instructions retired;
the game's ISR ran fine, so TCP frame counters kept ticking — the freeze
was invisible to every telemetry-only validation). Even PSX_FORCE_INTERP
missed it: normalize() shadowed the address before the interp path saw it.
Fix: post-game-start, the WHOLE shell-overlap window routes to
dirty_ram_dispatch before normalize() — real hardware executes the RAM
bytes regardless of who loaded them (boot EXE, overlay loader, DMA stream).
Pre-start shell addresses still normalize to the shell as before.
Also:
- interrupts/debug_server: nestgate_* per-guard veto-attribution counters on
the nested-delivery gate (freeze_check), used to exonerate the IEc/SR
guards and redirect the hunt from interrupts.c to dispatch.
- tools/window_capture.py: PrintWindow-based SDL-window capture with
LIVE/FROZEN series verdict — window-liveness ground truth; TCP/VRAM
screenshots provably lie about presentation (this bug's entire history).
Validated: fresh launch now runs logo -> intro FMV -> title -> attract with
sustained full-frame motion (window-level capture series).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
platform.system() under an MSYS/Cygwin python returns 'MSYS_NT-...', not
'Windows', so cache_arch_abi() filed every compiled overlay DLL under
gcc/linux-x64/ while the Windows loader reads gcc/win-x64/. Result: the
loader found no native overlays (interp forever), while autocompile kept
checking its own linux-x64 dir and reporting 'already covered — no new
native code to build'. Measured on Tomba2 attract: ~14M interp insns/s,
dirty dispatches 534-695/s, 55-57fps / 91-95%% guest speed. After fixing
the tag (+ migrating the misfiled DLLs): 58-60fps, 33.8M cycles/s = 100%%
speed, dirty dispatches down 3x. Also fixes the -fPIC-on-Windows flag
picked by the same broken check.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- [video] fmv_skip_no_xa: broaden FMV detection to MDEC activity alone.
Silent RAM-preloaded movies never stream XA, so the MDEC+XA detector
missed them and no skip mechanism was even attempted. Per-game opt-in.
- tools/host_thread_probe.py: suspend/sample RIP+RSP of every host
thread of a live or wedged runtime, stack-scan return addresses into
the main exe, addr2line-symbolize (RelWithDebInfo). Diagnosed the
fatal-halt-serve loop in minutes; use whenever a runtime 'freezes'.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Root cause of the Tomba pause-menu wedge (loaded saves only): autocompiled
overlay shards were emitted by a recompiler binary OLDER than the last emitter
change, yet stamped with the CURRENT cg tag (compile_overlays derives the tag
from --runtime-include; nothing tied it to the emitting binary). The stale
shards corrupted the display-list task queue during load-game. NOT the
IRQ-resume class the prior handoff claimed - disproven with the new ring.
- recompiler: bake the emitter-source hash (shared canonical list
runtime/codegen_hash_sources.cmake + hash_codegen.cmake) into psxrecomp-game;
new --codegen-hash flag prints it (machine-parseable, pre-banner).
- compile_overlays.py: HARD-FAIL when the recompiler binary's baked hash
differs from the cg tag hash it stamps (or the binary lacks the flag).
Applies to both cache and --static modes. Closes the read==write-tag /
stale-content drift class structurally.
- runtime.cmake: codegen-hash source list moved to the shared canonical file.
- interrupts.c: always-on exception-EXIT half added to irqctx_ring (take_pc,
real_epc, exit_pc, escape reason, same-thread/restore flags, v1/ra
saved-vs-exit, RestoreState redirect count) - proves per delivery which
escape path fired and whether the interrupted GPRs were restored. Kept the
same_thread_resume GPR-restore refinement (ring-verified equal-value in
practice; faithful: hardware restores all GPRs on every exception return).
- debug_server.c: irqctx_ring dump extended with the exit-half fields.
Validated live: Tomba menu opens on loaded save (Stormy Mountain) with the
cache fully enabled after regenerating shards with a rebuilt recompiler
(build-cosim + build-prod caches). Guard verified positive and negative.
MMX6 gate passed: boot -> title -> opening cutscene -> gameplay handoff OK.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The cycle-accuracy emitter emits calls to 12 timing runtime functions
(psx_cyc_load_*, psx_icache_fetch, psx_muldiv_*, psx_gte_*, psx_slice_block,
...) + debug_server_cyc_observe + g_debug_last_store_pc. Overlay DLLs never
got these symbols, so every overlay failed to link -> 0 native shards ->
100% interpreter (unplayable). Fix (option A): expose the 12 via the overlay
callback ABI.
- overlay_api.h: +12 timing callbacks in OverlayCallbacks, ABI 8 -> 9.
- overlay_loader.c: populate the 12; add warn_on_cgtag_mismatch() (loud
warning when shards exist under the wrong cg-hash).
- compile_overlays.py: 12 preamble forwarders + local g_debug_last_store_pc
+ -DPSX_NO_DEBUG_TOOLS.
- cosim.c/main.cpp: input-cosim setpad + held pad (deterministic input).
- cosim.py: env-overridable COSIM_EXE/CWD/GAME.
KNOWN REGRESSION (patched on a follow-up branch): Tomba pause menu (triangle)
wedges. Root-caused: an IRQ taken mid-block resumes at the function entry
leader instead of the interrupted PC, re-zeroing the menu's VSync wait
counter (scratchpad 0x1F8001E8) every frame so it never reaches its target.
Not the scheduler (LLE reproduces) and not timing math -- interrupt-resume
re-entry.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New always-on ring (device_trace.{c,h}, compiles into both binaries like
parity_trace) recording every hardware IRQ-raise edge with its guest cycle,
source (I_STAT bit) and detail. All native device sources funnel their raise
through one choke point psx_irq_raise(bit,detail) in interrupts.c (VBLANK/GPU/
CDROM/DMA/timers/SIO routed; DMA passes DICR channel flags, timers pass index),
so the device-timing audit sees every event from one place. devtrace_dump/ctl
debug commands (cycle-window + source filters) + tools/devtrace_diff.py align
the two device-event timelines by source ordinal and flag where cycles drift.
The Beetle oracle side (mednafen IRQ_Assert rising-edge hook) lives in the main
checkout.
This is the level below parity_trace: parity names the divergent control-flow /
gate input; device_trace names the divergent HARDWARE EVENT behind it.
First result (MMX6 cutscene asset-load over-fast wedge): the load's CD+DMA
activity is compressed into ~232M cycles on native vs ~384M on Beetle (~1.65x),
completing the func_8001CB3C scene setup ~100M cycles / ~176 frames early. The
producer is CD-read / DMA-completion timing (DMA channels 2/3/4 = GPU/CD/SPU
asset streaming) -- confirming the device-completion-latency hypothesis and
ruling in the CDROM/DMA cycle model as the over-fast subsystem. (VBlank verified
correct: native VBlank edges are exactly 564480 cyc apart.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extends the two-process parity ring so each row carries, per watch word, the
exact store PC + guest cycle + frame + (approx) writing TCB of the most recent
write to that word. parity_trace_note_write() maintains the table from BOTH
processes' write paths (native: memory.c psx_write_* with g_debug_last_store_pc;
Beetle: beetle_libretro.cpp wtrace_callback with the store pc). Both parity_dump
serializers emit the new wwpc/wwcy/wwf/wwt arrays (identical wire format), and
parity_diff.py prints a side-by-side WATCH PROVENANCE block at the divergence.
This turns "the gate-input value diverged" into "who/when produced it" -- the
instrument ChatGPT recommended for the MMX6 over-fast wedge.
First result (MMX6 cutscene->gameplay, watch 0x80071088/0x800CD3F8/0x8006D9AC/
0x800CD3FC/0x800CD404/0x800200F4): the cutscene-setup func_8001CB3C (writer pc
0x8001CB7C of countdown 0x800CD3FC) runs ~99.7M guest cycles / ~176 frames
EARLIER on native (cyc 998,597,937 frame 1769) than Beetle (cyc 1,098,296,890
frame 1942) -- same writer PC, fired early. On Beetle the state struct
0x800CD3F8 is never touched by game code (stays BIOS-init 0 -> setter path); on
native it advances. The over-fast wait is localized one level above the state
machine, at whatever schedules func_8001CB3C.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
End the "somewhere upstream" blindness with a reusable first-divergence
instrument (PRINCIPLES.md "find the first divergence; is the value WRONG or the
behavior MISSING"). A TCB-scoped, freeze-on-trigger control-flow ring with an
identical struct + JSON wire format on both processes, so a generic sequence-diff
(tools/parity_diff.py) locates the first logical divergence between recomp and
oracle — not a frame diff.
- parity_trace.{h,c}: armed-from-boot ring, gated on current_tcb==watched_tcb,
latches on a configured trigger target so the pre-divergence window survives
the wedge loop. Up to 6 watch-word snapshots/row. Self-contained TU shared by
both binaries; frame source decoupled via parity_host_frame().
- fntrace.c: per-dispatch producer hook (cheap armed gate on the hot path).
- traps.c: YIELD control-event hook in the scheduler switch.
- main.cpp: env-armed from boot (PSX_PARITY_TRACE=1; TCB/trigger/watch tunable),
defaults target the MMX6 cutscene wedge.
- debug_server.c: parity_dump + parity_ctl commands.
- tools/parity_diff.py: pulls both rings, sequence-diffs by (kind,pc,target).
First use (MMX6 freeze) confirmed MISSING behavior, not a wrong value: native
thread1 is permanently parked at func_8002000C resume (TCB EPC 0x800200F4),
handshake flag 0x8006D9AC never set (0 across 16384 rows), setter func_8001344C
never runs; thread1 re-resumes a dead func_8002000C frame whose saved-ra slot was
reused -> jr 0x800CD3F8 wedge. Upstream context-staleness/scheduling root.
Beetle mirror producer + the cross-process diff are the next step.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
recompiler/code_generator.cpp: stamp g_debug_last_store_pc before EVERY game
store (SB/SH/SWL/SW/SWR, SWC2, + the ws-backdrop / game-option special paths)
and declare the extern in the game-C preamble. Previously only the BIOS emitter
(strict_translator.cpp) stamped it, so game stores inherited the last BIOS store
pc -> wtrace/readtrace mis-attributed game writes to the BIOS. VERIFIED LIVE:
MMX6's producer trace for the 0x800CD3F8 write now reports the real game store
pc 0x8001A708 (matches Beetle), not the stale 0xBFC11BF0.
runtime: g_pc0_reason slice-exit provenance, fail-closed exception-sentinel
asserts, named thread-event kinds, freeze-heartbeat RVA emission (diagnostics,
default-on, additive).
Carries the in-progress faithful-timing / MDEC work (cpu_state.h, memory.c,
dma.c, interrupts.c, dirty_ram_interp.c, full_function_emitter.cpp) as an
uncommitted-WIP checkpoint. tools/: regen_bios.sh, bios_emitter_fingerprint.sh.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The faithful dequant/IDCT/YUV transcription was correct, but the 15bpp pack did
(c+4)>>3 on `r ^ 0x80` as a SIGNED int. For negative Mask9ClampS8 results that
yields a negative int (r=-1 → -129) → garbage 5-bit channel → B saturated, R/G
doubled = the rainbow fringing on real FMV. Beetle's RGB_to_RGB555 takes uint8
params (truncates to 0..255); the 24bpp path already cast to uint8_t. Fix:
rgb_to_555_chan(uint8_t) so the caller's int truncates on the call.
Caught only by comparing our VRAM to the REAL Beetle process (new vram_peek on
the Beetle side, held-FMV diff) — both standalone pins shared the bug in their
self-transcribed reference and falsely passed. After the fix the per-channel VRAM
delta vs Beetle collapsed from mean (12,16,27) to (0.7,0.9,1.3) and the FMV renders
natural colours. Full-path pin (mdec_e2e, corrected uint8 oracle) is byte-exact.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The isolated dequant/IDCT/YUV pin matches Beetle byte-exact on all stages, but the
WIP numeric-only change REGRESSES the live FMV (rainbow fringing; baseline is clean)
because the block-assembly (append_color_macroblock) was not transcribed to the new
block layout. Proper fix needs Beetle's EncodeImage assembly + a full-path pin. See
tools/mdec_pin/FINDING.md. DO NOT MERGE the WIP until the full-path pin + live FMV pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the placeholder region cost (RAM +3 / everything-else +0) in
psx_cyc_readmem with the full Beetle MemRW device-region read-wait table
(libretro.cpp), size-aware:
main RAM (<0x800000) +3
SPU 0x1F801C00-1FFF +36 (32-bit) / +16 (8/16-bit)
CDC 0x1F801800-180F +6 * size
GPU/MDEC/SysCtrl/FIO/SIO/IRQ/DMA/Timers (0x1F801000-113F) +1
BIOS ROM / Expansion-PIO / unmatched / scratchpad +0
Thread the access size (1/2/4) from psx_cyc_load_word/half/byte and
psx_cyc_lwc2_read into psx_cyc_readmem (SPU/CDC waits are width-dependent).
The device wait combines with the existing +2 completion (+1 LWC2) and fudge
exactly as Beetle ReadMemory (LDAbsorb = region + completion). Runtime-only:
psx_cyc_load_* signatures unchanged, no emitter regen.
New ruler #2 loops mmio_timer (Timer0 read -> +3) and mmio_spu (32-bit SPU
read -> +38) added to gen_testrom.py. Validated: cyctest compiled (4600) ==
Beetle (4382) EXACT on all 15 loops incl. mmio_timer +3 / mmio_spu +38; the 13
prior loops unchanged; Tomba 2 boots past the BIOS to its SCEA-Presents intro
splash, no freeze.
Documented residual: DMACycleSteal (dynamic per-read DMA bus-steal) remains
unmodeled — needs the live steal count from the DMA controller; not statically
rulerable.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New runtime/src/psx_icache.c: direct-mapped (4 KB / 256-line) instruction-
cache fetch cost, transcribed from Beetle PS_CPU::ReadInstruction:
HIT (TV == addr) -> +0, no give-back clear
MISS, KSEG1/uncached -> +4 (BIOS ROM)
MISS, cached (KSEG0/KUSEG) -> +3, then +1 per word refilled from the
missing word to the line end (earlier words
stay invalid); also clears the load give-back.
Mirrors only the per-word TV tag array (cost + tag state; the value is already
in hand). Wired into the dirty-RAM interpreter (exec_one) per instruction,
charged BEFORE the §1 base — Beetle order.
New ruler #2 loop `icache_miss`: loop top + a victim 0x1000 apart alias the
same direct-mapped line with different tags, forcing a refill miss every
iteration (other loops are all-hits after warm-up).
VALIDATED interp vs Beetle (PSX_FORCE_INTERP=1 PSX_ICACHE=1): icache_miss
native +14 == Beetle +14 (per-iter 17); the other 12 loops unchanged at +0
fetch. So the I-cache hit AND refill-miss costs are MEASURED equal to the
oracle on the interp path.
Opt-in via PSX_ICACHE=1 (default OFF) — charging fetch cost in only one
backend would fork mixed compiled/interp timing. Production activation +
the compiled-path emit (per cache-line leader) + ruler #1 cold-spike
validation are Stage 2. Default-off => no production change (the interp's
fetch call is a no-op without the env).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds a force-interp knob (dirty_ram_is_dirty reports all RAM above the
kernel window dirty when PSX_FORCE_INTERP=1) so the dispatcher routes
clean compiled game text through the dirty-RAM interpreter — the SAME
path overlays take — with no emitter/dispatch change. This is the
interp-path Δ-ruler enabler: it lets the cyctest isolation loops be
measured native-INTERP vs Beetle.
RESULT — the dirty-RAM interpreter == Beetle EXACT on all 12 ruler#2
components (baseline/alu/load/load2/load_use/div/div_spaced/mult/
gte_rtps/gte_nclip/gte_read_use/ld_div). The interpreter's per-instruction
load-delay + GTE/muldiv interlock is now MEASURED equal to the oracle and
to the compiled backend, not merely shared-by-construction.
Process notes (cost real time, now documented in the cyctest README):
- launch psx-cyctest via PowerShell Start-Process; a bash '&' launch
fails to boot (pc=0).
- sample cyc_watch / freeze_check at STEADY STATE: an early query (before
the BIOS boots to the EXE entry) reports dirty_ram_insns=0 / warm-up
values. This premature-sampling artifact was the whole "dispatch
paradox" — the interp engages only after the EXE entry is reached.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds three cycle-isolation loops that exercise the load-delay give-back
TERMINATION and the two not-yet-modeled give-back interactions, each
measured against the Beetle oracle AT STEADY STATE (Beetle's warm-up
window reports the no-give-back value — the source of the handoff's
mis-recorded "gte_rtps +15 EXACT"; true steady Beetle = +11):
- load_use (lw; addu deps the loaded reg): native +5 == Beetle +5 —
Δ-validates the GPR_DEP give-back termination shipped in the prior
commit (load/load2 load into dead regs and never exercise it).
- gte_read_use / gte_rtps / gte_nclip: native OVER-counts by 4-5 — the
MFC2/CFC2 LDAbsorb give-back (gte_ts_done-now) is unmodeled.
- ld_div (lw; divu; mflo): native UNDER-counts by 4 — the muldiv stall
must CONSUME the pending lw give-back (Beetle decrements ReadAbsorb
while stalling MFLO to muldiv_ts_done).
The 8 CPU-load/alu/div/mult components all MATCH at steady state, so the
prior load-interlock commit is solid. The 4 confirmed divergences are
the Δ-gates for the GTE-read/MFC0/muldiv give-back follow-up; spec +
targets in accuracy/load_readfudge_ldabsorb.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Models the GTE command latency + stall-on-COP2-access, the next Stage-2 cycle
component. Mirrors the mult/div completion-stall mechanism:
- cpu->gte_ts_done (new CPUState field, appended): a GTE command sets it to
psx_cycle_count + (cost-1); back-to-back commands serialize (psx_gte_set
stalls to the prior deadline first). Any COP2 register access
(MFC2/CFC2/MTC2/CTC2/LWC2/SWC2) calls psx_gte_stall, advancing guest cycles
to the deadline if the op is still in flight.
- Cost table (psx_cycles.c, psx_gte_cmd_latency): per-op cost-1, transcribed +
verified from beetle-psx gte.cpp op returns (RTPS15 NCLIP8 MVMVA8 NCDS19
NCDT44 RTPT23 NCCT39 ... AVSZ4=5 not 6). cost-1 because the COP2 instruction's
own +1 base is charged separately (Beetle GTE_Instruction returns ret-1).
- Set armed in the shared gte_execute() (one site for both backends); stall
emitted before every COP2 reg access in both emitters (code_generator,
strict_translator) and the dirty interp. The compiled "+1 at top" vs interp
"base after exec_one" offset cancels (same as muldiv).
Validated EXACT on ruler #2 (new gte_rtps + gte_nclip loops): native +15/+8 ==
Beetle +15/+8. All other components unchanged (no regression); Tomba2 boots to
FMV. Requires regen (emitters changed); runtime+BIOS+game regenerated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Stage-2's biggest cost gap closed. MULT/MULTU/DIV/DIVU now set a completion
deadline; MFLO/MFHI stall guest cycles until it — exactly like Beetle's
muldiv_ts_done. Latencies transcribed from cpu.cpp: DIV/DIVU=37; MULT via
MULT_Tab24 (14/10/7) on the first-operand magnitude. Native previously charged
ZERO for this.
- Per-instruction cycle charging is now the DEFAULT on this audit branch
(PSX_CODEGEN_CYCLE_PER_INSN), required so the stall absorbs correctly (the
running cycle count must be accurate mid-block). Validated cycle-neutral for the
identity base model; Tomba 2 still reaches the FMV.
- runtime: CPUState.muldiv_ts_done + psx_muldiv_set/stall + psx_mult_latency_s/u
in psx_cycles.c. Emitted at the mult/div/mflo/mfhi sites in the GAME emitter
(gated PSX_ENABLE_BLOCK_CYCLES). Stall fires even for mflo/mfhi to $zero.
- ruler #2 native half: tools/cycle_testrom/CMakeLists.txt builds psx-cyctest
(links the recompiled test EXE), boots disc/cyctest.cue (serial CYCT-00101 so
disc-identity matches, no warning dialog). measure.py --port 4600.
VALIDATION (native vs Beetle oracle, per-iteration component cost):
div +38 vs +38 EXACT | div_spaced +38 vs +38 EXACT (absorb works) |
mult +15 vs +15 EXACT | baseline/alu match.
Remaining: load native +7 vs Beetle +5 (memory.c flat +6 wait-state needs
calibration to ~4) — a smaller refinement. BIOS emitter still block-mode (ruler
#1 unchanged at 30) — applying per-insn + muldiv there is the next step to close
ruler #1's div-stall gap.
No FMV regression (i_stat 0x8D, screenshot verified).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The test ROM now boots on a real BIOS. The license region (required or the BIOS
drops to its shell) is extracted from a disc the developer OWNS via the in-tree
dumpsxiso and embedded by mkpsxiso — LOCAL ONLY, gitignored, never redistributed.
Both backends boot disc/cyctest.cue identically (real BIOS -> SYSTEM.CNF -> EXE),
so CPU/cache state matches.
- disc/cyctest.xml + disc/SYSTEM.CNF: mkpsxiso project (tracked). license_data.dat
+ built .bin/.cue gitignored (copyrighted Sony data, local only).
- measure.py: per-anchor consecutive-hit Δ = per-iteration cycle cost; reports
per-component cost via baseline subtraction.
- README: boot-disc build + license-extraction recipe + the oracle results.
Beetle ORACLE component costs (the HW targets for the cost model):
baseline 3 | alu +1 | load +5 | load2 +11 (2nd load +1 = ReadFudge)
| div +38 (~36 stall) | div_spaced +38 (fillers ABSORBED) | mult +15 (~13 stall)
Native currently charges 0 for div/mult stall — the dominant gaps. div_spaced==div
proves the stall must be a completion-timestamp + stall-on-mflo-read (absorbable),
NOT a flat charge on the divu.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Ruler #2 (the in-tree cycle-model integration-test ecosystem):
- tools/cycle_testrom/gen_testrom.py: hand-encoded MIPS PS-X EXE of cycle-
isolation loops (baseline/alu/load/load2/div/div_spaced/mult). Each loop top is
a block leader, so a cyc_watch single-anchor Δ = one iteration; baseline
subtraction isolates each component (overhead + fetch cancel). Encoding
validated by a clean recompile (divu -> gpr[10]/gpr[11], etc.).
- game.toml/seeds.txt/README.md; .gitignore for the regenerable EXE/json/generated.
Universal per-block-leader observe:
- Added debug_server_cyc_observe at every block leader in the GAME emitter
(code_generator.cpp) to match the BIOS emitter, so ANY recompiled code (game,
BIOS, overlay) is anchorable on both backends — not just function entries.
#ifndef PSX_NO_DEBUG_TOOLS so prod = zero overhead.
Regression-checked: regen Tomba 2 + rebuild, FMV still streams (i_stat 0x8D), no
visual regression. Builds clean.
Remaining for ruler #2: native test-runtime delivery (boot->entry handoff) +
Beetle .exe sideload + the per-component compare harness (see README TODO).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
cyc_watch + cycle_compare.py now take an optional END anchor: each recorded value
is Δcycles of one START->END pass (cost of the known code path between two dispatch
points). [A,A] = Δ between consecutive A hits (call-to-call cost). Mirrored EXACTLY
on both backends (native debug_server.c + Beetle libretro.cpp/beetle_debug_server.c,
the latter uncommitted in main checkout, additive). VALIDATED: region [0x80017FC4,
0x80017FC4] = native Δ 56 vs Beetle Δ 91, gap -35, ZERO variance across 8 passes —
a stable, offset-independent per-region gate to drive Stage-2 components to
convergence (faithful components shrink the gap monotonically; can't overshoot).
Note: [A,B] with distinct B needs B to reliably follow A (a dispatch point on the
deterministic path); B=0x8001EFFC failed (func has multiple exits) — pick wrapper/
single-exit pairs or use [A,A].
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Native cyc_watch (ported from agent branch) + the missing universal hook:
debug_server_log_call_entry now samples cyc_watch_observe at every compiled
function entry (game AND BIOS; the agent's debug_server_trace_dispatch hook only
covered the BIOS trampoline, so game-text anchors fired 0). tools/cycle_compare.py
arms the same anchor on native(4500)+Beetle(4382) and diffs per-hit cycles.
Beetle side (libretro.cpp cyc_watch state/observe + cpu.cpp RunReal loop hook +
beetle_debug_server.c commands) is in the MAIN checkout (additive diagnostic,
uncommitted) and built into psx-beetle.
VALIDATED end-to-end: both backends capture 16 hits at game-text anchor 0x80017FC4.
FINDING: native is ~121M cycles BEHIND Beetle at that anchor = a gross boot-cycle-
path divergence (turbo-loads/fast-CD + overlay/dirty charge far fewer cycles than
Beetle's real-time CD boot), NOT the -8 fine drift. => absolute-cycle comparison
through boot is dominated by load-model differences; the -8 needs a controlled
anchor (early/pre-divergence or an interp-convergence point). Relocated BIOS-shell
funcs (phys 0x30000-0x5AFFF, e.g. 0x80050B0C) dispatch at a different native phys —
anchor on game-text or BIOS-ROM funcs, or the relocated phys.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
In-progress investigation work snapshotted before rebasing onto the
tcc-refactored psxrecomp master. Includes overlay ABI v8
(check_interrupts_at resume-PC callback for async RFE recovery) and
code_generator/full_function_emitter delay-slot cycle accounting.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Dev tooling only (never shipped: tools/ is not bundled in releases, and the
debug-server commands it drives are all PSX_NO_DEBUG_TOOLS-stripped in release
builds). Saved so the first-divergence workflow that root-caused the Tomba2
logo->FMV stall is never re-invented.
- t2probe.py layer 1 fingerprint cap/diff; layer 2 unified-recorder dump +
recdiff/recdiff_state (state fork, addr/val only); resolves the
debug_client import relative to tools/.
- waitf.py retry-connect + poll-until-frame (survives socket aborts).
- shot.py screenshot to PNG for VISUAL end-state verification.
- README.md the 3-layer method (always-on fingerprint ring -> divergent
frame; frame-gated recorder -> divergent access; read-watch +
per-entry cyc -> value/pointer/timing), env seeds, and the Tomba2
worked example.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root-caused the Tomba2 native logo-stuck bug: it is a guest-cycle-timeline
divergence between the compiled-overlay backend and the dirty-RAM interpreter,
surfacing through a Timer1 (0x1F801110) debounce loop at game pc 0x8008592C that
reads 1 (native) vs 2 (interp), forking logo->FMV. The handoff's "jalr target
divergence" was a benign store-PC artifact (read-watch proves a0=0xB870 and
t3=[a0]=0x800896E0 are identical in both backends); the scratchpad blind spot
was ruled out (separate sp fingerprint forks at the same frame).
First-divergence observability (debug_server.c, memory.c):
- frame_fingerprint: added separate sp (scratchpad) cumulative hash column +
per-entry cyc, so a scratchpad or cycle fork is classified, not hidden.
- Unified ordered access recorder: one buffer, kind = ramw/spw/mmiow/mmior/ramr,
array index = true execution order, per-entry cyc (psx_get_cycle_count).
record_frame_dump emits the unified stream; record_reads_dump filters reads.
- Targeted main-RAM read-watch (PSX_READ_WATCH="lo,hi" + g_ram_read_watch_active
guard in psx_read_word) — proves a load's value/address per backend.
- PSX_RECORD_FRAME env seed: arm the recorder from boot (race-free), never
arm-then-time.
Fix A (cycle-model unification, part 1): compiled-overlay code was built WITHOUT
PSX_ENABLE_BLOCK_CYCLES, so every psx_advance_cycles() in overlay code was
compiled out -> a function run as a native overlay charged ~0 cycles while the
same function via the interp charged per-instruction (~32x), the entire drift.
- overlay_api.h: OverlayCallbacks gains advance_cycles; ABI v6->v7 (rejects
stale zero-cycle DLLs).
- compile_overlays.py: build overlays with -DPSX_ENABLE_BLOCK_CYCLES=1 and
emit a psx_advance_cycles shim forwarding to the shared host timeline.
- overlay_loader.c: wire s_callbacks.advance_cycles = psx_advance_cycles.
Result: the first debounce now reads Timer1=2 in native (was 1); overlay cycle
charge went 9.5k -> 380k (interp 306k). Native still logo-stuck: a residual,
perfectly linear ~1 cycle/write over-charge remains (block.instruction_count
counts branch + delay-slot as 2 while the interp charges the pair as 1 step),
which flips a later debounce 3 vs 2. Fix A pt2 (cycle-exact block totals) is the
next step. wt/tomba2 only; no pin bump, no master merge, no other-game regen.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Compiles the FMV-driver class of hot code. After the region compiles, a DECOUPLED
pass finds DISPATCH_INTERIOR PCs that ACTUALLY EXECUTED this session but that no
built DLL covers (orphan interiors whose host static analysis never discovered, so
the region can't alias them) and compiles each as its OWN isolated island shard:
a single dispatch_root seed entered AT the interior PC, covering the recompiler's
worklist-discovered reachable CFG (out-of-island branches become dispatcher
exits). No host recovery.
Per ChatGPT architecture review: keep speculative/interior units in a SEPARATE
failure domain from the trusted region DLL. So each fragment is its own
<region>_<key>.dll; a fragment that fails the generated-C audit is dropped ALONE
(the region's DLL is untouched), and the pass runs even when the region's own
compile audit-failed. Safe because dispatch_root is exempt from the boundary
re-check that would reject a mid-function start, the recompiler translates
mid-function code literally (no synthesized frame; the caller set it up), and the
loader's per-function live-RAM CRC gate rejects any non-matching fragment. Gated
to EXECUTED orphans (resident code, not cross-variant data). Validated: probe
shows a clean mid-entry island for 0x80083094 (audit-clean); full run builds 34/47
executed orphans at 0x38000 as isolated shards and 0x80083094 is now covered,
even though that region's normal compile audit-failed on a bad capture variant.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The build artifact was keyed by a CRC32 of the WHOLE captured region, so any
volatile byte (animation counters, data tables) changed the key and minted a
brand-new DLL even when every compiled function was byte-identical -> piles of
redundant DLLs, skip-if-exists never fired.
The loader already content-matches each function by (entry, code_crc) across ALL
DLLs sharing a region_start, so a new DLL is only needed when it adds a NEW
function identity. Gate the gcc compile on exactly that: compute the capture's
(entry, code_crc) set from the recompiler manifest, and skip the build if every
function is already covered by an existing .ranges at that region_start.
- Refactor write_overlay_ranges -> parse_overlay_func_ids (identity list) +
write_overlay_ranges_from (emit). Add load_region_coverage.
- main(): warm per-region coverage set; skip gcc when this capture's function
set is a subset of what's already built.
No loader/runtime change, no filename change. Logic unit-tested: identical code
+ different data dedups; a new function still builds.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Brings the fail-closed native entry guard from wt/tomba2 (b5a6940 / bd60381) to
master, where it had never been merged — which is why Tomba (and any master-built
title) still wedged. This is the long-standing "blue screen" / shared-RAM-
divergence-between-interp-and-native class.
Root cause: an overlay function dispatched at a FOREIGN INTERIOR PC (a range-
ownership mismatch routed a PC the function does not own into it) hit the CPS
entry-switch `default: break`, which fell through and ran the function FROM ITS
TOP — re-running its prologue and corrupting shared CPU/RAM state, so the guest's
main loop then waits forever on a condition that never gets set (the 0xCF0 SIO
spin / blue screen). Reproduced on Tomba's Village of All Beginnings: a native
0x000E7000-region shard function entered at a foreign PC wedged; the interpreter
ran the same bytes correctly.
Fix: the generated CPS entry-switch for OVERLAY functions (config_.overlay_mode)
now FAILS CLOSED on an unknown non-zero continuation PC — it restores the PC and
calls psx_native_bad_entry instead of running from the top. overlay_loader_dispatch
consumes g_native_bad_entry and routes the PC to the sanctioned dirty-RAM
interpreter (the live bytes run correctly — NOT a stub/HLE). True-prologue entry
still runs from the top. Static (BIOS/boot-EXE) functions keep the legacy fall-
through. ABI bumped 5->6 so stale ABI-5 caches are rejected and rebuilt with the
guard.
Minimal port: kept only the fail-closed mechanism + its ABI plumbing. The Tomba2
host_aliases interior-routing and the native-block / CPS-probe diagnostics (which
depend on the overlay-cache-v2 multi-range Candidate fields not on master) were
intentionally omitted — the fail-closed default alone is the safety net.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two framework bugs that made overlay variants run interpreted forever (the
"agonizing lag" in overlay-heavy areas like Tomba's dwarf village):
1. Cache read/write divergence. The loader reads <exe>/cache and the runtime
writes captures to <exe>/overlay_captures.json, but the gcc autocompile path
used the game.toml command verbatim, whose hard-coded --out-dir/--captures
pointed elsewhere (build-stable/build-modern). So compiled shards landed where
the loader never looked and captures were read stale -> the variant never went
native. Fix: the runtime now injects PSX_OVERLAY_CACHE_DIR / PSX_OVERLAY_CAPTURES
(the canonical loader paths) into every autocompile spawn; compile_overlays.py
honors them over any CLI flag (--captures now optional). One source of truth for
the cache location, uniform across all games, dev or prod. gcc/tcc stay separated
by the existing <compiler> namespace.
2. Overlay DLL preamble missing psx_ws_cull_sltiu. auto_screen_x emits this helper
at every width-cull site into EVERY build's C (identity at 4:3), and the main EXE
links it from gpu.c, but the overlay DLL preamble never defined it -> any overlay
variant with cull sites failed to link -> no shard -> interpreter. Fix: provide
the self-contained helper (depends only on the already-forwarded psx_ws_x_margin)
in the DLL preamble.
Proven on Tomba dwarf village: dispatch_native 29/s -> 11454/s (was ~97% interp,
now ~97% native).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Compiler-neutral tier-selection policy extracted from the mis-named
overlay_sljit.{c,h} into NEW overlay_backend.{c,h} (OverlayBackend enum +
resolve/name/active). overlay_sljit.{c,h} is now just the deprecated emitter.
New tcc tier. gcc and tcc run the IDENTICAL pipeline (recompiler -> C ->
compiler -> DLL -> loader); only the compiler binary differs, so both map to the
autocompile-spawn provider (code_provider.c) and the COMMAND is chosen by the
resolved backend (main.cpp). compile_overlays.py gains --compiler gcc|tcc: tcc
writes a separate tcc/ cache namespace, BOM-strips the C + a copy of the include
headers (tcc 0.9.27 can't skip a UTF-8 BOM), and uses tcc's built-in linker.
Backends (overlay_backend): auto (gcc if a gcc TOOLCHAIN is present, else tcc --
was sljit), gcc, tcc, auto-no-gcc (dev/test: force tcc even with gcc present, to
exercise the toolchain-less user path), sljit (DEPRECATED, gated off, lowest).
Priority: static > gcc shard > tcc shard > sljit(off) > interp. The loader scans
gcc/ then tcc/ and dedups by region+crc (filename) so a gcc shard always wins a
tie.
Self-contained tcc: with no explicit overlay_autocompile_cmd_tcc, the runtime
CONSTRUCTS the tcc command from a toolchain bundled beside the exe
(<exe>/overlay_toolchain/: embedded python + tcc + recompiler + compile_overlays.py
+ runtime headers) -- no system python or gcc needed. config_loader gains
overlay_autocompile_cmd_tcc (a dev override). autocompile cmd buffers enlarged for
the constructed path-heavy command.
Compilation stays OFF the dispatch/render thread (off-process autocompile
subprocess, same model as gcc); the main thread only spawns it and dlopens the
finished DLL.
Validated: MMX6 with overlay_backend=auto-no-gcc fills the tcc/ namespace live
from the bundled toolchain and plays through save-load + gameplay -- the exact
path the sljit tier softlocked on. The gcc path is untouched.
Runtime-only, no regen.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The overlay autocompile_cmd uses paths RELATIVE to the project root so the
shipped game.toml stays portable across machines. But on Windows,
subprocess/CreateProcess does not resolve a relative *executable* against the
child cwd, so the relative --recompiler path failed with WinError 2 and the
overlay cache NEVER warmed -- a silent failure that left games ~100%
interpreted (sluggish) with no native payoff. abspath --recompiler and
--runtime-include up front (matching what compile_dll already does for gcc -I)
so relative config resolves correctly everywhere.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The launcher Hybrid controller mode (built first, for Tomba, to mimic Tomba:
Special Edition) flips the emulated pad's reported type 0x41<->0x73 on
d-pad<->stick. After the real DualShock config state machine was added for MMX6
(98aa688), that flip regressed every game's libpad re-detect into a 1-frame
"pad unplugged": Tomba unpauses its menu / drops input, MMX6 phantom-dashes.
The modern config SM is correct and required by MMX6, so instead of weakening it
globally, restore the pre-98aa688 "config commands always answer 0xF3" behaviour
as a per-game opt-in: [controller] legacy_pad_config (default OFF). Only Tomba
sets it; with the flag off the modern pad path is byte-for-byte unchanged, so no
other title is affected. A/B verified live: legacy = clean Tomba flip / MMX6
input; modern = Tomba disconnect + MMX6 dash.
Explicitly LEGACY: a correct findpad/re-detect refactor would make the flip
benign for every game with no per-game branch (real hardware tolerates unlimited
analog-button presses); when that lands this whole feature set is to be removed.
Full rationale in sio.c (g_pad_legacy_cfg).
Also includes the deferred coherent pad-type apply (pad_type_req, applied only at
an idle/non-config bus boundary so a flip can't split a poll/handshake) and a
`pad_cfg` debug command + tools/padcfg.py + tools/sio_flip_trace.py used to A/B
the config SM live.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
mingw gcc invoked by absolute path (the runtime's cmd.exe autocompile spawn,
or any launch without mingw64/bin on PATH) fails SILENTLY — exit 1, empty
stderr — because cc1/as/ld and the runtime DLLs resolve via PATH. _toolchain_env
auto-detects the bin dir ($PSX_MINGW_BIN, the --gcc path's dir, `which gcc`, then
common locations) and prepends it for the gcc subprocess. Fixes overlay-DLL
autocompile so it works regardless of launch environment.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The manual PSX_OVERLAY_CODEGEN_VER must be hand-bumped on every emitter change;
when it isn't (e.g. between CPS emitter iterations), a stale-but-cgN DLL is reused
-> the v0.3.0 black screen. Now hash_codegen.cmake hashes the recompiler codegen
sources into a (gitignored) overlay_codegen_hash.h at build time; overlay_api.h
exposes PSX_OVERLAY_CODEGEN_HASH and the cache path becomes gcc/<arch-abi>/cg<N>_<hash>/.
The loader and compile_overlays.py read the same value, so any emitter change
auto-invalidates the cache (old dirs coexist, no stale load). Falls back to hash 0
when the header is absent (fresh checkout) — identical to today's single-cgN path.
Verified: build emits hash 0x347751c1; autocompile + loader both use cg4_347751c1.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Vulkan backend alongside software + OpenGL: geometry/textured-prim
pipelines, GPU-authoritative VRAM image, deferred-submit + drain-at-present
perf model, GPU-direct present (vk_renderer_present_vram), and a
full-VRAM-pack readback path (ensure_cpu rect_adds the whole VRAM rect
before pack_flush so screenshots/guest VRAM reads reflect live content,
not just the incremental pack-dirty rect — readback-only, no per-frame cost).
Runtime plumbing: --debug-port / --renderer / --window-title CLI overrides
(multi-instance fleet), latency_ring, SPIR-V shaders + embed_spirv.py,
config_loader renderer selection, debug_server screenshot wiring.
All on feat/vulkan-renderer; snapshot in backup/vulkan-wip-20260617.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
RECURSION_BUG.md §25.5. Brings the overlay native path under continuation-passing so the
idle-freeze leak fix holds with overlay_cache=true (the production gcc>sljit>interp path).
Runtime CPS-mode flag g_psx_cps_mode (overlay_loader.c) set by a __attribute__((constructor))
emitted in the BIOS + game CPS dispatch (full_function_emitter.cpp / main_psx.cpp). Gates all
overlay/sljit CPS behavior; legacy unaffected.
sljit JIT (overlay_sljit.c), scoped:
- jr $ra publishes cpu->pc (was a bare return). jr rX already tail-transferred.
- call-containing fragments DECLINE to the interpreter (CPS-safe, depth-1) — leaf shards still
JIT. (Full sljit call coverage / per-fragment entry-switch is future work.)
gcc-DLL path — FULL native overlay coverage (validated):
- compile_overlays.py --cps sets PSX_CPS for the recompiler -> CPS overlay C (tail-transfer +
per-function entry-switch; --overlay mode only affects widescreen, not CPS). DISPATCH_PREAMBLE
psx_syscall decl fixed void->int.
- PSX_OVERLAY_CODEGEN_VER 3->4: fresh cg4/ cache namespace; legacy unit-model DLLs (cg3) ignored.
- overlay_loader_dispatch: under CPS, when a dispatched addr is not an overlay ENTRY but lies
inside a registered candidate's code range (a tail-transferred call's continuation),
overlay_find_by_range() re-enters that function with cpu->pc=addr so its entry-switch routes
to the block. gcc DLLs are the trusted tier (diff-validated at entry) -> run native directly.
This is the CPS equivalent of the old unit-model that fixed the dwarf->overworld blue screen.
Validated: CPS overlay DLLs compile + load + dispatch_native>0; with continuation routing
dispatch_interp_fallback -> 0 (full native), ce_profile FLAT (6 KB), boots clean. Addresses are
KSEG0-consistent.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
[widescreen.cull] auto_backdrop — generic far-backdrop column preload for
16:9. A shared detector (runtime/include/ws_backdrop_detect.h) finds each
scrolling-backdrop column-window generator by its /96-magic + 0x176-camX +
sra,5 divide-tail invariant and records the window's START/END bound
instructions + width (window_cols). All four execution paths route those
bounds through psx_ws_backdrop_value(orig, is_end, window_cols), which widens
the camera-tracked window by exactly the 16:9 reveal (margin scales with
window_cols; identity at 4:3 so the build stays byte-identical with widescreen
off). gcc emit + sljit + dirty-RAM interp + overlay ABI v4 / codegen ver 3.
Also: CodeGenConfig::overlay_mode — explicit [widescreen.*] per-address site
shape checks now SKIP (apply only where bytes match) instead of hard-exit in
overlay mode, because overlay regions reuse the same address for different
code across scene variants. Main-EXE mode keeps the hard error.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three coupled landings on top of the sljit merge:
1. Shared widescreen render-cull helper. auto_screen_x now emits a call to one
runtime helper psx_ws_cull_sltiu(sx,imm) (gpu.c) instead of an inline expr, and
the sljit JIT (overlay_sljit.c) + dirty-RAM interpreter (dirty_ram_interp.c)
route flagged render-funnel sltiu 0x140/0x141 through the SAME helper. So an
overlay's GTE render funnel widens identically whether it runs as a gcc DLL, a
sljit shard, or interpreted. Helper is byte-identical to vanilla at 4:3
(margin 0), so widescreen-off stays bit-exact. psx_ws_func_has_screen_cull
gates the widen to real render funnels (sltiu 0x140 + sltiu 0xE0 signature).
2. Versioned overlay cache path. gcc cache moves to <game_id>/gcc/<arch-abi>/cg<N>/
where N = PSX_OVERLAY_CODEGEN_VER (overlay_api.h), parsed identically by the
runtime (overlay_loader.c) and compile_overlays.py so they can't drift. A build
whose emitter output changed (e.g. this auto_screen_x change) reads+writes a
FRESH dir and never reuses a stale DLL; old versions coexist (no auto-delete).
Pre-1.0: legacy flat/unversioned fallbacks dropped.
3. sljit default-on (validated). gcc and sljit are complementary, not exclusive:
gcc fills the cache via prebuilt DLLs + async batch (compile_fragment NULL),
sljit JITs gcc-absent regions synchronously on-miss. New code_provider_sljit()
exposes sljit for the JIT-on-miss gap-fill independent of the "active" backend;
the dispatch gate fires whenever sljit is available (head<0 == gcc-absent), and
apply_live_policy defaults live ON when sljit is available. Priority gcc > sljit
> interp; shards still route through the same-state diff gate (device-touch +
diverging shards pinned to interp) so default-on does NOT reintroduce the
save-load wedge; a declined/failing shard falls to interp (never a crash).
Verified live: live=1, 62 shards compiled, 2 declines->interp, native:interp
ratio ~31%->~87%, gcc still primary.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SLJIT.md §7 step 1. gcc-compiled overlay DLLs now live under
<cache>/<game_id>/gcc/<arch-abi>/ so a same-OS different-arch DLL (and,
later, sljit blobs under sljit/<arch-abi>/) can never be confused for one
another. The backend-neutral coverage currency stays overlay_captures.json.
- overlay_loader.c: canonical PSX_OVERLAY_ARCH_ABI tag ("<os>-<arch>");
scan_cache_dir() refactored to scan the namespaced gcc dir AND the legacy
flat layout (<game_id>/*.dll) for back-compat — existing caches keep
loading with no migration. The legacy scan path is a provably-equivalent
refactor (same indexed paths as before).
- compile_overlays.py: cache_arch_abi() (lockstep with the C macro) writes
DLLs to the namespaced gcc dir; prior-ranges coverage merge falls back to
the legacy flat dir so pre-namespacing coverage still merges forward.
Verified: C and Python tags both resolve to win-x64 on this host; clean
rebuild; the 16 existing flat-layout Tomba DLLs remain findable.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 16:9 edge pop-in is the parallax 2D BACKDROP layer (ocean/cloud/mountain/
grass), drawn by overlay actor handlers (FUN_801216bc family) that compute
screenX = (worldX - camX) >> parallax in pure integer math and store it to the
object's +0x12 field WITHOUT the GTE — so the GTE X-squash that gives 3D the
wider 16:9 FOV never reaches them and far pieces sit past the 320px edge,
clipped (the blue void / half-rectangles at the edges). See TombaRecomp
ISSUES.md #8C for the full root-cause trail.
Fix = psx_ws_backdrop_x() (gpu.c): squash screenX around screen-centre by the
same ws factor as the GTE, identity at 4:3. Applied on BOTH execution paths
because overlay code runs interpreted when no cache DLL is loaded (the common
case in dev — see #8G):
- Native: recompiler emits write_half(.., psx_ws_backdrop_x(rt)) at the
[widescreen.backdrop] x_sites store (config_loader + code_generator +
main_psx --ws-config so the overlay compile gets the sites). Forwarded into
overlay DLLs via OverlayCallbacks {ws_backdrop_x, ws_x_margin, ws_sprite_tag}
(overlay ABI v2->v3 as the cache discriminator); compile_overlays.py glue +
--ws-config passthrough.
- Interp: dirty_ram_interp SH hook keyed by psx_ws_is_backdrop_site(pc); the
runtime registers the x_sites from game.toml (main.cpp).
Verified in data: backdrop slot screenX squashes 371 -> 318 (pulled on-screen).
PARTIAL: visible culling persists -> remaining cause is the backdrop spawn/
despawn window (separate from screenX clip); see #8C "REMAINING CAUSE".
Also added (earlier this session): ws_census draw ring, ws_margin override,
gpu_state ws fields.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Observability for the 16:9 backdrop-cull investigation (runtime-only,
4:3-identity-safe, no regen):
- gpu.c: always-on per-prim draw-census ring (ws_census). Every drawn GP0
primitive (0x20-0x7F) records {frame, src_addr, camX/camY, first-vertex
screen x/y, opcode} so object spawn/despawn and edge culls are observable
in data instead of screenshots. gpu_ws_census_dump() writes a frame range
to CSV (large dumps must not ride TCP).
- gpu.c: gpu_ws_set_margin_override() forces psx_ws_x_margin() to any value
for a live cull sweep at a fixed camera; gpu_ws_get_debug() snapshots live
widescreen state.
- debug_server.c: gpu_state now reports ws.{configured,active,game_mode,
present_native_43,x_margin,squash,cur_frame,last_tag_frame}; new ws_margin
and ws_census TCP commands.
- gpu.h: GpuWsDebug + census/override API.
- debug_client.py: ws_margin and ws_census subcommands.
Used to prove the shipped [widescreen.cull] sites are a no-op for 8C (their
4:3 margins already cover the 16:9 view) and that the real pop is an
overlay-resident backdrop layer. See TombaRecomp ISSUES.md #8C.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Overlay coverage (overlay_captures.json + the gcc-compiled cache DLLs) normally
lives only in gitignored build dirs, so a `cmake` clean or fresh worktree wipes
it (re-derivable only by replaying those areas). coverage_vault.py maintains one
additive vault OUTSIDE any build dir: `merge` unions captures by variant
(load_addr + bytes hash; same variant unions executed/entry PCs, distinct
variants all kept) and copies content-keyed cache DLLs (copy-if-absent/newer).
Wired into Tomba's overlay_autocompile_cmd so every compile backs up additively;
also runnable by hand to consolidate scattered build dirs. The vault holds
game-derived bytes, so it stays private/gitignored (this script does not).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The overlay cache keys DLLs by guest-bytes CRC, which is flavor-blind — the
same guest overlay yields the same filename whether compiled by the base
recompiler or a variant (e.g. widescreen, which emits psx_ws_*/GTE-adjust
symbols). Two flavors sharing a cache dir would collide: a base runtime could
pick up a widescreen DLL (or vice versa) with no guard, since both report ABI
v2.
Fold a codegen-flavor id into overlay_abi() (high 16 bits) and check the
combined tag in the loader's existing ABI gate. Base/master flavor is 0, so the
tag stays == PSX_OVERLAY_ABI_VERSION — fully backward compatible with every
existing base DLL (verified: an 8-DLL base cache loads with zero rejections).
A variant build sets PSX_OVERLAY_FLAVOR (e.g. widescreen: -DPSX_OVERLAY_FLAVOR=1
for the runtime, --flavor 1 for compile_overlays.py); its overlays then report a
distinct tag, so the base runtime rejects+regenerates them and vice versa. The
caches stay separate automatically, even in a shared directory — no new paths.
overlay_api.h: PSX_OVERLAY_FLAVOR (default 0) + PSX_OVERLAY_ABI_TAG.
overlay_loader.c: both ABI gates check the tag.
compile_overlays.py: --flavor arg -> -DPSX_OVERLAY_FLAVOR, baked into overlay_abi().
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per-player input routing with a real device dropdown (None / Keyboard /
each plugged-in SDL controller) and a digital-vs-DualShock pad-type toggle,
wired end-to-end through the runtime.
- sio.c/.h: per-slot pad state (pad_buttons[2]) + per-slot pad type. Analog
poll emulation (id 0x73 + 4 stick axes) when a slot is in DualShock mode;
digital (0x41) stays the default so existing games are unaffected. New
setters: sio_set_pad_state_slot / sio_set_pad_analog / sio_set_pad_connected,
getters sio_get_pad_{buttons,connected,analog}_slot.
- config_loader: settings.toml [controller] p1_device/p2_device (none/keyboard/
<SDL GUID>) + p1_analog/p2_analog; load + save round-trip.
- main.cpp: replaced the single auto-opened controller with a two-slot
PlayerInput model. Devices opened by GUID (with a first-free fallback so a
re-paired pad still works), keyboard or controller feeds each PSX port, and
analog stick axes drive the DualShock poll. Hot-plug re-resolves both slots.
Defaults: P1 keyboard/digital, P2 none.
- launcher: enumerates SDL controllers (name+GUID); each player card has a
custom click-to-open dropdown (option list injected via data-rml, the
data-for path being unreliable in this RmlUi build) and a DualShock switch
that drives the pad art (digital/analog) + status line. Zero-config default
selects the first plugged-in pad. A saved-but-unplugged controller is kept
as an "(offline)" option so the selection survives. Choices persist on launch.
- debug_server: pad_status now reports both slots' buttons + connected + analog.
UI polish from review:
- Removed the redundant disc-verdict widget (the green checks already convey it);
Change ISO now anchors to the disc panel instead of floating at the page top.
- Added a prominent "Settings" button in the top bar (renderer/SS/AA/etc. were
only reachable via the easily-missed footer link).
- Fixed the player-card asymmetry on toggle: status text no longer wraps and the
player card body is a fixed height, so the pad art renders at a consistent size
(measured 216x110 analog vs 219x110 digital) regardless of pad type.
Verified: device dropdown enumerates a real DualSense; selection + DualShock
toggle update the card; pad_status confirms per-slot connect + analog routing
(slot0 analog / slot1 digital and the reverse); default keyboard P1 unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the procedural placeholder art (disc/controllers/memory card/
logo) with the real PlayStation hardware cropped straight out of the
reference mockup, with the dark background knocked out to transparent so
each asset blends seamlessly on the panels — no seams, no border-glow
artifacts.
tools/crop_launcher_assets.ps1: crops the five assets from the mockup
and removes the background. Photo assets use an edge flood-fill (flood
the dark background inward from the borders; bright object pixels are
walls, so interior dark detail — the disc's PlayStation logo and centre
hole, the controller buttons — is preserved). The logo uses a global
luminance key (thin bright glyphs on near-black). PowerShell [int] of a
division rounds, not floors — the flood uses [Math]::Floor for the row
index.
launcher.rml: add `contain` fit to the photo decorators so the crops
keep their aspect ratio, and nudge the art sizes closer to the mockup's
proportions (disc 216, pads 206x128, memcard 118).
The check/verdict icons remain the generated PNGs (already clean +
transparent). Pixel-verified: dashboard renders with authentic disc,
PS1 digital pad, DualShock, and memory cards, all seamless on the panels.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rebuild the launcher as a card-based dashboard (n64recomp-style) per the
reference design, replacing the flat settings list.
Image pipeline: RenderInterface_GL3 only decodes TGA, so subclass it
(LauncherRenderInterface) and override LoadTexture to decode PNG via a
vendored stb_image (third_party/stb_image.h, STBI_ONLY_PNG, separate
impl TU), falling back to the base TGA path. This is done without
touching the pinned RmlUi submodule. RCSS now uses decorator:image(...)
for all art.
Layout (launcher.rml):
- Top bar: PS-glyph logo + "<game> Recomp".
- Disc Verification panel (Phase 3 — fully wired): disc art, FILE /
REGION / SERIAL, a three-check VERIFICATION RESULTS row
(Header Match / CRC-Hash Match / Disc Verified, driven by the
DiscIdentity fields) and a colour-coded verdict (icon + title +
detail). "Change ISO" re-runs verification.
- Player 1 / 2 and Memory Card 1 / 2 cards: full visual shell with
live enable toggles; controller enumeration (Phase 5) and memcard
introspection (Phase 4) wire in later — the rest is placeholder.
- Settings view (footer "Settings" <-> "Back"): the video / audio /
system controls moved off the home screen.
- Footer: Settings, recompiler-ready status, LAUNCH GAME.
launcher.cpp: model gains disc_file/region/serial + v_header/v_crc/
v_verified + verdict_{title,detail,state}, a view toggle, and card
enable bools; refresh_disc_status maps DiscIdentity onto the three
checks and the verdict; new events change_iso / show_settings /
show_dashboard / toggle_p1.p2.mc1.mc2.
Window bumped to 1280x924 for the dashboard. tools/gen_launcher_assets.ps1
generates the placeholder art (disc, controllers, memory card, PS-glyph
logo, check/verdict icons) into assets/img — swap for real renders later;
the PNG pipeline is identical.
Verified end-to-end on Tomba: dashboard renders with all three checks
green + "Tomba! disc verified"; Settings view and Back work; player /
memcard cards and toggles render.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Matthew Stanley