Review of #561. Guest time now has two entry points (psx_cycles.h): the
CPU charge psx_cpu_charge() (instruction base, load fudge/region/completion,
i-cache refills, BIOS HLE costs), which applies [timing] guest_cycle_scale,
and the unscaled device advance psx_advance_cycles() (DMA, idle skips,
mul/div and GTE deadline waits). The emitters' block-mode charges emit
psx_cpu_charge; overlay DLLs accumulate it raw and publish through the
cpu_charge callback (was advance_cycles, same slot) into the same host
function, so nothing is scaled at the DLL boundary. The batched
psx_cyc_charge path, the dirty-RAM interpreter, psx_cyc_readmem, i-cache
refills and bios_hle.c all charge through it. At scale 1 it is exactly
psx_advance_cycles.
Gate in psxrecomp: [timing] guest_cycle_scale_gate, RAM predicates judged
at every VBlank edge (deterministic, works online, no mod needed), plus
psx_mod_set_guest_cycle_scale_gate() in mod_plugins.h for custom logic or
to tie the scale to a mod (guest_cycle_scale_gated = true; every mod
session reset shuts it). The carried fraction and both gate states are a
boot_state section (BS_SEC_GCS, written only when the scale is not 1), so
savestates and the netplay rollback snapshot re-simulate the same cycles.
psx_guest_cycle_scale_gate_test pins tier identity, the gates and rollback
re-simulation against the real psx_cycles.c.
OpenBIOS regenerated (emitter stamp); retail SCPH1001 is not.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bundled releases link OpenBIOS only; retail-BIOS-derived C never ships.
"Bring your own BIOS" now works without it: when a dump matching a shipped
profile (psx_bios_known_images.h) is selected, the runtime builds that
backend on the player's machine with the overlay toolchain the release
already carries, caches it, and loads it as one more registered backend.
- runtime/include/psx_bios_module.h: the module contract. An overlay-style
self-contained shared library (no imports from the executable) exporting
psx_bios_module_{abi,codegen_hash,init,backend}; reaches the runtime only
through OverlayCallbacks plus a new PsxBiosModuleCallbacks table (traps,
dirty-RAM/kernel-bless probes, fntrace, log_probe) and pointer aliases of
the globals the generated dispatcher touches. Its #defines rewrite the
emitter's own extern lines, so the generated C compiles unchanged.
- runtime/include/psx_bios_module_glue.c.inc: module-side forwarders and the
four exports, compiled after overlay_dispatch_preamble.c.inc.
- runtime/src/psx_bios_module.c: host side. Cache path
<exe>/cache/bios/<os-arch>/bm<abi>_<codegen hash>_f<flavor>/<STEM>_<crc>,
ABI + codegen-hash gated load, callback fill, and a blocking build via
overlay_toolchain/bios_module_build.py (gcc via compiler.txt or PATH,
else tcc), then psx_bios_register().
- psx_bios_backend.c: the registry is now a mutable table seeded from the
CMake-generated psx_bios_builtin_registry by a constructor, with
psx_bios_register(); every existing walker keeps working.
- main.cpp: validate_bios_for_launch acquires a module (announcing a first
build) before declaring a mismatch; discovery accepts a buildable dump;
the launcher verify reports it usable ("ready" / "compiled on first
launch") with no Generate & rebuild; the BIOS row shows whenever the
toolchain is present; bundled_only respects module support.
- emitter: the CPS-marker constructor is not emitted in a module build.
- overlay_api.h: OverlayCallbacks gains a struct tag;
overlay_loader_callbacks() exposes the host table before loader init.
- tools/bios_module_build.py (ships in the toolchain): identity + profile
SHA-256 gate, staged emitter tree, emit, three-TU compile, ctypes
self-check, atomic publish. release_stage.py stages psxrecomp-bios, every
bios/*.toml, the seeds and the builder into overlay_toolchain/.
- Tests: runtime/tests/test_bios_module_host.c against three fake modules
(good / wrong ABI / wrong hash) on real dlopen; recompiler/tests/
test_bios_module_build.py builds the bundled OpenBIOS end to end and reads
the descriptor back; guards extended.
Verified on Twisted Metal 4 (bundled build + staged toolchain, headless):
first launch with --bios SCPH1001.BIN builds and loads the module and boots
on image=SCPH-1001; the second launch loads it from cache.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
psx_call_contract and the BIOS dispatch loop compared a return PC with the
call's link physically ((a ^ b) & 0x1FFFFFFF). A PC carries a segment, and
the link a call writes is in the segment of the body that made it
(docs/SEGMENT_AWARE_CODE.md §5.5). A guest that returns to another
segment's alias of the call site did not return to that body, so letting
the suspended C continuation run there would execute it with the wrong
segment's PCs. §5.5 deferred these checks to this PR so one call contract
has one rule. All five are exact now:
- psx_call_contract (cpu_state.h): the bail-resolve PC and the $ra check;
- the emitted BIOS dispatch loop: the return boundary, the bail resolve, the
wild-return test and the exception-stack straddle.
Since B routes every baked link through runtime_pc() and C/D compile each
body for the segment it runs in, a legitimate return always matches in
full. The ledger gains the exact-return-contract guard: it compiles
psx_call_contract and checks that another segment's alias starts a bail and
resolves nothing, and that the emitted OpenBIOS dispatch has the four exact
checks and no masked one.
OpenBIOS and SCPH-1001 dispatch C change in those four lines; game C does
not change. cpu_state.h is in the codegen hash.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SCPH-1001's reset code enters its relocated kernel through the uncached
alias (lui $a0,0xA000; addiu $a0,$a0,0x500; jr $a0 at ROM 0xBFC0044C).
Beetle charges those four instructions as uncached fetches, 20 cycles; the
BIOS dispatch normalized 0xA0000500 to the kernel body compiled for runtime
0x00000500, which charges cached fetches, 11. That is the -9 cycles left at
SCPH-1001's kernel entry since #429 (docs/SEGMENT_AWARE_CODE.md §3.3).
A BIOS seed that is a runtime PC inside a relocated copy window, in another
segment than the window runs in, now asks for a segment variant (§5.4): the
function at those ROM bytes and its direct-edge closure (branch and J/JAL
targets that leave it, the call return and fall-through past its end) are
emitted again with the seed's segment. Every PC a variant computes from its
own PCs is in that segment (links, fetch tags and the uncached test, IRQ
resume, branch, J and fallthrough targets, store and exception PCs), a KSEG1
variant charges a fetch per instruction, and a jump-table case in another
segment is dispatched instead of continuing locally. Variants and their
continuations get exact-PC rows in a segment_variant_table. A variant seed
that names no emitted function entry stops the build, and so does one in a
segment that does not map physical memory (0x20000000-0x7FFFFFFF, KSEG2).
The dispatch still finds the word by its normalized key (and applies the
kernel-bless guard); a new psx_bios_hit_body() then picks the body for the
exact PC. A variant compiled for that PC runs. Any other PC in another
segment than the one the word's home body was compiled for still runs the
home body, as before, but is now recorded as a BIOS segment miss in the
segment-miss ring (kind "bios", with the home PC), so a new alias entry is
loud instead of silent (§5.5). psx_bios_key_home_pc(), emitted from the
address model, gives each key's home runtime PC: its copy window's runtime
address, or the ROM run in place. The ring, TCP segment_misses and the exit
report carry each entry's kind; tools/collect_game_misses.py lists a BIOS
segment miss for the BIOS seeds instead of writing it into the game's.
SCPH-1001's seeds gain 0xA0000500 (kernel_entry_kseg1): a 4-instruction
KSEG1 variant of kernel_trampoline_0. Every BIOS's dispatch C gains the two
helpers and calls psx_bios_hit_body() where it called the home body. A BIOS
without variant seeds (OpenBIOS, SCPH-101, SCPH-5552) keeps its full C;
SCPH-1001's full C gains the variant body and its dispatch C the one-row
segment_variant_table.
New bios_segment_variants test: OpenBIOS with KSEG1 variants of kernel
0x12BC (closure 0x6F8 by jal and 0x70C by fall-through) and 0x13F4 (0x1424
by a conditional branch only) and a KSEG0 variant of kernel memset 0x540 (a
KUSEG jump table), plus a static check of SCPH-1001's seed. It compiles the
emitted lookup, with and without variants, and queries every variant row,
every home key's home PC (checked against the PC each function body was
compiled for) and every other segment's alias of it. Seeds in a segment
that does not map physical memory stop the build, and a runtime PC in its
window's own segment stays a discovery seed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Segment-aware code, rollout PR B (docs/SEGMENT_AWARE_CODE.md §5.2).
Game/overlay emitter: CodeGenerator gains a code segment and
runtime_pc(addr) = code_seg | phys. Links (including a jalr in a branch
delay slot), fetch tags and the uncached-fetch test in emit_pre_icache,
interrupt resume PCs, CPS exit PCs and continuation keys, store-PC
stamps, the reserved-instruction EPC, the slice resume PC,
call_by_address and stale-guard targets, the call contract's return PC,
cosim PCs, the VSync-query hook's base PC, the data stubs'
unknown-dispatch PC and the dispatch rows' addr/resume_pc all go through
it. func_/block_ names, .ranges and entry-hook ids keep the compile
address. The VSync-query hook is not emitted for an uncached code
segment: its hand-timed body charges cached fetches only. The segment
defaults to the image's own, so runtime_pc(addr) == addr for every
existing input and KSEG0 titles regenerate byte for byte.
BIOS emitter: StrictTranslator::translate(d, runtime_pc) stamps stores,
the syscall EPC and the break and unaligned-access PCs with the PC the
code runs at; the psx_ldd_ temporaries keep the ROM address. The
full-function emitter passes relocate_ra() to every translate() call and
publishes the runtime PC on a fallthrough. Relocated kernel and shell
code therefore stamps its RAM address, as the interpreter does.
memory.c re-keys, in the same change, the seven SCPH-1001 store-PC keys
that named ROM addresses of relocated code: RAM-0 filters 0x00000F00
(ROM 0xBFC10A00), 0x80056EB4, 0x800585E4, 0x80058788, 0x80059C50 (ROM
0xBFC3EEB4/405E4/40788/41C50), the opt-in Tomba EvCB key 0x00001CE4 (ROM
0xBFC117E4) and the GP0 source key 0x80050B1C (ROM 0xBFC38B1C). Without
the re-key the new stamps would silently disable all seven. A RAM PC is
ambiguous where the ROM key was not, so each key goes through
scph1001_relocated_store(pc, rom): it matches only while the active
image is SCPH-1001 and the word at pc is still the ROM word copied
there. Other BIOSes and game code at those addresses never match, as
before.
Tests: emitter_runtime_pc_test (new, CTest, run with PSX_CPS unset and
set to 0) compiles a small program through generate_all_functions with
the code segment left default, KSEG0, KUSEG and KSEG1, in CPS,
CPS-overlay and legacy mode. It requires every emission path to be
reached and checks every guest-visible PC class, the identity keys,
physical-only differences and KSEG1 per-instruction fetch charging, and
the translator's runtime PCs in every PC-bearing form. The ledger gains
two regression guards: bios-runtime-pc (OpenBIOS output, plus every
translate() call site) and store-pc-keys-runtime (memory.c keys against
SCPH1001.toml's windows). KNOWN_GAPS is unchanged: B closes no ledger id
on its own (§5.2).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The cached line-leader rule charges a fetch at every 16-byte line start of
the runtime PC. The BIOS emitter tested bits[3:0] of the ROM address instead
and assumed every copy window preserves them. OpenBIOS copies its kernel from
ROM 0x1FC1E4D4 to RAM 0x500, which shifts them by 4. Each kernel line
crossing was charged one instruction early, where it is a hit, and the real
crossing went uncharged. 1,152 kernel instruction sites differed from the
interpreter.
The test now uses the runtime PC that relocate_ra already computes. In the
OpenBIOS kernel, 823 misplaced fetch sites go and 773 are added at runtime
line starts. Retail profiles are unaffected: SCPH-1001's copy windows are
16-byte aligned and SCPH-101/5552 declare none. The uncached_fetch_charge
ctest now also replays every cached OpenBIOS run from a cold cache against
the interpreter's fetch path and the Beetle transcription.
R4 (OpenBIOS, recompiled kernel, HLE boot) reaches the game entry 6,013
guest cycles later than with the uncached fix alone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Beetle's ReadInstruction never fills an I-cache line for a fetch at
0xA0000000 or above: every such fetch costs +4 and clears the pending load
give-back. The dirty-RAM interpreter charges that by fetching at every PC.
Both static emitters emitted psx_icache_fetch only at cache-line leaders,
which is exact for cached code only. In OpenBIOS, 5,477 of the 9,592
in-place ROM (KSEG1) instruction sites were uncharged: 21,908 cycles short
per pass through that code, each also missing its give-back clear.
Both emitters now charge a fetch before every instruction whose runtime PC
is uncached. The predicate is psx_fetch_uncached in psx_instr_cost.h, which
psx_icache.c now uses too. The A0/B0/C0 call-vector native stubs charge one
fetch per executed word, so a KSEG1 call pays like the interpreter. Cached
code keeps the leader rule. R4's regenerated game C is byte-identical, and
the OpenBIOS diff is insertions only. psx_instr_cost.h joins the codegen
hash list, since the emitters fold it into emitted code.
Tests:
- ctest uncached_fetch_charge runs psxrecomp-bios on OpenBIOS and compares
per-instruction fetch charges three ways: the compiled fetches, the
interpreter's fetch path (psx_icache_fetch_interp) at every PC, and a
Beetle transcription.
- uncached_fetch_codegen_test checks the game emitter at KSEG1 and KSEG0.
Both tests fail on master. This closes bios-kseg1-fetch-charge in the
segment_aware_codegen ledger (#419).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The generated psx_bios_try_native_call_stub reads the four live words at
the A0/B0/C0 call vectors and, when they match the expected stub, decodes
the handler address and tail-transfers to it without the dirty-RAM
interpreter. The guard knew one shape, the retail kernel's
lui $t0,hi; addiu $t0,$t0,lo; jr $t0; nop
OpenBIOS installs
addiu $t0,$zero,lo; jr $t0; nop; nop
so on OpenBIOS the guard failed closed and every kernel call paid an
interpreter entry for two instructions. Breath of Fire III yields through
B0:10h ChangeTh ~230 times per frame in the field: ~460 interpreted
instructions per frame, 87% of what OpenBIOS kernel RAM still interpreted
after the install-slot ranges (#346), and the same A/B reads 0 on retail.
Accept both shapes. The target is still decoded from live RAM on every
call and any other word pattern still fails closed (Rule 18). Cycle steps
charge the instructions the guest executes on each shape: four for the
retail stub, three for OpenBIOS (its jr's delay slot is word 2). Dispatch
totals are unchanged (OpenBIOS 3990, SCPH1001 13012).
Adds recompiler/tests/test_bios_vector_stub_shapes.py (registered with
CTest next to bios_backend_constant_codegen).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The BIOS exception handler ran interpreted for the life of every process,
1.22 billion instructions and 44.6% of all interpreted work in Breath of
Fire III, even though it is compiled and even though the stub that
unblessed it sat at exactly the address bios/SCPH1001.toml already
declared as an install slot. Two mechanisms existed and did not compose:
the emitter hooked the declared slot PC, while the runtime's bless
verifier memcmp'd the whole body including that slot, so the body was
MISMATCH forever and native code never reached the hook.
Three parts, none of which pays off alone:
1. Publish the declared ranges to the runtime. The emitter emits a
PsxKernelPatchRange table next to the kernel body table and routes it
through the backend descriptor; the bless verifier compares a body in
segments that skip them. CLEAN now means "every byte outside every
declared range still matches ROM", which is the claim the native code
actually depends on.
2. Generalise the slot model to what the patches are. The old model
assumed a 4-word lui/addiu/jalr/nop stub written over ROM zeros and
fired on `word != 0`. Measured against both images, only one of five
observed shapes fits: patchers also plant a `jr` over real ROM
instructions, rewrite an 11-word prologue and insert an `mfc0 Cause`,
and NOP out a driver routine. A slot is now a range with a length and
an explicit `resume` (jalr / fallthrough / none), detected by
comparing every word against its ROM-baked value.
3. Hand back from the interpreter at the range end. Kernel page 0 is
permanently dirty, because the handler saves registers there on every
exception, so straight-line interpretation would run the whole
function after the patch. The emitter registers the resume PC as a
continuation key and dirty_ram_interp surfaces there.
A PC inside a declared range must never be a native dispatch key: the
compiled bytes there are not what executes. Seven pre-existing
jal-return continuations violated that and wedged the boot at frame 0 in
an infinite dispatch loop (OpenBIOS RAM 0x357C). The emitter now drops
continuation keys inside a range so dispatch misses through to the
interpreter, and refuses to emit if a range covers a function entry,
which would silently remove a callable function instead.
Both profiles declare their measured ranges, and SCPH1001.toml finally
pins [program.image] sha256 to v2.2 (CRC32 37157331) so a v2.0 dump
refuses to generate rather than producing phantom discovery gaps.
Measured on Breath of Fire III (SLPS-00990, Psy-Q SDK 3.70), headless,
matched frames, PSX_KERNEL_PATCH_RANGES=0 for the OFF side:
OpenBIOS SCPH-1001 v2.2
bless mismatch 21 -> 0 74 -> 0
kernel bodies -87.7%/frame -97.8%/frame
all interp insns -42.6%/frame -94.2%/frame
The residual kernel-body work is exactly declared-words x entries on
every range, so nothing leaks past the hook. The A0/B0/C0 vector
trampolines are unchanged (+0.2%, noise): the profile excludes them by
design and they are a separate job. Card read traces are byte-identical
across the A/B on both images, all 32 entries, which exercises the two
ranges that replace the kernel's card handler.
Rule 18 holds throughout: the guest's patched instructions still execute
as written on the dirty-RAM interpreter. Only the code around them
changes backend. Nothing is synthesised.
Tests: runtime/tests/test_kernel_patch_ranges.c (new, the verifier's
accept/reject behaviour and the hand-back predicate) and the install-slot
range invariants in recompiler/tests/bios_address_model_test.cpp.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The launcher and studio ship as Retro Launcher and Retro Studio now,
from the RetroPortingToolKit org, so the old product name is gone from
display text, documentation and release asset filenames.
The split is by case, and it is not cosmetic. CamelCase RetComM was
always the product; lowercase retcomm and RETCOMM_ are identifiers --
the C++ namespace, the binaries, the data directories, the CMake and
environment variables, the Cloudflare service names, the readme markers.
Renaming those would orphan existing installs for no benefit, so they
are untouched: 1101 lowercase retcomm and 205 RETCOMM_ still stand in
the launcher alone.
Four things are deliberately still spelled RetComM, and three of them
would have been real breakage:
- RetComM-Data is CamelCase but it is a directory on disk --
<exe_dir>\RetComM-Data\{runtime,config,data} for a portable install.
Renaming it strands every portable user's config, toolchains and
catalog cache.
- TechnicallyComputers/RetComM-Launcher in readme_has_launcher() is the
historical half of a both-slugs test. Renaming it makes it a duplicate
of the line above and sends every port built before the transfer back
to reporting its launcher section missing.
- _LEGACY_LAUNCHER_RE matched "^## RetComM Launcher". A sweep renamed it,
which would have silently stopped it matching every README already out
there; it now accepts both headings.
- The portable-stub lookup gained the pre-rename filenames back as
fallbacks, because old zips stay on the releases page and a rollback
downloads one.
Release assets become Retro-Launcher-* / Retro-Studio-*. That is safe
for installed clients: pick_launcher_asset() gates on *linux*/*windows*/
*macos* plus extension and arch, not on the product name, and the
Windows portable stub is identified by its RCM1 trailer rather than its
filename.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013DpaQqVguHFpBuCmdoFYWY
The CFG emitter now points an LWL/LWR merge operand at the deferred
psx_ldd_<addr> temp, but full_function_emitter.cpp (BIOS / resident
functions) had the identical defect and was left untouched.
There, insn_reads_gpr already reports LWL/LWR as reading rt (it is the merge
base), so a `lw rt, ...` followed by `lwl/lwr rt, ...` was recorded as a
dependent pair and the load's writeback deferred. The successor then merged
cpu->gpr[rt] -- still the PRE-load value -- and because LWL/LWR
architecturally writes rt, the flush-vs-discard rule turned the deferred
writeback into `(void)psx_ldd_<addr>`. So the fetched word was dropped
entirely and the merge combined a stale register: silently wrong code, no
diagnostic.
This mirrors the CFG fix: record such pairs in ldd_lwlr_forward, exclude them
from the flush/discard normalisation, rewrite the successor's single
psx_old_rt merge-base initializer to read the temp, and emit a note instead
of a writeback. If strict_translator's emitted shape ever stops matching, the
emitter throws rather than quietly producing a stale merge.
Not currently live: neither generated BIOS contains a `(void)psx_ldd_` site
today, so this is latent-bug hardening. It matters because the interpreter
DOES forward here (dirty_ram_interp.c's s_ld_pend LWL/LWR exemption, added
after it wedged Tomba 2 at boot), so leaving the emitter unfixed keeps native
and interpreted execution in disagreement -- the divergence class that
produced the OpenBIOS memcard corruption.
Also adds recompiler/tests/test_lwlr_load_delay_forward.py, registered in
recompiler/CMakeLists.txt. It synthesises a PS-EXE with the dependent
lw/lwr pair and asserts the merge operand names the temp and that the
deferred load is not discarded. Verified to FAIL on master (merge operand
cpu->gpr[8], plus a discard) and pass with the fix.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An IRQ taken exactly at a polled call's return boundary swaps sp between
the task stack and the kernel exception/scratchpad stack, so a genuine
return (ra == stop_addr) can arrive with sp != sp_at_call in either
direction. The strict same-frame gate misread that as recursion and
re-dispatched the same delivery until the depth-257 guard tripped.
Accept the return only when ra matches the stop address and exactly one
of the two sp values lies in the scratchpad exception window; recursion
(different ra) and same-stack wild arrivals still take the strict gate,
so the Tomba Bug D protection is unchanged.
Fixes mission-1 freezes in Medal of Honor, MoH Underground, and Driver 2
(depth guard trip at 257 after tens of thousands of deliveries);
validated on those three plus Jersey Devil as control and by an
independent BIOS regen chain.
Groundwork for "no BIOS chosen -> OpenBIOS, explicit choice -> that BIOS"
(docs/BIOS_SELECTION.md). Shipping both images in one build removes the
per-BIOS build flavour, and with it the three restatements of the same
choice that nothing cross-checked.
Two recompiled BIOSes could not previously coexist: each generated
dispatch exported 16 globals, 14 of which were identical between images
(16 func_ bodies at coincident normalized addresses, the two dispatch
entry points, the per-image tables, and the dispatch depth).
Each image now exports exactly one descriptor, PsxBiosBackend
<STEM>_psx_bios_backend, carrying its image info, dispatch entry points
and kernel-body table. Everything else it defines is static or
stem-prefixed:
- func_ bodies and continuations take the stem prefix (fn_sym/cont_sym).
- psx_dispatch / psx_dispatch_call are renamed in the generated
preamble rather than at ~20 emit sites; the runtime keeps the
unprefixed names as forwarders through the active backend, so the
game's generated C and ~66 runtime call sites are untouched.
- the image info, kernel-body and native-stub tables become static.
psx_bios_image and psx_bios_kernel_bodies/_count are now published
by the runtime from the selected backend, so memory.c and the other
52 read sites are unchanged (the table is a pointer, indexing is
identical).
- g_psx_dispatch_depth moves to the runtime: it is shared dispatch
state, not per-image, and its duplicate definition was one of the
collisions.
The generated dispatch now includes psx_bios_backend.h instead of
re-declaring PsxKernelBody / PsxBiosImageInfo / PsxNativeStub locally —
those layouts had to be kept in sync with the runtime by hand, which the
old header comment admitted.
Verified by compiling both images and diffing their exported symbols:
dispatch objects export 3 stem-prefixed symbols each (was 16, 14
shared), full-body objects export 3911 and 13070, and the intersection
in both cases is empty.
Selection policy and the [runtime] openbios setting are not wired yet;
this is the linkability seam only.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds PCSX-Redux's MIT-licensed OpenBIOS as a second BIOS the framework can
statically recompile, next to retail SCPH1001. Because OpenBIOS is
redistributable, a game verified against it ships its BIOS with the build —
players supply only a disc, and the launcher BIOS picker disappears.
The mechanism is general, not OpenBIOS-specific: every BIOS image gets a
bios/<STEM>.toml profile that is the single source of truth for its identity
and layout.
Recompiler
- bios/<STEM>.toml profiles: [program.image] (sha256 pin, redistributable
flag), [recompiler.address_model] (boot-time ROM->RAM copy windows,
kernel-bless window), install_slots, and [recompiler.runtime_exports]
HLE anchors (0 = feature structurally unavailable on that BIOS).
- BiosAddressModel (new): validated model built from the profile, replacing
~9 hardcoded SCPH1001 window constants that had drifted apart across the
emitter / discovery / game codegen. The emitted C normalize() derives
from the same table. from_config rejects overlapping/misaligned windows
and orderings where sequential folding != first-match. A CTest-registered
equivalence test sweeps every RAM/ROM address against verbatim copies of
the old helpers.
- Orphaned delay slots (the OpenBIOS-critical emitter fixes): jr/jalr
slots are inlined before the jump resolves — previously their side
effect was silently LOST (e.g. a case-return fragment's addiu $sp pop,
which unwound a shifted frame and walked the exception handler's IntRP
cursor into a wild dispatch). Orphaned branches resume at ds+4 on
not-taken instead of re-executing the slot (a read-modify-write slot
doubling twice broke OpenBIOS readPad). Jump targets latch pre-slot.
- MIPS-I load-delay modeling for dependent load pairs: the successor sees
the pre-load value (OpenBIOS's cardfasttrack.s relies on it on purpose —
"gotta break those bad emulators"). Unmodeled shapes warn at build time.
- Discovery hardening: branch-target dispatchability closure to fixpoint;
jump tables placed at jr+8 no longer walked as code; cross-function
continuations routed by an instruction->owner map.
- Signed MULT support (unreachable from the retail seed set; modern-GCC
OpenBIOS emits it freely), latency-modeled like MULTU.
- Seeds from ELF symbol tables + data-section code-pointer harvest
(tools/gen_bios_seeds_from_elf.py) — a source-built BIOS needs no
disassembly pass. 646 OpenBIOS seeds, incl. 5 empirical dispatch-miss
seeds.
- Stemmed outputs (<out_stem>_full.c / _dispatch.c) so two BIOSes coexist
in one out_dir. tools/regen_bios.sh is profile-driven (--config).
Runtime
- psx_bios_image: the generated dispatch now self-describes the BIOS it
was compiled from (identity: size/CRC32/SHA-256/wordsum; kernel-bless
window; HLE anchors; bundled flag). Runtime SCPH1001 hardcodes removed.
- Launch check is a hard identity gate (a mismatched image under
statically recompiled code is a guaranteed wild jump). Bundled builds
hide the BIOS picker and treat a missing/tampered bundled image as an
installation defect.
- Faithfulness fixes surfaced by OpenBIOS's tighter timing loops (all
global, argued in code comments, regression-tested against SCPH1001):
- SIO pad RX is cycle-paced like the memcard path (RX_RDY at
shift-complete, ACK +170); total ack-IRQ latency unchanged.
PSX_SIO_PAD_SYNC_RX=1 restores the legacy synchronous path for A/B.
- SPUSTAT derived from SPUCNT instead of permanently-asserted busy.
- GPUSTAT.13 (interlace field) alternates per vblank while interlaced,
pinned 0 in progressive.
- CD Init(0x0A) completes in ~3.9 ms instead of borrowing Beetle's
34 ms Reset figure (OpenBIOS's short bounded retry loop livelocked).
- Exception EPC guard accepts ROM-window block leaders, not just RAM.
- New TCP debug commands (documented in TCP_COMMANDS.md): bios_info,
s3_smear_watch, callret_watch; write-trace ring gains $s0-$s5.
bios/openbios.bin is vendored (MIT — notice in bios/OpenBIOS.LICENSE,
attribution in THIRD_PARTY_ATTRIBUTION.md, source pin + build recipe in
bios/OpenBIOS.toml). The emitter fingerprint digest is now content-only:
existing generated/*.emitter.sha will read STALE once; re-run
tools/regen_bios.sh.
The tier mis-compiled and has been hard-gated off since 2026-06-25; overlay
gaps fall to gcc/tcc shards and the dirty-RAM interpreter, unchanged.
Re-do on master of the removal validated on the stale branch (6da8c02),
reconciled with everything master grew since the fork:
- Delete overlay_sljit.{c,h}, overlay_compile_worker.{c,h}, lib/sljit,
runtime/tests/sljit_*.c, and the runtime.cmake sources/include-dirs.
- Strip the sljit provider/backend from code_provider.{c,h},
overlay_backend.{c,h} ("sljit" in overlay_backend/PSX_OVERLAY_BACKEND now
degrades to auto so stale configs still boot), main.cpp, debug_server.c
(sljit_status/sljit_async/sljit_try TCP commands), overlay_loader.{c,h}
(live policy, JIT-on-miss, persisted-shard cache, async worker glue,
psx_sljit_call/cop2/memx host helpers — reachable only via the removed
cpu->sljit_helpers table), and cpu_state.h (SLJIT_HLP_* enum + the
sljit_helpers[] member; CPUState layout change reshards via the cg tag).
- Master-only entanglement resolved before deletion: 28165d6's Einhander WS
player-X bound emit existed in three paths (gcc code_generator, sljit,
dirty_ram_interp). The gcc shard link contract now forwards
psx_ws_player_x_bound (ABI v13, previous commit), so deleting the sljit
emit path loses no widescreen behavior; ffc6688's game-dispatch block
inside the dead psx_sljit_call helper goes with it.
- Reword comments that described sljit as a live backend (gpu.{c,h},
ws_*_detect.h, cosim_state.c, dirty_ram_interp.c, autocompile.h,
boot_state.h, config_loader.h, compile_overlays.py, and the CPS marker
comments the recompiler emits).
recompiler psxrecomp-game builds clean; game runtimes rebuilt + boot-verified
separately.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- psx_cyc.h / psx_cycles.c: _BitScanForward/_BitScanReverse shims for
__builtin_ctz/__builtin_clz under _MSC_VER
- full_function_emitter.cpp / main_psx.cpp: emit a portable .CRT$XCU
init-pointer instead of GCC-only __attribute__((constructor)) in
generated dispatch/game code
- main.cpp: declare cross-language globals extern "C" so MSVC links
them against their C definitions (no-op for GCC/Clang)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The generated dispatcher ran kernel-window table hits (RAM [0x500,0x8500),
the BIOS's boot-time copy of ROM 0x1FC10000+) natively with NO check that
the live RAM bytes still match the ROM image the functions were compiled
from — unsound for runtime-patched kernel code (the BIOS installs pad/SIO
handlers INSIDE compiled bodies; live diff: 15 clusters, 348 bytes). It
never misfired only because entry points happened to avoid patched keys.
A static kernel hit now requires psx_kernel_bless_dispatchable(): the
emitter emits psx_bios_kernel_bodies[] (key -> full RAM extent of the code
reachable from it; continuations carry their parent's extent, 499 entries),
and the runtime lazily byte-compares each entry's body against the ROM
source, caches the verdict, and invalidates precisely on writes into the
body (store chokepoint, DMA/EXE range marks, savestate restore). Clean ->
guarded native; patched/unverifiable -> the faithful dirty-RAM interpreter,
exactly as Rule 18 prescribes. This implements the runtime half of the
relocation-manifest contract (docs/RELOCATION_MANIFEST_FORMAT.md).
Measured on Tomba (SCUS-94236): 208-214 of 219 verified functions clean
(the 5 mismatches are the genuinely patched pad-driver family, faithfully
interpreted); millions of guarded native hits per boot; title 1.00x,
village 0.92x (parity — kernel keys were already native, now they are
PROVEN native); LLE and HLE boots + gameplay input verified. Kernel-window
interp measures 1.3% of interp burden (dirty_ram_stats), so this is a
soundness fix, not a speedup; the dominant interp cost is overlay-region
coverage (cg-tag hygiene), tracked separately.
Observability: TCP kernel_bless (entries/clean/mismatch/native_hits/
verifies/invalidations), always-on. PSX_KERNEL_BLESS=0 reverts to
unguarded-table behavior for A/B.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Boots Crash Bash (SCUS-94570) past its frame-~144 hang into the attract loop.
Root cause: the Class-A shell-window fix (normalize() maps RAM 0x30000-0x5AFFF ->
shell ROM) collides with games whose code lives in that window, and Crash Bash
RELOCATES a code page onto 0x30000 at runtime. Three interacting gaps sent that
real game code to dead shell ROM -> unknown-dispatch abort:
1. fntrace.c — game_started never latched: it only tripped on a dispatch to
entry_pc (0x2E7B0), but Crash Bash reaches entry via compiled internal flow,
so the latch never armed and the shell-window shadow stayed active. Now also
latches on the first dispatch to native-ok game text (safe pre-load: that RAM
still holds shell bytes that differ from the game image).
2. dirty_ram_interp.c — THE decisive fix. When a game-text address diverged from
the static EXE image (native_ok=0, i.e. runtime-relocated/overlaid code) but
was not "dirty", dirty_ram_dispatch_inner bailed (return 0) and the caller
shadowed it to shell ROM. Now a diverged game-text address is marked a clean
game-text miss so the dirty interpreter executes the live RAM bytes.
3. full_function_emitter.cpp (BIOS dispatch driver) — also route native-ok
game-text shell-window addresses to the game path regardless of the latch.
Likely redundant now that (1) latches correctly; flagged to slim during the
final cross-game validation.
NOT merged/validated against other titles yet (MMX6/Tomba2) — per plan that
happens after Crash is fully running. Verified: Crash Bash runs headless to
6000+ frames and windowed with no dispatch abort; the vblank fix delivers 60/s.
Known remaining issue: character flicker at the game-type menu (Bart's original
report) — under investigation, unrelated to this dispatch fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root cause of the fresh-launch logo freeze (present at every commit in the
stack; window-capture-verified 13/13 repros, never a regression): the BIOS
dispatch normalize() maps RAM phys 0x30000-0x5AFFF to the shell ROM alias.
The Class-A fix routed post-game-start shell-window targets to the game/dirty
path first, but its gate also required psx_game_address_in_text() — which
exempts runtime-loaded OVERLAY code above the boot-EXE text end. Tomba2's
text ends at 0x38800; its post-logo main-thread resume PC 0x50B80 fell
through to normalize() and executed func_1FC38B80, a stale shell-image
data-as-code self-loop, starving the main thread forever (one VBlank
delivery per frame at the same EPC, zero main-thread instructions retired;
the game's ISR ran fine, so TCP frame counters kept ticking — the freeze
was invisible to every telemetry-only validation). Even PSX_FORCE_INTERP
missed it: normalize() shadowed the address before the interp path saw it.
Fix: post-game-start, the WHOLE shell-overlap window routes to
dirty_ram_dispatch before normalize() — real hardware executes the RAM
bytes regardless of who loaded them (boot EXE, overlay loader, DMA stream).
Pre-start shell addresses still normalize to the shell as before.
Also:
- interrupts/debug_server: nestgate_* per-guard veto-attribution counters on
the nested-delivery gate (freeze_check), used to exonerate the IEc/SR
guards and redirect the hunt from interrupts.c to dispatch.
- tools/window_capture.py: PrintWindow-based SDL-window capture with
LIVE/FROZEN series verdict — window-liveness ground truth; TCP/VRAM
screenshots provably lie about presentation (this bug's entire history).
Validated: fresh launch now runs logo -> intro FMV -> title -> attract with
sustained full-frame motion (window-level capture series).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
USER PIVOT 2026-07-02 (CLAUDE.md §0 amendment included): psxrecomp now
carries an opt-in High-Level Emulation backend alongside LLE. LLE (the
recompiled BIOS) stays the default, the reference implementation, and the
correctness/timing oracle; every service HLE does not implement falls
through to LLE, so HLE is never load-bearing beyond what it covers.
Mechanism: full_function_emitter.cpp emits a null-by-default hook
g_psx_bios_hle_hook consulted at the top of every psx_dispatch_impl
iteration, on the pre-normalize physical address, BEFORE any backend
(game image / dirty-RAM / static table) claims the target. A handled
call resumes at $ra through the trampoline's existing return/tail
contract. NULL (default) = pure LLE, dispatch-identical. Regenerating
BIOS+game images is required (emitter change).
v1 call-HLE (runtime/src/bios_hle.c): the B0 event family — DeliverEvent
(incl. callback dispatch with the kernel-true $ra 0x1720), OpenEvent,
CloseEvent, TestEvent, EnableEvent, DisableEvent — each ground-truthed
instruction-by-instruction against the SCPH1001 kernel disassembly
(0xBFC11644..0xBFC11A84) and operating on the REAL guest EvCB table
([0x120]/[0x124], stride 0x1C). WaitEvent/threads/pads/card/A0/C0 fall
through to LLE. Cycle charges are documented approximations; LLE remains
the timing oracle. No-stubs stands.
Boot HLE: with the tier on (and no keep-intro opt-out) the hook one-shot
intercepts the first dispatch of the BIOS shell entry (RAM 0x30000 —
LoadRunShell's indirect call always routes through the dispatcher), so
BIOS Main() proceeds straight to SYSTEM.CNF + game EXE load with kernel
state built entirely by the real recompiled kernel init, unpaced (boot
turbo) until game handoff. This REPLACES the fast_boot snapshot
capture/restore; fast_boot=true now aliases the boot-skip alone.
Selection: [runtime] bios_hle / bios_hle_keep_intro (+ settings.toml
bios_hle mirror), PSX_BIOS_HLE / PSX_BIOS_HLE_KEEP_INTRO env overrides,
startup banner bios_backend=/bios_boot=. The PSX_HLE_SCHEDULER spike is
folded in as the tier's standing subsystem replacement: default from
[runtime] hle_scheduler via psx_hle_scheduler_set_default, env wins.
Observability: always-on 16K-entry HLE ring (route LLE-fallthrough /
HLE / boot-skip per record) + hle_dump TCP command (TCP_COMMANDS.md).
VALIDATED on Tomba (build-wt): LLE pass byte-identical behavior (real
intro, card check, save/load OK); HLE pass boots shell-less, TestEvent/
DeliverEvent route HLE, ChangeTh/ReturnFromException/OpenTh/CloseTh and
the card stack fall through to LLE, save/load OK (user-driven).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
recompiler/code_generator.cpp: stamp g_debug_last_store_pc before EVERY game
store (SB/SH/SWL/SW/SWR, SWC2, + the ws-backdrop / game-option special paths)
and declare the extern in the game-C preamble. Previously only the BIOS emitter
(strict_translator.cpp) stamped it, so game stores inherited the last BIOS store
pc -> wtrace/readtrace mis-attributed game writes to the BIOS. VERIFIED LIVE:
MMX6's producer trace for the 0x800CD3F8 write now reports the real game store
pc 0x8001A708 (matches Beetle), not the stale 0xBFC11BF0.
runtime: g_pc0_reason slice-exit provenance, fail-closed exception-sentinel
asserts, named thread-event kinds, freeze-heartbeat RVA emission (diagnostics,
default-on, additive).
Carries the in-progress faithful-timing / MDEC work (cpu_state.h, memory.c,
dma.c, interrupts.c, dirty_ram_interp.c, full_function_emitter.cpp) as an
uncommitted-WIP checkpoint. tools/: regen_bios.sh, bios_emitter_fingerprint.sh.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Charge the faithful R3000A I-cache fetch cost in BOTH static emitters at each
cache-line leader, completing the model (interp-only since Stage 1). A leader is
a block leader / mid-block jump-table target (a possibly-cold line entry) OR a
16-byte-line start (addr&0xC==0); intra-line fall-through followers are provable
hits (+0) and emit nothing. Emitted BEFORE the per-instruction interlock/load so
a fetch miss clears the pending load give-back first (Beetle ReadInstruction
order).
- code_generator.cpp (game): emit_pre_icache at the 3 timing sites (normal,
branch, delay slot); `addr` is already the KSEG0 runtime PC.
- full_function_emitter.cpp (BIOS): emit_icache_fetch at all 5 emit_insn_interlock
sites (main loop + 4 orphaned delay slots), mapping the ROM/compile addr to the
RUNTIME guest PC via relocate_ra (BIOS main stays KSEG1 0xBFC.. uncached;
kernel Part 2 -> 0x500+, shell -> 0x80030000+). Same address value the interp
passes for the same instruction, so the shared TV array stays consistent.
- psx_icache.c: flip psx_icache_enabled() DEFAULT ON (PSX_ICACHE=0 to disable).
Validated: ruler #2 compiled (4600) == Beetle (4382) EXACT on all 13 loops incl.
icache_miss +14 (was +0); ruler #1 steady delta 0 (56==56) with native cold-refill
spikes (77/84) now matching Beetle's range; Tomba 2 boots past the load wedge to
the intro FMV + PS BIOS logo, no freeze. Default-on path re-validated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implements the last unmodeled load-path cycle component — the R3000A
load-delay pipeline interlock (Beetle ReadAbsorb/ReadFudge/LDAbsorb/
LDWhich) — as ONE shared per-instruction model across the dirty interp
and BOTH static emitters, replacing the flat per-instruction +1 base and
the flat +4 main-RAM read wait-state.
Model (transcribed from beetle-psx cpu.cpp + libretro.cpp, verified Δ):
- §1 base: a pending load's per-register give-back (read_absorb) is
consumed INSTEAD of charging +1 (pipeline write-back overlap).
- GPR_DEPRES: each opcode's src/dst regs end the give-back on use.
- DO_LDS: commits the prior load's give-back + ReadFudge.
- load ReadMemory: +2 fudge iff the predecessor committed no load;
main-RAM region +3 + completion +2 (LWC2 +1) become the LDAbsorb
give-back; scratchpad +0.
New runtime/include/psx_cyc.h (inline §1/deps/DO_LDS + step) over new
CPUState timing fields; psx_cyc_load_*/psx_cyc_lwc2_read in memory.c
(region-aware, own the whole load interlock, value via uncharged
psx_read_*); psx_cyc_dep_res_mask in psx_instr_cost.h (per-opcode
GPR_DEP/RES, gen-time literal). cpu->read_* rewired to uncharged in
main.cpp; charge_main_ram_read removed. Wired before each instruction
body in code_generator.cpp (game) and full_function_emitter.cpp +
strict_translator.cpp (BIOS); loads route through psx_cyc_load_*.
Validated against the Beetle oracle:
- ruler #2 load2: native +10 -> +11 == Beetle (all other components
still exact: alu+1/load+5/div+38/mult+15/gte_rtps+15/gte_nclip+8).
- ruler #1 [0x80001C5C->0x80001CA4]: 54 -> 56 == Beetle steady-state
(residual 84/77 spikes = I-cache cold refill, the separate P2 axis).
- Tomba 2 boots to the intro FMV (pixels on screen, no regression).
Follow-up (separate commit): GTE-read (MFC2/CFC2) + MFC0 LDAbsorb
give-back, and muldiv-stall give-back consumption — needed for mixed
game/FMV faithfulness but do not affect the rulers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Apply the faithful mult/div completion-stall to the BIOS emitter so real kernel
code is cycle-accurate, not just the game-emitter test ROM.
- full_function_emitter.cpp: per-instruction cycle charging (default on, same
PSX_CODEGEN_CYCLE_PER_INSN gate as the game emitter). One psx_advance_cycles(1)
at the top of every in-function instruction (terminators, non-terminators, and
in-block delay slots are all separate addr_to_raw entries → execution order),
plus the 4 inlined orphaned-delay-slot sites. Block-up-front charge disabled in
per-insn mode. The +1-at-top convention keeps the divu-set / mflo-stall offsets
cancelling identically to the game emitter (verified exact).
- strict_translator.cpp: MULTU/DIV/DIVU emit psx_muldiv_set (DIV/DIVU=37, MULTU via
psx_mult_latency_u); MFHI/MFLO emit psx_muldiv_stall. (Signed MULT has zero BIOS
inventory, so it's unimplemented here as before.)
RESULT — ruler #1 [0x80001C5C->0x80001CA4] (BIOS EvCB-search prologue, divu+mflo):
native 30 -> 56, Beetle 56 — STEADY DELTA 0, EXACT. (The remaining 84-on-cold-hit
spikes are Beetle's I-cache line refill — the next component, fetch timing.)
Tomba 2 FMV no regression (i_stat 0x8D, full frame verified).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Build the game-independent cycle-attribution ruler the plan called for, and
fix two real bugs it exposed.
Tooling (debug_server.c, full_function_emitter.cpp):
- cyc_watch double-fire dedupe: observe fired from BOTH the dispatcher and the
function prologue at the same cycle, double-recording every dispatched entry.
Key on (phys,cycle); a genuine re-entry is always >=1 cycle later.
- Per-block-leader cycle observe (debug_server_cyc_observe), emitted by the BIOS
emitter under #ifndef PSX_NO_DEBUG_TOOLS (prod = zero overhead). Native now
samples at EVERY compiled block leader (matching Beetle's before-instruction
sample), so cyc_watch can anchor any block-leader PC, not just fn entries.
Emitted at normalize_address() so relocated-kernel anchors match the RAM space.
Bug fix (psx_instr_cost.h):
- Loads were double-counted. Stage-2 #1a (2ef47bd) returned 3 for loads (+2
data-access) in psx_instr_base_cycles WHILE memory.c charge_main_ram_read
already charged +6 per main-RAM read. The ruler region [0x80001C5C->0x80001CA4]
decomposed native=34 as 16(base) + 2 + 12(mem) with loads counted twice. The
opaque 0x80017FC4 window hid it because the load over-charge masked the entirely
unmodeled divu->mflo stall. Reverted the opcode fn to pure execute base (loads=1)
per its own stated contract; memory.c is the single address-keyed data-access
owner (like Beetle ReadMemory). Ruler: native 34->30 (exact), FMV unaffected.
Docs: CYCLE_MODEL_BEETLE.md (full R3000A model transcribed from in-tree Beetle
cpu.cpp, cited), plan + burndown updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Completes P3 step 1 across ALL cost producers: the BIOS emitter
(full_function_emitter.cpp) now sums psx_instr_base_cycles per block like the
interp and game emitter, instead of its own instruction count. Identity (1/insn)
=> BIOS regen byte-identical (verified). Stage-2 real R3000A costs now land in the
single psx_instr_base_cycles function and all three backends update together.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
In-progress investigation work snapshotted before rebasing onto the
tcc-refactored psxrecomp master. Includes overlay ABI v8
(check_interrupts_at resume-PC callback for async RFE recovery) and
code_generator/full_function_emitter delay-slot cycle accounting.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The continuation-passing leak fix (RECURSION_BUG.md §25) is validated (Tomba boots+plays,
ce_profile flat through a 56.9k-frame soak, full overlay CPS). Flip the gate in all three
emitter sites so a normal regen emits CPS; PSX_CPS=0 is the legacy escape hatch. Verified:
default regen -> CPS markers present; PSX_CPS=0 -> none; default-CPS Tomba boots clean, leak flat.
Doc §25.6.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
RECURSION_BUG.md §25.5. Brings the overlay native path under continuation-passing so the
idle-freeze leak fix holds with overlay_cache=true (the production gcc>sljit>interp path).
Runtime CPS-mode flag g_psx_cps_mode (overlay_loader.c) set by a __attribute__((constructor))
emitted in the BIOS + game CPS dispatch (full_function_emitter.cpp / main_psx.cpp). Gates all
overlay/sljit CPS behavior; legacy unaffected.
sljit JIT (overlay_sljit.c), scoped:
- jr $ra publishes cpu->pc (was a bare return). jr rX already tail-transferred.
- call-containing fragments DECLINE to the interpreter (CPS-safe, depth-1) — leaf shards still
JIT. (Full sljit call coverage / per-fragment entry-switch is future work.)
gcc-DLL path — FULL native overlay coverage (validated):
- compile_overlays.py --cps sets PSX_CPS for the recompiler -> CPS overlay C (tail-transfer +
per-function entry-switch; --overlay mode only affects widescreen, not CPS). DISPATCH_PREAMBLE
psx_syscall decl fixed void->int.
- PSX_OVERLAY_CODEGEN_VER 3->4: fresh cg4/ cache namespace; legacy unit-model DLLs (cg3) ignored.
- overlay_loader_dispatch: under CPS, when a dispatched addr is not an overlay ENTRY but lies
inside a registered candidate's code range (a tail-transferred call's continuation),
overlay_find_by_range() re-enters that function with cpu->pc=addr so its entry-switch routes
to the block. gcc DLLs are the trusted tier (diff-validated at entry) -> run native directly.
This is the CPS equivalent of the old unit-model that fixed the dwarf->overworld blue screen.
Validated: CPS overlay DLLs compile + load + dispatch_native>0; with continuation routing
dispatch_interp_fallback -> 0 (full native), ce_profile FLAT (6 KB), boots clean. Addresses are
KSEG0-consistent.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The complete §25 fix: make BOTH recompiler emitters emit guest calls as
TAIL-TRANSFERS driven by the unified flat trampoline + the guest's own $ra/$sp,
instead of nesting one host C frame per guest call. The host stack can no longer
mirror unbounded guest re-entrancy — the architectural root of the ~14-min idle
freeze. Gated gen-time by PSX_CPS; legacy codegen byte-identical when unset.
VALIDATED (user): full-CPS Tomba boots BIOS -> FMV -> New Game -> village idle and
plays; ce_profile cur_max_kb FLAT at 5 KB from frame ~9.9k through 56,890 (turbo
soak) — blew past the ~50k-frame freeze threshold with the host stack bounded
(legacy climbed +1.17 KB/frame -> 48 MB -> freeze).
KEY CORRECTION to the prior session's §25.1: there are TWO emitters. The BIOS
(full_function_emitter.cpp) was done last session but the §8 leaking chain
(func_8001A954 ...) is GAME code, emitted by code_generator.cpp (via main_psx.cpp)
— never converted. Also, §25.1's "continuations not in the dispatch table"
hypothesis was wrong (they were always added); the real boot regressions were
found by dumping the always-on fntrace ring on the abnormal psx_dispatch return.
Emitter changes (gated PSX_CPS):
- full_function_emitter.cpp (BIOS): BIOS A0/B0/C0 vector runtime-fallback
tail-transfers instead of nested psx_dispatch_call (was the frame-0 exit).
- code_generator.cpp/.h (game): jal/jalr/jr-$ra/jr-table/branch-split/j-split ->
tail-transfer (jalr captures rs before the link, alias-safe); each call's
return point registered as a dispatchable continuation; entry-switch (keyed on
cpu->pc) at the top of every function AND every overlapping-alias body. 0
psx_call_contract / call_by_address emitted under CPS.
- main_psx.cpp: psx_dispatch_game_compiled gains continuation cases
(case cont: cpu->pc=cont; func_owner(cpu)); entries clear cpu->pc.
- strict_translator.cpp: syscall emits `if (psx_syscall(...)) return;` so a void
syscall falls through to the inline post-syscall jr $ra (real-HW EPC+4).
Runtime changes (the legacy "return via nested dispatch / cpu->pc=0" idioms that
exit the flat CPS trampoline):
- traps.c: psx_syscall returns int (0=void/fall-through, 1=transfer);
Enter/ExitCriticalSection return 0; case-3 ChangeThread-success returns 0 — the
fiber scheduler leaves cpu->pc=0 on resume, which was the frame-712 exit.
- cpu_state.h / overlay_api.h: psx_syscall signature void->int; overlay callback
ABI bumped 4->5 (stale overlay caches rejected).
- dirty_ram_interp.c: gate interp_enter_compiled behind PSX_HAS_GAME_DISPATCH
(un-breaks the BIOS-only build).
- main.cpp: dump the always-on fntrace ring tail + stack to psx_cps_exit_trace.json
on an abnormal psx_dispatch return (diagnostic; CLAUDE.md ring-buffer model).
Docs: RECURSION_BUG.md §25.2-§25.4. Remaining: dwarf->overworld playtest;
Stage 3 (sljit/overlay CPS parity); then consider making CPS default.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The universal fix for the idle-freeze host-stack leak (RECURSION_BUG.md §25): make
the recompiler emit guest calls as TAIL-TRANSFERS driven by the trampoline + the
guest's own $ra/$sp, instead of nesting one psx_dispatch_impl per guest call frame.
This makes the host stack incapable of mirroring unbounded guest re-entrancy — the
architectural root of the leak — for ANY game. Perf cost accepted ("right before
fast").
Stage 1 (full_function_emitter.cpp), GATED behind gen-time env PSX_CPS:
- jal/jalr (orphaned-delay + pending-delay paths): emit `cpu->pc=target; return;`
(tail-transfer) + register the return address as a dispatchable continuation,
instead of nested psx_dispatch + psx_call_contract. JALR captures the target reg
BEFORE the link write (rd==rs alias-safe).
- jr $ra (all 3 sites): publish `cpu->pc = cpu->gpr[31]; return;` under CPS so the
trampoline dispatches the caller's continuation (legacy `return;` relied on the
nested-dispatch C-return). Required — without it the BIOS exits at frame 0.
GATING PROVEN SAFE: regen+build WITHOUT PSX_CPS boots Tomba normally — default
codegen byte-unaffected; CPS fully opt-in.
OPEN (CPS path only): with PSX_CPS=1 the BIOS boots main() then exits at frame 0
(execution completed, PC=0). Root cause (high confidence): the trampoline must
dispatch mid-function continuation addresses, but the dispatch table holds only
function entries → a return to a continuation misses → psx_unknown_dispatch →
pc=0 → exit. NEXT: add registered continuations to the dispatch table (map
continuation_addr → containing func), then re-regen PSX_CPS=1 and run the §25 gate
(ce_profile flat + oracle-identical + 15-min soak + dwarf->overworld). Stages 2
(interp interp_enter_compiled surface) + 3 (sljit/overlay parity) follow.
Safe revert point for the whole feature: commit ae47e69.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bug D root cause (Tomba chest freeze, also QA issue #1): the game's own
dispatcher func_80056E08 has a dead jumptable case (type 0x21 ->
jal 0x80120B3C) whose target is mid-function in EVERY overlay variant on
the disc (only the boot EXE references it; the real handler entry is
0x80120AC8). Entering mid-function skips the prologue, runs an epilogue
for a frame that was never pushed, and returns through garbage. Our C
model then resumed the suspended generated continuations on the moved
guest stack -> double-executed tails -> infinite zombie loop (~3K wild
calls/frame) -> MMIO fatal. Family root of Bugs A/C/D.
The contract: a generated C continuation may only run if the guest
actually arrived at it ($ra == call site return address, $sp == caller
sp at the call). Violations begin a "bail" unwind: cpu->pc := the wild
jr's true target, every generated frame returns immediately, and the
unwind resolves at the first enclosing site whose contract matches (or
flattens at the outermost dispatch loop, re-dispatching the wild target
on a clean host stack -- matching real-HW one-stack semantics).
- cpu_state.h: bail state + psx_call_contract() helper (+ overlay-DLL
pointer indirection under PSX_OVERLAY_DLL_BUILD)
- emitters: (ra, sp) checks at every BIOS and game jal/jalr site;
psx_dispatch_impl validates pc==0 and stop-hits against (stop, sp),
handles bail resolve/propagate/flatten
- dirty_ram_interp: surfaced calls validate the contract on return;
bail aborts the interp run; tail transfers into compiled code now run
with pc cleared (closes a latent double-run on plain C returns)
- overlay cache ABI v2: DLLs export overlay_abi(); loader rejects and
deletes stale DLLs (autocompile regenerates with contract checks)
- telemetry: bail_first/resolved/flattened/anomaly in heartbeat + dump
headers; anomaly guard at exception entry; TCP_COMMANDS.md docs
Verified: BIOS+Tomba regen, boot to intro at locked 60.0 fps, all bail
counters 0 through boot/title/attract (no false positives). Chest-area
gameplay verification pending (requires user play).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Pig-into-pig crash (TombaRecomp issue #1, v0.1.5): when a cached-DLL
native overlay function called uncached overlay code, the dirty interp's
overlay local-flow chained the callee's jr-ra straight past the nested
dispatch's return point (the target page is always dirty for overlays),
executing the native caller's tail in the interpreter. The suspended
native host frames then resumed and double-executed those tails against
a moved guest stack, restoring garbage callee-saved registers — main
loop's s1 mode-switch constant destroyed -> zombie loop (blue screen,
music on IRQs) or wild pointer -> fatal MMIO READ16.
Thread stop_addr (psx_dispatch_call's return_addr) through
dirty_ram_dispatch; local flow exits when any transfer or straight-line
fall-through reaches it, handing control back to the dispatch loop's
existing pc==stop_addr return. Signature change is deliberate: stale
generated dispatch fails to compile, forcing regen (BIOS + Tomba
regenerated; other titles regen on their own branches).
Sibling of the Bug A call-contract fix, on the return side. Root-caused
live via always-on rings on a wedged instance; user-verified fixed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
dispatch_stats TCP command: g_dispatch_static_hits incremented on every
binary-search hit in generated dispatch; exposed alongside miss_total and
miss_unique so static coverage ratio is measurable numerically.
fast_boot = true in [runtime]: first launch runs BIOS normally (logos
visible) and captures full system state at game handoff to
fast_boot_<bios_cksum>_<entry_pc>.bin adjacent to the BIOS. Every
subsequent launch restores the snapshot and dispatches directly to the
game entry PC — no BIOS execution, instant start.
Snapshot captures: CPU (gpr/hi/lo/cop0/gte), RAM (2 MB), scratchpad,
dirty-RAM bitmap, i_stat/i_mask, timer state. GPU/SPU/VRAM not included
(game reinitialises). Display-disabled vblanks now clear SDL to black so
the window never shows OS-background white during game crt0 setup.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
SIO handler alias (0x0CF0 → func_0000641C):
Runtime capture showed BIOS installs lui+addiu+jalr to RAM 0x641C at 0x0CF0.
New [[recompiler.bios_aliases]] TOML mechanism emits a one-liner wrapper and
registers it in the dispatch table. Eliminates 30K dirty_ram blocks/session.
Instant disc speed (disc_speed = "instant" in game.toml [runtime]):
cdrom_set_speed(0) collapses all seek/read delays to 1 cycle. Correct INT
sequence still fires; no artificial wait. "2x" and "4x" divisors available.
Default remains authentic 1x. Tomba set to instant.
Persistent game miss collector (tools/collect_game_misses.py):
Appends newly-seen dirty_ram dispatch misses in the game text range to a
file (TombaRecomp/seeds/dirty_ram_misses.txt) for later seeding.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Replaces dirty_ram_interp interpretation of BIOS vector trampolines with
statically-generated C switch handlers registered in the dispatch table.
For SCPH1001.BIN (a fixed binary), the A0/B0/C0 function pointer tables
are known at recompile time: A0 is pre-populated in ROM at 0xBFC04300,
B0 and C0 in the kernel image at 0xBFC10374 and 0xBFC10174 respectively.
The recompiler reads each table from the ROM binary and emits a switch
function (bios_vec_A0/B0/C0) registered as dispatch table entries for
RAM addresses 0xA0/0xB0/0xC0.
For kernel entries (ROM-encoded), calls go directly to the static C function
via the switch: zero interpretation overhead. For Shell-patched entries
(runtime-installed by the Shell at B0:0x32+ etc.), a RAM-table fallback reads
the live pointer via cpu->read_word and dispatches through psx_dispatch_call:
one memory read + dispatch instead of a 6-instruction trampoline interpretation.
New TOML fields in [[recompiler.bios_vectors]]:
ram_addr, index_reg, table_rom_addr, table_count, table_ram_addr (opt.)
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Henrique Guedes