Keep selector zero stock and seal all choices until the next session restart. Refuse nonzero online activation, scope non-stock save states by frozen choices, and preserve original instruction guards when combined with effective-address adapters. Use overlay ABI34 on current master without the old menu dependency stack.
Extract the audited adapters and card transport onto master without the held selector/menu stack. Keep adapters and codecs tied to mod activation, use overlay ABI 33, retain rejected-load protection on codec registration, and refuse extended-card codecs in netplay before frame zero. Preserve default atomic save publication and guest-visible write failures.
Review of #561. Guest time now has two entry points (psx_cycles.h): the
CPU charge psx_cpu_charge() (instruction base, load fudge/region/completion,
i-cache refills, BIOS HLE costs), which applies [timing] guest_cycle_scale,
and the unscaled device advance psx_advance_cycles() (DMA, idle skips,
mul/div and GTE deadline waits). The emitters' block-mode charges emit
psx_cpu_charge; overlay DLLs accumulate it raw and publish through the
cpu_charge callback (was advance_cycles, same slot) into the same host
function, so nothing is scaled at the DLL boundary. The batched
psx_cyc_charge path, the dirty-RAM interpreter, psx_cyc_readmem, i-cache
refills and bios_hle.c all charge through it. At scale 1 it is exactly
psx_advance_cycles.
Gate in psxrecomp: [timing] guest_cycle_scale_gate, RAM predicates judged
at every VBlank edge (deterministic, works online, no mod needed), plus
psx_mod_set_guest_cycle_scale_gate() in mod_plugins.h for custom logic or
to tie the scale to a mod (guest_cycle_scale_gated = true; every mod
session reset shuts it). The carried fraction and both gate states are a
boot_state section (BS_SEC_GCS, written only when the scale is not 1), so
savestates and the netplay rollback snapshot re-simulate the same cycles.
psx_guest_cycle_scale_gate_test pins tier identity, the gates and rollback
re-simulation against the real psx_cycles.c.
OpenBIOS regenerated (emitter stamp); retail SCPH1001 is not.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fixes Ape Escape terrain seams: CPU-built subdivision midpoints and lwl/lwr-copied
grid corners drew native next to precise neighbours. Adds pgxp_tri_ring/pgxp_store_ring.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Preserve current segmented dispatch, active-plan mod hooks and software scale limits. Extend function filters through the active plan, retain the last-store pointer with overlay ABI 25, and keep default config fingerprints stable. Refresh BIOS emitter stamps after byte-identical regeneration and reconcile regression fixtures.
Bundled releases link OpenBIOS only; retail-BIOS-derived C never ships.
"Bring your own BIOS" now works without it: when a dump matching a shipped
profile (psx_bios_known_images.h) is selected, the runtime builds that
backend on the player's machine with the overlay toolchain the release
already carries, caches it, and loads it as one more registered backend.
- runtime/include/psx_bios_module.h: the module contract. An overlay-style
self-contained shared library (no imports from the executable) exporting
psx_bios_module_{abi,codegen_hash,init,backend}; reaches the runtime only
through OverlayCallbacks plus a new PsxBiosModuleCallbacks table (traps,
dirty-RAM/kernel-bless probes, fntrace, log_probe) and pointer aliases of
the globals the generated dispatcher touches. Its #defines rewrite the
emitter's own extern lines, so the generated C compiles unchanged.
- runtime/include/psx_bios_module_glue.c.inc: module-side forwarders and the
four exports, compiled after overlay_dispatch_preamble.c.inc.
- runtime/src/psx_bios_module.c: host side. Cache path
<exe>/cache/bios/<os-arch>/bm<abi>_<codegen hash>_f<flavor>/<STEM>_<crc>,
ABI + codegen-hash gated load, callback fill, and a blocking build via
overlay_toolchain/bios_module_build.py (gcc via compiler.txt or PATH,
else tcc), then psx_bios_register().
- psx_bios_backend.c: the registry is now a mutable table seeded from the
CMake-generated psx_bios_builtin_registry by a constructor, with
psx_bios_register(); every existing walker keeps working.
- main.cpp: validate_bios_for_launch acquires a module (announcing a first
build) before declaring a mismatch; discovery accepts a buildable dump;
the launcher verify reports it usable ("ready" / "compiled on first
launch") with no Generate & rebuild; the BIOS row shows whenever the
toolchain is present; bundled_only respects module support.
- emitter: the CPS-marker constructor is not emitted in a module build.
- overlay_api.h: OverlayCallbacks gains a struct tag;
overlay_loader_callbacks() exposes the host table before loader init.
- tools/bios_module_build.py (ships in the toolchain): identity + profile
SHA-256 gate, staged emitter tree, emit, three-TU compile, ctypes
self-check, atomic publish. release_stage.py stages psxrecomp-bios, every
bios/*.toml, the seeds and the builder into overlay_toolchain/.
- Tests: runtime/tests/test_bios_module_host.c against three fake modules
(good / wrong ABI / wrong hash) on real dlopen; recompiler/tests/
test_bios_module_build.py builds the bundled OpenBIOS end to end and reads
the descriptor back; guards extended.
Verified on Twisted Metal 4 (bundled build + staged toolchain, headless):
first launch with --bios SCPH1001.BIN builds and loads the module and boots
on image=SCPH-1001; the second launch loads it from cache.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The emitter writes g_debug_last_store_pc before every guest store. The
overlay preamble gave each DLL a private copy on the theory that only debug
tooling reads it. The runtime actually reads it in every build:
- memory.c psx_write_word_raw keys its RAM 0x0-0xF store filters on it;
- the fingerprint pc/mmio/sp columns use it, and so do wtrace,
record_frame, the MMIO/GP1 rings, parity provenance and the freeze
heartbeat.
So after a native overlay store the host still named an older store, and
those consumers saw an instruction the interpreter would not have.
OverlayCallbacks gains last_store_pc, which the loader always points at
the runtime's copy. In PSX_OVERLAY_DLL_BUILD, cpu_state.h maps the
emitted name onto that pointer, the same pattern as the bail-state
pointers. Generated code is unchanged. A NULL host pointer keeps the old
private copy.
The ABI goes to 24, so the loader rejects and regenerates older shards.
The preamble change already moves the codegen-hash cache path.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The repository root carried seven markdown files that were not onboarding
material, plus two that had been superseded by the code they described. This
sorts them: reader-facing docs to docs/, working notes to docs/internal/, and
the two superseded ones merged into the docs that replaced them.
Moved to docs/: WIDESCREEN.md, ENHANCEMENTS.md, TCP_COMMANDS.md
Moved to docs/internal/: PRINCIPLES.md, ISSUES.md, DEBUG.md, GTE_LM_FIX.md,
and from docs/: bios-hardening.md, overlay-status.md,
overlay-plan.md, overlay-recompilation-design.md,
overlay-discovery.md, TOMBA2_EXC_CONSULT_RESPONSE.md,
CASE_A_AOT_GAP.md, audit_inventory.md
Moved to docs/internal/upstream/: UPSTREAM_2026-08-20.md
The root keeps only what belongs there: README.md, CONTRIBUTING.md, LICENSE,
CLAUDE.md, THIRD_PARTY_ATTRIBUTION.md, RELEASE_NOTES.md, the dotfiles and
psxrecomp_cli.py.
MERGES
SPLITGEN_MIGRATION.md folded into docs/BUILDING.md as "Split generated C". Its
migration instructions were stale in a way that mattered: it told every game
repo to hand-edit its CMakeLists and add a file(GLOB ...) over
<serial>_full*.c, but runtime/runtime.cmake now does that itself. GEN_FULL_GLOB
is a multi-value argument of psxrecomp_add_game_runtime(); the loop at
runtime.cmake:1985 makes each pattern absolute against CMAKE_CURRENT_SOURCE_DIR,
expands it with file(GLOB), and concatenates the hits into
GAME_GENERATED_FULL_C. So a title names the pattern and gets however many shards
a regeneration produced. The rewritten section describes that, the two-step
fallback when the globs match nothing (GEN_FULL_FALLBACK, else <serial>_full.c
derived from the dispatch marker by substituting _dispatch.c), and the one
caveat the code actually has: that file(GLOB) is plain, not CONFIGURE_DEPENDS,
so a regeneration which changes the shard COUNT needs a re-configure to be seen.
psxrecomp-toml-readme.md folded into docs/GAME_PROJECT_SETUP.md as "EXE-only
autofill (psxrecomp-toml)", placed after "Disc autofill (probe_disc.py)" and
before "Project Studio" so the two autofill paths sit together. Its documented
flags were checked against the usage text in recompiler/src/main_toml.cpp:28-42
and against the argument parser at :150-165 -- all seven options match with no
drift, so the flag list is reproduced as-is. Added a pointer that probe_disc.py
is preferable when a cue exists, since that tool does the same EXE-header work
plus disc identity and netplay gates.
FUNCTIONAL FIXES
tools/gen_tcp_commands.py wrote to REPO/"TCP_COMMANDS.md". After the move that
path does not exist, so the generator would have created a new stray root file
and --check would have failed against it forever. Retargeted to
REPO/"docs"/"TCP_COMMANDS.md", along with the seven docstring and message
strings that name the path.
Running it surfaced two further problems, both fixed here:
* The doc was genuinely stale -- pre-existing drift, not introduced by this
commit (the moved file is byte-identical to origin/master, and no
debug-server command table is touched here). Four commands registered since
the last regeneration were missing from the index -- cdrom_timing_dump,
savestate_status, ws_ui_groups, xprobe_watch -- and scanline had gained a
backend. Regenerated: 307 -> 311 commands, and the change is confined to the
AUTOGEN block (the hand-written head and tail are byte-identical to master).
tools/gen_tcp_commands.py --check now exits 0.
* DOC.write_text() has no newline argument, so Python translated every newline
to os.linesep. On Windows that rewrote all 585 lines of the file to CRLF and
turned a four-line index update into a whole-file diff -- measured: 589 CRLF
/ 0 LF after a run, against 0 CRLF / 585 LF in the repo. Now written through
an explicit newline="" handle, verified to leave the file at 0 CRLF.
runtime/include/debug_server.h:9 and runtime/src/debug_server.c:8 cited "TCP.md
and DEBUG.md". TCP.md has never existed in this repo under that name; both now
name docs/TCP_COMMANDS.md and docs/internal/DEBUG.md.
LINK AND REFERENCE FIXES
Every inbound reference to a moved file was located by grepping each basename
across the whole tracked tree, not just the markdown. Markdown hrefs in
README.md, CONTRIBUTING.md, docs/ARCHITECTURE.md, docs/COMPILING_OVERLAYS.md,
docs/AOT_OVERLAY_PLAN.md, docs/RELOCATION_MANIFEST_FORMAT.md,
docs/config_schema.md, docs/OVERLAY_XREF.md, docs/TCP_COMMANDS.md and several
docs/internal/ files now point at the new locations; where two files became
siblings under docs/internal/ the path was shortened rather than lengthened
(bios-hardening.md's reference to overlay-discovery.md, for instance).
docs/COMPILING_OVERLAYS.md's "in this folder" phrasing was wrong once two of the
three files it listed moved, so it now names internal/ explicitly.
24 source-comment mentions of ENHANCEMENTS.md sections G1.x across 19 files
(recompiler, runtime, cmake) were retargeted to docs/ENHANCEMENTS.md with the
section numbers untouched. docs/SHADOW_ENHANCEMENTS.md is a DIFFERENT file and
was deliberately excluded.
20 rationale strings citing docs/CASE_A_AOT_GAP.md across the four
recompiler/seeds/*.json files were updated. Done as a byte-level substring
replacement rather than a JSON round-trip: two of those files are stored with
CRLF endings and two with LF, and re-serializing them reformatted 5,700 lines.
The byte-level version changes exactly 20 lines and each file still parses.
References that look similar but point elsewhere were deliberately left alone,
and are worth naming so a future pass does not "fix" them:
* "recomp-template PRINCIPLES.md" / "NES recomp template PRINCIPLES.md" in
CLAUDE.md:43, docs/SHADOW_ENHANCEMENTS.md:28,
docs/internal/HLE_SCHEDULER_CARVEOUT_PLAN.md:5,
runtime/src/debug_server.c:4934 and tools/parity_diff.py:8 are a different
repository's file. Only the five that mean this repo's PRINCIPLES.md were
updated.
* Every remaining bare ISSUES.md mention is a GAME repo's tracker
(MegaManX6Recomp 2/7/8/9, TombaRecomp 6), not this repo's.
* docs/disc-speed.md:4,54,56 and docs/internal/bios-hardening.md:5 name
overlay-discovery as a BRANCH, not a file, so no path applies.
INDEXES
README.md's "New here?" table gains rows for docs/WIDESCREEN.md,
docs/ENHANCEMENTS.md and docs/TCP_COMMANDS.md. docs/internal/README.md gains
sections indexing all fourteen documents that moved in, plus an "Upstream
contributor reviews" section describing upstream/ and the new sweep file, plus a
"Removed" bullet for CYCLE_TIMING_ARCH.md (deleted in the previous commit)
pointing at FAITHFUL_TIMING_PLAN.md, CYCLE_MODEL_BEETLE.md and
PRECISE_IRQ_SLICE.md -- following the SLJIT_PERSIST_CACHE.md precedent already
in that file.
docs/ENHANCEMENTS.md:7 claimed framework bugs live in "game repos' ISSUES.md
until a framework tracker exists". A framework tracker does exist; it now says
so and names the Beads epic.
VERIFICATION
A link checker was written to scan every tracked *.md for relative links,
resolving each against the tree being scanned. Baseline at origin/master: 129
files, 137 relative links, 3 broken. After both commits: 127 files, 159 relative
links, 3 broken -- the SAME three, all pre-existing and untouched here:
docs/DATA_SHARDS.md -> INSTANT_LOADS_PARKED.md (never existed)
docs/DATA_SHARDS.md -> INSTANT_LOADS_SPIKE2.md (never existed)
docs/NETPLAY.md -> ../lib/recomp-net/README.md (inside a submodule)
Zero new broken links. The checker's own coverage was checked against a raw grep
for markdown link openers: the 3-link difference is C++ lambdas inside code
blocks, correctly not treated as links.
Also verified: py_compile over all 113 tracked tools/*.py plus psxrecomp_cli.py;
all four seed JSONs still parse; the test-registration guard still reports 123
test files all registered; recompiler/tests/test_coverage_vault.py still passes;
and no file in this commit had its line endings flipped (checked mechanically
across all 50 modified files, after two files were caught and redone).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The emitter emits a direct psx_game_text_native_ok() stale-static guard at block
entry (code_generator.cpp:115) and before every direct jal (:2402). That function
is DEFINED IN THE GENERATED DISPATCH SHARD (main_psx.cpp:1620), which links into
the executable. An overlay DLL is a separate module and cannot resolve it, and
overlay_dispatch_preamble.c.inc had no shim for it.
Found by bumping Tomba 2 from a 254-commit-old pin to master and regenerating its
shard cache against the identical vault:
before bump : built OK 2799, FAILED 1
after bump : built OK 2233, FAILED 109
107 of the 109 were the same error:
ld.exe: undefined reference to `psx_game_text_native_ok'
collect2.exe: error: ld returned 1 exit status
This is the fifth instance of one class, and overlay_api.h's own version history
records the previous four:
v9 faithful-timing callbacks overlay code failed to link
v10 gte_read_ctrl GTE-heavy overlays failed to LINK without shims,
silently blocking new coverage
v12 rfe_mark_escape without the shim any rfe-containing overlay
fails to LINK
v14 gte_precision_store_word the rfe / gte_read_ctrl class again
Every time the shape is identical: an emitter gains a runtime call, the overlay
ABI does not gain the shim, and the only symptom is shards that quietly fail to
build. The affected code then runs interpreted forever and nothing at runtime
says so. Severity is coverage, not correctness -- but that is exactly the defect
that let Ape Escape ship 100% interpreted without anyone noticing.
Fix follows the v10/v12/v14 pattern exactly:
- overlay_api.h: append game_text_native_ok to OverlayCallbacks, bump
PSX_OVERLAY_ABI_VERSION 21 -> 22, add the version-history note
- overlay_dispatch_preamble.c.inc: shim forwarding to g_cbs
- overlay_loader.c init_callbacks: host populates it from the dispatch shard
The NULL guard deliberately returns 0, not 1. Returning 0 routes the guest
through the dispatcher -- slower, never wrong. Returning 1 would let an overlay
run native code the guard was explicitly trying to reject.
Follow-up worth doing, tracked as beads-eio.3.97: five instances of one class
means the process is the bug, not the symbol. The emitter knows every helper name
it can emit and the preamble enumerates every shim; a test comparing those two
sets would have caught v9, v10, v12, v14 and this one at authoring time instead
of at some future title's link step.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Emitter side of the value-propagation engine (ENHANCEMENTS.md G1.10):
- recompiler/src/pgxp_hook_emitter.{h,cpp} (new, in the codegen-hash
surface): ONE hook grammar shared by BOTH emitters, wrapping loads/
stores/COP2 transfers and precision-carrying ALU ops (moves, add/sub,
or-merge, shifts, LUI, HI/LO, muldiv) with PGXP_*() macros. Clobberable
sources are captured into locals before the statement. Precision-
destroying ops (AND/XOR/SLT/...) are deliberately unhooked - validate-
on-read drops their stale shadows for free.
- code_generator.cpp: central append in translate_instruction; LWC2/SWC2
emissions capture the RAW loaded/stored word (a masked GTE register
write must validate against the word as loaded).
- strict_translator.cpp: translate() wraps translate_impl()'s c_code with
the shared grammar; deferred load-delay variants hook the psx_ldd temp.
- BIOS regenerated, fingerprints verified CHANGED (SCPH1001 + OpenBIOS;
42.6k/12.2k macro sites). Base runtime + 33/33 tests green against it.
- Gating: macros expand only under -DPSX_PGXP=1. runtime.cmake grows a
PGXP option on psxrecomp_add_runtime_target (defines PSX_PGXP=1 +
PSX_OVERLAY_FLAVOR=2, _pgxp exe suffix) and PSX_PGXP_VARIANT to build
the sibling target from the same generated tree (framework runtime +
game wrapper). No ABI version bump needed: the appended PGXPHooks
table is NULL-tolerant and the flavor half of the ABI tag rejects any
host/DLL flavor mix.
- Shard caches: flavor folded into the cache tag (cg<V>_<hash>_gc<hash>
_f<flavor>) in overlay_loader.c + compile_overlays.py so base and pgxp
caches never share a directory; compile_overlays.py passes
-DPSX_PGXP=1 when flavor bit 2 is set; overlay preamble forwards
psx_pgxp_* through the table. PSX_OVERLAY_CODEGEN_VER 9 -> 10.
- Env overrides PSX_GEOMETRY_CORRECTION / PSX_PERSPECTIVE_TEXTURING /
PSX_PGXP_CPU_MODE (validation path: free-running headless boots arm
from the first vertex); census grew produced/swc2_stores and the pgxp
verb reports suppress/active gate state.
Validated: PSXRecomp_pgxp.exe boots the recompiled BIOS with hooks armed
(engine active, per-vertex lookups flowing; the BIOS shell has no RTPS
content so produced=0 there is correct - coverage measurement moves to a
3D title in phase 3).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add opt-in [widescreen.cull] xclip_load_sites: a configured `lw rt,off(rs)`
loading a per-primitive X-reject bound (sltu-compared against ANDI-masked
u16 screen X) routes through psx_ws_xclip_bound() - INT32_MAX while the
margins are revealed (reject disabled; the wide-surface scissor clips the
overflow, wrapped off-left u16 coords pass), vanilla value at 4:3. Matched
native codegen emit and interpreter hook; overlay ABI v18 forwarder (NULL
preserves the vanilla loaded value); codegen ver 8. Parser, codegen and
overlay-mismatch regression coverage.
Split from #49: replaces the xclip_globals RAM poke with a recompile-time
instruction transform so native shards and the interpreter behave identically.
Add opt-in [widescreen.cull] plane_nx_sites: a configured `lw rt,off(rs)`
loading the X component of a side frustum-plane normal (sign-tested per
corner, dot = nx*px + nz*pz) routes through psx_ws_plane_nx() — scaled by
the inverse aspect factor (4*den)/(3*num) while the margins are revealed,
widening the plane cone by exactly atan((3*num)/(4*den)*tan(theta));
identity at 4:3. Matched native codegen emit and interpreter hook; overlay
ABI v17 forwarder (NULL preserves the vanilla normal); codegen ver 7.
Parser, codegen and overlay-mismatch regression coverage.
Split from #49: replaces the plane sign-test RAM pokes with a
recompile-time instruction transform so native shards and the interpreter
behave identically.
Add opt-in slti/sltiu far-bound scaling with matched native and interpreter semantics, correct immediate sign extension, and an overlay ABI forwarder for live aspect state.
Co-authored-by: Opok <novelasevilliousesp@gmail.com>
Add an opt-in horizontal low-edge transform for subu rd,zero,rt classifiers, with matching native and dirty-RAM interpreter behavior and guarded overlay variants.
Co-authored-by: Opok <novelasevilliousesp@gmail.com>
Two provably-correct framework fixes (independent of the AOT work) plus the
play-free overlay-extraction spike tooling and design doc.
Framework fixes (accurate, safe, complete):
- ABI v14: add gte_precision_store_word to OverlayCallbacks + preamble shim +
runtime population. The emitter emits a direct call for every swc2 (GTE
store-word); it is stateful runtime code (gte.cpp) so overlay DLLs must
forward or fail to LINK (undefined reference). Play-captured shards rarely
compiled a swc2; fuller static coverage surfaced the gap. Same pattern as
the v13 ws_player_x_bound / v10 gte_read_ctrl shims.
- Gate splitgen OFF in overlay mode (main_psx.cpp): the current recompiler's
unconditional split-TU output deletes <stem>_full.c and emits only shards,
which broke ALL overlay compilation (compile_overlays.py reads _full.c).
Overlays are single small TUs with no parallel-compile benefit; emit the
monolith. Restores overlay compile on the current tree.
AOT spike (docs/AOT_OVERLAY_PLAN.md + tools/aot_overlay_spike/, WIP):
- Play-free static overlay extraction from the disc image. Validated:
Tomba 1 reproduces 90% of played functions + ~1964 unplayed, live-verified
in-game (Dwarf Village runs native, zero runtime compile). Tomba 2 yields
661 functions byte-identical to the shipped/played vault (provably real),
all from the self-describing MAIN.EXE producer; the header-table BIN
producer needs loader-table RE (base recovery unreliable). Generality is
producer-dependent. Spike scripts are Tomba-specific proofs of concept
with hardcoded paths, not a finished tool.
- Correctness is guaranteed by the existing per-function code_crc dispatch
guard: a mis-positioned shard fails audit at compile or never fires at
runtime -> coverage loss, never wrong execution. Autocompile self-heals gaps.
CROSS-CUTTING: the ABI v14 change is part of the overlay codegen-hash source
set, so it ROLLS the cg tag (1f2ad026 -> e8484213). On master + a fw bump,
every title's overlay cache invalidates and needs regen+reshard. Branch only;
NOT pushed; other games' caches untouched. Needs user validation before merge.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
28165d6 (Einhander native-wide player bounds) rewrites configured
[widescreen] signed_x_bound LUI sites into psx_ws_player_x_bound(vanilla)
calls in code_generator.cpp — and that site loop runs in overlay_mode too, so
the call lands in overlay-resident code. But the shard link contract never
forwarded the symbol: any shard containing a bound site fails to LINK
(undefined psx_ws_player_x_bound) and that variant runs interpreted forever —
the same silent class as v10 gte_read_ctrl and v12 psx_rfe_mark_escape (now
loud under the shard observability suite, but still a coverage hole).
The clamp computes against the host's live widescreen state (gpu.c), so a
local copy can't work: add the ws_player_x_bound callback (appended last),
the preamble forwarder (identity fallback => correct 4:3 on a host that
predates it), and the loader init wiring. Bump the ABI to 13 so stale DLL
mixes are rejected and regenerate.
This also unblocks removing the sljit tier: sljit's emit_ws_player_x_bound
was the only other native path carrying the clamp; with the gcc shard path
forwarding it (and dirty_ram_interp already handling the site), the feature
no longer depends on sljit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The emitter emits psx_rfe_mark_escape() at every rfe — including in
overlay-resident exception-context code — but the dispatch preamble
never provided it, so any rfe-containing overlay failed to LINK
(undefined reference) and ran interpreted forever: the gte_read_ctrl
silent-coverage-blocker class again (11 of Tomba's vault shards hit
it). It mutates shared RFE-pending escape state (interrupts.c), so the
shim forwards unconditionally through OverlayCallbacks; ABI bump to 12
rejects pre-shim caches.
Found live during LOAD_TIME_ZERO E0 bring-up: the user's first pig
load triggered autocapture and the compile died on this link error.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The emitter routes mfc2/cfc2/mtc2/ctc2 on flag/IR/derived GTE registers
through direct gte_read_data/gte_read_ctrl/gte_write_data/gte_write_ctrl
calls (code_generator.cpp COP2 case), but the overlay DLL preamble had no
shims for them — any GTE-heavy overlay failed to LINK (undefined
gte_read_ctrl) and its region ran interpreted forever. That silently
blocked all new native coverage for the lava/Trolley attract-demo regions
after the caches rotated.
GTE state is shared runtime state, so the shims forward unconditionally
through OverlayCallbacks (a local copy would fork the flag/IR timeline).
ABI bumped 9 -> 10; the loader gate rejects+deletes pre-shim DLLs and
autocompile self-heals; plain gte_data[]/gte_ctrl[] accesses stay direct
CPUState reads in emitted code, matching the emitter.
Smoke-validated: previously-failing GTE-heavy captures now compile clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The cycle-accuracy emitter emits calls to 12 timing runtime functions
(psx_cyc_load_*, psx_icache_fetch, psx_muldiv_*, psx_gte_*, psx_slice_block,
...) + debug_server_cyc_observe + g_debug_last_store_pc. Overlay DLLs never
got these symbols, so every overlay failed to link -> 0 native shards ->
100% interpreter (unplayable). Fix (option A): expose the 12 via the overlay
callback ABI.
- overlay_api.h: +12 timing callbacks in OverlayCallbacks, ABI 8 -> 9.
- overlay_loader.c: populate the 12; add warn_on_cgtag_mismatch() (loud
warning when shards exist under the wrong cg-hash).
- compile_overlays.py: 12 preamble forwarders + local g_debug_last_store_pc
+ -DPSX_NO_DEBUG_TOOLS.
- cosim.c/main.cpp: input-cosim setpad + held pad (deterministic input).
- cosim.py: env-overridable COSIM_EXE/CWD/GAME.
KNOWN REGRESSION (patched on a follow-up branch): Tomba pause menu (triangle)
wedges. Root-caused: an IRQ taken mid-block resumes at the function entry
leader instead of the interrupted PC, re-zeroing the menu's VSync wait
counter (scratchpad 0x1F8001E8) every frame so it never reaches its target.
Not the scheduler (LLE reproduces) and not timing math -- interrupt-resume
re-entry.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
In-progress investigation work snapshotted before rebasing onto the
tcc-refactored psxrecomp master. Includes overlay ABI v8
(check_interrupts_at resume-PC callback for async RFE recovery) and
code_generator/full_function_emitter delay-slot cycle accounting.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root-caused the Tomba2 native logo-stuck bug: it is a guest-cycle-timeline
divergence between the compiled-overlay backend and the dirty-RAM interpreter,
surfacing through a Timer1 (0x1F801110) debounce loop at game pc 0x8008592C that
reads 1 (native) vs 2 (interp), forking logo->FMV. The handoff's "jalr target
divergence" was a benign store-PC artifact (read-watch proves a0=0xB870 and
t3=[a0]=0x800896E0 are identical in both backends); the scratchpad blind spot
was ruled out (separate sp fingerprint forks at the same frame).
First-divergence observability (debug_server.c, memory.c):
- frame_fingerprint: added separate sp (scratchpad) cumulative hash column +
per-entry cyc, so a scratchpad or cycle fork is classified, not hidden.
- Unified ordered access recorder: one buffer, kind = ramw/spw/mmiow/mmior/ramr,
array index = true execution order, per-entry cyc (psx_get_cycle_count).
record_frame_dump emits the unified stream; record_reads_dump filters reads.
- Targeted main-RAM read-watch (PSX_READ_WATCH="lo,hi" + g_ram_read_watch_active
guard in psx_read_word) — proves a load's value/address per backend.
- PSX_RECORD_FRAME env seed: arm the recorder from boot (race-free), never
arm-then-time.
Fix A (cycle-model unification, part 1): compiled-overlay code was built WITHOUT
PSX_ENABLE_BLOCK_CYCLES, so every psx_advance_cycles() in overlay code was
compiled out -> a function run as a native overlay charged ~0 cycles while the
same function via the interp charged per-instruction (~32x), the entire drift.
- overlay_api.h: OverlayCallbacks gains advance_cycles; ABI v6->v7 (rejects
stale zero-cycle DLLs).
- compile_overlays.py: build overlays with -DPSX_ENABLE_BLOCK_CYCLES=1 and
emit a psx_advance_cycles shim forwarding to the shared host timeline.
- overlay_loader.c: wire s_callbacks.advance_cycles = psx_advance_cycles.
Result: the first debounce now reads Timer1=2 in native (was 1); overlay cycle
charge went 9.5k -> 380k (interp 306k). Native still logo-stuck: a residual,
perfectly linear ~1 cycle/write over-charge remains (block.instruction_count
counts branch + delay-slot as 2 while the interp charges the pair as 1 step),
which flips a later debounce 3 vs 2. Fix A pt2 (cycle-exact block totals) is the
next step. wt/tomba2 only; no pin bump, no master merge, no other-game regen.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Brings the fail-closed native entry guard from wt/tomba2 (b5a6940 / bd60381) to
master, where it had never been merged — which is why Tomba (and any master-built
title) still wedged. This is the long-standing "blue screen" / shared-RAM-
divergence-between-interp-and-native class.
Root cause: an overlay function dispatched at a FOREIGN INTERIOR PC (a range-
ownership mismatch routed a PC the function does not own into it) hit the CPS
entry-switch `default: break`, which fell through and ran the function FROM ITS
TOP — re-running its prologue and corrupting shared CPU/RAM state, so the guest's
main loop then waits forever on a condition that never gets set (the 0xCF0 SIO
spin / blue screen). Reproduced on Tomba's Village of All Beginnings: a native
0x000E7000-region shard function entered at a foreign PC wedged; the interpreter
ran the same bytes correctly.
Fix: the generated CPS entry-switch for OVERLAY functions (config_.overlay_mode)
now FAILS CLOSED on an unknown non-zero continuation PC — it restores the PC and
calls psx_native_bad_entry instead of running from the top. overlay_loader_dispatch
consumes g_native_bad_entry and routes the PC to the sanctioned dirty-RAM
interpreter (the live bytes run correctly — NOT a stub/HLE). True-prologue entry
still runs from the top. Static (BIOS/boot-EXE) functions keep the legacy fall-
through. ABI bumped 5->6 so stale ABI-5 caches are rejected and rebuilt with the
guard.
Minimal port: kept only the fail-closed mechanism + its ABI plumbing. The Tomba2
host_aliases interior-routing and the native-block / CPS-probe diagnostics (which
depend on the overlay-cache-v2 multi-range Candidate fields not on master) were
intentionally omitted — the fail-closed default alone is the safety net.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The manual PSX_OVERLAY_CODEGEN_VER must be hand-bumped on every emitter change;
when it isn't (e.g. between CPS emitter iterations), a stale-but-cgN DLL is reused
-> the v0.3.0 black screen. Now hash_codegen.cmake hashes the recompiler codegen
sources into a (gitignored) overlay_codegen_hash.h at build time; overlay_api.h
exposes PSX_OVERLAY_CODEGEN_HASH and the cache path becomes gcc/<arch-abi>/cg<N>_<hash>/.
The loader and compile_overlays.py read the same value, so any emitter change
auto-invalidates the cache (old dirs coexist, no stale load). Falls back to hash 0
when the header is absent (fresh checkout) — identical to today's single-cgN path.
Verified: build emits hash 0x347751c1; autocompile + loader both use cg4_347751c1.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
RECURSION_BUG.md §25.5. Brings the overlay native path under continuation-passing so the
idle-freeze leak fix holds with overlay_cache=true (the production gcc>sljit>interp path).
Runtime CPS-mode flag g_psx_cps_mode (overlay_loader.c) set by a __attribute__((constructor))
emitted in the BIOS + game CPS dispatch (full_function_emitter.cpp / main_psx.cpp). Gates all
overlay/sljit CPS behavior; legacy unaffected.
sljit JIT (overlay_sljit.c), scoped:
- jr $ra publishes cpu->pc (was a bare return). jr rX already tail-transferred.
- call-containing fragments DECLINE to the interpreter (CPS-safe, depth-1) — leaf shards still
JIT. (Full sljit call coverage / per-fragment entry-switch is future work.)
gcc-DLL path — FULL native overlay coverage (validated):
- compile_overlays.py --cps sets PSX_CPS for the recompiler -> CPS overlay C (tail-transfer +
per-function entry-switch; --overlay mode only affects widescreen, not CPS). DISPATCH_PREAMBLE
psx_syscall decl fixed void->int.
- PSX_OVERLAY_CODEGEN_VER 3->4: fresh cg4/ cache namespace; legacy unit-model DLLs (cg3) ignored.
- overlay_loader_dispatch: under CPS, when a dispatched addr is not an overlay ENTRY but lies
inside a registered candidate's code range (a tail-transferred call's continuation),
overlay_find_by_range() re-enters that function with cpu->pc=addr so its entry-switch routes
to the block. gcc DLLs are the trusted tier (diff-validated at entry) -> run native directly.
This is the CPS equivalent of the old unit-model that fixed the dwarf->overworld blue screen.
Validated: CPS overlay DLLs compile + load + dispatch_native>0; with continuation routing
dispatch_interp_fallback -> 0 (full native), ce_profile FLAT (6 KB), boots clean. Addresses are
KSEG0-consistent.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The complete §25 fix: make BOTH recompiler emitters emit guest calls as
TAIL-TRANSFERS driven by the unified flat trampoline + the guest's own $ra/$sp,
instead of nesting one host C frame per guest call. The host stack can no longer
mirror unbounded guest re-entrancy — the architectural root of the ~14-min idle
freeze. Gated gen-time by PSX_CPS; legacy codegen byte-identical when unset.
VALIDATED (user): full-CPS Tomba boots BIOS -> FMV -> New Game -> village idle and
plays; ce_profile cur_max_kb FLAT at 5 KB from frame ~9.9k through 56,890 (turbo
soak) — blew past the ~50k-frame freeze threshold with the host stack bounded
(legacy climbed +1.17 KB/frame -> 48 MB -> freeze).
KEY CORRECTION to the prior session's §25.1: there are TWO emitters. The BIOS
(full_function_emitter.cpp) was done last session but the §8 leaking chain
(func_8001A954 ...) is GAME code, emitted by code_generator.cpp (via main_psx.cpp)
— never converted. Also, §25.1's "continuations not in the dispatch table"
hypothesis was wrong (they were always added); the real boot regressions were
found by dumping the always-on fntrace ring on the abnormal psx_dispatch return.
Emitter changes (gated PSX_CPS):
- full_function_emitter.cpp (BIOS): BIOS A0/B0/C0 vector runtime-fallback
tail-transfers instead of nested psx_dispatch_call (was the frame-0 exit).
- code_generator.cpp/.h (game): jal/jalr/jr-$ra/jr-table/branch-split/j-split ->
tail-transfer (jalr captures rs before the link, alias-safe); each call's
return point registered as a dispatchable continuation; entry-switch (keyed on
cpu->pc) at the top of every function AND every overlapping-alias body. 0
psx_call_contract / call_by_address emitted under CPS.
- main_psx.cpp: psx_dispatch_game_compiled gains continuation cases
(case cont: cpu->pc=cont; func_owner(cpu)); entries clear cpu->pc.
- strict_translator.cpp: syscall emits `if (psx_syscall(...)) return;` so a void
syscall falls through to the inline post-syscall jr $ra (real-HW EPC+4).
Runtime changes (the legacy "return via nested dispatch / cpu->pc=0" idioms that
exit the flat CPS trampoline):
- traps.c: psx_syscall returns int (0=void/fall-through, 1=transfer);
Enter/ExitCriticalSection return 0; case-3 ChangeThread-success returns 0 — the
fiber scheduler leaves cpu->pc=0 on resume, which was the frame-712 exit.
- cpu_state.h / overlay_api.h: psx_syscall signature void->int; overlay callback
ABI bumped 4->5 (stale overlay caches rejected).
- dirty_ram_interp.c: gate interp_enter_compiled behind PSX_HAS_GAME_DISPATCH
(un-breaks the BIOS-only build).
- main.cpp: dump the always-on fntrace ring tail + stack to psx_cps_exit_trace.json
on an abnormal psx_dispatch return (diagnostic; CLAUDE.md ring-buffer model).
Docs: RECURSION_BUG.md §25.2-§25.4. Remaining: dwarf->overworld playtest;
Stage 3 (sljit/overlay CPS parity); then consider making CPS default.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
[widescreen.cull] auto_backdrop — generic far-backdrop column preload for
16:9. A shared detector (runtime/include/ws_backdrop_detect.h) finds each
scrolling-backdrop column-window generator by its /96-magic + 0x176-camX +
sra,5 divide-tail invariant and records the window's START/END bound
instructions + width (window_cols). All four execution paths route those
bounds through psx_ws_backdrop_value(orig, is_end, window_cols), which widens
the camera-tracked window by exactly the 16:9 reveal (margin scales with
window_cols; identity at 4:3 so the build stays byte-identical with widescreen
off). gcc emit + sljit + dirty-RAM interp + overlay ABI v4 / codegen ver 3.
Also: CodeGenConfig::overlay_mode — explicit [widescreen.*] per-address site
shape checks now SKIP (apply only where bytes match) instead of hard-exit in
overlay mode, because overlay regions reuse the same address for different
code across scene variants. Main-EXE mode keeps the hard error.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three coupled landings on top of the sljit merge:
1. Shared widescreen render-cull helper. auto_screen_x now emits a call to one
runtime helper psx_ws_cull_sltiu(sx,imm) (gpu.c) instead of an inline expr, and
the sljit JIT (overlay_sljit.c) + dirty-RAM interpreter (dirty_ram_interp.c)
route flagged render-funnel sltiu 0x140/0x141 through the SAME helper. So an
overlay's GTE render funnel widens identically whether it runs as a gcc DLL, a
sljit shard, or interpreted. Helper is byte-identical to vanilla at 4:3
(margin 0), so widescreen-off stays bit-exact. psx_ws_func_has_screen_cull
gates the widen to real render funnels (sltiu 0x140 + sltiu 0xE0 signature).
2. Versioned overlay cache path. gcc cache moves to <game_id>/gcc/<arch-abi>/cg<N>/
where N = PSX_OVERLAY_CODEGEN_VER (overlay_api.h), parsed identically by the
runtime (overlay_loader.c) and compile_overlays.py so they can't drift. A build
whose emitter output changed (e.g. this auto_screen_x change) reads+writes a
FRESH dir and never reuses a stale DLL; old versions coexist (no auto-delete).
Pre-1.0: legacy flat/unversioned fallbacks dropped.
3. sljit default-on (validated). gcc and sljit are complementary, not exclusive:
gcc fills the cache via prebuilt DLLs + async batch (compile_fragment NULL),
sljit JITs gcc-absent regions synchronously on-miss. New code_provider_sljit()
exposes sljit for the JIT-on-miss gap-fill independent of the "active" backend;
the dispatch gate fires whenever sljit is available (head<0 == gcc-absent), and
apply_live_policy defaults live ON when sljit is available. Priority gcc > sljit
> interp; shards still route through the same-state diff gate (device-touch +
diverging shards pinned to interp) so default-on does NOT reintroduce the
save-load wedge; a declined/failing shard falls to interp (never a crash).
Verified live: live=1, 62 shards compiled, 2 declines->interp, native:interp
ratio ~31%->~87%, gcc still primary.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Matt Stanley