The tier mis-compiled and has been hard-gated off since 2026-06-25; overlay
gaps fall to gcc/tcc shards and the dirty-RAM interpreter, unchanged.
Re-do on master of the removal validated on the stale branch (6da8c02),
reconciled with everything master grew since the fork:
- Delete overlay_sljit.{c,h}, overlay_compile_worker.{c,h}, lib/sljit,
runtime/tests/sljit_*.c, and the runtime.cmake sources/include-dirs.
- Strip the sljit provider/backend from code_provider.{c,h},
overlay_backend.{c,h} ("sljit" in overlay_backend/PSX_OVERLAY_BACKEND now
degrades to auto so stale configs still boot), main.cpp, debug_server.c
(sljit_status/sljit_async/sljit_try TCP commands), overlay_loader.{c,h}
(live policy, JIT-on-miss, persisted-shard cache, async worker glue,
psx_sljit_call/cop2/memx host helpers — reachable only via the removed
cpu->sljit_helpers table), and cpu_state.h (SLJIT_HLP_* enum + the
sljit_helpers[] member; CPUState layout change reshards via the cg tag).
- Master-only entanglement resolved before deletion: 28165d6's Einhander WS
player-X bound emit existed in three paths (gcc code_generator, sljit,
dirty_ram_interp). The gcc shard link contract now forwards
psx_ws_player_x_bound (ABI v13, previous commit), so deleting the sljit
emit path loses no widescreen behavior; ffc6688's game-dispatch block
inside the dead psx_sljit_call helper goes with it.
- Reword comments that described sljit as a live backend (gpu.{c,h},
ws_*_detect.h, cosim_state.c, dirty_ram_interp.c, autocompile.h,
boot_state.h, config_loader.h, compile_overlays.py, and the CPS marker
comments the recompiler emits).
recompiler psxrecomp-game builds clean; game runtimes rebuilt + boot-verified
separately.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Make the repo approachable for outside contributors and an IDE integrator:
- New contributor docs:
- docs/EXECUTION_MODEL.md — semi-technical tour of the static / native-overlay
/ interpreter execution model and how coverage gaps get plugged.
- docs/ARCHITECTURE.md — component-level view (recompiler, runtime, LLE+HLE
BIOS, overlay backend static->gcc->tcc, renderers, oracle model).
- docs/BUILDING.md — accurate dependency table, per-platform prereqs, build
steps, and the submodule + local-junction framework-linking setup.
- CONTRIBUTING.md — distilled core rules, verification expectations, PR flow.
- Declutter root: move ~14 internal dev-logs into docs/internal/ (history
preserved via rename) with an index README; repoint CLAUDE.md references.
- README: correct build recipe, LLE+HLE + overlay-backend summary, links to the
new docs.
- code_provider.c: update the stale backend comment to the real static->gcc->tcc
resolution (dead SLJIT.md reference removed).
- config_schema.md: drop the obsolete "once Phase A2/A3 ships --config" note.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Linux port (2 fixes — the POSIX paths largely existed already):
- overlay_loader.c: portable atomic for the async-cache dirty flag
(InterlockedExchange/LONG behind _WIN32, __atomic_exchange_n on POSIX)
- code_provider.c: missing <stddef.h> for NULL
memory.c — main-RAM mirroring (the Kula World boot-wedge root cause):
the 2 MB RAM is mirrored 4x across the first 8 MB of each segment
(mem-ctrl RAM_SIZE default 0x0B88) and Kula World's crt0 parks $sp in
the 4th mirror (0x807FFFF8). Guest accesses to mirrors fell into the
open-bus no-op: stack writes vanished, $ra loaded back 0, and the game
jumped to address 0 at frame ~891 (fntrace-verified call chain). All
guest access paths now fold mirrors via psx_phys_addr().
fntrace.c — null-page dispatch guard: a dispatch whose phys target is
below 0x40 latches the capture freeze, so the always-on ring preserves
the window that led to a null jump instead of the wedge storm evicting
it. This is how the mirror bug was pinned: the frozen ring showed crt0
set sp=0x807FFFF8, then ra came back 0 from the first mirrored-stack
return.
runtime/CMakeLists.txt: kula-runtime target (SCES-01000), guarded on
the locally-generated game C existing so clones without the (untracked)
game dump still configure. Verified on Linux/Xvfb: BIOS-only psx-runtime
boots to the shell menu; kula-runtime boots the disc, game code runs
native (mirror stack live), VBLANK event callback ticks, CD streams the
HIRO directory. Software renderer + TCP debug server work unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdjxoRB1f9VQsNzZayCgGS
Compiler-neutral tier-selection policy extracted from the mis-named
overlay_sljit.{c,h} into NEW overlay_backend.{c,h} (OverlayBackend enum +
resolve/name/active). overlay_sljit.{c,h} is now just the deprecated emitter.
New tcc tier. gcc and tcc run the IDENTICAL pipeline (recompiler -> C ->
compiler -> DLL -> loader); only the compiler binary differs, so both map to the
autocompile-spawn provider (code_provider.c) and the COMMAND is chosen by the
resolved backend (main.cpp). compile_overlays.py gains --compiler gcc|tcc: tcc
writes a separate tcc/ cache namespace, BOM-strips the C + a copy of the include
headers (tcc 0.9.27 can't skip a UTF-8 BOM), and uses tcc's built-in linker.
Backends (overlay_backend): auto (gcc if a gcc TOOLCHAIN is present, else tcc --
was sljit), gcc, tcc, auto-no-gcc (dev/test: force tcc even with gcc present, to
exercise the toolchain-less user path), sljit (DEPRECATED, gated off, lowest).
Priority: static > gcc shard > tcc shard > sljit(off) > interp. The loader scans
gcc/ then tcc/ and dedups by region+crc (filename) so a gcc shard always wins a
tie.
Self-contained tcc: with no explicit overlay_autocompile_cmd_tcc, the runtime
CONSTRUCTS the tcc command from a toolchain bundled beside the exe
(<exe>/overlay_toolchain/: embedded python + tcc + recompiler + compile_overlays.py
+ runtime headers) -- no system python or gcc needed. config_loader gains
overlay_autocompile_cmd_tcc (a dev override). autocompile cmd buffers enlarged for
the constructed path-heavy command.
Compilation stays OFF the dispatch/render thread (off-process autocompile
subprocess, same model as gcc); the main thread only spawns it and dlopens the
finished DLL.
Validated: MMX6 with overlay_backend=auto-no-gcc fills the tcc/ namespace live
from the bundled toolchain and plays through save-load + gameplay -- the exact
path the sljit tier softlocked on. The gcc path is untouched.
Runtime-only, no regen.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Persist validated sljit shards across launches (SLJIT_PERSIST_CACHE.md). On JIT,
serialize the (position-independent, Stage 1) LIR and write it to
<cache>/<game_id>/sljit/<arch-abi>/cg<N>/<entry8>_<crc8>.sljit with a small header
(magic, format/helper-order versions, entry/lo/len/crc/size). At init, scan that
dir, deserialize + sljit_generate_code for THIS process, and register each as a
dll<0 candidate carrying the blob's crc; dispatch re-hashes live RAM against it so
a reloaded shard re-earns trust exactly like a fresh JIT. Priority gcc > sljit >
interp is unchanged (a gcc DLL still supersedes a reloaded shard via the obsolete
path). Persist is write-on-generate (measured negligible; crash-safe).
Threads the serialized blob through OverlaySljitResult -> CompiledFragment;
register_sljit_candidate gains a crc_override (reloaded shards use the header crc
since the overlay RAM may not be loaded at init). sljit_status exposes
reloaded / persist / reload-diagnostic counters.
Two gotchas found + fixed in testing (vs the original design notes):
- serialize MUST precede generate_code (generate sets compiler->error =
SLJIT_ERR_COMPILED, after which serialize returns NULL);
- serialize with options=0, not SLJIT_SERIALIZE_IGNORE_DEBUG, so an
argument-checks build can deserialize its own blobs.
Validated: two-launch test reloaded 12 shards (deserfail=0), ran them at a fresh
process base with no crash, gameplay renders correctly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three coupled landings on top of the sljit merge:
1. Shared widescreen render-cull helper. auto_screen_x now emits a call to one
runtime helper psx_ws_cull_sltiu(sx,imm) (gpu.c) instead of an inline expr, and
the sljit JIT (overlay_sljit.c) + dirty-RAM interpreter (dirty_ram_interp.c)
route flagged render-funnel sltiu 0x140/0x141 through the SAME helper. So an
overlay's GTE render funnel widens identically whether it runs as a gcc DLL, a
sljit shard, or interpreted. Helper is byte-identical to vanilla at 4:3
(margin 0), so widescreen-off stays bit-exact. psx_ws_func_has_screen_cull
gates the widen to real render funnels (sltiu 0x140 + sltiu 0xE0 signature).
2. Versioned overlay cache path. gcc cache moves to <game_id>/gcc/<arch-abi>/cg<N>/
where N = PSX_OVERLAY_CODEGEN_VER (overlay_api.h), parsed identically by the
runtime (overlay_loader.c) and compile_overlays.py so they can't drift. A build
whose emitter output changed (e.g. this auto_screen_x change) reads+writes a
FRESH dir and never reuses a stale DLL; old versions coexist (no auto-delete).
Pre-1.0: legacy flat/unversioned fallbacks dropped.
3. sljit default-on (validated). gcc and sljit are complementary, not exclusive:
gcc fills the cache via prebuilt DLLs + async batch (compile_fragment NULL),
sljit JITs gcc-absent regions synchronously on-miss. New code_provider_sljit()
exposes sljit for the JIT-on-miss gap-fill independent of the "active" backend;
the dispatch gate fires whenever sljit is available (head<0 == gcc-absent), and
apply_live_policy defaults live ON when sljit is available. Priority gcc > sljit
> interp; shards still route through the same-state diff gate (device-touch +
diverging shards pinned to interp) so default-on does NOT reintroduce the
save-load wedge; a declined/failing shard falls to interp (never a crash).
Verified live: live=1, 62 shards compiled, 2 declines->interp, native:interp
ratio ~31%->~87%, gcc still primary.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Documentation only. Prior sessions kept re-deriving state from breadcrumbs that
were no longer true:
- overlay_sljit.c / code_provider.{h,c}: corrected "declines every fragment" and
"compile_fragment not yet called from dispatch" — the MIPS->sljit emitter is
fully implemented and wired (then validated via the same-state diff).
- ISSUES.md: add Issue #7 (sljit live execution was unvalidated; the pure-live
save-load wedge), root cause and fix recorded.
(SLJIT.md — the §11 forward plan and architecture clarification — lives at the
workspace root F:/Projects/psxrecomp/SLJIT.md, outside this repo, updated
separately.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The MIPS->sljit emitter, paralleling dirty_ram_interp.c's exec_one but
emitting host code. First slice = single-block LEAF functions ending in
`jr $ra`; everything else aborts the whole fragment to the interpreter
(precision over recall).
- overlay_sljit.c: memory-backed GPRs, 32-bit ops (R3000A wraparound),
loads/stores via icall through the cpu read/write callbacks (same
MMIO/page-watch/dirty-tracking path as the interpreter). Opcodes:
SLL/SRL/SRA/SLLV/SRLV/SRAV, ADD(U)/SUB(U), AND/OR/XOR/NOR, SLT(U),
ADDI(U)/SLTI(U)/ANDI/ORI/XORI/LUI, LB/LH/LW/LBU/LHU/SB/SH/SW, SYNC.
- compile_fragment now returns CompiledFragment{fn,code_lo,code_len};
overlay_loader registers an sljit candidate (crc hashed from the live
bytes it JIT'd from, same live-byte validation as gcc candidates).
- Step 5 wiring is VALIDATION-GATED: JIT-on-miss fires only when
backend==sljit AND the same-state differential is armed (never in a
shadow run). Normal play stays byte-identical; live sljit execution
stays off until the differential gate passes.
- Debug: sljit_try <addr> probe; sljit_status reports shards_registered.
Verified: runtime/tests/sljit_emit_test.c (links overlay_sljit.c +
sljitLir.c only) JITs hand-built leaf functions and checks output —
26/26 pass incl. gpr0 invariant, sign/zero-ext, signed/unsigned compare,
and the decline cases. Full psx-runtime build clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Introduce one backend-agnostic code-production seam the overlay
capture/dispatch spine talks to, and make the validated gcc spawn->DLL
path its first implementation. Pure refactor; gcc stays the default and
its behavior is byte-identical.
- New code_provider.h/.c: a CodeProvider struct exposing BOTH production
lifecycles so the interface isn't gcc-shaped -- batch/async
(request/busy/poll_main, the gcc model) and sync-on-miss
(compile_fragment, the sljit model). compile_fragment is defined and
implemented per-provider but not yet called from dispatch (step 5).
gcc provider = direct pass-throughs to autocompile_* (compile_fragment
NULL: a compiler spawn is too slow for the dispatch path); sljit
provider wraps overlay_sljit_* (still declines every fragment).
- Wire up overlay_backend_resolve, which was previously dead code:
code_provider_init() resolves the backend at overlay-cache init and
caches the active provider; code_provider_active() is the accessor.
- Add [runtime] overlay_backend config (auto|gcc|sljit; empty=auto) so
resolution has a config source alongside PSX_OVERLAY_BACKEND.
- Route the 3 direct autocompile_* call sites through the active
provider: overlay_capture.c autocapture tick, main.cpp per-frame poll
+ the init call, debug_server.c autocompile_run (now reports backend).
- runtime.cmake += code_provider.c.
Builds + links clean (TombaRecomp/build-dev, MinGW gcc 15.2).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Matthew Stanley