The sljit Tier-2 in-process JIT was replaced by the tcc tier in c2a3f6ad
(2026-06-25) and removed outright in 5b7e69b4 (2026-07-15). That removal
reworded the C-level comments but touched no markdown, so the prose still
described sljit as a live, default-on backend.
Backends today are static -> gcc -> tcc: gcc is the development default when
it is on PATH, tcc is the toolchain-free path for players. Both are external
compilers invoked as a subprocess that produce a DLL; neither is an in-process
JIT. docs/ARCHITECTURE.md, docs/BUILDING.md, docs/EXECUTION_MODEL.md,
docs/COMPILING_OVERLAYS.md and README.md already said this correctly and are
unchanged here.
- THIRD_PARTY_ATTRIBUTION.md: the sljit entry claimed code vendored at
lib/sljit/ and compiled into the runtime. That directory does not exist.
Replaced with a TinyCC entry, which was missing entirely despite tcc being
the compiler intended to reach end users.
- docs/internal/SLJIT_PERSIST_CACHE.md: deleted. It documented the persisted
shard cache of a subsystem that no longer exists, as "implemented" and
"default-on". docs/internal/README.md updated to point at the current model
instead of a dead link.
- docs/ASYNC_OVERLAY_COMPILE.md: superseded-in-part header. Deliberately NOT
rewritten sljit->tcc: the passages about serializing .sljit blobs,
overlay_sljit_deserialize and an in-memory JIT registry describe a mechanism
tcc never used, so renaming it would trade one false claim for another. The
async worker/file-handoff design and the CPS work remain accurate.
- docs/OVERLAY_CACHE_V2.md: sljit dropped from the dispatch priority chain.
- ISSUES.md: Issue #7 closed as superseded — the backend it reported against
was deleted rather than fixed.
- docs/internal/COSIM_ORACLE.md: sljit_helpers left CPUState with the tier.
- Dated session logs (RECURSION_BUG.md 25.5, BACKDROP_PRELOAD.md,
ecosystem-watch.md) keep their narrative and get a pointer to the removal.
One thing this does not fix: docs assert the tcc toolchain is "downloaded at
release-packaging time and bundled beside the game exe". runtime/src/main.cpp
does look for <exe_dir>/overlay_toolchain/, but no script in this repository
creates it. The attribution entry now states only what the code does and notes
that if packaging supplies the bundle, it happens outside this repo and must
carry the TinyCC notice. Flagged rather than papered over.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Replace absolute F:/C: paths with repo-relative (../TombaRecomp, bios/, beetle-psx/)
or placeholders (<desktop>/<psxref>/<nesrecomp>/<psxrecomp-v4>) across accuracy docs,
WIDESCREEN/SHADOW/RECURSION_BUG/config_schema/ISSUES/README, and tools/*.py|.ps1.
Removed a personal C:\Users\Matthew\Desktop path (ISSUES.md). Left CLAUDE.md/PLAN.md
sibling-project + version-history paths intact (intentional layout documentation).
The continuation-passing leak fix (RECURSION_BUG.md §25) is validated (Tomba boots+plays,
ce_profile flat through a 56.9k-frame soak, full overlay CPS). Flip the gate in all three
emitter sites so a normal regen emits CPS; PSX_CPS=0 is the legacy escape hatch. Verified:
default regen -> CPS markers present; PSX_CPS=0 -> none; default-CPS Tomba boots clean, leak flat.
Doc §25.6.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
RECURSION_BUG.md §25.5. Brings the overlay native path under continuation-passing so the
idle-freeze leak fix holds with overlay_cache=true (the production gcc>sljit>interp path).
Runtime CPS-mode flag g_psx_cps_mode (overlay_loader.c) set by a __attribute__((constructor))
emitted in the BIOS + game CPS dispatch (full_function_emitter.cpp / main_psx.cpp). Gates all
overlay/sljit CPS behavior; legacy unaffected.
sljit JIT (overlay_sljit.c), scoped:
- jr $ra publishes cpu->pc (was a bare return). jr rX already tail-transferred.
- call-containing fragments DECLINE to the interpreter (CPS-safe, depth-1) — leaf shards still
JIT. (Full sljit call coverage / per-fragment entry-switch is future work.)
gcc-DLL path — FULL native overlay coverage (validated):
- compile_overlays.py --cps sets PSX_CPS for the recompiler -> CPS overlay C (tail-transfer +
per-function entry-switch; --overlay mode only affects widescreen, not CPS). DISPATCH_PREAMBLE
psx_syscall decl fixed void->int.
- PSX_OVERLAY_CODEGEN_VER 3->4: fresh cg4/ cache namespace; legacy unit-model DLLs (cg3) ignored.
- overlay_loader_dispatch: under CPS, when a dispatched addr is not an overlay ENTRY but lies
inside a registered candidate's code range (a tail-transferred call's continuation),
overlay_find_by_range() re-enters that function with cpu->pc=addr so its entry-switch routes
to the block. gcc DLLs are the trusted tier (diff-validated at entry) -> run native directly.
This is the CPS equivalent of the old unit-model that fixed the dwarf->overworld blue screen.
Validated: CPS overlay DLLs compile + load + dispatch_native>0; with continuation routing
dispatch_interp_fallback -> 0 (full native), ce_profile FLAT (6 KB), boots clean. Addresses are
KSEG0-consistent.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The complete §25 fix: make BOTH recompiler emitters emit guest calls as
TAIL-TRANSFERS driven by the unified flat trampoline + the guest's own $ra/$sp,
instead of nesting one host C frame per guest call. The host stack can no longer
mirror unbounded guest re-entrancy — the architectural root of the ~14-min idle
freeze. Gated gen-time by PSX_CPS; legacy codegen byte-identical when unset.
VALIDATED (user): full-CPS Tomba boots BIOS -> FMV -> New Game -> village idle and
plays; ce_profile cur_max_kb FLAT at 5 KB from frame ~9.9k through 56,890 (turbo
soak) — blew past the ~50k-frame freeze threshold with the host stack bounded
(legacy climbed +1.17 KB/frame -> 48 MB -> freeze).
KEY CORRECTION to the prior session's §25.1: there are TWO emitters. The BIOS
(full_function_emitter.cpp) was done last session but the §8 leaking chain
(func_8001A954 ...) is GAME code, emitted by code_generator.cpp (via main_psx.cpp)
— never converted. Also, §25.1's "continuations not in the dispatch table"
hypothesis was wrong (they were always added); the real boot regressions were
found by dumping the always-on fntrace ring on the abnormal psx_dispatch return.
Emitter changes (gated PSX_CPS):
- full_function_emitter.cpp (BIOS): BIOS A0/B0/C0 vector runtime-fallback
tail-transfers instead of nested psx_dispatch_call (was the frame-0 exit).
- code_generator.cpp/.h (game): jal/jalr/jr-$ra/jr-table/branch-split/j-split ->
tail-transfer (jalr captures rs before the link, alias-safe); each call's
return point registered as a dispatchable continuation; entry-switch (keyed on
cpu->pc) at the top of every function AND every overlapping-alias body. 0
psx_call_contract / call_by_address emitted under CPS.
- main_psx.cpp: psx_dispatch_game_compiled gains continuation cases
(case cont: cpu->pc=cont; func_owner(cpu)); entries clear cpu->pc.
- strict_translator.cpp: syscall emits `if (psx_syscall(...)) return;` so a void
syscall falls through to the inline post-syscall jr $ra (real-HW EPC+4).
Runtime changes (the legacy "return via nested dispatch / cpu->pc=0" idioms that
exit the flat CPS trampoline):
- traps.c: psx_syscall returns int (0=void/fall-through, 1=transfer);
Enter/ExitCriticalSection return 0; case-3 ChangeThread-success returns 0 — the
fiber scheduler leaves cpu->pc=0 on resume, which was the frame-712 exit.
- cpu_state.h / overlay_api.h: psx_syscall signature void->int; overlay callback
ABI bumped 4->5 (stale overlay caches rejected).
- dirty_ram_interp.c: gate interp_enter_compiled behind PSX_HAS_GAME_DISPATCH
(un-breaks the BIOS-only build).
- main.cpp: dump the always-on fntrace ring tail + stack to psx_cps_exit_trace.json
on an abnormal psx_dispatch return (diagnostic; CLAUDE.md ring-buffer model).
Docs: RECURSION_BUG.md §25.2-§25.4. Remaining: dwarf->overworld playtest;
Stage 3 (sljit/overlay CPS parity); then consider making CPS default.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The universal fix for the idle-freeze host-stack leak (RECURSION_BUG.md §25): make
the recompiler emit guest calls as TAIL-TRANSFERS driven by the trampoline + the
guest's own $ra/$sp, instead of nesting one psx_dispatch_impl per guest call frame.
This makes the host stack incapable of mirroring unbounded guest re-entrancy — the
architectural root of the leak — for ANY game. Perf cost accepted ("right before
fast").
Stage 1 (full_function_emitter.cpp), GATED behind gen-time env PSX_CPS:
- jal/jalr (orphaned-delay + pending-delay paths): emit `cpu->pc=target; return;`
(tail-transfer) + register the return address as a dispatchable continuation,
instead of nested psx_dispatch + psx_call_contract. JALR captures the target reg
BEFORE the link write (rd==rs alias-safe).
- jr $ra (all 3 sites): publish `cpu->pc = cpu->gpr[31]; return;` under CPS so the
trampoline dispatches the caller's continuation (legacy `return;` relied on the
nested-dispatch C-return). Required — without it the BIOS exits at frame 0.
GATING PROVEN SAFE: regen+build WITHOUT PSX_CPS boots Tomba normally — default
codegen byte-unaffected; CPS fully opt-in.
OPEN (CPS path only): with PSX_CPS=1 the BIOS boots main() then exits at frame 0
(execution completed, PC=0). Root cause (high confidence): the trampoline must
dispatch mid-function continuation addresses, but the dispatch table holds only
function entries → a return to a continuation misses → psx_unknown_dispatch →
pc=0 → exit. NEXT: add registered continuations to the dispatch table (map
continuation_addr → containing func), then re-regen PSX_CPS=1 and run the §25 gate
(ce_profile flat + oracle-identical + 15-min soak + dwarf->overworld). Stages 2
(interp interp_enter_compiled surface) + 3 (sljit/overlay parity) follow.
Safe revert point for the whole feature: commit ae47e69.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Oracle disambiguation (RECURSION_BUG.md §23–§24). Traced one func_8001A954
invocation on real HW (psx-beetle) vs the recomp in the IDENTICAL Tomba idle
state (New Game village dialogue; object 0x801FD950, state/scene/jumptable=0).
VERDICT: real HW's per-frame chain is BALANCED — func_8001A954 is entered once
per frame from main_loop (JAL 0x8001A630, sp=0x801FE3E0) and RETURNS once per
frame (jr ra @0x8001A9E8 -> 0x8001A638, SP fully restored); no re-entry, the
overlay 0x80063864 never calls back (refutes §21/§22's "call-cycle"). The recomp
runs the same guest flow and the guest SP unwinds identically, but the HOST stack
leaks +1.17 KB/frame (ce_profile, live-reconfirmed: 20099->22215 KB / 1807 frames)
because the per-frame compiled chain (func_8001A954->AC00->B2B4->B5A8->func_80046264,
emitted as DIRECT func_X(cpu) host calls) re-enters itself via func_80046264's
CD-DMA-dirty hop and never unwinds.
native_stack early-trip (PSX_STACK_GUARD_KB=6144, frame 7592, 6 MB) pins the
leaking host cycle. Cross-checked: xprobe mixed_depth bounded at 6, dirty-insn/
dirty-block show NO literal func_8001A954 transfer, and a954=1/frame is a SURFACED
guest tail-transfer (j/jr) to func_8001A954 that the oracle proves real HW never
makes — i.e. there is both (A) a correctness divergence (spurious per-frame
re-entry) and (B) the architectural host-mirroring limitation that amplifies it.
Fix scope CORRECTED from §23's "quick contained runtime patch": the UNIVERSAL fix
is §11 — route in-range guest calls through the depth-managed psx_dispatch_impl
trampoline instead of mirroring them as direct host calls, so the host stack stays
bounded regardless of guest re-entrancy (real or fabricated). §12 (CD-DMA dirty
precision) and fixing the specific divergence are surgical mitigations, not
universal. Performance loss accepted for now (static recomp: right before fast).
Instruments added (runtime-only, no regen):
- SP capture in the fntrace control-flow ring on BOTH backends (GPR[29] read live):
psx-beetle (beetle_libretro.cpp + beetle_debug_server.c) and the recomp
(fntrace.h/.c + debug_server.c) — enables the func_8001A954 SP/RA-lifecycle oracle.
- psx-beetle default debug port 4380 -> 4382 (+ --port N override): 4380 collided
with psxref and cdirecomp/CdiRuntime.exe; two LISTENers on one port silently
misroute connections (the phantom "oracle keeps dying" early in the session).
Oracle tooling: the control-flow oracle is psx-beetle (Beetle interprets -> records
every J/JAL/JR/JALR), NOT psxref (no PC trace; the assumed emu_step/emu_trace_addr
never existed). Loads Tomba via: psx-beetle.exe --disc tomba.cue <bios> --port 4382.
Next: implement the §11 universal fix (codegen + full regen).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pinned the exact leaking crossing (xprobe detail): entry to func_8001A954 is a single
JAL, src 0x8001A630 (the main loop's only call site), once/frame. Implemented (A):
surface interp JAL/JALR to game-text targets to the owner trampoline (gated by
psx_game_address_in_text so overlay-native keeps the nesting contract). §19 result: leak
STILL CLIMBS (~0.875 KB/frame, down from 1.17) -> reduced, not eliminated -> REVERTED.
It is a guest CALL-CYCLE (main_loop 0x8001A51C <-> func_8001A954): surfacing the interp
side just relocates the nest to the compiled-side jal back to the main loop. Can't fix a
call-cycle by surfacing one crossing. Leak is ~1 host frame PER FRAME (not per cycle
iteration) -> the loop mostly returns; exactly one call/frame fails to unwind. Real HW
idles forever (balanced), so either we mishandle one specific return/frame (Hyp C,
contained) or it needs the §11 continuation-passing redesign. DECISIVE NEXT = oracle
(psxref): does real HW return from func_8001A954 to 0x8001A634 each frame, or never?
Stop surfacing single crossings (block-loop §20 + JAL/JALR §22 both dead ends).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The ~14-min Tomba idle freeze is a gradual per-frame host-stack LEAK (not a
within-one-frame spiral, not the gradual-marking theory). Confirmed via a new
compiled-entry depth profile (§19): per-frame MAX guest-stack-used climbs dead-linear
+1.17 KB/frame from gameplay start to 48 MB on the 64 MB main-thread stack -> native
stack guard -> freeze. Once per frame an interpreted overlay (reached via
func_80046264's dirty hop jal 0x80063864) does jalr func_8001A954, nesting a host call
that never unwinds (the per-frame loop never returns). Full writeup RECURSION_BUG.md
§16-§21.
Instruments (always-on rings, runtime-only, no regen):
- §17 stack_profile: per-vblank host-stack sample (main.cpp; stack_profile TCP cmd).
- §18 xprobe / xprobe_arm: interp<->compiled boundary crossing flight recorder
(dirty_ram_interp.c; per-frame summary + per-crossing detail).
- §19 ce_profile: per-compiled-entry depth profile + raw TEB at the trip
(debug_server.c; ce_profile TCP cmd) — the trustworthy instrument; the regression
check for the fix (max_kb must stay FLAT).
- PSX_STACK_GUARD_KB early-trip + PSX_FIBER_STACK_KB (debug_server.c / traps.c).
Fix progress:
- Block-loop tail-transfer surface (dirty_ram_interp.c ~1514): surface compiled
tail-transfers to the trampoline instead of nesting. Correct for tail-transfers but
RULED OUT as the leak (§19 climb unchanged) — the leak is the CALL path (jalr/jal).
- Real fix (next): surface compiled-game CALLS to the owner trampoline, gated to
compiled-game targets only (a naive all-targets surface reintroduces the
dwarf->overworld lost-continuation bug). Needs a non-nesting membership test
(generated psx_dispatch_game_has) or §12 CD-DMA dirty-marking precision.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Run #1: interp->0x8001A954 edge is ORDINARY (1/frame). Run #2: dispatch_8001A954=0
(dirty_ram_dispatch never called with 0x8001A954) AND recursion_func VARIES
(0x8004DEE0/0x8004DFA0/0x8005FF60) -> address-specific counters are the wrong
instrument. dirty_block_tail shows the real cycle: dirty_ram_dispatch(overlay
0x8011-0x8013) called from main-EXE 0x8002-0x8005 = the per-frame MESSAGE/RENDER
overlay system recursing. SITE_CAP 256->32 fixes the report-JSON overflow. Pivot:
use data-driven rings (native_stack/dirty_block_tail/dispatch_tail) from a clean
report, not guessed addresses; then last-writer slice on the loop termination var.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per-frame count of the re-entry edge interp->func_8001A954 (the per-frame loop
head) + cycle counter, in a 256-frame ring the crash report dumps
(dirty_ram_re954_json). Answers ChatGPT's step-1 question: is the re-entry
ORDINARY bounded per-frame behavior that stops terminating at ~50k, or a new edge
— and whether cycles advance (it's within ONE frame; 23k laps ~= 23k cycles <<
565k/frame, so no VBlank before overflow => control-flow cycle, not starvation).
docs/RECURSION_BUG.md §15 captures the full strategy reset (stop RAM-diffing;
triggered control-flow recorder + backward last-writer slice; narrow oracle use
only on the resulting small address set). The separately-navigated oracle RAM-diff
(§14) was a dead end (alignment noise).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Capture everything for recall in a future session:
- Resolved the overworld-load regression (stale build artifact, not tooling/master).
- 3 organic captures (deterministic: v0=0x8004DEE0, ra=v0+8, pc=0; both New-Game
and Dwarf-Forest reproduce).
- The contract mechanism (S7): psx_dispatch_impl tail-call trampoline + psx_call_contract
bail keep COMPILED tail-loops flat; the dirty interpreter bypasses them
(exec_one JAL/JALR + dirty_ram_dispatch_inner:~1179 nest psx_dispatch_game_compiled).
Proof it's host-only: guest sp flat (~7KB) across ~23k laps.
- Ghidra map of the guest state machine (S8): func_8001A954 state[0x4a]==1 ->
func_8001AC00 chain -> func_80046264 jumptable (selector @0x8009BCDD, table
@0x80013C60) -> func_8004DFA0 (a render handler that advances no selector).
- External-review verdict (S9): Hypothesis A overwhelmingly likely; B has no
positive evidence; C (interp control-transfer bug fabricating the re-entry) must
be excluded by an oracle PC-trace.
- Oracle disambiguation plan (S10), the single mixed-dispatch-owner fix design
(S11, runtime-only / no regen), CD-DMA dirty-marking critique (S12), and the next
disassembly target around 0x80063864 (S13).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Snapshot of the uncommitted native_stack capture tool (crash_trace.c,
append_native_stack) that recovers the true compiled<->dirty-interp
recursion cycle the recent_fn ring can't show, plus docs/RECURSION_BUG.md
documenting the freeze root cause, the 4-instance soak reproduction
harness, and the overworld-load regression now blocking repro.
This is the build that currently insta-crashes soaks on overworld load;
preserved as-is before hardening so the regression stays reproducible.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Matthew Stanley