Reconstruct the shareable pieces from PR #58 onto current master: add contiguous MDEC DMA input bursts, reduce per-pixel/per-byte MDEC output overhead, add sparse IDCT fast paths, and keep the MDEC activity stamp sane across rematches.
Also quiet the heavy debug trace recorders while MDEC video is active so debug builds can query state without starving FMV playback.
Regression testing was done with non-debug OpenGL builds across multiple FMV-heavy titles. Debug-tool FMV playback was also exercised with the quiet gate enabled.
Co-authored-by: Alex Vanderveen <alex@technicallycomputers.ca>
Co-authored-by: Cursor <cursoragent@cursor.com>
Restore user savestates to their saved CPU PC instead of the game entry PC, and clear non-serialized CPU/GTE completion deadlines after loading a state so stale absolute guest-cycle deadlines cannot stall the first post-restore unit read.
Debounce load requests for 60 frames after a successful restore. This prevents repeated load requests from stacking multiple restore/recovery cycles into a long apparent softlock while keeping normal save/load behavior unchanged.
Regression testing was done. Caveat: restores can visibly pause, especially near CD/load transitions, and worst-case recovery may take several seconds; observed long recovery cases eventually resumed rather than crashing or permanently hanging.
Co-authored-by: Henrique Guedes <henrique.guedes@porttus.com>
Register configured range_sites with the dirty-RAM interpreter so overlay-resident classifiers use the same two-margin widening already emitted by native codegen.
Co-authored-by: Opok <novelasevilliousesp@gmail.com>
Add opt-in slti/sltiu far-bound scaling with matched native and interpreter semantics, correct immediate sign extension, and an overlay ABI forwarder for live aspect state.
Co-authored-by: Opok <novelasevilliousesp@gmail.com>
Landed while root-causing the Tomba 2 warm-cache boot death; each of these
was a hole that blocked attribution on the production binary:
- psx_fatal_halt: halt-and-serve is now keyed on the TCP listener being
live (debug_server_get_status) instead of the build flavor. A production
run with PSX_DEBUG_SERVER=1 + PSX_EXIT_HALT=1 now stays inspectable at
the fatal instead of exit(1)ing the evidence away.
- debug_server_get_status: was declared in debug_server.h but never
implemented; implemented (listener state, port, saved socket error).
- event_ring: EVENT_RING_ENABLED=1 unconditionally. The ring records
edges only; its production disable predated the production listener and
contradicted its own ALWAYS-ON design note.
- sp_ring (new): stack-domain (64 KB) transition ring at fntrace_record --
one entry per dispatch whose guest SP crossed a domain, i.e. the
provenance record for 'who installed this stack pointer'.
- disp_ring (new): 128-entry unconditional dispatch tail (target/ra/sp/
cycle) so the final control-flow sequence before a death is always
reconstructable post-mortem.
- overlay_loader: the native-call ring (overlay_native_ring recent[]) is
now recorded in production too -- each entry names the requested PC and
the claiming candidate CRC, which is what finally attributed the
wrong-variant native run.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add an opt-in horizontal low-edge transform for subu rd,zero,rt classifiers, with matching native and dirty-RAM interpreter behavior and guarded overlay variants.
Co-authored-by: Opok <novelasevilliousesp@gmail.com>
Audit fixes for the async shard-publication pipeline (45a2dda):
- autocompile_shutdown(): stops the pipeline, kills the compiler tree via a
kill-on-close job object (assigned while the child is SUSPENDED so cmd.exe
cannot spawn grandchildren outside it), and JOINS the watcher + preparer —
never abandoning a thread inside the Windows loader lock at ExitProcess.
Wired first into shutdown_runtime(). Crash exits reap the tree via the job.
- Failed off-thread prepares retry (3 attempts, 100 ms) on the worker before
the artifact silently reverts to a synchronous emu-thread first-map; retry
and giveup counters exposed in autocompile_status_json.
- Capacity caps fail closed and LOUD: cand_register shouts once at CAND_CAP;
load_one_dll_prepared refuses at MAX_LOADED_DLLS before registering, so
dll_already_loaded can never lose track and double-register on rescan.
- Publication confinement is filesystem-resolved: published_path_valid
canonicalizes the cache root and the pipe path (GetFinalPathNameByHandle /
realpath), prefix-checks canonical forms, and the worker maps the canonical
string while holding a no-delete-share guard handle (TOCTOU). `..`/junction
spellings that passed the lexical check are now rejected — proven by the
new overlay_path_canon_test boundary test.
- test_autocompile_publication gains a shutdown-mid-run scenario asserting
prepared == committed + discarded with nothing queued; perf guards updated
to require validate -> guard-handle -> canonical first-map ordering.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two provably-correct framework fixes (independent of the AOT work) plus the
play-free overlay-extraction spike tooling and design doc.
Framework fixes (accurate, safe, complete):
- ABI v14: add gte_precision_store_word to OverlayCallbacks + preamble shim +
runtime population. The emitter emits a direct call for every swc2 (GTE
store-word); it is stateful runtime code (gte.cpp) so overlay DLLs must
forward or fail to LINK (undefined reference). Play-captured shards rarely
compiled a swc2; fuller static coverage surfaced the gap. Same pattern as
the v13 ws_player_x_bound / v10 gte_read_ctrl shims.
- Gate splitgen OFF in overlay mode (main_psx.cpp): the current recompiler's
unconditional split-TU output deletes <stem>_full.c and emits only shards,
which broke ALL overlay compilation (compile_overlays.py reads _full.c).
Overlays are single small TUs with no parallel-compile benefit; emit the
monolith. Restores overlay compile on the current tree.
AOT spike (docs/AOT_OVERLAY_PLAN.md + tools/aot_overlay_spike/, WIP):
- Play-free static overlay extraction from the disc image. Validated:
Tomba 1 reproduces 90% of played functions + ~1964 unplayed, live-verified
in-game (Dwarf Village runs native, zero runtime compile). Tomba 2 yields
661 functions byte-identical to the shipped/played vault (provably real),
all from the self-describing MAIN.EXE producer; the header-table BIN
producer needs loader-table RE (base recovery unreliable). Generality is
producer-dependent. Spike scripts are Tomba-specific proofs of concept
with hardcoded paths, not a finished tool.
- Correctness is guaranteed by the existing per-function code_crc dispatch
guard: a mis-positioned shard fails audit at compile or never fires at
runtime -> coverage loss, never wrong execution. Autocompile self-heals gaps.
CROSS-CUTTING: the ABI v14 change is part of the overlay codegen-hash source
set, so it ROLLS the cg tag (1f2ad026 -> e8484213). On master + a fw bump,
every title's overlay cache invalidates and needs regen+reshard. Branch only;
NOT pushed; other games' caches untouched. Needs user validation before merge.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The tier mis-compiled and has been hard-gated off since 2026-06-25; overlay
gaps fall to gcc/tcc shards and the dirty-RAM interpreter, unchanged.
Re-do on master of the removal validated on the stale branch (6da8c02),
reconciled with everything master grew since the fork:
- Delete overlay_sljit.{c,h}, overlay_compile_worker.{c,h}, lib/sljit,
runtime/tests/sljit_*.c, and the runtime.cmake sources/include-dirs.
- Strip the sljit provider/backend from code_provider.{c,h},
overlay_backend.{c,h} ("sljit" in overlay_backend/PSX_OVERLAY_BACKEND now
degrades to auto so stale configs still boot), main.cpp, debug_server.c
(sljit_status/sljit_async/sljit_try TCP commands), overlay_loader.{c,h}
(live policy, JIT-on-miss, persisted-shard cache, async worker glue,
psx_sljit_call/cop2/memx host helpers — reachable only via the removed
cpu->sljit_helpers table), and cpu_state.h (SLJIT_HLP_* enum + the
sljit_helpers[] member; CPUState layout change reshards via the cg tag).
- Master-only entanglement resolved before deletion: 28165d6's Einhander WS
player-X bound emit existed in three paths (gcc code_generator, sljit,
dirty_ram_interp). The gcc shard link contract now forwards
psx_ws_player_x_bound (ABI v13, previous commit), so deleting the sljit
emit path loses no widescreen behavior; ffc6688's game-dispatch block
inside the dead psx_sljit_call helper goes with it.
- Reword comments that described sljit as a live backend (gpu.{c,h},
ws_*_detect.h, cosim_state.c, dirty_ram_interp.c, autocompile.h,
boot_state.h, config_loader.h, compile_overlays.py, and the CPS marker
comments the recompiler emits).
recompiler psxrecomp-game builds clean; game runtimes rebuilt + boot-verified
separately.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
28165d6 (Einhander native-wide player bounds) rewrites configured
[widescreen] signed_x_bound LUI sites into psx_ws_player_x_bound(vanilla)
calls in code_generator.cpp — and that site loop runs in overlay_mode too, so
the call lands in overlay-resident code. But the shard link contract never
forwarded the symbol: any shard containing a bound site fails to LINK
(undefined psx_ws_player_x_bound) and that variant runs interpreted forever —
the same silent class as v10 gte_read_ctrl and v12 psx_rfe_mark_escape (now
loud under the shard observability suite, but still a coverage hole).
The clamp computes against the host's live widescreen state (gpu.c), so a
local copy can't work: add the ws_player_x_bound callback (appended last),
the preamble forwarder (identity fallback => correct 4:3 on a host that
predates it), and the loader init wiring. Bump the ABI to 13 so stale DLL
mixes are rejected and regenerate.
This also unblocks removing the sljit tier: sljit's emit_ws_player_x_bound
was the only other native path carrying the clamp; with the gcc shard path
forwarding it (and dirty_ram_interp already handling the site), the feature
no longer depends on sljit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Re-land of the shard-observability suite from 6da8c02 (stale branch), merged
against current master (keeps master's overlay_ext() platform suffixes,
recompiler_patch hash sources, and the unconditional v12 rfe forwarder).
- compile_overlays.py: ShardStats tallies every shard outcome — per-shard
"SHARD FAIL [class]" lines, a SHARD BUILD SUMMARY, a machine-readable
"PSX_SHARD_RESULT ok=N failed=M skipped=K" line, and a non-zero exit when
any shard that should have built failed (previously a run that built zero
shards exited 0, indistinguishable from a clean run).
- New --check preflight: build every shard into a throwaway dir and report
pass/fail — answers "would this game's shards compile against the current
recompiler + headers?" without a runtime.
- Retain audit-failed interior-fragment C (<phys>_<interior>_fragment_FAILED.c)
for triage, plus a persistent doomed-interior memo (interior_fail_memo.txt)
keyed (phys,interior,region_crc) so autocompile stops re-walking
data-as-code fragments every cycle (~90x off the V8 fragment pass).
- Extract DISPATCH_PREAMBLE to runtime/include/overlay_dispatch_preamble.c.inc
(single source of truth for the shard link contract, loaded by
load_dispatch_preamble(); missing file is a hard error).
- codegen_hash_sources.cmake: fold the shard COMPILE surface (overlay_api.h,
cpu_state.h, the .inc) into the cg tag so header/ABI/link-contract drift
reshards same-tree instead of silently breaking every new shard compile.
- autocompile.c: parse PSX_SHARD_RESULT from the child-output tail, expose
shard_ok/shard_fail/shard_skipped/shard_fail_total over autocompile_status;
ALWAYS rescan on completion so shards that did build load even when the
run also reported failures.
Live proof on the stale branch: Tomba's first autocompile surfaced 9 shard
link failures (undefined psx_rfe_mark_escape — since fixed by v12) and V8
surfaced 91 benign data-as-code fragment failures, all previously silent.
Codegen hash changes => every title reshards once on next build (expected;
loader rejects/regenerates stale shards).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Port the reusable Gaussian interpolation slice from PR #16 while keeping its reverb engine on a separate branch.
Co-authored-by: Martin Penkava <mpenkava1337@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Apply the PS1 1023x511 size rule to every polygon triangle, line, and polyline continuation before widescreen transforms while preserving textured-command state latching.
Co-authored-by: Martin Penkava <mpenkava1337@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The emitter emits psx_rfe_mark_escape() at every rfe — including in
overlay-resident exception-context code — but the dispatch preamble
never provided it, so any rfe-containing overlay failed to LINK
(undefined reference) and ran interpreted forever: the gte_read_ctrl
silent-coverage-blocker class again (11 of Tomba's vault shards hit
it). It mutates shared RFE-pending escape state (interrupts.c), so the
shim forwards unconditionally through OverlayCallbacks; ABI bump to 12
rejects pre-shim caches.
Found live during LOAD_TIME_ZERO E0 bring-up: the user's first pig
load triggered autocapture and the compile died on this link error.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Port Douglas Vanderveer's manifest-driven macOS/Linux overlay discovery onto the current cache index and exact-dispatch loader. Validate both filename key fields, preserve ABI cleanup and rescan idempotence, and cover the POSIX primitives with a real shared-library fixture.
Co-authored-by: douglasjv <vanderveerdouglas@gmail.com>
- psx_cyc.h / psx_cycles.c: _BitScanForward/_BitScanReverse shims for
__builtin_ctz/__builtin_clz under _MSC_VER
- full_function_emitter.cpp / main_psx.cpp: emit a portable .CRT$XCU
init-pointer instead of GCC-only __attribute__((constructor)) in
generated dispatch/game code
- main.cpp: declare cross-language globals extern "C" so MSVC links
them against their C definitions (no-op for GCC/Clang)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The cue parser kept a single bin_filename that every FILE line
overwrote, so a redump-style dump (one .bin per track) mounted only
the LAST track file - the CD-DA audio bin - as the entire disc: no
ISO9660 filesystem, no boot EXE, black screen at startup (reported
against Tomba! 2, whose USA disc is data + one CD-DA track).
Replace the single ifstream with a BinSegment table: one entry per
FILE line, opened for the reader's lifetime and laid out at cumulative
disc-relative sector offsets in cue order. ReadSector/ReadRawSector
resolve a disc LBA to its owning segment; INDEX times, which a cue
states relative to the owning file, are converted to disc-relative
track starts once the segment layout is known. Single-file images are
a one-segment table with unchanged addressing.
Also fixed in passing: Open() never cleared tracks_ from a previous
open (reopening a reader accumulated stale TOC entries), per-read
sector-geometry probing is now computed once per segment, a cue whose
FILE is missing or non-BINARY (WAVE/MP3) fails Open() instead of
mis-mapping the TOC, and GetSectorCount() reports the whole-disc
total.
Verified: Tomba! 2 single-bin dump split at the track-2 boundary
(concatenation SHA1-identical to the original); a host-side test
opens both cues and confirms all 186430 sectors byte-identical and
the TOC identical (track 2 at LBA 174674, audio); the game boots to
intro FMV, title, and attract gameplay from the multi-file dump with
CD audio flowing, and the single-file dump boots identically on the
same binary.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Closes the latent Rule-18 soundness hole: kernel-window static dispatch
hits (RAM [0x500,0x8500), the BIOS's boot-time ROM copy) now require the
live RAM bytes of the function's full reachable extent to byte-match the
ROM source (lazy verify, cached, precisely invalidated on writes); patched
bodies divert to the faithful interpreter. Emitter ships
psx_bios_kernel_bodies[] (499 entries); runtime guard in memory.c; TCP
kernel_bless observability; PSX_KERNEL_BLESS=0 A/B switch.
USER-VALIDATED on Tomba (SCUS-94236): multiple play runs, LLE + HLE boots,
pads/menus/events, title 1.00x / village 0.92x parity; 208-214/219 kernel
functions verified clean, the 5 mismatches (runtime-patched pad-driver
family) faithfully interpreted as before.
NOTE: rolls the codegen hash — regen BIOS+game generated C and recompile
overlay shards with a same-tree recompiler on each title's next fw bump
(the overlay loader fail-closes loudly on tag mismatch).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The generated dispatcher ran kernel-window table hits (RAM [0x500,0x8500),
the BIOS's boot-time copy of ROM 0x1FC10000+) natively with NO check that
the live RAM bytes still match the ROM image the functions were compiled
from — unsound for runtime-patched kernel code (the BIOS installs pad/SIO
handlers INSIDE compiled bodies; live diff: 15 clusters, 348 bytes). It
never misfired only because entry points happened to avoid patched keys.
A static kernel hit now requires psx_kernel_bless_dispatchable(): the
emitter emits psx_bios_kernel_bodies[] (key -> full RAM extent of the code
reachable from it; continuations carry their parent's extent, 499 entries),
and the runtime lazily byte-compares each entry's body against the ROM
source, caches the verdict, and invalidates precisely on writes into the
body (store chokepoint, DMA/EXE range marks, savestate restore). Clean ->
guarded native; patched/unverifiable -> the faithful dirty-RAM interpreter,
exactly as Rule 18 prescribes. This implements the runtime half of the
relocation-manifest contract (docs/RELOCATION_MANIFEST_FORMAT.md).
Measured on Tomba (SCUS-94236): 208-214 of 219 verified functions clean
(the 5 mismatches are the genuinely patched pad-driver family, faithfully
interpreted); millions of guarded native hits per boot; title 1.00x,
village 0.92x (parity — kernel keys were already native, now they are
PROVEN native); LLE and HLE boots + gameplay input verified. Kernel-window
interp measures 1.3% of interp burden (dirty_ram_stats), so this is a
soundness fix, not a speedup; the dominant interp cost is overlay-region
coverage (cg-tag hygiene), tracked separately.
Observability: TCP kernel_bless (entries/clean/mismatch/native_hits/
verifies/invalidations), always-on. PSX_KERNEL_BLESS=0 reverts to
unguarded-table behavior for A/B.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Matthew Stanley