Commit Graph

73 Commits

Author SHA1 Message Date
Matthew Stanley 80ff96780a recompiler: the overlay guard word is a delay-slot source, never a block leader
The overlay capture format DELIBERATELY appends one coherent guard instruction
past the end of a dirty-page run (overlay_capture.c write_json_window, `size
+= 4u`, pinned by runtime/tests/test_interpreter_perf_guards.py:47): a MIPS
branch sitting at the run's final word (...FFC) always executes its delay slot
in the next page (...000), and the shard must be able to emit and hash that
word. So a capture whose size is a page multiple PLUS 4 is a CORRECT capture --
N bytes of overlay plus one guard word. Measured: 36 of 36 Ape Escape
(SCUS-94423) vault records and 846 of 946 Tomba 2 (SCUS-94454) records are that
shape.

The recompiler was never taught the asymmetry: the guard word is a legal
DELAY-SLOT SOURCE but an ILLEGAL BLOCK LEADER, because the word after it does
not exist in the image. It had exactly one bound -- end_address() -- used both
to read words and to decide what is code.

MEASURED on Ape Escape, its own single-source vault at framework
v0.3.2-alpha-139-g1bf70960, 36 variants:

  PSX_SHARD_RESULT ok=50 failed=17 skipped=32 capacity_fastpath=0

  ! [recompiler] overlay 0x80136000 crc 4BAB23C9: what():  cannot emit control
    flow at 0x80138000: mandatory delay slot 0x80138004 is outside the input
    image
  + 16 x [fragment] interior 0x801360.. @region 0x00136000, identical message

One root cause, seventeen shards. The whole region then falls back to the
dirty-RAM interpreter forever, and nothing at runtime says so.

Causal chain, every step read in source:

  1. overlay_capture.c:349-361 appends the guard word; region 0x80136000 gets
     size 0x2004, so the image is [0x80136000, 0x80138004).
  2. compile_overlays.py make_psxexe wraps those bytes verbatim with text size
     = len(data), so PS1Executable::end_address() = 0x80138004.
  3. function_analysis.cpp in_exe / in_function admit 0x80138000 as ordinary
     code, and the walk reaches it by fall-through from 0x80137FFC
     (0x24022000, addiu -- not control flow), so func.end_addr becomes the
     image end.
  4. control_flow.cpp makes the final block end_addr = func.end_addr - 4 =
     0x80138000: the block LEADER is the guard word.
  5. control_flow.cpp's "Also check last instruction itself" promotion finds
     0x80137FFC is not control flow and therefore promotes read_word(end_addr)
     -- the guard word 0x1062005A, a beq -- to block.exit_instr, with
     has_delay_slot = true and address == end_addr. THIS IS THE DEFECT.
  6. code_generator.cpp computes delay_slot_addr = 0x80138004, read_word
     returns nullopt, and it throws.

THE THROW AT STEP 6 IS CORRECT AND IS PRESERVED. Emitting a control transfer
without its architectural delay slot silently corrupts guest state, which is
what the code did before that throw was added. The gap is step 5.

Fix: give the image view two distinct bounds.

  end_address()           READ bound -- every byte the image supplies, so a
                          mandatory delay slot living at a guard word resolves.
  analysis_end_address()  ANALYSIS bound -- one past the last byte that may be
                          discovered, made a block leader, or made a block's
                          exit instruction.

read_word() keeps the READ bound. Discovery and block construction move to the
ANALYSIS bound: function_analysis.cpp in_exe / in_exe_p3 / add_forced_entry /
every hard_cap that used to be the image end / the jr-table producer_hi
defaults / the static jr-ra and JAL sweeps; control_flow.cpp gains
analysis_walk_hi(func) and uses it for cfg.function_end, alias entries,
add_boundary, the scan loop, the final block's end_addr and in_func;
code_generator.cpp's mid-function split pre-pass; main_psx.cpp seed admission
and the data-scan pointer promotion; analysis_db.cpp, main_toml.cpp,
main_cli.cpp target/seed admission. The clamp is applied only when it leaves a
non-empty walk, so a degenerate function starting at or past the bound keeps
failing closed rather than silently producing an empty CFG.

HOW THE BOUND IS DERIVED -- the producer STATES it; the recompiler is told.

The alternative was for the recompiler to infer the count from `size % 4096 ==
4`. That fits every capture the current writer has ever produced, but it is a
consumer guessing at a producer's intent, and this tree already has the
counter-example: 52 of the 946 SCUS-94454 records have residues from 112 to
3996 bytes (one not even 4-byte aligned) because they come from the STATIC
extractor, which records an exact file extent and appends no guard word. A rule
keyed to the page-run shape is a rule that shifts the moment a producer
changes.

So the count travels with the image, in a magic-tagged field in the otherwise
zero-filled tail of the 2048-byte PS-EXE header (exe_tag in ps1_exe_parser.h:
"PSXRGRD1" at 0x7E0, uint32 count at 0x7E8). Putting it in the header rather
than on the command line means every consumer gets it automatically and no
future invocation path can forget to pass it -- compile_overlays has three
make_psxexe sites and five recompiler spawn sites. An absent or malformed tag
means ZERO guard bytes, which is exactly what every genuine PS-X EXE is: the
BIOS and every retail main executable have no guard word, their analysis bound
equals their read bound, and their analysis is bit-for-bit unchanged.

Producer side:
  overlay_capture.c    writes "guard_bytes": 4|0 per region -- the writer
                       stating its own intent, additive to the v2 schema and
                       preserved through coverage_vault.py compaction.
  compile_overlays.py  capture_guard_bytes(): prefer that field; for captures
                       recorded before it existed, reconstruct the single
                       shape write_json_window can produce (page multiple + 4)
                       and return 0 for everything else, which is the
                       fail-closed answer -- an unrecognised guard word is
                       analysed as code and the step-6 throw still fires
                       loudly at generation time.
  make_psxexe          guard_bytes is keyword-only with NO default, so a new
                       call site is a TypeError, not a silently mis-analysed
                       shard. The count is threaded through
                       compile_fragment_batch / compile_interior_fragment /
                       generate_interior_fragment_static and into the fragment
                       fail-memo key (bumped to v3, so a memo recorded under
                       the old analysis cannot suppress a retry under the
                       corrected one).

SECOND-ORDER DEFECT FIXED IN THE SAME PASS. generate_ranges_manifest extends a
range by +4 for any block whose exit instruction IS its last word, so the
cloned delay slot joins the candidate CRC and the page-generation watch. That
extension was unclamped. Reachable TODAY, independent of the guard word: a
branch-likely opcode (0x14-0x17, RESERVED on the R3000A) as the image's very
last word is short-circuited by translate_basic_block into an inline RI raise
BEFORE any delay slot is read, so generation succeeds and the manifest claims 4
bytes past the image end. Measured on origin/master with a 28-byte synthetic
image: `R 80010000 20` for an image ending at 0x8001001C. The cache would then
validate the shard against bytes the shard never saw. Now clamped to the READ
bound -- not the analysis bound, because a guard word genuinely IS compiled in
when it serves as a delay slot and must participate in the CRC.

DEAD CODE, documented rather than trusted: basic_block.cpp's BasicBlockAnalyzer
bounds its walk correctly but nothing in the tree constructs it. It is still
compiled and still hashed into the codegen tag, so a change there invalidates
caches without moving an emitted byte. Its bounds move to the analysis bound
too -- wiring it up must not reintroduce this defect -- and a header comment now
says where the live walker actually is.

TESTS. Both registered, so ctest runs them and the configure-time
registration guard is satisfied.

recompiler/tests/test_overlay_guard_word_analysis_bound.py drives
psxrecomp-game over five synthetic images:
  A  tagged, 2 pages + guard word, last real word not control flow, guard word
     IS a beq -- the measured Ape shape. Must generate; the guard word must not
     be emitted as an instruction; the block must publish 0x80012000 for the
     runtime to dispatch; no range may claim it.
  B  byte-for-byte the same payload with NO tag. Must still fail closed with
     the mandatory-delay-slot diagnostic naming 0x80012000. A and B differ only
     in the header tag, which proves the tag is what changed behaviour and that
     the throw was not weakened.
  B2 a misaligned declaration (6). Must be rejected and behave like B.
  C  tagged, with a real branch at the last analysable word. The guard word
     must be emitted AS the delay slot and must be covered by the manifest.
  D  branch-likely as the final word. Generation must succeed and no range may
     exceed the image end.

Against origin/master's psxrecomp-game the same file reports exactly the two
things this commit changes, and passes B, B2 and C:
  FAIL: tagged guard word still fails generation: what():  cannot emit control
        flow at 0x80012000: mandatory delay slot 0x80012004 is outside the
        input image
  FAIL: range [0x80010000,0x80010020) claims 4 byte(s) past the image end
        0x8001001C

tools/tests/test_overlay_guard_bytes_declaration.py pins the producer half:
field precedence, the reconstruction table, rejection of nonsense
declarations, the tag's byte offsets, that make_psxexe with guard_bytes=0 is
byte-identical to the pre-fix wrapper, and that overlay_capture.c both appends
the guard word and now declares it.

MEASURED OUTCOME, Ape Escape, same vault, same flags, fresh cache:
  before  PSX_SHARD_RESULT ok=50 failed=17 skipped=32 capacity_fastpath=0
  after   PSX_SHARD_RESULT ok=31 failed=0  skipped=57 capacity_fastpath=0

ok falls because 19 isolated island fragment shards at 0x00136000 are no longer
needed once the region variant compiles as one shard. Coverage is what matters,
and it is measured from the DLL exports: 349 -> 361 unique func_XXXXXXXX
entries, ZERO lost, 12 gained -- 0x80136140, 0x8013614C, 0x80136154,
0x80136238, 0x801362D4, 0x801362DC, 0x801362F0, 0x80136324, 0x80136370,
0x80136400, 0x80136480, 0x80137950, all from the failure list. The other four
failing PCs were already served by a sibling variant's shard.

INERTNESS.
  BIOS (SCPH1001, psxrecomp-bios --config bios/SCPH1001.toml): all four
  generated artifacts BYTE-IDENTICAL, verified by sha256 rather than by exit
  code, because tools/regen_bios.sh exits 0 on failure.
  Tomba 2 main EXE (psxrecomp-game --config game.toml): all nine generated
  artifacts BYTE-IDENTICAL, and the console log is identical too.
  Tomba 2 overlays, 82 variants over 6 regions from its own vault: identical
  result line (ok=98 failed=0 skipped=103) and identical shard identity set,
  4901 unique func exports on both sides -- zero gained, zero lost. 97 of 98
  range manifests are byte-identical. The one that differs is region
  0x8010A000 variant A32DB188, size 0x2004, whose last function's range goes
  from 0x218 to 0x214 bytes: its guard word (0x24840008, addiu) had been
  emitted as an ordinary instruction and hashed, and now is not. That is
  precisely the intended change and nothing else moved.

Bead beads-eio.3.100.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-02 15:35:24 -07:00
Alexandros Mandravillis 62dc9b9bb4 recompiler: sync GTE read-helper classification with runtime accessor semantics
The emitter's mfc2/cfc2 raw-vs-helper classification
(gte_register_classification.h) was out of sync with the runtime's
gte_read_data/gte_read_ctrl special cases: compiled code read OTZ(7)
and SZ0-3(16-19) unmasked and VZ/IR-class regs (1,3,5,8-11) without
sign extension, and cfc2 missed ctrl regs 4/12/20 — while the
interpreter always goes through the accessors. Nine data regs + three
ctrl regs of backend semantic divergence, locked in by a stale
static_assert mask.

data_read_needs_helper now covers every gte_read_data special case
(masks 0xB08F8FAA data / 0xEC101010 ctrl); regenerate titles to pick
it up.
2026-08-30 18:15:51 +03:00
tetrisgm 5afe95b158 runtime: add title-scoped exact NCLIP branches
(cherry picked from commit 81ebee1346)
2026-08-27 20:44:13 -07:00
Matthew Stanley 5c876bd385 recompiler: add standalone static analyzer 2026-08-25 00:41:00 -07:00
Unchiga 50c730cadf Forward a pending load into an LWL/LWR merge
LWL/LWR take their destination register as both the merge base and the
destination, and they read that base late enough to receive the forwarded
result of an immediately preceding load. They are therefore not subject to the
MIPS-I load delay: after

    lw   $t0, 12($a3)
    lwr  $t0, 10($a3)

the LWR merges into the word the LW just fetched, not the stale $t0.

The load-delay pair emitter defers a load's writeback into a psx_ldd_<addr>
temporary, so an LWL/LWR successor was merging into cpu->gpr[rt], which still
held the pre-load value. The merge operand now names the deferred temporary
when the successor is an LWL/LWR writing the same register, and the ordinary
writeback is suppressed there because the merge has already consumed it.

Only the consumer side needs this. LWL/LWR as the producer never defer, so
their writeback is already immediate.

Verified against a full regeneration of a title that exercises the pair: 16
forwarding sites, and every emitted merge operand identical to the reference
output built with this fix in place.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-19 07:58:58 -07:00
Matthew Stanley 200b5bf003 Add opt-in widescreen culling keep hooks 2026-08-15 19:42:43 -07:00
Alex Vanderveen 94cb3807a9 Merge origin/master into integrate/game-codegen-hooks-into-rollback
Keep rollback/netplay multiplayer routing, dual-raster GL present, and TOC
fingerprints while taking master's CHD support, mod load acceleration,
adaptive widescreen, and geometry/perspective APIs.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-06 22:50:38 -04:00
Alex Vanderveen 652be7ba02 rollback FMV patches and overlay 2026-08-06 00:52:14 -04:00
Matthew Stanley b2af4e89c9 feat(mods): add trusted game enhancement hooks 2026-08-03 15:17:39 -07:00
Alex Vanderveen 3da61df355 Revert "ok"
This reverts commit b836f2198c.
2026-07-29 03:18:10 -04:00
Alex Vanderveen b836f2198c ok 2026-07-29 03:15:42 -04:00
Matthew Stanley a292a7c136 Add activation-only widescreen cull margin 2026-07-28 11:08:03 -07:00
Matthew Stanley a6bd9455ca fix: exclude fail-closed stubs from game dispatch 2026-07-27 10:48:41 -07:00
Matthew Stanley 2696c391b8 feat: add aspect-aware world participation 2026-07-27 10:48:41 -07:00
Matthew Stanley 436fe14101 widescreen: support guarded object participation overrides 2026-07-25 23:24:41 -07:00
Matthew Stanley 986d8be94c Merge remote-tracking branch 'origin/master' into integrate/openbios-bundle
# Conflicts:
#	recompiler/src/full_function_emitter.cpp
#	recompiler/src/full_function_emitter.h
#	recompiler/src/main_bios.cpp
#	runtime/runtime.cmake
#	runtime/src/main.cpp
2026-07-25 12:31:05 -07:00
Matthew Stanley 68667cd58b disc+bios: either file the user picks resolves, mounts, and verifies
Two path-resolution defects where the runtime silently accepted only one of
two equally-valid spellings of the same input.

1. Disc: picking the .cue threw away the disc's TOC.

normalize_disc_path_for_launch() swapped any .cue for its same-named .bin
unconditionally, and every disc entry point (game.toml, --disc, the cached
disc.cfg, the launcher's Change Disc, the first-run dialog) went through it.
That discarded the cue's table of contents -- every CD-DA track and pregap
with it -- even though iso_reader.cpp has full multi-FILE cue support that
the swap defeated before it was ever reached. In the other direction,
picking a redump audio-track bin mounted that track as the whole disc, which
cannot boot.

resolve_disc_path() (runtime/include/disc_path.h) now canonicalizes any pick:

  .cue that resolves            -> mount the cue (TOC intact)
  .cue with a missing bin       -> the payload it still names, else a
                                   stem-matched sibling image
  .bin/.iso/.img owned by a cue -> mount that cue
  bare image, no cue            -> mount it

identify_disc() resolves through the same code, so the header/serial/CRC32
verdict is read from the data track no matter which member was picked.

While here: the mount path and the verify path had two divergent private cue
parsers (quoted-only + case-sensitive vs quoted-or-bare + case-insensitive),
so they could disagree about the same file. Both now use one parser
(runtime/include/cue_sheet.h), which additionally handles CRLF, backslash
paths, and case-insensitive payload lookup for case-sensitive filesystems.

2. BIOS: the multi-BIOS rename made configs and dump folders incompatible.

SCPH*.toml name "US-PSX-SCPH1001.BIN" while every pre-existing dump folder,
bios.cfg and tools/regen_bios.sh default name "SCPH1001.BIN". Either
combination fataled with "cannot open BIOS file" despite the right ROM being
present. resolve_bios_rom() (recompiler/include/bios_rom_alias.h) matches on
the model token -- the last '-'-separated component of the stem -- so both
conventions resolve in both directions without a hardcoded alias table.
Applied at the config loader, the positional psxrecomp-bios CLI, and each
rung of the runtime's BIOS search.

The positional CLI also derived out_stem from the raw filename, so a
region-qualified ROM would have emitted US-PSX-SCPH1001_full.c -- a name
runtime.cmake cannot link. It now derives the model token.

Also fixes SCPH5552.toml, whose rom field omitted the bios/ directory that
its two siblings have.

Tests: test_disc_path_resolve (36 checks, five dump layouts) and
test_bios_rom_alias (15 checks, both directions + negative cases), both
registered in runtime/CMakeLists.txt. Registered the pre-existing
test_iso_reader_cdda, which was wired into no build.

Verified on real dumps: Ape Escape .cue and .bin both give
SCUS-94423 / NTSC-U / crc=C6F455BC; Tomba 2's multi-track dump gives
crc=1FB8C151 and a 2-track TOC from the .cue, the data bin, and the CD-DA
bin alike.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-24 22:17:52 -07:00
Henrique Guedes 6749b826d4 feat: OpenBIOS support — profile-driven BIOS recompilation
Adds PCSX-Redux's MIT-licensed OpenBIOS as a second BIOS the framework can
statically recompile, next to retail SCPH1001. Because OpenBIOS is
redistributable, a game verified against it ships its BIOS with the build —
players supply only a disc, and the launcher BIOS picker disappears.

The mechanism is general, not OpenBIOS-specific: every BIOS image gets a
bios/<STEM>.toml profile that is the single source of truth for its identity
and layout.

Recompiler
- bios/<STEM>.toml profiles: [program.image] (sha256 pin, redistributable
  flag), [recompiler.address_model] (boot-time ROM->RAM copy windows,
  kernel-bless window), install_slots, and [recompiler.runtime_exports]
  HLE anchors (0 = feature structurally unavailable on that BIOS).
- BiosAddressModel (new): validated model built from the profile, replacing
  ~9 hardcoded SCPH1001 window constants that had drifted apart across the
  emitter / discovery / game codegen. The emitted C normalize() derives
  from the same table. from_config rejects overlapping/misaligned windows
  and orderings where sequential folding != first-match. A CTest-registered
  equivalence test sweeps every RAM/ROM address against verbatim copies of
  the old helpers.
- Orphaned delay slots (the OpenBIOS-critical emitter fixes): jr/jalr
  slots are inlined before the jump resolves — previously their side
  effect was silently LOST (e.g. a case-return fragment's addiu $sp pop,
  which unwound a shifted frame and walked the exception handler's IntRP
  cursor into a wild dispatch). Orphaned branches resume at ds+4 on
  not-taken instead of re-executing the slot (a read-modify-write slot
  doubling twice broke OpenBIOS readPad). Jump targets latch pre-slot.
- MIPS-I load-delay modeling for dependent load pairs: the successor sees
  the pre-load value (OpenBIOS's cardfasttrack.s relies on it on purpose —
  "gotta break those bad emulators"). Unmodeled shapes warn at build time.
- Discovery hardening: branch-target dispatchability closure to fixpoint;
  jump tables placed at jr+8 no longer walked as code; cross-function
  continuations routed by an instruction->owner map.
- Signed MULT support (unreachable from the retail seed set; modern-GCC
  OpenBIOS emits it freely), latency-modeled like MULTU.
- Seeds from ELF symbol tables + data-section code-pointer harvest
  (tools/gen_bios_seeds_from_elf.py) — a source-built BIOS needs no
  disassembly pass. 646 OpenBIOS seeds, incl. 5 empirical dispatch-miss
  seeds.
- Stemmed outputs (<out_stem>_full.c / _dispatch.c) so two BIOSes coexist
  in one out_dir. tools/regen_bios.sh is profile-driven (--config).

Runtime
- psx_bios_image: the generated dispatch now self-describes the BIOS it
  was compiled from (identity: size/CRC32/SHA-256/wordsum; kernel-bless
  window; HLE anchors; bundled flag). Runtime SCPH1001 hardcodes removed.
- Launch check is a hard identity gate (a mismatched image under
  statically recompiled code is a guaranteed wild jump). Bundled builds
  hide the BIOS picker and treat a missing/tampered bundled image as an
  installation defect.
- Faithfulness fixes surfaced by OpenBIOS's tighter timing loops (all
  global, argued in code comments, regression-tested against SCPH1001):
  - SIO pad RX is cycle-paced like the memcard path (RX_RDY at
    shift-complete, ACK +170); total ack-IRQ latency unchanged.
    PSX_SIO_PAD_SYNC_RX=1 restores the legacy synchronous path for A/B.
  - SPUSTAT derived from SPUCNT instead of permanently-asserted busy.
  - GPUSTAT.13 (interlace field) alternates per vblank while interlaced,
    pinned 0 in progressive.
  - CD Init(0x0A) completes in ~3.9 ms instead of borrowing Beetle's
    34 ms Reset figure (OpenBIOS's short bounded retry loop livelocked).
  - Exception EPC guard accepts ROM-window block leaders, not just RAM.
- New TCP debug commands (documented in TCP_COMMANDS.md): bios_info,
  s3_smear_watch, callret_watch; write-trace ring gains $s0-$s5.

bios/openbios.bin is vendored (MIT — notice in bios/OpenBIOS.LICENSE,
attribution in THIRD_PARTY_ATTRIBUTION.md, source pin + build recipe in
bios/OpenBIOS.toml). The emitter fingerprint digest is now content-only:
existing generated/*.emitter.sha will read STALE once; re-run
tools/regen_bios.sh.
2026-07-23 17:10:23 -03:00
Claudio Varandas 64ca4c3dba recompiler: [widescreen.cull] bltz_sites (explicit left-edge funnel widen)
Left-edge counterpart to slti_sites: emit psx_ws_cull_bltz at configured
bltz PCs. detect_cull_bltz_sites only classifies left-edge bltz for
auto_screen_x-qualified functions, so an X-only funnel wired via explicit
slti_sites had no left-edge widen. bltz_sites closes that gap. Identity at
4:3; regen-class.
2026-07-22 22:36:41 +01:00
Alex Vanderveen 31015ceab1 feat: launcher polish and recomp-net lobby netplay
Wire MotK-style lobby (recomp-net submodule, WS client, match_caps),
netplay delay-sync, and launcher UX so host sim settings stay aligned
across peers.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-21 21:37:57 -07:00
Opok 4412bc7a7d widescreen: add per-primitive X-reject bound-load cull sites
Add opt-in [widescreen.cull] xclip_load_sites: a configured `lw rt,off(rs)`
loading a per-primitive X-reject bound (sltu-compared against ANDI-masked
u16 screen X) routes through psx_ws_xclip_bound() - INT32_MAX while the
margins are revealed (reject disabled; the wide-surface scissor clips the
overflow, wrapped off-left u16 coords pass), vanilla value at 4:3. Matched
native codegen emit and interpreter hook; overlay ABI v18 forwarder (NULL
preserves the vanilla loaded value); codegen ver 8. Parser, codegen and
overlay-mismatch regression coverage.

Split from #49: replaces the xclip_globals RAM poke with a recompile-time
instruction transform so native shards and the interpreter behave identically.
2026-07-21 15:02:20 -07:00
Opok a5fdf1ad9a widescreen: add side frustum-plane normal-X cull sites
Add opt-in [widescreen.cull] plane_nx_sites: a configured `lw rt,off(rs)`
loading the X component of a side frustum-plane normal (sign-tested per
corner, dot = nx*px + nz*pz) routes through psx_ws_plane_nx() — scaled by
the inverse aspect factor (4*den)/(3*num) while the margins are revealed,
widening the plane cone by exactly atan((3*num)/(4*den)*tan(theta));
identity at 4:3. Matched native codegen emit and interpreter hook; overlay
ABI v17 forwarder (NULL preserves the vanilla normal); codegen ver 7.
Parser, codegen and overlay-mismatch regression coverage.

Split from #49: replaces the plane sign-test RAM pokes with a
recompile-time instruction transform so native shards and the interpreter
behave identically.
2026-07-21 14:38:55 -07:00
Matthew Stanley 8f78ddbd89 integrate interpreter hotpath and AOT sharding 2026-07-20 15:58:43 -07:00
Matthew Stanley fa626bdb4c feat: add aspect-scaled depth cull sites
Add opt-in slti/sltiu far-bound scaling with matched native and interpreter semantics, correct immediate sign extension, and an overlay ABI forwarder for live aspect state.

Co-authored-by: Opok <novelasevilliousesp@gmail.com>
2026-07-20 12:14:18 -07:00
Matthew Stanley 288926111f feat: add masked-u16 widescreen cull sites
Add a shared native/interpreter helper for sltiu classifiers operating on ANDI-masked 16-bit screen X values, preserving MIPS immediate sign extension and overlay linkage.

Co-authored-by: Opok <novelasevilliousesp@gmail.com>
2026-07-20 12:11:58 -07:00
Matthew Stanley 5ded80e61e Merge master (AOT overlay generation) into perf/interpreter-hotpath
Integration branch: the interpreter-perf series (interpreter fallback +
additive shard autocompile + always-on production observability + the
fail-closed alias-body emitter fix) rebased-by-merge onto master's AOT
overlay generation (play-free extraction, bounded indirect discovery,
AOT recall, shadow-diff redesign).

Semantic-union highlights:
- gte.cpp: HEAD's direct-field marshaling fast paths kept; master's
  s_gte_replay_sandbox guards grafted onto every affected site.
- psx_icache.c: HEAD's global tag array + fetch_miss split kept;
  master's transactional shadow record/replay retargeted onto it.
- overlay_capture.c: HEAD's commit/retry/wait-pending pipeline kept;
  master's durable history + autocapture APIs wired into it
  (persist_history_snapshot fed from the immutable .d contribution).
- overlay_loader.c: master's interp-record/native-replay shadow-diff
  redesign kept as base with HEAD's scheduler-bail hardening grafted
  (new SHDIFF_SCHED divergence kind); strict manifest v2 parsing keeps
  master's P pair-id lines; always-on native-call ring preserved.
- main.cpp shutdown calls BOTH overlay_autocapture_shutdown() and
  overlay_capture_wait_pending() (the latter drains the preserve-write
  FIFO the former does not).
- recompiler: master's generate_alias_group refactor retains the
  fail-closed alias-body re-entry defaults from c85f1d1.

Validated on this tree: recompiler + runtime + tests build; all six
runtime unit tests PASS; interpreter perf guards PASS; recompiler
patch/structural tests PASS (l1 decoder oracle noise = 376 pre-existing
on both parents).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 00:43:41 -07:00
Matthew Stanley bdeb106e5c feat: add negsub widescreen cull sites
Add an opt-in horizontal low-edge transform for subu rd,zero,rt classifiers, with matching native and dirty-RAM interpreter behavior and guarded overlay variants.

Co-authored-by: Opok <novelasevilliousesp@gmail.com>
2026-07-19 23:25:03 -07:00
Matthew Stanley 504db26445 feat: add bounded indirect overlay discovery 2026-07-19 17:15:17 -07:00
Matthew Stanley 05f6ef3e02 Optimize GTE fallback and isolate interpreter perf runs 2026-07-19 14:23:36 -07:00
Matthew Stanley 91c9aea91f aot: make discovery and shard publication fail closed 2026-07-18 14:08:45 -07:00
Matthew Stanley 495bf2de53 aot: constrain calls across composite producers 2026-07-18 02:56:42 -07:00
Matthew Stanley 41370a64e4 recompiler: split SCUS_*_full.c into parallel-compilable shards
Emit a shared <stem>_decls.h (runtime extern prologue, static-inline
unaligned helpers, all func + alias-body forward decls) and distribute
function bodies across N ~40k-line <stem>_full_NN.c shards instead of one
monolithic <stem>_full.c. Overlapping-alias host bodies become externally
linked so shard bucketing has no atomicity constraint. runtime.cmake's
GAME_GENERATED_FULL_C is now multi-value (single path still works, so
other games are unaffected until they regen + glob).

Tomba: SCUS_942.36_full.c 41MB single TU -> 36 shards; compile (-O3)
4m35s -> 51s (-j16, ~5.4x). Generated function bytes verified byte-identical
(6245/6245 functions).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 11:54:11 -07:00
Matthew Stanley 411532bd87 Merge remote-tracking branch 'origin/master' into spike/load-time-zero 2026-07-14 23:09:27 -07:00
douglasjv 28165d6421 Expand Einhander player bounds for native widescreen 2026-07-14 23:06:05 -07:00
Matthew Stanley 44b895e331 Merge remote-tracking branch 'origin/master' into spike/load-time-zero 2026-07-14 22:48:36 -07:00
Matthew Stanley 64b084e2a7 load: add opt-in event horizon, warm routes, and audio sink 2026-07-14 22:44:31 -07:00
Matthew Stanley 12934ed2c5 data-shards spike: memoized pure-function replay (capture/verify/replay + cycle-faithful credit)
- runtime/src/data_shards.c + include: read-before-write recorder fed from
  the memory chokepoints; byte-verified replay (kernel-bless pattern);
  recorded-cycle credit in slices through psx_advance_cycles +
  psx_check_interrupts_at; purity poisons (MMIO/DMA/thread-switch/budget);
  .dss persistence under the unified cache dir; TCP data_shards cmd.
- emitter: [data_shards] funcs -> psx_datashard_enter at entry +
  psx_datashard_ret at jr-$ra sites (rolls cg-hash).
- obs: phase_hot set=static keyed on new g_psx_last_fn_entry entry stamp
  (current_func dispatch stamp survives post-IRQ resumption and pooled 55%
  of static samples on kernel 0xF40); TCP starv_ring query for the
  always-on PC-sample ring.
- docs/DATA_SHARDS.md: design, soundness model, gates, FMV exclusion.
2026-07-14 11:08:02 -07:00
Matthew Stanley bd705bfcec Add bounded reachable main-EXE discovery
Adapt Douglas's opt-in reachable discovery onto current exact/static dispatch. Validate configured static-analysis bounds before truncation, preserve the whole-image default and canonical page reservations, and leave unresolved indirect or out-of-bound targets to runtime fallback.

Source: https://github.com/douglasjv/mm8

Source-commit: https://github.com/douglasjv/mm8/commit/8415f3a95458c41c7f48fbe36caaf0ee82730720

Co-authored-by: douglasjv <vanderveerdouglas@gmail.com>
2026-07-13 21:56:43 -07:00
Matthew Stanley b39387eb00 Fix exact native and static overlay dispatch 2026-07-13 05:23:16 -07:00
Matthew Stanley dbfc124856 Merge master into feat/mmx4-widescreen-fw
Master brought the MMX5 true-wide generalization, which evolved this
branch's earlier X4 work (unified psx_ws_bg2d_* helpers, copied-a1 cull,
sltiu count sites, nw_left_hud_packet arena filter). Resolution keeps
master's evolved implementations and grafts the X4-only pieces on top:

- bg2d count_site accepts signed slti alongside sltiu (master style
  ?1u:0u emission); startx_site accepts subu rd,zero,rt (X4's
  -(scrollX & 15)).
- cull a1 move sites accept both move encodings (addu rD,a1,zero and
  addu rD,zero,a1) with a distinct addu-but-not-move error.
- HUD arena filter: master's nw_left_hud_packet_lo/hi is the single
  mechanism (X4's parallel nw_hud_source_lo/hi dropped); X4's broader
  prim coverage kept — ws_nw_hud_shift_vertices re-anchors polygons,
  lines and dots as rigid composites alongside the rect/sprite sites.
- mono_rect gets both ws_expand_fullscreen_rect and the HUD re-anchor.

Also carried from this branch: binary-search dispatch table and
Release-gated GL frame_perf instrumentation (auto-merged).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 10:52:04 -07:00
Matthew Stanley 97ea975580 widescreen: X4-generation 2D hooks — inline-count bg2d variants, copied-a1 cull, HUD source-range filter
Generalizes the Capcom 2D widescreen machinery for the MMX4 engine revision:

- [widescreen.bg2d] count_site accepts the inline slti/sltiu loop compare
  (X4/X5 style) alongside MMX6's li; startx_site accepts subu rd,zero,rt
  (X4's -(scrollX & 15)); inline-count sites route through the generic
  psx_ws_bg2d_* helpers that keep the emitted ring mask and skip MMX6's
  title-specific host-side refill state.
- [widescreen.cull] a1_sites now also accept a move rd,a1 (addu encoding):
  classifiers that copy the caller margin with no delay-slot nop widen the
  copied value instead.
- [widescreen] nw_hud_source_lo/hi: optional GP0 command-source range filter
  for nw_hud_corners, so a pure-2D game re-anchors only its dedicated HUD
  packet arena and never world sprites. 0/0 preserves legacy behavior.
  ws_nw_hud_shift_vertices re-anchors polygon/line prims as one rigid
  composite; mono rect/dot/8x8 sites shift too.

All helpers identity at 4:3; unset config keys are byte-identical.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 10:32:40 -07:00
Matthew Stanley 40b476c4e4 widescreen: finish MMX5 edge handling 2026-07-10 10:07:40 -07:00
Matthew Stanley 57ce271990 Merge branch 'master' into feat/mmx5-widescreen-fw
# Conflicts:
#	recompiler/src/config_loader.cpp
#	recompiler/src/config_loader.h
#	recompiler/src/main_psx.cpp
#	runtime/src/gpu.c
#	runtime/src/main.cpp
2026-07-10 08:52:29 -07:00
Matthew Stanley 79cdb12eed feat: support MMX6 finite-map widescreen 2026-07-10 01:47:29 -07:00
Matthew Stanley d9dae48c8e widescreen: generalize Capcom 2D background hooks for MMX5 2026-07-10 00:57:03 -07:00
Matthew Stanley ee09ab9bbb widescreen: generic Ape-Escape-class support + native-wide GL perf + cutscene flicker fix
Adds full 16:9/21:9 native-wide support for fully-3D titles (Ape Escape,
SCUS-94423) as a generic, opt-in, 4:3-byte-identical framework capability, plus
a native-wide GL compositor perf fix and a cutscene present-mode flicker fix.

Widescreen (generic, per-game opt-in via [widescreen*]):
- Shared screen-extent cull detector (runtime/include/ws_cull_detect.h) used by
  all three backends (recompiler, dirty-RAM interp, sljit). Per-game signature
  immediates ([widescreen.cull] screen_w_imms/screen_h_imms) — Ape's funnel
  tests 0x181/0xF1 on its 368px display, not Tomba's 0x140/0xE0.
- Three funnel idioms widened: unsigned per-vertex sltiu (psx_ws_cull_sltiu),
  signed min/max slti right edge + bare-bltz left edge (psx_ws_cull_slti/bltz,
  new helpers + SLJIT_HLP appends), and billboard center+/-halfwidth. Explicit
  [widescreen.cull] slti_sites for X-only funnels the auto-detector can't
  qualify.
- gte_game_mode: GTE-activity gameplay detector for 3D titles with no sprite-tag
  helper (gte.cpp notes every RTPS/RTPT). STICKY (>=3 verts, 45-frame hysteresis)
  so sparse cutscene frames never flip to a 4:3 pillarbox = fixes intro flicker.
- nw_hud_corners: push outer-third screen-space HUD sprites to the true wide
  corners in native-wide (else inset by the reveal).
- nw_backdrop: stretch a full-frame axis-aligned 2D backdrop quad to fill the
  wide frame (for titles with a flat sky plane).
- Pattern-scanned interp/sljit widen hooks now gated per-game (auto_screen_x /
  auto_backdrop): fixes an ungated auto_backdrop false-positive that rewrote a
  live GPR = the ~24s native-wide crash on Ape.

Native-wide GL perf (fixes 16:9 frame drops):
- The wide surface centre equals the canonical 4:3 framebuffer, so wide_blit_center
  copies canonical->wide-centre at present (both present paths) and the per-prim
  mirror SKIPS prims fully inside the 4:3 frame (mirror_x/geo/batch_center_only).
  Only reveal-margin prims re-rasterize. Peak frame time 38.8ms -> 17.5ms, no
  vsync misses. Toggle gl_wide_fast; 4:3 untouched (all gated on g_wide_cur).

Runtime-only except the recompiler cull emit (regen needed for cull widening).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 19:30:09 -07:00
Matthew Stanley 8ab24c5e62 wip(tomba2): ABI v8 check_interrupts_at + emitter charge-cycle work (pre-rebase snapshot)
In-progress investigation work snapshotted before rebasing onto the
tcc-refactored psxrecomp master. Includes overlay ABI v8
(check_interrupts_at resume-PC callback for async RFE recovery) and
code_generator/full_function_emitter delay-slot cycle accounting.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 23:26:04 -07:00
Matthew Stanley 598d1dd551 Revert "recompiler: roll back AE pre-prologue function-start discovery (Tomba regression)"
This reverts commit b9e38416ca.
2026-06-25 21:29:35 -07:00
Matthew Stanley b9e38416ca recompiler: roll back AE pre-prologue function-start discovery (Tomba regression)
The Ape Escape forward-port (e0c819c) added pre-prologue setup detection to
function_analysis (find_preprologue_setup_start / preprologue_window_is_valid +
is_bios_dispatch_thunk), extending function STARTS backward to absorb preceding
lui/lw global-load setup. On Tomba this mis-bounds a function in the Village-of-
All-Beginnings init path (function count 5562 -> 5492, ~70 functions absorbed),
producing a main-loop softlock: New Game / loading the Village save wedges with
the main loop spinning on the 0xCF0 SIO poll, never rendering the area (a "blue
screen"). Backend-independent (gcc + tcc), not overlay/interp related.

This worked in v0.5.x (pin 5007c46, pre-AE). Revert function_analysis.{cpp,h} to
that state. Confirmed: the 0xCF0 wedge signature disappears (repro cf0_frac
1.00 -> 0.02) and Tomba boots/plays again. Ape Escape is not a public release, so
rolling back its discovery changes is safe; MMX6's existing release is unaffected
(its binary was already built), but a future MMX6 rebuild should be re-verified
against this discovery. A follow-up can re-introduce the pre-prologue heuristic
correctly (game-gated or with a tighter boundary check) without the over-extension.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 19:53:05 -07:00
Matthew Stanley 01b4d91e8a WIP: forward-port Ape Escape framework (dev/ape-escape) onto master
Comprehensive 3-way merge of the dev/ape-escape branch into master for
the isolated wt/ape-escape worktree. Brought in every complementary ape
change automatically (dma.c, memory.c interrupt byte/half lanes,
function_analysis discovery, main_bios vector seeds, iso_reader
iso_sector_count, config_loader, crash-trace fields, headers). Resolved
the 13 conflicts by rule:

  - CPS dispatch / recompiler (code_generator, full_function_emitter,
    dirty_ram_interp, traps, main_psx) -> KEEP MASTER (evolved system).
  - CD model (cdrom.c) -> take ape (the model that booted Ape).
  - interrupts.c/.h -> reverted to MASTER (ape's interrupt-delivery is
    dispatch-coupled; the merged Frankenstein stormed).
  - sio/debug_server/freeze_heartbeat/main.cpp -> keep master (evolved).
  - crash_trace.c -> master (ape freeze-dump fns omitted to avoid the
    debug-ring cascade).

STATUS: does NOT boot yet. ape-cdrom + master-interrupts derails to
PC=0 at frame ~2092 (insta-close); ape-cdrom + ape-interrupts storms
(297M exception re-entries) at frame 2077. Root finding: ape's CD
anti-storm is coupled to ape's interrupt-delivery + call-depth + EPC
model, which master replaced with CPS continuation-passing, so the CD
subsystem does not forward-port as pure hardware simulation. Preserved
as the reference state for a design consult on bridging the two models.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-21 15:55:23 -07:00