Commit Graph

196 Commits

Author SHA1 Message Date
Matthew Stanley 4bf868da4f feat(bios): ship OpenBIOS as a runtime dependency 2026-07-25 18:11:57 -07:00
Matthew Stanley 986d8be94c Merge remote-tracking branch 'origin/master' into integrate/openbios-bundle
# Conflicts:
#	recompiler/src/full_function_emitter.cpp
#	recompiler/src/full_function_emitter.h
#	recompiler/src/main_bios.cpp
#	runtime/runtime.cmake
#	runtime/src/main.cpp
2026-07-25 12:31:05 -07:00
Matthew Stanley b8adb758d2 docs: specify BIOS selection — OpenBIOS by default, retail on request
Writes down the contract before implementing it, because one part is a
behaviour change someone would otherwise file as a bug.

The rule: no BIOS chosen -> OpenBIOS; a BIOS the player explicitly chose
-> that BIOS. A developer can require retail for a title with a verified
OpenBIOS incompatibility via [runtime] openbios = false.

"Chosen" means --bios or a launcher/settings pick, and nothing else.
Today resolve_bios_path() also searches directories near the executable,
so a stray SCPH1001.BIN would silently win even though the player never
asked — making behaviour depend on what happens to be on disk. Discovery
stops being a selection mechanism.

Also recorded: why both BIOSes are compiled into one binary (a build
flavour per BIOS needed the same choice restated in three places that
nothing cross-checked, and it compiled cleanly when they disagreed); why
that is affordable (game C is byte-identical across profiles — verified
on Tomba!, Tomba! 2 and Ape Escape — so only the BIOS is duplicated, and
the two collide on just 22 symbols); that the developer setting is not
player-overridable; and that savestates are BIOS-specific and must be
refused across BIOSes, which is the one silent-corruption risk here.

README gets a player-facing version: you probably don't need a BIOS,
here's how to use your own, saves work either way, savestates don't mix.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 11:31:39 -07:00
Matthew Stanley bfa760f43b build: fail early with a clear message when recompiled game C is missing
The runtime links generated/<serial>_{full,dispatch}.c, produced by a separate
recompiler step run before the build. Those paths are marked GENERATED so cmake
configure succeeds before that step has run — which meant skipping generation
surfaced only as a raw "cc1: fatal error: <file>: No such file or directory"
deep in the build, with no hint that a step was skipped or what produces the file
(exactly the trap a from-source builder on a fresh checkout hits).

Add runtime/check_generated_sources.cmake and wire it into
psxrecomp_add_runtime_target as a build-time dependency of the runtime target:
a missing game source now aborts at build start with the files named and the
exact fix (build recompiler -> extract EXE -> generate -> rebuild), and configure
emits an early non-fatal warning. Framework-wide, so every game inherits it.

Also fix the docs/BUILDING.md "build and run a game" recompiler invocation (the
framework is a submodule at psxrecomp/, not ../psxrecomp/) and add the
recompiler-build step.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 09:36:14 -07:00
Henrique Guedes 6749b826d4 feat: OpenBIOS support — profile-driven BIOS recompilation
Adds PCSX-Redux's MIT-licensed OpenBIOS as a second BIOS the framework can
statically recompile, next to retail SCPH1001. Because OpenBIOS is
redistributable, a game verified against it ships its BIOS with the build —
players supply only a disc, and the launcher BIOS picker disappears.

The mechanism is general, not OpenBIOS-specific: every BIOS image gets a
bios/<STEM>.toml profile that is the single source of truth for its identity
and layout.

Recompiler
- bios/<STEM>.toml profiles: [program.image] (sha256 pin, redistributable
  flag), [recompiler.address_model] (boot-time ROM->RAM copy windows,
  kernel-bless window), install_slots, and [recompiler.runtime_exports]
  HLE anchors (0 = feature structurally unavailable on that BIOS).
- BiosAddressModel (new): validated model built from the profile, replacing
  ~9 hardcoded SCPH1001 window constants that had drifted apart across the
  emitter / discovery / game codegen. The emitted C normalize() derives
  from the same table. from_config rejects overlapping/misaligned windows
  and orderings where sequential folding != first-match. A CTest-registered
  equivalence test sweeps every RAM/ROM address against verbatim copies of
  the old helpers.
- Orphaned delay slots (the OpenBIOS-critical emitter fixes): jr/jalr
  slots are inlined before the jump resolves — previously their side
  effect was silently LOST (e.g. a case-return fragment's addiu $sp pop,
  which unwound a shifted frame and walked the exception handler's IntRP
  cursor into a wild dispatch). Orphaned branches resume at ds+4 on
  not-taken instead of re-executing the slot (a read-modify-write slot
  doubling twice broke OpenBIOS readPad). Jump targets latch pre-slot.
- MIPS-I load-delay modeling for dependent load pairs: the successor sees
  the pre-load value (OpenBIOS's cardfasttrack.s relies on it on purpose —
  "gotta break those bad emulators"). Unmodeled shapes warn at build time.
- Discovery hardening: branch-target dispatchability closure to fixpoint;
  jump tables placed at jr+8 no longer walked as code; cross-function
  continuations routed by an instruction->owner map.
- Signed MULT support (unreachable from the retail seed set; modern-GCC
  OpenBIOS emits it freely), latency-modeled like MULTU.
- Seeds from ELF symbol tables + data-section code-pointer harvest
  (tools/gen_bios_seeds_from_elf.py) — a source-built BIOS needs no
  disassembly pass. 646 OpenBIOS seeds, incl. 5 empirical dispatch-miss
  seeds.
- Stemmed outputs (<out_stem>_full.c / _dispatch.c) so two BIOSes coexist
  in one out_dir. tools/regen_bios.sh is profile-driven (--config).

Runtime
- psx_bios_image: the generated dispatch now self-describes the BIOS it
  was compiled from (identity: size/CRC32/SHA-256/wordsum; kernel-bless
  window; HLE anchors; bundled flag). Runtime SCPH1001 hardcodes removed.
- Launch check is a hard identity gate (a mismatched image under
  statically recompiled code is a guaranteed wild jump). Bundled builds
  hide the BIOS picker and treat a missing/tampered bundled image as an
  installation defect.
- Faithfulness fixes surfaced by OpenBIOS's tighter timing loops (all
  global, argued in code comments, regression-tested against SCPH1001):
  - SIO pad RX is cycle-paced like the memcard path (RX_RDY at
    shift-complete, ACK +170); total ack-IRQ latency unchanged.
    PSX_SIO_PAD_SYNC_RX=1 restores the legacy synchronous path for A/B.
  - SPUSTAT derived from SPUCNT instead of permanently-asserted busy.
  - GPUSTAT.13 (interlace field) alternates per vblank while interlaced,
    pinned 0 in progressive.
  - CD Init(0x0A) completes in ~3.9 ms instead of borrowing Beetle's
    34 ms Reset figure (OpenBIOS's short bounded retry loop livelocked).
  - Exception EPC guard accepts ROM-window block leaders, not just RAM.
- New TCP debug commands (documented in TCP_COMMANDS.md): bios_info,
  s3_smear_watch, callret_watch; write-trace ring gains $s0-$s5.

bios/openbios.bin is vendored (MIT — notice in bios/OpenBIOS.LICENSE,
attribution in THIRD_PARTY_ATTRIBUTION.md, source pin + build recipe in
bios/OpenBIOS.toml). The emitter fingerprint digest is now content-only:
existing generated/*.emitter.sha will read STALE once; re-run
tools/regen_bios.sh.
2026-07-23 17:10:23 -03:00
Matthew Stanley f86087ad94 Remove stale RmlUi launcher remnants 2026-07-23 00:29:22 -07:00
Alex Vanderveen 7cf9958135 FMV optimizations 2026-07-21 21:38:04 -07:00
Alex Vanderveen 0b890cdc13 lobby game version control 2026-07-21 21:38:04 -07:00
Alex Vanderveen 31015ceab1 feat: launcher polish and recomp-net lobby netplay
Wire MotK-style lobby (recomp-net submodule, WS client, match_caps),
netplay delay-sync, and launcher UX so host sim settings stay aligned
across peers.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-21 21:37:57 -07:00
Matthew Stanley e03c743e87 tools: add Linux macOS dev setup helper
Add a conservative source-checkout setup helper based on the Linux quick-start intent from PR #54.

The script builds the CLI/recompiler tools, refreshes BIOS C when a BIOS is present, and builds the BIOS-only runtime only when BIOS/generated sources are available. It deliberately leaves game project creation to the existing CLI generator.

Co-authored-by: Keeganator <keegan.pan@gmail.com>
2026-07-21 17:02:05 -07:00
Matthew Stanley c4517c3b7c docs: credit PR16 Vulkan source developer
Record that PR #61 reconstructed the reusable Vulkan work from Martin Penkava's PR #16 branch.

Co-authored-by: Martin Penkava <mpenkava1337@gmail.com>
2026-07-21 16:42:54 -07:00
Matthew Stanley 9228f32c00 runtime: keep Vulkan opt-in and remove old launcher
Remove the in-tree launcher implementation and its submodules now that game projects use recomp-ui for launcher support.

Keep Vulkan hidden from the launcher unless a game explicitly sets [video] offer_vulkan = true, and clamp stale Vulkan settings back to OpenGL when it is not offered.

Co-authored-by: Martin Penkava <mpenkava1337@gmail.com>
2026-07-21 16:31:36 -07:00
Matthew Stanley da64f44e82 launcher: expose the Vulkan renderer
Cycle the renderer selector through Software, OpenGL, and Vulkan while retaining OpenGL-only option gating.

Co-authored-by: Martin Penkava <mpenkava1337@gmail.com>
2026-07-21 15:55:16 -07:00
Matthew Stanley f1fd8ec848 vulkan: expose present blocking telemetry
Record queue-wait, acquire, and present durations plus presentation timestamps in the existing vk_perf ring.

Co-authored-by: Martin Penkava <mpenkava1337@gmail.com>
2026-07-21 15:55:11 -07:00
Matthew Stanley f5498f796c vulkan: prefer MAILBOX for tear-free presentation
Let the frontend remain the sole frame pacer by requesting MAILBOX for tear-free delivery, with FIFO retained as the required fallback.

Co-authored-by: Martin Penkava <mpenkava1337@gmail.com>
2026-07-21 15:55:06 -07:00
Matthew Stanley af9de64951 vulkan: reuse persistently mapped staging buffers
Cache best-fit host-visible staging allocations across queue-idle reclamation points and safely detach long-lived CPU-present buffers during teardown.

Co-authored-by: Martin Penkava <mpenkava1337@gmail.com>
2026-07-21 15:54:42 -07:00
Matthew Stanley b5e9f763fe vulkan: batch work into one command buffer
Keep a shared work command buffer open across logical GPU operations and submit it at the existing synchronization boundary.

Co-authored-by: Martin Penkava <mpenkava1337@gmail.com>
2026-07-21 15:54:29 -07:00
Matthew Stanley 92cbb1a327 vulkan: order back-to-back color passes
Add same-layout color attachment barriers before geometry and wide-surface render passes so prior writes are visible to LOAD and blend operations.

Co-authored-by: Martin Penkava <mpenkava1337@gmail.com>

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 15:54:17 -07:00
Matthew Stanley 7f9f59a2bb vulkan: wait for acquire before transfer writes
Include the transfer stage in the swapchain acquire wait mask because the present command buffer clears and blits the acquired image with transfer operations.

Co-authored-by: Martin Penkava <mpenkava1337@gmail.com>

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 15:54:12 -07:00
Matthew Stanley c25931e3b7 vulkan: align packed upload offsets
Pad each packed R16 upload region to an even texel count so subsequent VkBufferImageCopy offsets retain the required four-byte alignment.

Co-authored-by: Martin Penkava <mpenkava1337@gmail.com>

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 15:54:07 -07:00
Matthew Stanley 8f78ddbd89 integrate interpreter hotpath and AOT sharding 2026-07-20 15:58:43 -07:00
Matthew Stanley ae3acc7db6 aot: deduplicate complete shard pairs safely 2026-07-20 14:13:46 -07:00
Matthew Stanley c2f4c8bf4b aot: close cross-variant coverage deterministically 2026-07-20 14:06:26 -07:00
Matthew Stanley fd0795cd65 aot: batch supplemental exact-root shards 2026-07-20 14:06:25 -07:00
Matthew Stanley 09d5818efd aot: recover play-free continuation fragments 2026-07-20 14:06:25 -07:00
Matthew Stanley 5ded80e61e Merge master (AOT overlay generation) into perf/interpreter-hotpath
Integration branch: the interpreter-perf series (interpreter fallback +
additive shard autocompile + always-on production observability + the
fail-closed alias-body emitter fix) rebased-by-merge onto master's AOT
overlay generation (play-free extraction, bounded indirect discovery,
AOT recall, shadow-diff redesign).

Semantic-union highlights:
- gte.cpp: HEAD's direct-field marshaling fast paths kept; master's
  s_gte_replay_sandbox guards grafted onto every affected site.
- psx_icache.c: HEAD's global tag array + fetch_miss split kept;
  master's transactional shadow record/replay retargeted onto it.
- overlay_capture.c: HEAD's commit/retry/wait-pending pipeline kept;
  master's durable history + autocapture APIs wired into it
  (persist_history_snapshot fed from the immutable .d contribution).
- overlay_loader.c: master's interp-record/native-replay shadow-diff
  redesign kept as base with HEAD's scheduler-bail hardening grafted
  (new SHDIFF_SCHED divergence kind); strict manifest v2 parsing keeps
  master's P pair-id lines; always-on native-call ring preserved.
- main.cpp shutdown calls BOTH overlay_autocapture_shutdown() and
  overlay_capture_wait_pending() (the latter drains the preserve-write
  FIFO the former does not).
- recompiler: master's generate_alias_group refactor retains the
  fail-closed alias-body re-entry defaults from c85f1d1.

Validated on this tree: recompiler + runtime + tests build; all six
runtime unit tests PASS; interpreter perf guards PASS; recompiler
patch/structural tests PASS (l1 decoder oracle noise = 376 pre-existing
on both parents).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 00:43:41 -07:00
Matthew Stanley ca1888a2f8 docs: FAITHFUL_TIMING_PLAN log entry for the alias re-entry root cause
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 00:07:34 -07:00
Matthew Stanley 5a4b74f5d3 fix: report AOT recall with exact semantics 2026-07-19 17:39:55 -07:00
Matthew Stanley 205438eb98 tools: clean TinyCC staging sidecars 2026-07-18 11:15:13 -07:00
Matthew Stanley 4af125cf07 tools: align TinyCC overlay build contract 2026-07-18 11:08:16 -07:00
Matthew Stanley dbe14dfadb aot: recover safe frameless and interior overlay coverage 2026-07-18 10:13:18 -07:00
Matthew Stanley 6af5437058 tools: publish overlay shards atomically 2026-07-18 09:05:41 -07:00
Matthew Stanley ae1f1e7f19 tools: report true AOT coverage gaps 2026-07-18 08:59:26 -07:00
Matthew Stanley 32cc2e81f4 aot: bound header dispatch fallback discovery 2026-07-18 08:27:09 -07:00
Matthew Stanley 5e8cc5b8e5 aot: enrich archive discovery with safe fallback 2026-07-18 07:49:29 -07:00
Matthew Stanley 41f498fd09 aot: recover linked archives and static aliases 2026-07-18 07:25:49 -07:00
Matthew Stanley 8f90df9b43 aot: preload exact-hash BIOS resident shards 2026-07-18 05:31:03 -07:00
Matthew Stanley baf8b5928c aot: recover cross-boundary branch roots 2026-07-18 04:40:07 -07:00
Matthew Stanley 4fb144a4fd aot: preserve additive static discovery 2026-07-18 04:16:51 -07:00
Matthew Stanley 65d89a06d7 runtime: compact persisted capture addenda 2026-07-18 02:56:53 -07:00
Matthew Stanley f53569dfea aot: account for native BIOS coverage 2026-07-18 01:32:49 -07:00
Matthew Stanley 6c67f81a33 docs: record Tomba 2 static attract recovery 2026-07-18 01:21:05 -07:00
Matthew Stanley 88e471274f aot: separate code-range recall from entry parity 2026-07-18 01:18:48 -07:00
Matthew Stanley bc24934048 aot: recover raw and adjacent overlay producers 2026-07-18 00:38:26 -07:00
Matthew Stanley a544809edf aot: persist overlay capture history atomically 2026-07-17 23:56:58 -07:00
Matthew Stanley 2834f0468d aot: recover call-sparse overlay bases 2026-07-17 23:29:38 -07:00
Matthew Stanley 43cd37624a aot: discover frameless static call targets 2026-07-17 22:58:09 -07:00
Matthew Stanley 414314d412 aot_overlay_spike: robust header-table base recovery via jal self-consistency
The old delta-sweep matched export-table pointers against 0x27BD prologues,
but those pointers point at mid-function DISPATCH entries, not prologues
(measured: 0/29 hit a prologue at the true base). It scattered bases by
2-16KB and decoded data as code.

Fix: j/jal targets are BASE-INDEPENDENT (0x80000000 | (imm26<<2), since all
game code lives in 0x800xxxxx). At the true base, the maximum number of
intra-overlay jal targets land on a real prologue. recover_base() votes every
(jal-target, prologue-offset) pair for base t-o within the pointer-containment
window; the peak is unique and sharp (~2x margin). Records are emitted
page-aligned with a FILL prefix, because the runtime keys shards by a
page-aligned region_start and compile_overlays takes load_addr verbatim.
Weak-signal files are skipped (score<4 or <1.5x runner-up) -- safe coverage
loss under the per-function code_crc dispatch guard. Also dedup overlays by
content CRC (position-fixed BINs are reused across AREA folders).

Validation: generic output is byte-identical (base, region envelope, seed set)
to the live-proven tomba1_extract.py hand tool for ALL 22 Tomba 1 overlays,
with no hardcoded constants. Tomba 2's 21 A*.BIN overlays converge on a single
consistent base 0x80108F9C (region 0x80108000 +3996).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 20:19:03 -07:00
Matthew Stanley 855f253dec overlay: ABI v14 gte_precision_store_word shim + splitgen gating; AOT spike
Two provably-correct framework fixes (independent of the AOT work) plus the
play-free overlay-extraction spike tooling and design doc.

Framework fixes (accurate, safe, complete):
- ABI v14: add gte_precision_store_word to OverlayCallbacks + preamble shim +
  runtime population. The emitter emits a direct call for every swc2 (GTE
  store-word); it is stateful runtime code (gte.cpp) so overlay DLLs must
  forward or fail to LINK (undefined reference). Play-captured shards rarely
  compiled a swc2; fuller static coverage surfaced the gap. Same pattern as
  the v13 ws_player_x_bound / v10 gte_read_ctrl shims.
- Gate splitgen OFF in overlay mode (main_psx.cpp): the current recompiler's
  unconditional split-TU output deletes <stem>_full.c and emits only shards,
  which broke ALL overlay compilation (compile_overlays.py reads _full.c).
  Overlays are single small TUs with no parallel-compile benefit; emit the
  monolith. Restores overlay compile on the current tree.

AOT spike (docs/AOT_OVERLAY_PLAN.md + tools/aot_overlay_spike/, WIP):
- Play-free static overlay extraction from the disc image. Validated:
  Tomba 1 reproduces 90% of played functions + ~1964 unplayed, live-verified
  in-game (Dwarf Village runs native, zero runtime compile). Tomba 2 yields
  661 functions byte-identical to the shipped/played vault (provably real),
  all from the self-describing MAIN.EXE producer; the header-table BIN
  producer needs loader-table RE (base recovery unreliable). Generality is
  producer-dependent. Spike scripts are Tomba-specific proofs of concept
  with hardcoded paths, not a finished tool.
- Correctness is guaranteed by the existing per-function code_crc dispatch
  guard: a mis-positioned shard fails audit at compile or never fires at
  runtime -> coverage loss, never wrong execution. Autocompile self-heals gaps.

CROSS-CUTTING: the ABI v14 change is part of the overlay codegen-hash source
set, so it ROLLS the cg tag (1f2ad026 -> e8484213). On master + a fw bump,
every title's overlay cache invalidates and needs regen+reshard. Branch only;
NOT pushed; other games' caches untouched. Needs user validation before merge.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 18:48:30 -07:00
Matthew Stanley 3ebddf408b docs: record MMX5 and MMX6 load smoke passes 2026-07-14 23:59:50 -07:00