Commit Graph

201 Commits

Author SHA1 Message Date
Matthew Stanley 67757efac4 feat(runtime): add CHD discs and mod-owned loading modes 2026-08-03 22:32:17 -07:00
Matthew Stanley 5959ef2fdd feat(tools): shared macOS release packager for game repos
tools/package_release.ps1 packages this framework's BIOS-only runtime, and
each game repo carries its own Windows packager. There was no macOS
equivalent at either level, so the first macOS release (Crash Bandicoot) was
assembled by hand and TombaRecomp already carries a private copy of most of
this logic. Add the generic one here so titles configure rather than
duplicate it.

Takes an already-built runtime plus the per-game identity and produces a
signed, relocatable .app in a staged release folder, archived as
<package>-<version>-macos-<arch>.zip with a .sha256 sidecar — the naming the
game repos' Windows releases already use. It does not build; the game's
build script builds, then calls this.

Release gates, porting the intent of the Windows packager to macOS:
- self-containment via otool (only OS libraries or bundled Frameworks)
- no baked absolute BIOS path, and no build-machine home path. The Windows
  gate keys on a drive letter; the macOS tell is a path rooted under
  /Users, /Volumes and friends. Matching bare "/...SCPH*.BIN" instead would
  flag both the correct relative default "bios/SCPH1001.BIN" and the usage
  string "/path/to/SCPH1001.BIN" in every binary.
- the 512 KiB OpenBIOS image and its MIT notice are present
- every --require-mod package is present, so a title cannot ship a launcher
  whose Mods page silently vanished
- no disc / retail BIOS / memory-card / generated / local-state files
- saves/ ships empty
- the .app carries a valid signature

The mod catalog is taken from the repo's reviewed source, never the build
tree: a developer build dir accumulates mods/state.toml with enhancements
switched on, which would silently flip a title's default presentation for
every player. Packaging from a build tree without --mods-src is refused.

Bundle layout: staged data lives in Contents/Resources with symlinks from
Contents/MacOS. The runtime resolves bios/, assets/ and mods/ beside the
real executable, but codesign treats anything non-executable under
Contents/MacOS as nested code and rejects the whole bundle —
mods/packages/<pkg>/<version>/ parses as a versioned bundle. codesign seals
a symlink without descending, so this signs cleanly and path resolution is
unchanged. Signing is deliberately not --deep for the same reason.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 18:19:34 -07:00
Alex Vanderveen 3da61df355 Revert "ok"
This reverts commit b836f2198c.
2026-07-29 03:18:10 -04:00
Alex Vanderveen b836f2198c ok 2026-07-29 03:15:42 -04:00
Matthew Stanley 60cb60eee6 fix: repair CLI BIOS and launcher builds 2026-07-28 22:46:55 -07:00
Matthew Stanley 106e31d6e9 Make SDL3 the default runtime backend 2026-07-28 09:22:28 -07:00
Matthew Stanley 0c4dd2db85 docs(tcp): generate the command index; fix six wrong entries
TCP_COMMANDS.md instructed contributors to "update this file" on every new
handler. It described 47 commands while the two debug servers registered 292
between them (279 native + 61 Beetle, overlapping) -- 15% coverage. A
hand-maintained inventory that size does not stay correct, and a stale
reference is worse than none because people trust it.

New tools/gen_tcp_commands.py parses the s_commands[] (native) and CMDS[]
(Beetle) tables and regenerates a complete index between AUTOGEN markers,
marking which commands the curated prose already covers. The hand-written
"Command inventory" table is untouched -- it carries parameters and context
worth keeping. `--check` exits non-zero when the block drifts, so this can gate
CI; verified it fails on a simulated added handler and passes once regenerated.

Corrections to the curated table, each verified against the code first:

  write_ram       params are `addr`,`val` -- not `addr`,`hex` -- and it writes
                  exactly one byte per call (handle_write_ram)
  screenshot      writes a PNG (psx_screenshot.png), not a 24-bit BMP
                  (debug_server.c:7666,7856)
  wtrace_range    ring holds 262144 entries (WRITE_TRACE_CAP, 1<<18), not 1024
                  -- the documented figure was 256x too small
  sljit_async     row removed; the command is not registered anywhere (the
                  sljit tier was removed 2026-07-15)
  pause/continue/step/run_to_frame
                  documented as working. All four are registered but
                  unconditionally return an error (debug_server.c:7324-7342);
                  they were deliberately removed in favour of ring-buffer
                  queries. Marked REMOVED with the replacement each error
                  message names.

Also corrected the header: it presented DuckStation (port 4371) as one of the
two live servers, though DuckStation was retired as the oracle on 2026-05-05
and has no .gitmodules entry, while the page's own CLI examples already used
Beetle on 4380. The live pair is now Native 4370 / Beetle 4380, with a note
that the inventory's "D" column is historical, and the "add a handler" checklist
now points at beetle_debug_server.c and the generator instead of DuckStation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 11:49:31 -07:00
Matthew Stanley fa408aae50 Merge trusted mod plugin support 2026-07-26 21:05:03 -07:00
Matthew Stanley 45dd4210cf Remove Tomba legacy hybrid controller path 2026-07-26 20:52:12 -07:00
Matthew Stanley 8b77d8a730 fix: make overlay autocompile cache-safe 2026-07-26 09:39:24 -07:00
Matthew Stanley 436fe14101 widescreen: support guarded object participation overrides 2026-07-25 23:24:41 -07:00
Matthew Stanley 316eaa44a7 feat: add versioned runtime mod packages 2026-07-25 21:14:27 -07:00
Matthew Stanley 929b2d17f0 build: stage only redistributable BIOS files 2026-07-25 18:19:19 -07:00
Matthew Stanley 4bf868da4f feat(bios): ship OpenBIOS as a runtime dependency 2026-07-25 18:11:57 -07:00
Matthew Stanley 986d8be94c Merge remote-tracking branch 'origin/master' into integrate/openbios-bundle
# Conflicts:
#	recompiler/src/full_function_emitter.cpp
#	recompiler/src/full_function_emitter.h
#	recompiler/src/main_bios.cpp
#	runtime/runtime.cmake
#	runtime/src/main.cpp
2026-07-25 12:31:05 -07:00
Henrique Guedes 6749b826d4 feat: OpenBIOS support — profile-driven BIOS recompilation
Adds PCSX-Redux's MIT-licensed OpenBIOS as a second BIOS the framework can
statically recompile, next to retail SCPH1001. Because OpenBIOS is
redistributable, a game verified against it ships its BIOS with the build —
players supply only a disc, and the launcher BIOS picker disappears.

The mechanism is general, not OpenBIOS-specific: every BIOS image gets a
bios/<STEM>.toml profile that is the single source of truth for its identity
and layout.

Recompiler
- bios/<STEM>.toml profiles: [program.image] (sha256 pin, redistributable
  flag), [recompiler.address_model] (boot-time ROM->RAM copy windows,
  kernel-bless window), install_slots, and [recompiler.runtime_exports]
  HLE anchors (0 = feature structurally unavailable on that BIOS).
- BiosAddressModel (new): validated model built from the profile, replacing
  ~9 hardcoded SCPH1001 window constants that had drifted apart across the
  emitter / discovery / game codegen. The emitted C normalize() derives
  from the same table. from_config rejects overlapping/misaligned windows
  and orderings where sequential folding != first-match. A CTest-registered
  equivalence test sweeps every RAM/ROM address against verbatim copies of
  the old helpers.
- Orphaned delay slots (the OpenBIOS-critical emitter fixes): jr/jalr
  slots are inlined before the jump resolves — previously their side
  effect was silently LOST (e.g. a case-return fragment's addiu $sp pop,
  which unwound a shifted frame and walked the exception handler's IntRP
  cursor into a wild dispatch). Orphaned branches resume at ds+4 on
  not-taken instead of re-executing the slot (a read-modify-write slot
  doubling twice broke OpenBIOS readPad). Jump targets latch pre-slot.
- MIPS-I load-delay modeling for dependent load pairs: the successor sees
  the pre-load value (OpenBIOS's cardfasttrack.s relies on it on purpose —
  "gotta break those bad emulators"). Unmodeled shapes warn at build time.
- Discovery hardening: branch-target dispatchability closure to fixpoint;
  jump tables placed at jr+8 no longer walked as code; cross-function
  continuations routed by an instruction->owner map.
- Signed MULT support (unreachable from the retail seed set; modern-GCC
  OpenBIOS emits it freely), latency-modeled like MULTU.
- Seeds from ELF symbol tables + data-section code-pointer harvest
  (tools/gen_bios_seeds_from_elf.py) — a source-built BIOS needs no
  disassembly pass. 646 OpenBIOS seeds, incl. 5 empirical dispatch-miss
  seeds.
- Stemmed outputs (<out_stem>_full.c / _dispatch.c) so two BIOSes coexist
  in one out_dir. tools/regen_bios.sh is profile-driven (--config).

Runtime
- psx_bios_image: the generated dispatch now self-describes the BIOS it
  was compiled from (identity: size/CRC32/SHA-256/wordsum; kernel-bless
  window; HLE anchors; bundled flag). Runtime SCPH1001 hardcodes removed.
- Launch check is a hard identity gate (a mismatched image under
  statically recompiled code is a guaranteed wild jump). Bundled builds
  hide the BIOS picker and treat a missing/tampered bundled image as an
  installation defect.
- Faithfulness fixes surfaced by OpenBIOS's tighter timing loops (all
  global, argued in code comments, regression-tested against SCPH1001):
  - SIO pad RX is cycle-paced like the memcard path (RX_RDY at
    shift-complete, ACK +170); total ack-IRQ latency unchanged.
    PSX_SIO_PAD_SYNC_RX=1 restores the legacy synchronous path for A/B.
  - SPUSTAT derived from SPUCNT instead of permanently-asserted busy.
  - GPUSTAT.13 (interlace field) alternates per vblank while interlaced,
    pinned 0 in progressive.
  - CD Init(0x0A) completes in ~3.9 ms instead of borrowing Beetle's
    34 ms Reset figure (OpenBIOS's short bounded retry loop livelocked).
  - Exception EPC guard accepts ROM-window block leaders, not just RAM.
- New TCP debug commands (documented in TCP_COMMANDS.md): bios_info,
  s3_smear_watch, callret_watch; write-trace ring gains $s0-$s5.

bios/openbios.bin is vendored (MIT — notice in bios/OpenBIOS.LICENSE,
attribution in THIRD_PARTY_ATTRIBUTION.md, source pin + build recipe in
bios/OpenBIOS.toml). The emitter fingerprint digest is now content-only:
existing generated/*.emitter.sha will read STALE once; re-run
tools/regen_bios.sh.
2026-07-23 17:10:23 -03:00
Matthew Stanley f86087ad94 Remove stale RmlUi launcher remnants 2026-07-23 00:29:22 -07:00
Alex Vanderveen 0aa50e514f Update bios_emitter_fingerprint.sh 2026-07-21 21:38:05 -07:00
Alex Vanderveen 4eb1794abb Update regen_bios.sh 2026-07-21 21:38:05 -07:00
Matthew Stanley e03c743e87 tools: add Linux macOS dev setup helper
Add a conservative source-checkout setup helper based on the Linux quick-start intent from PR #54.

The script builds the CLI/recompiler tools, refreshes BIOS C when a BIOS is present, and builds the BIOS-only runtime only when BIOS/generated sources are available. It deliberately leaves game project creation to the existing CLI generator.

Co-authored-by: Keeganator <keegan.pan@gmail.com>
2026-07-21 17:02:05 -07:00
Matthew Stanley 9228f32c00 runtime: keep Vulkan opt-in and remove old launcher
Remove the in-tree launcher implementation and its submodules now that game projects use recomp-ui for launcher support.

Keep Vulkan hidden from the launcher unless a game explicitly sets [video] offer_vulkan = true, and clamp stale Vulkan settings back to OpenGL when it is not offered.

Co-authored-by: Martin Penkava <mpenkava1337@gmail.com>
2026-07-21 16:31:36 -07:00
Matthew Stanley 8f78ddbd89 integrate interpreter hotpath and AOT sharding 2026-07-20 15:58:43 -07:00
Matthew Stanley ae3acc7db6 aot: deduplicate complete shard pairs safely 2026-07-20 14:13:46 -07:00
Matthew Stanley a59cf8a88d aot: make saturated caches a deterministic fixed point 2026-07-20 14:06:26 -07:00
Matthew Stanley c2f4c8bf4b aot: close cross-variant coverage deterministically 2026-07-20 14:06:26 -07:00
Matthew Stanley 71257f6424 aot: persist exact candidate coverage gaps 2026-07-20 14:06:26 -07:00
Matthew Stanley fd0795cd65 aot: batch supplemental exact-root shards 2026-07-20 14:06:25 -07:00
Matthew Stanley 09d5818efd aot: recover play-free continuation fragments 2026-07-20 14:06:25 -07:00
Matthew Stanley 5ded80e61e Merge master (AOT overlay generation) into perf/interpreter-hotpath
Integration branch: the interpreter-perf series (interpreter fallback +
additive shard autocompile + always-on production observability + the
fail-closed alias-body emitter fix) rebased-by-merge onto master's AOT
overlay generation (play-free extraction, bounded indirect discovery,
AOT recall, shadow-diff redesign).

Semantic-union highlights:
- gte.cpp: HEAD's direct-field marshaling fast paths kept; master's
  s_gte_replay_sandbox guards grafted onto every affected site.
- psx_icache.c: HEAD's global tag array + fetch_miss split kept;
  master's transactional shadow record/replay retargeted onto it.
- overlay_capture.c: HEAD's commit/retry/wait-pending pipeline kept;
  master's durable history + autocapture APIs wired into it
  (persist_history_snapshot fed from the immutable .d contribution).
- overlay_loader.c: master's interp-record/native-replay shadow-diff
  redesign kept as base with HEAD's scheduler-bail hardening grafted
  (new SHDIFF_SCHED divergence kind); strict manifest v2 parsing keeps
  master's P pair-id lines; always-on native-call ring preserved.
- main.cpp shutdown calls BOTH overlay_autocapture_shutdown() and
  overlay_capture_wait_pending() (the latter drains the preserve-write
  FIFO the former does not).
- recompiler: master's generate_alias_group refactor retains the
  fail-closed alias-body re-entry defaults from c85f1d1.

Validated on this tree: recompiler + runtime + tests build; all six
runtime unit tests PASS; interpreter perf guards PASS; recompiler
patch/structural tests PASS (l1 decoder oracle noise = 376 pre-existing
on both parents).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 00:43:41 -07:00
Matthew Stanley 9446038dbb autocompile: survive quoted interpreter paths + pin Windows Python in launcher
Two independent failures made every live reshard fail SILENTLY (runs
counted, shard_ok stayed 0, the game just never got shards):

- autocompile.c: the child line is executed via 'cmd.exe /C <line>'.
  When the line BEGINS with a quoted path and contains further quotes,
  cmd.exe strips the first and last quote characters and mangles the
  command ('The filename, directory name, or volume label syntax is
  incorrect'). Wrap the whole line in one extra pair of quotes -- cmd
  then strips exactly those and executes the inner command verbatim.
- launch_tomba2_interp_perf.ps1: the autocompile command used an
  unqualified 'python'; with an MSYS python first on the inherited
  PATH, it cannot open the Windows-style script path ('can't open
  file'). Use the canonical Windows Python312 explicitly and fail the
  launch loudly if it is absent.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 00:07:33 -07:00
Matthew Stanley 156d9b9b34 Add self-contained media-to-source CLI release 2026-07-20 00:00:22 -07:00
Matthew Stanley b3bfdd9330 Fix in-handler RFE contract gaps that livelock the boot splash (Tomba 2 frame 1385)
Three-layer faithful-core fix for the exception-return contract, root-caused
live over TCP on a deterministic dbg-build repro (vblank authority vetoed
forever while the guest spun in the kernel idle loop, in_exception stuck 1):

1. Interpreter RFE backend parity: dirty-RAM RFE now calls
   psx_rfe_mark_escape() exactly like the recompiled opcode and every
   overlay shard (ABI v12), and dirty_ram_dispatch_inner takes a pending
   escape at the committed transfer before local dirty flow can keep
   interpreting the resumed code inside psx_check_interrupts' window.
2. Legacy-sentinel deliveries arm the escape too: the old blanket decline
   assumed a sentinel delivery always resumes AT the sentinel, but guest
   TCBs never hold the sentinel (Fix B anti-leak), so a kernel
   ReturnFromException restore resumes at an approximate real PC with no
   exit path armed. The take site preserves the legacy contract: a resume
   landing ON the sentinel still exits through the sentinel detector.
3. Same-thread completion of a sentinel delivery via explicit RFE /
   ReturnFromException converts to the legacy exit (restore interrupted
   GPRs, pc=0, continue the live host chain) keyed on the structural
   entry_tcb == exit_tcb test - otherwise the approximate TCB PC surfaces
   up the trampoline, severs the live chain, and the run dies one hop
   later on an un-re-enterable mid-function PC ("execution completed").

Validated: production build passes the previously-fatal splash transition
(frames 4717+ at 60fps, was deterministic death at 1385 x3 runs); dbg build
advances past the wedge (was frozen at exactly 1385); all runtime tests +
structural guards pass (guards extended to pin the interp RFE contract).
Known follow-up: the PSX_DEBUG_TOOLS build still loses one kernel CD event
through a third timing path at the same site - tracked as an open
foundation item; production config unaffected.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 19:00:29 -07:00
Matthew Stanley 5a4b74f5d3 fix: report AOT recall with exact semantics 2026-07-19 17:39:55 -07:00
Matthew Stanley 504db26445 feat: add bounded indirect overlay discovery 2026-07-19 17:15:17 -07:00
Matthew Stanley 76257fcf48 tools: point interp-perf launcher at the isolated game worktree/build
Two changes to finish the interpreter-perf / AOT-investigation split:

1. Build dir switches from runtime/build-tomba2-perf-rel (configured
   against the SHARED Tomba2Recomp checkout, CMAKE_HOME_DIRECTORY=
   F:/Projects/psxrecomp/Tomba2Recomp) to the new isolated
   runtime/build-tomba2-interp-iso-rel, configured against
   _wt-tomba2-interp-perf (branch interp/game-isolated, forked from
   Tomba2Recomp's master). That build passes
   -DPSXRECOMP_GAME_EXE_NAME_OVERRIDE=Tomba2InterpPerf, so it emits
   Tomba2InterpPerf.exe natively. The old build-tomba2-perf-rel is left
   untouched on disk as evidence of the original measurement; only this
   script's target moves.

2. Drops the Copy-Item alias hack: since the isolated build already
   produces the isolated exe name directly, there is nothing left to
   alias.

3. GameRoot now defaults to the isolated game worktree instead of the
   shared Tomba2Recomp checkout, so this script no longer reads
   game.toml from a tree whose HEAD (feat/aot-overlay-spike) belongs to
   a different investigation and has already drifted from master
   (overlay_capture_history).

4. The pre-launch guard now refuses to start if EITHER Tomba2InterpPerf
   OR Tomba2Recomp is already running (previously only checked its own
   process name) — the two builds can no longer collide on disk, but
   still share the BIOS file, debug-port range, and taskkill-by-name
   cleanup surface.

State paths (memcards/captures/logs/overlay cache) were already scoped
under _wt-interpreter-perf and are unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 15:54:48 -07:00
Matthew Stanley 02f29b979d aot: harden exact interior fragment recovery 2026-07-19 14:29:57 -07:00
Matthew Stanley 05f6ef3e02 Optimize GTE fallback and isolate interpreter perf runs 2026-07-19 14:23:36 -07:00
Matthew Stanley 7376e6b042 Harden additive overlay shards and speed interpreter fetch 2026-07-19 12:11:56 -07:00
Matthew Stanley 91c9aea91f aot: make discovery and shard publication fail closed 2026-07-18 14:08:45 -07:00
Matthew Stanley aac755b0ae tools: accelerate cumulative coverage scoring 2026-07-18 13:14:57 -07:00
Matthew Stanley 205438eb98 tools: clean TinyCC staging sidecars 2026-07-18 11:15:13 -07:00
Matthew Stanley 4af125cf07 tools: align TinyCC overlay build contract 2026-07-18 11:08:16 -07:00
Matthew Stanley dbe14dfadb aot: recover safe frameless and interior overlay coverage 2026-07-18 10:13:18 -07:00
Matthew Stanley 6af5437058 tools: publish overlay shards atomically 2026-07-18 09:05:41 -07:00
Matthew Stanley ae1f1e7f19 tools: report true AOT coverage gaps 2026-07-18 08:59:26 -07:00
Matthew Stanley 3949854981 Optimize interpreter fallback and additive overlay autocompile 2026-07-18 08:32:42 -07:00
Matthew Stanley 32cc2e81f4 aot: bound header dispatch fallback discovery 2026-07-18 08:27:09 -07:00
Matthew Stanley 5e8cc5b8e5 aot: enrich archive discovery with safe fallback 2026-07-18 07:49:29 -07:00
Matthew Stanley 41f498fd09 aot: recover linked archives and static aliases 2026-07-18 07:25:49 -07:00
Matthew Stanley 731f43235e tools: make large disc reads linear 2026-07-18 06:30:37 -07:00