Commit Graph

510 Commits

Author SHA1 Message Date
Matthew Stanley 6534b3ddad Merge latest master for segment design; preserve landed IsC and segment notes 2026-09-30 15:02:17 -07:00
Matt Stanley fcc685ce1d Merge pull request #435 from RetroPortingToolKit/fix/isc-icache-invalidate
Apply cache-isolated (IsC) stores to the caches, as Beetle does
2026-09-30 15:00:19 -07:00
Alex Vanderveen 9a5c6e0eab docs: BIOS selection in bundled builds, and the planned player-side backend build
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-30 17:55:18 -04:00
Alex Vanderveen 874df9b5bb release: ship bundled, compiled games from committed generated/ C
Retire the setup-host release model (CI wiped generated/ and shipped
emitters + sources + a wizard so every player regenerated and rebuilt the
title). Titles now commit generated/ and CI ships the compiled game.

- docs/ci/templates/game-release.yml replaces setup-release.yml: gate on
  committed game C (tools/ci/check_generated.sh), emit the OpenBIOS backend
  in CI (tools/ci/generate_openbios.sh), configure with
  PSXRECOMP_REQUIRE_GAME_C=ON and assert the full path was taken, package
  with tools/package_game_release.sh, verify the zip is a game not a kit.
- tools/package_game_release.sh: exe, assets, bundled OpenBIOS, configs, mod
  catalog + overlay toolchain via release_stage.py, notices; refuses sources,
  emitters, generated C, BIOS dumps. clear_generated.sh removed;
  package_setup_host.sh marked retired.
- Scaffold: gitignore templates track generated/, package_release.sh.in
  wrapper, setup_project.{sh,ps1} commit generated/ after Generate.
- Project Studio: audit flags ignored/untracked generated/, retail BIOS C
  and setup-host workflows; merge_gitignore un-ignores generated/; ops emit
  the bundled packager + workflow; generate_ci uses the new template.
- Docs: docs/ci/BUNDLED_RELEASES.md replaces HOST_ONLY_RELEASES.md.
- Tests: tools/tests/test_bundled_release_layout.py; fix pre-existing
  false positives in the packager-surface and overlay-static tests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-30 17:11:14 -04:00
Alexandros Mandravillis 5c7a37146c netplay: say why a match ended, and end a boot mismatch in 3 s
A netplay match that ends early returns both players to the lobby. The
runtime knew why (netplay_soft_exit's origin) but only printed it to
stdout, so players saw the lobby again with no reason. A boot-digest
mismatch (peers booted different BIOS images or boot settings) first
held a black window for the full 20 s admit-stall watchdog.

netplay_soft_exit now records a sentence for its origin, and the soft
return to the lobby puts it on the launcher's status line through
last_error (the room and browser views already show it). The dig0 gate
timestamps a mismatch; once the same pair of digests has held for 3 s,
the watchdog ends the match with its own reason. A peer that gets the
other side's BYE while it sees the same mismatch reports the mismatch,
not "the other player left". A CLI match prints the sentence to stderr.

The origin-to-text map and the grace decision live in
netplay_exit_reason.c, covered by netplay_exit_reason_test. Builds on
the previous commit's psx_lobby_set_last_error.
2026-09-30 15:59:58 +03:00
Alexandros Mandravillis 5fbd11921a netplay: settle the lobby BIOS by image, not by "any retail"
The lobby's BIOS identity was a two-value token, openbios or scph1001,
and scph1001 meant "some retail dump". An SCPH-5552 peer and an
SCPH-1001 peer both offered it, the host settled it, each peer booted
its own image, and the rollback boot digest never matched: about 20 s
of black window, then both players back in the lobby with no reason
(reported in #243).

Each offer now carries the CRC-32 of the retail image that peer would
boot (online bios_offer.retail_crc; LAN JOIN line 6). The host settles
retail only when every seat offers the same CRC, else OpenBIOS when
every seat links it, else refuses to start and puts the reason on the
launcher's status line ("Players use different BIOS images (SCPH-5552
and SCPH-1001) ..."). The settled CRC ships in match_caps
(session_bios_crc) and on MOTK1 START line 7, and each peer boots only a
dump with that CRC. Rooms where every peer has the same image keep
working, including SCPH-5552-only builds with no OpenBIOS. Old peers
send no CRC, so a mixed old/new room falls back to OpenBIOS or refuses;
a session from an old host still accepts any retail dump. The lobby
server relays bios_offer and match_caps as opaque objects, so it needs
no change.

The rule lives in netplay_bios_settle.c, shared by the online and LAN
paths and covered by netplay_bios_settle_test. last_error grows to fit
one sentence, with psx_lobby_set_last_error to set it.
2026-09-30 15:58:52 +03:00
Shokunin 4630a0d9f7 docs: record PR A (#429) as landed in the segment-aware design
#429 charges psx_icache_fetch before every instruction whose runtime PC is
uncached, in both emitters, and makes the BIOS emitter's cache-line start
test use the runtime PC. The OpenBIOS check finds no uncharged KSEG1 site,
so segment_aware_codegen passes with 10 known gaps; bios-kseg1-fetch-charge
has not been in KNOWN_GAPS since the ledger's first commit in this series.
The check stays as a regression guard: an uncharged site reports the id as
a new gap.

docs/SEGMENT_AWARE_CODE.md:
- §5.6 describes the charge as landed, including the runtime-PC line-start
  fix. The game emitter already has the predicate, but emit_pre_icache
  tests the compile address: PR B must route both its psx_fetch_uncached()
  argument and the fetch tag through runtime_pc(), and D adds the KSEG1
  variants they reach.
- §7.1 lists ten open ids and records how A's id left the ledger.
- §8 marks A merged, with its Beetle parity (shell entry -126 on OpenBIOS,
  -454 on SCPH-1001; the residuals are the IsC gap and SCPH-1001's KSEG1
  kernel entry). D now only needs the KSEG1 variants; kseg1-fetch-charge
  stays open until D. The macOS oracle recipe is docs/beetle-macos.md (#431).
- §1, §3.5 and the header say the BIOS undercharge is fixed. §10 is
  unchanged.
- §3.1, §3.5 and §5.2 cite the emitter lines as they are after #429
  (code_generator.cpp +6, full_function_emitter.cpp +10).

FAITHFUL_TIMING_PLAN §5 gets a log entry for the rebase.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:20:52 -07:00
Shokunin 591cf8ff73 docs: segment-aware code after the #417/#418/#420 merges and the first Beetle runs
- Rebased onto master: #417 (the old base), #418 and #420 merged on
  2026-09-29. The status line says so, and every cited source line is
  refreshed for current master.
- PR A is #429. §7.2 records its passed LLE land gate against live Beetle
  (shell entry -126 OpenBIOS, -454 SCPH-1001; ruler #1 64/64, ruler #2
  14/14). The residue is the IsC gap (§9, now with per-flush numbers and the
  prototype result) and SCPH-1001's KSEG1 kernel entry 0xA0000500 (§3.3, -9),
  which PR D closes (§8).
- §7.2 records the synthetic EXE's first Beetle runs (macOS oracle recipe:
  #431). Disc boot works on both BIOSes; sideloading does not, because
  Beetle's loader forces a KSEG0 start. With the probe fixes, Beetle reads
  56/56/82 cycles for KUSEG/KSEG0/KSEG1, a 26-cycle surcharge equal to the
  fetch model's.
- Ledger values follow the new probe layout (probe_run 0x000100F0; 30 fetch
  tags; leaf 22 vs Beetle 11). Still 10 ids.
- ACCURACY_BURNDOWN points the KSEG1 fetch item at #429; FAITHFUL_TIMING_PLAN
  gets a dated log entry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:20:52 -07:00
Shokunin 3afa4a0b41 docs: add the GP0 source key to the store-PC re-key list; PR C closes eight ids
memory.c:1292 also reads g_debug_last_store_pc in every build: a GP0 write
from 0xBFC38B1C gets a RAM source key for the widescreen prim matching. That
key lies in SCPH1001's Shell window (runtime 0x80050B1C), so it has to be
re-keyed with the other six when the BIOS stamp moves to runtime_pc().
List it in §3.1, §5.2, §9 and the FAITHFUL_TIMING_PLAN entry, and give each
relocated key its runtime PC.

§8: PR C closes eight ledger ids (§5.3's seven plus segment-miss), so
C and D close all ten; A (#429) closed the eleventh gap,
bios-kseg1-fetch-charge, on master. B's store-PC hole ships with the re-key of
all seven keys.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:20:52 -07:00
Shokunin d51a34a766 docs: record the segment-aware decisions and the store-PC correction
The owner settled §10's four questions on 2026-09-29, all as recommended:
- I-cache tags follow Beetle; the hardware bit-31 difference is recorded
  in ACCURACY_BURNDOWN axis 4.
- Segment misses in static game code interpret loudly until the title is
  regenerated.
- Extra segments are requested through segment-qualified seeds.
- The overlay cache uses per-segment subdirectories.

The doc called g_debug_last_store_pc a debug/identity hook. It is not:
memory.c's RAM 0x0-0xF store filters compare it with exact PCs in every
build, the interpreter stamps the full PC, and #420 (ABI v24) forwards
overlay stamps to the host. §3.1 now lists it as a baked PC, and §5.2
routes it through runtime_pc(). Consequences:
- Option A, re-measured with the 1,261 store-PC stamps in the three R4
  shards: +3.8 % / +27.6 % __TEXT (was +3.2 % / +27.2 %).
- New ledger id store-pc-segment (10 KSEG0 stamps in the synthetic EXE's
  home bodies), closed by §5.3. The ledger now has 10 known gaps.
- §9: several filter keys are ROM addresses of BIOS code that runs
  relocated in SCPH1001. The BIOS emitter stamps ROM addresses and the
  interpreter stamps runtime PCs, so the keys must be re-keyed when the
  BIOS stamp moves to runtime_pc().

§8 notes that rollout PR A (fix/uncached-fetch-per-insn, based on master)
is being prepared. A has since landed as #429, so the ledger does not
list bios-kseg1-fetch-charge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:20:52 -07:00
Shokunin 9f8ee72f4d docs: design segment-aware code, with a KUSEG-linked acceptance ledger
Compiled code bakes KSEG0 into link values, CPS/IRQ resume PCs and I-cache
tags. KUSEG-linked EXEs run KUSEG PCs through those bodies (and their KUSEG
seeds are dropped), and overlay code executed at KUSEG/KSEG1 stays
interpreted after #417. Until #429, both emitters also charged the uncached
KSEG1 fetch only at cache-line leaders, so compiled BIOS ROM code was 4
cycles short on 5,477 of OpenBIOS's 9,592 KSEG1 instruction sites (Beetle
and the interp charge every fetch).

docs/SEGMENT_AWARE_CODE.md recommends per-segment compiled variants (code
identity = full VA, emitter segment as a per-compile constant, exact-segment
dispatch, per-instruction fetch in uncached bodies, segment-aware capture
and cache keys) over segment-relative emission, which measures +3.2 % /
+27.2 % __TEXT on R4 shards.

tools/segment_testrom/gen_segment_exe.py emits a synthetic KUSEG-linked
PS-X EXE that also works as an oracle probe. The new ctest
segment_aware_codegen runs the real recompilers on it and on OpenBIOS and
checks emitted PCs, the emitted dispatch lookup and fetch cycles (through
runtime/src/psx_icache.c) against a Beetle ReadInstruction transcription.
It passes only while the observed gaps equal its ledger (9 today).

#429 (rollout A) fixed the BIOS undercharge on master before this landed,
so KNOWN_GAPS does not list bios-kseg1-fetch-charge: the OpenBIOS check
finds no uncharged KSEG1 site and stays as a regression guard that reports
the id as a new gap. The last commit of this series brings the doc up to
date with #429.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:20:52 -07:00
Shokunin 825a89a9c5 runtime: keep host stores out of IsC, as DMA is
Only CPU stores are cache-isolated. Mods, the FMV skip, debug pokes, the
bg2d ring fill and the exception sentinel write guest memory from the host;
they now go through psx_host_write_*, which bypasses isc_store the way DMA
does, so a host store inside a FlushCache window reaches memory instead of
rewriting an I-cache tag. The render-pass checkpoint restores the new depth
counter like the DMA depth. isc_store_test covers it (a lane-0 host store in
tag-test mode would otherwise mark a cold line valid).

Docs: the SCPH-1001 residual starts at kernel init 0x598, and without #429
the fix removes 113 of the 126 OpenBIOS flush cycles.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 17:55:26 -07:00
Shokunin 6de75122a3 docs: record the Beetle gate for IsC stores and tick the burndown item
FAITHFUL_TIMING_PLAN §5 (2026-09-29): with #429 on master, the LLE boot
now matches Beetle at every anchor to the shell on OpenBIOS (was -126)
and sits at -9 on SCPH-1001 (was -454), the KSEG1 kernel entry PR D
closes. Also measured without #429, the rulers (unchanged), R4's game
entry (+342 default boot, +18 LLE boot, 0 dispatch misses) and the
unchanged codegen hash.

ACCURACY_BURNDOWN axis 4: IsC item ticked; the SWL/SWR corner cases are
a new open item; the P2 scratchpad-under-IsC note points at Beetle's
rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 17:33:35 -07:00
Matt Stanley 0ffab1ceb9 Merge pull request #429 from RetroPortingToolKit/fix/uncached-fetch-per-insn
Charge the uncached (KSEG1) instruction fetch on every instruction
2026-09-29 16:31:20 -07:00
Matt Stanley 9a8737d579 Merge pull request #431 from RetroPortingToolKit/docs/beetle-macos
docs: build the psx-beetle oracle on macOS (plus the missing Beetle hook patches)
2026-09-29 16:31:07 -07:00
Shokunin 1caf4a6897 docs: record the land-gate re-check after the rebase onto 470f03b7
The branch was rebased onto master 470f03b7, which now carries #418, so
the gate was re-checked on that tree without local merges. The
regenerated OpenBIOS and SCPH-1001 C is byte-identical to the gated
build's. Every boot anchor gives the same cycle (native - Beetle at the
shell entry: -126 on OpenBIOS, -454 on SCPH-1001). Ruler #1 matches Beetle
on all 64 passes, and ruler #2 on all 14 components on an HLE and an LLE
boot. uncached_fetch_charge and uncached_fetch_codegen_test pass. The
recompiler suite passes apart from the three environmental failures
master also shows on this Mac.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:36:13 -07:00
Shokunin 035115392b docs: make the land-gate method reproducible from the tree
The land-gate entry said cyc_watch was armed from power-on on both
backends, but did not say how, and cited a local branch for the psx-beetle
build. The arming is PSX_CYC_WATCH / PSX_CYC_WATCH_N on psx-runtime and
psx-beetle (plus PSX_BEETLE_UNPACED=1 for a headless Beetle), documented in
docs/beetle-macos.md step 7 with the psx-beetle build; the entry now cites
both.

It also listed the shell entry as 0x80030000 on both sides. Both BIOS
profiles declare the shell window dispatch_key = "rom", so native keys it
at its ROM copy: the native anchor was 0xBFC0A500 (OpenBIOS) and
0xBFC18000 (SCPH-1001), and a native watch on 0x80030000 records nothing.
The numbers are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:27:58 -07:00
Shokunin 0cf03770d4 docs: record the Beetle land gate for the uncached fetch fix
SEGMENT_AWARE_CODE.md §7.2 gates this change on an LLE boot with per-anchor
cycle parity against psx-beetle to the shell, plus ruler #1. Both ran on
macOS, each twice: with OpenBIOS (SHA-1 95419841b5104d552b14810b1ecbe6c1358bcdf1)
and with the owner's own SCPH-1001 dump (SHA-1
10155d8d6e6e832d6ea66db9bc098321fb5e8ebf), the same image on both sides. Native
master and this branch were built with #418 merged locally, so the mult/div
flush could not mask the result.

At the shell entry, native minus Beetle goes from -2,165,591 to -126 cycles on
OpenBIOS and from -7,049,462 to -454 on SCPH-1001. The ROM boot path is exact
on both. Two known gaps explain what is left:
- IsC stores do not reach the I-cache model, so a FlushCache leaves native
  kernel lines valid that Beetle refills. A local prototype of Beetle's IsC
  tag write takes OpenBIOS to 0 and SCPH-1001 to -9. It is recorded as an
  axis-4 item here, not part of this change.
- SCPH-1001 enters its copied kernel through the KSEG1 alias 0xA0000500. The
  four trampoline words run uncached in Beetle and cached natively (-9). That
  is the KSEG1-alias gap in SEGMENT_AWARE_CODE.md §3.3.

Ruler #1 on SCPH-1001 matches Beetle on all 64 passes, on master and this
branch. Ruler #2 stays exact on all 15 loops. Beetle also confirms the memcpy
cost the branch now charges: 39 cycles per byte during boot, and 42 per byte,
860 and 3,884 cycles per call in R4.

The land gate item is ticked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:27:58 -07:00
Shokunin b3d902f80b docs: record the Beetle land gate and in-game cost of the uncached fetch fix
The 2026-09-29 log entry marked the live-Beetle validation OPEN inside a
done item. SEGMENT_AWARE_CODE.md §7.2 makes it a precondition for landing:
an LLE boot with per-anchor cycle parity against psx-beetle to the shell,
plus ruler #1. ACCURACY_BURNDOWN now carries it as its own unchecked gate
item, and the timing-plan entry says the PR stays draft until it runs on an
oracle machine.

The entry also framed the change as boot-only. OpenBIOS runs its A0/B0/C0
services in place, and 148 of its 192 A0 entries point into ROM, so the
per-instruction charge also applies during gameplay. Measured in an R4
race: memcpy's byte loop goes from 25 to 42 cycles per byte, and R4's two
memcpy calls per game frame go from 2,824 to 4,744 cycles (about 0.17% of
the frame). An LLE boot of R4 reaches the game entry 2,934,910 cycles
(1.10%) later, identical on two boots of each build, with 0 dispatch misses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:27:57 -07:00
Shokunin c981b3a2d7 bios emitter: test cache-line starts on the runtime PC
The cached line-leader rule charges a fetch at every 16-byte line start of
the runtime PC. The BIOS emitter tested bits[3:0] of the ROM address instead
and assumed every copy window preserves them. OpenBIOS copies its kernel from
ROM 0x1FC1E4D4 to RAM 0x500, which shifts them by 4. Each kernel line
crossing was charged one instruction early, where it is a hit, and the real
crossing went uncharged. 1,152 kernel instruction sites differed from the
interpreter.

The test now uses the runtime PC that relocate_ra already computes. In the
OpenBIOS kernel, 823 misplaced fetch sites go and 773 are added at runtime
line starts. Retail profiles are unaffected: SCPH-1001's copy windows are
16-byte aligned and SCPH-101/5552 declare none. The uncached_fetch_charge
ctest now also replays every cached OpenBIOS run from a cold cache against
the interpreter's fetch path and the Beetle transcription.

R4 (OpenBIOS, recompiled kernel, HLE boot) reaches the game entry 6,013
guest cycles later than with the uncached fix alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:27:57 -07:00
Shokunin 7342e6bcd1 emitters: charge an uncached (KSEG1) fetch before every instruction
Beetle's ReadInstruction never fills an I-cache line for a fetch at
0xA0000000 or above: every such fetch costs +4 and clears the pending load
give-back. The dirty-RAM interpreter charges that by fetching at every PC.
Both static emitters emitted psx_icache_fetch only at cache-line leaders,
which is exact for cached code only. In OpenBIOS, 5,477 of the 9,592
in-place ROM (KSEG1) instruction sites were uncharged: 21,908 cycles short
per pass through that code, each also missing its give-back clear.

Both emitters now charge a fetch before every instruction whose runtime PC
is uncached. The predicate is psx_fetch_uncached in psx_instr_cost.h, which
psx_icache.c now uses too. The A0/B0/C0 call-vector native stubs charge one
fetch per executed word, so a KSEG1 call pays like the interpreter. Cached
code keeps the leader rule. R4's regenerated game C is byte-identical, and
the OpenBIOS diff is insertions only. psx_instr_cost.h joins the codegen
hash list, since the emitters fold it into emitted code.

Tests:
- ctest uncached_fetch_charge runs psxrecomp-bios on OpenBIOS and compares
  per-instruction fetch charges three ways: the compiled fetches, the
  interpreter's fetch path (psx_icache_fetch_interp) at every PC, and a
  Beetle transcription.
- uncached_fetch_codegen_test checks the game emitter at KSEG1 and KSEG0.

Both tests fail on master. This closes bios-kseg1-fetch-charge in the
segment_aware_codegen ledger (#419).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:27:57 -07:00
Shokunin 7118734d04 docs: correct the ruler #2 icache_miss value on the 15-loop ROM
The 2026-06-27 MMIO entry in FAITHFUL_TIMING_PLAN.md says the 13 prior
loops were unchanged when mmio_timer and mmio_spu were added. icache_miss
was not: the two new loops sit before it and move its loop top from
0x80010144 to 0x80010170, so each of its two misses costs one cycle more
and it reads +16 on both backends (re-measured against live psx-beetle; the
cycle_testrom README already explains the shift). The entry now carries a
dated correction, and the beetle-macos.md intro no longer points readers
at +14 as the current value.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:26:45 -07:00
Shokunin 216eb31ef2 cyc_watch: arm boot anchors from power-on on both backends
Boot anchors (reset, the kernel copy, the first A0/B0/C0 calls, the shell)
pass within the first frames, before a TCP cyc_watch arm can land, so a
native-vs-Beetle boot comparison could not be run from the tree.

psx-runtime (debug_server.c) and psx-beetle (beetle_main.cpp) now read
PSX_CYC_WATCH ("<pc>" or "<pc>-<end>" for region mode) and PSX_CYC_WATCH_N
(default 16) at start-up and arm the watch before the first guest
instruction. The native arm shares the TCP handler's code path
(cyc_watch_arm); psx-beetle calls beetle_cyc_watch_arm. With neither
variable set nothing changes. tools/cycle_compare.py --no-arm then reads
both rings.

PSX_BEETLE_UNPACED=1 skips psx-beetle's 59.94 Hz wall-clock pacing for
headless oracle runs. Guest timing is unaffected.

docs/beetle-macos.md step 7 documents the variables and the anchor-key
difference for BIOS copy windows declared dispatch_key = "rom": the shell
is keyed natively at its ROM address (OpenBIOS 0xBFC0A500, SCPH-1001
0xBFC18000) while Beetle sees 0x80030000. Checked on macOS: the documented
example records the OpenBIOS shell entry at 5,015,670 in Beetle, the value
of the #429 land gate, and the native watch arms from boot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:26:16 -07:00
Matt Stanley 470f03b782 Merge pull request #430 from RetroPortingToolKit/feat/fp-identity-tool
tools: generic A/B guest-identity check (fp_identity.py)
2026-09-29 11:24:04 -07:00
Matthew Stanley 58bdf987d9 Merge master into feat/internal-resolution: combine with #422/#423/#424
Keep #422's 1x GL_LINES batch (gated s_hr_scale == 1 && !s_hiw) beside the >1x quad batch; FLIP capture uses hiw_capture_size; render passes use s_hr_scale and are refused under s_hiw.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 11:23:28 -07:00
Matthew Stanley 2bf4739f44 Merge master into feat/internal-resolution-settings: regenerate TCP_COMMANDS.md
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 11:22:53 -07:00
Matt Stanley f8a6f4f185 Merge pull request #425 from RetroPortingToolKit/feat/internal-resolution-gl
Raise the GL internal-resolution ceiling to the driver's limits
2026-09-29 11:22:28 -07:00
Matthew Stanley 914d8cb305 Merge master into feat/frame-rate: reset_on_load onto #420's fingerprint struct
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 11:22:02 -07:00
Matt Stanley d5e41d7b7d Merge pull request #423 from RetroPortingToolKit/feat/frame-rate-blend
Add a flip-aware frame-interpolation source (opt-in)
2026-09-29 11:21:11 -07:00
Matt Stanley b2f72d7cce Merge pull request #422 from RetroPortingToolKit/feat/adaptive-widescreen
Add [widescreen.cull] bgez_sites and clip_edge_x_load_sites
2026-09-29 11:20:58 -07:00
Matt Stanley df4e905ee8 Merge pull request #428 from RetroPortingToolKit/fix/rematch-session-start
Run the full session start on the lobby rematch
2026-09-29 11:20:45 -07:00
Matt Stanley 4ea39cf942 Merge pull request #421 from RetroPortingToolKit/fix/mod-session-hygiene
Reset mod-owned presentation state at every session start
2026-09-29 11:20:12 -07:00
Shokunin 055c33aaa1 docs: complete the Beetle oracle patch set; add a macOS recipe
beetle-linux.md told builders to re-land several Beetle hooks by hand
because they were never exported to docs/: the rtrace and IRQ callbacks,
the guest-cycle clock, PS_CDC::PSXRecomp_GetDecodeVolume and the CDC/DMA
IRQ peeks that beetle_libretro.cpp links against, plus the SPU event
calls and the per-instruction hook the real cyc_watch now needs. Ship
them, one patch per hook, all against the 5759277b base:

- beetle_rtrace_irq_hook.patch
- beetle_guest_cycles_hook.patch
- beetle_cdc_dma_peek_hook.patch
- beetle_spu_event_hook.patch

beetle_cdcmd_trace_hook.patch was generated from a tree that already had
the wtrace block, so applying it after the wtrace patch redefined the
wtrace globals and typedefs and needed a manual fix. It is now a diff on
top of the wtrace and SIO patches; the resulting tree is unchanged.

beetle-macos.md is the macOS recipe, verified from a fresh official
clone on Apple Silicon: `git apply` of all seven patches, the osx static
build, the psx-beetle configure on current master, OpenBIOS under the
retail file names (Beetle otherwise boots its own, different OpenBIOS),
headless runs through SDL's dummy driver, and ruler #2 as the check
before trusting a build. beetle-linux.md now uses the full patch set,
notes the current configure requirements and the headless option, and
gives psx-beetle's actual default port (4382).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 11:06:14 -07:00
Shokunin c4e50d92e0 tools: fp_identity review fixes (run failures, mixed FMV-quiet, Windows)
- run: a crash, hang-up or socket timeout on the final frame_fingerprint /
  dispatch_stats queries is a RunError (exit 3, INCOMPLETE), not a
  JSONDecodeError traceback with exit 1 (MISMATCH). Seeding, log and dump
  I/O errors and a runtime that survives SIGKILL also exit 3. A missing
  output directory is a usage error. The dump is written to OUT.tmp and
  renamed.
- compare: with different FMV-quiet settings, once a quiet frame occurred
  (qc > 0, a different frame set, or no qc column) the runs are not
  comparable. Report INCOMPLETE and skip the DIVERGE, missing-frame and
  locator lines, which were artefacts of the setting. Mixed settings with no
  quiet frame are judged as usual, with a note.
- --launch on Windows keeps backslashes (shlex with no escape character),
  so C:\psx\build\game-runtime.exe is not mangled into C:psxbuild...

Self-test: 59 cases (11 new), ctest fp_identity passes. R4 dumps re-judged
and fresh warm/cold runs repeat the IDENTICAL results with 497 tolerated
straddles.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 00:48:36 -07:00
Shokunin d36701ba7b docs: log the fp_identity A/B tool and its R4 measurement
FAITHFUL_TIMING_PLAN gets the dated session entry and ACCURACY_BURNDOWN's
backend-equivalence item names the tool and the R4 warm/cold result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 00:24:20 -07:00
Shokunin e8b17c394f tools: add fp_identity.py, a generic A/B guest-identity check
Moves the warm/cold A/B tool out of R4 so every title can use it. `run`
launches the runtime from a command template ({build} {port} {headless}
{state_dir}) or from --runtime/--game/--disc, optionally installs a saved
overlay state (--seed), turbos to frame N and saves the frame_fingerprint
ring plus the dispatch-miss count. Runs default to PSX_DEBUG_FMV_QUIET=0 and
PSX_OVERLAY_AUTOCOMPILE_OFF=1, and the dump records what they used.

`compare` judges on the guest-fact columns (cyc mmio mc sp sc wc ws qc) and
reports wr/pc only as the frame where the runs part. It tolerates the
one-write VBlank straddle from docs/TCP_COMMANDS.md (ws/wc off for one frame,
equal again at the next) and, when FMV-quiet was on, the wc+qc shift with ws
judged against the carried offset. Every tolerated frame is counted and
listed; --strict turns them off. A straddle on the last frame, a judge column
an older runtime does not report, or different FMV-quiet settings make the
result INCOMPLETE. Exit codes are unchanged from the R4 tool: IDENTICAL 0,
MISMATCH 1, usage 2, INCOMPLETE 3, MISSES 4.

The self-test (ctest fp_identity) covers column judging, both tolerances,
truncated and mismatched frame sets, dumps without ws/qc, and drives `run`
against a fake debug server.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 00:08:49 -07:00
Shokunin bd65c814f7 Run the full session start on the lobby rematch
The lobby rematch re-enters the emulator through goto session_reboot,
below the first-boot session block. After its commit (offline) or
netplay clear it reset mod-owned presentation state, but it never ran
mod_runtime_activate_plugins() or the controller, load-acceleration and
disc-speed resets. So an offline rematch with mods enabled applied the
plan's main-EXE and disc patches and ran its VBlank callbacks, but no
activation callback ran, and the function-entry hook table, which the
commit empties and only activation rebuilds, stayed empty. It also
mounted the stock disc even when the plan had built a derived disc
image.

start_mod_session() now holds the first-boot sequence, unchanged:
clear the controller-mode overrides and presentation policies, load
acceleration and disc speed; run reset_mod_owned_presentation();
activate; then apply the netplay local viewport, the controller-mode
overrides and load acceleration. The first boot and the rematch both
call it after their commit or netplay clear, and both then take the
mounted disc from session_disc_path(), which returns the plan's derived
image when there is one and otherwise the stock disc. Activation still
precedes renderer and window creation on the rematch, because those
happen after session_reboot on every session. The rematch re-applies
its 4:3 Settings clamp before the session start, so a plugin's
activation or the netplay local viewport can still set the aspect.

A netplay rematch clears the plan, so activating it runs nothing and
the match stays vanilla. The first boot's behaviour is unchanged.

Tests: test_mod_owned_display_controls.py now requires activation and
the mod-owned reset to happen only inside start_mod_session, in the
order reset, activate, apply. It requires both paths to call it after
their commit and before the disc mount, and the rematch 4:3 clamp to
come before it. mod_runtime_test adds the lobby rematch sequence:
activating after a netplay clear runs no callback, VBlank plugin or
hook; the offline rematch's commit and activation re-run the
activation callback and VBlank plugin and arm exactly the plan's hook.
test_mod_load_acceleration.py follows the reset block into the lambda body.
Two comments that said a soft return never re-applies the controller-mode
override are corrected: test_launcher_pad_mode_wiring.py and
test_launcher_pad_mode_resolve.cpp. MOD_PACKAGES.md now describes the
rematch as a full session start.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 23:42:17 -07:00
Shokunin 4bb58c2a81 Render passes: measure a stale pass-cost estimate again
Only passes that run are measured. The first passes of a race can all run in
a transient (a busy host, code run for the first time): in earlier Grand
Prix runs the three warm-up passes measured 13-15 ms against a steady 3-7 ms
at the same setting. An estimate that high prices most plans out, no pass
runs to correct it, and a plugin that crossfades while passes are shed (R4
Frame Rate) stayed on the crossfade for most or all of the run: 4, 9 and 29
passes in 22 s at 300 Hz, 7 in a verify run at 120 Hz, against about 1400
when the first passes were fast.

render_pass_cost_note_plan() (render_pass_plan.c) counts the plans that
wanted passes since the last measured pass. After 30 (about a second of a
30 Hz game) the warm-up restarts, so plans ask for one pass until three have
been measured again. When the new median is more than 3/4 of the old
estimate, the old one was right (a size that is truly too expensive, or a
host at its limit) and the next wait doubles, up to 960 plans, so these
one-pass warm-ups stay rare; a re-measure that finds the estimate stale
keeps the wait at 30. render_pass_stats reports cost_rewarms.

render_pass_plan_test covers the restart after 30 plans, an estimate that
passes keep measuring never restarting, the minimum wait after a stale
estimate, and the doubling waits when re-measures confirm the estimate
(47 ms at 4K, 10 ms re-measured as 9); render_pass_guards pins the planner's
call and its one-pass re-plan.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:19:08 -07:00
Shokunin c7ae3c86b2 Guard bltz_sites and branch_keep_sites in the main EXE
docs/WIDESCREEN.md said every explicit screen-X cull kind rejects a wrong
opcode at build time, but only slti, slti_lower, bgez and clip-edge sites
did. A bltz_sites entry was used only when the word was bltz, and
branch_keep_sites wrapped whatever branch was there, so a mistyped address
generated silently and the edge stayed narrow.

One check, CodeGenerator::check_explicit_branch_site, now covers the three
branch kinds in the main EXE: bltz_sites needs bltz (REGIMM rt=0),
bgez_sites bgez (REGIMM rt=1), branch_keep_sites a conditional branch
(REGIMM, beq, bne, blez, bgtz). It runs in generate_branch_condition, in
translate_instruction (non-branch words) and at the block exit, which also
catches a jump at a listed address (jumps never reached either guard, for
bgez_sites too). psxrecomp-game prints the site and exits 1; overlay
variants stay vanilla as before.

ws_cull_edge_codegen_test and recompiler_error_exit cover each kind on a
wrong opcode, a non-branch and a jump, valid sites, and overlay variants;
both fail against the previous emitter. The codegen test also pins that a
matching overlay instruction emits the helpers (the preamble defines them).
Docs list the required instruction per kind.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 22:00:17 -07:00
Shokunin 29a49eb4d5 Render passes: one pass per plan until the cost is measured at this size
render_pass_plan_phases() treated an unknown pass cost (0) as free, so the
first plans of a process scheduled every phase up to the slot cap. The
passes that allocate textures are left out of the average (up to 8 in a
row), so the first two frames ran at full count, and after an aspect or
resolution change the stale cost of the old size stayed in force. Passes
run on the emulation thread: at 4K (46.9 ms a pass) the first race frames
stalled the guest by several passes each.

- An unknown cost with a budget now plans one pass: it measures the cost
  and costs the guest at most one pass per frame while it is learnt.
- The cost is a RenderPassCost (render_pass_plan.c), replacing
  render_pass_cost_sample: the same allocation rule, then a warm-up of
  three kept samples whose median seeds the average. Until then the
  estimate is unknown. With one-pass plans a single slow first pass (a busy
  host) would otherwise set the average alone, and a plan that sheds every
  pass never measures again.
- The renderer keys the cost to the presented image size: a plan at
  another size passes an unknown cost, and the first pass there starts the
  cost over.
- RENDER_PASSES.md no longer claims passes never slow the guest, and says
  a cost above every frame's budget keeps passes shed until the size
  changes.

render_pass_plan_test covers the one-pass plan (none without budget), the
warm-up median (one outlier ignored; a truly expensive size still prices
itself out) and the allocation rule; render_pass_guards pins the size key.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 21:02:48 -07:00
Shokunin 3d705f73c3 Render passes: hold a late frame's newest image until the flip
A generation's timeline is fixed at plan time (t_len = plan period x
VBlank). When the game frame stayed up longer (a 30 Hz tick that lags to
three VBlanks, or host-interval drift), pass_gen_present expired the
generation past phase 1 + 0.5/period and the presenter fell back to its own
capture, which in FLIP mode is the frame's phase-0 image: the output ran
0 -> .25 -> .5 -> .75, stepped back to 0, then jumped to the next frame.

render_pass_gen_select() (render_pass_plan.c) now keeps render_pass_select's
hold of the newest image past phase 1 and only expires a generation after
RENDER_PASS_GEN_HOLD_MAX (4) frame lengths without a flip, i.e. a game that
stopped flipping. render_pass_stats reports late_presents (presents more
than 1/64 of a frame past its planned end). The plan keeps the plugin's
period: its phases were placed on the output deadlines of a frame of that
length, and the flip tracker's period is the previous frame's, which cannot
predict a sporadic lag.

render_pass_plan_test covers phases 1 to 4 (hold) and past 4 / NaN
(expire); render_pass_guards pins pass_gen_present to the rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 21:02:48 -07:00
Shokunin cbaf7a3a6c docs: render passes: register the pass hook under the plugin id
The pass point is a function-entry hook. Say how to register it with
psx_mod_register_function_entry_plugin() under the manifest [[plugin]] id
(it then runs only while the mod plan activates the plugin), to register
once because a repeated id and address returns 0, and that entry hooks
also fire for guest functions a pass calls.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 21:02:48 -07:00
Shokunin d300319b7e Render passes: follow the live 2/8 MiB RAM geometry
Upstream now has one live main-RAM geometry: retail 2 MiB mirrored four
times across the 8 MiB decode window, or 8 MiB unique with the opt-in
8 MB RAM mod (psx_memory.h). memory.c's RAM array is sized for the
largest geometry, so the pass store policy was handed 8 MiB even on a
retail boot, and a store to a mirror (0x80200000..) would have landed in
the unused upper backing bytes instead of folding onto RAM. With the
8 MB map live, the checkpoint copied and restored only the low 2 MiB.

- memory.c passes the store policy psx_ram_live_bytes(), so pass stores
  fold exactly as psx_ram_map_write does outside a pass.
- render_pass.c checkpoints, restores and (PSX_RENDER_PASS_VERIFY)
  hashes memory_get_ram_bytes() bytes; the copy grows if a later session
  boots with the larger map.
- render_pass_sandbox_test: policy and a real pass with 8 MiB live
  (unique addresses up to 0x7FFFFF, all 8 MiB restored), then a retail
  pass folding mirrors again. Restoring a fixed 2 MiB fails it twice;
  folding with the backing size fails it twice. render_pass_guards pins
  the live size in memory.c and in the checkpoint, restore and hash.
- psx_ram_runtime_map_test (upstream's 2/8 MiB test over the real
  memory.c and dma.c) links those files with section GC. The pass paths
  make six more symbols reachable there (the pass flag and store policy,
  the synchronous GPU linked-list walk); its stubs double them, trapping
  if a pass path ever runs. With the macOS-only link gaps stubbed as well,
  the test passes on this branch as on master.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 21:02:48 -07:00
Shokunin ab2ddbc351 Render passes: journal out-of-rect writes under native-wide too
Found in the R4 integration build (widescreen + Interpolated): R4 copies
2x1 VRAM pixels at 0,0, outside the pass rect, every frame. The journal
backs such writes up and restores them, but pass_journal_protect() refused
whenever a native-wide surface existed, so with the widescreen view on
every pass was rolled back ("render pass refused copy 0,0 2x1 outside
0,240 320x240", "rolled back (VRAM write outside the pass rect)") and no
in-between frame was ever shown.

The refusal was not needed: fills (gpu_fill), copies (gpu_copy_rect),
uploads (glb_vram_transfer_in / flush_cpu_upload) and pokes
(glb_vram_write) write the hr surface, the raw mirror and the CPU VRAM
rows only -- exactly what the journal backs up -- never a wide surface.
The wide margins come from mirrored draws, which the pass scissor already
keeps inside the pass band.

test_render_pass_guards pins both halves: pass_journal_protect() must not
test g_wide, and none of those five write paths may reference a wide
surface. With the old refusal put back the guard fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 21:02:48 -07:00
Shokunin 3e262c128d Render passes: unit-test the sandbox (store drops, frozen devices, journal)
The store policy and the CPU side of the out-of-rect VRAM journal move into
render_pass_plan.c as pure functions (render_pass_store_to,
render_pass_vram_policy, render_pass_journal_add/_rollback); memory.c and
gpu_gl_renderer.c only wire them to their arrays and GL copies. Behaviour is
unchanged.

render_pass_sandbox_test covers what the plan's U5a list asked for:
- every device class a pass may not reach (SPU key-on and control, CD,
  timer, other DMA channels, GP1, SIO, memory control, MDEC, expansion,
  KSEG2) is dropped under its class and never delivered, while GP0,
  GP1 DMA mode/info, I_STAT/I_MASK, GPU/OTC DMA and DPCR reach the device;
- a real pass (render_pass.c, psx_cycles.c, timers.c) that makes those
  stores, acknowledges and masks every interrupt, runs 10^6 cycles while
  timer 2 is armed to interrupt every 1000, and changes a timer behind the
  store policy's back: drops counted per class, no device advanced, no
  interrupt raised, RAM, scratchpad, I_STAT/I_MASK, timers and clock
  restored, and the timer interrupts again once time is live;
- fast-forward and backend status refuse the pass without running it;
- VRAM writes inside the rect are allowed, outside ones journaled once
  (covered writes not twice, clipped to VRAM) and rolled back exactly,
  newest first; a full journal or a mode that cannot journal refuses.

Mutations it catches: SPU or CD mis-classed, device service not frozen,
rollback order or entry skipped, I_STAT, timer, RAM or scratchpad restore
removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 21:02:48 -07:00
Shokunin 673e75ec86 Render passes: reuse the VRAM backup for later passes of a frame
Every pass backed up the display rect (hr colour + stencil and raw mirror
blits, CPU rows) although the previous pass of the same frame had just
restored exactly that backup. On Apple's OpenGL each blit ends the render
pass and submits a command buffer, which was about half of a pass's begin
time. A pass now reuses the previous pass's backup when render_pass.c
vouches that no guest code ran since its restore (same plan, guest clock and
store count unchanged) and the rect, scale and native-wide source match; the
first pass of every frame still takes a fresh backup.

R4 Grand Prix race, 240 Hz interpolated: average pass begin 0.28 -> 0.16 ms.
render_pass_stats reports backups_reused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 21:02:48 -07:00
Shokunin 69e6b6b552 Render passes: refuse during fast-forward; say why passes are unavailable
Manual fast-forward presents every second VBlank, so the plan still saw
presents and kept rendering passes, spending host time the fast-forward
wants, although the API said turbo refuses them. main.cpp now records when
presentation is sped up (manual fast-forward, turbo-through-loads, FMV
auto-skip, TCP turbo) and the plan and passes refuse meanwhile.

psx_mod_render_pass_status() tells a plugin why passes cannot run, the
host-time budget aside: NO_PRESENTER, BACKEND (the renderer declines passes
in its current mode), DISABLED (after repeated faults), SESSION, FAST_FORWARD
or BUSY. The GL backend reports its part through
gl_renderer_pass_unavailable(), where a renderer mode that cannot host
passes reports BACKEND. psx_mod_set_frame_interpolation_blend() now also
takes effect when called from a hook after activation, so a plugin can show a
crossfade while passes are unavailable and return to HOLD afterwards.

Debug: PSX_RENDER_PASS_REFUSE=1 or TCP render_pass_refuse on=1 makes the
backend decline passes, to test a plugin's fallback; render_pass_stats
reports the status.

Default off: nothing reads the new flag or status unless a plugin uses
render passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 21:02:48 -07:00
Shokunin 856cbabffe Render passes: make pass image textures as slots fill
pass_slot_cap() limits a generation to what two generations fit in 256 MiB,
but pass_gen_textures() allocated all 17 slots of each generation at
internal-resolution size as soon as passes ran, whatever the cap: 34
textures, about 167 MB at 4x and 1.5 GB at 12x for at most two used slots.

Slot textures are now made on first use, never past the cap, so the budget
holds at any internal scale; a size change frees the old set, and context
teardown frees every pass texture, backup and journal FBO and forgets their
names (a new context no longer reuses dead ones). render_pass_stats reports
image_textures and image_bytes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 21:02:48 -07:00
Shokunin ea1e67a423 Render passes: PSX_RENDER_PASS_WATCHDOG to exercise the rollback path
A debug override that lowers the guest-cycle watchdog (1000 up to the 8 M
default), so a title's real passes can be driven through the abort path and
checked with fingerprints against a run without passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 21:02:48 -07:00
Shokunin 1a8f382085 Render passes: put host nesting back after a watchdog abort
The watchdog leaves a runaway pass by longjmp from inside guest code, which
skips the exits of every frame it leaves: generated functions' bb_defer
cleanup, overlay_loader_call_native's unit depth, the native-shard stack and
cycle-flush hook, the DMA kick's exec depth, and the dirty-RAM interpreter's
active/phase/precise flags, resume latch and pending load. The checkpoint
now saves all of it and the restore puts it back (bb_defer with the cycle
freeze), as the scheduler and exception landings do for theirs. An overlay
shard's unpublished pass cycles are flushed into the frozen clock at the
landing so they never reach the live one.

PSX_RENDER_PASS_VERIFY also checks that a pass which returned normally left
the nesting balanced, since the restore would hide that. render_pass_stats
gains nesting_repairs and the fault log names what the landing undid.

render_pass_abort_test aborts a pass from five nested frames (real
render_pass.c and psx_cycles.c, mock devices and presenter) and checks every
value, the clock and RAM; it fails without this change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 21:02:47 -07:00