Commit Graph

1951 Commits

Author SHA1 Message Date
Matthew Stanley 718f2b227e docs: record IPO-on title validation 2026-09-13 00:21:45 -07:00
Matthew Stanley 20ec948ab1 build: add opt-in runtime IPO experiment with validation gates 2026-09-12 23:31:28 -07:00
Matt Stanley ee1d72543c Merge pull request #354 from RetroPortingToolKit/fix/cfg-metadata-integrity
fix(recompiler): repair live CFG metadata and natural back edges
2026-09-12 23:30:17 -07:00
Matthew Stanley 2677fccce3 docs: include bounded input smoke results and limits 2026-09-12 23:29:52 -07:00
Matthew Stanley c5b3b7a214 docs: record CFG repair title regression and merge boundaries 2026-09-12 23:28:23 -07:00
Matthew Stanley 90c62097e0 test(recompiler): cover CFG reachability in fallthrough emission 2026-09-12 23:24:51 -07:00
Matthew Stanley 6c55d698bb fix(recompiler): rebuild CFG metadata and prove natural back edges 2026-09-12 23:24:51 -07:00
Matt Stanley de8b942fb4 Merge pull request #355 from RetroPortingToolKit/fix/optional-netplay-auth-header
fix(runtime): restore builds without optional netplay auth
2026-09-12 23:24:14 -07:00
Matthew Stanley ce7f646f42 fix(runtime): keep chat-report dependency inside enabled lobby path 2026-09-12 23:10:42 -07:00
Matthew Stanley fa655333fb fix(runtime): guard optional netplay authentication dependencies 2026-09-12 23:06:05 -07:00
Alex Vanderveen 3a275d4939 launcher: Allow Spectators is a preference, not an in-room setting
The toggle lives in the Create Lobby modal, before any room exists, and
its value goes out with `create`. The setter refused unless the client
was already seated in a server lobby, so the modal read the unchanged
preference back every frame and the checkbox snapped off. Refuse only in
a LAN / Direct-IP room, which has no gallery to allow -- the SNES host's
rule.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FPXfGGhBBBxQPcEzaMsauF
2026-09-12 20:54:12 -04:00
Alex Vanderveen a9bfbf4d15 Merge pull request #351 from RetroPortingToolKit/feat/lobby-parity
Feat/lobby parity
2026-09-12 20:06:06 -04:00
Alex Vanderveen b9ca600778 netplay: block list, account keys, list scope and automatch for the launcher
Brings the PSX runtime level with what recomp-ui's netplay pages now ask a
host for, and with the SNES runner that already answers them
(snes_lobby_client.c / snes_host_lobby.c):

- Chat lines, seat rows and players-online rows carry the server's opaque
  account key, so the launcher's ignore/block list survives a rename.
- psx_lobby_set_blocks pushes that list to the server as `set_blocks`, since
  refusing to pair or seat the two together is the server's job; recomp-ui
  re-sends it on every reconnect.
- list_scope_set: a LAN player no longer sees online rooms they have no
  connection for, and an online player no longer sees their own LAN rows.
- Automatch: rulesets discovery, the relay latency probe (14-byte LE frame,
  type 200 -> 201, nonce-matched), queue with the disc fingerprint, the
  accept gate with a locally counted deadline, requeue/cancel/cooldown, and
  claiming automatch's `error` codes only while a ticket is in flight.
  Leaving an automatch room after the match: it is the server's room, has
  no host to rematch with, and holding the seat blocks the next ticket.
- Account: init the auth client once per URL rather than every pump (the
  login worker reads the host), anchor the device key to the executable
  directory, and publish the signed-in handle as the lobby display name.

mods_enabled is asserted false on PSX: every netplay launch clears the mod
plan or refuses to start, and the caps a match runs are the ruleset's.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FPXfGGhBBBxQPcEzaMsauF
2026-09-12 19:39:09 -04:00
Alex Vanderveen 615d35fe22 Merge remote-tracking branch 'origin/master' into feat/lobby-parity
# Conflicts:
#	tools/new_project_layout/project_studio/readme_metrics.py
2026-09-12 19:30:47 -04:00
Alex Vanderveen fddc8b8516 netplay: report chat lines for moderation, and pin recomp-net for it
PsxLobbyChatMsg gains the server's `mid` for a line, taken off `chat` and
`server_chat`, and psx_lobby_report_chat hands a list of them to
rnet_chat_report_build. The frame carries ids and never the text, so a
report cannot fabricate what somebody said. Same eight lines as the SNES
copy; what a report contains lives in recomp-net.

lib/recomp-net -> 859ba28 (support chat reporting), which is also where
rnet_account_set_secret_path comes from.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FPXfGGhBBBxQPcEzaMsauF
2026-09-12 19:29:57 -04:00
Alex Vanderveen 84bc48df5b sync 2026-09-12 19:14:50 -04:00
Matt Stanley 0baf7bb11d Merge pull request #349 from RetroPortingToolKit/fix/observed-overlay-interiors
fix(overlays): recover executed dispatch entries without a shared host
2026-09-12 10:58:25 -07:00
Matthew Stanley 4683e923ea fix(overlays): retain executed dispatch demands after host rejection 2026-09-12 10:45:50 -07:00
Matt Stanley 89db8168db Merge pull request #348 from RetroPortingToolKit/integrate/wo6-bios343-346
Integrate WO-6 dispatch lookup and BIOS #343/#346 with game regressions
2026-09-11 23:52:03 -07:00
Matthew Stanley 1c7e62a58a Record matched MMX6 demo fade-out validation 2026-09-11 23:51:43 -07:00
Matthew Stanley b9e49074ea Record combined Tomba and MMX6 BIOS regression evidence 2026-09-11 23:50:15 -07:00
Matthew Stanley 9576c6323f Refresh combined BIOS and dispatch validation against latest master 2026-09-11 23:39:47 -07:00
Matthew Stanley fae85cc886 Repair Windows and stale regression test harness assumptions 2026-09-11 23:39:47 -07:00
Matthew Stanley 63b205df78 Index immutable resident dispatch keys and clarify owner telemetry 2026-09-11 23:39:46 -07:00
Matthew Stanley fd84134c19 Guard patched BIOS ranges before control flow and cycle charging 2026-09-11 23:39:33 -07:00
Matthew Stanley ac0e40628a Match BIOS setup detection to configured linkable backends 2026-09-11 23:39:11 -07:00
Matthew Stanley b4ea4c37e9 Expand opt-in DMA arenas without aliasing CPU BIOS addresses 2026-09-11 23:05:07 -07:00
Matthew Stanley 2bcc7ae0ba Integrate original-file tagged relocation into AOT profiles 2026-09-11 23:03:37 -07:00
Matthew Stanley 860bd6a237 Add verified two-bit tagged MIPS relocation reader 2026-09-11 22:59:03 -07:00
Matthew Stanley a1e7db08e7 Preserve verified resident helper markers in AOT packages 2026-09-11 22:56:16 -07:00
Matthew Stanley c5390e4265 Merge kernel patch ranges PR #346 for combined validation 2026-09-11 22:53:07 -07:00
Matthew Stanley 9550b3e579 Merge BIOS setup-host PR #343 for combined validation 2026-09-11 22:53:05 -07:00
Matthew Stanley 736c002a1a Refuse AOT releases whose packaged runtime disables native overlays 2026-09-11 22:40:33 -07:00
Matthew Stanley 06a93dc8aa Forward widescreen hooks in native overlays and verify fallback intervals 2026-09-11 22:28:03 -07:00
Matthew Stanley fdd0bd8eeb Decode aligned LZSS banks for original-disc AOT coverage 2026-09-11 22:08:36 -07:00
Matthew Stanley 070c92c647 Add declarative original-disc structural inventory inspection 2026-09-11 21:27:53 -07:00
Matthew Stanley 0e06446310 Extract archive-relative sector extents for original-disc AOT 2026-09-11 21:11:25 -07:00
Matthew Stanley 52dd4dffbc Add verified packed sector-table AOT overlay method 2026-09-11 20:37:37 -07:00
Matthew Stanley 3a174fab9f Accept exact fragment recovery after an empty primary overlay scan 2026-09-11 19:33:31 -07:00
Matthew Stanley 51ea5559f2 Add method-based original-disc AOT release pipeline 2026-09-11 19:24:46 -07:00
Matt Stanley da4434cfb9 Merge pull request #347 from RetroPortingToolKit/fix/overlay-scheduled-switch
Support scheduled overlay switches and document disc-derived AOT sharding
2026-09-11 16:33:18 -07:00
Matthew Stanley c3daaaa7f0 Document disc-derived AOT sharding and preserve full producer extents 2026-09-11 16:28:01 -07:00
Kevin baca0a8a42 tools: find install-slot ranges and A/B what they buy, for any title
The install-slot range work is only trustworthy if a second title can
reproduce it, and the two tools that produced its evidence lived in one
game repo with that game's paths baked in. Moved into the framework and
generalised: every path defaults off --project-root (cwd) and the BIOS
profile's out_stem, and the exe is found by globbing the build tree,
because its name is derived from the window title and so differs per
title. Run either from a game project root.

kernel_patch_diff.py boots headless and diffs live kernel RAM against
the ROM copy it came from, joining each differing word to the compiled
kernel bodies. It reads the emitted PsxKernelPatchRange table, so it
separates a DECLARED patch (the verifier skips it, the body still runs
native) from one that genuinely unblesses a body. Without that split its
verdict line calls every declared range "these interpret", which is
exactly backwards once the ranges work. This is the tool you use to find
ranges for a new image or a new Psy-Q release.

kernel_patch_ab.py answers "what did declaring them buy". It boots the
same binary twice to the same frame, once with the ranges live and once
with PSX_KERNEL_PATCH_RANGES=0, which drops them and restores the
whole-body bless memcmp. Holding the binary constant holds the overlays,
the codegen and the disc constant too, so only the bless verdict moves.
Headless runs uncapped, so it normalises per frame rather than comparing
raw totals at different frame counts.

Two things it gets right that cost real time to learn:

- It does NOT discount the A0/B0/C0 vectors from the headline. An earlier
  version did, reasoning that they interpret on both sides and would mask
  the result. True on one title, false in general: on Mega Man X6 with
  OpenBIOS the B0 vector burns 2,361,470 instructions over 1,180,735
  entries with the ranges off and vanishes entirely with them on, because
  a clean bless verdict lets the vector body dispatch native. Discounting
  it hid a 130 insn/frame win. Kernel and all-interp are the headline;
  the bodies/vectors split is printed underneath as diagnosis.
- It fails fast on a wedge, detected as a frame counter that stops
  advancing while the debug server still answers, instead of burning the
  whole timeout. That is how a dispatch key inside a declared range was
  caught spinning the dispatch loop at frame 0.

debug_client.query grew an optional timeout so these can use it instead
of carrying their own transport; the 10 s default is too short for
dirty_ram_stats and a whole-window read_ram.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-11 17:57:01 -04:00
Kevin 3cbe6c1cde kernel-bless: publish install-slot ranges so patched BIOS bodies run native
The BIOS exception handler ran interpreted for the life of every process,
1.22 billion instructions and 44.6% of all interpreted work in Breath of
Fire III, even though it is compiled and even though the stub that
unblessed it sat at exactly the address bios/SCPH1001.toml already
declared as an install slot. Two mechanisms existed and did not compose:
the emitter hooked the declared slot PC, while the runtime's bless
verifier memcmp'd the whole body including that slot, so the body was
MISMATCH forever and native code never reached the hook.

Three parts, none of which pays off alone:

1. Publish the declared ranges to the runtime. The emitter emits a
   PsxKernelPatchRange table next to the kernel body table and routes it
   through the backend descriptor; the bless verifier compares a body in
   segments that skip them. CLEAN now means "every byte outside every
   declared range still matches ROM", which is the claim the native code
   actually depends on.

2. Generalise the slot model to what the patches are. The old model
   assumed a 4-word lui/addiu/jalr/nop stub written over ROM zeros and
   fired on `word != 0`. Measured against both images, only one of five
   observed shapes fits: patchers also plant a `jr` over real ROM
   instructions, rewrite an 11-word prologue and insert an `mfc0 Cause`,
   and NOP out a driver routine. A slot is now a range with a length and
   an explicit `resume` (jalr / fallthrough / none), detected by
   comparing every word against its ROM-baked value.

3. Hand back from the interpreter at the range end. Kernel page 0 is
   permanently dirty, because the handler saves registers there on every
   exception, so straight-line interpretation would run the whole
   function after the patch. The emitter registers the resume PC as a
   continuation key and dirty_ram_interp surfaces there.

A PC inside a declared range must never be a native dispatch key: the
compiled bytes there are not what executes. Seven pre-existing
jal-return continuations violated that and wedged the boot at frame 0 in
an infinite dispatch loop (OpenBIOS RAM 0x357C). The emitter now drops
continuation keys inside a range so dispatch misses through to the
interpreter, and refuses to emit if a range covers a function entry,
which would silently remove a callable function instead.

Both profiles declare their measured ranges, and SCPH1001.toml finally
pins [program.image] sha256 to v2.2 (CRC32 37157331) so a v2.0 dump
refuses to generate rather than producing phantom discovery gaps.

Measured on Breath of Fire III (SLPS-00990, Psy-Q SDK 3.70), headless,
matched frames, PSX_KERNEL_PATCH_RANGES=0 for the OFF side:

                      OpenBIOS            SCPH-1001 v2.2
  bless mismatch      21 -> 0             74 -> 0
  kernel bodies       -87.7%/frame        -97.8%/frame
  all interp insns    -42.6%/frame        -94.2%/frame

The residual kernel-body work is exactly declared-words x entries on
every range, so nothing leaks past the hook. The A0/B0/C0 vector
trampolines are unchanged (+0.2%, noise): the profile excludes them by
design and they are a separate job. Card read traces are byte-identical
across the A/B on both images, all 32 entries, which exercises the two
ranges that replace the kernel's card handler.

Rule 18 holds throughout: the guest's patched instructions still execute
as written on the dirty-RAM interpreter. Only the code around them
changes backend. Nothing is synthesised.

Tests: runtime/tests/test_kernel_patch_ranges.c (new, the verifier's
accept/reject behaviour and the hand-back predicate) and the install-slot
range invariants in recompiler/tests/bios_address_model_test.cpp.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-11 16:38:34 -04:00
Matthew Stanley fe2c0046ba Recognize bounded MIPS switches with scheduled table constants 2026-09-11 13:38:06 -07:00
Matt Stanley 6f77dcc31f Merge pull request #345 from RetroPortingToolKit/fix/pgo-train-clean-exit
Flush Windows PGO profiles through clean runtime exit
2026-09-10 21:40:19 -07:00
Matthew Stanley 62ca12bd31 Flush PGO profiles through clean runtime exit 2026-09-10 21:40:06 -07:00
Matt Stanley 7d23cedb8d Merge pull request #344 from RetroPortingToolKit/pe-feedback
Fix Clang overlay exports, PGO, and overlay-cache scaffold
2026-09-10 17:52:02 -07:00
Matthew Stanley 2113defcfc Fix overlay Clang exports, PGO, and scaffold cache 2026-09-10 17:23:03 -07:00
Alexandros Mandravillis c2f9c3d59a setup: detect any recompiled BIOS stem and stop the first-run setup loop
Focused port of the fork's fix/setup-host-bios-deadlock feature (10 commits) onto current upstream master, per the 2026-09-10 decision to port rather than rebase (the fork line and upstream share no merge base).

Content: the codegen host accepts any recompiled BIOS stem instead of only OpenBIOS/SCPH1001 (psx_bios_known_images.h); the runtime checks the setup host before validating a chosen BIOS and the first-run loop breaker reports a cause-specific reason; the CLI decodes subprocess output as UTF-8; --setup-selfcheck verifies a headless setup; the rbengine preflight names the recursive init. Two port conflicts resolved: the new CTest registrations in recompiler/CMakeLists.txt are kept (the names exist only in this change), and runtime.cmake takes the new recomp-ui guidance text. Cubic P3 finding fixed: the sibling PSX_NETPLAY FATAL_ERROR now also names --recursive.

Evidence: python syntax checks pass for psxrecomp_cli.py and the new tests; runtime/tests/test_codegen_host_bios_stems.py runs or skips only on a missing RECOMP_UI_ROOT. No full runtime build or retail route is claimed here. Developed with AI assistance (Cline, 2026-09-10).
2026-09-11 01:26:22 +03:00