Commit Graph

1943 Commits

Author SHA1 Message Date
Matthew Stanley ce7f646f42 fix(runtime): keep chat-report dependency inside enabled lobby path 2026-09-12 23:10:42 -07:00
Matthew Stanley fa655333fb fix(runtime): guard optional netplay authentication dependencies 2026-09-12 23:06:05 -07:00
Alex Vanderveen 3a275d4939 launcher: Allow Spectators is a preference, not an in-room setting
The toggle lives in the Create Lobby modal, before any room exists, and
its value goes out with `create`. The setter refused unless the client
was already seated in a server lobby, so the modal read the unchanged
preference back every frame and the checkbox snapped off. Refuse only in
a LAN / Direct-IP room, which has no gallery to allow -- the SNES host's
rule.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FPXfGGhBBBxQPcEzaMsauF
2026-09-12 20:54:12 -04:00
Alex Vanderveen a9bfbf4d15 Merge pull request #351 from RetroPortingToolKit/feat/lobby-parity
Feat/lobby parity
2026-09-12 20:06:06 -04:00
Alex Vanderveen b9ca600778 netplay: block list, account keys, list scope and automatch for the launcher
Brings the PSX runtime level with what recomp-ui's netplay pages now ask a
host for, and with the SNES runner that already answers them
(snes_lobby_client.c / snes_host_lobby.c):

- Chat lines, seat rows and players-online rows carry the server's opaque
  account key, so the launcher's ignore/block list survives a rename.
- psx_lobby_set_blocks pushes that list to the server as `set_blocks`, since
  refusing to pair or seat the two together is the server's job; recomp-ui
  re-sends it on every reconnect.
- list_scope_set: a LAN player no longer sees online rooms they have no
  connection for, and an online player no longer sees their own LAN rows.
- Automatch: rulesets discovery, the relay latency probe (14-byte LE frame,
  type 200 -> 201, nonce-matched), queue with the disc fingerprint, the
  accept gate with a locally counted deadline, requeue/cancel/cooldown, and
  claiming automatch's `error` codes only while a ticket is in flight.
  Leaving an automatch room after the match: it is the server's room, has
  no host to rematch with, and holding the seat blocks the next ticket.
- Account: init the auth client once per URL rather than every pump (the
  login worker reads the host), anchor the device key to the executable
  directory, and publish the signed-in handle as the lobby display name.

mods_enabled is asserted false on PSX: every netplay launch clears the mod
plan or refuses to start, and the caps a match runs are the ruleset's.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FPXfGGhBBBxQPcEzaMsauF
2026-09-12 19:39:09 -04:00
Alex Vanderveen 615d35fe22 Merge remote-tracking branch 'origin/master' into feat/lobby-parity
# Conflicts:
#	tools/new_project_layout/project_studio/readme_metrics.py
2026-09-12 19:30:47 -04:00
Alex Vanderveen fddc8b8516 netplay: report chat lines for moderation, and pin recomp-net for it
PsxLobbyChatMsg gains the server's `mid` for a line, taken off `chat` and
`server_chat`, and psx_lobby_report_chat hands a list of them to
rnet_chat_report_build. The frame carries ids and never the text, so a
report cannot fabricate what somebody said. Same eight lines as the SNES
copy; what a report contains lives in recomp-net.

lib/recomp-net -> 859ba28 (support chat reporting), which is also where
rnet_account_set_secret_path comes from.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FPXfGGhBBBxQPcEzaMsauF
2026-09-12 19:29:57 -04:00
Alex Vanderveen 84bc48df5b sync 2026-09-12 19:14:50 -04:00
Matt Stanley 0baf7bb11d Merge pull request #349 from RetroPortingToolKit/fix/observed-overlay-interiors
fix(overlays): recover executed dispatch entries without a shared host
2026-09-12 10:58:25 -07:00
Matthew Stanley 4683e923ea fix(overlays): retain executed dispatch demands after host rejection 2026-09-12 10:45:50 -07:00
Matt Stanley 89db8168db Merge pull request #348 from RetroPortingToolKit/integrate/wo6-bios343-346
Integrate WO-6 dispatch lookup and BIOS #343/#346 with game regressions
2026-09-11 23:52:03 -07:00
Matthew Stanley 1c7e62a58a Record matched MMX6 demo fade-out validation 2026-09-11 23:51:43 -07:00
Matthew Stanley b9e49074ea Record combined Tomba and MMX6 BIOS regression evidence 2026-09-11 23:50:15 -07:00
Matthew Stanley 9576c6323f Refresh combined BIOS and dispatch validation against latest master 2026-09-11 23:39:47 -07:00
Matthew Stanley fae85cc886 Repair Windows and stale regression test harness assumptions 2026-09-11 23:39:47 -07:00
Matthew Stanley 63b205df78 Index immutable resident dispatch keys and clarify owner telemetry 2026-09-11 23:39:46 -07:00
Matthew Stanley fd84134c19 Guard patched BIOS ranges before control flow and cycle charging 2026-09-11 23:39:33 -07:00
Matthew Stanley ac0e40628a Match BIOS setup detection to configured linkable backends 2026-09-11 23:39:11 -07:00
Matthew Stanley b4ea4c37e9 Expand opt-in DMA arenas without aliasing CPU BIOS addresses 2026-09-11 23:05:07 -07:00
Matthew Stanley 2bcc7ae0ba Integrate original-file tagged relocation into AOT profiles 2026-09-11 23:03:37 -07:00
Matthew Stanley 860bd6a237 Add verified two-bit tagged MIPS relocation reader 2026-09-11 22:59:03 -07:00
Matthew Stanley a1e7db08e7 Preserve verified resident helper markers in AOT packages 2026-09-11 22:56:16 -07:00
Matthew Stanley c5390e4265 Merge kernel patch ranges PR #346 for combined validation 2026-09-11 22:53:07 -07:00
Matthew Stanley 9550b3e579 Merge BIOS setup-host PR #343 for combined validation 2026-09-11 22:53:05 -07:00
Matthew Stanley 736c002a1a Refuse AOT releases whose packaged runtime disables native overlays 2026-09-11 22:40:33 -07:00
Matthew Stanley 06a93dc8aa Forward widescreen hooks in native overlays and verify fallback intervals 2026-09-11 22:28:03 -07:00
Matthew Stanley fdd0bd8eeb Decode aligned LZSS banks for original-disc AOT coverage 2026-09-11 22:08:36 -07:00
Matthew Stanley 070c92c647 Add declarative original-disc structural inventory inspection 2026-09-11 21:27:53 -07:00
Matthew Stanley 0e06446310 Extract archive-relative sector extents for original-disc AOT 2026-09-11 21:11:25 -07:00
Matthew Stanley 52dd4dffbc Add verified packed sector-table AOT overlay method 2026-09-11 20:37:37 -07:00
Matthew Stanley 3a174fab9f Accept exact fragment recovery after an empty primary overlay scan 2026-09-11 19:33:31 -07:00
Matthew Stanley 51ea5559f2 Add method-based original-disc AOT release pipeline 2026-09-11 19:24:46 -07:00
Matt Stanley da4434cfb9 Merge pull request #347 from RetroPortingToolKit/fix/overlay-scheduled-switch
Support scheduled overlay switches and document disc-derived AOT sharding
2026-09-11 16:33:18 -07:00
Matthew Stanley c3daaaa7f0 Document disc-derived AOT sharding and preserve full producer extents 2026-09-11 16:28:01 -07:00
Kevin baca0a8a42 tools: find install-slot ranges and A/B what they buy, for any title
The install-slot range work is only trustworthy if a second title can
reproduce it, and the two tools that produced its evidence lived in one
game repo with that game's paths baked in. Moved into the framework and
generalised: every path defaults off --project-root (cwd) and the BIOS
profile's out_stem, and the exe is found by globbing the build tree,
because its name is derived from the window title and so differs per
title. Run either from a game project root.

kernel_patch_diff.py boots headless and diffs live kernel RAM against
the ROM copy it came from, joining each differing word to the compiled
kernel bodies. It reads the emitted PsxKernelPatchRange table, so it
separates a DECLARED patch (the verifier skips it, the body still runs
native) from one that genuinely unblesses a body. Without that split its
verdict line calls every declared range "these interpret", which is
exactly backwards once the ranges work. This is the tool you use to find
ranges for a new image or a new Psy-Q release.

kernel_patch_ab.py answers "what did declaring them buy". It boots the
same binary twice to the same frame, once with the ranges live and once
with PSX_KERNEL_PATCH_RANGES=0, which drops them and restores the
whole-body bless memcmp. Holding the binary constant holds the overlays,
the codegen and the disc constant too, so only the bless verdict moves.
Headless runs uncapped, so it normalises per frame rather than comparing
raw totals at different frame counts.

Two things it gets right that cost real time to learn:

- It does NOT discount the A0/B0/C0 vectors from the headline. An earlier
  version did, reasoning that they interpret on both sides and would mask
  the result. True on one title, false in general: on Mega Man X6 with
  OpenBIOS the B0 vector burns 2,361,470 instructions over 1,180,735
  entries with the ranges off and vanishes entirely with them on, because
  a clean bless verdict lets the vector body dispatch native. Discounting
  it hid a 130 insn/frame win. Kernel and all-interp are the headline;
  the bodies/vectors split is printed underneath as diagnosis.
- It fails fast on a wedge, detected as a frame counter that stops
  advancing while the debug server still answers, instead of burning the
  whole timeout. That is how a dispatch key inside a declared range was
  caught spinning the dispatch loop at frame 0.

debug_client.query grew an optional timeout so these can use it instead
of carrying their own transport; the 10 s default is too short for
dirty_ram_stats and a whole-window read_ram.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-11 17:57:01 -04:00
Kevin 3cbe6c1cde kernel-bless: publish install-slot ranges so patched BIOS bodies run native
The BIOS exception handler ran interpreted for the life of every process,
1.22 billion instructions and 44.6% of all interpreted work in Breath of
Fire III, even though it is compiled and even though the stub that
unblessed it sat at exactly the address bios/SCPH1001.toml already
declared as an install slot. Two mechanisms existed and did not compose:
the emitter hooked the declared slot PC, while the runtime's bless
verifier memcmp'd the whole body including that slot, so the body was
MISMATCH forever and native code never reached the hook.

Three parts, none of which pays off alone:

1. Publish the declared ranges to the runtime. The emitter emits a
   PsxKernelPatchRange table next to the kernel body table and routes it
   through the backend descriptor; the bless verifier compares a body in
   segments that skip them. CLEAN now means "every byte outside every
   declared range still matches ROM", which is the claim the native code
   actually depends on.

2. Generalise the slot model to what the patches are. The old model
   assumed a 4-word lui/addiu/jalr/nop stub written over ROM zeros and
   fired on `word != 0`. Measured against both images, only one of five
   observed shapes fits: patchers also plant a `jr` over real ROM
   instructions, rewrite an 11-word prologue and insert an `mfc0 Cause`,
   and NOP out a driver routine. A slot is now a range with a length and
   an explicit `resume` (jalr / fallthrough / none), detected by
   comparing every word against its ROM-baked value.

3. Hand back from the interpreter at the range end. Kernel page 0 is
   permanently dirty, because the handler saves registers there on every
   exception, so straight-line interpretation would run the whole
   function after the patch. The emitter registers the resume PC as a
   continuation key and dirty_ram_interp surfaces there.

A PC inside a declared range must never be a native dispatch key: the
compiled bytes there are not what executes. Seven pre-existing
jal-return continuations violated that and wedged the boot at frame 0 in
an infinite dispatch loop (OpenBIOS RAM 0x357C). The emitter now drops
continuation keys inside a range so dispatch misses through to the
interpreter, and refuses to emit if a range covers a function entry,
which would silently remove a callable function instead.

Both profiles declare their measured ranges, and SCPH1001.toml finally
pins [program.image] sha256 to v2.2 (CRC32 37157331) so a v2.0 dump
refuses to generate rather than producing phantom discovery gaps.

Measured on Breath of Fire III (SLPS-00990, Psy-Q SDK 3.70), headless,
matched frames, PSX_KERNEL_PATCH_RANGES=0 for the OFF side:

                      OpenBIOS            SCPH-1001 v2.2
  bless mismatch      21 -> 0             74 -> 0
  kernel bodies       -87.7%/frame        -97.8%/frame
  all interp insns    -42.6%/frame        -94.2%/frame

The residual kernel-body work is exactly declared-words x entries on
every range, so nothing leaks past the hook. The A0/B0/C0 vector
trampolines are unchanged (+0.2%, noise): the profile excludes them by
design and they are a separate job. Card read traces are byte-identical
across the A/B on both images, all 32 entries, which exercises the two
ranges that replace the kernel's card handler.

Rule 18 holds throughout: the guest's patched instructions still execute
as written on the dirty-RAM interpreter. Only the code around them
changes backend. Nothing is synthesised.

Tests: runtime/tests/test_kernel_patch_ranges.c (new, the verifier's
accept/reject behaviour and the hand-back predicate) and the install-slot
range invariants in recompiler/tests/bios_address_model_test.cpp.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-11 16:38:34 -04:00
Matthew Stanley fe2c0046ba Recognize bounded MIPS switches with scheduled table constants 2026-09-11 13:38:06 -07:00
Matt Stanley 6f77dcc31f Merge pull request #345 from RetroPortingToolKit/fix/pgo-train-clean-exit
Flush Windows PGO profiles through clean runtime exit
2026-09-10 21:40:19 -07:00
Matthew Stanley 62ca12bd31 Flush PGO profiles through clean runtime exit 2026-09-10 21:40:06 -07:00
Matt Stanley 7d23cedb8d Merge pull request #344 from RetroPortingToolKit/pe-feedback
Fix Clang overlay exports, PGO, and overlay-cache scaffold
2026-09-10 17:52:02 -07:00
Matthew Stanley 2113defcfc Fix overlay Clang exports, PGO, and scaffold cache 2026-09-10 17:23:03 -07:00
Alexandros Mandravillis c2f9c3d59a setup: detect any recompiled BIOS stem and stop the first-run setup loop
Focused port of the fork's fix/setup-host-bios-deadlock feature (10 commits) onto current upstream master, per the 2026-09-10 decision to port rather than rebase (the fork line and upstream share no merge base).

Content: the codegen host accepts any recompiled BIOS stem instead of only OpenBIOS/SCPH1001 (psx_bios_known_images.h); the runtime checks the setup host before validating a chosen BIOS and the first-run loop breaker reports a cause-specific reason; the CLI decodes subprocess output as UTF-8; --setup-selfcheck verifies a headless setup; the rbengine preflight names the recursive init. Two port conflicts resolved: the new CTest registrations in recompiler/CMakeLists.txt are kept (the names exist only in this change), and runtime.cmake takes the new recomp-ui guidance text. Cubic P3 finding fixed: the sibling PSX_NETPLAY FATAL_ERROR now also names --recursive.

Evidence: python syntax checks pass for psxrecomp_cli.py and the new tests; runtime/tests/test_codegen_host_bios_stems.py runs or skips only on a missing RECOMP_UI_ROOT. No full runtime build or retail route is claimed here. Developed with AI assistance (Cline, 2026-09-10).
2026-09-11 01:26:22 +03:00
Matt Stanley 3d10366fba Merge pull request #341 from RetroPortingToolKit/chore/retro-rename
Rename the product to Retro, and follow recomp-ui to the org
2026-09-10 13:52:01 -07:00
Matt Stanley ed55299be3 Merge pull request #342 from RetroPortingToolKit/license/polyform-org
Ship third-party notices with releases; name the licensing contact
2026-09-09 14:59:15 -07:00
Matthew Stanley 2a909c6017 Ship the third-party notices with releases; add the maintainer notice
libchdr (BSD-3-Clause) and toml11 (MIT) are statically linked into the
runtime and both require the notice in binary redistributions;
runtime/licenses/ existed but nothing copied it into a package.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-09 14:55:17 -07:00
Alex Vanderveen 922e8c5a07 Rename the product from RetComM to Retro
The launcher and studio ship as Retro Launcher and Retro Studio now,
from the RetroPortingToolKit org, so the old product name is gone from
display text, documentation and release asset filenames.

The split is by case, and it is not cosmetic. CamelCase RetComM was
always the product; lowercase retcomm and RETCOMM_ are identifiers --
the C++ namespace, the binaries, the data directories, the CMake and
environment variables, the Cloudflare service names, the readme markers.
Renaming those would orphan existing installs for no benefit, so they
are untouched: 1101 lowercase retcomm and 205 RETCOMM_ still stand in
the launcher alone.

Four things are deliberately still spelled RetComM, and three of them
would have been real breakage:

- RetComM-Data is CamelCase but it is a directory on disk --
  <exe_dir>\RetComM-Data\{runtime,config,data} for a portable install.
  Renaming it strands every portable user's config, toolchains and
  catalog cache.

- TechnicallyComputers/RetComM-Launcher in readme_has_launcher() is the
  historical half of a both-slugs test. Renaming it makes it a duplicate
  of the line above and sends every port built before the transfer back
  to reporting its launcher section missing.

- _LEGACY_LAUNCHER_RE matched "^## RetComM Launcher". A sweep renamed it,
  which would have silently stopped it matching every README already out
  there; it now accepts both headings.

- The portable-stub lookup gained the pre-rename filenames back as
  fallbacks, because old zips stay on the releases page and a rollback
  downloads one.

Release assets become Retro-Launcher-* / Retro-Studio-*. That is safe
for installed clients: pick_launcher_asset() gates on *linux*/*windows*/
*macos* plus extension and arch, not on the product name, and the
Windows portable stub is identified by its RCM1 trailer rather than its
filename.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013DpaQqVguHFpBuCmdoFYWY
2026-09-09 17:32:19 -04:00
Alex Vanderveen cf9648e7ea Rename the product from RetComM to Retro
The launcher and studio ship as Retro Launcher and Retro Studio now,
from the RetroPortingToolKit org, so the old product name is gone from
display text, documentation and release asset filenames.

The split is by case, and it is not cosmetic. CamelCase RetComM was
always the product; lowercase retcomm and RETCOMM_ are identifiers --
the C++ namespace, the binaries, the data directories, the CMake and
environment variables, the Cloudflare service names, the readme markers.
Renaming those would orphan existing installs for no benefit, so they
are untouched: 1101 lowercase retcomm and 205 RETCOMM_ still stand in
the launcher alone.

Four things are deliberately still spelled RetComM, and three of them
would have been real breakage:

- RetComM-Data is CamelCase but it is a directory on disk --
  <exe_dir>\RetComM-Data\{runtime,config,data} for a portable install.
  Renaming it strands every portable user's config, toolchains and
  catalog cache.

- TechnicallyComputers/RetComM-Launcher in readme_has_launcher() is the
  historical half of a both-slugs test. Renaming it makes it a duplicate
  of the line above and sends every port built before the transfer back
  to reporting its launcher section missing.

- _LEGACY_LAUNCHER_RE matched "^## RetComM Launcher". A sweep renamed it,
  which would have silently stopped it matching every README already out
  there; it now accepts both headings.

- The portable-stub lookup gained the pre-rename filenames back as
  fallbacks, because old zips stay on the releases page and a rollback
  downloads one.

Release assets become Retro-Launcher-* / Retro-Studio-*. That is safe
for installed clients: pick_launcher_asset() gates on *linux*/*windows*/
*macos* plus extension and arch, not on the product name, and the
Windows portable stub is identified by its RCM1 trailer rather than its
filename.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013DpaQqVguHFpBuCmdoFYWY
2026-09-09 17:29:48 -04:00
Matt Stanley 2079e65eda Correct copyright holder to Matthew Stanley
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-09 13:30:08 -07:00
Alex Vanderveen 1a64f61153 Merge pull request #340 from mstan/chore/retroportingtoolkit-rename
Follow the RetroPortingToolKit transfer in submodule URLs and tooling
2026-09-09 00:48:39 -04:00
Alex Vanderveen 3b5bf48305 Follow the RetroPortingToolKit transfer in submodule URLs and tooling
recomp-net, retcomm-rbengine (now rbengine), retcomm-toolchains (now
RetroPorting-Toolchains), the catalog and the launcher moved from
TechnicallyComputers to the RetroPortingToolKit org. GitHub still
redirects the old repo URLs, so nothing is broken today — but this repo
is what every port inherits its .gitmodules from, so the old owner
propagates into each new clone until it is fixed here.

.gitmodules now names the new URLs. The toolchain repo is updated in the
CI action default, the codegen host, fetch_toolchain.sh, toolchain_pack
and package_setup_host, and the scaffolder's recomp-net / rbengine
defaults follow.

Two things the mechanical sweep would have missed:

- render_launcher_block() held a second copy of the launcher slug split
  across two string literals, so a search for the full slug did not see
  it and every regenerated README kept the dead owner. It is now derived
  from LAUNCHER_REPO, which is the only place the slug lives.

- readme_has_launcher() matched one exact slug. Pointed at the new one
  alone it would report every port built before the transfer as missing
  its launcher section, and append a second copy to any README carrying
  neither the markers nor a "## RetComM Launcher" heading. It now
  accepts both.

The scaffolder's default GitHub owner for new ports becomes
RetroPortingToolKit. Existing ports are unaffected — they carry their
own owner in the README and remote.

psxrecomp, snesrecomp and recomp-ui did not move and still point at
mstan. Submodule paths (lib/recomp-net, lib/retcomm-rbengine) are
unchanged: they are pinned by every port and are not repo slugs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018xfKkiEkh87zL1EpE5Ty55
2026-09-09 00:39:14 -04:00