Commit Graph

1253 Commits

Author SHA1 Message Date
Matt Stanley 1a0897ca50 Merge pull request #373 from Alexbeav/up/autocompile-posix
runtime: spawn overlay autocompile on POSIX hosts
2026-09-17 14:10:45 -05:00
Matt Stanley 2697a78846 Merge pull request #376 from Alexbeav/up/toolchain-staging
cli: stage the overlay toolchain beside every built product
2026-09-17 14:10:42 -05:00
Matt Stanley b1d8d4cead Merge pull request #374 from Alexbeav/up/frame-report
runtime: report frame rate from every product, not just debug builds
2026-09-17 14:10:28 -05:00
Matt Stanley ba5a70d5de Merge pull request #375 from Alexbeav/up/launcher-dialogs
runtime: show launcher warnings through SDL on every platform
2026-09-17 14:10:25 -05:00
Matt Stanley 234bfc4dac Merge pull request #378 from Alexbeav/up/freeze-mdec
runtime: do not classify MDEC-active stillness as a spin freeze
2026-09-17 14:10:21 -05:00
Alexandros Mandravillis 59f17edf6a runtime: do not classify MDEC-active stillness as a spin freeze
During a boot FMV the guest parks in BIOS MDEC code with the dirty-RAM counters
legitimately still while frames advance, which is exactly the spin signature.
Every boot FMV wrote a 47-76 MB dump, 1.8 GB per kit in one session, inside
timed windows. MDEC activity within the window now rules the classification
out. Verified on NCII: heartbeat live through the FMV, zero dumps.

(cherry picked from commit bf60c128218949e3e3af7ea867d50cc7a1e8a774)
2026-09-16 20:00:33 +03:00
Alexandros Mandravillis 1da867ea99 runtime: recompute the text-modified bitmap after a state restore
dirty_ram_text_guard_resync_after_restore() cleared text_modified_bitmap,
and the page-clean fast path in dirty_ram_text_native_ok_ranges_from()
then trusted a clear bit as "bytes still equal the reference" and skipped
the compare. Restored RAM may carry self-modified text (the CMR2 class)
that no live write will ever re-flag: the bitmap is not part of the state
image, and the writes happened in the process that produced it. After a
save-state load into a fresh process, a stale native body could be
admitted for those pages.

Recompute the bitmap from the bytes: one pass over the reference range (at
most 2 MB, once per restore), setting the bit for every page that differs,
so those pages stay under the exact compare.

(cherry picked from commit ab186edf6, memory.c part)
2026-09-16 20:00:33 +03:00
Alexandros Mandravillis c10bad2ebb cli: stage the overlay toolchain beside every built product
No setup package ever staged one, so overlay_cache = true alone left every
streamed overlay on the interpreter. rebuild now stages the pinned Python,
tcc, this pin's recompiler, headers and the build's codegen hash beside the
normal and diagnostic products, and writes compiler.txt naming the compiler
the wizard already has; the runtime treats that as a gcc-tier compiler, so a
player with nothing on PATH gets -O2 shards. Verified on the NCII kit: gcc
tier reported and four shards built with no compiler on PATH.

(cherry picked from commit 03865e10f6ee198d58c47c74526df5db46bb35a9)
2026-09-16 19:59:17 +03:00
Alexandros Mandravillis 6451508633 runtime: spawn overlay autocompile on POSIX hosts
autocompile_request() returned 0 on every non-Windows host, so a Linux or
macOS player never had a shard compiled at runtime: captures were written,
the packager staged a working toolchain, the boot log said "overlay
autocompile enabled", and every overlay ran interpreted for the whole
session (Parasite Eve on Rocky 9: 1.35e9 interpreted instructions by frame
13k with /usr/bin/gcc beside the exe).

Mirror the Windows spawner with fork/exec: /bin/sh -c runs the configured
command in its own process group at nice 10, stdout+stderr flow through one
pipe into the same tail ring (streaming PSX_SHARD_RESULT parse), a pthread
watcher drains the pipe and reaps the child, and the emulation thread's poll
turns DONE into the one idempotent cache rescan. No PSX_SHARD_PUBLISHED
handoff off Windows: the rescan is the fallback the Windows path already
takes when a marker is cut off. Shutdown SIGTERMs the group and joins;
PR_SET_PDEATHSIG on Linux plus the dying pipe cover a runtime that never
reaches shutdown. The PSX_OVERLAY_* pins are set once at configure time
(glibc setenv is not safe against a concurrent getenv). Configure names the
interpreter it will run and marks the run degraded when the path is not
executable. Link Threads unconditionally on Unix.

The child-line buffer, shard_result_values, shard_result_line_locked and
publish_parse_locked move out of the Windows block; child_line_locked is
per-platform. Both Windows unit tests still pass.

Verified on Rocky 9 (PE diagnostic product): 0 -> 68 shards in 150 s,
interpreted instructions by frame 14.5k 1.35e9 -> 0.39e9, curve flat from
frame ~4k, counters visible over the debug server, quit and SIGKILL
mid-compile leave no orphans or zombies.

(cherry picked from commit 854540c9c96e86b0cc1eea0dd99ef285169784d9)
2026-09-16 19:56:55 +03:00
Alexandros Mandravillis 09a59af86c runtime: report frame rate from every product, not just debug builds
s_frame_count is already maintained under PSX_NO_DEBUG_TOOLS, but everything
that exposes it — the debug server, the freeze heartbeat, screenshots — is
compiled out of production. A shipped binary could not report its own speed, so
throughput could only be measured on the diagnostic product, which carries
instrumentation of its own and is not what players run.

That gap is why a 30% throughput regression reached a release candidate
unmeasured, and why the first four attempts to attribute it were all made
against the wrong binary.

Adds an opt-in periodic line on stdout, outside every debug guard:

  PSX_FRAME_REPORT_MS=2000
  psxrecomp: frames=1234 elapsed_ms=20000 fps=61.7 avg_fps=61.7

Unset, it is one branch on a cached int per vblank. Line-oriented on stdout so a
harness can read it without a debug connection.

(cherry picked from commit aac4dbf58331a413b4ac8be6890c7ff499c55ca0)
(cherry picked from commit c1fbb313d15fe00256bc68a805ba0df255feb249)
2026-09-16 19:56:55 +03:00
Alexandros Mandravillis 3b541c681f runtime: show launcher warnings through SDL on every platform
launcher_warning/launcher_info only ever raised a dialog under _WIN32, so on
macOS and Linux a first run with no BIOS printed to a terminal nobody was
looking at and quit. SDL3 is already linked; SDL_ShowSimpleMessageBox covers
all three platforms and keeps the headless guard.

(cherry picked from commit 22f4322eb33d46946f9b953f943bedf070eb75a6)
2026-09-16 19:56:55 +03:00
Alexandros Mandravillis 6acf1976d2 runtime: refuse an over-long overlay autocompile command instead of truncating it
s_cmd was 4096 bytes; a 208-address --force-interior list produced a 6392-char
command, snprintf cut it mid-argument, every compile failed the same way and
autocompile latched degraded for the process lifetime. Buffer is now 8192
(cmd.exe's own ceiling), the cmd.exe wrapper is sized from it, and an over-long
command is reported and disabled loudly.

(cherry picked from commit b0fd04076ca4b0cc7f8d813bc6f750f94df30bf9)
2026-09-16 19:56:54 +03:00
Pegasus 5968162fe9 fix: publish deferred CPU work before GTE deadline checks
(cherry picked from commit 43abd4a5c8)
2026-09-14 17:26:34 +03:00
Pegasus 77a8480363 Read disc region from the actual SYSTEM.CNF directory entry
Prefer the ISO9660 boot file over the bounded raw-image scan. Biohazard stores SYSTEM.CNF outside that scan, leaving fresh builds at the default SCEA response. Add raw and cooked late-file regressions with an early decoy serial. The full-profile prefix again matches the retained native baseline through 5999 returns; the independent-source frontier remains return 464.

(cherry picked from commit 82c3de13ef)
2026-09-14 17:26:33 +03:00
Pegasus 324690cb97 Preserve fully qualified Windows network asset paths
(cherry picked from commit 51d5d56a27)
2026-09-14 17:26:32 +03:00
Pegasus 5e23de8393 feat: add PS1 conformance harness and fix GTE fidelity
(cherry picked from commit 98d617b5f3)
2026-09-14 17:26:31 +03:00
Alexandros Mandravillis d4abc2f05d Fix non-RA JALR interpreter handoff
(cherry picked from commit 2f8f627b24)
2026-09-14 17:26:30 +03:00
Matthew Stanley 8b689140dd Add opt-in world scene classification and immutable terrain batching 2026-09-13 21:06:58 -07:00
Matthew Stanley b4d41b173c Merge current master and preserve both runtime regression test groups 2026-09-13 19:40:33 -07:00
Matthew Stanley f2352e3d32 Keep retained scene aspect until the next displayed buffer flip 2026-09-13 19:27:03 -07:00
Matthew Stanley e3beb365ba Add opt-in native-wide presentation hold for retained loading scenes 2026-09-13 19:11:21 -07:00
Alexandros Mandravillis 693c7be0fc Fix GPU upload history overflow after retention fills (#360)
Co-authored-by: Pegasus <pegasus@psxrecomp.local>
2026-09-13 17:01:55 -07:00
Matthew Stanley 9037e7bd23 Add opt-in retained texture banks and enhancement memory snapshots 2026-09-13 16:50:26 -07:00
Matt Stanley d4c45adf62 fix(sio): remove title-specific card IRQ repair and opt-in (#362) 2026-09-13 16:33:46 -07:00
Matthew Stanley 7025bb5f88 Fix rewind keyboard load conflicting with cancel aliases 2026-09-13 14:16:20 -07:00
Aliet Exposito Garcia ab2aaf95be fix(runtime): fix compilation failure when netplay is disabled
When `PSX_NETPLAY` is turned off, the build fails because `ae_np_account_sync` is referenced without being declared in that configuration. This issue caused automated continuous integration pipelines and local builds to fail unless netplay was explicitly forced on.

Wrapping this function call ensures that the runtime compiles cleanly for projects that do not require netplay support, restoring the ability to build with netplay disabled.
2026-09-13 13:33:16 -04:00
Matt Stanley 0690e5c164 Merge pull request #357 from RetroPortingToolKit/rfc/fmv-irq-batching-invariants
RFC: FMV IRQ batching correctness gates and counterexample
2026-09-13 00:30:51 -07:00
Matt Stanley 0b898494c8 Merge pull request #356 from RetroPortingToolKit/experiment/runtime-ipo-validation
experiment(build): opt-in runtime IPO and PGO validation gates
2026-09-13 00:30:23 -07:00
Matt Stanley bd01da696a Merge pull request #353 from plinkr/feat/gl-renderer-post-gamma
feat(OpenGL renderer): add post presentation gamma control
2026-09-13 00:29:57 -07:00
Matthew Stanley 20ec948ab1 build: add opt-in runtime IPO experiment with validation gates 2026-09-12 23:31:28 -07:00
Matthew Stanley 847c2baf37 docs: stage FMV IRQ batching RFC and cache-state counterexample 2026-09-12 23:30:41 -07:00
Matthew Stanley ce7f646f42 fix(runtime): keep chat-report dependency inside enabled lobby path 2026-09-12 23:10:42 -07:00
Matthew Stanley fa655333fb fix(runtime): guard optional netplay authentication dependencies 2026-09-12 23:06:05 -07:00
Aliet Exposito Garcia 72b3ead892 feat(OpenGL renderer): add post presentation gamma control
Some titles and enhancement mods/plugins might require display level gamma correction to improve shadow detail and overall visibility without altering guest rendering state. This introduces a public gamma adjustment interface for the OpenGL renderer via `gl_renderer_set_post_gamma`, that enables external plugins to adjust the display curve cleanly at runtime.

This adjustment applies exclusively to game content and temporal interpolation passes while leaving host interface elements, bezels, and previously composed frames untouched. Modifying the gamma setting immediately invalidates the presentation buffer so that changes become visible on screen without requiring new video memory writes from the guest system.
2026-09-13 01:30:10 -04:00
Alex Vanderveen 3a275d4939 launcher: Allow Spectators is a preference, not an in-room setting
The toggle lives in the Create Lobby modal, before any room exists, and
its value goes out with `create`. The setter refused unless the client
was already seated in a server lobby, so the modal read the unchanged
preference back every frame and the checkbox snapped off. Refuse only in
a LAN / Direct-IP room, which has no gallery to allow -- the SNES host's
rule.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FPXfGGhBBBxQPcEzaMsauF
2026-09-12 20:54:12 -04:00
Alex Vanderveen b9ca600778 netplay: block list, account keys, list scope and automatch for the launcher
Brings the PSX runtime level with what recomp-ui's netplay pages now ask a
host for, and with the SNES runner that already answers them
(snes_lobby_client.c / snes_host_lobby.c):

- Chat lines, seat rows and players-online rows carry the server's opaque
  account key, so the launcher's ignore/block list survives a rename.
- psx_lobby_set_blocks pushes that list to the server as `set_blocks`, since
  refusing to pair or seat the two together is the server's job; recomp-ui
  re-sends it on every reconnect.
- list_scope_set: a LAN player no longer sees online rooms they have no
  connection for, and an online player no longer sees their own LAN rows.
- Automatch: rulesets discovery, the relay latency probe (14-byte LE frame,
  type 200 -> 201, nonce-matched), queue with the disc fingerprint, the
  accept gate with a locally counted deadline, requeue/cancel/cooldown, and
  claiming automatch's `error` codes only while a ticket is in flight.
  Leaving an automatch room after the match: it is the server's room, has
  no host to rematch with, and holding the seat blocks the next ticket.
- Account: init the auth client once per URL rather than every pump (the
  login worker reads the host), anchor the device key to the executable
  directory, and publish the signed-in handle as the lobby display name.

mods_enabled is asserted false on PSX: every netplay launch clears the mod
plan or refuses to start, and the caps a match runs are the ruleset's.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FPXfGGhBBBxQPcEzaMsauF
2026-09-12 19:39:09 -04:00
Alex Vanderveen 615d35fe22 Merge remote-tracking branch 'origin/master' into feat/lobby-parity
# Conflicts:
#	tools/new_project_layout/project_studio/readme_metrics.py
2026-09-12 19:30:47 -04:00
Alex Vanderveen fddc8b8516 netplay: report chat lines for moderation, and pin recomp-net for it
PsxLobbyChatMsg gains the server's `mid` for a line, taken off `chat` and
`server_chat`, and psx_lobby_report_chat hands a list of them to
rnet_chat_report_build. The frame carries ids and never the text, so a
report cannot fabricate what somebody said. Same eight lines as the SNES
copy; what a report contains lives in recomp-net.

lib/recomp-net -> 859ba28 (support chat reporting), which is also where
rnet_account_set_secret_path comes from.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FPXfGGhBBBxQPcEzaMsauF
2026-09-12 19:29:57 -04:00
Matthew Stanley 9576c6323f Refresh combined BIOS and dispatch validation against latest master 2026-09-11 23:39:47 -07:00
Matthew Stanley fae85cc886 Repair Windows and stale regression test harness assumptions 2026-09-11 23:39:47 -07:00
Matthew Stanley 63b205df78 Index immutable resident dispatch keys and clarify owner telemetry 2026-09-11 23:39:46 -07:00
Matthew Stanley fd84134c19 Guard patched BIOS ranges before control flow and cycle charging 2026-09-11 23:39:33 -07:00
Matthew Stanley ac0e40628a Match BIOS setup detection to configured linkable backends 2026-09-11 23:39:11 -07:00
Matthew Stanley b4ea4c37e9 Expand opt-in DMA arenas without aliasing CPU BIOS addresses 2026-09-11 23:05:07 -07:00
Matthew Stanley c5390e4265 Merge kernel patch ranges PR #346 for combined validation 2026-09-11 22:53:07 -07:00
Matthew Stanley 9550b3e579 Merge BIOS setup-host PR #343 for combined validation 2026-09-11 22:53:05 -07:00
Matthew Stanley 06a93dc8aa Forward widescreen hooks in native overlays and verify fallback intervals 2026-09-11 22:28:03 -07:00
Kevin 3cbe6c1cde kernel-bless: publish install-slot ranges so patched BIOS bodies run native
The BIOS exception handler ran interpreted for the life of every process,
1.22 billion instructions and 44.6% of all interpreted work in Breath of
Fire III, even though it is compiled and even though the stub that
unblessed it sat at exactly the address bios/SCPH1001.toml already
declared as an install slot. Two mechanisms existed and did not compose:
the emitter hooked the declared slot PC, while the runtime's bless
verifier memcmp'd the whole body including that slot, so the body was
MISMATCH forever and native code never reached the hook.

Three parts, none of which pays off alone:

1. Publish the declared ranges to the runtime. The emitter emits a
   PsxKernelPatchRange table next to the kernel body table and routes it
   through the backend descriptor; the bless verifier compares a body in
   segments that skip them. CLEAN now means "every byte outside every
   declared range still matches ROM", which is the claim the native code
   actually depends on.

2. Generalise the slot model to what the patches are. The old model
   assumed a 4-word lui/addiu/jalr/nop stub written over ROM zeros and
   fired on `word != 0`. Measured against both images, only one of five
   observed shapes fits: patchers also plant a `jr` over real ROM
   instructions, rewrite an 11-word prologue and insert an `mfc0 Cause`,
   and NOP out a driver routine. A slot is now a range with a length and
   an explicit `resume` (jalr / fallthrough / none), detected by
   comparing every word against its ROM-baked value.

3. Hand back from the interpreter at the range end. Kernel page 0 is
   permanently dirty, because the handler saves registers there on every
   exception, so straight-line interpretation would run the whole
   function after the patch. The emitter registers the resume PC as a
   continuation key and dirty_ram_interp surfaces there.

A PC inside a declared range must never be a native dispatch key: the
compiled bytes there are not what executes. Seven pre-existing
jal-return continuations violated that and wedged the boot at frame 0 in
an infinite dispatch loop (OpenBIOS RAM 0x357C). The emitter now drops
continuation keys inside a range so dispatch misses through to the
interpreter, and refuses to emit if a range covers a function entry,
which would silently remove a callable function instead.

Both profiles declare their measured ranges, and SCPH1001.toml finally
pins [program.image] sha256 to v2.2 (CRC32 37157331) so a v2.0 dump
refuses to generate rather than producing phantom discovery gaps.

Measured on Breath of Fire III (SLPS-00990, Psy-Q SDK 3.70), headless,
matched frames, PSX_KERNEL_PATCH_RANGES=0 for the OFF side:

                      OpenBIOS            SCPH-1001 v2.2
  bless mismatch      21 -> 0             74 -> 0
  kernel bodies       -87.7%/frame        -97.8%/frame
  all interp insns    -42.6%/frame        -94.2%/frame

The residual kernel-body work is exactly declared-words x entries on
every range, so nothing leaks past the hook. The A0/B0/C0 vector
trampolines are unchanged (+0.2%, noise): the profile excludes them by
design and they are a separate job. Card read traces are byte-identical
across the A/B on both images, all 32 entries, which exercises the two
ranges that replace the kernel's card handler.

Rule 18 holds throughout: the guest's patched instructions still execute
as written on the dirty-RAM interpreter. Only the code around them
changes backend. Nothing is synthesised.

Tests: runtime/tests/test_kernel_patch_ranges.c (new, the verifier's
accept/reject behaviour and the hand-back predicate) and the install-slot
range invariants in recompiler/tests/bios_address_model_test.cpp.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-11 16:38:34 -04:00
Matthew Stanley 2113defcfc Fix overlay Clang exports, PGO, and scaffold cache 2026-09-10 17:23:03 -07:00
Alexandros Mandravillis c2f9c3d59a setup: detect any recompiled BIOS stem and stop the first-run setup loop
Focused port of the fork's fix/setup-host-bios-deadlock feature (10 commits) onto current upstream master, per the 2026-09-10 decision to port rather than rebase (the fork line and upstream share no merge base).

Content: the codegen host accepts any recompiled BIOS stem instead of only OpenBIOS/SCPH1001 (psx_bios_known_images.h); the runtime checks the setup host before validating a chosen BIOS and the first-run loop breaker reports a cause-specific reason; the CLI decodes subprocess output as UTF-8; --setup-selfcheck verifies a headless setup; the rbengine preflight names the recursive init. Two port conflicts resolved: the new CTest registrations in recompiler/CMakeLists.txt are kept (the names exist only in this change), and runtime.cmake takes the new recomp-ui guidance text. Cubic P3 finding fixed: the sibling PSX_NETPLAY FATAL_ERROR now also names --recursive.

Evidence: python syntax checks pass for psxrecomp_cli.py and the new tests; runtime/tests/test_codegen_host_bios_stems.py runs or skips only on a missing RECOMP_UI_ROOT. No full runtime build or retail route is claimed here. Developed with AI assistance (Cline, 2026-09-10).
2026-09-11 01:26:22 +03:00