Commit Graph

1999 Commits

Author SHA1 Message Date
Alex Vanderveen 32831ff758 Update package_setup_host.sh 2026-09-18 19:38:53 -04:00
Matt Stanley 1a0897ca50 Merge pull request #373 from Alexbeav/up/autocompile-posix
runtime: spawn overlay autocompile on POSIX hosts
2026-09-17 14:10:45 -05:00
Matt Stanley 2697a78846 Merge pull request #376 from Alexbeav/up/toolchain-staging
cli: stage the overlay toolchain beside every built product
2026-09-17 14:10:42 -05:00
Matt Stanley baf8023dc6 Merge pull request #370 from Alexbeav/feat/chd-python-reader
setup: read .chd through libchdr in the Python disc tools
2026-09-17 14:10:38 -05:00
Matt Stanley 1f9a11d3b8 Merge pull request #367 from kerokline/fix/static-overlay-reconcile-cache-dir
compile_overlays: only reconcile empty primary scans in DLL mode
2026-09-17 14:10:35 -05:00
Matt Stanley 572cc5ed2d Merge pull request #377 from Alexbeav/up/packaging-gate
packaging: gate overlay_cache = true and report it from --setup-selfcheck
2026-09-17 14:10:31 -05:00
Matt Stanley b1d8d4cead Merge pull request #374 from Alexbeav/up/frame-report
runtime: report frame rate from every product, not just debug builds
2026-09-17 14:10:28 -05:00
Matt Stanley ba5a70d5de Merge pull request #375 from Alexbeav/up/launcher-dialogs
runtime: show launcher warnings through SDL on every platform
2026-09-17 14:10:25 -05:00
Matt Stanley 234bfc4dac Merge pull request #378 from Alexbeav/up/freeze-mdec
runtime: do not classify MDEC-active stillness as a spin freeze
2026-09-17 14:10:21 -05:00
Matt Stanley 5bde8b8714 Merge pull request #379 from Alexbeav/up/text-restore-recompute
runtime: recompute the text-modified bitmap after a state restore
2026-09-17 14:10:17 -05:00
Alexandros Mandravillis d2e2c92887 setup: keep the CLI's reason when disc verification fails
cli_fail_msg replaced whatever the CLI said on exit 3 with a flat
"Disc verification failed (wrong dump)." With the CHD reader, the CLI's
own line can be "... is a CHD and the CHD reader (libchdr) is not built
for this kit ... this does not mean your dump is bad", and the wizard then
contradicted it. cli_tail_note already captures that line; show it, and
fall back to the old text only when the CLI said nothing.

Carried over from #369, which was closed in favour of this change.
2026-09-16 21:20:05 +03:00
Alexandros Mandravillis b53ab4bd41 release_stage: pin the overlay toolchain interpreter for macOS
TOOLCHAIN_PINS had entries for win and linux only, and stage_toolchain
picked 'linux' for every non-Windows host, so a Mac either died with "no
toolchain pins" or staged a Linux interpreter it cannot run. No Mac player
ever got a bundled toolchain. The stock /usr/bin/python3 on macOS 13/14 is
3.9, which cannot run compile_overlays.py (tomllib, PEP 604 unions), so the
system interpreter is no fallback either.

Add macos-x64 and macos-arm64 pins: the same python-build-standalone release
and CPython 3.13.1 as the Linux pin, install_only_stripped assets, SHA256
measured by downloading both and matched against the publisher's .sha256
sidecars. host_platform_tag() picks the key from sys.platform and the
machine architecture. No prebuilt tcc on macOS: the bundled interpreter
drives the system clang.

Verified on an Intel Mac (macOS 13, 8 GB): toolchain staged beside the
Parasite Eve diagnostic product (45 MB), and with the POSIX spawner the
runtime compiled 9 shards in the first 24 s of a headless boot.

(cherry picked from commit 5c49e4d3cf835ed026964cd24998907c41a8be4c)
2026-09-16 20:35:29 +03:00
Alexandros Mandravillis 59f17edf6a runtime: do not classify MDEC-active stillness as a spin freeze
During a boot FMV the guest parks in BIOS MDEC code with the dirty-RAM counters
legitimately still while frames advance, which is exactly the spin signature.
Every boot FMV wrote a 47-76 MB dump, 1.8 GB per kit in one session, inside
timed windows. MDEC activity within the window now rules the classification
out. Verified on NCII: heartbeat live through the FMV, zero dumps.

(cherry picked from commit bf60c128218949e3e3af7ea867d50cc7a1e8a774)
2026-09-16 20:00:33 +03:00
Alexandros Mandravillis 1da867ea99 runtime: recompute the text-modified bitmap after a state restore
dirty_ram_text_guard_resync_after_restore() cleared text_modified_bitmap,
and the page-clean fast path in dirty_ram_text_native_ok_ranges_from()
then trusted a clear bit as "bytes still equal the reference" and skipped
the compare. Restored RAM may carry self-modified text (the CMR2 class)
that no live write will ever re-flag: the bitmap is not part of the state
image, and the writes happened in the process that produced it. After a
save-state load into a fresh process, a stale native body could be
admitted for those pages.

Recompute the bitmap from the bytes: one pass over the reference range (at
most 2 MB, once per restore), setting the bit for every page that differs,
so those pages stay under the exact compare.

(cherry picked from commit ab186edf6, memory.c part)
2026-09-16 20:00:33 +03:00
Alexandros Mandravillis 68ff52c14a packaging: gate overlay_cache = true and report it from --setup-selfcheck
78 of 105 published releases lack the key, and the runtime never initialises
the overlay loader without it. The packager now refuses a staged game.toml
without it unless --ship-without-overlay-cache-because states why, and the
setup host's self-check reports overlay_cache_configured so CI can assert it.

(cherry picked from commit a030f4843d8e55eafed4cb42eafc6d74bb89c086)
2026-09-16 19:59:56 +03:00
Alexandros Mandravillis c10bad2ebb cli: stage the overlay toolchain beside every built product
No setup package ever staged one, so overlay_cache = true alone left every
streamed overlay on the interpreter. rebuild now stages the pinned Python,
tcc, this pin's recompiler, headers and the build's codegen hash beside the
normal and diagnostic products, and writes compiler.txt naming the compiler
the wizard already has; the runtime treats that as a gcc-tier compiler, so a
player with nothing on PATH gets -O2 shards. Verified on the NCII kit: gcc
tier reported and four shards built with no compiler on PATH.

(cherry picked from commit 03865e10f6ee198d58c47c74526df5db46bb35a9)
2026-09-16 19:59:17 +03:00
Alexandros Mandravillis 6451508633 runtime: spawn overlay autocompile on POSIX hosts
autocompile_request() returned 0 on every non-Windows host, so a Linux or
macOS player never had a shard compiled at runtime: captures were written,
the packager staged a working toolchain, the boot log said "overlay
autocompile enabled", and every overlay ran interpreted for the whole
session (Parasite Eve on Rocky 9: 1.35e9 interpreted instructions by frame
13k with /usr/bin/gcc beside the exe).

Mirror the Windows spawner with fork/exec: /bin/sh -c runs the configured
command in its own process group at nice 10, stdout+stderr flow through one
pipe into the same tail ring (streaming PSX_SHARD_RESULT parse), a pthread
watcher drains the pipe and reaps the child, and the emulation thread's poll
turns DONE into the one idempotent cache rescan. No PSX_SHARD_PUBLISHED
handoff off Windows: the rescan is the fallback the Windows path already
takes when a marker is cut off. Shutdown SIGTERMs the group and joins;
PR_SET_PDEATHSIG on Linux plus the dying pipe cover a runtime that never
reaches shutdown. The PSX_OVERLAY_* pins are set once at configure time
(glibc setenv is not safe against a concurrent getenv). Configure names the
interpreter it will run and marks the run degraded when the path is not
executable. Link Threads unconditionally on Unix.

The child-line buffer, shard_result_values, shard_result_line_locked and
publish_parse_locked move out of the Windows block; child_line_locked is
per-platform. Both Windows unit tests still pass.

Verified on Rocky 9 (PE diagnostic product): 0 -> 68 shards in 150 s,
interpreted instructions by frame 14.5k 1.35e9 -> 0.39e9, curve flat from
frame ~4k, counters visible over the debug server, quit and SIGKILL
mid-compile leave no orphans or zombies.

(cherry picked from commit 854540c9c96e86b0cc1eea0dd99ef285169784d9)
2026-09-16 19:56:55 +03:00
Alexandros Mandravillis 09a59af86c runtime: report frame rate from every product, not just debug builds
s_frame_count is already maintained under PSX_NO_DEBUG_TOOLS, but everything
that exposes it — the debug server, the freeze heartbeat, screenshots — is
compiled out of production. A shipped binary could not report its own speed, so
throughput could only be measured on the diagnostic product, which carries
instrumentation of its own and is not what players run.

That gap is why a 30% throughput regression reached a release candidate
unmeasured, and why the first four attempts to attribute it were all made
against the wrong binary.

Adds an opt-in periodic line on stdout, outside every debug guard:

  PSX_FRAME_REPORT_MS=2000
  psxrecomp: frames=1234 elapsed_ms=20000 fps=61.7 avg_fps=61.7

Unset, it is one branch on a cached int per vblank. Line-oriented on stdout so a
harness can read it without a debug connection.

(cherry picked from commit aac4dbf58331a413b4ac8be6890c7ff499c55ca0)
(cherry picked from commit c1fbb313d15fe00256bc68a805ba0df255feb249)
2026-09-16 19:56:55 +03:00
Alexandros Mandravillis 3b541c681f runtime: show launcher warnings through SDL on every platform
launcher_warning/launcher_info only ever raised a dialog under _WIN32, so on
macOS and Linux a first run with no BIOS printed to a terminal nobody was
looking at and quit. SDL3 is already linked; SDL_ShowSimpleMessageBox covers
all three platforms and keeps the headless guard.

(cherry picked from commit 22f4322eb33d46946f9b953f943bedf070eb75a6)
2026-09-16 19:56:55 +03:00
Alexandros Mandravillis 6acf1976d2 runtime: refuse an over-long overlay autocompile command instead of truncating it
s_cmd was 4096 bytes; a 208-address --force-interior list produced a 6392-char
command, snprintf cut it mid-argument, every compile failed the same way and
autocompile latched degraded for the process lifetime. Buffer is now 8192
(cmd.exe's own ceiling), the cmd.exe wrapper is sized from it, and an over-long
command is reported and disabled loudly.

(cherry picked from commit b0fd04076ca4b0cc7f8d813bc6f750f94df30bf9)
2026-09-16 19:56:54 +03:00
Alexandros Mandravillis 8cb24ee19a tests: compare resolved paths in the CHD receipt check 2026-09-16 12:00:02 +03:00
Alexandros Mandravillis d17676a5c7 setup: read .chd through libchdr in the Python disc tools
The runtime mounts a .chd natively and the launcher lists it in the disc
picker, so a player could select one and see a valid disc identity, then have
first-run setup fail: verify-disc, prepare_disc.py and probe_disc.py hash and
parse raw 2352-byte track data, which a CHD stores compressed. Nothing on the
Python side could read it, and telling players to install MAME for chdman is
not an answer.

libchdr is already vendored and compiled for the runtime and the native media
CLI. Build it once more as a shared library in recompiler/ (CMake target chdr,
output next to the emitters, static CRT on Windows so a plain python.exe can
load it) and read the CHD from Python through ctypes. tools/psx_chd.py mirrors
the layout the runtime's iso_reader.cpp uses -- 2448-byte frames, 4-frame
track padding, stored versus virtual pregaps, big-endian audio -- and
reproduces the .bin files a Redump dump of the same disc contains, in either
of Redump's two layouts (one bin per track, or one bin for the disc).

verify-disc hashes those reconstructed track bytes against [prepare_disc]
instead of the container. prepare_disc.py writes the tracks into out_dir in
whichever layout the recorded digests match and then continues down its
normal cue path, so the data track is written once, SBI companions are looked
up next to the CHD, and the receipt names the CHD the player chose.
probe_disc.py extracts to a scratch directory and probes the cue. The CLI
builds target chdr on demand for a kit whose emitters predate it, and falls
back to the extract-it-first message when it cannot.

Verified against two discs from a Redump library converted with chdman: a
single-track MODE2 disc and a four-track disc with stored audio pregaps both
reproduce their catalogued track-1 size, MD5 and SHA-1 exactly; verify-disc
and prepare_disc.py accept the four-track one end to end. Tests build a
synthetic uncompressed v5 CHD around a cooked ISO image, so no retail data is
needed; the decode tests skip with a reason when no libchdr is built.
2026-09-16 11:49:59 +03:00
Kevin 4a792379a1 compile_overlays: only reconcile empty primary scans in DLL mode
3a174fab added reconcile_empty_primary_scans() at the tail of main() and
called it unconditionally with cache_dir. cache_dir is bound only on the
DLL path; --static never assigns it. So every --static run now ends in

    UnboundLocalError: cannot access local variable 'cache_dir' where it
    is not associated with a value

after generated/overlays_static.c and all its translation units have
already been written. The output is complete, but the process exits 1,
and callers that gate on the compiler's exit (axis_b_loop.sh expects 0
or 2) abort before the runtime is rebuilt. Found regenerating Breath of
Fire III against master 193a60b8.

The reconciliation has nothing to do in static mode: _do_capture hands
each capture to static_capture_job and returns before the only
pending_empty_primary.append(), and the check it performs reads the
per-game DLL cache, which static mode does not have. Guard the call on
the non-static path. In static mode, raise instead of silently skipping
if anything was ever queued, so a future change that routes static
captures through the DLL path fails loudly rather than dropping a root.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-15 16:54:28 -04:00
Matt Stanley 193a60b805 Merge pull request #366 from Alexbeav/split/01-standalone-fixes
Standalone fixes from the TAS integration: JALR handoff, conformance harness, UNC paths, disc region, GTE deadline
2026-09-14 12:43:11 -07:00
Pegasus 5968162fe9 fix: publish deferred CPU work before GTE deadline checks
(cherry picked from commit 43abd4a5c8)
2026-09-14 17:26:34 +03:00
Pegasus 77a8480363 Read disc region from the actual SYSTEM.CNF directory entry
Prefer the ISO9660 boot file over the bounded raw-image scan. Biohazard stores SYSTEM.CNF outside that scan, leaving fresh builds at the default SCEA response. Add raw and cooked late-file regressions with an early decoy serial. The full-profile prefix again matches the retained native baseline through 5999 returns; the independent-source frontier remains return 464.

(cherry picked from commit 82c3de13ef)
2026-09-14 17:26:33 +03:00
Pegasus 324690cb97 Preserve fully qualified Windows network asset paths
(cherry picked from commit 51d5d56a27)
2026-09-14 17:26:32 +03:00
Pegasus 5e23de8393 feat: add PS1 conformance harness and fix GTE fidelity
(cherry picked from commit 98d617b5f3)
2026-09-14 17:26:31 +03:00
Alexandros Mandravillis d4abc2f05d Fix non-RA JALR interpreter handoff
(cherry picked from commit 2f8f627b24)
2026-09-14 17:26:30 +03:00
Matt Stanley 4be26f162e Merge pull request #364 from RetroPortingToolKit/codex/crash-intro-widescreen-fw-20260913
Opt-in world scene classification and immutable terrain batching
2026-09-13 21:08:26 -07:00
Matthew Stanley 8b689140dd Add opt-in world scene classification and immutable terrain batching 2026-09-13 21:06:58 -07:00
Matt Stanley 7699cceaa4 Merge pull request #363 from RetroPortingToolKit/codex/crash-host-terrain-20260913
Add opt-in retained-terrain services and loading-scene presentation holds
2026-09-13 19:42:02 -07:00
Matthew Stanley b4d41b173c Merge current master and preserve both runtime regression test groups 2026-09-13 19:40:33 -07:00
Matthew Stanley f2352e3d32 Keep retained scene aspect until the next displayed buffer flip 2026-09-13 19:27:03 -07:00
Matthew Stanley e3beb365ba Add opt-in native-wide presentation hold for retained loading scenes 2026-09-13 19:11:21 -07:00
Alexandros Mandravillis 693c7be0fc Fix GPU upload history overflow after retention fills (#360)
Co-authored-by: Pegasus <pegasus@psxrecomp.local>
2026-09-13 17:01:55 -07:00
Matthew Stanley 9037e7bd23 Add opt-in retained texture banks and enhancement memory snapshots 2026-09-13 16:50:26 -07:00
Matt Stanley d4c45adf62 fix(sio): remove title-specific card IRQ repair and opt-in (#362) 2026-09-13 16:33:46 -07:00
Matthew Stanley 7025bb5f88 Fix rewind keyboard load conflicting with cancel aliases 2026-09-13 14:16:20 -07:00
Alex Vanderveen 85cd26f05c Merge pull request #359 from plinkr/fix/psx-netplay-off-undeclared-ae-np-account-sync
fix(runtime): fix compilation failure when netplay is disabled
2026-09-13 13:41:57 -04:00
Aliet Exposito Garcia ab2aaf95be fix(runtime): fix compilation failure when netplay is disabled
When `PSX_NETPLAY` is turned off, the build fails because `ae_np_account_sync` is referenced without being declared in that configuration. This issue caused automated continuous integration pipelines and local builds to fail unless netplay was explicitly forced on.

Wrapping this function call ensures that the runtime compiles cleanly for projects that do not require netplay support, restoring the ability to build with netplay disabled.
2026-09-13 13:33:16 -04:00
Matt Stanley 0690e5c164 Merge pull request #357 from RetroPortingToolKit/rfc/fmv-irq-batching-invariants
RFC: FMV IRQ batching correctness gates and counterexample
2026-09-13 00:30:51 -07:00
Matt Stanley 0b898494c8 Merge pull request #356 from RetroPortingToolKit/experiment/runtime-ipo-validation
experiment(build): opt-in runtime IPO and PGO validation gates
2026-09-13 00:30:23 -07:00
Matt Stanley bd01da696a Merge pull request #353 from plinkr/feat/gl-renderer-post-gamma
feat(OpenGL renderer): add post presentation gamma control
2026-09-13 00:29:57 -07:00
Matthew Stanley 75012474c7 docs: promote FMV batching RFC from draft 2026-09-13 00:29:05 -07:00
Matthew Stanley 543896dbe7 docs: promote IPO experiment from draft 2026-09-13 00:27:46 -07:00
Matthew Stanley 718f2b227e docs: record IPO-on title validation 2026-09-13 00:21:45 -07:00
Matthew Stanley 20ec948ab1 build: add opt-in runtime IPO experiment with validation gates 2026-09-12 23:31:28 -07:00
Matthew Stanley 847c2baf37 docs: stage FMV IRQ batching RFC and cache-state counterexample 2026-09-12 23:30:41 -07:00
Matt Stanley ee1d72543c Merge pull request #354 from RetroPortingToolKit/fix/cfg-metadata-integrity
fix(recompiler): repair live CFG metadata and natural back edges
2026-09-12 23:30:17 -07:00