Commit Graph

196 Commits

Author SHA1 Message Date
Pegasus 5e23de8393 feat: add PS1 conformance harness and fix GTE fidelity
(cherry picked from commit 98d617b5f3)
2026-09-14 17:26:31 +03:00
Matthew Stanley 9037e7bd23 Add opt-in retained texture banks and enhancement memory snapshots 2026-09-13 16:50:26 -07:00
Matthew Stanley 20ec948ab1 build: add opt-in runtime IPO experiment with validation gates 2026-09-12 23:31:28 -07:00
Matthew Stanley ac0e40628a Match BIOS setup detection to configured linkable backends 2026-09-11 23:39:11 -07:00
Matthew Stanley c5390e4265 Merge kernel patch ranges PR #346 for combined validation 2026-09-11 22:53:07 -07:00
Matthew Stanley 9550b3e579 Merge BIOS setup-host PR #343 for combined validation 2026-09-11 22:53:05 -07:00
Kevin 3cbe6c1cde kernel-bless: publish install-slot ranges so patched BIOS bodies run native
The BIOS exception handler ran interpreted for the life of every process,
1.22 billion instructions and 44.6% of all interpreted work in Breath of
Fire III, even though it is compiled and even though the stub that
unblessed it sat at exactly the address bios/SCPH1001.toml already
declared as an install slot. Two mechanisms existed and did not compose:
the emitter hooked the declared slot PC, while the runtime's bless
verifier memcmp'd the whole body including that slot, so the body was
MISMATCH forever and native code never reached the hook.

Three parts, none of which pays off alone:

1. Publish the declared ranges to the runtime. The emitter emits a
   PsxKernelPatchRange table next to the kernel body table and routes it
   through the backend descriptor; the bless verifier compares a body in
   segments that skip them. CLEAN now means "every byte outside every
   declared range still matches ROM", which is the claim the native code
   actually depends on.

2. Generalise the slot model to what the patches are. The old model
   assumed a 4-word lui/addiu/jalr/nop stub written over ROM zeros and
   fired on `word != 0`. Measured against both images, only one of five
   observed shapes fits: patchers also plant a `jr` over real ROM
   instructions, rewrite an 11-word prologue and insert an `mfc0 Cause`,
   and NOP out a driver routine. A slot is now a range with a length and
   an explicit `resume` (jalr / fallthrough / none), detected by
   comparing every word against its ROM-baked value.

3. Hand back from the interpreter at the range end. Kernel page 0 is
   permanently dirty, because the handler saves registers there on every
   exception, so straight-line interpretation would run the whole
   function after the patch. The emitter registers the resume PC as a
   continuation key and dirty_ram_interp surfaces there.

A PC inside a declared range must never be a native dispatch key: the
compiled bytes there are not what executes. Seven pre-existing
jal-return continuations violated that and wedged the boot at frame 0 in
an infinite dispatch loop (OpenBIOS RAM 0x357C). The emitter now drops
continuation keys inside a range so dispatch misses through to the
interpreter, and refuses to emit if a range covers a function entry,
which would silently remove a callable function instead.

Both profiles declare their measured ranges, and SCPH1001.toml finally
pins [program.image] sha256 to v2.2 (CRC32 37157331) so a v2.0 dump
refuses to generate rather than producing phantom discovery gaps.

Measured on Breath of Fire III (SLPS-00990, Psy-Q SDK 3.70), headless,
matched frames, PSX_KERNEL_PATCH_RANGES=0 for the OFF side:

                      OpenBIOS            SCPH-1001 v2.2
  bless mismatch      21 -> 0             74 -> 0
  kernel bodies       -87.7%/frame        -97.8%/frame
  all interp insns    -42.6%/frame        -94.2%/frame

The residual kernel-body work is exactly declared-words x entries on
every range, so nothing leaks past the hook. The A0/B0/C0 vector
trampolines are unchanged (+0.2%, noise): the profile excludes them by
design and they are a separate job. Card read traces are byte-identical
across the A/B on both images, all 32 entries, which exercises the two
ranges that replace the kernel's card handler.

Rule 18 holds throughout: the guest's patched instructions still execute
as written on the dirty-RAM interpreter. Only the code around them
changes backend. Nothing is synthesised.

Tests: runtime/tests/test_kernel_patch_ranges.c (new, the verifier's
accept/reject behaviour and the hand-back predicate) and the install-slot
range invariants in recompiler/tests/bios_address_model_test.cpp.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-11 16:38:34 -04:00
Matthew Stanley 2113defcfc Fix overlay Clang exports, PGO, and scaffold cache 2026-09-10 17:23:03 -07:00
Alexandros Mandravillis c2f9c3d59a setup: detect any recompiled BIOS stem and stop the first-run setup loop
Focused port of the fork's fix/setup-host-bios-deadlock feature (10 commits) onto current upstream master, per the 2026-09-10 decision to port rather than rebase (the fork line and upstream share no merge base).

Content: the codegen host accepts any recompiled BIOS stem instead of only OpenBIOS/SCPH1001 (psx_bios_known_images.h); the runtime checks the setup host before validating a chosen BIOS and the first-run loop breaker reports a cause-specific reason; the CLI decodes subprocess output as UTF-8; --setup-selfcheck verifies a headless setup; the rbengine preflight names the recursive init. Two port conflicts resolved: the new CTest registrations in recompiler/CMakeLists.txt are kept (the names exist only in this change), and runtime.cmake takes the new recomp-ui guidance text. Cubic P3 finding fixed: the sibling PSX_NETPLAY FATAL_ERROR now also names --recursive.

Evidence: python syntax checks pass for psxrecomp_cli.py and the new tests; runtime/tests/test_codegen_host_bios_stems.py runs or skips only on a missing RECOMP_UI_ROOT. No full runtime build or retail route is claimed here. Developed with AI assistance (Cline, 2026-09-10).
2026-09-11 01:26:22 +03:00
Alex Vanderveen 922e8c5a07 Rename the product from RetComM to Retro
The launcher and studio ship as Retro Launcher and Retro Studio now,
from the RetroPortingToolKit org, so the old product name is gone from
display text, documentation and release asset filenames.

The split is by case, and it is not cosmetic. CamelCase RetComM was
always the product; lowercase retcomm and RETCOMM_ are identifiers --
the C++ namespace, the binaries, the data directories, the CMake and
environment variables, the Cloudflare service names, the readme markers.
Renaming those would orphan existing installs for no benefit, so they
are untouched: 1101 lowercase retcomm and 205 RETCOMM_ still stand in
the launcher alone.

Four things are deliberately still spelled RetComM, and three of them
would have been real breakage:

- RetComM-Data is CamelCase but it is a directory on disk --
  <exe_dir>\RetComM-Data\{runtime,config,data} for a portable install.
  Renaming it strands every portable user's config, toolchains and
  catalog cache.

- TechnicallyComputers/RetComM-Launcher in readme_has_launcher() is the
  historical half of a both-slugs test. Renaming it makes it a duplicate
  of the line above and sends every port built before the transfer back
  to reporting its launcher section missing.

- _LEGACY_LAUNCHER_RE matched "^## RetComM Launcher". A sweep renamed it,
  which would have silently stopped it matching every README already out
  there; it now accepts both headings.

- The portable-stub lookup gained the pre-rename filenames back as
  fallbacks, because old zips stay on the releases page and a rollback
  downloads one.

Release assets become Retro-Launcher-* / Retro-Studio-*. That is safe
for installed clients: pick_launcher_asset() gates on *linux*/*windows*/
*macos* plus extension and arch, not on the product name, and the
Windows portable stub is identified by its RCM1 trailer rather than its
filename.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013DpaQqVguHFpBuCmdoFYWY
2026-09-09 17:32:19 -04:00
Matthew Stanley 93a93c8e07 Fix game runtime wrapper setup gating 2026-09-08 13:36:54 -07:00
Alex Vanderveen 0bdc648604 cmake patch 2026-09-07 14:51:39 -04:00
Matt Stanley 1d414fc7b1 Merge pull request #314 from mstan/chore/repo-cleanup
chore: repo hygiene — purge dead scratch tooling, relocate root docs/fixtures/BIOS profiles
2026-09-03 11:30:32 -07:00
Matthew Stanley 772a3a956b Merge remote-tracking branch 'origin/master' into pr293-vulkan-stage-idempotent 2026-09-03 11:21:11 -07:00
Matthew Stanley bcd048f184 CMake: make Vulkan header staging idempotent 2026-09-03 11:21:10 -07:00
Matthew Stanley e6ce5006b4 docs: move root and stale docs into docs/ and docs/internal/
The repository root carried seven markdown files that were not onboarding
material, plus two that had been superseded by the code they described. This
sorts them: reader-facing docs to docs/, working notes to docs/internal/, and
the two superseded ones merged into the docs that replaced them.

Moved to docs/:            WIDESCREEN.md, ENHANCEMENTS.md, TCP_COMMANDS.md
Moved to docs/internal/:   PRINCIPLES.md, ISSUES.md, DEBUG.md, GTE_LM_FIX.md,
                           and from docs/: bios-hardening.md, overlay-status.md,
                           overlay-plan.md, overlay-recompilation-design.md,
                           overlay-discovery.md, TOMBA2_EXC_CONSULT_RESPONSE.md,
                           CASE_A_AOT_GAP.md, audit_inventory.md
Moved to docs/internal/upstream/: UPSTREAM_2026-08-20.md

The root keeps only what belongs there: README.md, CONTRIBUTING.md, LICENSE,
CLAUDE.md, THIRD_PARTY_ATTRIBUTION.md, RELEASE_NOTES.md, the dotfiles and
psxrecomp_cli.py.

MERGES

SPLITGEN_MIGRATION.md folded into docs/BUILDING.md as "Split generated C". Its
migration instructions were stale in a way that mattered: it told every game
repo to hand-edit its CMakeLists and add a file(GLOB ...) over
<serial>_full*.c, but runtime/runtime.cmake now does that itself. GEN_FULL_GLOB
is a multi-value argument of psxrecomp_add_game_runtime(); the loop at
runtime.cmake:1985 makes each pattern absolute against CMAKE_CURRENT_SOURCE_DIR,
expands it with file(GLOB), and concatenates the hits into
GAME_GENERATED_FULL_C. So a title names the pattern and gets however many shards
a regeneration produced. The rewritten section describes that, the two-step
fallback when the globs match nothing (GEN_FULL_FALLBACK, else <serial>_full.c
derived from the dispatch marker by substituting _dispatch.c), and the one
caveat the code actually has: that file(GLOB) is plain, not CONFIGURE_DEPENDS,
so a regeneration which changes the shard COUNT needs a re-configure to be seen.

psxrecomp-toml-readme.md folded into docs/GAME_PROJECT_SETUP.md as "EXE-only
autofill (psxrecomp-toml)", placed after "Disc autofill (probe_disc.py)" and
before "Project Studio" so the two autofill paths sit together. Its documented
flags were checked against the usage text in recompiler/src/main_toml.cpp:28-42
and against the argument parser at :150-165 -- all seven options match with no
drift, so the flag list is reproduced as-is. Added a pointer that probe_disc.py
is preferable when a cue exists, since that tool does the same EXE-header work
plus disc identity and netplay gates.

FUNCTIONAL FIXES

tools/gen_tcp_commands.py wrote to REPO/"TCP_COMMANDS.md". After the move that
path does not exist, so the generator would have created a new stray root file
and --check would have failed against it forever. Retargeted to
REPO/"docs"/"TCP_COMMANDS.md", along with the seven docstring and message
strings that name the path.

Running it surfaced two further problems, both fixed here:

  * The doc was genuinely stale -- pre-existing drift, not introduced by this
    commit (the moved file is byte-identical to origin/master, and no
    debug-server command table is touched here). Four commands registered since
    the last regeneration were missing from the index -- cdrom_timing_dump,
    savestate_status, ws_ui_groups, xprobe_watch -- and scanline had gained a
    backend. Regenerated: 307 -> 311 commands, and the change is confined to the
    AUTOGEN block (the hand-written head and tail are byte-identical to master).
    tools/gen_tcp_commands.py --check now exits 0.
  * DOC.write_text() has no newline argument, so Python translated every newline
    to os.linesep. On Windows that rewrote all 585 lines of the file to CRLF and
    turned a four-line index update into a whole-file diff -- measured: 589 CRLF
    / 0 LF after a run, against 0 CRLF / 585 LF in the repo. Now written through
    an explicit newline="" handle, verified to leave the file at 0 CRLF.

runtime/include/debug_server.h:9 and runtime/src/debug_server.c:8 cited "TCP.md
and DEBUG.md". TCP.md has never existed in this repo under that name; both now
name docs/TCP_COMMANDS.md and docs/internal/DEBUG.md.

LINK AND REFERENCE FIXES

Every inbound reference to a moved file was located by grepping each basename
across the whole tracked tree, not just the markdown. Markdown hrefs in
README.md, CONTRIBUTING.md, docs/ARCHITECTURE.md, docs/COMPILING_OVERLAYS.md,
docs/AOT_OVERLAY_PLAN.md, docs/RELOCATION_MANIFEST_FORMAT.md,
docs/config_schema.md, docs/OVERLAY_XREF.md, docs/TCP_COMMANDS.md and several
docs/internal/ files now point at the new locations; where two files became
siblings under docs/internal/ the path was shortened rather than lengthened
(bios-hardening.md's reference to overlay-discovery.md, for instance).
docs/COMPILING_OVERLAYS.md's "in this folder" phrasing was wrong once two of the
three files it listed moved, so it now names internal/ explicitly.

24 source-comment mentions of ENHANCEMENTS.md sections G1.x across 19 files
(recompiler, runtime, cmake) were retargeted to docs/ENHANCEMENTS.md with the
section numbers untouched. docs/SHADOW_ENHANCEMENTS.md is a DIFFERENT file and
was deliberately excluded.

20 rationale strings citing docs/CASE_A_AOT_GAP.md across the four
recompiler/seeds/*.json files were updated. Done as a byte-level substring
replacement rather than a JSON round-trip: two of those files are stored with
CRLF endings and two with LF, and re-serializing them reformatted 5,700 lines.
The byte-level version changes exactly 20 lines and each file still parses.

References that look similar but point elsewhere were deliberately left alone,
and are worth naming so a future pass does not "fix" them:

  * "recomp-template PRINCIPLES.md" / "NES recomp template PRINCIPLES.md" in
    CLAUDE.md:43, docs/SHADOW_ENHANCEMENTS.md:28,
    docs/internal/HLE_SCHEDULER_CARVEOUT_PLAN.md:5,
    runtime/src/debug_server.c:4934 and tools/parity_diff.py:8 are a different
    repository's file. Only the five that mean this repo's PRINCIPLES.md were
    updated.
  * Every remaining bare ISSUES.md mention is a GAME repo's tracker
    (MegaManX6Recomp 2/7/8/9, TombaRecomp 6), not this repo's.
  * docs/disc-speed.md:4,54,56 and docs/internal/bios-hardening.md:5 name
    overlay-discovery as a BRANCH, not a file, so no path applies.

INDEXES

README.md's "New here?" table gains rows for docs/WIDESCREEN.md,
docs/ENHANCEMENTS.md and docs/TCP_COMMANDS.md. docs/internal/README.md gains
sections indexing all fourteen documents that moved in, plus an "Upstream
contributor reviews" section describing upstream/ and the new sweep file, plus a
"Removed" bullet for CYCLE_TIMING_ARCH.md (deleted in the previous commit)
pointing at FAITHFUL_TIMING_PLAN.md, CYCLE_MODEL_BEETLE.md and
PRECISE_IRQ_SLICE.md -- following the SLJIT_PERSIST_CACHE.md precedent already
in that file.

docs/ENHANCEMENTS.md:7 claimed framework bugs live in "game repos' ISSUES.md
until a framework tracker exists". A framework tracker does exist; it now says
so and names the Beads epic.

VERIFICATION

A link checker was written to scan every tracked *.md for relative links,
resolving each against the tree being scanned. Baseline at origin/master: 129
files, 137 relative links, 3 broken. After both commits: 127 files, 159 relative
links, 3 broken -- the SAME three, all pre-existing and untouched here:

    docs/DATA_SHARDS.md -> INSTANT_LOADS_PARKED.md   (never existed)
    docs/DATA_SHARDS.md -> INSTANT_LOADS_SPIKE2.md   (never existed)
    docs/NETPLAY.md     -> ../lib/recomp-net/README.md  (inside a submodule)

Zero new broken links. The checker's own coverage was checked against a raw grep
for markdown link openers: the 3-link difference is C++ lambdas inside code
blocks, correctly not treated as links.

Also verified: py_compile over all 113 tracked tools/*.py plus psxrecomp_cli.py;
all four seed JSONs still parse; the test-registration guard still reports 123
test files all registered; recompiler/tests/test_coverage_vault.py still passes;
and no file in this commit had its line endings flipped (checked mechanically
across all 50 modified files, after two files were caught and redone).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 07:39:07 -07:00
Matt Stanley d08d84a3df Merge pull request #296 from kerokline/perf/static-overlay-parallel
overlays: parallel static compile, linear resume-wrapper pass, split output
2026-09-03 07:30:43 -07:00
Matthew Stanley 94e3ef610d release: ONE staging implementation for both platforms (2 titles had no Linux release path)
tools/release_overlay_stage.ps1 exists because a packager that re-derived the
overlay cache tag in PowerShell went stale the moment the framework appended an
_f<flavor> field to it: the packager's filter matched nothing, a perfectly
regenerated cache staged ZERO shards, and the release shipped with no native
overlays (beads-eio.2.6). The fix was correct -- expose cache_tag() and route
the packager through it -- and it fixed Windows only.

Linux was never consolidated at all. Three title repositories each carry a
forked tools/package_appimage.sh, and each hand-builds the same tag string,
inside a python heredoc that has just imported the module that owns it:

    spec = importlib.util.spec_from_file_location('co', mod_path)
    m = importlib.util.module_from_spec(spec); spec.loader.exec_module(m)
    print(<the tag format, reproduced by hand> % (m.codegen_ver(inc),
          m.codegen_hash(inc), m.overlay_config_hash(exe, gt)))

MEASURED 2026-09-02, reading each repository's origin/master (not a working
tree -- the Ape checkout is parked on codex/ape-issue3-diagnostic-prerelease,
behind master, and reading it gives the wrong answer, including a wrong line
count):

  repo             lines  calls cache_tag()  composes _f  stages toolchain
  ApeEscapeRecomp    416         no              yes            no
  Tomba2Recomp       436         no              NO             no
  MegaManX6Recomp    408         no              NO             no

ApeEscapeRecomp 4a17272 (2026-08-28, "release: stage flavored Linux overlay
cache") hand-appended the field to ONE of the three forks. It did not adopt
cache_tag(); it wrote cache_tag="${cg_tag}_f${overlay_flavor}" and switched the
filters to it. So Ape is correct today and one character from the same failure
on the next tag-shape change, and the other two are broken right now:
find -path "*/$cg_tag/*" requires a separator immediately after the tag, the
real directory is named ..._f0, so the pattern cannot match. shards evaluates
to 0 and the packager exits 1.

That is not hypothetical. Tomba 2 v0.0.9, released 2026-09-02, has exactly two
assets: Tomba2Recomp-v0.0.9-windows-x64.zip and its .sha256. There is no
AppImage in that release at all, which is what a packager that exits 1 looks
like from the outside.

Two implementations, one per platform, have to be kept in step by review. This
bug is the proof that review does not hold: the identical defect had ALREADY
been fixed on the other platform, and the reason cache_tag() exists is that
fix. So there is now ONE implementation.

  tools/release_stage.py            the implementation (new)
  tools/release_overlay_stage.ps1   thin wrapper, surface UNCHANGED
  tools/release_overlay_stage.sh    thin wrapper, new (Linux had none)

Both wrappers are argument marshalling. Neither contains a tag format string, a
shard filter, an extension list, an arch-abi segment or a catalog rule. Every
sh function forwards "$@" verbatim, so a new option needs no edit there at all.
No title's package_release.ps1 needs a change: every parameter, switch and
return value of Get-OverlayCgTag / Add-OverlayCache / Add-OverlayToolchain /
Add-ModCatalog / Get-PinnedArchive is preserved, verified by dot-sourcing the
module in PowerShell 5.1 and exercising all five.

WHAT THE SHARED IMPLEMENTATION DOES DIFFERENTLY

* The tag is cache_tag(). Not "a call that produces the same string" -- the
  call. release_stage.py does not know the tag's shape and must never learn it.
  It also takes the arch-abi and the shared-library suffix from
  compile_overlays.cache_arch_abi()/overlay_ext() rather than naming them, so
  ".so under linux-x64" and ".dll under win-x64" are one decision, made in the
  module the loader is kept in lockstep with, not two.

* Shards are selected by the tag DIRECTORY'S IDENTITY, not by a path substring.
  -path "*/<tag>/*" is simultaneously too loose (it accepts a foreign arch-abi)
  and too strict (it rejected the real _f0 directory and staged nothing). The
  staged layout is then asserted -- exactly one tag directory, and it must be
  this build's -- because the loader scans cache/<id>/<tier>/<arch-abi>/<tag>/
  literally, and a shard anywhere else is worth exactly as much as no shard,
  which no count can distinguish.

* Selection is an ALLOWLIST of {<ext>, .ranges, .resident}, so a new artifact
  kind in a producer's cache is excluded by default rather than shipped by
  default, and the staged tree is re-scanned afterwards to prove that
  .abi_<tag>.ok, *.c and *.pair-lock did not ship. The sweep memo is the one
  that matters: it tells a player's first launch that an ABI sweep it never ran
  already passed, so a stale shard gets loaded instead of rejected.

* --allow-no-cache is gone from the Linux surface entirely. It is the
  relaxation that let a packager staging nothing read as a successful build.
  What remains is --ship-without-overlay-cache-because '<reason>', which is
  unwieldy on purpose, prints the reason, and is not the documented path in any
  recipe. PowerShell's -AllowNoCache is kept (unmigrated title packagers still
  pass it) and maps onto that flag, so even the escape hatch is one
  implementation.

* The mod catalog is verified against the manifest the BUILD published.
  runtime.cmake's _psxrt_stage_mod_catalog already writes
  psx_mod_catalog_<target>.txt -- the sorted, deduplicated id list that build
  staged -- which is a strictly better authority than re-globbing source trees,
  and it handles Tomba 2's Italian catalog overriding framework psx.* packages
  at the same ids without a per-variant number.

LINUX GETS AN OVERLAY TOOLCHAIN, AND THE RUNTIME CAN NOW SEE IT

No AppImage packager has ever staged one (grep -c overlay_toolchain = 0 in all
three forks), so on Linux the shipped AOT cache was all a player ever got: the
capture-and-compile fail-safe could not fire, and an overlay outside the cache
was interpreted forever.

Staging one alone would not have worked. runtime/src/main.cpp gated autocompile
on overlay_toolchain/python/python.exe, and built the command from
psxrecomp-game.exe and tcc/tcc.exe -- hardcoded Windows spellings, so on Linux
the gate could never be true no matter what a packager staged. A bundled Linux
toolchain would have been dead weight the runtime never looked at. Those three
names are now per-platform; the layout is identical, only the spellings differ,
and they are kept in lockstep with release_stage.py's TOOLCHAIN_PY_REL /
TOOLCHAIN_RECOMPILER.

The Linux interpreter pin is a python-build-standalone CPython 3.13.1 --
deliberately the same version as the Windows embeddable-zip pin -- fetched and
SHA256-verified on every use by the same code path as the Windows one, so the
platforms cannot drift on retry, cache or verification behaviour either. The
_stripped asset (21 MB download, 78 MB unpacked) is used; compile_overlays.py
needs only the stdlib, verified by import. tinycc publishes no prebuilt Linux
binary, so there is no pinned tcc there and the bundle drives a gcc from PATH;
main.cpp already says so on stdout rather than silently doing nothing. The
SHA256 was measured by downloading the asset and matching the publisher's own
.sha256 sidecar, not copied from a listing.

THE FLAVOR IS NO LONGER GUESSED

PSX_OVERLAY_FLAVOR is the high half of overlay_abi() and the _f<n> field of the
tag. It is a property of the BINARY, decided in runtime.cmake (0 base, 2 for a
PGXP target), and it is NOT platform-dependent -- Windows and Linux builds of
the same target share it, so nobody should ever add a platform branch for it.
Every packager simply assumed 0, and that assumption is invisible when wrong: a
PGXP runtime reads ..._f2/ while the packager stages ..._f0/, the package ships
a cache the binary ignores completely, and nothing fails.

runtime.cmake now publishes psxrecomp_overlay_flavor-<target>.txt next to the
exe-name publication it already writes, for the same stated reason: so nothing
downstream re-derives it. release_stage.py cg-tag takes either --flavor or
--flavor-from-build, and there is no default -- a packager either states the
value or reads what the build published.

TESTS

runtime/tests/test_packagers_never_format_cache_tag.py asserts the rule the two
fixes above were supposed to establish: the tag format appears only in
tools/compile_overlays.py and runtime/src/overlay_loader.c (measured: 5
occurrences, all inside those two files -- note overlay_loader.c has four, at
1745, 1773, 1924 and 1930, the last two being the actual read path) and nowhere
else, and a release packager that touches the overlay cache must reference the
shared staging surface. It carries a positive control -- each owner must
actually contain the pattern -- because the investigation of this very bug once
reported "no cache present" from a probe that could not have found a cache at
all. It is also the SHARED check a title runs against its own tree
(--root <repo>), so the rule about not duplicating logic is not itself
duplicated per repository.

Run against Tomba2Recomp origin/master it reports, pre-fix:

  tools\package_appimage.sh:292 formats the overlay cache tag itself
  tools\package_appimage.sh is a release packager but does not reference the
  framework shared staging surface

runtime/tests/test_release_stage_cache.py pins the behaviour, hermetically, on
synthetic cache trees -- no recompiler, no captures, no build, which is the
point: this is the assertion that could have run in CI on the day the tag grew
a field. 15 cases, including that a tag one field short selects nothing AND
raises rather than warns, that a foreign tag namespace / arch-abi / the removed
sljit tier are not shipped, that the three forbidden artifact kinds never ship,
and that developer state (mods/installed, mods/state.toml) never ships.

Both are registered in runtime/CMakeLists.txt, so the configure-time
test-registration guard covers them and ctest runs them.

NOT DONE, deliberately: tools/package_release.ps1,
tools/package_release_macos.sh and tools/package_setup_host.sh each carry their
own hand-written mod-catalog staging. That is the same duplication defect one
layer over and they should route through stage-mods too, but it is a separate
change needing separate proof (macOS cannot be built or verified here), so the
new test scopes its packager assertion to packagers that touch the overlay
cache rather than pretending to cover them.

Bead: beads-eio.3.102

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-02 18:28:24 -07:00
Matthew Stanley 57648745a3 mods: the framework stages the title's catalog too (5 titles could not package)
4cc04be3 ("rebump") renamed the staged mod catalog from <exe>/mods/packages to
<exe>/mods/bundled so that a rebuild could wipe build output without deleting
the player's own installed mods. runtime.cmake followed the rename. The shared
release module followed it (release_overlay_stage.ps1's Add-ModCatalog reads
mods/bundled). docs/MOD_PACKAGES.md already described the destination as "the
framework's mods/builtin/packages PLUS the title's mods/preloaded/packages".

No title CMakeLists followed, because the second half of that sentence was
implemented five times, once per repository, as a hand-written

    add_custom_command(TARGET psx-runtime POST_BUILD
        COMMAND ${CMAKE_COMMAND} -E copy_directory
            "${<TITLE>_PRELOADED_MODS}" "$<TARGET_FILE_DIR:psx-runtime>/mods")

in ApeEscapeRecomp, Tomba2Recomp and MegaManX4/5/6Recomp. That names the
destination as a STRING. There is no symbol, no header, no link dependency --
so renaming a staged output directory breaks nothing a configure, compile or
link step can see, and the diffstat of the commit that does it looks cosmetic.
The five titles went on staging into a directory neither the launcher nor any
packager reads, and discovery was deferred to whichever title next ran a
release packager, in a different repository, on a later day.

MEASURED, on real build output. Framework origin/master 22fbbfca plus
Tomba2Recomp origin/master cdf00af, both unmodified, both built:

  build-dir mods/
    bundled/psx.enhancement.cd-speed          <- framework, correct
    bundled/psx.enhancement.fast-loading
    bundled/psx.enhancement.pgxp
    bundled/psx.presentation.bezel
    packages/tomba2.debug.debug-menu          <- title, stranded
    packages/tomba2.enhancement.skip-fmvs
    packages/tomba2.enhancement.widescreen
    packages/tomba2.experimental.interpolated-frame-rate

and Add-ModCatalog run against that tree, from this commit's parent:

  Mod catalog is missing package(s) the sources define:
  tomba2.debug.debug-menu, tomba2.enhancement.skip-fmvs,
  tomba2.enhancement.widescreen, tomba2.experimental.interpolated-frame-rate.
  They exist in the source tree but did not reach ...\mods\bundled, so the
  release would ship a Mods page the dev build does not have.

Shipped releases are NOT affected. Tomba 2 v0.0.9 pins 04d9184b, which is an
ancestor of 4cc04be3 and contains zero occurrences of migrate_legacy_root and
bundled_root; its artifact is a self-consistent mods/packages layout. This is a
forward break only.

THE FIX PUTS THE LAYOUT IN ONE PLACE.

psxrecomp_add_runtime_target() gains a one-value keyword PRELOADED_MODS_DIR.
A title declares WHERE its packages live and never HOW they are laid out:

    set(APE_PRELOADED_MODS "${CMAKE_CURRENT_SOURCE_DIR}/mods/preloaded")
    psxrecomp_add_runtime_target(psx-runtime
        ...
        PRELOADED_MODS_DIR "${APE_PRELOADED_MODS}")

A keyword argument rather than a positional one or a variable read out of the
caller's scope, for three reasons. It is what the other fourteen one-value
arguments already are, so it needs no new convention and reads the same as the
LAUNCHER_BOXART line next to it. A pre-set variable would be invisible at the
call site and would silently do nothing if set after the call -- the same
"fails quietly in the caller's repository" shape as the bug being fixed. And it
is PER TARGET, which a global variable cannot be: Tomba 2 has two runtime
targets needing two different catalogs (mods/preloaded, mods/preloaded_ita).
psxrecomp_v4_add_runtime_target and psxrecomp_add_game_runtime both forward
their remaining arguments, so it reaches through both wrappers unchanged.

_psxrt_stage_mod_catalog() now emits, in one POST_BUILD command sequence per
runtime target: wipe mods/bundled; remove exactly the ids it is about to stage
from any pre-existing mods/packages; copy the framework builtins; copy the
title's packages; copy <dir>/README.md to mods/README.md; verify. Consequences
worth stating:

  * mods/README.md previously reached mods/ only as a side effect of copying
    the whole mods/preloaded tree. Now that only packages/ is staged it is
    copied on purpose, or it would silently disappear. Checked per title: all
    five carry mods/preloaded/README.md and none carries a root mods/README.md,
    so mods/preloaded/README.md is the only source there ever was.
  * a title may still OVERRIDE a framework builtin at the same id and version,
    because the framework copy runs first and the title copy overwrites it.
    Tomba 2's Italian catalog relies on exactly this, shipping localized psx.*
    manifests. The id LIST is deduplicated for the assertion and the reported
    count while the copies keep both, or the build claims to stage 11 packages
    for a catalog that holds 7.
  * COSIM targets stage nothing. They have no launcher and no Mods page, and
    they land in the same output directory as the real runtime, so letting one
    stage had it wipe and re-stage the runtime's catalog with only the
    framework half whenever -DPSX_BUILD_COSIM=ON built it last.

TWO GUARDS, because the failure mode being closed is silence.

Configure time: a project with packages under mods/preloaded/packages that does
not declare PRELOADED_MODS_DIR is a FATAL_ERROR naming the packages and the
argument to add. That is exactly the missed-title state and it is detectable
before any build runs. An empty packages/ is fine (the scaffold creates one),
and PRELOADED_MODS_DIR NONE is the explicit opt-out.

Build time: runtime/psx_check_mod_catalog.cmake fails the build if a declared
package did not reach mods/bundled, or if any package this build stages turned
up under mods/packages -- which after this change can only mean a CMakeLists
still stages its own catalog by hand. It runs as the LAST POST_BUILD step, via
cmake_language(DEFER), because POST_BUILD commands execute in registration
order and the framework block registers BEFORE anything a title adds after its
psxrecomp_add_runtime_target() call. Verified by inspecting build.ninja: the
deferred command lands after a title-registered POST_BUILD, not before it.

The deferred pass takes NO arguments and reads its work list out of global
properties. cmake_language(DEFER CALL <fn> "${var}") does not carry a
function-local variable through -- measured with cmake 4.2.2, the callee
receives an empty argument and an add_custom_command built from it fails at
directory end with "CMakeLists.txt:DEFERRED". Recorded so the next reader does
not re-derive it.

Ids the build does NOT stage are deliberately left alone in mods/packages. On a
self-compiling setup release that directory can hold packages the player
installed under the pre-split layout, and mod_packages.cpp's
migrate_legacy_root() relocates those into mods/installed on the next launch.
Removing only the ids this build stages is also what lets a build directory
made before the split be migrated silently, so the guard's "an id turned up in
mods/packages" report has no false positives.

The same script is registered as the ctest psx_staged_mod_catalog_test in every
project that stages a catalog, with REQUIRE_STAGED=0 so a tree where the
runtime was never built reports a skip rather than a spurious failure. Where
two runtime targets share an output directory it accepts either target's
complete catalog, since after a full build mods/bundled holds whichever linked
last -- which is correct for the packager, as it builds exactly one target per
variant, and each target's own POST_BUILD guard already verified its catalog at
stage time.

TESTS. runtime/tests/test_mod_catalog_layout.py exercises the guard against
on-disk fixtures: a reproduction of the broken state as it actually appeared
(bundled/ holding the four psx.* ids, packages/ holding the four game ids), a
declared package simply absent, a package directory with no manifest.toml, a
FOREIGN legacy package that must NOT fail the build, the require-staged split
between the two callers, and the shared-output-directory sibling case. It also
pins two source invariants in runtime.cmake so the layout cannot drift back
into per-title copies. Registered in runtime/CMakeLists.txt; the
test-registration guard reports 123 test file(s), all registered.

Framework ctest: 59/59 pass, 2 pre-existing DISABLED.

Also updated so the next title does not repeat this: the project scaffold
template now passes PRELOADED_MODS_DIR (it staged no catalog at all before, so
a scaffolded project with mods/preloaded/packages shipped none), and
docs/MOD_PACKAGES.md gains a "How bundled/ gets staged" section written at
titles.

Bead beads-eio.3.101. The five title-side patches must land AFTER this commit:
they reference PRELOADED_MODS_DIR, which does not exist before it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-02 16:37:19 -07:00
Alex Vanderveen 4cc04be3e6 rebump 2026-09-02 13:26:06 -04:00
Kevin 7ab698caa5 overlays: parallel static compile, linear resume-wrapper pass, split output
Static mode (--static) was serial end to end and produced one translation
unit, so a whole title compiled on one core twice: once in Python, once in
gcc. Measured on BoF3 (339 captures, 79,688 identities) in a build-dbg loop:
phase "compile all overlay bands" ~12 min, runtime build ~4 min at -O0.

Three changes, each independently verified byte-identical:

1. static_capture_job(): the per-capture static work (recompiler subprocess,
   C audit, range audit, namespacing) is now a module-level, picklable job
   that returns its part + captured log. --jobs runs it in a
   ProcessPoolExecutor; results merge in capture order, so static_parts and
   entry sources are order-identical to the sequential loop. --jobs 1 and
   --jobs 3 on three bands: diff -r of the outputs is empty.

2. add_cps_resume_cases(): the universal CPS resume-wrapper pass called
   add_cps_resume_case() once per entry, and each call re-scanned and
   rebuilt the WHOLE part source. cProfile on three bands: 7,386 entries ->
   22,159 re.search calls, 366 s of 424 s total. The batch form locates
   every signature once, edits each host's segment in isolation, rebuilds
   once. Same three bands: 418 s -> 5 s, outputs byte-identical (diff -r
   against the per-entry output). The per-entry function is kept; a unit
   test asserts the two agree on fresh-switch, existing-switch, missing
   label and missing host cases.

3. write_static_outputs(): overlays_static.c now holds the dispatcher plus an
   extern prototype per compiled entry, and every part becomes its own unit
   overlays_static_NNNN.c (each already carries its #includes and a private
   namespace). runtime.cmake globs the siblings with CONFIGURE_DEPENDS, so a
   changed part count needs no reconfigure. Unchanged units are not
   rewritten (mtime preserved -> not recompiled); stale units are removed.
   --static-single-file restores the monolithic file.

Result, same BoF3 loop on a 12-core box: phase 5a 12 min -> 20 s, runtime
build 4 min -> 69 s (358 units in parallel), whole loop ~16 min -> 90 s.
Shard summary unchanged (332 ok / 7 benign audit fails), identity count
unchanged. Headless boot of the rebuilt runtime: 485,650 static checks,
485,244 hits (99.92%), miss_total 0, 60 emu fps through the LOGO overlay.

Note: tools/test_compile_overlays_additive.py already fails on master (it
references removed functions: compile_and_publish_dll, load_additive_captures,
validated_cached_bundle_ids); untouched here.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 09:20:32 -04:00
Alex Vanderveen 67fed036b5 Vulkan: stage the headers instead of dropping the renderer
When the located Vulkan headers are not reachable from the compiler, the
build already detects it and falls back to the software stub. That is
correct but costs the renderer on a whole class of toolchain: the usual
case is _vk_inc=/usr/include, which CMake refuses to emit into an include
list, on a clang packing its own sysroot that never had the host include
root on its search path. Such a build is left with no Vulkan even though
the headers are sitting right there.

Stage a private include dir holding only the Vulkan subtrees and retry
before giving up. That directory is not /usr/include, so CMake emits it,
and it brings none of the rest of the host include root ahead of the
sysroot's own libc headers -- which is what makes it safe where adding
_vk_inc itself is not. vk_video/ is staged alongside vulkan/ because
vulkan_core.h includes the H.264/H.265 codec headers from it; staging
vulkan/ alone still fails to compile.

The stub fallback is unchanged and still runs when staging does not help.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016k5TMHeJ2xbzjKdP4H19vY
2026-09-02 02:01:05 -04:00
Alex Vanderveen 1305cefc7a netplay: stop ENABLE_NETPLAY_IF_PRESENT from failing silently
The argument cannot turn netplay on by itself, and said nothing when it
did not. By the time psxrecomp_add_game_runtime() reads it, runtime.cmake
has already been included, and that include both resolved recomp-net and
decided whether the netplay TUs compile for real or as stubs. The
option(PSX_NETPLAY ... OFF) near the top has also already made the
`if(NOT DEFINED PSX_NETPLAY)` below it false, so the assignment it guards
never runs on a configured tree.

The switch that works is the pre-include block the scaffold emits above
the include(). Warn when the flag is passed but netplay is still off, and
name that block -- separately reporting the case where recomp-net simply
is not checked out, since the fix there is a submodule init rather than a
CMake edit.

No behaviour change beyond the diagnostic: the flag was already inert.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016k5TMHeJ2xbzjKdP4H19vY
2026-09-02 01:59:35 -04:00
Matt Stanley be19a02902 Merge pull request #285 from Alexbeav/pr/zero-bios-registry
Make zero-BIOS registries portable to MSVC
2026-08-31 12:06:01 -07:00
Alexandros Mandravillis eb12cb2411 Fix zero-BIOS MSVC registry build 2026-08-31 09:16:14 +03:00
Alexandros Mandravillis fe6a06d298 Preserve explicit recomp UI root 2026-08-31 09:15:51 +03:00
Alexandros Mandravillis 8af48ae986 runtime: slice GPU linked-list DMA 2026-08-30 23:17:24 +03:00
Matt Stanley 3aeef99f47 Merge pull request #258 from mstan/fix/hermetic-toolchain-deps
build: find dependencies the hermetic toolchain can actually compile
2026-08-29 16:35:17 -07:00
Matthew Stanley 9ebd8b374d build: publish final runtime executable name 2026-08-29 16:29:05 -07:00
Alex Vanderveen 9e829086aa build: find dependencies the hermetic toolchain can actually compile
A fresh Final Fantasy 7 scaffold failed to build with SDL3/SDL.h file not
found on every translation unit that includes psx_sdl.h, even though CMake
had reported "using prebuilt/system SDL3".

cmake-clang-v1 compiles against its own sysroot and never searches the host
/usr/include:

  #include <...> search starts here:
   .../v1.0.14/sysroot/usr/include/...
  End of search list.

so a host package can be found and still be unusable. Two paths hit that.

SDL3: the toolchain ships a compatible SDL3 under deps/lib/cmake/SDL3, and
runtime.cmake already prefers it — but only via _PSX_TOOLCHAIN_PREFIX_HINTS,
which was built purely from RETCOMM_TOOLCHAIN_DIR and friends. The setup
wizard passes -DCMAKE_C_COMPILER=<toolchain>/bin/cc without exporting any of
them, so the hint list was empty and find_package fell through to
/usr/lib/cmake/SDL3. The compiler's own location is the hint that cannot be
forgotten, so it is now derived from CMAKE_C_COMPILER as well.

Vulkan: find_path landed on /usr/include, and CMake drops that directory from
include lists (adding it explicitly would disturb the system header order), so
the -I never reached the compiler at all. The block already knows how to build
gpu_vk_renderer.c as an inert stub when Vulkan is absent; it just never got
there, because "found" was decided by a path existing rather than by the header
compiling.

Both now verify before they believe, via a _psx_header_compiles() helper that
actually compiles the header. It takes LIBRARIES for an imported target and
INCLUDES for a bare directory, and the distinction is load-bearing: the first
attempt read INTERFACE_INCLUDE_DIRECTORIES off SDL3::SDL3, which is NOTFOUND
because SDL3 carries its headers on a transitive SDL3::Headers target, so the
check rejected a perfectly good SDL3. Asking through CMAKE_REQUIRED_LIBRARIES
resolves the interface transitively and discriminates correctly: the
toolchain's SDL3 passes, the host's SDL3 under this compiler fails.

Vulkan falls back to the stub it already had. SDL3 fails loudly instead,
naming the toolchain copy to point SDL3_DIR at — silently switching to
FetchContent there would collide with the SDL3::SDL3 target find_package has
already defined, and a wrong SDL is worse than a stopped configure.

Verified on the Final Fantasy 7 project that surfaced this: from a clean build
directory, with no overrides and with every toolchain environment variable
unset, configure selects the toolchain's SDL3, reports the Vulkan stub with the
reason, and psx-runtime builds and links (321/321).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017AwqKPQFnK3x7xQbK4PQKj
2026-08-29 00:27:06 -04:00
Alex Vanderveen cc448e20a1 build: let CMake own the runtime executable name
The runtime's filename was derived three times from three copies of the same
window title. runtime.cmake sets OUTPUT_NAME from
MAKE_C_IDENTIFIER(WINDOW_TITLE); psxrecomp_cli.py rebuilt it from --exe-name;
and the in-runtime self-compiler rebuilt it again from
codegen_setup.exe_basename. The rule is identical in all three, so they agree
right up until a game is renamed and only one copy is refreshed. After that the
build links <new>.exe while both consumers look for <old>.exe, and the failure
is reported as

    build succeeded but binary missing: ...\<old>.exe
    Build failed. Fix the errors above, then rebuild manually.

on a build that had no errors at all, with the executable sitting in the build
directory under the name CMake chose. Seen on Revelations: Persona, where CMake
emitted Revelations__Persona_Recompiled.exe and the setup host wanted
Revelations_Persona__Recompiled.exe — same algorithm, colon in a different
place in each side's copy of the title.

runtime.cmake now writes the OUTPUT_NAME it actually set to
<build-dir>/psxrecomp_exe_name-<target>.txt, and both consumers prefer that
over their own derivation. Written at configure time with file(WRITE) rather
than file(GENERATE): _psxrt_exe_name is a plain variable carrying no generator
expression and does not vary per configuration, so one fixed filename is
correct on multi-config generators too. The marker holds the name only, never a
path — the consumers already know their build directory and only ever got the
basename wrong.

A build tree without the marker still falls back to exe_basename, so older
trees and foreign build directories behave exactly as before.

The error messages were misleading in both directions and are now specific: a
name disagreement says the build succeeded and names both sides, while a
genuinely absent binary lists the executables that are present. The PGO
training path resolved the binary the same guessing way and now shares the
resolver.

Adds test_runtime_exe_name.py, which pins all four behaviours against the real
Revelations: Persona names, and relaxes the GAME_PROJECT_SETUP.md note that
previously just warned people to keep the two values in sync by hand.

Verified: the resolver against a real Ninja build tree whose OUTPUT_NAME and
marker were produced by the same two lines runtime.cmake uses, including the
exact stale-name case that failed in the wild; test_cli_project_packaging still
passes; the self-compiler change compiles clean. The runtime.cmake edit itself
was exercised in an isolated reproduction rather than a full runtime configure
— this checkout has no BIOS dump or generated game C to build one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017AwqKPQFnK3x7xQbK4PQKj
2026-08-28 23:48:56 -04:00
Matt Stanley 645253e7e5 Merge pull request #254 from Alexbeav/eagle-codex/t41-freeze-watchdog
runtime: bound automatic freeze dumps
2026-08-28 19:00:11 -07:00
Alexandros Mandravillis 0153fe13ef runtime: bound automatic freeze dumps 2026-08-29 04:33:56 +03:00
tetrisgm 3bf7bf1361 auto_ui_squash: join on submission order; instrument the silent gates
Three changes, one fix and two pieces of visibility.

1. ws_ui_group_assign gains a third join: consecutive in the ordering-table
walk AND touching. WipEout 3's speed/shield readout ends at x=288 and the
meter bar it belongs to starts at x=306 with a different CLUT, so neither
shared columns nor a matching key could join them. The readout stood alone,
its bbox midpoint 256.5 landed in the middle third against a screen centre of
254, and it stayed pinned near centre while its own meters went to the right
edge -- a 2 px gap in 4:3 opening to 159 framebuffer pixels at 32:9.

No anchor threshold can fix that: the genuinely centred "pit ... check"
message sits FARTHER RIGHT (midpoint 300.5) than the mis-anchored readout, so
any threshold reaching the readout drags the message with it. Submission order
is the only signal that separates them, and the array already carries it.

Simulated against seven captures spanning three aspects and two code states,
this forms three unions and changes exactly one anchor: [225,288] centre
becomes [225,486] right. The other two unions ([208,240]+[240,361],
[364,447]+[450,473]) are within an anchor. Confirmed in-race at 32:9: 7 runs,
lap readout left, speed readout and meters together on the right.

2. ws_ui_groups reports why primitives were NOT admitted to the partition.
Each gate in ws_ui_prepass_add is a silent return, so a primitive left at its
raw 4:3 X while its neighbours are squashed -- which is how a HUD mark ends up
stranded mid-screen -- was indistinguishable from one never drawn. In-race at
32:9: opcode 367, not_axis 410, everything else 0.

3. geom_correction reports perspective-UV arming with a real denominator.
perspective_triangles could previously only be compared against gp0_draw,
which counts untextured primitives that are correctly never armed, so it read
as a coverage figure without being one. texcorr.attempts counts exactly the
textured triangles reaching the predicate. In-race at 32:9: 199599 submitted,
96.15% armed, 3.85% rejected for a missing per-vertex Z, 0% for correction
disabled or a missing packet address.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit 572b9210d8)
2026-08-27 21:17:20 -07:00
Matthew Stanley 88f07bc9e3 runtime: keep temporal blending on the render thread 2026-08-27 13:21:17 -07:00
Alex Vanderveen 7efc712dd3 Windows/MinGW self-build: pin cxx_std_17 and stop capturing a structured binding
Two defects that only bite a MinGW C++17 build, both still live on master.

runtime.cmake declared only `c_std_11` on the runtime target, leaving the C++
dialect to whatever the parent project happened to set. A game CMakeLists that
omits CMAKE_CXX_STANDARD therefore compiled mod_packages.cpp as a pre-17
dialect. State the runtime's real requirement instead of inheriting it.

mod_packages.cpp captured a structured binding (`for (const auto& [id, ...])`
with `id` used inside a lambda). That is a C++20 extension; clang accepts it
silently, MinGW/GCC at C++17 rejects it. Bind the map entry to ordinary locals
before the lambda.

Cherry-picked from ae9432a3 on feat/rbengine-bak, minus that commit's unrelated
lib/recomp-net submodule bump. Verified by building psx-runtime on Linux.
2026-08-25 12:24:05 -04:00
Alex Vanderveen db2c03e109 hotfix for offline build deps
(cherry picked from commit 986de03fcf)
2026-08-25 11:17:44 -04:00
Alex Vanderveen e0037d7433 cmake patch hardening for steamos
(cherry picked from commit 8e0cad4685)
2026-08-25 11:17:44 -04:00
Matt Stanley d1df20514f Merge pull request #206 from mstan/fix/pr199-headless-no-recomp-ui-setup-host
runtime: fix headless generated-game builds without recomp-ui
2026-08-24 17:26:39 -07:00
Matthew Stanley 69f851f0f3 runtime: omit setup host sources without recomp-ui 2026-08-24 17:25:24 -07:00
Matthew Stanley 9ba703e5ed Merge remote-tracking branch 'origin/master' into split/pr169-presentation-bezel
# Conflicts:
#	runtime/runtime.cmake
2026-08-24 15:16:40 -07:00
Matthew Stanley c8e3dac678 presentation: expose bezel artwork as a mod 2026-08-24 15:15:15 -07:00
Tomasz Finc 2bd2208c8b Fix psxrecomp_add_game_runtime for --no-recomp-ui headless builds
psxrecomp_add_game_runtime() (added in #110) is unusable with
PSX_RECOMP_UI=OFF and real generated game C linked — a documented,
supported config ("PSX_RECOMP_UI OFF for headless/generated builds",
BUILDING.md) that nothing currently exercises. Found while bringing up
a new title with recomp-ui intentionally left out initially.

Two independent breaks in that combination:

1. main.cpp calls psx_game_codegen_forward_if_built() from main()
   whenever PSX_HAS_GAME_CODEGEN is set (i.e. any time real game C is
   linked), but only *declares* it under the narrower RECOMP_LAUNCHER
   guard. Any no-launcher build with generated game C fails with
   "use of undeclared identifier". Introduced a precise
   PSX_HAS_CODEGEN_SETUP_HOST macro (set only when a codegen-setup
   host is actually linked) and re-guarded both the declaration and
   the call site on it instead of the mismatched pair of guards.

2. psxrecomp_codegen_host.c was unconditionally added to every game
   target's extras, but it unconditionally #includes recomp_launcher.h
   from the recomp-ui submodule. Any --no-recomp-ui build fails with
   "recomp_launcher.h file not found". Gated its inclusion on
   PSX_RECOMP_UI.

Verification:
- Built and ran a new title (headless: PSX_RECOMP_UI=OFF, real
  generated game C, no launcher) end to end: OpenBIOS -> game entry ->
  continuous gameplay, clean atexit shutdown, no crash.
- Regression-checked the one existing title that actually calls
  psxrecomp_add_game_runtime() (TombaRecomp; every other shipped title
  calls the lower-level psxrecomp_add_runtime_target directly and
  never reaches this code): built TombaRecomp's real production config
  (PSX_RECOMP_UI=ON, ENABLE_SETUP_WIZARD, recomp-ui launcher, mod
  catalog, setup-host/no-disc mode) against this branch via
  -DPSXRECOMP_ROOT override, no submodule surgery. Clean build, ran to
  the launcher with no regressions. Both changes are conditionally
  gated on flags Tomba always sets the opposite way, so this is
  expected and was confirmed empirically, not just by inspection.

Not covered by this PR: CMAKE_CXX_STANDARD 17 is also never inherited
by a game repo that include()s runtime.cmake directly (as opposed to
add_subdirectory()-ing runtime/CMakeLists.txt), which silently
compiles text_xlate.cpp's std::filesystem usage against whatever the
host compiler's default standard is. Six of seven other title repos
work around this by setting it themselves; happy to send a follow-up
if that's wanted here instead of per-repo.
2026-08-23 23:12:24 -07:00
tetrisgm d10d76563a release: allowlist title framework packages
(cherry picked from commit 03aba66423)
2026-08-23 13:19:09 -07:00
Henrique Guedes aebb11b3d8 build: link iphlpapi into Windows runtime targets
main.cpp includes iphlpapi.h and calls GetAdaptersAddresses for the
launcher's netplay local-address discovery, but no CMake file links the
library. MinGW builds link regardless; MSVC does not link iphlpapi
implicitly, so every MSVC runtime build fails at link with an
unresolved GetAdaptersAddresses.
2026-08-21 21:39:12 -03:00
Alexandros Mandravillis 73eb746479 fix(runtime): validate full AOT entry before straight-line handoff
The dirty-RAM interpreter reused a suffix-aware validity check when resuming generated code after a patched call site. If an entry prologue had been overwritten but its resume suffix still matched, execution could enter stale AOT code and fall through to another generated piece without redispatch.

Emit a whole-entry predicate for straight-line handoffs, retain suffix checks for explicit guest transfers, and fail closed for older generated dispatchers until regeneration.

Validated with the MGS PAL 6,875-frame route, manual codec-to-heliport gameplay, focused continuation-guard tests, and a current-upstream recompiler build.
2026-08-20 14:51:26 +03:00
Matthew Stanley d31ce5ac6d pgxp: framework-owned PGXP Precision mod, every game, default off
psx.enhancement.pgxp (game_id \"*\") in the builtin catalog: one Mods-page
toggle that arms sub-pixel vertex precision + perspective-correct
texturing with the user-validated defaults (tolerance clamp 0.5px), plus
a cpu-mode option. Default off = the faithful floor (G1.9). Fully
effective on PGXP-variant binaries; reduced-but-safe coverage elsewhere
(the always-emitted swc2 tier + interpreters, validate-on-read).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 20:52:21 -07:00
Matthew Stanley e3824b47f5 pgxp: auto-clone the _pgxp variant inside psxrecomp_add_runtime_target
Titles that call psxrecomp_add_runtime_target directly (Ape Escape)
never went through the game wrapper, so the variant only existed for
wrapper callers. The clone now replays ARGN recursively at the end of
the target builder itself under PSX_PGXP_VARIANT, which covers every
caller; the wrapper folds an existing clone into its tail config and
the framework CMakeLists drops its now-redundant explicit block.

First real-title census (Ape Escape, worktree build vs this framework,
memory-mode only, boot->title->attract): 79-90% dataflow hits per
10s window vs the 5.2% position-cache ceiling, 0 value mismatches,
0 truncation rejects, every dataflow hit carrying depth; residual
native is 2D HUD/sprite content that must stay native. The G1.9
census gate is passed; visual screenshot_hires A/B requires a windowed
run (headless has no hi-res mirror - scale:1 fallback).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 19:17:30 -07:00
Matthew Stanley 38bdeeaade pgxp: emit hook macros + dual build targets + flavor-namespaced caches (phase 2)
Emitter side of the value-propagation engine (ENHANCEMENTS.md G1.10):

- recompiler/src/pgxp_hook_emitter.{h,cpp} (new, in the codegen-hash
  surface): ONE hook grammar shared by BOTH emitters, wrapping loads/
  stores/COP2 transfers and precision-carrying ALU ops (moves, add/sub,
  or-merge, shifts, LUI, HI/LO, muldiv) with PGXP_*() macros. Clobberable
  sources are captured into locals before the statement. Precision-
  destroying ops (AND/XOR/SLT/...) are deliberately unhooked - validate-
  on-read drops their stale shadows for free.
- code_generator.cpp: central append in translate_instruction; LWC2/SWC2
  emissions capture the RAW loaded/stored word (a masked GTE register
  write must validate against the word as loaded).
- strict_translator.cpp: translate() wraps translate_impl()'s c_code with
  the shared grammar; deferred load-delay variants hook the psx_ldd temp.
- BIOS regenerated, fingerprints verified CHANGED (SCPH1001 + OpenBIOS;
  42.6k/12.2k macro sites). Base runtime + 33/33 tests green against it.
- Gating: macros expand only under -DPSX_PGXP=1. runtime.cmake grows a
  PGXP option on psxrecomp_add_runtime_target (defines PSX_PGXP=1 +
  PSX_OVERLAY_FLAVOR=2, _pgxp exe suffix) and PSX_PGXP_VARIANT to build
  the sibling target from the same generated tree (framework runtime +
  game wrapper). No ABI version bump needed: the appended PGXPHooks
  table is NULL-tolerant and the flavor half of the ABI tag rejects any
  host/DLL flavor mix.
- Shard caches: flavor folded into the cache tag (cg<V>_<hash>_gc<hash>
  _f<flavor>) in overlay_loader.c + compile_overlays.py so base and pgxp
  caches never share a directory; compile_overlays.py passes
  -DPSX_PGXP=1 when flavor bit 2 is set; overlay preamble forwards
  psx_pgxp_* through the table. PSX_OVERLAY_CODEGEN_VER 9 -> 10.
- Env overrides PSX_GEOMETRY_CORRECTION / PSX_PERSPECTIVE_TEXTURING /
  PSX_PGXP_CPU_MODE (validation path: free-running headless boots arm
  from the first vertex); census grew produced/swc2_stores and the pgxp
  verb reports suppress/active gate state.

Validated: PSXRecomp_pgxp.exe boots the recompiled BIOS with hooks armed
(engine active, per-vertex lookups flowing; the BIOS shell has no RTPS
content so produced=0 there is correct - coverage measurement moves to a
3D title in phase 3).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 19:03:36 -07:00
Matthew Stanley 2ba74f3b4b pgxp: land clean-room value-propagation engine (phases 0+1, issue #92)
The G1.2/G1.3 mechanism: per-word RAM/scratchpad + per-GPR + per-GTE-reg
shadows carrying sub-pixel projections BY PROVENANCE, validated against
the actual guest word on every read (never guessed from rounded position
- the measured 93.1%-ambiguous G1.4 dead end). Clean-room from psx-spx +
public design docs; vendored duckstation (CC BY-NC-ND) and beetle-psx
(GPL) trees remain black-box oracles only - no code ported.

- runtime/src/pgxp.cpp + pgxp.h: engine, per-vertex GetPreciseVertex
  (dataflow -> ambiguity-gated position cache -> native) with truncation
  agreement + tolerance safeguards; stats; test accessors.
- pgxp_hooks.h: 5-entry hook funnel; PGXP_*() macros no-op unless
  -DPSX_PGXP=1 (codegen-time gate over ONE generated tree); appended
  PGXPHooks* table member in OverlayCallbacks (ABI bump ships with the
  Phase-2 emitter change).
- Both interpreters wired: loads/stores/COP2 + precision-carrying
  arithmetic (moves, add/sub, or-merge, 16-bit shifts, LUI, HI/LO).
- gte.cpp: RTPS feeds the GTE reg shadows; retired s_precise_sxy FIFO,
  hashed s_precision_store, and plain-store invalidation (redundant
  under validate-on-read; store hot path gets 3 calls back).
- gpu.c: prepare_precise_triangle rewritten per-vertex; perspective UVs
  share the same provenance and compose per triangle.
- [video] pgxp_cpu_mode / pgxp_tolerance keys; TCP pgxp verb live-
  toggles geometry/texture/cpu_mode/tolerance; geom_correction census
  grew the pgxp dataflow counters (G1.9 gate metric).
- tests: new pgxp_test (registered); gte_register_access_test updated
  to precise invalidation contract. 33/33 enabled tests green.

Also fixed while landing (both broke a fresh master tree):
- runtime.cmake: once-only psx_game_version.txt stamp (multi-target
  file(GENERATE) collision broke every fresh single-config configure).
- test_overlay_capture_retry: SDL3 rot (SDL_SetMainReady gone, inverted
  SDL_Init check) -> psx_sdl_init.

Tracked as beads-eio.3.48; postmortem + this landing: ENHANCEMENTS.md
G1-G1.10.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 18:34:47 -07:00