Commit Graph

2291 Commits

Author SHA1 Message Date
Alexandros Mandravillis ff4be21c90 bios: add the SCPH-5500 and SCPH-5501 profiles
The v3.0 Japanese (SCPH-5500, 09/09/96) and US (SCPH-5501, 11/18/96)
retail BIOS images join SCPH-1001, SCPH-101 and SCPH-5552, so every v3.0
region (J, A, E) has a profile.

Both images match SCPH-1001 v2.2 byte for byte in the kernel,
ROM [0x80, 0x18000). SCPH-5501's reset vector is identical too; SCPH-5500's
differs in 24 bytes. The shell differs in both. So each profile is
SCPH1001.toml with its own identity (name, id, rom, sha256 pin, stem): the
copy windows and install slots are kernel properties and carry over
unchanged. The shell runs through the dirty-RAM interpreter, as for
SCPH5552.

Each image gets its own seed list, derived by the same rule as
phase2_ghidra_seeds_SCPH5552.json: the SCPH-1001 corpus filtered to seeds
whose 64-byte ROM window is identical in both images (534 kept, 569
dropped), plus the two architectural vectors and 6 A0/B0/C0 kernel-table
targets. The files carry only addresses, labels and rationales, plus the
SHA-256 of both ROMs.

- psx_bios_known_images.h: CRC-32 FF3EEB8C (SCPH-5500) and 8D8CB7E4
  (SCPH-5501), both 512 KiB. A build that does not link these backends
  builds one from the player's own dump as a loadable module, like
  SCPH-5552, and the netplay lobby can name the image.
- main_cli.cpp: map SCPH5500 / SCPH5501 dumps to their profiles.
- build_cli.py: package both profiles with the CLI framework.
- docs/BIOS_SELECTION.md: list them among the module-built images.
2026-10-01 09:28:13 +03:00
Matt Stanley fd019bbc84 Merge pull request #450 from RetroPortingToolKit/fix/mmx6-composite-screen-tags-20261001
Keep Ilumina and other large screen composites assembled in widescreen
2026-09-30 21:40:37 -07:00
Matthew Stanley 757790debe Keep large screen-space composites anchored as one group 2026-09-30 21:38:17 -07:00
Alex Vanderveen c9445d00cb chore: sync 2026-09-30 23:52:33 -04:00
Matt Stanley 93f2a70965 Merge pull request #448 from RetroPortingToolKit/integrate/mmx6-adaptive-20261001
Integrate opt-in MMX6 adaptive renderer with current framework
2026-09-30 20:27:40 -07:00
Matthew Stanley 7c7133fbb4 Integrate MMX6 adaptive renderer with current framework master
Preserve current segmented dispatch, active-plan mod hooks and software scale limits. Extend function filters through the active plan, retain the last-store pointer with overlay ABI 25, and keep default config fingerprints stable. Refresh BIOS emitter stamps after byte-identical regeneration and reconcile regression fixtures.
2026-09-30 20:23:16 -07:00
Matt Stanley 6aafacbc43 Merge pull request #447 from RetroPortingToolKit/fix/ape-issue18-aot
Fix discovery of scheduled overlay prologues and delayed RA saves
2026-09-30 19:11:15 -07:00
Matthew Stanley 9ea70f3566 Merge current master while retaining the Ape Escape patch release pin 2026-09-30 19:09:45 -07:00
Matthew Stanley 6901aac851 Discover scheduled overlay prologues through their first basic block 2026-09-30 18:58:25 -07:00
Alex Vanderveen 9f72435dce migrate_bundled: a pre-moved submodule gitlink is not a dirty tree
Retro Studio's bulk tab bumps the title's psxrecomp pin first so the script
it runs is the one the title will pin; the clean-tree gate must not refuse
that. Submodule-internal changes are still refused at bump time.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-30 21:16:36 -04:00
Alex Vanderveen 11df3be1ff tools: migrate_bundled_release.py moves an existing title onto bundled releases
One pass per title: bump psxrecomp/recomp-ui to a ref that carries
docs/ci/templates/game-release.yml and the committed BIOS backends (refused
otherwise), un-ignore generated/, ensure [runtime] overlay_cache = true, emit
scripts/package_release.sh (retiring package_setup_release.sh) and
release.yml from the pinned template, rebuild emitters + regenerate the game
C from the title's disc, refresh framework_pins.txt, run check_boot_exe and
check_generated, commit; --push optional, --dry-run writes nothing.

The operation lives in project_studio/migrate_bundled.py; the tools/ script,
`git migrate-bundled` and `git bulk-migrate-bundled` (map_repos, per-repo
streaming) are its faces. Exercised end to end on a scratch copy of
Bomberman Party Edition (48 generated files committed, zip prefix kept,
generate_ci --check current).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-30 21:15:57 -04:00
Alex Vanderveen efcb48fbcb bios fingerprint: compute it with cmake -P, not a bash found on PATH
The windows-2022 release job accepted the committed backends in the step
shell (Git Bash) and refused them at configure: runtime.cmake ran the bash
fingerprint through whichever bash find_program() located, and that
environment hashed a byte-identical tree differently. The fingerprint is now
tools/bios_emitter_fingerprint.cmake, run with the configuring cmake itself
on every platform; the shell script is a thin wrapper around it. Same digest
shape as before, so existing stamps stay valid (verified on an LF tree and
an autocrlf clone).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-30 20:49:23 -04:00
Alex Vanderveen 52c7b7447b ci: verify BIOS backend stamps before configure, with per-input digests
The Windows release job reports the committed backends as stale while
Linux and macOS accept the same tree; the configure error names no input.
check_bios_stamps.sh compares each stem's stamp with the local fingerprint
and, on mismatch, prints every hashed input's digest and the tool versions,
so the next run attributes the difference to a file.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-30 20:44:54 -04:00
Alex Vanderveen aa4cfea81b bios fingerprint: hash sources with carriage returns stripped
Git for Windows checks text files out with CRLF, so the windows-2022 release
job hashed a byte-identical tree to a different value and BIOS_STALE_FATAL
refused the committed backends there while Linux and macOS passed. Stripping
CR before hashing makes the fingerprint line-ending independent (verified:
an LF tree and a CRLF copy of it now agree). Stamps rewritten to the new
canonical value; the generated C is unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-30 20:38:18 -04:00
Alex Vanderveen 34191cadcc generate_ci: a workflow still running a retired step is stale
--check only looked for template steps the installed file lacked, so a copy
that still ran the removed OpenBIOS emission (or the setup-host wipe) read as
current and failed on the runner.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-30 20:33:10 -04:00
Alex Vanderveen f180a32ec1 bios: commit the OpenBIOS and SCPH-1001 backends; releases link both
Decision (Alex, 2026-09-30): the recompiled BIOS carries the same risk as
the recompiled game, which titles already commit, so the retail backend is
compiled in rather than built on the player's machine. The retail image is
still never shipped: the runtime needs its data at run time and accepts only
the exact image the backend was built from, so the player supplies their
own dump as before.

- generated/{OpenBIOS,SCPH1001}_{full,dispatch}.c and their .emitter.sha
  stamps are tracked (.gitignore un-ignores exactly those); regenerated with
  the current emitter.
- bios_emitter_fingerprint.sh no longer hashes the ROM image: the profile's
  SHA-256 pin stands for it and the emitter refuses any other image, and CI
  has no dump, so hashing it made every CI configure read "stale".
- runtime.cmake: the staleness check covers every LINKED stem and
  -DPSXRECOMP_BIOS_STALE_FATAL=ON makes a stale stamp fail configure.
- game-release.yml: no CI-side OpenBIOS emission (generate_openbios.sh is
  gone); configure with BIOS_STALE_FATAL and assert both backends linked.
  check_generated.sh asserts the submodule carries both backends tracked
  and no dump is tracked anywhere.
- The player-side module build stays for a known image a build does not
  link (SCPH-5552).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-30 20:27:41 -04:00
Matthew Stanley ca14a118f0 fix(runtime): grow software present buffers for adaptive widths 2026-09-30 17:11:39 -07:00
Alex Vanderveen 170370a440 launcher: prepare a retail BIOS from the wizard with progress, not at Play
A buildable dump used to verify as usable and then block at Play behind a
modal while the module compiled. Now:

- ae_bios_verify reports a buildable, not-yet-cached dump as needs_regen,
  and the runtime wires recomp-ui's new bios_prepare_with_progress job
  (ae_bios_prepare) whenever the toolchain is present and no Generate host
  is: the launcher runs the build as a progress job from the wizard's
  "Prepare BIOS" button or the Play prompt, re-verifies, and resumes the
  wizard. Generate & rebuild is never offered for a BIOS in a bundled build.
- psx_bios_module.c gains a progress sink fed from the builder's output
  lines (dump / emit / compile / self-check / published map to stage
  percentages with readable messages); run_command streams lines on both
  platforms.
- The launch-time build stays as the fallback for --bios / --no-launcher.

Requires recomp-ui 9f86c49 (GameInfo.bios_prepare_with_progress).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-30 20:10:57 -04:00
Alex Vanderveen 60919f5a87 bios: build a retail backend from the player's own dump as a loadable module
Bundled releases link OpenBIOS only; retail-BIOS-derived C never ships.
"Bring your own BIOS" now works without it: when a dump matching a shipped
profile (psx_bios_known_images.h) is selected, the runtime builds that
backend on the player's machine with the overlay toolchain the release
already carries, caches it, and loads it as one more registered backend.

- runtime/include/psx_bios_module.h: the module contract. An overlay-style
  self-contained shared library (no imports from the executable) exporting
  psx_bios_module_{abi,codegen_hash,init,backend}; reaches the runtime only
  through OverlayCallbacks plus a new PsxBiosModuleCallbacks table (traps,
  dirty-RAM/kernel-bless probes, fntrace, log_probe) and pointer aliases of
  the globals the generated dispatcher touches. Its #defines rewrite the
  emitter's own extern lines, so the generated C compiles unchanged.
- runtime/include/psx_bios_module_glue.c.inc: module-side forwarders and the
  four exports, compiled after overlay_dispatch_preamble.c.inc.
- runtime/src/psx_bios_module.c: host side. Cache path
  <exe>/cache/bios/<os-arch>/bm<abi>_<codegen hash>_f<flavor>/<STEM>_<crc>,
  ABI + codegen-hash gated load, callback fill, and a blocking build via
  overlay_toolchain/bios_module_build.py (gcc via compiler.txt or PATH,
  else tcc), then psx_bios_register().
- psx_bios_backend.c: the registry is now a mutable table seeded from the
  CMake-generated psx_bios_builtin_registry by a constructor, with
  psx_bios_register(); every existing walker keeps working.
- main.cpp: validate_bios_for_launch acquires a module (announcing a first
  build) before declaring a mismatch; discovery accepts a buildable dump;
  the launcher verify reports it usable ("ready" / "compiled on first
  launch") with no Generate & rebuild; the BIOS row shows whenever the
  toolchain is present; bundled_only respects module support.
- emitter: the CPS-marker constructor is not emitted in a module build.
- overlay_api.h: OverlayCallbacks gains a struct tag;
  overlay_loader_callbacks() exposes the host table before loader init.
- tools/bios_module_build.py (ships in the toolchain): identity + profile
  SHA-256 gate, staged emitter tree, emit, three-TU compile, ctypes
  self-check, atomic publish. release_stage.py stages psxrecomp-bios, every
  bios/*.toml, the seeds and the builder into overlay_toolchain/.
- Tests: runtime/tests/test_bios_module_host.c against three fake modules
  (good / wrong ABI / wrong hash) on real dlopen; recompiler/tests/
  test_bios_module_build.py builds the bundled OpenBIOS end to end and reads
  the descriptor back; guards extended.

Verified on Twisted Metal 4 (bundled build + staged toolchain, headless):
first launch with --bios SCPH1001.BIN builds and loads the module and boots
on image=SCPH-1001; the second launch loads it from cache.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-30 19:26:29 -04:00
Matt Stanley c317f711b7 Merge pull request #446 from RetroPortingToolKit/test/netplay-bios-wire-20260930
tests: cover lobby BIOS CRC wire roundtrip and fresh local offers
2026-09-30 15:22:20 -07:00
Matthew Stanley 9fc5b50d19 Merge remote-tracking branch 'origin/master' into test/netplay-bios-wire-20260930 2026-09-30 15:21:24 -07:00
Matt Stanley fae2b5eab3 Merge pull request #439 from RetroPortingToolKit/feat/overlay-segment-keys
Segment-aware code E: overlay shards keyed by segment (capture schema v3, per-segment cache directories)
2026-09-30 15:07:12 -07:00
Matthew Stanley e63a23b74c Merge latest master for segment E; preserve completed IsC accuracy notes 2026-09-30 15:06:59 -07:00
Matt Stanley 93ad29cf75 Merge pull request #438 from RetroPortingToolKit/feat/segment-variants
Segment-aware code D: per-segment compiled variants from segment-qualified seeds; exact call contract
2026-09-30 15:06:12 -07:00
Matthew Stanley 4717aacd63 Merge latest master for segment D; retain both landed accuracy fixes 2026-09-30 15:05:40 -07:00
Matt Stanley 2a8f2521c6 Merge pull request #437 from RetroPortingToolKit/feat/kuseg-linked-exe
Segment-aware code C: compile KUSEG-linked EXEs for their link segment; exact dispatch
2026-09-30 15:04:18 -07:00
Matthew Stanley 913822f358 Merge latest master for segment C; retain IsC notes and bundled-release test 2026-09-30 15:04:16 -07:00
Matt Stanley 1b99be0b8e Merge pull request #436 from RetroPortingToolKit/refactor/emitter-runtime-pc
Segment-aware code B: route every baked guest PC through runtime_pc()
2026-09-30 15:02:46 -07:00
Matt Stanley 6e932d009e Merge pull request #419 from RetroPortingToolKit/feat/segment-aware-code
Design: segment-aware code (KUSEG/KSEG1 links, EPCs, fetch tags, uncached fetch)
2026-09-30 15:02:39 -07:00
Matthew Stanley 6534b3ddad Merge latest master for segment design; preserve landed IsC and segment notes 2026-09-30 15:02:17 -07:00
Matt Stanley fcc685ce1d Merge pull request #435 from RetroPortingToolKit/fix/isc-icache-invalidate
Apply cache-isolated (IsC) stores to the caches, as Beetle does
2026-09-30 15:00:19 -07:00
Matt Stanley befdfea2db Merge pull request #442 from RetroPortingToolKit/fix/windows-framework-validation-20260930
fix: unblock Windows framework regression builds and tests
2026-09-30 14:58:38 -07:00
Alex Vanderveen 9a5c6e0eab docs: BIOS selection in bundled builds, and the planned player-side backend build
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-30 17:55:18 -04:00
Alex Vanderveen 0b6baa49c4 launcher: never offer "Generate & rebuild" for a BIOS in a bundled build
A shipped bundled build links OpenBIOS only and carries no CLI, sources
or toolchain, yet the codegen host forced the BIOS row on and a retail
pick answered with needs_regen ("Generate & rebuild to switch"), a step
that cannot run there.

- The BIOS row is shown only when a choice can mean something: a retail
  backend is linked, nothing is linked yet (setup host), or the codegen
  host actually wired Generate (prepare_with_progress). main.cpp decides
  after host_apply instead of the host forcing has_bios.
- ae_bios_verify short-circuits a bundled-only build with a plain
  "runs its bundled OpenBIOS only" message before the identity checks,
  whose "this build expects SCPH-1001" wording names the pinned stem,
  not a linked backend, and sets needs_regen only when Generate exists.
- test_bios_selection_guards pins both.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-30 17:54:53 -04:00
Matthew Stanley 243cc855ba tests: use native Git Bash for bundled release fixtures on Windows 2026-09-30 14:37:57 -07:00
Matthew Stanley c472e4ee3b tests: exercise lobby BIOS CRC wire roundtrip and fresh local offers 2026-09-30 14:30:38 -07:00
Matthew Stanley 962d949f1e Merge remote-tracking branch 'origin/master' into fix/framework-tests-followup-20260930
# Conflicts:
#	runtime/tests/test_packagers_never_format_cache_tag.py
2026-09-30 14:27:29 -07:00
Alex Vanderveen 74325e7446 package_game_release: ship the whole staged assets/ tree; allow BIOS profile TOMLs
The icon copy iterated a literal path nullglob does not cover and died on
the first real title (runtime.cmake stages the icon at assets/psxrecomp.png,
not beside the exe). And the dump gate rejected overlay_toolchain/bios/
SCPH1001.toml, a profile the toolchain needs; only image files are
forbidden.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-30 17:23:57 -04:00
Alex Vanderveen 61aa87f04c cli: prefer the project's build-recompiler emitters over the framework's build dir
A stale psxrecomp/recompiler/build/psxrecomp-game from August shadowed the
build-recompiler tree built from the pinned submodule, so generate emitted
a dispatch calling psx_ram_canon_code_addr, which the pinned runtime had
removed; the bundled release failed at link on every platform. The
project's emitter tree is what CI and the scaffold build, so it wins, and
the resolved paths are logged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-30 17:21:25 -04:00
Alex Vanderveen a7c0db2c63 tests: register test_bundled_release_layout with CTest
check_test_registration.cmake fails configure for an unregistered test
file, which broke every title's emitter build at the new pin.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-30 17:15:01 -04:00
Alex Vanderveen 874df9b5bb release: ship bundled, compiled games from committed generated/ C
Retire the setup-host release model (CI wiped generated/ and shipped
emitters + sources + a wizard so every player regenerated and rebuilt the
title). Titles now commit generated/ and CI ships the compiled game.

- docs/ci/templates/game-release.yml replaces setup-release.yml: gate on
  committed game C (tools/ci/check_generated.sh), emit the OpenBIOS backend
  in CI (tools/ci/generate_openbios.sh), configure with
  PSXRECOMP_REQUIRE_GAME_C=ON and assert the full path was taken, package
  with tools/package_game_release.sh, verify the zip is a game not a kit.
- tools/package_game_release.sh: exe, assets, bundled OpenBIOS, configs, mod
  catalog + overlay toolchain via release_stage.py, notices; refuses sources,
  emitters, generated C, BIOS dumps. clear_generated.sh removed;
  package_setup_host.sh marked retired.
- Scaffold: gitignore templates track generated/, package_release.sh.in
  wrapper, setup_project.{sh,ps1} commit generated/ after Generate.
- Project Studio: audit flags ignored/untracked generated/, retail BIOS C
  and setup-host workflows; merge_gitignore un-ignores generated/; ops emit
  the bundled packager + workflow; generate_ci uses the new template.
- Docs: docs/ci/BUNDLED_RELEASES.md replaces HOST_ONLY_RELEASES.md.
- Tests: tools/tests/test_bundled_release_layout.py; fix pre-existing
  false positives in the packager-surface and overlay-static tests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-30 17:11:14 -04:00
Matthew Stanley 50e292748b tests: keep synthetic DLL loader failures noninteractive on Windows 2026-09-30 14:07:37 -07:00
Alex Vanderveen 2b3f8c66c5 Merge pull request #445 from Alexbeav/up/netplay-exit-reason
netplay: say why a match ended, and end a boot mismatch in 3 s
2026-09-30 16:58:02 -04:00
Alex Vanderveen d8b644bf6d Merge pull request #444 from Alexbeav/up/netplay-bios-identity
netplay: settle the lobby BIOS by image, not by "any retail"
2026-09-30 16:52:27 -04:00
Alex Vanderveen 4ea620deb0 Merge pull request #243 from Alexbeav/pr/scph5552-profile
bios: replace the stale SCPH5552 profile
2026-09-30 16:05:24 -04:00
Matthew Stanley f77e7a570c test: read generated overlay C fixtures byte-preservingly 2026-09-30 13:00:11 -07:00
Matthew Stanley 7450dee7dd fix: unblock Windows runtime builds and current framework tests 2026-09-30 12:55:56 -07:00
Matt Stanley c604cea4d5 Merge pull request #432 from Cyanoxide/feat/configurable-depth24-trailing-margin
Make FMV depth24 trailing margin configurable per title
2026-09-30 11:09:04 -07:00
Alex Vanderveen f8b4773cd4 Merge pull request #440 from Alexbeav/up/netplay-libjuice-offline
build: vendor libjuice so netplay builds configure offline
2026-09-30 09:59:16 -04:00
Alex Vanderveen 451e3d1828 Merge pull request #441 from Alexbeav/up/lobby-chat-ring-test
tests: pass mid and account to chat_push in the lobby chat fixture
2026-09-30 09:58:58 -04:00