release: ship bundled, compiled games from committed generated/ C

Retire the setup-host release model (CI wiped generated/ and shipped
emitters + sources + a wizard so every player regenerated and rebuilt the
title). Titles now commit generated/ and CI ships the compiled game.

- docs/ci/templates/game-release.yml replaces setup-release.yml: gate on
  committed game C (tools/ci/check_generated.sh), emit the OpenBIOS backend
  in CI (tools/ci/generate_openbios.sh), configure with
  PSXRECOMP_REQUIRE_GAME_C=ON and assert the full path was taken, package
  with tools/package_game_release.sh, verify the zip is a game not a kit.
- tools/package_game_release.sh: exe, assets, bundled OpenBIOS, configs, mod
  catalog + overlay toolchain via release_stage.py, notices; refuses sources,
  emitters, generated C, BIOS dumps. clear_generated.sh removed;
  package_setup_host.sh marked retired.
- Scaffold: gitignore templates track generated/, package_release.sh.in
  wrapper, setup_project.{sh,ps1} commit generated/ after Generate.
- Project Studio: audit flags ignored/untracked generated/, retail BIOS C
  and setup-host workflows; merge_gitignore un-ignores generated/; ops emit
  the bundled packager + workflow; generate_ci uses the new template.
- Docs: docs/ci/BUNDLED_RELEASES.md replaces HOST_ONLY_RELEASES.md.
- Tests: tools/tests/test_bundled_release_layout.py; fix pre-existing
  false positives in the packager-surface and overlay-static tests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Alex Vanderveen
2026-09-30 17:11:10 -04:00
parent 2b3f8c66c5
commit 874df9b5bb
42 changed files with 1872 additions and 690 deletions
+2 -2
View File
@@ -27,9 +27,9 @@ record; the submodule pointer is the source of truth). The framework evolves on
its own cadence; a game only moves to a newer framework when someone deliberately
bumps that submodule pointer.
**Starting a new title or shipping a setup-host zip?** Follow
**Starting a new title or shipping a release?** Follow
[`docs/GAME_PROJECT_SETUP.md`](docs/GAME_PROJECT_SETUP.md) (layout, CI template
under `docs/ci/templates/setup-release.yml`, release checklist). Also see
under `docs/ci/templates/game-release.yml`, release checklist). Also see
[Linking the framework](docs/BUILDING.md#linking-the-framework) and
[Framework changes and the pin](#framework-changes-and-the-pin) below.
+3 -3
View File
@@ -58,7 +58,7 @@ see [Release Package](#release-package) below.
Bringing up a title of your own? Start with
[`docs/GAME_PROJECT_SETUP.md`](docs/GAME_PROJECT_SETUP.md) (submodules,
setup-host CI template, release checklist), then
bundled-release CI template, release checklist), then
[`CONTRIBUTING.md`](CONTRIBUTING.md). Community projects are listed here
alongside the rest.
@@ -230,7 +230,7 @@ These are the 3 most important folders to be aware of:
| Build the framework | [`docs/BUILDING.md`](docs/BUILDING.md) |
| **Ship a game repo** (submodules + CI + release checklist) | [`docs/GAME_PROJECT_SETUP.md`](docs/GAME_PROJECT_SETUP.md) |
| **Netplay** (rollback, SFU/ICE, dual-raster, disc gates) | [`docs/NETPLAY.md`](docs/NETPLAY.md) |
| Setup-host CI template | [`docs/ci/templates/setup-release.yml`](docs/ci/templates/setup-release.yml) |
| Bundled-release CI template | [`docs/ci/templates/game-release.yml`](docs/ci/templates/game-release.yml) |
| Local Generate & rebuild CLI | [`docs/LOCAL_CODEGEN_SDK.md`](docs/LOCAL_CODEGEN_SDK.md) |
| Mods | [`docs/MOD_PACKAGES.md`](docs/MOD_PACKAGES.md) |
| Widescreen / native-wide | [`docs/WIDESCREEN.md`](docs/WIDESCREEN.md) |
@@ -813,7 +813,7 @@ regenerate), and a change proves itself against the Beetle oracle / on screen
rather than by assertion. Game-specific work lives in the game repos, which pin
exact framework and UI commits as root-level submodules.
- New title / setup-host release:
- New title / bundled release:
[`docs/GAME_PROJECT_SETUP.md`](docs/GAME_PROJECT_SETUP.md)
- Framework PRs: [`CONTRIBUTING.md`](CONTRIBUTING.md) (rules, verification,
regression checklist, how a fix reaches a game through its pin)
+5 -3
View File
@@ -191,9 +191,11 @@ with exactly the audited files plus `AOT_STATIC_AUDIT.json`. Point the game's
runtime flavor the DLL release cannot target (for example a PGXP build); pass
`--cps` when the runtime is continuation-passing, as the compiler requires.
**Players get it from Generate.** Titles ship without game bytes, so the shard
has to be built on the player's machine. A profile opts in by declaring where
the build links it:
**It ships compiled.** The shard is generated on the developer's machine,
committed with the rest of `generated/`, and linked by release CI into the
shipped executable (`docs/ci/BUNDLED_RELEASES.md`); the overlay *cache* is
never built in CI, so this shard is the native overlay coverage a release
carries. A profile opts in by declaring where the build links it:
```json
"static_output": "generated/overlays_static.c"
+81 -106
View File
@@ -2,8 +2,10 @@
This guide is for **title developers**: you keep game code and config in your
own repository, pin this framework and `recomp-ui` as **root-level submodules**,
and (optionally) ship a **setup-host** zip that users generate from a legal disc
locally — the same model as Bomberman Party Edition / Masters of Teräs Käsi.
commit the recompiled game C under `generated/`, and ship a **bundled** zip:
the compiled game, built by CI from that committed C. Players bring their own
disc and play; nothing is generated or built on their machine
(`docs/ci/BUNDLED_RELEASES.md`).
Related docs:
@@ -79,7 +81,7 @@ Non-interactive / CI (`--yes` / `-Yes` or `PSXRECOMP_SETUP_YES=1`): requires
| Netplay lobby URL? | Only if netplay = Y (players ≥ 2); default host `netplay.retcomm.net` → `ws://…:8765` |
| GitHub Actions release workflow? | Y → `.github/workflows/release.yml` (logs submodule SHAs via `record_pins.sh`) |
| Fetch libretro boxart? | Needs network |
| Run Generate now? | Emitters + OpenBIOS + game C (`generated/` gitignored) |
| Run Generate now? | Emitters + OpenBIOS + game C; `generated/` is **committed** (release CI builds from it) |
| Configure & build after Generate? | Only if Generate = Y |
| Create GitHub repo (`gh`)? | Opt-in; needs `gh` auth |
@@ -196,8 +198,8 @@ disc identity, digests and netplay gates.
Older titles (e.g. `psxrecomp-v4` submodule, `psxrecomp_add_runtime_target`,
`packaging/` prebuilt zips) can be audited and migrated onto this layout with
**Project Studio**. Releases remain **setup-host only** — the tool does not
create prebuilt generated-C packages.
**Project Studio**. Releases are **bundled**: the tool un-ignores and expects
committed `generated/`, and emits the packager + CI that ship the compiled game.
| Entry | Role |
| ----- | ---- |
@@ -214,13 +216,13 @@ python3 tools/new_project_layout/migrate_project.py gui
```
Typical apply order: rename `psxrecomp-v4` → emit `codegen_setup` → rewrite
CMake (`psxrecomp_add_game_runtime` + wizard) → setup-host packager + CI →
CMake (`psxrecomp_add_game_runtime` + wizard) → bundled packager + CI →
optional `probe_disc` / pins. CMake rewrites keep
`CMakeLists.txt.pre_migrate.bak` plus `CMakeLists.migrate_extras.txt` when the
old file had tests / `EXTRAS_SOURCES` / mod `POST_BUILD` hooks.
`apply` always enables **recomp-ui + setup wizard** (required for
`PSXRECOMP_FORCE_SETUP_HOST`). Netplay stays opt-in (`--enable-netplay` / GUI).
`apply` always enables **recomp-ui + setup wizard** (the shipped game's
first-run disc picker). Netplay stays opt-in (`--enable-netplay` / GUI).
### After scaffold (still not automatic)
@@ -230,9 +232,9 @@ playable tree (OpenBIOS / optional retail BIOS C). You still must:
1. **Boot / soak** — fix missing seeds, overlays, FMV/runtime quirks in `game.toml`
2. **Netplay QA** — LAN then lobby; confirm digests + TOC fp; pin `VERSION`
3. **Polish** — more symbols in `symbols.toml`, boxart name mismatches
4. **Ship** — scaffold already creates the repo and pushes once at the end when
you opt in; otherwise push manually. Enable Actions, tag `vX.Y.Z` (CI ships
setup-host **without** `generated/` — end users run Generate locally / via wizard)
4. **Ship** — commit `generated/` (the scaffold does this after a successful
Generate), push, enable Actions, tag `vX.Y.Z`. CI builds the committed C and
attaches the compiled game for every platform.
**Legacy layout** (CLI `psxrecomp build`, nested UI under older trees,
`tools/setup_dev.sh` for framework-only boots) remains supported. Prefer the
@@ -332,20 +334,20 @@ YourGameRecomp/ # your git repo
│ # your own POST_BUILD command — see
│ # docs/MOD_PACKAGES.md ("How bundled/ gets staged").
├── scripts/
│ └── package_setup_release.sh # scaffold fills from package_setup_release.sh.in
│ └── package_release.sh # scaffold fills from package_release.sh.in
├── .github/workflows/
│ └── release.yml # scaffold fills from docs/ci/templates/setup-release.yml
│ └── release.yml # scaffold fills from docs/ci/templates/game-release.yml
├── psxrecomp/ # submodule → framework + CLI + codegen host + CI tools
│ ├── host/psxrecomp_codegen_host.*
│ └── lib/recomp-net/ # nested submodule (netplay)
├── recomp-ui/ # submodule → launcher UI
├── generated/ # local only — gitignore (not in CI setup zip)
├── generated/ # COMMITTED recompiled game C — release CI builds it
└── disc/ # local disc working tree — gitignore
```
Minimal `CMakeLists.txt` shape for a **setup-host** title (CI zip / first-run
Generate & rebuild). Wizard is required whenever you ship
`PSXRECOMP_FORCE_SETUP_HOST=ON` — without it the launcher never opens first-run.
Minimal `CMakeLists.txt` shape. The wizard is the shipped game's first-run disc
picker; `GEN_MARKER` / `GEN_FULL_GLOB` name the committed `generated/` C, which
release CI requires (`-DPSXRECOMP_REQUIRE_GAME_C=ON`).
```cmake
set(PSXRECOMP_ROOT "${CMAKE_CURRENT_SOURCE_DIR}/psxrecomp")
@@ -407,12 +409,16 @@ After scaffold (or a manual clone):
python3 psxrecomp/psxrecomp_cli.py generate \
--config game.toml --project-root . --disc disc/game.cue
```
5. **Build the playable runtime**:
5. **Commit the generated C** — it is the release's input:
```bash
git add generated && git commit -m "Regenerate game C"
```
6. **Build the playable runtime**:
```bash
cmake -S . -B build-release -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build build-release --target psx-runtime -j"$(nproc)"
```
6. **Soak** — offline boot → LAN netplay → lobby. Tune `[runtime]` quirks only
7. **Soak** — offline boot → LAN netplay → lobby. Tune `[runtime]` quirks only
after you have evidence (see MotK `game.toml` comments for the pattern).
Details: `[LOCAL_CODEGEN_SDK.md](LOCAL_CODEGEN_SDK.md)`, `[BUILDING.md](BUILDING.md)`,
@@ -420,34 +426,24 @@ Details: `[LOCAL_CODEGEN_SDK.md](LOCAL_CODEGEN_SDK.md)`, `[BUILDING.md](BUILDING
---
## Setup-host releases (self-build zip)
## Bundled releases (compiled game zip)
CI ships a zip **without** `generated/` game C, retail BIOS dumps, or an
embedded `toolchain/`. The zip includes:
CI builds the committed `generated/` C and ships a zip that is the game:
- Setup host exe (`recomp-ui` + generate/rebuild wizard)
- Your game sources (`game.toml`, seeds, CMake, host glue, …)
- `psxrecomp/` (runtime + CLI + OpenBIOS profiles + emitters)
- `recomp-ui/` sources (needed to rebuild)
- On Windows: MinGW runtime DLLs beside the host and emitters
- The game executable (signed on Windows when a certificate is configured)
- `assets/`, `mods/bundled/` (catalog verified against the build's manifest),
`game.toml` / `game_options.toml`
- `bios/openbios.bin` + its MIT notice — the release runs on OpenBIOS
- `overlay_toolchain/` so overlays outside the compiled static shard can be
turned native from the player's disc at runtime
- `licenses/` (third-party notices), `README.txt`
Players (or [Retro](https://github.com/RetroPortingToolKit/Retro-Launcher))
run **Generate once** (wizard or Retro Build & Install) with a legal disc.
Retro / the wizard download `cmake-clang-v1` from
[retcomm-toolchains](https://github.com/RetroPortingToolKit/RetroPorting-Toolchains)
(or accept an offline zip / `RETCOMM_TOOLCHAIN_DIR`). Pass
`--embed-toolchain` to `package_setup_host.sh` only for special offline-first
packs.
### Player updates (after first Generate)
| Action | Meaning |
|--------|---------|
| **Update** (Retro) | New setup-host zip → refresh source → cmake rebuild. Skips disc→C when `codegen-cache` fingerprints (ROM/BIOS/emitters) still match. |
| **Generate & Rebuild** | Force regenerate game C from the disc, then rebuild. Use when emit inputs change or cache is wrong. |
Ordinary host/UI releases do **not** require Generate & Rebuild. Details:
[`ci/HOST_ONLY_RELEASES.md`](ci/HOST_ONLY_RELEASES.md). CI tip: leave the
It contains **no** sources, emitters at the root, CLI, `psxrecomp/` or
`recomp-ui/` tree, generated C, disc, or BIOS dump. Players (or
[Retro](https://github.com/RetroPortingToolKit/Retro-Launcher)) extract it,
pick their legal disc on first run, and play; an update is a raw zip extract
over the previous one. Details and the exact CI steps:
[`ci/BUNDLED_RELEASES.md`](ci/BUNDLED_RELEASES.md). CI tip: leave the
`psxrecomp` gitlink pinned and use `reuse_cached_emitters` so release jobs skip
rebuilding emitters when only host sources moved.
@@ -461,13 +457,14 @@ Do **not** set `PSX_PGO` in CI. PGO stays user-local when `[pgo] enabled = true`
| Tool | Role |
| ------------------------------- | -------------------------------------------------------- |
| `tools/ci/normalize_version.sh` | `vX.Y.Z` → `VERSION` / `TAG` |
| `tools/ci/clear_generated.sh` | Wipe `generated/` for setup-host CI |
| `tools/ci/check_generated.sh` | Committed game C present + tracked; no retail BIOS C |
| `tools/ci/generate_openbios.sh` | Emit the OpenBIOS backend in CI from the bundled image |
| `tools/ci/record_pins.sh` | Log submodule SHAs |
| `tools/ci/build_emitters.sh` | Build `psxrecomp-game` + `psxrecomp-bios` |
| `tools/fetch_toolchain.sh` | Optional download/unpack (embed packs only) |
| `tools/stage_setup_sdk.sh` | Emitters + OpenBIOS + optional `toolchain/` + MinGW DLLs |
| `tools/fetch_toolchain.sh` | Windows emitter build on the CI machine (never shipped) |
| `tools/bundle_mingw_dlls.sh` | Windows runtime DLL copy |
| `tools/package_setup_host.sh` | Full setup-host zip (title args) |
| `tools/package_game_release.sh` | Bundled zip: compiled game + runtime data (title args) |
| `tools/release_stage.py` | Mod catalog / overlay toolchain / overlay cache staging |
Composite actions (from the game repo after checkout):
@@ -502,16 +499,16 @@ powershell -File psxrecomp\tools\generate_ci.ps1
It is the single-purpose face of Project Studio's `emit_ci_workflow`: the
template comes from the project's own psxrecomp submodule, the zip prefix from
`scripts/package_setup_release.sh` so CI uploads the zips the packager built,
`scripts/package_release.sh` so CI uploads the zips the packager built,
and an installed workflow is compared by step name -- `--check` exits 1 when
it is missing or stale, and only `--force` overwrites one.
The New Project Layout scaffold **copies and fills** this for you after
submodules land:
- `.github/workflows/release.yml` ← `docs/ci/templates/setup-release.yml`
- `.github/workflows/release.yml` ← `docs/ci/templates/game-release.yml`
(Linux x64, Windows x64, macOS arm64 + Intel)
- `scripts/package_setup_release.sh` ← zip prefix / exe / display name
- `scripts/package_release.sh` ← zip prefix / exe / display name
`--exe-name` / `codegen_setup.exe_basename` should match CMake `OUTPUT_NAME`
(`MAKE_C_IDENTIFIER(WINDOW_TITLE)`, e.g. `TwistedMetal4_Recompiled`), not the
@@ -538,16 +535,16 @@ Manual install (existing repos):
```bash
mkdir -p .github/workflows scripts
cp psxrecomp/docs/ci/templates/setup-release.yml .github/workflows/release.yml
cp psxrecomp/docs/ci/templates/game-release.yml .github/workflows/release.yml
# replace YOUR_ZIP_PREFIX / YOUR_GAME_TITLE / yourgame-release
# add scripts/package_setup_release.sh (see MotK / BPE)
# add scripts/package_release.sh (fill tools/new_project_layout/templates/package_release.sh.in)
```
Or re-run token fill against the submodule template:
```bash
python3 psxrecomp/tools/new_project_layout/fill_tokens.py \
psxrecomp/docs/ci/templates/setup-release.yml \
psxrecomp/docs/ci/templates/game-release.yml \
.github/workflows/release.yml \
--ci-placeholders \
--set ZIP_PREFIX=mygame \
@@ -560,8 +557,7 @@ Full action reference: `[ci/README.md](ci/README.md)`.
## Bundled release checklist
Use this before tagging a setup-host release that matches other titles
(BPE / MotK / Retro).
Use this before tagging a release.
### Repository
@@ -584,73 +580,52 @@ Use this before tagging a setup-host release that matches other titles
```
`VERSION` matches the release you will tag
```
- [ ] Disc images, `generated/`, BIOS, logs, and runtime dumps are gitignored
```
(scaffold writes a rich `.gitignore`; see also `docs/ci/templates/game.gitignore`)
```
- [ ] Disc images, BIOS, logs, and runtime dumps are gitignored; `generated/`
is **not** (scaffold writes a rich `.gitignore`; see also
`docs/ci/templates/game.gitignore`)
- [ ] `generated/<boot>_dispatch.c` + `_full_*.c` (+ `overlays_static*.c` when
the AOT profile declares it) are committed and current for the pinned
framework (`tools/ci/check_generated.sh` passes locally)
- [ ] No `generated/SCPH*` anywhere (retail-BIOS-derived C never ships)
- [ ] Submodule gitlinks (`psxrecomp` / `recomp-ui` / nested `recomp-net`) are
```
the SHAs you intend to ship (CI builds those; `record_pins.sh` only logs)
```
- [ ] Setup-host CMake path builds with **no** game C and **no** BIOS backends
```
CI: `-DPSXRECOMP_FORCE_SETUP_HOST=ON -DPSXRECOMP_ALLOW_NO_BIOS=ON
-DPSX_SETUP_WIZARD=ON` (CMakeLists must also set `PSX_SETUP_WIZARD` /
`ENABLE_SETUP_WIZARD` — FORCE_SETUP_HOST alone does not open the wizard;
`psxrecomp_add_game_runtime` FATAL_ERRORs if FORCE_SETUP_HOST is on without
the wizard). clear_generated.sh wipes OpenBIOS C; users Generate via wizard.
```
- [ ] Full build configures from the committed C: CI passes
`-DPSXRECOMP_REQUIRE_GAME_C=ON -DPSX_SETUP_WIZARD=ON` and asserts the
configure log says `linking generated game C (full runtime)` and
`BIOS backends linked: OpenBIOS`
- [ ] `[runtime] overlay_cache = true` in `game.toml` (the packager refuses
otherwise)
- [ ] Thin `codegen_setup.c` + `psxrecomp_add_game_runtime` (codegen host is in
```
`psxrecomp/host/`). Must export `psx_game_codegen_forward_if_built` (see
`codegen_setup.c.in`) — setup-host CI links `main.cpp`, which always calls it.
```
`codegen_setup.c.in`) — `main.cpp` always calls it; in a full build it is a
no-op
### Packaging (shared helpers)
- [ ] CI uses `./psxrecomp/.github/actions/build-emitters`
- [ ] Packager calls `psxrecomp/tools/package_setup_host.sh` (lean zip by
```
default; optional `--embed-toolchain`) or `stage_setup_sdk.sh` after a
custom stage
```
- [ ] Staged tree includes `psxrecomp/psxrecomp_cli.py`
- [ ] Staged `psxrecomp/bios/` has `OpenBIOS.toml`, `openbios.bin`,
```
`OpenBIOS.LICENSE`, `SCPH1001.toml` — and **no** retail `.BIN`
```
- [ ] Windows zip has `libstdc++-6.dll` + `libgcc_s_seh-1.dll` next to both
```
emitters (and host deps such as `zlib1.dll` next to the exe)
```
- [ ] Zip mtimes are normalized (packager `touch`) so Ninja does not see
```
“future” files after extract
```
- [ ] CI uses `./psxrecomp/.github/actions/build-emitters` (OpenBIOS emission
and the shipped overlay toolchain need them)
- [ ] `scripts/package_release.sh` calls `psxrecomp/tools/package_game_release.sh`
- [ ] Staged tree has the game exe, `psx_game_version.txt`, `assets/`,
`mods/bundled/`, `game.toml`, `bios/openbios.bin` + `OpenBIOS.LICENSE`,
`overlay_toolchain/`, `licenses/` — and **no** `psxrecomp/`, `recomp-ui/`,
CLI, root-level emitters, sources, generated C, retail `.BIN`
- [ ] Windows zip has the runtime DLLs the exe imports beside it
(`bundle_mingw_dlls.sh`), signed when a certificate is configured
### Zip contents smoke test
- [ ] Linux / macOS / Windows artifacts named consistently
```
(`YOUR_PREFIX-<ver>-<linux-x64|windows-x64|macos-arm64|macos-x64>.zip`)
```
- [ ] Extract → run host → Generate with a legal disc succeeds end-to-end
- [ ] After rebuild, game launches; saves/settings land beside the exe
- [ ] Extract → run → pick a legal disc → the game boots; saves/settings land
beside the exe
- [ ] Retro install/update (if you publish a catalog entry) uses this **same**
```
zip — no separate tools pack required; Update rebuilds via codegen-cache
(not raw zip extract over a Play binary)
```
- [ ] Release notes: first Generate once; later Updates are host/UI rebuilds
```
unless emitters/ROM/BIOS fingerprints change (see ci/HOST_ONLY_RELEASES.md)
```
zip as a prebuilt Play binary (raw zip extract; no build step)
### Publish
- [ ] Tag `vX.Y.Z`; GitHub Release attaches all four platform zips
- [ ] Release notes say the zip is a setup host (no disc / no retail BIOS /
```
no pre-generated game C)
```
- [ ] Release notes say the zip is the compiled game: bring your own disc,
no BIOS, no build
- [ ] Catalog / Retro entry points at the release assets when ready
+18 -13
View File
@@ -7,17 +7,21 @@ automation. This does **not** redistribute disc images.
PGO (optional) runs **only on the user’s machine** during local rebuild when
`game.toml` has `[pgo] enabled = true`. CI must not set `PSX_PGO`.
## Setup host (CI without game/BIOS generated C)
## Where this sits now (bundled releases)
Games can ship a **setup host**: `psx-runtime` linked **without** game C and
**without** BIOS backends (`-DPSXRECOMP_FORCE_SETUP_HOST=ON`, or legacy
`-DBPE_FORCE_SETUP_HOST=ON`) via `psxrecomp_add_game_runtime(...)`, and with
`PSX_SETUP_WIZARD=ON` / `ENABLE_SETUP_WIZARD` so first-run Generate & rebuild
actually appears. CI never
needs BIOS dumps or private assets. First-run Generate emits OpenBIOS (from
bundled `openbios.bin`) and optional SCPH1001 (player dump), then game C, then
rebuild links everything into `build-release/` (or the title’s
`build_dir_name`).
Public releases are **bundled**: the title commits `generated/`, CI builds the
compiled game and ships it (`docs/ci/BUNDLED_RELEASES.md`). Players never run
Generate. The contract below is the **developer** and Retro-automation path
for producing that committed C, and the runtime's own Generate & rebuild
wizard for a locally built setup host (`-DPSXRECOMP_FORCE_SETUP_HOST=ON`,
a developer convenience — not what CI ships).
A setup host is `psx-runtime` linked **without** game C and **without** BIOS
backends via `psxrecomp_add_game_runtime(...)`, with `PSX_SETUP_WIZARD=ON` /
`ENABLE_SETUP_WIZARD` so first-run Generate & rebuild appears. First-run
Generate emits OpenBIOS (from bundled `openbios.bin`) and optional SCPH1001
(local dump), then game C, then rebuild links everything into `build-release/`
(or the title’s `build_dir_name`).
**Layout after Generate & rebuild:** the zip-root exe stays the setup host;
the playable product (exe + `bios/` + `mods/` + `assets/` + `settings.toml`)
@@ -31,15 +35,16 @@ product binary (`psxrecomp_codegen_host_forward_if_built`). Opt out with
| Product exe | `build-release/<exe>` — Play, bios/mods/assets/settings |
| Game zip `psxrecomp/` | submodule tree: CLI, tools, emitters, OpenBIOS profiles |
| `psxrecomp_cli.py` | Generate / rebuild / verify-disc (in the submodule) |
| `tools/package_setup_host.sh` | Universal setup-host zip packager |
| `tools/ci/*.sh` | normalize version, clear generated, record pins, build emitters |
| `tools/package_game_release.sh` | Bundled zip packager (what CI ships) |
| `tools/package_setup_host.sh` | Retired setup-host zip packager (not used by release CI) |
| `tools/ci/*.sh` | normalize version, check generated, generate OpenBIOS, record pins, build emitters |
| `tools/fetch_toolchain.sh` | Optional: download/unpack `cmake-clang-v1` (CI embed or local) |
| `tools/toolchain_pack.py` | CLI: resolve / download / unpack into `toolchain/` + shared cache |
| `tools/stage_setup_sdk.sh` | Pack: emitters, OpenBIOS checks, optional `toolchain/`, MinGW DLLs |
| `tools/bundle_mingw_dlls.sh` | Windows: copy MinGW runtime DLLs next to host + emitters |
| Project `toolchain/` | Stamp file `.psxrecomp-bin` pointing at the shared pack `bin/` |
| Shared toolchain cache | `%LOCALAPPDATA%/retcomm/toolchains/cmake-clang-v1/` (Windows) or `~/.local/share/retcomm/toolchains/cmake-clang-v1/` — same tree Retro uses |
| `docs/ci/` | Composite actions + [`templates/setup-release.yml`](ci/templates/setup-release.yml) |
| `docs/ci/` | Composite actions + [`templates/game-release.yml`](ci/templates/game-release.yml) |
| `docs/GAME_PROJECT_SETUP.md` | Submodules, CI template usage, bundled-release checklist |
| Game sources | `game.toml`, seeds, `CMakeLists.txt` at repo root; `psxrecomp/`, `recomp-ui/` submodules |
+105
View File
@@ -0,0 +1,105 @@
# Bundled releases
Public CI builds and ships the **compiled game**. The title commits its
recompiled game C under `generated/`; CI links it, packages the executable with
its runtime data, and attaches one zip per platform. Players download, point
the game at their own disc, and play. Nothing is generated or compiled on a
player's machine.
This replaces the setup-host model (2026-09-30), under which CI wiped
`generated/`, shipped emitters plus sources plus a wizard, and every player
regenerated and rebuilt the title locally. That shipped uncompiled work, made
first-run depend on a toolchain download, and put every generate/build failure
on the player's machine.
## What the title commits
| Path | Status |
|------|--------|
| `generated/<boot>_dispatch.c`, `generated/<boot>_full_*.c`, `generated/<boot>_decls.h`, `.ranges` | **Committed.** `psxrecomp_cli.py generate` writes them; commit after every regenerate. Shards are ~1.4 MB each, far under GitHub's blob limit. |
| `generated/overlays_static*.c`, `AOT_STATIC_AUDIT.json` | **Committed** when the AOT profile declares `static_output` (`docs/AOT_SHARDING.md`). This is the overlay coverage a release links. |
| `psxrecomp/generated/` (BIOS backends) | Not committable: it is inside the framework submodule. CI regenerates OpenBIOS from the bundled image. |
| `generated/SCPH*`, `psxrecomp/generated/SCPH*` | **Never.** Retail-BIOS-derived C. `tools/ci/check_generated.sh` fails the release if it is present. |
| Disc images, BIOS dumps | Never. |
`.gitignore` must not carry `/generated/`; the scaffold (`gitignore.in`,
`docs/ci/templates/game.gitignore`) no longer writes it, and Project Studio's
`merge_gitignore` removes it from an existing file.
## What CI does
Template: [`templates/game-release.yml`](templates/game-release.yml).
1. `check_boot_exe.sh` — the three copies of the boot-EXE name agree, so the
build looks for the C the checkout actually has.
2. `check_generated.sh` — `generated/<boot>_dispatch.c` and its shards exist
**and are tracked**; no retail BIOS C; no tracked BIOS dump.
3. Build `psxrecomp-game` / `psxrecomp-bios` (or restore them from the Actions
cache when the `psxrecomp` submodule SHA matches a prior run).
4. `generate_openbios.sh` — `psxrecomp-bios --config bios/OpenBIOS.toml` into
`psxrecomp/generated/`, plus the emitter fingerprint runtime.cmake checks.
5. Configure with `-DPSXRECOMP_REQUIRE_GAME_C=ON` (a checkout without game C
fails configure instead of quietly producing a setup host), assert the
configure log says `linking generated game C (full runtime)` and
`BIOS backends linked: OpenBIOS`, build `psx-runtime`.
6. `scripts/package_release.sh` → `tools/package_game_release.sh` — stage the
executable, `assets/`, `bios/openbios.bin` + notice, `game.toml`,
`game_options.toml`, the mod catalog (verified against the manifest the
build published, developer-channel packages pruned), `overlay_toolchain/`,
third-party notices; refuse if anything kit-shaped is in the stage; sign on
Windows; zip.
7. Verify the zip: executable + OpenBIOS + catalog present; no `psxrecomp/`,
`recomp-ui/`, CLI, emitters at the root, sources, generated C, dumps,
per-machine mod state.
`PSX_SETUP_WIZARD` stays on. In a full build it is the first-run disc picker;
it never offers Generate & rebuild because the binary already carries the game.
## Overlays
The overlay **shard cache** is compiled from the disc, so CI cannot build one.
A release relies on two things instead:
- the **static AOT shard** committed under `generated/` and linked into the
executable (titles with an `aot/overlays.json` `static_output`);
- the bundled **`overlay_toolchain/`** (pinned relocatable Python +
`psxrecomp-game` + headers, tcc on Windows), which lets the runtime compile
any other overlay it streams from the player's disc.
A developer packaging locally with a cache can ship it:
`PSX_OVERLAY_CACHE_ROOT=<cache root> scripts/package_release.sh …`. The packager
routes it through `tools/release_stage.py`, the only place that knows the cache
tag and layout.
## Faster host/UI bumps
`workflow_dispatch` input **`reuse_cached_emitters`** (default `true`):
| Situation | Emitters |
|-----------|----------|
| Cache miss (new `psxrecomp` SHA) | Always rebuild |
| Cache hit + reuse on | Skip rebuild; use cached binaries |
| Cache hit + reuse off | Rebuild anyway |
Linux/macOS game compiles use **ccache**, so a host/UI-only bump is close to a
link-only rebuild of the tens of MB of generated C.
## What players / Retro do
| Step | Meaning |
|------|---------|
| Install | Extract the zip. Run the game. Pick the disc on first run. |
| Update | Extract the new zip over the old one (raw zip extract: this is a prebuilt Play binary). Saves and settings live beside the exe and are preserved. |
| Retail BIOS | Not needed and not shipped; the release runs on the bundled OpenBIOS. |
## Title checklist (short)
- `generated/` committed and regenerated whenever seeds, `game.toml` codegen
settings, or the framework pin change.
- `.gitignore` does not ignore `generated/`.
- `scripts/package_release.sh` (scaffold template `package_release.sh.in`).
- `.github/workflows/release.yml` from `templates/game-release.yml`
(`tools/generate_ci` writes and checks it).
- `[runtime] overlay_cache = true` in `game.toml`.
- Release notes say the zip is the compiled game: bring your own disc, no
BIOS, no build.
-64
View File
@@ -1,64 +0,0 @@
# Host-only / setup-host releases
Public CI builds a **setup-host** zip: wizard + game sources + emitters +
framework pins. It never ships retail discs, BIOS dumps, or `generated/` game C.
## What CI does
1. `clear_generated.sh` — empty `generated/` so CMake takes the setup-host path.
2. Build `psxrecomp-game` / `psxrecomp-bios` (or restore them from Actions cache
when the `psxrecomp` submodule SHA matches a prior run).
3. Configure with `-DPSXRECOMP_FORCE_SETUP_HOST=ON` + wizard flags; build the
host binary only.
4. `package_setup_host.sh` — lean zip (no embedded toolchain by default).
Requires `psx_game_version.txt` beside the host exe (stamped at compile
time). Refuses to ship if `RELEASE_VERSION` / `VERSION` disagree with that
stamp — a mismatched pin makes netplay lobby browsers filter each other out.
Template: [`templates/setup-release.yml`](templates/setup-release.yml).
### Faster host/UI bumps
`workflow_dispatch` input **`reuse_cached_emitters`** (default `true`):
| Situation | Emitters |
|-----------|----------|
| Cache miss (new `psxrecomp` SHA) | Always rebuild |
| Cache hit + reuse on | Skip rebuild; package cached binaries |
| Cache hit + reuse off | Rebuild anyway |
Linux/macOS setup-host compiles also use **ccache**. Windows emitters still use
the portable llvm-mingw fetch; host compile stays on MSYS2 g++.
Bump the `psxrecomp` gitlink when emitter/CLI behavior must change; leave it
pinned for pure host/UI/`recomp-ui` releases so CI stays cmake-time.
## What players / Retro do
| Step | When |
|------|------|
| **Generate** (wizard or Retro Build & Install) | First install, or after ROM/BIOS/emitter fingerprint changes |
| **Update** (Retro) | New setup-host zip → refresh source → cmake rebuild; **codegen-cache** skips regenerate when fingerprints match |
| **Generate & Rebuild** | Force disc→C again (`force_generate`) |
Players do **not** need Generate & Rebuild for ordinary host/UI updates after the
first successful generate. Retro keeps `apps/<title>/codegen-cache/` keyed by
ROM/BIOS + emitter fingerprints (see Retro `docs/BUILD_PACKS.md`).
Generate also writes the static overlay shard when the title's
`aot/overlays.json` declares `static_output` (`generated/overlays_static*.c`
plus `AOT_STATIC_AUDIT.json`; see `docs/AOT_SHARDING.md`). A cache that
restores `generated/` without regenerating has to carry those files and
include the profile in its key. Otherwise the restored build links a stale
shard or none. Generate itself reuses an unchanged shard, so re-running it
costs little.
Raw zip extract is for true prebuilt Play binaries only. Setup-host catalog
entries use local generate+cmake (`install_title_auto` / `update_title_auto`).
## Title checklist (short)
- Ship wizard (`PSX_SETUP_WIZARD` / `ENABLE_SETUP_WIZARD`).
- Thin `codegen_setup` forward for setup-host link.
- Catalog: `build.enabled` + release asset globs pointing at the setup zip.
- Document that first-run Generate is expected; later Updates are host rebuilds.
+42 -55
View File
@@ -1,50 +1,47 @@
# CI helpers for setup-host releases
# CI helpers for bundled releases
Shared scripts and GitHub Actions so every PSXRecomp title uses one
standardized release flow.
standardized release flow: CI builds the title's **committed `generated/`
game C** and ships the **compiled game**.
**Start here:** [`../GAME_PROJECT_SETUP.md`](../GAME_PROJECT_SETUP.md)
**Start here:** [`../GAME_PROJECT_SETUP.md`](../GAME_PROJECT_SETUP.md), then
[`BUNDLED_RELEASES.md`](BUNDLED_RELEASES.md) for what ships and why.
## Workflow template
**New Project Layout** (`tools/new_project_layout/setup_project.{sh,ps1}`) copies
this template into the title repo and replaces `YOUR_*` / `yourgame-release`
using `--zip-prefix` (or a derived acronym) plus the window title. It also
writes `scripts/package_setup_release.sh`, `README.md`, and an optional
writes `scripts/package_release.sh`, `README.md`, and an optional
`framework_pins.txt` snapshot, and can opt in wizard/netplay via
`--enable-wizard` / `--enable-netplay`. Release CI pins via submodule gitlinks
and only logs SHAs with `record_pins.sh` (no `verify_pins` gate).
**Migrating an older title** onto this CI shape: Project Studio
(`tools/new_project_layout/migrate_project.py apply` / `gui`) emits the same
packager + filled `release.yml`. Setup-host only — no prebuilt game-C zips.
**Existing titles:** `tools/generate_ci` writes (and `--check`s) the filled
workflow; Project Studio (`tools/new_project_layout/migrate_project.py apply` /
`gui`) emits the same packager wrapper + `release.yml` and un-ignores
`generated/`.
Manual:
```bash
mkdir -p .github/workflows
cp psxrecomp/docs/ci/templates/setup-release.yml .github/workflows/release.yml
cp psxrecomp/docs/ci/templates/game-release.yml .github/workflows/release.yml
# edit YOUR_* placeholders — or use fill_tokens.py --ci-placeholders
# Release name is YAML-quoted so titles with ':' (e.g. Marvel vs. Capcom: …) parse.
```
Template: [`templates/setup-release.yml`](templates/setup-release.yml)
Template: [`templates/game-release.yml`](templates/game-release.yml)
**Versioning:** `workflow_dispatch` with an empty `version` auto-bumps the next
`X.Y.Z` from the latest `v*.*.*` git tag (`bump`: patch / minor / major; default
patch). Pass an explicit version to override. Shared helper:
`tools/ci/normalize_version.sh --next [patch|minor|major]`.
**Host-only model / player updates:** [`HOST_ONLY_RELEASES.md`](HOST_ONLY_RELEASES.md)
(CI never ships game C; Retro Update reuses codegen-cache for host/UI bumps;
`reuse_cached_emitters` + ccache speed up release jobs when the `psxrecomp` pin
is unchanged).
Matrix: `ubuntu-24.04` (linux-x64), `windows-2022` (windows-x64),
`macos-15` (macos-arm64), `macos-15-intel` (macos-x64).
Release hosts always configure with Vulkan headers + `glslc` (MSYS2 /
Release builds always configure with Vulkan headers + `glslc` (MSYS2 /
apt / Homebrew packages), verify `vulkan.h` + `glslc` before configure, and
fail the job if CMake does not select `PSX_HAVE_VULKAN`. Runtime still loads
the ICD dynamically via SDL; CI only needs headers and the shader compiler.
@@ -54,58 +51,50 @@ the ICD dynamically via SDL; CI only needs headers and the shader compiler.
| Script | Role |
|--------|------|
| `ci/normalize_version.sh` | Normalize / write `VERSION` + `TAG`; `--next` auto-bumps from latest tag |
| `ci/clear_generated.sh` | Clear `generated/` for setup-host CI |
| `ci/check_boot_exe.sh` | The three copies of the boot-EXE name agree |
| `ci/check_generated.sh` | Committed game C present + tracked; no retail BIOS C; no BIOS dump |
| `ci/generate_openbios.sh` | Emit the OpenBIOS backend C in CI from the bundled MIT image |
| `ci/record_pins.sh` | Log `psxrecomp` / `recomp-ui` / `recomp-net` SHAs (CI + scaffold) |
| `ci/verify_pins.sh` | Optional local check vs `framework_pins.txt` (not used by release CI) |
| `ci/build_emitters.sh` | Build `psxrecomp-game` + `psxrecomp-bios` |
| `fetch_toolchain.sh` | Download/unpack cmake-clang-v1 (Windows emitter builds; optional embed) |
| `stage_setup_sdk.sh` | Emitters, OpenBIOS, optional `toolchain/`, MinGW DLLs |
| `ci/sign_windows.sh` | Authenticode-sign the staged Windows binaries (no-op without a certificate) |
| `fetch_toolchain.sh` | Download/unpack cmake-clang-v1 (Windows emitter builds on the CI machine only) |
| `package_game_release.sh` | Bundled zip: compiled game + runtime data + OpenBIOS + mods + overlay toolchain |
| `release_stage.py` | Shared staging surface (mod catalog, overlay toolchain, overlay cache tag/shards) |
| `bundle_mingw_dlls.sh` | Copy imported non-system DLLs next to Windows PEs |
| `package_setup_host.sh` | Lean setup-host zip (optional `--embed-toolchain`) |
| `../cmake/toolchain-mingw-w64.cmake` | Linux→Windows MinGW cross toolchain |
| `../host/psxrecomp_codegen_host.*` | Portable Generate & rebuild host (via CMake helper) |
| `templates/game.gitignore` | Suggested gitignore for title repos |
| `templates/game.gitignore` | Suggested gitignore for title repos (`generated/` tracked) |
Also use `psxrecomp_add_game_runtime(...)` in `runtime/runtime.cmake` for
setup-host / full-game wiring. The CLI (`psxrecomp_cli.py`) lives in this
submodule — there is no separate `psxrecomp-sdk/` overlay.
`package_setup_host.sh` / `stage_setup_sdk.sh` are the retired setup-host
packagers; they are not part of the release flow.
## Composite actions
```yaml
- uses: ./psxrecomp/.github/actions/build-emitters
# Prefer package_setup_host.sh (allow-no-toolchain by default).
# fetch-toolchain + --embed-toolchain only for offline-first packs.
- uses: ./psxrecomp/.github/actions/stage-setup-sdk
with:
stage: dist/stage-setup-${{ matrix.artifact }}
recompiler-build: build-recompiler
allow-no-toolchain: 'true'
runtime-bin: /mingw64/bin # Windows / MSYS2
- uses: ./psxrecomp/.github/actions/fetch-toolchain # Windows emitter build only
```
## Title responsibilities
Keep only this in the game repo:
- Setup-host CMake flags and exe basename
- `generated/` committed, regenerated (and re-committed) whenever seeds,
codegen settings, or the framework pin change
- CMake via `psxrecomp_add_game_runtime` with `GEN_MARKER` / `GEN_FULL_GLOB`
naming the committed C, and the wizard on (first-run disc picker)
- Thin `codegen_setup.c` / `.h` with `psx_game_codegen_forward_if_built`
(scaffold: `tools/new_project_layout/templates/codegen_setup.c.in`)
(scaffold: `tools/new_project_layout/templates/codegen_setup.c.in`) —
`main.cpp` always calls it; in a full build it is a no-op
- For netplay titles: `PSX_NETPLAY ON` **before** `include(runtime.cmake)`
(`ENABLE_NETPLAY_IF_PRESENT` alone is too late), plus `-DPSX_NETPLAY=ON`
in the release workflow configure step
- Thin `scripts/package_setup_release.sh` wrapping `package_setup_host.sh`
- Thin `scripts/package_release.sh` wrapping `package_game_release.sh`
- Zip prefix / display name / disc hint in that wrapper
- Release notes / GitHub Release job naming
Release CI configures with `-DPSXRECOMP_FORCE_SETUP_HOST=ON`,
`-DPSXRECOMP_ALLOW_NO_BIOS=ON`, and `-DPSX_SETUP_WIZARD=ON` after
`clear_generated.sh`. Titles must also set `PSX_SETUP_WIZARD` /
`ENABLE_SETUP_WIZARD` in `CMakeLists.txt` — without the wizard flag the
setup-host zip never opens first-run / Generate & rebuild. The zip ships
emitters + OpenBIOS.toml; end users Generate (OpenBIOS always; SCPH1001 if
they have a dump) via the wizard, then rebuild.
Release CI configures with `-DPSXRECOMP_REQUIRE_GAME_C=ON` after
`check_generated.sh` and `generate_openbios.sh`, and asserts from the configure
log that the generated game C and the OpenBIOS backend were linked.
## Windows code signing
@@ -114,9 +103,9 @@ executable only if it carries a valid Authenticode signature or its exact
hash already has reputation. Every CI build is a new hash, so an unsigned
release is blocked outright, with no "run anyway".
`package_setup_host.sh` therefore runs `tools/ci/sign_windows.sh` over the
staged tree before zipping. It signs the host exe, every DLL, and the
emitters (an embedded `toolchain/` is left alone) with SHA-256 and an RFC 3161
`package_game_release.sh` therefore runs `tools/ci/sign_windows.sh` over the
staged tree before zipping. It signs the game exe and every DLL (the
`overlay_toolchain/` bundle is left alone) with SHA-256 and an RFC 3161
timestamp, then verifies each file.
| Secret / variable | Meaning |
@@ -128,7 +117,7 @@ timestamp, then verifies each file.
**No certificate = no signing, and the build still succeeds** (a notice is
printed). A certificate that is present but fails to sign is fatal. The
workflow template passes the two secrets to the *Package setup zip* step;
workflow template passes the two secrets to the *Package game zip* step;
forks without them package unsigned exactly as before.
Certificate choice: an OV certificate satisfies Smart App Control at once,
@@ -137,12 +126,10 @@ or Azure Trusted Signing is trusted by both immediately. Trusted Signing
does not hand out a .pfx; wiring it means a different signing step, not
this script.
What signing cannot cover: a setup-host title is **rebuilt on the player's
machine** after Generate. That locally built game exe is a new, unsigned
binary and Smart App Control judges it on its own. Signing covers the
shipped host/wizard, emitters and DLLs; a machine with Smart App Control
*On* will still refuse the locally built game until the policy is turned
off (Windows Security → App & browser control), which is permanent.
Because the shipped executable **is** the game, the signature now covers
what the player runs. The setup-host model could not offer that: the binary
players ran was built on their own machine after Generate and carried no
signature.
## Release checklist
-23
View File
@@ -1,23 +0,0 @@
# Deprecated stub — use the full template:
#
# docs/ci/templates/setup-release.yml
#
# Install:
# cp psxrecomp/docs/ci/templates/setup-release.yml .github/workflows/release.yml
#
# Guide: docs/GAME_PROJECT_SETUP.md
# Actions: docs/ci/README.md
name: Release builds (see docs/ci/templates/setup-release.yml)
on:
workflow_dispatch:
jobs:
redirect:
runs-on: ubuntu-24.04
steps:
- run: |
echo "Copy psxrecomp/docs/ci/templates/setup-release.yml to .github/workflows/release.yml"
echo "See psxrecomp/docs/GAME_PROJECT_SETUP.md"
exit 1
@@ -1,30 +1,41 @@
# Setup-host multi-platform release template for a PSXRecomp *game* repository.
# Bundled multi-platform release template for a PSXRecomp *game* repository.
#
# Install:
# mkdir -p .github/workflows
# cp psxrecomp/docs/ci/templates/setup-release.yml .github/workflows/release.yml
# Then replace every YOUR_* token (or let setup_project / fill_tokens.py do it).
# See docs/GAME_PROJECT_SETUP.md.
# cp psxrecomp/docs/ci/templates/game-release.yml .github/workflows/release.yml
# Then replace every YOUR_* token (or let setup_project / fill_tokens.py /
# tools/generate_ci do it). See docs/GAME_PROJECT_SETUP.md.
#
# WHAT THIS SHIPS. A compiled game. The title COMMITS its generated/ game C
# (psxrecomp_cli.py generate, run once by the developer from a legal disc);
# CI builds the emitters, emits the OpenBIOS backend from the bundled MIT
# image, links the committed game C into the runtime, and packages the
# executable with its runtime data. Players download, point it at their disc,
# and play. Nothing is generated or compiled on a player's machine, and no
# emitters, sources, or CLI ship at the zip root.
#
# WHAT NEVER SHIPS. Disc images. Retail BIOS dumps and any C derived from one
# (tools/ci/check_generated.sh fails the job if generated/SCPH* exists). The
# overlay shard cache: it is compiled FROM THE DISC, which CI does not have,
# so the committed static AOT shard (generated/overlays_static*.c) is linked
# and the bundled overlay_toolchain/ compiles anything else from the player's
# disc at runtime. Do not set PSX_PGO in CI.
#
# Shared (do not fork these into the title):
# ./psxrecomp/.github/actions/{build-emitters,fetch-toolchain (optional embed),stage-setup-sdk}
# psxrecomp/tools/ci/*.sh
# psxrecomp/tools/{fetch_toolchain,stage_setup_sdk,bundle_mingw_dlls,package_setup_host}.sh
# psxrecomp/psxrecomp_cli.py (CLI lives in the submodule — no psxrecomp-sdk/)
# ./psxrecomp/.github/actions/{build-emitters,fetch-toolchain}
# psxrecomp/tools/ci/{check_boot_exe,check_generated,generate_openbios,record_pins}.sh
# psxrecomp/tools/{package_game_release,bundle_mingw_dlls}.sh
# psxrecomp/tools/release_stage.py (mod catalog + overlay toolchain staging)
#
# Title-owned:
# CMake setup-host flags, thin scripts/package_setup_release.sh wrapper
# CMakeLists.txt flags, the thin scripts/package_release.sh wrapper
#
# No private CI assets. No BIOS dumps. No generated/ required.
# Do not set PSX_PGO in CI (PGO stays user-local after Generate).
# Release hosts always build with PSX_HAVE_VULKAN (headers + glslc); runtime
# still loads the ICD dynamically via SDL.
# Release hosts always build with PSX_HAVE_VULKAN (headers + glslc); the
# runtime still loads the ICD dynamically via SDL.
#
# workflow_dispatch: leave version empty to auto-bump patch (or choose bump).
# Host-focused releases: CI never ships game C. Players Generate once; Retro
# Update reuses codegen-cache when ROM/BIOS/emitters match (cmake-only).
# See docs/ci/HOST_ONLY_RELEASES.md. reuse_cached_emitters skips rebuilding
# psxrecomp-game/bios when the psxrecomp SHA cache hits (host/UI bumps).
# reuse_cached_emitters skips rebuilding psxrecomp-game/bios when the
# psxrecomp submodule SHA cache hits. See docs/ci/BUNDLED_RELEASES.md.
name: Release builds
@@ -195,17 +206,19 @@ jobs:
shell: bash
run: bash psxrecomp/tools/ci/record_pins.sh
# A skew between game.toml, CMakeLists GEN_MARKER and codegen_setup.c
# makes every player's self-build loop generate+build forever. The CLI
# also refuses the skew at generate time, but failing the release is
# cheaper than failing the player.
# game.toml, CMakeLists GEN_MARKER and codegen_setup.c each carry the
# boot-EXE name; the committed generated/ files are named after it, so a
# skew means the build looks for C the checkout does not have.
- name: Lint boot-EXE name consistency
shell: bash
run: bash psxrecomp/tools/ci/check_boot_exe.sh .
- name: Setup host mode (no game C / BIOS backends in CI)
# The release is built from COMMITTED game C. Fail here, at the top,
# when it is absent or untracked, and when any retail-BIOS-derived C
# or BIOS dump is in the tree.
- name: Verify committed game C
shell: bash
run: bash psxrecomp/tools/ci/clear_generated.sh
run: bash psxrecomp/tools/ci/check_generated.sh --root .
- name: Setup MSYS2 (Windows)
if: runner.os == 'Windows'
@@ -223,6 +236,7 @@ jobs:
mingw-w64-x86_64-shaderc
curl
zip
unzip
- name: Install deps (Linux)
if: runner.os == 'Linux'
@@ -234,7 +248,7 @@ jobs:
libsdl2-dev libgl1-mesa-dev libxtst-dev \
libx11-dev libxext-dev libxrandr-dev libxcursor-dev \
libxfixes-dev libxi-dev libxss-dev libxkbcommon-dev \
libasound2-dev libpulse-dev libudev-dev zip \
libasound2-dev libpulse-dev libudev-dev zip unzip \
libvulkan-dev glslc
- name: Install deps (macOS)
@@ -247,7 +261,7 @@ jobs:
# Headers + glslc are enough to compile PSX_HAVE_VULKAN (runtime still
# loads the ICD dynamically via SDL). Fail early if the stub path would
# be selected so release hosts always ship the real Vulkan backend.
# be selected so releases always ship the real Vulkan backend.
- name: Verify Vulkan build tools
run: |
set -euo pipefail
@@ -276,11 +290,11 @@ jobs:
fi
echo "Found ${HDR}"
# Ubuntu 24.04 has no libsdl3-dev; Windows MSYS ships SDL2. Setup-host
# therefore FetchContents the pinned SDL3 tarball. CMake's HTTP/2 download
# from github.com release assets intermittently dies with REFUSED_STREAM
# (curl status 56). Prefetch with curl --http1.1 and point FetchContent
# at the extracted tree (macOS uses brew sdl3 and skips this).
# Ubuntu 24.04 has no libsdl3-dev; Windows MSYS ships SDL2. The runtime
# therefore FetchContents the pinned SDL3 tarball. CMake's HTTP/2
# download from github.com release assets intermittently dies with
# REFUSED_STREAM (curl status 56). Prefetch with curl --http1.1 and point
# FetchContent at the extracted tree (macOS uses brew sdl3 and skips this).
- name: Cache SDL3 source
if: runner.os != 'macOS'
uses: actions/cache@v4
@@ -340,16 +354,19 @@ jobs:
rm -rf "${OUT}.extracting"
sdl3_env
# Windows: build emitters with llvm-mingw from cmake-clang-v1 (static CLI).
# Zip still omits toolchain/ — fetch is for the build machine only.
# Windows: build emitters with llvm-mingw from cmake-clang-v1 (static
# CLI). The pack is for the build machine only; it never ships.
- name: Fetch portable toolchain (Windows emitters)
if: runner.os == 'Windows'
uses: ./psxrecomp/.github/actions/fetch-toolchain
with:
artifact: ${{ matrix.artifact }}
# Host/UI-only bumps keep the same psxrecomp gitlink → cache hit skips
# rebuilding emitters (see reuse_cached_emitters / HOST_ONLY_RELEASES.md).
# Emitters are needed for two things in a bundled release: psxrecomp-bios
# emits the OpenBIOS backend C, and psxrecomp-game ships inside
# overlay_toolchain/ so the runtime can compile overlays from the
# player's disc. Host/UI-only bumps keep the same psxrecomp gitlink →
# cache hit skips rebuilding them (see reuse_cached_emitters).
- name: Emitter cache key
id: emitter_cache
shell: bash
@@ -377,7 +394,7 @@ jobs:
shell: bash
run: |
set -euo pipefail
# Restored cache or fresh build must leave packager inputs in place.
# Restored cache or fresh build must leave both emitters in place.
test -d build-recompiler
find build-recompiler -type f \( -name 'psxrecomp-game*' -o -name 'psxrecomp-bios*' \) | head
count="$(find build-recompiler -type f \( -name 'psxrecomp-game*' -o -name 'psxrecomp-bios*' \) | wc -l | tr -d ' ')"
@@ -386,14 +403,23 @@ jobs:
exit 1
fi
- name: Setup ccache (Linux/macOS host build)
# BIOS C lives in the framework submodule's generated/, which a title
# cannot commit. OpenBIOS is MIT and its image is bundled, so CI emits
# that one backend here. No retail backend is ever produced in CI.
- name: Generate OpenBIOS backend C
shell: bash
run: bash psxrecomp/tools/ci/generate_openbios.sh --framework psxrecomp --recompiler-build build-recompiler
# The generated game C is tens of MB per title; ccache turns a host/UI
# bump into a link-only rebuild on every platform.
- name: Setup ccache (Linux/macOS)
if: runner.os != 'Windows'
uses: hendrikmuhs/ccache-action@v1.2
with:
key: setup-host-${{ matrix.artifact }}
max-size: 800M
key: game-${{ matrix.artifact }}
max-size: 2G
- name: Configure & build setup host
- name: Configure & build the game
env:
CMAKE_OSX_DEPLOYMENT_TARGET: ${{ matrix.deployment_target || '' }}
run: |
@@ -445,39 +471,21 @@ jobs:
fi
JOBS="$(nproc 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 4)"
# Emitters may have set PSXRECOMP_TOOLCHAIN_DIR (llvm-mingw) via
# GITHUB_ENV. Clear it before the MSYS2 g++ setup-host configure so
# pack lib/ cannot win -L for -static-libstdc++ (codecvt link errors).
# GITHUB_ENV. Clear it before the MSYS2 g++ configure so the pack's
# lib/ cannot win -L for -static-libstdc++ (codecvt link errors).
if [[ "$(uname -s)" == MINGW* || "$(uname -s)" == MSYS* ]]; then
unset PSXRECOMP_TOOLCHAIN_DIR TOOLCHAIN_DIR BPE_TOOLCHAIN_DIR RETCOMM_TOOLCHAIN_DIR || true
unset CMAKE_PREFIX_PATH SDL3_DIR ZLIB_ROOT || true
fi
# Setup host: no game C, no BIOS backends linked. clear_generated.sh
# already wiped generated/; end users Generate OpenBIOS (+ optional
# SCPH1001) locally via the wizard, then rebuild.
# Prefer -DPSXRECOMP_FORCE_SETUP_HOST=ON (psxrecomp_add_game_runtime).
# Legacy -DYOURGAME_FORCE_SETUP_HOST=ON still works if the title wires it.
# PSX_SETUP_WIZARD is required with FORCE_SETUP_HOST (CMakeLists
# ENABLE_SETUP_WIZARD + -D here). Without it the zip never opens
# first-run / Generate & rebuild.
# Title root codegen_setup.c must define
# psx_game_codegen_forward_if_built (see codegen_setup.c.in) or link
# fails — main.cpp always calls it, including setup-host CI builds.
# Full game build: the committed generated/ C is REQUIRED, so a
# checkout without it fails configure instead of silently producing
# a setup host. OpenBIOS was emitted above; runtime.cmake links every
# backend whose C is present, and only OpenBIOS can be.
# PSX_SETUP_WIZARD stays on: it is the first-run disc picker.
# Netplay titles: set PSX_NETPLAY ON in CMakeLists *before*
# include(runtime.cmake). ENABLE_NETPLAY_IF_PRESENT alone is too late
# (option() already cached OFF) and psx_netplay_sched.c still needs
# recomp_net/session.h. Also add -DPSX_NETPLAY=ON to this cmake
# invocation for CI belt-and-suspenders (do not leave it commented
# mid-continuation — that breaks the shell line).
# With PSX_NETPLAY, runtime.cmake defaults RNET_ENABLE_ICE=ON.
# recomp-net FetchContents libjuice via a pinned URL tarball (not
# git clone) so AppImage / mismatched libcurl hosts still configure.
# Offline: vendor under psxrecomp/lib/recomp-net/third_party/libjuice
# or pass -DRNET_LIBJUICE_ROOT=… / -DRNET_ENABLE_ICE=OFF.
# include(runtime.cmake) (ENABLE_NETPLAY_IF_PRESENT alone is too
# late) and add -DPSX_NETPLAY=ON below as belt-and-suspenders.
# Add title-specific -D flags here if needed (e.g. -DMOTK_NATIVE=OFF).
# Netplay titles: also pass -DPSX_NETPLAY=ON here (belt-and-suspenders
# when CMakeLists already FORCE-caches it before include(runtime.cmake)).
# Prefer pre-fetched SDL3 (see Prefetch SDL3 source) so configure does
# not call CMake file(DOWNLOAD) against GitHub release assets.
if [[ -n "${PSX_SDL3_SOURCE_DIR:-}" && -f "${PSX_SDL3_SOURCE_DIR}/CMakeLists.txt" ]]; then
EXTRA+=("-DFETCHCONTENT_SOURCE_DIR_SDL3=${PSX_SDL3_SOURCE_DIR}")
elif [[ -f "${GITHUB_WORKSPACE}/.cache/sdl3-src/CMakeLists.txt" ]]; then
@@ -485,34 +493,51 @@ jobs:
fi
cmake -S . -B build-ci -G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DPSXRECOMP_FORCE_SETUP_HOST=ON \
-DPSXRECOMP_ALLOW_NO_BIOS=ON \
-DPSXRECOMP_REQUIRE_GAME_C=ON \
-DPSXRECOMP_FORCE_SETUP_HOST=OFF \
-DPSXRECOMP_ALLOW_NO_BIOS=OFF \
-DPSX_SETUP_WIZARD=ON \
-DPSX_GAME_VERSION="${RELEASE_VERSION}" \
"${EXTRA[@]}" 2>&1 | tee build-ci-configure.log
# Assert the configure took the FULL path, not the setup-host one.
if ! grep -q 'linking generated game C (full runtime)' build-ci-configure.log; then
echo "::error::CMake did not link the generated game C (see configure log)"
exit 1
fi
if ! grep -qE 'BIOS backends linked: .*OpenBIOS' build-ci-configure.log; then
echo "::error::CMake did not link the OpenBIOS backend (generate_openbios.sh output missing?)"
exit 1
fi
if grep -qE 'BIOS backends linked: .*SCPH' build-ci-configure.log; then
echo "::error::a retail BIOS backend was linked — that C must never be in a CI checkout"
exit 1
fi
if ! grep -q 'Vulkan backend: headers' build-ci-configure.log; then
echo "::error::CMake did not enable PSX_HAVE_VULKAN (see configure log)"
exit 1
fi
# Ninja does not emit flags.make (Makefiles generator only). Probe
# whichever compile-flag artifact exists for PSX_HAVE_VULKAN.
FOUND_VK_DEFINE=0
for f in \
build-ci/build.ninja \
build-ci/compile_commands.json \
build-ci/CMakeFiles/psx-runtime.dir/flags.make; do
if [[ -f "${f}" ]] && grep -q 'PSX_HAVE_VULKAN' "${f}"; then
FOUND_VK_DEFINE=1
echo "Found PSX_HAVE_VULKAN in ${f}"
break
# whichever compile-flag artifact exists for the defines the release
# depends on.
for define in PSX_HAVE_VULKAN PSX_HAS_GAME_DISPATCH; do
FOUND=0
for f in \
build-ci/build.ninja \
build-ci/compile_commands.json \
build-ci/CMakeFiles/psx-runtime.dir/flags.make; do
if [[ -f "${f}" ]] && grep -q "${define}" "${f}"; then
FOUND=1
echo "Found ${define} in ${f}"
break
fi
done
if [[ "${FOUND}" -ne 1 ]]; then
echo "::error::${define} missing from psx-runtime compile flags"
exit 1
fi
done
if [[ "${FOUND_VK_DEFINE}" -ne 1 ]]; then
echo "::error::PSX_HAVE_VULKAN missing from psx-runtime compile flags"
exit 1
fi
cmake --build build-ci --target psx-runtime -j"${JOBS}"
# Lobby list pin must match RELEASE_VERSION (package_setup_host enforces).
# Lobby list pin must match RELEASE_VERSION (the packager enforces).
STAMP=""
for cand in \
build-ci/psx_game_version.txt \
@@ -522,7 +547,6 @@ jobs:
if [[ -f "${cand}" ]]; then STAMP="${cand}"; break; fi
done
if [[ -z "${STAMP}" ]]; then
# Fallback: stamp next to the host exe
STAMP="$(find build-ci -name psx_game_version.txt -type f 2>/dev/null | head -n1 || true)"
fi
if [[ -z "${STAMP}" || ! -f "${STAMP}" ]]; then
@@ -537,70 +561,95 @@ jobs:
fi
echo "lobby pin ok: ${BUILT} (${STAMP})"
- name: Package setup zip
- name: Package game zip
# Windows code signing (Smart App Control / SmartScreen): the packager
# signs the host exe, DLLs and emitters when these secrets exist and
# skips signing, with a notice, when they do not. Put the .pfx in the
# signs the game exe and its DLLs when these secrets exist and skips
# signing, with a notice, when they do not. Put the .pfx in the
# repo's Actions secrets base64-encoded (`base64 -w0 cert.pfx`).
# See psxrecomp/docs/ci/README.md, "Windows code signing".
# See psxrecomp/docs/ci/README.md, "Windows code signing". Because the
# shipped exe IS the game, the signature now covers what the player
# runs — there is no unsigned locally built binary any more.
env:
WINDOWS_SIGN_PFX_BASE64: ${{ secrets.WINDOWS_SIGN_PFX_BASE64 }}
WINDOWS_SIGN_PFX_PASSWORD: ${{ secrets.WINDOWS_SIGN_PFX_PASSWORD }}
run: |
set -euo pipefail
chmod +x scripts/package_setup_release.sh
chmod +x scripts/package_release.sh
if [[ "${{ runner.os }}" == "Windows" ]]; then
export PSXRECOMP_RUNTIME_BIN_DIR=/mingw64/bin
export BPE_RUNTIME_BIN_DIR=/mingw64/bin
fi
export RELEASE_VERSION
# Lean zip (no embedded toolchain/). Retro / wizard download
# cmake-clang-v1; pass --embed-toolchain only for offline-first packs.
scripts/package_setup_release.sh build-ci "${{ matrix.artifact }}" build-recompiler
scripts/package_release.sh build-ci "${{ matrix.artifact }}" build-recompiler
ls -la dist/
- name: Verify zip ships the mod catalog
- name: Verify zip contents
shell: bash
run: |
set -euo pipefail
# Gate the release on the catalog actually being inside the zip.
# The packager stages mods/ by default, but this shipped broken for
# a long time precisely because nothing checked: the Mods page was
# empty in every release and the build logs looked fine. Assert the
# artifact, not the intent.
# Assert the artifact, not the intent: every rule below was once
# "the packager does that by default" and shipped broken anyway.
ZIP=$(ls dist/YOUR_ZIP_PREFIX-*-"${{ matrix.artifact }}".zip | head -1)
echo "checking ${ZIP}"
MANIFESTS=$(unzip -l "${ZIP}" | grep -c 'mods/bundled/.*/manifest\.toml' || true)
LIST="$(unzip -Z1 "${ZIP}")"
# 1. It is a game: the executable, the bundled OpenBIOS, and the
# runtime data sit at the root.
for must in 'bios/openbios.bin' 'bios/OpenBIOS.LICENSE' 'game.toml' 'psx_game_version.txt' 'assets/fonts/' 'mods/bundled/'; do
if ! grep -q "^${must}" <<<"${LIST}"; then
echo "::error::release zip lacks ${must}" >&2
printf '%s\n' "${LIST}" | head -40 >&2
exit 1
fi
done
MANIFESTS=$(grep -c 'mods/bundled/.*/manifest\.toml' <<<"${LIST}" || true)
echo "mod manifests in zip: ${MANIFESTS}"
if [[ "${MANIFESTS}" -lt 1 ]]; then
echo "::error::release zip ships no mod catalog (mods/bundled/*/manifest.toml)" >&2
unzip -l "${ZIP}" | head -40 >&2
exit 1
fi
# Per-machine mod state must never ship: preloaded catalogs are
# default-disabled, so a leaked state.toml makes one machine's
# selections everybody's defaults.
if unzip -l "${ZIP}" | grep -q 'state\.toml'; then
# 2. It is NOT a build kit: no framework tree, no CLI, no emitters at
# the root, no sources, no generated C.
for never in '^psxrecomp/' '^recomp-ui/' '^psxrecomp_cli.py' '^psxrecomp-game' '^psxrecomp-bios' \
'^generated/' '^seeds/' '^CMakeLists.txt' '^codegen_setup\.' '^toolchain/' '^src/'; do
if grep -qE "${never}" <<<"${LIST}"; then
echo "::error::release zip contains build-kit content matching ${never}" >&2
grep -E "${never}" <<<"${LIST}" | head -20 >&2
exit 1
fi
done
if grep -E '\.c$' <<<"${LIST}" | grep -qv '^overlay_toolchain/'; then
echo "::error::release zip contains C sources outside overlay_toolchain/" >&2
grep -E '\.c$' <<<"${LIST}" | grep -v '^overlay_toolchain/' | head -20 >&2
exit 1
fi
# 3. No BIOS dump, disc, or memory card; the only image is OpenBIOS.
if grep -iE '(^|/)SCPH|\.(cue|iso|img|chd|mcd|mcr)$' <<<"${LIST}" | grep -qv '\.toml$'; then
echo "::error::release zip contains a BIOS dump, disc, or memory card" >&2
grep -iE '(^|/)SCPH|\.(cue|iso|img|chd|mcd|mcr)$' <<<"${LIST}" | grep -v '\.toml$' >&2
exit 1
fi
if grep -iE '\.bin$' <<<"${LIST}" | grep -qv '^bios/openbios.bin$'; then
echo "::error::release zip contains a .bin other than bios/openbios.bin" >&2
grep -iE '\.bin$' <<<"${LIST}" | grep -v '^bios/openbios.bin$' >&2
exit 1
fi
# 4. Per-machine mod state never ships: state.toml is one machine's
# selections, mods/installed/ is one developer's installs, and
# developer-channel packages are unfinished work.
if grep -q 'state\.toml' <<<"${LIST}"; then
echo "::error::release zip contains mods/state.toml (per-machine state)" >&2
exit 1
fi
# Same rule, other half: mods/installed/ is the launcher-owned tree of
# .psxmod archives a developer installed as a player would. Only
# mods/bundled/ is build output, and only build output ships.
if unzip -l "${ZIP}" | grep -q 'mods/installed/'; then
if grep -q 'mods/installed/' <<<"${LIST}"; then
echo "::error::release zip contains mods/installed (per-machine installs)" >&2
exit 1
fi
# Developer-channel packages are unfinished work that ships with
# local builds only. The packager drops them under CI automatically
# (it keys off $CI); assert the outcome so a future change to that
# default cannot quietly publish in-progress mods.
if unzip -p "${ZIP}" 'mods/**/manifest.toml' 2>/dev/null \
| grep -Eq '^[[:space:]]*channel[[:space:]]*=[[:space:]]*"developer"'; then
echo "::error::release zip contains developer-channel mod packages" >&2
exit 1
fi
echo "mod catalog verified in ${ZIP} (no developer-channel packages)"
echo "zip verified: compiled game, bundled OpenBIOS, mod catalog, no kit / dumps / dev mods"
- name: Upload artifact
uses: actions/upload-artifact@v4
+4 -3
View File
@@ -8,9 +8,10 @@
/out/
/.happypath/
# Setup-host: leave generated/ gitignored (users generate locally).
# Titles that still CI-build from committed game C may track generated/ instead.
/generated/
# generated/ is COMMITTED: releases ship the compiled game, and CI builds it
# from the game C in this repo (psxrecomp_cli.py generate, then git add
# generated). Never ignore it. Only the framework submodule's own generated/
# (BIOS backends) stays untracked, and that is the submodule's rule.
# psxrecomp_cli.py work areas (overlay toolchain download, static overlay
# shard evidence -- the latter holds bytes read from the disc).
/.cache/
@@ -117,8 +117,14 @@ SKIP_DIRS = {'.git', 'build', 'generated', 'node_modules', '__pycache__',
# but that is a separate change with a separate proof (macOS in particular
# cannot be built or verified here), so this test does not pretend to cover it.
PACKAGER_RE = re.compile(r'^package_.*\.(sh|ps1)$')
# `overlay_cache` followed by `=` (in any grep/sed spelling of whitespace) is
# the game.toml KEY `[runtime] overlay_cache = true`, which packagers gate on
# (Wave-5 F4) without touching the cache itself, and `overlay_cache_...` is an
# identifier continuing (SHIP_WITHOUT_OVERLAY_CACHE_BECAUSE, the reason flag).
# Neither is tag consumption.
TAG_CONSUMER_RE = re.compile(
r'cg_?tag|CgTag|cache_tag|overlay_cache|overlay_toolchain', re.I)
r'cg_?tag|CgTag|cache_tag|overlay_cache(?!\w|(\[\[:space:\]\]|\\s|\s)*\*?=)|overlay_toolchain',
re.I)
SHARED_SURFACE_RE = re.compile(r'release_overlay_stage|release_stage\.py')
+131
View File
@@ -0,0 +1,131 @@
#!/usr/bin/env bash
# Gate a bundled release on the committed game C being present and clean.
#
# Releases ship a compiled game: CI builds from the generated/ tree the title
# COMMITS, so a checkout missing it (or carrying the wrong marker name) must
# fail here, at the top of the job, not twenty minutes later at link -- and
# never on a player's machine, which is where the old setup-host model put
# every failure of this kind.
#
# Asserts, from the game repo root:
# 1. generated/<boot>_dispatch.c exists, where <boot> is the boot-EXE name
# game.toml declares (the same name check_boot_exe.sh cross-checks), and
# at least one generated/<boot>_full*.c shard sits beside it.
# 2. Those files are tracked by git. An untracked generated/ builds fine on
# the developer's machine and is absent from every CI checkout, which is
# the exact gap between "works for me" and a red release.
# 3. No retail-BIOS-derived C is present anywhere it could be linked:
# generated/SCPH*_{dispatch,full}.c and psxrecomp/generated/SCPH*. Those
# are derivatives of a copyrighted Sony image and must not reach a public
# artifact. OpenBIOS is the only backend a release links, and CI emits it
# from the bundled MIT image (tools/ci/generate_openbios.sh).
# 4. No BIOS dump is tracked (SCPH*.BIN / *.bin under bios/).
#
# Usage: check_generated.sh [--root DIR] (default: cwd)
# Runs on bash 3.2+ (macOS runners).
set -euo pipefail
ROOT="${PWD}"
while [[ $# -gt 0 ]]; do
case "$1" in
--root) ROOT="${2:?}"; shift 2 ;;
-h|--help)
sed -n '2,25p' "$0" | sed 's/^# \{0,1\}//'
exit 0
;;
*)
echo "error: unknown arg: $1" >&2
exit 2
;;
esac
done
ROOT="$(cd "${ROOT}" && pwd)"
toml_get() { # <section> <key> -- first quoted value of key inside [section]
awk -v sec="$1" -v key="$2" '
/^[[:space:]]*\[/ { in_sec = ($0 ~ ("^[[:space:]]*\\[" sec "\\]")) }
in_sec && $0 ~ ("^[[:space:]]*" key "[[:space:]]*=") {
if (match($0, /"[^"]*"/)) { print substr($0, RSTART + 1, RLENGTH - 2) }
exit
}' "${ROOT}/game.toml"
}
fail() { echo "error: $*" >&2; exit 1; }
[[ -f "${ROOT}/game.toml" ]] || fail "${ROOT}/game.toml missing -- not a game repo root"
boot="$(toml_get prepare_disc boot_exe || true)"
if [[ -z "${boot}" ]]; then
exe="$(toml_get game exe || true)"
boot="${exe##*/}"
fi
if [[ -z "${boot}" && -f "${ROOT}/CMakeLists.txt" ]]; then
boot="$(sed -n 's/.*GEN_MARKER[[:space:]]*"generated\/\(.*\)_dispatch\.c".*/\1/p' \
"${ROOT}/CMakeLists.txt" | head -n1)"
fi
[[ -n "${boot}" ]] || fail "cannot determine the boot-EXE name (game.toml [prepare_disc] boot_exe / [game] exe, or CMakeLists GEN_MARKER)"
gen="${ROOT}/generated"
dispatch="${gen}/${boot}_dispatch.c"
if [[ ! -f "${dispatch}" ]]; then
echo "error: ${dispatch#"${ROOT}"/} is missing." >&2
echo " Bundled releases build from COMMITTED game C. Generate it once from a" >&2
echo " legal disc and commit generated/:" >&2
echo " python3 psxrecomp/psxrecomp_cli.py generate --config game.toml \\" >&2
echo " --project-root . --disc disc/<game>.cue" >&2
echo " git add generated && git commit -m 'Add generated game C'" >&2
if grep -qE '^/?generated/?[[:space:]]*$' "${ROOT}/.gitignore" 2>/dev/null; then
echo " .gitignore still ignores generated/ (the old setup-host rule) -- remove" >&2
echo " that line; see psxrecomp/docs/ci/templates/game.gitignore." >&2
fi
exit 1
fi
shards=0
for f in "${gen}/${boot}_full"*.c; do
[[ -f "${f}" ]] && shards=$((shards + 1))
done
[[ "${shards}" -gt 0 ]] || fail "no generated/${boot}_full*.c beside ${boot}_dispatch.c -- regenerate"
# 2. Tracked, not merely present.
if git -C "${ROOT}" rev-parse --is-inside-work-tree >/dev/null 2>&1; then
untracked="$(git -C "${ROOT}" ls-files --others --exclude-standard -- "generated/${boot}_dispatch.c" "generated/${boot}_full"*.c 2>/dev/null || true)"
ignored="$(git -C "${ROOT}" ls-files --others --ignored --exclude-standard -- "generated/${boot}_dispatch.c" "generated/${boot}_full"*.c 2>/dev/null || true)"
if [[ -n "${untracked}${ignored}" ]]; then
echo "error: generated game C is present but NOT tracked by git:" >&2
printf '%s\n' ${untracked} ${ignored} | sed 's/^/ /' >&2
echo " A release builds from the checkout, so these files never reach CI." >&2
echo " Remove any /generated/ rule from .gitignore, then: git add generated" >&2
exit 1
fi
fi
# 3. No retail-BIOS-derived C anywhere it could link.
retail=()
for d in "${gen}" "${ROOT}/psxrecomp/generated"; do
[[ -d "${d}" ]] || continue
for f in "${d}"/SCPH*_dispatch.c "${d}"/SCPH*_full.c "${d}"/SCPH*_full_*.c; do
[[ -f "${f}" ]] && retail+=("${f#"${ROOT}"/}")
done
done
if [[ "${#retail[@]}" -gt 0 ]]; then
echo "error: retail-BIOS-derived C is present and would be linked into the release:" >&2
printf ' %s\n' "${retail[@]}" >&2
echo " Only the OpenBIOS backend ships (CI emits it from the bundled MIT image)." >&2
echo " Delete these from the checkout; they must never be committed." >&2
exit 1
fi
# 4. No BIOS dump tracked.
if git -C "${ROOT}" rev-parse --is-inside-work-tree >/dev/null 2>&1; then
dumps="$(git -C "${ROOT}" ls-files -- 'bios/*' '*.BIN' 'SCPH*' 2>/dev/null \
| grep -Ei '\.(bin|rom)$' | grep -v -i 'openbios' || true)"
if [[ -n "${dumps}" ]]; then
echo "error: BIOS image(s) tracked in the repo:" >&2
printf '%s\n' "${dumps}" | sed 's/^/ /' >&2
exit 1
fi
fi
echo "generated game C ok: generated/${boot}_dispatch.c + ${shards} full shard(s), tracked, no retail BIOS C"
-25
View File
@@ -1,25 +0,0 @@
#!/usr/bin/env bash
# Clear generated game/BIOS C so CI can build a BIOS-free setup host.
#
# Usage (from game repo root):
# psxrecomp/tools/ci/clear_generated.sh [--root DIR]
set -euo pipefail
ROOT="${PWD}"
while [[ $# -gt 0 ]]; do
case "$1" in
--root) ROOT="${2:?}"; shift 2 ;;
-h|--help)
sed -n '2,7p' "$0" | sed 's/^# \{0,1\}//'
exit 0
;;
*)
echo "error: unknown arg: $1" >&2
exit 2
;;
esac
done
ROOT="$(cd "${ROOT}" && pwd)"
rm -rf "${ROOT}/generated" "${ROOT}/psxrecomp/generated"
echo "cleared generated/ under ${ROOT} (setup-host mode)"
+82
View File
@@ -0,0 +1,82 @@
#!/usr/bin/env bash
# Emit the OpenBIOS backend C in CI so a bundled release links a BIOS.
#
# A bundled release compiles the title's COMMITTED game C, but BIOS C lives in
# the framework submodule's generated/ (psxrecomp/generated/), which a title
# cannot commit. OpenBIOS is MIT and its image ships in psxrecomp/bios/, so CI
# regenerates that one backend from the emitter it just built. Retail BIOS
# backends are never produced here: their C derives from a Sony image and must
# not reach a public artifact (tools/ci/check_generated.sh refuses them).
#
# This is the same emission tools/regen_bios.sh performs locally, minus that
# script's "rebuild the emitter first" step (CI has just built or restored it)
# and its search for a recompiler build dir under the framework root (CI
# builds emitters out of tree, at <game root>/build-recompiler).
#
# Usage (from game repo root):
# psxrecomp/tools/ci/generate_openbios.sh [--framework psxrecomp] \
# [--recompiler-build build-recompiler]
set -euo pipefail
FRAMEWORK="psxrecomp"
RECOMPILER_BUILD="build-recompiler"
while [[ $# -gt 0 ]]; do
case "$1" in
--framework) FRAMEWORK="${2:?}"; shift 2 ;;
--recompiler-build) RECOMPILER_BUILD="${2:?}"; shift 2 ;;
-h|--help)
sed -n '2,18p' "$0" | sed 's/^# \{0,1\}//'
exit 0
;;
*)
echo "error: unknown arg: $1" >&2
exit 2
;;
esac
done
[[ -d "${FRAMEWORK}" ]] || { echo "error: framework dir missing: ${FRAMEWORK}" >&2; exit 1; }
FRAMEWORK="$(cd "${FRAMEWORK}" && pwd)"
[[ -d "${RECOMPILER_BUILD}" ]] || { echo "error: recompiler build dir missing: ${RECOMPILER_BUILD}" >&2; exit 1; }
RECOMPILER_BUILD="$(cd "${RECOMPILER_BUILD}" && pwd)"
EMITTER=""
for cand in "${RECOMPILER_BUILD}/psxrecomp-bios" "${RECOMPILER_BUILD}/psxrecomp-bios.exe" \
"${RECOMPILER_BUILD}/Release/psxrecomp-bios.exe"; do
if [[ -f "${cand}" ]]; then EMITTER="${cand}"; break; fi
done
[[ -n "${EMITTER}" ]] || { echo "error: psxrecomp-bios not found under ${RECOMPILER_BUILD} (run build_emitters.sh)" >&2; exit 1; }
PROFILE="bios/OpenBIOS.toml"
for req in "${PROFILE}" bios/openbios.bin bios/OpenBIOS.LICENSE tools/bios_emitter_fingerprint.sh; do
[[ -f "${FRAMEWORK}/${req}" ]] || { echo "error: ${FRAMEWORK}/${req} missing" >&2; exit 1; }
done
toml_value() {
sed -n "s/^[[:space:]]*$1[[:space:]]*=[[:space:]]*\"\([^\"]*\)\".*/\1/p" "${FRAMEWORK}/${PROFILE}" | head -1
}
OUT="$(toml_value out_dir)"; OUT="${OUT:-generated}"
STEM="$(toml_value out_stem)"
[[ -n "${STEM}" ]] || { echo "error: no out_stem in ${PROFILE}" >&2; exit 1; }
# The profile's rom / seeds paths are framework-relative, so run from there.
mkdir -p "${FRAMEWORK}/${OUT}"
echo "generate_openbios: ${EMITTER} --config ${PROFILE} (stem ${STEM}) -> ${FRAMEWORK}/${OUT}"
( cd "${FRAMEWORK}" && "${EMITTER}" --config "${PROFILE}" )
# Same fingerprint regen_bios.sh records, so runtime.cmake's staleness check
# sees a matching stamp instead of warning on every CI configure.
( cd "${FRAMEWORK}" && bash tools/bios_emitter_fingerprint.sh "${PROFILE}" > "${OUT}/${STEM}.emitter.sha" )
dispatch="${FRAMEWORK}/${OUT}/${STEM}_dispatch.c"
full="${FRAMEWORK}/${OUT}/${STEM}_full.c"
for f in "${dispatch}" "${full}"; do
[[ -f "${f}" ]] || { echo "error: emitter produced no ${f}" >&2; exit 1; }
done
# runtime.cmake links a stem only when its descriptor is present; probe the
# same thing so a stale emitter fails here rather than as a silent skip.
if ! grep -q "${STEM}_psx_bios_backend" "${dispatch}"; then
echo "error: ${dispatch} defines no ${STEM}_psx_bios_backend descriptor -- emitter predates the backend mechanism" >&2
exit 1
fi
echo "generate_openbios: ok ($(grep -m1 -o 'Dispatch entries: [0-9]*' "${dispatch}" || echo 'descriptor present'))"
+1 -1
View File
@@ -1,6 +1,6 @@
<#
.SYNOPSIS
Write the setup-host release workflow into an existing PSX project (Windows).
Write the bundled release workflow into an existing PSX project (Windows).
.DESCRIPTION
The logic is tools/generate_ci.py; this only finds Python and passes every
+25 -17
View File
@@ -1,6 +1,9 @@
#!/usr/bin/env python3
"""generate_ci.py -- write the setup-host release workflow into an EXISTING
PSX project. Nothing else: no commit, no push, no migration plan.
"""generate_ci.py -- write the bundled release workflow into an EXISTING PSX
project. Nothing else: no commit, no push, no migration plan.
The workflow builds the title's COMMITTED generated/ game C and ships the
compiled game (docs/ci/BUNDLED_RELEASES.md).
python3 psxrecomp/tools/generate_ci.py # cwd is the project
python3 psxrecomp/tools/generate_ci.py ~/src/MyGameRecomp
@@ -10,14 +13,14 @@ PSX project. Nothing else: no commit, no push, no migration plan.
This is the thin, single-purpose face of Project Studio's emit_ci_workflow
operation (tools/new_project_layout/project_studio/ops.py), which is also
what `migrate_project.py apply` and `git install-ci` run. The template is
docs/ci/templates/setup-release.yml from the project's own psxrecomp
docs/ci/templates/game-release.yml from the project's own psxrecomp
submodule when it has one, so the workflow matches the framework the project
pins; this checkout is the fallback.
Where the values come from:
ZIP_PREFIX scripts/package_setup_release.sh --zip-prefix <x> (the
packager's prefix, so the zips CI uploads are the zips it
built), else derived from the project name, else --zip-prefix
ZIP_PREFIX scripts/package_release.sh --zip-prefix <x> (the packager's
prefix, so the zips CI uploads are the zips it built), else
derived from the project name, else --zip-prefix
GAME_TITLE CMakeLists.txt / game.toml window_title, else --title
An existing, filled release.yml is not overwritten silently: its step names
@@ -40,13 +43,17 @@ WORKFLOW_REL = pathlib.Path(".github") / "workflows" / "release.yml"
def zip_prefix_from_packager(root: pathlib.Path) -> str:
try:
text = (root / "scripts" / "package_setup_release.sh").read_text(
encoding="utf-8", errors="replace")
except OSError:
return ""
m = re.search(r"--zip-prefix\s+([^\s\\'\"]+)", text)
return m.group(1) if m else ""
# package_setup_release.sh is the retired setup-host wrapper; its prefix is
# still the right one for a title that has not re-emitted the packager.
for name in ("package_release.sh", "package_setup_release.sh"):
try:
text = (root / "scripts" / name).read_text(encoding="utf-8", errors="replace")
except OSError:
continue
m = re.search(r"--zip-prefix\s+([^\s\\'\"]+)", text)
if m:
return m.group(1)
return ""
def main() -> int:
@@ -70,16 +77,17 @@ def main() -> int:
try:
from project_studio.models import MigrateOptions
from project_studio.ops import op_emit_ci_workflow, _ci_steps_missing_from
from project_studio.paths import ci_setup_release_template
from project_studio.paths import ci_release_template
except ImportError as exc:
print(f"generate_ci: cannot import Project Studio from {HERE / 'new_project_layout'}: {exc}",
file=sys.stderr)
return 2
template = ci_setup_release_template(root)
template = ci_release_template(root)
if template is None:
print("generate_ci: no docs/ci/templates/setup-release.yml found (neither the "
"project's psxrecomp submodule nor this checkout)", file=sys.stderr)
print("generate_ci: no docs/ci/templates/game-release.yml found (neither the "
"project's psxrecomp submodule nor this checkout -- a submodule pinned "
"before bundled releases must be bumped first)", file=sys.stderr)
return 2
dst = root / WORKFLOW_REL
+1 -1
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env sh
# Write the setup-host release workflow into an existing PSX project (Linux /
# Write the bundled release workflow into an existing PSX project (Linux /
# macOS / WSL / Git Bash). The logic is tools/generate_ci.py; this finds Python.
#
# sh psxrecomp/tools/generate_ci.sh # cwd is the project
+9 -6
View File
@@ -1,8 +1,9 @@
# New Project Layout + Project Studio
Scaffold **new** titles and **migrate** older ones onto the setup-host layout.
Scaffold **new** titles and **migrate** older ones onto the New Project Layout.
**Policy:** public releases are **setup-host only** (no prebuilt generated game C).
**Policy:** public releases are **bundled** — `generated/` is committed and CI
builds and ships the compiled game (`docs/ci/BUNDLED_RELEASES.md`).
## New project
@@ -85,11 +86,13 @@ For **bulk ops across many titles / platforms**, use the sibling tool
| `ensure_recomp_ui_submodule` | Add `recomp-ui` |
| `emit_codegen_setup` | Thin `codegen_setup.c/.h` |
| `rewrite_cmake_setup_host` | `psxrecomp_add_game_runtime` + wizard |
| `emit_packager` | `scripts/package_setup_release.sh` |
| `emit_ci_workflow` | Setup-host `release.yml` |
| `merge_gitignore` | Disc/BIOS/dist rules; **un-ignores** `generated/` |
| `emit_packager` | `scripts/package_release.sh` (bundled) |
| `emit_ci_workflow` | Bundled `release.yml` |
| `probe_disc_refresh` | TOC / catalog / seeds (needs `--disc`) |
| `annotate_legacy_packaging` | Mark old prebuilt packagers obsolete |
| `annotate_legacy_packaging` | Mark hand-forked packagers obsolete |
Wizard + `recomp-ui` are forced on for `apply` (setup-host requirement).
Wizard + `recomp-ui` are forced on for `apply` (the shipped game's first-run
disc picker).
Helpers reused: `probe_disc.py`, `fill_tokens.py`, `sync_symbols.py`, `write_recomp_json.py` (`.recomp.json`, [recomp.fyi spec](https://recomp.fyi/spec)), `templates/*`.
@@ -1,6 +1,6 @@
"""PSXRecomp Project Studio — audit / plan / apply for New Project Layout.
Setup-host releases only (no prebuilt game-C packaging path).
Bundled releases: generated/ is committed and CI ships the compiled game.
"""
from __future__ import annotations
@@ -848,7 +848,7 @@ def launch_status() -> str:
# a broken package (an empty Mods page) survived so long -- nobody produced one
# locally to look inside.
#
# This runs the SAME wrapper CI runs (scripts/package_setup_release.sh), so a
# This runs the SAME wrapper CI runs (scripts/package_release.sh), so a
# local export and a CI release are the same artifact by construction rather
# than by convention.
# ---------------------------------------------------------------------------
@@ -891,11 +891,13 @@ def export_release(
packages dropped. A local export keeps them by default, because the point
of exporting locally is to test the work in progress.
"""
wrapper = root / "scripts" / "package_setup_release.sh"
if not wrapper.is_file():
from .detect import packager_wrapper
wrapper = packager_wrapper(root)
if wrapper is None:
return CmdResult(
False,
"No scripts/package_setup_release.sh",
"No scripts/package_release.sh",
"Emit it first (Migrate -> packager), or install release CI.",
)
build_path = root / build_dir
@@ -907,7 +907,7 @@ def bulk_install_ci(
jobs: int = 1,
on_repo: OnRepoResults | None = None,
) -> list[CmdResult]:
"""Install/push setup-host ``release.yml`` on each selected game repo."""
"""Install/push the bundled ``release.yml`` on each selected game repo."""
from fill_tokens import derive_zip_prefix
def one(label: str, root: Path) -> list[CmdResult]:
@@ -49,7 +49,7 @@ def _print_audit(report, *, as_json: bool) -> int:
fails = sum(1 for c in report.checks if c.status.value == "fail")
warns = sum(1 for c in report.checks if c.status.value == "warn")
print()
print(f"Summary: {fails} fail, {warns} warn (setup-host releases only)")
print(f"Summary: {fails} fail, {warns} warn (bundled releases: committed generated/ C)")
return 1 if fails else 0
@@ -116,7 +116,7 @@ def cmd_plan(args: argparse.Namespace) -> int:
def cmd_apply(args: argparse.Namespace) -> int:
root = Path(args.root).expanduser().resolve()
opts = _options_from_args(args)
# Setup-host exclusive hard rules
# Hard rules: recomp-ui + wizard are the shipped game's first-run disc picker
opts.enable_wizard = True
opts.enable_recomp_ui = True
if opts.players < 2:
@@ -1600,7 +1600,7 @@ def build_parser() -> argparse.ArgumentParser:
prog="migrate_project",
description=(
"PSXRecomp Project Studio — migrate / update title repos to the "
"New Project Layout (setup-host releases only)."
"New Project Layout (bundled releases: committed generated/ C, compiled game shipped)."
),
)
ap.add_argument("--version", action="version", version=f"%(prog)s {__version__}")
@@ -1625,9 +1625,9 @@ def build_parser() -> argparse.ArgumentParser:
p.add_argument("--enable-netplay", action="store_true")
p.add_argument("--lobby-url", default="ws://netplay.retcomm.net:8765")
p.add_argument("--no-recomp-ui", action="store_true",
help="Ignored for setup-host apply (forced on)")
help="Ignored for apply (forced on)")
p.add_argument("--no-wizard", action="store_true",
help="Ignored for setup-host apply (forced on)")
help="Ignored for apply (forced on)")
p.add_argument("--no-ci", action="store_true")
p.add_argument("--no-boxart", action="store_true")
p.add_argument("--no-rewrite-cmake", action="store_true")
@@ -2206,7 +2206,7 @@ def build_parser() -> argparse.ArgumentParser:
p_gbci = git_sub.add_parser(
"bulk-install-ci",
help="Install/push setup-host release.yml on selected indexed repos",
help="Install/push bundled release.yml on selected indexed repos",
)
add_bulk_select(p_gbci)
p_gbci.add_argument(
@@ -2233,7 +2233,7 @@ def build_parser() -> argparse.ArgumentParser:
p_gci = git_sub.add_parser(
"install-ci",
help="Write psxrecomp setup-release.yml, commit, and push",
help="Write psxrecomp game-release.yml, commit, and push",
)
add_git_root(p_gci)
p_gci.add_argument("--zip-prefix", default="", help="CI asset zip prefix")
@@ -2281,12 +2281,12 @@ def build_parser() -> argparse.ArgumentParser:
p_bm.add_argument(
"--setup-host",
action="store_true",
help="CI-parity setup-host configure (FORCE_SETUP_HOST + wizard)",
help="Retired: setup-host configure parity (releases build the committed game C)",
)
p_bm.add_argument(
"--package",
action="store_true",
help="Run scripts/package_setup_release.sh after build",
help="Run scripts/package_release.sh after build",
)
p_bm.add_argument(
"--package-only",
+170 -30
View File
@@ -1,4 +1,4 @@
"""Audit an existing title repo against the New Project Layout (setup-host)."""
"""Audit an existing title repo against the New Project Layout (bundled releases)."""
from __future__ import annotations
@@ -144,6 +144,26 @@ def _cmake_source_refs(cmake: str) -> tuple[set[str], set[str]]:
return refs, guarded
# The bundled-release wrapper and the retired setup-host one. Audits accept
# either as "a packager is present" but only the first as current.
PACKAGER_WRAPPER = "package_release.sh"
LEGACY_PACKAGER_WRAPPER = "package_setup_release.sh"
def packager_wrapper(root: Path) -> Path | None:
"""The title's packager wrapper: the bundled one, else the retired one."""
for name in (PACKAGER_WRAPPER, LEGACY_PACKAGER_WRAPPER):
p = root / "scripts" / name
if p.is_file():
return p
return None
def packager_is_bundled(packager: Path) -> bool:
"""True when the wrapper calls the shared bundled packager (no source allowlist)."""
return "package_game_release.sh" in _read(packager)
def packager_staged_paths(packager_text: str) -> set[str]:
"""Top-level paths the packager passes as --project-file / --project-dir."""
out: set[str] = set()
@@ -165,8 +185,8 @@ def packager_missing_paths(root: Path) -> list[str]:
own pre-zip gate applies, so the two cannot disagree.
"""
cmake_path = root / "CMakeLists.txt"
packager = root / "scripts" / "package_setup_release.sh"
if not cmake_path.is_file() or not packager.is_file():
packager = packager_wrapper(root)
if not cmake_path.is_file() or packager is None or packager_is_bundled(packager):
return []
refs, guarded = _cmake_source_refs(_read(cmake_path))
staged = packager_staged_paths(_read(packager))
@@ -190,8 +210,8 @@ def packager_missing_optional_paths(root: Path) -> list[str]:
select, so the mod builds and never enables.
"""
cmake_path = root / "CMakeLists.txt"
packager = root / "scripts" / "package_setup_release.sh"
if not cmake_path.is_file() or not packager.is_file():
packager = packager_wrapper(root)
if not cmake_path.is_file() or packager is None or packager_is_bundled(packager):
return []
refs, guarded = _cmake_source_refs(_read(cmake_path))
staged = packager_staged_paths(_read(packager))
@@ -457,15 +477,29 @@ def audit_project(root: Path) -> AuditReport:
)
# Packager
packager = root / "scripts" / "package_setup_release.sh"
if packager.is_file():
packager = packager_wrapper(root)
if packager is not None and packager_is_bundled(packager):
checks.append(
CheckResult(
id="setup_host_packager",
title="scripts/package_setup_release.sh",
title=f"scripts/{PACKAGER_WRAPPER}",
status=CheckStatus.PASS,
severity=Severity.REQUIRED,
detail="Setup-host packager wrapper present.",
detail="Bundled-release packager wrapper present (ships the compiled game).",
)
)
elif packager is not None:
checks.append(
CheckResult(
id="setup_host_packager",
title=f"scripts/{PACKAGER_WRAPPER}",
status=CheckStatus.FAIL,
severity=Severity.REQUIRED,
detail=(
f"Only the retired setup-host wrapper scripts/{packager.name} is present; "
"releases now ship the compiled game. Re-emit the packager (force)."
),
fix_op="emit_packager",
)
)
# Present is not the same as current: the allowlist inside it has to
@@ -521,10 +555,10 @@ def audit_project(root: Path) -> AuditReport:
checks.append(
CheckResult(
id="setup_host_packager",
title="scripts/package_setup_release.sh",
title=f"scripts/{PACKAGER_WRAPPER}",
status=CheckStatus.FAIL,
severity=Severity.REQUIRED,
detail="Missing setup-host packager (required; prebuilt releases are not used).",
detail="Missing bundled-release packager wrapper (required).",
fix_op="emit_packager",
)
)
@@ -537,18 +571,32 @@ def audit_project(root: Path) -> AuditReport:
checks.append(
CheckResult(
id="ci_release",
title="Setup-host release CI",
title="Bundled release CI",
status=CheckStatus.WARN,
severity=Severity.REQUIRED,
detail="release.yml still has YOUR_* placeholders.",
fix_op="emit_ci_workflow",
)
)
elif "clear_generated.sh" in wtext or "FORCE_SETUP_HOST=ON" in wtext:
checks.append(
CheckResult(
id="ci_release",
title="Bundled release CI",
status=CheckStatus.FAIL,
severity=Severity.REQUIRED,
detail=(
"release.yml is the retired setup-host workflow (wipes generated/ and "
"ships an uncompiled kit). Re-emit it (force) to ship the compiled game."
),
fix_op="emit_ci_workflow",
)
)
else:
checks.append(
CheckResult(
id="ci_release",
title="Setup-host release CI",
title="Bundled release CI",
status=CheckStatus.PASS,
severity=Severity.REQUIRED,
detail=".github/workflows/release.yml present.",
@@ -558,28 +606,31 @@ def audit_project(root: Path) -> AuditReport:
checks.append(
CheckResult(
id="ci_release",
title="Setup-host release CI",
title="Bundled release CI",
status=CheckStatus.FAIL,
severity=Severity.REQUIRED,
detail="Missing setup-host release.yml workflow.",
detail="Missing release.yml workflow.",
fix_op="emit_ci_workflow",
)
)
# Legacy prebuilt packaging — warn only (setup-host exclusive policy)
# Hand-forked per-title packagers — warn only. The shared bundled packager
# (scripts/package_release.sh -> psxrecomp/tools/package_game_release.sh)
# supersedes them; forks are where the cache-tag drift lived.
legacy_pack = False
if (root / "packaging").is_dir() or (root / "tools" / "package_release.ps1").is_file():
if (root / "packaging").is_dir() or (root / "tools" / "package_release.ps1").is_file() \
or (root / "tools" / "package_appimage.sh").is_file():
legacy_pack = True
checks.append(
CheckResult(
id="legacy_prebuilt",
title="Legacy prebuilt packaging",
title="Hand-forked packaging scripts",
status=CheckStatus.WARN,
severity=Severity.RECOMMENDED,
detail=(
"Found packaging/ or tools/package_release.ps1. "
"This studio only ships setup-host zips — leave prebuilt scripts unused "
"or remove them after CI is green."
"Found packaging/, tools/package_release.ps1 or tools/package_appimage.sh. "
"Releases go through scripts/package_release.sh + release.yml; "
"remove the forks once CI is green."
),
fix_op="annotate_legacy_packaging",
)
@@ -588,10 +639,10 @@ def audit_project(root: Path) -> AuditReport:
checks.append(
CheckResult(
id="legacy_prebuilt",
title="Legacy prebuilt packaging",
title="Hand-forked packaging scripts",
status=CheckStatus.PASS,
severity=Severity.INFO,
detail="No legacy prebuilt packager detected.",
detail="No hand-forked packager detected.",
)
)
@@ -813,25 +864,42 @@ def audit_project(root: Path) -> AuditReport:
gi = _read(root / ".gitignore")
missing_gi = [
pat
for pat in ("/generated/", "/disc/", "/bios/", "/dist/", "/analysis/")
for pat in ("/disc/", "/bios/", "/dist/", "/analysis/")
if pat not in gi and pat.rstrip("/") not in gi
]
ignores_generated = any(
re.match(r"^\s*/?generated/?\s*$", line) for line in gi.splitlines()
)
if not (root / ".gitignore").is_file():
checks.append(
CheckResult(
id="gitignore",
title=".gitignore setup-host rules",
title=".gitignore rules",
status=CheckStatus.FAIL,
severity=Severity.RECOMMENDED,
detail="Missing .gitignore.",
fix_op="merge_gitignore",
)
)
elif ignores_generated:
checks.append(
CheckResult(
id="gitignore",
title=".gitignore rules",
status=CheckStatus.FAIL,
severity=Severity.REQUIRED,
detail=(
".gitignore ignores generated/ (the retired setup-host rule). Releases "
"build the committed game C, so CI would find nothing to compile."
),
fix_op="merge_gitignore",
)
)
elif missing_gi:
checks.append(
CheckResult(
id="gitignore",
title=".gitignore setup-host rules",
title=".gitignore rules",
status=CheckStatus.WARN,
severity=Severity.RECOMMENDED,
detail="Missing patterns: " + ", ".join(missing_gi),
@@ -842,10 +910,82 @@ def audit_project(root: Path) -> AuditReport:
checks.append(
CheckResult(
id="gitignore",
title=".gitignore setup-host rules",
title=".gitignore rules",
status=CheckStatus.PASS,
severity=Severity.RECOMMENDED,
detail="Essential setup-host ignore rules present.",
detail="Disc / BIOS / dist / analysis ignored; generated/ tracked.",
)
)
# Committed game C: the release's input. Present and tracked, or the
# workflow has nothing to build.
if boot:
marker = root / "generated" / f"{boot}_dispatch.c"
tracked = False
if marker.is_file():
try:
r = subprocess.run(
["git", "-C", str(root), "ls-files", "--error-unmatch",
f"generated/{boot}_dispatch.c"],
capture_output=True, text=True, check=False,
)
tracked = r.returncode == 0
except OSError:
tracked = False
if not marker.is_file():
checks.append(
CheckResult(
id="generated_committed",
title="Committed game C",
status=CheckStatus.FAIL,
severity=Severity.REQUIRED,
detail=(
f"generated/{boot}_dispatch.c is missing. Generate once from a legal "
"disc (psxrecomp_cli.py generate) and commit generated/."
),
)
)
elif not tracked:
checks.append(
CheckResult(
id="generated_committed",
title="Committed game C",
status=CheckStatus.FAIL,
severity=Severity.REQUIRED,
detail=(
f"generated/{boot}_dispatch.c exists but is not tracked by git; "
"release CI checks out the repo and would find nothing to build. "
"git add generated."
),
)
)
else:
checks.append(
CheckResult(
id="generated_committed",
title="Committed game C",
status=CheckStatus.PASS,
severity=Severity.REQUIRED,
detail=f"generated/{boot}_dispatch.c tracked.",
)
)
retail_c = sorted(
p.relative_to(root).as_posix()
for d in (root / "generated", root / "psxrecomp" / "generated")
if d.is_dir()
for p in d.glob("SCPH*_*.c")
)
if retail_c:
checks.append(
CheckResult(
id="retail_bios_c",
title="Retail BIOS C absent",
status=CheckStatus.FAIL,
severity=Severity.REQUIRED,
detail=(
"Retail-BIOS-derived C present: " + ", ".join(retail_c[:4])
+ ". It must never be committed or shipped; only OpenBIOS links in a release."
),
)
)
@@ -946,7 +1086,7 @@ def audit_project(root: Path) -> AuditReport:
if legacy_pack or "psxrecomp_add_runtime_target" in cmake or has_v4:
layout = LayoutClass.LEGACY_PACKAGING
elif fails:
if packager.is_file() or wizard_ok or (root / "codegen_setup.c").is_file():
if packager is not None or wizard_ok or (root / "codegen_setup.c").is_file():
layout = LayoutClass.SETUP_HOST_PARTIAL
else:
layout = LayoutClass.UNKNOWN
@@ -958,7 +1098,7 @@ def audit_project(root: Path) -> AuditReport:
if boot is None:
notes.append("Could not parse [game].exe from game.toml — set --boot-exe when applying.")
notes.append(
"Policy: setup-host releases only (no prebuilt generated-C zips)."
"Policy: bundled releases -- generated/ is committed and CI ships the compiled game."
)
return AuditReport(
@@ -1463,7 +1463,7 @@ def install_and_push_release_ci(
push_remote: bool = True,
dry_run: bool = False,
) -> CmdResult:
"""Write psxrecomp setup-release.yml (+ packager), commit, and push.
"""Write psxrecomp game-release.yml (+ packager), commit, and push.
Uses the same template as ``setup_project --enable-ci`` /
``op_emit_ci_workflow``. After push, best-effort checks that Actions has
@@ -1493,7 +1493,7 @@ def install_and_push_release_ci(
if not paths and not dry_run:
# Already present — still allow push of any uncommitted workflow edits.
wf = root / ".github" / "workflows" / "release.yml"
pkg = root / "scripts" / "package_setup_release.sh"
pkg = root / "scripts" / "package_release.sh"
for p in (wf, pkg):
if p.is_file():
rel = str(p.relative_to(root)).replace("\\", "/")
@@ -1538,7 +1538,7 @@ def install_and_push_release_ci(
root,
"commit",
"-m",
"ci: add setup-host release.yml (psxrecomp template)",
"ci: add bundled release.yml (psxrecomp template)",
)
if code != 0:
return CmdResult(False, "git commit failed", err or out)
@@ -523,7 +523,7 @@ class ProjectStudioApp:
).pack(side="left")
ctk.CTkLabel(
header,
text="Migrate, audit, and GitHub ops for setup-host game repos",
text="Migrate, audit, and GitHub ops for PSXRecomp game repos",
text_color=("gray40", "gray65"),
font=ctk.CTkFont(size=13),
).pack(side="left", padx=(12, 0), pady=(6, 0))
@@ -657,7 +657,7 @@ class ProjectStudioApp:
opts.pack(fill="x", padx=4, pady=4)
ctk.CTkLabel(
opts,
text="Options · setup-host only",
text="Options · bundled releases",
font=ctk.CTkFont(size=13, weight="bold"),
).pack(anchor="w", padx=12, pady=(10, 4))
@@ -707,7 +707,7 @@ class ProjectStudioApp:
ctk.CTkLabel(
opts,
text="Wizard + recomp-ui are always enabled. Releases are setup-host only (no prebuilt game C).",
text="Wizard + recomp-ui are always enabled. Releases build the committed generated/ C and ship the compiled game.",
text_color=("gray40", "gray60"),
font=ctk.CTkFont(size=12),
wraplength=900,
@@ -1935,7 +1935,7 @@ class ProjectStudioApp:
).pack(side="left")
ctk.CTkLabel(
rel,
text="Install & push CI writes psxrecomp setup-release.yml → "
text="Install & push CI writes psxrecomp game-release.yml → "
".github/workflows/release.yml, commits, pushes, and registers Actions.",
text_color=("gray30", "gray70"),
anchor="w",
@@ -5000,7 +5000,7 @@ class ProjectStudioApp:
elif not self._git_dry():
if not messagebox.askyesno(
"Project Studio",
"Install psxrecomp setup-release.yml as .github/workflows/release.yml,\n"
"Install psxrecomp game-release.yml as .github/workflows/release.yml,\n"
"commit, push to origin, and register Actions?\n\n"
f"{root}",
parent=self.root,
@@ -76,7 +76,7 @@ class AuditReport:
@dataclass
class MigrateOptions:
"""User choices for apply (setup-host exclusively)."""
"""User choices for apply (bundled releases: committed generated/ C)."""
disc: str | None = None
project_name: str | None = None
@@ -170,7 +170,7 @@ def netplay_configured(root: Path) -> bool:
def ci_workflow_present(root: Path) -> bool:
"""True if setup-host ``.github/workflows/release.yml`` exists."""
"""True if ``.github/workflows/release.yml`` exists."""
root = root.expanduser().resolve()
return (root / ".github" / "workflows" / "release.yml").is_file()
+83 -48
View File
@@ -1,4 +1,4 @@
"""Apply migration plan steps (setup-host exclusively)."""
"""Apply migration plan steps (bundled releases: committed generated/ C, compiled game shipped)."""
from __future__ import annotations
@@ -23,7 +23,14 @@ from .naming import (
window_title_from_cmake,
window_title_from_name,
)
from .paths import ci_setup_release_template, psxrecomp_root_from_toolkit, templates_dir, toolkit_dir
from .paths import ci_release_template, psxrecomp_root_from_toolkit, templates_dir, toolkit_dir
# The title's packager wrapper (bundled release) and the retired setup-host one.
PACKAGER_WRAPPER = "package_release.sh"
LEGACY_PACKAGER_WRAPPER = "package_setup_release.sh"
# A .gitignore line that hides the release's inputs: `/generated/`, `generated/`,
# `generated`, with optional surrounding whitespace.
_GENERATED_IGNORE_RE = re.compile(r"^\s*/?generated/?\s*$")
def _run(cmd: list[str], cwd: Path, dry_run: bool) -> tuple[bool, str]:
@@ -660,8 +667,9 @@ def op_merge_gitignore(root: Path, options: MigrateOptions) -> ApplyResult:
dst = root / ".gitignore"
existing = dst.read_text(encoding="utf-8") if dst.is_file() else ""
# generated/ is deliberately NOT here: releases build from the committed
# game C, so an ignore rule for it is removed below, never added.
required = [
"/generated/",
"/disc/",
"/bios/",
"/dist/",
@@ -684,21 +692,35 @@ def op_merge_gitignore(root: Path, options: MigrateOptions) -> ApplyResult:
_write(dst, template if not template.startswith("@") else "\n".join(required) + "\n", options.dry_run)
return ApplyResult("merge_gitignore", True, "Wrote .gitignore from template", [".gitignore"])
if not additions:
# The setup-host layout ignored generated/; a bundled release is built
# from it, so that rule hides the release's inputs from CI. Drop it.
kept_lines = []
dropped = 0
for line in existing.splitlines():
if _GENERATED_IGNORE_RE.match(line):
dropped += 1
continue
kept_lines.append(line)
if dropped:
existing = "\n".join(kept_lines) + ("\n" if existing.endswith("\n") else "")
if not additions and not dropped:
return ApplyResult("merge_gitignore", True, "No gitignore changes needed", [])
block = (
"\n# --- PSXRecomp setup-host (project studio) ---\n"
+ "\n".join(additions)
+ "\n"
)
_write(dst, existing.rstrip() + "\n" + block, options.dry_run)
return ApplyResult(
"merge_gitignore",
True,
f"Appended {len(additions)} ignore rules",
[".gitignore"],
)
text = existing.rstrip() + "\n"
if additions:
text += (
"\n# --- PSXRecomp (project studio) ---\n"
+ "\n".join(additions)
+ "\n"
)
_write(dst, text, options.dry_run)
parts = []
if additions:
parts.append(f"appended {len(additions)} ignore rule(s)")
if dropped:
parts.append("un-ignored generated/ (releases build from the committed game C)")
return ApplyResult("merge_gitignore", True, "; ".join(parts).capitalize(), [".gitignore"])
def op_emit_mods_preloaded(root: Path, options: MigrateOptions) -> ApplyResult:
@@ -851,9 +873,8 @@ def op_ensure_app_icon(root: Path, options: MigrateOptions) -> ApplyResult:
def op_rewrite_cmake_setup_host(root: Path, options: MigrateOptions) -> ApplyResult:
tokens = _resolve_tokens(root, options)
# Force wizard for setup-host policy
# The wizard is the shipped game's first-run disc picker; keep it on.
if not options.enable_wizard and options.enable_recomp_ui:
# Still rewrite with wizard ON — exclusive setup-host
options = MigrateOptions(**{**options.to_dict(), "enable_wizard": True})
tokens = _resolve_tokens(root, options)
@@ -886,7 +907,7 @@ def op_rewrite_cmake_setup_host(root: Path, options: MigrateOptions) -> ApplyRes
if extras_bits and not options.dry_run:
extras_note.write_text(
"Preserved notes after setup-host CMake rewrite:\n\n"
"Preserved notes after the CMake rewrite:\n\n"
+ "\n".join(f"- {b}" for b in extras_bits)
+ "\n\nFull previous file: CMakeLists.txt.pre_migrate.bak\n",
encoding="utf-8",
@@ -896,25 +917,25 @@ def op_rewrite_cmake_setup_host(root: Path, options: MigrateOptions) -> ApplyRes
return ApplyResult(
"rewrite_cmake_setup_host",
True,
"Wrote setup-host CMakeLists.txt (backup .pre_migrate.bak)",
"Wrote CMakeLists.txt (psxrecomp_add_game_runtime; backup .pre_migrate.bak)",
changed,
)
def op_emit_packager(root: Path, options: MigrateOptions) -> ApplyResult:
tokens = _resolve_tokens(root, options)
dst = root / "scripts" / "package_setup_release.sh"
dst = root / "scripts" / PACKAGER_WRAPPER
if dst.is_file() and not options.force:
return ApplyResult("emit_packager", True, "Packager already present", [])
_fill(templates_dir() / "package_setup_release.sh.in", dst, tokens, options.dry_run)
_fill(templates_dir() / (PACKAGER_WRAPPER + ".in"), dst, tokens, options.dry_run)
if not options.dry_run:
dst.chmod(dst.stat().st_mode | 0o111)
return ApplyResult(
"emit_packager",
True,
"Wrote scripts/package_setup_release.sh",
["scripts/package_setup_release.sh"],
)
msg = f"Wrote scripts/{PACKAGER_WRAPPER} (bundled release: compiled game)"
legacy = root / "scripts" / LEGACY_PACKAGER_WRAPPER
if legacy.is_file():
msg += (f"; scripts/{LEGACY_PACKAGER_WRAPPER} is the retired setup-host "
"wrapper and can be deleted")
return ApplyResult("emit_packager", True, msg, [f"scripts/{PACKAGER_WRAPPER}"])
_PACKAGER_BLURB = {
@@ -937,13 +958,24 @@ def op_sync_packager_project_dirs(root: Path, options: MigrateOptions) -> ApplyR
routinely hand-extended with extra --project-file lines, and rewriting from
the template would silently drop them.
"""
from .detect import packager_missing_optional_paths, packager_missing_paths
from .detect import (
packager_missing_optional_paths,
packager_missing_paths,
packager_wrapper,
packager_is_bundled,
)
dst = root / "scripts" / "package_setup_release.sh"
if not dst.is_file():
dst = packager_wrapper(root)
if dst is None:
return ApplyResult(
"sync_packager_project_dirs", False, "No scripts/package_setup_release.sh", []
"sync_packager_project_dirs", False, f"No scripts/{PACKAGER_WRAPPER}", []
)
if packager_is_bundled(dst):
# A bundled release ships the compiled game and no sources, so there
# is no source allowlist to keep in step with CMakeLists.txt.
return ApplyResult(
"sync_packager_project_dirs", True,
"Bundled packager ships no sources; nothing to stage", [])
missing = sorted(set(packager_missing_paths(root)) |
set(packager_missing_optional_paths(root)))
if not missing:
@@ -977,11 +1009,12 @@ def op_sync_packager_project_dirs(root: Path, options: MigrateOptions) -> ApplyR
text = text.replace(anchor, "\n" + block + anchor, 1)
_write(dst, text, options.dry_run)
rel = dst.relative_to(root).as_posix()
return ApplyResult(
"sync_packager_project_dirs",
True,
"Staged " + ", ".join(added) + " in scripts/package_setup_release.sh",
["scripts/package_setup_release.sh"],
"Staged " + ", ".join(added) + f" in {rel}",
[rel],
)
@@ -1015,12 +1048,13 @@ def _ci_steps_missing_from(installed: Path, template: Path) -> list[str]:
def op_emit_ci_workflow(root: Path, options: MigrateOptions) -> ApplyResult:
tokens = _resolve_tokens(root, options)
src = ci_setup_release_template(root)
src = ci_release_template(root)
if src is None:
return ApplyResult(
"emit_ci_workflow",
False,
"Cannot find docs/ci/templates/setup-release.yml (need psxrecomp submodule)",
"Cannot find docs/ci/templates/game-release.yml (need a psxrecomp "
"submodule pinned at or after bundled releases)",
[],
)
dst = root / ".github" / "workflows" / "release.yml"
@@ -1054,36 +1088,37 @@ def op_emit_ci_workflow(root: Path, options: MigrateOptions) -> ApplyResult:
return ApplyResult(
"emit_ci_workflow",
True,
"Wrote .github/workflows/release.yml (setup-host)",
"Wrote .github/workflows/release.yml (bundled release: builds the committed generated/ C)",
[".github/workflows/release.yml"],
)
def op_annotate_legacy_packaging(root: Path, options: MigrateOptions) -> ApplyResult:
note = root / "packaging" / "SETUP_HOST_MIGRATION.txt"
note = root / "packaging" / "PACKAGING_MIGRATION.txt"
text = (
"This title is migrating to setup-host releases only.\n"
"Do not ship prebuilt generated game C.\n"
"Use scripts/package_setup_release.sh + .github/workflows/release.yml\n"
"(see psxrecomp/docs/GAME_PROJECT_SETUP.md and docs/ci/HOST_ONLY_RELEASES.md).\n"
"Legacy package_release scripts in this folder are obsolete for public releases.\n"
"This title releases through the shared bundled-release flow:\n"
f"scripts/{PACKAGER_WRAPPER} + .github/workflows/release.yml build the\n"
"COMMITTED generated/ game C and ship the compiled game\n"
"(see psxrecomp/docs/GAME_PROJECT_SETUP.md and docs/ci/BUNDLED_RELEASES.md).\n"
"Hand-forked packagers in this folder (package_release.ps1,\n"
"package_appimage.sh, ...) are superseded and can be deleted once CI is green.\n"
)
if (root / "packaging").is_dir():
_write(note, text, options.dry_run)
return ApplyResult(
"annotate_legacy_packaging",
True,
"Wrote packaging/SETUP_HOST_MIGRATION.txt",
["packaging/SETUP_HOST_MIGRATION.txt"],
"Wrote packaging/PACKAGING_MIGRATION.txt",
["packaging/PACKAGING_MIGRATION.txt"],
)
# tools-only legacy
tools_note = root / "tools" / "SETUP_HOST_MIGRATION.txt"
tools_note = root / "tools" / "PACKAGING_MIGRATION.txt"
_write(tools_note, text, options.dry_run)
return ApplyResult(
"annotate_legacy_packaging",
True,
"Wrote tools/SETUP_HOST_MIGRATION.txt",
["tools/SETUP_HOST_MIGRATION.txt"],
"Wrote tools/PACKAGING_MIGRATION.txt",
["tools/PACKAGING_MIGRATION.txt"],
)
@@ -23,17 +23,28 @@ def psxrecomp_root_from_toolkit() -> Path | None:
return None
def ci_setup_release_template(game_root: Path | None = None) -> Path | None:
"""Prefer game submodule copy; fall back to toolkit's parent psxrecomp."""
CI_RELEASE_TEMPLATE = "game-release.yml"
def ci_release_template(game_root: Path | None = None) -> Path | None:
"""The bundled-release workflow template (docs/ci/templates/game-release.yml).
Prefer the game's own psxrecomp submodule so the workflow matches the
framework the project pins; fall back to the toolkit's parent psxrecomp.
A submodule pinned before bundled releases has no such template, and the
caller says so rather than silently installing the retired setup-host one.
"""
if game_root is not None:
p = game_root / "psxrecomp" / "docs" / "ci" / "templates" / "setup-release.yml"
if p.is_file():
return p
p = game_root / "psxrecomp-v4" / "docs" / "ci" / "templates" / "setup-release.yml"
if p.is_file():
return p
for sub in ("psxrecomp", "psxrecomp-v4"):
p = game_root / sub / "docs" / "ci" / "templates" / CI_RELEASE_TEMPLATE
if p.is_file():
return p
root = psxrecomp_root_from_toolkit()
if root is None:
return None
p = root / "docs" / "ci" / "templates" / "setup-release.yml"
p = root / "docs" / "ci" / "templates" / CI_RELEASE_TEMPLATE
return p if p.is_file() else None
# Retired name, kept so a caller pinned to it still resolves.
ci_setup_release_template = ci_release_template
@@ -42,16 +42,16 @@ _OP_TITLES = {
"emit_version": "Emit VERSION",
"emit_symbols_toml": "Emit symbols.toml stub",
"emit_sync_symbols": "Install tools/sync_symbols.py",
"merge_gitignore": "Merge setup-host .gitignore rules",
"merge_gitignore": "Merge .gitignore rules (un-ignore generated/)",
"emit_mods_preloaded": "Stub mods/preloaded catalog",
"relocate_boxart": "Relocate boxart → launcher_assets/",
"emit_boxart_stub": "Create launcher_assets stub dir",
"ensure_app_icon": "Install assets/psxrecomp app icon",
"rewrite_cmake_setup_host": "Rewrite CMakeLists.txt (setup-host)",
"emit_packager": "Emit scripts/package_setup_release.sh",
"rewrite_cmake_setup_host": "Rewrite CMakeLists.txt (psxrecomp_add_game_runtime)",
"emit_packager": "Emit scripts/package_release.sh (bundled release)",
"sync_packager_project_dirs": "Stage src/ + mods/ the release zip is missing",
"emit_ci_workflow": "Emit setup-host release.yml",
"annotate_legacy_packaging": "Annotate legacy prebuilt packaging",
"emit_ci_workflow": "Emit bundled release.yml",
"annotate_legacy_packaging": "Annotate hand-forked packaging scripts",
"probe_disc_refresh": "Refresh disc identity via probe_disc.py",
"record_framework_pins": "Write framework_pins.txt",
"patch_readme_metrics": "Patch README badges, Retro Launcher, and R.A.I.D. footer",
@@ -101,7 +101,7 @@ def build_plan(
if options.enable_ci:
wanted.add("emit_ci_workflow")
# Always ensure wizard/codegen for setup-host policy when rewriting
# Always ensure wizard/codegen (first-run disc picker) when rewriting
if "rewrite_cmake_setup_host" in wanted:
wanted.add("emit_codegen_setup")
+16 -7
View File
@@ -196,7 +196,7 @@ if (-not $useRecompUi) {
}
Write-Host " (recomp-ui declined -- skipping wizard/netplay; PSX_RECOMP_UI=OFF)"
} else {
# Default ON (interactive + non-interactive): setup-host CI needs the wizard.
# Default ON (interactive + non-interactive): the wizard is the first-run disc picker.
$useWizard = Resolve-BoolOpt -Enable:$EnableWizard -No:$NoWizard -PromptDefault:$true `
-Question "Enable first-run setup wizard + Generate & rebuild?"
if (-not $interactive -and -not $NoWizard) { $useWizard = $true }
@@ -230,7 +230,7 @@ $useCi = Resolve-BoolOpt -Enable:$EnableCi -No:$NoCi -PromptDefault:$true `
$doBoxart = Resolve-BoolOpt -Enable:$FetchBoxart -No:$NoFetchBoxart -PromptDefault:$true `
-Question "Fetch libretro boxart now? (needs network)"
$doGenerate = Resolve-BoolOpt -Enable:$Generate -No:$NoGenerate -PromptDefault:$false `
-Question "Run Generate now (emitters + OpenBIOS + game C)?"
-Question "Run Generate now (emitters + OpenBIOS + game C; generated/ is committed for release CI)?"
$doBuild = $false
if ($doGenerate) {
@@ -476,11 +476,11 @@ if ($useNetplay -and -not (Test-Path "psxrecomp\lib\recomp-net\CMakeLists.txt"))
}
Write-Host "== Packager stub =="
Fill-Template (Join-Path $TemplateDir "package_setup_release.sh.in") (Join-Path $Root "scripts\package_setup_release.sh")
Fill-Template (Join-Path $TemplateDir "package_release.sh.in") (Join-Path $Root "scripts\package_release.sh")
if ($useCi) {
Write-Host "== CI release workflow =="
$WfSrc = Join-Path $Root "psxrecomp\docs\ci\templates\setup-release.yml"
$WfSrc = Join-Path $Root "psxrecomp\docs\ci\templates\game-release.yml"
$WfDst = Join-Path $Root ".github\workflows\release.yml"
if (Test-Path -LiteralPath $WfSrc) {
New-Item -ItemType Directory -Force -Path (Split-Path -Parent $WfDst) | Out-Null
@@ -731,7 +731,15 @@ if ($doGenerate) {
}
python @genArgs
$GeneratedOk = $true
Write-Host " generate OK (generated/ is gitignored -- not committed)"
# Release CI builds the compiled game from this tree, so it is part
# of the repo, not a local by-product.
git add generated 2>$null
git -c user.email=setup@localhost -c user.name=setup commit -q -m "Add generated game C" 2>$null
if ($LASTEXITCODE -eq 0) {
Write-Host " generate OK (generated/ committed -- release CI builds from it)"
} else {
Write-Host " generate OK -- commit generated/ before tagging a release: git add generated"
}
} catch {
Write-Warning "Generate failed -- fix seeds/disc and re-run generate by hand."
$doBuild = $false
@@ -845,16 +853,17 @@ Build playable runtime:
"@
} else {
@"
Build emitters, Generate, then playable runtime:
Build emitters, Generate (and commit it -- release CI builds from generated/), then playable runtime:
bash psxrecomp/tools/ci/build_emitters.sh
python psxrecomp\psxrecomp_cli.py generate --config game.toml --project-root . --disc $GenDiscHint
git add generated; git commit -m "Add generated game C"
cmake -S . -B build-release -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build build-release --target psx-runtime
"@
}
$CiNote = if ($useCi) {
$n = "CI: .github/workflows/release.yml ready (zip prefix=$ZipPrefix; submodule gitlinks pin the build)."
$n = "CI: .github/workflows/release.yml ready (zip prefix=$ZipPrefix; builds the committed generated/ C into the shipped game; submodule gitlinks pin the build)."
if ($githubPushed) { $n += " Pushed -- open Actions -> Release builds (workflow_dispatch)." }
$n
} else {
+19 -18
View File
@@ -342,8 +342,8 @@ if [ "$ENABLE_RECOMP_UI" -eq 0 ]; then
echo " (recomp-ui declined — skipping wizard/netplay; PSX_RECOMP_UI=OFF)"
else
if [ "$SET_WIZARD" -eq 0 ]; then
# Default ON: setup-host CI zips need the wizard (FORCE_SETUP_HOST
# without PSX_SETUP_WIZARD opens no first-run UI).
# Default ON: the wizard is the first-run disc picker of the shipped
# (compiled) game.
if [ "$YES_MODE" -eq 1 ] || ! is_tty; then
ENABLE_WIZARD=1
else
@@ -388,14 +388,6 @@ if [ "$SET_CI" -eq 0 ]; then
fi
fi
# Setup-host release CI needs PSX_SETUP_WIZARD (FORCE_SETUP_HOST alone is not enough).
if [ "${ENABLE_CI:-0}" -eq 1 ] && [ "${ENABLE_RECOMP_UI:-0}" -eq 1 ] &&
[ "${ENABLE_WIZARD:-0}" -eq 0 ]; then
echo "error: release CI requires the first-run setup wizard." >&2
echo " Pass --enable-wizard (or omit --no-wizard)." >&2
exit 1
fi
if [ "$SET_BOXART" -eq 0 ]; then
if [ "$YES_MODE" -eq 1 ] || ! is_tty; then
FETCH_BOXART_FLAG=0
@@ -408,7 +400,7 @@ if [ "$SET_GENERATE" -eq 0 ]; then
if [ "$YES_MODE" -eq 1 ] || ! is_tty; then
DO_GENERATE=0
else
prompt_yn "Run Generate now (emitters + OpenBIOS + game C)?" DO_GENERATE 0
prompt_yn "Run Generate now (emitters + OpenBIOS + game C; generated/ is committed for release CI)?" DO_GENERATE 0
fi
fi
@@ -706,14 +698,14 @@ if [ "$ENABLE_NETPLAY" -eq 1 ] && [ ! -f psxrecomp/lib/recomp-net/CMakeLists.txt
fi
echo "== Packager stub =="
fill_template "$TEMPLATE_DIR/package_setup_release.sh.in" \
"$ROOT/scripts/package_setup_release.sh"
chmod +x "$ROOT/scripts/package_setup_release.sh"
fill_template "$TEMPLATE_DIR/package_release.sh.in" \
"$ROOT/scripts/package_release.sh"
chmod +x "$ROOT/scripts/package_release.sh"
CI_WORKFLOW_OK=0
if [ "$ENABLE_CI" -eq 1 ]; then
echo "== CI release workflow =="
WF_SRC="$ROOT/psxrecomp/docs/ci/templates/setup-release.yml"
WF_SRC="$ROOT/psxrecomp/docs/ci/templates/game-release.yml"
if [ -f "$WF_SRC" ]; then
mkdir -p "$ROOT/.github/workflows"
python3 "$FILL_TOKENS" "$WF_SRC" "$ROOT/.github/workflows/release.yml" \
@@ -964,7 +956,15 @@ if [ "$DO_GENERATE" -eq 1 ]; then
python3 psxrecomp/psxrecomp_cli.py generate "$@"
) && GENERATED_OK=1
if [ "$GENERATED_OK" -eq 1 ]; then
echo " generate OK (generated/ is gitignored — not committed)"
# Release CI builds the compiled game from this tree, so it is part
# of the repo, not a local by-product.
if git add generated 2>/dev/null && \
git -c user.email=setup@localhost -c user.name=setup \
commit -q -m "Add generated game C" 2>/dev/null; then
echo " generate OK (generated/ committed — release CI builds from it)"
else
echo " generate OK — commit generated/ before tagging a release: git add generated"
fi
else
echo "warning: generate failed — fix seeds/disc and re-run generate by hand." >&2
DO_BUILD=0
@@ -1086,17 +1086,18 @@ elif [ "$GENERATED_OK" -eq 1 ]; then
$BUILD_HINT"
else
STEP2="Build emitters, Generate, then playable runtime:
STEP2="Build emitters, Generate (and commit it — release CI builds from generated/), then playable runtime:
./psxrecomp/tools/ci/build_emitters.sh
python3 psxrecomp/psxrecomp_cli.py generate \\
--config game.toml --project-root . --disc \"$GEN_DISC_HINT\"
git add generated && git commit -m 'Add generated game C'
$BUILD_HINT"
fi
CI_NOTE="CI workflow not installed (declined)."
if [ "$ENABLE_CI" -eq 1 ] && [ "$CI_WORKFLOW_OK" -eq 1 ]; then
CI_NOTE="CI: .github/workflows/release.yml ready (zip prefix=$ZIP_PREFIX; submodule gitlinks pin the build)."
CI_NOTE="CI: .github/workflows/release.yml ready (zip prefix=$ZIP_PREFIX; builds the committed generated/ C into the shipped game; submodule gitlinks pin the build)."
if [ "$GITHUB_PUSHED" -eq 1 ]; then
CI_NOTE="$CI_NOTE Pushed — open Actions → Release builds (workflow_dispatch)."
fi
@@ -13,8 +13,9 @@ project(@PROJECT_CMAKE_NAME@ LANGUAGES C CXX)
# --- Opt-in product surfaces (OFF by default in the framework) -------------
# Scaffold prompts / flags fill these. Without recomp-ui, PSX_RECOMP_UI is OFF.
# Setup-host CI requires WIZARD_CMAKE_BLOCK / ENABLE_SETUP_WIZARD (FORCE_SETUP_HOST
# alone does not open first-run / Generate & rebuild).
# The wizard (WIZARD_CMAKE_BLOCK / ENABLE_SETUP_WIZARD) is the first-run disc
# picker; releases are compiled from the committed generated/ C, so it never
# needs to generate or rebuild anything on a player's machine.
@RECOMP_UI_CMAKE_BLOCK@
@NETPLAY_CMAKE_BLOCK@
@WIZARD_CMAKE_BLOCK@
@@ -40,6 +41,9 @@ psxrecomp_add_game_runtime(psx-runtime
MAX_PLAYERS @PLAYERS@
@BOXART_CMAKE_ARG@
@APP_ICON_CMAKE_ARG@
# Committed game C (psxrecomp_cli.py generate; git add generated). Release
# CI configures with -DPSXRECOMP_REQUIRE_GAME_C=ON so a checkout without
# it fails instead of building a setup host.
GEN_MARKER "generated/@BOOT_EXE@_dispatch.c"
GEN_FULL_GLOB "generated/@BOOT_EXE@_full_*.c"
@CODEGEN_SETUP_ARG@
@@ -25,9 +25,8 @@ Scaffolded with the New Project Layout. See
<!-- retcomm-readme-launcher -->
## Retro Launcher
You can run this title **standalone** (release zip + the built-in recomp-ui
Generate & Build flow), or manage installs, updates, ROM/BIOS wiring, and queued
builds more intuitively with
You can run this title **standalone** (download the release zip, point it at
your disc, play), or manage installs, updates, and disc/BIOS wiring with
**[Retro Launcher](https://github.com/RetroPortingToolKit/Retro-Launcher)** —
the Retro Compilation Manager hub for self-compiling recomps.
@@ -42,16 +41,19 @@ the Retro Compilation Manager hub for self-compiling recomps.
<img src="https://raw.githubusercontent.com/RetroPortingToolKit/Retro-Launcher/main/docs/screenshots/queue-and-background-build.png" alt="Background cmake build with titles queued" width="720">
</p>
Retro checks for updates, rebuilds with existing build data when possible,
shares the portable toolchain used by per-title launchers, and automates
BIOS/ROM/save plumbing so you are not stuck repeating each game’s wizard by hand.
Retro checks for updates, installs the prebuilt release zips, and automates
BIOS/ROM/save plumbing so you are not stuck repeating each game’s first run by hand.
<!-- /retcomm-readme-launcher -->
## Legal
You must own the original game. Disc images under `disc/` are gitignored and
must never be committed. Retail BIOS dumps are not redistributed; OpenBIOS is
used for Generate unless you supply your own SCPH locally.
must never be committed. Retail BIOS dumps are not redistributed and no C
derived from one may be committed; releases run on the bundled MIT OpenBIOS.
`generated/` (the recompiled game C) **is committed**: releases ship the
compiled game, built by CI from that tree. Regenerate and commit it whenever
seeds or the framework pin change.
Default app icon: `assets/psxrecomp.ico` (and `.png` / `.svg`) — Retro-themed controller mark from `psxrecomp/assets/`. Windows builds embed it via `APP_ICON`.
@@ -66,11 +68,15 @@ git submodule update --init --recursive
./psxrecomp/tools/ci/build_emitters.sh
python3 psxrecomp/psxrecomp_cli.py generate \
--config game.toml --project-root . --disc disc/<your>.cue
git add generated && git commit -m "Regenerate game C"
cmake -S . -B build-release -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build build-release --target psx-runtime
```
Zip prefix for CI artifacts: `@ZIP_PREFIX@`.
Releases: tag `vX.Y.Z` (or run the *Release builds* workflow). CI builds the
committed `generated/` C on Linux, Windows and macOS and attaches
`@ZIP_PREFIX@-<version>-<platform>.zip`, the compiled game. Locally:
`scripts/package_release.sh build-release linux-x64`.
## Symbols
@@ -7,8 +7,10 @@
/out/
/.happypath/
# Setup-host: players Generate game C locally (CI clears generated/).
/generated/
# generated/ is COMMITTED: releases ship the compiled game, and CI builds it
# from the game C in this repo (psxrecomp_cli.py generate, then git add
# generated). Never ignore it. Only the framework submodule's own generated/
# (BIOS backends) stays untracked, and that is the submodule's rule.
# psxrecomp_cli.py work areas (overlay toolchain download, static overlay
# shard evidence -- the latter holds bytes read from the disc).
/.cache/
@@ -0,0 +1,62 @@
#!/usr/bin/env bash
# Thin wrapper around the shared psxrecomp bundled-release packager.
# Autofilled by tools/new_project_layout/setup_project.{sh,ps1}.
#
# Ships the COMPILED game built from this repo's committed generated/ C:
# executable, runtime data, bundled OpenBIOS, mod catalog, overlay toolchain.
# No sources, emitters, CLI, generated C, or BIOS dumps.
#
# Usage:
# scripts/package_release.sh <build-dir> <artifact-tag> [recompiler-build-dir]
#
# Writes: dist/@ZIP_PREFIX@-<VERSION>-<artifact-tag>.zip
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
BUILD_DIR="${1:-}"
ARTIFACT_TAG="${2:-}"
RECOMPILER_BUILD="${3:-build-recompiler}"
if [[ -z "${BUILD_DIR}" || -z "${ARTIFACT_TAG}" ]]; then
echo "usage: $0 <build-dir> <artifact-tag> [recompiler-build-dir]" >&2
exit 2
fi
PACKAGER="${ROOT}/psxrecomp/tools/package_game_release.sh"
if [[ ! -f "${PACKAGER}" ]]; then
echo "error: missing ${PACKAGER} (psxrecomp submodule predates bundled releases -- bump it)" >&2
exit 1
fi
chmod +x "${PACKAGER}" 2>/dev/null || true
EXTRA=()
# Overlay shard cache. It is compiled FROM THE DISC, so CI (no disc) cannot
# build one: the committed static AOT shard is linked into the executable and
# the bundled overlay_toolchain/ compiles the rest from the player's disc at
# runtime. A developer packaging locally with a cache can ship it instead:
# PSX_OVERLAY_CACHE_ROOT=/path/to/cache scripts/package_release.sh ...
if [[ -n "${PSX_OVERLAY_CACHE_ROOT:-}" ]]; then
EXTRA+=(--overlay-cache-root "${PSX_OVERLAY_CACHE_ROOT}")
else
EXTRA+=(--ship-without-overlay-cache-because \
"no disc at package time: the committed static AOT shard is compiled in and overlay_toolchain/ autocompiles the rest on the player's machine")
fi
# Extra docs shipped at the zip root (DISC.md tells players which dump works).
for _doc in DISC.md; do
if [[ -f "${ROOT}/${_doc}" ]]; then
EXTRA+=(--doc "${_doc}")
fi
done
cd "${ROOT}"
exec bash "${PACKAGER}" \
--root "${ROOT}" \
--build-dir "${BUILD_DIR}" \
--artifact "${ARTIFACT_TAG}" \
--zip-prefix @ZIP_PREFIX@ \
--exe-name @EXE_BASENAME@ \
--display-name "@WINDOW_TITLE@" \
--recompiler-build "${RECOMPILER_BUILD}" \
--version-env RELEASE_VERSION \
--disc-hint "@DISC_HINT@" \
"${EXTRA[@]}"
@@ -1,94 +0,0 @@
#!/usr/bin/env bash
# Thin wrapper around the shared psxrecomp setup-host packager.
# Autofilled by tools/new_project_layout/setup_project.{sh,ps1}.
#
# Usage:
# scripts/package_setup_release.sh <build-dir> <artifact-tag> [recompiler-build-dir]
#
# Writes: dist/@ZIP_PREFIX@-<VERSION>-<artifact-tag>.zip
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
BUILD_DIR="${1:-}"
ARTIFACT_TAG="${2:-}"
RECOMPILER_BUILD="${3:-build-recompiler}"
if [[ -z "${BUILD_DIR}" || -z "${ARTIFACT_TAG}" ]]; then
echo "usage: $0 <build-dir> <artifact-tag> [recompiler-build-dir]" >&2
exit 2
fi
# mods/ is staged by the framework packager by default (catalog next to the
# exe, plus the source catalog for the setup-host rebuild) -- no flag needed.
# --project-dir src ships game-owned trusted-plugin sources; CMakeLists.txt
# references them unconditionally, so the rebuild cannot configure without it.
# copy_proj skips a path that does not exist, so it is inert for titles with
# no src/.
PACKAGER="${ROOT}/psxrecomp/tools/package_setup_host.sh"
if [[ ! -f "${PACKAGER}" ]]; then
echo "error: missing ${PACKAGER} (psxrecomp submodule)" >&2
exit 1
fi
chmod +x "${PACKAGER}" 2>/dev/null || true
EXTRA_PROJECT=()
if [[ -f "${ROOT}/catalog_identity.json" ]]; then
EXTRA_PROJECT+=(--project-file catalog_identity.json)
fi
if [[ -f "${ROOT}/framework_pins.txt" ]]; then
EXTRA_PROJECT+=(--project-file framework_pins.txt)
fi
# Optional in-game options defaults (titles that POST_BUILD-copy this file).
if [[ -f "${ROOT}/game_options.toml" ]]; then
EXTRA_PROJECT+=(--project-file game_options.toml)
fi
# Game-owned C that CMakeLists.txt compiles into the runtime — mod activation
# plugins live here (CODEGEN_SETUP_SOURCES "src/<game>_mods.c"). Leaving this
# out ships a CMakeLists that references a file the zip does not contain, so
# the build dies at configure with "Cannot find source file".
if [[ -d "${ROOT}/src" ]]; then
EXTRA_PROJECT+=(--project-dir src)
fi
# Preloaded mod packages (mods/preloaded/packages/<id>). The activation plugin
# above is compiled in, but selects itself through these marker files; without
# them the mod is built and never enabled.
if [[ -d "${ROOT}/mods" ]]; then
EXTRA_PROJECT+=(--project-dir mods)
fi
# Desktop/app icon sources. CMakeLists.txt passes APP_ICON
# "${CMAKE_CURRENT_SOURCE_DIR}/assets/psxrecomp.ico" unguarded, so a zip
# without assets/ cannot configure. runtime.cmake also stages the sibling
# psxrecomp.png from this directory as the SDL window icon, so the rebuilt exe
# would otherwise come out unbranded even if configure had survived.
if [[ -d "${ROOT}/assets" ]]; then
EXTRA_PROJECT+=(--project-dir assets)
fi
# Generated symbol headers. Game-owned C in src/ includes these by relative
# path ("../psx_symbols.h"), so they are compile inputs of the shipped tree even
# though nothing in CMakeLists.txt names them.
for _sym in psx_symbols.h psx_symbols_overlays.h; do
if [[ -f "${ROOT}/${_sym}" ]]; then
EXTRA_PROJECT+=(--project-file "${_sym}")
fi
done
cd "${ROOT}"
exec bash "${PACKAGER}" \
--root "${ROOT}" \
--build-dir "${BUILD_DIR}" \
--artifact "${ARTIFACT_TAG}" \
--zip-prefix @ZIP_PREFIX@ \
--exe-name @EXE_BASENAME@ \
--display-name "@WINDOW_TITLE@" \
--recompiler-build "${RECOMPILER_BUILD}" \
--version-env RELEASE_VERSION \
--disc-hint "@DISC_HINT@" \
--project-file CMakeLists.txt \
--project-file game.toml \
--project-file VERSION \
--project-file codegen_setup.c \
--project-file codegen_setup.h \
--project-file README.md \
--project-dir seeds \
--project-dir launcher_assets \
"${EXTRA_PROJECT[@]}"
+466
View File
@@ -0,0 +1,466 @@
#!/usr/bin/env bash
# Bundled-release zip packager for PSXRecomp game repos: ships the COMPILED
# game, not a kit that compiles it.
#
# A release built from the title's committed generated/ C is a finished
# product. The zip carries the game executable and everything the runtime
# reads beside it -- assets, the mod catalog, the bundled OpenBIOS image and
# its notice, game.toml / game_options.toml, third-party notices -- plus the
# overlay toolchain so overlays outside the compiled static shard can still be
# turned native on the player's machine. It carries NO sources, NO emitters at
# the root, NO psxrecomp/ or recomp-ui/ tree, NO CLI, NO generated C and NO
# BIOS dump. The player supplies a legal disc and plays.
#
# Usage (from game repo root):
# psxrecomp/tools/package_game_release.sh \
# --build-dir build-ci \
# --artifact linux-x64 \
# --zip-prefix bpe \
# --exe-name Bomberman_Party_Edition_Recompiled \
# --display-name "Bomberman Party Edition Recompiled" \
# --recompiler-build build-recompiler \
# [--runtime-file REL]... # extra exe-relative files (keybinds.ini, ...)
# [--runtime-dir NAME]... # extra exe-relative dirs (bezels, ...)
# [--doc REL]... # extra docs from the repo (DISC.md, ...)
# [--overlay-cache-root DIR] # ship a prebuilt overlay shard cache
# [--ship-without-overlay-cache-because REASON]
# [--no-overlay-toolchain] # do not bundle overlay_toolchain/
# [--disc-hint "your legally owned disc"] [--version-env RELEASE_VERSION]
#
# Overlay cache. The runtime's overlay shard cache is compiled FROM THE DISC,
# so a CI job -- which has no disc -- cannot build one; the committed static
# AOT shard (generated/overlays_static*.c) is compiled into the executable
# instead, and the bundled overlay_toolchain/ compiles the rest from the
# player's own disc at runtime. A developer packaging locally with a cache
# passes --overlay-cache-root and it is staged through release_stage.py
# (which owns the cache tag and layout; nothing here knows either). Without a
# cache the caller must say why, in words, and the reason is printed: this is
# the same deliberately unwieldy switch release_stage.py exposes, because a
# packager that quietly staged nothing once looked like a good release twice.
#
# DEVELOPER CHANNEL. mods with channel = "developer" are pruned from public
# releases (EXCLUDE_DEV_MODS=1, which CI sets via $CI) and kept in local
# exports; --include-dev-mods / --exclude-dev-mods override. Pruning is by
# package directory via tools/mod_channel_filter.py, never by editing a
# manifest.
#
# Env:
# RELEASE_VERSION / <version-env> / VERSION file (must match binary stamp)
# PSXRECOMP_RUNTIME_BIN_DIR | BPE_RUNTIME_BIN_DIR (Windows MinGW DLL search)
# PSX_RELEASE_STAGE_PYTHON (python3 override)
#
# Lobby pin: the exe must have been built with current runtime.cmake so
# $<TARGET_FILE_DIR>/psx_game_version.txt exists. This script refuses to ship
# a VERSION that disagrees with that stamp (netplay list filter bug).
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
FRAMEWORK="$(cd "${SCRIPT_DIR}/.." && pwd)"
ROOT="$(pwd)"
BUILD_DIR=""
ARTIFACT=""
ZIP_PREFIX=""
EXE_NAME=""
DISPLAY_NAME=""
RECOMPILER_BUILD="build-recompiler"
VERSION_ENV="RELEASE_VERSION"
DISC_HINT="your legally owned game disc"
RUNTIME_TARGET="psx-runtime"
RUNTIME_FILES=()
RUNTIME_DIRS=()
DOCS=()
OVERLAY_CACHE_ROOT=""
SHIP_WITHOUT_OVERLAY_CACHE_BECAUSE=""
SHIP_WITHOUT_OVERLAY_CACHE_KEY_BECAUSE=""
STAGE_OVERLAY_TOOLCHAIN=1
if [[ -z "${EXCLUDE_DEV_MODS:-}" ]]; then
if [[ -n "${CI:-}" ]]; then EXCLUDE_DEV_MODS=1; else EXCLUDE_DEV_MODS=0; fi
fi
RUNTIME_BIN_DIR="${PSXRECOMP_RUNTIME_BIN_DIR:-${BPE_RUNTIME_BIN_DIR:-/usr/x86_64-w64-mingw32/bin}}"
usage() {
sed -n '2,32p' "$0" | sed 's/^# \{0,1\}//'
exit "${1:-2}"
}
while [[ $# -gt 0 ]]; do
case "$1" in
-h|--help) usage 0 ;;
--build-dir) BUILD_DIR="${2:?}"; shift 2 ;;
--artifact) ARTIFACT="${2:?}"; shift 2 ;;
--zip-prefix) ZIP_PREFIX="${2:?}"; shift 2 ;;
--exe-name) EXE_NAME="${2:?}"; shift 2 ;;
--display-name) DISPLAY_NAME="${2:?}"; shift 2 ;;
--recompiler-build) RECOMPILER_BUILD="${2:?}"; shift 2 ;;
--runtime-target) RUNTIME_TARGET="${2:?}"; shift 2 ;;
--version-env) VERSION_ENV="${2:?}"; shift 2 ;;
--disc-hint) DISC_HINT="${2:?}"; shift 2 ;;
--runtime-file) RUNTIME_FILES+=("${2:?}"); shift 2 ;;
--runtime-dir) RUNTIME_DIRS+=("${2:?}"); shift 2 ;;
--doc) DOCS+=("${2:?}"); shift 2 ;;
--overlay-cache-root) OVERLAY_CACHE_ROOT="${2:?}"; shift 2 ;;
--ship-without-overlay-cache-because) SHIP_WITHOUT_OVERLAY_CACHE_BECAUSE="${2:?}"; shift 2 ;;
--ship-without-overlay-cache-key-because) SHIP_WITHOUT_OVERLAY_CACHE_KEY_BECAUSE="${2:?}"; shift 2 ;;
--no-overlay-toolchain) STAGE_OVERLAY_TOOLCHAIN=0; shift ;;
--exclude-dev-mods) EXCLUDE_DEV_MODS=1; shift ;;
--include-dev-mods) EXCLUDE_DEV_MODS=0; shift ;;
--runtime-bin) RUNTIME_BIN_DIR="${2:?}"; shift 2 ;;
--root) ROOT="${2:?}"; shift 2 ;;
*)
echo "error: unknown arg: $1" >&2
usage 2
;;
esac
done
if [[ -z "${BUILD_DIR}" || -z "${ARTIFACT}" || -z "${ZIP_PREFIX}" || -z "${EXE_NAME}" ]]; then
echo "error: --build-dir, --artifact, --zip-prefix, and --exe-name are required" >&2
usage 2
fi
[[ -n "${DISPLAY_NAME}" ]] || DISPLAY_NAME="${EXE_NAME}"
ROOT="$(cd "${ROOT}" && pwd)"
if [[ -d "${ROOT}/${BUILD_DIR}" ]]; then
BUILD_DIR="$(cd "${ROOT}/${BUILD_DIR}" && pwd)"
elif [[ -d "${BUILD_DIR}" ]]; then
BUILD_DIR="$(cd "${BUILD_DIR}" && pwd)"
else
echo "error: build dir not found: ${BUILD_DIR}" >&2
exit 1
fi
if [[ -d "${ROOT}/${RECOMPILER_BUILD}" ]]; then
RECOMPILER_BUILD="$(cd "${ROOT}/${RECOMPILER_BUILD}" && pwd)"
elif [[ -d "${RECOMPILER_BUILD}" ]]; then
RECOMPILER_BUILD="$(cd "${RECOMPILER_BUILD}" && pwd)"
fi
# The shared staging surface: cache tag / shards / toolchain / mod catalog are
# release_stage.py's, and only its. This script marshals arguments.
# shellcheck source=release_overlay_stage.sh
. "${SCRIPT_DIR}/release_overlay_stage.sh"
psx_release_stage_init "${FRAMEWORK}"
PY="${_psx_py}"
# --- Version: the compile-time stamp is authoritative ------------------------
REQUESTED=""
if [[ -n "${VERSION_ENV}" ]]; then REQUESTED="${!VERSION_ENV:-}"; fi
if [[ -z "${REQUESTED}" && -n "${RELEASE_VERSION:-}" ]]; then REQUESTED="${RELEASE_VERSION}"; fi
if [[ -z "${REQUESTED}" && -f "${ROOT}/VERSION" ]]; then REQUESTED="$(tr -d '[:space:]' <"${ROOT}/VERSION")"; fi
REQUESTED="$(printf '%s' "${REQUESTED}" | tr -d '[:space:]')"
REQUESTED="${REQUESTED#v}"
EXE=""
for cand in "${BUILD_DIR}/${EXE_NAME}" "${BUILD_DIR}/${EXE_NAME}.exe" "${BUILD_DIR}/Release/${EXE_NAME}.exe"; do
if [[ -f "${cand}" ]]; then EXE="${cand}"; break; fi
done
if [[ -z "${EXE}" ]]; then
# CMake records the OUTPUT_NAME it chose; prefer it over the wrapper's copy
# so a renamed title does not read as "binary missing".
marker="${BUILD_DIR}/psxrecomp_exe_name-${RUNTIME_TARGET}.txt"
if [[ -f "${marker}" ]]; then
chosen="$(tr -d '[:space:]' <"${marker}")"
for cand in "${BUILD_DIR}/${chosen}" "${BUILD_DIR}/${chosen}.exe"; do
if [[ -f "${cand}" ]]; then EXE="${cand}"; break; fi
done
if [[ -n "${EXE}" ]]; then
echo "note: --exe-name ${EXE_NAME} does not match CMake OUTPUT_NAME ${chosen}; using the marker" >&2
EXE_NAME="${chosen}"
fi
fi
fi
if [[ -z "${EXE}" ]]; then
echo "error: game executable '${EXE_NAME}' not found under ${BUILD_DIR}" >&2
ls -la "${BUILD_DIR}" >&2 || true
exit 1
fi
EXE_BASENAME="$(basename "${EXE}")"
EXE_DIR="$(dirname "${EXE}")"
# A bundled release is a FULL build. The build publishes which it is: the
# PSX_HAS_GAME_DISPATCH define only exists when generated C was linked.
# Refuse to package a setup host as a game.
if [[ -f "${BUILD_DIR}/CMakeCache.txt" ]] &&
grep -q '^PSXRECOMP_FORCE_SETUP_HOST:BOOL=ON' "${BUILD_DIR}/CMakeCache.txt"; then
echo "error: ${BUILD_DIR} was configured with PSXRECOMP_FORCE_SETUP_HOST=ON -- that is a setup host, not a game." >&2
echo " Reconfigure with -DPSXRECOMP_REQUIRE_GAME_C=ON (generated/ committed) and rebuild." >&2
exit 1
fi
if ! grep -qs 'PSX_HAS_GAME_DISPATCH' "${BUILD_DIR}/build.ninja" "${BUILD_DIR}/compile_commands.json" \
"${BUILD_DIR}/CMakeFiles/${RUNTIME_TARGET}.dir/flags.make" 2>/dev/null; then
echo "error: ${BUILD_DIR} did not compile generated game C (no PSX_HAS_GAME_DISPATCH in the build flags)." >&2
echo " A bundled release links the committed generated/ tree; configure with -DPSXRECOMP_REQUIRE_GAME_C=ON." >&2
exit 1
fi
normalize_ver() { local v; v="$(printf '%s' "${1:-}" | tr -d '[:space:]')"; printf '%s' "${v#v}"; }
BUILT=""
for stamp in "${EXE_DIR}/psx_game_version.txt" "${BUILD_DIR}/psx_game_version.txt" "${BUILD_DIR}/Release/psx_game_version.txt"; do
if [[ -f "${stamp}" ]]; then
BUILT="$(normalize_ver "$(cat "${stamp}")")"
echo "lobby pin stamp: ${stamp} -> ${BUILT}"
break
fi
done
if [[ -z "${BUILT}" ]]; then
echo "error: missing psx_game_version.txt next to ${EXE}" >&2
echo " Rebuild with current psxrecomp runtime.cmake so the lobby pin is stamped." >&2
exit 1
fi
if [[ -n "${REQUESTED}" && "${REQUESTED}" != "${BUILT}" ]]; then
echo "error: RELEASE_VERSION/VERSION=${REQUESTED} but binary stamp=${BUILT}" >&2
echo " Rebuild with -DPSX_GAME_VERSION=${REQUESTED} (or pin VERSION then reconfigure)." >&2
exit 1
fi
VERSION="${BUILT}"
printf '%s\n' "${VERSION}" >"${ROOT}/VERSION"
DIST="${ROOT:?}/dist"
STAGE="${DIST:?}/stage-game-${ARTIFACT:?}"
ZIP_NAME="${ZIP_PREFIX:?}-${VERSION:?}-${ARTIFACT:?}.zip"
rm -rf "${STAGE:?}"
mkdir -p "${STAGE}" "${DIST}"
rm -f "${DIST:?}/${ZIP_NAME:?}"
# --- The executable and what CMake staged beside it --------------------------
cp -a "${EXE}" "${STAGE}/"
if [[ -f "${EXE_DIR}/psx_game_version.txt" ]]; then
cp -a "${EXE_DIR}/psx_game_version.txt" "${STAGE}/psx_game_version.txt"
else
printf '%s\n' "${VERSION}" >"${STAGE}/psx_game_version.txt"
fi
if [[ "${EXE_BASENAME}" == *.exe ]]; then
shopt -s nullglob
for dll in "${EXE_DIR}"/*.dll "${EXE_DIR}"/*.DLL; do
cp -a "${dll}" "${STAGE}/"
echo "staged sibling DLL $(basename "${dll}")"
done
shopt -u nullglob
fi
if [[ ! -d "${EXE_DIR}/assets/fonts" || ! -d "${EXE_DIR}/assets/img" ]]; then
echo "error: ${EXE_DIR}/assets/{fonts,img} missing -- rebuild ${RUNTIME_TARGET}" >&2
exit 1
fi
mkdir -p "${STAGE}/assets"
cp -a "${EXE_DIR}/assets/fonts" "${STAGE}/assets/"
cp -a "${EXE_DIR}/assets/img" "${STAGE}/assets/"
if [[ ! -f "${STAGE}/assets/img/boxart.tga" && -f "${ROOT}/launcher_assets/img/boxart.tga" ]]; then
cp -a "${ROOT}/launcher_assets/img/boxart.tga" "${STAGE}/assets/img/boxart.tga"
fi
# Window icon staged by runtime.cmake beside the exe (see APP_ICON).
shopt -s nullglob
for icon in "${EXE_DIR}"/psxrecomp.png "${EXE_DIR}"/*.png; do
cp -a "${icon}" "${STAGE}/"
break
done
shopt -u nullglob
# Bundled OpenBIOS: runtime.cmake stages the exact image the compiled backend
# consumed plus its MIT notice at <exe>/bios/. The runtime resolves
# "bios/openbios.bin" relative to the executable, so ship the directory as a
# unit. Nothing else from bios/ may ship.
if [[ ! -f "${EXE_DIR}/bios/openbios.bin" || ! -f "${EXE_DIR}/bios/OpenBIOS.LICENSE" ]]; then
echo "error: ${EXE_DIR}/bios/{openbios.bin,OpenBIOS.LICENSE} missing -- the build did not link the OpenBIOS backend" >&2
exit 1
fi
mkdir -p "${STAGE}/bios"
cp -a "${EXE_DIR}/bios/openbios.bin" "${EXE_DIR}/bios/OpenBIOS.LICENSE" "${STAGE}/bios/"
# Runtime configs: game.toml is what the runtime reads beside the exe; the
# build stages it there, so take the staged copy (it is the packaged config
# the cache tag derives from). game_options.toml is optional.
for cfg in game.toml game_options.toml; do
if [[ -f "${EXE_DIR}/${cfg}" ]]; then
cp -a "${EXE_DIR}/${cfg}" "${STAGE}/${cfg}"
elif [[ -f "${ROOT}/${cfg}" ]]; then
cp -a "${ROOT}/${cfg}" "${STAGE}/${cfg}"
fi
done
[[ -f "${STAGE}/game.toml" ]] || { echo "error: no game.toml to ship (build tree or repo root)" >&2; exit 1; }
# Wave-5 F4: without overlay_cache = true the runtime never initialises the
# overlay loader and every streamed overlay runs interpreted for every player.
if ! grep -qE '^[[:space:]]*overlay_cache[[:space:]]*=[[:space:]]*true' "${STAGE}/game.toml"; then
if [[ -z "${SHIP_WITHOUT_OVERLAY_CACHE_KEY_BECAUSE}" ]]; then
echo "error: REFUSING TO PACKAGE: ${STAGE}/game.toml has no '[runtime] overlay_cache = true'." >&2
echo " Add the key, or pass --ship-without-overlay-cache-key-because '<reason>'." >&2
exit 1
fi
echo "warning: packaging without overlay_cache = true (reason: ${SHIP_WITHOUT_OVERLAY_CACHE_KEY_BECAUSE})" >&2
fi
for rel in "${RUNTIME_FILES[@]+"${RUNTIME_FILES[@]}"}"; do
[[ -n "${rel}" ]] || continue
src=""
for c in "${EXE_DIR}/${rel}" "${ROOT}/${rel}"; do [[ -f "${c}" ]] && { src="${c}"; break; }; done
[[ -n "${src}" ]] || { echo "error: --runtime-file ${rel} not found (build tree or repo)" >&2; exit 1; }
mkdir -p "$(dirname "${STAGE}/${rel}")"
cp -a "${src}" "${STAGE}/${rel}"
done
for name in "${RUNTIME_DIRS[@]+"${RUNTIME_DIRS[@]}"}"; do
[[ -n "${name}" ]] || continue
[[ -d "${EXE_DIR}/${name}" ]] || { echo "error: --runtime-dir ${name} missing under ${EXE_DIR}" >&2; exit 1; }
mkdir -p "${STAGE}/${name}"
cp -a "${EXE_DIR}/${name}/." "${STAGE}/${name}/"
echo "staged runtime dir ${name}/"
done
# --- Mod catalog, verified against what THIS build declared -------------------
MANIFEST="$(find "${BUILD_DIR}" -maxdepth 3 -name "psx_mod_catalog_${RUNTIME_TARGET}.txt" -type f 2>/dev/null | head -n1 || true)"
mods_args=(--build-path "${EXE_DIR}" --stage "${STAGE}" --runtime-target "${RUNTIME_TARGET}")
[[ -n "${MANIFEST}" ]] && mods_args+=(--catalog-manifest "${MANIFEST}")
psx_add_mod_catalog "${mods_args[@]}"
prune_dev_mods() {
if ! "${PY}" "${SCRIPT_DIR}/mod_channel_filter.py" "$@"; then
echo "error: developer-channel filtering failed" >&2
exit 1
fi
}
if [[ "${EXCLUDE_DEV_MODS}" -eq 1 ]]; then
echo "excluding developer-channel mods from this package"
prune_dev_mods "${STAGE}/mods/bundled"
left=$( { grep -rlE '^[[:space:]]*channel[[:space:]]*=[[:space:]]*"developer"[[:space:]]*$' \
"${STAGE}/mods" --include=manifest.toml 2>/dev/null || true; } | wc -l)
if [[ "${left}" -ne 0 ]]; then
echo "error: ${left} developer manifest(s) survived pruning" >&2
exit 1
fi
else
n=$( { grep -rlE '^[[:space:]]*channel[[:space:]]*=[[:space:]]*"developer"[[:space:]]*$' \
"${STAGE}/mods" --include=manifest.toml 2>/dev/null || true; } | wc -l)
[[ "${n}" -eq 0 ]] || echo "note: package includes ${n} developer-channel manifest(s); pass --exclude-dev-mods for a public release"
fi
# --- Overlay shard cache (developer-local only) and toolchain -----------------
GAME_BIN=""
for cand in "${RECOMPILER_BUILD}/psxrecomp-game" "${RECOMPILER_BUILD}/psxrecomp-game.exe" "${RECOMPILER_BUILD}/Release/psxrecomp-game.exe"; do
if [[ -f "${cand}" ]]; then GAME_BIN="${cand}"; break; fi
done
if [[ -n "${OVERLAY_CACHE_ROOT}" ]]; then
[[ -n "${GAME_BIN}" ]] || { echo "error: --overlay-cache-root needs psxrecomp-game under ${RECOMPILER_BUILD} (cache tag derivation)" >&2; exit 1; }
game_id="$(sed -n 's/^[[:space:]]*id[[:space:]]*=[[:space:]]*"\([^"]*\)".*/\1/p' "${STAGE}/game.toml" | head -1)"
[[ -n "${game_id}" ]] || { echo "error: [game] id missing from the staged game.toml (cache namespace)" >&2; exit 1; }
cg_tag="$(psx_overlay_cg_tag --runtime-include "${FRAMEWORK}/runtime/include" \
--recompiler "${GAME_BIN}" --game-toml "${STAGE}/game.toml" \
--flavor-from-build "${BUILD_DIR}" --runtime-target "${RUNTIME_TARGET}")"
[[ -n "${cg_tag}" ]] || { echo "error: could not derive the overlay cache tag" >&2; exit 1; }
psx_add_overlay_cache --game-id "${game_id}" --cache-src-root "${OVERLAY_CACHE_ROOT}" \
--stage "${STAGE}" --cg-tag "${cg_tag}"
else
if [[ -z "${SHIP_WITHOUT_OVERLAY_CACHE_BECAUSE}" ]]; then
echo "error: no --overlay-cache-root given. A shard cache is compiled from the disc, so a" >&2
echo " release without one relies on the compiled static shard plus runtime autocompile." >&2
echo " State that in words: --ship-without-overlay-cache-because '<reason>'." >&2
exit 1
fi
echo "overlay cache: NONE shipped -- ${SHIP_WITHOUT_OVERLAY_CACHE_BECAUSE}"
fi
if [[ "${STAGE_OVERLAY_TOOLCHAIN}" -eq 1 ]]; then
[[ -n "${GAME_BIN}" ]] || { echo "error: overlay toolchain needs psxrecomp-game under ${RECOMPILER_BUILD} (pass --recompiler-build, or --no-overlay-toolchain)" >&2; exit 1; }
case "${ARTIFACT}" in
windows-*) platform=win ;;
linux-*) platform=linux ;;
macos-arm64|macos-aarch64) platform=macos-arm64 ;;
macos-x64|macos-x86_64) platform=macos-x64 ;;
*) echo "error: cannot map artifact '${ARTIFACT}' to an overlay toolchain platform" >&2; exit 1 ;;
esac
tc_args=(--stage "${STAGE}" --recomp-dir "$(dirname "${GAME_BIN}")" \
--recomp-tools "${FRAMEWORK}/tools" --recomp-include "${FRAMEWORK}/runtime/include" \
--dl-cache "${DIST}/.dl-cache" --platform "${platform}")
[[ "${platform}" == win ]] && tc_args+=(--mingw-bin "${RUNTIME_BIN_DIR}")
psx_add_overlay_toolchain "${tc_args[@]}"
fi
# --- Notices and docs ---------------------------------------------------------
mkdir -p "${STAGE}/licenses"
if [[ -d "${FRAMEWORK}/runtime/licenses" ]]; then
cp -a "${FRAMEWORK}/runtime/licenses/." "${STAGE}/licenses/"
fi
[[ -f "${FRAMEWORK}/THIRD_PARTY_ATTRIBUTION.md" ]] && cp -a "${FRAMEWORK}/THIRD_PARTY_ATTRIBUTION.md" "${STAGE}/licenses/"
[[ -f "${FRAMEWORK}/LICENSE" ]] && cp -a "${FRAMEWORK}/LICENSE" "${STAGE}/licenses/psxrecomp-LICENSE"
if [[ -z "$(ls -A "${STAGE}/licenses" 2>/dev/null)" ]]; then
echo "error: no third-party notices staged (${FRAMEWORK}/runtime/licenses empty?)" >&2
exit 1
fi
for d in "${DOCS[@]+"${DOCS[@]}"}"; do
[[ -n "${d}" ]] || continue
[[ -f "${ROOT}/${d}" ]] || { echo "error: --doc ${d} not found" >&2; exit 1; }
cp -a "${ROOT}/${d}" "${STAGE}/$(basename "${d}")"
done
mkdir -p "${STAGE}/saves"
cat >"${STAGE}/README.txt" <<EOF
${DISPLAY_NAME} ${VERSION}
Platform: ${ARTIFACT}
This is the compiled game. Nothing needs to be generated or built.
1. Run ${EXE_BASENAME}.
2. On first run, point it at ${DISC_HINT}. The disc is never copied here.
3. Play. Saves land in saves/; settings beside the executable.
The game runs on the bundled OpenBIOS (bios/, MIT licensed). No retail BIOS
is included or required. Third-party notices are in licenses/.
overlay_toolchain/ lets the game turn code it streams from your disc into
native code on this machine; it is not a build kit and needs no action.
EOF
# --- Gates: what must NOT be in a bundled release -------------------------------
forbidden=()
for p in psxrecomp recomp-ui generated seeds CMakeLists.txt codegen_setup.c codegen_setup.h \
psxrecomp_cli.py psxrecomp-game psxrecomp-game.exe psxrecomp-bios psxrecomp-bios.exe \
toolchain disc src; do
[[ -e "${STAGE}/${p}" ]] && forbidden+=("${p}")
done
if [[ "${#forbidden[@]}" -gt 0 ]]; then
echo "error: bundled release stage contains build-kit content:" >&2
printf ' %s\n' "${forbidden[@]}" >&2
exit 1
fi
bad_bios="$(find "${STAGE}" -type f \( -iname 'SCPH*' -o -iname '*.BIN' -o -iname '*.bin' \) \
! -path "${STAGE}/bios/openbios.bin" 2>/dev/null || true)"
if [[ -n "${bad_bios}" ]]; then
echo "error: BIOS image(s) other than the bundled OpenBIOS in the stage:" >&2
printf '%s\n' "${bad_bios}" | sed 's/^/ /' >&2
exit 1
fi
bad_disc="$(find "${STAGE}" -type f \( -iname '*.cue' -o -iname '*.iso' -o -iname '*.img' -o -iname '*.chd' -o -iname '*.mcd' -o -iname '*.mcr' \) 2>/dev/null || true)"
if [[ -n "${bad_disc}" ]]; then
echo "error: disc / memory-card files in the stage:" >&2
printf '%s\n' "${bad_disc}" | sed 's/^/ /' >&2
exit 1
fi
if find "${STAGE}" -type f -name '*.c' ! -path "${STAGE}/overlay_toolchain/*" | grep -q .; then
echo "error: C sources in the stage outside overlay_toolchain/:" >&2
find "${STAGE}" -type f -name '*.c' ! -path "${STAGE}/overlay_toolchain/*" | sed 's/^/ /' >&2
exit 1
fi
# --- Windows: DLLs beside the exe, then Authenticode -------------------------
if [[ "${EXE_BASENAME}" == *.exe ]]; then
BUNDLE="${SCRIPT_DIR}/bundle_mingw_dlls.sh"
[[ -f "${BUNDLE}" ]] || { echo "error: missing ${BUNDLE}" >&2; exit 1; }
bash "${BUNDLE}" \
--runtime-bin "${RUNTIME_BIN_DIR}" --search-dir "${EXE_DIR}" --search-dir "${BUILD_DIR}" \
--exe "${STAGE}/${EXE_BASENAME}" --dest "${STAGE}" --label "${EXE_BASENAME}" \
--require libgcc_s_seh-1.dll --require libstdc++-6.dll --require libwinpthread-1.dll \
--require libssp-0.dll --require zlib1.dll --require z.dll
SIGN_SH="${SCRIPT_DIR}/ci/sign_windows.sh"
if [[ -f "${SIGN_SH}" ]]; then
bash "${SIGN_SH}" "${STAGE}"
else
echo "note: ${SIGN_SH} missing; Windows binaries ship unsigned" >&2
fi
fi
find "${STAGE}" -exec touch -c {} + 2>/dev/null || find "${STAGE}" -exec touch {} +
"${PY}" "${SCRIPT_DIR}/create_release_zip.py" --source "${STAGE}" --output "${DIST}/${ZIP_NAME}" >/dev/null
echo "Wrote ${DIST}/${ZIP_NAME}"
du -h "${DIST}/${ZIP_NAME}"
+7 -2
View File
@@ -1,4 +1,9 @@
#!/usr/bin/env bash
# RETIRED (2026-09-30): setup-host zip packager. Public releases are bundled
# (tools/package_game_release.sh ships the compiled game built from the
# title's committed generated/ C; docs/ci/BUNDLED_RELEASES.md). Kept only for
# a locally built self-compile kit; no release workflow calls it.
#
# Universal setup-host zip packager for PSXRecomp game repos.
#
# Stages the host exe, title sources, filtered psxrecomp/ + recomp-ui/, then
@@ -553,7 +558,7 @@ if [[ -f "${STAGE}/CMakeLists.txt" ]]; then
if (( ${#missing_refs[@]} )); then
echo "error: CMakeLists.txt references paths that are not staged in the zip:" >&2
for r in "${missing_refs[@]}"; do echo " - ${r}" >&2; done
echo " add them via --project-file / --project-dir in scripts/package_setup_release.sh" >&2
echo " add them via --project-file / --project-dir in the title's setup-host wrapper" >&2
exit 1
fi
fi
@@ -597,7 +602,7 @@ done
if (( ${#missing_incs[@]} )); then
echo "error: staged project sources include files that are not in the zip:" >&2
printf ' - %s\n' "${missing_incs[@]}" >&2
echo " add them via --project-file / --project-dir in scripts/package_setup_release.sh" >&2
echo " add them via --project-file / --project-dir in the title's setup-host wrapper" >&2
exit 1
fi
+289
View File
@@ -0,0 +1,289 @@
#!/usr/bin/env python3
"""Releases ship the compiled game built from COMMITTED generated/ C.
WHY THIS EXISTS
===============
Until 2026-09-30 the release flow was "setup-host": CI wiped generated/,
shipped emitters + sources + a wizard, and every player regenerated and
rebuilt the title on their own machine. That shipped uncompiled work. The
pivot to bundled releases touches four places that can drift back
independently -- the CI template, the scaffold's .gitignore, the scaffold's
packager wrapper, and Project Studio's audit/apply -- so this test pins each
one to the bundled contract rather than trusting the docs.
Every check here is against the ARTIFACT (the template text, the wrapper it
emits, the script's exit status on a fixture tree), never the intent.
Run: python3 tools/tests/test_bundled_release_layout.py
"""
import os
import re
import shutil
import subprocess
import sys
import tempfile
import unittest
from pathlib import Path
FW = Path(__file__).resolve().parents[2]
TOOLKIT = FW / "tools" / "new_project_layout"
sys.path.insert(0, str(TOOLKIT))
TEMPLATE = FW / "docs" / "ci" / "templates" / "game-release.yml"
CHECK_GENERATED = FW / "tools" / "ci" / "check_generated.sh"
BOOT = "SLUS_012.34"
def _git(root, *args):
return subprocess.run(["git", "-C", str(root), *args], capture_output=True,
text=True, check=False)
def _fixture_title(root: Path, *, ignore_generated=False, track=True,
retail=False):
"""A minimal game repo with committed generated C."""
root.mkdir(parents=True, exist_ok=True)
_git(root, "init", "-q")
_git(root, "config", "user.email", "t@example.invalid")
_git(root, "config", "user.name", "t")
(root / "game.toml").write_text(
f'[game]\nname = "Fixture"\nexe = "disc/{BOOT}"\n'
'[runtime]\noverlay_cache = true\n', encoding="utf-8")
(root / "CMakeLists.txt").write_text(
'psxrecomp_add_game_runtime(psx-runtime\n'
f' GEN_MARKER "generated/{BOOT}_dispatch.c"\n'
f' GEN_FULL_GLOB "generated/{BOOT}_full_*.c"\n)\n', encoding="utf-8")
gen = root / "generated"
gen.mkdir(exist_ok=True)
(gen / f"{BOOT}_dispatch.c").write_text("/* dispatch */\n", encoding="utf-8")
(gen / f"{BOOT}_full_00.c").write_text("/* full */\n", encoding="utf-8")
if retail:
(gen / "SCPH1001_dispatch.c").write_text("/* retail */\n", encoding="utf-8")
(gen / "SCPH1001_full.c").write_text("/* retail */\n", encoding="utf-8")
(root / ".gitignore").write_text(
("/generated/\n" if ignore_generated else "") + "/disc/\n/bios/\n/dist/\n/analysis/\n",
encoding="utf-8")
_git(root, "add", "game.toml", "CMakeLists.txt", ".gitignore")
if track and not ignore_generated:
_git(root, "add", "generated")
_git(root, "commit", "-q", "-m", "fixture")
return root
class TemplateContract(unittest.TestCase):
def setUp(self):
self.text = TEMPLATE.read_text(encoding="utf-8")
def test_template_exists_under_its_new_name(self):
self.assertTrue(TEMPLATE.is_file(), TEMPLATE)
self.assertFalse((TEMPLATE.parent / "setup-release.yml").exists(),
"the setup-host template must not linger beside the bundled one")
def test_never_clears_generated(self):
self.assertNotIn("clear_generated", self.text)
self.assertFalse((FW / "tools" / "ci" / "clear_generated.sh").exists(),
"clear_generated.sh wipes the release's inputs; it must be gone")
def test_gates_on_committed_game_c_before_building(self):
self.assertIn("tools/ci/check_generated.sh", self.text)
self.assertLess(self.text.index("check_generated.sh"),
self.text.index("Configure & build the game"))
def test_emits_openbios_in_ci(self):
self.assertIn("tools/ci/generate_openbios.sh", self.text)
def test_configures_a_full_build(self):
self.assertIn("-DPSXRECOMP_REQUIRE_GAME_C=ON", self.text)
self.assertNotIn("-DPSXRECOMP_FORCE_SETUP_HOST=ON", self.text)
self.assertNotIn("-DPSXRECOMP_ALLOW_NO_BIOS=ON", self.text)
self.assertIn("linking generated game C (full runtime)", self.text)
self.assertIn("BIOS backends linked: .*OpenBIOS", self.text)
def test_packages_through_the_bundled_wrapper(self):
self.assertIn("scripts/package_release.sh build-ci", self.text)
self.assertNotIn("package_setup_release.sh", self.text)
self.assertNotIn("package_setup_host.sh", self.text)
def test_zip_verification_rejects_kit_content(self):
for pat in ("'^psxrecomp/'", "'^psxrecomp_cli.py'", "'^generated/'", "'^psxrecomp-game'"):
self.assertIn(pat, self.text, f"zip gate must reject {pat}")
self.assertIn("bios/openbios.bin", self.text)
def test_parses_as_yaml_when_pyyaml_is_available(self):
try:
import yaml # noqa: F401
except ImportError:
self.skipTest("pyyaml not installed")
import yaml
doc = yaml.safe_load(self.text)
self.assertEqual(set(doc["jobs"]), {"prepare", "build", "release"})
names = [s.get("name", "") for s in doc["jobs"]["build"]["steps"]]
self.assertIn("Verify committed game C", names)
self.assertIn("Generate OpenBIOS backend C", names)
self.assertIn("Package game zip", names)
class ScaffoldContract(unittest.TestCase):
def test_gitignore_templates_track_generated(self):
for p in (TOOLKIT / "templates" / "gitignore.in",
FW / "docs" / "ci" / "templates" / "game.gitignore"):
lines = p.read_text(encoding="utf-8").splitlines()
ignored = [ln for ln in lines if re.match(r"^\s*/?generated/?\s*$", ln)]
self.assertEqual(ignored, [], f"{p} ignores generated/")
def test_packager_wrapper_template_is_bundled(self):
tpl = TOOLKIT / "templates" / "package_release.sh.in"
self.assertTrue(tpl.is_file(), tpl)
self.assertFalse((TOOLKIT / "templates" / "package_setup_release.sh.in").exists())
text = tpl.read_text(encoding="utf-8")
self.assertIn("package_game_release.sh", text)
self.assertNotIn("package_setup_host.sh", text)
self.assertNotIn("--project-dir", text, "a bundled release ships no sources")
def test_scaffold_scripts_use_the_new_template_and_wrapper(self):
for script in (TOOLKIT / "setup_project.sh", TOOLKIT / "setup_project.ps1"):
text = script.read_text(encoding="utf-8")
self.assertIn("game-release.yml", text, script)
self.assertIn("package_release.sh.in", text, script)
self.assertNotIn("setup-release.yml", text, script)
self.assertNotIn("package_setup_release", text, script)
self.assertIn("git add generated", text, script)
def test_fill_tokens_fills_the_template(self):
with tempfile.TemporaryDirectory() as td:
out = Path(td) / "release.yml"
r = subprocess.run([sys.executable, str(TOOLKIT / "fill_tokens.py"),
str(TEMPLATE), str(out), "--ci-placeholders",
"--set", "ZIP_PREFIX=fx", "--set", "GAME_TITLE=Fixture: Recompiled"],
capture_output=True, text=True, check=False)
self.assertEqual(r.returncode, 0, r.stderr)
text = out.read_text(encoding="utf-8")
self.assertNotIn("YOUR_ZIP_PREFIX", text)
self.assertNotIn("YOUR_GAME_TITLE", text)
self.assertIn("dist/fx-*-${{ matrix.artifact }}.zip", text)
self.assertIn('name: "Fixture: Recompiled ${{ needs.prepare.outputs.tag }}"', text)
class CheckGeneratedScript(unittest.TestCase):
def run_check(self, root):
return subprocess.run(["bash", str(CHECK_GENERATED), "--root", str(root)],
capture_output=True, text=True, check=False)
def test_passes_on_committed_game_c(self):
with tempfile.TemporaryDirectory() as td:
root = _fixture_title(Path(td) / "t")
r = self.run_check(root)
self.assertEqual(r.returncode, 0, r.stdout + r.stderr)
self.assertIn("generated game C ok", r.stdout)
def test_fails_when_marker_is_missing(self):
with tempfile.TemporaryDirectory() as td:
root = _fixture_title(Path(td) / "t")
(root / "generated" / f"{BOOT}_dispatch.c").unlink()
r = self.run_check(root)
self.assertNotEqual(r.returncode, 0)
self.assertIn("is missing", r.stderr)
def test_fails_when_generated_is_ignored(self):
with tempfile.TemporaryDirectory() as td:
root = _fixture_title(Path(td) / "t", ignore_generated=True)
r = self.run_check(root)
self.assertNotEqual(r.returncode, 0)
self.assertIn("NOT tracked", r.stderr)
def test_fails_when_generated_is_untracked(self):
with tempfile.TemporaryDirectory() as td:
root = _fixture_title(Path(td) / "t", track=False)
r = self.run_check(root)
self.assertNotEqual(r.returncode, 0)
self.assertIn("NOT tracked", r.stderr)
def test_fails_on_retail_bios_c(self):
with tempfile.TemporaryDirectory() as td:
root = _fixture_title(Path(td) / "t", retail=True)
r = self.run_check(root)
self.assertNotEqual(r.returncode, 0)
self.assertIn("retail-BIOS-derived", r.stderr)
class ProjectStudioContract(unittest.TestCase):
def test_audit_flags_ignored_generated_and_missing_c(self):
from project_studio.detect import audit_project
with tempfile.TemporaryDirectory() as td:
root = _fixture_title(Path(td) / "t", ignore_generated=True)
report = audit_project(root)
by_id = {c.id: c for c in report.checks}
self.assertEqual(by_id["gitignore"].status.value, "fail")
self.assertEqual(by_id["generated_committed"].status.value, "fail")
def test_audit_passes_committed_generated(self):
from project_studio.detect import audit_project
with tempfile.TemporaryDirectory() as td:
root = _fixture_title(Path(td) / "t")
report = audit_project(root)
by_id = {c.id: c for c in report.checks}
self.assertEqual(by_id["generated_committed"].status.value, "pass")
self.assertNotIn("retail_bios_c", by_id)
def test_merge_gitignore_unignores_generated(self):
from project_studio.models import MigrateOptions
from project_studio.ops import op_merge_gitignore
with tempfile.TemporaryDirectory() as td:
root = _fixture_title(Path(td) / "t", ignore_generated=True)
res = op_merge_gitignore(root, MigrateOptions())
self.assertTrue(res.ok, res.message)
lines = (root / ".gitignore").read_text(encoding="utf-8").splitlines()
self.assertFalse(any(re.match(r"^\s*/?generated/?\s*$", ln) for ln in lines))
self.assertIn("/disc/", lines)
def test_emit_packager_and_ci_workflow(self):
from project_studio.models import MigrateOptions
from project_studio.ops import op_emit_ci_workflow, op_emit_packager
with tempfile.TemporaryDirectory() as td:
root = _fixture_title(Path(td) / "t")
opts = MigrateOptions(zip_prefix="fx", window_title="Fixture Recompiled",
project_name="FixtureRecomp")
pk = op_emit_packager(root, opts)
self.assertTrue(pk.ok, pk.message)
wrapper = root / "scripts" / "package_release.sh"
self.assertTrue(wrapper.is_file())
self.assertIn("package_game_release.sh", wrapper.read_text(encoding="utf-8"))
ci = op_emit_ci_workflow(root, opts)
self.assertTrue(ci.ok, ci.message)
wf = (root / ".github" / "workflows" / "release.yml").read_text(encoding="utf-8")
self.assertIn("check_generated.sh", wf)
self.assertIn("dist/fx-*", wf)
# generate_ci --check agrees the installed workflow is current.
r = subprocess.run([sys.executable, str(FW / "tools" / "generate_ci.py"),
str(root), "--check"], capture_output=True, text=True,
check=False)
self.assertEqual(r.returncode, 0, r.stdout + r.stderr)
class PackagerScriptContract(unittest.TestCase):
def test_bundled_packager_routes_through_release_stage(self):
text = (FW / "tools" / "package_game_release.sh").read_text(encoding="utf-8")
self.assertIn("release_overlay_stage.sh", text)
self.assertIn("psx_add_mod_catalog", text)
self.assertIn("psx_add_overlay_toolchain", text)
self.assertIn("PSXRECOMP_FORCE_SETUP_HOST:BOOL=ON", text,
"must refuse to package a setup host as a game")
# No source allowlist of any kind: nothing project-owned is shipped.
self.assertNotIn("--project-file", text)
self.assertNotIn("--project-dir", text)
def test_scripts_are_executable_and_parse(self):
for rel in ("tools/package_game_release.sh", "tools/ci/check_generated.sh",
"tools/ci/generate_openbios.sh"):
p = FW / rel
self.assertTrue(os.access(p, os.X_OK), f"{rel} not executable")
r = subprocess.run(["bash", "-n", str(p)], capture_output=True, text=True)
self.assertEqual(r.returncode, 0, r.stderr)
if __name__ == "__main__":
if shutil.which("git") is None or shutil.which("bash") is None:
print("git and bash are required", file=sys.stderr)
sys.exit(2)
unittest.main(verbosity=1)
@@ -108,11 +108,13 @@ message(STATUS "PROBE_SOURCES=${{srcs}}")
self.profile(static_output='generated/overlays_static.c')
code, out, _ = self.configure(None)
self.assertNotEqual(code, 0)
self.assertIn('passes no GAME_OVERLAY_STATIC_C', out)
# CMake re-wraps message() text at ~72 columns, so this phrase can
# straddle a line break; compare with whitespace collapsed.
self.assertIn('passes no GAME_OVERLAY_STATIC_C', ' '.join(out.split()))
other = self.project / 'elsewhere' / 'overlays_static.c'
code, out, _ = self.configure(other)
self.assertNotEqual(code, 0)
self.assertIn('Make them name the same file', out)
self.assertIn('Make them name the same file', ' '.join(out.split()))
def test_undeclared_profiles_and_capture_built_shards_are_unaffected(self):
# A release/DLL-cache title: profile without static_output, no static C.