mirror/psxrecomp
1
0
Fork 0
mirror of https://github.com/RetroPortingToolKit/psxrecomp synced 2026-10-25 18:05:12 -04:00
Code Issues Packages Projects Releases Wiki Activity

mods: let titles exclude framework builtins with EXCLUDE_BUILTIN_MODS

Browse Source 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Matthew Stanley
2026-09-26 12:11:39 -07:00
parent 71336b44d2
commit 141eb39002
11 changed files with 690 additions and 38 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/MOD_PACKAGES.md
+63 -3
View File
@@ -18,9 +18,10 @@ Two catalog roots sit beside the executable, split by who owns the files:
```text
<exe>/mods/
bundled/ build output — the framework's mods/builtin/packages plus the
title's mods/preloaded/packages. Every build WIPES and re-stages
this tree, so nothing a player owns may live here.
bundled/ build output — the framework's mods/builtin/packages (minus any
the title declines with EXCLUDE_BUILTIN_MODS) plus the title's
mods/preloaded/packages. Every build WIPES and re-stages this
tree, so nothing a player owns may live here.
installed/ launcher-owned — .psxmod archives installed through the Mods
manager. No build ever touches this tree.
state.toml user selection state (enabled features, option values).
@@ -81,6 +82,52 @@ Pass `PRELOADED_MODS_DIR NONE` to declare that a target intentionally ships no
game catalog. A target built with `COSIM` stages nothing: it has no launcher,
and it shares an output directory with the real runtime.
### Declining a framework builtin (`EXCLUDE_BUILTIN_MODS`)
Every package under the framework's `mods/builtin/packages` targets
`game_id = "*"`, so by default every title ships all of them. A title that does
not want one — because it ships its own replacement, or because the feature is
wrong for it — names it:
```cmake
psxrecomp_add_game_runtime(psx-runtime
...
PRELOADED_MODS_DIR "${CMAKE_CURRENT_SOURCE_DIR}/mods/preloaded"
# WipEout 3's own team-mark bezel replaces the generic file-picker one,
# and host-paced fast loading is not offered for it.
EXCLUDE_BUILTIN_MODS
psx.enhancement.fast-loading
psx.presentation.bezel
)
```
`EXCLUDE_BUILTIN_MODS` is accepted by both `psxrecomp_add_game_runtime()` and
`psxrecomp_add_runtime_target()` (the PGXP clone inherits it). An excluded
package is **absent, not hidden**, the same rule as the developer channel:
- it is never copied into `<exe-dir>/mods/bundled`, so the launcher cannot list
it and every release packager — which all ship the build's staged tree —
cannot ship it either;
- the build publishes the exclusions beside the catalog manifest as
`psx_mod_catalog_<target>.excluded.txt`. The build-time guard fails if an
excluded id turns up in `mods/bundled` anyway, and `tools/release_stage.py
stage-mods` both refuses such a catalog and stops counting an excluded
builtin as *missing* when a caller passes the framework's `mods/builtin` as a
`--mod-source`;
- the framework itself is unchanged: the package, its plugin and its C API stay
available to every other title.
Configure fails loudly when an excluded id is not a framework builtin (a typo
would otherwise leave the package shipping while the CMakeLists reads as if it
did not), and when the title's own catalog provides the same id — that is an
override, and excluding and overriding one id at once contradicts itself. The
selection lives in `runtime/psx_mod_catalog_select.cmake`; both it and the guard
are exercised by `runtime/tests/test_mod_catalog_layout.py`.
A player's `mods/state.toml` written by an earlier build may still name an
excluded package. That selection is dormant, not an error: see
[State and migration](#state-and-migration).
**Do NOT write your own `copy_directory` into `<exe-dir>/mods`.** Five titles
did, and the reason it is now a build error is worth stating: a hand-written
copy names the destination as a *string*, so when framework commit `4cc04be3`
@@ -527,6 +574,19 @@ State format 1 and package-only manifests remain readable as a migration aid.
They appear through one synthetic legacy feature. New packages should use
explicit features.
**Selections for packages the catalog does not hold are dormant.** State can
outlive the package it names: the player deleted an installed archive, a
release build stripped a developer-only package, or the title now declines a
framework builtin with `EXCLUDE_BUILTIN_MODS`. Resolution only visits packages
that are present, so such a selection contributes nothing to the plan (and
does not change its fingerprint), and it is not a launch error — the Mods page
no longer lists the package, so the player would have no way to clear one.
`save_state()` keeps the entry verbatim, so the choice applies again if the
package ever returns, and the runtime names each one at startup
(`mod selection kept but inactive: <id> is not in this build's mod catalog`).
A selection pinned to a *version* that is missing while other versions of the
package are present remains an error, because the Mods page can fix that one.
The old `derived_disc` VCDIFF mechanism is legacy conversion scaffolding only.
Feature-style manifests reject it. It is not a product mod primitive, fallback,
or image-selection workflow; patched discs may be used offline as parity
runtime/include/mod_packages.h
+9
View File
@@ -437,6 +437,15 @@ public:
* package is absent from the list. */
const std::vector<std::string>& scan_errors() const { return scan_errors_; }
/* Package ids that mods/state.toml carries a selection for but that no
* catalog root holds -- a package the player removed, a developer-only
* package a release build stripped, or a framework builtin the title
* excludes (EXCLUDE_BUILTIN_MODS). Such a selection is DORMANT: resolve()
* never visits it, so it contributes nothing to the plan, and save_state()
* keeps it verbatim so the choice returns if the package ever does. The
* runtime names these once at startup instead of dropping them silently. */
std::vector<std::string> dormant_selections() const;
private:
/* One-time move of a pre-split <exe>/mods/packages tree into the two
* owned roots. Anything the build also staged into bundled/ is dropped;
runtime/psx_check_mod_catalog.cmake
+30
View File
@@ -28,6 +28,7 @@
# -----
# cmake -DPSX_MODS_DIR=<build>/mods
# -DPSX_CATALOG_MANIFEST=<file> # ids this target must stage
# [-DPSX_CATALOG_EXCLUDED=<file>] # builtins it must NOT stage
# [-DPSX_CATALOG_ALT_MANIFESTS=<f>|<f>] # sibling targets' id sets
# [-DPSX_REQUIRE_STAGED=1] # mods/ MUST already exist
# [-DPSX_LABEL=psx-runtime]
@@ -282,6 +283,35 @@ if(_manifestless)
"will silently ignore it.")
endif()
# ---- 5. builtins the target EXCLUDES must be absent ------------------------
#
# EXCLUDE_BUILTIN_MODS means "this title does not ship that package". The
# staging step never copies an excluded id, so finding one here means some
# other mechanism put it back (a hand-written copy, a stale merge) and the
# release would ship a package the title declined. A sibling target sharing
# this output directory may legitimately stage the id, so ids that belong to a
# sibling's declared set are left to that sibling's own guard.
if(DEFINED PSX_CATALOG_EXCLUDED AND NOT PSX_CATALOG_EXCLUDED STREQUAL "")
_psx_read_manifest("${PSX_CATALOG_EXCLUDED}" _excluded)
set(_resurrected "")
foreach(_id IN LISTS _excluded)
if("${_id}" IN_LIST _staged AND NOT "${_id}" IN_LIST _all_known)
list(APPEND _resurrected "${_id}")
endif()
endforeach()
if(_resurrected)
list(JOIN _resurrected "\n " _pretty)
message(FATAL_ERROR
"[${PSX_LABEL}] package(s) this target EXCLUDES "
"(EXCLUDE_BUILTIN_MODS) are present in the staged catalog:\n"
" ${_pretty}\n under ${_bundled}\n\n"
"The framework's staging never copies an excluded builtin, so "
"something else put it there -- most likely a hand-written copy "
"into <exe-dir>/mods. A release built from this tree would ship a "
"package the title declined. See docs/MOD_PACKAGES.md.")
endif()
endif()
# Extra ids are reported but not fatal: mods/bundled is wiped and re-staged on
# every build, so an unexpected id is either a sibling target's package or
# something staged by a mechanism the framework does not know about. Worth
runtime/psx_mod_catalog_select.cmake
+117
View File
@@ -0,0 +1,117 @@
# Which framework builtin mod packages a runtime target stages.
#
# Every builtin under mods/builtin/packages targets game_id "*", so by default
# every title ships every one of them. A title may decline one it does not
# want -- because it ships its own replacement (WipEout 3's team-mark bezel
# supersedes the generic file-picker bezel) or because the feature is wrong for
# it -- by naming the package in EXCLUDE_BUILTIN_MODS on
# psxrecomp_add_runtime_target() / psxrecomp_add_game_runtime().
#
# An excluded package is ABSENT, not hidden: it is never copied into
# <exe-dir>/mods/bundled, so the launcher cannot list it, no release packager
# can ship it, and a stale state.toml naming it has nothing to reach.
#
# Kept in its own file, with no project-mode commands, so that
# runtime/tests/test_mod_catalog_layout.py can drive the real selection logic
# through `cmake -P` without configuring a whole runtime.
#
# psx_select_builtin_mod_ids(<out_var>
# LABEL <target name, for messages>
# AVAILABLE <ids>... # package directories under mods/builtin/packages
# [ALLOWLIST <ids>...] # PSX_BUILTIN_MOD_ALLOWLIST, when set
# [EXCLUDE <ids>...] # the target's EXCLUDE_BUILTIN_MODS
# [TITLE <ids>...] # the target's own PRELOADED_MODS_DIR package ids
# )
#
# Fails configure loudly on:
# * an excluded id that is not a framework builtin (a typo would otherwise
# leave the package shipping while the CMakeLists reads as if it did not);
# * an excluded id the title's own catalog also provides (a title catalog may
# OVERRIDE a builtin at the same id -- excluding and overriding the same id
# at once contradicts itself, and either reading would surprise someone).
# psx_split_exclude_builtin_mods(<ids_var> <tail_var> <tokens>...)
#
# psxrecomp_add_game_runtime() parses EXCLUDE_BUILTIN_MODS as a multi-value
# argument but forwards keywords it does not know (GAME_OVERLAY_STATIC_C,
# APP_ICON, ...) to psxrecomp_add_runtime_target(). cmake_parse_arguments keeps
# collecting a multi-value list until it meets one of ITS OWN keywords, so a
# forwarded keyword written after EXCLUDE_BUILTIN_MODS -- and its value -- would
# be swallowed as package ids. Package ids are lower-case dotted names, never
# an upper-case keyword, so everything from the first keyword-shaped token on
# is handed back as the tail to forward.
function(psx_split_exclude_builtin_mods ids_var tail_var)
set(_ids "")
set(_tail "")
set(_in_tail FALSE)
foreach(_tok IN LISTS ARGN)
if(NOT _in_tail AND _tok MATCHES "^[A-Z][A-Z0-9_]*$")
set(_in_tail TRUE)
endif()
if(_in_tail)
list(APPEND _tail "${_tok}")
else()
list(APPEND _ids "${_tok}")
endif()
endforeach()
set(${ids_var} "${_ids}" PARENT_SCOPE)
set(${tail_var} "${_tail}" PARENT_SCOPE)
endfunction()
function(psx_select_builtin_mod_ids out_var)
cmake_parse_arguments(_psxsel "" "LABEL" "AVAILABLE;ALLOWLIST;EXCLUDE;TITLE" ${ARGN})
if(NOT _psxsel_LABEL)
set(_psxsel_LABEL "runtime")
endif()
set(_selected ${_psxsel_AVAILABLE})
if(_psxsel_ALLOWLIST)
foreach(_id IN LISTS _psxsel_ALLOWLIST)
if(NOT "${_id}" IN_LIST _psxsel_AVAILABLE)
message(FATAL_ERROR
"PSX_BUILTIN_MOD_ALLOWLIST names missing package: ${_id}")
endif()
endforeach()
set(_selected ${_psxsel_ALLOWLIST})
endif()
set(_unknown "")
set(_overridden "")
set(_exclude ${_psxsel_EXCLUDE})
list(REMOVE_DUPLICATES _exclude)
foreach(_id IN LISTS _exclude)
if(NOT "${_id}" IN_LIST _psxsel_AVAILABLE)
list(APPEND _unknown "${_id}")
elseif("${_id}" IN_LIST _psxsel_TITLE)
list(APPEND _overridden "${_id}")
endif()
endforeach()
if(_unknown)
list(JOIN _unknown "\n " _pretty_unknown)
list(JOIN _psxsel_AVAILABLE "\n " _pretty_available)
message(FATAL_ERROR
"EXCLUDE_BUILTIN_MODS for target '${_psxsel_LABEL}' names package(s) "
"that are not framework builtins:\n ${_pretty_unknown}\n\n"
"The framework's builtin packages (mods/builtin/packages) are:\n"
" ${_pretty_available}\n\n"
"An unknown id is an error rather than a no-op: a misspelled exclusion "
"would leave the package shipping while the CMakeLists reads as though "
"it does not. See docs/MOD_PACKAGES.md.")
endif()
if(_overridden)
list(JOIN _overridden "\n " _pretty_overridden)
message(FATAL_ERROR
"EXCLUDE_BUILTIN_MODS for target '${_psxsel_LABEL}' excludes package(s) "
"its own PRELOADED_MODS_DIR catalog also provides:\n"
" ${_pretty_overridden}\n\n"
"A title catalog package with a builtin's id OVERRIDES that builtin, so "
"excluding the same id contradicts it. Drop the id from "
"EXCLUDE_BUILTIN_MODS to ship the title's override, or delete the "
"title's package to ship neither.")
endif()
if(_exclude)
list(REMOVE_ITEM _selected ${_exclude})
endif()
set(${out_var} "${_selected}" PARENT_SCOPE)
endfunction()
runtime/runtime.cmake
+101 -33
View File
@@ -916,6 +916,11 @@ endfunction()
# anything the title itself registered) and fails the build if a declared
# package did not reach mods/bundled, or if any package this build stages
# turned up in the legacy mods/packages instead.
#
# A title may also decline individual framework builtins with
# EXCLUDE_BUILTIN_MODS; the selection (and its loud failures) lives in
# psx_mod_catalog_select.cmake so the tests can drive it via `cmake -P`.
include("${CMAKE_CURRENT_LIST_DIR}/psx_mod_catalog_select.cmake")
# Immediate subdirectory names of `root` (package ids), sorted.
function(_psxrt_package_ids root out_var)
@@ -958,6 +963,7 @@ function(_psxrt_finalize_mod_catalog_guards)
get_property(_targets GLOBAL PROPERTY PSXRECOMP_MOD_CATALOG_TARGETS)
get_property(_manifests GLOBAL PROPERTY PSXRECOMP_MOD_CATALOG_MANIFESTS)
get_property(_dirs GLOBAL PROPERTY PSXRECOMP_MOD_CATALOG_DIRS)
get_property(_excludeds GLOBAL PROPERTY PSXRECOMP_MOD_CATALOG_EXCLUDED)
list(LENGTH _targets _n)
if(_n EQUAL 0)
return()
@@ -970,13 +976,16 @@ function(_psxrt_finalize_mod_catalog_guards)
# deferred pass per directory and each pass handles only its own.
set(_here_targets "")
set(_here_manifests "")
set(_here_excludeds "")
foreach(_i RANGE 0 ${_last})
list(GET _dirs ${_i} _d)
if(_d STREQUAL "${CMAKE_CURRENT_SOURCE_DIR}")
list(GET _targets ${_i} _t)
list(GET _manifests ${_i} _m)
list(GET _excludeds ${_i} _x)
list(APPEND _here_targets "${_t}")
list(APPEND _here_manifests "${_m}")
list(APPEND _here_excludeds "${_x}")
endif()
endforeach()
list(LENGTH _here_targets _n_here)
@@ -988,6 +997,7 @@ function(_psxrt_finalize_mod_catalog_guards)
foreach(_i RANGE 0 ${_last_here})
list(GET _here_targets ${_i} _t)
list(GET _here_manifests ${_i} _own)
list(GET _here_excludeds ${_i} _own_excluded)
# Sibling runtime targets in this directory. Two of them can share one
# output directory (Tomba 2's US and Italian runtimes both land in the
@@ -1007,6 +1017,7 @@ function(_psxrt_finalize_mod_catalog_guards)
COMMAND ${CMAKE_COMMAND}
"-DPSX_MODS_DIR=$<TARGET_FILE_DIR:${_t}>/mods"
"-DPSX_CATALOG_MANIFEST=${_own}"
"-DPSX_CATALOG_EXCLUDED=${_own_excluded}"
"-DPSX_CATALOG_ALT_MANIFESTS=${_alts_joined}"
"-DPSX_REQUIRE_STAGED=1"
"-DPSX_LABEL=${_t}"
@@ -1025,6 +1036,7 @@ function(_psxrt_finalize_mod_catalog_guards)
set_property(GLOBAL PROPERTY PSXRECOMP_MOD_CATALOG_TEST_ADDED TRUE)
list(GET _here_targets 0 _first)
list(GET _here_manifests 0 _first_manifest)
list(GET _here_excludeds 0 _first_excluded)
set(_first_alts "")
foreach(_j RANGE 0 ${_last_here})
if(NOT _j EQUAL 0)
@@ -1037,6 +1049,7 @@ function(_psxrt_finalize_mod_catalog_guards)
COMMAND ${CMAKE_COMMAND}
"-DPSX_MODS_DIR=$<TARGET_FILE_DIR:${_first}>/mods"
"-DPSX_CATALOG_MANIFEST=${_first_manifest}"
"-DPSX_CATALOG_EXCLUDED=${_first_excluded}"
"-DPSX_CATALOG_ALT_MANIFESTS=${_first_alts_joined}"
"-DPSX_REQUIRE_STAGED=0"
"-DPSX_LABEL=${_first}"
@@ -1046,39 +1059,21 @@ endfunction()
# Stage the framework's builtin packages and the title's own packages into
# <exe-dir>/mods/bundled, and register the guards described above.
#
# Trailing arguments are the target's EXCLUDE_BUILTIN_MODS: framework builtins
# this title declines to ship (runtime/psx_mod_catalog_select.cmake).
function(_psxrt_stage_mod_catalog target preloaded_dir)
set(_excluded_builtins ${ARGN})
set(_out "$<TARGET_FILE_DIR:${target}>")
set(_ids "")
set(_copy "")
# ---- framework-owned builtins (psx.*) ---------------------------------
# These target game_id "*" -- emulated-hardware features rather than
# per-disc content -- so every game gets them without carrying a copy of
# the manifests.
set(_builtin_root "${PSXRECOMP_ROOT}/mods/builtin/packages")
if(EXISTS "${_builtin_root}")
if(DEFINED PSX_BUILTIN_MOD_ALLOWLIST AND NOT
"${PSX_BUILTIN_MOD_ALLOWLIST}" STREQUAL "")
set(_builtin_ids ${PSX_BUILTIN_MOD_ALLOWLIST})
foreach(_id IN LISTS _builtin_ids)
if(NOT EXISTS "${_builtin_root}/${_id}")
message(FATAL_ERROR
"PSX_BUILTIN_MOD_ALLOWLIST names missing package: ${_id}")
endif()
endforeach()
else()
_psxrt_package_ids("${_builtin_root}" _builtin_ids)
endif()
foreach(_id IN LISTS _builtin_ids)
list(APPEND _ids "${_id}")
list(APPEND _copy
COMMAND ${CMAKE_COMMAND} -E copy_directory
"${_builtin_root}/${_id}"
"${_out}/mods/bundled/${_id}")
endforeach()
endif()
set(_builtin_copy "")
set(_game_copy "")
set(_game_ids "")
# ---- title-owned packages ---------------------------------------------
# Enumerated first so builtin selection can refuse an id the title both
# excludes and overrides; still COPIED after the builtins (see below).
set(_readme_copy "")
if(preloaded_dir STREQUAL "")
# The missed-title tripwire. A title whose source tree carries a mod
@@ -1125,7 +1120,6 @@ function(_psxrt_stage_mod_catalog target preloaded_dir)
# scaffold's initial state and must still configure -- only a bad path
# is an error, because a wrong path is exactly how a title ends up
# shipping an empty Mods page.
set(_game_ids "")
if(IS_DIRECTORY "${preloaded_dir}/packages")
_psxrt_package_ids("${preloaded_dir}/packages" _game_ids)
endif()
@@ -1135,8 +1129,7 @@ function(_psxrt_stage_mod_catalog target preloaded_dir)
"${preloaded_dir}/packages; staging framework packages only")
endif()
foreach(_id IN LISTS _game_ids)
list(APPEND _ids "${_id}")
list(APPEND _copy
list(APPEND _game_copy
COMMAND ${CMAKE_COMMAND} -E copy_directory
"${preloaded_dir}/packages/${_id}"
"${_out}/mods/bundled/${_id}")
@@ -1152,6 +1145,45 @@ function(_psxrt_stage_mod_catalog target preloaded_dir)
endif()
endif()
# ---- framework-owned builtins (psx.*) ---------------------------------
# These target game_id "*" -- emulated-hardware features rather than
# per-disc content -- so every game gets them without carrying a copy of
# the manifests, minus any the title declines with EXCLUDE_BUILTIN_MODS.
set(_builtin_root "${PSXRECOMP_ROOT}/mods/builtin/packages")
set(_available_builtins "")
if(EXISTS "${_builtin_root}")
_psxrt_package_ids("${_builtin_root}" _available_builtins)
endif()
set(_allowlist "")
if(DEFINED PSX_BUILTIN_MOD_ALLOWLIST AND NOT
"${PSX_BUILTIN_MOD_ALLOWLIST}" STREQUAL "")
set(_allowlist ${PSX_BUILTIN_MOD_ALLOWLIST})
endif()
psx_select_builtin_mod_ids(_builtin_ids
LABEL "${target}"
AVAILABLE ${_available_builtins}
ALLOWLIST ${_allowlist}
EXCLUDE ${_excluded_builtins}
TITLE ${_game_ids})
if(_excluded_builtins)
set(_excluded_pretty ${_excluded_builtins})
list(REMOVE_DUPLICATES _excluded_pretty)
list(JOIN _excluded_pretty ", " _excluded_pretty)
message(STATUS
"psxrecomp: ${target} excludes framework builtin mod(s): "
"${_excluded_pretty}")
endif()
foreach(_id IN LISTS _builtin_ids)
list(APPEND _ids "${_id}")
list(APPEND _builtin_copy
COMMAND ${CMAKE_COMMAND} -E copy_directory
"${_builtin_root}/${_id}"
"${_out}/mods/bundled/${_id}")
endforeach()
list(APPEND _ids ${_game_ids})
# Framework first, title second: the title's copy of a shared id wins.
set(_copy ${_builtin_copy} ${_game_copy})
# The COPY commands above may legitimately name an id twice -- a title's
# catalog is allowed to OVERRIDE a framework builtin at the same id and
# version, and Tomba 2's Italian catalog does exactly that with localized
@@ -1202,9 +1234,23 @@ function(_psxrt_stage_mod_catalog target preloaded_dir)
list(SORT _ids)
string(JOIN "\n" _manifest_text ${_ids})
_psxrt_write_if_changed("${_manifest}" "${_manifest_text}\n")
# ...and the builtins this target declined, which must be ABSENT. Written
# even when empty so a packager reading it (tools/release_stage.py) never
# has to guess whether "no file" means "nothing excluded" or "old build".
set(_excluded_manifest
"${CMAKE_CURRENT_BINARY_DIR}/psx_mod_catalog_${target}.excluded.txt")
set(_excluded_sorted ${_excluded_builtins})
list(REMOVE_DUPLICATES _excluded_sorted)
list(SORT _excluded_sorted)
set(_excluded_text "# framework builtin mod packages ${target} excludes (EXCLUDE_BUILTIN_MODS)\n")
foreach(_id IN LISTS _excluded_sorted)
string(APPEND _excluded_text "${_id}\n")
endforeach()
_psxrt_write_if_changed("${_excluded_manifest}" "${_excluded_text}")
set_property(GLOBAL APPEND PROPERTY PSXRECOMP_MOD_CATALOG_TARGETS "${target}")
set_property(GLOBAL APPEND PROPERTY PSXRECOMP_MOD_CATALOG_MANIFESTS "${_manifest}")
set_property(GLOBAL APPEND PROPERTY PSXRECOMP_MOD_CATALOG_EXCLUDED "${_excluded_manifest}")
set_property(GLOBAL APPEND PROPERTY PSXRECOMP_MOD_CATALOG_DIRS
"${CMAKE_CURRENT_SOURCE_DIR}")
# Schedule the guard pass once per directory, not once per target.
@@ -1256,7 +1302,12 @@ function(psxrecomp_add_runtime_target target)
# monolithic full.c, so this argument may carry 1..N paths. A single path
# is just a one-element list, so games still passing one file are
# unaffected.
set(multiValueArgs EXTRAS_SOURCES GAME_GENERATED_FULL_C)
#
# EXCLUDE_BUILTIN_MODS names framework builtin packages (ids under
# mods/builtin/packages) this target does not ship. They are left out of
# <exe-dir>/mods/bundled entirely -- absent, not hidden -- so no packager
# can ship them either. An id that is not a builtin fails configure.
set(multiValueArgs EXTRAS_SOURCES GAME_GENERATED_FULL_C EXCLUDE_BUILTIN_MODS)
cmake_parse_arguments(PSXRT "${options}" "${oneValueArgs}" "${multiValueArgs}" ${ARGN})
# DEBUG_PORT and WINDOW_TITLE were previously required cmake-time defaults.
@@ -1857,7 +1908,8 @@ function(psxrecomp_add_runtime_target target)
# letting it stage would have it wipe and re-stage the runtime's catalog
# with only the framework half.
if(NOT PSXRT_COSIM)
_psxrt_stage_mod_catalog("${target}" "${PSXRT_PRELOADED_MODS_DIR}")
_psxrt_stage_mod_catalog("${target}" "${PSXRT_PRELOADED_MODS_DIR}"
${PSXRT_EXCLUDE_BUILTIN_MODS})
endif()
endif()
@@ -2319,6 +2371,8 @@ endfunction()
# MAX_PLAYERS 2
# ENABLE_NETPLAY_IF_PRESENT
# ENABLE_SETUP_WIZARD
# PRELOADED_MODS_DIR "${CMAKE_CURRENT_SOURCE_DIR}/mods/preloaded"
# EXCLUDE_BUILTIN_MODS psx.presentation.bezel # optional; builtins not shipped
# )
#
# Remaining args are forwarded to psxrecomp_add_runtime_target.
@@ -2333,7 +2387,10 @@ function(psxrecomp_add_game_runtime target)
NETPLAY_LOBBY_URL
PRELOADED_MODS_DIR
)
set(multiValueArgs GEN_FULL_GLOB CODEGEN_SETUP_SOURCES)
# EXCLUDE_BUILTIN_MODS is parsed here (not left in the unparsed tail) for
# the same reason as PRELOADED_MODS_DIR: an unknown keyword written after
# CODEGEN_SETUP_SOURCES would otherwise be swallowed as more source files.
set(multiValueArgs GEN_FULL_GLOB CODEGEN_SETUP_SOURCES EXCLUDE_BUILTIN_MODS)
cmake_parse_arguments(PSXG "${options}" "${oneValueArgs}" "${multiValueArgs}" ${ARGN})
if(NOT PSXRECOMP_ROOT)
@@ -2518,6 +2575,17 @@ function(psxrecomp_add_game_runtime target)
list(APPEND _psxg_forwarded_args
PRELOADED_MODS_DIR "${PSXG_PRELOADED_MODS_DIR}")
endif()
if(PSXG_EXCLUDE_BUILTIN_MODS)
# Hand back any forwarded keyword (and its values) the multi-value
# parse swallowed after EXCLUDE_BUILTIN_MODS; see the helper.
psx_split_exclude_builtin_mods(_psxg_exclude_ids _psxg_exclude_tail
${PSXG_EXCLUDE_BUILTIN_MODS})
if(_psxg_exclude_ids)
list(APPEND _psxg_forwarded_args
EXCLUDE_BUILTIN_MODS ${_psxg_exclude_ids})
endif()
list(APPEND _psxg_forwarded_args ${_psxg_exclude_tail})
endif()
set(_psxg_rt_args
GAME_VERSION "${PSX_GAME_VERSION}"
runtime/src/mod_packages.cpp
+11
View File
@@ -2987,6 +2987,17 @@ bool ModPackageManager::set_feature_resource_path(
return true;
}
std::vector<std::string> ModPackageManager::dormant_selections() const {
std::vector<std::string> dormant;
for (const auto& [id, selection] : selections_) {
(void)selection;
const auto found = packages_.find(id);
if (found == packages_.end() || found->second.empty())
dormant.push_back(id);
}
return dormant;
}
const ModPackage* ModPackageManager::selected_package(const std::string& id) const {
const auto selection = selections_.find(id);
const ModSelection blank;
runtime/src/mod_runtime.cpp
+8
View File
@@ -1167,6 +1167,14 @@ bool mod_runtime_initialize(const std::filesystem::path& root,
for (const std::string& scan_error : s.manager.scan_errors())
std::fprintf(stderr, "psxrecomp: mod manifest ignored: %s\n",
scan_error.c_str());
/* state.toml may name a package this catalog does not hold (removed,
* stripped from a release, or a builtin the title excludes). resolve()
* never visits it and save_state() keeps it; say so once. */
for (const std::string& dormant : s.manager.dormant_selections())
std::fprintf(stdout,
"psxrecomp: mod selection kept but inactive: %s is not "
"in this build's mod catalog\n",
dormant.c_str());
if (!sha256_file(exe_path, s.exe_sha256, &s.error)) {
/* Release installs commonly do not carry a loose PS-X EXE; game-id and
* expected-byte guards remain available in that case. */
runtime/tests/test_mod_catalog_layout.py
+165
View File
@@ -85,6 +85,7 @@ def run_guard(
*,
require_staged: int = 1,
alt_manifests: list[Path] | None = None,
excluded: Path | None = None,
label: str = "psx-runtime",
) -> subprocess.CompletedProcess[str]:
argv = [
@@ -94,6 +95,8 @@ def run_guard(
f"-DPSX_REQUIRE_STAGED={require_staged}",
f"-DPSX_LABEL={label}",
]
if excluded is not None:
argv.append(f"-DPSX_CATALOG_EXCLUDED={excluded.as_posix()}")
if alt_manifests:
joined = "|".join(p.as_posix() for p in alt_manifests)
argv.append(f"-DPSX_CATALOG_ALT_MANIFESTS={joined}")
@@ -101,6 +104,46 @@ def run_guard(
return subprocess.run(argv, capture_output=True, text=True)
SELECT = REPO / "runtime" / "psx_mod_catalog_select.cmake"
def run_select(
cmake: str,
tmp: Path,
*,
available: list[str],
exclude: list[str] = (),
title: list[str] = (),
allowlist: list[str] = (),
) -> tuple[subprocess.CompletedProcess[str], list[str] | None]:
"""Drive the REAL builtin selection (psx_mod_catalog_select.cmake) that
runtime.cmake's _psxrt_stage_mod_catalog calls, through `cmake -P`."""
def cm_list(name: str, ids) -> str:
return f" {name} " + " ".join(ids) if ids else f" {name}"
driver = tmp / "select_driver.cmake"
driver.write_text(
f'include("{SELECT.as_posix()}")\n'
"psx_select_builtin_mod_ids(_out LABEL psx-runtime"
+ cm_list("AVAILABLE", available)
+ cm_list("ALLOWLIST", allowlist)
+ cm_list("EXCLUDE", exclude)
+ cm_list("TITLE", title)
+ ")\n"
'list(JOIN _out "," _joined)\n'
'message(STATUS "SELECTED=[${_joined}]")\n',
encoding="utf-8",
)
r = subprocess.run([cmake, "-P", str(driver)], capture_output=True, text=True)
selected = None
for line in (r.stdout + r.stderr).splitlines():
if "SELECTED=[" in line:
body = line.split("SELECTED=[", 1)[1].rsplit("]", 1)[0]
selected = [s for s in body.split(",") if s]
return r, selected
def main() -> int:
ap = argparse.ArgumentParser()
ap.add_argument("--cmake", default=shutil.which("cmake") or "cmake")
@@ -241,6 +284,111 @@ def main() -> int:
"does NOT accept a catalog matching neither target's id set",
)
# ---- 9. EXCLUDE_BUILTIN_MODS: builtin selection ------------------
# WipEout 3 ships its own team-mark bezel and has no use for host-paced
# fast loading, so it declines both framework builtins. Excluded means
# ABSENT from mods/bundled, and a typo must never read as success.
print("[9] EXCLUDE_BUILTIN_MODS builtin selection")
builtins = sorted(FW_IDS + ["psx.enhancement.8mb-ram"])
r, sel = run_select(args.cmake, tmp, available=builtins)
check(r.returncode == 0 and sel == builtins,
"no exclusion stages every builtin")
r, sel = run_select(
args.cmake, tmp, available=builtins,
exclude=["psx.enhancement.fast-loading", "psx.presentation.bezel"],
title=["wipeout3.enhancement.bezel"])
check(r.returncode == 0, "a valid exclusion configures")
check(sel == ["psx.enhancement.8mb-ram", "psx.enhancement.cd-speed",
"psx.enhancement.pgxp"],
f"excluded builtins are dropped, the rest kept in order (got {sel})")
r, sel = run_select(
args.cmake, tmp, available=builtins,
exclude=["psx.enhancement.fast-loading",
"psx.enhancement.fast-loading"])
check(r.returncode == 0 and sel is not None
and "psx.enhancement.fast-loading" not in sel
and len(sel) == len(builtins) - 1,
"a repeated id is excluded once, not an error")
r, sel = run_select(
args.cmake, tmp, available=builtins,
exclude=["psx.enhancement.fast-laoding"])
out = r.stdout + r.stderr
check(r.returncode != 0, "an unknown excluded id FAILS configure")
check("psx.enhancement.fast-laoding" in out, "names the unknown id")
check("psx.enhancement.fast-loading" in out,
"lists the real builtins so the typo is obvious")
check(sel is None, "produces no selection after the failure")
r, sel = run_select(
args.cmake, tmp, available=builtins,
exclude=["wipeout3.enhancement.bezel"],
title=["wipeout3.enhancement.bezel"])
check(r.returncode != 0,
"excluding a TITLE package (not a builtin) FAILS configure")
r, sel = run_select(
args.cmake, tmp, available=builtins,
exclude=["psx.presentation.bezel"],
title=["psx.presentation.bezel"])
out = r.stdout + r.stderr
check(r.returncode != 0,
"excluding a builtin the title also overrides FAILS configure")
check("OVERRIDES" in out, "explains the exclude/override contradiction")
r, sel = run_select(
args.cmake, tmp, available=builtins,
allowlist=["psx.enhancement.pgxp", "psx.presentation.bezel"],
exclude=["psx.presentation.bezel"])
check(r.returncode == 0 and sel == ["psx.enhancement.pgxp"],
f"exclusion applies on top of PSX_BUILTIN_MOD_ALLOWLIST (got {sel})")
r, sel = run_select(
args.cmake, tmp, available=[],
exclude=["psx.presentation.bezel"])
check(r.returncode != 0,
"an exclusion with no builtin catalog at all FAILS configure")
# psxrecomp_add_game_runtime forwards keywords it does not parse, so
# its EXCLUDE_BUILTIN_MODS list can swallow one written after it.
split = tmp / "split_driver.cmake"
split.write_text(
f'include("{SELECT.as_posix()}")\n'
"psx_split_exclude_builtin_mods(_ids _tail psx.enhancement.fast-loading "
"psx.presentation.bezel GAME_OVERLAY_STATIC_C generated/x.c APP_ICON a.ico)\n"
'list(JOIN _ids "," _i)\nlist(JOIN _tail "," _t)\n'
'message(STATUS "IDS=[${_i}] TAIL=[${_t}]")\n', encoding="utf-8")
r = subprocess.run([args.cmake, "-P", str(split)], capture_output=True, text=True)
out = r.stdout + r.stderr
check(r.returncode == 0
and "IDS=[psx.enhancement.fast-loading,psx.presentation.bezel]" in out
and "TAIL=[GAME_OVERLAY_STATIC_C,generated/x.c,APP_ICON,a.ico]" in out,
"a forwarded keyword after EXCLUDE_BUILTIN_MODS is handed back, not "
"swallowed as a package id")
# ---- 10. build-time guard: excluded ids must stay absent ---------
print("[10] build-time guard over excluded builtins")
kept = sorted(i for i in all_ids if i != "psx.presentation.bezel")
kept_manifest = write_manifest(tmp / "expected_excl.txt", kept)
excluded_file = tmp / "expected_excl.excluded.txt"
excluded_file.write_text(
"# framework builtin mod packages psx-runtime excludes\n"
"psx.presentation.bezel\n", encoding="utf-8")
mods = tmp / "excluded_ok" / "mods"
for pid in kept:
make_package(mods / "bundled", pid)
r = run_guard(args.cmake, mods, kept_manifest, excluded=excluded_file)
check(r.returncode == 0, "accepts a catalog with the excluded builtin absent")
mods = tmp / "excluded_back" / "mods"
for pid in kept + ["psx.presentation.bezel"]:
make_package(mods / "bundled", pid)
r = run_guard(args.cmake, mods, kept_manifest, excluded=excluded_file)
out = r.stdout + r.stderr
check(r.returncode != 0,
"FAILS the build when an excluded builtin reappears in mods/bundled")
check("psx.presentation.bezel" in out and "EXCLUDES" in out,
"names the resurrected package and why it is wrong")
sibling = write_manifest(tmp / "expected_sib.txt", all_ids)
r = run_guard(args.cmake, mods, kept_manifest, excluded=excluded_file,
alt_manifests=[sibling])
check(r.returncode == 0,
"a sibling target that ships the id may own the shared directory")
# ---- 8. source invariants in runtime.cmake ---------------------------
print("[8] runtime.cmake source invariants")
text = RUNTIME_CMAKE.read_text(encoding="utf-8", errors="replace")
@@ -262,6 +410,23 @@ def main() -> int:
"psx_check_mod_catalog.cmake" in text,
"runtime.cmake wires the guard script into the build",
)
check(
"psx_mod_catalog_select.cmake" in text
and "psx_select_builtin_mod_ids(" in text,
"runtime.cmake stages builtins through the tested selection function",
)
check(
text.count("EXCLUDE_BUILTIN_MODS)") >= 2
and "${PSXRT_EXCLUDE_BUILTIN_MODS}" in text
and "psx_split_exclude_builtin_mods(_psxg_exclude_ids" in text
and "EXCLUDE_BUILTIN_MODS ${_psxg_exclude_ids}" in text,
"both psxrecomp_add_runtime_target and psxrecomp_add_game_runtime "
"parse EXCLUDE_BUILTIN_MODS as a multi-value arg and pass it on",
)
check(
"-DPSX_CATALOG_EXCLUDED=" in text and ".excluded.txt" in text,
"the excluded-id list is published and handed to the build-time guard",
)
forwarded_args = "${_psxg_forwarded_args}"
wrapper_extras = "EXTRAS_SOURCES ${_psxg_extras}"
check(
runtime/tests/test_mod_packages.cpp
+100
View File
@@ -1597,6 +1597,106 @@ int main() {
"an unknown channel name must be rejected");
}
/* A stale state.toml naming a package the catalog no longer holds: a
* framework builtin the title now excludes (EXCLUDE_BUILTIN_MODS), a
* developer package a release stripped, or one the player deleted. The
* selection must be dormant -- not a launch error the player cannot clear
* from a Mods page that no longer lists the package, never applied, and
* never silently thrown away. */
{
const fs::path stale_root = root / "stale";
const std::string kept =
"format_version = 5\n"
"id = \"kept.mod\"\n"
"version = \"1.0.0\"\n"
"name = \"Kept\"\n"
"[[target]]\n"
"game_id = \"*\"\n"
"[[feature]]\n"
"id = \"on\"\n"
"name = \"On\"\n"
"[[patch]]\n"
"feature = \"on\"\n"
"target = \"main_exe\"\n"
"address = 2147487744\n"
"expected = \"01 02 03 04\"\n"
"replace = \"05 06 07 08\"\n";
write_text(stale_root / "bundled/kept.mod/1.0.0/manifest.toml", kept);
write_text(stale_root / "state.toml",
"format_version = 2\n"
"\n[[package]]\nid = \"kept.mod\"\n"
"\n[[package]]\nid = \"psx.enhancement.fast-loading\"\n"
"\n[[package]]\nid = \"psx.presentation.bezel\"\n"
"version = \"1.0.0\"\n"
"\n[[feature]]\npackage_id = \"kept.mod\"\nid = \"on\"\n"
"enabled = true\n"
"\n[[feature]]\npackage_id = \"psx.enhancement.fast-loading\"\n"
"id = \"fast-loading\"\nenabled = true\n"
"[feature.values]\nmultiplier = \"8\"\n"
"\n[[feature]]\npackage_id = \"psx.presentation.bezel\"\n"
"id = \"bezel\"\nenabled = true\n"
"[feature.resources]\nartwork = \"C:/art.png\"\n");
ModPackageManager stale(stale_root);
check(stale.scan(&error), error.c_str());
check(stale.load_state(&error), error.c_str());
const ModResolution plan = stale.resolve("SLUS-TEST");
check(plan.ok, "a selection naming an absent package must not fail resolution");
check(plan.ordered.size() == 1 && plan.ordered[0]->id == "kept.mod",
"only the present, enabled package may enter the plan");
check(plan.writes.size() == 1,
"the present package's operations must still resolve");
const std::vector<std::string> dormant = stale.dormant_selections();
check(dormant.size() == 2 &&
std::find(dormant.begin(), dormant.end(),
"psx.enhancement.fast-loading") != dormant.end() &&
std::find(dormant.begin(), dormant.end(),
"psx.presentation.bezel") != dormant.end(),
"both absent packages must be reported as dormant selections");
check(std::find(dormant.begin(), dormant.end(), "kept.mod") == dormant.end(),
"a present package is never dormant");
/* Saving (every launch commit does) must keep the dormant choices. */
check(stale.save_state(&error), error.c_str());
ModPackageManager reread(stale_root);
check(reread.scan(&error), error.c_str());
check(reread.load_state(&error), error.c_str());
const auto& sel = reread.selections();
const auto fast = sel.find("psx.enhancement.fast-loading");
check(fast != sel.end() &&
fast->second.features.count("fast-loading") == 1 &&
fast->second.features.at("fast-loading").enabled &&
fast->second.features.at("fast-loading").values.at("multiplier") == "8",
"a dormant selection must survive save_state verbatim");
const auto bezel = sel.find("psx.presentation.bezel");
check(bezel != sel.end() && bezel->second.version == "1.0.0" &&
bezel->second.features.count("bezel") == 1 &&
bezel->second.features.at("bezel").resources.at("artwork") ==
"C:/art.png",
"a dormant pinned version and resource must survive save_state");
check(reread.resolve("SLUS-TEST").fingerprint == plan.fingerprint,
"dormant selections must not perturb the plan fingerprint");
/* When the package comes back, the preserved choice applies again. */
write_text(stale_root / "bundled/psx.enhancement.fast-loading/1.0.0/manifest.toml",
"format_version = 5\n"
"id = \"psx.enhancement.fast-loading\"\n"
"version = \"1.0.0\"\n"
"name = \"Fast\"\n"
"[[target]]\n"
"game_id = \"*\"\n"
"[[feature]]\n"
"id = \"fast-loading\"\n"
"name = \"Fast\"\n");
ModPackageManager restored(stale_root);
check(restored.scan(&error), error.c_str());
check(restored.load_state(&error), error.c_str());
check(restored.feature_enabled("psx.enhancement.fast-loading", "fast-loading"),
"a restored package must pick its preserved selection back up");
check(restored.dormant_selections().size() == 1,
"only the still-absent package remains dormant");
}
fs::remove_all(root, ec);
if (failures) {
std::cerr << failures << " mod package test(s) failed\n";
runtime/tests/test_release_stage_cache.py
+43
View File
@@ -254,6 +254,49 @@ class ModCatalogStagingTest(unittest.TestCase):
self.assertFalse(os.path.isdir(
os.path.join(self.stage, 'mods', 'installed')))
def _exclude(self, *ids):
return touch(
os.path.join(self.build, 'psx_mod_catalog_psx-runtime.excluded.txt'),
'# framework builtin mod packages psx-runtime excludes\n'
+ ''.join('%s\n' % i for i in ids))
def _framework_source(self):
src = os.path.join(self.tmp, 'fw_builtin')
for i in ('psx.enhancement.cd-speed', 'psx.enhancement.pgxp',
'psx.enhancement.fast-loading', 'psx.presentation.bezel'):
touch(os.path.join(src, 'packages', i, '1.0.0', 'manifest.toml'))
return src
def test_excluded_builtins_are_not_missing(self):
# Add-ModCatalog hands the framework's mods/builtin over as a
# --mod-source; the builtins the title declined with
# EXCLUDE_BUILTIN_MODS must count as absent, not as missing.
self._exclude('psx.enhancement.fast-loading', 'psx.presentation.bezel')
n = rs.stage_mods(self.build, self.stage, runtime_target='psx-runtime',
extra_sources=[self._framework_source()],
log=self.log)
self.assertEqual(n, 3)
self.assertTrue(any('verified absent' in m and 'psx.presentation.bezel' in m
for m in self.logged), self.logged)
def test_framework_source_without_exclusions_still_demands_every_builtin(self):
self._exclude() # published, but empty
with self.assertRaises(rs.StageError) as cm:
rs.stage_mods(self.build, self.stage, runtime_target='psx-runtime',
extra_sources=[self._framework_source()],
log=self.log)
self.assertIn('psx.enhancement.fast-loading', str(cm.exception))
def test_an_excluded_builtin_that_reached_the_catalog_is_refused(self):
self._exclude('psx.presentation.bezel')
touch(os.path.join(self.build, 'mods', 'bundled',
'psx.presentation.bezel', 'manifest.toml'))
with self.assertRaises(rs.StageError) as cm:
rs.stage_mods(self.build, self.stage, runtime_target='psx-runtime',
log=self.log)
self.assertIn('EXCLUDES', str(cm.exception))
self.assertIn('psx.presentation.bezel', str(cm.exception))
def test_unverifiable_catalog_is_refused(self):
os.remove(self.manifest)
with self.assertRaises(rs.StageError) as cm:
tools/release_stage.py
+43 -2
View File
@@ -796,6 +796,28 @@ def _find_catalog_manifest(build_path, runtime_target):
return None
def _excluded_manifest_for(manifest):
"""psx_mod_catalog_<target>.txt -> psx_mod_catalog_<target>.excluded.txt.
runtime.cmake writes the pair side by side: the ids the target staged, and
the framework builtins it declined with EXCLUDE_BUILTIN_MODS.
"""
if not manifest:
return None
root, ext = os.path.splitext(manifest)
return root + '.excluded' + (ext or '.txt')
def _read_id_list(path):
ids = set()
with open(path) as f:
for ln in f:
ln = ln.strip()
if ln and not ln.startswith('#'):
ids.add(ln)
return ids
def stage_mods(build_path, stage, runtime_target=None, catalog_manifest=None,
extra_sources=(), log=print):
mods_src = os.path.join(build_path, 'mods')
@@ -834,16 +856,23 @@ def stage_mods(build_path, stage, runtime_target=None, catalog_manifest=None,
manifest = _find_catalog_manifest(build_path, runtime_target)
want = set()
origin = None
excluded = set()
if manifest and os.path.isfile(manifest):
with open(manifest) as f:
want = {ln.strip() for ln in f if ln.strip()}
want = _read_id_list(manifest)
origin = manifest
excluded_manifest = _excluded_manifest_for(manifest)
if excluded_manifest and os.path.isfile(excluded_manifest):
excluded = _read_id_list(excluded_manifest)
for src in extra_sources:
pkgs = os.path.join(src, 'packages')
if os.path.isdir(pkgs):
want |= {d for d in os.listdir(pkgs)
if os.path.isdir(os.path.join(pkgs, d))}
origin = origin or 'source trees'
# A --mod-source tree (the framework's mods/builtin, typically) lists every
# builtin; the ones this title declined with EXCLUDE_BUILTIN_MODS are
# meant to be absent, not missing.
want -= excluded
if not want:
_die('cannot verify the mod catalog: neither the build-published '
'manifest (psx_mod_catalog_<target>.txt in %s) nor any --mod-source '
@@ -858,11 +887,23 @@ def stage_mods(build_path, stage, runtime_target=None, catalog_manifest=None,
'Mods page the dev build does not have.'
% (', '.join(missing), origin, staged_pkg_dir))
shipped_excluded = sorted(excluded & set(staged_ids))
if shipped_excluded:
_die('mod catalog contains package(s) the title EXCLUDES '
'(EXCLUDE_BUILTIN_MODS, per %s): %s. The build never stages an '
'excluded builtin, so something else put it into %s; the release '
'would ship a package the title declined.'
% (_excluded_manifest_for(manifest), ', '.join(shipped_excluded),
staged_pkg_dir))
fw = [i for i in staged_ids if i.startswith('psx.')]
game = [i for i in staged_ids if not i.startswith('psx.')]
log('Bundled mod catalog: %d package(s) = %d game-owned + %d '
'framework-owned (verified against %s)'
% (len(staged_ids), len(game), len(fw), origin))
if excluded:
log('Excluded framework builtin(s), verified absent: %s'
% ', '.join(sorted(excluded)))
return len(staged_ids)