mirror/psxrecomp
1
0
Fork 0
mirror of https://github.com/RetroPortingToolKit/psxrecomp synced 2026-10-26 18:25:35 -04:00
Code Issues Packages Projects Releases Wiki Activity

rebump

Browse Source
This commit is contained in:
Alex Vanderveen
2026-09-02 13:26:06 -04:00
parent b420752574
commit 4cc04be3e6
13 changed files with 957 additions and 91 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/MOD_PACKAGES.md
+88 -17
View File
@@ -12,10 +12,49 @@ The player selects a verified stock BIN/CUE. Resolution produces guarded native
operations and sparse disc overlays without rewriting or replacing that stock
image.
## Where packages live
Two catalog roots sit beside the executable, split by who owns the files:
```text
<exe>/mods/
bundled/ build output — the framework's mods/builtin/packages plus the
title's mods/preloaded/packages. Every build WIPES and re-stages
this tree, so nothing a player owns may live here.
installed/ launcher-owned — .psxmod archives installed through the Mods
manager. No build ever touches this tree.
state.toml user selection state (enabled features, option values).
```
Both roots use the same `<package-id>/<version>/manifest.toml` layout and are
scanned into one catalog, bundled first. An installed package with the same id
as a bundled one deliberately shadows it and records that it did so, so an
override is visible rather than decided by directory-iteration order.
A bundled package is not removable from the Mods page: deleting build output
would succeed and then be undone by the next build. Remove it from the title's
`mods/preloaded/packages` instead.
Only `bundled/` ships in a release. `installed/` and `state.toml` are the local
machine's and are excluded by every packager.
**Migration.** A pre-split install has one `mods/packages/` tree holding both.
The first scan moves each package the current build did not also stage into
`installed/`, drops the rest as redundant build output, and removes the old
tree — but only once every version directory has been dealt with. Anything that
could not be moved is left exactly where it is and reported.
A manifest that fails to parse is never skipped silently: it is reported by
`scan_errors()` and logged, naming the path and the reason.
## Feature manifest
Write new manifests at the current format version, which is **6**. Older
versions stay readable so installed packages survive an update, and each
section below notes the version a field first required.
```toml
format_version = 1
format_version = 6
id = "example.localization"
version = "1.2.0"
name = "Example Localization Pack"
@@ -261,31 +300,63 @@ Sparse fields and integer predicates are still pre-boot plan construction.
They do not provide a general expression evaluator, masks, arithmetic beyond
the checked field addend, package code execution, or per-frame dispatch.
## Developer channel
## Channels
A package may declare itself unfinished:
A feature declares how finished it is. Format 6 puts `channel` on the
**feature**, not the package:
```toml
format_version = 5
format_version = 6
id = "example.enhancements"
channel = "developer"
[[feature]]
id = "widescreen"
channel = "experimental" # ships, badged, default off
[[feature]]
id = "hook-trace"
channel = "developer" # absent from any release build
```
`channel = "developer"` marks work in progress. Such a package is staged next
to the executable by an ordinary build and ships in a locally exported
package, so it can be tested exactly as a player would see it — but release
packaging drops it, so unfinished work is never published.
| Channel | Ships | In the launcher |
|---|---|---|
| `stable` (default) | yes | no tag — the absence is the stable case |
| `experimental` | yes | amber `EXP` tag, and a line saying it is unvalidated |
| `developer` | **no** | secondary-accent `DEV` tag; only ever visible on a local build |
Packaging defaults the exclusion from `$CI`: on under any CI provider, off
An absent `channel` means `stable`. A package may still declare one, which its
features inherit unless they state their own — which is how a format-5 manifest
carrying a package-level `channel` keeps working unchanged.
**Why the feature and not the package.** A package is the installation and
**trust** boundary; how finished one of its features is has nothing to do with
that. When the marker sat on the package, a catalog holding one player-ready
feature and one developer instrument had to declare itself entirely
developer — so neither shipped, and the documented workaround was to split the
catalog into two packages. Channels per feature remove that trade.
**"Developer does not ship" means absent, not hidden.** Two mechanisms, one per
catalog root:
- **`bundled/` is filtered when it is staged.** `tools/mod_channel_filter.py`
emits a manifest without the developer features and without the `[[option]]`,
`[[patch]]`, `[[overlay]]`, `[[plugin]]`, `[[resource]]` and `[[constraint]]`
entries that only served them; a package whose every feature is developer has
its directory removed. This is generation, not rewriting: the staged catalog
is build output and the author's manifest in the repo is never touched.
- **`installed/` is refused at load.** A third-party archive is never modified,
so the runtime declines to surface developer features from one instead.
The runtime gate is the build definition `PSX_MOD_DEVELOPER_CHANNEL`, which
`runtime.cmake` sets for a local build and clears under `$CI`. A contributor
reaches developer features by cloning the repo and building; a release build
carries neither the features nor their operations, so a stale `state.toml`
naming one cannot reach them either.
Packaging defaults the exclusion from `$CI` — on under any CI provider, off
locally. `--exclude-dev-mods` / `--include-dev-mods` (or `EXCLUDE_DEV_MODS=0|1`)
override it, and `project_studio build export --exclude-dev-mods` reproduces
what a release would contain. Any absent `channel` means stable.
Pruning removes the whole package directory. A manifest is never rewritten
during packaging — a package's content stays exactly what its author shipped,
so the maturity marker is package-level. Split a player-ready feature into its
own stable package rather than promoting a catalog that still holds
in-progress work.
what a release would contain.
## Trusted static plugins
docs/ci/templates/setup-release.yml
+9 -2
View File
@@ -562,10 +562,10 @@ jobs:
# artifact, not the intent.
ZIP=$(ls dist/YOUR_ZIP_PREFIX-*-"${{ matrix.artifact }}".zip | head -1)
echo "checking ${ZIP}"
MANIFESTS=$(unzip -l "${ZIP}" | grep -c 'mods/packages/.*/manifest\.toml' || true)
MANIFESTS=$(unzip -l "${ZIP}" | grep -c 'mods/bundled/.*/manifest\.toml' || true)
echo "mod manifests in zip: ${MANIFESTS}"
if [[ "${MANIFESTS}" -lt 1 ]]; then
echo "::error::release zip ships no mod catalog (mods/packages/*/manifest.toml)" >&2
echo "::error::release zip ships no mod catalog (mods/bundled/*/manifest.toml)" >&2
unzip -l "${ZIP}" | head -40 >&2
exit 1
fi
@@ -576,6 +576,13 @@ jobs:
echo "::error::release zip contains mods/state.toml (per-machine state)" >&2
exit 1
fi
# Same rule, other half: mods/installed/ is the launcher-owned tree of
# .psxmod archives a developer installed as a player would. Only
# mods/bundled/ is build output, and only build output ships.
if unzip -l "${ZIP}" | grep -q 'mods/installed/'; then
echo "::error::release zip contains mods/installed (per-machine installs)" >&2
exit 1
fi
# Developer-channel packages are unfinished work that ships with
# local builds only. The packager drops them under CI automatically
# (it keys off $CI); assert the outcome so a future change to that
runtime/include/mod_packages.h
+77
View File
@@ -9,6 +9,36 @@
namespace PSXRecompV4 {
/* How finished a feature is, and therefore who is allowed to see it.
*
* Stable ships and is offered to everyone. Experimental ships, badged, and is
* default-off: it works, but has not been validated on this title. Developer
* does not ship at all -- contributors reach those features by cloning the
* repo and building locally.
*
* This lives on the FEATURE, not the package. A package is the trust and
* installation boundary; how finished one of its features is has nothing to do
* with that, and tying the two forced a whole catalog down to the maturity of
* its least finished feature. A package may still declare a channel, which its
* features inherit unless they state their own. */
enum class ModChannel {
Stable = 0,
Experimental = 1,
Developer = 2,
};
/* Whether this build may see developer-channel features. A local build may; a
* CI/release build may not, which is what "developer does not ship" means in
* practice. runtime.cmake defines PSX_MOD_DEVELOPER_CHANNEL for local builds. */
#if defined(PSX_MOD_DEVELOPER_CHANNEL) && PSX_MOD_DEVELOPER_CHANNEL
inline constexpr bool kDeveloperChannelDefault = true;
#else
inline constexpr bool kDeveloperChannelDefault = false;
#endif
const char* mod_channel_name(ModChannel channel);
bool mod_channel_from_name(const std::string& name, ModChannel& out);
enum class ModOptionType {
Boolean,
Choice,
@@ -45,6 +75,7 @@ struct ModFeature {
std::string author;
std::string description;
std::string group = "General";
ModChannel channel = ModChannel::Stable;
bool default_enabled = false;
bool hidden = false;
bool legacy = false;
@@ -190,6 +221,20 @@ struct ModAuthorLink {
std::string url;
};
/* Where a package's files came from, and therefore who owns them on disk.
*
* Bundled packages are build output: the framework and the title stage them
* into <exe>/mods/bundled, which every build wipes and re-stages. Nothing the
* player owns may live there. Installed packages are written by the launcher
* into <exe>/mods/installed and are never touched by a build.
*
* Before this split there was one <exe>/mods/packages tree serving both roles,
* so a rebuild deleted every mod the player had installed. */
enum class ModPackageOrigin {
Bundled,
Installed,
};
struct ModPackage {
uint32_t format_version = 0;
std::string id;
@@ -203,6 +248,11 @@ struct ModPackage {
std::string source_url;
std::string resolver = "declarative";
std::string save_compatibility = "shared";
/* Default channel for features that do not declare their own. */
ModChannel channel = ModChannel::Stable;
ModPackageOrigin origin = ModPackageOrigin::Installed;
/* Set when an installed package of the same id shadows a bundled one. */
bool shadows_bundled = false;
std::filesystem::path root;
std::vector<ModTarget> targets;
std::vector<ModRequirement> dependencies;
@@ -312,6 +362,18 @@ public:
void set_root(std::filesystem::path mods_root);
const std::filesystem::path& root() const { return root_; }
/* Developer-channel features are stripped at scan and install time unless
* this is set. Defaults to kDeveloperChannelDefault; tests set it
* explicitly so they exercise both a local and a release build. */
void set_developer_channel_visible(bool visible) {
developer_channel_ = visible;
}
bool developer_channel_visible() const { return developer_channel_; }
/* Build-owned catalog: wiped and re-staged by every build. */
std::filesystem::path bundled_root() const { return root_ / "bundled"; }
/* Launcher-owned catalog: a build never touches this tree. */
std::filesystem::path installed_root() const { return root_ / "installed"; }
bool scan(std::string* error = nullptr);
bool load_state(std::string* error = nullptr);
@@ -367,8 +429,23 @@ public:
static bool read_manifest(const std::filesystem::path& path, ModPackage& out,
std::string* error = nullptr);
/* Manifests that failed to parse during the last scan(), as
* "<path>: <reason>". A package that cannot be read must never vanish
* silently -- the launcher surfaces these so a mod author sees why their
* package is absent from the list. */
const std::vector<std::string>& scan_errors() const { return scan_errors_; }
private:
/* One-time move of a pre-split <exe>/mods/packages tree into the two
* owned roots. Anything the build also staged into bundled/ is dropped;
* everything else is the player's and moves to installed/. */
void migrate_legacy_root();
bool scan_root(const std::filesystem::path& packages_root,
ModPackageOrigin origin, std::string* error);
std::filesystem::path root_;
bool developer_channel_ = kDeveloperChannelDefault;
std::vector<std::string> scan_errors_;
std::map<std::string, std::map<std::string, ModPackage>> packages_;
std::map<std::string, ModSelection> selections_;
};
runtime/runtime.cmake
+28 -6
View File
@@ -1377,8 +1377,14 @@ function(psxrecomp_add_runtime_target target)
# Clear first: copy_directory MERGES, so a mod deleted from source
# would otherwise survive in the build output forever and keep
# appearing on the Mods page (and inflate release catalog assertions).
# Scoped to mods/packages, not mods/: state.toml is launcher-owned user
# state and must survive a rebuild.
#
# Scoped to mods/bundled, which is build output and nothing else.
# mods/installed/ is the launcher's (player-installed .psxmod archives)
# and mods/state.toml is user selection state; a build must never touch
# either. Before that split this wipe was scoped to mods/packages,
# which ALSO held everything the player had installed -- so a rebuild,
# including the one a self-compiling setup release runs on the player's
# own machine, deleted their mods without a word.
set(_psx_builtin_mod_copy_commands "")
if(DEFINED PSX_BUILTIN_MOD_ALLOWLIST AND NOT
"${PSX_BUILTIN_MOD_ALLOWLIST}" STREQUAL "")
@@ -1390,17 +1396,17 @@ function(psxrecomp_add_runtime_target target)
list(APPEND _psx_builtin_mod_copy_commands
COMMAND ${CMAKE_COMMAND} -E copy_directory
"${PSXRECOMP_ROOT}/mods/builtin/packages/${_psx_builtin_mod}"
"$<TARGET_FILE_DIR:${target}>/mods/packages/${_psx_builtin_mod}")
"$<TARGET_FILE_DIR:${target}>/mods/bundled/${_psx_builtin_mod}")
endforeach()
else()
list(APPEND _psx_builtin_mod_copy_commands
COMMAND ${CMAKE_COMMAND} -E copy_directory
"${PSXRECOMP_ROOT}/mods/builtin"
"$<TARGET_FILE_DIR:${target}>/mods")
"${PSXRECOMP_ROOT}/mods/builtin/packages"
"$<TARGET_FILE_DIR:${target}>/mods/bundled")
endif()
add_custom_command(TARGET ${target} POST_BUILD
COMMAND ${CMAKE_COMMAND} -E rm -rf
"$<TARGET_FILE_DIR:${target}>/mods/packages"
"$<TARGET_FILE_DIR:${target}>/mods/bundled"
${_psx_builtin_mod_copy_commands}
COMMENT "Staging framework-owned mod catalog"
VERBATIM)
@@ -1419,6 +1425,22 @@ function(psxrecomp_add_runtime_target target)
if(PSX_SHELLWIN_INTERP)
target_compile_definitions(${target} PRIVATE PSX_SHELLWIN_INTERP_DEFAULT=1)
endif()
# Developer-channel mod features do not ship. A contributor reaches them by
# cloning the repo and building locally; a release build must not carry
# them at all -- not hidden behind a toggle, absent. Keying the default off
# $CI matches what the packagers already do for EXCLUDE_DEV_MODS, so one
# rule covers the catalog on disk and the catalog in the binary.
if(NOT DEFINED PSX_MOD_DEVELOPER_CHANNEL)
if(DEFINED ENV{CI} AND NOT "$ENV{CI}" STREQUAL "")
set(PSX_MOD_DEVELOPER_CHANNEL OFF)
else()
set(PSX_MOD_DEVELOPER_CHANNEL ON)
endif()
endif()
if(PSX_MOD_DEVELOPER_CHANNEL)
target_compile_definitions(${target} PRIVATE PSX_MOD_DEVELOPER_CHANNEL=1)
endif()
if(has_game_dispatch)
target_compile_definitions(${target} PRIVATE PSX_HAS_GAME_DISPATCH=1)
endif()
runtime/src/mod_packages.cpp
+195 -8
View File
@@ -22,7 +22,7 @@ namespace PSXRecompV4 {
namespace {
constexpr uint32_t kMinFormatVersion = 1;
constexpr uint32_t kMaxFormatVersion = 5;
constexpr uint32_t kMaxFormatVersion = 6;
constexpr uint64_t kMaxArchiveBytes = 256ull * 1024ull * 1024ull;
constexpr uint32_t kMaxArchiveFiles = 4096;
@@ -1471,6 +1471,65 @@ void mod_clear_plugins_for_tests() {
registered_plugins().clear();
}
const char* mod_channel_name(ModChannel channel) {
switch (channel) {
case ModChannel::Experimental: return "experimental";
case ModChannel::Developer: return "developer";
case ModChannel::Stable: break;
}
return "stable";
}
bool mod_channel_from_name(const std::string& name, ModChannel& out) {
if (name == "stable") { out = ModChannel::Stable; return true; }
if (name == "experimental") { out = ModChannel::Experimental; return true; }
if (name == "developer") { out = ModChannel::Developer; return true; }
return false;
}
namespace {
/* Remove every developer-channel feature and everything that only existed to
* serve one. "Developer does not ship" means absent, not hidden: a release
* build must not carry the operations either, or a stale state.toml naming a
* dropped feature could still reach them.
*
* Returns false when nothing is left, so the caller can drop the package
* rather than list one with no features. */
bool strip_developer_features(ModPackage& package) {
std::set<std::string> dropped;
for (const ModFeature& feature : package.features)
if (feature.channel == ModChannel::Developer)
dropped.insert(feature.id);
if (dropped.empty()) return !package.features.empty();
const auto orphaned = [&dropped](const std::string& feature_id) {
return dropped.count(feature_id) != 0;
};
const auto prune = [&orphaned](auto& entries) {
entries.erase(std::remove_if(entries.begin(), entries.end(),
[&orphaned](const auto& entry) {
return orphaned(entry.feature_id);
}),
entries.end());
};
package.features.erase(
std::remove_if(package.features.begin(), package.features.end(),
[&orphaned](const ModFeature& feature) {
return orphaned(feature.id);
}),
package.features.end());
prune(package.options);
prune(package.constraints);
prune(package.patches);
prune(package.overlays);
prune(package.plugins);
prune(package.resources);
return !package.features.empty();
}
} // namespace
ModPackageManager::ModPackageManager(fs::path mods_root) : root_(std::move(mods_root)) {}
void ModPackageManager::set_root(fs::path mods_root) {
@@ -1517,6 +1576,16 @@ bool ModPackageManager::read_manifest(const fs::path& path, ModPackage& out,
cfg.contains("resolver") ? toml::find<std::string>(cfg, "resolver") : "declarative";
out.save_compatibility = cfg.contains("save_compatibility")
? toml::find<std::string>(cfg, "save_compatibility") : "shared";
/* Accepted at every format version: v5 manifests already carried a
* package-level channel, enforced then only by a packaging grep over
* the manifest text. Now it parses, and it seeds the per-feature
* default rather than quarantining the whole package. */
if (cfg.contains("channel")) {
const std::string name = toml::find<std::string>(cfg, "channel");
if (!mod_channel_from_name(name, out.channel))
throw std::runtime_error(
"channel must be stable, experimental or developer");
}
out.root = path.parent_path();
if (out.format_version < kMinFormatVersion ||
out.format_version > kMaxFormatVersion)
@@ -1573,6 +1642,16 @@ bool ModPackageManager::read_manifest(const fs::path& path, ModPackage& out,
feature.default_enabled =
toml::find_or<bool>(v, "default_enabled", false);
feature.hidden = toml::find_or<bool>(v, "hidden", false);
feature.channel = out.channel;
if (v.contains("channel")) {
if (out.format_version < 6)
throw std::runtime_error(
"per-feature channel requires format_version 6");
const std::string name = toml::find<std::string>(v, "channel");
if (!mod_channel_from_name(name, feature.channel))
throw std::runtime_error(
"channel must be stable, experimental or developer");
}
if (!valid_id(feature.id) ||
!feature_ids.insert(feature.id).second)
throw std::runtime_error("invalid or duplicate feature id");
@@ -1588,6 +1667,7 @@ bool ModPackageManager::read_manifest(const fs::path& path, ModPackage& out,
feature.name = out.name;
feature.author = out.author;
feature.description = out.description;
feature.channel = out.channel;
feature.legacy = true;
out.features.push_back(std::move(feature));
}
@@ -2305,10 +2385,68 @@ bool ModPackageManager::read_manifest(const fs::path& path, ModPackage& out,
}
}
bool ModPackageManager::scan(std::string* error) {
packages_.clear();
void ModPackageManager::migrate_legacy_root() {
std::error_code ec;
const fs::path legacy = root_ / "packages";
if (!fs::exists(legacy, ec)) return;
const fs::path installed = installed_root();
const fs::path bundled = bundled_root();
size_t stranded = 0;
for (const fs::directory_entry& id_dir : fs::directory_iterator(legacy, ec)) {
if (ec) break;
if (!id_dir.is_directory()) continue;
const std::string id = id_dir.path().filename().string();
for (const fs::directory_entry& version_dir :
fs::directory_iterator(id_dir.path(), ec)) {
if (ec) break;
if (!version_dir.is_directory()) continue;
const std::string version = version_dir.path().filename().string();
std::error_code move_ec;
/* Already staged by the build: this copy is redundant output, not
* anything the player owns. */
if (fs::exists(bundled / id / version, move_ec)) {
fs::remove_all(version_dir.path(), move_ec);
continue;
}
const fs::path destination = installed / id / version;
if (fs::exists(destination, move_ec)) {
fs::remove_all(version_dir.path(), move_ec);
continue;
}
fs::create_directories(destination.parent_path(), move_ec);
if (move_ec) { ++stranded; continue; }
fs::rename(version_dir.path(), destination, move_ec);
if (move_ec) {
/* Cross-device or a locked file: copy rather than lose it. */
move_ec.clear();
fs::copy(version_dir.path(), destination,
fs::copy_options::recursive, move_ec);
if (move_ec) {
++stranded;
continue;
}
fs::remove_all(version_dir.path(), move_ec);
}
}
}
/* Drop the legacy tree only when every version directory was dealt with.
* A blanket remove_all here would be the very data loss this migration
* exists to end, so anything that could not be moved stays exactly where
* it is and is reported instead. */
if (stranded != 0 || ec) {
scan_errors_.push_back(
legacy.string() + ": could not migrate " +
std::to_string(stranded) + " package version(s) to " +
installed.string() + "; the old catalog was left in place");
return;
}
ec.clear();
fs::remove_all(legacy, ec);
}
bool ModPackageManager::scan_root(const fs::path& packages_root,
ModPackageOrigin origin, std::string* error) {
std::error_code ec;
const fs::path packages_root = root_ / "packages";
if (!fs::exists(packages_root, ec)) return true;
for (const fs::directory_entry& id_dir : fs::directory_iterator(packages_root, ec)) {
if (ec) break;
@@ -2321,13 +2459,34 @@ bool ModPackageManager::scan(std::string* error) {
ModPackage package;
std::string parse_error;
if (!read_manifest(manifest, package, &parse_error)) {
/* A package that cannot be parsed must say so. Silently
* skipping it leaves a mod author with a mod that simply does
* not exist and nothing anywhere explaining why. */
scan_errors_.push_back(manifest.string() + ": " + parse_error);
continue;
}
if (package.id != id_dir.path().filename().string() ||
package.version != version_dir.path().filename().string()) {
set_error(error, "package path does not match manifest id/version: " +
manifest.string());
return false;
scan_errors_.push_back(
manifest.string() +
": package path does not match manifest id/version");
continue;
}
if (!developer_channel_ && !strip_developer_features(package)) {
/* Every feature was developer-channel, so on this build the
* package has nothing to offer. Not an error: it is the
* intended outcome of shipping without them. */
continue;
}
package.origin = origin;
if (origin == ModPackageOrigin::Installed) {
const auto existing = packages_.find(package.id);
package.shadows_bundled =
existing != packages_.end() &&
std::any_of(existing->second.begin(), existing->second.end(),
[](const std::pair<const std::string, ModPackage>& e) {
return e.second.origin == ModPackageOrigin::Bundled;
});
}
packages_[package.id][package.version] = std::move(package);
}
@@ -2339,6 +2498,18 @@ bool ModPackageManager::scan(std::string* error) {
return true;
}
bool ModPackageManager::scan(std::string* error) {
packages_.clear();
scan_errors_.clear();
migrate_legacy_root();
/* Bundled first, then installed: an installed package of the same id and
* version deliberately shadows the build-staged one, and records that it
* did so, rather than the two racing on directory-iteration order. */
if (!scan_root(bundled_root(), ModPackageOrigin::Bundled, error)) return false;
if (!scan_root(installed_root(), ModPackageOrigin::Installed, error)) return false;
return true;
}
bool ModPackageManager::load_state(std::string* error) {
selections_.clear();
const fs::path path = root_ / "state.toml";
@@ -2536,7 +2707,16 @@ bool ModPackageManager::install_archive(const fs::path& archive,
fs::remove_all(staging, ec);
return false;
}
const fs::path destination = root_ / "packages" / package.id / package.version;
if (!developer_channel_ && !strip_developer_features(package)) {
fs::remove_all(staging, ec);
set_error(error,
"this package contains only developer-channel features, "
"which are not available in this build");
return false;
}
/* Player installs land in the launcher-owned root. They must never share
* a tree with the build-staged catalog, which every build wipes. */
const fs::path destination = installed_root() / package.id / package.version;
if (fs::exists(destination)) {
fs::remove_all(staging, ec);
set_error(error, "package version is already installed");
@@ -2555,6 +2735,7 @@ bool ModPackageManager::install_archive(const fs::path& archive,
return false;
}
package.root = destination;
package.origin = ModPackageOrigin::Installed;
packages_[package.id][package.version] = package;
if (installed_id) *installed_id = package.id;
if (installed_version) *installed_version = package.version;
@@ -2595,6 +2776,12 @@ bool ModPackageManager::remove_version(const std::string& id, const std::string&
set_error(error, "package version is not installed");
return false;
}
if (pit->second.at(version).origin == ModPackageOrigin::Bundled) {
/* Deleting build output would succeed and then be undone by the next
* build, which is not a model any player can reason about. */
set_error(error, "cannot remove a bundled package; it ships with the game");
return false;
}
const fs::path path = pit->second.at(version).root;
std::error_code ec;
fs::remove_all(path, ec);
runtime/src/mod_runtime.cpp
+25 -3
View File
@@ -626,7 +626,10 @@ int provider_package_get(void*, int index, RecompLauncherCModPackage* out) {
copy_text(out->source_url, sizeof(out->source_url), package->source_url);
out->enabled = package_has_enabled_feature(*package);
out->option_count = (int)package->options.size();
out->removable = !out->enabled;
/* A bundled package is build output. Offering to remove it would succeed
* and then be silently undone by the next build. */
out->removable = !out->enabled &&
package->origin == ModPackageOrigin::Installed;
return 1;
}
@@ -742,6 +745,19 @@ int provider_feature_get(void*, int index, RecompLauncherCModFeature* out) {
copy_text(out->source_url, sizeof(out->source_url), package->source_url);
copy_text(out->group, sizeof(out->group), feature->group);
out->hidden = feature->hidden ? 1 : 0;
switch (feature->channel) {
case ModChannel::Experimental:
out->channel = RECOMP_MOD_CHANNEL_EXPERIMENTAL;
break;
case ModChannel::Developer:
/* Only reachable on a local developer build: a shipped catalog
* carries no developer-channel feature to report. */
out->channel = RECOMP_MOD_CHANNEL_DEVELOPER;
break;
case ModChannel::Stable:
out->channel = RECOMP_MOD_CHANNEL_STABLE;
break;
}
out->enabled =
state().manager.feature_enabled(package->id, feature->id) ? 1 : 0;
out->option_count =
@@ -957,8 +973,9 @@ int provider_version_get(void*, const char* package_id, int index,
copy_text(out->version, sizeof(out->version), version->first);
const ModPackage* selected = selected_package(package_id);
out->selected = selected && selected->version == version->first;
out->removable = !out->selected ||
!selected || !package_has_enabled_feature(*selected);
out->removable = (!out->selected || !selected ||
!package_has_enabled_feature(*selected)) &&
version->second.origin == ModPackageOrigin::Installed;
return 1;
}
@@ -1107,6 +1124,11 @@ bool mod_runtime_initialize(const std::filesystem::path& root,
if (error) *error = s.error;
return false;
}
/* A manifest that fails to parse used to be skipped in silence, so a mod
* author's typo produced a mod that simply did not exist. Name every one. */
for (const std::string& scan_error : s.manager.scan_errors())
std::fprintf(stderr, "psxrecomp: mod manifest ignored: %s\n",
scan_error.c_str());
if (!sha256_file(exe_path, s.exe_sha256, &s.error)) {
/* Release installs commonly do not carry a loose PS-X EXE; game-id and
* expected-byte guards remain available in that case. */
runtime/tests/test_mod_packages.cpp
+194
View File
@@ -1356,6 +1356,200 @@ int main() {
check(!ModPackageManager::read_manifest(root / "bad.toml", invalid, &error),
"unsafe package id must be rejected");
/* Catalog roots. mods/bundled is build output that every build wipes and
* re-stages; mods/installed belongs to the launcher and a build never
* touches it. Before the split both lived in mods/packages, so a rebuild
* deleted every mod the player had installed. */
{
const fs::path split_root = root / "split";
/* Pre-split layout: a build-staged package and a player install,
* indistinguishable in the same tree. */
write_text(split_root / "packages/psx.builtin/1.0.0/manifest.toml",
manifest("psx.builtin", "1.0.0"));
write_text(split_root / "packages/player.mod/1.0.0/manifest.toml",
manifest("player.mod", "1.0.0"));
/* The current build has already staged its own copy of the builtin. */
write_text(split_root / "bundled/psx.builtin/1.0.0/manifest.toml",
manifest("psx.builtin", "1.0.0"));
ModPackageManager split(split_root);
check(split.scan(&error), error.c_str());
check(!fs::exists(split_root / "packages"),
"the legacy mods/packages tree must be migrated away");
check(fs::exists(split_root / "installed/player.mod/1.0.0/manifest.toml"),
"a player-installed package must migrate into mods/installed");
check(!fs::exists(split_root / "installed/psx.builtin"),
"a package the build also staged must not migrate to installed");
check(split.packages().at("psx.builtin").at("1.0.0").origin ==
ModPackageOrigin::Bundled,
"a package under mods/bundled must be Bundled");
check(split.packages().at("player.mod").at("1.0.0").origin ==
ModPackageOrigin::Installed,
"a migrated player package must be Installed");
check(!split.remove_version("psx.builtin", "1.0.0", &error),
"a bundled package must not be removable");
/* What a rebuild does: wipe bundled/ and re-stage it. The player's
* installed catalog must survive untouched. */
fs::remove_all(split_root / "bundled", ec);
write_text(split_root / "bundled/psx.builtin/1.0.0/manifest.toml",
manifest("psx.builtin", "1.0.0"));
ModPackageManager rebuilt(split_root);
check(rebuilt.scan(&error), error.c_str());
check(rebuilt.packages().count("player.mod") == 1,
"a rebuild must not delete a player-installed package");
check(rebuilt.packages().count("psx.builtin") == 1,
"a rebuild must re-stage the bundled catalog");
/* An installed package of the same id shadows the bundled one, and
* says so rather than winning silently. */
write_text(split_root / "installed/psx.builtin/1.0.0/manifest.toml",
manifest("psx.builtin", "1.0.0"));
ModPackageManager shadowed(split_root);
check(shadowed.scan(&error), error.c_str());
const ModPackage& shadow =
shadowed.packages().at("psx.builtin").at("1.0.0");
check(shadow.origin == ModPackageOrigin::Installed &&
shadow.shadows_bundled,
"an installed package must shadow the bundled one and record it");
/* A manifest that cannot be parsed must be named, never skipped in
* silence -- an author's typo used to produce a mod that simply did
* not exist, with nothing anywhere explaining why. */
write_text(split_root / "installed/broken.mod/1.0.0/manifest.toml",
"format_version = 5\nid = \"broken.mod\"\n");
ModPackageManager broken(split_root);
check(broken.scan(&error), error.c_str());
check(broken.packages().count("broken.mod") == 0,
"an unparseable package must not load");
check(std::any_of(broken.scan_errors().begin(),
broken.scan_errors().end(),
[](const std::string& e) {
return e.find("broken.mod") != std::string::npos;
}),
"an unparseable manifest must be reported by scan_errors()");
}
/* Channels. A feature carries its own maturity: a package is the trust and
* installation boundary, and tying the two forced a whole catalog down to
* the maturity of its least finished feature. */
{
const fs::path chan_root = root / "channel";
const std::string mixed =
"format_version = 6\n"
"id = \"chan.mod\"\n"
"version = \"1.0.0\"\n"
"name = \"Channel Mod\"\n"
"[[target]]\n"
"game_id = \"SLUS-TEST\"\n"
"[[feature]]\n"
"id = \"solid\"\n"
"name = \"Solid\"\n"
"[[feature]]\n"
"id = \"rough\"\n"
"name = \"Rough\"\n"
"channel = \"experimental\"\n"
"[[feature]]\n"
"id = \"probe\"\n"
"name = \"Probe\"\n"
"channel = \"developer\"\n"
"[[option]]\n"
"feature = \"probe\"\n"
"id = \"depth\"\n"
"label = \"Depth\"\n"
"type = \"boolean\"\n"
"default = \"false\"\n";
write_text(chan_root / "bundled/chan.mod/1.0.0/manifest.toml", mixed);
/* A local build sees all three. */
ModPackageManager local(chan_root);
local.set_developer_channel_visible(true);
check(local.scan(&error), error.c_str());
const ModPackage& all = local.packages().at("chan.mod").at("1.0.0");
check(all.features.size() == 3, "a local build must see every channel");
check(all.features[0].channel == ModChannel::Stable &&
all.features[1].channel == ModChannel::Experimental &&
all.features[2].channel == ModChannel::Developer,
"each feature must carry its own channel");
/* A release build must not carry the developer feature at all --
* absent, not hidden -- nor the options that only served it. */
ModPackageManager shipped(chan_root);
shipped.set_developer_channel_visible(false);
check(shipped.scan(&error), error.c_str());
const ModPackage& published = shipped.packages().at("chan.mod").at("1.0.0");
check(published.features.size() == 2,
"a release build must drop developer-channel features");
check(std::none_of(published.features.begin(), published.features.end(),
[](const ModFeature& f) { return f.id == "probe"; }),
"the developer feature must be absent from a release build");
check(std::none_of(published.options.begin(), published.options.end(),
[](const ModOption& o) { return o.feature_id == "probe"; }),
"options serving a dropped feature must go with it");
check(published.features[1].channel == ModChannel::Experimental,
"an experimental feature still ships");
/* A package whose every feature is developer-channel simply is not
* there on a release build, rather than listing with nothing in it. */
write_text(chan_root / "bundled/dev.only/1.0.0/manifest.toml",
"format_version = 6\n"
"id = \"dev.only\"\n"
"version = \"1.0.0\"\n"
"name = \"Dev Only\"\n"
"channel = \"developer\"\n"
"[[target]]\n"
"game_id = \"SLUS-TEST\"\n"
"[[feature]]\n"
"id = \"instrument\"\n"
"name = \"Instrument\"\n");
ModPackageManager dev_only(chan_root);
dev_only.set_developer_channel_visible(false);
check(dev_only.scan(&error), error.c_str());
check(dev_only.packages().count("dev.only") == 0,
"a developer-only package must not load in a release build");
ModPackageManager dev_seen(chan_root);
dev_seen.set_developer_channel_visible(true);
check(dev_seen.scan(&error), error.c_str());
check(dev_seen.packages().count("dev.only") == 1,
"a developer-only package must load in a local build");
check(dev_seen.packages().at("dev.only").at("1.0.0")
.features[0].channel == ModChannel::Developer,
"a feature must inherit its package's channel by default");
/* The key is version-gated like every other format addition, and only
* the three names are accepted. */
ModPackage rejected;
write_text(chan_root / "v5-feature-channel.toml",
"format_version = 5\n"
"id = \"old.mod\"\n"
"version = \"1.0.0\"\n"
"name = \"Old\"\n"
"[[target]]\n"
"game_id = \"SLUS-TEST\"\n"
"[[feature]]\n"
"id = \"f\"\n"
"name = \"F\"\n"
"channel = \"experimental\"\n");
check(!ModPackageManager::read_manifest(
chan_root / "v5-feature-channel.toml", rejected, &error),
"a per-feature channel must require format_version 6");
write_text(chan_root / "bad-channel.toml",
"format_version = 6\n"
"id = \"bad.mod\"\n"
"version = \"1.0.0\"\n"
"name = \"Bad\"\n"
"channel = \"beta\"\n"
"[[target]]\n"
"game_id = \"SLUS-TEST\"\n"
"[[feature]]\n"
"id = \"f\"\n"
"name = \"F\"\n");
check(!ModPackageManager::read_manifest(
chan_root / "bad-channel.toml", rejected, &error),
"an unknown channel name must be rejected");
}
fs::remove_all(root, ec);
if (failures) {
std::cerr << failures << " mod package test(s) failed\n";
tools/mod_channel_filter.py
+255
View File
@@ -0,0 +1,255 @@
#!/usr/bin/env python3
"""Drop developer-channel mod features from a staged catalog tree.
Developer-channel work does not ship. A contributor reaches it by cloning the
repo and building locally; a release must not carry it at all -- absent, not
hidden behind a toggle.
Channels are per FEATURE, so pruning by package directory is no longer right:
a catalog may hold a stable feature and an instrument side by side, and dropping
the directory would take the player-facing feature with it. That is exactly what
a line-anchored `grep '^channel = "developer"'` does, because it cannot tell a
package-level key from one inside a [[feature]] block.
What this does instead:
* package-level channel = "developer", or every feature developer
-> remove the version directory (nothing in it ships)
* some features developer
-> emit a manifest without those features, and without the
[[option]] / [[patch]] / [[overlay]] / [[plugin]] / [[resource]] /
[[constraint]] entries that only served them
Emitting is not rewriting someone's archive. A staged catalog is build output
generated from the title's mods/preloaded/ source (or the framework's
mods/builtin/); the author's manifest is the one in the repo and is never
touched. Installed .psxmod archives are a different matter -- those are never
modified, and the runtime declines to surface developer features from them
instead (see PSX_MOD_DEVELOPER_CHANNEL).
Usage:
mod_channel_filter.py <catalog-dir> [<catalog-dir> ...] [--quiet]
A catalog dir directly contains <package-id>/<version>/manifest.toml. Missing
directories are skipped. Exits non-zero if any manifest could not be parsed or
if a filtered manifest failed to re-parse, so packaging fails closed rather
than publishing unfinished work.
"""
from __future__ import annotations
import argparse
import re
import shutil
import sys
from pathlib import Path
try:
import tomllib
except ModuleNotFoundError: # pragma: no cover - Python < 3.11
tomllib = None
try:
import tomli as tomllib # type: ignore[no-redef]
except ModuleNotFoundError:
pass
DEVELOPER = "developer"
# A top-level array-of-tables header starts a block. A dotted header
# ([[option.choice]]) belongs to the block above it and rides along with it.
_TABLE = re.compile(r"^\s*\[\[\s*([A-Za-z0-9_.]+)\s*\]\]\s*(?:#.*)?$")
# Only used on lines inside one block, so it cannot pick up a package-level key.
_FEATURE_KEY = re.compile(r"""^\s*feature\s*=\s*["']([^"']*)["']\s*(?:#.*)?$""")
_ID_KEY = re.compile(r"""^\s*id\s*=\s*["']([^"']*)["']\s*(?:#.*)?$""")
# Entries that name the feature they serve and are meaningless without it.
DEPENDENT_TABLES = {
"option", "patch", "overlay", "plugin", "resource", "constraint",
}
class ManifestError(Exception):
pass
def _split_blocks(text: str) -> tuple[str, list[tuple[str, str]]]:
"""Return (preamble, [(table-name, block-text), ...]).
The preamble is everything before the first top-level [[table]] -- the
package keys. Each block keeps its own trailing blank lines and any comment
lines that precede its header, so removing one leaves the rest untouched.
"""
lines = text.splitlines(keepends=True)
preamble: list[str] = []
blocks: list[tuple[str, list[str]]] = []
pending: list[str] = [] # comments/blanks that belong to the NEXT header
for line in lines:
match = _TABLE.match(line)
name = match.group(1) if match else None
if name is not None and "." not in name:
if blocks:
blocks[-1][1].extend([])
blocks.append((name, pending + [line]))
pending = []
continue
if not blocks:
if line.strip().startswith("#") or not line.strip():
pending.append(line)
else:
preamble.extend(pending)
pending = []
preamble.append(line)
continue
if line.strip().startswith("#") or not line.strip():
pending.append(line)
continue
blocks[-1][1].extend(pending)
pending = []
blocks[-1][1].append(line)
if blocks:
blocks[-1][1].extend(pending)
else:
preamble.extend(pending)
return "".join(preamble), [(n, "".join(b)) for n, b in blocks]
def _block_value(block: str, pattern: re.Pattern[str]) -> str | None:
for line in block.splitlines():
if _TABLE.match(line):
continue
match = pattern.match(line)
if match:
return match.group(1)
return None
def _block_channel(block: str) -> str | None:
for line in block.splitlines():
if _TABLE.match(line):
continue
match = re.match(r"""^\s*channel\s*=\s*["']([^"']*)["']\s*(?:#.*)?$""", line)
if match:
return match.group(1)
return None
def filter_manifest(text: str) -> tuple[str | None, list[str]]:
"""Return (filtered text or None to drop the package, dropped feature ids).
None means nothing in this package ships: the package itself is
developer-channel, or every feature it defines is.
"""
if tomllib is None:
raise ManifestError(
"no TOML parser available (need Python 3.11+, or tomli installed)")
try:
parsed = tomllib.loads(text)
except Exception as exc: # tomllib.TOMLDecodeError, and anything it raises
raise ManifestError(f"unparseable manifest: {exc}") from exc
package_channel = parsed.get("channel")
features = parsed.get("feature") or []
if package_channel == DEVELOPER:
return None, [str(f.get("id", "")) for f in features]
dropped = [
str(f.get("id", ""))
for f in features
if f.get("channel", package_channel) == DEVELOPER
]
if not dropped:
return text, []
if len(dropped) == len(features):
return None, dropped
preamble, blocks = _split_blocks(text)
kept: list[str] = []
dropped_option_ids: set[tuple[str, str]] = set()
for name, block in blocks:
if name == "feature":
if _block_value(block, _ID_KEY) in dropped:
continue
elif name in DEPENDENT_TABLES:
owner = _block_value(block, _FEATURE_KEY)
if owner in dropped:
if name == "option":
dropped_option_ids.add((owner, _block_value(block, _ID_KEY) or ""))
continue
kept.append(block)
filtered = preamble + "".join(kept)
try:
reparsed = tomllib.loads(filtered)
except Exception as exc:
raise ManifestError(f"filtered manifest does not parse: {exc}") from exc
surviving = {str(f.get("id", "")) for f in (reparsed.get("feature") or [])}
if surviving & set(dropped):
raise ManifestError(
"developer feature survived filtering: "
+ ", ".join(sorted(surviving & set(dropped))))
if not surviving:
raise ManifestError("filtering left a package with no features")
return filtered, dropped
def filter_catalog(catalog: Path, quiet: bool = False) -> tuple[int, int]:
"""Filter every package under one catalog dir. Returns (removed, edited)."""
removed = edited = 0
if not catalog.is_dir():
return 0, 0
for manifest in sorted(catalog.glob("*/*/manifest.toml")):
version_dir = manifest.parent
package_dir = version_dir.parent
try:
filtered, dropped = filter_manifest(
manifest.read_text(encoding="utf-8"))
except ManifestError as exc:
raise ManifestError(f"{manifest}: {exc}") from exc
if filtered is None:
shutil.rmtree(version_dir)
try:
package_dir.rmdir()
except OSError:
pass
removed += 1
if not quiet:
print(f" excluded developer package: "
f"{package_dir.name}/{version_dir.name}")
continue
if dropped:
manifest.write_text(filtered, encoding="utf-8")
edited += 1
if not quiet:
print(f" excluded developer feature(s) from "
f"{package_dir.name}/{version_dir.name}: "
f"{', '.join(sorted(dropped))}")
return removed, edited
def main() -> int:
parser = argparse.ArgumentParser(
description="Drop developer-channel mod features from staged catalogs.")
parser.add_argument("catalog", type=Path, nargs="+",
help="directory holding <id>/<version>/manifest.toml")
parser.add_argument("--quiet", action="store_true")
args = parser.parse_args()
removed = edited = 0
try:
for catalog in args.catalog:
r, e = filter_catalog(catalog, args.quiet)
removed += r
edited += e
except ManifestError as exc:
print(f"error: {exc}", file=sys.stderr)
return 1
if not args.quiet:
print(f"developer-channel filter: {removed} package(s) removed, "
f"{edited} manifest(s) filtered")
return 0
if __name__ == "__main__":
raise SystemExit(main())
tools/new_project_layout/setup_project.ps1
+10 -4
View File
@@ -398,7 +398,7 @@ New-Item -ItemType Directory -Force -Path (Join-Path $Root "scripts") | Out-Null
New-Item -ItemType Directory -Force -Path (Join-Path $Root "tools") | Out-Null
New-Item -ItemType Directory -Force -Path (Join-Path $Root "mods\preloaded\packages") | Out-Null
New-Item -ItemType Directory -Force -Path (Join-Path $Root "assets") | Out-Null
# Empty mod catalog tree (runtime copies mods/preloaded -> beside the exe as mods/).
# Empty mod catalog tree (build stages mods/preloaded/packages -> <exe>/mods/bundled).
@'
# Preloaded mods
@@ -410,9 +410,15 @@ packages/<package-id>/<version>/
...
```
Build wiring copies `mods/preloaded` next to the game executable as `mods/`.
Install player `.psxmod` archives through the launcher Mods manager instead of
committing them here. See `psxrecomp/docs/MOD_PACKAGES.md`.
Build wiring copies `mods/preloaded/packages` next to the game executable as
`mods/bundled/`. That tree is build output: every build wipes and re-stages it,
so nothing you place there by hand survives.
Player-installed `.psxmod` archives live in `mods/installed/`, which the
launcher owns and no build ever touches. Install them through the launcher Mods
manager rather than committing them here.
See `psxrecomp/docs/MOD_PACKAGES.md`.
'@ | Set-Content -Encoding utf8 (Join-Path $Root "mods\preloaded\README.md")
New-Item -ItemType File -Force -Path (Join-Path $Root "mods\preloaded\packages\.gitkeep") | Out-Null
Copy-Item (Join-Path $ScriptDir "sync_symbols.py") (Join-Path $Root "tools\sync_symbols.py")
tools/new_project_layout/setup_project.sh
+10 -4
View File
@@ -620,7 +620,7 @@ chmod +x "$ROOT/tools/sync_symbols.py"
if [ -f "$TEMPLATE_DIR/raid-discord.png" ]; then
cp "$TEMPLATE_DIR/raid-discord.png" "$ROOT/.github/raid-discord.png"
fi
# Empty mod catalog tree (runtime copies mods/preloaded → beside the exe as mods/).
# Empty mod catalog tree (build stages mods/preloaded/packages → <exe>/mods/bundled).
cat > "$ROOT/mods/preloaded/README.md" <<'EOF'
# Preloaded mods
@@ -632,9 +632,15 @@ packages/<package-id>/<version>/
…
```
Build wiring copies `mods/preloaded` next to the game executable as `mods/`.
Install player `.psxmod` archives through the launcher Mods manager instead of
committing them here. See `psxrecomp/docs/MOD_PACKAGES.md`.
Build wiring copies `mods/preloaded/packages` next to the game executable as
`mods/bundled/`. That tree is build output: every build wipes and re-stages it,
so nothing you place there by hand survives.
Player-installed `.psxmod` archives live in `mods/installed/`, which the
launcher owns and no build ever touches. Install them through the launcher Mods
manager rather than committing them here.
See `psxrecomp/docs/MOD_PACKAGES.md`.
EOF
: > "$ROOT/mods/preloaded/packages/.gitkeep"
tools/package_release_macos.sh
+18 -15
View File
@@ -40,7 +40,7 @@
# --mods-src DIR override catalog source (default: the build tree's
# mods/, which carries framework builtins AND the title's
# own packages; per-machine state.toml is stripped)
# --require-mod PKG path under mods/packages that must exist (repeatable)
# --require-mod PKG path under mods/bundled that must exist (repeatable)
# --data FILE file the runtime reads BESIDE the executable, relative to
# repo, e.g. keybinds.ini (repeatable)
# --doc FILE extra doc to ship, relative to repo (repeatable)
@@ -142,7 +142,7 @@ cp "$BIN" "$APPDIR/Contents/MacOS/$BIN_NAME"
# they must be reachable from Contents/MacOS. They cannot literally live there:
# codesign treats every non-executable under Contents/MacOS as a nested-code
# candidate and rejects the whole bundle ("bundle format unrecognized, invalid,
# or unsuitable") — mods/packages/<pkg>/<version>/ parses as a versioned bundle,
# or unsuitable") — mods/bundled/<pkg>/<version>/ parses as a versioned bundle,
# assets/img/*.tga as stray nested code. So data lives in Contents/Resources and
# Contents/MacOS carries symlinks: codesign seals a symlink as a symlink instead
# of descending into it, and runtime path resolution is unchanged.
@@ -189,19 +189,22 @@ _mod_manifests=$(find "$APPDIR/Contents/Resources/mods" -name manifest.toml 2>/d
|| die "staged mods/ contains no manifest.toml; the catalog would ship empty"
[ -z "$(find "$APPDIR/Contents/Resources/mods" -name 'state.toml*' 2>/dev/null)" ] \
|| die "per-machine mods/state.toml survived staging"
# Developer-channel packages (channel = "developer") are unfinished work that
# ships with local builds but must not be published. Prune by PACKAGE
# directory; never rewrite a manifest during packaging.
# Developer-channel work is unfinished: it ships with local builds and must
# never be published. Channels are per FEATURE, so this cannot be a grep -- a
# line-anchored match cannot tell a package-level key from one inside a
# [[feature]] block, and would drop a whole catalog over one instrument.
# mod_channel_filter.py parses instead, dropping the version directory when
# nothing in it ships and otherwise emitting a manifest without the developer
# features. The staged catalog is build output, so emitting it filtered is
# generation; the author's manifest in the repo is never touched.
if [ "${EXCLUDE_DEV_MODS:-0}" = "1" ]; then
note "excluding developer-channel mods"
find "$APPDIR/Contents/Resources/mods/packages" -mindepth 3 -maxdepth 3 \
-name manifest.toml 2>/dev/null | while read -r m; do
if grep -Eq '^[[:space:]]*channel[[:space:]]*=[[:space:]]*"developer"[[:space:]]*$' "$m"; then
v=$(dirname "$m"); p=$(dirname "$v")
rm -rf "$v"; rmdir "$p" 2>/dev/null || true
note " excluded developer package: $(basename "$p")/$(basename "$v")"
fi
done
command -v python3 >/dev/null 2>&1 \
|| die "no python3 on PATH; cannot filter developer-channel mods"
python3 "$(dirname "$0")/mod_channel_filter.py" \
"$APPDIR/Contents/Resources/mods/bundled" \
"$APPDIR/Contents/Resources/mods/packages" \
|| die "developer-channel filtering failed"
_dev_left=$( { grep -rlE '^[[:space:]]*channel[[:space:]]*=[[:space:]]*"developer"[[:space:]]*$' \
"$APPDIR" --include=manifest.toml 2>/dev/null || true; } | wc -l)
[ "$_dev_left" -eq 0 ] || die "developer manifest(s) survived pruning: $_dev_left"
@@ -334,8 +337,8 @@ note "bundled OpenBIOS (512 KiB) + MIT notice"
# 4. Required mod packages. A silently mod-less launcher looks fine but has no
# Mods page at all, which is indistinguishable from the feature being cut.
for m in "${REQUIRE_MODS[@]}"; do
[ -f "$APPDIR/Contents/Resources/mods/packages/$m" ] \
|| die "required mod package missing: mods/packages/$m"
[ -f "$APPDIR/Contents/Resources/mods/bundled/$m" ] \
|| die "required mod package missing: mods/bundled/$m"
done
[ ${#REQUIRE_MODS[@]} -eq 0 ] || note "mod catalog: ${#REQUIRE_MODS[@]} required package(s) present"
tools/package_setup_host.sh
+37 -29
View File
@@ -298,9 +298,11 @@ fi
# Exe-relative runtime data (mods catalog, bezels, ...): the runtime resolves
# these from the exe's own directory, so ship them at the stage root exactly
# as the build tree staged them. mods/state.toml is the LOCAL machine's mod
# enable/disable state -- preloaded catalogs ship default-disabled, so a dev
# machine's selections must never leak into a release.
# as the build tree staged them. Two things under mods/ belong to the local
# machine and must never leak into a release: state.toml is this machine's mod
# enable/disable state (preloaded catalogs ship default-disabled), and
# installed/ holds .psxmod archives the developer installed as a player would.
# Only mods/bundled/ is build output, and only build output ships.
copy_runtime_dir() {
local name="$1" optional="$2"
if [[ ! -d "${EXE_DIR}/${name}" ]]; then
@@ -315,36 +317,40 @@ copy_runtime_dir() {
mkdir -p "${STAGE}/${name}"
if command -v rsync >/dev/null 2>&1; then
rsync -a --exclude 'state.toml' --exclude 'state.toml.tmp' \
--exclude 'installed' --exclude 'installed/***' \
"${EXE_DIR}/${name}/" "${STAGE}/${name}/"
else
cp -a "${EXE_DIR}/${name}/." "${STAGE}/${name}/"
rm -f "${STAGE}/${name}/state.toml" "${STAGE}/${name}/state.toml.tmp"
rm -rf "${STAGE}/${name}/installed"
fi
echo "staged runtime dir ${name}/"
}
# Remove every channel = "developer" package from a staged catalog tree.
# <tree>/packages/<id>/<version>/manifest.toml is the layout; drop the VERSION
# directory that declares the channel, then the package directory once it has
# no versions left. Matching is on the manifest's own declaration so a package
# carries its own status -- nothing here maintains a list of names to exclude.
prune_dev_mods() { # prune_dev_mods <staged-catalog-root>
local tree="$1" manifest pkgdir verdir dropped=0
[[ -d "${tree}/packages" ]] || return 0
while IFS= read -r manifest; do
# Tolerate spacing/quoting variants; anchor to a top-level key so a
# feature description mentioning the word cannot trigger a drop.
if grep -Eq '^[[:space:]]*channel[[:space:]]*=[[:space:]]*"developer"[[:space:]]*$' \
"${manifest}"; then
verdir="$(dirname "${manifest}")"
pkgdir="$(dirname "${verdir}")"
rm -rf "${verdir}"
rmdir "${pkgdir}" 2>/dev/null || true
echo " excluded developer package: $(basename "${pkgdir}")/$(basename "${verdir}")"
dropped=$((dropped + 1))
fi
done < <(find "${tree}/packages" -mindepth 3 -maxdepth 3 -name manifest.toml 2>/dev/null)
return 0
# Drop developer-channel work from a staged catalog tree.
#
# Channels are per FEATURE, so this cannot be a grep: a line-anchored match on
# `channel = "developer"` cannot tell a package-level key from one inside a
# [[feature]] block, and would drop a whole catalog because one instrument in
# it is unfinished. mod_channel_filter.py parses the manifest, removes the
# version directory when nothing in it ships, and otherwise emits a manifest
# without the developer features and the entries that only served them.
#
# Fails closed: if the filter cannot run, packaging stops rather than
# publishing unfinished work.
prune_dev_mods() { # prune_dev_mods <dir of <id>/<version>/manifest.toml>...
local python=""
for candidate in python3 python; do
if command -v "${candidate}" >/dev/null 2>&1; then python="${candidate}"; break; fi
done
if [[ -z "${python}" ]]; then
echo "error: no python3 on PATH; cannot filter developer-channel mods" >&2
exit 1
fi
if ! "${python}" "${SCRIPT_DIR}/mod_channel_filter.py" "$@"; then
echo "error: developer-channel filtering failed" >&2
exit 1
fi
}
if ((${#RUNTIME_DIRS[@]})); then
@@ -384,14 +390,16 @@ for d in "${PROJECT_DIRS[@]}"; do
done
# Developer-channel pruning runs AFTER every catalog is staged, and covers BOTH
# copies: the runtime catalog under mods/packages that the launcher lists, and
# copies: the runtime catalog under mods/bundled that the launcher lists, and
# the source catalog under mods/preloaded that the setup-host rebuild re-stages
# from. Pruning only the first would publish the package in source form and let
# the player's own rebuild put it straight back.
# the player's own rebuild put it straight back. mods/packages is the pre-split
# layout, kept here so an older staged tree is still pruned rather than shipped.
if [[ "${EXCLUDE_DEV_MODS}" -eq 1 ]]; then
echo "excluding developer-channel mods from this package"
prune_dev_mods "${STAGE}/mods"
prune_dev_mods "${STAGE}/mods/preloaded"
prune_dev_mods "${STAGE}/mods/bundled" \
"${STAGE}/mods/preloaded/packages" \
"${STAGE}/mods/packages"
_dev_left=$( { grep -rlE '^[[:space:]]*channel[[:space:]]*=[[:space:]]*"developer"[[:space:]]*$' \
"${STAGE}" --include=manifest.toml 2>/dev/null || true; } | wc -l)
if [[ "${_dev_left}" -ne 0 ]]; then
tools/release_overlay_stage.ps1
+11 -3
View File
@@ -298,17 +298,25 @@ silently drops them (pattern and rationale from MegaManX6).
#>
function Add-ModCatalog {
param(
[Parameter(Mandatory)][string]$BuildPath, # build dir (has mods/packages)
[Parameter(Mandatory)][string]$BuildPath, # build dir (has mods/bundled)
[Parameter(Mandatory)][string]$Stage,
[string]$GameModSource = "", # <repo>/mods/preloaded
[string]$FrameworkModSource = "" # <framework>/mods/builtin
)
$ModsSrc = Join-Path $BuildPath "mods"
if (-not (Test-Path (Join-Path $ModsSrc "packages"))) {
if (-not (Test-Path (Join-Path $ModsSrc "bundled"))) {
throw "No mod catalog staged at $ModsSrc - build the runtime first (the framework stages its builtin packages there)"
}
Copy-Item -Recurse -Force $ModsSrc (Join-Path $Stage "mods")
$StagedPkgDir = Join-Path $Stage "mods\packages"
# mods/bundled is build output and ships. Two things under mods/ belong to
# this machine and must never reach a release: installed/ holds .psxmod
# archives the developer installed as a player would, and state.toml is
# their own enable/disable selection over a catalog that ships default-off.
$StagedMods = Join-Path $Stage "mods"
Remove-Item -Recurse -Force (Join-Path $StagedMods "installed") -ErrorAction SilentlyContinue
Remove-Item -Force (Join-Path $StagedMods "state.toml") -ErrorAction SilentlyContinue
Remove-Item -Force (Join-Path $StagedMods "state.toml.tmp") -ErrorAction SilentlyContinue
$StagedPkgDir = Join-Path $Stage "mods\bundled"
$stagedIds = @(Get-ChildItem $StagedPkgDir -Directory -ErrorAction SilentlyContinue |
ForEach-Object { $_.Name })