mirror of
https://github.com/RetroPortingToolKit/psxrecomp
synced 2026-10-26 02:11:36 -04:00
packaging: a developer channel, kept out of published releases
Mods under active development had nowhere to live: a package was either in the catalog and therefore in every release, or not built at all. Widescreen is the immediate case -- it engages, but it is unvalidated as gameplay and FF7's pre-rendered field backdrops still cull at the 4:3 bound, so it is not something to put in front of players yet. A package now declares its own maturity with channel = "developer". It still ships with ordinary local builds and local exports, so it can be tested exactly as a player would see it, and release packaging drops it. The exclusion defaults from $CI -- on under any CI provider, off locally. That is deliberate: a per-workflow opt-in is precisely the shape that left 21 of 23 ports shipping no mods at all, so this must not need 23 release workflows edited to take effect. --exclude-dev-mods / --include-dev-mods and EXCLUDE_DEV_MODS=0|1 override in either direction, and Studio's `build export --exclude-dev-mods` reproduces what a release contains. Applied in all three packagers (setup-host, Windows, macOS). Each prunes the whole PACKAGE directory and then asserts no developer manifest survived -- including the SOURCE catalog under mods/preloaded, since the setup-host zip rebuilds on the player's machine and would otherwise re-stage the package the release just removed. A manifest is never rewritten during packaging: editing a package while packaging it would change content meant to be verifiable, which is why the marker is package-level rather than per-feature. Verified by packaging: local keeps 5 catalogs, CI=true drops to 4 with no ff7.enhancements entry anywhere in the zip, and EXCLUDE_DEV_MODS=0 under CI=true restores it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0194TYJAqgyG9a9PJe9uUcgQ
This commit is contained in:
@@ -261,6 +261,32 @@ Sparse fields and integer predicates are still pre-boot plan construction.
|
||||
They do not provide a general expression evaluator, masks, arithmetic beyond
|
||||
the checked field addend, package code execution, or per-frame dispatch.
|
||||
|
||||
## Developer channel
|
||||
|
||||
A package may declare itself unfinished:
|
||||
|
||||
```toml
|
||||
format_version = 5
|
||||
id = "example.enhancements"
|
||||
channel = "developer"
|
||||
```
|
||||
|
||||
`channel = "developer"` marks work in progress. Such a package is staged next
|
||||
to the executable by an ordinary build and ships in a locally exported
|
||||
package, so it can be tested exactly as a player would see it — but release
|
||||
packaging drops it, so unfinished work is never published.
|
||||
|
||||
Packaging defaults the exclusion from `$CI`: on under any CI provider, off
|
||||
locally. `--exclude-dev-mods` / `--include-dev-mods` (or `EXCLUDE_DEV_MODS=0|1`)
|
||||
override it, and `project_studio build export --exclude-dev-mods` reproduces
|
||||
what a release would contain. Any absent `channel` means stable.
|
||||
|
||||
Pruning removes the whole package directory. A manifest is never rewritten
|
||||
during packaging — a package's content stays exactly what its author shipped,
|
||||
so the maturity marker is package-level. Split a player-ready feature into its
|
||||
own stable package rather than promoting a catalog that still holds
|
||||
in-progress work.
|
||||
|
||||
## Trusted static plugins
|
||||
|
||||
Package format 5 can activate a game-owned plugin that is already statically
|
||||
|
||||
@@ -882,9 +882,15 @@ def export_release(
|
||||
artifact_tag: str | None = None,
|
||||
recompiler_build: str = "build-recompiler",
|
||||
*,
|
||||
exclude_dev_mods: bool = False,
|
||||
log: LogFn | None = None,
|
||||
) -> CmdResult:
|
||||
"""Package a distributable zip from an existing build. Returns its path."""
|
||||
"""Package a distributable zip from an existing build. Returns its path.
|
||||
|
||||
exclude_dev_mods reproduces what CI publishes -- channel = "developer"
|
||||
packages dropped. A local export keeps them by default, because the point
|
||||
of exporting locally is to test the work in progress.
|
||||
"""
|
||||
wrapper = root / "scripts" / "package_setup_release.sh"
|
||||
if not wrapper.is_file():
|
||||
return CmdResult(
|
||||
@@ -909,6 +915,9 @@ def export_release(
|
||||
tag = artifact_tag or default_artifact_tag()
|
||||
env = dict(os.environ)
|
||||
env.setdefault("RELEASE_VERSION", release_version(root))
|
||||
# Explicit either way: the packager defaults this from $CI, and a studio
|
||||
# export must not inherit that by accident on a CI-flavoured machine.
|
||||
env["EXCLUDE_DEV_MODS"] = "1" if exclude_dev_mods else "0"
|
||||
res = _run_stream(
|
||||
["bash", str(wrapper), build_dir, tag, recompiler_build],
|
||||
root, log=log, env=env,
|
||||
|
||||
@@ -1523,6 +1523,7 @@ def cmd_build_export(args: argparse.Namespace) -> int:
|
||||
build_dir=args.build_dir,
|
||||
artifact_tag=args.artifact or None,
|
||||
recompiler_build=args.recompiler_build,
|
||||
exclude_dev_mods=args.exclude_dev_mods,
|
||||
log=print,
|
||||
)
|
||||
print(f"[{'OK' if r.ok else 'FAIL'}] {r.message}")
|
||||
@@ -2388,6 +2389,12 @@ def build_parser() -> argparse.ArgumentParser:
|
||||
"Matches the CI matrix so local and released zips share names.",
|
||||
)
|
||||
p_bx.add_argument("--recompiler-build", default="build-recompiler")
|
||||
p_bx.add_argument(
|
||||
"--exclude-dev-mods",
|
||||
action="store_true",
|
||||
help='Drop channel = "developer" packages, reproducing what CI '
|
||||
"publishes. Local exports keep them by default.",
|
||||
)
|
||||
p_bx.set_defaults(func=cmd_build_export)
|
||||
|
||||
p_br = build_sub.add_parser("run", help="Launch product binary with env")
|
||||
|
||||
@@ -2622,6 +2622,10 @@ class ProjectStudioApp:
|
||||
return export_release(
|
||||
root,
|
||||
build_dir=self.build_dir_var.get().strip() or "build-release",
|
||||
# Local export keeps developer-channel mods: the reason to
|
||||
# export locally is to test what you are working on. CI drops
|
||||
# them; `build export --exclude-dev-mods` reproduces that.
|
||||
exclude_dev_mods=False,
|
||||
log=lambda m: self.root.after(0, lambda line=m: self._log(line)),
|
||||
)
|
||||
|
||||
|
||||
@@ -1,6 +1,9 @@
|
||||
param(
|
||||
[string]$Version = "v0.3.2-alpha",
|
||||
[string]$BuildDir = "runtime/build-release"
|
||||
[string]$BuildDir = "runtime/build-release",
|
||||
# Drop channel = "developer" packages. Off by default so a local package
|
||||
# still carries in-progress work; CI sets EXCLUDE_DEV_MODS=1 instead.
|
||||
[switch]$ExcludeDevMods
|
||||
)
|
||||
|
||||
$ErrorActionPreference = "Stop"
|
||||
@@ -99,6 +102,36 @@ $manifestCount = (Get-ChildItem $ModsDst -Recurse -File -Filter "manifest.toml"
|
||||
if ($manifestCount -lt 1) {
|
||||
throw "Staged mods/ contains no manifest.toml; the catalog would ship empty"
|
||||
}
|
||||
# Developer-channel packages (channel = "developer") are unfinished work that
|
||||
# ships with local builds but must never be published. Prune by PACKAGE
|
||||
# directory; never rewrite a manifest during packaging.
|
||||
$ExcludeDev = $ExcludeDevMods -or ($env:EXCLUDE_DEV_MODS -eq "1")
|
||||
if ($ExcludeDev) {
|
||||
Write-Host "Excluding developer-channel mods"
|
||||
Get-ChildItem (Join-Path $ModsDst "packages") -Recurse -File -Filter "manifest.toml" `
|
||||
-ErrorAction SilentlyContinue | ForEach-Object {
|
||||
if ((Get-Content $_.FullName) -match '^\s*channel\s*=\s*"developer"\s*$') {
|
||||
$verDir = $_.Directory
|
||||
$pkgDir = $verDir.Parent
|
||||
Remove-Item -Recurse -Force $verDir.FullName
|
||||
if (-not (Get-ChildItem $pkgDir.FullName -ErrorAction SilentlyContinue)) {
|
||||
Remove-Item -Force $pkgDir.FullName
|
||||
}
|
||||
Write-Host " excluded developer package: $($pkgDir.Name)/$($verDir.Name)"
|
||||
}
|
||||
}
|
||||
$devLeft = Get-ChildItem $Stage -Recurse -File -Filter "manifest.toml" `
|
||||
-ErrorAction SilentlyContinue |
|
||||
Where-Object { (Get-Content $_.FullName) -match '^\s*channel\s*=\s*"developer"\s*$' }
|
||||
if ($devLeft) {
|
||||
throw "developer manifest(s) survived pruning: $(($devLeft | ForEach-Object FullName) -join '; ')"
|
||||
}
|
||||
$manifestCount = (Get-ChildItem $ModsDst -Recurse -File -Filter "manifest.toml" `
|
||||
-ErrorAction SilentlyContinue | Measure-Object).Count
|
||||
if ($manifestCount -lt 1) {
|
||||
throw "every staged package was developer-channel; the catalog would ship empty"
|
||||
}
|
||||
}
|
||||
Write-Host "Staged mod catalog: $manifestCount manifest(s)"
|
||||
|
||||
# No user-machine or copyrighted files may ride along in the stage. OpenBIOS
|
||||
|
||||
@@ -36,6 +36,7 @@
|
||||
# --version VER release version, e.g. v0.0.1 (default: dev)
|
||||
# --out DIR output directory (default: <repo>/release-macos)
|
||||
# --game-config P game.toml to bundle, relative to repo
|
||||
# --exclude-dev-mods drop channel = "developer" packages (also EXCLUDE_DEV_MODS=1)
|
||||
# --mods-src DIR override catalog source (default: the build tree's
|
||||
# mods/, which carries framework builtins AND the title's
|
||||
# own packages; per-machine state.toml is stripped)
|
||||
@@ -77,6 +78,7 @@ while [ $# -gt 0 ]; do
|
||||
--version) VERSION="$2"; shift 2;;
|
||||
--out) OUT="$2"; shift 2;;
|
||||
--game-config) GAME_CONFIG="$2"; shift 2;;
|
||||
--exclude-dev-mods) EXCLUDE_DEV_MODS=1; shift;;
|
||||
--mods-src) MODS_SRC="$2"; MODS_SRC_EXPLICIT=1; shift 2;;
|
||||
--require-mod) REQUIRE_MODS+=("$2"); shift 2;;
|
||||
--data) DATA_FILES+=("$2"); shift 2;;
|
||||
@@ -187,6 +189,24 @@ _mod_manifests=$(find "$APPDIR/Contents/Resources/mods" -name manifest.toml 2>/d
|
||||
|| die "staged mods/ contains no manifest.toml; the catalog would ship empty"
|
||||
[ -z "$(find "$APPDIR/Contents/Resources/mods" -name 'state.toml*' 2>/dev/null)" ] \
|
||||
|| die "per-machine mods/state.toml survived staging"
|
||||
# Developer-channel packages (channel = "developer") are unfinished work that
|
||||
# ships with local builds but must not be published. Prune by PACKAGE
|
||||
# directory; never rewrite a manifest during packaging.
|
||||
if [ "${EXCLUDE_DEV_MODS:-0}" = "1" ]; then
|
||||
note "excluding developer-channel mods"
|
||||
find "$APPDIR/Contents/Resources/mods/packages" -mindepth 3 -maxdepth 3 \
|
||||
-name manifest.toml 2>/dev/null | while read -r m; do
|
||||
if grep -Eq '^[[:space:]]*channel[[:space:]]*=[[:space:]]*"developer"[[:space:]]*$' "$m"; then
|
||||
v=$(dirname "$m"); p=$(dirname "$v")
|
||||
rm -rf "$v"; rmdir "$p" 2>/dev/null || true
|
||||
note " excluded developer package: $(basename "$p")/$(basename "$v")"
|
||||
fi
|
||||
done
|
||||
_dev_left=$( { grep -rlE '^[[:space:]]*channel[[:space:]]*=[[:space:]]*"developer"[[:space:]]*$' \
|
||||
"$APPDIR" --include=manifest.toml 2>/dev/null || true; } | wc -l)
|
||||
[ "$_dev_left" -eq 0 ] || die "developer manifest(s) survived pruning: $_dev_left"
|
||||
_mod_manifests=$(find "$APPDIR/Contents/Resources/mods" -name manifest.toml 2>/dev/null | wc -l)
|
||||
fi
|
||||
note "mod catalog: ${_mod_manifests} manifest(s), no per-machine state"
|
||||
|
||||
if [ -n "$GAME_CONFIG" ]; then
|
||||
|
||||
@@ -34,6 +34,14 @@
|
||||
# --runtime-dir mods. Enforce it here rather than in each title's wrapper.
|
||||
# A title that genuinely ships no catalog passes --no-mods.
|
||||
#
|
||||
# DEVELOPER CHANNEL. A package manifest may declare channel = "developer".
|
||||
# Those are work-in-progress catalogs that should stay on the maintainer's
|
||||
# machine: they ship with ordinary local builds and local exports, but
|
||||
# --exclude-dev-mods (or EXCLUDE_DEV_MODS=1, which CI sets) drops them from a
|
||||
# public release. Pruning is by PACKAGE directory, never by rewriting a
|
||||
# manifest -- editing a package during packaging would change content that is
|
||||
# meant to be verifiable.
|
||||
#
|
||||
# Env:
|
||||
# RELEASE_VERSION / <version-env> / VERSION file (must match binary stamp)
|
||||
# PSXRECOMP_TOOLCHAIN_DIR | TOOLCHAIN_DIR | BPE_TOOLCHAIN_DIR (only with --embed-toolchain)
|
||||
@@ -62,6 +70,18 @@ RUNTIME_DIRS=()
|
||||
RUNTIME_DIRS_OPTIONAL=()
|
||||
# mods/ ships by default; --no-mods opts a catalog-less title out.
|
||||
STAGE_MODS=1
|
||||
# Drop channel = "developer" packages from the shipped catalog.
|
||||
#
|
||||
# Defaults to ON under CI and OFF locally, which is the whole point: a
|
||||
# maintainer's local build and local export keep the mods they are working on,
|
||||
# while anything a release workflow publishes does not. Keying off $CI (set by
|
||||
# GitHub Actions and every other provider) means this holds for every title
|
||||
# without editing 23 separate release workflows -- a per-workflow opt-in is
|
||||
# exactly the shape that left 21 of 23 ports shipping no mods at all.
|
||||
# --include-dev-mods / --exclude-dev-mods override either way.
|
||||
if [[ -z "${EXCLUDE_DEV_MODS:-}" ]]; then
|
||||
if [[ -n "${CI:-}" ]]; then EXCLUDE_DEV_MODS=1; else EXCLUDE_DEV_MODS=0; fi
|
||||
fi
|
||||
RUNTIME_BIN_DIR="${PSXRECOMP_RUNTIME_BIN_DIR:-${BPE_RUNTIME_BIN_DIR:-/usr/x86_64-w64-mingw32/bin}}"
|
||||
EMBED_TOOLCHAIN=0
|
||||
if [[ "${PSXRECOMP_EMBED_TOOLCHAIN:-0}" == "1" ]]; then
|
||||
@@ -87,6 +107,8 @@ while [[ $# -gt 0 ]]; do
|
||||
--project-file) PROJECT_FILES+=("${2:?}"); shift 2 ;;
|
||||
--project-dir) PROJECT_DIRS+=("${2:?}"); shift 2 ;;
|
||||
--no-mods) STAGE_MODS=0; shift ;;
|
||||
--exclude-dev-mods) EXCLUDE_DEV_MODS=1; shift ;;
|
||||
--include-dev-mods) EXCLUDE_DEV_MODS=0; shift ;;
|
||||
--runtime-dir) RUNTIME_DIRS+=("${2:?}"); shift 2 ;;
|
||||
--runtime-dir-optional) RUNTIME_DIRS_OPTIONAL+=("${2:?}"); shift 2 ;;
|
||||
--runtime-bin) RUNTIME_BIN_DIR="${2:?}"; shift 2 ;;
|
||||
@@ -301,6 +323,30 @@ copy_runtime_dir() {
|
||||
echo "staged runtime dir ${name}/"
|
||||
}
|
||||
|
||||
# Remove every channel = "developer" package from a staged catalog tree.
|
||||
# <tree>/packages/<id>/<version>/manifest.toml is the layout; drop the VERSION
|
||||
# directory that declares the channel, then the package directory once it has
|
||||
# no versions left. Matching is on the manifest's own declaration so a package
|
||||
# carries its own status -- nothing here maintains a list of names to exclude.
|
||||
prune_dev_mods() { # prune_dev_mods <staged-catalog-root>
|
||||
local tree="$1" manifest pkgdir verdir dropped=0
|
||||
[[ -d "${tree}/packages" ]] || return 0
|
||||
while IFS= read -r manifest; do
|
||||
# Tolerate spacing/quoting variants; anchor to a top-level key so a
|
||||
# feature description mentioning the word cannot trigger a drop.
|
||||
if grep -Eq '^[[:space:]]*channel[[:space:]]*=[[:space:]]*"developer"[[:space:]]*$' \
|
||||
"${manifest}"; then
|
||||
verdir="$(dirname "${manifest}")"
|
||||
pkgdir="$(dirname "${verdir}")"
|
||||
rm -rf "${verdir}"
|
||||
rmdir "${pkgdir}" 2>/dev/null || true
|
||||
echo " excluded developer package: $(basename "${pkgdir}")/$(basename "${verdir}")"
|
||||
dropped=$((dropped + 1))
|
||||
fi
|
||||
done < <(find "${tree}/packages" -mindepth 3 -maxdepth 3 -name manifest.toml 2>/dev/null)
|
||||
return 0
|
||||
}
|
||||
|
||||
if ((${#RUNTIME_DIRS[@]})); then
|
||||
for d in "${RUNTIME_DIRS[@]}"; do
|
||||
copy_runtime_dir "${d}" 0
|
||||
@@ -337,6 +383,32 @@ for d in "${PROJECT_DIRS[@]}"; do
|
||||
copy_proj "${d}"
|
||||
done
|
||||
|
||||
# Developer-channel pruning runs AFTER every catalog is staged, and covers BOTH
|
||||
# copies: the runtime catalog under mods/packages that the launcher lists, and
|
||||
# the source catalog under mods/preloaded that the setup-host rebuild re-stages
|
||||
# from. Pruning only the first would publish the package in source form and let
|
||||
# the player's own rebuild put it straight back.
|
||||
if [[ "${EXCLUDE_DEV_MODS}" -eq 1 ]]; then
|
||||
echo "excluding developer-channel mods from this package"
|
||||
prune_dev_mods "${STAGE}/mods"
|
||||
prune_dev_mods "${STAGE}/mods/preloaded"
|
||||
_dev_left=$( { grep -rlE '^[[:space:]]*channel[[:space:]]*=[[:space:]]*"developer"[[:space:]]*$' \
|
||||
"${STAGE}" --include=manifest.toml 2>/dev/null || true; } | wc -l)
|
||||
if [[ "${_dev_left}" -ne 0 ]]; then
|
||||
echo "error: ${_dev_left} developer manifest(s) survived pruning:" >&2
|
||||
grep -rlE '^[[:space:]]*channel[[:space:]]*=[[:space:]]*"developer"[[:space:]]*$' \
|
||||
"${STAGE}" --include=manifest.toml >&2 || true
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
_dev_in=$( { grep -rlE '^[[:space:]]*channel[[:space:]]*=[[:space:]]*"developer"[[:space:]]*$' \
|
||||
"${STAGE}" --include=manifest.toml 2>/dev/null || true; } | wc -l)
|
||||
if [[ "${_dev_in}" -ne 0 ]]; then
|
||||
echo "note: package includes ${_dev_in} developer-channel manifest(s);" \
|
||||
"pass --exclude-dev-mods (or EXCLUDE_DEV_MODS=1) for a public release"
|
||||
fi
|
||||
fi
|
||||
|
||||
copy_tree_filtered() {
|
||||
local src="$1" dest="$2"
|
||||
shift 2
|
||||
|
||||
Reference in New Issue
Block a user