packaging: a developer channel, kept out of published releases

Mods under active development had nowhere to live: a package was either in
the catalog and therefore in every release, or not built at all. Widescreen is
the immediate case -- it engages, but it is unvalidated as gameplay and FF7's
pre-rendered field backdrops still cull at the 4:3 bound, so it is not
something to put in front of players yet.

A package now declares its own maturity with channel = "developer". It still
ships with ordinary local builds and local exports, so it can be tested
exactly as a player would see it, and release packaging drops it.

The exclusion defaults from $CI -- on under any CI provider, off locally.
That is deliberate: a per-workflow opt-in is precisely the shape that left 21
of 23 ports shipping no mods at all, so this must not need 23 release
workflows edited to take effect. --exclude-dev-mods / --include-dev-mods and
EXCLUDE_DEV_MODS=0|1 override in either direction, and Studio's
`build export --exclude-dev-mods` reproduces what a release contains.

Applied in all three packagers (setup-host, Windows, macOS). Each prunes the
whole PACKAGE directory and then asserts no developer manifest survived --
including the SOURCE catalog under mods/preloaded, since the setup-host zip
rebuilds on the player's machine and would otherwise re-stage the package the
release just removed.

A manifest is never rewritten during packaging: editing a package while
packaging it would change content meant to be verifiable, which is why the
marker is package-level rather than per-feature.

Verified by packaging: local keeps 5 catalogs, CI=true drops to 4 with no
ff7.enhancements entry anywhere in the zip, and EXCLUDE_DEV_MODS=0 under
CI=true restores it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194TYJAqgyG9a9PJe9uUcgQ
This commit is contained in:
Alex Vanderveen
2026-08-29 20:24:39 -04:00
parent 647b233015
commit d2aeac70a5
7 changed files with 173 additions and 2 deletions
+26
View File
@@ -261,6 +261,32 @@ Sparse fields and integer predicates are still pre-boot plan construction.
They do not provide a general expression evaluator, masks, arithmetic beyond
the checked field addend, package code execution, or per-frame dispatch.
## Developer channel
A package may declare itself unfinished:
```toml
format_version = 5
id = "example.enhancements"
channel = "developer"
```
`channel = "developer"` marks work in progress. Such a package is staged next
to the executable by an ordinary build and ships in a locally exported
package, so it can be tested exactly as a player would see it — but release
packaging drops it, so unfinished work is never published.
Packaging defaults the exclusion from `$CI`: on under any CI provider, off
locally. `--exclude-dev-mods` / `--include-dev-mods` (or `EXCLUDE_DEV_MODS=0|1`)
override it, and `project_studio build export --exclude-dev-mods` reproduces
what a release would contain. Any absent `channel` means stable.
Pruning removes the whole package directory. A manifest is never rewritten
during packaging — a package's content stays exactly what its author shipped,
so the maturity marker is package-level. Split a player-ready feature into its
own stable package rather than promoting a catalog that still holds
in-progress work.
## Trusted static plugins
Package format 5 can activate a game-owned plugin that is already statically
@@ -882,9 +882,15 @@ def export_release(
artifact_tag: str | None = None,
recompiler_build: str = "build-recompiler",
*,
exclude_dev_mods: bool = False,
log: LogFn | None = None,
) -> CmdResult:
"""Package a distributable zip from an existing build. Returns its path."""
"""Package a distributable zip from an existing build. Returns its path.
exclude_dev_mods reproduces what CI publishes -- channel = "developer"
packages dropped. A local export keeps them by default, because the point
of exporting locally is to test the work in progress.
"""
wrapper = root / "scripts" / "package_setup_release.sh"
if not wrapper.is_file():
return CmdResult(
@@ -909,6 +915,9 @@ def export_release(
tag = artifact_tag or default_artifact_tag()
env = dict(os.environ)
env.setdefault("RELEASE_VERSION", release_version(root))
# Explicit either way: the packager defaults this from $CI, and a studio
# export must not inherit that by accident on a CI-flavoured machine.
env["EXCLUDE_DEV_MODS"] = "1" if exclude_dev_mods else "0"
res = _run_stream(
["bash", str(wrapper), build_dir, tag, recompiler_build],
root, log=log, env=env,
@@ -1523,6 +1523,7 @@ def cmd_build_export(args: argparse.Namespace) -> int:
build_dir=args.build_dir,
artifact_tag=args.artifact or None,
recompiler_build=args.recompiler_build,
exclude_dev_mods=args.exclude_dev_mods,
log=print,
)
print(f"[{'OK' if r.ok else 'FAIL'}] {r.message}")
@@ -2388,6 +2389,12 @@ def build_parser() -> argparse.ArgumentParser:
"Matches the CI matrix so local and released zips share names.",
)
p_bx.add_argument("--recompiler-build", default="build-recompiler")
p_bx.add_argument(
"--exclude-dev-mods",
action="store_true",
help='Drop channel = "developer" packages, reproducing what CI '
"publishes. Local exports keep them by default.",
)
p_bx.set_defaults(func=cmd_build_export)
p_br = build_sub.add_parser("run", help="Launch product binary with env")
@@ -2622,6 +2622,10 @@ class ProjectStudioApp:
return export_release(
root,
build_dir=self.build_dir_var.get().strip() or "build-release",
# Local export keeps developer-channel mods: the reason to
# export locally is to test what you are working on. CI drops
# them; `build export --exclude-dev-mods` reproduces that.
exclude_dev_mods=False,
log=lambda m: self.root.after(0, lambda line=m: self._log(line)),
)
+34 -1
View File
@@ -1,6 +1,9 @@
param(
[string]$Version = "v0.3.2-alpha",
[string]$BuildDir = "runtime/build-release"
[string]$BuildDir = "runtime/build-release",
# Drop channel = "developer" packages. Off by default so a local package
# still carries in-progress work; CI sets EXCLUDE_DEV_MODS=1 instead.
[switch]$ExcludeDevMods
)
$ErrorActionPreference = "Stop"
@@ -99,6 +102,36 @@ $manifestCount = (Get-ChildItem $ModsDst -Recurse -File -Filter "manifest.toml"
if ($manifestCount -lt 1) {
throw "Staged mods/ contains no manifest.toml; the catalog would ship empty"
}
# Developer-channel packages (channel = "developer") are unfinished work that
# ships with local builds but must never be published. Prune by PACKAGE
# directory; never rewrite a manifest during packaging.
$ExcludeDev = $ExcludeDevMods -or ($env:EXCLUDE_DEV_MODS -eq "1")
if ($ExcludeDev) {
Write-Host "Excluding developer-channel mods"
Get-ChildItem (Join-Path $ModsDst "packages") -Recurse -File -Filter "manifest.toml" `
-ErrorAction SilentlyContinue | ForEach-Object {
if ((Get-Content $_.FullName) -match '^\s*channel\s*=\s*"developer"\s*$') {
$verDir = $_.Directory
$pkgDir = $verDir.Parent
Remove-Item -Recurse -Force $verDir.FullName
if (-not (Get-ChildItem $pkgDir.FullName -ErrorAction SilentlyContinue)) {
Remove-Item -Force $pkgDir.FullName
}
Write-Host " excluded developer package: $($pkgDir.Name)/$($verDir.Name)"
}
}
$devLeft = Get-ChildItem $Stage -Recurse -File -Filter "manifest.toml" `
-ErrorAction SilentlyContinue |
Where-Object { (Get-Content $_.FullName) -match '^\s*channel\s*=\s*"developer"\s*$' }
if ($devLeft) {
throw "developer manifest(s) survived pruning: $(($devLeft | ForEach-Object FullName) -join '; ')"
}
$manifestCount = (Get-ChildItem $ModsDst -Recurse -File -Filter "manifest.toml" `
-ErrorAction SilentlyContinue | Measure-Object).Count
if ($manifestCount -lt 1) {
throw "every staged package was developer-channel; the catalog would ship empty"
}
}
Write-Host "Staged mod catalog: $manifestCount manifest(s)"
# No user-machine or copyrighted files may ride along in the stage. OpenBIOS
+20
View File
@@ -36,6 +36,7 @@
# --version VER release version, e.g. v0.0.1 (default: dev)
# --out DIR output directory (default: <repo>/release-macos)
# --game-config P game.toml to bundle, relative to repo
# --exclude-dev-mods drop channel = "developer" packages (also EXCLUDE_DEV_MODS=1)
# --mods-src DIR override catalog source (default: the build tree's
# mods/, which carries framework builtins AND the title's
# own packages; per-machine state.toml is stripped)
@@ -77,6 +78,7 @@ while [ $# -gt 0 ]; do
--version) VERSION="$2"; shift 2;;
--out) OUT="$2"; shift 2;;
--game-config) GAME_CONFIG="$2"; shift 2;;
--exclude-dev-mods) EXCLUDE_DEV_MODS=1; shift;;
--mods-src) MODS_SRC="$2"; MODS_SRC_EXPLICIT=1; shift 2;;
--require-mod) REQUIRE_MODS+=("$2"); shift 2;;
--data) DATA_FILES+=("$2"); shift 2;;
@@ -187,6 +189,24 @@ _mod_manifests=$(find "$APPDIR/Contents/Resources/mods" -name manifest.toml 2>/d
|| die "staged mods/ contains no manifest.toml; the catalog would ship empty"
[ -z "$(find "$APPDIR/Contents/Resources/mods" -name 'state.toml*' 2>/dev/null)" ] \
|| die "per-machine mods/state.toml survived staging"
# Developer-channel packages (channel = "developer") are unfinished work that
# ships with local builds but must not be published. Prune by PACKAGE
# directory; never rewrite a manifest during packaging.
if [ "${EXCLUDE_DEV_MODS:-0}" = "1" ]; then
note "excluding developer-channel mods"
find "$APPDIR/Contents/Resources/mods/packages" -mindepth 3 -maxdepth 3 \
-name manifest.toml 2>/dev/null | while read -r m; do
if grep -Eq '^[[:space:]]*channel[[:space:]]*=[[:space:]]*"developer"[[:space:]]*$' "$m"; then
v=$(dirname "$m"); p=$(dirname "$v")
rm -rf "$v"; rmdir "$p" 2>/dev/null || true
note " excluded developer package: $(basename "$p")/$(basename "$v")"
fi
done
_dev_left=$( { grep -rlE '^[[:space:]]*channel[[:space:]]*=[[:space:]]*"developer"[[:space:]]*$' \
"$APPDIR" --include=manifest.toml 2>/dev/null || true; } | wc -l)
[ "$_dev_left" -eq 0 ] || die "developer manifest(s) survived pruning: $_dev_left"
_mod_manifests=$(find "$APPDIR/Contents/Resources/mods" -name manifest.toml 2>/dev/null | wc -l)
fi
note "mod catalog: ${_mod_manifests} manifest(s), no per-machine state"
if [ -n "$GAME_CONFIG" ]; then
+72
View File
@@ -34,6 +34,14 @@
# --runtime-dir mods. Enforce it here rather than in each title's wrapper.
# A title that genuinely ships no catalog passes --no-mods.
#
# DEVELOPER CHANNEL. A package manifest may declare channel = "developer".
# Those are work-in-progress catalogs that should stay on the maintainer's
# machine: they ship with ordinary local builds and local exports, but
# --exclude-dev-mods (or EXCLUDE_DEV_MODS=1, which CI sets) drops them from a
# public release. Pruning is by PACKAGE directory, never by rewriting a
# manifest -- editing a package during packaging would change content that is
# meant to be verifiable.
#
# Env:
# RELEASE_VERSION / <version-env> / VERSION file (must match binary stamp)
# PSXRECOMP_TOOLCHAIN_DIR | TOOLCHAIN_DIR | BPE_TOOLCHAIN_DIR (only with --embed-toolchain)
@@ -62,6 +70,18 @@ RUNTIME_DIRS=()
RUNTIME_DIRS_OPTIONAL=()
# mods/ ships by default; --no-mods opts a catalog-less title out.
STAGE_MODS=1
# Drop channel = "developer" packages from the shipped catalog.
#
# Defaults to ON under CI and OFF locally, which is the whole point: a
# maintainer's local build and local export keep the mods they are working on,
# while anything a release workflow publishes does not. Keying off $CI (set by
# GitHub Actions and every other provider) means this holds for every title
# without editing 23 separate release workflows -- a per-workflow opt-in is
# exactly the shape that left 21 of 23 ports shipping no mods at all.
# --include-dev-mods / --exclude-dev-mods override either way.
if [[ -z "${EXCLUDE_DEV_MODS:-}" ]]; then
if [[ -n "${CI:-}" ]]; then EXCLUDE_DEV_MODS=1; else EXCLUDE_DEV_MODS=0; fi
fi
RUNTIME_BIN_DIR="${PSXRECOMP_RUNTIME_BIN_DIR:-${BPE_RUNTIME_BIN_DIR:-/usr/x86_64-w64-mingw32/bin}}"
EMBED_TOOLCHAIN=0
if [[ "${PSXRECOMP_EMBED_TOOLCHAIN:-0}" == "1" ]]; then
@@ -87,6 +107,8 @@ while [[ $# -gt 0 ]]; do
--project-file) PROJECT_FILES+=("${2:?}"); shift 2 ;;
--project-dir) PROJECT_DIRS+=("${2:?}"); shift 2 ;;
--no-mods) STAGE_MODS=0; shift ;;
--exclude-dev-mods) EXCLUDE_DEV_MODS=1; shift ;;
--include-dev-mods) EXCLUDE_DEV_MODS=0; shift ;;
--runtime-dir) RUNTIME_DIRS+=("${2:?}"); shift 2 ;;
--runtime-dir-optional) RUNTIME_DIRS_OPTIONAL+=("${2:?}"); shift 2 ;;
--runtime-bin) RUNTIME_BIN_DIR="${2:?}"; shift 2 ;;
@@ -301,6 +323,30 @@ copy_runtime_dir() {
echo "staged runtime dir ${name}/"
}
# Remove every channel = "developer" package from a staged catalog tree.
# <tree>/packages/<id>/<version>/manifest.toml is the layout; drop the VERSION
# directory that declares the channel, then the package directory once it has
# no versions left. Matching is on the manifest's own declaration so a package
# carries its own status -- nothing here maintains a list of names to exclude.
prune_dev_mods() { # prune_dev_mods <staged-catalog-root>
local tree="$1" manifest pkgdir verdir dropped=0
[[ -d "${tree}/packages" ]] || return 0
while IFS= read -r manifest; do
# Tolerate spacing/quoting variants; anchor to a top-level key so a
# feature description mentioning the word cannot trigger a drop.
if grep -Eq '^[[:space:]]*channel[[:space:]]*=[[:space:]]*"developer"[[:space:]]*$' \
"${manifest}"; then
verdir="$(dirname "${manifest}")"
pkgdir="$(dirname "${verdir}")"
rm -rf "${verdir}"
rmdir "${pkgdir}" 2>/dev/null || true
echo " excluded developer package: $(basename "${pkgdir}")/$(basename "${verdir}")"
dropped=$((dropped + 1))
fi
done < <(find "${tree}/packages" -mindepth 3 -maxdepth 3 -name manifest.toml 2>/dev/null)
return 0
}
if ((${#RUNTIME_DIRS[@]})); then
for d in "${RUNTIME_DIRS[@]}"; do
copy_runtime_dir "${d}" 0
@@ -337,6 +383,32 @@ for d in "${PROJECT_DIRS[@]}"; do
copy_proj "${d}"
done
# Developer-channel pruning runs AFTER every catalog is staged, and covers BOTH
# copies: the runtime catalog under mods/packages that the launcher lists, and
# the source catalog under mods/preloaded that the setup-host rebuild re-stages
# from. Pruning only the first would publish the package in source form and let
# the player's own rebuild put it straight back.
if [[ "${EXCLUDE_DEV_MODS}" -eq 1 ]]; then
echo "excluding developer-channel mods from this package"
prune_dev_mods "${STAGE}/mods"
prune_dev_mods "${STAGE}/mods/preloaded"
_dev_left=$( { grep -rlE '^[[:space:]]*channel[[:space:]]*=[[:space:]]*"developer"[[:space:]]*$' \
"${STAGE}" --include=manifest.toml 2>/dev/null || true; } | wc -l)
if [[ "${_dev_left}" -ne 0 ]]; then
echo "error: ${_dev_left} developer manifest(s) survived pruning:" >&2
grep -rlE '^[[:space:]]*channel[[:space:]]*=[[:space:]]*"developer"[[:space:]]*$' \
"${STAGE}" --include=manifest.toml >&2 || true
exit 1
fi
else
_dev_in=$( { grep -rlE '^[[:space:]]*channel[[:space:]]*=[[:space:]]*"developer"[[:space:]]*$' \
"${STAGE}" --include=manifest.toml 2>/dev/null || true; } | wc -l)
if [[ "${_dev_in}" -ne 0 ]]; then
echo "note: package includes ${_dev_in} developer-channel manifest(s);" \
"pass --exclude-dev-mods (or EXCLUDE_DEV_MODS=1) for a public release"
fi
fi
copy_tree_filtered() {
local src="$1" dest="$2"
shift 2