Matched Ape Escape retail SCPH1001 LLE builds identify 0661ce77 as the visible boot regression: parent 155e269b reaches game code, while 0661ce77 stalls near BIOS frame 375. Revert its four dependent follow-ups too; retain PR #326 readback bounds and PR #328 Vulkan aspect changes.
The row-by-row raster dispatcher amplifies an existing malformed DMA list into massive GL work. Keep the broader CPU/DMA arbitration experiment separate for compatibility testing. Document attribution, validation, and the temporarily restored interlace limitation.
Validation: full Ape target reaches title/menu and frame 11669 on retail LLE; OpenBIOS control reaches frame 3701. Eight focused tests pass; retained GL readback fixture passes 71 checks at both 1x and 4x after linking local SDL with iconv. Beads: beads-iisy; DMA follow-up: beads-eio.3.122.
Smart App Control blocks unsigned executables outright. The packager now
runs tools/ci/sign_windows.sh over the staged tree (host exe, DLLs,
emitters) with SHA-256 + RFC 3161 timestamp; no certificate in the
environment means a notice and an unsigned package as before, while a
configured certificate that fails to sign is fatal. The workflow template
passes WINDOWS_SIGN_PFX_BASE64 / WINDOWS_SIGN_PFX_PASSWORD to the package
step. Docs cover the secrets and the limit: the game exe the player builds
locally after Generate is unsigned by nature.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1dT6h1ZshMVUUVKTPxsCD
- Host may sit in a spectator seat: session slot 0 stays the sim authority
with a muted pad and no port; slot->port map keeps players on the port of
their lobby seat (host first). Rematch path copies the same plan.
- Seat self-service on LAN (MOTK5 SEATMOVE / SWAPASK / SWAPANS / SWAPRES)
relayed by the host; online swap requests accept a gallery seat as target.
- Lobby chat ring + join/leave lines; country codes from the server.
- Keybinds: a key drives one input per player (binding S to Cross unbinds it
from Circle), and keybinds.ini is re-read after the rematch launcher too.
- LAN registry file anchored to the exe directory.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1dT6h1ZshMVUUVKTPxsCD
Seat 1 (P2) can bring its LOCAL slot-1 card to a match: it uploads the card
to the host at launch over the new recomp-net MEMCARD op, the host installs
it as the match's slot-2 card, and the usual card broadcast carries it to
everyone. The host's real slot-2 file is rebound to netplay/guest_card2.mcd
for the session and restored on shutdown; guests already sandbox both slots.
Lobby side: a per-seat memcard_offer on set_ready (has_card, share), a host
allow flag in match_caps, and guest_memcard_active settled ONCE by the host
at start and shipped with the launch, so a toggle racing the start cannot
split the room. LAN carries the same over MOTK5 UPDATE / MEMCARD / JOIN
tail / a trailing START line. PSX_NET_GUEST_MEMCARD=1 for CLI sessions;
a host waiting on a card that never comes stalls at a named phase
(mc_guest_wait), not a silent "ok".
Two latent guest-sandbox bugs fixed on the way: a disabled slot 2 had no
bound path, so a host blob carrying a card for it failed to import and the
guest stalled forever; and leaving the sandbox re-bound a disabled slot to
the sandbox file. Both slots are now bound in the sandbox and restored per
slot (memcard_rebind_path, which can also unbind).
Lobby chat: the client keeps a ring fed by the server's echo (op:chat);
LAN rooms relay through the host (MOTK5 CHATREQ guest->host, MOTK5 CHAT
host->everyone). Wired to the launcher's chat_send/chat_count/chat_get.
Also carries the spectator-seat work that was in the tree alongside this
(spectator seats in the lobby client, spectator_wire_slot in the launch).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1dT6h1ZshMVUUVKTPxsCD
sh psxrecomp/tools/generate_ci.sh # in the project
powershell -File psxrecomp\tools\generate_ci.ps1
... [root] [--check] [--force] [--dry-run] [--zip-prefix X] [--title X]
Project Studio already knows how to emit the setup-host release workflow
(emit_ci_workflow, behind `apply` and `git install-ci`), but reaching it
meant a migration plan or a commit-and-push. This is the single-purpose
face of that operation: it resolves the template from the project's own
psxrecomp submodule, takes the zip prefix from
scripts/package_setup_release.sh so CI uploads the zips the packager
built, writes .github/workflows/release.yml, and stops.
--check reports missing / placeholder / stale (step names the template
has and the file lacks) with exit 1, so a fleet can be audited without
writing anything; only --force overwrites. The .sh and .ps1 are thin
launchers over the one Python implementation.
Verified read-only against two ports: ApeEscapeRecomp reports stale
(four emitter/ccache steps missing), TombaRecomp reports current.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0173EhspvAdRQc9DZdDr97Hg
The static isolated-fragment pass resolved demands by bare entry address across
every capture: once any part had a variant at an address, that address counted
as served for every other capture of the band, and static_entry_sources kept
only the last capture's bytes. The dispatcher validates each variant against
the exact bytes it was compiled from, so in a band with N occupants exactly one
occupant ever received fragments (BoF3: BOSS055 35/35, PLP678 19/19, SCENA19
20/20; the 128-occupant and 181-occupant bands got none). 23,728 (entry, image)
demands were unserved; at the title screen the resident SCENA16 ran every
observed entry on the interpreter while every spanning piece came from
SCENA19/04/06.
- Demands are keyed (entry, image crc); parts carry image_crc;
select_static_fragment_demands / served_static_variant_keys are the pure
helpers (unit-tested).
- Fragment compiles run on the --jobs pool; the image table is sent once per
worker (initializer) instead of once per demand; results merge in demand
order and duplicates already served by an earlier-merged part are dropped,
so the output equals the sequential loop's.
- generate_interior_fragment_static returns (part, reason). Deterministic
verdicts on the bytes (generated-C audit, requested-entry audit, and a walk
that runs off the image, which the recompiler reports by throwing) are
memoized in <out-dir>/interior_fail_memo.txt and counted as skipped, never
as shard failures; --force-interior retries an entry. Only a
non-deterministic recompiler failure is a shard failure.
- Fragment parts are grouped one translation unit per image (BoF3: 779 units
instead of 20,635 files).
BoF3 full run: 20,250 fragments built, 3,570 memoized rejections, 238 s cold,
nothing lost from the previous dispatcher (99,778 -> 138,148 served keys).
Title screen: 12 interpreted SCENARIO-band entries -> 2, both unharvested PCs.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Make a bare interpreted PC explainable from the session-long per_pc snapshot
alone. DirtyRamPcEntry gains occ_crc / occ_ok (the static overlay variant whose
code ranges span the PC, via a per-PC scan of the static-match cache, and
whether it validated at that moment) and last_ext_ra (gpr[31] at the most
recent external entry, i.e. the caller that reached this interior). Both are
stamped only on external entries -- arrivals from native dispatch, not interp
block chaining -- and emitted per row in dirty_ram_stats.per_pc as occ_crc,
occ_ok and ext_ra.
Reading the pair: occ_ok=1 is an interior gap inside live native code (an
alias seed fixes it); occ_ok=0 with a CRC means the spanning piece is resident
but CRC-missing, so it was compiled from another section's bytes or rewritten
at run time and no seed helps; 0 means nothing compiled spans the PC.
psx_overlay_resident_crc_at() records each static-match cache entry's code
span (lo_min/hi_max) so the scan is a range test; DLL-path titles fall back to
the loader's last residency hash. The debug-server reply buffer grows 32 -> 64
KiB for the wider rows; the tail reserve still cuts between rows.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The repository root carried seven markdown files that were not onboarding
material, plus two that had been superseded by the code they described. This
sorts them: reader-facing docs to docs/, working notes to docs/internal/, and
the two superseded ones merged into the docs that replaced them.
Moved to docs/: WIDESCREEN.md, ENHANCEMENTS.md, TCP_COMMANDS.md
Moved to docs/internal/: PRINCIPLES.md, ISSUES.md, DEBUG.md, GTE_LM_FIX.md,
and from docs/: bios-hardening.md, overlay-status.md,
overlay-plan.md, overlay-recompilation-design.md,
overlay-discovery.md, TOMBA2_EXC_CONSULT_RESPONSE.md,
CASE_A_AOT_GAP.md, audit_inventory.md
Moved to docs/internal/upstream/: UPSTREAM_2026-08-20.md
The root keeps only what belongs there: README.md, CONTRIBUTING.md, LICENSE,
CLAUDE.md, THIRD_PARTY_ATTRIBUTION.md, RELEASE_NOTES.md, the dotfiles and
psxrecomp_cli.py.
MERGES
SPLITGEN_MIGRATION.md folded into docs/BUILDING.md as "Split generated C". Its
migration instructions were stale in a way that mattered: it told every game
repo to hand-edit its CMakeLists and add a file(GLOB ...) over
<serial>_full*.c, but runtime/runtime.cmake now does that itself. GEN_FULL_GLOB
is a multi-value argument of psxrecomp_add_game_runtime(); the loop at
runtime.cmake:1985 makes each pattern absolute against CMAKE_CURRENT_SOURCE_DIR,
expands it with file(GLOB), and concatenates the hits into
GAME_GENERATED_FULL_C. So a title names the pattern and gets however many shards
a regeneration produced. The rewritten section describes that, the two-step
fallback when the globs match nothing (GEN_FULL_FALLBACK, else <serial>_full.c
derived from the dispatch marker by substituting _dispatch.c), and the one
caveat the code actually has: that file(GLOB) is plain, not CONFIGURE_DEPENDS,
so a regeneration which changes the shard COUNT needs a re-configure to be seen.
psxrecomp-toml-readme.md folded into docs/GAME_PROJECT_SETUP.md as "EXE-only
autofill (psxrecomp-toml)", placed after "Disc autofill (probe_disc.py)" and
before "Project Studio" so the two autofill paths sit together. Its documented
flags were checked against the usage text in recompiler/src/main_toml.cpp:28-42
and against the argument parser at :150-165 -- all seven options match with no
drift, so the flag list is reproduced as-is. Added a pointer that probe_disc.py
is preferable when a cue exists, since that tool does the same EXE-header work
plus disc identity and netplay gates.
FUNCTIONAL FIXES
tools/gen_tcp_commands.py wrote to REPO/"TCP_COMMANDS.md". After the move that
path does not exist, so the generator would have created a new stray root file
and --check would have failed against it forever. Retargeted to
REPO/"docs"/"TCP_COMMANDS.md", along with the seven docstring and message
strings that name the path.
Running it surfaced two further problems, both fixed here:
* The doc was genuinely stale -- pre-existing drift, not introduced by this
commit (the moved file is byte-identical to origin/master, and no
debug-server command table is touched here). Four commands registered since
the last regeneration were missing from the index -- cdrom_timing_dump,
savestate_status, ws_ui_groups, xprobe_watch -- and scanline had gained a
backend. Regenerated: 307 -> 311 commands, and the change is confined to the
AUTOGEN block (the hand-written head and tail are byte-identical to master).
tools/gen_tcp_commands.py --check now exits 0.
* DOC.write_text() has no newline argument, so Python translated every newline
to os.linesep. On Windows that rewrote all 585 lines of the file to CRLF and
turned a four-line index update into a whole-file diff -- measured: 589 CRLF
/ 0 LF after a run, against 0 CRLF / 585 LF in the repo. Now written through
an explicit newline="" handle, verified to leave the file at 0 CRLF.
runtime/include/debug_server.h:9 and runtime/src/debug_server.c:8 cited "TCP.md
and DEBUG.md". TCP.md has never existed in this repo under that name; both now
name docs/TCP_COMMANDS.md and docs/internal/DEBUG.md.
LINK AND REFERENCE FIXES
Every inbound reference to a moved file was located by grepping each basename
across the whole tracked tree, not just the markdown. Markdown hrefs in
README.md, CONTRIBUTING.md, docs/ARCHITECTURE.md, docs/COMPILING_OVERLAYS.md,
docs/AOT_OVERLAY_PLAN.md, docs/RELOCATION_MANIFEST_FORMAT.md,
docs/config_schema.md, docs/OVERLAY_XREF.md, docs/TCP_COMMANDS.md and several
docs/internal/ files now point at the new locations; where two files became
siblings under docs/internal/ the path was shortened rather than lengthened
(bios-hardening.md's reference to overlay-discovery.md, for instance).
docs/COMPILING_OVERLAYS.md's "in this folder" phrasing was wrong once two of the
three files it listed moved, so it now names internal/ explicitly.
24 source-comment mentions of ENHANCEMENTS.md sections G1.x across 19 files
(recompiler, runtime, cmake) were retargeted to docs/ENHANCEMENTS.md with the
section numbers untouched. docs/SHADOW_ENHANCEMENTS.md is a DIFFERENT file and
was deliberately excluded.
20 rationale strings citing docs/CASE_A_AOT_GAP.md across the four
recompiler/seeds/*.json files were updated. Done as a byte-level substring
replacement rather than a JSON round-trip: two of those files are stored with
CRLF endings and two with LF, and re-serializing them reformatted 5,700 lines.
The byte-level version changes exactly 20 lines and each file still parses.
References that look similar but point elsewhere were deliberately left alone,
and are worth naming so a future pass does not "fix" them:
* "recomp-template PRINCIPLES.md" / "NES recomp template PRINCIPLES.md" in
CLAUDE.md:43, docs/SHADOW_ENHANCEMENTS.md:28,
docs/internal/HLE_SCHEDULER_CARVEOUT_PLAN.md:5,
runtime/src/debug_server.c:4934 and tools/parity_diff.py:8 are a different
repository's file. Only the five that mean this repo's PRINCIPLES.md were
updated.
* Every remaining bare ISSUES.md mention is a GAME repo's tracker
(MegaManX6Recomp 2/7/8/9, TombaRecomp 6), not this repo's.
* docs/disc-speed.md:4,54,56 and docs/internal/bios-hardening.md:5 name
overlay-discovery as a BRANCH, not a file, so no path applies.
INDEXES
README.md's "New here?" table gains rows for docs/WIDESCREEN.md,
docs/ENHANCEMENTS.md and docs/TCP_COMMANDS.md. docs/internal/README.md gains
sections indexing all fourteen documents that moved in, plus an "Upstream
contributor reviews" section describing upstream/ and the new sweep file, plus a
"Removed" bullet for CYCLE_TIMING_ARCH.md (deleted in the previous commit)
pointing at FAITHFUL_TIMING_PLAN.md, CYCLE_MODEL_BEETLE.md and
PRECISE_IRQ_SLICE.md -- following the SLJIT_PERSIST_CACHE.md precedent already
in that file.
docs/ENHANCEMENTS.md:7 claimed framework bugs live in "game repos' ISSUES.md
until a framework tracker exists". A framework tracker does exist; it now says
so and names the Beads epic.
VERIFICATION
A link checker was written to scan every tracked *.md for relative links,
resolving each against the tree being scanned. Baseline at origin/master: 129
files, 137 relative links, 3 broken. After both commits: 127 files, 159 relative
links, 3 broken -- the SAME three, all pre-existing and untouched here:
docs/DATA_SHARDS.md -> INSTANT_LOADS_PARKED.md (never existed)
docs/DATA_SHARDS.md -> INSTANT_LOADS_SPIKE2.md (never existed)
docs/NETPLAY.md -> ../lib/recomp-net/README.md (inside a submodule)
Zero new broken links. The checker's own coverage was checked against a raw grep
for markdown link openers: the 3-link difference is C++ lambdas inside code
blocks, correctly not treated as links.
Also verified: py_compile over all 113 tracked tools/*.py plus psxrecomp_cli.py;
all four seed JSONs still parse; the test-registration guard still reports 123
test files all registered; recompiler/tests/test_coverage_vault.py still passes;
and no file in this commit had its line endings flipped (checked mechanically
across all 50 modified files, after two files were caught and redone).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Removes 204 tools/ scripts (203 .py/.sh plus tools/_missing_funcs.json) that
no tracked file references. Verified mechanically, not by eye: for each of the
204 stems, a word-boundary grep across every tracked text file in the repo,
excluding files that are themselves being deleted, returns zero hits. So
nothing kept imports, execs, or documents any of them. Most are the
underscore-prefixed one-shot probes the .gitignore convention already treats
as scratch (tools/_*.py is ignored; these predate that rule and were force-
added), plus a long tail of session-specific dump/decode/poll scripts with
hardcoded local paths and addresses.
Also removed at the root:
* card2.mcd.bak.before_beetle_copy -- a stray backup of a memcard.
* .mcp.json -- local MCP server wiring, not repo configuration.
* SCPH1001.toml -- a root duplicate of bios/SCPH1001.toml, still on the
pre-refactor [[recompiler.bios_vectors]] schema while the bios/ copy has
moved to [recompiler.address_model]. Every reference in the tree already
names bios/SCPH1001.toml, so the root copy was a stale trap: editing it
would have had no effect on any build.
* docs/CYCLE_TIMING_ARCH.md -- superseded by
docs/internal/FAITHFUL_TIMING_PLAN.md, CYCLE_MODEL_BEETLE.md and
PRECISE_IRQ_SLICE.md; nothing linked to it.
Memcard fixtures move out of the repo root into
runtime/tests/fixtures/memcards/ with a README recording what each of the
four images actually contains (directory entries read out of the images, not
guessed). Nothing in the build read them from a fixed path -- the four
hardcoded consumers (tools/check_card.py, inspect_card.py, peek_cards.py,
verify_card_buffer.py) are all in the deletion above.
.gitignore is corrected to match: runtime memcards a session creates stay
ignored (*.mcr anywhere, /*.mcd at the root, which is where the runtime drops
card1.mcd/card2.mcd when no path is configured), while the tracked fixtures
are re-included. The old !dummy.*.mcr negation existed only to keep the two
root oracle cards tracked and is replaced by
!runtime/tests/fixtures/memcards/*. Verified with git check-ignore --no-index
on each moved file, on the pre-existing runtime/tests/test_card_synth.mcd and
runtime/tests/test_dir/card1.mcd fixtures (all not-ignored), and on
hypothetical root card1.mcd / card2.mcd / saves/card1.mcr (all ignored).
beetle_libretro.cpp's warn-only memcard sanity check keeps its behaviour; its
WARNING now names where populated fixtures can be copied from. The cards are
opened relative to the process's working directory because the libretro
GET_SAVE_DIRECTORY environment callback answers "." (same file, line 419), so
the message says to copy them into the CWD rather than implying a repo path.
BIOS profiles SCPH101.toml and SCPH5552.toml move from the root into bios/,
joining OpenBIOS.toml and SCPH1001.toml so that bios/*.toml is the whole set.
Behaviour-preserving: config paths inside a profile resolve against
find_project_root() (config_loader.cpp:821), which walks up from the profile's
directory for .gitignore/.git/CMakeLists.txt -- bios/ has none of those, so it
lands on the same repo root as before, and bios/SCPH1001.toml already relied
on exactly that. tools/build_cli.py's staging map is updated to the new
sources; it already copied them to framework/bios/, so the packaged layout is
unchanged. recompiler/src/main_cli.cpp's bios/-first lookup keeps its root
fallback (harmless, and it is what makes older layouts still resolve).
Fixes a real latent break while in bios/SCPH5552.toml: seeds pointed at
"seeds/phase2_ghidra_seeds_SCPH5552.json", but there is no seeds/ directory at
the repo root -- the file lives at
recompiler/seeds/phase2_ghidra_seeds_SCPH5552.json, which is what SCPH101.toml
already named. Porting these two profiles to the address_model schema is
deliberately NOT done here; it is a separate tracked follow-up.
Test files that no build system knew about:
* tools/test_coverage_vault.py PASSES (8/8). Moved to
recompiler/tests/test_coverage_vault.py and registered as the ctest
coverage_vault, which puts it inside runtime/check_test_registration.cmake's
glob so it can never silently rot again. Its ROOT is adjusted from
parents[1] to parents[2] for the new depth, matching every other python
test in that directory. Re-run from the new location and from an unrelated
CWD: 8/8. The registration guard was run standalone before and after, and
with the add_test() deliberately broken as a negative control -- it does
name recompiler/tests/test_coverage_vault.py as an orphan when
unregistered, so the registration is load-bearing rather than decorative.
* tools/test_compile_overlays_additive.py FAILS, 10 of 14 tests erroring
against an API that has moved on, so it is removed rather than wired up.
It calls four module attributes that no longer exist in
tools/compile_overlays.py -- load_additive_captures,
validated_cached_bundle_ids, cached_bundle_pairs, compile_and_publish_dll
(zero occurrences each) -- and two signatures have changed under it:
parse_overlay_func_ids() no longer takes require_stored_crc, and
load_region_coverage() now takes 2-3 positional arguments where the test
passes 5. Reviving it means rewriting it against the current additive
capture API, which is a change to overlay-cache test coverage and needs its
own commit and its own evidence, not a rename.
Verification: py_compile over all 113 remaining tracked tools/*.py plus
psxrecomp_cli.py passes; bash -n over every remaining tracked tools/*.sh
passes; importing sdk_progress, pad_delivery and cursor_writers from tools/
succeeds.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
4cc04be3 ("rebump") renamed the staged mod catalog from <exe>/mods/packages to
<exe>/mods/bundled so that a rebuild could wipe build output without deleting
the player's own installed mods. runtime.cmake followed the rename. The shared
release module followed it (release_overlay_stage.ps1's Add-ModCatalog reads
mods/bundled). docs/MOD_PACKAGES.md already described the destination as "the
framework's mods/builtin/packages PLUS the title's mods/preloaded/packages".
No title CMakeLists followed, because the second half of that sentence was
implemented five times, once per repository, as a hand-written
add_custom_command(TARGET psx-runtime POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_directory
"${<TITLE>_PRELOADED_MODS}" "$<TARGET_FILE_DIR:psx-runtime>/mods")
in ApeEscapeRecomp, Tomba2Recomp and MegaManX4/5/6Recomp. That names the
destination as a STRING. There is no symbol, no header, no link dependency --
so renaming a staged output directory breaks nothing a configure, compile or
link step can see, and the diffstat of the commit that does it looks cosmetic.
The five titles went on staging into a directory neither the launcher nor any
packager reads, and discovery was deferred to whichever title next ran a
release packager, in a different repository, on a later day.
MEASURED, on real build output. Framework origin/master 22fbbfca plus
Tomba2Recomp origin/master cdf00af, both unmodified, both built:
build-dir mods/
bundled/psx.enhancement.cd-speed <- framework, correct
bundled/psx.enhancement.fast-loading
bundled/psx.enhancement.pgxp
bundled/psx.presentation.bezel
packages/tomba2.debug.debug-menu <- title, stranded
packages/tomba2.enhancement.skip-fmvs
packages/tomba2.enhancement.widescreen
packages/tomba2.experimental.interpolated-frame-rate
and Add-ModCatalog run against that tree, from this commit's parent:
Mod catalog is missing package(s) the sources define:
tomba2.debug.debug-menu, tomba2.enhancement.skip-fmvs,
tomba2.enhancement.widescreen, tomba2.experimental.interpolated-frame-rate.
They exist in the source tree but did not reach ...\mods\bundled, so the
release would ship a Mods page the dev build does not have.
Shipped releases are NOT affected. Tomba 2 v0.0.9 pins 04d9184b, which is an
ancestor of 4cc04be3 and contains zero occurrences of migrate_legacy_root and
bundled_root; its artifact is a self-consistent mods/packages layout. This is a
forward break only.
THE FIX PUTS THE LAYOUT IN ONE PLACE.
psxrecomp_add_runtime_target() gains a one-value keyword PRELOADED_MODS_DIR.
A title declares WHERE its packages live and never HOW they are laid out:
set(APE_PRELOADED_MODS "${CMAKE_CURRENT_SOURCE_DIR}/mods/preloaded")
psxrecomp_add_runtime_target(psx-runtime
...
PRELOADED_MODS_DIR "${APE_PRELOADED_MODS}")
A keyword argument rather than a positional one or a variable read out of the
caller's scope, for three reasons. It is what the other fourteen one-value
arguments already are, so it needs no new convention and reads the same as the
LAUNCHER_BOXART line next to it. A pre-set variable would be invisible at the
call site and would silently do nothing if set after the call -- the same
"fails quietly in the caller's repository" shape as the bug being fixed. And it
is PER TARGET, which a global variable cannot be: Tomba 2 has two runtime
targets needing two different catalogs (mods/preloaded, mods/preloaded_ita).
psxrecomp_v4_add_runtime_target and psxrecomp_add_game_runtime both forward
their remaining arguments, so it reaches through both wrappers unchanged.
_psxrt_stage_mod_catalog() now emits, in one POST_BUILD command sequence per
runtime target: wipe mods/bundled; remove exactly the ids it is about to stage
from any pre-existing mods/packages; copy the framework builtins; copy the
title's packages; copy <dir>/README.md to mods/README.md; verify. Consequences
worth stating:
* mods/README.md previously reached mods/ only as a side effect of copying
the whole mods/preloaded tree. Now that only packages/ is staged it is
copied on purpose, or it would silently disappear. Checked per title: all
five carry mods/preloaded/README.md and none carries a root mods/README.md,
so mods/preloaded/README.md is the only source there ever was.
* a title may still OVERRIDE a framework builtin at the same id and version,
because the framework copy runs first and the title copy overwrites it.
Tomba 2's Italian catalog relies on exactly this, shipping localized psx.*
manifests. The id LIST is deduplicated for the assertion and the reported
count while the copies keep both, or the build claims to stage 11 packages
for a catalog that holds 7.
* COSIM targets stage nothing. They have no launcher and no Mods page, and
they land in the same output directory as the real runtime, so letting one
stage had it wipe and re-stage the runtime's catalog with only the
framework half whenever -DPSX_BUILD_COSIM=ON built it last.
TWO GUARDS, because the failure mode being closed is silence.
Configure time: a project with packages under mods/preloaded/packages that does
not declare PRELOADED_MODS_DIR is a FATAL_ERROR naming the packages and the
argument to add. That is exactly the missed-title state and it is detectable
before any build runs. An empty packages/ is fine (the scaffold creates one),
and PRELOADED_MODS_DIR NONE is the explicit opt-out.
Build time: runtime/psx_check_mod_catalog.cmake fails the build if a declared
package did not reach mods/bundled, or if any package this build stages turned
up under mods/packages -- which after this change can only mean a CMakeLists
still stages its own catalog by hand. It runs as the LAST POST_BUILD step, via
cmake_language(DEFER), because POST_BUILD commands execute in registration
order and the framework block registers BEFORE anything a title adds after its
psxrecomp_add_runtime_target() call. Verified by inspecting build.ninja: the
deferred command lands after a title-registered POST_BUILD, not before it.
The deferred pass takes NO arguments and reads its work list out of global
properties. cmake_language(DEFER CALL <fn> "${var}") does not carry a
function-local variable through -- measured with cmake 4.2.2, the callee
receives an empty argument and an add_custom_command built from it fails at
directory end with "CMakeLists.txt:DEFERRED". Recorded so the next reader does
not re-derive it.
Ids the build does NOT stage are deliberately left alone in mods/packages. On a
self-compiling setup release that directory can hold packages the player
installed under the pre-split layout, and mod_packages.cpp's
migrate_legacy_root() relocates those into mods/installed on the next launch.
Removing only the ids this build stages is also what lets a build directory
made before the split be migrated silently, so the guard's "an id turned up in
mods/packages" report has no false positives.
The same script is registered as the ctest psx_staged_mod_catalog_test in every
project that stages a catalog, with REQUIRE_STAGED=0 so a tree where the
runtime was never built reports a skip rather than a spurious failure. Where
two runtime targets share an output directory it accepts either target's
complete catalog, since after a full build mods/bundled holds whichever linked
last -- which is correct for the packager, as it builds exactly one target per
variant, and each target's own POST_BUILD guard already verified its catalog at
stage time.
TESTS. runtime/tests/test_mod_catalog_layout.py exercises the guard against
on-disk fixtures: a reproduction of the broken state as it actually appeared
(bundled/ holding the four psx.* ids, packages/ holding the four game ids), a
declared package simply absent, a package directory with no manifest.toml, a
FOREIGN legacy package that must NOT fail the build, the require-staged split
between the two callers, and the shared-output-directory sibling case. It also
pins two source invariants in runtime.cmake so the layout cannot drift back
into per-title copies. Registered in runtime/CMakeLists.txt; the
test-registration guard reports 123 test file(s), all registered.
Framework ctest: 59/59 pass, 2 pre-existing DISABLED.
Also updated so the next title does not repeat this: the project scaffold
template now passes PRELOADED_MODS_DIR (it staged no catalog at all before, so
a scaffolded project with mods/preloaded/packages shipped none), and
docs/MOD_PACKAGES.md gains a "How bundled/ gets staged" section written at
titles.
Bead beads-eio.3.101. The five title-side patches must land AFTER this commit:
they reference PRELOADED_MODS_DIR, which does not exist before it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Static mode (--static) was serial end to end and produced one translation
unit, so a whole title compiled on one core twice: once in Python, once in
gcc. Measured on BoF3 (339 captures, 79,688 identities) in a build-dbg loop:
phase "compile all overlay bands" ~12 min, runtime build ~4 min at -O0.
Three changes, each independently verified byte-identical:
1. static_capture_job(): the per-capture static work (recompiler subprocess,
C audit, range audit, namespacing) is now a module-level, picklable job
that returns its part + captured log. --jobs runs it in a
ProcessPoolExecutor; results merge in capture order, so static_parts and
entry sources are order-identical to the sequential loop. --jobs 1 and
--jobs 3 on three bands: diff -r of the outputs is empty.
2. add_cps_resume_cases(): the universal CPS resume-wrapper pass called
add_cps_resume_case() once per entry, and each call re-scanned and
rebuilt the WHOLE part source. cProfile on three bands: 7,386 entries ->
22,159 re.search calls, 366 s of 424 s total. The batch form locates
every signature once, edits each host's segment in isolation, rebuilds
once. Same three bands: 418 s -> 5 s, outputs byte-identical (diff -r
against the per-entry output). The per-entry function is kept; a unit
test asserts the two agree on fresh-switch, existing-switch, missing
label and missing host cases.
3. write_static_outputs(): overlays_static.c now holds the dispatcher plus an
extern prototype per compiled entry, and every part becomes its own unit
overlays_static_NNNN.c (each already carries its #includes and a private
namespace). runtime.cmake globs the siblings with CONFIGURE_DEPENDS, so a
changed part count needs no reconfigure. Unchanged units are not
rewritten (mtime preserved -> not recompiled); stale units are removed.
--static-single-file restores the monolithic file.
Result, same BoF3 loop on a 12-core box: phase 5a 12 min -> 20 s, runtime
build 4 min -> 69 s (358 units in parallel), whole loop ~16 min -> 90 s.
Shard summary unchanged (332 ok / 7 benign audit fails), identity count
unchanged. Headless boot of the rebuilt runtime: 485,650 static checks,
485,244 hits (99.92%), miss_total 0, 60 emu fps through the LOGO overlay.
Note: tools/test_compile_overlays_additive.py already fails on master (it
references removed functions: compile_and_publish_dll, load_additive_captures,
validated_cached_bundle_ids); untouched here.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Follow-ups to the marker-skew hardening, closing the remaining legs of
the generate+build loop class:
- Loop breaker: host_prepare_generate stamps a successful generate into
.psxrecomp_last_generate at the project root. When apply() is about to
reopen the wizard because sources are missing and the stamp is recent,
the wizard's prepare note becomes a diagnostic naming exactly which
gate is unmet (the expected *_dispatch.c and/or the BIOS backends) and
what generated/ actually contains, instead of silently asking for
another round. The stamp is dropped once sources are visible so a much
later manual wipe of generated/ reads as a fresh first run.
- Preflight: generate now refuses up front when the project root sits
under the Windows temp directory (Explorer's run-from-inside-the-ZIP
extraction, which evaporates) or is not writable, with instructions,
instead of failing downstream in ways that present as the same loop.
- CI lint: tools/ci/check_boot_exe.sh cross-checks the three baked
copies of the boot-EXE name (game.toml boot_exe/exe, CMakeLists
GEN_MARKER, codegen_setup.c gen_marker_relpath) and the setup-release
template gains a fail-fast "Lint boot-EXE name consistency" step, so a
skew fails the maintainer's release instead of the player's install.
op_emit_ci_workflow's step-name drift detection flags older titles
automatically. Script is bash-3.2-clean for macOS runners; verified
against fixture repos (consistent / skewed / boot_exe override /
single-source) and real ports - where it immediately caught
Tomba2Recomp's placeholder gen_marker_relpath ("SLUS_01234" vs
SCUS_944.54), a live instance of the reported loop.
Both gitignore templates learn the new sidecar names.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KfPHUBEYiSCwaGiwRc8ea7
Matthew Stanley