Merge remote-tracking branch 'origin/master' into split/pr169-presentation-bezel

# Conflicts:
#	runtime/runtime.cmake
This commit is contained in:
Matthew Stanley
2026-08-24 15:16:40 -07:00
28 changed files with 3677 additions and 138 deletions
+203
View File
@@ -0,0 +1,203 @@
# Host optimization contract
This document is the acceptance contract for changes intended to make
psxrecomp run faster on the host. It applies to generated code, the runtime,
the flat CPS trampoline, overlay dispatch and validation, graphics and audio
support code, and host-side caches or fast paths.
Correctness fixes and deliberate guest-visible enhancements still need focused
verification, but the 5% performance retention gate below applies specifically
to optimizations.
## Non-negotiable rules
1. Measure on a quiet host with zero compiler processes. Finish all AOT,
overlay, shader, and helper compilation before the timed region. Confirm
that no compiler, linker, build driver, or autocompile worker is alive while
sampling. A run in which one appears is contaminated and is discarded, not
explained away.
2. Interleave A and B. Never measure all baseline runs and then all candidate
runs. The minimum acceptable order for three pairs is `A1 B1 A2 B2 A3 B3`;
reversing the first member of alternate campaigns is encouraged. Keep the
same binary and use a forced runtime selector when practical.
3. Report every raw run, the min-of-N for A and B, the percentage change
between those minima, both medians, and the percentage change between the
medians. N is at least three. State whether lower or higher is better.
4. Discard a pair when either member is contaminated. Reasons include a
compiler process, overlay compilation, shader-cache construction, a changed
power or thermal state, unexpected foreground load, a different overlay
population, or a different pacing owner. Record the discarded pair and its
reason in the experiment ledger.
5. A candidate that adds complexity, persistent state, generated-code ABI,
configuration or debug ABI, or another maintained fast path is retained
only when the agreed primary metric improves by at least 5%. A pure
simplification may be retained at flat cost, but its measured numbers must
still be recorded. Correctness fixes are judged on correctness, not this
speed threshold; do not disguise an optimization as a correctness fix.
6. With the optimization forced on for every eligible operation, guest-visible
state must be byte-identical to the faithful path at deterministic
checkpoints. An automatic fallback is not proof: it can hide the exact case
the fast path mishandles.
7. The faithful reference computation remains linked and selectable. A fast
path is an implementation of that same computation, not a replacement model
that merely looks plausible for one title.
8. Do not copy another console project's opcode handlers. It is valid to reuse
its experimental process, evidence format, test structure, or review
checklist. Implement R3000A behavior from PlayStation evidence and
psxrecomp's own faithful path.
## Preparing comparable psxrecomp runs
Prefer one executable containing both paths and an explicit selector with
three states: faithful, candidate forced on, and normal policy. Record the
selector and assert in telemetry that the requested state took effect. If two
binaries are unavoidable, build both before the campaign and record their
commits, toolchains, options, and artifact hashes.
Static recompilation makes build quietness part of the benchmark definition.
Pre-generate the game and BIOS images, warm every overlay needed by the
workload, and prevent runtime autocompile during the measured interval. Record
the static image identity and the exact overlay inventory. A benchmark is not
comparable if A interprets an overlay while B uses a compiled shard.
Content-verified overlay variants require an additional check. A and B must
use the same captured bytes, identity ranges, flavor, cache generation, and
selected variant at each checkpoint. Pin and report the overlay flavor. A fast
identity check may avoid a CRC or range scan, but the faithful verifier must
remain callable and both must select the same content-verified variant.
The flat CPS trampoline is part of observable execution semantics. Compare the
published continuation PC, dispatch result, exception redirect, dispatch depth,
and same-site resume consumption at block and function boundaries. A host
function returning with `cpu->pc == 0` is not equivalent to tail-publishing the
next guest PC, even if common images never cross that boundary.
Use a deterministic workload with fixed inputs and a fixed capture or replay
when possible. Keep renderer, audio backend, window state, display refresh,
vsync/pacer ownership, CPU overclock, CRTC multiplier, mods, RAM size, and
instrumentation identical. Warm caches outside the timed region unless cold
behavior is the stated metric.
## Correctness proof
The forced-candidate run must exercise the candidate, and its checkpoints must
match a run forced through the faithful reference. Hash or serialize all state
that can affect later guest execution, including:
- GPR, HI/LO, PC, COP0, GTE registers and completion timestamps;
- main RAM, scratchpad, VRAM, SPU RAM, and relevant device registers;
- the R3000A cycle counter, deferred cycle charges, event deadlines, IRQ state,
load-delay state, and pending exception continuation;
- dispatch and overlay identity state that changes which guest bytes execute;
- deterministic mod-owned guest state and savestate-visible sections.
Host-only counters may be excluded only when they cannot influence selection,
timing, invalidation, or later guest state. Document every exclusion.
Compare at more than the final frame. Include checkpoints before the candidate
first fires, immediately after it fires, at overlay or exception boundaries it
touches, and after a long soak. For caches and memos, force hits, misses,
invalidation, wrap or saturation, and savestate/rewind restore. For generated
code, cover the game emitter, BIOS full-function emitter, compiled overlays,
static overlay bakes, and the dirty interpreter wherever the contract crosses
execution tiers.
For cycle-affecting work, use `tools/cycle_testrom/measure.py` as the existing
measurement seam. Run the same test-ROM anchors against the native compiled
path, `PSX_FORCE_INTERP=1`, and the Beetle oracle as applicable. The script's
per-iteration deltas establish guest-cycle correctness; they do not replace a
separate interleaved wall-time or throughput campaign for host performance.
The faithful per-instruction R3000A cycle model remains authoritative: an
optimization may reduce host work but may not remove base charges, wait states,
interlocks, GTE completion timing, I-cache behavior, or peripheral deadlines.
## Performance campaign and evidence
Choose the primary metric before running the campaign. Frame or workload time
is preferred when the work is bounded; throughput is acceptable for a fixed
steady-state loop. Report secondary metrics, but do not change the retention
decision after seeing the results.
The evidence block for every candidate contains:
```text
Candidate: <short name>
Commits/artifacts: A=<hash> B=<hash> selector=<setting>
Workload: <capture, interval, overlay inventory, backend and host>
Primary metric: <name and unit; lower/higher is better>
Run order: A1 B1 A2 B2 A3 B3
Raw A: <a1>, <a2>, <a3>
Raw B: <b1>, <b2>, <b3>
Min-of-3: A=<min-a> B=<min-b> change=<percent>
Medians: A=<median-a> B=<median-b> change=<percent>
Correctness: faithful=<hashes> forced-on=<hashes> result=<identical/not identical>
Contamination: <none, or discarded pairs and reasons>
Complexity/ABI: <none, or exact added surface>
Decision: ACCEPT/REJECT - <reason>
```
Worked example, using workload time in milliseconds where lower is better:
```text
Run order: A1 B1 A2 B2 A3 B3
Raw A: 12.000, 11.000, 13.000 ms
Raw B: 9.744, 9.493, 10.100 ms
Min-of-3: A=11.000 B=9.493; improves 13.7%
Medians: A=12.000 B=9.744; improves 18.8%
Correctness: faithful=7a... forced-on=7a...; byte-identical
Complexity/ABI: two per-variant words plus one generated helper call
Decision: ACCEPT - both summaries exceed the 5% retention gate
```
Do not report only a profiler share, a best screenshot, an FPS range, or a
sequential before/after observation. Those are useful for finding candidates,
not for accepting them.
## Experiment ledger
Every investigated candidate gets a ledger row, including rejected,
superseded, reverted, and flat candidates. The ledger is append-only evidence,
not a highlights page. Keep it beside the optimization or campaign notes and
include links to raw data when available.
Required fields are:
| Field | Required contents |
|---|---|
| ID and date | Stable candidate ID and campaign date |
| Change | Commit(s), artifact hashes, selector, and concise hypothesis |
| Scope | Workload, renderer/audio backend, host, toolchain, config, overlay inventory and flavor |
| Correctness | Faithful and forced-on checkpoint hashes; exact exclusions |
| Runs | Interleaved order, all raw accepted runs, and every discarded pair with reason |
| Summary | A/B min-of-N and medians with percentage changes |
| Cost | Added code, state, ABI, configuration, maintenance burden, or simplification |
| Verdict | Accepted or rejected, threshold used, and reason |
Accepted and rejected results are equally visible. A compact ledger index may
look like this, with each row linking to its full evidence block:
| ID | Min-of-N change | Median change | State result | Cost | Verdict |
|---|---:|---:|---|---|---|
| EX-A | improves 13.7% | improves 18.8% | byte-identical | two per-variant words and one helper call | ACCEPT - clears 5% |
| EX-R | improves 3.0% | improves 1.0% | byte-identical | new cache and invalidation path | REJECT - below 5% |
Use an explicit rejected entry even when the candidate was reverted before a
formal campaign. Record the numbers that motivated the experiment and state
which required evidence is missing. Do not silently delete failed ideas: the
ledger prevents the same attractive but invalid optimization from being tried
again without new evidence.
## Review checklist
- Host was quiet and had zero compiler processes during every retained run.
- A/B was interleaved; contaminated pairs were discarded and recorded.
- Raw runs, min-of-N, and medians are present.
- The forced candidate and faithful reference produced byte-identical
guest-visible state.
- The faithful path remains linked and selectable.
- Overlay identities and execution tiers were held constant and reported.
- Cycle-model changes were checked through the cycle-test-ROM seam.
- Added complexity or ABI clears 5%; flat pure simplifications have numbers.
- Accepted and rejected candidates are both in the experiment ledger.
- R3000A implementation was derived independently; no other console's opcode
handlers were copied.
+262
View File
@@ -0,0 +1,262 @@
# Upstream contribution audit — 2026-08-20
## Summary
This worktree is a cross-cutting psxrecomp framework hardening and host-performance pass discovered while bringing up WipEout 3 SE: it fixes overlay-DLL generation and byte-variant identity, makes cycle/IRQ/cache shortcuts preserve guest-observable timing, closes several PGXP/GTE provenance holes, extends HD-texture tracking to copied VRAM, separates presentation content policy from the configured outer canvas, and adds reusable profiling/test surfaces. The mechanisms are generally applicable to recompiled PS1 titles; the few title-specific observations and enablement conclusions are isolated below and should not be upstreamed as framework policy.
## Audit basis and limits
- The audited delta is the **current working tree against local `HEAD` `4f0b07b1491e` on `feat/ws-cull-widen`**, because that is the boundary that matches the reported session body: 58 tracked files (`3,068` insertions, `871` deletions) plus 16 meaningful untracked source/doc/test files = **74 meaningful paths**. Untracked `build-pgxp-focused/` and `build-texpack-focused/` are build artifacts and were excluded.
- `fork/feat/ws-cull-widen` is `e27eac73663f`, an ancestor 27 commits behind local `HEAD`. Diffing to that remote ref would mix those already-committed changes into this working-tree contribution and produce 81 meaningful paths instead of the reported approximately 76.
- Per instruction, no build, test, benchmark, or executable was run during this audit. “Covered” below means the test source exists and targets the invariant; it is **not** a fresh pass result.
- A number is called “measured” only when the audited files contain an actual observation/result. Loop counts and cache capacities are reported as designed coverage/structure, not as performance results. Where the delta has no result, the measurement is **UNKNOWN**.
## Corrections to the supplied headlines
1. **Overlay shim anchor: mechanism supported; manifestation counts unverified.** `patch_generated_c` now anchors after the unconditional `#include "psx_runtime.h"` and fails if it is missing. The delta does not contain a linker transcript or result establishing “~10k undefined runtime symbols” or “zero overlay DLLs”; those numbers are **UNKNOWN** here.
2. **Overlay stale-PC/variant identity: supported.** An executed-word overwrite retires the whole page epoch, incoherent capture roots are rejected against current bytes, and static requests are keyed by `(load address, size, bytes)`.
3. **Cycle specialization: supported with a qualification.** Generation selects arity 0/1/2/3 helpers, but masks with more than three nonzero dependencies deliberately retain the generic mask helper. Main-RAM load charges and the pre-deadline fast path are present.
4. **LTO: feature supported; quoted deltas unsupported.** `PSX_ENABLE_LTO`, the no-inline cycle wrappers, and a deterministic benchmark target exist. No checked-in output contains `-45.92%`, `+10.45%`, or `+32.69%`; all three are **UNKNOWN** for this audit.
5. **PGXP/GTE: the correctness changes are supported; “fake never-incremented counters removed” is not.** The old `s_nclip_corrected` counter was incremented on both sign-disagreement branches, and the removed `s_texcorr.z_interp` field was incremented when missing Z values were mean-filled. The change converts NCLIP to non-mutating disagreement telemetry, removes partial-Z interpolation, and adds real primitive/provenance counters; no demonstrably never-incremented counter removal was found.
6. **HD texture pack: supported.** Tile occupancy gates invalidation scans, and VRAM copies can become upload-equivalent candidates, with deferred readback for unresolved copies.
7. **HD text gate: headline mechanism is wrong.** The change uses an atomic aggregate boolean (`g_dispatch_armed`) derived from capture, string-table, and pending-patch state. It is **not** a monotonic mutation epoch. The separate native-text guard does gain a monotonic epoch, but that is a different subsystem.
8. **Presentation: supported as an opt-in capability.** With `fixed_outer_aspect=true`, scene classification selects only the inner content rect; the outer rect comes from configured aspect. Legacy behavior remains when the option is false.
9. **Card-write diagnostic: only true for release builds.** `card_data_writes_check` and `card_data_writes_arm` preprocess away under `PSX_NO_DEBUG_TOOLS`; debug builds still execute them.
10. **GTE bridge narrowing: supported.** The command exporter stops rewriting inputs/control registers and its source comment counts **40 redundant stores removed per COP2 command**; per-write whole-state canonicalization is removed from data/control writes.
11. **Widescreen `items_join`: not in this delta.** `runtime/src/ws_ui_group.c` and its test are unchanged against local `HEAD`. Submission-order joining was implemented earlier in ancestor commit `572b921`; it must be summarized from that commit separately, not claimed as part of this worktree.
Significant changes missing from the supplied list are the exact IRQ-pending cache and writer audit, I-cache replay isolation, two generated-dispatch caches, cycle-observer flushes, live CRTC cadence reconciliation/config exposure, PGXP rectangle-state disarming, release diagnostic pruning, and new host-optimization/keyboard regression surfaces.
## Improvement inventory
Tests named in the last column were inspected but not executed.
### Correctness bugs
| ID | Title | Files touched | What it fixes | Why it is general | Test(s) that cover it |
|---|---|---|---|---|---|
| C1 | Overlay shim anchored to an unconditional include | `tools/compile_overlays.py`; `runtime/include/{cpu_state.h,psx_cycles.h}`; overlay shim tests | A conditional include could make the whole DLL callback shim disappear under `PSX_OVERLAY_DLL_BUILD`, leaving unresolved direct runtime calls. The new anchor is the emitter-guaranteed unconditional runtime include and absence is fatal. | Any generated overlay can acquire conditional headers; no title address or symbol is involved. | `recompiler/tests/test_overlay_store_barrier_codegen.py`; `test_overlay_shim_compile.py` |
| C2 | Overlay execution epoch and exact byte-variant identity | `runtime/src/memory.c`; `tools/compile_overlays.py`; overlay discovery/perf-guard tests | Prevents an outgoing image's recorded PCs from being compiled against a later image's bytes; current-byte CFG rejection and exact-image static requests fail closed to interpretation. | Self-modifying code, streamed overlays, and same-address variants are generic PS1 behaviors. | `recompiler/tests/test_aot_overlay_discovery.py`; `runtime/tests/test_interpreter_perf_guards.py` |
| C3 | Exact cached INTC output and Cause.IP2 ownership | `runtime/include/interrupts.h`; `runtime/src/{interrupts,boot_state,memory,overlay_loader,sio,psx_interpreter}.c`; IRQ tests | Replaces repeated `i_stat & i_mask` hot reads with one cache while making a single refresh owner update both the full pending value and Cause.IP2 after every writer. | Every title uses the PS1 INTC; correctness depends on hardware-register mutation, not game logic. | `runtime/tests/test_cause_ip2_combinational.c`; `test_perf_round3_guards.py` |
| C4 | Deferred-cycle observers flush before reading time | `runtime/src/psx_cycles.c`; cycle tests | MUL/DIV and GTE issue/read/stall paths no longer observe a guest clock that is still sitting in the deferred batch. | These are architectural R3000A/GTE completion timestamps shared by all guests. | `runtime/tests/test_psx_cycle_event_boundaries.c`; `test_perf_round3_guards.py` |
| C5 | I-cache replay cannot mutate tags or charge misses | `runtime/include/psx_icache.h`; `runtime/tests/test_psx_icache_fastpath.c` | Replay now suppresses both tag evolution and miss charges while preserving normal cold initialization, hit/miss behavior, and the disabled-cache giveback path. | Lockstep/replay correctness is independent of title code. | `runtime/tests/test_psx_icache_fastpath.c` |
| C6 | Primitive-atomic PGXP position/depth and frame-canonical endpoints | `runtime/include/pgxp_vertex_cache.h`; `runtime/src/{pgxp_vertex_cache,gpu}.c`; `ENHANCEMENTS.md`; cache test/CMake | Eliminates corrected/native cracks on shared raster endpoints, prevents the two halves of a quad from making different decisions, and rejects incomplete depth instead of mean-filling missing Z. First use of `(frame, final native x, final native y)` owns the endpoint for the frame. | Shared-edge identity, quad splitting, and incomplete projection depth occur in any PGXP-enabled title. | `runtime/tests/test_pgxp_vertex_cache.c`; depth completeness is exercised indirectly by existing GTE/PGXP tests, but **no focused incomplete-Z primitive test was found.** |
| C7 | Per-component PGXP projection identity | `runtime/src/pgxp.cpp`; `runtime/include/pgxp.h`; PGXP tests | Identical packed bits from different projections can no longer be mistaken for coherent X/Y/Z provenance, including after halfword reconstruction. | Packed-word aliasing is a generic provenance problem in MIPS dataflow. | `runtime/tests/test_pgxp.cpp` |
| C8 | Destructive PGXP writes invalidate stale provenance everywhere | `recompiler/src/{pgxp_hook_emitter.cpp,pgxp_hook_emitter.h}`; `runtime/include/{pgxp.h,pgxp_hooks.h,overlay_dispatch_preamble.c.inc}`; `runtime/src/pgxp.cpp`; `runtime/src/{memory,psx_interpreter,dirty_ram_interp,overlay_loader}.c`; emitter/PGXP coverage tests | AND/XOR/NOR/SLT families, link writes, COP0/COP2 destinations, and raw RAM/scratch stores now kill old provenance even when the resulting integer bytes are unchanged. Sparse GPR/page-live gates avoid paying the callback when no value is live. | All recompilers/interpreters must agree on writer semantics; no opcode handling is title-specific. | `recompiler/tests/full_function_emitter_test.cpp`; `runtime/tests/test_pgxp.cpp`; `test_pgxp_destructive_write_coverage.py` |
| C9 | SXYP shadow FIFO follows architectural FIFO movement | `runtime/src/{gte.cpp,pgxp.cpp}`; `runtime/include/pgxp.h`; GTE test | A write to register 15 shifts shadows `12 <- 13`, `13 <- 14`, then invalidates the new `14/15`, instead of retaining stale SXY0/SXY1 provenance. | SXYP alias/FIFO semantics are architectural GTE behavior. | `runtime/tests/test_gte_register_access.cpp` |
| C10 | Precise NCLIP is telemetry-only and guest-bit-exact | `runtime/src/{gte.cpp,gpu.c,debug_server.c}`; `runtime/include/cpu_state.h`; `recompiler/src/config_loader.h`; GTE test | Exact 16.16 winding is still observed, but it never changes guest MAC0 or face visibility; the old saturation-rail clamp rescue can no longer smuggle far-off precise coordinates into rasterization. | Host precision enhancements must not alter guest branches or bypass native GTE saturation in any title. | `runtime/tests/test_gte_register_access.cpp::test_precise_nclip_never_flips_guest_sign` |
| C11 | Polygon-to-rectangle boundary clears triangle-only PGXP state | `runtime/src/{gpu.c,gpu_render.c}` | PGXP-armed polygon fast paths are no longer collapsed to rectangles, and native GP0 rectangles clear any prior triangle precision, perspective, or depth state. | Renderer state leakage and primitive fast-path conversion can affect any sequence mixing polygon and rectangle paths. | **No focused test found.** |
| C12 | Live CRTC changes reconcile host pacing at VBlank | `runtime/include/gpu.h`; `runtime/src/gpu.c`; `runtime/src/main.cpp`; `runtime/tests/test_host_refresh_sanitize.py` | Host pacing/swap policy is no longer startup-latched: a GP1(08h), restore, or plugin period change resets old pacing debt and is applied before waits; banners defer until the guest programs display mode. | PAL/NTSC switching, restores, and timing plugins are framework concerns. | `runtime/tests/test_host_refresh_sanitize.py` |
### Performance
| ID | Title | Files touched | What it fixes | Why it is general | Test(s) that cover it |
|---|---|---|---|---|---|
| P1 | Generation-time cycle dependency arity | `recompiler/src/{code_generator,full_function_emitter}.cpp`; `runtime/include/psx_cyc.h`; `runtime/src/psx_cyc_steps.c`; emitter/cycle tests | Removes the runtime dependency-mask loop for the common 0–3 register cases while retaining a generic fallback for larger masks; shared no-inline wrappers bound LTO cloning. | Every generated instruction uses the load-delay/cycle dependency machinery. | `runtime/tests/test_psx_cyc_batch.c`; `recompiler/tests/full_function_emitter_test.cpp`; `runtime/tests/test_perf_round3_guards.py` |
| P2 | Fused load charge and exact pre-deadline fast charge | `runtime/include/{psx_cyc.h,psx_cycles.h}`; `runtime/src/{memory,psx_cycles,psx_interpreter,dirty_ram_interp}.c`; cycle/perf tests | Main-RAM loads publish base + read fudge + wait/completion as one exact charge; eligible CPU charges advance directly only through `next_service_cycle - 1`, including exact Q16 overclock remainder handling. | Load timing, event deadlines, replay, and overclock transforms are common runtime machinery. | `runtime/tests/test_psx_cyc_batch.c`; `test_psx_cycle_event_boundaries.c`; `test_interpreter_perf_guards.py` |
| P3 | Exact-address compiled dispatch caches | `recompiler/src/{full_function_emitter,main_psx}.cpp`; emitter test | Adds a 256-slot TLS cache in generated static dispatch and a 1,024-slot TLS positive/negative cache in game lookup; every hit rechecks the full canonical address before bypassing binary search. | BIOS services, tail-call loops, and large dispatch tables occur across recompiled games. | `recompiler/tests/full_function_emitter_test.cpp` covers the generated-dispatch cache; **no focused test found for the game lookup cache.** |
| P4 | Two-level native-text guard epoch | `runtime/src/memory.c`; structural guard test | Stable no-write intervals return after one epoch comparison; after a mutation, the existing exact page-generation sum decides whether the cached verdict remains valid. | Dirty-RAM/native-code guards are shared by every title using mutable text. | `runtime/tests/test_perf_round3_guards.py` |
| P5 | Aggregate text-translation dispatch gate | `runtime/include/text_xlate.h`; `runtime/src/{text_xlate.cpp,fntrace.c}`; structural guard test | Avoids invoking the translation callback at every dispatch once capture is off and no string-table or unsettled RAM patch needs it. This is an atomic boolean, not an epoch. | Capture, shipped translations, and pending patch classes are framework state. | `runtime/tests/test_perf_round3_guards.py` |
| P6 | Tile-indexed texture invalidation | `runtime/include/tex_pack_rect_index.h`; `runtime/src/tex_pack.cpp`; index test/CMake | VRAM writes first query conservative 16×16-tile occupancy instead of always linearly scanning up to 8,192 upload/palette/pending-copy records; uncertainty returns “maybe” and preserves the slow exact scan. | VRAM invalidation volume and upload count vary by title, while the VRAM geometry is common. | `runtime/tests/test_tex_pack_rect_index.cpp` |
| P7 | Narrow GTE command export and incremental canonicalization | `runtime/src/gte.cpp`; `runtime/tests/test_gte_register_access.cpp`; `runtime/CMakeLists.txt` | Commands export only architectural results (`D7..D29`, `C31`) and writes maintain aliases/masks incrementally instead of sweeping the entire backing state after every write. | COP2 register semantics are architectural and shared. | `runtime/tests/test_gte_register_access.cpp` |
| P8 | Debug-only store diagnostics compile out of release | `runtime/src/{memory,sio}.c`; structural guard test | Removes universal RAM-store calls to card-write and debug trace consumers, plus SIO arming, when `PSX_NO_DEBUG_TOOLS` has no consumer. Debug behavior is retained. | This is build-mode ownership, not a title exception. | `runtime/tests/test_perf_round3_guards.py` |
### New capability
| ID | Title | Files touched | What it fixes/adds | Why it is general | Test(s) that cover it |
|---|---|---|---|---|---|
| N1 | Opt-in CMake IPO/LTO and generated-hotpath benchmark | `runtime/runtime.cmake`; `runtime/CMakeLists.txt`; `runtime/tests/bench_psx_runtime_hotpath.c`; `runtime/src/psx_cyc_steps.c` | Adds toolchain-checked `PSX_ENABLE_LTO` (default off), applies IPO to runtime targets, and provides a semantic-checksummed generated-style benchmark. | Toolchain optimization and code-shape regression measurement apply to every game target. | Benchmark is deliberately not registered in CTest; **no result is checked in or run here.** |
| N2 | HD replacements from VRAM-to-VRAM copies | `runtime/include/tex_pack.h`; `runtime/src/{tex_pack.cpp,gpu_render.c}`; copy test/CMake | Copy destinations can reuse an exact tracked source preimage or become a lazily read-back candidate; CPU upload and copy records share the same position/size/hash semantics and replacement lookup. | Many PS1 engines stage textures in VRAM rather than uploading final destinations directly. | `runtime/tests/test_tex_pack_copy.cpp` |
| N3 | Configurable CRTC refresh multiplier | `recompiler/src/{config_loader.cpp,config_loader.h}`; `runtime/include/{gpu.h,mod_plugins.h}`; `runtime/src/gpu.c`; `runtime/src/main.cpp`; host refresh test | Exposes `[video] crtc_refresh_multiplier` in the range 1..8 as the baseline timing setting, with activation plugins allowed to override it. | It turns an existing runtime timing control into a validated per-project framework option. | `runtime/tests/test_host_refresh_sanitize.py` covers live x2 cadence ownership, not parser bounds 1..8. |
| N4 | Stable configured outer aspect with scene-specific inner content | `WIDESCREEN.md`; `recompiler/src/{config_loader.cpp,config_loader.h}`; `runtime/include/{gpu_gl_renderer.h,gpu_vk_renderer.h,present_ring.h,ws_present_layout.h}`; `runtime/src/{gpu.c,gte.cpp,main.cpp,gpu_gl_renderer.c,gpu_vk_renderer.c,ws_present_layout.c}`; layout test/CMake | Adds opt-in `fixed_outer_aspect`: menus/FMV can remain centered 4:3 content without changing the outer configured canvas; GL, Vulkan, interpolation, hold-last, and CPU paths use the separated policy. | Outer user preference and content correction are independent concepts for any widescreen title. | `runtime/tests/test_ws_present_layout.c` |
### Diagnostics and verification surfaces
| ID | Title | Files touched | What it fixes/adds | Why it is general | Test(s) that cover it |
|---|---|---|---|---|---|
| D1 | Actionable PGXP and copied-texture telemetry | `runtime/include/{cpu_state.h,pgxp.h}`; `runtime/src/{gpu.c,gte.cpp,pgxp.cpp,debug_server.c,tex_pack.cpp}` | Reports primitive armed/rejected/mixed decisions, endpoint promotion/demotion/cache overflow, memory invalidations, mixed projection rejects, NCLIP sign disagreements, and copy queued/resolved/tracked/invalidated/dropped counts. | These counters diagnose framework coverage and fail-closed behavior without game addresses. | Counter transitions are partially asserted by `runtime/tests/{test_pgxp.cpp,test_gte_register_access.cpp,test_tex_pack_copy.cpp}`. |
| D2 | Host optimization acceptance contract | `docs/HOST_OPTIMIZATION_CONTRACT.md` | Defines an interleaved A/B protocol, semantic equivalence requirement, raw-run retention, and a 5% acceptance gate for complexity/ABI-bearing host optimizations. | It is a reusable maintainer process rather than title behavior. | Documentation only. |
| D3 | Keyboard Select+Start chord regression | `runtime/tests/test_keyboard_pad_chord.c`; `runtime/CMakeLists.txt` | Adds a focused test for default bindings, simultaneous and consecutive polls, release, and re-press behavior. It does not change production keybind code. | Multi-button active-low pad words are common input behavior. | `runtime/tests/test_keyboard_pad_chord.c` |
## Concrete evidence and measurements
### Correctness evidence
**C1 — overlay shim anchor.** The root cause and new invariant are stated at `tools/compile_overlays.py:2078-2089`: the syntactically last include may be inside the per-instruction I-cache `#ifdef/#else`, and `PSX_OVERLAY_DLL_BUILD` would preprocess away the callback definitions. The insertion now uses the exact unconditional include at `tools/compile_overlays.py:2084-2091`; absence raises `ValueError`. `recompiler/tests/test_overlay_store_barrier_codegen.py:109-118` asserts the flush definition precedes the DLL conditional, and `test_overlay_shim_compile.py:43-59` compiles the expanded PGXP hook surface with the DLL macros when that test is run. Windows exports for the cycle barrier are at `runtime/include/cpu_state.h:76-81` and `runtime/include/psx_cycles.h:192-198`. **Measured result:** UNKNOWN. The supplied ~10k unresolved-symbol and zero-linked-DLL claims are absent from the audited delta.
**C2 — overlay epoch/identity.** `runtime/src/memory.c:1080-1114` preserves evidence for data-only writes but clears both full-page PC bitmaps and the page gate when any executed word is replaced. `tools/compile_overlays.py:1546-1572` rejects capture-derived roots whose current capped CFG reaches invalid instructions; `tools/compile_overlays.py:2867-2930` keys static demand by `(load_addr, size, data)` and resolves it only against current-byte range identities; isolated generation also reports `INCOHERENT_VARIANT_BYTES` at `tools/compile_overlays.py:3074-3083`. Focused regressions begin at `recompiler/tests/test_aot_overlay_discovery.py:340` and `:366`. **Measured result:** UNKNOWN; no shard ok/failed count or coverage percentage for this change is present.
**C3 — IRQ cache.** The owner computes `pending = i_stat & i_mask`, stores the full value, and then derives Cause.IP2 at `runtime/src/interrupts.c:149-160`. The structural audit at `runtime/tests/test_perf_round3_guards.py:22-46` currently finds exactly **19** direct `I_STAT`/`I_MASK` mutations across five writer files and requires a refresh near each; it also verifies the cache is stored before the nullable Cause pointer can return (`:55-63`). `runtime/tests/test_cause_ip2_combinational.c:97` repeatedly asserts exact cache equality. **Measured result:** UNKNOWN; 19 is a static writer-set count, not a runtime speed result.
**C4 — cycle observers.** Flushes precede cycle-dependent work in `psx_muldiv_set`, `psx_muldiv_stall`, `psx_gte_read`, `psx_gte_set`, and `psx_gte_stall` at `runtime/src/psx_cycles.c:696-795`. `runtime/tests/test_psx_cycle_event_boundaries.c` includes explicit delayed GTE read/stall assertions and ends by checking cross-device `D-1 + 1` causality. **Measured result:** UNKNOWN.
**C5 — I-cache replay.** `runtime/include/psx_icache.h:20-32` separates active-state initialization from replay and calls the miss path only outside replay. Warmed-hit and would-be-miss replay cases are at `runtime/tests/test_psx_icache_fastpath.c:79-103`. **Measured result:** UNKNOWN.
**C6 — primitive/canonical endpoint.** The cache is a fixed **65,536-entry** table (`runtime/include/pgxp_vertex_cache.h:21-35`) and resolves frame/native-coordinate identity at `runtime/src/pgxp_vertex_cache.c:10-38`. Whole primitive resolution, including all four quad vertices, starts at `runtime/src/gpu.c:3434-3609`; depth arms only when `depth_count == count` (`:3570-3605`), replacing the old mean-filled partial-Z path. The focused cache test covers corrected→native, native→corrected, transform identity, and frame identity; its native→corrected candidate is deliberately **0.75 px**, above the old 0.5 workaround (`runtime/tests/test_pgxp_vertex_cache.c:12-64`). **Measured result:** no performance or in-game crack count is recorded; UNKNOWN.
**C7 — component IDs.** `PGXPValue` gains independent X/Y/Z projection IDs at `runtime/src/pgxp.cpp:62-68`; coherence predicates require matching nonzero IDs at `:287-307`, and reconstructed halves propagate their component's ID around `:450-526`. `runtime/tests/test_pgxp.cpp:205-220` asserts that mixed projections are rejected even when packed integer geometry can otherwise look usable. **Measured result:** UNKNOWN.
**C8 — destructive writes.** The hot GPR macro consults `g_pgxp_gpr_live_mask` before calling the invalidator (`runtime/include/pgxp_hooks.h:93-101`); the invalidator itself is at `runtime/src/pgxp.cpp:396-403`. Raw word invalidation and partial-half handling begin at `runtime/src/pgxp.cpp:371`, while six central RAM/scratch write branches are gated in `runtime/src/memory.c:1968-2520`. `runtime/tests/test_pgxp_destructive_write_coverage.py:69-77` requires two 1-byte, two 2-byte, and two 4-byte paths plus both interpreter/emitter surfaces. `runtime/tests/test_pgxp.cpp:137-203` includes byte-identical GPR and RAM writes. The existing source comment says the earlier broad ALU-hook gate accounted for **3.6% of the emulation thread** in a high-refresh workload (`runtime/src/pgxp.cpp:91-101`), but there is no measured delta for the new sparse invalidation gates. **Measured result for this change:** UNKNOWN.
**C9 — SXYP.** Architectural backing shifts at `runtime/src/gte.cpp:2027-2047`; the shadow shift/reset is implemented in `runtime/src/pgxp.cpp:1033-1053`. `runtime/tests/test_gte_register_access.cpp:501-526` checks the precise FIFO before and after register 15 and all SXY write classes. **Measured result:** UNKNOWN.
**C10 — NCLIP.** The old double-based sign replacement is removed; `runtime/src/gte.cpp:962-983` computes an exact `int64_t` determinant, increments disagreement telemetry, and always writes native `out` to MAC0. The GPU-side comment at `runtime/src/gpu.c:3644-3650` makes NCLIP audit-only and removes the saturation-rail rescue route to far-off precise raster coordinates. The regression constructs native `+1` versus exact negative and requires guest MAC0 to remain `+1` while the disagreement counter increments (`runtime/tests/test_gte_register_access.cpp:575-602`). The removed code incremented `s_nclip_corrected` in both mismatch branches, and the removed partial-Z path incremented `s_texcorr.z_interp`, so the “never-incremented” description is false. **Measured result:** one deterministic disagreement fixture; no field count or performance result.
**C11 — rectangle boundary.** PGXP-aware polygon fast paths require position/depth to be unarmed before collapsing a quad to a rectangle (`runtime/src/gpu.c:4049-4054`, `:4287-4330`). `runtime/src/gpu_render.c:135-141` clears precise position, perspective, and depth state; each flat/textured/scaled rect entry invokes it at `:270`, `:288`, and `:296`. **Test/measurement:** no focused test found; UNKNOWN.
**C12 — live CRTC cadence.** Guest display-mode observation is recorded on GP1(08h) and restore at `runtime/src/gpu.c:6018` and `:6202`. `refresh_live_crtc_cadence` (`runtime/src/main.cpp:2916-2949`) keys transitions by exact guest period, resets old wall-clock debt, reapplies present cadence, and logs only after guest programming; it is called before frame performance/pacing work at `runtime/src/main.cpp:6250`. The structural test's exact arithmetic establishes PAL×2 = **100.0 Hz** matches a 100 Hz panel while NTSC×2 = **120.0 Hz** does not (`runtime/tests/test_host_refresh_sanitize.py:73-86`). These are formula assertions, not a live-run result. **Measured runtime result:** UNKNOWN.
### Performance evidence
**P1 — arity helpers.** Both emitters choose specialized helpers and retain the generic fallback at `recompiler/src/code_generator.cpp:51-69` and `recompiler/src/full_function_emitter.cpp:35-53`. The shared wrappers and deliberate no-inline boundary are at `runtime/src/psx_cyc_steps.c:5-68`; overlay DLLs retain inline equivalents at `runtime/include/psx_cyc.h:219-247`. `runtime/tests/test_psx_cyc_batch.c:244-264` defines **98,304 oracle comparisons** (`4,096 seeds × 6 modes × 4 arities`) and compares every `CPUState` byte plus deferred cycles. **Measured performance:** UNKNOWN.
**P2 — fused/fast charge.** Main-RAM word and half loads compute a single `charge + fudge + 5` publication (`wait 3 + completion 2`) at `runtime/include/psx_cyc.h:280-337`; slow paths carry the same base charge through `runtime/src/memory.c:2316-2383`. `psx_cycles_try_fast_charge` previews and commits both scaled cycles and Q16 remainder only below the published limit (`runtime/include/psx_cycles.h:110-139`), which is set to `next_service_cycle - 1` at `runtime/src/psx_cycles.c:189-194` and invalidated at MMIO/exact/restore boundaries. Tests define **4,096 fused-load oracle cases** (`2,048 × 2 overclock modes`, `runtime/tests/test_psx_cyc_batch.c:267-307`) and **1,152 fast-charge oracle cases** (`2 scales × 16 accumulators × 3 charges × 12 deadline rooms`, `:311-334`). **Measured performance:** UNKNOWN.
**P3 — dispatch caches.** Generated static dispatch emits the 256-slot table and exact-address hit check at `recompiler/src/full_function_emitter.cpp:1834-1865`; the game lookup emits 1,024 TLS slots and caches both hits and misses at `recompiler/src/main_psx.cpp:1594-1629`. `recompiler/tests/full_function_emitter_test.cpp:199-204` checks the generated helper and address validation. **Measured result:** UNKNOWN; no hit rate, comparison count, or benchmark delta is present.
**P4 — native-text guard epoch.** The monotonic epoch and global mutation helper are at `runtime/src/memory.c:617-624`; the one-compare hit and page-generation fallback are at `:772-829`. The existing source comment records a motivating profile in which `psx_game_text_native_ok` was **6.8% of the emulation thread** with the older memo never hitting (`runtime/src/memory.c:625-632`). That is a baseline attribution, not a measured post-change gain. **Measured gain:** UNKNOWN.
**P5 — text-translation aggregate gate.** `refresh_dispatch_armed` derives one atomic gate from capture, string-table, and three pending patch counts (`runtime/src/text_xlate.cpp:441-453`); `fntrace_record` checks it at `runtime/src/fntrace.c:170-171`. The source records two historical baselines: capture scanning at roughly **33k dispatches/frame**, **18.6% of samples**, with capture disabled yielding **+22% fps over guest frames 800–2200** (`runtime/src/text_xlate.cpp:423-435`), and the otherwise-useless fixes-only argument scan at **4.2% of the emulation thread** (`:898-905`). None is a direct A/B of the new outer aggregate gate. **Measured gain:** UNKNOWN.
**P6 — texture tile index.** The conservative grid uses 16×16 tiles over a 2,048×1,024 domain and returns “maybe” for unindexed/uncertain rectangles (`runtime/include/tex_pack_rect_index.h:15-83`). `invalidate_locked` checks upload, pending-copy, and palette occupancy before any vector scan (`runtime/src/tex_pack.cpp:562-620`); the tracker cap remains **8,192** records (`runtime/src/tex_pack.cpp:54`). The deterministic test runs **20,000 randomized add/remove/query iterations** and asserts there are no false negatives (`runtime/tests/test_tex_pack_rect_index.cpp:47-77`). **Measured performance:** UNKNOWN.
**P7 — GTE bridge.** `runtime/src/gte.cpp:1702-1729` exports only command results and explicitly counts **40 redundant stores removed per command**. Data/control writes now maintain their own masked/aliased state at `:2012-2081`; full canonicalization remains only at the raw import boundary (`:2099`). The oracle suite covers **2,112 command cases** (`22 functions × 96`), **256** MVMVA selector combinations, and **8,192** stateful stream steps (`64 × 128`) at `runtime/tests/test_gte_register_access.cpp:382-473`, in addition to register-write tests. **Measured throughput:** UNKNOWN.
**P8 — release diagnostics.** Declarations and call sites are guarded around `runtime/src/memory.c:1298-1301`, `:2006-2012`, `:2142-2150`, and `:2485-2493`; SIO arming receives the same build guard. `runtime/tests/test_perf_round3_guards.py:65-73` scans every card diagnostic call for a release guard. **Measured performance:** UNKNOWN.
### Capability evidence
**N1 — LTO/benchmark.** `runtime/runtime.cmake:26-44` defines default-off, C/C++ toolchain-checked `PSX_ENABLE_LTO`; target IPO is applied at `:937-939`. The wrappers resist whole-program cloning at `runtime/src/psx_cyc_steps.c:28-38`. The benchmark models **32 guest instructions/block**, defaults to **4,000,000 blocks** (**128,000,000 guest instructions/sample**), takes **7 samples**, reports median/min/max, and rejects checksum drift (`runtime/tests/bench_psx_runtime_hotpath.c:25-28`, `:227-284`). It is intentionally not a CTest (`runtime/CMakeLists.txt:46-58`). **Measured result:** UNKNOWN; the supplied `-45.92%`, `+10.45%`, and `+32.69%` values are not in the audited repository state.
**N2 — VRAM copies.** `gr_copy_rect` classifies mask/wrap/overlap preservation and reports the copy before backend mutation (`runtime/src/gpu_render.c:143-156`). `tex_pack_on_copy` (`runtime/src/tex_pack.cpp:817-893`) shares exact full-source preimages, hashes subrects, invalidates the destination in the original order, and otherwise queues a fail-closed candidate. Draw-time query/resolution is at `:896-989`; CPU and copy records meet in the shared builder at `:625-657`. `runtime/tests/test_tex_pack_copy.cpp` checks CPU/copy key parity, invalidation, and replacement lookup. Its optional, unregistered benchmark is designed for **200,000 copies**, **5,000,000 no-pending queries**, and **50,000 hashes** (`:168-208`), but no output is checked in or run. **Measured performance:** UNKNOWN.
**N3 — CRTC config.** Parsing and range rejection are at `recompiler/src/config_loader.cpp:573-579`; the default/range contract is at `recompiler/src/config_loader.h:412-414`. Config is applied before activation plugins at `runtime/src/main.cpp:11534-11538`. The available test checks x2 live cadence only; it does not test TOML parser bounds or multipliers 3–8. **Measured compatibility/coverage for 3–8:** UNKNOWN.
**N4 — outer/content layout.** The API contract explicitly forbids scene classification from changing the outer rect when fixed mode is enabled (`runtime/include/ws_present_layout.h:20-32`); the independent rect calculation is at `runtime/src/ws_present_layout.c:33-60`. GL uses it at `runtime/src/gpu_gl_renderer.c:3452-3461`; Vulkan uses it at `runtime/src/gpu_vk_renderer.c:1688-1706`; scene classification was renamed/content-scoped in `runtime/src/gpu.c:324-390` and `runtime/src/main.cpp:7022-7391`. The focused test checks exact 32:9-in-16:9 and 21:9 layouts plus unchanged legacy behavior (`runtime/tests/test_ws_present_layout.c:14-43`). **Measured visual/performance result:** UNKNOWN.
### Diagnostic evidence
**D1 — telemetry.** Primitive counters are defined and exported at `runtime/src/gpu.c:3460-3498`; PGXP memory/mixed-projection counters are maintained in `runtime/src/pgxp.cpp`; NCLIP disagreements are at `runtime/src/gte.cpp:941-983`; debug JSON publishes them at `runtime/src/debug_server.c:5216-5276`. Copy lifecycle counters are defined at `runtime/src/tex_pack.cpp:204-207` and published at `:1465-1487`. **Measured field values:** UNKNOWN; the change adds counters but the delta contains no captured report.
**D2 — optimization contract.** `docs/HOST_OPTIMIZATION_CONTRACT.md:29-44` defines the 5% retention and semantic-equivalence gates; `:103-110` requires a separate interleaved performance campaign; `:185-200` specifies evidence retention. The example percentages at `:181-182` are explicitly examples, not results from this work. **Measured result:** not applicable.
**D3 — keyboard chord test.** `runtime/tests/test_keyboard_pad_chord.c:24-50` checks default Return/RShift bindings and six pad states: released, Select held, first and consecutive Select+Start polls, Start release, and Start re-press. `runtime/CMakeLists.txt:86-95` registers it. **Production change/measurement:** none; this is regression coverage only, and it was not run here.
## RISK / REVIEW FOCUS
1. **Overlay byte identity and shim placement (C1/C2) — highest review priority.** Preserve three invariants: (a) the callback preamble is outside every conditional and missing the guaranteed runtime include is a hard failure; (b) a compiled body satisfies a request only when load address, size, and exact bytes/range identities agree; (c) overwriting an executed word retires all PC/dispatch evidence for that page's outgoing image, while an unrelated data-only write does not erase valid execution evidence. False positives execute wrong bytes; overly broad invalidation silently destroys AOT coverage.
2. **Guest cycle exactness and device deadlines (C4/P1/P2) — highest review priority.** Specialized helpers must be byte-for-byte equivalent to the generic dependency mask, preserve GPR0, read-absorb and load-delay ownership, and retain COSIM/replay/conservative/device-service fallbacks. A fused load must apply base, read fudge, wait, completion, and Q16 overclock remainder exactly once. Fast charge must never reach or cross `next_service_cycle`, overflow the clock, bypass a local/deferred batch, or leave stale eligibility after MMIO, exact service, or restore. Every observer of absolute guest time must flush first.
3. **IRQ pending cache and live CRTC timing (C3/C12/N3) — highest review priority.** Every direct `I_STAT`/`I_MASK` writer, including init, restore, acknowledgment, and temporary CD/SIO masking, must call the refresh owner. The full cached value must remain exactly `i_stat & i_mask`; only Cause.IP2 is masked to the architectural 11 bits. Cached snapshots must be refreshed after any idle skip/device service. CRTC period changes must be reconciled before the next host wait without changing plugin override precedence or inheriting pacing debt from the old period. Adding a twentieth writer without extending the structural audit is a correctness bug.
4. **PGXP provenance and guest visibility (C6–C11).** NCLIP telemetry must never write a host-corrected sign to guest MAC0. Projection IDs must survive only operations that prove component identity; every destructive writer must invalidate across both interpreters, generated code, and overlay DLLs. Endpoint-cache collision/overflow behavior must remain deterministic and fail to native geometry. Rect paths must clear every triangle-only backend latch.
5. **VRAM copy tracking (P6/N2).** Masked, wrapping, or overlapping copies must never claim a pre-copy source hash as the post-copy destination. Pending readback must use the authoritative post-copy backend, be invalidated by later writes/restores, and preserve CPU-upload key semantics including position. Tile indices may have false positives but must never have false negatives.
6. **LTO/code-shape claims (P1/N1).** Review generated symbol visibility on MSVC/Clang/GCC and ensure IPO does not clone the guarded wrappers or drop DLL exports. Do not accept the quoted LTO percentages without raw commands, compiler/linker versions, interleaved samples, and semantic checksums; none is present in this delta.
7. **Presentation backends (N4).** Fixed mode must produce the same outer rect for gameplay, menu, FMV, interpolation, hold-last, native VRAM, CPU, and wide paths; scene state may change only the inner rect. Legacy mode must remain byte/geometry compatible for projects that do not opt in, especially Vulkan's pre-existing 4:3 policy.
## NOT UPSTREAMABLE
- The WipEout-specific conclusion in `ENHANCEMENTS.md:879-883` that `pgxp_tolerance = 1.0` is valid for WipEout 3 is title validation/configuration, not a framework default. Upstream the canonical-endpoint mechanism and neutral explanation; keep the title's chosen value in the game repository.
- The explanatory phrase “the WipEout 8 MB payload depends on that” at `runtime/src/memory.c:1090` should be generalized before submission. The whole-page execution-epoch mechanism itself is upstreamable.
- `docs/internal/upstream/WIPEOUT3_SESSION_ENGINE_DRAFTS_2026-08-20.md` is a WipEout session/commit audit, not framework product documentation; do not include it in an upstream code PR.
- `build-pgxp-focused/` and `build-texpack-focused/` are generated build trees and must not be committed.
- Enabling `fixed_outer_aspect`, a non-stock CRTC multiplier, PGXP tolerance 1.0, or any other title profile belongs in the game repository. The parser/runtime capabilities and default-off framework behavior are upstreamable.
- No new WipEout-specific executable address hook was found in the working-tree delta. Existing title references outside the added lines are not evidence that the general mechanisms above are per-game hacks.
- The `items_join` UI grouping change is potentially upstreamable, but it is **not part of this delta**; submit/review it from ancestor commit `572b921` with its own history and evidence rather than folding it into this worktree summary.
## Suggested PR split and landing order
1. **Overlay DLL shim placement/export fix (C1).** Small, independently testable, and prerequisite to trusting any overlay-DLL validation.
2. **Overlay byte-identity correctness (C2).** Land after the shim fix so generated overlay tests exercise a valid DLL surface. Keep the page-epoch producer fix and byte-keyed consumer/static-request logic together because either half alone leaves a stale-image path.
3. **IRQ pending-cache ownership (C3).** Separate high-risk PR. Include all current writers and the 19-writer structural audit in the same change.
4. **Cycle exactness foundation (C4, C5, P2 without code-shape claims).** Land observer flushes, replay-safe I-cache behavior, fused load accounting, deadline publication/invalidation, and oracle tests together. This establishes equivalence before performance-oriented code generation changes.
5. **Cycle arity specialization and LTO surface (P1, N1).** Depend on PR 4. Keep the benchmark and no-inline rationale with the specialization; publish new raw benchmark results instead of the unsupported percentages in the supplied headline.
6. **Dispatch/text/release hot-path pruning (P3, P4, P5, P8).** These are independent of guest timing after PR 4. Prefer three reviewable commits/PRs if maintainers want narrower invalidation reasoning: dispatch caches; text guards/gates; debug-only store hooks.
7. **GTE/PGXP provenance correctness (C7–C10, P7).** Land component IDs, destructive-write coverage, SXYP FIFO, NCLIP telemetry-only behavior, and narrowed architectural export first. These define trustworthy provenance and guest-bit-exact boundaries.
8. **PGXP primitive/canonical endpoint and renderer boundary (C6, C11, D1 PGXP portion).** Depend on PR 7. Add a focused rectangle-state regression before submission; keep the WipEout tolerance conclusion out of the framework PR.
9. **Texture-pack index, then copy candidates (P6 followed by N2 and copy telemetry).** The copy queue uses the conservative index and shared upload storage, so land/test the no-false-negative index first. Review copy/mask/wrap/overlap and savestate invalidation as a separate capability PR.
10. **Presentation layout separation (N4).** Independent UI/rendering PR covering the shared layout helper and both GL/Vulkan paths. Preserve default-off legacy behavior.
11. **CRTC configuration and live cadence (C12/N3).** Separate timing-sensitive PR after IRQ/cycle work. Add parser tests for 1, 8, 0, and 9 and either validate or narrow the advertised 1..8 range before landing.
12. **Maintainer diagnostics (D2/D3).** The optimization contract can land as documentation if the project accepts that policy; the keyboard regression can land independently. Neither should block correctness PRs.
---
## Appendix: session measurements (recorded 2026-08-20)
The audit above correctly marks several deltas UNKNOWN because no result transcript is
checked in. These numbers WERE observed live during the session that produced this work.
They are recorded here so a reviewer can see the basis; they are single-machine
observations, not a portable benchmark suite.
**Host:** AMD Ryzen 7 9800X3D (8C/16T, 4.7 GHz, X3D cache), Windows 11, pinned
retcomm cmake-clang-v1 (Clang 22.1.8). Title: WipEout 3 SE (PAL, SCES-02845) with
hueponik's official PAL 100 fps patch, 8 MB RAM, max in-game detail,
`cpu_overclock = 900`, `crtc_refresh_multiplier = 2` (PAL 50 Hz x2 = 100 Hz).
### LTO / codegen benchmark
`runtime/tests/bench_psx_runtime_hotpath.c`, median-of-three rotated runs, seven samples
each, 256M guest instructions per sample. EVERY variant produced identical results
(guest cycles 33,774,169; oc accumulator 60,416; services 0; digest 3d88e743440954f1),
which is the equivalence evidence for this workload.
| Variant | Guest instructions/s | vs baseline |
|---|---:|---:|
| Baseline `-O3` | 451,062,216 | - |
| Plain ThinLTO | 243,919,125 | -45.92% |
| Plain full LTO | 585,251,881 | +29.75% |
| ThinLTO + guarded wrappers | 498,195,210 | +10.45% |
| **Full LTO + guarded wrappers** | **598,526,036** | **+32.69%** |
| `-march=native` only | 457,336,000 | +1.39% |
| `-march=znver5 -mtune=znver5` | 441,476,740 | -2.13% |
| Helpers moved to headers/unity | 162,246,886 | -64.03% |
Plain ThinLTO REGRESSES badly by cloning hot interlock bodies; the `noinline` guard at
`runtime/src/psx_cyc_steps.c:27` is what makes LTO a win. ThinLTO executable size fell
97,148,928 -> 86,249,472 bytes (-11.22%) once guarded, confirming the over-cloning.
### End-to-end guest speed (identical BIOS boot phase each run, so directly comparable)
| Build state | Guest speed |
|---|---:|
| Session start (no static bake; overlay autocompile degraded) | 0.36x |
| + partial static bake (25/62 shards) | 0.42x |
| + clean static bake (26 shards, 0 failed) | 0.49x |
| + overlay DLL link fix; PGXP hooks and rewind disarmed | 0.58x |
| + cycle-accounting specialization, PGXP RE-ENABLED, 100% texture pack | 0.50x |
| **+ full LTO** | **0.68-0.76x** |
LTO's end-to-end contribution on this identical phase was ~+35%, consistent with the
+32.69% synthetic result. 3D attract-mode scenes were observed sustaining
`VBlank 100/s 1.00x, render 100 fps`.
### Overlay pipeline
| Metric | Before | After |
|---|---:|---:|
| Overlay DLL link errors | 10,395 diagnostics / 20 symbol names | 0 |
| Overlay DLLs produced by autocompile | 0 | 1,442 in cache |
| Static bake shards | ok=25 failed=23 skipped=14 | ok=26 failed=0 skipped=14 |
| Runtime dispatch | `disp_native=0`, `disp_interp=4,456,079` | native dispatch active |
The `disp_native=0` figure is from the runtime's own `psx_last_run_report.json`, which also
carried `autocompile_degraded=1` with the reason string naming the cause.
### HD texture pack
| Metric | Before | After |
|---|---:|---:|
| Requested (texture,palette) pairs covered | 39 / 439 (8.88%) | 439 / 439 (100%) |
| Pack entries | 292 | 445 generated (692 live incl. legacy) |
| PNG payload | 2.61 MiB | 4.32 MiB |
Verification on the generated set: alpha subset of source 402/402, RGB subset 402/402,
all dimensions <= 4096, F500 font atlases byte-identical (untouched).
### PGXP
Hook cost measured by the 1 kHz IP sampler on the emulation thread, in race:
`psx_pgxp_load` 7.66% + `psx_pgxp_alu` 3.63% + `psx_pgxp_store` 2.35% +
`psx_pgxp_cop2` 1.20% = **14.84%** while both correction features were DISABLED, because
`precise_nclip` and `pgxp_cpu_mode` arm the hook surface independently. After disarming,
the same buckets fell to 0.69% total.
### Font atlases (game-side asset work, listed for completeness)
| Metric | Before | After |
|---|---:|---:|
| Detached ink components (ringing lobes) | 986 | 0 |
| Dark ink pixels (black halo) | 87,650 | 0 |
| Alpha masks | - | byte-identical before/after |
+2 -2
View File
@@ -2329,14 +2329,14 @@ UserSettings load_user_settings(const fs::path& path) {
const toml::value& h = toml::find(doc, "hotkeys");
if (h.contains("rewind_pad")) try_get([&]{
const auto n = toml::find<int64_t>(h, "rewind_pad");
if (n >= 0 && n < 256) {
if (pad_bind_value_ok(n)) {
s.hotkey_pad_rewind = (int)n;
s.has_hotkey_pad_rewind = true;
}
});
if (h.contains("save_state_menu_pad")) try_get([&]{
const auto n = toml::find<int64_t>(h, "save_state_menu_pad");
if (n >= 0 && n < 256) {
if (pad_bind_value_ok(n)) {
s.hotkey_pad_save_state_menu = (int)n;
s.has_hotkey_pad_save_state_menu = true;
}
+14
View File
@@ -55,6 +55,20 @@ inline constexpr int VIDEO_RENDERER_OPENGL = 1;
inline constexpr int VIDEO_RENDERER_VULKAN = 2;
inline constexpr int DEFAULT_VIDEO_RENDERER = VIDEO_RENDERER_OPENGL;
// Controller-hotkey bind encoding, mirroring recomp-ui's RECOMP_LAUNCHER_PAD_*
// (recomp_launcher.h): 0 = unbound, 1..99 = button (1 + SDL button code),
// 100..999 = axis, 1000+ = a CHORD, encoded as 1000 + a bitmask of SDL button
// codes. Two buttons held together is the normal case (select+r3), so a real
// value here is routinely five digits.
//
// The bound matters: a naive `< 256` check silently rejects every chord, so a
// saved rewind_pad = 1272 was dropped on load and pad hotkeys were only ever
// configurable to single buttons. SDL defines about 21 gamepad buttons, so cap
// the mask at 32 bits' worth and let the runtime ignore bits no controller can
// produce.
inline constexpr int PAD_BIND_MAX = 1000 + (1 << 21);
inline bool pad_bind_value_ok(long long n) { return n >= 0 && n < PAD_BIND_MAX; }
struct WidescreenSignedBoundSite {
uint32_t address = 0;
uint32_t expected = 0; // guarded LUI instruction
+3
View File
@@ -403,6 +403,9 @@ if(BUILD_TESTING)
add_test(NAME rewind_toggle_combo_test
COMMAND ${Python3_EXECUTABLE}
${CMAKE_CURRENT_SOURCE_DIR}/tests/test_rewind_toggle_combo.py)
add_test(NAME savestate_status_protocol_test
COMMAND ${Python3_EXECUTABLE}
${CMAKE_CURRENT_SOURCE_DIR}/tests/test_savestate_status_protocol.py)
endif()
# DISABLED, not de-registered. It links and runs, but 147 of its 309 checks
+3
View File
@@ -97,6 +97,9 @@ int savestate_request_load_blob_protocol(const void* data, size_t size);
/* 1 while a staged save/load has not yet been consumed by savestate_poll. */
int savestate_pending(void);
/* Machine-readable lifecycle receipt for deterministic debug harnesses. */
void savestate_status_json(char* buf, size_t cap);
/* 1 once after a successful load restore (before scheduler longjmp). Clears. */
int savestate_take_load_completed(void);
+27 -9
View File
@@ -1264,19 +1264,37 @@ function(psxrecomp_add_runtime_target target)
# own POST_BUILD copy so a title may still override an id if it ever needs
# to; copy_directory merges rather than replacing the tree.
if(EXISTS "${PSXRECOMP_ROOT}/mods/builtin/packages")
# Clear first: copy_directory MERGES, so a mod deleted from source
# would otherwise survive in the build output forever and keep
# appearing on the Mods page (and inflate release catalog assertions).
# Scoped to mods/packages, not mods/: state.toml is launcher-owned user
# state and must survive a rebuild.
set(_psx_builtin_mod_copy_commands "")
if(DEFINED PSX_BUILTIN_MOD_ALLOWLIST AND NOT
"${PSX_BUILTIN_MOD_ALLOWLIST}" STREQUAL "")
foreach(_psx_builtin_mod IN LISTS PSX_BUILTIN_MOD_ALLOWLIST)
if(NOT EXISTS "${PSXRECOMP_ROOT}/mods/builtin/packages/${_psx_builtin_mod}")
message(FATAL_ERROR
"PSX_BUILTIN_MOD_ALLOWLIST names missing package: ${_psx_builtin_mod}")
endif()
list(APPEND _psx_builtin_mod_copy_commands
COMMAND ${CMAKE_COMMAND} -E copy_directory
"${PSXRECOMP_ROOT}/mods/builtin/packages/${_psx_builtin_mod}"
"$<TARGET_FILE_DIR:${target}>/mods/packages/${_psx_builtin_mod}")
endforeach()
else()
list(APPEND _psx_builtin_mod_copy_commands
COMMAND ${CMAKE_COMMAND} -E copy_directory
"${PSXRECOMP_ROOT}/mods/builtin"
"$<TARGET_FILE_DIR:${target}>/mods")
endif()
add_custom_command(TARGET ${target} POST_BUILD
# Clear first: copy_directory MERGES, so a mod deleted from source
# would otherwise survive in the build output forever and keep
# appearing on the Mods page (and inflate the release packagers'
# catalog assertions). This runs before the game's own staging, so
# both catalogs land on a clean slate.
COMMAND ${CMAKE_COMMAND} -E rm -rf
"$<TARGET_FILE_DIR:${target}>/mods"
COMMAND ${CMAKE_COMMAND} -E copy_directory
"${PSXRECOMP_ROOT}/mods/builtin"
"$<TARGET_FILE_DIR:${target}>/mods"
"$<TARGET_FILE_DIR:${target}>/mods/packages"
${_psx_builtin_mod_copy_commands}
COMMENT "Staging framework-owned mod catalog"
VERBATIM)
unset(_psx_builtin_mod_copy_commands)
endif()
endif()
+14
View File
@@ -15,6 +15,10 @@
# include <windows.h>
#endif
#ifndef PSX_OVERLAY_FLAVOR
#define PSX_OVERLAY_FLAVOR 0
#endif
static char s_cmd[4096]; /* large: the runtime-constructed bundled tcc cmd has
* many absolute paths (python+script+recompiler+tcc+...) */
static char s_cwd[512];
@@ -855,6 +859,16 @@ int autocompile_request(void) {
if (s_cache_dir[0]) SetEnvironmentVariableA("PSX_OVERLAY_CACHE_DIR", s_cache_dir);
if (s_captures[0]) SetEnvironmentVariableA("PSX_OVERLAY_CAPTURES", s_captures);
SetEnvironmentVariableA("PSX_OVERLAY_LIVE_AUTOCOMPILE", "1");
/* Pin the FLAVOR the same way: the tool's --flavor defaults to 0 (play),
* so an instrumented (flavor 2) runtime spawned a compile whose shards it
* could never load — the loader's flavor guard rejected them and EVERY
* overlay ran interpreted, forever, on debug builds. The child must
* always write the flavor this runtime reads. */
{
char flavor_buf[16];
snprintf(flavor_buf, sizeof flavor_buf, "%d", (int)PSX_OVERLAY_FLAVOR);
SetEnvironmentVariableA("PSX_OVERLAY_FLAVOR", flavor_buf);
}
/* cmd.exe /C resolves the command via PATH and supports the relative
* paths in the configured line (cwd = project root). The WHOLE command is
+13
View File
@@ -8044,6 +8044,18 @@ static void handle_savestate(int id, const char *json)
send_fmt("{\"id\":%d,\"ok\":true,\"op\":\"%s\",\"slot\":%d}", id, op, slot);
}
/* Deterministic harness receipt: unlike the savestate acknowledgement above,
* this reports the safe-boundary completion generation after savestate_poll
* has actually applied or rejected the request. */
static void handle_savestate_status(int id, const char *json)
{
(void)json;
extern void savestate_status_json(char *buf, size_t cap);
char status[256];
savestate_status_json(status, sizeof status);
send_fmt("{\"id\":%d,\"ok\":true,%s}", id, status);
}
static void handle_turbo(int id, const char *json)
{
int enabled = json_get_int(json, "enabled", -1);
@@ -13514,6 +13526,7 @@ static const CmdEntry s_commands[] = {
{ "input_route_stop", handle_input_route_stop },
{ "input_route_status",handle_input_route_status },
{ "savestate", handle_savestate },
{ "savestate_status", handle_savestate_status },
{ "turbo", handle_turbo },
{ "turbo_state", handle_turbo_state },
{ "pause", handle_pause },
+66 -3
View File
@@ -1119,7 +1119,8 @@ static std::string g_bezel_path; /* mod-owned OpenGL margin artwork */
static int g_fullscreen = 0; /* tri-state: 0 windowed, 1 borderless (desktop)
* fullscreen, 2 exclusive fullscreen */
static int g_video_screen = 0; /* 0=raw,1=crt,2=composite,3=trinitron */
static int g_video_win_w = 1280; /* window width (height follows aspect) */
static int g_video_win_w = 0; /* 0 = fit the display; see clamp_window_aspect */
static bool g_video_win_w_explicit = false; /* user chose a width */
static bool g_audio_spu_hq = false; /* SPU float-shadow (env overrides) */
static int g_audio_freq = 44100; /* host device request */
static int g_auto_skip_fmv = 0; /* skip FMVs the instant they're detected */
@@ -1474,9 +1475,18 @@ static int g_logical_w = 640;
* the given aspect: height = width*den/num. */
static void clamp_window_aspect(int* w, int* h, int num, int den) {
int width = *w;
if (width < 640) width = 640;
SDL_Rect bounds;
if (SDL_GetDisplayUsableBounds(0, &bounds) == 0 && bounds.w > 0 && bounds.h > 0) {
const int have_bounds =
(SDL_GetDisplayUsableBounds(0, &bounds) == 0 && bounds.w > 0 && bounds.h > 0);
/* 0 = "fit the display". The old default was a hardcoded 1280, which on a
* 4K or 8K panel opens a small window in the corner and, worse, makes the
* image far smaller than the internal render resolution the user chose --
* supersampling 16 rendering into a 1280-wide window throws almost all of
* it away. Fitting the usable bounds keeps the window proportional to the
* display it is actually on. An explicit width still wins. */
if (width <= 0) width = have_bounds ? bounds.w : 1280;
if (width < 640) width = 640;
if (have_bounds) {
if (width > bounds.w) width = bounds.w;
if (width * den / num > bounds.h) width = bounds.h * num / den;
}
@@ -11026,6 +11036,7 @@ int main(int argc, char** argv) {
#endif
if (us.has_supersampling) g_video_scale = us.supersampling;
if (us.has_window_width) g_video_win_w = us.window_width;
if (us.has_window_width && us.window_width > 0) g_video_win_w_explicit = true;
if (us.has_antialiasing) g_video_aa = us.antialiasing;
if (us.has_texture_filter) g_video_texfilter = us.texture_filter;
if (us.has_geometry_correction)
@@ -11439,6 +11450,24 @@ int main(int argc, char** argv) {
(!std::getenv("PSX_NO_LAUNCHER") && !force_no_launcher && !skip_launcher_setting);
if (want_launcher) {
launcher_boot_timing_mark("host:before_sdl_init");
/* Per-monitor DPI awareness, BEFORE any SDL_Init.
*
* Without it Windows virtualises everything this process sees: on a
* 7680x4320 panel at 400% scaling SDL_GetDisplayUsableBounds reports
* 1920x1032, so the window is clamped to roughly 1376 LOGICAL pixels and
* opens as a small box, while the desktop compositor then upscales it.
* The internal render resolution is unaffected -- which is the trap: the
* game renders at supersampling 16 and the result is thrown away scaling
* a 1376-wide window up to an 8K display.
*
* permonitorv2 makes SDL report physical pixels, so the window sizes
* against the real panel and the drawable matches it 1:1. */
#ifdef SDL_HINT_WINDOWS_DPI_AWARENESS
SDL_SetHint(SDL_HINT_WINDOWS_DPI_AWARENESS, "permonitorv2");
#endif
#ifdef SDL_HINT_WINDOWS_DPI_SCALING
SDL_SetHint(SDL_HINT_WINDOWS_DPI_SCALING, "0");
#endif
if (SDL_Init(SDL_INIT_VIDEO | SDL_INIT_GAMECONTROLLER) == 0) {
launcher_boot_timing_mark("host:after_sdl_init");
recomp_launcher_set_preserve_sdl(1);
@@ -12653,6 +12682,24 @@ session_reboot:
* default). Enable the HIDAPI Xbox driver so HIDAPI handles Xbox pads too. */
SDL_SetHint(SDL_HINT_JOYSTICK_HIDAPI_XBOX, "1");
if (!SDL_WasInit(SDL_INIT_VIDEO | SDL_INIT_GAMECONTROLLER)) {
/* Per-monitor DPI awareness, BEFORE any SDL_Init.
*
* Without it Windows virtualises everything this process sees: on a
* 7680x4320 panel at 400% scaling SDL_GetDisplayUsableBounds reports
* 1920x1032, so the window is clamped to roughly 1376 LOGICAL pixels and
* opens as a small box, while the desktop compositor then upscales it.
* The internal render resolution is unaffected -- which is the trap: the
* game renders at supersampling 16 and the result is thrown away scaling
* a 1376-wide window up to an 8K display.
*
* permonitorv2 makes SDL report physical pixels, so the window sizes
* against the real panel and the drawable matches it 1:1. */
#ifdef SDL_HINT_WINDOWS_DPI_AWARENESS
SDL_SetHint(SDL_HINT_WINDOWS_DPI_AWARENESS, "permonitorv2");
#endif
#ifdef SDL_HINT_WINDOWS_DPI_SCALING
SDL_SetHint(SDL_HINT_WINDOWS_DPI_SCALING, "0");
#endif
if (SDL_Init(SDL_INIT_VIDEO | SDL_INIT_GAMECONTROLLER) != 0) {
std::fprintf(stderr, "SDL_Init failed: %s\n", SDL_GetError());
return 1;
@@ -12737,6 +12784,22 @@ session_reboot:
}
psx_apply_window_icon(sdl_window, argv[0]);
/* Maximise instead of computing the frame size ourselves.
*
* clamp_window_aspect fits the CLIENT area to the usable bounds, but a
* window is client plus title bar and borders, so fitting the client to a
* full-height display produced a window taller than the screen that hung
* off the top. Deriving the decoration size first does not work either:
* SDL_GetWindowBordersSize reports nothing useful before the window is
* shown, so the correction silently did not apply.
*
* The window manager already solves this exactly. Maximise and let it fit
* the work area, decorations and taskbar included. Only when the request
* was "fit the display" (window_width unset) -- an explicit width is a
* deliberate choice and is left alone. */
if (!g_fullscreen && !g_video_win_w_explicit)
SDL_MaximizeWindow(sdl_window);
/* Host refresh: if the panel is within ~2% of 60 Hz, record it so driver
* vsync can own cadence (pacer skipped). Non-~60 Hz and unknown refresh
* (common on Wayland) keep PSX 59.94 Hz pacing and force swap interval 0
+38
View File
@@ -55,6 +55,11 @@ static int s_save_pending = -1; /* slot, or -1 */
static int s_load_pending = -1;
static int s_load_completed = 0;
static int s_load_failed = 0;
static uint32_t s_status_generation = 0;
static int s_status_pending = 0;
static int s_status_last_ok = 0;
static int s_status_last_load = 0;
static int s_status_last_slot = -1;
static int s_save_failed = 0;
static uint32_t s_last_save_pc = 0;
static int s_save_defer_slot = -1;
@@ -587,6 +592,9 @@ static int request_save_inner(int slot) {
s_last_save_pc = 0; /* block netplay transfer until this write stamps a PC */
s_save_defer_slot = -1;
s_save_pending = slot;
s_status_pending = 1;
s_status_last_load = 0;
s_status_last_slot = slot;
return 1;
}
@@ -603,6 +611,9 @@ static int request_load_inner(int slot) {
s_load_failed = 0;
s_load_completed = 0;
s_load_pending = slot;
s_status_pending = 1;
s_status_last_load = 1;
s_status_last_slot = slot;
return 1;
}
@@ -658,6 +669,16 @@ int savestate_pending(void) {
return (s_save_pending >= 0 || s_load_pending >= 0) ? 1 : 0;
}
void savestate_status_json(char* buf, size_t cap) {
if (!buf || cap == 0) return;
snprintf(buf, cap,
"\"generation\":%u,\"pending\":%d,\"last_ok\":%d,"
"\"last_op\":\"%s\",\"last_slot\":%d",
(unsigned)s_status_generation, s_status_pending,
s_status_last_ok, s_status_last_load ? "load" : "save",
s_status_last_slot);
}
int savestate_take_load_completed(void) {
int v = s_load_completed;
s_load_completed = 0;
@@ -705,6 +726,9 @@ void savestate_poll(CPUState* cpu, uint32_t resume_pc) {
s_save_defer_slot = -1;
s_last_save_pc = 0;
s_save_failed = 1;
s_status_pending = 0;
s_status_last_ok = 0;
s_status_generation++;
fprintf(stderr,
"savestate: SAVE FAILED slot %d — no safe resume PC "
"(hint=0x%08X)\n",
@@ -733,12 +757,18 @@ void savestate_poll(CPUState* cpu, uint32_t resume_pc) {
s_last_save_pc = 0;
s_save_failed = 1;
}
s_status_pending = 0;
s_status_last_ok = ok ? 1 : 0;
s_status_generation++;
fprintf(stderr, "savestate: %s slot %d @ pc=0x%08X -> %s\n",
ok ? "SAVED" : "SAVE FAILED", slot, (unsigned)pc, path);
psx_frontend_on_savestate_notify(0, slot, ok);
} else {
s_last_save_pc = 0;
s_save_failed = 1;
s_status_pending = 0;
s_status_last_ok = 0;
s_status_generation++;
psx_frontend_on_savestate_notify(0, slot, 0);
}
}
@@ -788,6 +818,11 @@ void savestate_poll(CPUState* cpu, uint32_t resume_pc) {
s_load_failed = 1;
psx_frontend_on_savestate_notify(1, slot, 0);
}
if (!loaded) {
s_status_pending = 0;
s_status_last_ok = 0;
s_status_generation++;
}
if (loaded) {
t_after_boot = savestate_mono_ms();
psx_cycles_resync_after_restore(cpu);
@@ -801,6 +836,9 @@ void savestate_poll(CPUState* cpu, uint32_t resume_pc) {
cdrom_accelerate_after_savestate();
/* Netplay post-load barrier observes this before the longjmp. */
s_load_completed = 1;
s_status_pending = 0;
s_status_last_ok = 1;
s_status_generation++;
/* Restage FBO/present latch so the restored frame is visible
* immediately (avoids disabled-display blank latch + stale smooth). */
psx_frontend_on_savestate_loaded();
@@ -0,0 +1,14 @@
"""Guard the deterministic savestate completion receipt used by load gates."""
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
HEADER = (ROOT / "include/savestate.h").read_text(encoding="utf-8")
STATE = (ROOT / "src/savestate.c").read_text(encoding="utf-8")
SERVER = (ROOT / "src/debug_server.c").read_text(encoding="utf-8")
assert "void savestate_status_json(char* buf, size_t cap);" in HEADER
assert '\\"generation\\"' in STATE and '\\"pending\\"' in STATE
assert "s_status_generation++" in STATE
assert '"savestate_status"' in SERVER
assert "savestate_status_json(status, sizeof status)" in SERVER
print("savestate status protocol guard passed")
+13
View File
@@ -5259,6 +5259,19 @@ def main():
if _env_cap != args.captures:
print(f'[cache] PSX_OVERLAY_CAPTURES overrides --captures: {_env_cap}')
args.captures = _env_cap
# Flavor is pinned by the spawning runtime the same way as the cache
# locations: --flavor defaults to 0 (play), so an instrumented (flavor 2)
# runtime used to spawn compiles whose shards its loader then rejected —
# every overlay ran interpreted on debug builds, silently.
_env_flavor = os.environ.get('PSX_OVERLAY_FLAVOR')
if _env_flavor:
try:
_fl = int(_env_flavor, 0)
except ValueError:
_fl = None
if _fl is not None and _fl != args.flavor:
print(f'[cache] PSX_OVERLAY_FLAVOR overrides --flavor: {_fl}')
args.flavor = _fl
if not args.captures:
ap.error('no captures file: set PSX_OVERLAY_CAPTURES (runtime injects it) '
'or pass --captures for manual/offline use')
+64 -20
View File
@@ -1,12 +1,13 @@
#!/usr/bin/env python3
"""Fetch PS1 Named_Boxarts from libretro-thumbnails and write launcher TGA.
"""Fetch PS1 Named_Boxarts from libretro-thumbnails (PNG + launcher TGA).
Sources (tried in order):
https://thumbnails.libretro.com/Sony - PlayStation/Named_Boxarts/<name>.png
https://raw.githubusercontent.com/libretro-thumbnails/Sony_-_PlayStation/master/Named_Boxarts/<name>.png
Writes 32-bit uncompressed TGA (bottom-up BGRA) for recomp-ui LAUNCHER_BOXART.
Also writes launcher_assets/img/BOXART_SOURCE.txt with URL + attribution.
Writes the original PNG (README) and a 32-bit uncompressed TGA (bottom-up BGRA)
for recomp-ui LAUNCHER_BOXART. Also writes launcher_assets/img/BOXART_SOURCE.txt
with URL + attribution.
Requires network. Prefer Pillow when installed; otherwise decodes common
8-bit RGBA/RGB PNGs with the stdlib.
@@ -120,7 +121,7 @@ def try_fetch(names: list[str]) -> tuple[bytes, str, str]:
except (urllib.error.HTTPError, urllib.error.URLError, TimeoutError, RuntimeError) as e:
last_err = e
continue
raise SystemExit(
raise FileNotFoundError(
f"boxart not found for candidates {names[:8]}… ({last_err})"
)
@@ -241,6 +242,40 @@ def write_tga_bgra(path: Path, width: int, height: int, rgba: bytes) -> None:
path.write_bytes(bytes(header) + bytes(bgra))
def fetch_to_paths(
tga_path: Path,
*,
cue_stem: str = "",
display_name: str = "",
extra_names: list[str] | None = None,
png_path: Path | None = None,
) -> tuple[Path, Path, str]:
"""Fetch libretro Named_Boxarts PNG, write PNG + TGA + BOXART_SOURCE.txt."""
names = candidate_names(cue_stem, display_name, *(extra_names or []))
if not names:
raise ValueError("pass cue_stem and/or display_name")
png, url, matched = try_fetch(names)
tga_path = Path(tga_path)
png_path = Path(png_path) if png_path else tga_path.with_suffix(".png")
png_path.parent.mkdir(parents=True, exist_ok=True)
png_path.write_bytes(png)
try:
w, h, rgba = png_to_rgba(png)
write_tga_bgra(tga_path, w, h, rgba)
except SystemExit as exc:
raise RuntimeError(str(exc) or "PNG decode failed") from exc
src_path = tga_path.parent / "BOXART_SOURCE.txt"
src_path.write_text(
"Box art sourced from libretro-thumbnails (Named_Boxarts).\n"
f"Matched name: {matched}\n"
f"URL: {url}\n"
"https://github.com/libretro-thumbnails/libretro-thumbnails\n"
"PNG is for README; TGA is for recomp-ui LAUNCHER_BOXART.\n",
encoding="utf-8",
)
return tga_path, png_path, url
def main() -> int:
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument(
@@ -256,6 +291,11 @@ def main() -> int:
default="",
help="attribution file (default: next to TGA as BOXART_SOURCE.txt)",
)
ap.add_argument(
"--png-out",
default="",
help="output PNG path (default: same stem as --out with .png)",
)
args = ap.parse_args()
names = candidate_names(args.cue_stem, args.display_name, *args.name)
@@ -264,24 +304,28 @@ def main() -> int:
return 2
print(f" searching libretro boxart ({len(names)} candidates)…", file=sys.stderr)
png, url, matched = try_fetch(names)
print(f" fetched {matched}", file=sys.stderr)
print(f" {url}", file=sys.stderr)
w, h, rgba = png_to_rgba(png)
out = Path(args.out)
write_tga_bgra(out, w, h, rgba)
print(f" wrote {out} ({w}x{h})", file=sys.stderr)
png_out = Path(args.png_out) if args.png_out else out.with_suffix(".png")
try:
tga_path, png_path, url = fetch_to_paths(
out,
cue_stem=args.cue_stem,
display_name=args.display_name,
extra_names=list(args.name),
png_path=png_out,
)
except (FileNotFoundError, ValueError, RuntimeError) as exc:
print(str(exc) or "boxart fetch failed", file=sys.stderr)
return 1
print(f" wrote {png_path}", file=sys.stderr)
print(f" wrote {tga_path}", file=sys.stderr)
print(f" {url}", file=sys.stderr)
src_path = Path(args.source_out) if args.source_out else out.parent / "BOXART_SOURCE.txt"
src_path.write_text(
"Box art sourced from libretro-thumbnails (Named_Boxarts).\n"
f"Matched name: {matched}\n"
f"URL: {url}\n"
"https://github.com/libretro-thumbnails/libretro-thumbnails\n"
"Convert: PNG → 32-bit TGA for recomp-ui LAUNCHER_BOXART.\n",
encoding="utf-8",
)
if args.source_out:
src_path.write_text(
(tga_path.parent / "BOXART_SOURCE.txt").read_text(encoding="utf-8"),
encoding="utf-8",
)
print(f" wrote {src_path}", file=sys.stderr)
return 0
+60
View File
@@ -8,6 +8,12 @@ import re
import sys
from pathlib import Path
GITHUB_ABOUT_DESCRIPTION = (
"Made with PSXrecomp, a Sony PlayStation game static recompiler ecosystem · "
"Part of the R.A.I.D. community"
)
GITHUB_ABOUT_HOMEPAGE = "https://discord.gg/Ad9BwSzctP"
def derive_zip_prefix(name: str) -> str:
base = re.sub(r"(?i)recomp(iled)?$", "", name).strip()
@@ -20,6 +26,60 @@ def derive_zip_prefix(name: str) -> str:
return (slug or "game")[:20]
def sanitize_github_name(name: str) -> str:
"""GitHub owner/repo slug: spaces → '-', drop illegal chars, keep [A-Za-z0-9._-]."""
s = (name or "").strip()
if not s:
return ""
s = re.sub(r"\s+", "-", s)
s = re.sub(r"[^A-Za-z0-9._-]+", "", s)
s = re.sub(r"-{2,}", "-", s)
s = s.strip(".-")
return s[:100] or "repo"
def install_dir_name(name: str) -> str:
"""Local checkout / launcher ``apps/`` folder — same slug as the GitHub repo name.
New-project wizard must create this folder (not the display name with spaces)
so Studio catalog matching and RetComM ``install_dir_name`` stay consistent.
"""
return sanitize_github_name(name)
def normalize_repo_key(name: str) -> str:
"""Casefold + collapse whitespace/underscores to hyphens for folder/catalog match.
Lets ``Wipeout 3 Special Edition Recomp`` match catalog
``Wipeout-3-Special-Edition-Recomp`` / github short name without renaming.
"""
s = (name or "").strip().lower()
if not s:
return ""
s = re.sub(r"[\s_]+", "-", s)
s = re.sub(r"[^a-z0-9._-]+", "", s)
s = re.sub(r"-{2,}", "-", s)
return s.strip(".-")
def repo_match_keys(name: str) -> set[str]:
"""Match keys for a folder / install_dir / github short name.
Includes the normalized form and a form with a trailing ``-recomp`` /
``-recompiled`` stripped so local ``… Recomp`` checkouts still match
catalog ``install_dir_name`` / github slugs that omit that suffix
(e.g. Klonoa).
"""
k = normalize_repo_key(name)
if not k:
return set()
out = {k}
stripped = re.sub(r"(-recomp(iled)?)+$", "", k).strip("-")
if stripped:
out.add(stripped)
return out
def main() -> int:
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("src")
@@ -509,7 +509,13 @@ def resolve_catalog_root(studio_toml: Path | None = None) -> Path | None:
def _catalog_name_keys(catalog_root: Path) -> set[str]:
"""Folder / github short-name keys (lowercased) for every catalog title."""
"""Folder / github short-name keys for every catalog title.
Stores raw lowercased names plus ``repo_match_keys`` forms so spacey /
``… Recomp`` legacy checkouts still match hyphenated ``install_dir_name``.
"""
from fill_tokens import repo_match_keys
index_path = catalog_root / "index.json"
titles_dir = catalog_root / "titles"
if not index_path.is_file() or not titles_dir.is_dir():
@@ -519,8 +525,16 @@ def _catalog_name_keys(catalog_root: Path) -> set[str]:
except (OSError, json.JSONDecodeError):
return set()
keys: set[str] = set()
def _add(raw: str) -> None:
s = (raw or "").strip()
if not s:
return
keys.add(s.lower())
keys.update(repo_match_keys(s))
for tid in ids:
keys.add(str(tid).lower())
_add(str(tid))
path = titles_dir / f"{tid}.json"
if not path.is_file():
continue
@@ -530,7 +544,7 @@ def _catalog_name_keys(catalog_root: Path) -> set[str]:
continue
name = str(raw.get("install_dir_name") or "").strip()
if name:
keys.add(name.lower())
_add(name)
for nest in (
("release", "github"),
("build", "source", "github"),
@@ -543,15 +557,141 @@ def _catalog_name_keys(catalog_root: Path) -> set[str]:
break
cur = cur[k]
if ok and isinstance(cur, str) and "/" in cur:
keys.add(cur.rsplit("/", 1)[-1].lower())
_add(cur.rsplit("/", 1)[-1])
homepage = str(raw.get("homepage") or "")
if "github.com/" in homepage.lower():
tail = homepage.rstrip("/").split("github.com/")[-1]
if "/" in tail:
keys.add(tail.split("/")[-1].lower())
_add(tail.split("/")[-1])
return keys
def catalog_title_id_for_root(
root: Path,
*,
catalog_root: Path | None = None,
) -> str | None:
"""Best-effort catalog title id for a local checkout (install_dir / github slug)."""
from fill_tokens import repo_match_keys
cat = catalog_root or resolve_catalog_root()
if cat is None:
return None
index_path = cat / "index.json"
titles_dir = cat / "titles"
if not index_path.is_file() or not titles_dir.is_dir():
return None
try:
ids = list(json.loads(index_path.read_text(encoding="utf-8")).get("titles") or [])
except (OSError, json.JSONDecodeError):
return None
want = set(repo_match_keys(root.name))
try:
from .gitops import _git
code, url, _ = _git(root, "remote", "get-url", "origin")
if code == 0 and url:
u = url.strip().rstrip("/")
if u.endswith(".git"):
u = u[:-4]
if "github.com" in u.lower():
# git@github.com:Owner/Repo or https://github.com/Owner/Repo
tail = u.replace(":", "/").split("github.com/")[-1]
if "/" in tail:
want.update(repo_match_keys(tail.split("/")[-1]))
except Exception:
pass
want.discard("")
for tid in ids:
path = titles_dir / f"{tid}.json"
if not path.is_file():
continue
try:
raw = json.loads(path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError):
continue
candidates = [str(tid), str(raw.get("install_dir_name") or "")]
gh = raw.get("release") if isinstance(raw.get("release"), dict) else {}
if isinstance(gh, dict) and isinstance(gh.get("github"), str) and "/" in gh["github"]:
candidates.append(gh["github"].rsplit("/", 1)[-1])
homepage = str(raw.get("homepage") or "")
if "github.com/" in homepage.lower():
tail = homepage.rstrip("/").split("github.com/")[-1]
if "/" in tail:
candidates.append(tail.split("/")[-1])
for c in candidates:
if repo_match_keys(c) & want:
return str(tid)
return None
def upsert_studio_toml_title(
title_id: str,
project_root: Path,
*,
studio_toml: Path | None = None,
) -> str | None:
"""Ensure ``[titles] "id" = "rel/path"`` in studio.toml. Returns note or None."""
cfg = studio_toml or find_studio_toml()
if cfg is None or not (title_id or "").strip():
return None
title_id = title_id.strip()
root = project_root.expanduser().resolve()
if not root.is_dir():
return None
try:
rel = os.path.relpath(root, cfg.parent.resolve()).replace("\\", "/")
except ValueError:
rel = str(root)
key_line = f'"{title_id}" = "{rel}"'
text = cfg.read_text(encoding="utf-8")
# Already present with any path → leave alone (user may have customized).
existing = re.search(
rf'^[ \t]*"{re.escape(title_id)}"\s*=\s*("[^"]*"|\'[^\']*\')',
text,
re.M,
)
if existing:
return f"studio.toml already maps {title_id}"
if re.search(r"^\[titles\]\s*$", text, re.M):
# Append after [titles] block's last assignment, or right after header.
lines = text.splitlines(keepends=True)
out: list[str] = []
in_titles = False
inserted = False
i = 0
while i < len(lines):
line = lines[i]
s = line.strip()
if s.startswith("[") and s.endswith("]"):
if in_titles and not inserted:
out.append(key_line + "\n")
inserted = True
in_titles = s == "[titles]"
out.append(line)
i += 1
continue
out.append(line)
i += 1
if in_titles and not inserted:
if out and not out[-1].endswith("\n"):
out[-1] = out[-1] + "\n"
out.append(key_line + "\n")
inserted = True
if not inserted:
return None
cfg.write_text("".join(out), encoding="utf-8")
else:
suffix = "" if text.endswith("\n") or not text else "\n"
cfg.write_text(
text + suffix + "\n[titles]\n" + key_line + "\n",
encoding="utf-8",
)
return f"studio.toml [titles] += {title_id}"
def _studio_title_paths(studio_toml: Path | None = None) -> set[Path]:
cfg = studio_toml or find_studio_toml()
if cfg is None:
@@ -576,6 +716,8 @@ def repo_has_catalog_entry(
studio_toml: Path | None = None,
) -> bool:
"""True if this indexed checkout maps to a retcomm-catalog title."""
from fill_tokens import repo_match_keys
try:
root = entry.resolved()
except OSError:
@@ -589,12 +731,34 @@ def repo_has_catalog_entry(
keys = _catalog_name_keys(cat)
if not keys:
return False
basename = root.name.lower()
if basename in keys:
def _hits(label: str) -> bool:
s = (label or "").strip()
if not s:
return False
if s.lower() in keys:
return True
return bool(repo_match_keys(s) & keys)
if _hits(root.name):
return True
name = (entry.name or "").strip().lower()
if name and name in keys:
if _hits(entry.name or ""):
return True
# GitHub origin short name (catalog often matches repo slug, not local folder).
try:
from .gitops import _git
code, url, _ = _git(root, "remote", "get-url", "origin")
if code == 0 and url:
u = url.strip().rstrip("/")
if u.endswith(".git"):
u = u[:-4]
if "github.com" in u.lower():
tail = u.replace(":", "/").split("github.com/")[-1]
if "/" in tail and _hits(tail.split("/")[-1]):
return True
except Exception:
pass
return False
File diff suppressed because it is too large Load Diff
@@ -14,6 +14,13 @@ from .models import (
Severity,
)
from .naming import boot_exe_from_game_toml, infer_project_name
from .readme_metrics import (
boxart_png_present,
readme_has_boxart,
readme_has_launcher,
readme_has_metrics,
readme_has_raid,
)
def _read(path: Path) -> str:
@@ -749,6 +756,47 @@ def audit_project(root: Path) -> AuditReport:
)
)
# README download badges + boxart + RetComM Launcher (idempotent migrate op)
readme_path = root / "README.md"
readme_text = _read(readme_path) if readme_path.is_file() else ""
missing_readme: list[str] = []
if not readme_path.is_file():
missing_readme.append("README.md")
else:
if not readme_has_metrics(readme_text):
missing_readme.append("download badges")
if not readme_has_boxart(readme_text):
missing_readme.append("libretro boxart")
if not readme_has_launcher(readme_text):
missing_readme.append("RetComM Launcher section")
if not readme_has_raid(readme_text):
missing_readme.append("R.A.I.D. Discord footer")
if not (root / ".github" / "raid-discord.png").is_file():
missing_readme.append(".github/raid-discord.png")
if not boxart_png_present(root):
missing_readme.append("launcher_assets/img/boxart.png")
if missing_readme:
checks.append(
CheckResult(
id="readme_metrics",
title="README download metrics / launcher / RAID / boxart",
status=CheckStatus.WARN,
severity=Severity.RECOMMENDED,
detail="Missing: " + ", ".join(missing_readme),
fix_op="patch_readme_metrics",
)
)
else:
checks.append(
CheckResult(
id="readme_metrics",
title="README download metrics / launcher / RAID / boxart",
status=CheckStatus.PASS,
severity=Severity.RECOMMENDED,
detail="Download badges, boxart, RetComM Launcher, and R.A.I.D. footer present.",
)
)
# Classify layout
fails = {c.id for c in checks if c.status == CheckStatus.FAIL}
rec_warns = [
@@ -83,6 +83,8 @@ class MigrateOptions:
boot_exe: str | None = None
players: int = 2
zip_prefix: str | None = None
github_owner: str | None = None
github_repo: str | None = None
window_title: str | None = None
enable_recomp_ui: bool = True
enable_wizard: bool = True
@@ -5,7 +5,7 @@ from __future__ import annotations
import re
from pathlib import Path
from fill_tokens import derive_zip_prefix
from fill_tokens import derive_zip_prefix, sanitize_github_name
DEFAULT_LOBBY_HOST = "netplay.retcomm.net"
@@ -220,6 +220,8 @@ def build_token_map(
enable_wizard: bool = True,
enable_netplay: bool = False,
has_boxart: bool = False,
github_owner: str | None = None,
github_repo: str | None = None,
) -> dict[str, str]:
cmake_name = project_cmake_name(name)
title = window_title or window_title_from_name(name)
@@ -294,6 +296,10 @@ def build_token_map(
"ENV_PREFIX": env_prefix(cmake_name),
"EXE_BASENAME": exe_basename(title),
"ZIP_PREFIX": zp,
"GITHUB_OWNER": sanitize_github_name(
(github_owner or "").strip() or "TechnicallyComputers"
),
"GITHUB_REPO": sanitize_github_name((github_repo or "").strip() or name),
"DISC_HINT": f"your legally owned {game} disc",
"GAME_TITLE": title,
"PLAYERS": str(players),
@@ -0,0 +1,374 @@
"""New-project wizard: drive setup_project.sh / .ps1 non-interactively."""
from __future__ import annotations
import os
import platform
import shutil
import subprocess
import threading
from collections.abc import Callable
from dataclasses import dataclass
from pathlib import Path
from .gitops import CmdResult, switch_modules
from .paths import toolkit_dir
@dataclass
class NewProjectOptions:
"""Mirrors setup_project.sh / setup_project.ps1 CLI flags."""
name: str = ""
disc: str = ""
parent_dir: str = ""
bios: str = ""
boot_exe: str = ""
players: int = 2
zip_prefix: str = ""
github_owner: str = ""
github_repo: str = ""
description: str = ""
publisher: str = ""
year: str = ""
region: str = "USA"
enable_recomp_ui: bool = True
enable_wizard: bool = True
# GUI defaults: all on except GitHub; netplay follows players (≥2).
enable_netplay: bool = False
lobby_url: str = "netplay.retcomm.net"
enable_ci: bool = True
fetch_boxart: bool = True
stage_disc: bool = True
do_generate: bool = True
do_build: bool = True
create_github: bool = False
github_visibility: str = "private" # public | private | internal
psxrecomp_ref: str = "master"
recomp_ui_ref: str = "master"
recomp_net_ref: str = "" # empty = keep psxrecomp pin
rbengine_ref: str = "" # post-setup switch only (script has no flag)
dry_run: bool = False
def setup_script_paths() -> tuple[Path, Path]:
"""Return (setup_project.sh, setup_project.ps1) under the toolkit."""
base = toolkit_dir()
return base / "setup_project.sh", base / "setup_project.ps1"
def is_windows() -> bool:
return platform.system().lower().startswith("win") or os.name == "nt"
def project_folder_name(opts: NewProjectOptions) -> str:
"""Checkout folder = GitHub/catalog install_dir slug (not display name with spaces)."""
from fill_tokens import install_dir_name, sanitize_github_name
repo = (opts.github_repo or "").strip()
if repo:
return sanitize_github_name(repo)
return install_dir_name(opts.name or "")
def project_root_for(opts: NewProjectOptions) -> Path:
parent = Path(opts.parent_dir or ".").expanduser()
if not parent.is_absolute():
parent = parent.resolve()
else:
parent = parent.resolve()
folder = project_folder_name(opts)
if not folder:
folder = (opts.name or "").strip() or "repo"
return (parent / folder).resolve()
def validate_options(opts: NewProjectOptions) -> list[str]:
errs: list[str] = []
name = (opts.name or "").strip()
disc = (opts.disc or "").strip()
if not name:
errs.append("Project name is required")
if not disc:
errs.append("Disc .cue path is required")
else:
p = Path(disc).expanduser()
if not p.is_file():
errs.append(f"Disc not found: {disc}")
if opts.bios:
bp = Path(opts.bios).expanduser()
if not bp.is_file():
errs.append(f"BIOS not found: {opts.bios}")
if opts.players < 1 or opts.players > 8:
errs.append("Players must be 1–8")
if opts.do_build and not opts.do_generate:
errs.append("Build requires Generate")
# Wizard/netplay without UI (and 1P netplay) are auto-corrected at run time.
vis = (opts.github_visibility or "private").strip().lower()
if vis not in ("public", "private", "internal"):
errs.append("GitHub visibility must be public/private/internal")
dest = project_root_for(opts)
if dest.exists():
if dest.is_file() or (dest.is_dir() and any(dest.iterdir())):
errs.append(f"Destination already exists: {dest}")
return errs
def build_command(opts: NewProjectOptions) -> tuple[list[str], dict[str, str]]:
"""Build argv + env for the OS-appropriate setup script.
Always passes ``--yes`` / ``-Yes`` so the GUI/CLI supply every choice.
"""
sh, ps1 = setup_script_paths()
env = os.environ.copy()
env["PSXRECOMP_SETUP_YES"] = "1"
if (opts.recomp_net_ref or "").strip():
env["RECOMP_NET_REF"] = opts.recomp_net_ref.strip()
if is_windows():
if not ps1.is_file():
raise FileNotFoundError(f"Missing setup script: {ps1}")
powershell = (
shutil.which("pwsh")
or shutil.which("powershell")
or "powershell"
)
cmd: list[str] = [
powershell,
"-NoProfile",
"-ExecutionPolicy",
"Bypass",
"-File",
str(ps1),
"-Disc",
str(Path(opts.disc).expanduser().resolve()),
"-Name",
opts.name.strip(),
"-Yes",
"-Players",
str(int(opts.players)),
"-PsxrecompRef",
(opts.psxrecomp_ref or "master").strip(),
"-RecompUiRef",
(opts.recomp_ui_ref or "master").strip(),
"-GithubVisibility",
(opts.github_visibility or "private").strip().lower(),
]
parent = (opts.parent_dir or ".").strip() or "."
cmd.extend(["-Dir", str(Path(parent).expanduser().resolve())])
if opts.bios:
cmd.extend(["-Bios", str(Path(opts.bios).expanduser().resolve())])
if opts.boot_exe:
cmd.extend(["-BootExe", opts.boot_exe.strip()])
if opts.zip_prefix:
cmd.extend(["-ZipPrefix", opts.zip_prefix.strip()])
if opts.github_owner:
cmd.extend(["-GithubOwner", opts.github_owner.strip()])
if opts.github_repo:
cmd.extend(["-GithubRepo", opts.github_repo.strip()])
if opts.description:
cmd.extend(["-Description", opts.description.strip()])
if opts.publisher:
cmd.extend(["-Publisher", opts.publisher.strip()])
if opts.year:
cmd.extend(["-Year", opts.year.strip()])
if opts.region:
cmd.extend(["-Region", opts.region.strip()])
if opts.lobby_url:
cmd.extend(["-LobbyUrl", opts.lobby_url.strip()])
def sw(yes: bool, on: str, off: str) -> None:
cmd.append(on if yes else off)
sw(opts.enable_recomp_ui, "-EnableRecompUi", "-NoRecompUi")
sw(opts.enable_wizard, "-EnableWizard", "-NoWizard")
sw(opts.enable_netplay, "-EnableNetplay", "-NoNetplay")
sw(opts.enable_ci, "-EnableCi", "-NoCi")
sw(opts.fetch_boxart, "-FetchBoxart", "-NoFetchBoxart")
sw(opts.do_generate, "-Generate", "-NoGenerate")
sw(opts.do_build, "-EnableBuild", "-NoBuild")
sw(opts.create_github, "-CreateGithub", "-NoGithub")
if opts.stage_disc:
cmd.append("-StageDisc")
return cmd, env
if not sh.is_file():
raise FileNotFoundError(f"Missing setup script: {sh}")
cmd = [
"sh",
str(sh),
"--yes",
"--disc",
str(Path(opts.disc).expanduser().resolve()),
"--name",
opts.name.strip(),
"--players",
str(int(opts.players)),
"--psxrecomp-ref",
(opts.psxrecomp_ref or "master").strip(),
"--recomp-ui-ref",
(opts.recomp_ui_ref or "master").strip(),
"--github-visibility",
(opts.github_visibility or "private").strip().lower(),
]
parent = (opts.parent_dir or ".").strip() or "."
cmd.extend(["--dir", str(Path(parent).expanduser().resolve())])
if opts.bios:
cmd.extend(["--bios", str(Path(opts.bios).expanduser().resolve())])
if opts.boot_exe:
cmd.extend(["--boot-exe", opts.boot_exe.strip()])
if opts.zip_prefix:
cmd.extend(["--zip-prefix", opts.zip_prefix.strip()])
if opts.github_owner:
cmd.extend(["--github-owner", opts.github_owner.strip()])
if opts.github_repo:
cmd.extend(["--github-repo", opts.github_repo.strip()])
if opts.description:
cmd.extend(["--description", opts.description.strip()])
if opts.publisher:
cmd.extend(["--publisher", opts.publisher.strip()])
if opts.year:
cmd.extend(["--year", opts.year.strip()])
if opts.region:
cmd.extend(["--region", opts.region.strip()])
if opts.lobby_url:
cmd.extend(["--lobby-url", opts.lobby_url.strip()])
if (opts.recomp_net_ref or "").strip():
cmd.extend(["--recomp-net-ref", opts.recomp_net_ref.strip()])
def flag(yes: bool, on: str, off: str) -> None:
cmd.append(on if yes else off)
flag(opts.enable_recomp_ui, "--enable-recomp-ui", "--no-recomp-ui")
flag(opts.enable_wizard, "--enable-wizard", "--no-wizard")
flag(opts.enable_netplay, "--enable-netplay", "--no-netplay")
flag(opts.enable_ci, "--enable-ci", "--no-ci")
flag(opts.fetch_boxart, "--fetch-boxart", "--no-fetch-boxart")
flag(opts.do_generate, "--generate", "--no-generate")
flag(opts.do_build, "--enable-build", "--no-build")
flag(opts.create_github, "--create-github", "--no-github")
cmd.append("--stage-disc" if opts.stage_disc else "--no-stage-disc")
return cmd, env
def run_new_project(
opts: NewProjectOptions,
*,
on_line: Callable[[str], None] | None = None,
cancel_event: threading.Event | None = None,
) -> CmdResult:
"""Run setup_project end-to-end; stream lines via ``on_line``."""
errs = validate_options(opts)
if errs:
return CmdResult(False, "Invalid new-project options", "\n".join(errs))
# Match script policy: no UI ⇒ no wizard/netplay; 1P ⇒ no netplay.
if not opts.enable_recomp_ui:
opts.enable_wizard = False
opts.enable_netplay = False
if opts.players < 2:
opts.enable_netplay = False
try:
cmd, env = build_command(opts)
except FileNotFoundError as exc:
return CmdResult(False, str(exc))
if opts.dry_run:
preview = " ".join(cmd)
if on_line:
on_line(f"[dry-run] {preview}")
return CmdResult(True, "dry-run: would run setup_project", preview)
if on_line:
on_line(f"$ {' '.join(cmd)}")
try:
proc = subprocess.Popen(
cmd,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True,
encoding="utf-8",
errors="replace",
env=env,
cwd=str(Path(opts.parent_dir or ".").expanduser().resolve()),
)
except OSError as exc:
return CmdResult(False, f"Failed to start setup: {exc}")
assert proc.stdout is not None
for line in proc.stdout:
if cancel_event is not None and cancel_event.is_set():
proc.terminate()
return CmdResult(False, "New project cancelled")
text = line.rstrip("\n")
if on_line:
on_line(text)
code = proc.wait()
root = project_root_for(opts)
if code != 0:
return CmdResult(
False,
f"setup_project failed (exit {code})",
str(root),
)
# Optional nested rbengine (and net on Windows, where script has no ref flag)
post_notes: list[str] = []
if root.is_dir():
nested_branches: dict[str, str] = {}
net = (opts.recomp_net_ref or "").strip()
rb = (opts.rbengine_ref or "").strip()
if net and is_windows():
nested_branches["lib/recomp-net"] = net
if rb:
nested_branches["lib/retcomm-rbengine"] = rb
if nested_branches:
if on_line:
on_line("== Post-setup nested lib branch switch ==")
for r in switch_modules(
root,
nested=True,
branch_by_path=nested_branches,
paths=list(nested_branches.keys()),
set_tracking=True,
dry_run=False,
):
post_notes.append(r.message)
if on_line:
on_line(f" [{'OK' if r.ok else 'FAIL'}] {r.message}")
detail = str(root)
if post_notes:
detail += "\n" + "\n".join(post_notes)
return CmdResult(True, f"Created project at {root}", detail)
def index_new_project(
root: Path,
*,
name: str = "",
cue: str = "",
) -> CmdResult:
"""Add/update the Studio repo index for a freshly created project."""
from .bulkops import (
catalog_title_id_for_root,
upsert_studio_toml_title,
)
from .repo_index import add_repo, load_index
if not root.is_dir():
return CmdResult(False, f"Project root missing: {root}")
idx = load_index()
entry = add_repo(idx, root, name=name or root.name, cue=cue)
notes: list[str] = [entry.path]
tid = catalog_title_id_for_root(root)
if tid:
note = upsert_studio_toml_title(tid, root)
if note:
notes.append(note)
return CmdResult(True, f"Indexed {entry.label()}", "\n".join(notes))
@@ -2,6 +2,7 @@
from __future__ import annotations
import json
import re
import shutil
import subprocess
@@ -15,6 +16,7 @@ from .naming import (
boot_exe_from_game_toml,
build_token_map,
entry_pc_from_game_toml,
game_name_from_project,
infer_project_name,
normalize_lobby_url,
players_from_cmake,
@@ -221,6 +223,8 @@ def _resolve_tokens(root: Path, options: MigrateOptions) -> dict[str, str]:
enable_wizard=options.enable_wizard if options.enable_recomp_ui else False,
enable_netplay=enable_netplay if options.enable_recomp_ui else False,
has_boxart=has_boxart,
github_owner=options.github_owner,
github_repo=options.github_repo,
)
@@ -1080,6 +1084,136 @@ def op_record_framework_pins(root: Path, options: MigrateOptions) -> ApplyResult
)
def _boxart_hints(root: Path, options: MigrateOptions) -> tuple[str, str]:
cue = ""
display = ""
if options.disc:
cue = Path(options.disc).name
cat = root / "catalog_identity.json"
if cat.is_file():
try:
data = json.loads(cat.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError):
data = {}
if isinstance(data, dict):
rom = data.get("rom_identity") if isinstance(data.get("rom_identity"), dict) else {}
cue = cue or str(rom.get("cue_name") or "")
game = data.get("game") if isinstance(data.get("game"), dict) else {}
display = str(game.get("name") or "")
name = options.project_name or infer_project_name(root)
display = options.window_title or display or game_name_from_project(name)
return cue, display
def _ensure_boxart_png(root: Path, options: MigrateOptions) -> list[str]:
"""Fetch libretro boxart PNG/TGA when the README PNG is missing. Never raises."""
png = root / "launcher_assets" / "img" / "boxart.png"
tga = root / "launcher_assets" / "img" / "boxart.tga"
if png.is_file():
return []
if options.dry_run:
return ["launcher_assets/img/boxart.png"]
cue, display = _boxart_hints(root, options)
if not cue and not display:
return []
try:
sys.path.insert(0, str(toolkit_dir()))
from fetch_boxart import fetch_to_paths # type: ignore
except ImportError:
return []
try:
fetch_to_paths(tga, cue_stem=cue, display_name=display)
except Exception:
return []
changed: list[str] = []
if png.is_file():
changed.append("launcher_assets/img/boxart.png")
if tga.is_file():
changed.append("launcher_assets/img/boxart.tga")
src = tga.parent / "BOXART_SOURCE.txt"
if src.is_file():
changed.append("launcher_assets/img/BOXART_SOURCE.txt")
return changed
def op_patch_readme_metrics(root: Path, options: MigrateOptions) -> ApplyResult:
"""Upsert download badges, libretro boxart, RetComM Launcher, and R.A.I.D. footer."""
from .readme_metrics import (
apply_github_about,
boxart_png_present,
render_boxart_block,
render_launcher_block,
render_metrics_block,
render_raid_block,
resolve_github_slug,
upsert_readme_blocks,
)
name = options.project_name or infer_project_name(root)
owner, repo = resolve_github_slug(root, options.github_owner, options.github_repo)
zp = options.zip_prefix or derive_zip_prefix(name)
fetched = _ensure_boxart_png(root, options)
display = options.window_title or game_name_from_project(name)
boxart_block = None
if boxart_png_present(root) or (
options.dry_run and "launcher_assets/img/boxart.png" in fetched
):
boxart_block = render_boxart_block(display)
path = root / "README.md"
if path.is_file():
old = path.read_text(encoding="utf-8", errors="replace")
else:
title = options.window_title or window_title_from_name(name)
old = f"# {title}\n"
new = upsert_readme_blocks(
old,
render_metrics_block(owner, repo, zp),
render_launcher_block(),
render_raid_block(),
boxart=boxart_block,
)
changed: list[str] = list(fetched)
if new != old:
_write(path, new, options.dry_run)
if "README.md" not in changed:
changed.append("README.md")
src_img = templates_dir() / "raid-discord.png"
dst_img = root / ".github" / "raid-discord.png"
if src_img.is_file() and not dst_img.is_file():
if not options.dry_run:
dst_img.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(src_img, dst_img)
changed.append(".github/raid-discord.png")
about_ok, about_msg = apply_github_about(owner, repo, dry_run=options.dry_run)
parts: list[str] = []
if changed:
parts.append(
f"Patched README badges, boxart, RetComM Launcher, and R.A.I.D. footer ({owner}/{repo})"
)
if fetched:
parts.append("fetched libretro boxart")
if about_ok:
parts.append(about_msg)
else:
parts.append("GitHub About skipped: " + about_msg)
if not changed and not about_ok:
return ApplyResult(
"patch_readme_metrics",
True,
f"README metrics already current ({owner}/{repo}); {about_msg}",
[],
)
return ApplyResult(
"patch_readme_metrics",
True,
"; ".join(parts),
changed,
)
_OPS = {
"rename_psxrecomp_submodule": op_rename_psxrecomp_submodule,
"ensure_psxrecomp_submodule": op_ensure_psxrecomp_submodule,
@@ -1100,6 +1234,7 @@ _OPS = {
"annotate_legacy_packaging": op_annotate_legacy_packaging,
"probe_disc_refresh": op_probe_disc_refresh,
"record_framework_pins": op_record_framework_pins,
"patch_readme_metrics": op_patch_readme_metrics,
}
@@ -29,6 +29,7 @@ _OP_ORDER = [
"annotate_legacy_packaging",
"probe_disc_refresh",
"record_framework_pins",
"patch_readme_metrics",
]
_OP_TITLES = {
@@ -51,6 +52,7 @@ _OP_TITLES = {
"annotate_legacy_packaging": "Annotate legacy prebuilt packaging",
"probe_disc_refresh": "Refresh disc identity via probe_disc.py",
"record_framework_pins": "Write framework_pins.txt",
"patch_readme_metrics": "Patch README badges, RetComM Launcher, and R.A.I.D. footer",
}
@@ -0,0 +1,348 @@
"""README download-badge + RetComM Launcher blocks for scaffold and migrate."""
from __future__ import annotations
import json
import re
import subprocess
from pathlib import Path
from fill_tokens import (
GITHUB_ABOUT_DESCRIPTION,
GITHUB_ABOUT_HOMEPAGE,
sanitize_github_name,
)
METRICS_BEGIN = "<!-- retcomm-readme-metrics -->"
METRICS_END = "<!-- /retcomm-readme-metrics -->"
BOXART_BEGIN = "<!-- retcomm-readme-boxart -->"
BOXART_END = "<!-- /retcomm-readme-boxart -->"
LAUNCHER_BEGIN = "<!-- retcomm-readme-launcher -->"
LAUNCHER_END = "<!-- /retcomm-readme-launcher -->"
RAID_BEGIN = "<!-- retcomm-readme-raid -->"
RAID_END = "<!-- /retcomm-readme-raid -->"
BOXART_REL = "launcher_assets/img/boxart.png"
_METRICS_RE = re.compile(
re.escape(METRICS_BEGIN) + r".*?" + re.escape(METRICS_END) + r"\n?",
re.S,
)
_BOXART_RE = re.compile(
re.escape(BOXART_BEGIN) + r".*?" + re.escape(BOXART_END) + r"\n?",
re.S,
)
_LAUNCHER_RE = re.compile(
re.escape(LAUNCHER_BEGIN) + r".*?" + re.escape(LAUNCHER_END) + r"\n?",
re.S,
)
_RAID_RE = re.compile(
re.escape(RAID_BEGIN) + r".*?" + re.escape(RAID_END) + r"\n?",
re.S,
)
_LEGACY_LAUNCHER_RE = re.compile(
r"^## RetComM Launcher\n.*?(?=^## |\Z)",
re.M | re.S,
)
_LEGACY_RAID_RE = re.compile(
r"(?:\n---\s*)?\n*<p align=\"center\">\s*\n\s*<sub><b>R\.A\.I\.D\..*\Z",
re.S,
)
_REMOTE_RE = re.compile(
r"github\.com[:/](?P<owner>[^/\s]+)/(?P<repo>[^/\s]+)",
re.I,
)
DEFAULT_GITHUB_OWNER = "TechnicallyComputers"
LAUNCHER_REPO = "https://github.com/TechnicallyComputers/RetComM-Launcher"
def parse_github_remote(url: str) -> tuple[str, str] | None:
url = (url or "").strip().rstrip("/")
m = _REMOTE_RE.search(url)
if not m:
return None
owner = m.group("owner")
repo = m.group("repo")
if repo.endswith(".git"):
repo = repo[: -len(".git")]
repo = repo.rstrip("/")
if not owner or not repo:
return None
return owner, repo
def infer_github_slug(root: Path) -> tuple[str, str] | None:
"""Best-effort owner/repo from origin, then catalog homepage."""
try:
proc = subprocess.run(
["git", "-C", str(root), "remote", "get-url", "origin"],
capture_output=True,
text=True,
check=False,
)
except OSError:
proc = None
if proc is not None and proc.returncode == 0:
parsed = parse_github_remote(proc.stdout.strip())
if parsed:
return parsed
cat = root / "catalog_identity.json"
if cat.is_file():
try:
data = json.loads(cat.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError):
data = {}
if isinstance(data, dict):
for key in ("homepage", "github", "repository", "url"):
val = data.get(key)
if isinstance(val, str):
parsed = parse_github_remote(val)
if parsed:
return parsed
return None
def resolve_github_slug(
root: Path,
owner: str | None = None,
repo: str | None = None,
) -> tuple[str, str]:
o = sanitize_github_name(owner or "")
r = sanitize_github_name(repo or "")
if o and r:
return o, r
inferred = infer_github_slug(root)
if inferred:
return (
o or sanitize_github_name(inferred[0]),
r or sanitize_github_name(inferred[1]),
)
return (o or DEFAULT_GITHUB_OWNER, r or sanitize_github_name(root.name))
def apply_github_about(
owner: str,
repo: str,
*,
dry_run: bool = False,
) -> tuple[bool, str]:
"""Set GitHub About description + Discord homepage via ``gh repo edit``."""
slug = f"{sanitize_github_name(owner) or DEFAULT_GITHUB_OWNER}/{sanitize_github_name(repo) or 'repo'}"
cmd = [
"gh",
"repo",
"edit",
slug,
"--description",
GITHUB_ABOUT_DESCRIPTION,
"--homepage",
GITHUB_ABOUT_HOMEPAGE,
]
if dry_run:
return True, "dry-run: " + " ".join(cmd)
try:
proc = subprocess.run(cmd, capture_output=True, text=True, check=False)
except OSError as exc:
return False, f"gh not available ({exc})"
if proc.returncode != 0:
err = (proc.stderr or proc.stdout or "").strip()
return False, err or f"gh repo edit exit {proc.returncode}"
return True, f"GitHub About set on {slug}"
def render_metrics_block(owner: str, repo: str, zip_prefix: str = "") -> str:
# Per-OS latest badges need an exact asset name. Title zips are
# `{prefix}-{version}-{platform}.zip`, so globs always show 0 on shields.io.
# Keep all-time / latest totals + version; zip_prefix is unused here.
del zip_prefix
owner = sanitize_github_name(owner) or DEFAULT_GITHUB_OWNER
repo = sanitize_github_name(repo) or "repo"
base = f"https://github.com/{owner}/{repo}"
img = "https://img.shields.io/github"
lines = [
METRICS_BEGIN,
f"[![GitHub downloads (all assets, all releases)]({img}/downloads/{owner}/{repo}/total)]({base}/releases)",
f"[![GitHub downloads (latest release)]({img}/downloads/{owner}/{repo}/latest/total)]({base}/releases/latest)",
f"[![GitHub release]({img}/v/release/{owner}/{repo})]({base}/releases/latest)",
METRICS_END,
]
return "\n".join(lines)
def render_boxart_block(game_name: str) -> str:
alt = (game_name or "Game").strip() or "Game"
alt = alt.replace('"', "'")
return "\n".join(
[
BOXART_BEGIN,
'<p align="center">',
f' <img src="{BOXART_REL}" alt="{alt} box art" width="280">',
"</p>",
BOXART_END,
]
)
def render_launcher_block() -> str:
shots = (
"https://raw.githubusercontent.com/TechnicallyComputers/"
"RetComM-Launcher/main/docs/screenshots"
)
return "\n".join(
[
LAUNCHER_BEGIN,
"## RetComM Launcher",
"",
"You can run this title **standalone** (release zip + the built-in recomp-ui",
"Generate & Build flow), or manage installs, updates, ROM/BIOS wiring, and queued",
"builds more intuitively with",
f"**[RetComM Launcher]({LAUNCHER_REPO})** —",
"the Retro Compilation Manager hub for self-compiling recomps.",
"",
f"[Downloads]({LAUNCHER_REPO}/releases) ·",
f"[Full README & features]({LAUNCHER_REPO}#readme)",
"",
"<p align=\"center\">",
f' <img src="{shots}/hub-and-game-launcher.png" alt="RetComM hub with a background build, next to a title’s recomp-ui launcher" width="720">',
"</p>",
"",
"<p align=\"center\">",
f' <img src="{shots}/queue-and-background-build.png" alt="Background cmake build with titles queued" width="720">',
"</p>",
"",
"RetComM checks for updates, rebuilds with existing build data when possible,",
"shares the portable toolchain used by per-title launchers, and automates",
"BIOS/ROM/save plumbing so you are not stuck repeating each game’s wizard by hand.",
LAUNCHER_END,
]
)
def render_raid_block() -> str:
return "\n".join(
[
RAID_BEGIN,
"---",
"",
"<p align=\"center\">",
" <sub><b>R.A.I.D. — Retro AI Development</b> · a Discord for AI-assisted retro reverse-engineering, decomp &amp; recomp</sub>",
"</p>",
"",
"<p align=\"center\">",
' <a href="https://discord.gg/Ad9BwSzctP"><img src=".github/raid-discord.png" alt="Join the Retro AI Development (R.A.I.D.) Discord" width="200"></a>',
"</p>",
RAID_END,
]
)
def readme_has_metrics(text: str) -> bool:
return "img.shields.io/github/downloads/" in text
def readme_has_boxart(text: str) -> bool:
return BOXART_BEGIN in text or BOXART_REL in text
def boxart_png_present(root: Path) -> bool:
return (root / BOXART_REL).is_file()
def readme_has_launcher(text: str) -> bool:
return "TechnicallyComputers/RetComM-Launcher" in text
def readme_has_raid(text: str) -> bool:
return "discord.gg/Ad9BwSzctP" in text or "R.A.I.D. — Retro AI Development" in text
def _apply_boxart(text: str, boxart_block: str) -> str:
boxart_block = boxart_block.strip() + "\n"
if _BOXART_RE.search(text):
return _BOXART_RE.sub(boxart_block, text, count=1)
m = _METRICS_RE.search(text)
if m:
rest = text[m.end() :].lstrip("\n")
return text[: m.end()] + "\n" + boxart_block + "\n" + rest
hm = re.match(r"(#[^\n]+\n)", text)
if hm:
rest = text[hm.end() :].lstrip("\n")
return text[: hm.end()] + "\n" + boxart_block + "\n" + rest
return boxart_block + "\n" + text
def upsert_boxart_block(text: str, game_name: str) -> str:
if not text.endswith("\n"):
text += "\n"
return _apply_boxart(text, render_boxart_block(game_name))
def inject_readme_boxart(readme: Path, game_name: str) -> bool:
"""Insert/replace the README boxart block if ``boxart.png`` exists."""
readme = Path(readme)
if not readme.is_file() or not boxart_png_present(readme.parent):
return False
old = readme.read_text(encoding="utf-8", errors="replace")
new = upsert_boxart_block(old, game_name)
if new == old:
return False
readme.write_text(new, encoding="utf-8")
return True
def upsert_readme_blocks(
text: str,
metrics: str,
launcher: str,
raid: str | None = None,
boxart: str | None = None,
) -> str:
"""Insert or replace marked metrics, boxart, launcher, and RAID footer; keep custom body."""
if not text.endswith("\n"):
text += "\n"
metrics_block = metrics.strip() + "\n"
launcher_block = launcher.strip() + "\n"
raid_block = (raid if raid is not None else render_raid_block()).strip() + "\n"
if _METRICS_RE.search(text):
text = _METRICS_RE.sub(metrics_block, text, count=1)
elif not readme_has_metrics(text):
m = re.match(r"(#[^\n]+\n)", text)
if m:
text = (
text[: m.end()]
+ "\n"
+ metrics_block
+ "\n"
+ text[m.end() :].lstrip("\n")
)
else:
text = metrics_block + "\n" + text
if boxart:
text = _apply_boxart(text, boxart)
if _LAUNCHER_RE.search(text):
text = _LAUNCHER_RE.sub(launcher_block, text, count=1)
elif _LEGACY_LAUNCHER_RE.search(text):
text = _LEGACY_LAUNCHER_RE.sub(launcher_block + "\n", text, count=1)
elif not readme_has_launcher(text):
insert_at = None
for heading in ("## Legal", "## Quick start", "## Layout", "## Symbols"):
idx = text.find("\n" + heading)
if idx >= 0:
insert_at = idx + 1
break
if insert_at is None:
text = text.rstrip() + "\n\n" + launcher_block
else:
text = text[:insert_at] + launcher_block + "\n" + text[insert_at:]
if _RAID_RE.search(text):
text = _RAID_RE.sub(raid_block, text, count=1)
elif _LEGACY_RAID_RE.search(text):
text = _LEGACY_RAID_RE.sub("\n" + raid_block, text, count=1)
elif not readme_has_raid(text):
text = text.rstrip() + "\n\n" + raid_block
return text if text.endswith("\n") else text + "\n"
+71 -6
View File
@@ -22,6 +22,8 @@ param(
[string]$BootExe = "SLUS_01234",
[int]$Players = 0,
[string]$ZipPrefix = "",
[string]$GithubOwner = "",
[string]$GithubRepo = "",
[switch]$EnableRecompUi,
[switch]$NoRecompUi,
[switch]$EnableNetplay,
@@ -142,6 +144,24 @@ if (-not $ZipPrefix) {
}
}
if (-not $GithubOwner) {
if ($interactive) {
$GithubOwner = Prompt-Line "GitHub owner / org (README download badges)" "TechnicallyComputers"
} else {
$GithubOwner = "TechnicallyComputers"
}
}
$derivedRepo = (python -c "from fill_tokens import sanitize_github_name; import sys; print(sanitize_github_name(sys.argv[1]))" $Name).Trim()
if (-not $GithubRepo) {
if ($interactive) {
$GithubRepo = Prompt-Line "GitHub repo name (README download badges)" $derivedRepo
} else {
$GithubRepo = $derivedRepo
}
}
$GithubOwner = (python -c "from fill_tokens import sanitize_github_name; import sys; print(sanitize_github_name(sys.argv[1]))" $GithubOwner).Trim()
$GithubRepo = (python -c "from fill_tokens import sanitize_github_name; import sys; print(sanitize_github_name(sys.argv[1]))" $GithubRepo).Trim()
if (-not $PSBoundParameters.ContainsKey("Description")) {
if ($interactive) { $Description = Prompt-Line "Short game description (optional)" "" }
}
@@ -246,8 +266,13 @@ $PublisherDisp = if ($Publisher) { $Publisher } else { "-" }
$YearDisp = if ($Year) { $Year } else { "-" }
$DescriptionMd = if ($Description) { $Description } else { "_Add a short pitch in catalog_identity.json / README._" }
$Root = Join-Path (Resolve-Path $Dir) $Name
# Folder = GitHub/catalog install_dir slug (hyphenated), not display $Name with spaces.
$InstallDirName = $GithubRepo
$Root = Join-Path (Resolve-Path $Dir) $InstallDirName
if (Test-Path $Root) { throw "Target already exists: $Root" }
if ($InstallDirName -ne $Name) {
Write-Host "note: project folder is '$InstallDirName' (catalog/GitHub slug; display name stays '$Name')"
}
$NetplayBlockFile = Join-Path $env:TEMP "psxrecomp_netplay_block.cmake"
$WizardBlockFile = Join-Path $env:TEMP "psxrecomp_wizard_block.cmake"
@@ -316,6 +341,8 @@ function Fill-Template([string]$src, [string]$dst) {
--set "ENV_PREFIX=$EnvPrefix" `
--set "EXE_BASENAME=$ExeBasename" `
--set "ZIP_PREFIX=$ZipPrefix" `
--set "GITHUB_OWNER=$GithubOwner" `
--set "GITHUB_REPO=$GithubRepo" `
--set "DISC_HINT=$DiscHint" `
--set "GAME_TITLE=$WindowTitle" `
--set "PLAYERS=$Players" `
@@ -339,6 +366,7 @@ Write-Host "== New Project Layout =="
Write-Host " repo: $Root"
Write-Host " disc: $Disc"
Write-Host " zip prefix: $ZipPrefix"
Write-Host " gh slug: $GithubOwner/$GithubRepo"
Write-Host " players: $Players"
Write-Host " recomp-ui: $useRecompUi"
Write-Host " wizard: $useWizard"
@@ -564,6 +592,32 @@ print(m.group(1) if m else '')
Write-Warning "-Disc is not a .cue; skipped probe autofill."
}
if ($doBoxart) {
Write-Host "== Fetching libretro boxart =="
$cueHint = if ($DiscBasename) { $DiscBasename } else { $GameName }
$prevPy = $env:PYTHONPATH
try {
python $FetchBoxartPy `
--out (Join-Path $Root "launcher_assets\img\boxart.tga") `
--cue-stem $cueHint `
--display-name $GameName
if ($LASTEXITCODE -eq 0) {
$HasBoxart = $true
Set-Content -Encoding UTF8 -Path $BoxartBlockFile -Value ' LAUNCHER_BOXART "${CMAKE_CURRENT_SOURCE_DIR}/launcher_assets/img/boxart.tga"'
Fill-Template (Join-Path $TemplateDir "CMakeLists.txt.in") (Join-Path $Root "CMakeLists.txt")
Write-Host " wired LAUNCHER_BOXART in CMakeLists.txt"
$env:PYTHONPATH = $ScriptDir
python -c "from pathlib import Path; import sys; from project_studio.readme_metrics import inject_readme_boxart; inject_readme_boxart(Path(sys.argv[1]), sys.argv[2])" (Join-Path $Root "README.md") $GameName
Write-Host " injected boxart into README.md"
} else {
Write-Warning "boxart fetch failed — leave LAUNCHER_BOXART commented."
}
} catch {
Write-Warning "boxart fetch failed — leave LAUNCHER_BOXART commented."
} finally {
if ($null -eq $prevPy) { Remove-Item Env:PYTHONPATH -ErrorAction SilentlyContinue } else { $env:PYTHONPATH = $prevPy }
}
}
Write-Host "== Sync symbols header =="
Push-Location $Root
@@ -571,8 +625,8 @@ try { python tools/sync_symbols.py --game $GameName } catch { Write-Warning "syn
Pop-Location
git add CMakeLists.txt game.toml codegen_setup.c codegen_setup.h .gitignore VERSION README.md seeds scripts tools mods framework_pins.txt symbols.toml psx_symbols.h 2>$null
if (Test-Path (Join-Path $Root ".github\workflows")) {
git add .github/workflows 2>$null
if (Test-Path (Join-Path $Root ".github")) {
git add .github 2>$null
}
if (Test-Path (Join-Path $Root "catalog_identity.json")) {
git add catalog_identity.json 2>$null
@@ -581,7 +635,7 @@ if (Test-Path (Join-Path $Root "disc_probe.json")) {
git add disc_probe.json 2>$null
}
if ($HasBoxart) {
git add launcher_assets/img/boxart.tga launcher_assets/img/BOXART_SOURCE.txt 2>$null
git add launcher_assets/img/boxart.tga launcher_assets/img/boxart.png launcher_assets/img/BOXART_SOURCE.txt 2>$null
}
$committed = $false
try {
@@ -606,15 +660,18 @@ if ($createGithub) {
"internal" { "--internal" }
default { "--private" }
}
$ghSlug = "$GithubOwner/$GithubRepo"
$ghDesc = (python -c "from fill_tokens import GITHUB_ABOUT_DESCRIPTION; print(GITHUB_ABOUT_DESCRIPTION)").Trim()
$ghHome = (python -c "from fill_tokens import GITHUB_ABOUT_HOMEPAGE; print(GITHUB_ABOUT_HOMEPAGE)").Trim()
try {
gh repo create $Name $vis --source=$Root --remote=origin
gh repo create $ghSlug $vis --source=$Root --remote=origin --description "$ghDesc" --homepage "$ghHome"
Write-Host " created origin ($GithubVisibility); push deferred until end"
$githubCreated = $true
} catch {
Write-Warning "gh repo create failed -- if the repo already exists, add origin and push at the end."
if (-not (git remote get-url origin 2>$null)) {
try {
$ghUrl = (gh repo view $Name --json url -q .url 2>$null)
$ghUrl = (gh repo view $ghSlug --json url -q .url 2>$null)
if ($ghUrl) {
git remote add origin $ghUrl
Write-Host " attached existing origin -> $ghUrl"
@@ -625,6 +682,14 @@ if ($createGithub) {
$githubCreated = $true
}
}
if ($githubCreated) {
try {
gh repo edit $ghSlug --description "$ghDesc" --homepage "$ghHome"
Write-Host " GitHub About: R.A.I.D. description + Discord homepage"
} catch {
Write-Warning "gh repo edit (About) failed — set description/homepage by hand."
}
}
}
}
+67 -10
View File
@@ -9,11 +9,13 @@
# --stage-disc Copy full cue+bins into repo disc/ (large; optional)
# --no-stage-disc Default: probe in place, extract boot EXE only
# --psxrecomp-ref / --recomp-ui-ref / --recomp-net-ref / URLs
# --github-owner <org> README download-badge owner (default TechnicallyComputers)
# --github-repo <name> README download-badge repo (default project name)
#
# Everything else is prompted on a TTY (or passed via flags / --yes defaults):
# name, players (1–8, default 2), zip-prefix, marketing blurb, recomp-ui,
# wizard, netplay, lobby URL (if netplay), CI workflow, fetch-boxart,
# generate, build (if generate), GitHub repo (gh)
# name, players (1–8, default 2), zip-prefix, github owner/repo (README badges),
# marketing blurb, recomp-ui, wizard, netplay, lobby URL (if netplay), CI
# workflow, fetch-boxart, generate, build (if generate), GitHub repo (gh)
#
# Non-interactive: --yes (or PSXRECOMP_SETUP_YES=1) requires --name and --disc;
# boolean opts stay off unless explicitly flagged; lobby URL defaults to
@@ -55,6 +57,8 @@ DO_GENERATE=""
DO_BUILD=""
CREATE_GITHUB=""
GITHUB_VISIBILITY="private"
GITHUB_OWNER=""
GITHUB_REPO=""
BIOS_PATH=""
LOBBY_URL=""
DESCRIPTION=""
@@ -183,6 +187,8 @@ while [ $# -gt 0 ]; do
--create-github) CREATE_GITHUB=1; SET_GITHUB=1; shift ;;
--no-github) CREATE_GITHUB=0; SET_GITHUB=1; shift ;;
--github-visibility) GITHUB_VISIBILITY=$2; shift 2 ;;
--github-owner) GITHUB_OWNER=$2; shift 2 ;;
--github-repo) GITHUB_REPO=$2; shift 2 ;;
--description) DESCRIPTION=$2; SET_DESCRIPTION=1; shift 2 ;;
--publisher) PUBLISHER=$2; SET_PUBLISHER=1; shift 2 ;;
--year) YEAR=$2; SET_YEAR=1; shift 2 ;;
@@ -264,6 +270,28 @@ if [ -z "$ZIP_PREFIX" ]; then
fi
fi
if [ -z "$GITHUB_OWNER" ]; then
if [ "$YES_MODE" -eq 1 ] || ! is_tty; then
GITHUB_OWNER=TechnicallyComputers
else
prompt_line "GitHub owner / org (README download badges)" GITHUB_OWNER \
"TechnicallyComputers"
fi
fi
if [ -z "$GITHUB_REPO" ]; then
_derived_repo=$(PYTHONPATH="$SCRIPT_DIR${PYTHONPATH:+:$PYTHONPATH}" \
python3 -c 'from fill_tokens import sanitize_github_name; import sys; print(sanitize_github_name(sys.argv[1]))' "$NAME")
if [ "$YES_MODE" -eq 1 ] || ! is_tty; then
GITHUB_REPO=$_derived_repo
else
prompt_line "GitHub repo name (README download badges)" GITHUB_REPO "$_derived_repo"
fi
fi
GITHUB_OWNER=$(PYTHONPATH="$SCRIPT_DIR${PYTHONPATH:+:$PYTHONPATH}" \
python3 -c 'from fill_tokens import sanitize_github_name; import sys; print(sanitize_github_name(sys.argv[1]))' "$GITHUB_OWNER")
GITHUB_REPO=$(PYTHONPATH="$SCRIPT_DIR${PYTHONPATH:+:$PYTHONPATH}" \
python3 -c 'from fill_tokens import sanitize_github_name; import sys; print(sanitize_github_name(sys.argv[1]))' "$GITHUB_REPO")
# Optional marketing (catalog_identity.json + README)
if [ "$SET_DESCRIPTION" -eq 0 ]; then
if [ "$YES_MODE" -eq 1 ] || ! is_tty; then
@@ -448,11 +476,16 @@ if [ -z "$DESCRIPTION_MD" ]; then
DESCRIPTION_MD="_Add a short pitch in catalog_identity.json / README._"
fi
ROOT=$(CDPATH= cd -- "$PARENT" && pwd)/"$NAME"
# Folder = GitHub/catalog install_dir slug (hyphenated), not display $NAME with spaces.
INSTALL_DIR_NAME="$GITHUB_REPO"
ROOT=$(CDPATH= cd -- "$PARENT" && pwd)/"$INSTALL_DIR_NAME"
if [ -e "$ROOT" ]; then
echo "error: target already exists: $ROOT" >&2
exit 1
fi
if [ "$INSTALL_DIR_NAME" != "$NAME" ]; then
echo "note: project folder is '$INSTALL_DIR_NAME' (catalog/GitHub slug; display name stays '$NAME')" >&2
fi
# --- CMake block files -----------------------------------------------------
NETPLAY_BLOCK_FILE=$(mktemp)
@@ -527,6 +560,8 @@ fill_template() {
--set "ENV_PREFIX=$ENV_PREFIX" \
--set "EXE_BASENAME=$EXE_BASENAME" \
--set "ZIP_PREFIX=$ZIP_PREFIX" \
--set "GITHUB_OWNER=$GITHUB_OWNER" \
--set "GITHUB_REPO=$GITHUB_REPO" \
--set "DISC_HINT=$DISC_HINT" \
--set "GAME_TITLE=$WINDOW_TITLE" \
--set "PLAYERS=$PLAYERS" \
@@ -550,6 +585,7 @@ echo "== New Project Layout =="
echo " repo: $ROOT"
echo " disc: $DISC"
echo " zip prefix: $ZIP_PREFIX"
echo " gh slug: $GITHUB_OWNER/$GITHUB_REPO"
echo " players: $PLAYERS"
echo " recomp-ui: $ENABLE_RECOMP_UI"
echo " wizard: $ENABLE_WIZARD"
@@ -578,9 +614,12 @@ fill_template "$TEMPLATE_DIR/VERSION.in" "$ROOT/VERSION"
fill_template "$TEMPLATE_DIR/README.md.in" "$ROOT/README.md"
fill_template "$TEMPLATE_DIR/symbols.toml.in" "$ROOT/symbols.toml"
mkdir -p "$ROOT/seeds" "$ROOT/launcher_assets/img" "$ROOT/scripts" "$ROOT/tools" \
"$ROOT/mods/preloaded/packages" "$ROOT/assets"
"$ROOT/mods/preloaded/packages" "$ROOT/assets" "$ROOT/.github"
cp "$SCRIPT_DIR/sync_symbols.py" "$ROOT/tools/sync_symbols.py"
chmod +x "$ROOT/tools/sync_symbols.py"
if [ -f "$TEMPLATE_DIR/raid-discord.png" ]; then
cp "$TEMPLATE_DIR/raid-discord.png" "$ROOT/.github/raid-discord.png"
fi
# Empty mod catalog tree (runtime copies mods/preloaded → beside the exe as mods/).
cat > "$ROOT/mods/preloaded/README.md" <<'EOF'
# Preloaded mods
@@ -801,6 +840,11 @@ if [ "$FETCH_BOXART_FLAG" -eq 1 ]; then
>"$BOXART_BLOCK_FILE"
fill_template "$TEMPLATE_DIR/CMakeLists.txt.in" "$ROOT/CMakeLists.txt"
echo " wired LAUNCHER_BOXART in CMakeLists.txt"
PYTHONPATH="$SCRIPT_DIR${PYTHONPATH:+:$PYTHONPATH}" python3 -c \
'from pathlib import Path; import sys; from project_studio.readme_metrics import inject_readme_boxart; inject_readme_boxart(Path(sys.argv[1]), sys.argv[2])' \
"$ROOT/README.md" "$GAME_NAME" \
&& echo " injected boxart into README.md" \
|| echo "warning: README boxart inject failed" >&2
else
echo "warning: boxart fetch failed — leave LAUNCHER_BOXART commented." >&2
fi
@@ -813,8 +857,8 @@ echo "== Sync symbols header =="
)
git add CMakeLists.txt game.toml codegen_setup.c codegen_setup.h .gitignore VERSION README.md seeds scripts tools mods framework_pins.txt symbols.toml psx_symbols.h || true
if [ -d "$ROOT/.github/workflows" ]; then
git add .github/workflows || true
if [ -d "$ROOT/.github" ]; then
git add .github || true
fi
if [ -f "$ROOT/catalog_identity.json" ]; then
git add catalog_identity.json || true
@@ -823,7 +867,7 @@ if [ -f "$ROOT/disc_probe.json" ]; then
git add disc_probe.json || true
fi
if [ "$HAS_BOXART" -eq 1 ]; then
git add launcher_assets/img/boxart.tga launcher_assets/img/BOXART_SOURCE.txt || true
git add launcher_assets/img/boxart.tga launcher_assets/img/boxart.png launcher_assets/img/BOXART_SOURCE.txt || true
fi
COMMITTED=0
if git -c user.email=setup@localhost -c user.name=setup \
@@ -847,14 +891,20 @@ if [ "${CREATE_GITHUB:-0}" -eq 1 ]; then
VIS_FLAG="--private"
[ "$GITHUB_VISIBILITY" = "public" ] && VIS_FLAG="--public"
[ "$GITHUB_VISIBILITY" = "internal" ] && VIS_FLAG="--internal"
if gh repo create "$NAME" $VIS_FLAG --source="$ROOT" --remote=origin; then
GH_SLUG="$GITHUB_OWNER/$GITHUB_REPO"
GH_DESC=$(PYTHONPATH="$SCRIPT_DIR${PYTHONPATH:+:$PYTHONPATH}" \
python3 -c 'from fill_tokens import GITHUB_ABOUT_DESCRIPTION; print(GITHUB_ABOUT_DESCRIPTION)')
GH_HOME=$(PYTHONPATH="$SCRIPT_DIR${PYTHONPATH:+:$PYTHONPATH}" \
python3 -c 'from fill_tokens import GITHUB_ABOUT_HOMEPAGE; print(GITHUB_ABOUT_HOMEPAGE)')
if gh repo create "$GH_SLUG" $VIS_FLAG --source="$ROOT" --remote=origin \
--description "$GH_DESC" --homepage "$GH_HOME"; then
echo " created origin ($GITHUB_VISIBILITY); push deferred until end"
GITHUB_CREATED=1
else
echo "warning: gh repo create failed — if the repo already exists," >&2
echo " add origin and push manually at the end." >&2
if ! git remote get-url origin >/dev/null 2>&1; then
_gh_url=$(gh repo view "$NAME" --json url -q .url 2>/dev/null || true)
_gh_url=$(gh repo view "$GH_SLUG" --json url -q .url 2>/dev/null || true)
if [ -n "$_gh_url" ]; then
git remote add origin "$_gh_url"
echo " attached existing origin → $_gh_url"
@@ -864,6 +914,13 @@ if [ "${CREATE_GITHUB:-0}" -eq 1 ]; then
GITHUB_CREATED=1
fi
fi
if [ "$GITHUB_CREATED" -eq 1 ]; then
if gh repo edit "$GH_SLUG" --description "$GH_DESC" --homepage "$GH_HOME"; then
echo " GitHub About: R.A.I.D. description + Discord homepage"
else
echo "warning: gh repo edit (About) failed — set description/homepage by hand." >&2
fi
fi
fi
fi
@@ -1,5 +1,11 @@
# @WINDOW_TITLE@
<!-- retcomm-readme-metrics -->
[![GitHub downloads (all assets, all releases)](https://img.shields.io/github/downloads/@GITHUB_OWNER@/@GITHUB_REPO@/total)](https://github.com/@GITHUB_OWNER@/@GITHUB_REPO@/releases)
[![GitHub downloads (latest release)](https://img.shields.io/github/downloads/@GITHUB_OWNER@/@GITHUB_REPO@/latest/total)](https://github.com/@GITHUB_OWNER@/@GITHUB_REPO@/releases/latest)
[![GitHub release](https://img.shields.io/github/v/release/@GITHUB_OWNER@/@GITHUB_REPO@)](https://github.com/@GITHUB_OWNER@/@GITHUB_REPO@/releases/latest)
<!-- /retcomm-readme-metrics -->
Static recompilation of **@GAME_NAME@** built on
[psxrecomp](https://github.com/mstan/psxrecomp) and
[recomp-ui](https://github.com/mstan/recomp-ui).
@@ -16,6 +22,31 @@ Static recompilation of **@GAME_NAME@** built on
Scaffolded with the New Project Layout. See
`psxrecomp/docs/GAME_PROJECT_SETUP.md` for the full flow.
<!-- retcomm-readme-launcher -->
## RetComM Launcher
You can run this title **standalone** (release zip + the built-in recomp-ui
Generate & Build flow), or manage installs, updates, ROM/BIOS wiring, and queued
builds more intuitively with
**[RetComM Launcher](https://github.com/TechnicallyComputers/RetComM-Launcher)** —
the Retro Compilation Manager hub for self-compiling recomps.
[Downloads](https://github.com/TechnicallyComputers/RetComM-Launcher/releases) ·
[Full README & features](https://github.com/TechnicallyComputers/RetComM-Launcher#readme)
<p align="center">
<img src="https://raw.githubusercontent.com/TechnicallyComputers/RetComM-Launcher/main/docs/screenshots/hub-and-game-launcher.png" alt="RetComM hub with a background build, next to a title’s recomp-ui launcher" width="720">
</p>
<p align="center">
<img src="https://raw.githubusercontent.com/TechnicallyComputers/RetComM-Launcher/main/docs/screenshots/queue-and-background-build.png" alt="Background cmake build with titles queued" width="720">
</p>
RetComM checks for updates, rebuilds with existing build data when possible,
shares the portable toolchain used by per-title launchers, and automates
BIOS/ROM/save plumbing so you are not stuck repeating each game’s wizard by hand.
<!-- /retcomm-readme-launcher -->
## Legal
You must own the original game. Disc images under `disc/` are gitignored and
@@ -39,12 +70,6 @@ cmake -S . -B build-release -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build build-release --target psx-runtime
```
`psxrecomp/runtime/runtime.cmake` must exist before configuring. If CMake says
the runtime is missing, the framework submodule is not checked out; run
`git submodule update --init --recursive`, or pass
`-DPSXRECOMP_ROOT=<path-to-psxrecomp>` when intentionally building against a
separate framework checkout.
Zip prefix for CI artifacts: `@ZIP_PREFIX@`.
## Symbols
@@ -59,3 +84,15 @@ are authoritative. `framework_pins.txt` is an optional scaffold snapshot;
release CI logs SHAs with `record_pins.sh` but builds whatever the gitlinks
resolve to. Bump submodules deliberately — do not float on `main`/`master`
in release CI.
<!-- retcomm-readme-raid -->
---
<p align="center">
<sub><b>R.A.I.D. — Retro AI Development</b> · a Discord for AI-assisted retro reverse-engineering, decomp &amp; recomp</sub>
</p>
<p align="center">
<a href="https://discord.gg/Ad9BwSzctP"><img src=".github/raid-discord.png" alt="Join the Retro AI Development (R.A.I.D.) Discord" width="200"></a>
</p>
<!-- /retcomm-readme-raid -->
Binary file not shown.

After

Width:  |  Height:  |  Size: 560 KiB