diff --git a/docs/HOST_OPTIMIZATION_CONTRACT.md b/docs/HOST_OPTIMIZATION_CONTRACT.md new file mode 100644 index 00000000..29146544 --- /dev/null +++ b/docs/HOST_OPTIMIZATION_CONTRACT.md @@ -0,0 +1,203 @@ +# Host optimization contract + +This document is the acceptance contract for changes intended to make +psxrecomp run faster on the host. It applies to generated code, the runtime, +the flat CPS trampoline, overlay dispatch and validation, graphics and audio +support code, and host-side caches or fast paths. + +Correctness fixes and deliberate guest-visible enhancements still need focused +verification, but the 5% performance retention gate below applies specifically +to optimizations. + +## Non-negotiable rules + +1. Measure on a quiet host with zero compiler processes. Finish all AOT, + overlay, shader, and helper compilation before the timed region. Confirm + that no compiler, linker, build driver, or autocompile worker is alive while + sampling. A run in which one appears is contaminated and is discarded, not + explained away. +2. Interleave A and B. Never measure all baseline runs and then all candidate + runs. The minimum acceptable order for three pairs is `A1 B1 A2 B2 A3 B3`; + reversing the first member of alternate campaigns is encouraged. Keep the + same binary and use a forced runtime selector when practical. +3. Report every raw run, the min-of-N for A and B, the percentage change + between those minima, both medians, and the percentage change between the + medians. N is at least three. State whether lower or higher is better. +4. Discard a pair when either member is contaminated. Reasons include a + compiler process, overlay compilation, shader-cache construction, a changed + power or thermal state, unexpected foreground load, a different overlay + population, or a different pacing owner. Record the discarded pair and its + reason in the experiment ledger. +5. A candidate that adds complexity, persistent state, generated-code ABI, + configuration or debug ABI, or another maintained fast path is retained + only when the agreed primary metric improves by at least 5%. A pure + simplification may be retained at flat cost, but its measured numbers must + still be recorded. Correctness fixes are judged on correctness, not this + speed threshold; do not disguise an optimization as a correctness fix. +6. With the optimization forced on for every eligible operation, guest-visible + state must be byte-identical to the faithful path at deterministic + checkpoints. An automatic fallback is not proof: it can hide the exact case + the fast path mishandles. +7. The faithful reference computation remains linked and selectable. A fast + path is an implementation of that same computation, not a replacement model + that merely looks plausible for one title. +8. Do not copy another console project's opcode handlers. It is valid to reuse + its experimental process, evidence format, test structure, or review + checklist. Implement R3000A behavior from PlayStation evidence and + psxrecomp's own faithful path. + +## Preparing comparable psxrecomp runs + +Prefer one executable containing both paths and an explicit selector with +three states: faithful, candidate forced on, and normal policy. Record the +selector and assert in telemetry that the requested state took effect. If two +binaries are unavoidable, build both before the campaign and record their +commits, toolchains, options, and artifact hashes. + +Static recompilation makes build quietness part of the benchmark definition. +Pre-generate the game and BIOS images, warm every overlay needed by the +workload, and prevent runtime autocompile during the measured interval. Record +the static image identity and the exact overlay inventory. A benchmark is not +comparable if A interprets an overlay while B uses a compiled shard. + +Content-verified overlay variants require an additional check. A and B must +use the same captured bytes, identity ranges, flavor, cache generation, and +selected variant at each checkpoint. Pin and report the overlay flavor. A fast +identity check may avoid a CRC or range scan, but the faithful verifier must +remain callable and both must select the same content-verified variant. + +The flat CPS trampoline is part of observable execution semantics. Compare the +published continuation PC, dispatch result, exception redirect, dispatch depth, +and same-site resume consumption at block and function boundaries. A host +function returning with `cpu->pc == 0` is not equivalent to tail-publishing the +next guest PC, even if common images never cross that boundary. + +Use a deterministic workload with fixed inputs and a fixed capture or replay +when possible. Keep renderer, audio backend, window state, display refresh, +vsync/pacer ownership, CPU overclock, CRTC multiplier, mods, RAM size, and +instrumentation identical. Warm caches outside the timed region unless cold +behavior is the stated metric. + +## Correctness proof + +The forced-candidate run must exercise the candidate, and its checkpoints must +match a run forced through the faithful reference. Hash or serialize all state +that can affect later guest execution, including: + +- GPR, HI/LO, PC, COP0, GTE registers and completion timestamps; +- main RAM, scratchpad, VRAM, SPU RAM, and relevant device registers; +- the R3000A cycle counter, deferred cycle charges, event deadlines, IRQ state, + load-delay state, and pending exception continuation; +- dispatch and overlay identity state that changes which guest bytes execute; +- deterministic mod-owned guest state and savestate-visible sections. + +Host-only counters may be excluded only when they cannot influence selection, +timing, invalidation, or later guest state. Document every exclusion. + +Compare at more than the final frame. Include checkpoints before the candidate +first fires, immediately after it fires, at overlay or exception boundaries it +touches, and after a long soak. For caches and memos, force hits, misses, +invalidation, wrap or saturation, and savestate/rewind restore. For generated +code, cover the game emitter, BIOS full-function emitter, compiled overlays, +static overlay bakes, and the dirty interpreter wherever the contract crosses +execution tiers. + +For cycle-affecting work, use `tools/cycle_testrom/measure.py` as the existing +measurement seam. Run the same test-ROM anchors against the native compiled +path, `PSX_FORCE_INTERP=1`, and the Beetle oracle as applicable. The script's +per-iteration deltas establish guest-cycle correctness; they do not replace a +separate interleaved wall-time or throughput campaign for host performance. +The faithful per-instruction R3000A cycle model remains authoritative: an +optimization may reduce host work but may not remove base charges, wait states, +interlocks, GTE completion timing, I-cache behavior, or peripheral deadlines. + +## Performance campaign and evidence + +Choose the primary metric before running the campaign. Frame or workload time +is preferred when the work is bounded; throughput is acceptable for a fixed +steady-state loop. Report secondary metrics, but do not change the retention +decision after seeing the results. + +The evidence block for every candidate contains: + +```text +Candidate: +Commits/artifacts: A= B= selector= +Workload: +Primary metric: +Run order: A1 B1 A2 B2 A3 B3 +Raw A: , , +Raw B: , , +Min-of-3: A= B= change= +Medians: A= B= change= +Correctness: faithful= forced-on= result= +Contamination: +Complexity/ABI: +Decision: ACCEPT/REJECT - +``` + +Worked example, using workload time in milliseconds where lower is better: + +```text +Run order: A1 B1 A2 B2 A3 B3 +Raw A: 12.000, 11.000, 13.000 ms +Raw B: 9.744, 9.493, 10.100 ms +Min-of-3: A=11.000 B=9.493; improves 13.7% +Medians: A=12.000 B=9.744; improves 18.8% +Correctness: faithful=7a... forced-on=7a...; byte-identical +Complexity/ABI: two per-variant words plus one generated helper call +Decision: ACCEPT - both summaries exceed the 5% retention gate +``` + +Do not report only a profiler share, a best screenshot, an FPS range, or a +sequential before/after observation. Those are useful for finding candidates, +not for accepting them. + +## Experiment ledger + +Every investigated candidate gets a ledger row, including rejected, +superseded, reverted, and flat candidates. The ledger is append-only evidence, +not a highlights page. Keep it beside the optimization or campaign notes and +include links to raw data when available. + +Required fields are: + +| Field | Required contents | +|---|---| +| ID and date | Stable candidate ID and campaign date | +| Change | Commit(s), artifact hashes, selector, and concise hypothesis | +| Scope | Workload, renderer/audio backend, host, toolchain, config, overlay inventory and flavor | +| Correctness | Faithful and forced-on checkpoint hashes; exact exclusions | +| Runs | Interleaved order, all raw accepted runs, and every discarded pair with reason | +| Summary | A/B min-of-N and medians with percentage changes | +| Cost | Added code, state, ABI, configuration, maintenance burden, or simplification | +| Verdict | Accepted or rejected, threshold used, and reason | + +Accepted and rejected results are equally visible. A compact ledger index may +look like this, with each row linking to its full evidence block: + +| ID | Min-of-N change | Median change | State result | Cost | Verdict | +|---|---:|---:|---|---|---| +| EX-A | improves 13.7% | improves 18.8% | byte-identical | two per-variant words and one helper call | ACCEPT - clears 5% | +| EX-R | improves 3.0% | improves 1.0% | byte-identical | new cache and invalidation path | REJECT - below 5% | + +Use an explicit rejected entry even when the candidate was reverted before a +formal campaign. Record the numbers that motivated the experiment and state +which required evidence is missing. Do not silently delete failed ideas: the +ledger prevents the same attractive but invalid optimization from being tried +again without new evidence. + +## Review checklist + +- Host was quiet and had zero compiler processes during every retained run. +- A/B was interleaved; contaminated pairs were discarded and recorded. +- Raw runs, min-of-N, and medians are present. +- The forced candidate and faithful reference produced byte-identical + guest-visible state. +- The faithful path remains linked and selectable. +- Overlay identities and execution tiers were held constant and reported. +- Cycle-model changes were checked through the cycle-test-ROM seam. +- Added complexity or ABI clears 5%; flat pure simplifications have numbers. +- Accepted and rejected candidates are both in the experiment ledger. +- R3000A implementation was derived independently; no other console's opcode + handlers were copied. diff --git a/docs/UPSTREAM_2026-08-20.md b/docs/UPSTREAM_2026-08-20.md new file mode 100644 index 00000000..d69b5cca --- /dev/null +++ b/docs/UPSTREAM_2026-08-20.md @@ -0,0 +1,262 @@ +# Upstream contribution audit — 2026-08-20 + +## Summary + +This worktree is a cross-cutting psxrecomp framework hardening and host-performance pass discovered while bringing up WipEout 3 SE: it fixes overlay-DLL generation and byte-variant identity, makes cycle/IRQ/cache shortcuts preserve guest-observable timing, closes several PGXP/GTE provenance holes, extends HD-texture tracking to copied VRAM, separates presentation content policy from the configured outer canvas, and adds reusable profiling/test surfaces. The mechanisms are generally applicable to recompiled PS1 titles; the few title-specific observations and enablement conclusions are isolated below and should not be upstreamed as framework policy. + +## Audit basis and limits + +- The audited delta is the **current working tree against local `HEAD` `4f0b07b1491e` on `feat/ws-cull-widen`**, because that is the boundary that matches the reported session body: 58 tracked files (`3,068` insertions, `871` deletions) plus 16 meaningful untracked source/doc/test files = **74 meaningful paths**. Untracked `build-pgxp-focused/` and `build-texpack-focused/` are build artifacts and were excluded. +- `fork/feat/ws-cull-widen` is `e27eac73663f`, an ancestor 27 commits behind local `HEAD`. Diffing to that remote ref would mix those already-committed changes into this working-tree contribution and produce 81 meaningful paths instead of the reported approximately 76. +- Per instruction, no build, test, benchmark, or executable was run during this audit. “Covered” below means the test source exists and targets the invariant; it is **not** a fresh pass result. +- A number is called “measured” only when the audited files contain an actual observation/result. Loop counts and cache capacities are reported as designed coverage/structure, not as performance results. Where the delta has no result, the measurement is **UNKNOWN**. + +## Corrections to the supplied headlines + +1. **Overlay shim anchor: mechanism supported; manifestation counts unverified.** `patch_generated_c` now anchors after the unconditional `#include "psx_runtime.h"` and fails if it is missing. The delta does not contain a linker transcript or result establishing “~10k undefined runtime symbols” or “zero overlay DLLs”; those numbers are **UNKNOWN** here. +2. **Overlay stale-PC/variant identity: supported.** An executed-word overwrite retires the whole page epoch, incoherent capture roots are rejected against current bytes, and static requests are keyed by `(load address, size, bytes)`. +3. **Cycle specialization: supported with a qualification.** Generation selects arity 0/1/2/3 helpers, but masks with more than three nonzero dependencies deliberately retain the generic mask helper. Main-RAM load charges and the pre-deadline fast path are present. +4. **LTO: feature supported; quoted deltas unsupported.** `PSX_ENABLE_LTO`, the no-inline cycle wrappers, and a deterministic benchmark target exist. No checked-in output contains `-45.92%`, `+10.45%`, or `+32.69%`; all three are **UNKNOWN** for this audit. +5. **PGXP/GTE: the correctness changes are supported; “fake never-incremented counters removed” is not.** The old `s_nclip_corrected` counter was incremented on both sign-disagreement branches, and the removed `s_texcorr.z_interp` field was incremented when missing Z values were mean-filled. The change converts NCLIP to non-mutating disagreement telemetry, removes partial-Z interpolation, and adds real primitive/provenance counters; no demonstrably never-incremented counter removal was found. +6. **HD texture pack: supported.** Tile occupancy gates invalidation scans, and VRAM copies can become upload-equivalent candidates, with deferred readback for unresolved copies. +7. **HD text gate: headline mechanism is wrong.** The change uses an atomic aggregate boolean (`g_dispatch_armed`) derived from capture, string-table, and pending-patch state. It is **not** a monotonic mutation epoch. The separate native-text guard does gain a monotonic epoch, but that is a different subsystem. +8. **Presentation: supported as an opt-in capability.** With `fixed_outer_aspect=true`, scene classification selects only the inner content rect; the outer rect comes from configured aspect. Legacy behavior remains when the option is false. +9. **Card-write diagnostic: only true for release builds.** `card_data_writes_check` and `card_data_writes_arm` preprocess away under `PSX_NO_DEBUG_TOOLS`; debug builds still execute them. +10. **GTE bridge narrowing: supported.** The command exporter stops rewriting inputs/control registers and its source comment counts **40 redundant stores removed per COP2 command**; per-write whole-state canonicalization is removed from data/control writes. +11. **Widescreen `items_join`: not in this delta.** `runtime/src/ws_ui_group.c` and its test are unchanged against local `HEAD`. Submission-order joining was implemented earlier in ancestor commit `572b921`; it must be summarized from that commit separately, not claimed as part of this worktree. + +Significant changes missing from the supplied list are the exact IRQ-pending cache and writer audit, I-cache replay isolation, two generated-dispatch caches, cycle-observer flushes, live CRTC cadence reconciliation/config exposure, PGXP rectangle-state disarming, release diagnostic pruning, and new host-optimization/keyboard regression surfaces. + +## Improvement inventory + +Tests named in the last column were inspected but not executed. + +### Correctness bugs + +| ID | Title | Files touched | What it fixes | Why it is general | Test(s) that cover it | +|---|---|---|---|---|---| +| C1 | Overlay shim anchored to an unconditional include | `tools/compile_overlays.py`; `runtime/include/{cpu_state.h,psx_cycles.h}`; overlay shim tests | A conditional include could make the whole DLL callback shim disappear under `PSX_OVERLAY_DLL_BUILD`, leaving unresolved direct runtime calls. The new anchor is the emitter-guaranteed unconditional runtime include and absence is fatal. | Any generated overlay can acquire conditional headers; no title address or symbol is involved. | `recompiler/tests/test_overlay_store_barrier_codegen.py`; `test_overlay_shim_compile.py` | +| C2 | Overlay execution epoch and exact byte-variant identity | `runtime/src/memory.c`; `tools/compile_overlays.py`; overlay discovery/perf-guard tests | Prevents an outgoing image's recorded PCs from being compiled against a later image's bytes; current-byte CFG rejection and exact-image static requests fail closed to interpretation. | Self-modifying code, streamed overlays, and same-address variants are generic PS1 behaviors. | `recompiler/tests/test_aot_overlay_discovery.py`; `runtime/tests/test_interpreter_perf_guards.py` | +| C3 | Exact cached INTC output and Cause.IP2 ownership | `runtime/include/interrupts.h`; `runtime/src/{interrupts,boot_state,memory,overlay_loader,sio,psx_interpreter}.c`; IRQ tests | Replaces repeated `i_stat & i_mask` hot reads with one cache while making a single refresh owner update both the full pending value and Cause.IP2 after every writer. | Every title uses the PS1 INTC; correctness depends on hardware-register mutation, not game logic. | `runtime/tests/test_cause_ip2_combinational.c`; `test_perf_round3_guards.py` | +| C4 | Deferred-cycle observers flush before reading time | `runtime/src/psx_cycles.c`; cycle tests | MUL/DIV and GTE issue/read/stall paths no longer observe a guest clock that is still sitting in the deferred batch. | These are architectural R3000A/GTE completion timestamps shared by all guests. | `runtime/tests/test_psx_cycle_event_boundaries.c`; `test_perf_round3_guards.py` | +| C5 | I-cache replay cannot mutate tags or charge misses | `runtime/include/psx_icache.h`; `runtime/tests/test_psx_icache_fastpath.c` | Replay now suppresses both tag evolution and miss charges while preserving normal cold initialization, hit/miss behavior, and the disabled-cache giveback path. | Lockstep/replay correctness is independent of title code. | `runtime/tests/test_psx_icache_fastpath.c` | +| C6 | Primitive-atomic PGXP position/depth and frame-canonical endpoints | `runtime/include/pgxp_vertex_cache.h`; `runtime/src/{pgxp_vertex_cache,gpu}.c`; `ENHANCEMENTS.md`; cache test/CMake | Eliminates corrected/native cracks on shared raster endpoints, prevents the two halves of a quad from making different decisions, and rejects incomplete depth instead of mean-filling missing Z. First use of `(frame, final native x, final native y)` owns the endpoint for the frame. | Shared-edge identity, quad splitting, and incomplete projection depth occur in any PGXP-enabled title. | `runtime/tests/test_pgxp_vertex_cache.c`; depth completeness is exercised indirectly by existing GTE/PGXP tests, but **no focused incomplete-Z primitive test was found.** | +| C7 | Per-component PGXP projection identity | `runtime/src/pgxp.cpp`; `runtime/include/pgxp.h`; PGXP tests | Identical packed bits from different projections can no longer be mistaken for coherent X/Y/Z provenance, including after halfword reconstruction. | Packed-word aliasing is a generic provenance problem in MIPS dataflow. | `runtime/tests/test_pgxp.cpp` | +| C8 | Destructive PGXP writes invalidate stale provenance everywhere | `recompiler/src/{pgxp_hook_emitter.cpp,pgxp_hook_emitter.h}`; `runtime/include/{pgxp.h,pgxp_hooks.h,overlay_dispatch_preamble.c.inc}`; `runtime/src/pgxp.cpp`; `runtime/src/{memory,psx_interpreter,dirty_ram_interp,overlay_loader}.c`; emitter/PGXP coverage tests | AND/XOR/NOR/SLT families, link writes, COP0/COP2 destinations, and raw RAM/scratch stores now kill old provenance even when the resulting integer bytes are unchanged. Sparse GPR/page-live gates avoid paying the callback when no value is live. | All recompilers/interpreters must agree on writer semantics; no opcode handling is title-specific. | `recompiler/tests/full_function_emitter_test.cpp`; `runtime/tests/test_pgxp.cpp`; `test_pgxp_destructive_write_coverage.py` | +| C9 | SXYP shadow FIFO follows architectural FIFO movement | `runtime/src/{gte.cpp,pgxp.cpp}`; `runtime/include/pgxp.h`; GTE test | A write to register 15 shifts shadows `12 <- 13`, `13 <- 14`, then invalidates the new `14/15`, instead of retaining stale SXY0/SXY1 provenance. | SXYP alias/FIFO semantics are architectural GTE behavior. | `runtime/tests/test_gte_register_access.cpp` | +| C10 | Precise NCLIP is telemetry-only and guest-bit-exact | `runtime/src/{gte.cpp,gpu.c,debug_server.c}`; `runtime/include/cpu_state.h`; `recompiler/src/config_loader.h`; GTE test | Exact 16.16 winding is still observed, but it never changes guest MAC0 or face visibility; the old saturation-rail clamp rescue can no longer smuggle far-off precise coordinates into rasterization. | Host precision enhancements must not alter guest branches or bypass native GTE saturation in any title. | `runtime/tests/test_gte_register_access.cpp::test_precise_nclip_never_flips_guest_sign` | +| C11 | Polygon-to-rectangle boundary clears triangle-only PGXP state | `runtime/src/{gpu.c,gpu_render.c}` | PGXP-armed polygon fast paths are no longer collapsed to rectangles, and native GP0 rectangles clear any prior triangle precision, perspective, or depth state. | Renderer state leakage and primitive fast-path conversion can affect any sequence mixing polygon and rectangle paths. | **No focused test found.** | +| C12 | Live CRTC changes reconcile host pacing at VBlank | `runtime/include/gpu.h`; `runtime/src/gpu.c`; `runtime/src/main.cpp`; `runtime/tests/test_host_refresh_sanitize.py` | Host pacing/swap policy is no longer startup-latched: a GP1(08h), restore, or plugin period change resets old pacing debt and is applied before waits; banners defer until the guest programs display mode. | PAL/NTSC switching, restores, and timing plugins are framework concerns. | `runtime/tests/test_host_refresh_sanitize.py` | + +### Performance + +| ID | Title | Files touched | What it fixes | Why it is general | Test(s) that cover it | +|---|---|---|---|---|---| +| P1 | Generation-time cycle dependency arity | `recompiler/src/{code_generator,full_function_emitter}.cpp`; `runtime/include/psx_cyc.h`; `runtime/src/psx_cyc_steps.c`; emitter/cycle tests | Removes the runtime dependency-mask loop for the common 0–3 register cases while retaining a generic fallback for larger masks; shared no-inline wrappers bound LTO cloning. | Every generated instruction uses the load-delay/cycle dependency machinery. | `runtime/tests/test_psx_cyc_batch.c`; `recompiler/tests/full_function_emitter_test.cpp`; `runtime/tests/test_perf_round3_guards.py` | +| P2 | Fused load charge and exact pre-deadline fast charge | `runtime/include/{psx_cyc.h,psx_cycles.h}`; `runtime/src/{memory,psx_cycles,psx_interpreter,dirty_ram_interp}.c`; cycle/perf tests | Main-RAM loads publish base + read fudge + wait/completion as one exact charge; eligible CPU charges advance directly only through `next_service_cycle - 1`, including exact Q16 overclock remainder handling. | Load timing, event deadlines, replay, and overclock transforms are common runtime machinery. | `runtime/tests/test_psx_cyc_batch.c`; `test_psx_cycle_event_boundaries.c`; `test_interpreter_perf_guards.py` | +| P3 | Exact-address compiled dispatch caches | `recompiler/src/{full_function_emitter,main_psx}.cpp`; emitter test | Adds a 256-slot TLS cache in generated static dispatch and a 1,024-slot TLS positive/negative cache in game lookup; every hit rechecks the full canonical address before bypassing binary search. | BIOS services, tail-call loops, and large dispatch tables occur across recompiled games. | `recompiler/tests/full_function_emitter_test.cpp` covers the generated-dispatch cache; **no focused test found for the game lookup cache.** | +| P4 | Two-level native-text guard epoch | `runtime/src/memory.c`; structural guard test | Stable no-write intervals return after one epoch comparison; after a mutation, the existing exact page-generation sum decides whether the cached verdict remains valid. | Dirty-RAM/native-code guards are shared by every title using mutable text. | `runtime/tests/test_perf_round3_guards.py` | +| P5 | Aggregate text-translation dispatch gate | `runtime/include/text_xlate.h`; `runtime/src/{text_xlate.cpp,fntrace.c}`; structural guard test | Avoids invoking the translation callback at every dispatch once capture is off and no string-table or unsettled RAM patch needs it. This is an atomic boolean, not an epoch. | Capture, shipped translations, and pending patch classes are framework state. | `runtime/tests/test_perf_round3_guards.py` | +| P6 | Tile-indexed texture invalidation | `runtime/include/tex_pack_rect_index.h`; `runtime/src/tex_pack.cpp`; index test/CMake | VRAM writes first query conservative 16×16-tile occupancy instead of always linearly scanning up to 8,192 upload/palette/pending-copy records; uncertainty returns “maybe” and preserves the slow exact scan. | VRAM invalidation volume and upload count vary by title, while the VRAM geometry is common. | `runtime/tests/test_tex_pack_rect_index.cpp` | +| P7 | Narrow GTE command export and incremental canonicalization | `runtime/src/gte.cpp`; `runtime/tests/test_gte_register_access.cpp`; `runtime/CMakeLists.txt` | Commands export only architectural results (`D7..D29`, `C31`) and writes maintain aliases/masks incrementally instead of sweeping the entire backing state after every write. | COP2 register semantics are architectural and shared. | `runtime/tests/test_gte_register_access.cpp` | +| P8 | Debug-only store diagnostics compile out of release | `runtime/src/{memory,sio}.c`; structural guard test | Removes universal RAM-store calls to card-write and debug trace consumers, plus SIO arming, when `PSX_NO_DEBUG_TOOLS` has no consumer. Debug behavior is retained. | This is build-mode ownership, not a title exception. | `runtime/tests/test_perf_round3_guards.py` | + +### New capability + +| ID | Title | Files touched | What it fixes/adds | Why it is general | Test(s) that cover it | +|---|---|---|---|---|---| +| N1 | Opt-in CMake IPO/LTO and generated-hotpath benchmark | `runtime/runtime.cmake`; `runtime/CMakeLists.txt`; `runtime/tests/bench_psx_runtime_hotpath.c`; `runtime/src/psx_cyc_steps.c` | Adds toolchain-checked `PSX_ENABLE_LTO` (default off), applies IPO to runtime targets, and provides a semantic-checksummed generated-style benchmark. | Toolchain optimization and code-shape regression measurement apply to every game target. | Benchmark is deliberately not registered in CTest; **no result is checked in or run here.** | +| N2 | HD replacements from VRAM-to-VRAM copies | `runtime/include/tex_pack.h`; `runtime/src/{tex_pack.cpp,gpu_render.c}`; copy test/CMake | Copy destinations can reuse an exact tracked source preimage or become a lazily read-back candidate; CPU upload and copy records share the same position/size/hash semantics and replacement lookup. | Many PS1 engines stage textures in VRAM rather than uploading final destinations directly. | `runtime/tests/test_tex_pack_copy.cpp` | +| N3 | Configurable CRTC refresh multiplier | `recompiler/src/{config_loader.cpp,config_loader.h}`; `runtime/include/{gpu.h,mod_plugins.h}`; `runtime/src/gpu.c`; `runtime/src/main.cpp`; host refresh test | Exposes `[video] crtc_refresh_multiplier` in the range 1..8 as the baseline timing setting, with activation plugins allowed to override it. | It turns an existing runtime timing control into a validated per-project framework option. | `runtime/tests/test_host_refresh_sanitize.py` covers live x2 cadence ownership, not parser bounds 1..8. | +| N4 | Stable configured outer aspect with scene-specific inner content | `WIDESCREEN.md`; `recompiler/src/{config_loader.cpp,config_loader.h}`; `runtime/include/{gpu_gl_renderer.h,gpu_vk_renderer.h,present_ring.h,ws_present_layout.h}`; `runtime/src/{gpu.c,gte.cpp,main.cpp,gpu_gl_renderer.c,gpu_vk_renderer.c,ws_present_layout.c}`; layout test/CMake | Adds opt-in `fixed_outer_aspect`: menus/FMV can remain centered 4:3 content without changing the outer configured canvas; GL, Vulkan, interpolation, hold-last, and CPU paths use the separated policy. | Outer user preference and content correction are independent concepts for any widescreen title. | `runtime/tests/test_ws_present_layout.c` | + +### Diagnostics and verification surfaces + +| ID | Title | Files touched | What it fixes/adds | Why it is general | Test(s) that cover it | +|---|---|---|---|---|---| +| D1 | Actionable PGXP and copied-texture telemetry | `runtime/include/{cpu_state.h,pgxp.h}`; `runtime/src/{gpu.c,gte.cpp,pgxp.cpp,debug_server.c,tex_pack.cpp}` | Reports primitive armed/rejected/mixed decisions, endpoint promotion/demotion/cache overflow, memory invalidations, mixed projection rejects, NCLIP sign disagreements, and copy queued/resolved/tracked/invalidated/dropped counts. | These counters diagnose framework coverage and fail-closed behavior without game addresses. | Counter transitions are partially asserted by `runtime/tests/{test_pgxp.cpp,test_gte_register_access.cpp,test_tex_pack_copy.cpp}`. | +| D2 | Host optimization acceptance contract | `docs/HOST_OPTIMIZATION_CONTRACT.md` | Defines an interleaved A/B protocol, semantic equivalence requirement, raw-run retention, and a 5% acceptance gate for complexity/ABI-bearing host optimizations. | It is a reusable maintainer process rather than title behavior. | Documentation only. | +| D3 | Keyboard Select+Start chord regression | `runtime/tests/test_keyboard_pad_chord.c`; `runtime/CMakeLists.txt` | Adds a focused test for default bindings, simultaneous and consecutive polls, release, and re-press behavior. It does not change production keybind code. | Multi-button active-low pad words are common input behavior. | `runtime/tests/test_keyboard_pad_chord.c` | + +## Concrete evidence and measurements + +### Correctness evidence + +**C1 — overlay shim anchor.** The root cause and new invariant are stated at `tools/compile_overlays.py:2078-2089`: the syntactically last include may be inside the per-instruction I-cache `#ifdef/#else`, and `PSX_OVERLAY_DLL_BUILD` would preprocess away the callback definitions. The insertion now uses the exact unconditional include at `tools/compile_overlays.py:2084-2091`; absence raises `ValueError`. `recompiler/tests/test_overlay_store_barrier_codegen.py:109-118` asserts the flush definition precedes the DLL conditional, and `test_overlay_shim_compile.py:43-59` compiles the expanded PGXP hook surface with the DLL macros when that test is run. Windows exports for the cycle barrier are at `runtime/include/cpu_state.h:76-81` and `runtime/include/psx_cycles.h:192-198`. **Measured result:** UNKNOWN. The supplied ~10k unresolved-symbol and zero-linked-DLL claims are absent from the audited delta. + +**C2 — overlay epoch/identity.** `runtime/src/memory.c:1080-1114` preserves evidence for data-only writes but clears both full-page PC bitmaps and the page gate when any executed word is replaced. `tools/compile_overlays.py:1546-1572` rejects capture-derived roots whose current capped CFG reaches invalid instructions; `tools/compile_overlays.py:2867-2930` keys static demand by `(load_addr, size, data)` and resolves it only against current-byte range identities; isolated generation also reports `INCOHERENT_VARIANT_BYTES` at `tools/compile_overlays.py:3074-3083`. Focused regressions begin at `recompiler/tests/test_aot_overlay_discovery.py:340` and `:366`. **Measured result:** UNKNOWN; no shard ok/failed count or coverage percentage for this change is present. + +**C3 — IRQ cache.** The owner computes `pending = i_stat & i_mask`, stores the full value, and then derives Cause.IP2 at `runtime/src/interrupts.c:149-160`. The structural audit at `runtime/tests/test_perf_round3_guards.py:22-46` currently finds exactly **19** direct `I_STAT`/`I_MASK` mutations across five writer files and requires a refresh near each; it also verifies the cache is stored before the nullable Cause pointer can return (`:55-63`). `runtime/tests/test_cause_ip2_combinational.c:97` repeatedly asserts exact cache equality. **Measured result:** UNKNOWN; 19 is a static writer-set count, not a runtime speed result. + +**C4 — cycle observers.** Flushes precede cycle-dependent work in `psx_muldiv_set`, `psx_muldiv_stall`, `psx_gte_read`, `psx_gte_set`, and `psx_gte_stall` at `runtime/src/psx_cycles.c:696-795`. `runtime/tests/test_psx_cycle_event_boundaries.c` includes explicit delayed GTE read/stall assertions and ends by checking cross-device `D-1 + 1` causality. **Measured result:** UNKNOWN. + +**C5 — I-cache replay.** `runtime/include/psx_icache.h:20-32` separates active-state initialization from replay and calls the miss path only outside replay. Warmed-hit and would-be-miss replay cases are at `runtime/tests/test_psx_icache_fastpath.c:79-103`. **Measured result:** UNKNOWN. + +**C6 — primitive/canonical endpoint.** The cache is a fixed **65,536-entry** table (`runtime/include/pgxp_vertex_cache.h:21-35`) and resolves frame/native-coordinate identity at `runtime/src/pgxp_vertex_cache.c:10-38`. Whole primitive resolution, including all four quad vertices, starts at `runtime/src/gpu.c:3434-3609`; depth arms only when `depth_count == count` (`:3570-3605`), replacing the old mean-filled partial-Z path. The focused cache test covers corrected→native, native→corrected, transform identity, and frame identity; its native→corrected candidate is deliberately **0.75 px**, above the old 0.5 workaround (`runtime/tests/test_pgxp_vertex_cache.c:12-64`). **Measured result:** no performance or in-game crack count is recorded; UNKNOWN. + +**C7 — component IDs.** `PGXPValue` gains independent X/Y/Z projection IDs at `runtime/src/pgxp.cpp:62-68`; coherence predicates require matching nonzero IDs at `:287-307`, and reconstructed halves propagate their component's ID around `:450-526`. `runtime/tests/test_pgxp.cpp:205-220` asserts that mixed projections are rejected even when packed integer geometry can otherwise look usable. **Measured result:** UNKNOWN. + +**C8 — destructive writes.** The hot GPR macro consults `g_pgxp_gpr_live_mask` before calling the invalidator (`runtime/include/pgxp_hooks.h:93-101`); the invalidator itself is at `runtime/src/pgxp.cpp:396-403`. Raw word invalidation and partial-half handling begin at `runtime/src/pgxp.cpp:371`, while six central RAM/scratch write branches are gated in `runtime/src/memory.c:1968-2520`. `runtime/tests/test_pgxp_destructive_write_coverage.py:69-77` requires two 1-byte, two 2-byte, and two 4-byte paths plus both interpreter/emitter surfaces. `runtime/tests/test_pgxp.cpp:137-203` includes byte-identical GPR and RAM writes. The existing source comment says the earlier broad ALU-hook gate accounted for **3.6% of the emulation thread** in a high-refresh workload (`runtime/src/pgxp.cpp:91-101`), but there is no measured delta for the new sparse invalidation gates. **Measured result for this change:** UNKNOWN. + +**C9 — SXYP.** Architectural backing shifts at `runtime/src/gte.cpp:2027-2047`; the shadow shift/reset is implemented in `runtime/src/pgxp.cpp:1033-1053`. `runtime/tests/test_gte_register_access.cpp:501-526` checks the precise FIFO before and after register 15 and all SXY write classes. **Measured result:** UNKNOWN. + +**C10 — NCLIP.** The old double-based sign replacement is removed; `runtime/src/gte.cpp:962-983` computes an exact `int64_t` determinant, increments disagreement telemetry, and always writes native `out` to MAC0. The GPU-side comment at `runtime/src/gpu.c:3644-3650` makes NCLIP audit-only and removes the saturation-rail rescue route to far-off precise raster coordinates. The regression constructs native `+1` versus exact negative and requires guest MAC0 to remain `+1` while the disagreement counter increments (`runtime/tests/test_gte_register_access.cpp:575-602`). The removed code incremented `s_nclip_corrected` in both mismatch branches, and the removed partial-Z path incremented `s_texcorr.z_interp`, so the “never-incremented” description is false. **Measured result:** one deterministic disagreement fixture; no field count or performance result. + +**C11 — rectangle boundary.** PGXP-aware polygon fast paths require position/depth to be unarmed before collapsing a quad to a rectangle (`runtime/src/gpu.c:4049-4054`, `:4287-4330`). `runtime/src/gpu_render.c:135-141` clears precise position, perspective, and depth state; each flat/textured/scaled rect entry invokes it at `:270`, `:288`, and `:296`. **Test/measurement:** no focused test found; UNKNOWN. + +**C12 — live CRTC cadence.** Guest display-mode observation is recorded on GP1(08h) and restore at `runtime/src/gpu.c:6018` and `:6202`. `refresh_live_crtc_cadence` (`runtime/src/main.cpp:2916-2949`) keys transitions by exact guest period, resets old wall-clock debt, reapplies present cadence, and logs only after guest programming; it is called before frame performance/pacing work at `runtime/src/main.cpp:6250`. The structural test's exact arithmetic establishes PAL×2 = **100.0 Hz** matches a 100 Hz panel while NTSC×2 = **120.0 Hz** does not (`runtime/tests/test_host_refresh_sanitize.py:73-86`). These are formula assertions, not a live-run result. **Measured runtime result:** UNKNOWN. + +### Performance evidence + +**P1 — arity helpers.** Both emitters choose specialized helpers and retain the generic fallback at `recompiler/src/code_generator.cpp:51-69` and `recompiler/src/full_function_emitter.cpp:35-53`. The shared wrappers and deliberate no-inline boundary are at `runtime/src/psx_cyc_steps.c:5-68`; overlay DLLs retain inline equivalents at `runtime/include/psx_cyc.h:219-247`. `runtime/tests/test_psx_cyc_batch.c:244-264` defines **98,304 oracle comparisons** (`4,096 seeds × 6 modes × 4 arities`) and compares every `CPUState` byte plus deferred cycles. **Measured performance:** UNKNOWN. + +**P2 — fused/fast charge.** Main-RAM word and half loads compute a single `charge + fudge + 5` publication (`wait 3 + completion 2`) at `runtime/include/psx_cyc.h:280-337`; slow paths carry the same base charge through `runtime/src/memory.c:2316-2383`. `psx_cycles_try_fast_charge` previews and commits both scaled cycles and Q16 remainder only below the published limit (`runtime/include/psx_cycles.h:110-139`), which is set to `next_service_cycle - 1` at `runtime/src/psx_cycles.c:189-194` and invalidated at MMIO/exact/restore boundaries. Tests define **4,096 fused-load oracle cases** (`2,048 × 2 overclock modes`, `runtime/tests/test_psx_cyc_batch.c:267-307`) and **1,152 fast-charge oracle cases** (`2 scales × 16 accumulators × 3 charges × 12 deadline rooms`, `:311-334`). **Measured performance:** UNKNOWN. + +**P3 — dispatch caches.** Generated static dispatch emits the 256-slot table and exact-address hit check at `recompiler/src/full_function_emitter.cpp:1834-1865`; the game lookup emits 1,024 TLS slots and caches both hits and misses at `recompiler/src/main_psx.cpp:1594-1629`. `recompiler/tests/full_function_emitter_test.cpp:199-204` checks the generated helper and address validation. **Measured result:** UNKNOWN; no hit rate, comparison count, or benchmark delta is present. + +**P4 — native-text guard epoch.** The monotonic epoch and global mutation helper are at `runtime/src/memory.c:617-624`; the one-compare hit and page-generation fallback are at `:772-829`. The existing source comment records a motivating profile in which `psx_game_text_native_ok` was **6.8% of the emulation thread** with the older memo never hitting (`runtime/src/memory.c:625-632`). That is a baseline attribution, not a measured post-change gain. **Measured gain:** UNKNOWN. + +**P5 — text-translation aggregate gate.** `refresh_dispatch_armed` derives one atomic gate from capture, string-table, and three pending patch counts (`runtime/src/text_xlate.cpp:441-453`); `fntrace_record` checks it at `runtime/src/fntrace.c:170-171`. The source records two historical baselines: capture scanning at roughly **33k dispatches/frame**, **18.6% of samples**, with capture disabled yielding **+22% fps over guest frames 800–2200** (`runtime/src/text_xlate.cpp:423-435`), and the otherwise-useless fixes-only argument scan at **4.2% of the emulation thread** (`:898-905`). None is a direct A/B of the new outer aggregate gate. **Measured gain:** UNKNOWN. + +**P6 — texture tile index.** The conservative grid uses 16×16 tiles over a 2,048×1,024 domain and returns “maybe” for unindexed/uncertain rectangles (`runtime/include/tex_pack_rect_index.h:15-83`). `invalidate_locked` checks upload, pending-copy, and palette occupancy before any vector scan (`runtime/src/tex_pack.cpp:562-620`); the tracker cap remains **8,192** records (`runtime/src/tex_pack.cpp:54`). The deterministic test runs **20,000 randomized add/remove/query iterations** and asserts there are no false negatives (`runtime/tests/test_tex_pack_rect_index.cpp:47-77`). **Measured performance:** UNKNOWN. + +**P7 — GTE bridge.** `runtime/src/gte.cpp:1702-1729` exports only command results and explicitly counts **40 redundant stores removed per command**. Data/control writes now maintain their own masked/aliased state at `:2012-2081`; full canonicalization remains only at the raw import boundary (`:2099`). The oracle suite covers **2,112 command cases** (`22 functions × 96`), **256** MVMVA selector combinations, and **8,192** stateful stream steps (`64 × 128`) at `runtime/tests/test_gte_register_access.cpp:382-473`, in addition to register-write tests. **Measured throughput:** UNKNOWN. + +**P8 — release diagnostics.** Declarations and call sites are guarded around `runtime/src/memory.c:1298-1301`, `:2006-2012`, `:2142-2150`, and `:2485-2493`; SIO arming receives the same build guard. `runtime/tests/test_perf_round3_guards.py:65-73` scans every card diagnostic call for a release guard. **Measured performance:** UNKNOWN. + +### Capability evidence + +**N1 — LTO/benchmark.** `runtime/runtime.cmake:26-44` defines default-off, C/C++ toolchain-checked `PSX_ENABLE_LTO`; target IPO is applied at `:937-939`. The wrappers resist whole-program cloning at `runtime/src/psx_cyc_steps.c:28-38`. The benchmark models **32 guest instructions/block**, defaults to **4,000,000 blocks** (**128,000,000 guest instructions/sample**), takes **7 samples**, reports median/min/max, and rejects checksum drift (`runtime/tests/bench_psx_runtime_hotpath.c:25-28`, `:227-284`). It is intentionally not a CTest (`runtime/CMakeLists.txt:46-58`). **Measured result:** UNKNOWN; the supplied `-45.92%`, `+10.45%`, and `+32.69%` values are not in the audited repository state. + +**N2 — VRAM copies.** `gr_copy_rect` classifies mask/wrap/overlap preservation and reports the copy before backend mutation (`runtime/src/gpu_render.c:143-156`). `tex_pack_on_copy` (`runtime/src/tex_pack.cpp:817-893`) shares exact full-source preimages, hashes subrects, invalidates the destination in the original order, and otherwise queues a fail-closed candidate. Draw-time query/resolution is at `:896-989`; CPU and copy records meet in the shared builder at `:625-657`. `runtime/tests/test_tex_pack_copy.cpp` checks CPU/copy key parity, invalidation, and replacement lookup. Its optional, unregistered benchmark is designed for **200,000 copies**, **5,000,000 no-pending queries**, and **50,000 hashes** (`:168-208`), but no output is checked in or run. **Measured performance:** UNKNOWN. + +**N3 — CRTC config.** Parsing and range rejection are at `recompiler/src/config_loader.cpp:573-579`; the default/range contract is at `recompiler/src/config_loader.h:412-414`. Config is applied before activation plugins at `runtime/src/main.cpp:11534-11538`. The available test checks x2 live cadence only; it does not test TOML parser bounds or multipliers 3–8. **Measured compatibility/coverage for 3–8:** UNKNOWN. + +**N4 — outer/content layout.** The API contract explicitly forbids scene classification from changing the outer rect when fixed mode is enabled (`runtime/include/ws_present_layout.h:20-32`); the independent rect calculation is at `runtime/src/ws_present_layout.c:33-60`. GL uses it at `runtime/src/gpu_gl_renderer.c:3452-3461`; Vulkan uses it at `runtime/src/gpu_vk_renderer.c:1688-1706`; scene classification was renamed/content-scoped in `runtime/src/gpu.c:324-390` and `runtime/src/main.cpp:7022-7391`. The focused test checks exact 32:9-in-16:9 and 21:9 layouts plus unchanged legacy behavior (`runtime/tests/test_ws_present_layout.c:14-43`). **Measured visual/performance result:** UNKNOWN. + +### Diagnostic evidence + +**D1 — telemetry.** Primitive counters are defined and exported at `runtime/src/gpu.c:3460-3498`; PGXP memory/mixed-projection counters are maintained in `runtime/src/pgxp.cpp`; NCLIP disagreements are at `runtime/src/gte.cpp:941-983`; debug JSON publishes them at `runtime/src/debug_server.c:5216-5276`. Copy lifecycle counters are defined at `runtime/src/tex_pack.cpp:204-207` and published at `:1465-1487`. **Measured field values:** UNKNOWN; the change adds counters but the delta contains no captured report. + +**D2 — optimization contract.** `docs/HOST_OPTIMIZATION_CONTRACT.md:29-44` defines the 5% retention and semantic-equivalence gates; `:103-110` requires a separate interleaved performance campaign; `:185-200` specifies evidence retention. The example percentages at `:181-182` are explicitly examples, not results from this work. **Measured result:** not applicable. + +**D3 — keyboard chord test.** `runtime/tests/test_keyboard_pad_chord.c:24-50` checks default Return/RShift bindings and six pad states: released, Select held, first and consecutive Select+Start polls, Start release, and Start re-press. `runtime/CMakeLists.txt:86-95` registers it. **Production change/measurement:** none; this is regression coverage only, and it was not run here. + +## RISK / REVIEW FOCUS + +1. **Overlay byte identity and shim placement (C1/C2) — highest review priority.** Preserve three invariants: (a) the callback preamble is outside every conditional and missing the guaranteed runtime include is a hard failure; (b) a compiled body satisfies a request only when load address, size, and exact bytes/range identities agree; (c) overwriting an executed word retires all PC/dispatch evidence for that page's outgoing image, while an unrelated data-only write does not erase valid execution evidence. False positives execute wrong bytes; overly broad invalidation silently destroys AOT coverage. +2. **Guest cycle exactness and device deadlines (C4/P1/P2) — highest review priority.** Specialized helpers must be byte-for-byte equivalent to the generic dependency mask, preserve GPR0, read-absorb and load-delay ownership, and retain COSIM/replay/conservative/device-service fallbacks. A fused load must apply base, read fudge, wait, completion, and Q16 overclock remainder exactly once. Fast charge must never reach or cross `next_service_cycle`, overflow the clock, bypass a local/deferred batch, or leave stale eligibility after MMIO, exact service, or restore. Every observer of absolute guest time must flush first. +3. **IRQ pending cache and live CRTC timing (C3/C12/N3) — highest review priority.** Every direct `I_STAT`/`I_MASK` writer, including init, restore, acknowledgment, and temporary CD/SIO masking, must call the refresh owner. The full cached value must remain exactly `i_stat & i_mask`; only Cause.IP2 is masked to the architectural 11 bits. Cached snapshots must be refreshed after any idle skip/device service. CRTC period changes must be reconciled before the next host wait without changing plugin override precedence or inheriting pacing debt from the old period. Adding a twentieth writer without extending the structural audit is a correctness bug. +4. **PGXP provenance and guest visibility (C6–C11).** NCLIP telemetry must never write a host-corrected sign to guest MAC0. Projection IDs must survive only operations that prove component identity; every destructive writer must invalidate across both interpreters, generated code, and overlay DLLs. Endpoint-cache collision/overflow behavior must remain deterministic and fail to native geometry. Rect paths must clear every triangle-only backend latch. +5. **VRAM copy tracking (P6/N2).** Masked, wrapping, or overlapping copies must never claim a pre-copy source hash as the post-copy destination. Pending readback must use the authoritative post-copy backend, be invalidated by later writes/restores, and preserve CPU-upload key semantics including position. Tile indices may have false positives but must never have false negatives. +6. **LTO/code-shape claims (P1/N1).** Review generated symbol visibility on MSVC/Clang/GCC and ensure IPO does not clone the guarded wrappers or drop DLL exports. Do not accept the quoted LTO percentages without raw commands, compiler/linker versions, interleaved samples, and semantic checksums; none is present in this delta. +7. **Presentation backends (N4).** Fixed mode must produce the same outer rect for gameplay, menu, FMV, interpolation, hold-last, native VRAM, CPU, and wide paths; scene state may change only the inner rect. Legacy mode must remain byte/geometry compatible for projects that do not opt in, especially Vulkan's pre-existing 4:3 policy. + +## NOT UPSTREAMABLE + +- The WipEout-specific conclusion in `ENHANCEMENTS.md:879-883` that `pgxp_tolerance = 1.0` is valid for WipEout 3 is title validation/configuration, not a framework default. Upstream the canonical-endpoint mechanism and neutral explanation; keep the title's chosen value in the game repository. +- The explanatory phrase “the WipEout 8 MB payload depends on that” at `runtime/src/memory.c:1090` should be generalized before submission. The whole-page execution-epoch mechanism itself is upstreamable. +- `docs/internal/upstream/WIPEOUT3_SESSION_ENGINE_DRAFTS_2026-08-20.md` is a WipEout session/commit audit, not framework product documentation; do not include it in an upstream code PR. +- `build-pgxp-focused/` and `build-texpack-focused/` are generated build trees and must not be committed. +- Enabling `fixed_outer_aspect`, a non-stock CRTC multiplier, PGXP tolerance 1.0, or any other title profile belongs in the game repository. The parser/runtime capabilities and default-off framework behavior are upstreamable. +- No new WipEout-specific executable address hook was found in the working-tree delta. Existing title references outside the added lines are not evidence that the general mechanisms above are per-game hacks. +- The `items_join` UI grouping change is potentially upstreamable, but it is **not part of this delta**; submit/review it from ancestor commit `572b921` with its own history and evidence rather than folding it into this worktree summary. + +## Suggested PR split and landing order + +1. **Overlay DLL shim placement/export fix (C1).** Small, independently testable, and prerequisite to trusting any overlay-DLL validation. +2. **Overlay byte-identity correctness (C2).** Land after the shim fix so generated overlay tests exercise a valid DLL surface. Keep the page-epoch producer fix and byte-keyed consumer/static-request logic together because either half alone leaves a stale-image path. +3. **IRQ pending-cache ownership (C3).** Separate high-risk PR. Include all current writers and the 19-writer structural audit in the same change. +4. **Cycle exactness foundation (C4, C5, P2 without code-shape claims).** Land observer flushes, replay-safe I-cache behavior, fused load accounting, deadline publication/invalidation, and oracle tests together. This establishes equivalence before performance-oriented code generation changes. +5. **Cycle arity specialization and LTO surface (P1, N1).** Depend on PR 4. Keep the benchmark and no-inline rationale with the specialization; publish new raw benchmark results instead of the unsupported percentages in the supplied headline. +6. **Dispatch/text/release hot-path pruning (P3, P4, P5, P8).** These are independent of guest timing after PR 4. Prefer three reviewable commits/PRs if maintainers want narrower invalidation reasoning: dispatch caches; text guards/gates; debug-only store hooks. +7. **GTE/PGXP provenance correctness (C7–C10, P7).** Land component IDs, destructive-write coverage, SXYP FIFO, NCLIP telemetry-only behavior, and narrowed architectural export first. These define trustworthy provenance and guest-bit-exact boundaries. +8. **PGXP primitive/canonical endpoint and renderer boundary (C6, C11, D1 PGXP portion).** Depend on PR 7. Add a focused rectangle-state regression before submission; keep the WipEout tolerance conclusion out of the framework PR. +9. **Texture-pack index, then copy candidates (P6 followed by N2 and copy telemetry).** The copy queue uses the conservative index and shared upload storage, so land/test the no-false-negative index first. Review copy/mask/wrap/overlap and savestate invalidation as a separate capability PR. +10. **Presentation layout separation (N4).** Independent UI/rendering PR covering the shared layout helper and both GL/Vulkan paths. Preserve default-off legacy behavior. +11. **CRTC configuration and live cadence (C12/N3).** Separate timing-sensitive PR after IRQ/cycle work. Add parser tests for 1, 8, 0, and 9 and either validate or narrow the advertised 1..8 range before landing. +12. **Maintainer diagnostics (D2/D3).** The optimization contract can land as documentation if the project accepts that policy; the keyboard regression can land independently. Neither should block correctness PRs. + +--- + +## Appendix: session measurements (recorded 2026-08-20) + +The audit above correctly marks several deltas UNKNOWN because no result transcript is +checked in. These numbers WERE observed live during the session that produced this work. +They are recorded here so a reviewer can see the basis; they are single-machine +observations, not a portable benchmark suite. + +**Host:** AMD Ryzen 7 9800X3D (8C/16T, 4.7 GHz, X3D cache), Windows 11, pinned +retcomm cmake-clang-v1 (Clang 22.1.8). Title: WipEout 3 SE (PAL, SCES-02845) with +hueponik's official PAL 100 fps patch, 8 MB RAM, max in-game detail, +`cpu_overclock = 900`, `crtc_refresh_multiplier = 2` (PAL 50 Hz x2 = 100 Hz). + +### LTO / codegen benchmark +`runtime/tests/bench_psx_runtime_hotpath.c`, median-of-three rotated runs, seven samples +each, 256M guest instructions per sample. EVERY variant produced identical results +(guest cycles 33,774,169; oc accumulator 60,416; services 0; digest 3d88e743440954f1), +which is the equivalence evidence for this workload. + +| Variant | Guest instructions/s | vs baseline | +|---|---:|---:| +| Baseline `-O3` | 451,062,216 | - | +| Plain ThinLTO | 243,919,125 | -45.92% | +| Plain full LTO | 585,251,881 | +29.75% | +| ThinLTO + guarded wrappers | 498,195,210 | +10.45% | +| **Full LTO + guarded wrappers** | **598,526,036** | **+32.69%** | +| `-march=native` only | 457,336,000 | +1.39% | +| `-march=znver5 -mtune=znver5` | 441,476,740 | -2.13% | +| Helpers moved to headers/unity | 162,246,886 | -64.03% | + +Plain ThinLTO REGRESSES badly by cloning hot interlock bodies; the `noinline` guard at +`runtime/src/psx_cyc_steps.c:27` is what makes LTO a win. ThinLTO executable size fell +97,148,928 -> 86,249,472 bytes (-11.22%) once guarded, confirming the over-cloning. + +### End-to-end guest speed (identical BIOS boot phase each run, so directly comparable) +| Build state | Guest speed | +|---|---:| +| Session start (no static bake; overlay autocompile degraded) | 0.36x | +| + partial static bake (25/62 shards) | 0.42x | +| + clean static bake (26 shards, 0 failed) | 0.49x | +| + overlay DLL link fix; PGXP hooks and rewind disarmed | 0.58x | +| + cycle-accounting specialization, PGXP RE-ENABLED, 100% texture pack | 0.50x | +| **+ full LTO** | **0.68-0.76x** | + +LTO's end-to-end contribution on this identical phase was ~+35%, consistent with the ++32.69% synthetic result. 3D attract-mode scenes were observed sustaining +`VBlank 100/s 1.00x, render 100 fps`. + +### Overlay pipeline +| Metric | Before | After | +|---|---:|---:| +| Overlay DLL link errors | 10,395 diagnostics / 20 symbol names | 0 | +| Overlay DLLs produced by autocompile | 0 | 1,442 in cache | +| Static bake shards | ok=25 failed=23 skipped=14 | ok=26 failed=0 skipped=14 | +| Runtime dispatch | `disp_native=0`, `disp_interp=4,456,079` | native dispatch active | + +The `disp_native=0` figure is from the runtime's own `psx_last_run_report.json`, which also +carried `autocompile_degraded=1` with the reason string naming the cause. + +### HD texture pack +| Metric | Before | After | +|---|---:|---:| +| Requested (texture,palette) pairs covered | 39 / 439 (8.88%) | 439 / 439 (100%) | +| Pack entries | 292 | 445 generated (692 live incl. legacy) | +| PNG payload | 2.61 MiB | 4.32 MiB | + +Verification on the generated set: alpha subset of source 402/402, RGB subset 402/402, +all dimensions <= 4096, F500 font atlases byte-identical (untouched). + +### PGXP +Hook cost measured by the 1 kHz IP sampler on the emulation thread, in race: +`psx_pgxp_load` 7.66% + `psx_pgxp_alu` 3.63% + `psx_pgxp_store` 2.35% + +`psx_pgxp_cop2` 1.20% = **14.84%** while both correction features were DISABLED, because +`precise_nclip` and `pgxp_cpu_mode` arm the hook surface independently. After disarming, +the same buckets fell to 0.69% total. + +### Font atlases (game-side asset work, listed for completeness) +| Metric | Before | After | +|---|---:|---:| +| Detached ink components (ringing lobes) | 986 | 0 | +| Dark ink pixels (black halo) | 87,650 | 0 | +| Alpha masks | - | byte-identical before/after | diff --git a/recompiler/src/config_loader.cpp b/recompiler/src/config_loader.cpp index f9d75249..6b25477e 100644 --- a/recompiler/src/config_loader.cpp +++ b/recompiler/src/config_loader.cpp @@ -2329,14 +2329,14 @@ UserSettings load_user_settings(const fs::path& path) { const toml::value& h = toml::find(doc, "hotkeys"); if (h.contains("rewind_pad")) try_get([&]{ const auto n = toml::find(h, "rewind_pad"); - if (n >= 0 && n < 256) { + if (pad_bind_value_ok(n)) { s.hotkey_pad_rewind = (int)n; s.has_hotkey_pad_rewind = true; } }); if (h.contains("save_state_menu_pad")) try_get([&]{ const auto n = toml::find(h, "save_state_menu_pad"); - if (n >= 0 && n < 256) { + if (pad_bind_value_ok(n)) { s.hotkey_pad_save_state_menu = (int)n; s.has_hotkey_pad_save_state_menu = true; } diff --git a/recompiler/src/config_loader.h b/recompiler/src/config_loader.h index 9e7e829f..a993421e 100644 --- a/recompiler/src/config_loader.h +++ b/recompiler/src/config_loader.h @@ -55,6 +55,20 @@ inline constexpr int VIDEO_RENDERER_OPENGL = 1; inline constexpr int VIDEO_RENDERER_VULKAN = 2; inline constexpr int DEFAULT_VIDEO_RENDERER = VIDEO_RENDERER_OPENGL; +// Controller-hotkey bind encoding, mirroring recomp-ui's RECOMP_LAUNCHER_PAD_* +// (recomp_launcher.h): 0 = unbound, 1..99 = button (1 + SDL button code), +// 100..999 = axis, 1000+ = a CHORD, encoded as 1000 + a bitmask of SDL button +// codes. Two buttons held together is the normal case (select+r3), so a real +// value here is routinely five digits. +// +// The bound matters: a naive `< 256` check silently rejects every chord, so a +// saved rewind_pad = 1272 was dropped on load and pad hotkeys were only ever +// configurable to single buttons. SDL defines about 21 gamepad buttons, so cap +// the mask at 32 bits' worth and let the runtime ignore bits no controller can +// produce. +inline constexpr int PAD_BIND_MAX = 1000 + (1 << 21); +inline bool pad_bind_value_ok(long long n) { return n >= 0 && n < PAD_BIND_MAX; } + struct WidescreenSignedBoundSite { uint32_t address = 0; uint32_t expected = 0; // guarded LUI instruction diff --git a/runtime/CMakeLists.txt b/runtime/CMakeLists.txt index 982fe204..0088aea4 100644 --- a/runtime/CMakeLists.txt +++ b/runtime/CMakeLists.txt @@ -403,6 +403,9 @@ if(BUILD_TESTING) add_test(NAME rewind_toggle_combo_test COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/tests/test_rewind_toggle_combo.py) + add_test(NAME savestate_status_protocol_test + COMMAND ${Python3_EXECUTABLE} + ${CMAKE_CURRENT_SOURCE_DIR}/tests/test_savestate_status_protocol.py) endif() # DISABLED, not de-registered. It links and runs, but 147 of its 309 checks diff --git a/runtime/include/savestate.h b/runtime/include/savestate.h index c9c8f138..7d2a380a 100644 --- a/runtime/include/savestate.h +++ b/runtime/include/savestate.h @@ -97,6 +97,9 @@ int savestate_request_load_blob_protocol(const void* data, size_t size); /* 1 while a staged save/load has not yet been consumed by savestate_poll. */ int savestate_pending(void); +/* Machine-readable lifecycle receipt for deterministic debug harnesses. */ +void savestate_status_json(char* buf, size_t cap); + /* 1 once after a successful load restore (before scheduler longjmp). Clears. */ int savestate_take_load_completed(void); diff --git a/runtime/runtime.cmake b/runtime/runtime.cmake index 4fa2729c..6dc313ff 100644 --- a/runtime/runtime.cmake +++ b/runtime/runtime.cmake @@ -1264,19 +1264,37 @@ function(psxrecomp_add_runtime_target target) # own POST_BUILD copy so a title may still override an id if it ever needs # to; copy_directory merges rather than replacing the tree. if(EXISTS "${PSXRECOMP_ROOT}/mods/builtin/packages") + # Clear first: copy_directory MERGES, so a mod deleted from source + # would otherwise survive in the build output forever and keep + # appearing on the Mods page (and inflate release catalog assertions). + # Scoped to mods/packages, not mods/: state.toml is launcher-owned user + # state and must survive a rebuild. + set(_psx_builtin_mod_copy_commands "") + if(DEFINED PSX_BUILTIN_MOD_ALLOWLIST AND NOT + "${PSX_BUILTIN_MOD_ALLOWLIST}" STREQUAL "") + foreach(_psx_builtin_mod IN LISTS PSX_BUILTIN_MOD_ALLOWLIST) + if(NOT EXISTS "${PSXRECOMP_ROOT}/mods/builtin/packages/${_psx_builtin_mod}") + message(FATAL_ERROR + "PSX_BUILTIN_MOD_ALLOWLIST names missing package: ${_psx_builtin_mod}") + endif() + list(APPEND _psx_builtin_mod_copy_commands + COMMAND ${CMAKE_COMMAND} -E copy_directory + "${PSXRECOMP_ROOT}/mods/builtin/packages/${_psx_builtin_mod}" + "$/mods/packages/${_psx_builtin_mod}") + endforeach() + else() + list(APPEND _psx_builtin_mod_copy_commands + COMMAND ${CMAKE_COMMAND} -E copy_directory + "${PSXRECOMP_ROOT}/mods/builtin" + "$/mods") + endif() add_custom_command(TARGET ${target} POST_BUILD - # Clear first: copy_directory MERGES, so a mod deleted from source - # would otherwise survive in the build output forever and keep - # appearing on the Mods page (and inflate the release packagers' - # catalog assertions). This runs before the game's own staging, so - # both catalogs land on a clean slate. COMMAND ${CMAKE_COMMAND} -E rm -rf - "$/mods" - COMMAND ${CMAKE_COMMAND} -E copy_directory - "${PSXRECOMP_ROOT}/mods/builtin" - "$/mods" + "$/mods/packages" + ${_psx_builtin_mod_copy_commands} COMMENT "Staging framework-owned mod catalog" VERBATIM) + unset(_psx_builtin_mod_copy_commands) endif() endif() diff --git a/runtime/src/autocompile.c b/runtime/src/autocompile.c index 2018428e..a9b2a79e 100644 --- a/runtime/src/autocompile.c +++ b/runtime/src/autocompile.c @@ -15,6 +15,10 @@ # include #endif +#ifndef PSX_OVERLAY_FLAVOR +#define PSX_OVERLAY_FLAVOR 0 +#endif + static char s_cmd[4096]; /* large: the runtime-constructed bundled tcc cmd has * many absolute paths (python+script+recompiler+tcc+...) */ static char s_cwd[512]; @@ -855,6 +859,16 @@ int autocompile_request(void) { if (s_cache_dir[0]) SetEnvironmentVariableA("PSX_OVERLAY_CACHE_DIR", s_cache_dir); if (s_captures[0]) SetEnvironmentVariableA("PSX_OVERLAY_CAPTURES", s_captures); SetEnvironmentVariableA("PSX_OVERLAY_LIVE_AUTOCOMPILE", "1"); + /* Pin the FLAVOR the same way: the tool's --flavor defaults to 0 (play), + * so an instrumented (flavor 2) runtime spawned a compile whose shards it + * could never load — the loader's flavor guard rejected them and EVERY + * overlay ran interpreted, forever, on debug builds. The child must + * always write the flavor this runtime reads. */ + { + char flavor_buf[16]; + snprintf(flavor_buf, sizeof flavor_buf, "%d", (int)PSX_OVERLAY_FLAVOR); + SetEnvironmentVariableA("PSX_OVERLAY_FLAVOR", flavor_buf); + } /* cmd.exe /C resolves the command via PATH and supports the relative * paths in the configured line (cwd = project root). The WHOLE command is diff --git a/runtime/src/debug_server.c b/runtime/src/debug_server.c index cccf2daf..f2fd4794 100644 --- a/runtime/src/debug_server.c +++ b/runtime/src/debug_server.c @@ -8044,6 +8044,18 @@ static void handle_savestate(int id, const char *json) send_fmt("{\"id\":%d,\"ok\":true,\"op\":\"%s\",\"slot\":%d}", id, op, slot); } +/* Deterministic harness receipt: unlike the savestate acknowledgement above, + * this reports the safe-boundary completion generation after savestate_poll + * has actually applied or rejected the request. */ +static void handle_savestate_status(int id, const char *json) +{ + (void)json; + extern void savestate_status_json(char *buf, size_t cap); + char status[256]; + savestate_status_json(status, sizeof status); + send_fmt("{\"id\":%d,\"ok\":true,%s}", id, status); +} + static void handle_turbo(int id, const char *json) { int enabled = json_get_int(json, "enabled", -1); @@ -13514,6 +13526,7 @@ static const CmdEntry s_commands[] = { { "input_route_stop", handle_input_route_stop }, { "input_route_status",handle_input_route_status }, { "savestate", handle_savestate }, + { "savestate_status", handle_savestate_status }, { "turbo", handle_turbo }, { "turbo_state", handle_turbo_state }, { "pause", handle_pause }, diff --git a/runtime/src/main.cpp b/runtime/src/main.cpp index 5b8a85d0..faa78fe1 100644 --- a/runtime/src/main.cpp +++ b/runtime/src/main.cpp @@ -1119,7 +1119,8 @@ static std::string g_bezel_path; /* mod-owned OpenGL margin artwork */ static int g_fullscreen = 0; /* tri-state: 0 windowed, 1 borderless (desktop) * fullscreen, 2 exclusive fullscreen */ static int g_video_screen = 0; /* 0=raw,1=crt,2=composite,3=trinitron */ -static int g_video_win_w = 1280; /* window width (height follows aspect) */ +static int g_video_win_w = 0; /* 0 = fit the display; see clamp_window_aspect */ +static bool g_video_win_w_explicit = false; /* user chose a width */ static bool g_audio_spu_hq = false; /* SPU float-shadow (env overrides) */ static int g_audio_freq = 44100; /* host device request */ static int g_auto_skip_fmv = 0; /* skip FMVs the instant they're detected */ @@ -1474,9 +1475,18 @@ static int g_logical_w = 640; * the given aspect: height = width*den/num. */ static void clamp_window_aspect(int* w, int* h, int num, int den) { int width = *w; - if (width < 640) width = 640; SDL_Rect bounds; - if (SDL_GetDisplayUsableBounds(0, &bounds) == 0 && bounds.w > 0 && bounds.h > 0) { + const int have_bounds = + (SDL_GetDisplayUsableBounds(0, &bounds) == 0 && bounds.w > 0 && bounds.h > 0); + /* 0 = "fit the display". The old default was a hardcoded 1280, which on a + * 4K or 8K panel opens a small window in the corner and, worse, makes the + * image far smaller than the internal render resolution the user chose -- + * supersampling 16 rendering into a 1280-wide window throws almost all of + * it away. Fitting the usable bounds keeps the window proportional to the + * display it is actually on. An explicit width still wins. */ + if (width <= 0) width = have_bounds ? bounds.w : 1280; + if (width < 640) width = 640; + if (have_bounds) { if (width > bounds.w) width = bounds.w; if (width * den / num > bounds.h) width = bounds.h * num / den; } @@ -11026,6 +11036,7 @@ int main(int argc, char** argv) { #endif if (us.has_supersampling) g_video_scale = us.supersampling; if (us.has_window_width) g_video_win_w = us.window_width; + if (us.has_window_width && us.window_width > 0) g_video_win_w_explicit = true; if (us.has_antialiasing) g_video_aa = us.antialiasing; if (us.has_texture_filter) g_video_texfilter = us.texture_filter; if (us.has_geometry_correction) @@ -11439,6 +11450,24 @@ int main(int argc, char** argv) { (!std::getenv("PSX_NO_LAUNCHER") && !force_no_launcher && !skip_launcher_setting); if (want_launcher) { launcher_boot_timing_mark("host:before_sdl_init"); + /* Per-monitor DPI awareness, BEFORE any SDL_Init. + * + * Without it Windows virtualises everything this process sees: on a + * 7680x4320 panel at 400% scaling SDL_GetDisplayUsableBounds reports + * 1920x1032, so the window is clamped to roughly 1376 LOGICAL pixels and + * opens as a small box, while the desktop compositor then upscales it. + * The internal render resolution is unaffected -- which is the trap: the + * game renders at supersampling 16 and the result is thrown away scaling + * a 1376-wide window up to an 8K display. + * + * permonitorv2 makes SDL report physical pixels, so the window sizes + * against the real panel and the drawable matches it 1:1. */ +#ifdef SDL_HINT_WINDOWS_DPI_AWARENESS + SDL_SetHint(SDL_HINT_WINDOWS_DPI_AWARENESS, "permonitorv2"); +#endif +#ifdef SDL_HINT_WINDOWS_DPI_SCALING + SDL_SetHint(SDL_HINT_WINDOWS_DPI_SCALING, "0"); +#endif if (SDL_Init(SDL_INIT_VIDEO | SDL_INIT_GAMECONTROLLER) == 0) { launcher_boot_timing_mark("host:after_sdl_init"); recomp_launcher_set_preserve_sdl(1); @@ -12653,6 +12682,24 @@ session_reboot: * default). Enable the HIDAPI Xbox driver so HIDAPI handles Xbox pads too. */ SDL_SetHint(SDL_HINT_JOYSTICK_HIDAPI_XBOX, "1"); if (!SDL_WasInit(SDL_INIT_VIDEO | SDL_INIT_GAMECONTROLLER)) { + /* Per-monitor DPI awareness, BEFORE any SDL_Init. + * + * Without it Windows virtualises everything this process sees: on a + * 7680x4320 panel at 400% scaling SDL_GetDisplayUsableBounds reports + * 1920x1032, so the window is clamped to roughly 1376 LOGICAL pixels and + * opens as a small box, while the desktop compositor then upscales it. + * The internal render resolution is unaffected -- which is the trap: the + * game renders at supersampling 16 and the result is thrown away scaling + * a 1376-wide window up to an 8K display. + * + * permonitorv2 makes SDL report physical pixels, so the window sizes + * against the real panel and the drawable matches it 1:1. */ +#ifdef SDL_HINT_WINDOWS_DPI_AWARENESS + SDL_SetHint(SDL_HINT_WINDOWS_DPI_AWARENESS, "permonitorv2"); +#endif +#ifdef SDL_HINT_WINDOWS_DPI_SCALING + SDL_SetHint(SDL_HINT_WINDOWS_DPI_SCALING, "0"); +#endif if (SDL_Init(SDL_INIT_VIDEO | SDL_INIT_GAMECONTROLLER) != 0) { std::fprintf(stderr, "SDL_Init failed: %s\n", SDL_GetError()); return 1; @@ -12737,6 +12784,22 @@ session_reboot: } psx_apply_window_icon(sdl_window, argv[0]); + /* Maximise instead of computing the frame size ourselves. + * + * clamp_window_aspect fits the CLIENT area to the usable bounds, but a + * window is client plus title bar and borders, so fitting the client to a + * full-height display produced a window taller than the screen that hung + * off the top. Deriving the decoration size first does not work either: + * SDL_GetWindowBordersSize reports nothing useful before the window is + * shown, so the correction silently did not apply. + * + * The window manager already solves this exactly. Maximise and let it fit + * the work area, decorations and taskbar included. Only when the request + * was "fit the display" (window_width unset) -- an explicit width is a + * deliberate choice and is left alone. */ + if (!g_fullscreen && !g_video_win_w_explicit) + SDL_MaximizeWindow(sdl_window); + /* Host refresh: if the panel is within ~2% of 60 Hz, record it so driver * vsync can own cadence (pacer skipped). Non-~60 Hz and unknown refresh * (common on Wayland) keep PSX 59.94 Hz pacing and force swap interval 0 diff --git a/runtime/src/savestate.c b/runtime/src/savestate.c index f8a09e94..f2f753ee 100644 --- a/runtime/src/savestate.c +++ b/runtime/src/savestate.c @@ -55,6 +55,11 @@ static int s_save_pending = -1; /* slot, or -1 */ static int s_load_pending = -1; static int s_load_completed = 0; static int s_load_failed = 0; +static uint32_t s_status_generation = 0; +static int s_status_pending = 0; +static int s_status_last_ok = 0; +static int s_status_last_load = 0; +static int s_status_last_slot = -1; static int s_save_failed = 0; static uint32_t s_last_save_pc = 0; static int s_save_defer_slot = -1; @@ -587,6 +592,9 @@ static int request_save_inner(int slot) { s_last_save_pc = 0; /* block netplay transfer until this write stamps a PC */ s_save_defer_slot = -1; s_save_pending = slot; + s_status_pending = 1; + s_status_last_load = 0; + s_status_last_slot = slot; return 1; } @@ -603,6 +611,9 @@ static int request_load_inner(int slot) { s_load_failed = 0; s_load_completed = 0; s_load_pending = slot; + s_status_pending = 1; + s_status_last_load = 1; + s_status_last_slot = slot; return 1; } @@ -658,6 +669,16 @@ int savestate_pending(void) { return (s_save_pending >= 0 || s_load_pending >= 0) ? 1 : 0; } +void savestate_status_json(char* buf, size_t cap) { + if (!buf || cap == 0) return; + snprintf(buf, cap, + "\"generation\":%u,\"pending\":%d,\"last_ok\":%d," + "\"last_op\":\"%s\",\"last_slot\":%d", + (unsigned)s_status_generation, s_status_pending, + s_status_last_ok, s_status_last_load ? "load" : "save", + s_status_last_slot); +} + int savestate_take_load_completed(void) { int v = s_load_completed; s_load_completed = 0; @@ -705,6 +726,9 @@ void savestate_poll(CPUState* cpu, uint32_t resume_pc) { s_save_defer_slot = -1; s_last_save_pc = 0; s_save_failed = 1; + s_status_pending = 0; + s_status_last_ok = 0; + s_status_generation++; fprintf(stderr, "savestate: SAVE FAILED slot %d — no safe resume PC " "(hint=0x%08X)\n", @@ -733,12 +757,18 @@ void savestate_poll(CPUState* cpu, uint32_t resume_pc) { s_last_save_pc = 0; s_save_failed = 1; } + s_status_pending = 0; + s_status_last_ok = ok ? 1 : 0; + s_status_generation++; fprintf(stderr, "savestate: %s slot %d @ pc=0x%08X -> %s\n", ok ? "SAVED" : "SAVE FAILED", slot, (unsigned)pc, path); psx_frontend_on_savestate_notify(0, slot, ok); } else { s_last_save_pc = 0; s_save_failed = 1; + s_status_pending = 0; + s_status_last_ok = 0; + s_status_generation++; psx_frontend_on_savestate_notify(0, slot, 0); } } @@ -788,6 +818,11 @@ void savestate_poll(CPUState* cpu, uint32_t resume_pc) { s_load_failed = 1; psx_frontend_on_savestate_notify(1, slot, 0); } + if (!loaded) { + s_status_pending = 0; + s_status_last_ok = 0; + s_status_generation++; + } if (loaded) { t_after_boot = savestate_mono_ms(); psx_cycles_resync_after_restore(cpu); @@ -801,6 +836,9 @@ void savestate_poll(CPUState* cpu, uint32_t resume_pc) { cdrom_accelerate_after_savestate(); /* Netplay post-load barrier observes this before the longjmp. */ s_load_completed = 1; + s_status_pending = 0; + s_status_last_ok = 1; + s_status_generation++; /* Restage FBO/present latch so the restored frame is visible * immediately (avoids disabled-display blank latch + stale smooth). */ psx_frontend_on_savestate_loaded(); diff --git a/runtime/tests/test_savestate_status_protocol.py b/runtime/tests/test_savestate_status_protocol.py new file mode 100644 index 00000000..88a8764f --- /dev/null +++ b/runtime/tests/test_savestate_status_protocol.py @@ -0,0 +1,14 @@ +"""Guard the deterministic savestate completion receipt used by load gates.""" +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[1] +HEADER = (ROOT / "include/savestate.h").read_text(encoding="utf-8") +STATE = (ROOT / "src/savestate.c").read_text(encoding="utf-8") +SERVER = (ROOT / "src/debug_server.c").read_text(encoding="utf-8") + +assert "void savestate_status_json(char* buf, size_t cap);" in HEADER +assert '\\"generation\\"' in STATE and '\\"pending\\"' in STATE +assert "s_status_generation++" in STATE +assert '"savestate_status"' in SERVER +assert "savestate_status_json(status, sizeof status)" in SERVER +print("savestate status protocol guard passed") diff --git a/tools/compile_overlays.py b/tools/compile_overlays.py index b3f7a3c0..d9d8105f 100644 --- a/tools/compile_overlays.py +++ b/tools/compile_overlays.py @@ -5259,6 +5259,19 @@ def main(): if _env_cap != args.captures: print(f'[cache] PSX_OVERLAY_CAPTURES overrides --captures: {_env_cap}') args.captures = _env_cap + # Flavor is pinned by the spawning runtime the same way as the cache + # locations: --flavor defaults to 0 (play), so an instrumented (flavor 2) + # runtime used to spawn compiles whose shards its loader then rejected — + # every overlay ran interpreted on debug builds, silently. + _env_flavor = os.environ.get('PSX_OVERLAY_FLAVOR') + if _env_flavor: + try: + _fl = int(_env_flavor, 0) + except ValueError: + _fl = None + if _fl is not None and _fl != args.flavor: + print(f'[cache] PSX_OVERLAY_FLAVOR overrides --flavor: {_fl}') + args.flavor = _fl if not args.captures: ap.error('no captures file: set PSX_OVERLAY_CAPTURES (runtime injects it) ' 'or pass --captures for manual/offline use') diff --git a/tools/new_project_layout/fetch_boxart.py b/tools/new_project_layout/fetch_boxart.py index edc4cd80..8eebffff 100755 --- a/tools/new_project_layout/fetch_boxart.py +++ b/tools/new_project_layout/fetch_boxart.py @@ -1,12 +1,13 @@ #!/usr/bin/env python3 -"""Fetch PS1 Named_Boxarts from libretro-thumbnails and write launcher TGA. +"""Fetch PS1 Named_Boxarts from libretro-thumbnails (PNG + launcher TGA). Sources (tried in order): https://thumbnails.libretro.com/Sony - PlayStation/Named_Boxarts/.png https://raw.githubusercontent.com/libretro-thumbnails/Sony_-_PlayStation/master/Named_Boxarts/.png -Writes 32-bit uncompressed TGA (bottom-up BGRA) for recomp-ui LAUNCHER_BOXART. -Also writes launcher_assets/img/BOXART_SOURCE.txt with URL + attribution. +Writes the original PNG (README) and a 32-bit uncompressed TGA (bottom-up BGRA) +for recomp-ui LAUNCHER_BOXART. Also writes launcher_assets/img/BOXART_SOURCE.txt +with URL + attribution. Requires network. Prefer Pillow when installed; otherwise decodes common 8-bit RGBA/RGB PNGs with the stdlib. @@ -120,7 +121,7 @@ def try_fetch(names: list[str]) -> tuple[bytes, str, str]: except (urllib.error.HTTPError, urllib.error.URLError, TimeoutError, RuntimeError) as e: last_err = e continue - raise SystemExit( + raise FileNotFoundError( f"boxart not found for candidates {names[:8]}… ({last_err})" ) @@ -241,6 +242,40 @@ def write_tga_bgra(path: Path, width: int, height: int, rgba: bytes) -> None: path.write_bytes(bytes(header) + bytes(bgra)) +def fetch_to_paths( + tga_path: Path, + *, + cue_stem: str = "", + display_name: str = "", + extra_names: list[str] | None = None, + png_path: Path | None = None, +) -> tuple[Path, Path, str]: + """Fetch libretro Named_Boxarts PNG, write PNG + TGA + BOXART_SOURCE.txt.""" + names = candidate_names(cue_stem, display_name, *(extra_names or [])) + if not names: + raise ValueError("pass cue_stem and/or display_name") + png, url, matched = try_fetch(names) + tga_path = Path(tga_path) + png_path = Path(png_path) if png_path else tga_path.with_suffix(".png") + png_path.parent.mkdir(parents=True, exist_ok=True) + png_path.write_bytes(png) + try: + w, h, rgba = png_to_rgba(png) + write_tga_bgra(tga_path, w, h, rgba) + except SystemExit as exc: + raise RuntimeError(str(exc) or "PNG decode failed") from exc + src_path = tga_path.parent / "BOXART_SOURCE.txt" + src_path.write_text( + "Box art sourced from libretro-thumbnails (Named_Boxarts).\n" + f"Matched name: {matched}\n" + f"URL: {url}\n" + "https://github.com/libretro-thumbnails/libretro-thumbnails\n" + "PNG is for README; TGA is for recomp-ui LAUNCHER_BOXART.\n", + encoding="utf-8", + ) + return tga_path, png_path, url + + def main() -> int: ap = argparse.ArgumentParser(description=__doc__) ap.add_argument( @@ -256,6 +291,11 @@ def main() -> int: default="", help="attribution file (default: next to TGA as BOXART_SOURCE.txt)", ) + ap.add_argument( + "--png-out", + default="", + help="output PNG path (default: same stem as --out with .png)", + ) args = ap.parse_args() names = candidate_names(args.cue_stem, args.display_name, *args.name) @@ -264,24 +304,28 @@ def main() -> int: return 2 print(f" searching libretro boxart ({len(names)} candidates)…", file=sys.stderr) - png, url, matched = try_fetch(names) - print(f" fetched {matched}", file=sys.stderr) - print(f" {url}", file=sys.stderr) - - w, h, rgba = png_to_rgba(png) out = Path(args.out) - write_tga_bgra(out, w, h, rgba) - print(f" wrote {out} ({w}x{h})", file=sys.stderr) - + png_out = Path(args.png_out) if args.png_out else out.with_suffix(".png") + try: + tga_path, png_path, url = fetch_to_paths( + out, + cue_stem=args.cue_stem, + display_name=args.display_name, + extra_names=list(args.name), + png_path=png_out, + ) + except (FileNotFoundError, ValueError, RuntimeError) as exc: + print(str(exc) or "boxart fetch failed", file=sys.stderr) + return 1 + print(f" wrote {png_path}", file=sys.stderr) + print(f" wrote {tga_path}", file=sys.stderr) + print(f" {url}", file=sys.stderr) src_path = Path(args.source_out) if args.source_out else out.parent / "BOXART_SOURCE.txt" - src_path.write_text( - "Box art sourced from libretro-thumbnails (Named_Boxarts).\n" - f"Matched name: {matched}\n" - f"URL: {url}\n" - "https://github.com/libretro-thumbnails/libretro-thumbnails\n" - "Convert: PNG → 32-bit TGA for recomp-ui LAUNCHER_BOXART.\n", - encoding="utf-8", - ) + if args.source_out: + src_path.write_text( + (tga_path.parent / "BOXART_SOURCE.txt").read_text(encoding="utf-8"), + encoding="utf-8", + ) print(f" wrote {src_path}", file=sys.stderr) return 0 diff --git a/tools/new_project_layout/fill_tokens.py b/tools/new_project_layout/fill_tokens.py index 012ca948..43e6c048 100755 --- a/tools/new_project_layout/fill_tokens.py +++ b/tools/new_project_layout/fill_tokens.py @@ -8,6 +8,12 @@ import re import sys from pathlib import Path +GITHUB_ABOUT_DESCRIPTION = ( + "Made with PSXrecomp, a Sony PlayStation game static recompiler ecosystem · " + "Part of the R.A.I.D. community" +) +GITHUB_ABOUT_HOMEPAGE = "https://discord.gg/Ad9BwSzctP" + def derive_zip_prefix(name: str) -> str: base = re.sub(r"(?i)recomp(iled)?$", "", name).strip() @@ -20,6 +26,60 @@ def derive_zip_prefix(name: str) -> str: return (slug or "game")[:20] +def sanitize_github_name(name: str) -> str: + """GitHub owner/repo slug: spaces → '-', drop illegal chars, keep [A-Za-z0-9._-].""" + s = (name or "").strip() + if not s: + return "" + s = re.sub(r"\s+", "-", s) + s = re.sub(r"[^A-Za-z0-9._-]+", "", s) + s = re.sub(r"-{2,}", "-", s) + s = s.strip(".-") + return s[:100] or "repo" + + +def install_dir_name(name: str) -> str: + """Local checkout / launcher ``apps/`` folder — same slug as the GitHub repo name. + + New-project wizard must create this folder (not the display name with spaces) + so Studio catalog matching and RetComM ``install_dir_name`` stay consistent. + """ + return sanitize_github_name(name) + + +def normalize_repo_key(name: str) -> str: + """Casefold + collapse whitespace/underscores to hyphens for folder/catalog match. + + Lets ``Wipeout 3 Special Edition Recomp`` match catalog + ``Wipeout-3-Special-Edition-Recomp`` / github short name without renaming. + """ + s = (name or "").strip().lower() + if not s: + return "" + s = re.sub(r"[\s_]+", "-", s) + s = re.sub(r"[^a-z0-9._-]+", "", s) + s = re.sub(r"-{2,}", "-", s) + return s.strip(".-") + + +def repo_match_keys(name: str) -> set[str]: + """Match keys for a folder / install_dir / github short name. + + Includes the normalized form and a form with a trailing ``-recomp`` / + ``-recompiled`` stripped so local ``… Recomp`` checkouts still match + catalog ``install_dir_name`` / github slugs that omit that suffix + (e.g. Klonoa). + """ + k = normalize_repo_key(name) + if not k: + return set() + out = {k} + stripped = re.sub(r"(-recomp(iled)?)+$", "", k).strip("-") + if stripped: + out.add(stripped) + return out + + def main() -> int: ap = argparse.ArgumentParser(description=__doc__) ap.add_argument("src") diff --git a/tools/new_project_layout/project_studio/bulkops.py b/tools/new_project_layout/project_studio/bulkops.py index 79a08e0c..a91ee709 100644 --- a/tools/new_project_layout/project_studio/bulkops.py +++ b/tools/new_project_layout/project_studio/bulkops.py @@ -509,7 +509,13 @@ def resolve_catalog_root(studio_toml: Path | None = None) -> Path | None: def _catalog_name_keys(catalog_root: Path) -> set[str]: - """Folder / github short-name keys (lowercased) for every catalog title.""" + """Folder / github short-name keys for every catalog title. + + Stores raw lowercased names plus ``repo_match_keys`` forms so spacey / + ``… Recomp`` legacy checkouts still match hyphenated ``install_dir_name``. + """ + from fill_tokens import repo_match_keys + index_path = catalog_root / "index.json" titles_dir = catalog_root / "titles" if not index_path.is_file() or not titles_dir.is_dir(): @@ -519,8 +525,16 @@ def _catalog_name_keys(catalog_root: Path) -> set[str]: except (OSError, json.JSONDecodeError): return set() keys: set[str] = set() + + def _add(raw: str) -> None: + s = (raw or "").strip() + if not s: + return + keys.add(s.lower()) + keys.update(repo_match_keys(s)) + for tid in ids: - keys.add(str(tid).lower()) + _add(str(tid)) path = titles_dir / f"{tid}.json" if not path.is_file(): continue @@ -530,7 +544,7 @@ def _catalog_name_keys(catalog_root: Path) -> set[str]: continue name = str(raw.get("install_dir_name") or "").strip() if name: - keys.add(name.lower()) + _add(name) for nest in ( ("release", "github"), ("build", "source", "github"), @@ -543,15 +557,141 @@ def _catalog_name_keys(catalog_root: Path) -> set[str]: break cur = cur[k] if ok and isinstance(cur, str) and "/" in cur: - keys.add(cur.rsplit("/", 1)[-1].lower()) + _add(cur.rsplit("/", 1)[-1]) homepage = str(raw.get("homepage") or "") if "github.com/" in homepage.lower(): tail = homepage.rstrip("/").split("github.com/")[-1] if "/" in tail: - keys.add(tail.split("/")[-1].lower()) + _add(tail.split("/")[-1]) return keys +def catalog_title_id_for_root( + root: Path, + *, + catalog_root: Path | None = None, +) -> str | None: + """Best-effort catalog title id for a local checkout (install_dir / github slug).""" + from fill_tokens import repo_match_keys + + cat = catalog_root or resolve_catalog_root() + if cat is None: + return None + index_path = cat / "index.json" + titles_dir = cat / "titles" + if not index_path.is_file() or not titles_dir.is_dir(): + return None + try: + ids = list(json.loads(index_path.read_text(encoding="utf-8")).get("titles") or []) + except (OSError, json.JSONDecodeError): + return None + + want = set(repo_match_keys(root.name)) + try: + from .gitops import _git + + code, url, _ = _git(root, "remote", "get-url", "origin") + if code == 0 and url: + u = url.strip().rstrip("/") + if u.endswith(".git"): + u = u[:-4] + if "github.com" in u.lower(): + # git@github.com:Owner/Repo or https://github.com/Owner/Repo + tail = u.replace(":", "/").split("github.com/")[-1] + if "/" in tail: + want.update(repo_match_keys(tail.split("/")[-1])) + except Exception: + pass + want.discard("") + + for tid in ids: + path = titles_dir / f"{tid}.json" + if not path.is_file(): + continue + try: + raw = json.loads(path.read_text(encoding="utf-8")) + except (OSError, json.JSONDecodeError): + continue + candidates = [str(tid), str(raw.get("install_dir_name") or "")] + gh = raw.get("release") if isinstance(raw.get("release"), dict) else {} + if isinstance(gh, dict) and isinstance(gh.get("github"), str) and "/" in gh["github"]: + candidates.append(gh["github"].rsplit("/", 1)[-1]) + homepage = str(raw.get("homepage") or "") + if "github.com/" in homepage.lower(): + tail = homepage.rstrip("/").split("github.com/")[-1] + if "/" in tail: + candidates.append(tail.split("/")[-1]) + for c in candidates: + if repo_match_keys(c) & want: + return str(tid) + return None + + +def upsert_studio_toml_title( + title_id: str, + project_root: Path, + *, + studio_toml: Path | None = None, +) -> str | None: + """Ensure ``[titles] "id" = "rel/path"`` in studio.toml. Returns note or None.""" + cfg = studio_toml or find_studio_toml() + if cfg is None or not (title_id or "").strip(): + return None + title_id = title_id.strip() + root = project_root.expanduser().resolve() + if not root.is_dir(): + return None + try: + rel = os.path.relpath(root, cfg.parent.resolve()).replace("\\", "/") + except ValueError: + rel = str(root) + key_line = f'"{title_id}" = "{rel}"' + text = cfg.read_text(encoding="utf-8") + # Already present with any path → leave alone (user may have customized). + existing = re.search( + rf'^[ \t]*"{re.escape(title_id)}"\s*=\s*("[^"]*"|\'[^\']*\')', + text, + re.M, + ) + if existing: + return f"studio.toml already maps {title_id}" + if re.search(r"^\[titles\]\s*$", text, re.M): + # Append after [titles] block's last assignment, or right after header. + lines = text.splitlines(keepends=True) + out: list[str] = [] + in_titles = False + inserted = False + i = 0 + while i < len(lines): + line = lines[i] + s = line.strip() + if s.startswith("[") and s.endswith("]"): + if in_titles and not inserted: + out.append(key_line + "\n") + inserted = True + in_titles = s == "[titles]" + out.append(line) + i += 1 + continue + out.append(line) + i += 1 + if in_titles and not inserted: + if out and not out[-1].endswith("\n"): + out[-1] = out[-1] + "\n" + out.append(key_line + "\n") + inserted = True + if not inserted: + return None + cfg.write_text("".join(out), encoding="utf-8") + else: + suffix = "" if text.endswith("\n") or not text else "\n" + cfg.write_text( + text + suffix + "\n[titles]\n" + key_line + "\n", + encoding="utf-8", + ) + return f"studio.toml [titles] += {title_id}" + + def _studio_title_paths(studio_toml: Path | None = None) -> set[Path]: cfg = studio_toml or find_studio_toml() if cfg is None: @@ -576,6 +716,8 @@ def repo_has_catalog_entry( studio_toml: Path | None = None, ) -> bool: """True if this indexed checkout maps to a retcomm-catalog title.""" + from fill_tokens import repo_match_keys + try: root = entry.resolved() except OSError: @@ -589,12 +731,34 @@ def repo_has_catalog_entry( keys = _catalog_name_keys(cat) if not keys: return False - basename = root.name.lower() - if basename in keys: + + def _hits(label: str) -> bool: + s = (label or "").strip() + if not s: + return False + if s.lower() in keys: + return True + return bool(repo_match_keys(s) & keys) + + if _hits(root.name): return True - name = (entry.name or "").strip().lower() - if name and name in keys: + if _hits(entry.name or ""): return True + # GitHub origin short name (catalog often matches repo slug, not local folder). + try: + from .gitops import _git + + code, url, _ = _git(root, "remote", "get-url", "origin") + if code == 0 and url: + u = url.strip().rstrip("/") + if u.endswith(".git"): + u = u[:-4] + if "github.com" in u.lower(): + tail = u.replace(":", "/").split("github.com/")[-1] + if "/" in tail and _hits(tail.split("/")[-1]): + return True + except Exception: + pass return False diff --git a/tools/new_project_layout/project_studio/cli.py b/tools/new_project_layout/project_studio/cli.py index 87aabf30..cc2a65ac 100644 --- a/tools/new_project_layout/project_studio/cli.py +++ b/tools/new_project_layout/project_studio/cli.py @@ -62,6 +62,8 @@ def _options_from_args(args: argparse.Namespace) -> MigrateOptions: boot_exe=args.boot_exe, players=args.players, zip_prefix=args.zip_prefix, + github_owner=getattr(args, "github_owner", None) or None, + github_repo=getattr(args, "github_repo", None) or None, window_title=args.window_title, enable_recomp_ui=not args.no_recomp_ui, enable_wizard=not args.no_wizard, @@ -152,10 +154,394 @@ def cmd_ops(_: argparse.Namespace) -> int: def cmd_gui(args: argparse.Namespace) -> int: - from project_studio.gui import run_gui + """Launch the native Dear ImGui Studio binary when available.""" + import os + import shutil + import subprocess root = Path(args.root).expanduser().resolve() if args.root else None - return run_gui(initial_root=root) + candidates: list[Path] = [] + env_bin = (os.environ.get("RETCOMM_STUDIO_BIN") or "").strip() + if env_bin: + candidates.append(Path(env_bin)) + here = Path(__file__).resolve() + # tools/new_project_layout/project_studio/cli.py → repo root + repo = here.parents[3] if len(here.parents) > 3 else here.parents[2] + for rel in ( + Path("build") / "RetComM-Studio", + Path("build") / "retcomm-studio", + Path("build-release") / "RetComM-Studio", + Path("out") / "bin" / "RetComM-Studio", + ): + candidates.append(repo / rel) + which = shutil.which("RetComM-Studio") or shutil.which("retcomm-studio") + if which: + candidates.append(Path(which)) + + for cand in candidates: + if cand.is_file() and os.access(cand, os.X_OK): + cmd = [str(cand)] + if root is not None: + os.environ["RETCOMM_STUDIO_INITIAL_ROOT"] = str(root) + return int(subprocess.call(cmd)) + + print( + "RetComM Studio GUI is Dear ImGui (native).\n" + "Build it first:\n" + " cmake -S . -B build && cmake --build build\n" + " ./build/RetComM-Studio\n" + "Or set RETCOMM_STUDIO_BIN to the executable path.", + file=sys.stderr, + ) + return 2 + + +def _index_to_json_dict(index) -> dict: + from project_studio.repo_index import labels_for_repos + + from project_studio.bulkops import filter_indexed_catalog + from project_studio.naming import configured_players + + data = index.to_dict() + labels = labels_for_repos(index.repos) + catalog_hits, _note = filter_indexed_catalog(index) + catalog_paths = {e.path for e in catalog_hits} + repos_out = [] + for entry, label in zip(index.repos, labels): + try: + players = configured_players(entry.resolved()) + except OSError: + players = 2 + repos_out.append( + { + "path": entry.path, + "name": entry.name, + "cue": entry.cue, + "label": label, + "in_catalog": entry.path in catalog_paths, + "players": players, + } + ) + data["repos"] = repos_out + return data + + +def _print_index_json(index) -> int: + print(json.dumps(_index_to_json_dict(index), indent=2)) + return 0 + + +def cmd_repos_list(_: argparse.Namespace) -> int: + from project_studio.repo_index import load_index + + return _print_index_json(load_index()) + + +def cmd_repos_add(args: argparse.Namespace) -> int: + from project_studio.repo_index import add_repo, load_index, looks_like_game_repo + + root = Path(args.path).expanduser().resolve() + if not root.is_dir(): + print(f"error: not a directory: {root}", file=sys.stderr) + return 2 + if not looks_like_game_repo(root) and not args.force: + print( + f"error: does not look like a game repo (pass --force): {root}", + file=sys.stderr, + ) + return 2 + idx = load_index() + add_repo(idx, root, name=args.name or "", cue=args.cue or "") + return _print_index_json(idx) + + +def cmd_repos_remove(args: argparse.Namespace) -> int: + from project_studio.repo_index import load_index, remove_repo + + idx = load_index() + if not remove_repo(idx, args.path): + print(f"error: not in index: {args.path}", file=sys.stderr) + return 2 + return _print_index_json(idx) + + +def cmd_repos_set_cue(args: argparse.Namespace) -> int: + from project_studio.repo_index import load_index, set_repo_cue + + idx = load_index() + entry = set_repo_cue(idx, args.path, args.cue) + if entry is None: + print(f"error: not in index: {args.path}", file=sys.stderr) + return 2 + return _print_index_json(idx) + + +def cmd_repos_set_last(args: argparse.Namespace) -> int: + from project_studio.repo_index import load_index, set_last + + idx = load_index() + set_last(idx, args.path) + return _print_index_json(idx) + + +def cmd_repos_set_flags(args: argparse.Namespace) -> int: + from project_studio.repo_index import load_index, save_index + + idx = load_index() + if args.catalog_only is not None: + idx.catalog_only = str(args.catalog_only).strip() in ("1", "true", "yes", "on") + if args.bulk_jobs is not None: + try: + jobs = int(args.bulk_jobs) + except (TypeError, ValueError): + jobs = idx.bulk_jobs + idx.bulk_jobs = max(1, min(4, jobs)) + if args.log_height is not None: + try: + h = int(args.log_height) + except (TypeError, ValueError): + h = idx.log_height + idx.log_height = max(100, min(800, h)) + save_index(idx) + return _print_index_json(idx) + + +def cmd_repos_filter_catalog(_: argparse.Namespace) -> int: + """JSON paths for indexed repos that map to retcomm-catalog titles.""" + from project_studio.bulkops import filter_indexed_catalog + from project_studio.repo_index import load_index + + hits, note = filter_indexed_catalog(load_index()) + print( + json.dumps( + { + "note": note, + "paths": [e.path for e in hits], + "labels": [e.label() for e in hits], + }, + indent=2, + ) + ) + return 0 + + +def cmd_repos_filter_catalog_contributors(_: argparse.Namespace) -> int: + """JSON paths for catalog repos where the viewer has WRITE+ (gh).""" + from project_studio.bulkops import filter_indexed_catalog_contributors + from project_studio.repo_index import load_index + + hits, note, logs = filter_indexed_catalog_contributors(load_index()) + for line in logs: + print(line, file=sys.stderr) + print( + json.dumps( + { + "note": note, + "paths": [e.path for e in hits], + "labels": [e.label() for e in hits], + "logs": logs, + }, + indent=2, + ) + ) + return 0 + + +def cmd_updates_check(args: argparse.Namespace) -> int: + from project_studio.updater import check_updates, check_updates_on_startup_enabled + + if getattr(args, "startup", False) and not check_updates_on_startup_enabled(): + if args.json: + print( + json.dumps( + { + "ok": True, + "skipped": True, + "message": "Startup update check disabled in config.", + "studio": {"available": False}, + "toolchain": {"available": False}, + }, + indent=2, + ) + ) + else: + print("Startup update check disabled in config.") + return 0 + + def prog(msg: str) -> None: + print(msg, file=sys.stderr) + + result = check_updates(on_progress=prog) + if args.json: + def comp(c) -> dict: + return { + "kind": c.kind, + "current": c.current, + "latest": c.latest, + "available": c.available, + "supported": c.supported, + "message": c.message, + "asset_name": c.asset_name, + } + + print( + json.dumps( + { + "ok": result.ok, + "message": result.message, + "studio": comp(result.studio), + "toolchain": comp(result.toolchain), + }, + indent=2, + ) + ) + else: + print(result.studio.message) + print(result.toolchain.message) + if result.message: + print(result.message) + return 0 if result.ok else 1 + + +def cmd_updates_apply(args: argparse.Namespace) -> int: + from project_studio.updater import apply_updates, check_updates + + def prog(msg: str) -> None: + print(msg, file=sys.stderr) + + result = check_updates(on_progress=prog) + summary, should_exit = apply_updates( + result, + update_studio=not args.toolchain_only, + update_toolchain=not args.studio_only, + on_progress=prog, + ) + print(summary) + if should_exit: + return 0 + return 0 + + +def cmd_updates_ensure_toolchain(args: argparse.Namespace) -> int: + from project_studio.updater import ensure_toolchain, resolve_toolchain_python + + def prog(msg: str) -> None: + print(msg, file=sys.stderr) + + ok, msg = ensure_toolchain(on_progress=prog) + print(f"[{'OK' if ok else 'FAIL'}] {msg}") + py = resolve_toolchain_python() + if py: + print(f"python: {py}") + return 0 if ok else 1 + + +def cmd_new_project(args: argparse.Namespace) -> int: + from project_studio.newproject import ( + NewProjectOptions, + index_new_project, + project_root_for, + run_new_project, + validate_options, + ) + + opts = NewProjectOptions( + name=(args.name or "").strip(), + disc=(args.disc or "").strip(), + parent_dir=(args.dir or ".").strip(), + bios=(getattr(args, "bios", None) or "").strip(), + boot_exe=(getattr(args, "boot_exe", None) or "").strip(), + players=int(getattr(args, "players", 2) or 2), + zip_prefix=(getattr(args, "zip_prefix", None) or "").strip(), + github_owner=(getattr(args, "github_owner", None) or "").strip(), + github_repo=(getattr(args, "github_repo", None) or "").strip(), + description=(getattr(args, "description", None) or "").strip(), + publisher=(getattr(args, "publisher", None) or "").strip(), + year=(getattr(args, "year", None) or "").strip(), + region=(getattr(args, "region", None) or "USA").strip(), + enable_recomp_ui=not bool(getattr(args, "no_recomp_ui", False)), + enable_wizard=not bool(getattr(args, "no_wizard", False)), + enable_netplay=bool(getattr(args, "enable_netplay", False)), + lobby_url=(getattr(args, "lobby_url", None) or "netplay.retcomm.net").strip(), + enable_ci=not bool(getattr(args, "no_ci", False)), + fetch_boxart=not bool(getattr(args, "no_fetch_boxart", False)), + stage_disc=bool(getattr(args, "stage_disc", False)), + do_generate=bool(getattr(args, "generate", False)), + do_build=bool(getattr(args, "enable_build", False)), + create_github=bool(getattr(args, "create_github", False)), + github_visibility=( + getattr(args, "github_visibility", None) or "private" + ).strip(), + psxrecomp_ref=(getattr(args, "psxrecomp_ref", None) or "master").strip(), + recomp_ui_ref=(getattr(args, "recomp_ui_ref", None) or "master").strip(), + recomp_net_ref=(getattr(args, "recomp_net_ref", None) or "").strip(), + rbengine_ref=(getattr(args, "rbengine_ref", None) or "").strip(), + dry_run=bool(getattr(args, "dry_run", False)), + ) + + if bool(getattr(args, "autofill_meta", False)): + from project_studio.discmeta import apply_hit_to_options, lookup_cue + + print("Looking up disc metadata (Redump / libretro / catalog)…", flush=True) + hit = lookup_cue(opts.disc) + for note in hit.notes: + print(f" meta: {note}", flush=True) + filled = apply_hit_to_options(opts, hit, only_empty=True) + if filled: + print(" filled: " + ", ".join(filled), flush=True) + else: + print(f" no empty fields filled (source={hit.source})", flush=True) + + errs = validate_options(opts) + if errs: + for e in errs: + print(f"error: {e}", file=sys.stderr) + return 2 + + def on_line(msg: str) -> None: + print(msg, flush=True) + + r = run_new_project(opts, on_line=on_line) + print(f"[{'OK' if r.ok else 'FAIL'}] {r.message}") + if not r.ok: + return 1 + if opts.dry_run: + return 0 + root = project_root_for(opts) + ir = index_new_project(root, name=opts.name, cue=opts.disc) + print(f"[{'OK' if ir.ok else 'FAIL'}] {ir.message}") + return 0 if ir.ok else 1 + + +def cmd_lookup_disc_meta(args: argparse.Namespace) -> int: + from project_studio.discmeta import lookup_cue, lookup_digests + + force = bool(getattr(args, "force_refresh", False)) + if getattr(args, "disc", None): + hit = lookup_cue(args.disc, force_refresh=force) + else: + hit = lookup_digests( + crc32=getattr(args, "crc32", "") or "", + md5=getattr(args, "md5", "") or "", + sha1=getattr(args, "sha1", "") or "", + serial=getattr(args, "serial", "") or "", + force_refresh=force, + ) + if args.json: + print(json.dumps(hit.to_dict(), indent=2)) + else: + print(f"source: {hit.source}") + print(f"name: {hit.name}") + print(f"serial: {hit.serial}") + print(f"region: {hit.region}") + print(f"players: {hit.players}") + print(f"publisher: {hit.publisher}") + print(f"year: {hit.year}") + print(f"description: {(hit.description or '')[:200]}") + print(f"crc32/md5/sha1: {hit.crc32} / {hit.md5} / {hit.sha1}") + for n in hit.notes: + print(f"note: {n}") + return 0 if hit.source != "none" else 1 def _root_or_die(args: argparse.Namespace) -> Path | None: @@ -166,6 +552,105 @@ def _root_or_die(args: argparse.Namespace) -> Path | None: return root +def cmd_git_branches(args: argparse.Namespace) -> int: + """List game + module branch names as JSON for Studio dropdowns.""" + from project_studio.gitops import ( + DEFAULT_PSXRECOMP_URL, + DEFAULT_RBENGINE_URL, + DEFAULT_RECOMP_NET_URL, + DEFAULT_RECOMP_UI_URL, + current_branch, + list_branches, + list_module_branches, + list_remote_head_branches, + resolve_module_dir, + ) + + def prefer_current(names: list[str], cur: str) -> list[str]: + cur = (cur or "").strip() + if not cur: + return names + rest = [n for n in names if n != cur] + return [cur, *rest] + + fetch = bool(getattr(args, "fetch", False)) + root_s = (getattr(args, "root", None) or "").strip() + out: dict = { + "game": [], + "psxrecomp": [], + "recomp-ui": [], + "recomp-net": [], + "rbengine": [], + "current": { + "game": "", + "psxrecomp": "", + "recomp-ui": "", + "recomp-net": "", + "rbengine": "", + }, + } + if root_s: + root = Path(root_s).expanduser().resolve() + if not root.is_dir(): + print(f"error: not a directory: {root}", file=sys.stderr) + return 2 + out["game"] = list_branches(root, remotes=True, fetch=fetch) + out["psxrecomp"] = list_module_branches( + root, "psxrecomp", remotes=True, fetch=fetch, url_fallback=DEFAULT_PSXRECOMP_URL + ) + out["recomp-ui"] = list_module_branches( + root, "recomp-ui", remotes=True, fetch=fetch, url_fallback=DEFAULT_RECOMP_UI_URL + ) + out["recomp-net"] = list_module_branches( + root, + "lib/recomp-net", + nested=True, + remotes=True, + fetch=fetch, + url_fallback=DEFAULT_RECOMP_NET_URL, + ) + out["rbengine"] = list_module_branches( + root, + "lib/retcomm-rbengine", + nested=True, + remotes=True, + fetch=fetch, + url_fallback=DEFAULT_RBENGINE_URL, + ) + cur = out["current"] + cur["game"] = current_branch(root) or "" + for key, path, nested in ( + ("psxrecomp", "psxrecomp", False), + ("recomp-ui", "recomp-ui", False), + ("recomp-net", "lib/recomp-net", True), + ("rbengine", "lib/retcomm-rbengine", True), + ): + sub = resolve_module_dir(root, path, nested=nested) + cur[key] = (current_branch(sub) or "") if sub is not None else "" + # Put live checkouts first so dropdowns open on the active branch. + out["game"] = prefer_current(out["game"], cur["game"]) + out["psxrecomp"] = prefer_current(out["psxrecomp"], cur["psxrecomp"]) + out["recomp-ui"] = prefer_current(out["recomp-ui"], cur["recomp-ui"]) + out["recomp-net"] = prefer_current(out["recomp-net"], cur["recomp-net"]) + out["rbengine"] = prefer_current(out["rbengine"], cur["rbengine"]) + else: + # New-project / no checkout: ls-remote default module URLs. + out["psxrecomp"] = list_remote_head_branches(DEFAULT_PSXRECOMP_URL) + out["recomp-ui"] = list_remote_head_branches(DEFAULT_RECOMP_UI_URL) + out["recomp-net"] = list_remote_head_branches(DEFAULT_RECOMP_NET_URL) + out["rbengine"] = list_remote_head_branches(DEFAULT_RBENGINE_URL) + # Sensible game-branch placeholders when scaffolding. + out["game"] = ["main", "master"] + + # Always offer (default) for nested/ref menus that support it. + for key in ("recomp-net", "rbengine"): + if "(default)" not in out[key]: + out[key] = ["(default)", *out[key]] + + print(json.dumps(out, indent=2)) + return 0 + + def cmd_git_status(args: argparse.Namespace) -> int: from project_studio.gitops import repo_status @@ -261,8 +746,8 @@ def cmd_git_ensure_nested(args: argparse.Namespace) -> int: def cmd_git_set_branch(args: argparse.Namespace) -> int: from project_studio.gitops import ( set_nested_branch, - set_repo_branch, set_submodule_branch, + switch_branch, ) root = _root_or_die(args) @@ -278,7 +763,7 @@ def cmd_git_set_branch(args: argparse.Namespace) -> int: root, args.submodule, args.branch, dry_run=args.dry_run ) else: - r = set_repo_branch( + r = switch_branch( root, args.branch, create=args.create, dry_run=args.dry_run ) print(f"[{'OK' if r.ok else 'FAIL'}] {r.message}") @@ -287,6 +772,158 @@ def cmd_git_set_branch(args: argparse.Namespace) -> int: return 0 if r.ok else 1 +def _git_target_flags(args: argparse.Namespace) -> dict[str, bool]: + """Resolve Game / Modules / Nested / psxrecomp targets for a single-root op. + + When no target flags are set, default to the game root (legacy ``git pull`` / + ``git switch --branch``). + """ + game = bool(getattr(args, "game", False)) + modules = bool(getattr(args, "modules", False)) + nested = bool(getattr(args, "nested", False)) + psx = bool(getattr(args, "psxrecomp", False)) + if not (game or modules or nested or psx): + game = True + return { + "game": game, + "modules": modules, + "nested": nested, + "psxrecomp": psx, + } + + +def cmd_git_switch(args: argparse.Namespace) -> int: + from project_studio.gitops import ( + CmdResult, + default_module_paths, + switch_branch, + switch_modules, + switch_psxrecomp, + ) + + root = _root_or_die(args) + if root is None: + return 2 + branch = (getattr(args, "branch", None) or "").strip() + create = bool(getattr(args, "create", False)) + set_tracking = not bool(getattr(args, "no_track", False)) + submodule = (getattr(args, "submodule", None) or "").strip() + psx_branch = (getattr(args, "psxrecomp_branch", None) or "").strip() + ui_branch = (getattr(args, "ui_branch", None) or "").strip() + net_branch = (getattr(args, "net_branch", None) or "").strip() + rb_branch = (getattr(args, "rb_branch", None) or "").strip() + per_module = bool(psx_branch or ui_branch or net_branch or rb_branch) + multi = bool(getattr(args, "game", False)) or ( + sum( + bool(x) + for x in ( + getattr(args, "modules", False), + getattr(args, "nested", False), + getattr(args, "psxrecomp", False), + ) + ) + > 1 + ) + + # Single-path legacy: --submodule PATH [--nested] [--branch] + if submodule and not multi and not per_module and not getattr(args, "game", False): + nested = bool(getattr(args, "nested", False)) + paths = [submodule] + branch_by_path = None + if branch: + branch_by_path = {submodule.strip().replace("\\", "/"): branch} + results = switch_modules( + root, + paths=paths, + nested=nested, + branch_by_path=branch_by_path, + create=create, + set_tracking=set_tracking, + dry_run=args.dry_run, + ) + return _print_module_results(results) + + t = _git_target_flags(args) + results: list[CmdResult] = [] + + if t["game"]: + if not branch: + print("error: --game / game root requires --branch NAME", file=sys.stderr) + return 2 + r = switch_branch(root, branch, create=create, dry_run=args.dry_run) + results.append(CmdResult(r.ok, r.message, r.detail)) + + if t["modules"]: + branch_by_path: dict[str, str] | None = None + paths = _module_paths_from_args(args) + if psx_branch or ui_branch: + branch_by_path = {} + if psx_branch: + branch_by_path["psxrecomp"] = psx_branch + if ui_branch: + branch_by_path["recomp-ui"] = ui_branch + paths = list(branch_by_path.keys()) + elif branch and not t["game"]: + # Legacy: ``git switch --modules --branch X`` applies X to all modules. + want = paths or list(default_module_paths(nested=False)) + branch_by_path = {p.strip().replace("\\", "/"): branch for p in want} + results.extend( + switch_modules( + root, + paths=paths, + nested=False, + branch_by_path=branch_by_path, + create=create, + set_tracking=set_tracking, + dry_run=args.dry_run, + ) + ) + elif t["psxrecomp"]: + use = psx_branch or branch + if not use: + print( + "error: --psxrecomp requires --branch or --psxrecomp-branch", + file=sys.stderr, + ) + return 2 + r = switch_psxrecomp(root, use, create=create, dry_run=args.dry_run) + results.append(CmdResult(r.ok, r.message, r.detail)) + + if t["nested"]: + branch_by_path = None + paths = _module_paths_from_args(args) + if net_branch or rb_branch: + branch_by_path = {} + if net_branch: + branch_by_path["lib/recomp-net"] = net_branch + if rb_branch: + branch_by_path["lib/retcomm-rbengine"] = rb_branch + paths = list(branch_by_path.keys()) + elif branch and not t["modules"] and not t["game"]: + # Legacy: ``git switch --nested --branch X`` applies X to all nested. + want = paths or list(default_module_paths(nested=True)) + branch_by_path = {p.strip().replace("\\", "/"): branch for p in want} + results.extend( + switch_modules( + root, + paths=paths, + nested=True, + branch_by_path=branch_by_path, + create=create, + set_tracking=set_tracking, + dry_run=args.dry_run, + ) + ) + + if not results: + print( + "error: nothing to switch (pass --game / --modules / --nested)", + file=sys.stderr, + ) + return 2 + return _print_module_results(results) + + def cmd_git_update_submodules(args: argparse.Namespace) -> int: from project_studio.gitops import update_submodules @@ -353,92 +990,319 @@ def _print_module_results(results: list) -> int: return 1 if failed else 0 +def _bulk_select_from_args(args: argparse.Namespace) -> list[str] | None: + raw = getattr(args, "select", None) or "" + parts = [p.strip() for p in raw.split(",") if p.strip()] + return parts or None + + +def _bulk_repos_or_die(args: argparse.Namespace): + from project_studio.bulkops import indexed_repos + + repos = indexed_repos(select=_bulk_select_from_args(args)) + if not repos: + print("No indexed repos matched (Add… in GUI, or --select filter).") + return None + return repos + + +def _bulk_targets_from_args(args: argparse.Namespace) -> dict: + game = bool(getattr(args, "game", False)) + modules = bool(getattr(args, "modules", False)) + psx = bool(getattr(args, "psxrecomp", False)) + nested = bool(getattr(args, "nested", False)) + # Default: game root only when nothing specified. + if not (game or modules or psx or nested): + game = True + return { + "game": game, + "modules": modules, + "psxrecomp": psx, + "nested": nested, + } + + +def cmd_git_bulk_status(args: argparse.Namespace) -> int: + from project_studio.bulkops import bulk_status + + repos = _bulk_repos_or_die(args) + if repos is None: + return 2 + return _print_module_results(bulk_status(repos)) + + +def cmd_git_bulk_pull(args: argparse.Namespace) -> int: + from project_studio.bulkops import bulk_pull + + repos = _bulk_repos_or_die(args) + if repos is None: + return 2 + return _print_module_results( + bulk_pull( + repos, + **_bulk_targets_from_args(args), + mode=getattr(args, "mode", None) or "ff-only", + dirty=getattr(args, "dirty", None) or "fail", + dry_run=args.dry_run, + ) + ) + + +def cmd_git_bulk_push(args: argparse.Namespace) -> int: + from project_studio.bulkops import bulk_push + + repos = _bulk_repos_or_die(args) + if repos is None: + return 2 + return _print_module_results( + bulk_push( + repos, + **_bulk_targets_from_args(args), + dry_run=args.dry_run, + ) + ) + + +def cmd_git_bulk_commit(args: argparse.Namespace) -> int: + from project_studio.bulkops import bulk_commit + + repos = _bulk_repos_or_die(args) + if repos is None: + return 2 + t = _bulk_targets_from_args(args) + return _print_module_results( + bulk_commit( + repos, + args.message, + game=t["game"], + modules=t["modules"], + nested=t["nested"], + dry_run=args.dry_run, + ) + ) + + +def cmd_git_bulk_switch(args: argparse.Namespace) -> int: + from project_studio.bulkops import bulk_switch + + repos = _bulk_repos_or_die(args) + if repos is None: + return 2 + t = _bulk_targets_from_args(args) + # For switch, defaulting to game-only when no flags is surprising if they + # only passed --psxrecomp-branch — require at least one explicit target. + if not any( + ( + getattr(args, "game", False), + getattr(args, "modules", False), + getattr(args, "psxrecomp", False), + getattr(args, "nested", False), + ) + ): + print( + "error: pass --game / --modules / --psxrecomp / --nested " + "(which checkouts to switch)", + file=sys.stderr, + ) + return 2 + return _print_module_results( + bulk_switch( + repos, + game=t["game"], + modules=t["modules"], + psxrecomp=t["psxrecomp"], + nested=t["nested"], + game_branch=getattr(args, "branch", None) or "", + psxrecomp_branch=getattr(args, "psxrecomp_branch", None) or "", + recomp_ui_branch=getattr(args, "ui_branch", None) or "", + recomp_net_branch=getattr(args, "net_branch", None) or "", + rbengine_branch=getattr(args, "rb_branch", None) or "", + create=bool(getattr(args, "create", False)), + set_tracking=not bool(getattr(args, "no_track", False)), + dry_run=args.dry_run, + ) + ) + + def cmd_git_pull(args: argparse.Namespace) -> int: - from project_studio.gitops import pull, pull_modules, pull_psxrecomp + from project_studio.gitops import CmdResult, pull, pull_modules, pull_psxrecomp root = _root_or_die(args) if root is None: return 2 - if getattr(args, "psxrecomp", False): - r = pull_psxrecomp(root, dry_run=args.dry_run) - print(f"[{'OK' if r.ok else 'FAIL'}] {r.message}") - if r.detail: - print(r.detail) - return 0 if r.ok else 1 - if getattr(args, "modules", False) or getattr(args, "nested", False): - results = pull_modules( - root, - paths=_module_paths_from_args(args), - nested=bool(getattr(args, "nested", False)), - dry_run=args.dry_run, + mode = getattr(args, "mode", None) or "ff-only" + dirty = getattr(args, "dirty", None) or "fail" + t = _git_target_flags(args) + results: list[CmdResult] = [] + if t["game"]: + r = pull(root, mode=mode, dirty=dirty, dry_run=args.dry_run) + results.append(CmdResult(r.ok, r.message, r.detail)) + if t["modules"]: + results.extend( + pull_modules( + root, + paths=_module_paths_from_args(args), + nested=False, + mode=mode, + dirty=dirty, + dry_run=args.dry_run, + ) ) - return _print_module_results(results) - r = pull(root, dry_run=args.dry_run) - print(f"[{'OK' if r.ok else 'FAIL'}] {r.message}") - if r.detail: - print(r.detail) - return 0 if r.ok else 1 + elif t["psxrecomp"]: + r = pull_psxrecomp(root, mode=mode, dirty=dirty, dry_run=args.dry_run) + results.append(CmdResult(r.ok, r.message, r.detail)) + if t["nested"]: + results.extend( + pull_modules( + root, + paths=_module_paths_from_args(args), + nested=True, + mode=mode, + dirty=dirty, + dry_run=args.dry_run, + ) + ) + if not results: + print("error: nothing to pull (pass --game / --modules / --nested)", file=sys.stderr) + return 2 + return _print_module_results(results) def cmd_git_commit(args: argparse.Namespace) -> int: - from project_studio.gitops import commit_all, commit_modules + from project_studio.gitops import CmdResult, commit_all, commit_modules root = _root_or_die(args) if root is None: return 2 - if getattr(args, "modules", False) or getattr(args, "nested", False): - results = commit_modules( - root, - args.message, - paths=_module_paths_from_args(args), - nested=bool(getattr(args, "nested", False)), - dry_run=args.dry_run, + t = _git_target_flags(args) + results: list[CmdResult] = [] + if t["game"]: + r = commit_all(root, args.message, dry_run=args.dry_run) + results.append(CmdResult(r.ok, r.message, r.detail)) + if t["modules"]: + results.extend( + commit_modules( + root, + args.message, + paths=_module_paths_from_args(args), + nested=False, + dry_run=args.dry_run, + ) ) - return _print_module_results(results) - r = commit_all(root, args.message, dry_run=args.dry_run) - print(f"[{'OK' if r.ok else 'FAIL'}] {r.message}") - if r.detail: - print(r.detail) - return 0 if r.ok else 1 + if t["nested"]: + results.extend( + commit_modules( + root, + args.message, + paths=_module_paths_from_args(args), + nested=True, + dry_run=args.dry_run, + ) + ) + if not results: + print( + "error: nothing to commit (pass --game / --modules / --nested)", + file=sys.stderr, + ) + return 2 + return _print_module_results(results) def cmd_git_push(args: argparse.Namespace) -> int: - from project_studio.gitops import push, push_modules, push_psxrecomp + from project_studio.gitops import ( + CmdResult, + default_module_paths, + push, + push_modules, + push_psxrecomp, + ) root = _root_or_die(args) if root is None: return 2 branch = (getattr(args, "branch", None) or "").strip() - if getattr(args, "psxrecomp", False): - r = push_psxrecomp(root, branch=branch, dry_run=args.dry_run) - print(f"[{'OK' if r.ok else 'FAIL'}] {r.message}") - if r.detail: - print(r.detail) - return 0 if r.ok else 1 - if getattr(args, "modules", False) or getattr(args, "nested", False): + t = _git_target_flags(args) + results: list[CmdResult] = [] + if t["game"]: + r = push(root, branch=branch, dry_run=args.dry_run) + results.append(CmdResult(r.ok, r.message, r.detail)) + if t["modules"]: paths = _module_paths_from_args(args) branch_by_path = None if branch and paths and len(paths) == 1: branch_by_path = {paths[0]: branch} elif branch and not paths: - # Apply same branch hint to all default paths (detached rescue). - nested = bool(getattr(args, "nested", False)) - from project_studio.gitops import default_module_paths - - branch_by_path = {p: branch for p in default_module_paths(nested=nested)} - results = push_modules( - root, - paths=paths, - nested=bool(getattr(args, "nested", False)), - branch_by_path=branch_by_path, - dry_run=args.dry_run, + branch_by_path = {p: branch for p in default_module_paths(nested=False)} + results.extend( + push_modules( + root, + paths=paths, + nested=False, + branch_by_path=branch_by_path, + dry_run=args.dry_run, + ) ) - return _print_module_results(results) - r = push(root, branch=branch, dry_run=args.dry_run) - print(f"[{'OK' if r.ok else 'FAIL'}] {r.message}") - if r.detail: - print(r.detail) - return 0 if r.ok else 1 + elif t["psxrecomp"]: + r = push_psxrecomp(root, branch=branch, dry_run=args.dry_run) + results.append(CmdResult(r.ok, r.message, r.detail)) + if t["nested"]: + paths = _module_paths_from_args(args) + branch_by_path = None + if branch and paths and len(paths) == 1: + branch_by_path = {paths[0]: branch} + elif branch and not paths and not t["modules"] and not t["game"]: + branch_by_path = {p: branch for p in default_module_paths(nested=True)} + results.extend( + push_modules( + root, + paths=paths, + nested=True, + branch_by_path=branch_by_path, + dry_run=args.dry_run, + ) + ) + if not results: + print( + "error: nothing to push (pass --game / --modules / --nested)", + file=sys.stderr, + ) + return 2 + return _print_module_results(results) + + +def cmd_git_bulk_release(args: argparse.Namespace) -> int: + from project_studio.bulkops import bulk_release + + repos = _bulk_repos_or_die(args) + if repos is None: + return 2 + results = bulk_release( + repos, + version=getattr(args, "version", "") or "", + bump=getattr(args, "bump", "patch") or "patch", + publish=not bool(getattr(args, "no_publish", False)), + reuse_cached_emitters=not bool( + getattr(args, "no_reuse_cached_emitters", False) + ), + dry_run=bool(getattr(args, "dry_run", False)), + skip_missing_workflow=not bool(getattr(args, "strict", False)), + ) + return _print_module_results(results) + + +def cmd_git_bulk_install_ci(args: argparse.Namespace) -> int: + from project_studio.bulkops import bulk_install_ci + + repos = _bulk_repos_or_die(args) + if repos is None: + return 2 + results = bulk_install_ci( + repos, + force=bool(getattr(args, "force", False)), + push_remote=not bool(getattr(args, "no_push", False)), + dry_run=bool(getattr(args, "dry_run", False)), + ) + return _print_module_results(results) def cmd_git_release(args: argparse.Namespace) -> int: @@ -461,6 +1325,96 @@ def cmd_git_release(args: argparse.Namespace) -> int: return 0 if r.ok else 1 +def cmd_git_install_ci(args: argparse.Namespace) -> int: + from project_studio.gitops import install_and_push_release_ci + + root = _root_or_die(args) + if root is None: + return 2 + r = install_and_push_release_ci( + root, + zip_prefix=getattr(args, "zip_prefix", "") or "", + force=bool(getattr(args, "force", False)), + push_remote=not bool(getattr(args, "no_push", False)), + dry_run=args.dry_run, + ) + print(f"[{'OK' if r.ok else 'FAIL'}] {r.message}") + if r.detail: + print(r.detail) + return 0 if r.ok else 1 + + +def cmd_build_mingw(args: argparse.Namespace) -> int: + """Linux → Windows MinGW cross-build via scripts/build_windows_mingw.sh.""" + import json + import shutil + import subprocess + + script = _TOOLKIT / "scripts" / "build_windows_mingw.sh" + if not script.is_file(): + print(f"error: missing {script}", file=sys.stderr) + return 2 + + bash = shutil.which("bash") + if not bash: + print("error: bash not found (required for MinGW cross script)", file=sys.stderr) + return 2 + + cmd: list[str] = [bash, str(script)] + if getattr(args, "root", None): + cmd += ["--root", str(Path(args.root).expanduser().resolve())] + if getattr(args, "name", None): + cmd += ["--name", args.name] + if getattr(args, "last", False) or ( + not getattr(args, "root", None) and not getattr(args, "name", None) + ): + cmd.append("--last") + if args.build_dir and args.build_dir != "build-mingw": + cmd += ["--build-dir", args.build_dir] + if getattr(args, "package_only", False): + cmd.append("--package-only") + if getattr(args, "setup_host", False): + cmd.append("--setup-host") + if getattr(args, "package", False) and not getattr(args, "package_only", False): + cmd.append("--package") + if getattr(args, "dynamic", False): + cmd.append("--dynamic") + if getattr(args, "ensure", False) and not getattr(args, "package_only", False): + cmd.append("--ensure") + if getattr(args, "jobs", 0): + cmd += ["--jobs", str(args.jobs)] + if getattr(args, "dry_run", False): + cmd.append("--dry-run") + if getattr(args, "extra", ""): + cmd += ["--extra", args.extra] + + print("+", " ".join(cmd), flush=True) + r = subprocess.run(cmd, check=False) + # Studio Bundle+Export parses the JSON trailer / MINGW_ZIP line. + root = "" + if getattr(args, "root", None): + root = str(Path(args.root).expanduser().resolve()) + zip_path = "" + if root: + dist = Path(root) / "dist" + if dist.is_dir(): + zips = sorted( + dist.glob("*-windows-x64-mingw.zip"), + key=lambda p: p.stat().st_mtime, + reverse=True, + ) + if not zips: + zips = sorted(dist.glob("*.zip"), key=lambda p: p.stat().st_mtime, reverse=True) + if zips: + zip_path = str(zips[0].resolve()) + if zip_path and r.returncode == 0: + print(f"MINGW_ZIP={zip_path}", flush=True) + print(json.dumps({"ok": True, "zip": zip_path}), flush=True) + elif getattr(args, "package_only", False) or getattr(args, "package", False): + print(json.dumps({"ok": False, "zip": "", "error": "no zip produced"}), flush=True) + return int(r.returncode) + + def cmd_build_configure(args: argparse.Namespace) -> int: import shlex @@ -478,11 +1432,67 @@ def cmd_build_configure(args: argparse.Namespace) -> int: extra_args=extra, dry_run=args.dry_run, log=print, + ensure_bios=not bool(getattr(args, "skip_bios", False)), ) print(f"[{'OK' if r.ok else 'FAIL'}] {r.message}") return 0 if r.ok else 1 +def cmd_build_ensure_bios(args: argparse.Namespace) -> int: + from project_studio.buildops import ensure_bios_backends + + root = _root_or_die(args) + if root is None: + return 2 + r = ensure_bios_backends( + root, + force=bool(getattr(args, "force", False)), + include_scph1001=not bool(getattr(args, "openbios_only", False)), + dry_run=args.dry_run, + log=print, + ) + print(f"[{'OK' if r.ok else 'FAIL'}] {r.message}") + return 0 if r.ok else 1 + + +def cmd_build_generate(args: argparse.Namespace) -> int: + from project_studio.buildops import generate_rom_and_bios + + root = _root_or_die(args) + if root is None: + return 2 + r = generate_rom_and_bios( + root, + disc=getattr(args, "disc", "") or "", + bios=getattr(args, "bios", "") or "", + force_bios=not bool(getattr(args, "no_force_bios", False)), + dry_run=args.dry_run, + log=print, + ) + print(f"[{'OK' if r.ok else 'FAIL'}] {r.message}") + if r.detail and not r.ok: + print(r.detail) + return 0 if r.ok else 1 + + +def cmd_build_ensure_emitters(args: argparse.Namespace) -> int: + from project_studio.buildops import ensure_emitters + + root = _root_or_die(args) + if root is None: + return 2 + r = ensure_emitters( + root, + force=bool(getattr(args, "force", False)), + dry_run=args.dry_run, + log=print, + ) + print(f"[{'OK' if r.ok else 'FAIL'}] {r.message}") + if r.detail and not r.ok: + print(r.detail) + return 0 if r.ok else 1 + + def cmd_build_compile(args: argparse.Namespace) -> int: from project_studio.buildops import build @@ -511,6 +1521,11 @@ def cmd_build_run(args: argparse.Namespace) -> int: if root is None: return 2 extra = shlex.split(args.args, posix=os.name != "nt") if args.args else [] + + def _log(line: str) -> None: + # Piped into Studio — must flush or activity log stalls until buffer fill. + print(line, flush=True) + r = launch( root, build_dir=args.build_dir, @@ -518,9 +1533,10 @@ def cmd_build_run(args: argparse.Namespace) -> int: env_text=args.env or "", extra_args=extra, dry_run=args.dry_run, - log=print, + log=_log, + wait=True, ) - print(f"[{'OK' if r.ok else 'FAIL'}] {r.message}") + print(f"[{'OK' if r.ok else 'FAIL'}] {r.message}", flush=True) return 0 if r.ok else 1 @@ -528,7 +1544,7 @@ def cmd_build_stop(args: argparse.Namespace) -> int: from project_studio.buildops import stop_launch r = stop_launch() - print(f"[{'OK' if r.ok else 'FAIL'}] {r.message}") + print(f"[{'OK' if r.ok else 'FAIL'}] {r.message}", flush=True) return 0 if r.ok else 1 @@ -580,6 +1596,8 @@ def build_parser() -> argparse.ArgumentParser: p.add_argument("--boot-exe", help="Boot EXE basename (e.g. SCUS_944.23)") p.add_argument("--players", type=int, default=2) p.add_argument("--zip-prefix", help="CI/zip prefix") + p.add_argument("--github-owner", help="GitHub owner/org for README download badges") + p.add_argument("--github-repo", help="GitHub repo name for README download badges") p.add_argument("--window-title", help="WINDOW_TITLE override") p.add_argument("--enable-netplay", action="store_true") p.add_argument("--lobby-url", default="ws://netplay.retcomm.net:8765") @@ -618,10 +1636,138 @@ def build_parser() -> argparse.ArgumentParser: p_ops = sub.add_parser("ops", help="List op ids") p_ops.set_defaults(func=cmd_ops) - p_gui = sub.add_parser("gui", help="Open Project Studio GUI") + p_gui = sub.add_parser("gui", help="Open Project Studio GUI (Dear ImGui)") p_gui.add_argument("--root", default=None, help="Optional initial game root") p_gui.set_defaults(func=cmd_gui) + p_repos = sub.add_parser("repos", help="Manage local game-repo index") + repos_sub = p_repos.add_subparsers(dest="repos_cmd", required=True) + p_rl = repos_sub.add_parser("list", help="Print index as JSON") + p_rl.add_argument("--json", action="store_true", default=True, + help="Always JSON (accepted for ImGui runner symmetry)") + p_rl.set_defaults(func=cmd_repos_list) + p_ra = repos_sub.add_parser("add", help="Add a game repo") + p_ra.add_argument("--path", required=True) + p_ra.add_argument("--name", default="") + p_ra.add_argument("--cue", default="") + p_ra.add_argument("--force", action="store_true") + p_ra.add_argument("--json", action="store_true", default=True) + p_ra.set_defaults(func=cmd_repos_add) + p_rr = repos_sub.add_parser("remove", help="Remove a game repo") + p_rr.add_argument("--path", required=True) + p_rr.add_argument("--json", action="store_true", default=True) + p_rr.set_defaults(func=cmd_repos_remove) + p_rsc = repos_sub.add_parser("set-cue", help="Set .cue for an indexed repo") + p_rsc.add_argument("--path", required=True) + p_rsc.add_argument("--cue", required=True) + p_rsc.add_argument("--json", action="store_true", default=True) + p_rsc.set_defaults(func=cmd_repos_set_cue) + p_rsl = repos_sub.add_parser("set-last", help="Remember last-selected repo") + p_rsl.add_argument("--path", required=True) + p_rsl.add_argument("--json", action="store_true", default=True) + p_rsl.set_defaults(func=cmd_repos_set_last) + p_rsf = repos_sub.add_parser("set-flags", help="Update catalog_only / bulk_jobs / log_height") + p_rsf.add_argument("--catalog-only", default=None) + p_rsf.add_argument("--bulk-jobs", default=None) + p_rsf.add_argument("--log-height", default=None) + p_rsf.add_argument("--json", action="store_true", default=True) + p_rsf.set_defaults(func=cmd_repos_set_flags) + + p_rfc = repos_sub.add_parser( + "filter-catalog", + help="List indexed repos that have a retcomm-catalog entry (JSON paths)", + ) + p_rfc.add_argument("--json", action="store_true", default=True) + p_rfc.set_defaults(func=cmd_repos_filter_catalog) + + p_rfcc = repos_sub.add_parser( + "filter-catalog-contributors", + help="Catalog-backed indexed repos with WRITE+ for the current gh user", + ) + p_rfcc.add_argument("--json", action="store_true", default=True) + p_rfcc.set_defaults(func=cmd_repos_filter_catalog_contributors) + + p_upd = sub.add_parser("updates", help="Studio + toolchain updates") + upd_sub = p_upd.add_subparsers(dest="updates_cmd", required=True) + p_uc = upd_sub.add_parser("check", help="Check for updates") + p_uc.add_argument("--json", action="store_true") + p_uc.add_argument( + "--startup", + action="store_true", + help="Honor check_updates_on_startup from studio/hub config (skip when false)", + ) + p_uc.set_defaults(func=cmd_updates_check) + p_ua = upd_sub.add_parser("apply", help="Apply available updates") + p_ua.add_argument("--studio-only", action="store_true") + p_ua.add_argument("--toolchain-only", action="store_true") + p_ua.set_defaults(func=cmd_updates_apply) + p_uet = upd_sub.add_parser( + "ensure-toolchain", + help="Install cmake-clang-v1 if missing (github.com downloads, no API listing)", + ) + p_uet.set_defaults(func=cmd_updates_ensure_toolchain) + + p_np = sub.add_parser( + "new-project", + help="Run setup_project.sh/.ps1 (OS-routed) then index the new repo", + ) + p_np.add_argument("--name", required=True, help="Project folder / display name") + p_np.add_argument("--disc", required=True, help="Redump .cue path") + p_np.add_argument( + "--dir", + default=".", + help="Parent directory for the new repo (default: .)", + ) + p_np.add_argument("--bios", default="", help="Optional SCPH1001.BIN") + p_np.add_argument("--boot-exe", default="") + p_np.add_argument("--players", type=int, default=2) + p_np.add_argument("--zip-prefix", default="") + p_np.add_argument("--github-owner", default="") + p_np.add_argument("--github-repo", default="") + p_np.add_argument("--description", default="") + p_np.add_argument("--publisher", default="") + p_np.add_argument("--year", default="") + p_np.add_argument("--region", default="USA") + p_np.add_argument("--lobby-url", default="netplay.retcomm.net") + p_np.add_argument("--no-recomp-ui", action="store_true") + p_np.add_argument("--no-wizard", action="store_true") + p_np.add_argument("--enable-netplay", action="store_true") + p_np.add_argument("--no-ci", action="store_true") + p_np.add_argument("--no-fetch-boxart", action="store_true") + p_np.add_argument("--stage-disc", action="store_true") + p_np.add_argument("--generate", action="store_true") + p_np.add_argument("--enable-build", action="store_true") + p_np.add_argument("--create-github", action="store_true") + p_np.add_argument( + "--github-visibility", + choices=("private", "public", "internal"), + default="private", + ) + p_np.add_argument("--psxrecomp-ref", default="master") + p_np.add_argument("--recomp-ui-ref", default="master") + p_np.add_argument("--recomp-net-ref", default="") + p_np.add_argument("--rbengine-ref", default="") + p_np.add_argument("--dry-run", action="store_true") + p_np.add_argument( + "--autofill-meta", + action="store_true", + help="Fill empty players/description/publisher/year/region from disc digests", + ) + p_np.set_defaults(func=cmd_new_project) + + p_meta = sub.add_parser( + "lookup-disc-meta", + help="Lookup players/description/publisher/year/region from disc digests", + ) + p_meta.add_argument("--disc", default="", help="Redump .cue to probe") + p_meta.add_argument("--crc32", default="") + p_meta.add_argument("--md5", default="") + p_meta.add_argument("--sha1", default="") + p_meta.add_argument("--serial", default="") + p_meta.add_argument("--force-refresh", action="store_true") + p_meta.add_argument("--json", action="store_true") + p_meta.set_defaults(func=cmd_lookup_disc_meta) + # --- git / GitHub --- p_git = sub.add_parser("git", help="Git / GitHub ops on a game repo") git_sub = p_git.add_subparsers(dest="git_cmd", required=True) @@ -635,6 +1781,23 @@ def build_parser() -> argparse.ArgumentParser: p_gs.add_argument("--json", action="store_true") p_gs.set_defaults(func=cmd_git_status) + p_gbrs = git_sub.add_parser( + "branches", + help="List game/module branch names (JSON) for Studio dropdowns", + ) + p_gbrs.add_argument( + "--root", + default="", + help="Game repo root (omit to ls-remote default module URLs)", + ) + p_gbrs.add_argument( + "--fetch", + action="store_true", + help="git fetch --prune before listing local checkouts", + ) + p_gbrs.add_argument("--json", action="store_true", default=True) + p_gbrs.set_defaults(func=cmd_git_branches) + p_ge = git_sub.add_parser("ensure-submodules", help="Add psxrecomp + recomp-ui") add_git_root(p_ge) p_ge.add_argument("--psxrecomp-branch", default="master") @@ -652,13 +1815,13 @@ def build_parser() -> argparse.ArgumentParser: p_gb = git_sub.add_parser( "set-branch", - help="Set game branch (checkout) or submodule tracking branch", + help="Switch game branch (git switch) or set submodule tracking in .gitmodules", ) add_git_root(p_gb) p_gb.add_argument("--branch", required=True) p_gb.add_argument( "--submodule", - help="Submodule path (e.g. psxrecomp). Omit to checkout game branch.", + help="Submodule path (e.g. psxrecomp). Omit to switch the game working tree.", ) p_gb.add_argument( "--nested", @@ -668,10 +1831,82 @@ def build_parser() -> argparse.ArgumentParser: p_gb.add_argument( "--create", action="store_true", - help="Create/reset game branch (-B) when not using --submodule", + help="Create the game branch (-c) if it does not exist", ) p_gb.set_defaults(func=cmd_git_set_branch) + p_gsw = git_sub.add_parser( + "switch", + help="git switch on game / --modules / --nested (combinable)", + ) + add_git_root(p_gsw) + p_gsw.add_argument( + "--branch", + default="", + help="Game-root branch (with --game / default). " + "With --modules/--nested alone, omit to use each .gitmodules tracking branch.", + ) + p_gsw.add_argument( + "--game", + action="store_true", + help="Switch the game repo root (default when no other target flags)", + ) + p_gsw.add_argument( + "--modules", + action="store_true", + help="Switch game submodules (psxrecomp, recomp-ui)", + ) + p_gsw.add_argument( + "--nested", + action="store_true", + help="Switch nested libs inside psxrecomp (recomp-net, rbengine)", + ) + p_gsw.add_argument( + "--psxrecomp", + action="store_true", + help="Switch the psxrecomp checkout itself", + ) + p_gsw.add_argument( + "--psxrecomp-branch", + default="", + help="psxrecomp branch (with --modules or --psxrecomp)", + ) + p_gsw.add_argument( + "--ui-branch", + default="", + help="recomp-ui branch (with --modules)", + ) + p_gsw.add_argument( + "--net-branch", + default="", + help="lib/recomp-net branch (with --nested)", + ) + p_gsw.add_argument( + "--rb-branch", + default="", + help="lib/retcomm-rbengine branch (with --nested)", + ) + p_gsw.add_argument( + "--submodule", + default="", + help="Single module path (with --nested if inside psxrecomp)", + ) + p_gsw.add_argument( + "--paths", + help="Comma-separated module paths (defaults depend on --modules/--nested)", + ) + p_gsw.add_argument( + "--create", + action="store_true", + help="Create the branch (-c) if it does not exist locally or remotely", + ) + p_gsw.add_argument( + "--no-track", + action="store_true", + help="Do not update .gitmodules branch= when switching modules", + ) + p_gsw.set_defaults(func=cmd_git_switch) + p_gu = git_sub.add_parser("update-submodules", help="git submodule update") add_git_root(p_gu) p_gu.add_argument( @@ -706,9 +1941,14 @@ def build_parser() -> argparse.ArgumentParser: p_gpull = git_sub.add_parser( "pull", - help="git pull --ff-only (game root, --modules, --nested, or --psxrecomp)", + help="git pull (game / --modules / --nested; combinable)", ) add_git_root(p_gpull) + p_gpull.add_argument( + "--game", + action="store_true", + help="Pull the game repo root (default when no other target flags)", + ) p_gpull.add_argument( "--modules", action="store_true", @@ -728,14 +1968,33 @@ def build_parser() -> argparse.ArgumentParser: "--paths", help="Comma-separated module paths (defaults depend on --modules/--nested)", ) + p_gpull.add_argument( + "--mode", + choices=("ff-only", "rebase", "merge", "reset"), + default="ff-only", + help="ff-only (default), rebase, merge (--no-rebase), or reset " + "(fetch + reset --hard upstream = match origin)", + ) + p_gpull.add_argument( + "--dirty", + choices=("fail", "stash", "discard"), + default="fail", + help="If working tree dirty: fail (default), stash, or discard " + "(reset --hard HEAD). Ignored for --mode reset.", + ) p_gpull.set_defaults(func=cmd_git_pull) p_gc = git_sub.add_parser( "commit", - help="git add -A && git commit (game root, --modules, or --nested)", + help="git add -A && git commit (game / --modules / --nested; combinable)", ) add_git_root(p_gc) p_gc.add_argument("-m", "--message", required=True) + p_gc.add_argument( + "--game", + action="store_true", + help="Commit the game repo root (default when no other target flags)", + ) p_gc.add_argument( "--modules", action="store_true", @@ -754,9 +2013,14 @@ def build_parser() -> argparse.ArgumentParser: p_gpush = git_sub.add_parser( "push", - help="git push -u origin HEAD (game root, --modules, --nested, or --psxrecomp)", + help="git push -u origin HEAD (game / --modules / --nested; combinable)", ) add_git_root(p_gpush) + p_gpush.add_argument( + "--game", + action="store_true", + help="Push the game repo root (default when no other target flags)", + ) p_gpush.add_argument( "--modules", action="store_true", @@ -783,6 +2047,157 @@ def build_parser() -> argparse.ArgumentParser: ) p_gpush.set_defaults(func=cmd_git_push) + def add_bulk_select(p: argparse.ArgumentParser) -> None: + p.add_argument( + "--select", + default="", + help="Comma-separated name/path filters (default: all indexed repos)", + ) + p.add_argument("--dry-run", action="store_true") + + def add_bulk_targets(p: argparse.ArgumentParser) -> None: + p.add_argument( + "--game", + action="store_true", + help="Operate on game repo root (default if no target flags)", + ) + p.add_argument( + "--modules", + action="store_true", + help="Operate on psxrecomp + recomp-ui", + ) + p.add_argument( + "--psxrecomp", + action="store_true", + help="Operate on the psxrecomp checkout", + ) + p.add_argument( + "--nested", + action="store_true", + help="Operate on nested libs (recomp-net, rbengine)", + ) + + p_gbs = git_sub.add_parser( + "bulk-status", + help="Status for indexed repos (project_studio_repos.json)", + ) + add_bulk_select(p_gbs) + p_gbs.set_defaults(func=cmd_git_bulk_status) + + p_gbp = git_sub.add_parser( + "bulk-pull", + help="Pull indexed repos (see --mode / --dirty / targets)", + ) + add_bulk_select(p_gbp) + add_bulk_targets(p_gbp) + p_gbp.add_argument( + "--mode", + choices=("ff-only", "rebase", "merge", "reset"), + default="ff-only", + ) + p_gbp.add_argument( + "--dirty", + choices=("fail", "stash", "discard"), + default="fail", + ) + p_gbp.set_defaults(func=cmd_git_bulk_pull) + + p_gbpush = git_sub.add_parser( + "bulk-push", + help="Push indexed repos (game / modules / psxrecomp / nested)", + ) + add_bulk_select(p_gbpush) + add_bulk_targets(p_gbpush) + p_gbpush.set_defaults(func=cmd_git_bulk_push) + + p_gbc = git_sub.add_parser( + "bulk-commit", + help="Commit indexed repos (game / modules / nested)", + ) + add_bulk_select(p_gbc) + add_bulk_targets(p_gbc) + p_gbc.add_argument("-m", "--message", required=True) + p_gbc.set_defaults(func=cmd_git_bulk_commit) + + p_gbsw = git_sub.add_parser( + "bulk-switch", + help="git switch submodule / nested / game branches on indexed repos", + ) + add_bulk_select(p_gbsw) + add_bulk_targets(p_gbsw) + p_gbsw.add_argument( + "--branch", + default="", + help="Game-root branch (with --game)", + ) + p_gbsw.add_argument( + "--psxrecomp-branch", + default="", + help="psxrecomp branch (with --modules or --psxrecomp)", + ) + p_gbsw.add_argument( + "--ui-branch", + default="", + help="recomp-ui branch (with --modules)", + ) + p_gbsw.add_argument( + "--net-branch", + default="", + help="lib/recomp-net branch (with --nested)", + ) + p_gbsw.add_argument( + "--rb-branch", + default="", + help="lib/retcomm-rbengine branch (with --nested)", + ) + p_gbsw.add_argument( + "--create", + action="store_true", + help="Create the branch (-c) if missing", + ) + p_gbsw.add_argument( + "--no-track", + action="store_true", + help="Do not update .gitmodules branch= tracking", + ) + p_gbsw.set_defaults(func=cmd_git_bulk_switch) + + + p_gbr = git_sub.add_parser( + "bulk-release", + help="Dispatch release.yml on selected indexed repos (gh workflow run)", + ) + add_bulk_select(p_gbr) + p_gbr.add_argument("--version", default="", help="Empty = auto-bump per repo") + p_gbr.add_argument( + "--bump", choices=("patch", "minor", "major"), default="patch" + ) + p_gbr.add_argument("--no-publish", action="store_true") + p_gbr.add_argument("--no-reuse-cached-emitters", action="store_true") + p_gbr.add_argument( + "--strict", + action="store_true", + help="Fail hard when release.yml is missing (default: skip)", + ) + p_gbr.set_defaults(func=cmd_git_bulk_release) + + p_gbci = git_sub.add_parser( + "bulk-install-ci", + help="Install/push setup-host release.yml on selected indexed repos", + ) + add_bulk_select(p_gbci) + p_gbci.add_argument( + "--force", + action="store_true", + help="Overwrite an already-filled release.yml", + ) + p_gbci.add_argument( + "--no-push", + action="store_true", + help="Commit locally only (do not push)", + ) + p_gbci.set_defaults(func=cmd_git_bulk_install_ci) + p_gr = git_sub.add_parser("release", help="gh workflow run release.yml") add_git_root(p_gr) p_gr.add_argument("--version", default="", help="Empty = auto-bump") @@ -793,6 +2208,24 @@ def build_parser() -> argparse.ArgumentParser: p_gr.add_argument("--no-reuse-cached-emitters", action="store_true") p_gr.set_defaults(func=cmd_git_release) + p_gci = git_sub.add_parser( + "install-ci", + help="Write psxrecomp setup-release.yml, commit, and push", + ) + add_git_root(p_gci) + p_gci.add_argument("--zip-prefix", default="", help="CI asset zip prefix") + p_gci.add_argument( + "--force", + action="store_true", + help="Overwrite an already-filled release.yml", + ) + p_gci.add_argument( + "--no-push", + action="store_true", + help="Commit locally only (do not push)", + ) + p_gci.set_defaults(func=cmd_git_install_ci) + # --- local cmake build / launch --- p_build = sub.add_parser("build", help="Local CMake configure / build / launch") build_sub = p_build.add_subparsers(dest="build_cmd", required=True) @@ -802,13 +2235,119 @@ def build_parser() -> argparse.ArgumentParser: p.add_argument("--dry-run", action="store_true") p.add_argument("--build-dir", default="build-release") + p_bm = build_sub.add_parser( + "mingw", + help="Linux→Windows MinGW cross-build (local testing; no GitHub CI)", + ) + p_bm.add_argument( + "--root", + default="", + help="Game repository root (default: Studio index --last / --name)", + ) + p_bm.add_argument( + "--name", + default="", + help="Match a Studio-indexed title by name (substring)", + ) + p_bm.add_argument( + "--last", + action="store_true", + help="Use Studio index 'last' project (default when --root/--name omitted)", + ) + p_bm.add_argument("--build-dir", default="build-mingw") + p_bm.add_argument( + "--setup-host", + action="store_true", + help="CI-parity setup-host configure (FORCE_SETUP_HOST + wizard)", + ) + p_bm.add_argument( + "--package", + action="store_true", + help="Run scripts/package_setup_release.sh after build", + ) + p_bm.add_argument( + "--package-only", + action="store_true", + help="Skip configure/build; package existing MinGW build dir into dist/*.zip", + ) + p_bm.add_argument( + "--dynamic", + action="store_true", + help="PSX_STATIC_RUNTIME=OFF (ship SDL2/libgcc DLLs)", + ) + p_bm.add_argument( + "--ensure", + action="store_true", + help="Host ensure-emitters / ensure-bios / generate before cross-build", + ) + p_bm.add_argument("--jobs", type=int, default=0) + p_bm.add_argument("--extra", default="", help="Extra cmake -D args (one shell string)") + p_bm.add_argument("--dry-run", action="store_true") + p_bm.set_defaults(func=cmd_build_mingw) + p_bc = build_sub.add_parser("configure", help="cmake -S . -B ") add_build_root(p_bc) p_bc.add_argument("--build-type", default="Release") p_bc.add_argument("--generator", default=None, help="Empty = auto") p_bc.add_argument("--extra", default="", help="Extra cmake args (shell-quoted)") + p_bc.add_argument( + "--skip-bios", + action="store_true", + help="Do not auto-regen missing OpenBIOS before cmake", + ) p_bc.set_defaults(func=cmd_build_configure) + p_beb = build_sub.add_parser( + "ensure-bios", + help="Regen OpenBIOS (+ SCPH1001 if dump present) under psxrecomp/generated", + ) + add_build_root(p_beb) + p_beb.add_argument( + "--force", + action="store_true", + help="Regen even when generated backends already exist", + ) + p_beb.add_argument( + "--openbios-only", + action="store_true", + help="Skip SCPH1001 even if bios/SCPH1001.BIN exists", + ) + p_beb.set_defaults(func=cmd_build_ensure_bios) + + p_bg = build_sub.add_parser( + "generate", + help="psxrecomp_cli generate: BIOS backends + disc prepare + game C", + ) + add_build_root(p_bg) + p_bg.add_argument( + "--disc", + default="", + help="Source .cue (default: game.toml game.disc / indexed cue)", + ) + p_bg.add_argument( + "--bios", + default="", + help="Optional retail SCPH1001.BIN (omit for OpenBIOS only)", + ) + p_bg.add_argument( + "--no-force-bios", + action="store_true", + help="Skip BIOS regen when backends already exist", + ) + p_bg.set_defaults(func=cmd_build_generate) + + p_bee = build_sub.add_parser( + "ensure-emitters", + help="Build psxrecomp-game + psxrecomp-bios (psxrecomp_cli ensure-emitters)", + ) + add_build_root(p_bee) + p_bee.add_argument( + "--force", + action="store_true", + help="Rebuild even if emitter binaries already exist", + ) + p_bee.set_defaults(func=cmd_build_ensure_emitters) + p_bb = build_sub.add_parser("compile", help="cmake --build (alias: build)") add_build_root(p_bb) p_bb.add_argument("--target", default="psx-runtime") diff --git a/tools/new_project_layout/project_studio/detect.py b/tools/new_project_layout/project_studio/detect.py index 7db869ba..b60c0070 100644 --- a/tools/new_project_layout/project_studio/detect.py +++ b/tools/new_project_layout/project_studio/detect.py @@ -14,6 +14,13 @@ from .models import ( Severity, ) from .naming import boot_exe_from_game_toml, infer_project_name +from .readme_metrics import ( + boxart_png_present, + readme_has_boxart, + readme_has_launcher, + readme_has_metrics, + readme_has_raid, +) def _read(path: Path) -> str: @@ -749,6 +756,47 @@ def audit_project(root: Path) -> AuditReport: ) ) + # README download badges + boxart + RetComM Launcher (idempotent migrate op) + readme_path = root / "README.md" + readme_text = _read(readme_path) if readme_path.is_file() else "" + missing_readme: list[str] = [] + if not readme_path.is_file(): + missing_readme.append("README.md") + else: + if not readme_has_metrics(readme_text): + missing_readme.append("download badges") + if not readme_has_boxart(readme_text): + missing_readme.append("libretro boxart") + if not readme_has_launcher(readme_text): + missing_readme.append("RetComM Launcher section") + if not readme_has_raid(readme_text): + missing_readme.append("R.A.I.D. Discord footer") + if not (root / ".github" / "raid-discord.png").is_file(): + missing_readme.append(".github/raid-discord.png") + if not boxart_png_present(root): + missing_readme.append("launcher_assets/img/boxart.png") + if missing_readme: + checks.append( + CheckResult( + id="readme_metrics", + title="README download metrics / launcher / RAID / boxart", + status=CheckStatus.WARN, + severity=Severity.RECOMMENDED, + detail="Missing: " + ", ".join(missing_readme), + fix_op="patch_readme_metrics", + ) + ) + else: + checks.append( + CheckResult( + id="readme_metrics", + title="README download metrics / launcher / RAID / boxart", + status=CheckStatus.PASS, + severity=Severity.RECOMMENDED, + detail="Download badges, boxart, RetComM Launcher, and R.A.I.D. footer present.", + ) + ) + # Classify layout fails = {c.id for c in checks if c.status == CheckStatus.FAIL} rec_warns = [ diff --git a/tools/new_project_layout/project_studio/models.py b/tools/new_project_layout/project_studio/models.py index 640dbe44..94ad8eb1 100644 --- a/tools/new_project_layout/project_studio/models.py +++ b/tools/new_project_layout/project_studio/models.py @@ -83,6 +83,8 @@ class MigrateOptions: boot_exe: str | None = None players: int = 2 zip_prefix: str | None = None + github_owner: str | None = None + github_repo: str | None = None window_title: str | None = None enable_recomp_ui: bool = True enable_wizard: bool = True diff --git a/tools/new_project_layout/project_studio/naming.py b/tools/new_project_layout/project_studio/naming.py index 0921d3d1..a5e2a0e5 100644 --- a/tools/new_project_layout/project_studio/naming.py +++ b/tools/new_project_layout/project_studio/naming.py @@ -5,7 +5,7 @@ from __future__ import annotations import re from pathlib import Path -from fill_tokens import derive_zip_prefix +from fill_tokens import derive_zip_prefix, sanitize_github_name DEFAULT_LOBBY_HOST = "netplay.retcomm.net" @@ -220,6 +220,8 @@ def build_token_map( enable_wizard: bool = True, enable_netplay: bool = False, has_boxart: bool = False, + github_owner: str | None = None, + github_repo: str | None = None, ) -> dict[str, str]: cmake_name = project_cmake_name(name) title = window_title or window_title_from_name(name) @@ -294,6 +296,10 @@ def build_token_map( "ENV_PREFIX": env_prefix(cmake_name), "EXE_BASENAME": exe_basename(title), "ZIP_PREFIX": zp, + "GITHUB_OWNER": sanitize_github_name( + (github_owner or "").strip() or "TechnicallyComputers" + ), + "GITHUB_REPO": sanitize_github_name((github_repo or "").strip() or name), "DISC_HINT": f"your legally owned {game} disc", "GAME_TITLE": title, "PLAYERS": str(players), diff --git a/tools/new_project_layout/project_studio/newproject.py b/tools/new_project_layout/project_studio/newproject.py new file mode 100644 index 00000000..15315d3b --- /dev/null +++ b/tools/new_project_layout/project_studio/newproject.py @@ -0,0 +1,374 @@ +"""New-project wizard: drive setup_project.sh / .ps1 non-interactively.""" + +from __future__ import annotations + +import os +import platform +import shutil +import subprocess +import threading +from collections.abc import Callable +from dataclasses import dataclass +from pathlib import Path + +from .gitops import CmdResult, switch_modules +from .paths import toolkit_dir + + +@dataclass +class NewProjectOptions: + """Mirrors setup_project.sh / setup_project.ps1 CLI flags.""" + + name: str = "" + disc: str = "" + parent_dir: str = "" + bios: str = "" + boot_exe: str = "" + players: int = 2 + zip_prefix: str = "" + github_owner: str = "" + github_repo: str = "" + description: str = "" + publisher: str = "" + year: str = "" + region: str = "USA" + + enable_recomp_ui: bool = True + enable_wizard: bool = True + # GUI defaults: all on except GitHub; netplay follows players (≥2). + enable_netplay: bool = False + lobby_url: str = "netplay.retcomm.net" + enable_ci: bool = True + fetch_boxart: bool = True + stage_disc: bool = True + do_generate: bool = True + do_build: bool = True + create_github: bool = False + github_visibility: str = "private" # public | private | internal + + psxrecomp_ref: str = "master" + recomp_ui_ref: str = "master" + recomp_net_ref: str = "" # empty = keep psxrecomp pin + rbengine_ref: str = "" # post-setup switch only (script has no flag) + + dry_run: bool = False + + +def setup_script_paths() -> tuple[Path, Path]: + """Return (setup_project.sh, setup_project.ps1) under the toolkit.""" + base = toolkit_dir() + return base / "setup_project.sh", base / "setup_project.ps1" + + +def is_windows() -> bool: + return platform.system().lower().startswith("win") or os.name == "nt" + + +def project_folder_name(opts: NewProjectOptions) -> str: + """Checkout folder = GitHub/catalog install_dir slug (not display name with spaces).""" + from fill_tokens import install_dir_name, sanitize_github_name + + repo = (opts.github_repo or "").strip() + if repo: + return sanitize_github_name(repo) + return install_dir_name(opts.name or "") + + +def project_root_for(opts: NewProjectOptions) -> Path: + parent = Path(opts.parent_dir or ".").expanduser() + if not parent.is_absolute(): + parent = parent.resolve() + else: + parent = parent.resolve() + folder = project_folder_name(opts) + if not folder: + folder = (opts.name or "").strip() or "repo" + return (parent / folder).resolve() + + +def validate_options(opts: NewProjectOptions) -> list[str]: + errs: list[str] = [] + name = (opts.name or "").strip() + disc = (opts.disc or "").strip() + if not name: + errs.append("Project name is required") + if not disc: + errs.append("Disc .cue path is required") + else: + p = Path(disc).expanduser() + if not p.is_file(): + errs.append(f"Disc not found: {disc}") + if opts.bios: + bp = Path(opts.bios).expanduser() + if not bp.is_file(): + errs.append(f"BIOS not found: {opts.bios}") + if opts.players < 1 or opts.players > 8: + errs.append("Players must be 1–8") + if opts.do_build and not opts.do_generate: + errs.append("Build requires Generate") + # Wizard/netplay without UI (and 1P netplay) are auto-corrected at run time. + vis = (opts.github_visibility or "private").strip().lower() + if vis not in ("public", "private", "internal"): + errs.append("GitHub visibility must be public/private/internal") + dest = project_root_for(opts) + if dest.exists(): + if dest.is_file() or (dest.is_dir() and any(dest.iterdir())): + errs.append(f"Destination already exists: {dest}") + return errs + + +def build_command(opts: NewProjectOptions) -> tuple[list[str], dict[str, str]]: + """Build argv + env for the OS-appropriate setup script. + + Always passes ``--yes`` / ``-Yes`` so the GUI/CLI supply every choice. + """ + sh, ps1 = setup_script_paths() + env = os.environ.copy() + env["PSXRECOMP_SETUP_YES"] = "1" + if (opts.recomp_net_ref or "").strip(): + env["RECOMP_NET_REF"] = opts.recomp_net_ref.strip() + + if is_windows(): + if not ps1.is_file(): + raise FileNotFoundError(f"Missing setup script: {ps1}") + powershell = ( + shutil.which("pwsh") + or shutil.which("powershell") + or "powershell" + ) + cmd: list[str] = [ + powershell, + "-NoProfile", + "-ExecutionPolicy", + "Bypass", + "-File", + str(ps1), + "-Disc", + str(Path(opts.disc).expanduser().resolve()), + "-Name", + opts.name.strip(), + "-Yes", + "-Players", + str(int(opts.players)), + "-PsxrecompRef", + (opts.psxrecomp_ref or "master").strip(), + "-RecompUiRef", + (opts.recomp_ui_ref or "master").strip(), + "-GithubVisibility", + (opts.github_visibility or "private").strip().lower(), + ] + parent = (opts.parent_dir or ".").strip() or "." + cmd.extend(["-Dir", str(Path(parent).expanduser().resolve())]) + if opts.bios: + cmd.extend(["-Bios", str(Path(opts.bios).expanduser().resolve())]) + if opts.boot_exe: + cmd.extend(["-BootExe", opts.boot_exe.strip()]) + if opts.zip_prefix: + cmd.extend(["-ZipPrefix", opts.zip_prefix.strip()]) + if opts.github_owner: + cmd.extend(["-GithubOwner", opts.github_owner.strip()]) + if opts.github_repo: + cmd.extend(["-GithubRepo", opts.github_repo.strip()]) + if opts.description: + cmd.extend(["-Description", opts.description.strip()]) + if opts.publisher: + cmd.extend(["-Publisher", opts.publisher.strip()]) + if opts.year: + cmd.extend(["-Year", opts.year.strip()]) + if opts.region: + cmd.extend(["-Region", opts.region.strip()]) + if opts.lobby_url: + cmd.extend(["-LobbyUrl", opts.lobby_url.strip()]) + + def sw(yes: bool, on: str, off: str) -> None: + cmd.append(on if yes else off) + + sw(opts.enable_recomp_ui, "-EnableRecompUi", "-NoRecompUi") + sw(opts.enable_wizard, "-EnableWizard", "-NoWizard") + sw(opts.enable_netplay, "-EnableNetplay", "-NoNetplay") + sw(opts.enable_ci, "-EnableCi", "-NoCi") + sw(opts.fetch_boxart, "-FetchBoxart", "-NoFetchBoxart") + sw(opts.do_generate, "-Generate", "-NoGenerate") + sw(opts.do_build, "-EnableBuild", "-NoBuild") + sw(opts.create_github, "-CreateGithub", "-NoGithub") + if opts.stage_disc: + cmd.append("-StageDisc") + return cmd, env + + if not sh.is_file(): + raise FileNotFoundError(f"Missing setup script: {sh}") + cmd = [ + "sh", + str(sh), + "--yes", + "--disc", + str(Path(opts.disc).expanduser().resolve()), + "--name", + opts.name.strip(), + "--players", + str(int(opts.players)), + "--psxrecomp-ref", + (opts.psxrecomp_ref or "master").strip(), + "--recomp-ui-ref", + (opts.recomp_ui_ref or "master").strip(), + "--github-visibility", + (opts.github_visibility or "private").strip().lower(), + ] + parent = (opts.parent_dir or ".").strip() or "." + cmd.extend(["--dir", str(Path(parent).expanduser().resolve())]) + if opts.bios: + cmd.extend(["--bios", str(Path(opts.bios).expanduser().resolve())]) + if opts.boot_exe: + cmd.extend(["--boot-exe", opts.boot_exe.strip()]) + if opts.zip_prefix: + cmd.extend(["--zip-prefix", opts.zip_prefix.strip()]) + if opts.github_owner: + cmd.extend(["--github-owner", opts.github_owner.strip()]) + if opts.github_repo: + cmd.extend(["--github-repo", opts.github_repo.strip()]) + if opts.description: + cmd.extend(["--description", opts.description.strip()]) + if opts.publisher: + cmd.extend(["--publisher", opts.publisher.strip()]) + if opts.year: + cmd.extend(["--year", opts.year.strip()]) + if opts.region: + cmd.extend(["--region", opts.region.strip()]) + if opts.lobby_url: + cmd.extend(["--lobby-url", opts.lobby_url.strip()]) + if (opts.recomp_net_ref or "").strip(): + cmd.extend(["--recomp-net-ref", opts.recomp_net_ref.strip()]) + + def flag(yes: bool, on: str, off: str) -> None: + cmd.append(on if yes else off) + + flag(opts.enable_recomp_ui, "--enable-recomp-ui", "--no-recomp-ui") + flag(opts.enable_wizard, "--enable-wizard", "--no-wizard") + flag(opts.enable_netplay, "--enable-netplay", "--no-netplay") + flag(opts.enable_ci, "--enable-ci", "--no-ci") + flag(opts.fetch_boxart, "--fetch-boxart", "--no-fetch-boxart") + flag(opts.do_generate, "--generate", "--no-generate") + flag(opts.do_build, "--enable-build", "--no-build") + flag(opts.create_github, "--create-github", "--no-github") + cmd.append("--stage-disc" if opts.stage_disc else "--no-stage-disc") + return cmd, env + + +def run_new_project( + opts: NewProjectOptions, + *, + on_line: Callable[[str], None] | None = None, + cancel_event: threading.Event | None = None, +) -> CmdResult: + """Run setup_project end-to-end; stream lines via ``on_line``.""" + errs = validate_options(opts) + if errs: + return CmdResult(False, "Invalid new-project options", "\n".join(errs)) + + # Match script policy: no UI ⇒ no wizard/netplay; 1P ⇒ no netplay. + if not opts.enable_recomp_ui: + opts.enable_wizard = False + opts.enable_netplay = False + if opts.players < 2: + opts.enable_netplay = False + + try: + cmd, env = build_command(opts) + except FileNotFoundError as exc: + return CmdResult(False, str(exc)) + + if opts.dry_run: + preview = " ".join(cmd) + if on_line: + on_line(f"[dry-run] {preview}") + return CmdResult(True, "dry-run: would run setup_project", preview) + + if on_line: + on_line(f"$ {' '.join(cmd)}") + + try: + proc = subprocess.Popen( + cmd, + stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, + text=True, + encoding="utf-8", + errors="replace", + env=env, + cwd=str(Path(opts.parent_dir or ".").expanduser().resolve()), + ) + except OSError as exc: + return CmdResult(False, f"Failed to start setup: {exc}") + + assert proc.stdout is not None + for line in proc.stdout: + if cancel_event is not None and cancel_event.is_set(): + proc.terminate() + return CmdResult(False, "New project cancelled") + text = line.rstrip("\n") + if on_line: + on_line(text) + code = proc.wait() + root = project_root_for(opts) + if code != 0: + return CmdResult( + False, + f"setup_project failed (exit {code})", + str(root), + ) + + # Optional nested rbengine (and net on Windows, where script has no ref flag) + post_notes: list[str] = [] + if root.is_dir(): + nested_branches: dict[str, str] = {} + net = (opts.recomp_net_ref or "").strip() + rb = (opts.rbengine_ref or "").strip() + if net and is_windows(): + nested_branches["lib/recomp-net"] = net + if rb: + nested_branches["lib/retcomm-rbengine"] = rb + if nested_branches: + if on_line: + on_line("== Post-setup nested lib branch switch ==") + for r in switch_modules( + root, + nested=True, + branch_by_path=nested_branches, + paths=list(nested_branches.keys()), + set_tracking=True, + dry_run=False, + ): + post_notes.append(r.message) + if on_line: + on_line(f" [{'OK' if r.ok else 'FAIL'}] {r.message}") + + detail = str(root) + if post_notes: + detail += "\n" + "\n".join(post_notes) + return CmdResult(True, f"Created project at {root}", detail) + + +def index_new_project( + root: Path, + *, + name: str = "", + cue: str = "", +) -> CmdResult: + """Add/update the Studio repo index for a freshly created project.""" + from .bulkops import ( + catalog_title_id_for_root, + upsert_studio_toml_title, + ) + from .repo_index import add_repo, load_index + + if not root.is_dir(): + return CmdResult(False, f"Project root missing: {root}") + idx = load_index() + entry = add_repo(idx, root, name=name or root.name, cue=cue) + notes: list[str] = [entry.path] + tid = catalog_title_id_for_root(root) + if tid: + note = upsert_studio_toml_title(tid, root) + if note: + notes.append(note) + return CmdResult(True, f"Indexed {entry.label()}", "\n".join(notes)) diff --git a/tools/new_project_layout/project_studio/ops.py b/tools/new_project_layout/project_studio/ops.py index 2fa1e2a5..54d5a675 100644 --- a/tools/new_project_layout/project_studio/ops.py +++ b/tools/new_project_layout/project_studio/ops.py @@ -2,6 +2,7 @@ from __future__ import annotations +import json import re import shutil import subprocess @@ -15,6 +16,7 @@ from .naming import ( boot_exe_from_game_toml, build_token_map, entry_pc_from_game_toml, + game_name_from_project, infer_project_name, normalize_lobby_url, players_from_cmake, @@ -221,6 +223,8 @@ def _resolve_tokens(root: Path, options: MigrateOptions) -> dict[str, str]: enable_wizard=options.enable_wizard if options.enable_recomp_ui else False, enable_netplay=enable_netplay if options.enable_recomp_ui else False, has_boxart=has_boxart, + github_owner=options.github_owner, + github_repo=options.github_repo, ) @@ -1080,6 +1084,136 @@ def op_record_framework_pins(root: Path, options: MigrateOptions) -> ApplyResult ) +def _boxart_hints(root: Path, options: MigrateOptions) -> tuple[str, str]: + cue = "" + display = "" + if options.disc: + cue = Path(options.disc).name + cat = root / "catalog_identity.json" + if cat.is_file(): + try: + data = json.loads(cat.read_text(encoding="utf-8")) + except (OSError, json.JSONDecodeError): + data = {} + if isinstance(data, dict): + rom = data.get("rom_identity") if isinstance(data.get("rom_identity"), dict) else {} + cue = cue or str(rom.get("cue_name") or "") + game = data.get("game") if isinstance(data.get("game"), dict) else {} + display = str(game.get("name") or "") + name = options.project_name or infer_project_name(root) + display = options.window_title or display or game_name_from_project(name) + return cue, display + + +def _ensure_boxart_png(root: Path, options: MigrateOptions) -> list[str]: + """Fetch libretro boxart PNG/TGA when the README PNG is missing. Never raises.""" + png = root / "launcher_assets" / "img" / "boxart.png" + tga = root / "launcher_assets" / "img" / "boxart.tga" + if png.is_file(): + return [] + if options.dry_run: + return ["launcher_assets/img/boxart.png"] + cue, display = _boxart_hints(root, options) + if not cue and not display: + return [] + try: + sys.path.insert(0, str(toolkit_dir())) + from fetch_boxart import fetch_to_paths # type: ignore + except ImportError: + return [] + try: + fetch_to_paths(tga, cue_stem=cue, display_name=display) + except Exception: + return [] + changed: list[str] = [] + if png.is_file(): + changed.append("launcher_assets/img/boxart.png") + if tga.is_file(): + changed.append("launcher_assets/img/boxart.tga") + src = tga.parent / "BOXART_SOURCE.txt" + if src.is_file(): + changed.append("launcher_assets/img/BOXART_SOURCE.txt") + return changed + + +def op_patch_readme_metrics(root: Path, options: MigrateOptions) -> ApplyResult: + """Upsert download badges, libretro boxart, RetComM Launcher, and R.A.I.D. footer.""" + from .readme_metrics import ( + apply_github_about, + boxart_png_present, + render_boxart_block, + render_launcher_block, + render_metrics_block, + render_raid_block, + resolve_github_slug, + upsert_readme_blocks, + ) + + name = options.project_name or infer_project_name(root) + owner, repo = resolve_github_slug(root, options.github_owner, options.github_repo) + zp = options.zip_prefix or derive_zip_prefix(name) + fetched = _ensure_boxart_png(root, options) + display = options.window_title or game_name_from_project(name) + boxart_block = None + if boxart_png_present(root) or ( + options.dry_run and "launcher_assets/img/boxart.png" in fetched + ): + boxart_block = render_boxart_block(display) + + path = root / "README.md" + if path.is_file(): + old = path.read_text(encoding="utf-8", errors="replace") + else: + title = options.window_title or window_title_from_name(name) + old = f"# {title}\n" + new = upsert_readme_blocks( + old, + render_metrics_block(owner, repo, zp), + render_launcher_block(), + render_raid_block(), + boxart=boxart_block, + ) + changed: list[str] = list(fetched) + if new != old: + _write(path, new, options.dry_run) + if "README.md" not in changed: + changed.append("README.md") + + src_img = templates_dir() / "raid-discord.png" + dst_img = root / ".github" / "raid-discord.png" + if src_img.is_file() and not dst_img.is_file(): + if not options.dry_run: + dst_img.parent.mkdir(parents=True, exist_ok=True) + shutil.copy2(src_img, dst_img) + changed.append(".github/raid-discord.png") + + about_ok, about_msg = apply_github_about(owner, repo, dry_run=options.dry_run) + parts: list[str] = [] + if changed: + parts.append( + f"Patched README badges, boxart, RetComM Launcher, and R.A.I.D. footer ({owner}/{repo})" + ) + if fetched: + parts.append("fetched libretro boxart") + if about_ok: + parts.append(about_msg) + else: + parts.append("GitHub About skipped: " + about_msg) + if not changed and not about_ok: + return ApplyResult( + "patch_readme_metrics", + True, + f"README metrics already current ({owner}/{repo}); {about_msg}", + [], + ) + return ApplyResult( + "patch_readme_metrics", + True, + "; ".join(parts), + changed, + ) + + _OPS = { "rename_psxrecomp_submodule": op_rename_psxrecomp_submodule, "ensure_psxrecomp_submodule": op_ensure_psxrecomp_submodule, @@ -1100,6 +1234,7 @@ _OPS = { "annotate_legacy_packaging": op_annotate_legacy_packaging, "probe_disc_refresh": op_probe_disc_refresh, "record_framework_pins": op_record_framework_pins, + "patch_readme_metrics": op_patch_readme_metrics, } diff --git a/tools/new_project_layout/project_studio/plan.py b/tools/new_project_layout/project_studio/plan.py index 8270012c..9204f39e 100644 --- a/tools/new_project_layout/project_studio/plan.py +++ b/tools/new_project_layout/project_studio/plan.py @@ -29,6 +29,7 @@ _OP_ORDER = [ "annotate_legacy_packaging", "probe_disc_refresh", "record_framework_pins", + "patch_readme_metrics", ] _OP_TITLES = { @@ -51,6 +52,7 @@ _OP_TITLES = { "annotate_legacy_packaging": "Annotate legacy prebuilt packaging", "probe_disc_refresh": "Refresh disc identity via probe_disc.py", "record_framework_pins": "Write framework_pins.txt", + "patch_readme_metrics": "Patch README badges, RetComM Launcher, and R.A.I.D. footer", } diff --git a/tools/new_project_layout/project_studio/readme_metrics.py b/tools/new_project_layout/project_studio/readme_metrics.py new file mode 100644 index 00000000..bf771a50 --- /dev/null +++ b/tools/new_project_layout/project_studio/readme_metrics.py @@ -0,0 +1,348 @@ +"""README download-badge + RetComM Launcher blocks for scaffold and migrate.""" + +from __future__ import annotations + +import json +import re +import subprocess +from pathlib import Path + +from fill_tokens import ( + GITHUB_ABOUT_DESCRIPTION, + GITHUB_ABOUT_HOMEPAGE, + sanitize_github_name, +) + +METRICS_BEGIN = "" +METRICS_END = "" +BOXART_BEGIN = "" +BOXART_END = "" +LAUNCHER_BEGIN = "" +LAUNCHER_END = "" +RAID_BEGIN = "" +RAID_END = "" + +BOXART_REL = "launcher_assets/img/boxart.png" + +_METRICS_RE = re.compile( + re.escape(METRICS_BEGIN) + r".*?" + re.escape(METRICS_END) + r"\n?", + re.S, +) +_BOXART_RE = re.compile( + re.escape(BOXART_BEGIN) + r".*?" + re.escape(BOXART_END) + r"\n?", + re.S, +) +_LAUNCHER_RE = re.compile( + re.escape(LAUNCHER_BEGIN) + r".*?" + re.escape(LAUNCHER_END) + r"\n?", + re.S, +) +_RAID_RE = re.compile( + re.escape(RAID_BEGIN) + r".*?" + re.escape(RAID_END) + r"\n?", + re.S, +) +_LEGACY_LAUNCHER_RE = re.compile( + r"^## RetComM Launcher\n.*?(?=^## |\Z)", + re.M | re.S, +) +_LEGACY_RAID_RE = re.compile( + r"(?:\n---\s*)?\n*

\s*\n\s*R\.A\.I\.D\..*\Z", + re.S, +) +_REMOTE_RE = re.compile( + r"github\.com[:/](?P[^/\s]+)/(?P[^/\s]+)", + re.I, +) + +DEFAULT_GITHUB_OWNER = "TechnicallyComputers" +LAUNCHER_REPO = "https://github.com/TechnicallyComputers/RetComM-Launcher" + + +def parse_github_remote(url: str) -> tuple[str, str] | None: + url = (url or "").strip().rstrip("/") + m = _REMOTE_RE.search(url) + if not m: + return None + owner = m.group("owner") + repo = m.group("repo") + if repo.endswith(".git"): + repo = repo[: -len(".git")] + repo = repo.rstrip("/") + if not owner or not repo: + return None + return owner, repo + + +def infer_github_slug(root: Path) -> tuple[str, str] | None: + """Best-effort owner/repo from origin, then catalog homepage.""" + try: + proc = subprocess.run( + ["git", "-C", str(root), "remote", "get-url", "origin"], + capture_output=True, + text=True, + check=False, + ) + except OSError: + proc = None + if proc is not None and proc.returncode == 0: + parsed = parse_github_remote(proc.stdout.strip()) + if parsed: + return parsed + cat = root / "catalog_identity.json" + if cat.is_file(): + try: + data = json.loads(cat.read_text(encoding="utf-8")) + except (OSError, json.JSONDecodeError): + data = {} + if isinstance(data, dict): + for key in ("homepage", "github", "repository", "url"): + val = data.get(key) + if isinstance(val, str): + parsed = parse_github_remote(val) + if parsed: + return parsed + return None + + +def resolve_github_slug( + root: Path, + owner: str | None = None, + repo: str | None = None, +) -> tuple[str, str]: + o = sanitize_github_name(owner or "") + r = sanitize_github_name(repo or "") + if o and r: + return o, r + inferred = infer_github_slug(root) + if inferred: + return ( + o or sanitize_github_name(inferred[0]), + r or sanitize_github_name(inferred[1]), + ) + return (o or DEFAULT_GITHUB_OWNER, r or sanitize_github_name(root.name)) + + +def apply_github_about( + owner: str, + repo: str, + *, + dry_run: bool = False, +) -> tuple[bool, str]: + """Set GitHub About description + Discord homepage via ``gh repo edit``.""" + slug = f"{sanitize_github_name(owner) or DEFAULT_GITHUB_OWNER}/{sanitize_github_name(repo) or 'repo'}" + cmd = [ + "gh", + "repo", + "edit", + slug, + "--description", + GITHUB_ABOUT_DESCRIPTION, + "--homepage", + GITHUB_ABOUT_HOMEPAGE, + ] + if dry_run: + return True, "dry-run: " + " ".join(cmd) + try: + proc = subprocess.run(cmd, capture_output=True, text=True, check=False) + except OSError as exc: + return False, f"gh not available ({exc})" + if proc.returncode != 0: + err = (proc.stderr or proc.stdout or "").strip() + return False, err or f"gh repo edit exit {proc.returncode}" + return True, f"GitHub About set on {slug}" + + +def render_metrics_block(owner: str, repo: str, zip_prefix: str = "") -> str: + # Per-OS latest badges need an exact asset name. Title zips are + # `{prefix}-{version}-{platform}.zip`, so globs always show 0 on shields.io. + # Keep all-time / latest totals + version; zip_prefix is unused here. + del zip_prefix + owner = sanitize_github_name(owner) or DEFAULT_GITHUB_OWNER + repo = sanitize_github_name(repo) or "repo" + base = f"https://github.com/{owner}/{repo}" + img = "https://img.shields.io/github" + lines = [ + METRICS_BEGIN, + f"[![GitHub downloads (all assets, all releases)]({img}/downloads/{owner}/{repo}/total)]({base}/releases)", + f"[![GitHub downloads (latest release)]({img}/downloads/{owner}/{repo}/latest/total)]({base}/releases/latest)", + f"[![GitHub release]({img}/v/release/{owner}/{repo})]({base}/releases/latest)", + METRICS_END, + ] + return "\n".join(lines) + + +def render_boxart_block(game_name: str) -> str: + alt = (game_name or "Game").strip() or "Game" + alt = alt.replace('"', "'") + return "\n".join( + [ + BOXART_BEGIN, + '

', + f' {alt} box art', + "

", + BOXART_END, + ] + ) + + +def render_launcher_block() -> str: + shots = ( + "https://raw.githubusercontent.com/TechnicallyComputers/" + "RetComM-Launcher/main/docs/screenshots" + ) + return "\n".join( + [ + LAUNCHER_BEGIN, + "## RetComM Launcher", + "", + "You can run this title **standalone** (release zip + the built-in recomp-ui", + "Generate & Build flow), or manage installs, updates, ROM/BIOS wiring, and queued", + "builds more intuitively with", + f"**[RetComM Launcher]({LAUNCHER_REPO})** —", + "the Retro Compilation Manager hub for self-compiling recomps.", + "", + f"[Downloads]({LAUNCHER_REPO}/releases) ·", + f"[Full README & features]({LAUNCHER_REPO}#readme)", + "", + "

", + f' RetComM hub with a background build, next to a title’s recomp-ui launcher', + "

", + "", + "

", + f' Background cmake build with titles queued', + "

", + "", + "RetComM checks for updates, rebuilds with existing build data when possible,", + "shares the portable toolchain used by per-title launchers, and automates", + "BIOS/ROM/save plumbing so you are not stuck repeating each game’s wizard by hand.", + LAUNCHER_END, + ] + ) + + +def render_raid_block() -> str: + return "\n".join( + [ + RAID_BEGIN, + "---", + "", + "

", + " R.A.I.D. — Retro AI Development · a Discord for AI-assisted retro reverse-engineering, decomp & recomp", + "

", + "", + "

", + ' Join the Retro AI Development (R.A.I.D.) Discord', + "

", + RAID_END, + ] + ) + + +def readme_has_metrics(text: str) -> bool: + return "img.shields.io/github/downloads/" in text + + +def readme_has_boxart(text: str) -> bool: + return BOXART_BEGIN in text or BOXART_REL in text + + +def boxart_png_present(root: Path) -> bool: + return (root / BOXART_REL).is_file() + + +def readme_has_launcher(text: str) -> bool: + return "TechnicallyComputers/RetComM-Launcher" in text + + +def readme_has_raid(text: str) -> bool: + return "discord.gg/Ad9BwSzctP" in text or "R.A.I.D. — Retro AI Development" in text + + +def _apply_boxart(text: str, boxart_block: str) -> str: + boxart_block = boxart_block.strip() + "\n" + if _BOXART_RE.search(text): + return _BOXART_RE.sub(boxart_block, text, count=1) + m = _METRICS_RE.search(text) + if m: + rest = text[m.end() :].lstrip("\n") + return text[: m.end()] + "\n" + boxart_block + "\n" + rest + hm = re.match(r"(#[^\n]+\n)", text) + if hm: + rest = text[hm.end() :].lstrip("\n") + return text[: hm.end()] + "\n" + boxart_block + "\n" + rest + return boxart_block + "\n" + text + + +def upsert_boxart_block(text: str, game_name: str) -> str: + if not text.endswith("\n"): + text += "\n" + return _apply_boxart(text, render_boxart_block(game_name)) + + +def inject_readme_boxart(readme: Path, game_name: str) -> bool: + """Insert/replace the README boxart block if ``boxart.png`` exists.""" + readme = Path(readme) + if not readme.is_file() or not boxart_png_present(readme.parent): + return False + old = readme.read_text(encoding="utf-8", errors="replace") + new = upsert_boxart_block(old, game_name) + if new == old: + return False + readme.write_text(new, encoding="utf-8") + return True + + +def upsert_readme_blocks( + text: str, + metrics: str, + launcher: str, + raid: str | None = None, + boxart: str | None = None, +) -> str: + """Insert or replace marked metrics, boxart, launcher, and RAID footer; keep custom body.""" + if not text.endswith("\n"): + text += "\n" + metrics_block = metrics.strip() + "\n" + launcher_block = launcher.strip() + "\n" + raid_block = (raid if raid is not None else render_raid_block()).strip() + "\n" + + if _METRICS_RE.search(text): + text = _METRICS_RE.sub(metrics_block, text, count=1) + elif not readme_has_metrics(text): + m = re.match(r"(#[^\n]+\n)", text) + if m: + text = ( + text[: m.end()] + + "\n" + + metrics_block + + "\n" + + text[m.end() :].lstrip("\n") + ) + else: + text = metrics_block + "\n" + text + + if boxart: + text = _apply_boxart(text, boxart) + + if _LAUNCHER_RE.search(text): + text = _LAUNCHER_RE.sub(launcher_block, text, count=1) + elif _LEGACY_LAUNCHER_RE.search(text): + text = _LEGACY_LAUNCHER_RE.sub(launcher_block + "\n", text, count=1) + elif not readme_has_launcher(text): + insert_at = None + for heading in ("## Legal", "## Quick start", "## Layout", "## Symbols"): + idx = text.find("\n" + heading) + if idx >= 0: + insert_at = idx + 1 + break + if insert_at is None: + text = text.rstrip() + "\n\n" + launcher_block + else: + text = text[:insert_at] + launcher_block + "\n" + text[insert_at:] + + if _RAID_RE.search(text): + text = _RAID_RE.sub(raid_block, text, count=1) + elif _LEGACY_RAID_RE.search(text): + text = _LEGACY_RAID_RE.sub("\n" + raid_block, text, count=1) + elif not readme_has_raid(text): + text = text.rstrip() + "\n\n" + raid_block + return text if text.endswith("\n") else text + "\n" diff --git a/tools/new_project_layout/setup_project.ps1 b/tools/new_project_layout/setup_project.ps1 index f37edaf5..061a43b0 100644 --- a/tools/new_project_layout/setup_project.ps1 +++ b/tools/new_project_layout/setup_project.ps1 @@ -22,6 +22,8 @@ param( [string]$BootExe = "SLUS_01234", [int]$Players = 0, [string]$ZipPrefix = "", + [string]$GithubOwner = "", + [string]$GithubRepo = "", [switch]$EnableRecompUi, [switch]$NoRecompUi, [switch]$EnableNetplay, @@ -142,6 +144,24 @@ if (-not $ZipPrefix) { } } +if (-not $GithubOwner) { + if ($interactive) { + $GithubOwner = Prompt-Line "GitHub owner / org (README download badges)" "TechnicallyComputers" + } else { + $GithubOwner = "TechnicallyComputers" + } +} +$derivedRepo = (python -c "from fill_tokens import sanitize_github_name; import sys; print(sanitize_github_name(sys.argv[1]))" $Name).Trim() +if (-not $GithubRepo) { + if ($interactive) { + $GithubRepo = Prompt-Line "GitHub repo name (README download badges)" $derivedRepo + } else { + $GithubRepo = $derivedRepo + } +} +$GithubOwner = (python -c "from fill_tokens import sanitize_github_name; import sys; print(sanitize_github_name(sys.argv[1]))" $GithubOwner).Trim() +$GithubRepo = (python -c "from fill_tokens import sanitize_github_name; import sys; print(sanitize_github_name(sys.argv[1]))" $GithubRepo).Trim() + if (-not $PSBoundParameters.ContainsKey("Description")) { if ($interactive) { $Description = Prompt-Line "Short game description (optional)" "" } } @@ -246,8 +266,13 @@ $PublisherDisp = if ($Publisher) { $Publisher } else { "-" } $YearDisp = if ($Year) { $Year } else { "-" } $DescriptionMd = if ($Description) { $Description } else { "_Add a short pitch in catalog_identity.json / README._" } -$Root = Join-Path (Resolve-Path $Dir) $Name +# Folder = GitHub/catalog install_dir slug (hyphenated), not display $Name with spaces. +$InstallDirName = $GithubRepo +$Root = Join-Path (Resolve-Path $Dir) $InstallDirName if (Test-Path $Root) { throw "Target already exists: $Root" } +if ($InstallDirName -ne $Name) { + Write-Host "note: project folder is '$InstallDirName' (catalog/GitHub slug; display name stays '$Name')" +} $NetplayBlockFile = Join-Path $env:TEMP "psxrecomp_netplay_block.cmake" $WizardBlockFile = Join-Path $env:TEMP "psxrecomp_wizard_block.cmake" @@ -316,6 +341,8 @@ function Fill-Template([string]$src, [string]$dst) { --set "ENV_PREFIX=$EnvPrefix" ` --set "EXE_BASENAME=$ExeBasename" ` --set "ZIP_PREFIX=$ZipPrefix" ` + --set "GITHUB_OWNER=$GithubOwner" ` + --set "GITHUB_REPO=$GithubRepo" ` --set "DISC_HINT=$DiscHint" ` --set "GAME_TITLE=$WindowTitle" ` --set "PLAYERS=$Players" ` @@ -339,6 +366,7 @@ Write-Host "== New Project Layout ==" Write-Host " repo: $Root" Write-Host " disc: $Disc" Write-Host " zip prefix: $ZipPrefix" +Write-Host " gh slug: $GithubOwner/$GithubRepo" Write-Host " players: $Players" Write-Host " recomp-ui: $useRecompUi" Write-Host " wizard: $useWizard" @@ -564,6 +592,32 @@ print(m.group(1) if m else '') Write-Warning "-Disc is not a .cue; skipped probe autofill." } +if ($doBoxart) { + Write-Host "== Fetching libretro boxart ==" + $cueHint = if ($DiscBasename) { $DiscBasename } else { $GameName } + $prevPy = $env:PYTHONPATH + try { + python $FetchBoxartPy ` + --out (Join-Path $Root "launcher_assets\img\boxart.tga") ` + --cue-stem $cueHint ` + --display-name $GameName + if ($LASTEXITCODE -eq 0) { + $HasBoxart = $true + Set-Content -Encoding UTF8 -Path $BoxartBlockFile -Value ' LAUNCHER_BOXART "${CMAKE_CURRENT_SOURCE_DIR}/launcher_assets/img/boxart.tga"' + Fill-Template (Join-Path $TemplateDir "CMakeLists.txt.in") (Join-Path $Root "CMakeLists.txt") + Write-Host " wired LAUNCHER_BOXART in CMakeLists.txt" + $env:PYTHONPATH = $ScriptDir + python -c "from pathlib import Path; import sys; from project_studio.readme_metrics import inject_readme_boxart; inject_readme_boxart(Path(sys.argv[1]), sys.argv[2])" (Join-Path $Root "README.md") $GameName + Write-Host " injected boxart into README.md" + } else { + Write-Warning "boxart fetch failed — leave LAUNCHER_BOXART commented." + } + } catch { + Write-Warning "boxart fetch failed — leave LAUNCHER_BOXART commented." + } finally { + if ($null -eq $prevPy) { Remove-Item Env:PYTHONPATH -ErrorAction SilentlyContinue } else { $env:PYTHONPATH = $prevPy } + } +} Write-Host "== Sync symbols header ==" Push-Location $Root @@ -571,8 +625,8 @@ try { python tools/sync_symbols.py --game $GameName } catch { Write-Warning "syn Pop-Location git add CMakeLists.txt game.toml codegen_setup.c codegen_setup.h .gitignore VERSION README.md seeds scripts tools mods framework_pins.txt symbols.toml psx_symbols.h 2>$null -if (Test-Path (Join-Path $Root ".github\workflows")) { - git add .github/workflows 2>$null +if (Test-Path (Join-Path $Root ".github")) { + git add .github 2>$null } if (Test-Path (Join-Path $Root "catalog_identity.json")) { git add catalog_identity.json 2>$null @@ -581,7 +635,7 @@ if (Test-Path (Join-Path $Root "disc_probe.json")) { git add disc_probe.json 2>$null } if ($HasBoxart) { - git add launcher_assets/img/boxart.tga launcher_assets/img/BOXART_SOURCE.txt 2>$null + git add launcher_assets/img/boxart.tga launcher_assets/img/boxart.png launcher_assets/img/BOXART_SOURCE.txt 2>$null } $committed = $false try { @@ -606,15 +660,18 @@ if ($createGithub) { "internal" { "--internal" } default { "--private" } } + $ghSlug = "$GithubOwner/$GithubRepo" + $ghDesc = (python -c "from fill_tokens import GITHUB_ABOUT_DESCRIPTION; print(GITHUB_ABOUT_DESCRIPTION)").Trim() + $ghHome = (python -c "from fill_tokens import GITHUB_ABOUT_HOMEPAGE; print(GITHUB_ABOUT_HOMEPAGE)").Trim() try { - gh repo create $Name $vis --source=$Root --remote=origin + gh repo create $ghSlug $vis --source=$Root --remote=origin --description "$ghDesc" --homepage "$ghHome" Write-Host " created origin ($GithubVisibility); push deferred until end" $githubCreated = $true } catch { Write-Warning "gh repo create failed -- if the repo already exists, add origin and push at the end." if (-not (git remote get-url origin 2>$null)) { try { - $ghUrl = (gh repo view $Name --json url -q .url 2>$null) + $ghUrl = (gh repo view $ghSlug --json url -q .url 2>$null) if ($ghUrl) { git remote add origin $ghUrl Write-Host " attached existing origin -> $ghUrl" @@ -625,6 +682,14 @@ if ($createGithub) { $githubCreated = $true } } + if ($githubCreated) { + try { + gh repo edit $ghSlug --description "$ghDesc" --homepage "$ghHome" + Write-Host " GitHub About: R.A.I.D. description + Discord homepage" + } catch { + Write-Warning "gh repo edit (About) failed — set description/homepage by hand." + } + } } } diff --git a/tools/new_project_layout/setup_project.sh b/tools/new_project_layout/setup_project.sh index 8f9ea735..6f7a658f 100755 --- a/tools/new_project_layout/setup_project.sh +++ b/tools/new_project_layout/setup_project.sh @@ -9,11 +9,13 @@ # --stage-disc Copy full cue+bins into repo disc/ (large; optional) # --no-stage-disc Default: probe in place, extract boot EXE only # --psxrecomp-ref / --recomp-ui-ref / --recomp-net-ref / URLs +# --github-owner README download-badge owner (default TechnicallyComputers) +# --github-repo README download-badge repo (default project name) # # Everything else is prompted on a TTY (or passed via flags / --yes defaults): -# name, players (1–8, default 2), zip-prefix, marketing blurb, recomp-ui, -# wizard, netplay, lobby URL (if netplay), CI workflow, fetch-boxart, -# generate, build (if generate), GitHub repo (gh) +# name, players (1–8, default 2), zip-prefix, github owner/repo (README badges), +# marketing blurb, recomp-ui, wizard, netplay, lobby URL (if netplay), CI +# workflow, fetch-boxart, generate, build (if generate), GitHub repo (gh) # # Non-interactive: --yes (or PSXRECOMP_SETUP_YES=1) requires --name and --disc; # boolean opts stay off unless explicitly flagged; lobby URL defaults to @@ -55,6 +57,8 @@ DO_GENERATE="" DO_BUILD="" CREATE_GITHUB="" GITHUB_VISIBILITY="private" +GITHUB_OWNER="" +GITHUB_REPO="" BIOS_PATH="" LOBBY_URL="" DESCRIPTION="" @@ -183,6 +187,8 @@ while [ $# -gt 0 ]; do --create-github) CREATE_GITHUB=1; SET_GITHUB=1; shift ;; --no-github) CREATE_GITHUB=0; SET_GITHUB=1; shift ;; --github-visibility) GITHUB_VISIBILITY=$2; shift 2 ;; + --github-owner) GITHUB_OWNER=$2; shift 2 ;; + --github-repo) GITHUB_REPO=$2; shift 2 ;; --description) DESCRIPTION=$2; SET_DESCRIPTION=1; shift 2 ;; --publisher) PUBLISHER=$2; SET_PUBLISHER=1; shift 2 ;; --year) YEAR=$2; SET_YEAR=1; shift 2 ;; @@ -264,6 +270,28 @@ if [ -z "$ZIP_PREFIX" ]; then fi fi +if [ -z "$GITHUB_OWNER" ]; then + if [ "$YES_MODE" -eq 1 ] || ! is_tty; then + GITHUB_OWNER=TechnicallyComputers + else + prompt_line "GitHub owner / org (README download badges)" GITHUB_OWNER \ + "TechnicallyComputers" + fi +fi +if [ -z "$GITHUB_REPO" ]; then + _derived_repo=$(PYTHONPATH="$SCRIPT_DIR${PYTHONPATH:+:$PYTHONPATH}" \ + python3 -c 'from fill_tokens import sanitize_github_name; import sys; print(sanitize_github_name(sys.argv[1]))' "$NAME") + if [ "$YES_MODE" -eq 1 ] || ! is_tty; then + GITHUB_REPO=$_derived_repo + else + prompt_line "GitHub repo name (README download badges)" GITHUB_REPO "$_derived_repo" + fi +fi +GITHUB_OWNER=$(PYTHONPATH="$SCRIPT_DIR${PYTHONPATH:+:$PYTHONPATH}" \ + python3 -c 'from fill_tokens import sanitize_github_name; import sys; print(sanitize_github_name(sys.argv[1]))' "$GITHUB_OWNER") +GITHUB_REPO=$(PYTHONPATH="$SCRIPT_DIR${PYTHONPATH:+:$PYTHONPATH}" \ + python3 -c 'from fill_tokens import sanitize_github_name; import sys; print(sanitize_github_name(sys.argv[1]))' "$GITHUB_REPO") + # Optional marketing (catalog_identity.json + README) if [ "$SET_DESCRIPTION" -eq 0 ]; then if [ "$YES_MODE" -eq 1 ] || ! is_tty; then @@ -448,11 +476,16 @@ if [ -z "$DESCRIPTION_MD" ]; then DESCRIPTION_MD="_Add a short pitch in catalog_identity.json / README._" fi -ROOT=$(CDPATH= cd -- "$PARENT" && pwd)/"$NAME" +# Folder = GitHub/catalog install_dir slug (hyphenated), not display $NAME with spaces. +INSTALL_DIR_NAME="$GITHUB_REPO" +ROOT=$(CDPATH= cd -- "$PARENT" && pwd)/"$INSTALL_DIR_NAME" if [ -e "$ROOT" ]; then echo "error: target already exists: $ROOT" >&2 exit 1 fi +if [ "$INSTALL_DIR_NAME" != "$NAME" ]; then + echo "note: project folder is '$INSTALL_DIR_NAME' (catalog/GitHub slug; display name stays '$NAME')" >&2 +fi # --- CMake block files ----------------------------------------------------- NETPLAY_BLOCK_FILE=$(mktemp) @@ -527,6 +560,8 @@ fill_template() { --set "ENV_PREFIX=$ENV_PREFIX" \ --set "EXE_BASENAME=$EXE_BASENAME" \ --set "ZIP_PREFIX=$ZIP_PREFIX" \ + --set "GITHUB_OWNER=$GITHUB_OWNER" \ + --set "GITHUB_REPO=$GITHUB_REPO" \ --set "DISC_HINT=$DISC_HINT" \ --set "GAME_TITLE=$WINDOW_TITLE" \ --set "PLAYERS=$PLAYERS" \ @@ -550,6 +585,7 @@ echo "== New Project Layout ==" echo " repo: $ROOT" echo " disc: $DISC" echo " zip prefix: $ZIP_PREFIX" +echo " gh slug: $GITHUB_OWNER/$GITHUB_REPO" echo " players: $PLAYERS" echo " recomp-ui: $ENABLE_RECOMP_UI" echo " wizard: $ENABLE_WIZARD" @@ -578,9 +614,12 @@ fill_template "$TEMPLATE_DIR/VERSION.in" "$ROOT/VERSION" fill_template "$TEMPLATE_DIR/README.md.in" "$ROOT/README.md" fill_template "$TEMPLATE_DIR/symbols.toml.in" "$ROOT/symbols.toml" mkdir -p "$ROOT/seeds" "$ROOT/launcher_assets/img" "$ROOT/scripts" "$ROOT/tools" \ - "$ROOT/mods/preloaded/packages" "$ROOT/assets" + "$ROOT/mods/preloaded/packages" "$ROOT/assets" "$ROOT/.github" cp "$SCRIPT_DIR/sync_symbols.py" "$ROOT/tools/sync_symbols.py" chmod +x "$ROOT/tools/sync_symbols.py" +if [ -f "$TEMPLATE_DIR/raid-discord.png" ]; then + cp "$TEMPLATE_DIR/raid-discord.png" "$ROOT/.github/raid-discord.png" +fi # Empty mod catalog tree (runtime copies mods/preloaded → beside the exe as mods/). cat > "$ROOT/mods/preloaded/README.md" <<'EOF' # Preloaded mods @@ -801,6 +840,11 @@ if [ "$FETCH_BOXART_FLAG" -eq 1 ]; then >"$BOXART_BLOCK_FILE" fill_template "$TEMPLATE_DIR/CMakeLists.txt.in" "$ROOT/CMakeLists.txt" echo " wired LAUNCHER_BOXART in CMakeLists.txt" + PYTHONPATH="$SCRIPT_DIR${PYTHONPATH:+:$PYTHONPATH}" python3 -c \ + 'from pathlib import Path; import sys; from project_studio.readme_metrics import inject_readme_boxart; inject_readme_boxart(Path(sys.argv[1]), sys.argv[2])' \ + "$ROOT/README.md" "$GAME_NAME" \ + && echo " injected boxart into README.md" \ + || echo "warning: README boxart inject failed" >&2 else echo "warning: boxart fetch failed — leave LAUNCHER_BOXART commented." >&2 fi @@ -813,8 +857,8 @@ echo "== Sync symbols header ==" ) git add CMakeLists.txt game.toml codegen_setup.c codegen_setup.h .gitignore VERSION README.md seeds scripts tools mods framework_pins.txt symbols.toml psx_symbols.h || true -if [ -d "$ROOT/.github/workflows" ]; then - git add .github/workflows || true +if [ -d "$ROOT/.github" ]; then + git add .github || true fi if [ -f "$ROOT/catalog_identity.json" ]; then git add catalog_identity.json || true @@ -823,7 +867,7 @@ if [ -f "$ROOT/disc_probe.json" ]; then git add disc_probe.json || true fi if [ "$HAS_BOXART" -eq 1 ]; then - git add launcher_assets/img/boxart.tga launcher_assets/img/BOXART_SOURCE.txt || true + git add launcher_assets/img/boxart.tga launcher_assets/img/boxart.png launcher_assets/img/BOXART_SOURCE.txt || true fi COMMITTED=0 if git -c user.email=setup@localhost -c user.name=setup \ @@ -847,14 +891,20 @@ if [ "${CREATE_GITHUB:-0}" -eq 1 ]; then VIS_FLAG="--private" [ "$GITHUB_VISIBILITY" = "public" ] && VIS_FLAG="--public" [ "$GITHUB_VISIBILITY" = "internal" ] && VIS_FLAG="--internal" - if gh repo create "$NAME" $VIS_FLAG --source="$ROOT" --remote=origin; then + GH_SLUG="$GITHUB_OWNER/$GITHUB_REPO" + GH_DESC=$(PYTHONPATH="$SCRIPT_DIR${PYTHONPATH:+:$PYTHONPATH}" \ + python3 -c 'from fill_tokens import GITHUB_ABOUT_DESCRIPTION; print(GITHUB_ABOUT_DESCRIPTION)') + GH_HOME=$(PYTHONPATH="$SCRIPT_DIR${PYTHONPATH:+:$PYTHONPATH}" \ + python3 -c 'from fill_tokens import GITHUB_ABOUT_HOMEPAGE; print(GITHUB_ABOUT_HOMEPAGE)') + if gh repo create "$GH_SLUG" $VIS_FLAG --source="$ROOT" --remote=origin \ + --description "$GH_DESC" --homepage "$GH_HOME"; then echo " created origin ($GITHUB_VISIBILITY); push deferred until end" GITHUB_CREATED=1 else echo "warning: gh repo create failed — if the repo already exists," >&2 echo " add origin and push manually at the end." >&2 if ! git remote get-url origin >/dev/null 2>&1; then - _gh_url=$(gh repo view "$NAME" --json url -q .url 2>/dev/null || true) + _gh_url=$(gh repo view "$GH_SLUG" --json url -q .url 2>/dev/null || true) if [ -n "$_gh_url" ]; then git remote add origin "$_gh_url" echo " attached existing origin → $_gh_url" @@ -864,6 +914,13 @@ if [ "${CREATE_GITHUB:-0}" -eq 1 ]; then GITHUB_CREATED=1 fi fi + if [ "$GITHUB_CREATED" -eq 1 ]; then + if gh repo edit "$GH_SLUG" --description "$GH_DESC" --homepage "$GH_HOME"; then + echo " GitHub About: R.A.I.D. description + Discord homepage" + else + echo "warning: gh repo edit (About) failed — set description/homepage by hand." >&2 + fi + fi fi fi diff --git a/tools/new_project_layout/templates/README.md.in b/tools/new_project_layout/templates/README.md.in index f978342d..3f31de03 100644 --- a/tools/new_project_layout/templates/README.md.in +++ b/tools/new_project_layout/templates/README.md.in @@ -1,5 +1,11 @@ # @WINDOW_TITLE@ + +[![GitHub downloads (all assets, all releases)](https://img.shields.io/github/downloads/@GITHUB_OWNER@/@GITHUB_REPO@/total)](https://github.com/@GITHUB_OWNER@/@GITHUB_REPO@/releases) +[![GitHub downloads (latest release)](https://img.shields.io/github/downloads/@GITHUB_OWNER@/@GITHUB_REPO@/latest/total)](https://github.com/@GITHUB_OWNER@/@GITHUB_REPO@/releases/latest) +[![GitHub release](https://img.shields.io/github/v/release/@GITHUB_OWNER@/@GITHUB_REPO@)](https://github.com/@GITHUB_OWNER@/@GITHUB_REPO@/releases/latest) + + Static recompilation of **@GAME_NAME@** built on [psxrecomp](https://github.com/mstan/psxrecomp) and [recomp-ui](https://github.com/mstan/recomp-ui). @@ -16,6 +22,31 @@ Static recompilation of **@GAME_NAME@** built on Scaffolded with the New Project Layout. See `psxrecomp/docs/GAME_PROJECT_SETUP.md` for the full flow. + +## RetComM Launcher + +You can run this title **standalone** (release zip + the built-in recomp-ui +Generate & Build flow), or manage installs, updates, ROM/BIOS wiring, and queued +builds more intuitively with +**[RetComM Launcher](https://github.com/TechnicallyComputers/RetComM-Launcher)** — +the Retro Compilation Manager hub for self-compiling recomps. + +[Downloads](https://github.com/TechnicallyComputers/RetComM-Launcher/releases) · +[Full README & features](https://github.com/TechnicallyComputers/RetComM-Launcher#readme) + +

+ RetComM hub with a background build, next to a title’s recomp-ui launcher +

+ +

+ Background cmake build with titles queued +

+ +RetComM checks for updates, rebuilds with existing build data when possible, +shares the portable toolchain used by per-title launchers, and automates +BIOS/ROM/save plumbing so you are not stuck repeating each game’s wizard by hand. + + ## Legal You must own the original game. Disc images under `disc/` are gitignored and @@ -39,12 +70,6 @@ cmake -S . -B build-release -G Ninja -DCMAKE_BUILD_TYPE=Release cmake --build build-release --target psx-runtime ``` -`psxrecomp/runtime/runtime.cmake` must exist before configuring. If CMake says -the runtime is missing, the framework submodule is not checked out; run -`git submodule update --init --recursive`, or pass -`-DPSXRECOMP_ROOT=` when intentionally building against a -separate framework checkout. - Zip prefix for CI artifacts: `@ZIP_PREFIX@`. ## Symbols @@ -59,3 +84,15 @@ are authoritative. `framework_pins.txt` is an optional scaffold snapshot; release CI logs SHAs with `record_pins.sh` but builds whatever the gitlinks resolve to. Bump submodules deliberately — do not float on `main`/`master` in release CI. + + +--- + +

+ R.A.I.D. — Retro AI Development · a Discord for AI-assisted retro reverse-engineering, decomp & recomp +

+ +

+ Join the Retro AI Development (R.A.I.D.) Discord +

+ diff --git a/tools/new_project_layout/templates/raid-discord.png b/tools/new_project_layout/templates/raid-discord.png new file mode 100644 index 00000000..f10e0193 Binary files /dev/null and b/tools/new_project_layout/templates/raid-discord.png differ