Commit Graph

2087 Commits

Author SHA1 Message Date
Shokunin c4e50d92e0 tools: fp_identity review fixes (run failures, mixed FMV-quiet, Windows)
- run: a crash, hang-up or socket timeout on the final frame_fingerprint /
  dispatch_stats queries is a RunError (exit 3, INCOMPLETE), not a
  JSONDecodeError traceback with exit 1 (MISMATCH). Seeding, log and dump
  I/O errors and a runtime that survives SIGKILL also exit 3. A missing
  output directory is a usage error. The dump is written to OUT.tmp and
  renamed.
- compare: with different FMV-quiet settings, once a quiet frame occurred
  (qc > 0, a different frame set, or no qc column) the runs are not
  comparable. Report INCOMPLETE and skip the DIVERGE, missing-frame and
  locator lines, which were artefacts of the setting. Mixed settings with no
  quiet frame are judged as usual, with a note.
- --launch on Windows keeps backslashes (shlex with no escape character),
  so C:\psx\build\game-runtime.exe is not mangled into C:psxbuild...

Self-test: 59 cases (11 new), ctest fp_identity passes. R4 dumps re-judged
and fresh warm/cold runs repeat the IDENTICAL results with 497 tolerated
straddles.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 00:48:36 -07:00
Shokunin d36701ba7b docs: log the fp_identity A/B tool and its R4 measurement
FAITHFUL_TIMING_PLAN gets the dated session entry and ACCURACY_BURNDOWN's
backend-equivalence item names the tool and the R4 warm/cold result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 00:24:20 -07:00
Shokunin e8b17c394f tools: add fp_identity.py, a generic A/B guest-identity check
Moves the warm/cold A/B tool out of R4 so every title can use it. `run`
launches the runtime from a command template ({build} {port} {headless}
{state_dir}) or from --runtime/--game/--disc, optionally installs a saved
overlay state (--seed), turbos to frame N and saves the frame_fingerprint
ring plus the dispatch-miss count. Runs default to PSX_DEBUG_FMV_QUIET=0 and
PSX_OVERLAY_AUTOCOMPILE_OFF=1, and the dump records what they used.

`compare` judges on the guest-fact columns (cyc mmio mc sp sc wc ws qc) and
reports wr/pc only as the frame where the runs part. It tolerates the
one-write VBlank straddle from docs/TCP_COMMANDS.md (ws/wc off for one frame,
equal again at the next) and, when FMV-quiet was on, the wc+qc shift with ws
judged against the carried offset. Every tolerated frame is counted and
listed; --strict turns them off. A straddle on the last frame, a judge column
an older runtime does not report, or different FMV-quiet settings make the
result INCOMPLETE. Exit codes are unchanged from the R4 tool: IDENTICAL 0,
MISMATCH 1, usage 2, INCOMPLETE 3, MISSES 4.

The self-test (ctest fp_identity) covers column judging, both tolerances,
truncated and mismatched frame sets, dumps without ws/qc, and drives `run`
against a fake debug server.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 00:08:49 -07:00
Shokunin 255d05e8e4 debug_server: judge frame_fingerprint on guest-visible facts
frame_fingerprint's wr column is an ordered hash over main-RAM writes.
DMA and device writes (MDEC-out, CD, SPU, GPU->RAM) are recorded in host
device-service order. Native shards flush cycles at every store barrier,
while batched interpreted and static code services devices after the
block. So identical guest writes gave a different wr forever after.

- ws: an order-independent sum of a splitmix64 term per (addr, val) over
  main RAM. Together with wc it is the main-RAM judge column. wr and pc
  stay as ordered locators.
- qc: fingerprint-eligible writes that FMV-quiet kept out. If a straddled
  write falls into a quiet frame in one run only, the cumulative columns
  never agree again, and qc is where that shows.
- The accumulator moves to runtime/include/frame_fingerprint.h so the
  semantics are unit-tested (frame_fingerprint_test).
- The JSON buffer is sized from the real maximum entry length (284 B),
  which the old 224 B per entry could overflow.

docs/TCP_COMMANDS.md documents the column roles, the one-frame snapshot
straddle and the FMV-quiet caveat (set PSX_DEBUG_FMV_QUIET=0 for A/B
identity runs). Existing columns keep their meaning. Nothing outside
debug-tools builds changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 18:30:24 -07:00
Shokunin 649aceba7b overlay: write the runtime's store-PC breadcrumb from shards (ABI v24)
The emitter writes g_debug_last_store_pc before every guest store. The
overlay preamble gave each DLL a private copy on the theory that only debug
tooling reads it. The runtime actually reads it in every build:

- memory.c psx_write_word_raw keys its RAM 0x0-0xF store filters on it;
- the fingerprint pc/mmio/sp columns use it, and so do wtrace,
  record_frame, the MMIO/GP1 rings, parity provenance and the freeze
  heartbeat.

So after a native overlay store the host still named an older store, and
those consumers saw an instruction the interpreter would not have.

OverlayCallbacks gains last_store_pc, which the loader always points at
the runtime's copy. In PSX_OVERLAY_DLL_BUILD, cpu_state.h maps the
emitted name onto that pointer, the same pattern as the bail-state
pointers. Generated code is unchanged. A NULL host pointer keeps the old
private copy.

The ABI goes to 24, so the loader rejects and regenerates older shards.
The preamble change already moves the codegen-hash cache path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 18:03:43 -07:00
Matt Stanley 44a45d3c3d Merge pull request #416 from RetroPortingToolKit/fix/disc-picker-ascii-20260928
runtime: remove redundant missing-disc modal on Windows
2026-09-28 17:33:24 -07:00
Matt Stanley 6ef8877324 Merge pull request #412 from shaneomac1337/fix/vulkan-opaque-present
vulkan: present through an opaque shader pass (fixes AMD see-through window, #411)
2026-09-28 17:01:12 -07:00
Matthew Stanley 3382f86c59 runtime: keep disc-picker dialog text ASCII-safe 2026-09-28 16:54:33 -07:00
Matt Stanley 16382d22db Merge pull request #413 from shaneomac1337/fix/project-studio-windows
project studio: make migrate_project.py work from a Windows console
2026-09-28 15:34:52 -07:00
Matt Stanley f9d8830721 Merge pull request #410 from shaneomac1337/fix/debug-server-socket-noinherit
debug_server: make the listen socket non-inheritable on Windows
2026-09-28 15:34:48 -07:00
Matt Stanley 13aa8dbf12 Merge pull request #409 from shaneomac1337/fix/vulkan-validation-ds-staging
vulkan: fix depth/stencil aspect + usage and staging-buffer usage (validation errors)
2026-09-28 15:34:44 -07:00
Martin Penkava 3a4e166545 project studio: make migrate_project.py work from a Windows console
Two failures hit while migrating a title on Windows 11:

- `audit` crashed with UnicodeEncodeError on the "->" mark (U+2192) when the
  console code page can't encode it (cp1250 here). main() now reconfigures
  stdout/stderr with errors="replace", so reports degrade those characters
  instead of aborting.
- record_framework_pins failed. subprocess.run(["bash", ...]) lets
  CreateProcess search System32 first, which starts WSL's bash; it can't see
  D:/ paths, and a CRLF checkout of record_pins.sh breaks any bash anyway.
  It now resolves bash via shutil.which, passes POSIX-style paths, and falls
  back to the existing git rev-parse snapshot when the script can't run.

Most of this was done by Claude Opus 5.5; I reviewed and tested it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 14:11:39 +02:00
Martin Penkava 7ad8519128 vulkan: present through an opaque shader pass instead of a blit
The plain present path copied the VRAM image into the swapchain with
vkCmdBlitImage, alpha included. VRAM alpha is the PSX mask/STP bit, 0 on
most pixels, and AMD's compositor honours it even with an OPAQUE
compositeAlpha: parts of the window went see-through (8x8 black/white
blocks of whatever sat behind it). Reported in #411.

vk_renderer_present_vram/_wide now draw the displayed rect with a
fullscreen triangle whose fragment shader writes alpha = 1 (new
present.vert/.frag). The pass is self-contained and created lazily; if
any of it fails to build, presents fall back to the old blit. Its render
pass ends in TRANSFER_DST_OPTIMAL so finish_present (OSD + present
barrier) is unchanged. Wide surfaces gain SAMPLED usage for it.

Checked on an RX 7900 XT: intro/logo screens render correctly, no new
validation-layer messages.

Most of this was written by Claude Opus 5.5; I reviewed and tested it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 13:41:03 +02:00
Martin Penkava c619fe2b75 debug_server: make the listen socket non-inheritable on Windows
Winsock sockets are inheritable by default. Any child process the runtime
spawns (e.g. the overlay autocompile's `cmd.exe /C python compile_overlays.py`)
inherits the debug server's listening socket; if the game exits while the
child is still running, the child keeps 127.0.0.1:<debug_port> bound and the
next launch cannot bind it (clients get connection refused / talk to a dead
listener).

Observed: after closing the game mid-autocompile, `netstat -ano` showed :4370
LISTENING under the exited game's PID, held by the surviving python child;
killing the compile freed it. Fix: SetHandleInformation(HANDLE_FLAG_INHERIT, 0)
right after socket(). No behavior change on non-Windows.

This change was made with AI assistance (Claude Opus 5.5, Anthropic) and
reviewed/tested by the submitter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 13:11:23 +02:00
Martin Penkava a8f91653d1 vulkan: fix depth/stencil aspect + usage and staging-buffer usage (validation errors)
Found with the Khronos validation layer on AMD RX 7900 XT (Windows, mingw),
where D24S8 is unavailable and choose_ds_format() picks the combined
D32_SFLOAT_S8_UINT:

- VUID-VkImageMemoryBarrier-image-03320: every view/barrier/clear on the
  canonical and wide depth-stencil images named only the STENCIL aspect.
  Without separateDepthStencilLayouts, a combined depth/stencil format must
  be transitioned with both aspects. Added DS_ASPECT and used it for the
  image views, barriers and vkCmdClearDepthStencilImage ranges.
  (vkCmdClearAttachments keeps the stencil-only mask - that is legal.)
- VUID-vkCmdClearDepthStencilImage-pRanges-02659: the canonical DS image is
  cleared with vkCmdClearDepthStencilImage but lacked
  VK_IMAGE_USAGE_TRANSFER_DST_BIT (the wide DS images already had it).
- VUID-vkCmdCopyImageToBuffer-dstBuffer-00191: staging buffers are used for
  readbacks (vkCmdCopyImageToBuffer) but were created with TRANSFER_SRC only.

Validation messages over the same 20 s boot (SmackDown 2, Vulkan):
53 -> 3 (the remaining VUID-vkQueueSubmit-pSignalSemaphores-00067 swapchain
semaphore reuse is untouched). The fix also cleared the downstream
VkImageMemoryBarrier-oldLayout-01213 and vkCmdDraw-None-09600 reports.

This change was made with AI assistance (Claude Opus 5.5, Anthropic) and
reviewed/tested by the submitter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 13:11:02 +02:00
Alex Vanderveen 5c1839675d Merge pull request #407 from RetroPortingToolKit/feat/recomp-json-scaffold
scaffold: setup_project writes a starter .recomp.json (recomp.fyi spec)
2026-09-27 00:13:09 -04:00
Alex Vanderveen 833c591720 scaffold: setup_project writes a starter .recomp.json (recomp.fyi spec)
New projects describe themselves to community lists (recomp.board et al.)
via .recomp.json at the repo root: game, system PS1, type recomp,
toolchain psxrecomp, status exploring, and original region + serial.

write_recomp_json.py takes the serial from disc_probe.json and derives
region from the Sony serial prefix, falling back to --region (which
defaults to USA and would mislabel PAL discs). It only fills missing
fields, so re-running it never clobbers a maintainer's edits.

Both gitignore templates ignored /*.json at the root, which would have
silently dropped the file from the initial commit; un-ignore it.

Closes #406

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 00:07:26 -04:00
Alex Vanderveen 5002187c17 bump 2026-09-27 00:00:46 -04:00
Alex Vanderveen 94552532a2 bump 2026-09-26 23:57:20 -04:00
Matt Stanley 4bb7bd429d Merge pull request #405 from RetroPortingToolKit/fix/aot-feature-presentation-keys
aot: presentation keys and exclude_writes; mods: hide_hidden_mod_features
2026-09-26 16:53:56 -07:00
Matthew Stanley 78534deff4 mods: [runtime] hide_hidden_mod_features opts the launcher into never showing hidden features
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 16:41:47 -07:00
Matthew Stanley 4b4c4b8453 aot: transfer_entries.exclude_writes keeps package-written data out of the transfer inventory
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 16:41:46 -07:00
Matthew Stanley ed07541f13 aot: accept feature presentation keys (hidden, author, channel) in mod package views
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 16:04:43 -07:00
Matt Stanley 68d77ed6a9 Merge pull request #404 from RetroPortingToolKit/fix/vblank-follows-video-mode
runtime: VBlank rate follows the GPU video mode, not the disc region
2026-09-26 16:03:47 -07:00
Matt Stanley 6dd0d2ff5a Merge pull request #401 from RetroPortingToolKit/feat/exclude-builtin-mods
Let titles exclude framework builtin mods
2026-09-26 16:03:43 -07:00
Matthew Stanley 7caef77459 debug_server: report the live video standard and frame timing in vblank_rate
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 15:39:51 -07:00
Matthew Stanley f51617bb7d runtime: VBlank rate follows the GPU video mode (GP1(08h) bit 3), not the disc region
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 15:38:21 -07:00
Matthew Stanley c547f3b967 mods: implicit cross-package feature requirements ([[requirement]], format 7)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 13:58:16 -07:00
Matthew Stanley 7f1c81f1f4 aot: apply disc overlays to the boot EXE in mod package views
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 12:11:40 -07:00
Matthew Stanley 141eb39002 mods: let titles exclude framework builtins with EXCLUDE_BUILTIN_MODS
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 12:11:39 -07:00
Matt Stanley 71336b44d2 Merge pull request #400 from RetroPortingToolKit/feat/cli-generate-aot-static
Generate builds the static overlay shard an AOT profile declares
2026-09-26 10:45:04 -07:00
Matthew Stanley 0995f1bd19 tests: overlay_static_sources_cmake also runs under pytest
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 10:37:18 -07:00
Matthew Stanley bc5ec35998 cli: generate builds the static overlay shard an AOT profile declares
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 10:28:14 -07:00
Matt Stanley 321776b836 Merge pull request #399 from RetroPortingToolKit/feat/aot-nfe-lzss
AOT: indexed_lzss_members method for bit-stream LZSS packs
2026-09-26 10:04:42 -07:00
Matt Stanley 0487e0cb81 Merge pull request #398 from RetroPortingToolKit/feat/8mb-ram-opt-in
Opt-in 8 MB main RAM
2026-09-26 10:04:39 -07:00
Matt Stanley be0d0e7a05 Merge pull request #397 from RetroPortingToolKit/feat/aot-mod-package-images
AOT: images from mod packages; resolve function-entry mod plugins
2026-09-26 10:04:35 -07:00
Matthew Stanley b6a80e5d9e debug_server: dirty_ram_stats per_pc range filter, paging and completeness counts
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 08:17:34 -07:00
Matthew Stanley b43a7c9ea8 aot: indexed_lzss_members verifies identical pack copies, e.g. one linked into the boot EXE
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 08:17:34 -07:00
Matthew Stanley 12d13deff4 aot: indexed_lzss_members method for offset-indexed bit-stream LZSS packs
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 08:17:34 -07:00
Matthew Stanley c67327e42f aot: per-entry static overlay dispatch counters, queried by overlay_static_entries
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 08:17:28 -07:00
Matthew Stanley ec4c91cd94 runtime: GP0 source key and overlay_region_floor follow the 8 MiB window
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 08:03:58 -07:00
Matthew Stanley 1721739a73 debug_server: PSX_WTRACE_BOOT_RANGES gets reserved slots and is refused whole, with the reason in wtrace stats
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 08:03:58 -07:00
Matthew Stanley 0238117450 runtime: keep RAM-mirror PCs architectural; fold only bytes, through one live mask
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 08:03:57 -07:00
Matthew Stanley 18e3030440 savestate: validate every section before applying any; header records the live RAM size
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 08:03:39 -07:00
Matthew Stanley 15ed505656 recompiler: normalize KUSEG header addresses anywhere in the 8 MiB RAM window
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 07:53:18 -07:00
Matthew Stanley ab5d63e1f1 tests: register every offline-tool unit test and make the registration guard scan tools/ and tests/
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 07:36:33 -07:00
Matthew Stanley 199cfe01bc mods: flat function-entry hook table built at activation; interpreter skips it when empty
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 07:34:20 -07:00
Matthew Stanley b0aa7e108d aot: reject images crossing a retail RAM mirror unless the profile declares 8 MiB
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 07:30:03 -07:00
Matthew Stanley 71fdd712db debug_server: seed boot/transition write-ring ranges from PSX_WTRACE_BOOT_RANGES
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 06:43:12 -07:00
Matthew Stanley cd8135406b mods: give the 8 MB RAM builtin a current-schema manifest and parse-test every builtin
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 06:43:12 -07:00