psxrecomp/docs/PLAYTEST_DIAGNOSTICS.md

Shared playtest diagnostics

The PSX recorder follows the bounded background writer used by SNES Mega Man X and the current MediEvil II recorder. It records the controller state actually latched in SIO at each guest VBlank: both button words, analog capability, four stick bytes per pad and independent controller presence. A rotating digest reads 16 KiB of main RAM each VBlank, covering every word across 128 frames. It is a sampled divergence check, not a full machine-state checksum.

Each accepted row is copied to a bounded queue. Formatting and disk writes run on a worker that flushes every 100 ms, including while gameplay is paused. Normal shutdown drains accepted rows. Queue overflow or a write error stops capture explicitly; it never silently drops an input row or blocks gameplay waiting for queue space. Session creation, starting-card/settings copies and the once-per-process executable hash happen outside steady-state recording.

Add a title

Call after creating the runtime target:

include("${PSXRECOMP_ROOT}/runtime/playtest_diagnostics.cmake")
psxrecomp_add_playtest_diagnostics(psx-runtime
    GAME_ID "SLUS-00000" MOD_ID "game.diagnostics")

Create a title mod package with a recording feature whose plugin is game.diagnostics, and a separate input-display feature whose plugin is game.diagnostics.osd. Both features are always player-selectable mods (never channel = "developer"); the build decides their default (below). The helper also covers an existing psx-runtime-pgxp sibling and stages replay-diagnostics.py beside both executables. The bundled-release packager includes this script when present, as does the setup-host kit packager; AppImage wrapping preserves it in the payload. Input Display also works with Recording disabled, with no RAM digest or file capture. Session changes and netplay suspension clear the display. No texture feature, texture package or other enhancement is enabled or exposed by this helper.

Default state: public releases off, everything else on

Owner policy (2026-10-19): Recording and Input Display are off by default only in public production releases. Dev builds and playtest/pre-release packages default them on.

• The source manifest carries the production default: default_enabled = false on both the recording and osd features.
• PSX_PUBLIC_RELEASE is the one build-level notion of a public release. It is derived from PSX_RELEASE_CHANNEL (public => ON; playtest or unset => OFF; runtime/psx_release_channel.cmake). Releases are cut only by tools/package_title_release.py --channel public|playtest (RELEASING_A_TITLE.md); -DPSX_PUBLIC_RELEASE=ON without a channel fails configure.
• After the framework stages the catalog into <exe-dir>/mods/bundled, runtime/psx_playtest_diagnostics_defaults.cmake edits only the staged copy: with PSX_PUBLIC_RELEASE=OFF it sets both features to default_enabled = true; with ON it fails the build unless the source says false.
• psxrecomp_add_playtest_diagnostics applies this to its MOD_ID. A title with its own diagnostics package and recorder opts in with psxrecomp_playtest_diagnostics_defaults(psx-runtime MOD_ID game.diagnostics) (after psxrecomp_add_game_runtime; it also covers the -pgxp sibling).
• tools/ci/audit_playtest_diagnostics_package.py --mode public|playtest checks a finished ZIP: the staged mods/bundled copy must be false in public mode and true in playtest mode, and any shipped source copy must stay false. --mode is required.

A player's own choice in mods/state.toml always overrides the default.

The implementation reads no title-specific addresses. telemetry.csv stores host intervals, incident markers, CD activity, ordinary GP0 draw commands, host presentation calls, effective video configuration and optional render-pass state-verification counters. These are passive evidence and never feed replay or simulation. Press L3 + R3 together to mark an incident and queue a composed screenshot on supported presenters; holding the chord produces one marker/request. Game-specific gameplay samplers remain separate, validated adapters.

Feedback bundle

Recording starts at the first guest VBlank and creates saves/diagnostics/session-<timestamp> beneath the runtime data root. On normal desktop installs this is beside the executable; AppImage launchers use their writable data root. A blocked destination falls back to the OS temporary directory under psxrecomp-diagnostics/<game-id>/session-<timestamp>. diagnostics-location.txt, when writable beside the game, points to the actual folder. The runtime log also reports its full path.

Quit normally after reproducing a problem and zip the complete session folder:

• inputs.csv and session.json describe inputs and execution/build identity.
• telemetry.csv contains timing, incident, CD and renderer evidence.
• incident-<number>.png, when the presenter completed an incident capture, shows composed output rather than only native VRAM. OpenGL and software support this path; headless and Vulkan currently reject it explicitly.
• mods-state.toml, optional settings.toml, and starting cards or explicit absence markers preserve the original choices and save data.
• A slot-state restore starts a new segment with its complete baseline.pst. Rewind segments without a reusable slot image are marked unreplayable.

Discs and BIOS images are never copied into feedback bundles. Starting cards contain the player's saves. Recording and input display are independently controlled by the title's mod features. Netplay frames are skipped; this offline recorder does not claim to reproduce network sessions.

Coverage and effective video state

psxrecomp-telemetry-v2 preserves the original thirteen CSV columns and appends the following evidence. Input-v2 and strict replay checks are unchanged.

• gpu_draw_commands: cumulative ordinary GP0 drawing commands observed by the GPU front end. This is command activity, not a count of correct rendered pixels.
• host_presents / host_present_errors: cumulative returned/failed calls from the actual host present API. OpenGL's direct and threaded paths, Vulkan, and software presentation publish these without requiring debug tools. host_present_status_available says whether the latest host present API supplied a result. SDL2 has no return status, so its activity counter only proves the call returned; zero errors under SDL2 is missing error coverage. Vulkan success includes VK_SUBOPTIMAL_KHR. Neither API proves monitor scanout. Counts include repeated images, pause/overlay presents and generated frames; do not label them unique game frames. GP0 commands and presents need not be 1:1.
• video_renderer: the active backend, after fallback (0 none/headless, 1 software, 2 OpenGL, 3 Vulkan). video_resolution_preset, video_aspect_num/den and video_widescreen sample the emulation owner's settings after mod overrides, including changes during the session.
• video_snapshot_valid: whether a coherent passive backend snapshot was available. composed_* fields describe the last composed image's backend, actual internal scale, native source dimensions, drawable dimensions and fitted viewport dimensions. Dimensions unavailable on a path are zero. A threaded renderer can compose ahead of the swap counter; these fields do not identify the exact image a particular swap displayed. FMV/menus may use a narrower viewport than the configured widescreen mode.
• render_verify_enabled, existing render_verify_checks/mismatch, and render_verify_ever_executed: enablement, cumulative checks/failures and whether any optional extra render-pass state-preservation check has run. These do not monitor ordinary rendering or texture correctness. An enabled check that never executes supplies no coverage. To measure a recording interval, compare the counter increase, not just a preexisting nonzero value.
• visual_verify_available / visual_verify_checks: currently zero because this recorder has no independent pixel oracle. This is explicit missing coverage, never a successful texture test. Screenshots/clips and matched original-game comparisons remain necessary for visible defects.
• incident_capture_request: -1 means no request on this row, 0 rejected, and 1 queued on the existing composed-output capture path. The next presented image is written as incident-<number>.png if capture succeeds; an accepted request alone is not a completed screenshot. The analyzer lists accepted/rejected requests separately from image files present in the bundle. Capturing on an explicit incident may cost a readback/PNG write; ordinary frames do not request pixel capture. A pending diagnostic request can be replaced by a later explicit screenshot request before it is fulfilled.

The backend observer does not read pixels, query GL errors, synchronize the render thread or change guest state. Its snapshot makes at most three attempts and reports unavailable if a coherent publication cannot be sampled.

Analyze either v1 or v2 without launching a game:

python replay-diagnostics.py <session-dir> --analyze

The JSON report separates activity, optional state checks and visual coverage. Old v1 bundles explicitly report absent ordinary-renderer/effective-mode data. Missing probes, zero executed checks, sequence discontinuities and counter resets are visible; zero mismatches alone is never presented as a pass. A torn final row is ignored, while a malformed complete row is rejected.

A diagnostic recording is helpful for exact input/state reproduction, but is not a prerequisite for investigating a visual report. Reproduce known menu, attract-demo, transition and resize cases directly with a fresh private save directory and muted automated output. Capture the composed presentation, since raw/native VRAM screenshots can omit the very widescreen/HD transformation at issue. Preserve location, camera direction, effective mode and before/after images. Escalate to an incident-marked recording when the trigger is unclear or depends on intermittent timing; do not classify an untried case as improbable merely because the reporter supplied no log.

Replay

Use Python 3.8+ (install tomli before Python 3.11) with the recording's original native executable, disc and game configuration. The tool copies the executable, runtime dependencies and starting state into a newly created private directory. Personal settings/cards and the original installation are untouched.

python replay-diagnostics.py <session-dir> --exe <original-executable> \
    --disc <disc.cue> --game <game.toml> --output <new-private-directory>

The source copy is psxrecomp/tools/replay_diagnostics.py. For an AppImage, extract the original image, pass the native usr/bin/<game> executable and the payload's game.toml. Direct AppImage execution is rejected because its launcher can seed personal data. Automated playback uses dummy SDL audio output while retaining emulation; ordinary interactive play keeps normal audio.

Strict playback requires the same input format, execution identity, portable mod-plan identity and build identity. The build identity includes the game's serial/version, game/framework revisions and SHA-256 of the running executable. The binary hash also distinguishes source archives, dirty builds, compiler changes and signed packages. A baseline segment must restore exactly the saved image before inputs are consumed. Playback checks every accepted pad field and sampled RAM digest, stops at the first observed divergence, and reports success only after all complete rows have been checked. A torn final row is ignored.

Developer builds additionally accept --input-only to explore a route across changes. This checks accepted inputs but skips build/mod/RAM equivalence and reports route completion separately. Player builds reject it. Legacy v1 routes require an explicit --legacy-connected assumption because they lacked pad presence. These routes are never counted as verified replay.

Standalone validation

No game, BIOS, SDL window or speakers are needed:

cmake -S runtime/tests/playtest_diagnostics -B build-diagnostics
cmake --build build-diagnostics --parallel 2
ctest --test-dir build-diagnostics --output-on-failure

The fixtures cover queue saturation/owned rows/order/wall-time flushing, record/replay alignment, digital and analog controller removal, incident debouncing, first divergence, identity mismatches, save-state baselines, readonly output fallback, release-build input-only rejection and isolated replay setup. They establish the shared engine-boundary contract. Actual title/disc playtests are still needed to qualify each game's original-build replay. The video tests additionally cover concurrent passive snapshots, composition versus completed presentation, failed presents, live scale/aspect changes, ordinary activity with no optional checks, legacy coverage reporting and the analysis-only CLI.