mirror of
https://github.com/RetroPortingToolKit/psxrecomp
synced 2026-10-26 02:11:36 -04:00
Record real renderer activity and explicit playtest verification coverage
This commit is contained in:
@@ -36,9 +36,12 @@ No texture feature,
|
||||
texture package or other enhancement is enabled or exposed by this helper.
|
||||
|
||||
The implementation reads no title-specific addresses. `telemetry.csv` stores
|
||||
host intervals, incident markers, CD activity and render-pass counters. These
|
||||
are passive evidence and never feed replay or simulation. Press L3 + R3 together
|
||||
to mark an incident; holding the chord produces one marker. Game-specific
|
||||
host intervals, incident markers, CD activity, ordinary GP0 draw commands,
|
||||
host presentation calls, effective video configuration and optional render-pass
|
||||
state-verification counters. These are passive evidence and never feed replay
|
||||
or simulation. Press L3 + R3 together
|
||||
to mark an incident and queue a composed screenshot on supported presenters;
|
||||
holding the chord produces one marker/request. Game-specific
|
||||
gameplay samplers remain separate, validated adapters.
|
||||
|
||||
## Feedback bundle
|
||||
@@ -55,6 +58,9 @@ Quit normally after reproducing a problem and zip the complete session folder:
|
||||
|
||||
- `inputs.csv` and `session.json` describe inputs and execution/build identity.
|
||||
- `telemetry.csv` contains timing, incident, CD and renderer evidence.
|
||||
- `incident-<number>.png`, when the presenter completed an incident capture,
|
||||
shows composed output rather than only native VRAM. OpenGL and software
|
||||
support this path; headless and Vulkan currently reject it explicitly.
|
||||
- `mods-state.toml`, optional `settings.toml`, and starting cards or explicit
|
||||
absence markers preserve the original choices and save data.
|
||||
- A slot-state restore starts a new segment with its complete `baseline.pst`.
|
||||
@@ -65,6 +71,78 @@ contain the player's saves. Recording and input display are independently
|
||||
controlled by the title's mod features. Netplay frames are skipped; this
|
||||
offline recorder does not claim to reproduce network sessions.
|
||||
|
||||
## Coverage and effective video state
|
||||
|
||||
`psxrecomp-telemetry-v2` preserves the original thirteen CSV columns and appends
|
||||
the following evidence. Input-v2 and strict replay checks are unchanged.
|
||||
|
||||
- `gpu_draw_commands`: cumulative ordinary GP0 drawing commands observed by the
|
||||
GPU front end. This is command activity, not a count of correct rendered pixels.
|
||||
- `host_presents` / `host_present_errors`: cumulative returned/failed calls
|
||||
from the actual host present API. OpenGL's direct and threaded paths, Vulkan,
|
||||
and software presentation publish these without requiring debug tools.
|
||||
`host_present_status_available` says whether the latest host present API
|
||||
supplied a result. SDL2 has no return status, so its activity counter only
|
||||
proves the call returned; zero errors under SDL2 is missing error coverage. Vulkan
|
||||
success includes `VK_SUBOPTIMAL_KHR`. Neither API proves monitor scanout.
|
||||
Counts include repeated images, pause/overlay presents and generated frames;
|
||||
do not label them unique game frames. GP0 commands and presents need not be 1:1.
|
||||
- `video_renderer`: the active backend, after fallback (`0` none/headless,
|
||||
`1` software, `2` OpenGL, `3` Vulkan). `video_resolution_preset`,
|
||||
`video_aspect_num/den` and `video_widescreen` sample the emulation owner's
|
||||
settings after mod overrides, including changes during the session.
|
||||
- `video_snapshot_valid`: whether a coherent passive backend snapshot was
|
||||
available. `composed_*` fields describe the last composed image's backend,
|
||||
actual internal scale, native source dimensions, drawable dimensions and
|
||||
fitted viewport dimensions. Dimensions unavailable on a path are zero.
|
||||
A threaded renderer can compose ahead of the swap counter; these fields do
|
||||
not identify the exact image a particular swap displayed. FMV/menus may use
|
||||
a narrower viewport than the configured widescreen mode.
|
||||
- `render_verify_enabled`, existing `render_verify_checks/mismatch`, and
|
||||
`render_verify_ever_executed`: enablement, cumulative checks/failures and
|
||||
whether any optional extra render-pass state-preservation check has run.
|
||||
These do not monitor ordinary rendering or texture correctness. An enabled
|
||||
check that never executes supplies no coverage. To measure a recording
|
||||
interval, compare the counter increase, not just a preexisting nonzero value.
|
||||
- `visual_verify_available` / `visual_verify_checks`: currently zero because
|
||||
this recorder has no independent pixel oracle. This is explicit missing
|
||||
coverage, never a successful texture test. Screenshots/clips and matched
|
||||
original-game comparisons remain necessary for visible defects.
|
||||
- `incident_capture_request`: `-1` means no request on this row, `0` rejected,
|
||||
and `1` queued on the existing composed-output capture path. The next
|
||||
presented image is written as `incident-<number>.png` if capture succeeds;
|
||||
an accepted request alone is not a completed screenshot. The analyzer lists
|
||||
accepted/rejected requests separately from image files present in the bundle.
|
||||
Capturing on an explicit incident may cost a readback/PNG write; ordinary
|
||||
frames do not request pixel capture. A pending diagnostic request can be
|
||||
replaced by a later explicit screenshot request before it is fulfilled.
|
||||
|
||||
The backend observer does not read pixels, query GL errors, synchronize the
|
||||
render thread or change guest state. Its snapshot makes at most three attempts
|
||||
and reports unavailable if a coherent publication cannot be sampled.
|
||||
|
||||
Analyze either v1 or v2 without launching a game:
|
||||
|
||||
```text
|
||||
python replay-diagnostics.py <session-dir> --analyze
|
||||
```
|
||||
|
||||
The JSON report separates activity, optional state checks and visual coverage.
|
||||
Old v1 bundles explicitly report absent ordinary-renderer/effective-mode data.
|
||||
Missing probes, zero executed checks, sequence discontinuities and counter
|
||||
resets are visible; zero mismatches alone is never presented as a pass. A torn
|
||||
final row is ignored, while a malformed complete row is rejected.
|
||||
|
||||
A diagnostic recording is helpful for exact input/state reproduction, but is
|
||||
not a prerequisite for investigating a visual report. Reproduce known menu,
|
||||
attract-demo, transition and resize cases directly with a fresh private save
|
||||
directory and muted automated output. Capture the composed presentation, since
|
||||
raw/native VRAM screenshots can omit the very widescreen/HD transformation at
|
||||
issue. Preserve location, camera direction, effective mode and before/after
|
||||
images. Escalate to an incident-marked recording when the trigger is unclear
|
||||
or depends on intermittent timing; do not classify an untried case as
|
||||
improbable merely because the reporter supplied no log.
|
||||
|
||||
## Replay
|
||||
|
||||
Use Python 3.8+ (install `tomli` before Python 3.11) with the recording's original
|
||||
@@ -114,3 +192,7 @@ debouncing, first divergence, identity mismatches, save-state baselines, readonl
|
||||
output fallback, release-build input-only rejection and isolated replay setup.
|
||||
They establish the shared engine-boundary contract. Actual title/disc playtests
|
||||
are still needed to qualify each game's original-build replay.
|
||||
The video tests additionally cover concurrent passive snapshots, composition
|
||||
versus completed presentation, failed presents, live scale/aspect changes,
|
||||
ordinary activity with no optional checks, legacy coverage reporting and the
|
||||
analysis-only CLI.
|
||||
|
||||
@@ -4,8 +4,14 @@
|
||||
|
||||
namespace psx_playtest_diagnostic {
|
||||
// Host evidence only. No game-specific addresses or classification assumptions.
|
||||
using TelemetryRow = std::array<int64_t,13>;
|
||||
using TelemetryRow = std::array<int64_t,36>;
|
||||
constexpr const char *TelemetryHeader =
|
||||
"sequence,host_elapsed_us,host_interval_us,incident,cd_reading,cd_pending,"
|
||||
"cd_lba,cd_data_read,cd_xa,render_passes,render_aborted,render_verify_checks,render_verify_mismatch\n";
|
||||
"cd_lba,cd_data_read,cd_xa,render_passes,render_aborted,render_verify_checks,render_verify_mismatch,"
|
||||
"gpu_draw_commands,host_presents,host_present_errors,video_snapshot_valid,video_renderer,"
|
||||
"video_resolution_preset,video_aspect_num,video_aspect_den,video_widescreen,"
|
||||
"composed_renderer,composed_internal_scale,composed_source_width,composed_source_height,"
|
||||
"composed_drawable_width,composed_drawable_height,composed_viewport_width,composed_viewport_height,"
|
||||
"render_verify_enabled,render_verify_ever_executed,visual_verify_available,visual_verify_checks,"
|
||||
"incident_capture_request,host_present_status_available\n";
|
||||
}
|
||||
|
||||
@@ -75,6 +75,9 @@ typedef struct RenderPassStats {
|
||||
} RenderPassStats;
|
||||
|
||||
void render_pass_get_stats(RenderPassStats *out);
|
||||
/* Whether optional state-preservation verification is enabled. Zero executed
|
||||
* checks still means no coverage even when this is enabled. Not a pixel test. */
|
||||
int render_pass_verification_enabled(void);
|
||||
/* Clears counters and the sticky fault latch (new mod session). */
|
||||
void render_pass_reset_session(void);
|
||||
typedef struct RenderStereoStats {
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
#ifndef PSX_VIDEO_TELEMETRY_H
|
||||
#define PSX_VIDEO_TELEMETRY_H
|
||||
#include <stdint.h>
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
enum { PSX_VIDEO_NONE = 0, PSX_VIDEO_SOFTWARE = 1,
|
||||
PSX_VIDEO_OPENGL = 2, PSX_VIDEO_VULKAN = 3 };
|
||||
typedef struct PsxVideoTelemetry {
|
||||
uint64_t presents, present_errors;
|
||||
int valid, renderer, internal_scale, present_status_available;
|
||||
int source_width, source_height;
|
||||
int drawable_width, drawable_height;
|
||||
int viewport_width, viewport_height;
|
||||
} PsxVideoTelemetry;
|
||||
|
||||
/* Render-owner publication, before submission. These describe the latest
|
||||
* composed image, which can lead actual swaps when presentation is threaded.
|
||||
* Zero dimensions mean unavailable, not a zero-sized successful image.
|
||||
* A backend has one composing owner at a time. No readbacks or GL syncs. */
|
||||
void psx_video_telemetry_compose(int renderer, int scale, int source_w, int source_h,
|
||||
int drawable_w, int drawable_h, int viewport_w, int viewport_h);
|
||||
/* Call after the backend's swap/present API returns. A success means the host
|
||||
* API accepted/completed the call, not that a monitor displayed correct pixels.
|
||||
* Counts include repeats, overlays and generated frames. */
|
||||
/* result: 1 accepted, 0 failed, -1 returned but API supplies no status (SDL2). */
|
||||
void psx_video_telemetry_present(int result);
|
||||
/* Bounded passive read: valid=0 if a coherent composition was unavailable.
|
||||
* Completion/error counters remain available in that case. */
|
||||
void psx_video_telemetry_snapshot(PsxVideoTelemetry *out);
|
||||
|
||||
/* Emulation-owner settings after runtime mod overrides, sampled every VBlank.
|
||||
* Actual allocated scale/dimensions come from the separate backend snapshot. */
|
||||
typedef struct PsxVideoConfigTelemetry {
|
||||
int renderer, resolution_preset, aspect_num, aspect_den, widescreen;
|
||||
} PsxVideoConfigTelemetry;
|
||||
void psx_video_config_telemetry(PsxVideoConfigTelemetry *out);
|
||||
/* Queue one composed-output PNG on the existing presenter capture path.
|
||||
* Zero means rejected/unavailable. Acceptance is not proof the file was written.
|
||||
* Called only for an explicit owner incident, never for every ordinary frame. */
|
||||
int psx_video_telemetry_capture(const char *path);
|
||||
|
||||
#ifdef __cplusplus
|
||||
}
|
||||
#endif
|
||||
#endif
|
||||
@@ -440,6 +440,7 @@ set(PSXRECOMP_RUNTIME_SOURCES
|
||||
${PSXRECOMP_ROOT}/runtime/src/bios_hle_plan.c
|
||||
${PSXRECOMP_ROOT}/runtime/src/savestate.c
|
||||
${PSXRECOMP_ROOT}/runtime/src/video_texture_filter.c
|
||||
${PSXRECOMP_ROOT}/runtime/src/video_telemetry.cpp
|
||||
${PSXRECOMP_ROOT}/runtime/src/psx_savestate_menu.c
|
||||
${PSXRECOMP_ROOT}/runtime/src/psx_rewind.c
|
||||
${PSXRECOMP_ROOT}/runtime/src/host_osd.c
|
||||
|
||||
@@ -79,6 +79,7 @@
|
||||
#include "gl_scale_limits.h"
|
||||
#include "gpu_sw_renderer.h"
|
||||
#include "gpu_gl_renderer.h"
|
||||
#include "video_telemetry.h"
|
||||
#include "gpu_hd_textures.h"
|
||||
#include "mod_texture_banks.h"
|
||||
#include "gl_batch_policy.h"
|
||||
@@ -1401,6 +1402,10 @@ static int postfx_active(void);
|
||||
static void postfx_present(int fresh, int with_aa, int lx, int ly, int lw, int lh);
|
||||
static void pres_record(int path, int dx, int dy, int w, int h,
|
||||
int lx, int ly, int lw, int lh) {
|
||||
int drawable_w = 0, drawable_h = 0;
|
||||
SDL_GL_GetDrawableSize(s_win, &drawable_w, &drawable_h);
|
||||
psx_video_telemetry_compose(PSX_VIDEO_OPENGL, s_out_scale, w, h,
|
||||
drawable_w, drawable_h, lw, lh);
|
||||
/* The composed game image, before hold-last capture and the OSD: the
|
||||
* post-effect chains (bloom, [video] post_shader, AA; gl_postfx.inc). */
|
||||
if ((path == GL_PRES_VRAM || path == GL_PRES_WIDE || path == GL_PRES_INTERP) && postfx_active())
|
||||
@@ -9606,7 +9611,14 @@ static int pt_cb_ctx(void *user, int current) {
|
||||
|
||||
/* The one owned swap call: the composing thread's (direct) or the present
|
||||
* thread's. */
|
||||
static void gl_window_swap(void) { SDL_GL_SwapWindow(s_win); }
|
||||
static void gl_window_swap(void) {
|
||||
#if defined(PSX_SDL3)
|
||||
psx_video_telemetry_present(SDL_GL_SwapWindow(s_win) ? 1 : 0);
|
||||
#else
|
||||
SDL_GL_SwapWindow(s_win);
|
||||
psx_video_telemetry_present(-1); // SDL2 exposes no swap result.
|
||||
#endif
|
||||
}
|
||||
|
||||
static void *pt_cb_present(void *user, int slot, void *ready) {
|
||||
(void)user;
|
||||
|
||||
@@ -20,6 +20,7 @@
|
||||
#include "gpu.h"
|
||||
#include "gpu_render.h"
|
||||
#include "gpu_vk_renderer.h"
|
||||
#include "video_telemetry.h"
|
||||
#include "gpu_sw_renderer.h"
|
||||
#include "host_osd.h"
|
||||
#include "psx_savestate_menu.h"
|
||||
@@ -1675,7 +1676,10 @@ static void submit_present(VkCommandBuffer cb, uint32_t img_idx, uint32_t fr) {
|
||||
pi.swapchainCount = 1; pi.pSwapchains = &s_swapchain;
|
||||
pi.pImageIndices = &img_idx;
|
||||
uint64_t present_start = SDL_GetPerformanceCounter();
|
||||
p_vkQueuePresentKHR(s_queue, &pi);
|
||||
psx_video_telemetry_compose(PSX_VIDEO_VULKAN, s_scale, 0, 0,
|
||||
(int)s_sc_extent.width, (int)s_sc_extent.height, 0, 0);
|
||||
const VkResult present_result = p_vkQueuePresentKHR(s_queue, &pi);
|
||||
psx_video_telemetry_present(present_result == VK_SUCCESS || present_result == VK_SUBOPTIMAL_KHR);
|
||||
s_perf_cur.present_us += perf_elapsed_us(present_start);
|
||||
s_frame_idx++;
|
||||
}
|
||||
|
||||
+42
-4
@@ -123,6 +123,7 @@ extern "C" uint32_t g_guest_last_store_addr, g_guest_last_store_val;
|
||||
#include "mod_session_baseline.h"
|
||||
#include "mod_runtime.h"
|
||||
#include "video_texture_filter.h"
|
||||
#include "video_telemetry.h"
|
||||
#include "host_launch_timing.h"
|
||||
#include "mod_packages.h"
|
||||
#include "present_image_ring.h"
|
||||
@@ -2553,6 +2554,38 @@ extern "C" void psx_video_resolution_info(int *preset, int *ref_lines, int *requ
|
||||
if (px_h) *px_h = ph;
|
||||
}
|
||||
|
||||
extern "C" void psx_video_config_telemetry(PsxVideoConfigTelemetry *out) {
|
||||
if (!out) return;
|
||||
*out = {};
|
||||
out->renderer = g_gl_active ? PSX_VIDEO_OPENGL : g_vk_active ? PSX_VIDEO_VULKAN
|
||||
: g_headless ? PSX_VIDEO_NONE : PSX_VIDEO_SOFTWARE;
|
||||
out->resolution_preset = effective_internal_resolution();
|
||||
out->aspect_num = g_video_aspect_num;
|
||||
out->aspect_den = g_video_aspect_den;
|
||||
out->widescreen = g_ws_engaged && g_video_aspect_num * 3 != g_video_aspect_den * 4;
|
||||
}
|
||||
|
||||
#ifndef PSX_SDL_NO_RENDER
|
||||
/* All software swaps, including display-disabled and held frames. Query only
|
||||
* the owning SDL renderer; never synchronize another render thread. */
|
||||
static void software_present_with_telemetry(SDL_Renderer *renderer) {
|
||||
int width = 0, height = 0;
|
||||
#if defined(PSX_SDL3)
|
||||
SDL_GetRenderOutputSize(renderer, &width, &height);
|
||||
#else
|
||||
SDL_GetRendererOutputSize(renderer, &width, &height);
|
||||
#endif
|
||||
psx_video_telemetry_compose(PSX_VIDEO_SOFTWARE, gr_scale(), 0, 0,
|
||||
width, height, 0, 0);
|
||||
#if defined(PSX_SDL3)
|
||||
psx_video_telemetry_present(SDL_RenderPresent(renderer) ? 1 : 0);
|
||||
#else
|
||||
SDL_RenderPresent(renderer);
|
||||
psx_video_telemetry_present(-1); // SDL2's API has no return status.
|
||||
#endif
|
||||
}
|
||||
#endif
|
||||
|
||||
extern "C" int present_shot_request(const char *path)
|
||||
{
|
||||
if (!path || !*path) return 0;
|
||||
@@ -2572,6 +2605,11 @@ extern "C" int present_shot_request(const char *path)
|
||||
|
||||
/* Backend hook: atomically claim a staged shot (returns 1 and fills `out`),
|
||||
* then call present_shot_done(ok) once the PNG is written — or not written. */
|
||||
extern "C" int psx_video_telemetry_capture(const char *path) {
|
||||
if (!path || std::strlen(path) >= sizeof(s_present_shot_path)) return 0;
|
||||
return present_shot_request(path);
|
||||
}
|
||||
|
||||
extern "C" int present_shot_take(char *out, int n)
|
||||
{
|
||||
if (!out || n <= 0) return 0;
|
||||
@@ -7548,7 +7586,7 @@ static void netplay_hold_last_present_tick(void) {
|
||||
SDL_RenderClear(sdl_renderer);
|
||||
SDL_RenderCopy(sdl_renderer, sdl_texture, &s_sw_hold_src, &s_sw_hold_dst);
|
||||
host_osd_draw_sdl(sdl_renderer);
|
||||
SDL_RenderPresent(sdl_renderer);
|
||||
software_present_with_telemetry(sdl_renderer);
|
||||
did = 1;
|
||||
}
|
||||
#endif
|
||||
@@ -8253,7 +8291,7 @@ static void rewind_pause_present(void) {
|
||||
SDL_RenderCopy(sdl_renderer, sdl_texture, &s_sw_hold_src,
|
||||
&s_sw_hold_dst);
|
||||
host_osd_draw_sdl(sdl_renderer);
|
||||
SDL_RenderPresent(sdl_renderer);
|
||||
software_present_with_telemetry(sdl_renderer);
|
||||
}
|
||||
#endif
|
||||
}
|
||||
@@ -9394,7 +9432,7 @@ static NetplayVblankEpilogue sdl_vblank_present_body(void) {
|
||||
SDL_SetRenderDrawColor(sdl_renderer, 0, 0, 0, 255);
|
||||
SDL_RenderClear(sdl_renderer);
|
||||
host_osd_draw_sdl(sdl_renderer);
|
||||
SDL_RenderPresent(sdl_renderer);
|
||||
software_present_with_telemetry(sdl_renderer);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -9874,7 +9912,7 @@ static NetplayVblankEpilogue sdl_vblank_present_body(void) {
|
||||
{
|
||||
latency_ring_mark(LAT_SWAP_BEGIN);
|
||||
const Uint64 t0 = SDL_GetPerformanceCounter();
|
||||
SDL_RenderPresent(sdl_renderer);
|
||||
software_present_with_telemetry(sdl_renderer);
|
||||
const Uint64 t1 = SDL_GetPerformanceCounter();
|
||||
latency_ring_mark(LAT_SWAP_END);
|
||||
netplay_note_present();
|
||||
|
||||
@@ -22,6 +22,8 @@
|
||||
#include "playtest_diagnostic_telemetry.h"
|
||||
#include "cdrom.h"
|
||||
#include "render_pass.h"
|
||||
#include "gpu.h"
|
||||
#include "video_telemetry.h"
|
||||
#include "psx_sha256_file.h"
|
||||
#include <array>
|
||||
#include <algorithm>
|
||||
@@ -61,6 +63,7 @@ psx_playtest_diagnostic::Writer<4096, psx_playtest_diagnostic::TelemetryRow> tel
|
||||
std::chrono::steady_clock::time_point telemetry_start, telemetry_last;
|
||||
unsigned incident;
|
||||
bool incident_held, telemetry_failed;
|
||||
std::filesystem::path recording_folder;
|
||||
std::ifstream replay;
|
||||
Row pending{}; // replay row the next VBlank verifies
|
||||
bool pending_valid;
|
||||
@@ -285,7 +288,8 @@ void seed_first_row() {
|
||||
std::string session_json(bool netplay) {
|
||||
std::ostringstream out;
|
||||
out << "{\n \"format_version\": 1,\n \"input_format\": \"" PSX_DIAGNOSTIC_INPUT_FORMAT "\",\n"
|
||||
<< " \"telemetry_format\": \"psxrecomp-telemetry-v1\",\n"
|
||||
<< " \"telemetry_format\": \"psxrecomp-telemetry-v2\",\n"
|
||||
<< " \"visual_verification\": \"unavailable: no independent pixel comparison\",\n"
|
||||
<< " \"game_id\": \"" PSX_DIAGNOSTIC_GAME_ID "\",\n"
|
||||
<< " \"boundary\": \"guest-vblank-sio-pad-state\",\n"
|
||||
<< " \"digest\": \"fnv-mix over " << DigestWords << " main-RAM words per VBlank, stride "
|
||||
@@ -309,6 +313,7 @@ void begin_recording() {
|
||||
std::chrono::system_clock::now().time_since_epoch()).count();
|
||||
const auto folder = psx_playtest_diagnostic_session_directory(
|
||||
application_directory(), PSX_DIAGNOSTIC_GAME_ID, "session-" + std::to_string(stamp));
|
||||
recording_folder = folder;
|
||||
recording.open(folder / "inputs.csv", std::string("# " PSX_DIAGNOSTIC_INPUT_FORMAT "\n# execution ") +
|
||||
PSX_EXECUTION_ID + "\n# plan " + plan_identity() + "\n# build " + build_identity() + "\n");
|
||||
if (!recording) throw std::runtime_error(recording.error_text());
|
||||
@@ -384,8 +389,11 @@ void capture_telemetry(const Row& input) {
|
||||
row[2] = sequence ? std::chrono::duration_cast<std::chrono::microseconds>(now - telemetry_last).count() : 0;
|
||||
telemetry_last = now;
|
||||
const bool held = (input[ColButtons] & 6) == 0;
|
||||
row[34] = -1; // No capture requested on this sample.
|
||||
if (held && !incident_held) {
|
||||
++incident;
|
||||
const auto image = recording_folder / ("incident-" + std::to_string(incident) + ".png");
|
||||
row[34] = psx_video_telemetry_capture(psx_playtest_diagnostic_utf8(image).c_str()) ? 1 : 0;
|
||||
host_osd_push(("Playtest incident " + std::to_string(incident) + " marked").c_str(), 2000);
|
||||
}
|
||||
incident_held = held; row[3] = incident;
|
||||
@@ -396,6 +404,21 @@ void capture_telemetry(const Row& input) {
|
||||
RenderPassStats render{}; render_pass_get_stats(&render);
|
||||
row[9] = render.passes; row[10] = render.aborted;
|
||||
row[11] = render.verify_checks; row[12] = render.verify_mismatch;
|
||||
uint64_t nop = 0, fill = 0, draw_commands = 0, environment = 0, copy = 0;
|
||||
gpu_get_gp0_stats(&nop, &fill, &draw_commands, &environment, ©);
|
||||
PsxVideoTelemetry video{}; psx_video_telemetry_snapshot(&video);
|
||||
PsxVideoConfigTelemetry config{}; psx_video_config_telemetry(&config);
|
||||
row[13] = draw_commands; row[14] = video.presents; row[15] = video.present_errors;
|
||||
row[16] = video.valid; row[17] = config.renderer;
|
||||
row[18] = config.resolution_preset;
|
||||
row[19] = config.aspect_num; row[20] = config.aspect_den; row[21] = config.widescreen;
|
||||
row[22] = video.renderer; row[23] = video.internal_scale;
|
||||
row[24] = video.source_width; row[25] = video.source_height;
|
||||
row[26] = video.drawable_width; row[27] = video.drawable_height;
|
||||
row[28] = video.viewport_width; row[29] = video.viewport_height;
|
||||
row[30] = render_pass_verification_enabled(); row[31] = render.verify_checks != 0;
|
||||
row[32] = 0; row[33] = 0; // No independent visual oracle is wired here.
|
||||
row[35] = video.present_status_available;
|
||||
telemetry.native(row);
|
||||
if (!telemetry) {
|
||||
telemetry_failed = true;
|
||||
|
||||
@@ -324,6 +324,8 @@ void render_pass_get_stats(RenderPassStats *out) {
|
||||
out->dropped[i] = g_render_pass_dropped_writes[i];
|
||||
}
|
||||
|
||||
int render_pass_verification_enabled(void) { return verify_on(); }
|
||||
|
||||
void render_pass_reset_session(void) {
|
||||
s_netplay_allowed = 0;
|
||||
memset(&s_stats, 0, sizeof s_stats);
|
||||
|
||||
@@ -0,0 +1,46 @@
|
||||
#include "video_telemetry.h"
|
||||
#include <atomic>
|
||||
|
||||
namespace {
|
||||
std::atomic<uint64_t> presents{0}, errors{0};
|
||||
std::atomic<int> status_available{0};
|
||||
std::atomic<unsigned> generation{0};
|
||||
std::atomic<int> fields[8]{};
|
||||
}
|
||||
|
||||
extern "C" void psx_video_telemetry_compose(int renderer, int scale, int sw, int sh,
|
||||
int dw, int dh, int vw, int vh) {
|
||||
const int values[] = {renderer, scale, sw, sh, dw, dh, vw, vh};
|
||||
// Single composing owner. Atomic fields avoid data races even when the
|
||||
// bounded reader abandons an in-progress publication. This observer never
|
||||
// waits for the renderer, queries GL, hashes pixels or changes guest state.
|
||||
generation.fetch_add(1);
|
||||
for (unsigned i = 0; i < 8; ++i) fields[i].store(values[i]);
|
||||
generation.fetch_add(1);
|
||||
}
|
||||
|
||||
extern "C" void psx_video_telemetry_present(int result) {
|
||||
status_available.store(result >= 0, std::memory_order_relaxed);
|
||||
(result != 0 ? presents : errors).fetch_add(1, std::memory_order_relaxed);
|
||||
}
|
||||
|
||||
extern "C" void psx_video_telemetry_snapshot(PsxVideoTelemetry *out) {
|
||||
if (!out) return;
|
||||
*out = {};
|
||||
out->presents = presents.load(std::memory_order_relaxed);
|
||||
out->present_errors = errors.load(std::memory_order_relaxed);
|
||||
out->present_status_available = status_available.load(std::memory_order_relaxed);
|
||||
for (unsigned attempt = 0; attempt < 3; ++attempt) {
|
||||
const unsigned before = generation.load();
|
||||
if (!before || (before & 1u)) continue;
|
||||
int values[8];
|
||||
for (unsigned i = 0; i < 8; ++i) values[i] = fields[i].load();
|
||||
if (before != generation.load()) continue;
|
||||
out->valid = 1;
|
||||
out->renderer = values[0]; out->internal_scale = values[1];
|
||||
out->source_width = values[2]; out->source_height = values[3];
|
||||
out->drawable_width = values[4]; out->drawable_height = values[5];
|
||||
out->viewport_width = values[6]; out->viewport_height = values[7];
|
||||
return;
|
||||
}
|
||||
}
|
||||
@@ -13,7 +13,7 @@ foreach(_test writer boundary player)
|
||||
target_include_directories(playtest_${_test}_test PRIVATE ../../include)
|
||||
target_link_libraries(playtest_${_test}_test PRIVATE Threads::Threads)
|
||||
if(NOT _test STREQUAL "writer")
|
||||
target_sources(playtest_${_test}_test PRIVATE ../../src/psx_sha256_file.cpp ../../src/psx_sha256.c)
|
||||
target_sources(playtest_${_test}_test PRIVATE ../../src/psx_sha256_file.cpp ../../src/psx_sha256.c ../../src/video_telemetry.cpp)
|
||||
target_compile_definitions(playtest_${_test}_test PRIVATE PSX_SHA256_FILE_PORTABLE=1)
|
||||
endif()
|
||||
if(_test STREQUAL "player")
|
||||
@@ -23,9 +23,17 @@ foreach(_test writer boundary player)
|
||||
"${CMAKE_CURRENT_BINARY_DIR}/${_test}-evidence")
|
||||
set_tests_properties(playtest_${_test}_test PROPERTIES TIMEOUT 20)
|
||||
endforeach()
|
||||
add_executable(video_telemetry_test ../test_video_telemetry.cpp ../../src/video_telemetry.cpp)
|
||||
target_include_directories(video_telemetry_test PRIVATE ../../include)
|
||||
target_compile_features(video_telemetry_test PRIVATE cxx_std_17)
|
||||
target_link_libraries(video_telemetry_test PRIVATE Threads::Threads)
|
||||
add_test(NAME video_telemetry_test COMMAND video_telemetry_test)
|
||||
find_package(Python3 COMPONENTS Interpreter)
|
||||
if(Python3_FOUND)
|
||||
add_test(NAME replay_diagnostics_prepare_test
|
||||
COMMAND "${Python3_EXECUTABLE}" -I
|
||||
"${CMAKE_CURRENT_SOURCE_DIR}/../../../tools/tests/test_replay_diagnostics.py")
|
||||
add_test(NAME playtest_telemetry_analysis_test
|
||||
COMMAND "${Python3_EXECUTABLE}" -I
|
||||
"${CMAKE_CURRENT_SOURCE_DIR}/../../../tools/tests/test_playtest_telemetry_analysis.py")
|
||||
endif()
|
||||
|
||||
@@ -12,6 +12,8 @@
|
||||
#include "sio.h"
|
||||
#include "cdrom.h"
|
||||
#include "render_pass.h"
|
||||
#include "gpu.h"
|
||||
#include "video_telemetry.h"
|
||||
#include <array>
|
||||
#include <cstdio>
|
||||
#include <cstring>
|
||||
@@ -40,6 +42,9 @@ static std::string plan_text="fixture-plan";
|
||||
static void (*activation)(),(*vblank)(),(*savestate)(),(*osd_activation)(),(*osd_vblank)(),(*session_reset)();
|
||||
static void (*recording_activation)();
|
||||
static uint64_t ram_reads;
|
||||
static uint64_t ordinary_draws;
|
||||
static PsxVideoConfigTelemetry video_config{PSX_VIDEO_OPENGL, 1, 4, 3, 0};
|
||||
static std::vector<std::string> incident_images;
|
||||
|
||||
static void activate_recording_plan() {session_reset();recording_activation();}
|
||||
|
||||
@@ -78,6 +83,13 @@ void cdrom_debug_snapshot(CDROMDebugState *out) {*out={};}
|
||||
int cdrom_data_read_active(void) {return 0;}
|
||||
int cdrom_xa_stream_active(void) {return 0;}
|
||||
void render_pass_get_stats(RenderPassStats *out) {*out={};}
|
||||
int render_pass_verification_enabled(void) {return 0;}
|
||||
void gpu_get_gp0_stats(uint64_t *nop, uint64_t *fill, uint64_t *draw,
|
||||
uint64_t *environment, uint64_t *copy) {
|
||||
*nop = *fill = *environment = *copy = 0; *draw = ordinary_draws;
|
||||
}
|
||||
void psx_video_config_telemetry(PsxVideoConfigTelemetry *out) {*out=video_config;}
|
||||
int psx_video_telemetry_capture(const char *path) {incident_images.emplace_back(path);return 1;}
|
||||
void savestate_status_json(char *buf,size_t cap) {
|
||||
snprintf(buf,cap,"\"generation\":%u,\"pending\":%u,\"last_ok\":%d,\"last_op\":\"%s\",\"last_slot\":%d",
|
||||
status_generation,status_pending,status_ok,status_op,status_slot);
|
||||
@@ -153,6 +165,13 @@ static void guest_frame(unsigned t) {
|
||||
// VBlank of that frame to model a diverging guest.
|
||||
static void run_frames(unsigned from,unsigned to,int tamper=-1) {
|
||||
for(unsigned t=from;t<to;++t) {
|
||||
ordinary_draws += 17;
|
||||
if (t == 50) video_config = {PSX_VIDEO_OPENGL, 1080, 16, 9, 1};
|
||||
psx_video_telemetry_compose(PSX_VIDEO_OPENGL, t < 50 ? 1 : 5,
|
||||
t < 50 ? 320 : 426, 240,
|
||||
t < 50 ? 640 : 1920, t < 50 ? 480 : 1080,
|
||||
t < 50 ? 640 : 1920, t < 50 ? 480 : 1080);
|
||||
psx_video_telemetry_present(1);
|
||||
vblank();
|
||||
if(osd_enabled)osd_vblank();
|
||||
if((int)t==tamper)memory[0x80000]^=1;
|
||||
@@ -229,6 +248,26 @@ int main(int argc,char **argv) {
|
||||
auto timings=slurp(session/"telemetry.csv");
|
||||
CHECK(timings.rfind("sequence,host_elapsed_us,host_interval_us,",0)==0);
|
||||
CHECK(std::count(timings.begin(),timings.end(),'\n')==201);
|
||||
CHECK(timings.find("gpu_draw_commands,host_presents,host_present_errors")!=std::string::npos);
|
||||
CHECK(timings.find("visual_verify_available,visual_verify_checks")!=std::string::npos);
|
||||
// Ordinary frames are observable with no optional render passes/checks.
|
||||
// Runtime changes must be sampled, rather than copied from saved settings.
|
||||
{std::istringstream csv(timings);std::string line;std::getline(csv,line);unsigned capture_requests=0;
|
||||
for(unsigned n=0;n<200;++n) {
|
||||
CHECK(bool(std::getline(csv,line)));std::replace(line.begin(),line.end(),',',' ');
|
||||
std::istringstream values(line);psx_playtest_diagnostic::TelemetryRow row{};
|
||||
for(auto& value:row)CHECK(bool(values>>value));
|
||||
std::string extra;CHECK(!(values>>extra));
|
||||
CHECK(row[9]==0 && row[11]==0 && row[12]==0);
|
||||
CHECK(row[13]>0 && row[14]>0 && row[16]==1);
|
||||
CHECK(row[18]==(n<50?1:1080) && row[19]==(n<50?4:16));
|
||||
CHECK(row[23]==(n<50?1:5) && row[26]==(n<50?640:1920));
|
||||
CHECK(row[30]==0 && row[31]==0 && row[32]==0 && row[33]==0);
|
||||
CHECK(row[34]==-1 || row[34]==1);capture_requests+=row[34]==1;
|
||||
CHECK(row[35]==1);
|
||||
}
|
||||
CHECK(capture_requests==1 && incident_images.size()==1);
|
||||
CHECK(std::filesystem::u8path(incident_images[0])==(session/"incident-1.png"));}
|
||||
CHECK(incident==1); // holding the chord across five frames marks once
|
||||
// Row 0 is the initial SIO state; row 11 shows Cross, which the model's
|
||||
// script first sampled at the boundary after frame 10... recorded one
|
||||
@@ -239,6 +278,7 @@ int main(int argc,char **argv) {
|
||||
CHECK(std::filesystem::exists(session/"card2.absent"));
|
||||
CHECK(std::filesystem::exists(session/"mods-state.toml") && std::filesystem::exists(session/"settings.toml"));
|
||||
CHECK(slurp(session/"session.json").find("\"netplay\": false")!=std::string::npos);
|
||||
CHECK(slurp(session/"session.json").find("psxrecomp-telemetry-v2")!=std::string::npos);
|
||||
CHECK(slurp(session/"session.json").find("baseline")==std::string::npos);
|
||||
{auto location=slurp(std::filesystem::path(save_root)/"diagnostics-location.txt");
|
||||
CHECK(location.find(psx_playtest_diagnostic_utf8(std::filesystem::absolute(session).make_preferred()))!=std::string::npos &&
|
||||
|
||||
@@ -0,0 +1,57 @@
|
||||
#include "video_telemetry.h"
|
||||
#include <atomic>
|
||||
#include <cstdio>
|
||||
#include <thread>
|
||||
#define CHECK(value) do { if (!(value)) { std::fprintf(stderr, "line %d: %s\n", __LINE__, #value); return 1; } } while (0)
|
||||
|
||||
int main() {
|
||||
PsxVideoTelemetry sample{};
|
||||
psx_video_telemetry_snapshot(&sample);
|
||||
CHECK(!sample.valid && !sample.presents && !sample.present_errors);
|
||||
// A composition is not a swap. A failed present is not a completed one.
|
||||
psx_video_telemetry_compose(PSX_VIDEO_SOFTWARE, 1, 320, 240, 640, 480, 640, 480);
|
||||
psx_video_telemetry_snapshot(&sample);
|
||||
CHECK(sample.valid && sample.renderer == PSX_VIDEO_SOFTWARE && !sample.presents);
|
||||
psx_video_telemetry_present(0);
|
||||
psx_video_telemetry_snapshot(&sample);
|
||||
CHECK(sample.present_errors == 1 && sample.presents == 0);
|
||||
CHECK(sample.present_status_available);
|
||||
psx_video_telemetry_present(1);
|
||||
psx_video_telemetry_snapshot(&sample);
|
||||
CHECK(sample.presents == 1 && sample.present_errors == 1);
|
||||
psx_video_telemetry_present(-1); // SDL2: the call returned, no status exists.
|
||||
psx_video_telemetry_snapshot(&sample);
|
||||
CHECK(!sample.present_status_available && sample.presents == 2);
|
||||
// A live resize and scale change publish together. A busy writer permits
|
||||
// an unavailable snapshot, never a fabricated/mixed coherent result.
|
||||
std::atomic<bool> done{false};
|
||||
std::thread producer([&] {
|
||||
for (int scale = 1; scale <= 10000; ++scale) {
|
||||
psx_video_telemetry_compose(PSX_VIDEO_OPENGL, scale,
|
||||
320 * scale, 240 * scale,
|
||||
640 * scale, 480 * scale,
|
||||
640 * scale, 480 * scale);
|
||||
psx_video_telemetry_present(1);
|
||||
}
|
||||
done.store(true);
|
||||
});
|
||||
bool consistent = true;
|
||||
unsigned observations = 0;
|
||||
do {
|
||||
psx_video_telemetry_snapshot(&sample);
|
||||
if (sample.valid && sample.renderer == PSX_VIDEO_OPENGL) {
|
||||
consistent &= sample.source_width == sample.internal_scale * 320 &&
|
||||
sample.drawable_width == sample.internal_scale * 640 &&
|
||||
sample.viewport_height == sample.internal_scale * 480;
|
||||
++observations;
|
||||
}
|
||||
} while (!done.load());
|
||||
producer.join();
|
||||
psx_video_telemetry_snapshot(&sample);
|
||||
CHECK(consistent && observations > 0);
|
||||
CHECK(sample.valid && sample.internal_scale == 10000 && sample.presents == 10002);
|
||||
CHECK(sample.present_status_available);
|
||||
CHECK(sample.present_errors == 1);
|
||||
psx_video_telemetry_snapshot(nullptr);
|
||||
return 0;
|
||||
}
|
||||
+112
-3
@@ -22,6 +22,8 @@ does not verify build/mod identity or RAM. V1 routes require an explicit
|
||||
An input-only completion is never an exact replay, and player builds reject it.
|
||||
"""
|
||||
import argparse
|
||||
import csv
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
@@ -39,6 +41,107 @@ ENV_PREFIXES = ('PSX_', 'RNET_', 'LNG_', 'PSX_DIAGNOSTIC_', 'RECOMP_')
|
||||
LOG_PREFIX = 'psxrecomp.diagnostics: '
|
||||
|
||||
|
||||
def analyze_telemetry(recording):
|
||||
"""Read passive evidence without launching a game or certifying pixels.
|
||||
|
||||
Old bundles remain readable, with missing coverage stated explicitly.
|
||||
Counters may reset across state changes; report those resets separately.
|
||||
"""
|
||||
folder = recording if recording.is_dir() else recording.parent
|
||||
metadata_path = folder / 'session.json'
|
||||
metadata = json.loads(metadata_path.read_text(encoding='utf-8')) if metadata_path.exists() else {}
|
||||
version = metadata.get('telemetry_format')
|
||||
if version not in (None, 'psxrecomp-telemetry-v1', 'psxrecomp-telemetry-v2'):
|
||||
raise ValueError('Unsupported telemetry format: ' + str(version))
|
||||
raw = (folder / 'telemetry.csv').read_text(encoding='utf-8')
|
||||
lines = raw.splitlines(keepends=True)
|
||||
torn = bool(lines and not lines[-1].endswith('\n'))
|
||||
if torn:
|
||||
lines.pop()
|
||||
reader = csv.DictReader(io.StringIO(''.join(lines)))
|
||||
columns = set(reader.fieldnames or [])
|
||||
required = {'sequence', 'host_elapsed_us', 'host_interval_us', 'render_verify_checks',
|
||||
'render_verify_mismatch'}
|
||||
if not required <= columns:
|
||||
raise ValueError('Incomplete telemetry header')
|
||||
rows = []
|
||||
for row in reader:
|
||||
if None in row or any(value is None for value in row.values()):
|
||||
raise ValueError('Incomplete telemetry row')
|
||||
rows.append({key: int(value) for key, value in row.items()})
|
||||
if not rows:
|
||||
raise ValueError('No complete telemetry samples')
|
||||
|
||||
def counter(name):
|
||||
if name not in columns:
|
||||
return {'available': False}
|
||||
values = [row[name] for row in rows]
|
||||
return {'available': True, 'first': values[0], 'last': values[-1],
|
||||
'observed_increase': sum(max(b - a, 0) for a, b in zip(values, values[1:])),
|
||||
'resets': sum(b < a for a, b in zip(values, values[1:]))}
|
||||
|
||||
warnings = []
|
||||
draws, presents = counter('gpu_draw_commands'), counter('host_presents')
|
||||
checks, mismatches = counter('render_verify_checks'), counter('render_verify_mismatch')
|
||||
covered = checks['observed_increase'] > 0
|
||||
if not draws['available'] or not presents['available']:
|
||||
warnings.append('Ordinary drawing/presentation coverage was not recorded by this bundle.')
|
||||
status_samples = [row['host_present_status_available'] for previous, row in zip(rows, rows[1:])
|
||||
if row.get('host_presents') != previous.get('host_presents')
|
||||
or row.get('host_present_errors') != previous.get('host_present_errors')] if 'host_present_status_available' in columns else []
|
||||
present_status = all(status_samples) if 'host_present_status_available' in columns and status_samples else None
|
||||
if not present_status:
|
||||
warnings.append('Host present error status was not available for all samples; SDL2 only proves a present call returned.')
|
||||
if not covered:
|
||||
warnings.append('No render-pass state-preservation checks executed in the observed interval; zero mismatches is not a pass.')
|
||||
warnings.append('Activity counters and state-preservation checks do not verify texture correctness; independent visual comparison is required.')
|
||||
mode_fields = ('video_renderer', 'video_resolution_preset', 'video_aspect_num',
|
||||
'video_aspect_den', 'video_widescreen')
|
||||
modes = sorted({tuple(row[key] for key in mode_fields) for row in rows}) if set(mode_fields) <= columns else []
|
||||
composed_fields = ('composed_renderer', 'composed_internal_scale', 'composed_source_width',
|
||||
'composed_source_height', 'composed_drawable_width', 'composed_drawable_height',
|
||||
'composed_viewport_width', 'composed_viewport_height')
|
||||
composed = sorted({tuple(row[key] for key in composed_fields) for row in rows
|
||||
if row.get('video_snapshot_valid')}) if set(composed_fields) <= columns else []
|
||||
if not modes:
|
||||
warnings.append('Effective runtime video configuration was not recorded; saved settings and mod state are requests, not proof of the live mode.')
|
||||
gaps = sum(b['sequence'] != a['sequence'] + 1 for a, b in zip(rows, rows[1:]))
|
||||
if gaps:
|
||||
warnings.append('Telemetry sequence gaps or resets are present; do not treat the capture as contiguous.')
|
||||
enabled = any(row.get('render_verify_enabled', 0) for row in rows) if 'render_verify_enabled' in columns else None
|
||||
return {
|
||||
'telemetry_format': version or 'inferred from header',
|
||||
'samples': len(rows), 'sequence_discontinuities': gaps,
|
||||
'ignored_torn_final_row': torn,
|
||||
'host_elapsed_seconds': (rows[-1]['host_elapsed_us'] - rows[0]['host_elapsed_us']) / 1e6,
|
||||
'max_vblank_interval_ms': max(row['host_interval_us'] for row in rows) / 1000,
|
||||
'timing_scope': 'Host intervals between guest VBlanks; not unique game frames or presentation latency.',
|
||||
'ordinary_draw_commands': draws, 'host_present_calls': presents,
|
||||
'host_present_errors': counter('host_present_errors'),
|
||||
'host_present_status_available_all_observed_calls': present_status,
|
||||
'render_state_verification': {'enabled': enabled, 'covered_in_observed_interval': covered,
|
||||
'checks': checks, 'mismatches': mismatches,
|
||||
'scope': 'Optional extra render-pass state preservation, not normal rendering or pixels.'},
|
||||
'visual_verification': {'available': any(row.get('visual_verify_available', 0) for row in rows),
|
||||
'checks': counter('visual_verify_checks'), 'result': 'not evaluated'},
|
||||
'effective_video_modes': [dict(zip(mode_fields, values)) for values in modes[:32]],
|
||||
'effective_video_mode_count': len(modes),
|
||||
'composed_video_modes': [dict(zip(composed_fields, values)) for values in composed[:32]],
|
||||
'composed_video_mode_count': len(composed),
|
||||
'composed_video_samples': sum(row.get('video_snapshot_valid', 0) != 0 for row in rows),
|
||||
'incident_captures': {
|
||||
'requests_recorded': 'incident_capture_request' in columns,
|
||||
'accepted_requests': sum(row.get('incident_capture_request', -1) == 1 for row in rows),
|
||||
'rejected_requests': sum(row.get('incident_capture_request', -1) == 0 for row in rows),
|
||||
'files_present': sorted(path.name for path in folder.glob('incident-*.png')
|
||||
if re.fullmatch(r'incident-\d+\.png', path.name) and path.is_file()),
|
||||
'scope': 'Queued requests are not completed captures; files still need visual inspection.',
|
||||
},
|
||||
'composition_scope': 'Latest composed image may lead actual swaps with threaded presentation; zero dimensions mean unavailable.',
|
||||
'warnings': warnings,
|
||||
}
|
||||
|
||||
|
||||
def native_executable(path):
|
||||
executable = path.resolve(strict=True)
|
||||
with executable.open('rb') as source:
|
||||
@@ -215,17 +318,23 @@ def prepare(args):
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('recording', type=Path, help='Session directory or inputs.csv')
|
||||
parser.add_argument('--exe', type=Path, required=True, help='Original native PSX playtest build')
|
||||
parser.add_argument('--analyze', action='store_true', help='Summarize telemetry and coverage without launching a game')
|
||||
parser.add_argument('--exe', type=Path, help='Original native PSX playtest build')
|
||||
parser.add_argument('--bios', type=Path, help='BIOS image (default: bundled OpenBIOS)')
|
||||
parser.add_argument('--disc', type=Path, required=True)
|
||||
parser.add_argument('--disc', type=Path)
|
||||
parser.add_argument('--game', type=Path)
|
||||
parser.add_argument('--settings', type=Path, help='Settings used when recording')
|
||||
parser.add_argument('--output', type=Path, required=True, help='New isolated directory')
|
||||
parser.add_argument('--output', type=Path, help='New isolated replay directory')
|
||||
parser.add_argument('--timeout', type=float, default=900)
|
||||
parser.add_argument('--debug-port', type=int, help='Developer build TCP inspection port')
|
||||
parser.add_argument('--input-only', action='store_true', help='Developer build only: apply inputs across fixes without claiming build/mod/RAM equivalence')
|
||||
parser.add_argument('--legacy-connected', choices=('00','10','01','11'), help='Explicit P1/P2 presence assumption for an input-only v1 route')
|
||||
args = parser.parse_args()
|
||||
if args.analyze:
|
||||
print(json.dumps(analyze_telemetry(args.recording), indent=2))
|
||||
return
|
||||
if not args.exe or not args.disc or not args.output:
|
||||
parser.error('replay requires --exe, --disc and --output; --analyze only needs a session')
|
||||
disc = args.disc.resolve(strict=True)
|
||||
bios = args.bios.resolve(strict=True) if args.bios else None
|
||||
output, target, trace = prepare(args)
|
||||
|
||||
@@ -0,0 +1,141 @@
|
||||
"""Coverage claims must distinguish missing probes, inactive checks and activity."""
|
||||
import csv
|
||||
import importlib.util
|
||||
import json
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
SCRIPT = Path(__file__).resolve().parents[1] / 'replay_diagnostics.py'
|
||||
spec = importlib.util.spec_from_file_location('replay', SCRIPT)
|
||||
replay = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(replay)
|
||||
|
||||
|
||||
class Coverage(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self.temp.cleanup)
|
||||
self.folder = Path(self.temp.name)
|
||||
|
||||
def write(self, rows, version='v2'):
|
||||
(self.folder / 'session.json').write_text(json.dumps({'telemetry_format': 'psxrecomp-telemetry-' + version}))
|
||||
with (self.folder / 'telemetry.csv').open('w', newline='') as output:
|
||||
writer = csv.DictWriter(output, fieldnames=rows[0].keys())
|
||||
writer.writeheader()
|
||||
writer.writerows(rows)
|
||||
|
||||
def samples(self, modern=False):
|
||||
rows = [dict(sequence=i, host_elapsed_us=i*16667, host_interval_us=16667,
|
||||
render_verify_checks=0, render_verify_mismatch=0) for i in range(3)]
|
||||
if modern:
|
||||
for i, row in enumerate(rows):
|
||||
row.update(gpu_draw_commands=200*i, host_presents=i, host_present_errors=0,
|
||||
host_present_status_available=1,
|
||||
video_snapshot_valid=1, video_renderer=2,
|
||||
video_resolution_preset=1 if i == 0 else 1080,
|
||||
video_aspect_num=4 if i == 0 else 16,
|
||||
video_aspect_den=3 if i == 0 else 9, video_widescreen=int(i > 0),
|
||||
render_verify_enabled=0, visual_verify_available=0, visual_verify_checks=0)
|
||||
row.update(composed_renderer=2, composed_internal_scale=1 if i == 0 else 5,
|
||||
composed_source_width=320 if i == 0 else 426, composed_source_height=240,
|
||||
composed_drawable_width=640 if i == 0 else 1920,
|
||||
composed_drawable_height=480 if i == 0 else 1080,
|
||||
composed_viewport_width=640 if i == 0 else 1920,
|
||||
composed_viewport_height=480 if i == 0 else 1080)
|
||||
return rows
|
||||
|
||||
def test_legacy_zero_checks_are_unknown_coverage_not_success(self):
|
||||
self.write(self.samples(), 'v1')
|
||||
result = replay.analyze_telemetry(self.folder)
|
||||
self.assertFalse(result['ordinary_draw_commands']['available'])
|
||||
self.assertFalse(result['render_state_verification']['covered_in_observed_interval'])
|
||||
self.assertIsNone(result['render_state_verification']['enabled'])
|
||||
self.assertFalse(result['visual_verification']['available'])
|
||||
self.assertTrue(any('not recorded' in text for text in result['warnings']))
|
||||
|
||||
def test_normal_rendering_can_advance_with_no_extra_render_passes(self):
|
||||
self.write(self.samples(True))
|
||||
result = replay.analyze_telemetry(self.folder)
|
||||
self.assertEqual(result['ordinary_draw_commands']['observed_increase'], 400)
|
||||
self.assertEqual(result['host_present_calls']['observed_increase'], 2)
|
||||
self.assertFalse(result['render_state_verification']['covered_in_observed_interval'])
|
||||
self.assertEqual(result['visual_verification']['result'], 'not evaluated')
|
||||
self.assertEqual(result['effective_video_mode_count'], 2)
|
||||
self.assertEqual(result['effective_video_modes'][1]['video_resolution_preset'], 1080)
|
||||
self.assertEqual(result['composed_video_modes'][1]['composed_internal_scale'], 5)
|
||||
|
||||
def test_enabled_but_never_exercised_and_preexisting_checks_are_uncovered(self):
|
||||
rows = self.samples(True)
|
||||
for row in rows:
|
||||
row['render_verify_enabled'] = 1
|
||||
row['render_verify_checks'] = 9
|
||||
self.write(rows)
|
||||
result = replay.analyze_telemetry(self.folder)['render_state_verification']
|
||||
self.assertTrue(result['enabled'])
|
||||
self.assertFalse(result['covered_in_observed_interval'])
|
||||
|
||||
def test_checks_do_not_certify_pixels_and_counter_resets_remain_visible(self):
|
||||
rows = self.samples(True)
|
||||
for row, checks in zip(rows, (0, 3, 1)):
|
||||
row['render_verify_checks'] = checks
|
||||
row['render_verify_enabled'] = 1
|
||||
rows[-1]['host_present_errors'] = 1
|
||||
self.write(rows)
|
||||
result = replay.analyze_telemetry(self.folder)
|
||||
self.assertTrue(result['render_state_verification']['covered_in_observed_interval'])
|
||||
self.assertEqual(result['render_state_verification']['checks']['resets'], 1)
|
||||
self.assertEqual(result['host_present_errors']['observed_increase'], 1)
|
||||
self.assertFalse(result['visual_verification']['available'])
|
||||
|
||||
def test_torn_final_record_is_ignored_and_internal_malformed_rows_fail(self):
|
||||
self.write(self.samples(True))
|
||||
path = self.folder / 'telemetry.csv'
|
||||
with path.open('a') as output:
|
||||
output.write('3,partial')
|
||||
self.assertTrue(replay.analyze_telemetry(self.folder)['ignored_torn_final_row'])
|
||||
with path.open('a') as output:
|
||||
output.write('\n')
|
||||
with self.assertRaises(ValueError):
|
||||
replay.analyze_telemetry(self.folder)
|
||||
|
||||
def test_analysis_cli_needs_no_build_disc_or_replay_output(self):
|
||||
self.write(self.samples(True))
|
||||
completed = subprocess.run([sys.executable, '-I', str(SCRIPT), str(self.folder), '--analyze'],
|
||||
check=True, capture_output=True, text=True)
|
||||
self.assertEqual(json.loads(completed.stdout)['samples'], 3)
|
||||
self.assertFalse((self.folder / 'replay').exists())
|
||||
|
||||
def test_incident_request_acceptance_does_not_invent_a_written_image(self):
|
||||
rows = self.samples(True)
|
||||
for row, state in zip(rows, (-1, 1, 0)):
|
||||
row['incident_capture_request'] = state
|
||||
self.write(rows)
|
||||
result = replay.analyze_telemetry(self.folder)['incident_captures']
|
||||
self.assertEqual(result['accepted_requests'], 1)
|
||||
self.assertEqual(result['rejected_requests'], 1)
|
||||
self.assertEqual(result['files_present'], [])
|
||||
(self.folder / 'incident-1.png').write_bytes(b'fixture')
|
||||
self.assertEqual(replay.analyze_telemetry(self.folder)['incident_captures']['files_present'], ['incident-1.png'])
|
||||
|
||||
def test_sdl2_void_present_is_activity_with_unknown_failure_coverage(self):
|
||||
rows = self.samples(True)
|
||||
for row in rows:
|
||||
row['host_present_status_available'] = 0
|
||||
self.write(rows)
|
||||
result = replay.analyze_telemetry(self.folder)
|
||||
self.assertEqual(result['host_present_calls']['observed_increase'], 2)
|
||||
self.assertFalse(result['host_present_status_available_all_observed_calls'])
|
||||
self.assertTrue(any('SDL2' in warning for warning in result['warnings']))
|
||||
|
||||
def test_startup_before_first_present_is_not_a_missing_result_for_a_call(self):
|
||||
rows = self.samples(True)
|
||||
rows[0]['host_present_status_available'] = 0
|
||||
self.write(rows)
|
||||
self.assertTrue(replay.analyze_telemetry(self.folder)['host_present_status_available_all_observed_calls'])
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
Reference in New Issue
Block a user