psxrecomp/docs/RENDER_PASSES.md

Render passes: true in-between frames

Frame blending (FRAME_RATE.md) crossfades finished frames; it cannot show objects in positions the game never drew. A render pass lets a trusted game plugin redraw its scene between two logic ticks with the game's own draw code, with objects and camera placed part of the way from one tick to the next. The pass never changes the game: guest time is frozen while it runs and the machine is restored bit-for-bit afterwards. Its only product is an image the presenter shows between the game's own frames.

Default off. Nothing happens unless a plugin calls the API below.

API (mod_plugins.h)

uint32_t psx_mod_render_pass_plan(uint32_t period_vblanks,
                                  uint32_t shown_after_vblanks,
                                  uint32_t *alpha_q16, uint32_t max);
int psx_mod_render_pass(struct CPUState *cpu, const PSXModRenderPass *pass,
                        PSXModRenderPassFn fn, void *user);
uint32_t psx_mod_render_pass_status(void);

Call both from an emulation-thread function-entry hook at the point where the game has finished the logic of tick n+1 but the display still has to flip to frame n. For a PsyQ double-buffered loop that is the entry of the VSync(0) that precedes PutDispEnv.

Register that hook with psx_mod_register_function_entry_plugin() under the plugin's manifest [[plugin]] id (see MOD_PACKAGES.md): it then runs only while the resolved mod plan activates the plugin, so never while the package is disabled or in netplay. Register once, for example from the plugin's constructor; a repeated id and address returns 0. Entry hooks also fire for the guest functions a pass itself calls, the plugin's own included, so a hook that plans passes must ignore entries made inside one.

1. psx_mod_render_pass_plan(P, D, a, max) returns the phases (Q16, in (0, 1)) at which the presenter will show frame n: its output deadlines during the P VBlanks the frame stays on screen, starting after D more VBlank presents. When the host cannot afford them all, an evenly spread subset is returned. 0 means "no passes this frame" (see Gates).
2. For each phase, psx_mod_render_pass() with the display rect the next flip shows (the DISPENV rect). fn(cpu, user, alpha) may write guest RAM and call guest functions with psx_dispatch_call(); it returns nonzero to keep the image. The first pass after a plan also captures frame n's own image (phase 0).

The presenter uses PSX_MOD_FRAME_SOURCE_FLIP to see the flip, and a plugin that supplies passes normally selects PSX_MOD_FRAME_INTERPOLATION_HOLD, so wherever no pass image applies the newest game frame is repeated rather than crossfaded one frame late.

When passes are unavailable

psx_mod_render_pass_status() says why a plan would be empty, the host-time budget aside (an empty plan while it says READY was shed for time):

Status	Meaning	Lasts
READY (0)	passes can run
NO_PRESENTER (1)	not OpenGL, interpolation off or suspended (FMV), or not the FLIP source	until the presenter changes
BACKEND (2)	the renderer declines passes in its current mode	while that mode lasts
DISABLED (3)	switched off after repeated faults	the session
SESSION (4)	netplay, rollback, rewind, load/save replay, self-check resimulation	transient
FAST_FORWARD (5)	manual fast-forward, turbo-through-loads, FMV auto-skip, TCP turbo	transient
BUSY (6)	inside an exception, a pass or a GPU DMA walk, or no frame captured since the presenter's history restarted (a display mode change)	transient

A plugin that relies on passes should not leave the player on stock-rate frames while a lasting reason holds: it can switch the presenter to a crossfade with psx_mod_set_frame_interpolation_blend(), which may be called from a hook at any time and takes effect at the next present, and switch back to HOLD when the status is READY again. A backend mode that cannot host passes (for example a renderer path whose VRAM is not one texture) should report BACKEND from gl_renderer_pass_unavailable().

What a pass may and may not do

While fn runs (g_psx_render_pass_active):

Area	Behaviour	Where
Guest clock	cycles are counted (GTE and mult/div deadlines work) but no device is serviced, no VBlank or device event fires	psx_cycles.c freeze (psx_cycle_freeze.h, runtime-only: the codegen-hashed psx_cycles.h is untouched)
Interrupts	never delivered	interrupts.c
GPU DMA	linked lists and delayed completions finish synchronously	dma.c
RAM / scratchpad stores	written directly, bypassing code-page tracking, overlay watch, write traces and fingerprints; RAM addresses fold through the live geometry (2 MiB mirrored, or 8 MiB with the 8 MB RAM mod), as outside a pass	memory.c render_pass_store
MMIO stores	allowed: GP0, GP1 DMA mode / info, GPU and OTC DMA channels, DPCR/DICR, I_STAT/I_MASK. Dropped and counted: SPU (key-ons), CD, timers, SIO, MDEC, other DMA channels, memory control	memory.c
VRAM	only the declared rect; writes that bypass the scissor elsewhere (fills, copies, uploads, pokes: never a native-wide surface) are journaled and rolled back	gpu_gl_renderer.c
Runaway code	an 8 M guest-cycle watchdog rolls the pass back	render_pass.c

After fn (success or not) everything is restored: CPU state with the GTE, all of main RAM at its live size (2 MiB, or 8 MiB with the opt-in 8 MB RAM mod; psx_memory.h), scratchpad, I-cache tags, I_STAT/I_MASK, timers, DMA and GPU registers (without the widescreen side effects of a savestate load), the VRAM rect (hr colour, mask stencil, raw 16-bit mirror, native-wide band, CPU VRAM rows), the renderer's coherency bookkeeping, and every clock value. A watchdog abort leaves by longjmp from inside guest code, skipping the exits of the frames it leaves, so the host nesting those frames own is put back from the checkpoint too: the cycle-deferral depth, the native overlay unit depth, shard stack and cycle-flush hook, the DMA execution depth, and the interpreter's active/phase/precise flags, resume latch and pending load (render_pass_abort_test). A plugin must likewise not keep state that only its callback's normal return resets. After 8 faults (watchdog or refused writes) passes stay off for the session.

Gates

The plan returns 0 in netplay, rollback resimulation, self-check, rewind, lockstep, an exception, while a GPU DMA list is in flight, during manual fast-forward, turbo-through-loads, FMV auto-skip and TCP turbo, on anything but the OpenGL renderer with FLIP-source interpolation, while the presenter is suspended (FMV), after repeated faults, and when no VBlank was presented since the last plan (headless). psx_mod_render_pass() refuses under the same conditions.

Presentation and budget

The presenter keeps two generations of images: the frame on screen and the one being built for the next flip. A generation becomes current when the FLIP source sees the display flip to its rect; from that interval's start its phases map onto host time. At each output deadline the newest image at or before the deadline's phase is shown, crossfaded into the next one when passes were shed. Deadlines that fall due while passes run are presented between passes, so the frame on screen keeps moving. A frame that stays on screen longer than the plan's period (a lagging tick: three VBlanks instead of two) holds its newest image until the next flip, as a late stock frame holds; it never steps back to the game's own image of the frame, which is older (late_presents in render_pass_stats). Only a game that stops flipping for four frame lengths expires the frame's images (expired), and the presenter then shows its own newest capture.

Consecutive passes of one frame reuse the VRAM backup the first one took: each pass's restore leaves the rect exactly as backed up, and no guest code runs in between (the clock and guest store count are unchanged), so only the first pass copies the rect out (backups_reused in render_pass_stats).

Passes cost host time inside the game's frame: they run on the emulation thread. The budget per frame is PSX_RENDER_PASS_BUDGET percent (default 80) of the presenter's idle time plus the presents beyond two per frame, learnt from the previous frame, over a smoothed per-pass cost. When the budget runs out, fewer passes are rendered, and a pass that costs more than the budget is not planned at all. The cost is measured per presented image size: until three passes have been measured at the current size (the first plans, and after an aspect or internal-resolution change) a plan asks for one pass, so an expensive size costs at most one pass per frame while it is learnt. The three seed the average with their median, so one slow pass on a busy host does not price passes out. A pass that creates pass textures or framebuffers (the first frames at a size, or a slot filled for the first time) is not a cost sample, at most eight in a row. Only passes that run are measured, and the first passes of a race can all run in a transient (a busy host, code run for the first time) that costs several times the steady state; an estimate that high prices every plan out, so no pass would ever correct it. An estimate that no pass has been measured against for 30 plans that wanted passes (about a second of a 30 Hz game) is measured again: the warm-up restarts, with one pass per plan (cost_rewarms in render_pass_stats). When the new median is not at least a quarter below the old estimate, the old one was right (a size that is truly too expensive, or a host at its limit), and the next wait doubles, up to 960 plans, so these one-pass warm-ups stay rare; a re-measure that finds the old estimate stale keeps the wait at 30. A plugin can show a crossfade while passes are shed (see "When passes are unavailable").

Pass images are kept at internal resolution (the size the presenter captures). Their textures are made as the slots fill, never more slots than two generations fit in 256 MiB, which limits passes per frame at very high internal resolutions; a size change frees the old set.

Verifying a title

• render_pass_stats (TCP): passes, shedding, faults, dropped device stores, timing split (backup / guest code / capture / restore), presents made from pass images, and status.
• render_pass_refuse on=1 (TCP) or PSX_RENDER_PASS_REFUSE=1: the backend declines passes (BACKEND), to test a plugin's fallback.
• render_pass_dump path=<dir> count=<n>: PNGs of the next n frames' images (the game's own frame, then each pass in phase order).
• PSX_RENDER_PASS_VERIFY=1: hash CPU, RAM, scratchpad, I-cache, interrupt, timer, DMA and GPU state and read back the VRAM rect before and after every pass, and check that a pass that returned normally left the host nesting balanced; verify_mismatch must stay 0.
• PSX_RENDER_PASS_WATCHDOG=<guest cycles>: lower the watchdog (default 8 M) below a title's pass size to drive real passes through the rollback path; with fingerprints (below) the run must still match one without passes.
• frame_fingerprint reset_on_load=1 then a savestate load: the per-frame write/MMIO/cycle fingerprints of a run with passes must equal a run without them.

Tests (runtime ctest unless noted):

• render_pass_plan_test: phase planning per rate, shedding (one pass while the cost is unknown), the cost warm-up and allocation rule, re-measuring a stale estimate (and backing off when it stays too expensive), selection, holding the newest image when the next flip is late, the MMIO allow-list.
• render_pass_freeze_test: 10^6 frozen cycles, exact clock restore, watchdog.
• render_pass_sandbox_test: the store policy memory.c routes every pass store through (SPU, CD, timer, other-DMA, GP1 and other device stores dropped and counted per class, never delivered; GP0, I_STAT/I_MASK and GPU/OTC DMA delivered); a real pass over render_pass.c, psx_cycles.c and timers.c that makes those stores and runs 10^6 cycles while a timer is armed to interrupt every 1000 (no device advances, no interrupt, RAM, scratchpad, I_STAT/I_MASK, timers and clock restored); status gates; the same store folding and restore with 8 MiB RAM live (unique addresses up to 0x7FFFFF, restored after a pass, and a retail pass after it folding again); the out-of-rect VRAM journal policy and its exact rollback of the CPU VRAM rows.
• render_pass_abort_test: a watchdog abort from nested frames.
• render_pass_guards (source guard, recompiler ctest): the choke points above, and that the codegen-hashed headers do not carry the pass API.

The GPU half of the VRAM transaction (hr colour, stencil, raw mirror blits) needs an OpenGL context and is checked at runtime by PSX_RENDER_PASS_VERIFY=1.