psxrecomp/docs/RENDER_THREAD.md

Render thread ([video] render_thread)

Opt-in, OpenGL only, off by default. With it off nothing in this document runs: gl_backend_get() returns the existing table and the GL renderer is the synchronous path every title already uses.

[video]
render_thread = true   # game.toml; PSX_RENDER_THREAD=0/1 overrides

PSX_RENDER_THREAD_FRAMES=N (default 2) bounds the closed frames in flight.

Player settings. render_thread, present_thread and frame_generation are also read from the player's settings.toml [video], which wins over the game.toml default; the PSX_* env vars override both for one run and are never saved. The launcher shows them as Settings → Display rows (Render thread, Present thread, Smooth motion (frame generation); recomp-ui RECOMP_LAUNCHER_HAS_RENDER_PIPELINE, OpenGL only, the latter two disabled while Render thread is off). The pipeline starts once at boot, so a change from the in-game launcher is saved and applies at next launch.

HD textures. Texture replacements and dumping run on the render thread. Residency notes (upload, invalidate, copy) are recorded in command order and applied against the render thread's VRAM copy. Under HD authority native VRAM is CPU-rasterized on that copy and published to gpu.c's array whenever the emulation thread takes the context and at stop. A GP0(A0) header takes the context and keeps it until the payload commits, so streamed words, mask checks and partial readbacks are exact. The open upload is tracked whatever the HD state, so HD turning on mid-payload, or a savestate restoring gp0 mid-A0 (after its full-VRAM load), keeps the hold too. Generated frames do not dump. Smooth motion runs with HD on.

Why

Before this change one thread ran the recompiled guest, turned every GP0 command into GL calls (gpu_gl_renderer.c), presented, and paid every Apple GL-on-Metal stall: command-buffer waits, uploads, readbacks, the swap. A frame cost emu + GL. With the render thread a frame costs max(emu, GL), and the guest waits for rendering only at the sync points below.

Model

emulation thread                         render thread (owns the GL context)
----------------                         -----------------------------------
guest + gpu.c (GP0/GP1, VRAM array,      replays records in order through the
status, timing)                          unchanged glb_* functions; presents
  gr_* -> GL_RT_BACKEND (record) ---->   ring (32 MiB, SPSC) ---->  exec()
  present  (record + overlay copy)
  vblank: frame boundary --------------> frame marker; <= N closed frames
  sync point: drain, take context <----  release context, park
  ... direct GL calls for the rest of the frame ...
  frame boundary: hand context back ---> make current, continue

Pieces, with file:line on feat/render-thread:

• Core — runtime/src/render_thread.c, runtime/include/render_thread.h. Single-producer/single-consumer byte ring with monotonic 64-bit positions. A record never splits across the ring end (a pad record fills the tail); payloads are copied, and space is freed only after exec() returns. No lock is held while a record executes; a mutex and two condition variables guard sleeping only (each side raises its *_sleeping flag under the lock before re-checking, the waker publishes its position before reading the flag). The consumer spins briefly before sleeping; the producer wakes a sleeping consumer every 128 KiB / 256 records and always at a frame end, drain or sync point. On macOS the thread is created with QOS_CLASS_USER_INTERACTIVE: with the default QoS a busy host scheduled it onto efficiency cores and it fell behind the emulation thread (R4 4K race on a host at load average ~40: 40 Hz with the default class, 60 Hz with this one).
• Frame bound — rt_frame_end() blocks while more than N closed frames are unconsumed (backpressure). rt_frames_ahead() tells the replaying thread how many complete frames are queued behind the current one.
• Sync point — rt_acquire(): drain, have the render thread release the context (glFlush, SDL_GL_MakeCurrent(win, NULL)), make it current on the emulation thread. The emulation thread then holds the context and calls the backend directly, exactly as with no render thread, until the next frame boundary hands it back (rt_release()). So a sync point costs at most one hand-off per frame, and code that is not converted to recording stays correct.
• Recording table — GL_RT_BACKEND (gpu_gl_renderer.c:8469), selected by gl_backend_get() while the thread runs (gr_refresh_backend() in gpu_render.c). Every state call, primitive, fill, copy, CPU->VRAM upload, single-pixel write and native-wide call is recorded by value; queries are answered from emulation-side mirrors or are sync points (table below).
• Hand-off rules — gl_renderer_render_thread_frame_boundary() (:8653), called from sdl_vblank_present() after the present and before wall-clock pacing (main.cpp:8650). It releases a held context when the frame may run asynchronously (gl_rth_eligible, :8173), takes it when it may not, then closes the frame.

Guest-visible VRAM stays on the emulation thread

The GL backend is GPU-authoritative: s_hr_fbo holds VRAM, gpu.c's array is refreshed only by readbacks (ensure_cpu, :2395), and pending uploads stage from that same array (flush_cpu_upload, :838). gpu.c writes the array directly while a GP0(A0) payload streams in (gpu.c:6912) and hands the facade a copy at commit (gpu.c:3156). If the render thread staged from the live array it would upload pixels of a later transfer (and its sw_vram_transfer_in would write stale payloads back over the guest's array).

So the backend's s_vram is gpu.c's array whenever the emulation thread runs GL, and a private copy while the render thread does (gl_rth_acquire/gl_rth_release, :8181/:8188; the SW rasterizer's pointer moves with it, sw_renderer_rebind_vram). The private copy gets each upload payload in command order; a release first copies gpu.c's array into it, so it matches the guest at every hand-off. Every readback runs on the emulation thread under a sync point and writes gpu.c's array, as before.

Guest state the backend reads while drawing

State	Read by	Under the render thread
per-prim widescreen tags (psx_ws_prim_in_backdrop, psx_ws_prim_is_tagged)	bd_prim_gate :2533	captured per primitive in the record flags (rth_prim_flags) when native-wide and the backdrop stretch are on; replay reads them via ctx_prim_* (:379)
native-wide fast-path latch (gpu_ws_background_requires_full_composite, gpu_ws_netplay_local_viewport_width), flat backdrop	wide_fast_center_valid :2569 (also at batch flush and present)	sent as an RTH_STATE record before the first call that would see a change
depth24 display (gpu_display_is_depth24)	depth24_upload_policy :4096 and the depth24 helpers	a depth24 display is a sync point (rth_record_mode, :8202); the render thread only replays 15-bit frames, so ctx_depth24() is 0 there
host overlays (OSD, volume, rewind strip, savestate menu)	gl_swap_with_osd :7543	rasterized and copied at record time (rth_record_present, :8216); host_osd_present_done runs at record time
present-shot request	gl_swap_with_osd	already mutex-guarded (main.cpp present_shot_take)

Emulation-side answers without a sync point: gl_renderer_projective_supported (:3854) and gl_renderer_present_wide_fbo's return value (:7981) from a mirror of the wide-surface bookkeeping (rth_mirror_wide_for, :8121, the same lookup, limit check and slot allocation wide_fbo_for does), and gr_get_draw_area from the recorded draw area. gr_scale, gr_texture_filter and gl_renderer_fit_wide_aspect read values that only change while the emulation thread holds the context. The mirrors are re-read from the backend at every release.

Sync points found in the code

Guest-visible:

Where	Why
GPUREAD gpu_read_gpuread (gpu.c:3743) -> gr_vram_read	VRAM->CPU transfer reads the FBO (glb_vram_read -> ensure_cpu)
GP0(A0) with mask check (gpu.c:6912)	per-pixel mask test reads VRAM
gpu_vram_peek (gpu.c:6889)	debug/mod VRAM reads
savestate / rewind save (boot_state.c:148,159,189,356)	gr_vram_transfer_out of all VRAM
savestate load (gl_renderer_restage_vram_after_savestate, :5112)	full restage from gpu.c's array
depth24 (FMV) display	the CPU mirror is presented; the emulation thread draws those frames

Not guest-visible but GL-owned (each is GL_RT_SYNC at the top of the entry point): gr_render_display*, gr_render_wide_display, gr_wide_dump_full (screenshots, CPU present fallback), every gl_renderer_pass_*/stereo_* (render passes), psx_mod_openxr_*, texture-bank selection, settings (scanlines, aspect, gamma, bezel, swap interval, interpolation), the CPU present / blank / hold-last presents (pause, rewind, netplay hold), and the debug-server diagnostics (gl_* rings, frame_perf, gl_fbo_peek, gl_vram_diff). Two per-frame debug captures are queued instead of being sync points: the display ring's readback (gl_renderer_fbo_peek_deferred) and the --headless-opengl presented-image capture (gl_renderer_ring_capture); their ring readers sync first (gl_renderer_render_thread_sync). ensure_cpu refuses to run on the render thread: a readback there would land in the private upload source and mark the CPU side current without gpu.c's array seeing the pixels.

What is not a sync point: a frame of primitives, fills, copies, uploads, native-wide draws and the 15-bit present (gl_renderer_present_vram / present_wide_fbo). A Ridge Racer Type 4 race loaded from a savestate takes one sync point (the load) and then none.

Frames, backpressure, stale presents

At most N closed frames wait in the ring; past that the emulation thread blocks in rt_frame_end. The render thread must replay every frame's drawing: later frames sample what earlier frames rendered, so drawing cannot be skipped. Only the present can. When two newer frames are already recorded (the emulation thread is at the bound, waiting), the present of the frame being replayed is skipped (rth_replay_present, :8250), never twice in a row, so a saturated render thread still shows at least every other frame.

Consequence: the render thread takes the GL cost off the guest's frame, but if GL alone exceeds the frame budget the guest is still held back by the bound. Choosing an internal resolution that fits is the job of dynamic resolution (below), not of this layer.

Interoperation

Feature	Behaviour
savestates / rewind	load and save are sync points; the frame stays synchronous, the next frame boundary releases
netplay / rollback resimulation	psx_netplay_active() or dual-raster makes every frame ineligible: the emulation thread holds the context (synchronous path). Not started when netplay is configured at boot
render passes	first gl_renderer_pass_* call in a frame is a sync point; the rest of that frame is synchronous. Recommended: leave render_thread off with the frame-rate mod
frame interpolation	ineligible (held) while enabled; not started when it is on at boot
HD texture replacements / texture dumping	eligible; held only from a GP0(A0) header to its commit
OpenXR	ineligible while a session is active
Smooth motion (frame generation)	[video] frame_generation: in-between frames drawn by the render thread from recorded lists, docs/FRAME_GENERATION.md
native-wide / widescreen	recorded; tags, latch and wide-surface mirror as above
internal resolution changes	gr_set_scale and every resolution entry point are sync points; dynamic-resolution level steps are recorded (below)
screenshots / debug captures	screenshot* sync; present_shot is fulfilled on the render thread
headless	--headless (software) never starts it; --headless-opengl runs it on the hidden context (no present; the frame boundary still closes every frame), which is what fp_identity exercises
Vulkan / software	never started (log line says why)

Present thread ([video] present_thread)

Opt-in, under the render thread, off by default (present_thread = true; PSX_PRESENT_THREAD=0/1, PSX_PRESENT_THREAD_SLOTS=2..4, default 3). On macOS a swap on the render thread cost 6-9 ms of waiting on the window compositor ([NSOpenGLContext flushBuffer] on GL-on-Metal) in every frame, real or generated; with this it costs a fence and a hand-off.

• Slots. While it runs, framebuffer 0 of the composing context (the render thread's, or the emulation thread's while it holds the context at a sync point) is an offscreen slot: every glBindFramebuffer(..., 0) goes through gl_bind_fb_redirect and lands on the current slot, an RGBA8 (RGB8 if the window has no alpha) texture plus depth-stencil at the window's drawable size, reallocated when the window changes. Everything that used to be drawn into the back buffer (display quad, OSD, hold-last captures, present_shot, frame generation's kept real frame) is drawn into it unchanged; the two glReadBuffer(GL_BACK) reads read its attachment.
• Swap. gl_present_swap: a fence (glFenceSync), glFlush, queue the slot to the present thread, take the next free slot (waiting only while every other slot is queued or on screen: the display is the bottleneck; that wait is what the dynamic-resolution and frame-generation ledgers count as the swap), and glWaitSync on the fence the present thread left after its last copy out of that slot.
• Present thread (present_thread.c, backend-neutral core; GL callbacks in gpu_gl_renderer.c): owns a second context created on the same window with SDL_GL_SHARE_WITH_CURRENT_CONTEXT (textures and syncs are shared; FBOs are per context, so it keeps its own read FBO per slot). Per slot, in submission order: glWaitSync on the ready fence, a 1:1 GL_NEAREST blit to the real framebuffer 0 (scaled only if the window changed since the frame was composed), a done fence, the swap. The swap interval is applied on its context. QOS_CLASS_USER_INTERACTIVE on macOS, above-normal priority on Windows.
• Fallback. No sync objects, no shared context or a failed start: a log line and direct swaps, as without it. It starts and stops with the render thread (stop presents what is queued first).
• Pixels. The window shows exactly the composed image (gl_frame_gen_test reads the window back after each copy). Compared with composing straight into the window, the composed image is bit-identical at 1x and for every native-wide present; on Apple GL-on-Metal a VRAM present resolved down into a smaller window (area resolve) rounded 1 pixel by 1 LSB in 2 of 24 frames: rasterizing into a texture instead of the drawable. The test allows at most 1 LSB on 0.05 % of the pixels and reports it.
• Latency. Up to slots - 1 composed frames can wait for the display. Frame generation already paces presents to the refresh, so the queue normally holds at most one.
• R4 (Match display 10x ceiling, dynres, Fit widescreen, 120 Hz panel, host at load average 22-26): the render thread's swap went from 4.5-7.8 ms to 0.02-0.08 ms; the present thread's swap takes 8-10 ms. 2P VS guest 56.6-57.6 → 57.2-58.6 Hz; with generation forced (test knob) 1P guest 25 → 41-44 Hz and presented 50 → 78-85 Hz.

{"cmd":"render_thread"} carries present_thread: active, slots, submits, presents, waits and wait time (composing side blocked), swap time (avg/max, on the present thread), queue high water.

Dynamic resolution ([video] dynamic_resolution)

Opt-in as before (dynamic_resolution = true, dynamic_resolution_min; PSX_DYNRES=0/1, PSX_DYNRES_MIN for one run); no new keys. The level steps between the configured internal resolution (the ceiling, e.g. Match display, which the GPU limit and memory budget are checked against) and the minimum. Every surface (hr VRAM and native-wide) is allocated at the current level and reallocated at each step: the hr surface is reseeded into the new allocation from the 1x image plus the displayed rect and draw area at full detail, the wide surfaces' presented rows and margins are rescaled into theirs. On Apple's GL-on-Metal every render pass into an attachment loads and stores the whole attachment, so surfaces held at the ceiling cost the ceiling's bandwidth at every level; with a 10x ceiling, R4's 2P VS start (which needs about 4x) ran at 35-40 Hz for its first seconds at any level. If a new allocation fails, a step down rescales in place inside the old allocation and a step up past it is refused. With the render thread off nothing changes: the emulation thread's wall-time controller (dynres_* in dynamic_resolution.c) decides. With it on, the guest's frame no longer pays for GL, so its wall time says nothing about whether the resolution fits; a second controller (dynrt_*, same file) takes over when the render thread starts and is fed what the render thread measured. The one target is every guest frame at the guest's rate (60 on NTSC); resolution is the only lever.

Cost. Per replayed guest frame, between its first record and an RTH_FRAME marker the frame boundary records: the render thread's CPU time (wall time minus its idle waits for records and its time in the swap, the display's vsync) and the GPU time of the same span (a GL_TIME_ELAPSED query, read back when available, never waited on; GL_TIMESTAMP counters read 0 on macOS). cost = max(CPU, GPU), published as running totals (gl_renderer_render_thread_costs). A frame during which the context went to the emulation thread is dropped (its span holds the emulation thread's drawing). While this measures, frame_perf's own TIME_ELAPSED brackets stand aside (queries of one target cannot nest). PSX_DYNRES_GPU_TIMER=0 leaves CPU time only.

Decisions (0.25 s windows of guest time; dynamic_resolution.h has the full rules and dynrt_default_params the numbers):

• load = mean cost / the guest's nominal interval (one display refresh at 60 Hz). Budget 0.85 (15 % margin).
• Down, one level, after two consecutive windows over budget, or one window in which the emulation thread spent 5 % or more of the time blocked on the queue bound (backpressure: the render thread is behind) or the cost was over the whole interval. Never below the minimum.
• Guest-bound: when the guest ran below its rate, never waited on the queue and the render thread had slack in the interval it was given, the guest is the limit and no down step is taken.
• Judged: the window after a step settles (the cost lags the queue and the query readback), the next one judges it. A down step that removed less than 30 % of the predicted cost (and did not end the backpressure) is a strike; a second in a row means the cost is not the pixels: both steps are undone and down steps are blocked for 20 s, doubling to 320 s.
• Fast first descent: armed when the controller starts (session start, render-thread start, a new resolution) and on a savestate load, game entry or window resize, for 6 s from the first judged window. While armed, a window at or over the whole interval, or two over budget, jumps straight to the highest level whose pure-area prediction (load × (S′/S)²) is at or below 0.78; the jump is judged like any step and, if still over, jumps again from the new measurement. A fixed part of the cost only makes the target dearer than predicted, so a jump never lands below the level that fits, and the level above it is predicted over the 0.70 up threshold, so no up step follows. A fit after a step, an up step or a failed judgement disarm it; single steps fine-tune. R4 2P VS from a 10x ceiling (busy host): 10→5 or 10→6, then 4-3; guest at 59 Hz or more from 1.2-2.3 s after the load (one level per judged window: 3.3 s).
• Up, one level, after 3 s of windows with no backpressure and the next level predicted at 0.70 or less, 2 s after the last step, never above the ceiling. The prediction (load × ((1−f) + f·(S′/S)²)) learns f from each judged step. A level reached by an up step and left within 10 s is blocked for up steps, 10 s doubling to 160 s.
• Holds as before (FMV, fast-forward, overlay compiles, resim, resizes, boot); the savestate-load and game-entry tails, which only covered the wall-time model's settling, are cut to 0.5 s.

Steps are recorded (RTH_DYN_STEP) after the frame just closed, with the displayed rect captured at record time, and applied by the render thread in stream order (dyn_apply, the same reseed as without the thread). The emulation side's level (gr_scale, the wide-surface mirror, gl_renderer_dynres_stats) moves at record time. A frame the emulation thread holds steps directly, as without the thread.

Telemetry. {"cmd":"dynres"} reports "mode":"render_thread", level, load, budget, bp_share, guest_hz, cpu_ms/gpu_ms (last window's means), guest_bound, the step counters and up_blocked. PSX_DYNRES_TRACE=<csv> writes one line per window (t_s,level,load,bp_share,guest_hz,cpu_ms,gpu_ms,guest_bound,decision).

Out of scope

• Vulkan (the same split would apply; its backend has its own readbacks).
• Converting the remaining sync points to recorded commands (texture banks, render passes, CPU/hold presents). Each is correct as a sync point.
• Making replay cheaper (batching across records, fewer GL state changes).
• Diagnostic rings written by both threads (latency_ring, gpu_timeline) can interleave entries; they are diagnostics only.

Tests

• render_thread_test (runtime/tests/test_render_thread.c): the core against a fake executor and context token — ordering and payload integrity over 400 frames of random-size records through ring wraps, oversize refusal, the in-flight bound under a slow consumer (and that the one- and two-frames-behind cases both occur), 300 randomized acquire/release rounds with the context never current on two threads, stop while held. Clean under ThreadSanitizer.
• gl_render_thread_test (runtime/tests/run_gl_render_thread.py, test_gl_render_thread.c, label gpu;opengl;hardware): a real hidden GL context, a scripted stream through the facade, render thread off vs on at 1x and 4x; the values the guest read back, native VRAM, the frame at S and the native-wide surface must be identical. The stream has mask set/check, a mask-checked upload (per-pixel readback sync point), back-to-back uploads of one rect with a catch-up between gpu.c's writes and its commit, more than the ring's size recorded, readback frames, presents with a slow render-thread present, and widescreen tags that change per primitive and are poisoned after each call. It fails for each of these deliberate mistakes: replay reading the tags live, staging uploads from the guest array, dropping the stream state.
• video_enhancement_settings_test: the keys (render_thread, present_thread, frame_generation) default off and parse, and the settings.toml copies parse and round-trip through save_user_settings.
• present_thread_test (test_present_thread.c): the present core against a fake presenter: presentation order is submission order over 600 frames with 2-4 slots, a slot is never composed while queued or on screen, each done token reaches the composer that next takes its slot, a slow display blocks the composer, stop presents the queue and disposes leftover tokens, a context failure fails the start. Clean under ThreadSanitizer.
• gl_frame_gen_test present-thread runs: generation off and on, VRAM and native-wide, flip and late timing: the window shows every composed image, in order, bit for bit; real frames match the direct-swap run (see Pixels).
• Windows (MinGW): the runtime builds with the Win32 path of render_thread.c (SRW lock, condition variable, CreateThread), and render_thread_test and gl_render_thread_test run there (the GL test exits 77, a CTest skip, on a host with no GL 3.3 context).
• dynamic_resolution_rt_test (test_dynamic_resolution_rt.c): the render-thread controller against a synthetic two-stage pipeline (guest E, render a + b·S², two frames in flight, costs four frames late): overrun -> one level at a time to the highest level that fits, then no oscillation; headroom -> up only after the window; the hysteresis band; guest-bound -> no step; a cost that does not scale -> two strikes, undo, back-off; queue full with an under-reading meter -> down; floor and ceiling; a heavy stretch whose first window is mixed -> one strike, not an undo; holds; thin windows; pins; fast descent (10x→5x in one jump in 0.68 s vs 3.88 s one level at a time, no overshoot, no step in the next 120 s; re-armed by dynrt_arm_descent, single steps when not armed; light windows first do not use it up; a flat cost still ends in the undo; the target function).
• gl_render_thread_test dynres mode: the same level steps between frames with the thread off (applied at once) and on (recorded, and directly in held frames) give identical readbacks, VRAM, frame and wide surface.
• In game (R4, see the PR): tools/fp_identity.py guest identity render thread off vs on under --headless-opengl, and A/B frame rates from a race savestate at Native, 4K and Match display, 4:3 and widescreen.

Debug server

{"cmd":"render_thread"} — active, held, max_frames, records, bytes, frames produced/consumed, presents and presents_stale (skipped), acquires (sync points) with the 16 most recent reasons and wait times, backpressure / ring-full waits and time, render busy/idle time, ring high water. Reading it is not a sync point.