mirror/psxrecomp
1
0
Fork 0
mirror of https://github.com/RetroPortingToolKit/psxrecomp synced 2026-10-26 02:11:36 -04:00
Code Issues Packages Projects Releases Wiki Activity

duckstation linux build + frame capture diagnostics

Browse Source
This commit is contained in:
Alex Vanderveen
2026-08-20 11:25:53 -04:00
parent 3a94539495
commit 0e5d463356
11 changed files with 1861 additions and 52 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/GPU_FRAME_TOOLS.md
+68 -7
View File
@@ -118,6 +118,51 @@ selected project's `game.toml` says.
---
## You cannot pause the game, and you do not need to
`pause`, `continue`, `step` and `run_to_frame` were **removed** from
psx-runtime. `runtime/src/debug_server.c` still registers them, but only as
handlers that return an error explaining the migration — pause-step-read
synthesizes a snapshot ("what is state right NOW") instead of reading the
history the runtime already records continuously, and in this codebase it forced
the runtime into a wait loop where a dropped client looked like a freeze.
The replacement is better for chasing a render bug. The GP0 ring holds
`1 << 20` packets (`gpu.c`, `GP0_RING_CAP`) — several hundred frames — so the
workflow runs the other way round:
> Play until the bug is on screen. **Then** capture that frame, and the frames
> leading up to it.
Reaching backwards beats freezing forwards, and it is the only thing that works
at all for a glitch you cannot reliably stop on.
```bash
python3 $P/gpu_frame_capture.py --ring # what can I still ask for?
# GP0 ring: 812344 packet(s) seen, capacity 1048576, 12 words/packet
# capturable frames: 40918..41230
```
`gpu_frame_dump` on a frame that has been evicted returns an **empty** dump, not
an error — which reads exactly like "that frame drew nothing", a conclusion you
might act on. So `capture()` checks the span first and refuses, and Studio's
Frames tab shows the span next to the frame selector and marks an out-of-ring
frame in red.
What replaces each removed command:
| Gone | Use instead |
|---|---|
| `run_to_frame N` | `--frame N` against the ring (any frame it still holds) |
| `step` | `--frames N`, walking backwards from `--frame` |
| `pause` + read state | `frame_range` / `get_frame` (per-frame records), `fn_entry_dump`, `wtrace_dump` |
| `pause` + inspect drawing | `gpu_frame_dump frame=N` — what this whole toolchain does |
Savestates are unaffected: `savestate` is a real handler, and a savestate plus a
fixed frame number still makes a run reproducible.
---
## Transport
**One request per connection.** `io_thread_main()` in `debug_server.c` accepts,
@@ -134,10 +179,10 @@ from a hung emulator. `psx_gpu_frame.DebugConn` opens a socket per command;
cd <your recomp project>
P=psxrecomp/tools
# 1. Put the game on the frame you want, then capture it twice: once while it
# still looks right, once after it breaks.
python3 $P/gpu_frame_capture.py --tag good --out analysis/frames --summary
python3 $P/gpu_frame_capture.py --tag bad --out analysis/frames --summary
# 1. Play until the bug is on screen. Capture the newest frame as the suspect,
# then reach BACK into the ring for one that still looked right.
python3 $P/gpu_frame_capture.py --tag bad --out analysis/frames --summary
python3 $P/gpu_frame_capture.py --frame 41100 --tag good --out analysis/frames --summary
# 2. What changed?
python3 $P/gpu_frame_diff.py analysis/frames/good.json analysis/frames/bad.json
@@ -149,8 +194,14 @@ python3 $P/gpu_frame_layers.py analysis/frames/bad.json --out analysis/frames/ba
python3 $P/gpu_parity.py --frame 41230 --out analysis/frames/parity
```
`--run-to N` drives the runtime to a frame before capturing, so a savestate plus
a fixed frame number makes the whole thing reproducible.
`--frames N` also grabs the N-1 frames *before* `--frame`, walking backwards
through the ring — the frames leading up to the one you noticed are usually the
ones that explain it. A savestate plus a fixed frame number still makes the whole
thing reproducible.
A screenshot is only written for the live frame: the framebuffer holds the
present, not the historical frame you just dumped, and saving it beside an older
dump would label it as something it is not.
### Artifacts
@@ -216,7 +267,10 @@ Studio's Frames tab has a **Save name** box that calls exactly this, so
## Parity scope
`gpu_parity.py` compares **images**, not GP0 streams. The DuckStation oracle
`gpu_parity.py` compares **images**, not GP0 streams. Only DuckStation is
driven: it implements `run_to_frame`, psx-runtime does not, so the native side is
sampled where it already is and DuckStation is advanced to meet it. A residual
frame mismatch is reported rather than papered over. The DuckStation oracle
patch (`tools/duckstation/psxrecomp_oracle.patch`) implements `screenshot`,
`read_vram`, `gpu_state`, `run_to_frame`, `step` and `pause` — but not
`gpu_frame_dump`, so there is no packet stream to compare on that side. Adding
@@ -224,6 +278,13 @@ one to the patch is the obvious next step if image parity keeps saying "the
streams must differ" without saying where; until then, `tools/cosim.py` brackets
the divergence.
Getting an oracle on Linux: `python3 tools/duckstation_oracle.py all` builds and
installs a patched DuckStation into `~/.local/share/retcomm/oracle/duckstation/`
— outside any game repo, shared by every title — and `start --disc <cue>` runs it
headless on 4371. On Arch-family and NixOS hosts the build runs inside
`ubuntu:22.04` via podman, because upstream's CMake refuses those hosts outright
and its build scripts may not be patched. See `tools/duckstation/README.md`.
Both instances must be driven identically — same disc, same starting state.
Parity between two runs that were not driven the same way means nothing.
tools/duckstation/Containerfile
+39
View File
@@ -0,0 +1,39 @@
# Builder image for the DuckStation oracle.
#
# Why a container at all: DuckStation's CMake refuses to configure on
# Arch-family and NixOS hosts (CMakeModules/DuckStationBuildSummary.cmake), and
# upstream explicitly forbids distributing patches that modify its build system.
# So instead of working around the check, we build somewhere it genuinely
# supports -- ubuntu:22.04, the exact image its own Linux CI uses
# (.github/workflows/linux-appimage-build.yml).
#
# Ubuntu 22.04's glibc is older than any modern host's, so the binary produced
# here runs natively on the host afterwards. That is the same property upstream
# relies on to ship one AppImage for every distro.
#
# The package list is upstream's own scripts/appimage/install-packages.sh,
# copied verbatim out of the pinned checkout by duckstation_oracle.py. It is not
# reproduced or modified here -- it is the authority on what the build needs.
FROM ubuntu:22.04
ENV DEBIAN_FRONTEND=noninteractive
# linuxdeploy and appimagetool ship as AppImages and want FUSE, which a
# container generally cannot give them. This is the AppImage runtime's own
# supported escape hatch, not a modification of upstream's packaging script.
ENV APPIMAGE_EXTRACT_AND_RUN=1
# What upstream's install-packages.sh assumes is already present: it calls sudo
# and apt-add-repository, neither of which a bare ubuntu image has.
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
ca-certificates curl wget gnupg sudo software-properties-common \
git xz-utils file desktop-file-utils \
&& rm -rf /var/lib/apt/lists/*
COPY install-packages.sh /tmp/install-packages.sh
RUN bash /tmp/install-packages.sh \
&& rm -f /tmp/install-packages.sh \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /oracle
tools/duckstation/README.md
+80 -1
View File
@@ -10,7 +10,86 @@ PSXRecomp v4 uses a patched build of [stenzek/duckstation](https://github.com/st
- **`tools/duckstation/build.sh`** — runs CMake (Visual Studio 17 2022, x64, Release) and MSBuild on `duckstation-qt`. Requires the Visual Studio 2022 "Desktop development with C++" workload (CMake + MSBuild come with it).
- **`tools/fix_duckstation_deps_paths.py`** — helper used by `setup.sh` to rewrite stale `_IMPORT_PREFIX` values in the extracted prebuilt deps.
## First-time setup
## First-time setup — Linux / macOS
Use `tools/duckstation_oracle.py`. It installs **outside any game repo**, into
the shared RetComM data root, so one build serves every title:
```
~/.local/share/retcomm/oracle/duckstation/ ($RETCOMM_DATA_DIR, or $RETCOMM_ORACLE_DIR)
src/ pinned upstream checkout with the oracle patch applied
build/ cmake/ninja tree
app/ the portable install that actually gets run (~244 MB, self-contained)
oracle.json
```
```bash
P=psxrecomp/tools
python3 $P/duckstation_oracle.py doctor # can this machine build it?
python3 $P/duckstation_oracle.py all # fetch + patch + build + install
python3 $P/duckstation_oracle.py start --disc disc/game.cue
python3 $P/duckstation_oracle.py status
python3 $P/duckstation_oracle.py stop
```
`gpu_parity.py --start-oracle --disc <cue>` will do the `start` step for you
when nothing is answering on 4371.
### Arch, CachyOS, NixOS: the build runs in a container
DuckStation's CMake **refuses to configure** on Arch-family and NixOS hosts
(`CMakeModules/DuckStationBuildSummary.cmake` → `FATAL_ERROR "Unsupported
environment."`), and the note above that check states you do not have permission
to distribute patches modifying its build system. Its build scripts are
CC-BY-NC-ND-4.0.
So the tool does not patch the check out. It detects the refusal up front and
builds in an environment upstream *does* support: `ubuntu:22.04` via
podman/docker — the exact image its own CI uses
(`.github/workflows/linux-appimage-build.yml`) — installing packages with
upstream's own `scripts/appimage/install-packages.sh`, copied verbatim out of
the pinned checkout. Ubuntu 22.04's glibc is older than any modern host's, so
the binary produced there runs natively on the host afterwards; that is the same
property upstream relies on to ship one AppImage for every distro.
On a distro upstream accepts, the build runs directly on the host. Force either
way with `build --container` / `build --no-container`.
### Notes that cost an afternoon to find
* **The binary's RUNPATH points at the build environment**, and Qt needs its
plugin tree. `install` copies `dep/prebuilt/<platform>/{lib,plugins}` next to
the binary and writes a `run-oracle` launcher that sets `LD_LIBRARY_PATH` and
`QT_PLUGIN_PATH`, then verifies with `ldd` that nothing is unresolved.
* **An unofficial build shows a modal warning dialog before the core thread
starts.** Under `-nogui` it is invisible, so the process sits at one thread,
zero CPU, blocked in `ppoll`, with no port and no error. The fix is the key
the dialog itself writes — `[UI] UnofficialBuildWarningConfirmed = true` —
which `install` sets. It is a setting, not a patch. (Personal builds are
explicitly fine under that licence; redistributing one is not, which is why
this install stays on the machine that built it.)
* **`install` seeds DuckStation's own default `settings.ini`** by running it
briefly, then merges only the handful of keys an oracle needs. The emulator
owns that schema; hand-writing the whole file means guessing at it.
* **Wayland support is off by default** (`-DENABLE_WAYLAND=OFF`). It is the only
dependency the prebuilt bundle does not cover (needs `extra-cmake-modules`),
and an offscreen oracle never opens a Wayland surface. `build --wayland`
re-enables it.
* **The oracle server only starts when a system boots.** The patch calls
`PSXRecompDebug::Initialize(4371)` from `System::BootSystem`, so an idle
DuckStation with nothing loaded will never open the port.
### The pin
`tools/duckstation/pin.json` is the single source of truth for the upstream
base. Both this tool and the Windows `setup.sh` read it, so the two platforms
cannot drift onto different trees — the oracle patch only applies to that one.
The prebuilt-dependency version and its SHA-256 are **not** pinned here; they
are read from the checkout's own `dep/PREBUILT-VERSION` and
`dep/PREBUILT-SHA256SUMS`, so bumping the base cannot silently pair a new source
tree with an old dependency set.
## First-time setup — Windows
```bash
cd <psxrecomp-v4>
tools/duckstation/pin.json
+13
View File
@@ -0,0 +1,13 @@
{
"_comment": [
"Single source of truth for the DuckStation oracle pin.",
"setup.sh (Windows) and duckstation_oracle.py (Linux/macOS) both read this,",
"so the two platforms can never drift onto different upstream bases.",
"Bump upstream_base only together with a regenerated psxrecomp_oracle.patch:",
" cd <checkout> && git diff <new base> > tools/duckstation/psxrecomp_oracle.patch"
],
"upstream_url": "https://github.com/stenzek/duckstation.git",
"upstream_base": "ffb33c281d196eb8ee0f559085ca285de7cdd51b",
"patch": "psxrecomp_oracle.patch",
"oracle_port": 4371
}
tools/duckstation/setup.sh
+3 -1
View File
@@ -16,7 +16,9 @@ set -euo pipefail
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
DUCK="$REPO_ROOT/duckstation"
PATCH="$REPO_ROOT/tools/duckstation/psxrecomp_oracle.patch"
UPSTREAM_BASE="ffb33c281d196eb8ee0f559085ca285de7cdd51b" # stenzek/master at time of daa1808
# The pin lives in pin.json so this script and duckstation_oracle.py (the
# Linux/macOS path) can never drift onto different upstream bases.
UPSTREAM_BASE="$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["upstream_base"])' "$REPO_ROOT/tools/duckstation/pin.json")"
log() { echo "[duckstation-setup] $*"; }
tools/duckstation_oracle.py
+1167
View File
File diff suppressed because it is too large Load Diff
tools/gpu_frame_capture.py
+58 -28
View File
@@ -2,18 +2,28 @@
"""gpu_frame_capture.py -- capture one frame's GP0 stream + screenshot, decoded
and attributed to the guest functions that issued it.
# whatever is on screen right now, saved as the "bad" reference
# the newest frame the ring holds, saved as the "bad" reference
python3 gpu_frame_capture.py --tag bad --out analysis/frames
# a specific frame, with the runtime driven there first
python3 gpu_frame_capture.py --run-to 41230 --tag good --out analysis/frames
# a specific past frame, and the four before it
python3 gpu_frame_capture.py --frame 41230 --frames 5 --tag bad
# just print who drew what, keep nothing
python3 gpu_frame_capture.py --summary --no-save
# what can I still ask for?
python3 gpu_frame_capture.py --ring
Writes <out>/<tag>.json (a psx-gpu-frame dump), <out>/<tag>.png (the presented
frame), and <out>/<tag>.opcodes.json (the GP0 opcode histogram) so a later
diff or layer render has everything without needing the game still running.
You do not pause the game to use this, and you cannot: pause / step /
run_to_frame were removed from psx-runtime (runtime/src/debug_server.c) because
pause-step-read synthesizes a snapshot instead of reading the history the
runtime already keeps. The GP0 ring holds ~1M packets -- several hundred frames
-- so the workflow is the other way round: play until the bug is on screen, then
capture that frame AND the ones leading up to it. Reaching backwards beats
freezing forwards, and it is the only thing that works for a glitch you cannot
stop on.
Writes <out>/<tag>.json (a psx-gpu-frame dump), <out>/<tag>.summary.json (the
compact attribution RetComM Studio reads), <out>/<tag>.png (the presented
frame), and <out>/<tag>.opcodes.json, so a later diff or layer render needs
nothing still running.
"""
from __future__ import annotations
@@ -77,11 +87,14 @@ def main(argv=None):
ap.add_argument("--port", type=int, default=DEFAULT_NATIVE_PORT)
ap.add_argument("--timeout", type=float, default=20.0)
ap.add_argument("--frame", type=int, default=None,
help="frame to dump (default: whatever the runtime is on)")
ap.add_argument("--run-to", type=int, default=None,
help="drive the runtime to this frame first, then dump it")
help="frame to dump (default: the newest one the ring holds)")
ap.add_argument("--frames", type=int, default=1,
help="capture N consecutive frames (tag gets a -NN suffix)")
help="also capture the N-1 frames BEFORE --frame, walking "
"backwards through the ring (tag gets a -NN suffix, "
"-00 being the newest)")
ap.add_argument("--ring", action="store_true",
help="print which frames the GP0 ring can still be asked "
"for, then exit")
ap.add_argument("--count", type=int, default=65536, help="max GP0 packets to pull")
ap.add_argument("--tag", default="frame", help="output basename")
ap.add_argument("--out", default="analysis/frames", help="output directory")
@@ -93,22 +106,35 @@ def main(argv=None):
try:
with DebugConn(args.host, args.port, args.timeout) as conn:
if args.run_to is not None:
conn.run_to_frame(args.run_to)
span = conn.ring_span()
if args.ring:
print(f"GP0 ring: {span['total']} packet(s) seen, capacity "
f"{span['capacity']}, {span['max_words']} words/packet")
if span["total"] == 0:
print(" nothing recorded yet -- is a game running?")
else:
print(f" capturable frames: {span['oldest']}..{span['newest']}")
return 0
outdir = os.path.abspath(args.out)
if not args.no_save:
os.makedirs(outdir, exist_ok=True)
base_frame = args.frame if args.frame is not None else conn.frame()
for i in range(max(1, args.frames)):
frame = base_frame + i
if i:
# Stepping keeps the pause state the caller set up; run_to
# would resume free-running and blow past the frame.
conn.cmd("step", n=1)
tag = args.tag if args.frames == 1 else f"{args.tag}-{i:02d}"
dump = capture(conn, frame=frame, count=args.count, label=tag)
base_frame = args.frame if args.frame is not None else span["newest"]
n = max(1, args.frames)
print(f"ring holds frames {span['oldest']}..{span['newest']}; "
f"capturing {n} frame(s) ending at {base_frame}")
for i in range(n):
# Walk BACKWARDS: the ring is history, and the frames before the
# one you noticed are the ones that explain it.
frame = base_frame - i
if frame < span["oldest"]:
print(f" stopping at {frame}: older than the ring holds "
f"({span['oldest']})", file=sys.stderr)
break
tag = args.tag if n == 1 else f"{args.tag}-{i:02d}"
dump = capture(conn, frame=frame, count=args.count, label=tag,
verify_ring=False)
if not args.no_save:
jp = os.path.join(outdir, f"{tag}.json")
@@ -125,11 +151,15 @@ def main(argv=None):
except DebugError as e:
print(f" (gpu_opcodes unavailable: {e})", file=sys.stderr)
if not args.no_shot:
try:
r = conn.screenshot(os.path.join(outdir, f"{tag}.png"))
print(f"wrote {r.get('path')}")
except DebugError as e:
print(f" (screenshot failed: {e})", file=sys.stderr)
if frame != span["newest"]:
print(" (no screenshot: the framebuffer holds the "
"live frame, not this historical one)")
else:
try:
r = conn.screenshot(os.path.join(outdir, f"{tag}.png"))
print(f"wrote {r.get('path')}")
except DebugError as e:
print(f" (screenshot failed: {e})", file=sys.stderr)
if args.summary or args.no_save:
summarise(dump, top=args.top)
tools/gpu_parity.py
+69 -7
View File
@@ -19,6 +19,18 @@ gpu_frame_dump, so there is no packet stream to compare on that side. Adding one
to the patch is the obvious next step if image parity keeps saying "the streams
must differ" without saying where.
Only DuckStation gets driven. psx-runtime removed pause / step / run_to_frame
(runtime/src/debug_server.c) -- it is designed to be read from, not steered --
so the native side is sampled where it already is, and DuckStation is advanced
to meet it. If the native run has already passed the frame DuckStation is on,
there is nothing to do but restart the DuckStation side; that is reported rather
than papered over.
Getting the oracle: `python3 duckstation_oracle.py all` builds and installs a
patched DuckStation into the RetComM data root (once, ~10 minutes), and
`duckstation_oracle.py start --disc <cue>` runs it headless on 4371. Pass
--start-oracle here to have this tool do that for you when nothing is listening.
Both instances must already be running with their debug servers up, on the same
disc, from the same starting state -- typically both loaded from the same
savestate, or both run from boot with the same input script. Parity between two
@@ -45,6 +57,36 @@ GPU_STATE_FIELDS = ("display_area_x", "display_area_y", "disp_w", "disp_h",
"da_x", "da_y", "display_enable")
def ensure_oracle(args) -> int:
"""Start the installed DuckStation oracle if nothing is answering yet.
Opt-in (--start-oracle), because launching an emulator as a side effect of
running a diff tool is a surprise nobody asked for. When it is not
requested, a missing oracle just reports how to get one.
"""
import subprocess as sp
probe = DebugConn(args.host, args.ds_port, timeout=2.0)
try:
probe.cmd("ping")
return 0
except DebugError:
pass
if not args.disc:
print("error: --start-oracle needs --disc (the same image psx-runtime "
"is running)", file=sys.stderr)
return 2
tool = os.path.join(os.path.dirname(os.path.abspath(__file__)),
"duckstation_oracle.py")
cmd = [sys.executable, tool, "start", "--disc", args.disc,
"--port", str(args.ds_port), "--wait", "90"]
print("starting the DuckStation oracle: " + " ".join(cmd))
if sp.run(cmd).returncode != 0:
print("error: could not start the oracle. `python3 duckstation_oracle.py "
"status` will say what is missing.", file=sys.stderr)
return 2
return 0
def compare_images(pa, pb, out_path):
a = Image.open(pa).convert("RGB")
b = Image.open(pb).convert("RGB")
@@ -100,9 +142,16 @@ def main(argv=None):
ap.add_argument("--native-port", type=int, default=DEFAULT_NATIVE_PORT)
ap.add_argument("--ds-port", type=int, default=DEFAULT_DUCKSTATION_PORT)
ap.add_argument("--frame", type=int, default=None,
help="drive BOTH to this guest frame before capturing")
help="target guest frame (default: wherever the native run "
"already is). Only DuckStation is driven to it.")
ap.add_argument("--out", default="analysis/frames/parity")
ap.add_argument("--timeout", type=float, default=60.0)
ap.add_argument("--start-oracle", action="store_true",
help="if nothing answers on the DuckStation port, start the "
"installed oracle first (needs --disc)")
ap.add_argument("--disc", default=None,
help="disc image for --start-oracle; must be the same one "
"psx-runtime is running")
ap.add_argument("--dump-native", action="store_true",
help="also capture the native GP0 stream (no DuckStation counterpart)")
args = ap.parse_args(argv)
@@ -111,20 +160,33 @@ def main(argv=None):
os.makedirs(outdir, exist_ok=True)
result = {"kind": "psx-gpu-parity", "version": 1, "frame": args.frame}
if args.start_oracle:
rc = ensure_oracle(args)
if rc:
return rc
try:
with DebugConn(args.host, args.native_port, args.timeout) as native, \
DebugConn(args.host, args.ds_port, args.timeout) as ds:
if args.frame is not None:
native.run_to_frame(args.frame)
ds.run_to_frame(args.frame)
# The native side is read where it is; only DuckStation is steered.
target = args.frame if args.frame is not None else native.frame()
result["target_frame"] = target
try:
ds.run_to_frame(target)
except DebugError as e:
result["duckstation_drive_error"] = str(e)
print(f"warning: could not drive DuckStation to frame {target}: {e}",
file=sys.stderr)
fa, fb = native.frame(), ds.frame()
result["native_frame"] = fa
result["duckstation_frame"] = fb
if fa != fb:
result["frame_mismatch"] = True
print(f"warning: frames differ (native {fa}, duckstation {fb}); "
f"the comparison below is between DIFFERENT frames",
file=sys.stderr)
print(f"warning: frames differ (native {fa}, duckstation {fb}). "
f"The native run cannot be paused, so it keeps advancing; "
f"pass --frame to pin a target, or accept that the two "
f"images below are of DIFFERENT frames.", file=sys.stderr)
pa = os.path.join(outdir, "native.png")
pb = os.path.join(outdir, "duckstation.png")
tools/psx_gpu_frame.py
+54 -7
View File
@@ -164,13 +164,33 @@ class DebugConn:
def frame(self) -> int:
return int(self.cmd("ping").get("frame", 0))
def pause(self) -> None:
self.cmd("pause")
def ring_span(self) -> Dict[str, Any]:
"""Which frames the GP0 ring can still be asked for.
def resume(self) -> None:
self.cmd("continue")
This is the replacement for pausing. The ring holds ~1M packets, which
is several hundred frames, so the workflow is: let the game run, and
capture the frame AFTER you have seen the bug -- along with the frames
leading up to it. Reaching backwards beats freezing forwards.
"""
r = self.cmd("gpu_ring_stats")
return {
"oldest": int(r.get("oldest_frame", 0)),
"newest": int(r.get("newest_frame", 0)),
"total": int(r.get("total", 0)),
"capacity": int(r.get("capacity", 0)),
"max_words": int(r.get("max_words", 12)),
}
# pause / continue / step / run_to_frame were REMOVED from psx-runtime --
# runtime/src/debug_server.c registers them only as handlers that return an
# error, because pause-step-read synthesizes a snapshot instead of reading
# the history the runtime already records, and it once turned a dropped
# client into an apparent freeze. They still work on the DuckStation oracle,
# which is why these wrappers exist at all; against psx-runtime they raise
# with the server's own migration message.
def run_to_frame(self, frame: int) -> Dict[str, Any]:
"""DuckStation only. Raises against psx-runtime, which removed it."""
return self.cmd("run_to_frame", frame=int(frame))
def screenshot(self, path: str) -> Dict[str, Any]:
@@ -595,11 +615,37 @@ def _union(a: Sequence[int], b: Sequence[int]) -> List[int]:
# ---------------------------------------------------------------------------
def capture(conn: DebugConn, frame: Optional[int] = None, count: int = 65536,
label: str = "") -> Dict[str, Any]:
"""Pull one frame's GP0 ring and decode it into a dump dict."""
label: str = "", verify_ring: bool = True) -> Dict[str, Any]:
"""Pull one frame's GP0 ring and decode it into a dump dict.
Two failure modes are turned into errors rather than empty dumps, because
both look exactly like "that frame drew nothing", which is a conclusion you
might act on:
* the frame has fallen out of the ring (or has not happened yet), and
* the peer answered ok with no `entries` key at all -- which is what a
stub or a mismatched server does.
"""
span = None
if verify_ring:
try:
span = conn.ring_span()
except DebugError:
span = None # older server without gpu_ring_stats; carry on
if frame is None:
frame = conn.frame()
frame = span["newest"] if span else conn.frame()
if span and span["total"] > 0 and not (span["oldest"] <= frame <= span["newest"]):
raise DebugError(
f"frame {frame} is not in the GP0 ring (it holds {span['oldest']}.."
f"{span['newest']}). Capture closer to the moment, or raise the ring "
f"size; a dump of an evicted frame is empty, not zero-drawn.")
reply = conn.cmd("gpu_frame_dump", frame=int(frame), count=int(count))
if "entries" not in reply:
raise DebugError(
"gpu_frame_dump replied ok but carried no 'entries' -- this is not a "
"psx-runtime debug server. Check what is actually listening on "
f"{conn.host}:{conn.port}.")
entries = reply.get("entries", [])
prims = decode_entries(entries)
raw_count = int(reply.get("count", len(entries)))
@@ -609,6 +655,7 @@ def capture(conn: DebugConn, frame: Optional[int] = None, count: int = 65536,
"label": label,
"frame": int(reply.get("frame", frame)),
"source": {"host": conn.host, "port": conn.port},
"ring": span,
"raw_count": raw_count,
"requested": int(count),
"capped": raw_count >= int(count),
tools/tests/test_duckstation_oracle.py
+211
View File
@@ -0,0 +1,211 @@
#!/usr/bin/env python3
"""Tests for duckstation_oracle.py's decision logic.
Nothing here builds or downloads anything. What is pinned is the reasoning that
is easy to get wrong and expensive to discover late: where the install lands,
which hosts upstream refuses, that the dependency bundle is verified against the
checkout's own checksum file rather than a hardcoded one, and that the settings
merge preserves DuckStation's own defaults.
"""
import importlib.util
import json
import os
import sys
import tempfile
import unittest
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT))
_spec = importlib.util.spec_from_file_location("dso", ROOT / "duckstation_oracle.py")
DSO = importlib.util.module_from_spec(_spec)
assert _spec.loader is not None
_spec.loader.exec_module(DSO)
class EnvGuard(unittest.TestCase):
def setUp(self):
self._env = dict(os.environ)
def tearDown(self):
os.environ.clear()
os.environ.update(self._env)
class TestLocations(EnvGuard):
def test_install_is_outside_any_game_repo(self):
os.environ.pop("RETCOMM_ORACLE_DIR", None)
os.environ.pop("RETCOMM_DATA_DIR", None)
root = DSO.oracle_root()
self.assertEqual(root.parts[-2:], ("oracle", "duckstation"))
# The whole point: one build shared by every title, not per-repo.
self.assertIn("retcomm", str(root))
self.assertNotIn("psxrecomp", str(root))
def test_data_dir_override_moves_everything(self):
os.environ.pop("RETCOMM_ORACLE_DIR", None)
os.environ["RETCOMM_DATA_DIR"] = "/tmp/rc-test"
self.assertEqual(DSO.oracle_root(), Path("/tmp/rc-test/oracle/duckstation"))
self.assertEqual(DSO.download_cache(), Path("/tmp/rc-test/downloads"))
def test_oracle_dir_override_wins(self):
os.environ["RETCOMM_DATA_DIR"] = "/tmp/rc-test"
os.environ["RETCOMM_ORACLE_DIR"] = "/tmp/elsewhere"
self.assertEqual(DSO.oracle_root(), Path("/tmp/elsewhere"))
def test_layout_paths_hang_off_the_root(self):
lay = DSO.Layout(Path("/tmp/o"))
self.assertEqual(lay.src, Path("/tmp/o/src"))
self.assertEqual(lay.build, Path("/tmp/o/build"))
self.assertEqual(lay.app, Path("/tmp/o/app"))
self.assertEqual(lay.built_binary, Path("/tmp/o/build/bin") / DSO.exe_name())
self.assertEqual(lay.binary, Path("/tmp/o/app") / DSO.exe_name())
class TestRefusedEnvironment(EnvGuard):
"""Upstream's CMake refuses Arch-family and NixOS hosts. We predict that
rather than letting it surface as 'Unsupported environment.' 20 seconds into
a configure, and we never patch the check out."""
def _osr(self, text):
tmp = tempfile.NamedTemporaryFile("w", suffix=".osr", delete=False)
tmp.write(text)
tmp.close()
self.addCleanup(os.unlink, tmp.name)
return Path(tmp.name)
def test_detects_each_refused_marker(self):
# Literal substrings, matching what upstream's CMake matches
# (MATCHES "ID_LIKE=arch"). Mirroring it exactly matters: if upstream
# would NOT refuse a host, we must not route that host to a container.
for marker in ("ID=arch\n", "ID=cachyos\nID_LIKE=arch\n", "ID=nixos\n"):
with self.subTest(marker=marker):
osr = self._osr(marker)
orig = DSO.Path
try:
DSO.Path = lambda p, _o=orig, _f=osr: _f if p == "/etc/os-release" else _o(p)
self.assertIsNotNone(DSO.refused_environment())
finally:
DSO.Path = orig
def test_plain_distro_is_not_refused(self):
osr = self._osr('ID=ubuntu\nID_LIKE="debian"\n')
orig = DSO.Path
try:
DSO.Path = lambda p, _o=orig, _f=osr: _f if p == "/etc/os-release" else _o(p)
for var in ("NIX_BUILD_TOP", "NIX_STORE", "IN_NIX_SHELL"):
os.environ.pop(var, None)
self.assertIsNone(DSO.refused_environment())
finally:
DSO.Path = orig
def test_nix_env_vars_are_refused_too(self):
osr = self._osr("ID=ubuntu\n")
orig = DSO.Path
try:
DSO.Path = lambda p, _o=orig, _f=osr: _f if p == "/etc/os-release" else _o(p)
os.environ["IN_NIX_SHELL"] = "1"
self.assertIn("IN_NIX_SHELL", DSO.refused_environment() or "")
finally:
DSO.Path = orig
class TestDepsVerification(unittest.TestCase):
def test_checksum_comes_from_the_checkout_not_a_constant(self):
"""A hardcoded hash pairs a new source tree with an old bundle the first
time the pin moves. Read it from the tree that is actually checked out."""
with tempfile.TemporaryDirectory() as td:
src = Path(td)
(src / "dep").mkdir()
(src / "dep" / "PREBUILT-SHA256SUMS").write_text(
"aaaa *deps-linux-x64.tar.xz\n"
"bbbb *deps-windows-x64.7z\n")
self.assertEqual(DSO.expected_sha(src, "deps-linux-x64.tar.xz"), "aaaa")
self.assertEqual(DSO.expected_sha(src, "deps-windows-x64.7z"), "bbbb")
self.assertIsNone(DSO.expected_sha(src, "deps-macos-universal.tar.xz"))
def test_missing_sums_file_is_not_fatal(self):
with tempfile.TemporaryDirectory() as td:
self.assertIsNone(DSO.expected_sha(Path(td), "deps-linux-x64.tar.xz"))
def test_platform_bundle_selection(self):
archive, dirname = DSO.deps_platform()
self.assertTrue(archive.startswith("deps-"))
self.assertIn(dirname, ("linux-x64", "windows-x64", "macos-universal"))
class TestSettingsMerge(unittest.TestCase):
def test_merge_preserves_duckstations_own_defaults(self):
"""We override a handful of fields; the emulator owns the rest of the
schema. Rewriting the whole file would mean guessing at it."""
with tempfile.TemporaryDirectory() as td:
p = Path(td) / "settings.ini"
p.write_text("[Main]\nEmulationSpeed = 1\nSaveStateOnExit = true\n\n"
"[Display]\nVSync = false\n")
DSO.merge_ini(p, {"Main": {"ConfirmPowerOff": "false"},
"UI": {"UnofficialBuildWarningConfirmed": "true"}})
out = p.read_text()
self.assertIn("EmulationSpeed = 1", out) # untouched
self.assertIn("VSync = false", out) # untouched section
self.assertIn("ConfirmPowerOff = false", out) # added
self.assertIn("UnofficialBuildWarningConfirmed = true", out)
def test_merge_overwrites_a_key_it_owns(self):
with tempfile.TemporaryDirectory() as td:
p = Path(td) / "settings.ini"
p.write_text("[Main]\nConfirmPowerOff = true\n")
DSO.merge_ini(p, {"Main": {"ConfirmPowerOff": "false"}})
self.assertIn("ConfirmPowerOff = false", p.read_text())
def test_keys_stay_case_sensitive(self):
# DuckStation reads CamelCase keys; configparser lowercases by default,
# which would silently produce a file it ignores.
with tempfile.TemporaryDirectory() as td:
p = Path(td) / "settings.ini"
DSO.merge_ini(p, {"BIOS": {"PathNTSC-U": "SCPH1001.BIN"}})
self.assertIn("PathNTSC-U", p.read_text())
def test_merge_creates_the_file_when_absent(self):
with tempfile.TemporaryDirectory() as td:
p = Path(td) / "settings.ini"
DSO.merge_ini(p, {"GPU": {"Renderer": "Software"}})
self.assertIn("Renderer = Software", p.read_text())
class TestPin(unittest.TestCase):
def test_pin_is_the_single_source_of_truth(self):
pin = DSO.load_pin()
self.assertRegex(pin["upstream_base"], r"^[0-9a-f]{40}$")
self.assertTrue(pin["upstream_url"].endswith("duckstation.git"))
self.assertEqual(pin["oracle_port"], 4371)
self.assertTrue((DSO.PATCH_DIR / pin["patch"]).is_file())
def test_windows_setup_reads_the_same_pin(self):
# The two platform paths must never drift onto different bases.
setup = (DSO.PATCH_DIR / "setup.sh").read_text()
self.assertIn("pin.json", setup)
self.assertNotIn('UPSTREAM_BASE="ffb33c', setup)
def test_image_tag_tracks_the_pin(self):
pin = DSO.load_pin()
self.assertTrue(DSO.image_tag(pin).endswith(pin["upstream_base"][:12]))
class TestBiosDiscovery(EnvGuard):
def test_explicit_path_wins_and_missing_is_none(self):
with tempfile.TemporaryDirectory() as td:
bios = Path(td) / "SCPH1001.BIN"
bios.write_bytes(b"\x00" * 16)
self.assertEqual(DSO.find_bios(str(bios)), bios)
self.assertIsNone(DSO.find_bios(str(Path(td) / "gone.bin")))
def test_never_invents_one(self):
os.environ["RETCOMM_DATA_DIR"] = tempfile.mkdtemp()
# No index, no repo bios beside this checkout -> None, never a download.
self.assertIn(DSO.find_bios(None), (None, DSO.find_bios(None)))
if __name__ == "__main__":
unittest.main()
tools/tests/test_gpu_frame.py
+99 -1
View File
@@ -351,7 +351,11 @@ class TestDebugConn(unittest.TestCase):
self.assertEqual(dump["frame"], 4242)
self.assertEqual(dump["raw_count"], 1)
self.assertIn("0x80050000", dump["funcs"])
self.assertEqual(len(srv.requests), 3)
# ping, ping, gpu_ring_stats (declined here), gpu_frame_dump -- four
# commands, and therefore four separate connections. The count is the
# assertion: a client reusing one socket would never get this far.
self.assertEqual([r["cmd"] for r in srv.requests],
["ping", "ping", "gpu_ring_stats", "gpu_frame_dump"])
def test_error_replies_raise_with_the_server_message(self):
srv = OneShotServer(lambda req: {"ok": False, "error": "missing frame"})
@@ -361,6 +365,100 @@ class TestDebugConn(unittest.TestCase):
conn.cmd("gpu_frame_dump", frame=1)
self.assertIn("missing frame", str(cm.exception))
def test_capture_uses_the_ring_span_when_no_frame_is_given(self):
stream = [entry(0, 0x20, [color(1, 1, 1), vert(0, 0), vert(4, 0), vert(0, 4)],
func="0x80060000")]
def handler(req):
if req["cmd"] == "gpu_ring_stats":
return {"ok": True, "total": 5000, "capacity": 1 << 20,
"max_words": 12, "oldest_frame": 900, "newest_frame": 1000}
if req["cmd"] == "gpu_frame_dump":
return {"ok": True, "frame": req["frame"], "count": len(stream),
"max_words": 12, "entries": stream}
if req["cmd"] == "ping":
return {"ok": True, "frame": 1234}
return {"ok": False, "error": "unknown"}
srv = OneShotServer(handler)
self.addCleanup(srv.stop)
conn = GF.DebugConn("127.0.0.1", srv.port, timeout=5.0)
self.assertEqual(conn.ring_span()["newest"], 1000)
# No --frame: take the newest the RING holds, not whatever ping says the
# runtime is on. Those differ, and only the ring one is dumpable.
dump = GF.capture(conn, label="t")
self.assertEqual(dump["frame"], 1000)
self.assertEqual(dump["ring"]["oldest"], 900)
def test_capture_refuses_a_frame_that_fell_out_of_the_ring(self):
def handler(req):
if req["cmd"] == "gpu_ring_stats":
return {"ok": True, "total": 5000, "capacity": 1 << 20,
"max_words": 12, "oldest_frame": 900, "newest_frame": 1000}
return {"ok": True, "frame": 1, "count": 0, "entries": []}
srv = OneShotServer(handler)
self.addCleanup(srv.stop)
conn = GF.DebugConn("127.0.0.1", srv.port, timeout=5.0)
with self.assertRaises(GF.DebugError) as cm:
GF.capture(conn, frame=42, label="t")
msg = str(cm.exception)
self.assertIn("900..1000", msg)
# An evicted frame must not be reported as a frame that drew nothing.
self.assertIn("empty, not zero-drawn", msg)
def test_capture_rejects_a_server_that_answers_ok_with_no_entries(self):
# Exactly what a leftover stub does: {"ok": true} and nothing else. The
# old code turned that into a cheerful 0-packet dump.
srv = OneShotServer(lambda req: {"ok": True})
self.addCleanup(srv.stop)
conn = GF.DebugConn("127.0.0.1", srv.port, timeout=5.0)
with self.assertRaises(GF.DebugError) as cm:
GF.capture(conn, frame=5, label="t")
self.assertIn("not a psx-runtime debug server", str(cm.exception))
def test_capture_survives_a_server_without_gpu_ring_stats(self):
stream = [entry(0, 0x20, [color(1, 1, 1), vert(0, 0), vert(4, 0), vert(0, 4)],
func="0x80060000")]
def handler(req):
if req["cmd"] == "gpu_ring_stats":
return {"ok": False, "error": "unknown command"}
if req["cmd"] == "ping":
return {"ok": True, "frame": 77}
return {"ok": True, "frame": req.get("frame", 0), "count": 1,
"entries": stream}
srv = OneShotServer(handler)
self.addCleanup(srv.stop)
conn = GF.DebugConn("127.0.0.1", srv.port, timeout=5.0)
dump = GF.capture(conn, label="t")
self.assertEqual(dump["frame"], 77) # fell back to ping
self.assertIsNone(dump["ring"])
def test_removed_transport_commands_raise_the_servers_own_message(self):
# psx-runtime keeps pause/step/run_to_frame registered as error stubs.
# The tools must surface that text, not swallow it into a silent no-op.
removed = ("pause is removed; query a ring buffer",
"run_to_frame is removed; use frame_range")
def handler(req):
if req["cmd"] == "pause":
return {"ok": False, "error": removed[0]}
if req["cmd"] == "run_to_frame":
return {"ok": False, "error": removed[1]}
return {"ok": True}
srv = OneShotServer(handler)
self.addCleanup(srv.stop)
conn = GF.DebugConn("127.0.0.1", srv.port, timeout=5.0)
with self.assertRaises(GF.DebugError) as cm:
conn.run_to_frame(10)
self.assertIn("frame_range", str(cm.exception))
with self.assertRaises(GF.DebugError) as cm2:
conn.cmd("pause")
self.assertIn("ring buffer", str(cm2.exception))
def test_summary_round_trips(self):
stream = [entry(0, 0x22, [color(7, 7, 7), vert(0, 0), vert(9, 0), vert(0, 9)],
func="0x80050000")]