The BIOS exception handler ran interpreted for the life of every process,
1.22 billion instructions and 44.6% of all interpreted work in Breath of
Fire III, even though it is compiled and even though the stub that
unblessed it sat at exactly the address bios/SCPH1001.toml already
declared as an install slot. Two mechanisms existed and did not compose:
the emitter hooked the declared slot PC, while the runtime's bless
verifier memcmp'd the whole body including that slot, so the body was
MISMATCH forever and native code never reached the hook.
Three parts, none of which pays off alone:
1. Publish the declared ranges to the runtime. The emitter emits a
PsxKernelPatchRange table next to the kernel body table and routes it
through the backend descriptor; the bless verifier compares a body in
segments that skip them. CLEAN now means "every byte outside every
declared range still matches ROM", which is the claim the native code
actually depends on.
2. Generalise the slot model to what the patches are. The old model
assumed a 4-word lui/addiu/jalr/nop stub written over ROM zeros and
fired on `word != 0`. Measured against both images, only one of five
observed shapes fits: patchers also plant a `jr` over real ROM
instructions, rewrite an 11-word prologue and insert an `mfc0 Cause`,
and NOP out a driver routine. A slot is now a range with a length and
an explicit `resume` (jalr / fallthrough / none), detected by
comparing every word against its ROM-baked value.
3. Hand back from the interpreter at the range end. Kernel page 0 is
permanently dirty, because the handler saves registers there on every
exception, so straight-line interpretation would run the whole
function after the patch. The emitter registers the resume PC as a
continuation key and dirty_ram_interp surfaces there.
A PC inside a declared range must never be a native dispatch key: the
compiled bytes there are not what executes. Seven pre-existing
jal-return continuations violated that and wedged the boot at frame 0 in
an infinite dispatch loop (OpenBIOS RAM 0x357C). The emitter now drops
continuation keys inside a range so dispatch misses through to the
interpreter, and refuses to emit if a range covers a function entry,
which would silently remove a callable function instead.
Both profiles declare their measured ranges, and SCPH1001.toml finally
pins [program.image] sha256 to v2.2 (CRC32 37157331) so a v2.0 dump
refuses to generate rather than producing phantom discovery gaps.
Measured on Breath of Fire III (SLPS-00990, Psy-Q SDK 3.70), headless,
matched frames, PSX_KERNEL_PATCH_RANGES=0 for the OFF side:
OpenBIOS SCPH-1001 v2.2
bless mismatch 21 -> 0 74 -> 0
kernel bodies -87.7%/frame -97.8%/frame
all interp insns -42.6%/frame -94.2%/frame
The residual kernel-body work is exactly declared-words x entries on
every range, so nothing leaks past the hook. The A0/B0/C0 vector
trampolines are unchanged (+0.2%, noise): the profile excludes them by
design and they are a separate job. Card read traces are byte-identical
across the A/B on both images, all 32 entries, which exercises the two
ranges that replace the kernel's card handler.
Rule 18 holds throughout: the guest's patched instructions still execute
as written on the dirty-RAM interpreter. Only the code around them
changes backend. Nothing is synthesised.
Tests: runtime/tests/test_kernel_patch_ranges.c (new, the verifier's
accept/reject behaviour and the hand-back predicate) and the install-slot
range invariants in recompiler/tests/bios_address_model_test.cpp.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Make a bare interpreted PC explainable from the session-long per_pc snapshot
alone. DirtyRamPcEntry gains occ_crc / occ_ok (the static overlay variant whose
code ranges span the PC, via a per-PC scan of the static-match cache, and
whether it validated at that moment) and last_ext_ra (gpr[31] at the most
recent external entry, i.e. the caller that reached this interior). Both are
stamped only on external entries -- arrivals from native dispatch, not interp
block chaining -- and emitted per row in dirty_ram_stats.per_pc as occ_crc,
occ_ok and ext_ra.
Reading the pair: occ_ok=1 is an interior gap inside live native code (an
alias seed fixes it); occ_ok=0 with a CRC means the spanning piece is resident
but CRC-missing, so it was compiled from another section's bytes or rewritten
at run time and no seed helps; 0 means nothing compiled spans the PC.
psx_overlay_resident_crc_at() records each static-match cache entry's code
span (lo_min/hi_max) so the scan is a range test; DLL-path titles fall back to
the loader's last residency hash. The debug-server reply buffer grows 32 -> 64
KiB for the wider rows; the tail reserve still cuts between rows.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Add passive DMA2 ordering-table counters to dma_state so long or aborted linked-list walks are visible without relying on screenshots. The counters track starts, duplicate starts dropped while a walk is active, completions, cancels, last/max walk size and cycle cost, CHCR poll counts, the initiator PC, and the last eight cancel records.
This intentionally does not port PR #302's superseded walker changes or the PSX_GPU_LL_SYNC diagnostic lever; PR #304 remains the timing implementation.
Co-authored-by: Kevin Kline <9866117+kerokline@users.noreply.github.com>
The repository root carried seven markdown files that were not onboarding
material, plus two that had been superseded by the code they described. This
sorts them: reader-facing docs to docs/, working notes to docs/internal/, and
the two superseded ones merged into the docs that replaced them.
Moved to docs/: WIDESCREEN.md, ENHANCEMENTS.md, TCP_COMMANDS.md
Moved to docs/internal/: PRINCIPLES.md, ISSUES.md, DEBUG.md, GTE_LM_FIX.md,
and from docs/: bios-hardening.md, overlay-status.md,
overlay-plan.md, overlay-recompilation-design.md,
overlay-discovery.md, TOMBA2_EXC_CONSULT_RESPONSE.md,
CASE_A_AOT_GAP.md, audit_inventory.md
Moved to docs/internal/upstream/: UPSTREAM_2026-08-20.md
The root keeps only what belongs there: README.md, CONTRIBUTING.md, LICENSE,
CLAUDE.md, THIRD_PARTY_ATTRIBUTION.md, RELEASE_NOTES.md, the dotfiles and
psxrecomp_cli.py.
MERGES
SPLITGEN_MIGRATION.md folded into docs/BUILDING.md as "Split generated C". Its
migration instructions were stale in a way that mattered: it told every game
repo to hand-edit its CMakeLists and add a file(GLOB ...) over
<serial>_full*.c, but runtime/runtime.cmake now does that itself. GEN_FULL_GLOB
is a multi-value argument of psxrecomp_add_game_runtime(); the loop at
runtime.cmake:1985 makes each pattern absolute against CMAKE_CURRENT_SOURCE_DIR,
expands it with file(GLOB), and concatenates the hits into
GAME_GENERATED_FULL_C. So a title names the pattern and gets however many shards
a regeneration produced. The rewritten section describes that, the two-step
fallback when the globs match nothing (GEN_FULL_FALLBACK, else <serial>_full.c
derived from the dispatch marker by substituting _dispatch.c), and the one
caveat the code actually has: that file(GLOB) is plain, not CONFIGURE_DEPENDS,
so a regeneration which changes the shard COUNT needs a re-configure to be seen.
psxrecomp-toml-readme.md folded into docs/GAME_PROJECT_SETUP.md as "EXE-only
autofill (psxrecomp-toml)", placed after "Disc autofill (probe_disc.py)" and
before "Project Studio" so the two autofill paths sit together. Its documented
flags were checked against the usage text in recompiler/src/main_toml.cpp:28-42
and against the argument parser at :150-165 -- all seven options match with no
drift, so the flag list is reproduced as-is. Added a pointer that probe_disc.py
is preferable when a cue exists, since that tool does the same EXE-header work
plus disc identity and netplay gates.
FUNCTIONAL FIXES
tools/gen_tcp_commands.py wrote to REPO/"TCP_COMMANDS.md". After the move that
path does not exist, so the generator would have created a new stray root file
and --check would have failed against it forever. Retargeted to
REPO/"docs"/"TCP_COMMANDS.md", along with the seven docstring and message
strings that name the path.
Running it surfaced two further problems, both fixed here:
* The doc was genuinely stale -- pre-existing drift, not introduced by this
commit (the moved file is byte-identical to origin/master, and no
debug-server command table is touched here). Four commands registered since
the last regeneration were missing from the index -- cdrom_timing_dump,
savestate_status, ws_ui_groups, xprobe_watch -- and scanline had gained a
backend. Regenerated: 307 -> 311 commands, and the change is confined to the
AUTOGEN block (the hand-written head and tail are byte-identical to master).
tools/gen_tcp_commands.py --check now exits 0.
* DOC.write_text() has no newline argument, so Python translated every newline
to os.linesep. On Windows that rewrote all 585 lines of the file to CRLF and
turned a four-line index update into a whole-file diff -- measured: 589 CRLF
/ 0 LF after a run, against 0 CRLF / 585 LF in the repo. Now written through
an explicit newline="" handle, verified to leave the file at 0 CRLF.
runtime/include/debug_server.h:9 and runtime/src/debug_server.c:8 cited "TCP.md
and DEBUG.md". TCP.md has never existed in this repo under that name; both now
name docs/TCP_COMMANDS.md and docs/internal/DEBUG.md.
LINK AND REFERENCE FIXES
Every inbound reference to a moved file was located by grepping each basename
across the whole tracked tree, not just the markdown. Markdown hrefs in
README.md, CONTRIBUTING.md, docs/ARCHITECTURE.md, docs/COMPILING_OVERLAYS.md,
docs/AOT_OVERLAY_PLAN.md, docs/RELOCATION_MANIFEST_FORMAT.md,
docs/config_schema.md, docs/OVERLAY_XREF.md, docs/TCP_COMMANDS.md and several
docs/internal/ files now point at the new locations; where two files became
siblings under docs/internal/ the path was shortened rather than lengthened
(bios-hardening.md's reference to overlay-discovery.md, for instance).
docs/COMPILING_OVERLAYS.md's "in this folder" phrasing was wrong once two of the
three files it listed moved, so it now names internal/ explicitly.
24 source-comment mentions of ENHANCEMENTS.md sections G1.x across 19 files
(recompiler, runtime, cmake) were retargeted to docs/ENHANCEMENTS.md with the
section numbers untouched. docs/SHADOW_ENHANCEMENTS.md is a DIFFERENT file and
was deliberately excluded.
20 rationale strings citing docs/CASE_A_AOT_GAP.md across the four
recompiler/seeds/*.json files were updated. Done as a byte-level substring
replacement rather than a JSON round-trip: two of those files are stored with
CRLF endings and two with LF, and re-serializing them reformatted 5,700 lines.
The byte-level version changes exactly 20 lines and each file still parses.
References that look similar but point elsewhere were deliberately left alone,
and are worth naming so a future pass does not "fix" them:
* "recomp-template PRINCIPLES.md" / "NES recomp template PRINCIPLES.md" in
CLAUDE.md:43, docs/SHADOW_ENHANCEMENTS.md:28,
docs/internal/HLE_SCHEDULER_CARVEOUT_PLAN.md:5,
runtime/src/debug_server.c:4934 and tools/parity_diff.py:8 are a different
repository's file. Only the five that mean this repo's PRINCIPLES.md were
updated.
* Every remaining bare ISSUES.md mention is a GAME repo's tracker
(MegaManX6Recomp 2/7/8/9, TombaRecomp 6), not this repo's.
* docs/disc-speed.md:4,54,56 and docs/internal/bios-hardening.md:5 name
overlay-discovery as a BRANCH, not a file, so no path applies.
INDEXES
README.md's "New here?" table gains rows for docs/WIDESCREEN.md,
docs/ENHANCEMENTS.md and docs/TCP_COMMANDS.md. docs/internal/README.md gains
sections indexing all fourteen documents that moved in, plus an "Upstream
contributor reviews" section describing upstream/ and the new sweep file, plus a
"Removed" bullet for CYCLE_TIMING_ARCH.md (deleted in the previous commit)
pointing at FAITHFUL_TIMING_PLAN.md, CYCLE_MODEL_BEETLE.md and
PRECISE_IRQ_SLICE.md -- following the SLJIT_PERSIST_CACHE.md precedent already
in that file.
docs/ENHANCEMENTS.md:7 claimed framework bugs live in "game repos' ISSUES.md
until a framework tracker exists". A framework tracker does exist; it now says
so and names the Beads epic.
VERIFICATION
A link checker was written to scan every tracked *.md for relative links,
resolving each against the tree being scanned. Baseline at origin/master: 129
files, 137 relative links, 3 broken. After both commits: 127 files, 159 relative
links, 3 broken -- the SAME three, all pre-existing and untouched here:
docs/DATA_SHARDS.md -> INSTANT_LOADS_PARKED.md (never existed)
docs/DATA_SHARDS.md -> INSTANT_LOADS_SPIKE2.md (never existed)
docs/NETPLAY.md -> ../lib/recomp-net/README.md (inside a submodule)
Zero new broken links. The checker's own coverage was checked against a raw grep
for markdown link openers: the 3-link difference is C++ lambdas inside code
blocks, correctly not treated as links.
Also verified: py_compile over all 113 tracked tools/*.py plus psxrecomp_cli.py;
all four seed JSONs still parse; the test-registration guard still reports 123
test files all registered; recompiler/tests/test_coverage_vault.py still passes;
and no file in this commit had its line endings flipped (checked mechanically
across all 50 modified files, after two files were caught and redone).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Slicing the GPU linked-list DMA across guest cycles (8af48ae9, PR #277) also
changed what a walk costs: 8 clocks for every node header plus 5 setup clocks
per non-empty node, where the pre-slicing path charged one clock per word via
schedule_delayed_complete(2, actual_words, DMA_GPU_CYCLES_PER_WORD).
An ordering table is mostly EMPTY nodes -- Final Fantasy VII's runs to
ot_max 4094 -- and each empty slot still paid the full 8-cycle header for a
packet that does not exist. Measured on FF7 (SCUS-94163):
4,807 nodes / 12,757 words 52,384 cycles vs 17,564 (3.0x)
44,010 nodes / 117,507 words 499,187 cycles vs 161,517 (3.1x)
An NTSC frame is 564,480 cycles, so a single walk reached 88% of a frame.
FF7's libgpu polls CHCR bit 24, finds the channel still busy after 790-890
reads, and force-clears the start bit at 0x800484F0 -- the documented
stuck-channel path. cancel_async_transfer() then abandons the walk partway
through, and the rest of the frame's geometry is never sent to GP0. In a
battle 12.6% of ordering tables were aborted this way; models rendered with
missing or shattered limbs.
The guest is behaving correctly: it inspected the busy bit before clearing it.
The slicing is also correct and worth keeping -- a guest really can change a
packet after starting DMA and before DMA reaches it, and only a sliced walker
models that. Only the cost was wrong.
Header is one clock, setup is free, so a walk costs exactly (nodes + words).
When setup is free the payload boundary is the same service as the header, so
the header phase falls through instead of returning; cycles_to_event() never
reports zero, and returning would spend an unaccounted scheduler cycle per
non-empty node. The packet is still read at its own boundary, so #277's
mid-walk mutation behaviour is unchanged and its test still passes.
(nodes + words) is not a new assertion -- it is the cost this runtime shipped
and was validated against. A non-zero per-node overhead may well be real on
hardware, but it needs a measurement behind it, not a substituted constant.
Observability, because none of the above was visible:
dma_state gains a gpu_ot block -- starts, completes, cancels, starts_dropped,
nodes/words/cycles of the last walk and the max, plus guest reads of CHCR(2)
and a ring of the last 8 aborts carrying the guest PC, CHCR value, how far
the walk got, and how many times the guest polled before giving up. The
silent early return in start_async_gpu_linked_list() is now counted as
starts_dropped rather than losing an ordering table with nothing said.
PSX_GPU_LL_SYNC=1 drains a walk at start. Diagnostic A/B lever only: it
completes immediately as well, so it is not a faithful restoration of the
pre-slicing timing and must not be used as a fix.
The timing test now asserts the cost model rather than describing it: a table
of empty nodes must cost exactly one cycle per node. That is the check that
would have caught this.
Verified on FF7 SCUS-94163 from a save state: cycles_max 58,933 -> 16,972,
a 144-node/490-word walk costs exactly 490 cycles, and battle geometry renders
correctly with the walk still sliced.
Not fixed here: the same commit also broke gpu_frame_dump's func attribution.
GP0 writes now issue from dma_advance() on the cycle scheduler, so every
primitive is stamped with whatever the scheduler interrupted -- all 1,803
primitives in a captured battle frame attributed to BIOS 0x1FC02B50 instead of
the issuing game function. That disables gpu_frame_diff's "a function stopped
drawing" verdict and gpu_frame_layers entirely. Stamping the DMA initiator was
tried and does not work: g_debug_current_func_addr reads 0 at kick time. The
ra column still carries real game addresses as a fallback. Left as a separate
issue rather than shipped as a fix that does not fix it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L9v7e6AVZe3eRP8VXey9Pz
Adds an optional scanline effect to the GL present path: a soft sinusoidal
beam pitched to the PS1 display-line grid (one dark line per native scanline),
applied in both the present and interpolation shaders and gated by output
scale so it never shimmers below ~2x. Orthogonal to the crt_filter colour LUT
(the two compose).
Controls: [video] scanlines / scanline_strength (game.toml + settings.toml),
PSX_SCANLINES / PSX_SCANLINE_STRENGTH env overrides, an F6 hotkey
(HOST_KEYMAP_SCANLINES, rebindable via [KeyMap] Scanlines), and a `scanline`
TCP command for live A/B. Launcher Display-card wiring follows separately.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three changes, one fix and two pieces of visibility.
1. ws_ui_group_assign gains a third join: consecutive in the ordering-table
walk AND touching. WipEout 3's speed/shield readout ends at x=288 and the
meter bar it belongs to starts at x=306 with a different CLUT, so neither
shared columns nor a matching key could join them. The readout stood alone,
its bbox midpoint 256.5 landed in the middle third against a screen centre of
254, and it stayed pinned near centre while its own meters went to the right
edge -- a 2 px gap in 4:3 opening to 159 framebuffer pixels at 32:9.
No anchor threshold can fix that: the genuinely centred "pit ... check"
message sits FARTHER RIGHT (midpoint 300.5) than the mis-anchored readout, so
any threshold reaching the readout drags the message with it. Submission order
is the only signal that separates them, and the array already carries it.
Simulated against seven captures spanning three aspects and two code states,
this forms three unions and changes exactly one anchor: [225,288] centre
becomes [225,486] right. The other two unions ([208,240]+[240,361],
[364,447]+[450,473]) are within an anchor. Confirmed in-race at 32:9: 7 runs,
lap readout left, speed readout and meters together on the right.
2. ws_ui_groups reports why primitives were NOT admitted to the partition.
Each gate in ws_ui_prepass_add is a silent return, so a primitive left at its
raw 4:3 X while its neighbours are squashed -- which is how a HUD mark ends up
stranded mid-screen -- was indistinguishable from one never drawn. In-race at
32:9: opcode 367, not_axis 410, everything else 0.
3. geom_correction reports perspective-UV arming with a real denominator.
perspective_triangles could previously only be compared against gp0_draw,
which counts untextured primitives that are correctly never armed, so it read
as a coverage figure without being one. texcorr.attempts counts exactly the
textured triangles reaching the predicate. In-race at 32:9: 199599 submitted,
96.15% armed, 3.85% rejected for a missing per-vertex Z, 0% for correction
disabled or a missing packet address.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit 572b9210d8)
auto_ui_squash squashes each spatial run of UI primitives about that run's own
anchor, so a HUD element split across two runs gets two anchors and comes apart
as the frame widens -- elements drifting toward opposite edges, glyphs sliding
off the background box they sit on.
Diagnosing that required knowing which run each primitive landed in, and nothing
exposed it. The two things an observer could reach were both insufficient:
group.key is a hash of CLUT/texpage/Y-band/family, so unequal keys say two prims
differ without saying WHICH component differed; and the anchor takes only three
values (left/centre/right thirds), so equal anchors do not prove two prims
actually co-grouped rather than coincidentally landing on the same third.
ws_ui_groups dumps, for the last prepass, each primitive's op / key / raw key
inputs (y, h, and the derived 24px band and poly-vs-rect family) / union-find
root / final anchor, plus the frame-level active, squash, dense, rank, disp_x,
disp_w and join_gap. Root plus raw inputs together answer "did these two merge,
and if not, what split them".
Carries two small support changes:
- WsUiGroupItem gains a diagnostic `root`, written by ws_ui_group_assign in all
three paths (dense, alloc-failure fallback, and the union-find path). Never
read by the transform path.
- WsUiPrepassItem retains y/h/op, which the key otherwise folds away.
Read-only and query-driven, per the ring-buffer rule: capture is continuous and
the observer asks for a window later.
(cherry picked from commit 6de1ad4a05)
screenshot_hires passed the pixel width as out_pitch, but the resolves
(sw_render_display / sw_render_display_hires) step rows in BYTES, as the
present path does (main.cpp: sw * sizeof(uint32_t)). Each row advanced a
quarter of a row, tiling the frame 4x across and leaving the bottom three
quarters black. Verified on WipEout 3: content rows 65/256 -> 256/256.
The same handler could also emit untouched allocation: gr_scale() reports
the GL backend's internal scale, but sw_render_display_hires falls back to
the native resolve when the CPU hi-res mirror does not exist, filling w*h
of an ow*oh buffer while still returning non-zero. The buffer is now
calloc'd and the resolve must report FULL cover or the capture redoes
itself honestly at native size.
Add present_shot / present_shot_seq: a capture of the COMPOSED present
surface. Every existing capture resolves the display buffer BEFORE the
backend fits it to the window, so on a 508x256 display in a 4:3 window
they answer 508x256 while the player sees 640x480. That is correct for
faithfulness work but wrong for anything aspect-shaped -- a widescreen
change moves the GTE squash and the present fit, which is exactly the
stage those captures skip.
Fulfilled by whichever backend owns the present: SDL_RenderReadPixels on
the software path, glReadPixels before SwapWindow on GL (with the
bottom-up flip). Notes on the contract:
- GL fulfilment can run on the frame-interpolation thread (interp_present
-> gl_swap_with_osd), not just the emu thread, so the path buffer and
pending flag are mutex-guarded and the counters are atomic.
present_shot_take() claims the request under the lock, so two present
paths cannot fulfil the same shot.
- A staged shot is OVERWRITTEN by a later request rather than refused,
as savestate's request_save_inner does. A present that never arrives
therefore cannot wedge the command.
- Vulkan presents through its own swapchain and has no readback hook, so
the request is refused there instead of being accepted and never
fulfilled (CLAUDE.md rule 15).
- present_shot_seq advances on success AND failure so a polling client
always terminates; its `wrote` field reports whether a PNG landed.
- The SDL2 branch sizes its buffer from SDL_GetRendererOutputSize rather
than a viewport*scale reconstruction, which could be smaller than the
region SDL_RenderReadPixels fills.
Also: debug_client gains positional-path screenshot_file/present_shot and
turbo/turbo_state mappings (they previously fell through to the raw
key=value handler); TCP_COMMANDS.md marks screenshot_hires as native (it
is registered on the native server and the blank N column read as
Beetle-only), documents the two new commands, and the generated index is
regenerated via tools/gen_tcp_commands.py.
Tested on WipEout 3 SE (PAL, OpenGL, RelWithDebInfo + PSX_DEBUG_TOOLS=ON):
capture matches the on-screen window at 1280x960; seq/wrote reports 1 on
success and 0 for an unwritable path; back-to-back staging is accepted.
The SDL software path, the SDL2 branches and Vulkan refusal are
compile-verified only.
(cherry picked from commit 8fee64f178)
Legend of Mana's land-creation effect rendered as hard-edged coloured
wedges instead of a soft bloom. The cause was in the CD-ROM controller,
not the renderer.
The game streams a file by issuing Setloc(next)+ReadN roughly 10,800
guest cycles after each sector arrives. Every ReadN called
start_read_stream(), which clears the sector buffer -- destroying the
sector that had already been delivered and announced but not yet drained
by the guest. Measured at the palette load:
buffer psx-runtime DuckStation
0x0E1F18 125111 125111
0x0E2718 125113 125112 <- the effect's colour palette
0x0DCF18 125113 125113
Sector 125112 carries the effect's 5-colour palette (0x0888F8 =
(248,136,8), 0xB0F8F8 = (248,248,176), verified against the disc image).
Destroyed before the guest could take it, its buffer received 125113's
raw file bytes instead, and the colour routine faithfully scaled those
into 216 distinct vertex colours -- the wedges. The table now holds 5.
DuckStation only calls BeginReading -- the function that clears the
buffers and re-arms the read-start latency -- when the request actually
moves the drive:
if ((!setloc_pending || setloc_position.ToLBA() == GetNextSectorToBeRead())
&& (drive_state == Reading || ...))
"Ignoring read command with pending/same setloc, already reading"
else
BeginReading();
A Read issued while the drive already streams the sector the pending
Setloc names is the game saying "keep going", not "start over". The
command is still ACKed with the same INT3; only the restart is skipped.
Two things fall out of it. The un-drained sector survives, so the game
receives the consecutive sectors it expects (109,110,111,112,113 rather
than 109,111,111,113,113). And each request stops paying
initial_read_delay_cycles() again: inter-sector gaps drop from ~456,000
cycles to 225,792 -- exactly the 2x cadence -- so this load takes 2 guest
frames instead of 13, matching DuckStation. int1_pended goes 10 -> 0.
Also included, developed alongside and verified in the same runs:
- An 8-slot sector buffer ring, as the hardware has (nocash;
DuckStation NUM_SECTOR_BUFFERS = 8). A delivered sector fills the
NEXT slot and the INT1 that announces it latches the read slot, so a
sector arriving mid-drain no longer clobbers the FIFO. The read
pointer follows the slot each INT1 announces, captured when the INT1
is RAISED: this drive free-runs, so latching "newest" at
presentation time handed the guest a sector seven ahead of the one
it asked for.
- The seamless no-IRQ refill advances the read pointer, because it is
a continuation of the drain already in progress. Without it a
multi-sector DMA starves mid-transfer (measured: 1,128,960 starved
drains).
- A pended data-ready is released on a short schedule rather than
inside the guest's ack store, matching DuckStation's
QueueDeliverAsyncInterrupt.
- Flow control for accelerated (non-1x) reads: never overwrite a
sector the guest has not consumed while running faster than
hardware, capped at the authentic sector period. Inert at authentic
timing.
New cdrom_state counters ring_starved and ring_dropped, plus
cdrom_timing_dump for per-sector deadline/delivery records, are the
regression tripwires: ring_starved and int1_lost must stay 0.
Verified on Legend of Mana (SLUS-01013): the effect renders correctly,
the palette table holds 5 colour words, sectors arrive at the 2x
cadence, and no notification is lost.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three related defects in the interp observability rings. All three share a
failure mode that is worse than an error: the tool reports an empty result
that is indistinguishable from a real finding. "total: 0" reads as proof the
events never happened.
1. callret_watch armed on `lo != 0`, making address 0 an off switch rather
than a legal window floor. `lo=0 hi=0x200000` — the obvious spelling of
"watch the whole address space" — replied ok and recorded nothing.
Measured on one build, identical but for the floor:
lo=0x00000000 hi=0x00200000 -> total 0
lo=0x00000004 hi=0x00200000 -> total 468732
Four bytes of window between an empty ring and half a million call
resolutions. This cost real time during beads-eio.3.59: an
interp->compiled call route was briefly believed absent when the ring was
simply never on.
Arm on a NON-EMPTY window (hi > lo) instead. `lo=0` is now an ordinary
floor. Note that `{"lo":"0"}` was the DOCUMENTED disarm spelling
(TCP_COMMANDS.md), so that contract is kept explicitly rather than broken:
lo=0 with no hi still disarms, and now says so in the reply. Added
`disarm=true` as the unambiguous spelling. A `lo` with no `hi` is refused
outright instead of silently inheriting the previous ceiling (usually 0,
i.e. an empty window) — that was the second way to arm nothing by accident.
2. s3_smear_watch had the same `lo`-nonzero gate, so lo=0 with a real hi
recorded nothing while answering ok. Same fix. It already cleared `hi`
whenever `hi` was omitted, so `{"lo":"0"}` naturally lands on an empty
window and still disarms — no special case needed there.
Both commands now report `armed`, and callret_watch's dump carries `armed`
plus the active window, so an empty read can always be told apart from an
unarmed one.
3. The xprobe JAL/JALR watched-target list was six guest addresses from two
specific titles COMPILED INTO the shared runtime (an MMX6 card-load flow
and a Tomba target). Every other game therefore watched nothing, silently:
the `watched` dump came back empty and gave no hint that the build was
incapable of watching anything. Same defect class as beads-eio.3.21, and
the direct reason the JAL call site was unusable while measuring
beads-eio.3.59 — only the JALR callret ring could be used there.
The list is now configured, never compiled in: `xprobe_watch` TCP command
to get/set, or PSX_XPROBE_WATCH in the environment so targets are watched
from instruction zero without a rebuild. Default EMPTY — no title is
privileged — and a read with nothing watched says so rather than looking
like a negative result. Matched on the normalised address so KSEG and
physical forms both work.
Verified against Ape Escape, headless, RelWithDebInfo:
- PSX_XPROBE_WATCH=0x8002C0E4 -> watching that target from boot
- xprobe_watch targets=<two addrs> -> count 2; the `watched` dump then
carries 41 JALR records for 0x8002C0E4 and 21 for 0x8002C14C, with full
frame/cycle/src/sp/ra context. This title could previously watch NOTHING.
- xprobe_watch targets= (empty) -> count 0 plus an explanatory note
- callret_watch lo=0 hi=0x200000 -> armed true, total 77484
- callret_watch lo=0 -> disarmed, legacy spelling named
- callret_watch lo=0x40000 -> refused, window left unchanged
- callret_watch disarm=1 -> disarmed
- s3_smear_watch lo=0 hi=0x200000 -> armed true (previously silent)
- pre-fix build, callret lo=0 hi=0x200000 -> plain ok, total 0 (the trap)
TCP_COMMANDS.md regenerated for the new command. Note the regen also picks
up rows that had drifted on master beforehand: `--check` already failed on
pristine master (index said 292, the servers register 304), which is why the
count line jumps 292 -> 305 rather than by one.
A latched overlay_autocapture_tick() gate is invisible from outside:
triggers, futility and pressure counters simply stop moving, which
reads as "nothing to do" while in reality no new overlay variant will
ever be captured again and every freshly loaded RAM image stays
interpreted for the rest of the session. Observed live: a full disk
made the capture manifest write fail and requeue forever, and the only
outward symptom was the game getting slower with every new area.
Record which gate the tick last returned at and when the last pressure
sample completed, and surface both through autocompile_status as an
additive "autocapture" object: wedged (enabled yet no sample for 30 s),
gate, ms_since_sample, and the raw write-job/provider state. No
behavior change — observability only.
Emitter side of the value-propagation engine (ENHANCEMENTS.md G1.10):
- recompiler/src/pgxp_hook_emitter.{h,cpp} (new, in the codegen-hash
surface): ONE hook grammar shared by BOTH emitters, wrapping loads/
stores/COP2 transfers and precision-carrying ALU ops (moves, add/sub,
or-merge, shifts, LUI, HI/LO, muldiv) with PGXP_*() macros. Clobberable
sources are captured into locals before the statement. Precision-
destroying ops (AND/XOR/SLT/...) are deliberately unhooked - validate-
on-read drops their stale shadows for free.
- code_generator.cpp: central append in translate_instruction; LWC2/SWC2
emissions capture the RAW loaded/stored word (a masked GTE register
write must validate against the word as loaded).
- strict_translator.cpp: translate() wraps translate_impl()'s c_code with
the shared grammar; deferred load-delay variants hook the psx_ldd temp.
- BIOS regenerated, fingerprints verified CHANGED (SCPH1001 + OpenBIOS;
42.6k/12.2k macro sites). Base runtime + 33/33 tests green against it.
- Gating: macros expand only under -DPSX_PGXP=1. runtime.cmake grows a
PGXP option on psxrecomp_add_runtime_target (defines PSX_PGXP=1 +
PSX_OVERLAY_FLAVOR=2, _pgxp exe suffix) and PSX_PGXP_VARIANT to build
the sibling target from the same generated tree (framework runtime +
game wrapper). No ABI version bump needed: the appended PGXPHooks
table is NULL-tolerant and the flavor half of the ABI tag rejects any
host/DLL flavor mix.
- Shard caches: flavor folded into the cache tag (cg<V>_<hash>_gc<hash>
_f<flavor>) in overlay_loader.c + compile_overlays.py so base and pgxp
caches never share a directory; compile_overlays.py passes
-DPSX_PGXP=1 when flavor bit 2 is set; overlay preamble forwards
psx_pgxp_* through the table. PSX_OVERLAY_CODEGEN_VER 9 -> 10.
- Env overrides PSX_GEOMETRY_CORRECTION / PSX_PERSPECTIVE_TEXTURING /
PSX_PGXP_CPU_MODE (validation path: free-running headless boots arm
from the first vertex); census grew produced/swc2_stores and the pgxp
verb reports suppress/active gate state.
Validated: PSXRecomp_pgxp.exe boots the recompiled BIOS with hooks armed
(engine active, per-vertex lookups flowing; the BIOS shell has no RTPS
content so produced=0 there is correct - coverage measurement moves to a
3D title in phase 3).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Closes the four gaps named in issue #103 and lands the separable, verified
parts of PR #102. See docs/internal/SPU_FIDELITY_103.md for the per-piece
disposition and the oracle-verification queue.
SPU DSP (issue #103), clean-room from hardware documentation:
- Reverb: the documented 22050 Hz engine — vLIN/vRIN input, same-side and
crossed different-side IIR reflections, 4-tap comb early echo, both
all-pass filters, vLOUT/vROUT output, and the work-area address advance.
Gated on SPUCNT bit 7 for WRITES only; reads, IRQ checks, address advance
and output always run, as on hardware. Addressing is confined to
[mBASE, 0x80000). Per-voice sends from EON, CD send from SPUCNT bits 0+2.
- Noise: 16-bit LFSR with the documented parity feedback, clocked at the
SPUCNT bits 8-13 rate; NON voices substitute it for their ADPCM sample.
- Volume sweeps: all 24 voice L/R plus main L/R. Bit 15 selects sweep vs
direct; sweep is a live envelope on the same rate machinery as ADSR, and a
guest read of a sweeping register returns the live level.
- Capture buffers: CD L/R and voice 1/3 output written to SPU RAM, so a game
that parks the IRQ address there still gets its interrupt.
- SPU IRQ (I_STAT bit 9): checked at every SPU RAM access class — FIFO, DMA
both directions, ADPCM block fetch, capture writes, reverb work-area
accesses — gated on SPUCNT bit 6, mirrored in SPUSTAT bit 6, acked by
clearing SPUCNT bit 6.
The CD-only fast path in spu_render was removed rather than extended: the
SPU now has per-frame work that must run in every path, and an FMV (XA audio
with zero active voices) is exactly the CD-with-reverb case.
This is a clean-room implementation from the psx-spx/nocash register map and
documented algorithm. Beetle's source was deliberately not consulted: it is
GPL-2.0-or-later and this project is PolyForm Noncommercial. PR #16 was
parked for exactly that reason and PR #13's reverb was separately wrong (it
gated on bit 15 of dAPF1, an address offset, instead of SPUCNT bit 7). The
one place this knowingly deviates from hardware is the 22.05 -> 44.1 kHz
reconstruction filter, which the documentation does not specify; it is
isolated in a single function and flagged for oracle comparison.
CAUSE.IP2 (from PR #102):
IP2 is combinational on R3000A — it mirrors the interrupt line and falls the
instant the guest acks I_STAT or masks the source. It was being OR'd in at
delivery and never cleared, in TWO places (the compiled path and
psx_interpreter.c, the latter missed by PR #102), leaving a phantom pending
interrupt that can spin a kernel dispatcher looping on CAUSE.IP & SR.IM.
psx_irq_refresh_cause_ip2() is now the sole writer of bit 10, refreshed at
raise, I_STAT ack, I_MASK write, HLE context restore and power-on. Verified
against the Beetle oracle's irq.cpp/cpu.cpp rather than asserted, and
covered by test_cause_ip2_combinational.
Mid-dispatch audio pump (from PR #102), with a fix: routed through a gate
that mirrors sdl_audio_update's mute/sink decision. Pumping unconditionally
from the VBlank edge, as submitted, would have pushed real audio during every
turbo-load hard mute and defeated the freeze-in-place mute model.
SPU RAM DMA readback (from PR #102): DMA4 in the SPU->RAM direction wrote
literal zeros; SPU RAM is readable memory and titles carry state through it
across an Exec boundary.
Also fixes an unrelated build race found on the way: tools/embed_spirv.py
wrote its intermediate SPIR-V to <source>.spv, so the two runtime targets
that embed the same shaders raced on one path and one deleted the file the
other was reading.
Co-Authored-By: Alexandros Mandravillis <Alexbeav@live.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
screenshot/screenshot_file read native 15-bit VRAM via gpu_display_pixel_rgb.
That is correct for faithfulness work but BLIND to any enhancement that only
exists in the supersampled mirror: at supersampling >= 2 the geometry-correction
sub-pixel vertices, SSAA edges and perspective UVs are resolved in the hi-res
surface and gone once pixels are packed back to native.
This bit hard. Native captures of Ape Escape looked clean while the user was
looking at a visibly cracked frame, so a real regression got reported as fixed
and the argument moved to counters. An instrument that cannot show the defect
is worse than no instrument.
screenshot_hires routes gr_render_display_hires — the same present path the
window uses — and falls back to the native resolve (reporting scale 1) when
there is no hi-res surface, so it always answers truthfully about real output.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The geometry side cache was an 8192-entry HASHED table with no ambiguity check.
Unrelated screen positions shared a slot, so a vertex could be handed a
different vertex's sub-pixel fraction, and eviction changed which triangles got
corrected from frame to frame — moving seams across every mesh.
Both references index this table exactly, one slot per reachable SXY
(beetle-psx vertexCache[0x800*2][0x800*2]; DuckStation 2048x2048), and gate it
on an ambiguity flag so a pixel that two DIFFERENT projections rounded to is
never used. Do the same: direct index, first writer wins, flag on a conflicting
value, allocate lazily on enable so the faithful default costs nothing, and
fail closed to the faithful path if the allocation fails.
Adds a lookup census (attempted / hit / miss_unrecorded / miss_ambiguous) on
geom_correction. With an exact table a miss_unrecorded is a genuine tracking
coverage gap rather than a cache artifact, which is what distinguishes 'the
cache was lossy' from 'the vertex path never reaches us in a matchable form'.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both [video] enhancements are silent no-ops on anything they cannot prove is
projected geometry: a vertex whose sub-pixel fraction was never cached, or a
packet whose position words lack full GTE projection provenance, just draws the
faithful way. So the enable flag alone says nothing about whether a given title
benefits, and the existing hit counters had no reader.
geom_correction reports the flag plus the free-running geometry_vertex_hits and
perspective_triangles totals, so per-title bring-up can tell an engaged
correction from an inert one before anyone argues about screenshots. Sample
twice and diff for a per-window rate.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Extract the useful diagnostic from PR #13 into an opt-in, title-agnostic boot
write-range trace with TCP-queryable context.
Co-authored-by: NyperYuhgard <93950153+NyperYuhgard@users.noreply.github.com>
Two observability gaps hit while debugging the memory-card corruption
above; both are serializer-side, the data was already recorded.
crash_trace: the report carried cpu->pc but not the dirty-RAM
interpreter's unsupported-instruction block. A run that died there
showed only pc == 0, and ten separate sites set that, so the cause was
unattributable. Emit interp_unsupported{pc,insn,reason,block_entry,
entry_ra,entry_sp,insns_into_block,midblock_count}.
dirty_ram_interp log: the per-instruction ring captured v0/v1/a0-a3/
t0-t2/sp/ra but not $at or $k0/$k1 -- precisely the registers that
matter for BIOS exception-handler code, which is most of what this
interpreter runs, and the ones the cardfasttrack.s load-delay idiom
turns on.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds PCSX-Redux's MIT-licensed OpenBIOS as a second BIOS the framework can
statically recompile, next to retail SCPH1001. Because OpenBIOS is
redistributable, a game verified against it ships its BIOS with the build —
players supply only a disc, and the launcher BIOS picker disappears.
The mechanism is general, not OpenBIOS-specific: every BIOS image gets a
bios/<STEM>.toml profile that is the single source of truth for its identity
and layout.
Recompiler
- bios/<STEM>.toml profiles: [program.image] (sha256 pin, redistributable
flag), [recompiler.address_model] (boot-time ROM->RAM copy windows,
kernel-bless window), install_slots, and [recompiler.runtime_exports]
HLE anchors (0 = feature structurally unavailable on that BIOS).
- BiosAddressModel (new): validated model built from the profile, replacing
~9 hardcoded SCPH1001 window constants that had drifted apart across the
emitter / discovery / game codegen. The emitted C normalize() derives
from the same table. from_config rejects overlapping/misaligned windows
and orderings where sequential folding != first-match. A CTest-registered
equivalence test sweeps every RAM/ROM address against verbatim copies of
the old helpers.
- Orphaned delay slots (the OpenBIOS-critical emitter fixes): jr/jalr
slots are inlined before the jump resolves — previously their side
effect was silently LOST (e.g. a case-return fragment's addiu $sp pop,
which unwound a shifted frame and walked the exception handler's IntRP
cursor into a wild dispatch). Orphaned branches resume at ds+4 on
not-taken instead of re-executing the slot (a read-modify-write slot
doubling twice broke OpenBIOS readPad). Jump targets latch pre-slot.
- MIPS-I load-delay modeling for dependent load pairs: the successor sees
the pre-load value (OpenBIOS's cardfasttrack.s relies on it on purpose —
"gotta break those bad emulators"). Unmodeled shapes warn at build time.
- Discovery hardening: branch-target dispatchability closure to fixpoint;
jump tables placed at jr+8 no longer walked as code; cross-function
continuations routed by an instruction->owner map.
- Signed MULT support (unreachable from the retail seed set; modern-GCC
OpenBIOS emits it freely), latency-modeled like MULTU.
- Seeds from ELF symbol tables + data-section code-pointer harvest
(tools/gen_bios_seeds_from_elf.py) — a source-built BIOS needs no
disassembly pass. 646 OpenBIOS seeds, incl. 5 empirical dispatch-miss
seeds.
- Stemmed outputs (<out_stem>_full.c / _dispatch.c) so two BIOSes coexist
in one out_dir. tools/regen_bios.sh is profile-driven (--config).
Runtime
- psx_bios_image: the generated dispatch now self-describes the BIOS it
was compiled from (identity: size/CRC32/SHA-256/wordsum; kernel-bless
window; HLE anchors; bundled flag). Runtime SCPH1001 hardcodes removed.
- Launch check is a hard identity gate (a mismatched image under
statically recompiled code is a guaranteed wild jump). Bundled builds
hide the BIOS picker and treat a missing/tampered bundled image as an
installation defect.
- Faithfulness fixes surfaced by OpenBIOS's tighter timing loops (all
global, argued in code comments, regression-tested against SCPH1001):
- SIO pad RX is cycle-paced like the memcard path (RX_RDY at
shift-complete, ACK +170); total ack-IRQ latency unchanged.
PSX_SIO_PAD_SYNC_RX=1 restores the legacy synchronous path for A/B.
- SPUSTAT derived from SPUCNT instead of permanently-asserted busy.
- GPUSTAT.13 (interlace field) alternates per vblank while interlaced,
pinned 0 in progressive.
- CD Init(0x0A) completes in ~3.9 ms instead of borrowing Beetle's
34 ms Reset figure (OpenBIOS's short bounded retry loop livelocked).
- Exception EPC guard accepts ROM-window block leaders, not just RAM.
- New TCP debug commands (documented in TCP_COMMANDS.md): bios_info,
s3_smear_watch, callret_watch; write-trace ring gains $s0-$s5.
bios/openbios.bin is vendored (MIT — notice in bios/OpenBIOS.LICENSE,
attribution in THIRD_PARTY_ATTRIBUTION.md, source pin + build recipe in
bios/OpenBIOS.toml). The emitter fingerprint digest is now content-only:
existing generated/*.emitter.sha will read STALE once; re-run
tools/regen_bios.sh.
Matthew Stanley