Default = largest multiple of 240 lines within 2/3 of the usable height;
an explicit window_width is still honoured.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Overlay bands are shared address windows, so every observed interpreted PC
is demanded for every occupant of the band. classify_overlay_seeds promotes
such a demand to a linear walk root whenever _callable_legacy_seed accepts
it, and that predicate accepts three unequal grades of evidence: the image
entry, an `addiu sp, sp, -imm` prologue, or a `jr $ra` two words back.
The third grade fires exactly where it is least trustworthy. The first word
after a function's return is also the first word of whatever the linker laid
down next, and in an overlay image that is routinely a pointer table, a
packed record array or zero fill. The walk then starts at the table head,
never reaches a return, and runs to the image end; the generated-C audit
reports a column of UNSUPPORTED_INSTRUCTION and the shard fails.
Require the bounded CFG probe the discovery roots already use
(plausible_callable_target: reachable return, no fallthrough past the probe)
before a prologue-less dispatch entry may be a walk root. A failure demotes
it to DISPATCH_INTERIOR, which keeps the dispatch evidence and the isolated
fragment demand and only declines to start a walk there; the image entry and
the prologue keep their own evidence and are not probed.
Measured on Breath of Fire III (SLPS-00990), 406 overlay captures:
dispatch entries already accepted as walk roots 5,339
with a prologue 2,066 unchanged
on the preceding `jr $ra` alone 3,273
rejected by the probe 41
DISPATCH_ENTRY roots 1,141 -> 1,100 (-41)
DISPATCH_INTERIOR 81,474 -> 80,851 (-623, interiors of the
fabricated walks)
STATIC_DISCOVERY_ROOT 10,741 -> 10,741
STATIC_DISPATCH_ENTRY 194 -> 194
FUNCTION_POINTER_TARGET 20 -> 20
isolated fragment demands 105,891 -> 105,891 (nothing lost)
41 of 406 captures change at all. All 41 rejected addresses are data by
inspection: in-image code-pointer tables, packed 16-bit record arrays,
signed delta pairs, zero fill. 27 of them are precisely the 27 shards whose
all-bands compile failed the audit -- one bad root per failing shard. The
other 14 are the same defect in shards whose data happened to decode all the
way to the image end without meeting an unsupported opcode.
New CTest: frameless_dispatch_root_cfg_proof.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The bounded-jump-table resolver requires the guest's own bounds check
(`sltiu/beq`) and index scaling (`sll`), because those are what prove the
table's extent. A decoder that keeps its state as a pre-scaled byte offset
and dispatches through `lui/ori base; addu base,offset; lw; jr` has neither,
so the jr fell back to a bare CPS tail-transfer and every entry it reached
that was not already a dispatch key ran on the dirty-RAM interpreter. On
Breath of Fire III that is the file decoder's state dispatch at
`0x80164AB0` (five entries; the first entry is the word right after the
table), the last boot-EXE row a play session still interpreted after the
computed-stride resolver.
resolve_self_limited_jump_table recovers the extent from the table's own
layout instead: words are taken from the base while each is a 4-aligned
in-function code address outside every delay slot, and the table ends
where its lowest target begins, since a pointer table cannot overlap the
code it points at. As with the stride resolver the emitter switches on
the actual runtime target with the CPS tail-transfer as default, so the
recovered entries only add native cases; an index past the recovered
extent still takes the fallback. Wired as the third fallback at both
code-generator sites and in the exact-entry discovery walker.
On the title's boot EXE it matches exactly one site (five entries); every
other shard is byte-identical. The synthetic test gains a second function
with this shape.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A `jr R` with `R = base + index * stride` and no table in memory is how
unrolled copy loops enter their body in the middle (Duff's device). The
bounded-jump-table resolver only knows the `sltiu/beq; sll; addu; lw; jr`
chain, so the emitter fell back to a bare CPS tail-transfer; the trampoline
then found no dispatch key at the interior PC (straight-line code, no block
leader) and the dirty-RAM interpreter ran the rest of the copy on every
call. On Breath of Fire III that is the file decompressor's literal-copy
loop (`0x80164D1C`, 32 groups of `lbu/nop/sb`), which fires on every load
and was the whole non-kernel interpreted residual of a play session once
the kernel patch ranges (#346) and the call vectors (#381) ran native.
resolve_computed_stride_jump walks the chain back from the jr through
nearest definitions in straight-line code: `addu R, B, S` with S from
`sll S, I, k` or the two-sll sum `sll P,I,a; sll Q,I,b; addu S,P,Q`, and
then requires the code after the delay slot to be N >= 2 shape-identical
groups of `stride` bytes (same opcodes and register fields, immediates
free, no control flow inside a group). Targets are run_start + k * stride
for k = 0..N. The base register's value is never assumed: the emitter
switches on the actual runtime target with the CPS tail-transfer as the
default, so the recovered cases can only add native entries, never
redirect a jump that lands elsewhere. Wired into both code-generator
sites (interior labels + the switch) and the exact-entry discovery walker.
On the title's boot EXE it matches exactly one site, the decompressor loop
(33 entries); every other jr is untouched.
Adds recompiler/tests/test_computed_stride_jump_codegen.py (synthetic
PS-EXE with the idiom; asserts the switch, every case and the interior
labels), registered with CTest.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
test_compile_overlays_static_split.test_data_only_capture_reports_skip_with_captured_log
has been red on master since 4683e923 (2026-09-12). That commit reworded both
emit sites deliberately -- "no walk-root seeds (data-only region)" became
"no shared walk-root seeds; checking fragments separately", because the region
is no longer declared data-only: its executed dispatch demands are still
served by the isolated-fragment pass. The assertion still names the old text,
so the test fails on a clean checkout:
AssertionError: 'SKIP: no walk-root seeds' not found in '... SKIP: no
shared walk-root seeds; checking fragments separately ...'
The literal was duplicated at the two emit sites and copied a third time into
the test, so rewording two of the three left the third stale with nothing to
catch it. Give it a name -- NO_SHARED_WALK_ROOT_SEEDS_SKIP -- use it at both
emit sites, and have the test assert the constant rather than a copy of its
text. A future reword then updates all three at once or fails loudly.
The printed output is unchanged, verified byte for byte against the pre-change
text including both leading spaces and the trailing blank line.
tools/test_compile_overlays_static_split.py: 10 tests, was 9 passed 1 failed,
now 10 passed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The generated psx_bios_try_native_call_stub reads the four live words at
the A0/B0/C0 call vectors and, when they match the expected stub, decodes
the handler address and tail-transfers to it without the dirty-RAM
interpreter. The guard knew one shape, the retail kernel's
lui $t0,hi; addiu $t0,$t0,lo; jr $t0; nop
OpenBIOS installs
addiu $t0,$zero,lo; jr $t0; nop; nop
so on OpenBIOS the guard failed closed and every kernel call paid an
interpreter entry for two instructions. Breath of Fire III yields through
B0:10h ChangeTh ~230 times per frame in the field: ~460 interpreted
instructions per frame, 87% of what OpenBIOS kernel RAM still interpreted
after the install-slot ranges (#346), and the same A/B reads 0 on retail.
Accept both shapes. The target is still decoded from live RAM on every
call and any other word pattern still fails closed (Rule 18). Cycle steps
charge the instructions the guest executes on each shape: four for the
retail stub, three for OpenBIOS (its jr's delay slot is word 2). Dispatch
totals are unchanged (OpenBIOS 3990, SCPH1001 13012).
Adds recompiler/tests/test_bios_vector_stub_shapes.py (registered with
CTest next to bios_backend_constant_codegen).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
cli_fail_msg replaced whatever the CLI said on exit 3 with a flat
"Disc verification failed (wrong dump)." With the CHD reader, the CLI's
own line can be "... is a CHD and the CHD reader (libchdr) is not built
for this kit ... this does not mean your dump is bad", and the wizard then
contradicted it. cli_tail_note already captures that line; show it, and
fall back to the old text only when the CLI said nothing.
Carried over from #369, which was closed in favour of this change.
TOOLCHAIN_PINS had entries for win and linux only, and stage_toolchain
picked 'linux' for every non-Windows host, so a Mac either died with "no
toolchain pins" or staged a Linux interpreter it cannot run. No Mac player
ever got a bundled toolchain. The stock /usr/bin/python3 on macOS 13/14 is
3.9, which cannot run compile_overlays.py (tomllib, PEP 604 unions), so the
system interpreter is no fallback either.
Add macos-x64 and macos-arm64 pins: the same python-build-standalone release
and CPython 3.13.1 as the Linux pin, install_only_stripped assets, SHA256
measured by downloading both and matched against the publisher's .sha256
sidecars. host_platform_tag() picks the key from sys.platform and the
machine architecture. No prebuilt tcc on macOS: the bundled interpreter
drives the system clang.
Verified on an Intel Mac (macOS 13, 8 GB): toolchain staged beside the
Parasite Eve diagnostic product (45 MB), and with the POSIX spawner the
runtime compiled 9 shards in the first 24 s of a headless boot.
(cherry picked from commit 5c49e4d3cf835ed026964cd24998907c41a8be4c)
During a boot FMV the guest parks in BIOS MDEC code with the dirty-RAM counters
legitimately still while frames advance, which is exactly the spin signature.
Every boot FMV wrote a 47-76 MB dump, 1.8 GB per kit in one session, inside
timed windows. MDEC activity within the window now rules the classification
out. Verified on NCII: heartbeat live through the FMV, zero dumps.
(cherry picked from commit bf60c128218949e3e3af7ea867d50cc7a1e8a774)
dirty_ram_text_guard_resync_after_restore() cleared text_modified_bitmap,
and the page-clean fast path in dirty_ram_text_native_ok_ranges_from()
then trusted a clear bit as "bytes still equal the reference" and skipped
the compare. Restored RAM may carry self-modified text (the CMR2 class)
that no live write will ever re-flag: the bitmap is not part of the state
image, and the writes happened in the process that produced it. After a
save-state load into a fresh process, a stale native body could be
admitted for those pages.
Recompute the bitmap from the bytes: one pass over the reference range (at
most 2 MB, once per restore), setting the bit for every page that differs,
so those pages stay under the exact compare.
(cherry picked from commit ab186edf6, memory.c part)
78 of 105 published releases lack the key, and the runtime never initialises
the overlay loader without it. The packager now refuses a staged game.toml
without it unless --ship-without-overlay-cache-because states why, and the
setup host's self-check reports overlay_cache_configured so CI can assert it.
(cherry picked from commit a030f4843d8e55eafed4cb42eafc6d74bb89c086)
No setup package ever staged one, so overlay_cache = true alone left every
streamed overlay on the interpreter. rebuild now stages the pinned Python,
tcc, this pin's recompiler, headers and the build's codegen hash beside the
normal and diagnostic products, and writes compiler.txt naming the compiler
the wizard already has; the runtime treats that as a gcc-tier compiler, so a
player with nothing on PATH gets -O2 shards. Verified on the NCII kit: gcc
tier reported and four shards built with no compiler on PATH.
(cherry picked from commit 03865e10f6ee198d58c47c74526df5db46bb35a9)
autocompile_request() returned 0 on every non-Windows host, so a Linux or
macOS player never had a shard compiled at runtime: captures were written,
the packager staged a working toolchain, the boot log said "overlay
autocompile enabled", and every overlay ran interpreted for the whole
session (Parasite Eve on Rocky 9: 1.35e9 interpreted instructions by frame
13k with /usr/bin/gcc beside the exe).
Mirror the Windows spawner with fork/exec: /bin/sh -c runs the configured
command in its own process group at nice 10, stdout+stderr flow through one
pipe into the same tail ring (streaming PSX_SHARD_RESULT parse), a pthread
watcher drains the pipe and reaps the child, and the emulation thread's poll
turns DONE into the one idempotent cache rescan. No PSX_SHARD_PUBLISHED
handoff off Windows: the rescan is the fallback the Windows path already
takes when a marker is cut off. Shutdown SIGTERMs the group and joins;
PR_SET_PDEATHSIG on Linux plus the dying pipe cover a runtime that never
reaches shutdown. The PSX_OVERLAY_* pins are set once at configure time
(glibc setenv is not safe against a concurrent getenv). Configure names the
interpreter it will run and marks the run degraded when the path is not
executable. Link Threads unconditionally on Unix.
The child-line buffer, shard_result_values, shard_result_line_locked and
publish_parse_locked move out of the Windows block; child_line_locked is
per-platform. Both Windows unit tests still pass.
Verified on Rocky 9 (PE diagnostic product): 0 -> 68 shards in 150 s,
interpreted instructions by frame 14.5k 1.35e9 -> 0.39e9, curve flat from
frame ~4k, counters visible over the debug server, quit and SIGKILL
mid-compile leave no orphans or zombies.
(cherry picked from commit 854540c9c96e86b0cc1eea0dd99ef285169784d9)
s_frame_count is already maintained under PSX_NO_DEBUG_TOOLS, but everything
that exposes it — the debug server, the freeze heartbeat, screenshots — is
compiled out of production. A shipped binary could not report its own speed, so
throughput could only be measured on the diagnostic product, which carries
instrumentation of its own and is not what players run.
That gap is why a 30% throughput regression reached a release candidate
unmeasured, and why the first four attempts to attribute it were all made
against the wrong binary.
Adds an opt-in periodic line on stdout, outside every debug guard:
PSX_FRAME_REPORT_MS=2000
psxrecomp: frames=1234 elapsed_ms=20000 fps=61.7 avg_fps=61.7
Unset, it is one branch on a cached int per vblank. Line-oriented on stdout so a
harness can read it without a debug connection.
(cherry picked from commit aac4dbf58331a413b4ac8be6890c7ff499c55ca0)
(cherry picked from commit c1fbb313d15fe00256bc68a805ba0df255feb249)
launcher_warning/launcher_info only ever raised a dialog under _WIN32, so on
macOS and Linux a first run with no BIOS printed to a terminal nobody was
looking at and quit. SDL3 is already linked; SDL_ShowSimpleMessageBox covers
all three platforms and keeps the headless guard.
(cherry picked from commit 22f4322eb33d46946f9b953f943bedf070eb75a6)
s_cmd was 4096 bytes; a 208-address --force-interior list produced a 6392-char
command, snprintf cut it mid-argument, every compile failed the same way and
autocompile latched degraded for the process lifetime. Buffer is now 8192
(cmd.exe's own ceiling), the cmd.exe wrapper is sized from it, and an over-long
command is reported and disabled loudly.
(cherry picked from commit b0fd04076ca4b0cc7f8d813bc6f750f94df30bf9)
The runtime mounts a .chd natively and the launcher lists it in the disc
picker, so a player could select one and see a valid disc identity, then have
first-run setup fail: verify-disc, prepare_disc.py and probe_disc.py hash and
parse raw 2352-byte track data, which a CHD stores compressed. Nothing on the
Python side could read it, and telling players to install MAME for chdman is
not an answer.
libchdr is already vendored and compiled for the runtime and the native media
CLI. Build it once more as a shared library in recompiler/ (CMake target chdr,
output next to the emitters, static CRT on Windows so a plain python.exe can
load it) and read the CHD from Python through ctypes. tools/psx_chd.py mirrors
the layout the runtime's iso_reader.cpp uses -- 2448-byte frames, 4-frame
track padding, stored versus virtual pregaps, big-endian audio -- and
reproduces the .bin files a Redump dump of the same disc contains, in either
of Redump's two layouts (one bin per track, or one bin for the disc).
verify-disc hashes those reconstructed track bytes against [prepare_disc]
instead of the container. prepare_disc.py writes the tracks into out_dir in
whichever layout the recorded digests match and then continues down its
normal cue path, so the data track is written once, SBI companions are looked
up next to the CHD, and the receipt names the CHD the player chose.
probe_disc.py extracts to a scratch directory and probes the cue. The CLI
builds target chdr on demand for a kit whose emitters predate it, and falls
back to the extract-it-first message when it cannot.
Verified against two discs from a Redump library converted with chdman: a
single-track MODE2 disc and a four-track disc with stored audio pregaps both
reproduce their catalogued track-1 size, MD5 and SHA-1 exactly; verify-disc
and prepare_disc.py accept the four-track one end to end. Tests build a
synthetic uncompressed v5 CHD around a cooked ISO image, so no retail data is
needed; the decode tests skip with a reason when no libchdr is built.
3a174fab added reconcile_empty_primary_scans() at the tail of main() and
called it unconditionally with cache_dir. cache_dir is bound only on the
DLL path; --static never assigns it. So every --static run now ends in
UnboundLocalError: cannot access local variable 'cache_dir' where it
is not associated with a value
after generated/overlays_static.c and all its translation units have
already been written. The output is complete, but the process exits 1,
and callers that gate on the compiler's exit (axis_b_loop.sh expects 0
or 2) abort before the runtime is rebuilt. Found regenerating Breath of
Fire III against master 193a60b8.
The reconciliation has nothing to do in static mode: _do_capture hands
each capture to static_capture_job and returns before the only
pending_empty_primary.append(), and the check it performs reads the
per-game DLL cache, which static mode does not have. Guard the call on
the non-static path. In static mode, raise instead of silently skipping
if anything was ever queued, so a future change that routes static
captures through the DLL path fails loudly rather than dropping a root.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Prefer the ISO9660 boot file over the bounded raw-image scan. Biohazard stores SYSTEM.CNF outside that scan, leaving fresh builds at the default SCEA response. Add raw and cooked late-file regressions with an early decoy serial. The full-profile prefix again matches the retained native baseline through 5999 returns; the independent-source frontier remains return 464.
(cherry picked from commit 82c3de13ef)
Matthew Stanley