Some PlayStation 1 titles implement trivial rejection by folding the sign bits of projected vertex screen coordinates across triangle or quad primitives with a chain of AND instructions. Without adjusting these checks during widescreen presentation, geometry in the newly revealed horizontal area is still rejected by the original 4:3 bounds. Rather than embedding title specific instruction patterns directly into the recompiler, this introduces a generalized `[[widescreen.cull.sxy]]` configuration schema within `game.toml`.
Game configurations can now declare multi vertex trivial rejection sequences by their exact AND instruction words, vertex registers and the branch that consumes the result. The loader derives the expected chain from those words and rejects any other shape. Each AND keeps its original value except for the horizontal sign bit, which survives only while that vertex lies beyond the revealed left edge, so vertical and right side rejection are preserved and 4:3 output is bit identical. Shifted coordinate lower edge comparisons reuse the existing `screen_x_sites` handling through a new `slti rt, x, 0` form instead of adding a separate site family.
Full parity is maintained between recompiled code and the dirty RAM interpreter by sharing the same runtime helper and validating every fold and consumer instruction word on both paths, leaving mismatching overlay variants untouched. This eliminates hardcoded game logic from the recompiler while giving titles clean geometry retention in expanded aspect ratios.
Keep selector zero stock and seal all choices until the next session restart. Refuse nonzero online activation, scope non-stock save states by frozen choices, and preserve original instruction guards when combined with effective-address adapters. Use overlay ABI34 on current master without the old menu dependency stack.
Reconciled with #470 (PGXP): gpu.c precise vertices take PGXP only under geometry correction and count it, then native-wide edge recovery; the PGXP triangle stat stays PGXP-only. Stubs and tests from both kept; stamps recomputed.
Many PS1 renderers drop a primitive whose depth index falls past the end
of their ordering table. Raising that limit can write past the table, so
a title can now list such guards in game.toml with the register that
carries the index and the last safe value. While a trusted mod has called
psx_mod_set_draw_distance_clamp(1), generated main-EXE code clamps that
register before the guard runs, so far geometry is kept in the farthest
slot instead of popping in later.
- Off by default and reset at every session start, so netplay and every
title that lists no sites run unchanged; with no sites the generated
code and declarations are byte-identical.
- Main executable only: the full instruction word is the guard,
generation fails on a mismatch or a load-delay hazard, captured
overlay code keeps its own code (the sites are not part of the overlay
config identity), and the dirty-RAM interpreter applies the clamp
where the game's text image holds the listed word.
- Compatibility: code_generator.cpp is a codegen-hash input, so the
codegen hash and the overlay-cache tag change for every title, sites
or not. Savestates from earlier builds are refused, and overlay shards
rebuild once on first visit. Generated code and behaviour are
unchanged for a title without sites or with the switch off.
- Tests: draw_distance_codegen_test (config rules, emission, guards,
declarations, overlay identity) and draw_distance_interp_test (store,
switch, interpreted guards); the session-reset guard covers the switch.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Preserve current segmented dispatch, active-plan mod hooks and software scale limits. Extend function filters through the active plan, retain the last-store pointer with overlay ABI 25, and keep default config fingerprints stable. Refresh BIOS emitter stamps after byte-identical regeneration and reconcile regression fixtures.
A capture view is compiled at `segment | phys` in whichever segment its
entries ran (docs/SEGMENT_AWARE_CODE.md §5.7), but the recompiler matched
the game config's exact-match code sites as spelled. The config names
overlay code by its bytes, which run in any segment. The runtime keys mod
function-entry hooks by physical address, and the interpreter fires them at
every entry, whatever its segment. So a KUSEG or KSEG1 shard of a hooked
overlay function emitted no psx_mod_function_entry call: the hook fired on a
cold cache (interpreted) and not on a warm one (the new shard).
overlay_codegen_config() (segment_variants.cpp) moves every exact-match
site kind that D's rebase_codegen_config() moves, and whose physical address
lies in the image in a segment that maps RAM, into the view's segment. It
shares the site list with rebase_codegen_config(), which keeps its
link-segment match and its split pre-pass setting. main_psx applies it in
overlay mode. A KSEG0 view with KSEG0-spelled sites, as every pre-§5.7
overlay compile has, is unchanged.
Tests:
- segment_variants_test checks overlay_codegen_config() in each segment's
view: sites spelled in any RAM segment move, and addresses outside the
image, KSEG2 and 0x20000000-0x7FFFFFFF spellings and the physically
matched kinds stay.
- overlay_segment_keys compiles a v3 capture entered at all three segments
with mod_function_entry_funcs spelled at KSEG0 and KSEG1. Every segment's
shard must emit both hooks at its own PCs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A seed that names the image's bytes in another segment than its link
segment (0xA00100F0 for a KUSEG-linked EXE, 0x00012345 for a KSEG0 one)
now compiles a segment variant (docs/SEGMENT_AWARE_CODE.md §5.4): the home
functions the seed reaches through direct edges (calls, tail jumps, split
pieces, fall-throughs, call returns; a seed at a CPS continuation roots its
owner, an alias entry its alias group) are compiled again through a view of
the image in that segment. The variant's names, rows and every PC it bakes
are that segment's, and a KSEG1 variant charges a fetch per instruction
(#429's rule through B's runtime_pc()). A cached variant is the home C with
the segment swapped. Exact-match config sites move into the variant's
segment, so a variant gets every hook the home body gets. A seed that names
no home dispatch entry stops the build, and so does a seed in a segment
that does not map physical memory (0x20000000-0x7FFFFFFF or KSEG2: Beetle's
addr_mask leaves them unmasked, so they address no RAM and are no alias of
the image's bytes).
tools/collect_game_misses.py reports such a segment miss instead of writing
it as a seed.
The dispatch table holds a row per compiled segment of a word, adjacent and
in PC order; when any word has more than one row the lookup (indexed and
binary search) searches the word's rows for the exact PC. Two rows for one
PC remain a build error. A title without variants gets the same dispatch
source as before.
A jump-table case whose value is in another segment than its body is no
longer a local case: the default tail-transfer dispatches it with its full
PC. This applies to every compile. In a variant it is the common case (a
KSEG0 variant of KUSEG-linked code). A home or overlay compile has one only
when its table holds another segment's PCs of its own bytes, for example a
KSEG0-compiled overlay shard of code that runs at KUSEG. Such a case used to
continue in the KSEG0 body with KSEG0 links and fetch tags. Now it
dispatches: a game-text PC with no row in its segment is a recorded segment
miss, and #417's gate sends an overlay PC to the interpreter; both run it
at its own PC (PR E returns overlay code to native per-segment shards). R4's
game C and its 26 overlay shard sources are unchanged.
Ledger: segment-variants and kseg1-fetch-charge close (KNOWN_GAPS is empty)
and become regression guards; resume-pc-segment now requires each row's key
and resume PC to be in its body's segment, and segment-variants checks every
PC a variant body bakes. New segment_variants_test; kuseg_dispatch_lookup
and emitter_runtime_pc_test cover variant rows, closure, equality with the
home C, per-instruction KSEG1 charging, foreign jump-table cases at home and
in variants, data stubs (no row), alias groups declared for every shard and
seeds outside the physical segments.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A PS-X EXE header may carry KUSEG addresses (Alien Resurrection SLUS-00633,
Kula World SCES-01000); the BIOS then runs the program at KUSEG PCs, so its
links, EPCs and I-cache tags are KUSEG. The parser folded such a header to
KSEG0, so the whole program compiled for a segment it never runs in, and
the dispatch lookup matched any segment by physical address
(docs/SEGMENT_AWARE_CODE.md §3.2, §5.3, §5.5).
- EXE parser: keeps the header's addresses. link_segment() is the load
address's segment; the load address and entry must share it and lie in
the 8 MiB RAM window of KUSEG, KSEG0 or KSEG1 (KSEG1 compiles as an
uncached home segment). phys_load_address()/contains_phys() do byte
addressing; the BSS overlap check and the text_size bound are physical.
- function_analysis: the three JAL scans take the target's top bits from
the delay slot's PC instead of assuming KSEG0 (unchanged for KSEG0).
- Seeds are range-checked by physical address. A seed in the link segment
is an entry (0x000100E4 was silently dropped before). A seed for the same
bytes in another segment asks for a segment variant (§5.4): it is listed
in a WARNING, never folded into the home body. Seeds outside the image are
counted. The producer_range, cross_call_allow and hosted_interior
directives have no variant form: a foreign spelling is refused with the
link-segment one instead of a bare "invalid" error.
- Config code sites that are matched exactly (widescreen and cull sites,
mod hooks, hot/data-shard funcs, the VSync-query function and its
event-horizon return PCs, persisted options) and name the image's bytes
in another segment are refused with the spelling that matches, since they
would now match nothing. Kinds matched by physical address (byte patches,
the full-word-guarded cull keep/angle/aspect-cone and signed-X-bound
sites, dome call sites) reach the same bytes in any spelling and are not
refused. Overlay compiles are exempt until PR E.
- tools/collect_game_misses.py wrote every dirty-RAM miss as a KSEG0 seed,
which a KUSEG-linked image now reports instead of compiling. It writes
them in the link segment (--segment, --game-toml, or a segmented text
range; it refuses to guess from a physical one) and adds the
segment_misses rows at their full PC.
- Dispatch source moves to game_dispatch_emitter.cpp. The lookup indexes
the physical word, then requires row.addr == addr; a second row for one
word is an error until variants share the table. psx_game_address_in_text
stays physical, so a segment miss takes the clean-text-miss path (the
previous commit records it).
For a KSEG0 title the game C is byte-identical; its dispatch file changes
only in psx_game_find_entry (the exact-PC compare) and its comment.
Tests:
- segment_aware_codegen closes link-segment, fetch-tag-segment,
irq-resume-segment, resume-pc-segment, store-pc-segment,
home-seed-accepted, alias-fetch-coherence and segment-miss; they stay as
regression guards, and resume-pc-segment now also checks the dispatch
rows' keys and resume PCs.
- kuseg_dispatch_lookup: KUSEG rows, names and baked PCs; aliases miss while
text stays physical; variant seeds reported; a function reached only by
jal and a SaveState-style continuation are discovered in a KUSEG image
(kills the two discovery scans reverted to KSEG0; the third, the
invalid-JAL data heuristic, reads the same bits either way); mixed-segment
headers rejected; a KSEG1 image charged per instruction; foreign config
sites refused; the RAM-mirror contract as before.
- kuseg_dispatch_lookup also compiles the lookup's binary-search form (a
table spanning 2 MiB or more) and queries every segment's alias of each
row; refuses a KSEG0 event-horizon site and KSEG0 seeds directives with
their KUSEG spelling; accepts a KSEG0 signed_x_bound site and applies it.
- emitter_runtime_pc_test: the dispatch rows follow runtime_pc() and keep
the compile identity in func_ names (kills B's two surviving mutants: 76
and 12 failing checks); two rows on one physical word are an error.
- collect_game_misses (new, tools/tests): link-segment seeds from each
source, segment misses at their full PC, no guessing, no duplicates,
against a fake server using the runtime's own summary row format.
- reachable_discovery_test: the parser keeps KUSEG, accepts KSEG1, rejects
mixed segments and non-RAM load addresses.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Segment-aware code, rollout PR B (docs/SEGMENT_AWARE_CODE.md §5.2).
Game/overlay emitter: CodeGenerator gains a code segment and
runtime_pc(addr) = code_seg | phys. Links (including a jalr in a branch
delay slot), fetch tags and the uncached-fetch test in emit_pre_icache,
interrupt resume PCs, CPS exit PCs and continuation keys, store-PC
stamps, the reserved-instruction EPC, the slice resume PC,
call_by_address and stale-guard targets, the call contract's return PC,
cosim PCs, the VSync-query hook's base PC, the data stubs'
unknown-dispatch PC and the dispatch rows' addr/resume_pc all go through
it. func_/block_ names, .ranges and entry-hook ids keep the compile
address. The VSync-query hook is not emitted for an uncached code
segment: its hand-timed body charges cached fetches only. The segment
defaults to the image's own, so runtime_pc(addr) == addr for every
existing input and KSEG0 titles regenerate byte for byte.
BIOS emitter: StrictTranslator::translate(d, runtime_pc) stamps stores,
the syscall EPC and the break and unaligned-access PCs with the PC the
code runs at; the psx_ldd_ temporaries keep the ROM address. The
full-function emitter passes relocate_ra() to every translate() call and
publishes the runtime PC on a fallthrough. Relocated kernel and shell
code therefore stamps its RAM address, as the interpreter does.
memory.c re-keys, in the same change, the seven SCPH-1001 store-PC keys
that named ROM addresses of relocated code: RAM-0 filters 0x00000F00
(ROM 0xBFC10A00), 0x80056EB4, 0x800585E4, 0x80058788, 0x80059C50 (ROM
0xBFC3EEB4/405E4/40788/41C50), the opt-in Tomba EvCB key 0x00001CE4 (ROM
0xBFC117E4) and the GP0 source key 0x80050B1C (ROM 0xBFC38B1C). Without
the re-key the new stamps would silently disable all seven. A RAM PC is
ambiguous where the ROM key was not, so each key goes through
scph1001_relocated_store(pc, rom): it matches only while the active
image is SCPH-1001 and the word at pc is still the ROM word copied
there. Other BIOSes and game code at those addresses never match, as
before.
Tests: emitter_runtime_pc_test (new, CTest, run with PSX_CPS unset and
set to 0) compiles a small program through generate_all_functions with
the code segment left default, KSEG0, KUSEG and KSEG1, in CPS,
CPS-overlay and legacy mode. It requires every emission path to be
reached and checks every guest-visible PC class, the identity keys,
physical-only differences and KSEG1 per-instruction fetch charging, and
the translator's runtime PCs in every PC-bearing form. The ledger gains
two regression guards: bios-runtime-pc (OpenBIOS output, plus every
translate() call site) and store-pc-keys-runtime (memory.c keys against
SCPH1001.toml's windows). KNOWN_GAPS is unchanged: B closes no ledger id
on its own (§5.2).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Configuration and generation errors reach main() as exceptions: a malformed
game.toml (a bad hex address, an out-of-range [widescreen.cull]
clip_edge_width, a wrong TOML type) from load_game_config, and inputs the
emitter must refuse, such as an overlay whose mandatory delay slot lies
outside the image. Nothing caught them, so the process ended in
std::terminate -> abort(): SIGABRT and a macOS crash report (possibly a
crash dialog) instead of a diagnostic. Runtime overlay autocompile and
several ctests hit the generation case on purpose.
main() now runs the old body (psxrecomp_game_main) inside a catch of
std::exception, prints "ERROR: <message>" and returns 1. Every caller
(compile_overlays.py, psxrecomp_cli.py, the tests) already treats any
non-zero status as failure; the messages they match on are unchanged.
--overlay-config-hash keeps its own catch and status 2.
New ctest recompiler_error_exit runs psxrecomp-game and requires exit status
exactly 1 (a signal fails the test) plus the diagnostic for: clip_edge_width
0, 1025 and a string; a non-hex bgez_sites and clip_edge_x_load_sites entry;
an integer site entry (each through --config and --ws-config); main-EXE bgez
and clip-edge sites that hold the wrong opcode; and a truncated overlay
(mandatory delay slot outside the image). A valid bgez site must still
generate through psx_ws_cull_bgez.
main_psx.cpp is not in the codegen hash source list, so the overlay cache
tag does not change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two generic explicit cull-site kinds for native-wide titles whose screen-X
rejects the existing keys cannot express exactly:
- bgez_sites: `bgez SX, keep` in a signed per-vertex left-edge chain
(`bgez x0,keep; ...; bltz xN,reject`). Emitted through psx_ws_cull_bgez
(keep while SX >= -margin). bltz_sites alone still rejects a primitive
whose first vertices sit in the revealed band; with both kinds the chain
rejects exactly when every vertex is left of -margin.
- clip_edge_x_load_sites (+ clip_edge_width, default screen_w_imms[0]):
lh/lhu/lw of a clip-rectangle X bound. psx_ws_clip_edge_x moves a loaded 0
to -margin and a loaded width to width+margin and leaves interior bounds
(mirrors, split-screen halves) unchanged. xclip_load_sites is lw-only and
upper-bound-only, so it cannot express this.
Both are identity at 4:3, emit nothing when empty, and fail main-EXE
generation on an opcode mismatch (overlay variants stay vanilla). They enter
the overlay-cache hash only when non-empty, so every existing title keeps
its hash (pinned in the test against the previous layout).
Interpreter parity: the dirty-RAM interpreter now also honours explicit
bltz_sites, bgez_sites, branch_keep_sites and clip_edge_x_load_sites. Its
explicit site lists are stored sorted (binary-search lookup), the cap rises
from 64 to 256 per key, and an over-long list is logged instead of being
silently truncated.
Shared math lives in runtime/include/ws_cull_edge.h. New ctest
ws_cull_edge_codegen_test covers parsing, hash identity, emitted code,
the opcode guards and the helper math.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The overlay capture format DELIBERATELY appends one coherent guard instruction
past the end of a dirty-page run (overlay_capture.c write_json_window, `size
+= 4u`, pinned by runtime/tests/test_interpreter_perf_guards.py:47): a MIPS
branch sitting at the run's final word (...FFC) always executes its delay slot
in the next page (...000), and the shard must be able to emit and hash that
word. So a capture whose size is a page multiple PLUS 4 is a CORRECT capture --
N bytes of overlay plus one guard word. Measured: 36 of 36 Ape Escape
(SCUS-94423) vault records and 846 of 946 Tomba 2 (SCUS-94454) records are that
shape.
The recompiler was never taught the asymmetry: the guard word is a legal
DELAY-SLOT SOURCE but an ILLEGAL BLOCK LEADER, because the word after it does
not exist in the image. It had exactly one bound -- end_address() -- used both
to read words and to decide what is code.
MEASURED on Ape Escape, its own single-source vault at framework
v0.3.2-alpha-139-g1bf70960, 36 variants:
PSX_SHARD_RESULT ok=50 failed=17 skipped=32 capacity_fastpath=0
! [recompiler] overlay 0x80136000 crc 4BAB23C9: what(): cannot emit control
flow at 0x80138000: mandatory delay slot 0x80138004 is outside the input
image
+ 16 x [fragment] interior 0x801360.. @region 0x00136000, identical message
One root cause, seventeen shards. The whole region then falls back to the
dirty-RAM interpreter forever, and nothing at runtime says so.
Causal chain, every step read in source:
1. overlay_capture.c:349-361 appends the guard word; region 0x80136000 gets
size 0x2004, so the image is [0x80136000, 0x80138004).
2. compile_overlays.py make_psxexe wraps those bytes verbatim with text size
= len(data), so PS1Executable::end_address() = 0x80138004.
3. function_analysis.cpp in_exe / in_function admit 0x80138000 as ordinary
code, and the walk reaches it by fall-through from 0x80137FFC
(0x24022000, addiu -- not control flow), so func.end_addr becomes the
image end.
4. control_flow.cpp makes the final block end_addr = func.end_addr - 4 =
0x80138000: the block LEADER is the guard word.
5. control_flow.cpp's "Also check last instruction itself" promotion finds
0x80137FFC is not control flow and therefore promotes read_word(end_addr)
-- the guard word 0x1062005A, a beq -- to block.exit_instr, with
has_delay_slot = true and address == end_addr. THIS IS THE DEFECT.
6. code_generator.cpp computes delay_slot_addr = 0x80138004, read_word
returns nullopt, and it throws.
THE THROW AT STEP 6 IS CORRECT AND IS PRESERVED. Emitting a control transfer
without its architectural delay slot silently corrupts guest state, which is
what the code did before that throw was added. The gap is step 5.
Fix: give the image view two distinct bounds.
end_address() READ bound -- every byte the image supplies, so a
mandatory delay slot living at a guard word resolves.
analysis_end_address() ANALYSIS bound -- one past the last byte that may be
discovered, made a block leader, or made a block's
exit instruction.
read_word() keeps the READ bound. Discovery and block construction move to the
ANALYSIS bound: function_analysis.cpp in_exe / in_exe_p3 / add_forced_entry /
every hard_cap that used to be the image end / the jr-table producer_hi
defaults / the static jr-ra and JAL sweeps; control_flow.cpp gains
analysis_walk_hi(func) and uses it for cfg.function_end, alias entries,
add_boundary, the scan loop, the final block's end_addr and in_func;
code_generator.cpp's mid-function split pre-pass; main_psx.cpp seed admission
and the data-scan pointer promotion; analysis_db.cpp, main_toml.cpp,
main_cli.cpp target/seed admission. The clamp is applied only when it leaves a
non-empty walk, so a degenerate function starting at or past the bound keeps
failing closed rather than silently producing an empty CFG.
HOW THE BOUND IS DERIVED -- the producer STATES it; the recompiler is told.
The alternative was for the recompiler to infer the count from `size % 4096 ==
4`. That fits every capture the current writer has ever produced, but it is a
consumer guessing at a producer's intent, and this tree already has the
counter-example: 52 of the 946 SCUS-94454 records have residues from 112 to
3996 bytes (one not even 4-byte aligned) because they come from the STATIC
extractor, which records an exact file extent and appends no guard word. A rule
keyed to the page-run shape is a rule that shifts the moment a producer
changes.
So the count travels with the image, in a magic-tagged field in the otherwise
zero-filled tail of the 2048-byte PS-EXE header (exe_tag in ps1_exe_parser.h:
"PSXRGRD1" at 0x7E0, uint32 count at 0x7E8). Putting it in the header rather
than on the command line means every consumer gets it automatically and no
future invocation path can forget to pass it -- compile_overlays has three
make_psxexe sites and five recompiler spawn sites. An absent or malformed tag
means ZERO guard bytes, which is exactly what every genuine PS-X EXE is: the
BIOS and every retail main executable have no guard word, their analysis bound
equals their read bound, and their analysis is bit-for-bit unchanged.
Producer side:
overlay_capture.c writes "guard_bytes": 4|0 per region -- the writer
stating its own intent, additive to the v2 schema and
preserved through coverage_vault.py compaction.
compile_overlays.py capture_guard_bytes(): prefer that field; for captures
recorded before it existed, reconstruct the single
shape write_json_window can produce (page multiple + 4)
and return 0 for everything else, which is the
fail-closed answer -- an unrecognised guard word is
analysed as code and the step-6 throw still fires
loudly at generation time.
make_psxexe guard_bytes is keyword-only with NO default, so a new
call site is a TypeError, not a silently mis-analysed
shard. The count is threaded through
compile_fragment_batch / compile_interior_fragment /
generate_interior_fragment_static and into the fragment
fail-memo key (bumped to v3, so a memo recorded under
the old analysis cannot suppress a retry under the
corrected one).
SECOND-ORDER DEFECT FIXED IN THE SAME PASS. generate_ranges_manifest extends a
range by +4 for any block whose exit instruction IS its last word, so the
cloned delay slot joins the candidate CRC and the page-generation watch. That
extension was unclamped. Reachable TODAY, independent of the guard word: a
branch-likely opcode (0x14-0x17, RESERVED on the R3000A) as the image's very
last word is short-circuited by translate_basic_block into an inline RI raise
BEFORE any delay slot is read, so generation succeeds and the manifest claims 4
bytes past the image end. Measured on origin/master with a 28-byte synthetic
image: `R 80010000 20` for an image ending at 0x8001001C. The cache would then
validate the shard against bytes the shard never saw. Now clamped to the READ
bound -- not the analysis bound, because a guard word genuinely IS compiled in
when it serves as a delay slot and must participate in the CRC.
DEAD CODE, documented rather than trusted: basic_block.cpp's BasicBlockAnalyzer
bounds its walk correctly but nothing in the tree constructs it. It is still
compiled and still hashed into the codegen tag, so a change there invalidates
caches without moving an emitted byte. Its bounds move to the analysis bound
too -- wiring it up must not reintroduce this defect -- and a header comment now
says where the live walker actually is.
TESTS. Both registered, so ctest runs them and the configure-time
registration guard is satisfied.
recompiler/tests/test_overlay_guard_word_analysis_bound.py drives
psxrecomp-game over five synthetic images:
A tagged, 2 pages + guard word, last real word not control flow, guard word
IS a beq -- the measured Ape shape. Must generate; the guard word must not
be emitted as an instruction; the block must publish 0x80012000 for the
runtime to dispatch; no range may claim it.
B byte-for-byte the same payload with NO tag. Must still fail closed with
the mandatory-delay-slot diagnostic naming 0x80012000. A and B differ only
in the header tag, which proves the tag is what changed behaviour and that
the throw was not weakened.
B2 a misaligned declaration (6). Must be rejected and behave like B.
C tagged, with a real branch at the last analysable word. The guard word
must be emitted AS the delay slot and must be covered by the manifest.
D branch-likely as the final word. Generation must succeed and no range may
exceed the image end.
Against origin/master's psxrecomp-game the same file reports exactly the two
things this commit changes, and passes B, B2 and C:
FAIL: tagged guard word still fails generation: what(): cannot emit control
flow at 0x80012000: mandatory delay slot 0x80012004 is outside the
input image
FAIL: range [0x80010000,0x80010020) claims 4 byte(s) past the image end
0x8001001C
tools/tests/test_overlay_guard_bytes_declaration.py pins the producer half:
field precedence, the reconstruction table, rejection of nonsense
declarations, the tag's byte offsets, that make_psxexe with guard_bytes=0 is
byte-identical to the pre-fix wrapper, and that overlay_capture.c both appends
the guard word and now declares it.
MEASURED OUTCOME, Ape Escape, same vault, same flags, fresh cache:
before PSX_SHARD_RESULT ok=50 failed=17 skipped=32 capacity_fastpath=0
after PSX_SHARD_RESULT ok=31 failed=0 skipped=57 capacity_fastpath=0
ok falls because 19 isolated island fragment shards at 0x00136000 are no longer
needed once the region variant compiles as one shard. Coverage is what matters,
and it is measured from the DLL exports: 349 -> 361 unique func_XXXXXXXX
entries, ZERO lost, 12 gained -- 0x80136140, 0x8013614C, 0x80136154,
0x80136238, 0x801362D4, 0x801362DC, 0x801362F0, 0x80136324, 0x80136370,
0x80136400, 0x80136480, 0x80137950, all from the failure list. The other four
failing PCs were already served by a sibling variant's shard.
INERTNESS.
BIOS (SCPH1001, psxrecomp-bios --config bios/SCPH1001.toml): all four
generated artifacts BYTE-IDENTICAL, verified by sha256 rather than by exit
code, because tools/regen_bios.sh exits 0 on failure.
Tomba 2 main EXE (psxrecomp-game --config game.toml): all nine generated
artifacts BYTE-IDENTICAL, and the console log is identical too.
Tomba 2 overlays, 82 variants over 6 regions from its own vault: identical
result line (ok=98 failed=0 skipped=103) and identical shard identity set,
4901 unique func exports on both sides -- zero gained, zero lost. 97 of 98
range manifests are byte-identical. The one that differs is region
0x8010A000 variant A32DB188, size 0x2004, whose last function's range goes
from 0x218 to 0x214 bytes: its guard word (0x24840008, addiu) had been
emitted as an ordinary instruction and hashed, and now is not. That is
precisely the intended change and nothing else moved.
Bead beads-eio.3.100.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The dirty-RAM interpreter reused a suffix-aware validity check when resuming generated code after a patched call site. If an entry prologue had been overwritten but its resume suffix still matched, execution could enter stale AOT code and fall through to another generated piece without redispatch.
Emit a whole-entry predicate for straight-line handoffs, retain suffix checks for explicit guest transfers, and fail closed for older generated dispatchers until regeneration.
Validated with the MGS PAL 6,875-frame route, manual codec-to-heliport gameplay, focused continuation-guard tests, and a current-upstream recompiler build.
PS1 segments alias the same physical RAM. Most PS-X EXE headers carry KSEG
addresses, but a header may legally carry KUSEG ones instead, and the guest
then executes with a KUSEG PC. Alien Resurrection (SLUS-00633) is such a
title: its header holds load address 0x00010000 and entry PC 0x000155A4,
and get_registers on a live run reports PC=0x00010170.
The emitted dispatch table is keyed by the recompiler's KSEG-normalized
addresses, and psx_game_find_entry() compared those keys against the raw
incoming address. For a KUSEG PC the comparison can never succeed, because
0x0001xxxx is always below 0x8001xxxx: the binary search walks to the start
of the table and returns no entry. Every dispatch then falls through to the
interpreter.
The failure is silent and easy to misread. psx_game_address_in_text() does
mask, so it still reports the address as game text and the interpreter runs
the real RAM bytes via clean_game_text_miss. The game boots and renders
correctly, just entirely interpreted. Because no entry is ever found, the
text-image guard is not reached either, so its diagnostics stay at zero
(text_native_blocked=0, text_exact_mismatches=0) while dirty_ram_stats
reports over a billion interpreted instructions and native_handoffs=0 --
which reads as "nothing was rejected" rather than "nothing was looked up".
Compare 29-bit physical addresses on both sides of the search, and sort the
table by that same masked key so the binary-search invariant holds by
construction rather than by every entry happening to share a segment.
Adds tests/test_kuseg_dispatch_lookup.py: synthesizes a KUSEG-addressed
PS-EXE, runs the emitter, and asserts the lookup masks both the incoming
address and the stored key, and that the emitted table is ordered by
physical address.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
compile_overlays.py spawns psxrecomp-game with cwd = dirname(game.toml),
because the seeds and capture paths in the config are relative to it. That
cwd is also where the recompiler probes for bios/SCPH1001.toml when the
config carries no explicit [recompiler] bios_config -- and --ws-config, the
only config channel overlay compiles use, deliberately does not adopt the
config's paths, so bios_config is never consulted for them.
For a PACKAGED game.toml (packaging/release/game.toml) that directory holds
neither bios/SCPH1001.toml nor a vendored framework one level down. Every
shard then died with "FATAL: no BIOS profile found", ok=0, and the overlay
cache silently failed to build. The run still exits, so the only symptom is
that the title ships with no cache and every player's first session runs
interpreted. Reported as issue #72, and it is what the recipe printed by the
game repos' package_release.ps1 told people to run.
Fix:
- psxrecomp-game takes --project-root <dir> and resolves the profile against
it: the direct bios/SCPH1001.toml probe, the one-level vendored-framework
probe, and a relative [recompiler] bios_config (documented as relative to
the game project root, previously resolved against the CWD). Absent the
flag, behaviour is unchanged.
- Every "no BIOS profile found" / bios_config failure now names the absolute
directory it searched. Without it the message reads as a config error when
it is a CWD error, which is what cost two vault parses on the Tomba 2
v0.0.8 cut.
- compile_overlays.py forwards --project-root to all three recompiler spawn
sites, defaulting it to the framework root derived from the required
--runtime-include. The derivation is used ONLY when it actually locates a
profile, so it can repair a broken invocation but never alter one that
already worked -- and the recipe the packager prints now works as written,
with no game-repo change. Every attribute lookup degrades to "pass no
flag" so the in-process callers that build a partial args object keep
working.
This also un-breaks the framework's own overlay test suite, which was RED on
master for the same reason: test_aot_overlay_discovery.py compiles against
tools/cycle_testrom/game.toml, so the recompiler probed
tools/cycle_testrom/ -- which has no BIOS profile -- and every shard failed.
Measured on this tree with one recompiler binary, changing only the tool and
tests: master exits 1 at check_real_batched_fragment_publication; this commit
reaches ALL PASS.
Two test repairs were needed to get there, both pre-existing and both masked
by the profile failure that stopped the suite before it could reach them:
- check_full_candidate_cli_fastpath stages a SYNTHETIC runtime_include in a
temp dir, so the framework root is not derivable from it and the root must
be named explicitly.
- the same test staged its cache prefill under cg<ver>_<hash>, missing the
_gc<config-hash> component the cache path gained. A prefill one directory
off is not found, so the run compiled for real instead of taking the fast
path the test exists to assert.
The cache tag is unaffected: main_psx.cpp is not in codegen_hash_sources.cmake
(verified -- the baked hash is fbb44d49 before and after), and
--overlay-config-hash returns before any of this code runs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The no-bios_config fallback probed exactly two paths, the second of
which hardcoded the submodule directory name "psxrecomp/". Game repos
do not all use that name: ApeEscapeRecomp vendors the framework as
"psxrecomp-v4/", so `psxrecomp-game --config game.toml` failed outright
there with "no BIOS profile found" and could not regenerate at all.
Probe one directory level for any vendored <framework>/bios/SCPH1001.toml
instead, sorted so the pick is deterministic. Verified by regenerating
Ape Escape (SCUS-94423) with no [recompiler] bios_config set.
Also split the failure message: an explicit bios_config that does not
resolve now names the offending path, instead of reporting "no BIOS
profile found (set [recompiler] bios_config ...)" for a key that is in
fact set -- which sends you looking for the wrong problem.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds PCSX-Redux's MIT-licensed OpenBIOS as a second BIOS the framework can
statically recompile, next to retail SCPH1001. Because OpenBIOS is
redistributable, a game verified against it ships its BIOS with the build —
players supply only a disc, and the launcher BIOS picker disappears.
The mechanism is general, not OpenBIOS-specific: every BIOS image gets a
bios/<STEM>.toml profile that is the single source of truth for its identity
and layout.
Recompiler
- bios/<STEM>.toml profiles: [program.image] (sha256 pin, redistributable
flag), [recompiler.address_model] (boot-time ROM->RAM copy windows,
kernel-bless window), install_slots, and [recompiler.runtime_exports]
HLE anchors (0 = feature structurally unavailable on that BIOS).
- BiosAddressModel (new): validated model built from the profile, replacing
~9 hardcoded SCPH1001 window constants that had drifted apart across the
emitter / discovery / game codegen. The emitted C normalize() derives
from the same table. from_config rejects overlapping/misaligned windows
and orderings where sequential folding != first-match. A CTest-registered
equivalence test sweeps every RAM/ROM address against verbatim copies of
the old helpers.
- Orphaned delay slots (the OpenBIOS-critical emitter fixes): jr/jalr
slots are inlined before the jump resolves — previously their side
effect was silently LOST (e.g. a case-return fragment's addiu $sp pop,
which unwound a shifted frame and walked the exception handler's IntRP
cursor into a wild dispatch). Orphaned branches resume at ds+4 on
not-taken instead of re-executing the slot (a read-modify-write slot
doubling twice broke OpenBIOS readPad). Jump targets latch pre-slot.
- MIPS-I load-delay modeling for dependent load pairs: the successor sees
the pre-load value (OpenBIOS's cardfasttrack.s relies on it on purpose —
"gotta break those bad emulators"). Unmodeled shapes warn at build time.
- Discovery hardening: branch-target dispatchability closure to fixpoint;
jump tables placed at jr+8 no longer walked as code; cross-function
continuations routed by an instruction->owner map.
- Signed MULT support (unreachable from the retail seed set; modern-GCC
OpenBIOS emits it freely), latency-modeled like MULTU.
- Seeds from ELF symbol tables + data-section code-pointer harvest
(tools/gen_bios_seeds_from_elf.py) — a source-built BIOS needs no
disassembly pass. 646 OpenBIOS seeds, incl. 5 empirical dispatch-miss
seeds.
- Stemmed outputs (<out_stem>_full.c / _dispatch.c) so two BIOSes coexist
in one out_dir. tools/regen_bios.sh is profile-driven (--config).
Runtime
- psx_bios_image: the generated dispatch now self-describes the BIOS it
was compiled from (identity: size/CRC32/SHA-256/wordsum; kernel-bless
window; HLE anchors; bundled flag). Runtime SCPH1001 hardcodes removed.
- Launch check is a hard identity gate (a mismatched image under
statically recompiled code is a guaranteed wild jump). Bundled builds
hide the BIOS picker and treat a missing/tampered bundled image as an
installation defect.
- Faithfulness fixes surfaced by OpenBIOS's tighter timing loops (all
global, argued in code comments, regression-tested against SCPH1001):
- SIO pad RX is cycle-paced like the memcard path (RX_RDY at
shift-complete, ACK +170); total ack-IRQ latency unchanged.
PSX_SIO_PAD_SYNC_RX=1 restores the legacy synchronous path for A/B.
- SPUSTAT derived from SPUCNT instead of permanently-asserted busy.
- GPUSTAT.13 (interlace field) alternates per vblank while interlaced,
pinned 0 in progressive.
- CD Init(0x0A) completes in ~3.9 ms instead of borrowing Beetle's
34 ms Reset figure (OpenBIOS's short bounded retry loop livelocked).
- Exception EPC guard accepts ROM-window block leaders, not just RAM.
- New TCP debug commands (documented in TCP_COMMANDS.md): bios_info,
s3_smear_watch, callret_watch; write-trace ring gains $s0-$s5.
bios/openbios.bin is vendored (MIT — notice in bios/OpenBIOS.LICENSE,
attribution in THIRD_PARTY_ATTRIBUTION.md, source pin + build recipe in
bios/OpenBIOS.toml). The emitter fingerprint digest is now content-only:
existing generated/*.emitter.sha will read STALE once; re-run
tools/regen_bios.sh.
Left-edge counterpart to slti_sites: emit psx_ws_cull_bltz at configured
bltz PCs. detect_cull_bltz_sites only classifies left-edge bltz for
auto_screen_x-qualified functions, so an X-only funnel wired via explicit
slti_sites had no left-edge widen. bltz_sites closes that gap. Identity at
4:3; regen-class.
Add opt-in [widescreen.cull] xclip_load_sites: a configured `lw rt,off(rs)`
loading a per-primitive X-reject bound (sltu-compared against ANDI-masked
u16 screen X) routes through psx_ws_xclip_bound() - INT32_MAX while the
margins are revealed (reject disabled; the wide-surface scissor clips the
overflow, wrapped off-left u16 coords pass), vanilla value at 4:3. Matched
native codegen emit and interpreter hook; overlay ABI v18 forwarder (NULL
preserves the vanilla loaded value); codegen ver 8. Parser, codegen and
overlay-mismatch regression coverage.
Split from #49: replaces the xclip_globals RAM poke with a recompile-time
instruction transform so native shards and the interpreter behave identically.
Add opt-in [widescreen.cull] plane_nx_sites: a configured `lw rt,off(rs)`
loading the X component of a side frustum-plane normal (sign-tested per
corner, dot = nx*px + nz*pz) routes through psx_ws_plane_nx() — scaled by
the inverse aspect factor (4*den)/(3*num) while the margins are revealed,
widening the plane cone by exactly atan((3*num)/(4*den)*tan(theta));
identity at 4:3. Matched native codegen emit and interpreter hook; overlay
ABI v17 forwarder (NULL preserves the vanilla normal); codegen ver 7.
Parser, codegen and overlay-mismatch regression coverage.
Split from #49: replaces the plane sign-test RAM pokes with a
recompile-time instruction transform so native shards and the interpreter
behave identically.
Matthew Stanley