Preserve current segmented dispatch, active-plan mod hooks and software scale limits. Extend function filters through the active plan, retain the last-store pointer with overlay ABI 25, and keep default config fingerprints stable. Refresh BIOS emitter stamps after byte-identical regeneration and reconcile regression fixtures.
Retro Studio's bulk tab bumps the title's psxrecomp pin first so the script
it runs is the one the title will pin; the clean-tree gate must not refuse
that. Submodule-internal changes are still refused at bump time.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
One pass per title: bump psxrecomp/recomp-ui to a ref that carries
docs/ci/templates/game-release.yml and the committed BIOS backends (refused
otherwise), un-ignore generated/, ensure [runtime] overlay_cache = true, emit
scripts/package_release.sh (retiring package_setup_release.sh) and
release.yml from the pinned template, rebuild emitters + regenerate the game
C from the title's disc, refresh framework_pins.txt, run check_boot_exe and
check_generated, commit; --push optional, --dry-run writes nothing.
The operation lives in project_studio/migrate_bundled.py; the tools/ script,
`git migrate-bundled` and `git bulk-migrate-bundled` (map_repos, per-repo
streaming) are its faces. Exercised end to end on a scratch copy of
Bomberman Party Edition (48 generated files committed, zip prefix kept,
generate_ci --check current).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The windows-2022 release job accepted the committed backends in the step
shell (Git Bash) and refused them at configure: runtime.cmake ran the bash
fingerprint through whichever bash find_program() located, and that
environment hashed a byte-identical tree differently. The fingerprint is now
tools/bios_emitter_fingerprint.cmake, run with the configuring cmake itself
on every platform; the shell script is a thin wrapper around it. Same digest
shape as before, so existing stamps stay valid (verified on an LF tree and
an autocrlf clone).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The Windows release job reports the committed backends as stale while
Linux and macOS accept the same tree; the configure error names no input.
check_bios_stamps.sh compares each stem's stamp with the local fingerprint
and, on mismatch, prints every hashed input's digest and the tool versions,
so the next run attributes the difference to a file.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Git for Windows checks text files out with CRLF, so the windows-2022 release
job hashed a byte-identical tree to a different value and BIOS_STALE_FATAL
refused the committed backends there while Linux and macOS passed. Stripping
CR before hashing makes the fingerprint line-ending independent (verified:
an LF tree and a CRLF copy of it now agree). Stamps rewritten to the new
canonical value; the generated C is unchanged.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
--check only looked for template steps the installed file lacked, so a copy
that still ran the removed OpenBIOS emission (or the setup-host wipe) read as
current and failed on the runner.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Decision (Alex, 2026-09-30): the recompiled BIOS carries the same risk as
the recompiled game, which titles already commit, so the retail backend is
compiled in rather than built on the player's machine. The retail image is
still never shipped: the runtime needs its data at run time and accepts only
the exact image the backend was built from, so the player supplies their
own dump as before.
- generated/{OpenBIOS,SCPH1001}_{full,dispatch}.c and their .emitter.sha
stamps are tracked (.gitignore un-ignores exactly those); regenerated with
the current emitter.
- bios_emitter_fingerprint.sh no longer hashes the ROM image: the profile's
SHA-256 pin stands for it and the emitter refuses any other image, and CI
has no dump, so hashing it made every CI configure read "stale".
- runtime.cmake: the staleness check covers every LINKED stem and
-DPSXRECOMP_BIOS_STALE_FATAL=ON makes a stale stamp fail configure.
- game-release.yml: no CI-side OpenBIOS emission (generate_openbios.sh is
gone); configure with BIOS_STALE_FATAL and assert both backends linked.
check_generated.sh asserts the submodule carries both backends tracked
and no dump is tracked anywhere.
- The player-side module build stays for a known image a build does not
link (SCPH-5552).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Bundled releases link OpenBIOS only; retail-BIOS-derived C never ships.
"Bring your own BIOS" now works without it: when a dump matching a shipped
profile (psx_bios_known_images.h) is selected, the runtime builds that
backend on the player's machine with the overlay toolchain the release
already carries, caches it, and loads it as one more registered backend.
- runtime/include/psx_bios_module.h: the module contract. An overlay-style
self-contained shared library (no imports from the executable) exporting
psx_bios_module_{abi,codegen_hash,init,backend}; reaches the runtime only
through OverlayCallbacks plus a new PsxBiosModuleCallbacks table (traps,
dirty-RAM/kernel-bless probes, fntrace, log_probe) and pointer aliases of
the globals the generated dispatcher touches. Its #defines rewrite the
emitter's own extern lines, so the generated C compiles unchanged.
- runtime/include/psx_bios_module_glue.c.inc: module-side forwarders and the
four exports, compiled after overlay_dispatch_preamble.c.inc.
- runtime/src/psx_bios_module.c: host side. Cache path
<exe>/cache/bios/<os-arch>/bm<abi>_<codegen hash>_f<flavor>/<STEM>_<crc>,
ABI + codegen-hash gated load, callback fill, and a blocking build via
overlay_toolchain/bios_module_build.py (gcc via compiler.txt or PATH,
else tcc), then psx_bios_register().
- psx_bios_backend.c: the registry is now a mutable table seeded from the
CMake-generated psx_bios_builtin_registry by a constructor, with
psx_bios_register(); every existing walker keeps working.
- main.cpp: validate_bios_for_launch acquires a module (announcing a first
build) before declaring a mismatch; discovery accepts a buildable dump;
the launcher verify reports it usable ("ready" / "compiled on first
launch") with no Generate & rebuild; the BIOS row shows whenever the
toolchain is present; bundled_only respects module support.
- emitter: the CPS-marker constructor is not emitted in a module build.
- overlay_api.h: OverlayCallbacks gains a struct tag;
overlay_loader_callbacks() exposes the host table before loader init.
- tools/bios_module_build.py (ships in the toolchain): identity + profile
SHA-256 gate, staged emitter tree, emit, three-TU compile, ctypes
self-check, atomic publish. release_stage.py stages psxrecomp-bios, every
bios/*.toml, the seeds and the builder into overlay_toolchain/.
- Tests: runtime/tests/test_bios_module_host.c against three fake modules
(good / wrong ABI / wrong hash) on real dlopen; recompiler/tests/
test_bios_module_build.py builds the bundled OpenBIOS end to end and reads
the descriptor back; guards extended.
Verified on Twisted Metal 4 (bundled build + staged toolchain, headless):
first launch with --bios SCPH1001.BIN builds and loads the module and boots
on image=SCPH-1001; the second launch loads it from cache.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The icon copy iterated a literal path nullglob does not cover and died on
the first real title (runtime.cmake stages the icon at assets/psxrecomp.png,
not beside the exe). And the dump gate rejected overlay_toolchain/bios/
SCPH1001.toml, a profile the toolchain needs; only image files are
forbidden.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Retire the setup-host release model (CI wiped generated/ and shipped
emitters + sources + a wizard so every player regenerated and rebuilt the
title). Titles now commit generated/ and CI ships the compiled game.
- docs/ci/templates/game-release.yml replaces setup-release.yml: gate on
committed game C (tools/ci/check_generated.sh), emit the OpenBIOS backend
in CI (tools/ci/generate_openbios.sh), configure with
PSXRECOMP_REQUIRE_GAME_C=ON and assert the full path was taken, package
with tools/package_game_release.sh, verify the zip is a game not a kit.
- tools/package_game_release.sh: exe, assets, bundled OpenBIOS, configs, mod
catalog + overlay toolchain via release_stage.py, notices; refuses sources,
emitters, generated C, BIOS dumps. clear_generated.sh removed;
package_setup_host.sh marked retired.
- Scaffold: gitignore templates track generated/, package_release.sh.in
wrapper, setup_project.{sh,ps1} commit generated/ after Generate.
- Project Studio: audit flags ignored/untracked generated/, retail BIOS C
and setup-host workflows; merge_gitignore un-ignores generated/; ops emit
the bundled packager + workflow; generate_ci uses the new template.
- Docs: docs/ci/BUNDLED_RELEASES.md replaces HOST_ONLY_RELEASES.md.
- Tests: tools/tests/test_bundled_release_layout.py; fix pre-existing
false positives in the packager-surface and overlay-static tests.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
docs/SEGMENT_AWARE_CODE.md §5.7, §7. The synthetic probe gains an overlay
phase that loads code the way a game loads an overlay. After the three
probe_run sequences it:
- leaves the critical section the BIOS started it in, because the kernel's
CD driver waits on CD interrupts;
- reads its own EXE file back from the disc through the BIOS file calls
(B0:32 open, B0:34 read, B0:36 close; the kernel DMAs the sectors) into
RAM at 0x000A0000;
- calls the copy of a position-independent ov_run there through KUSEG,
KSEG0 and KSEG1, timing each run with Timer 2.
It records the T2 deltas and ov_getpc links at R+0x28-R+0x3C, and the file
descriptor and byte count at R+0x40/0x44. In the runtime that code is
CD-DMA'd RAM code. Capture records its entries with their segments, and
compile_overlays.py builds a shard per segment.
The phase replaces the `j .` spin with a `j` of the same size, so every
earlier label keeps its address. The static probe code and every result
word it wrote before are unchanged, and so are its probe_run cycle-watch
hits on Beetle and native. Beetle on both BIOSes: T2 56 / 56 / 82 and links
0x000A0A2C / 0x800A0A2C / 0xA00A0A2C, the same shape as probe_run.
The ledger gains the regression guard overlay-segment-shards (§5.7). It
builds the capture record overlay_capture.c writes for the probe's
disc-loaded ov_run (schema v3), splits it with capture_segment_views(), and
compiles each view as the fragment pass does, with compile_overlays.py's own
helpers and the real recompiler. It requires:
- every PC each shard bakes, and its names, to be in its segment;
- the manifest's S and F records;
- ov_run's straight run to cost Beetle's fetch-model cycles: line leaders
cached at KUSEG and KSEG0, +4 per fetch at KSEG1.
KNOWN_GAPS stays empty.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docs/SEGMENT_AWARE_CODE.md §5.7. compile_overlays.py splits each capture
into one view per segment with entries (capture_segment_views). A v2
capture has no dispatch_entry_segments and comes back unchanged, KSEG0 only;
in v3, a dispatch entry that no segment list names is a KSEG0 entry. A view
spells load_addr and every PC in its segment, holds only that segment's
dispatch entries, and carries all the execution evidence, which has no
segment. A malformed segment record fails that capture loudly.
Two kinds of record also get a view where they saw no dispatch entry:
- a record with no dispatch entry in any segment (execution evidence only,
which the runtime writes) is one KSEG0 view, exactly as a v2 reader reads
it, so classification, forced interiors, declared entries and prior
manifests apply as before;
- declared_view_segments(): a --force-interior PC inside the record (KSEG0)
asks for the KSEG0 view, and a game.toml [[overlays]] table asks for the
segment its load address, or else each of its entries inside the record,
is spelled in.
Each view compiles like any capture, region and fragment passes alike:
- the image is compiled at `segment | phys`, so the recompiler names and
bakes it in that segment;
- helpers that spell PCs without an image address use the view's segment
(image_segment(); KSEG0 by default, so a KSEG0 compile is unchanged);
- the shard lands in segment_cache_dir(): the tag directory for KSEG0,
seg-kuseg/ and seg-kseg1/ otherwise.
The manifest writer adds `S <segment>` for KUSEG/KSEG1. The runtime-contract
parser keeps F entries' full VA, applies the S rules, and refuses a
manifest whose segment differs from its directory's. Exports are looked up
by their full VA.
Capacity accounting counts each leaf's segment directories in the leaf's
namespace, since the runtime's candidate table is shared, and tells shards
apart by segment and name. The static generator emits the exact PC per row,
so KUSEG, KSEG0 and KSEG1 rows of one word coexist, with a segment gate
over the segments it compiled. A KSEG0-only table emits the same text as
before. Non-KSEG0 static namespaces are ov_s0_ (KUSEG) and ov_s5_ (KSEG1).
The static isolated-fragment pass keys its images by segment too, so each
(entry, image) demand compiles against its own segment's view and one entry
dispatched in two segments gets a fragment in each. --force-interior PCs
stay KSEG0 and force KSEG0 views.
coverage_vault.py unions dispatch_entry_segments when it merges variants,
making a v2 record's entries explicit KSEG0 entries, and crops it when it
compacts regions.
Output for v2 captures is unchanged: the 52 R4 shard sources and manifests
compiled from D's race captures are byte-identical to D's.
Tests:
- new overlay_segment_keys (CTest). It runs capture views, manifests and the
cache layout, then the real compile_overlays.py driver, in process, on a v3
capture of an overlay entered at KUSEG and KSEG1, and checks:
- each shard lands in its directory with S and full-VA F records;
- every baked PC is in its segment, and KSEG1 charges a fetch per
instruction;
- a second run is served from each segment's cache;
- a KSEG0 view compiles byte-identically to the same bytes captured as v2;
- --static emits per-segment rows and namespaces, and an isolated
fragment per segment for an entry with no callable boundary;
- a dispatch-less v3 record with --force-interior compiles byte-identically
to its v2 reading, and a KUSEG-only record still gets its forced KSEG0
fragment;
- a KUSEG/KSEG1 view compiled into a cache holding a KSEG0 manifest of the
same bytes merges none of it, and the empty-primary reconciliation counts
only the view's own segment's fragments;
- the real loader runs the shards: links, store-PC stamps and fetch tags in
each segment, fetch cycles equal to Beetle's model, KSEG0 interpreted.
- coverage_vault: segment unions and crops.
- compile_overlays_static_split: a KUSEG row beside a KSEG0 row dispatches
by exact PC; unmapped-segment entries are refused. This replaces the
KSEG0-only rejection.
- aot_overlay_discovery: the capacity namespace lists the segment
directories.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SCPH-1001's reset code enters its relocated kernel through the uncached
alias (lui $a0,0xA000; addiu $a0,$a0,0x500; jr $a0 at ROM 0xBFC0044C).
Beetle charges those four instructions as uncached fetches, 20 cycles; the
BIOS dispatch normalized 0xA0000500 to the kernel body compiled for runtime
0x00000500, which charges cached fetches, 11. That is the -9 cycles left at
SCPH-1001's kernel entry since #429 (docs/SEGMENT_AWARE_CODE.md §3.3).
A BIOS seed that is a runtime PC inside a relocated copy window, in another
segment than the window runs in, now asks for a segment variant (§5.4): the
function at those ROM bytes and its direct-edge closure (branch and J/JAL
targets that leave it, the call return and fall-through past its end) are
emitted again with the seed's segment. Every PC a variant computes from its
own PCs is in that segment (links, fetch tags and the uncached test, IRQ
resume, branch, J and fallthrough targets, store and exception PCs), a KSEG1
variant charges a fetch per instruction, and a jump-table case in another
segment is dispatched instead of continuing locally. Variants and their
continuations get exact-PC rows in a segment_variant_table. A variant seed
that names no emitted function entry stops the build, and so does one in a
segment that does not map physical memory (0x20000000-0x7FFFFFFF, KSEG2).
The dispatch still finds the word by its normalized key (and applies the
kernel-bless guard); a new psx_bios_hit_body() then picks the body for the
exact PC. A variant compiled for that PC runs. Any other PC in another
segment than the one the word's home body was compiled for still runs the
home body, as before, but is now recorded as a BIOS segment miss in the
segment-miss ring (kind "bios", with the home PC), so a new alias entry is
loud instead of silent (§5.5). psx_bios_key_home_pc(), emitted from the
address model, gives each key's home runtime PC: its copy window's runtime
address, or the ROM run in place. The ring, TCP segment_misses and the exit
report carry each entry's kind; tools/collect_game_misses.py lists a BIOS
segment miss for the BIOS seeds instead of writing it into the game's.
SCPH-1001's seeds gain 0xA0000500 (kernel_entry_kseg1): a 4-instruction
KSEG1 variant of kernel_trampoline_0. Every BIOS's dispatch C gains the two
helpers and calls psx_bios_hit_body() where it called the home body. A BIOS
without variant seeds (OpenBIOS, SCPH-101, SCPH-5552) keeps its full C;
SCPH-1001's full C gains the variant body and its dispatch C the one-row
segment_variant_table.
New bios_segment_variants test: OpenBIOS with KSEG1 variants of kernel
0x12BC (closure 0x6F8 by jal and 0x70C by fall-through) and 0x13F4 (0x1424
by a conditional branch only) and a KSEG0 variant of kernel memset 0x540 (a
KUSEG jump table), plus a static check of SCPH-1001's seed. It compiles the
emitted lookup, with and without variants, and queries every variant row,
every home key's home PC (checked against the PC each function body was
compiled for) and every other segment's alias of it. Seeds in a segment
that does not map physical memory stop the build, and a runtime PC in its
window's own segment stays a discovery seed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A seed that names the image's bytes in another segment than its link
segment (0xA00100F0 for a KUSEG-linked EXE, 0x00012345 for a KSEG0 one)
now compiles a segment variant (docs/SEGMENT_AWARE_CODE.md §5.4): the home
functions the seed reaches through direct edges (calls, tail jumps, split
pieces, fall-throughs, call returns; a seed at a CPS continuation roots its
owner, an alias entry its alias group) are compiled again through a view of
the image in that segment. The variant's names, rows and every PC it bakes
are that segment's, and a KSEG1 variant charges a fetch per instruction
(#429's rule through B's runtime_pc()). A cached variant is the home C with
the segment swapped. Exact-match config sites move into the variant's
segment, so a variant gets every hook the home body gets. A seed that names
no home dispatch entry stops the build, and so does a seed in a segment
that does not map physical memory (0x20000000-0x7FFFFFFF or KSEG2: Beetle's
addr_mask leaves them unmasked, so they address no RAM and are no alias of
the image's bytes).
tools/collect_game_misses.py reports such a segment miss instead of writing
it as a seed.
The dispatch table holds a row per compiled segment of a word, adjacent and
in PC order; when any word has more than one row the lookup (indexed and
binary search) searches the word's rows for the exact PC. Two rows for one
PC remain a build error. A title without variants gets the same dispatch
source as before.
A jump-table case whose value is in another segment than its body is no
longer a local case: the default tail-transfer dispatches it with its full
PC. This applies to every compile. In a variant it is the common case (a
KSEG0 variant of KUSEG-linked code). A home or overlay compile has one only
when its table holds another segment's PCs of its own bytes, for example a
KSEG0-compiled overlay shard of code that runs at KUSEG. Such a case used to
continue in the KSEG0 body with KSEG0 links and fetch tags. Now it
dispatches: a game-text PC with no row in its segment is a recorded segment
miss, and #417's gate sends an overlay PC to the interpreter; both run it
at its own PC (PR E returns overlay code to native per-segment shards). R4's
game C and its 26 overlay shard sources are unchanged.
Ledger: segment-variants and kseg1-fetch-charge close (KNOWN_GAPS is empty)
and become regression guards; resume-pc-segment now requires each row's key
and resume PC to be in its body's segment, and segment-variants checks every
PC a variant body bakes. New segment_variants_test; kuseg_dispatch_lookup
and emitter_runtime_pc_test cover variant rows, closure, equality with the
home C, per-instruction KSEG1 charging, foreign jump-table cases at home and
in variants, data stubs (no row), alias groups declared for every shard and
seeds outside the physical segments.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A PS-X EXE header may carry KUSEG addresses (Alien Resurrection SLUS-00633,
Kula World SCES-01000); the BIOS then runs the program at KUSEG PCs, so its
links, EPCs and I-cache tags are KUSEG. The parser folded such a header to
KSEG0, so the whole program compiled for a segment it never runs in, and
the dispatch lookup matched any segment by physical address
(docs/SEGMENT_AWARE_CODE.md §3.2, §5.3, §5.5).
- EXE parser: keeps the header's addresses. link_segment() is the load
address's segment; the load address and entry must share it and lie in
the 8 MiB RAM window of KUSEG, KSEG0 or KSEG1 (KSEG1 compiles as an
uncached home segment). phys_load_address()/contains_phys() do byte
addressing; the BSS overlap check and the text_size bound are physical.
- function_analysis: the three JAL scans take the target's top bits from
the delay slot's PC instead of assuming KSEG0 (unchanged for KSEG0).
- Seeds are range-checked by physical address. A seed in the link segment
is an entry (0x000100E4 was silently dropped before). A seed for the same
bytes in another segment asks for a segment variant (§5.4): it is listed
in a WARNING, never folded into the home body. Seeds outside the image are
counted. The producer_range, cross_call_allow and hosted_interior
directives have no variant form: a foreign spelling is refused with the
link-segment one instead of a bare "invalid" error.
- Config code sites that are matched exactly (widescreen and cull sites,
mod hooks, hot/data-shard funcs, the VSync-query function and its
event-horizon return PCs, persisted options) and name the image's bytes
in another segment are refused with the spelling that matches, since they
would now match nothing. Kinds matched by physical address (byte patches,
the full-word-guarded cull keep/angle/aspect-cone and signed-X-bound
sites, dome call sites) reach the same bytes in any spelling and are not
refused. Overlay compiles are exempt until PR E.
- tools/collect_game_misses.py wrote every dirty-RAM miss as a KSEG0 seed,
which a KUSEG-linked image now reports instead of compiling. It writes
them in the link segment (--segment, --game-toml, or a segmented text
range; it refuses to guess from a physical one) and adds the
segment_misses rows at their full PC.
- Dispatch source moves to game_dispatch_emitter.cpp. The lookup indexes
the physical word, then requires row.addr == addr; a second row for one
word is an error until variants share the table. psx_game_address_in_text
stays physical, so a segment miss takes the clean-text-miss path (the
previous commit records it).
For a KSEG0 title the game C is byte-identical; its dispatch file changes
only in psx_game_find_entry (the exact-PC compare) and its comment.
Tests:
- segment_aware_codegen closes link-segment, fetch-tag-segment,
irq-resume-segment, resume-pc-segment, store-pc-segment,
home-seed-accepted, alias-fetch-coherence and segment-miss; they stay as
regression guards, and resume-pc-segment now also checks the dispatch
rows' keys and resume PCs.
- kuseg_dispatch_lookup: KUSEG rows, names and baked PCs; aliases miss while
text stays physical; variant seeds reported; a function reached only by
jal and a SaveState-style continuation are discovered in a KUSEG image
(kills the two discovery scans reverted to KSEG0; the third, the
invalid-JAL data heuristic, reads the same bits either way); mixed-segment
headers rejected; a KSEG1 image charged per instruction; foreign config
sites refused; the RAM-mirror contract as before.
- kuseg_dispatch_lookup also compiles the lookup's binary-search form (a
table spanning 2 MiB or more) and queries every segment's alias of each
row; refuses a KSEG0 event-horizon site and KSEG0 seeds directives with
their KUSEG spelling; accepts a KSEG0 signed_x_bound site and applies it.
- emitter_runtime_pc_test: the dispatch rows follow runtime_pc() and keep
the compile identity in func_ names (kills B's two surviving mutants: 76
and 12 failing checks); two rows on one physical word are an error.
- collect_game_misses (new, tools/tests): link-segment seeds from each
source, segment misses at their full PC, no guessing, no duplicates,
against a fake server using the runtime's own summary row format.
- reachable_discovery_test: the parser keeps KUSEG, accepts KSEG1, rejects
mixed segments and non-RAM load addresses.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With the delay-slot fix the Beetle deltas were sane but not comparable: the
three unrolled call sequences sat at different I-cache line offsets, and
probe_run shared a line with seeded, so the KUSEG run found its first line
already filled. Beetle read 59/59/89 (KUSEG/KSEG0/KSEG1), where the fetch
model predicts a 23-cycle KSEG1 surcharge.
Pad each call sequence so its first T2 read starts a line, and start
probe_run on its own line. Beetle (disc boot, OpenBIOS and SCPH-1001 alike)
now reads 56/56/82: KUSEG equals KSEG0, and KSEG1 exceeds them by 26 cycles,
exactly the Beetle fetch-model difference for the 15 KSEG1 fetches (60 vs a
cold cached 34).
probe_run moves to 0x000100F0. Ledger ids are unchanged; fetch-tag-segment
counts 30 tags and alias-fetch-coherence reads 22 vs Beetle 11, because leaf
now straddles two lines.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Timer 2 subtraction sat in the second lw's load delay slot, so it read
the stale register, and the difference was not masked to T2's 16 bits. The
Beetle disc-boot run of the probe returned 0xFFFFFFF3-style values for
every delta. Add a nop after the load and an andi 0xFFFF.
The layout shifts by six words: probe_run moves from 0x000100C0 to
0x000100D8. The ledger ids are unchanged; fetch-tag-segment now counts 27
tags and alias-fetch-coherence reads 10 vs Beetle 5 for the moved leaf.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Compiled code bakes KSEG0 into link values, CPS/IRQ resume PCs and I-cache
tags. KUSEG-linked EXEs run KUSEG PCs through those bodies (and their KUSEG
seeds are dropped), and overlay code executed at KUSEG/KSEG1 stays
interpreted after #417. Until #429, both emitters also charged the uncached
KSEG1 fetch only at cache-line leaders, so compiled BIOS ROM code was 4
cycles short on 5,477 of OpenBIOS's 9,592 KSEG1 instruction sites (Beetle
and the interp charge every fetch).
docs/SEGMENT_AWARE_CODE.md recommends per-segment compiled variants (code
identity = full VA, emitter segment as a per-compile constant, exact-segment
dispatch, per-instruction fetch in uncached bodies, segment-aware capture
and cache keys) over segment-relative emission, which measures +3.2 % /
+27.2 % __TEXT on R4 shards.
tools/segment_testrom/gen_segment_exe.py emits a synthetic KUSEG-linked
PS-X EXE that also works as an oracle probe. The new ctest
segment_aware_codegen runs the real recompilers on it and on OpenBIOS and
checks emitted PCs, the emitted dispatch lookup and fetch cycles (through
runtime/src/psx_icache.c) against a Beetle ReadInstruction transcription.
It passes only while the observed gaps equal its ledger (9 today).
#429 (rollout A) fixed the BIOS undercharge on master before this landed,
so KNOWN_GAPS does not list bios-kseg1-fetch-charge: the OpenBIOS check
finds no uncharged KSEG1 site and stays as a regression guard that reports
the id as a new gap. The last commit of this series brings the doc up to
date with #429.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Normal-mode discovery ran psxrecomp-game with no --project-root and ignored its exit code, so from a cwd without bios/ every producer silently lost its discovered entries (Tomba 2 shipped 54 native pairs instead of 81).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Boot anchors (reset, the kernel copy, the first A0/B0/C0 calls, the shell)
pass within the first frames, before a TCP cyc_watch arm can land, so a
native-vs-Beetle boot comparison could not be run from the tree.
psx-runtime (debug_server.c) and psx-beetle (beetle_main.cpp) now read
PSX_CYC_WATCH ("<pc>" or "<pc>-<end>" for region mode) and PSX_CYC_WATCH_N
(default 16) at start-up and arm the watch before the first guest
instruction. The native arm shares the TCP handler's code path
(cyc_watch_arm); psx-beetle calls beetle_cyc_watch_arm. With neither
variable set nothing changes. tools/cycle_compare.py --no-arm then reads
both rings.
PSX_BEETLE_UNPACED=1 skips psx-beetle's 59.94 Hz wall-clock pacing for
headless oracle runs. Guest timing is unaffected.
docs/beetle-macos.md step 7 documents the variables and the anchor-key
difference for BIOS copy windows declared dispatch_key = "rom": the shell
is keyed natively at its ROM address (OpenBIOS 0xBFC0A500, SCPH-1001
0xBFC18000) while Beetle sees 0x80030000. Checked on macOS: the documented
example records the OpenBIOS shell entry at 5,015,670 in Beetle, the value
of the #429 land gate, and the native watch arms from boot.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ruler #2 no longer built from master:
- game.toml's text_size 0x800 is now rejected by the EXE parser ("a
truncated text_size must be 4 KiB page-aligned"). Use 0x2000, the
page-rounded size of the 0x1800-byte text. The icache_miss victim block
at +0x1000 is now inside the analysed range and compiled too.
- psxrecomp-game writes the text as shards (<exe>_full_NN.c), but
CMakeLists.txt still named a single <exe>_full.c. Glob the shards and
fail with the regeneration hint when there are none.
Add mkdisc.py, a pure-Python stand-in for mkpsxiso (Windows-only here):
it writes the ISO9660 boot disc as MODE2/2352 sectors with a correct
EDC/ECC and copies the license area from a disc the developer owns. The
built image stays gitignored.
README: the macOS/Linux flow (headless on both backends), and why
icache_miss reads +16 on the current ROM but +14 in the 2026-06-27 log:
the mmio loops moved the loop top from word 1 to word 0 of its cache
line, so each of its two misses refills one more word. Records the
Beetle values re-measured on macOS.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- run: a crash, hang-up or socket timeout on the final frame_fingerprint /
dispatch_stats queries is a RunError (exit 3, INCOMPLETE), not a
JSONDecodeError traceback with exit 1 (MISMATCH). Seeding, log and dump
I/O errors and a runtime that survives SIGKILL also exit 3. A missing
output directory is a usage error. The dump is written to OUT.tmp and
renamed.
- compare: with different FMV-quiet settings, once a quiet frame occurred
(qc > 0, a different frame set, or no qc column) the runs are not
comparable. Report INCOMPLETE and skip the DIVERGE, missing-frame and
locator lines, which were artefacts of the setting. Mixed settings with no
quiet frame are judged as usual, with a note.
- --launch on Windows keeps backslashes (shlex with no escape character),
so C:\psx\build\game-runtime.exe is not mangled into C:psxbuild...
Self-test: 59 cases (11 new), ctest fp_identity passes. R4 dumps re-judged
and fresh warm/cold runs repeat the IDENTICAL results with 497 tolerated
straddles.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Moves the warm/cold A/B tool out of R4 so every title can use it. `run`
launches the runtime from a command template ({build} {port} {headless}
{state_dir}) or from --runtime/--game/--disc, optionally installs a saved
overlay state (--seed), turbos to frame N and saves the frame_fingerprint
ring plus the dispatch-miss count. Runs default to PSX_DEBUG_FMV_QUIET=0 and
PSX_OVERLAY_AUTOCOMPILE_OFF=1, and the dump records what they used.
`compare` judges on the guest-fact columns (cyc mmio mc sp sc wc ws qc) and
reports wr/pc only as the frame where the runs part. It tolerates the
one-write VBlank straddle from docs/TCP_COMMANDS.md (ws/wc off for one frame,
equal again at the next) and, when FMV-quiet was on, the wc+qc shift with ws
judged against the carried offset. Every tolerated frame is counted and
listed; --strict turns them off. A straddle on the last frame, a judge column
an older runtime does not report, or different FMV-quiet settings make the
result INCOMPLETE. Exit codes are unchanged from the R4 tool: IDENTICAL 0,
MISMATCH 1, usage 2, INCOMPLETE 3, MISSES 4.
The self-test (ctest fp_identity) covers column judging, both tolerances,
truncated and mismatched frame sets, dumps without ws/qc, and drives `run`
against a fake debug server.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The emitter writes both helpers at a listed bgez_sites or
clip_edge_x_load_sites address whose instruction matches in overlay code
too, but overlay_dispatch_preamble.c.inc did not define them, so such a
shard failed to link (macOS, Windows) or to load (Linux dlopen RTLD_NOW)
and its overlay stayed interpreted.
The preamble now defines both over psx_ws_x_margin(), with the same pure
math gpu.c uses (ws_cull_edge.h, stdint-only, staged with the other
runtime headers). ws_cull_edge.h joins the shard compile surface in
codegen_hash_sources.cmake. overlay_shim_compile_contract links a shim
that calls both with the emitter's extern declarations, and
overlay_widescreen_callbacks runs them in an executable: identity with no
margin callback, widened at margin 120, interior clip edges unchanged.
Both fail without the definitions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Overlay shards are compiled at the KSEG0 address of their bytes: captures
record 0x80000000|offset and the manifest parser keys entries there. The
emitter bakes that segment into every PC a shard produces: the $ra a
jal/jalr writes, jump and exception-resume PCs, and I-cache fetch tags.
Dispatch matched candidates by the segment-stripped address, so a KUSEG or
KSEG1 alias of the same bytes ran the KSEG0 body at the wrong
architectural PC.
Found in R4 (SLUS-00797) with OpenBIOS: the exception handler enters its
RAM patch slots at KUSEG (0x0000281C, 0x0000357C). The kernel-page shard
linked $ra=0x8000282C where the interpreter (and hardware) gives
0x0000282C, and fetched 0x8000281C/0x8000357C, missing lines the kernel
had just filled with KUSEG tags: +4 cycles per slot, so the next VBlank
IRQ landed at a different instruction and warm (native) vs cold
(interpreted) overlay-cache runs saved different registers from frame
194 on. A KSEG1 entry would also be charged as cached.
PSX_OVERLAY_CODE_SEGMENT (overlay_loader.h) names the rule for capture,
the manifest parser, overlay_loader_dispatch and the generated static
overlay table: a PC outside it falls to the interpreter, which runs at
the PC it was handed. CPS continuations already failed closed this way
(their entry switch only knows KSEG0 PCs); function entries now match.
overlay_loader_status reports segment_alias_interp.
Static game code has the same shape for KUSEG-linked EXEs (the parser
normalizes their headers to KSEG0); that needs segment-relative emission
and is not changed here. Shard cache tags are unchanged.
Tests: test_overlay_segment_gate.py drives the real loader (pair-dedup
harness, which also gains the RAM-geometry globals it no longer linked
without) with a KSEG0 fixture shard: KUSEG and KSEG1 dispatches stay
interpreted and never call it, KSEG0 still runs native. It fails with the
gate removed. test_compile_overlays_static_split.py covers the generated
static table and rejects a non-KSEG0 entry.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two failures hit while migrating a title on Windows 11:
- `audit` crashed with UnicodeEncodeError on the "->" mark (U+2192) when the
console code page can't encode it (cp1250 here). main() now reconfigures
stdout/stderr with errors="replace", so reports degrade those characters
instead of aborting.
- record_framework_pins failed. subprocess.run(["bash", ...]) lets
CreateProcess search System32 first, which starts WSL's bash; it can't see
D:/ paths, and a CRLF checkout of record_pins.sh breaks any bash anyway.
It now resolves bash via shutil.which, passes POSIX-style paths, and falls
back to the existing git rev-parse snapshot when the script can't run.
Most of this was done by Claude Opus 5.5; I reviewed and tested it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
New projects describe themselves to community lists (recomp.board et al.)
via .recomp.json at the repo root: game, system PS1, type recomp,
toolchain psxrecomp, status exploring, and original region + serial.
write_recomp_json.py takes the serial from disc_probe.json and derives
region from the Sony serial prefix, falling back to --region (which
defaults to USA and would mislabel PAL discs). It only fills missing
fields, so re-running it never clobbers a maintainer's edits.
Both gitignore templates ignored /*.json at the root, which would have
silently dropped the file from the initial commit; un-ignore it.
Closes#406
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Matthew Stanley