docs/SEGMENT_AWARE_CODE.md §5.7. compile_overlays.py splits each capture
into one view per segment with entries (capture_segment_views). A v2
capture has no dispatch_entry_segments and comes back unchanged, KSEG0 only;
in v3, a dispatch entry that no segment list names is a KSEG0 entry. A view
spells load_addr and every PC in its segment, holds only that segment's
dispatch entries, and carries all the execution evidence, which has no
segment. A malformed segment record fails that capture loudly.
Two kinds of record also get a view where they saw no dispatch entry:
- a record with no dispatch entry in any segment (execution evidence only,
which the runtime writes) is one KSEG0 view, exactly as a v2 reader reads
it, so classification, forced interiors, declared entries and prior
manifests apply as before;
- declared_view_segments(): a --force-interior PC inside the record (KSEG0)
asks for the KSEG0 view, and a game.toml [[overlays]] table asks for the
segment its load address, or else each of its entries inside the record,
is spelled in.
Each view compiles like any capture, region and fragment passes alike:
- the image is compiled at `segment | phys`, so the recompiler names and
bakes it in that segment;
- helpers that spell PCs without an image address use the view's segment
(image_segment(); KSEG0 by default, so a KSEG0 compile is unchanged);
- the shard lands in segment_cache_dir(): the tag directory for KSEG0,
seg-kuseg/ and seg-kseg1/ otherwise.
The manifest writer adds `S <segment>` for KUSEG/KSEG1. The runtime-contract
parser keeps F entries' full VA, applies the S rules, and refuses a
manifest whose segment differs from its directory's. Exports are looked up
by their full VA.
Capacity accounting counts each leaf's segment directories in the leaf's
namespace, since the runtime's candidate table is shared, and tells shards
apart by segment and name. The static generator emits the exact PC per row,
so KUSEG, KSEG0 and KSEG1 rows of one word coexist, with a segment gate
over the segments it compiled. A KSEG0-only table emits the same text as
before. Non-KSEG0 static namespaces are ov_s0_ (KUSEG) and ov_s5_ (KSEG1).
The static isolated-fragment pass keys its images by segment too, so each
(entry, image) demand compiles against its own segment's view and one entry
dispatched in two segments gets a fragment in each. --force-interior PCs
stay KSEG0 and force KSEG0 views.
coverage_vault.py unions dispatch_entry_segments when it merges variants,
making a v2 record's entries explicit KSEG0 entries, and crops it when it
compacts regions.
Output for v2 captures is unchanged: the 52 R4 shard sources and manifests
compiled from D's race captures are byte-identical to D's.
Tests:
- new overlay_segment_keys (CTest). It runs capture views, manifests and the
cache layout, then the real compile_overlays.py driver, in process, on a v3
capture of an overlay entered at KUSEG and KSEG1, and checks:
- each shard lands in its directory with S and full-VA F records;
- every baked PC is in its segment, and KSEG1 charges a fetch per
instruction;
- a second run is served from each segment's cache;
- a KSEG0 view compiles byte-identically to the same bytes captured as v2;
- --static emits per-segment rows and namespaces, and an isolated
fragment per segment for an entry with no callable boundary;
- a dispatch-less v3 record with --force-interior compiles byte-identically
to its v2 reading, and a KUSEG-only record still gets its forced KSEG0
fragment;
- a KUSEG/KSEG1 view compiled into a cache holding a KSEG0 manifest of the
same bytes merges none of it, and the empty-primary reconciliation counts
only the view's own segment's fragments;
- the real loader runs the shards: links, store-PC stamps and fetch tags in
each segment, fetch cycles equal to Beetle's model, KSEG0 interpreted.
- coverage_vault: segment unions and crops.
- compile_overlays_static_split: a KUSEG row beside a KSEG0 row dispatches
by exact PC; unmapped-segment entries are refused. This replaces the
KSEG0-only rejection.
- aot_overlay_discovery: the capacity namespace lists the segment
directories.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docs/SEGMENT_AWARE_CODE.md §5.7 (decision 4). #417 made every KUSEG and
KSEG1 overlay dispatch fall to the interpreter, because every shard was
compiled for KSEG0. The loader now keys shards by segment, so a shard
compiled for KUSEG or KSEG1 runs for PCs of that segment.
Cache layout. KSEG0 shards keep their directory and {phys}_{crc} names, so
no existing cache is invalidated. KUSEG and KSEG1 shards live in the cache
tag's seg-kuseg/ and seg-kseg1/ subdirectories, under the same names; both
tiers scan them, and a shard's segment is its directory's
(cache_seg_from_path).
Manifest. A KUSEG or KSEG1 manifest starts with `S <segment>` (00000000 or
A0000000) before its first F. F entries are full VAs in that segment; the
parser no longer forces them to KSEG0. R records stay KSEG0-spelled byte
extents. Without S a manifest is KSEG0, as before. Refused, when indexing,
when loading and through live publication alike:
- a manifest whose segment differs from its directory's;
- an F entry outside its segment;
- S after an F, S twice, or S that is not a RAM segment base.
Dispatch. Each candidate records its segment. The physical indexes stay (the
bytes are one identity), and every lookup matches the PC's segment:
- the entry chain (idx_head_seg);
- CPS continuation ownership;
- the lazy manifest index and try_load_region;
- the interpreter's native-call contract.
The lazy-miss and range-owner memos are keyed by the full PC. A PC in a
segment that maps no RAM never runs a shard. Exports are resolved by their
full VA.
Counters. segment_alias_interp (from #417) now counts KUSEG/KSEG1 dispatches
the loader leaves to the interpreter because no shard of their segment is
cached or valid yet. The new segment_native counts those a shard of their
own segment runs. Both are in the TCP overlay_loader_status reply. A warm
cache takes the first to 0.
Full-VA compares:
- the overlay idle note's return test compares the resume PC with $ra
exactly, and counts it as internal only for a running shard of its segment;
- the shadow diff's two "returned to the caller" tests compare the exact PC;
- the diff mode's kernel-window exclusion tests the physical address, so
KSEG0 and KUSEG kernel candidates are treated alike. Before, it tested the
full PC, so only a KUSEG PC ever matched, and #417's gate never let one reach
it. This is opt-in (PSX_OVERLAY_DIFF).
Tests (loader harness, overlay_segment_gate):
- a KUSEG shard runs natively for its KUSEG PCs, entry and continuation,
while KSEG0/KSEG1 aliases are interpreted;
- one shard per segment for the same word, preloaded and lazy;
- a stale own-segment shard is interpreted rather than handed to another
segment's;
- a KUSEG continuation where KSEG0 has an entry at that word, and a lazy one
whose competing KSEG0 bundle is newer;
- refused manifests, including through live publication;
- the native-call contract;
- a source check that the interpreter records the segment from the full PC,
and that no site clears dispatch evidence without its segment bits.
The harness gains a segment-run scenario that drives real compiled shards
through a CPS dispatch loop. overlay_init_guard and overlay_retry_c11 follow
the per-segment lookup, and so do the disabled interpreter_perf_guards'
negative-miss and try_load_region checks (it still stops where it did on D).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docs/SEGMENT_AWARE_CODE.md §5.7. Overlay bytes are segment-free, but a PC
is not: a KUSEG, KSEG0 or KSEG1 entry of the same word links, traps, fetches
and stamps store PCs in its own segment, so each needs a shard compiled for
that segment. Capture kept only the physical word, so every shard was
compiled for KSEG0 and #417's gate interpreted the other segments forever.
dirty_ram_interp.h adds one sibling bitmap per segment that maps RAM
(g_dirty_ram_dispatch_seg_bitmap[3]; 768 KiB for the 8 MiB backing).
dirty_ram_dispatch_inner sets the bit for the full PC next to the existing
any-segment bit, once per interpreted dispatch. A PC in 0x20000000-0x7FFFFFFF
or KSEG2 addresses no RAM in Beetle and gets no bit.
dirty_ram_dispatch_evidence_clear() drops both together at every site that
dropped dispatch evidence: boot reset, a store into an executed page, DMA
preservation and the autocapture epoch. Snapshot jobs copy the siblings.
Capture schema v3 adds, per region,
"dispatch_entry_segments": {"kuseg": [...], "kseg0": [...], "kseg1": [...]}
with each PC spelled in its segment. dispatch_entry_pcs keeps its v2 meaning
(dispatched in any segment) and KSEG0 spelling, so existing readers are
unaffected. A record without the field is KSEG0 only.
Guest behaviour is unchanged: the bitmaps are host-only capture evidence and
are not saved in savestates. overlay_capture_retry_test checks the v3 record
through both writers (the background epoch and the synchronous final
capture), including that a cleared epoch's segment does not outlive it.
psx_ram_runtime_map_stubs.c defines the new bitmap, which memory.c's write
paths now reference, so psx_ram_runtime_map_test still links on GNU
toolchains. The disabled interpreter_perf_guards check follows the evidence
clear (it would otherwise contradict overlay_segment_gate's wiring check).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docs/SEGMENT_AWARE_CODE.md: §5.4 and §5.5 gain "Implemented in PR D"
(variant planning and compilation, the BIOS variants and BIOS segment
misses, the multi-row lookup, the five exact return checks, seeds outside
the physical segments); §3.3, §5.2 and §5.6 point at them; §5.2 and §5.4
say that the jump-table rule applies to every game and overlay compile, not
only to variants; the §7.1 ledger is at zero gaps with the new
exact-return-contract guard; §7.2 records the probe's compiled column after
D, the LLE boot after D (SCPH-1001 first hits at 0, shell entry -445), with
#435 on top (0 through the shell entry on both BIOSes) and after the review
fixes (every hit unchanged, no BIOS segment miss); §8 D and §9's IsC note
are updated.
FAITHFUL_TIMING_PLAN and ACCURACY_BURNDOWN get dated entries: the SCPH-1001
KSEG1 kernel entry and the masked call-contract checks are closed, and BIOS
alias entries without a variant are recorded.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
psx_call_contract and the BIOS dispatch loop compared a return PC with the
call's link physically ((a ^ b) & 0x1FFFFFFF). A PC carries a segment, and
the link a call writes is in the segment of the body that made it
(docs/SEGMENT_AWARE_CODE.md §5.5). A guest that returns to another
segment's alias of the call site did not return to that body, so letting
the suspended C continuation run there would execute it with the wrong
segment's PCs. §5.5 deferred these checks to this PR so one call contract
has one rule. All five are exact now:
- psx_call_contract (cpu_state.h): the bail-resolve PC and the $ra check;
- the emitted BIOS dispatch loop: the return boundary, the bail resolve, the
wild-return test and the exception-stack straddle.
Since B routes every baked link through runtime_pc() and C/D compile each
body for the segment it runs in, a legitimate return always matches in
full. The ledger gains the exact-return-contract guard: it compiles
psx_call_contract and checks that another segment's alias starts a bail and
resolves nothing, and that the emitted OpenBIOS dispatch has the four exact
checks and no masked one.
OpenBIOS and SCPH-1001 dispatch C change in those four lines; game C does
not change. cpu_state.h is in the codegen hash.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SCPH-1001's reset code enters its relocated kernel through the uncached
alias (lui $a0,0xA000; addiu $a0,$a0,0x500; jr $a0 at ROM 0xBFC0044C).
Beetle charges those four instructions as uncached fetches, 20 cycles; the
BIOS dispatch normalized 0xA0000500 to the kernel body compiled for runtime
0x00000500, which charges cached fetches, 11. That is the -9 cycles left at
SCPH-1001's kernel entry since #429 (docs/SEGMENT_AWARE_CODE.md §3.3).
A BIOS seed that is a runtime PC inside a relocated copy window, in another
segment than the window runs in, now asks for a segment variant (§5.4): the
function at those ROM bytes and its direct-edge closure (branch and J/JAL
targets that leave it, the call return and fall-through past its end) are
emitted again with the seed's segment. Every PC a variant computes from its
own PCs is in that segment (links, fetch tags and the uncached test, IRQ
resume, branch, J and fallthrough targets, store and exception PCs), a KSEG1
variant charges a fetch per instruction, and a jump-table case in another
segment is dispatched instead of continuing locally. Variants and their
continuations get exact-PC rows in a segment_variant_table. A variant seed
that names no emitted function entry stops the build, and so does one in a
segment that does not map physical memory (0x20000000-0x7FFFFFFF, KSEG2).
The dispatch still finds the word by its normalized key (and applies the
kernel-bless guard); a new psx_bios_hit_body() then picks the body for the
exact PC. A variant compiled for that PC runs. Any other PC in another
segment than the one the word's home body was compiled for still runs the
home body, as before, but is now recorded as a BIOS segment miss in the
segment-miss ring (kind "bios", with the home PC), so a new alias entry is
loud instead of silent (§5.5). psx_bios_key_home_pc(), emitted from the
address model, gives each key's home runtime PC: its copy window's runtime
address, or the ROM run in place. The ring, TCP segment_misses and the exit
report carry each entry's kind; tools/collect_game_misses.py lists a BIOS
segment miss for the BIOS seeds instead of writing it into the game's.
SCPH-1001's seeds gain 0xA0000500 (kernel_entry_kseg1): a 4-instruction
KSEG1 variant of kernel_trampoline_0. Every BIOS's dispatch C gains the two
helpers and calls psx_bios_hit_body() where it called the home body. A BIOS
without variant seeds (OpenBIOS, SCPH-101, SCPH-5552) keeps its full C;
SCPH-1001's full C gains the variant body and its dispatch C the one-row
segment_variant_table.
New bios_segment_variants test: OpenBIOS with KSEG1 variants of kernel
0x12BC (closure 0x6F8 by jal and 0x70C by fall-through) and 0x13F4 (0x1424
by a conditional branch only) and a KSEG0 variant of kernel memset 0x540 (a
KUSEG jump table), plus a static check of SCPH-1001's seed. It compiles the
emitted lookup, with and without variants, and queries every variant row,
every home key's home PC (checked against the PC each function body was
compiled for) and every other segment's alias of it. Seeds in a segment
that does not map physical memory stop the build, and a runtime PC in its
window's own segment stays a discovery seed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A seed that names the image's bytes in another segment than its link
segment (0xA00100F0 for a KUSEG-linked EXE, 0x00012345 for a KSEG0 one)
now compiles a segment variant (docs/SEGMENT_AWARE_CODE.md §5.4): the home
functions the seed reaches through direct edges (calls, tail jumps, split
pieces, fall-throughs, call returns; a seed at a CPS continuation roots its
owner, an alias entry its alias group) are compiled again through a view of
the image in that segment. The variant's names, rows and every PC it bakes
are that segment's, and a KSEG1 variant charges a fetch per instruction
(#429's rule through B's runtime_pc()). A cached variant is the home C with
the segment swapped. Exact-match config sites move into the variant's
segment, so a variant gets every hook the home body gets. A seed that names
no home dispatch entry stops the build, and so does a seed in a segment
that does not map physical memory (0x20000000-0x7FFFFFFF or KSEG2: Beetle's
addr_mask leaves them unmasked, so they address no RAM and are no alias of
the image's bytes).
tools/collect_game_misses.py reports such a segment miss instead of writing
it as a seed.
The dispatch table holds a row per compiled segment of a word, adjacent and
in PC order; when any word has more than one row the lookup (indexed and
binary search) searches the word's rows for the exact PC. Two rows for one
PC remain a build error. A title without variants gets the same dispatch
source as before.
A jump-table case whose value is in another segment than its body is no
longer a local case: the default tail-transfer dispatches it with its full
PC. This applies to every compile. In a variant it is the common case (a
KSEG0 variant of KUSEG-linked code). A home or overlay compile has one only
when its table holds another segment's PCs of its own bytes, for example a
KSEG0-compiled overlay shard of code that runs at KUSEG. Such a case used to
continue in the KSEG0 body with KSEG0 links and fetch tags. Now it
dispatches: a game-text PC with no row in its segment is a recorded segment
miss, and #417's gate sends an overlay PC to the interpreter; both run it
at its own PC (PR E returns overlay code to native per-segment shards). R4's
game C and its 26 overlay shard sources are unchanged.
Ledger: segment-variants and kseg1-fetch-charge close (KNOWN_GAPS is empty)
and become regression guards; resume-pc-segment now requires each row's key
and resume PC to be in its body's segment, and segment-variants checks every
PC a variant body bakes. New segment_variants_test; kuseg_dispatch_lookup
and emitter_runtime_pc_test cover variant rows, closure, equality with the
home C, per-instruction KSEG1 charging, foreign jump-table cases at home and
in variants, data stubs (no row), alias groups declared for every shard and
seeds outside the physical segments.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docs/SEGMENT_AWARE_CODE.md:
- Status names PR C. §3.1, §3.2 and §3.3 describe the folded parser and
the physical lookup in the past tense and point at §5.3/§5.5.
- §5.2: the dispatch rows are now tested (B's two surviving mutants), and
the code segment's default is the link segment.
- §5.3 and §5.5 record what PR C implements: link-segment compile, the JAL
scans, seed and config-site checks, KSEG1 home images, the exact lookup,
the segment-miss ring and its TCP/exit-report surfaces, and why
psx_call_contract stays masked until D (the BIOS dispatch loop's four
return checks share the contract).
- §5.4: until D a variant seed is reported, not compiled.
- §7.1: eight ids closed and kept as regression guards; two open.
- §7.2: the probe's compiled column after C equals Beetle and the
interpreter; cycle watch, spin totals and the segment-miss ring.
- §5.3 also records the seeds directives, collect_game_misses, which
config-site kinds are refused (exact matches) and which are not
(physical matches), and two consequences until later PRs: a KUSEG title's
overlay code runs interpreted until E, and a KSEG0 title's alias entries
into its own text run interpreted until D.
- §8: C's validation and cross-title impact (codegen hash: overlay caches
recompile once, pre-C savestates are refused); D also makes the five
masked return checks exact; E returns KUSEG titles' overlays to shards.
FAITHFUL_TIMING_PLAN gets the dated log entry. ACCURACY_BURNDOWN records the
closed axis-6 item, the axis-4 progress, and the masked call-contract checks
as an open item for D.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A PS-X EXE header may carry KUSEG addresses (Alien Resurrection SLUS-00633,
Kula World SCES-01000); the BIOS then runs the program at KUSEG PCs, so its
links, EPCs and I-cache tags are KUSEG. The parser folded such a header to
KSEG0, so the whole program compiled for a segment it never runs in, and
the dispatch lookup matched any segment by physical address
(docs/SEGMENT_AWARE_CODE.md §3.2, §5.3, §5.5).
- EXE parser: keeps the header's addresses. link_segment() is the load
address's segment; the load address and entry must share it and lie in
the 8 MiB RAM window of KUSEG, KSEG0 or KSEG1 (KSEG1 compiles as an
uncached home segment). phys_load_address()/contains_phys() do byte
addressing; the BSS overlap check and the text_size bound are physical.
- function_analysis: the three JAL scans take the target's top bits from
the delay slot's PC instead of assuming KSEG0 (unchanged for KSEG0).
- Seeds are range-checked by physical address. A seed in the link segment
is an entry (0x000100E4 was silently dropped before). A seed for the same
bytes in another segment asks for a segment variant (§5.4): it is listed
in a WARNING, never folded into the home body. Seeds outside the image are
counted. The producer_range, cross_call_allow and hosted_interior
directives have no variant form: a foreign spelling is refused with the
link-segment one instead of a bare "invalid" error.
- Config code sites that are matched exactly (widescreen and cull sites,
mod hooks, hot/data-shard funcs, the VSync-query function and its
event-horizon return PCs, persisted options) and name the image's bytes
in another segment are refused with the spelling that matches, since they
would now match nothing. Kinds matched by physical address (byte patches,
the full-word-guarded cull keep/angle/aspect-cone and signed-X-bound
sites, dome call sites) reach the same bytes in any spelling and are not
refused. Overlay compiles are exempt until PR E.
- tools/collect_game_misses.py wrote every dirty-RAM miss as a KSEG0 seed,
which a KUSEG-linked image now reports instead of compiling. It writes
them in the link segment (--segment, --game-toml, or a segmented text
range; it refuses to guess from a physical one) and adds the
segment_misses rows at their full PC.
- Dispatch source moves to game_dispatch_emitter.cpp. The lookup indexes
the physical word, then requires row.addr == addr; a second row for one
word is an error until variants share the table. psx_game_address_in_text
stays physical, so a segment miss takes the clean-text-miss path (the
previous commit records it).
For a KSEG0 title the game C is byte-identical; its dispatch file changes
only in psx_game_find_entry (the exact-PC compare) and its comment.
Tests:
- segment_aware_codegen closes link-segment, fetch-tag-segment,
irq-resume-segment, resume-pc-segment, store-pc-segment,
home-seed-accepted, alias-fetch-coherence and segment-miss; they stay as
regression guards, and resume-pc-segment now also checks the dispatch
rows' keys and resume PCs.
- kuseg_dispatch_lookup: KUSEG rows, names and baked PCs; aliases miss while
text stays physical; variant seeds reported; a function reached only by
jal and a SaveState-style continuation are discovered in a KUSEG image
(kills the two discovery scans reverted to KSEG0; the third, the
invalid-JAL data heuristic, reads the same bits either way); mixed-segment
headers rejected; a KSEG1 image charged per instruction; foreign config
sites refused; the RAM-mirror contract as before.
- kuseg_dispatch_lookup also compiles the lookup's binary-search form (a
table spanning 2 MiB or more) and queries every segment's alias of each
row; refuses a KSEG0 event-horizon site and KSEG0 seeds directives with
their KUSEG spelling; accepts a KSEG0 signed_x_bound site and applies it.
- emitter_runtime_pc_test: the dispatch rows follow runtime_pc() and keep
the compile identity in func_ names (kills B's two surviving mutants: 76
and 12 failing checks); two rows on one physical word are an error.
- collect_game_misses (new, tools/tests): link-segment seeds from each
source, segment misses at their full PC, no guessing, no duplicates,
against a fake server using the runtime's own summary row format.
- reachable_discovery_test: the parser keeps KUSEG, accepts KSEG1, rejects
mixed segments and non-RAM load addresses.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A PC carries a segment, and the game dispatch table is about to be keyed by
the full PC (docs/SEGMENT_AWARE_CODE.md §5.5). A PC whose physical word has a
compiled row only in another segment then has no body of its own: dispatch
interprets it through the existing clean-text-miss path and never runs the
other segment's body. Decision 2 in §10 is to interpret such a miss loudly
until the title is regenerated, so record it:
- psx_segment_miss.{c,h}: classification (the other segment's row at the
same word, through psx_game_is_function_entry), an always-on ring with the
full PC, the row that exists, $ra, $sp and the frame, and a per-PC count.
- dirty_ram_dispatch_inner hands every clean-text miss to
psx_segment_miss_note(), which records it when the PC has no row of its
own while another segment's alias does.
- TCP: new `segment_misses` (summary, or {"tail":N}); `dispatch_stats`
gains segment_miss_total/_unique; the emu-thread ping reports the total.
- psx_last_run_report.json gains a `segment_misses` section (top 50 PCs).
- psx_segment_miss_test covers classification, the note helper, the ring,
wrap-around and the summary order. segment_miss_wiring_test (a source
guard) pins the helper's call in dirty_ram_dispatch_inner, the TCP
command and its summary row shape, the dispatch_stats and ping fields and
the exit-report section, which no unit test can reach.
Nothing is recorded with a dispatcher generated before the exact lookup: its
physical lookup finds a row for every alias, so the check never fires.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docs/SEGMENT_AWARE_CODE.md:
- §3.1 names emitter functions instead of line numbers, adds the
delay-slot jalr link and the VSync-query hook, and moves cosim_* out of
the identity keys.
- §5.2 records what PR B routes through runtime_pc(), what keeps the
compile address, the code segment's default (the image's own), and
what the tests cover (the dispatch rows wait for PR C).
- §5.4 notes that D's variants must tail-transfer jump-table cases in
another segment.
- §5.6 records that B routes emit_pre_icache's uncached test.
- §7.1 lists the two new regression guards; KNOWN_GAPS stays at 10.
- §7.2 adds the probe's interpreter and compiled columns next to
Beetle's.
- §8 records B's validation.
- §9 records the re-keyed store-PC keys and their SCPH-1001 gate, the
filters' Beetle divergence, and the confirmed game-syscall EPC gap
(its own change).
FAITHFUL_TIMING_PLAN and ACCURACY_BURNDOWN get the dated entries; the
burndown also records the RAM 0x0-0xF boot-scratch divergence from
Beetle. STRING_TRANSLATION's note that attributed GP0 quads to ROM
0xBFC38B1C now names the runtime key.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PSX_FORCE_INTERP=1 (b0391bc7) is the tooling switch that sends every
dispatch into game text through the dirty-RAM interpreter, so the
interpreter's cycle model can be measured against Beetle. It works by
reporting the pages dirty. The native-safety checks later learned to
decide flagged pages by the actual bytes (dirty_ram_text_native_ok and
the exact-range dirty_ram_text_native_ok_ranges_from), and an untouched
page matches its reference image, so the compiled body ran anyway: the
switch had no effect on clean text.
Both checks now report no game text as native-safe while the switch is
on. Nothing changes without the environment variable.
Found while taking the interpreter column of the segment-aware probe
(docs/SEGMENT_AWARE_CODE.md §7.2): with the switch set, the probe's
links still came back KSEG0 and its cycles matched the compiled build
exactly. With this change the interpreter column matches Beetle on
OpenBIOS and SCPH-1001.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Segment-aware code, rollout PR B (docs/SEGMENT_AWARE_CODE.md §5.2).
Game/overlay emitter: CodeGenerator gains a code segment and
runtime_pc(addr) = code_seg | phys. Links (including a jalr in a branch
delay slot), fetch tags and the uncached-fetch test in emit_pre_icache,
interrupt resume PCs, CPS exit PCs and continuation keys, store-PC
stamps, the reserved-instruction EPC, the slice resume PC,
call_by_address and stale-guard targets, the call contract's return PC,
cosim PCs, the VSync-query hook's base PC, the data stubs'
unknown-dispatch PC and the dispatch rows' addr/resume_pc all go through
it. func_/block_ names, .ranges and entry-hook ids keep the compile
address. The VSync-query hook is not emitted for an uncached code
segment: its hand-timed body charges cached fetches only. The segment
defaults to the image's own, so runtime_pc(addr) == addr for every
existing input and KSEG0 titles regenerate byte for byte.
BIOS emitter: StrictTranslator::translate(d, runtime_pc) stamps stores,
the syscall EPC and the break and unaligned-access PCs with the PC the
code runs at; the psx_ldd_ temporaries keep the ROM address. The
full-function emitter passes relocate_ra() to every translate() call and
publishes the runtime PC on a fallthrough. Relocated kernel and shell
code therefore stamps its RAM address, as the interpreter does.
memory.c re-keys, in the same change, the seven SCPH-1001 store-PC keys
that named ROM addresses of relocated code: RAM-0 filters 0x00000F00
(ROM 0xBFC10A00), 0x80056EB4, 0x800585E4, 0x80058788, 0x80059C50 (ROM
0xBFC3EEB4/405E4/40788/41C50), the opt-in Tomba EvCB key 0x00001CE4 (ROM
0xBFC117E4) and the GP0 source key 0x80050B1C (ROM 0xBFC38B1C). Without
the re-key the new stamps would silently disable all seven. A RAM PC is
ambiguous where the ROM key was not, so each key goes through
scph1001_relocated_store(pc, rom): it matches only while the active
image is SCPH-1001 and the word at pc is still the ROM word copied
there. Other BIOSes and game code at those addresses never match, as
before.
Tests: emitter_runtime_pc_test (new, CTest, run with PSX_CPS unset and
set to 0) compiles a small program through generate_all_functions with
the code segment left default, KSEG0, KUSEG and KSEG1, in CPS,
CPS-overlay and legacy mode. It requires every emission path to be
reached and checks every guest-visible PC class, the identity keys,
physical-only differences and KSEG1 per-instruction fetch charging, and
the translator's runtime PCs in every PC-bearing form. The ledger gains
two regression guards: bios-runtime-pc (OpenBIOS output, plus every
translate() call site) and store-pc-keys-runtime (memory.c keys against
SCPH1001.toml's windows). KNOWN_GAPS is unchanged: B closes no ledger id
on its own (§5.2).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#429 charges psx_icache_fetch before every instruction whose runtime PC is
uncached, in both emitters, and makes the BIOS emitter's cache-line start
test use the runtime PC. The OpenBIOS check finds no uncharged KSEG1 site,
so segment_aware_codegen passes with 10 known gaps; bios-kseg1-fetch-charge
has not been in KNOWN_GAPS since the ledger's first commit in this series.
The check stays as a regression guard: an uncharged site reports the id as
a new gap.
docs/SEGMENT_AWARE_CODE.md:
- §5.6 describes the charge as landed, including the runtime-PC line-start
fix. The game emitter already has the predicate, but emit_pre_icache
tests the compile address: PR B must route both its psx_fetch_uncached()
argument and the fetch tag through runtime_pc(), and D adds the KSEG1
variants they reach.
- §7.1 lists ten open ids and records how A's id left the ledger.
- §8 marks A merged, with its Beetle parity (shell entry -126 on OpenBIOS,
-454 on SCPH-1001; the residuals are the IsC gap and SCPH-1001's KSEG1
kernel entry). D now only needs the KSEG1 variants; kseg1-fetch-charge
stays open until D. The macOS oracle recipe is docs/beetle-macos.md (#431).
- §1, §3.5 and the header say the BIOS undercharge is fixed. §10 is
unchanged.
- §3.1, §3.5 and §5.2 cite the emitter lines as they are after #429
(code_generator.cpp +6, full_function_emitter.cpp +10).
FAITHFUL_TIMING_PLAN §5 gets a log entry for the rebase.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Rebased onto master: #417 (the old base), #418 and #420 merged on
2026-09-29. The status line says so, and every cited source line is
refreshed for current master.
- PR A is #429. §7.2 records its passed LLE land gate against live Beetle
(shell entry -126 OpenBIOS, -454 SCPH-1001; ruler #1 64/64, ruler #2
14/14). The residue is the IsC gap (§9, now with per-flush numbers and the
prototype result) and SCPH-1001's KSEG1 kernel entry 0xA0000500 (§3.3, -9),
which PR D closes (§8).
- §7.2 records the synthetic EXE's first Beetle runs (macOS oracle recipe:
#431). Disc boot works on both BIOSes; sideloading does not, because
Beetle's loader forces a KSEG0 start. With the probe fixes, Beetle reads
56/56/82 cycles for KUSEG/KSEG0/KSEG1, a 26-cycle surcharge equal to the
fetch model's.
- Ledger values follow the new probe layout (probe_run 0x000100F0; 30 fetch
tags; leaf 22 vs Beetle 11). Still 10 ids.
- ACCURACY_BURNDOWN points the KSEG1 fetch item at #429; FAITHFUL_TIMING_PLAN
gets a dated log entry.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With the delay-slot fix the Beetle deltas were sane but not comparable: the
three unrolled call sequences sat at different I-cache line offsets, and
probe_run shared a line with seeded, so the KUSEG run found its first line
already filled. Beetle read 59/59/89 (KUSEG/KSEG0/KSEG1), where the fetch
model predicts a 23-cycle KSEG1 surcharge.
Pad each call sequence so its first T2 read starts a line, and start
probe_run on its own line. Beetle (disc boot, OpenBIOS and SCPH-1001 alike)
now reads 56/56/82: KUSEG equals KSEG0, and KSEG1 exceeds them by 26 cycles,
exactly the Beetle fetch-model difference for the 15 KSEG1 fetches (60 vs a
cold cached 34).
probe_run moves to 0x000100F0. Ledger ids are unchanged; fetch-tag-segment
counts 30 tags and alias-fetch-coherence reads 22 vs Beetle 11, because leaf
now straddles two lines.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Timer 2 subtraction sat in the second lw's load delay slot, so it read
the stale register, and the difference was not masked to T2's 16 bits. The
Beetle disc-boot run of the probe returned 0xFFFFFFF3-style values for
every delta. Add a nop after the load and an andi 0xFFFF.
The layout shifts by six words: probe_run moves from 0x000100C0 to
0x000100D8. The ledger ids are unchanged; fetch-tag-segment now counts 27
tags and alias-fetch-coherence reads 10 vs Beetle 5 for the moved leaf.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
memory.c:1292 also reads g_debug_last_store_pc in every build: a GP0 write
from 0xBFC38B1C gets a RAM source key for the widescreen prim matching. That
key lies in SCPH1001's Shell window (runtime 0x80050B1C), so it has to be
re-keyed with the other six when the BIOS stamp moves to runtime_pc().
List it in §3.1, §5.2, §9 and the FAITHFUL_TIMING_PLAN entry, and give each
relocated key its runtime PC.
§8: PR C closes eight ledger ids (§5.3's seven plus segment-miss), so
C and D close all ten; A (#429) closed the eleventh gap,
bios-kseg1-fetch-charge, on master. B's store-PC hole ships with the re-key of
all seven keys.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The owner settled §10's four questions on 2026-09-29, all as recommended:
- I-cache tags follow Beetle; the hardware bit-31 difference is recorded
in ACCURACY_BURNDOWN axis 4.
- Segment misses in static game code interpret loudly until the title is
regenerated.
- Extra segments are requested through segment-qualified seeds.
- The overlay cache uses per-segment subdirectories.
The doc called g_debug_last_store_pc a debug/identity hook. It is not:
memory.c's RAM 0x0-0xF store filters compare it with exact PCs in every
build, the interpreter stamps the full PC, and #420 (ABI v24) forwards
overlay stamps to the host. §3.1 now lists it as a baked PC, and §5.2
routes it through runtime_pc(). Consequences:
- Option A, re-measured with the 1,261 store-PC stamps in the three R4
shards: +3.8 % / +27.6 % __TEXT (was +3.2 % / +27.2 %).
- New ledger id store-pc-segment (10 KSEG0 stamps in the synthetic EXE's
home bodies), closed by §5.3. The ledger now has 10 known gaps.
- §9: several filter keys are ROM addresses of BIOS code that runs
relocated in SCPH1001. The BIOS emitter stamps ROM addresses and the
interpreter stamps runtime PCs, so the keys must be re-keyed when the
BIOS stamp moves to runtime_pc().
§8 notes that rollout PR A (fix/uncached-fetch-per-insn, based on master)
is being prepared. A has since landed as #429, so the ledger does not
list bios-kseg1-fetch-charge.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Compiled code bakes KSEG0 into link values, CPS/IRQ resume PCs and I-cache
tags. KUSEG-linked EXEs run KUSEG PCs through those bodies (and their KUSEG
seeds are dropped), and overlay code executed at KUSEG/KSEG1 stays
interpreted after #417. Until #429, both emitters also charged the uncached
KSEG1 fetch only at cache-line leaders, so compiled BIOS ROM code was 4
cycles short on 5,477 of OpenBIOS's 9,592 KSEG1 instruction sites (Beetle
and the interp charge every fetch).
docs/SEGMENT_AWARE_CODE.md recommends per-segment compiled variants (code
identity = full VA, emitter segment as a per-compile constant, exact-segment
dispatch, per-instruction fetch in uncached bodies, segment-aware capture
and cache keys) over segment-relative emission, which measures +3.2 % /
+27.2 % __TEXT on R4 shards.
tools/segment_testrom/gen_segment_exe.py emits a synthetic KUSEG-linked
PS-X EXE that also works as an oracle probe. The new ctest
segment_aware_codegen runs the real recompilers on it and on OpenBIOS and
checks emitted PCs, the emitted dispatch lookup and fetch cycles (through
runtime/src/psx_icache.c) against a Beetle ReadInstruction transcription.
It passes only while the observed gaps equal its ledger (9 today).
#429 (rollout A) fixed the BIOS undercharge on master before this landed,
so KNOWN_GAPS does not list bios-kseg1-fetch-charge: the OpenBIOS check
finds no uncharged KSEG1 site and stays as a regression guard that reports
the id as a new gap. The last commit of this series brings the doc up to
date with #429.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Normal-mode discovery ran psxrecomp-game with no --project-root and ignored its exit code, so from a cwd without bios/ every producer silently lost its discovered entries (Tomba 2 shipped 54 native pairs instead of 81).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The merge of #427 (58bdf987) kept the fixture's 8-argument
interp_capture(...) call, but #424 had already grown the function to 11
parameters (origin_x, origin_y, redrawn). The fixture is compiled when the
test runs, so CMake configures and builds cleanly and
gl_scale_invariance_test then fails in about 0.2 s with "too few arguments
to function call, expected 11, have 8".
Pass 0, 0, 1 (origin at the capture rectangle, frame redrawn), as the
pre-merge integration of #424 and #427 did. With it the test passes on
macOS arm64 (Metal GL 4.1): lines, masks, captures and render passes at
1x-18x, the window mode and both clamps, 0 failures.
Test-only; no runtime change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The branch was rebased onto master 470f03b7, which now carries #418, so
the gate was re-checked on that tree without local merges. The
regenerated OpenBIOS and SCPH-1001 C is byte-identical to the gated
build's. Every boot anchor gives the same cycle (native - Beetle at the
shell entry: -126 on OpenBIOS, -454 on SCPH-1001). Ruler #1 matches Beetle
on all 64 passes, and ruler #2 on all 14 components on an HLE and an LLE
boot. uncached_fetch_charge and uncached_fetch_codegen_test pass. The
recompiler suite passes apart from the three environmental failures
master also shows on this Mac.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The land-gate entry said cyc_watch was armed from power-on on both
backends, but did not say how, and cited a local branch for the psx-beetle
build. The arming is PSX_CYC_WATCH / PSX_CYC_WATCH_N on psx-runtime and
psx-beetle (plus PSX_BEETLE_UNPACED=1 for a headless Beetle), documented in
docs/beetle-macos.md step 7 with the psx-beetle build; the entry now cites
both.
It also listed the shell entry as 0x80030000 on both sides. Both BIOS
profiles declare the shell window dispatch_key = "rom", so native keys it
at its ROM copy: the native anchor was 0xBFC0A500 (OpenBIOS) and
0xBFC18000 (SCPH-1001), and a native watch on 0x80030000 records nothing.
The numbers are unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SEGMENT_AWARE_CODE.md §7.2 gates this change on an LLE boot with per-anchor
cycle parity against psx-beetle to the shell, plus ruler #1. Both ran on
macOS, each twice: with OpenBIOS (SHA-1 95419841b5104d552b14810b1ecbe6c1358bcdf1)
and with the owner's own SCPH-1001 dump (SHA-1
10155d8d6e6e832d6ea66db9bc098321fb5e8ebf), the same image on both sides. Native
master and this branch were built with #418 merged locally, so the mult/div
flush could not mask the result.
At the shell entry, native minus Beetle goes from -2,165,591 to -126 cycles on
OpenBIOS and from -7,049,462 to -454 on SCPH-1001. The ROM boot path is exact
on both. Two known gaps explain what is left:
- IsC stores do not reach the I-cache model, so a FlushCache leaves native
kernel lines valid that Beetle refills. A local prototype of Beetle's IsC
tag write takes OpenBIOS to 0 and SCPH-1001 to -9. It is recorded as an
axis-4 item here, not part of this change.
- SCPH-1001 enters its copied kernel through the KSEG1 alias 0xA0000500. The
four trampoline words run uncached in Beetle and cached natively (-9). That
is the KSEG1-alias gap in SEGMENT_AWARE_CODE.md §3.3.
Ruler #1 on SCPH-1001 matches Beetle on all 64 passes, on master and this
branch. Ruler #2 stays exact on all 15 loops. Beetle also confirms the memcpy
cost the branch now charges: 39 cycles per byte during boot, and 42 per byte,
860 and 3,884 cycles per call in R4.
The land gate item is ticked.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The 2026-09-29 log entry marked the live-Beetle validation OPEN inside a
done item. SEGMENT_AWARE_CODE.md §7.2 makes it a precondition for landing:
an LLE boot with per-anchor cycle parity against psx-beetle to the shell,
plus ruler #1. ACCURACY_BURNDOWN now carries it as its own unchecked gate
item, and the timing-plan entry says the PR stays draft until it runs on an
oracle machine.
The entry also framed the change as boot-only. OpenBIOS runs its A0/B0/C0
services in place, and 148 of its 192 A0 entries point into ROM, so the
per-instruction charge also applies during gameplay. Measured in an R4
race: memcpy's byte loop goes from 25 to 42 cycles per byte, and R4's two
memcpy calls per game frame go from 2,824 to 4,744 cycles (about 0.17% of
the frame). An LLE boot of R4 reaches the game entry 2,934,910 cycles
(1.10%) later, identical on two boots of each build, with 0 dispatch misses.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The cached line-leader rule charges a fetch at every 16-byte line start of
the runtime PC. The BIOS emitter tested bits[3:0] of the ROM address instead
and assumed every copy window preserves them. OpenBIOS copies its kernel from
ROM 0x1FC1E4D4 to RAM 0x500, which shifts them by 4. Each kernel line
crossing was charged one instruction early, where it is a hit, and the real
crossing went uncharged. 1,152 kernel instruction sites differed from the
interpreter.
The test now uses the runtime PC that relocate_ra already computes. In the
OpenBIOS kernel, 823 misplaced fetch sites go and 773 are added at runtime
line starts. Retail profiles are unaffected: SCPH-1001's copy windows are
16-byte aligned and SCPH-101/5552 declare none. The uncached_fetch_charge
ctest now also replays every cached OpenBIOS run from a cold cache against
the interpreter's fetch path and the Beetle transcription.
R4 (OpenBIOS, recompiled kernel, HLE boot) reaches the game entry 6,013
guest cycles later than with the uncached fix alone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Beetle's ReadInstruction never fills an I-cache line for a fetch at
0xA0000000 or above: every such fetch costs +4 and clears the pending load
give-back. The dirty-RAM interpreter charges that by fetching at every PC.
Both static emitters emitted psx_icache_fetch only at cache-line leaders,
which is exact for cached code only. In OpenBIOS, 5,477 of the 9,592
in-place ROM (KSEG1) instruction sites were uncharged: 21,908 cycles short
per pass through that code, each also missing its give-back clear.
Both emitters now charge a fetch before every instruction whose runtime PC
is uncached. The predicate is psx_fetch_uncached in psx_instr_cost.h, which
psx_icache.c now uses too. The A0/B0/C0 call-vector native stubs charge one
fetch per executed word, so a KSEG1 call pays like the interpreter. Cached
code keeps the leader rule. R4's regenerated game C is byte-identical, and
the OpenBIOS diff is insertions only. psx_instr_cost.h joins the codegen
hash list, since the emitters fold it into emitted code.
Tests:
- ctest uncached_fetch_charge runs psxrecomp-bios on OpenBIOS and compares
per-instruction fetch charges three ways: the compiled fetches, the
interpreter's fetch path (psx_icache_fetch_interp) at every PC, and a
Beetle transcription.
- uncached_fetch_codegen_test checks the game emitter at KSEG1 and KSEG0.
Both tests fail on master. This closes bios-kseg1-fetch-charge in the
segment_aware_codegen ledger (#419).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
beetle_main.cpp has defaulted to port 4382 since 4380 began colliding
with other oracles, but its header and the psx-beetle block in
runtime/CMakeLists.txt still said 4380. beetle_debug_server.c now says
4380 is only its fallback for a port <= 0. Comments only.
A correction to 9ef516a7 ("psx-beetle: build on macOS and send large
replies whole"): its message says none of its changes affect MinGW or
Linux. The send_raw change does. On EWOULDBLOCK every platform now waits
for the socket (up to 15 s) instead of dropping the client mid-reply.
That only turns a truncated reply into a complete one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The 2026-06-27 MMIO entry in FAITHFUL_TIMING_PLAN.md says the 13 prior
loops were unchanged when mmio_timer and mmio_spu were added. icache_miss
was not: the two new loops sit before it and move its loop top from
0x80010144 to 0x80010170, so each of its two misses costs one cycle more
and it reads +16 on both backends (re-measured against live psx-beetle; the
cycle_testrom README already explains the shift). The entry now carries a
dated correction, and the beetle-macos.md intro no longer points readers
at +14 as the current value.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Boot anchors (reset, the kernel copy, the first A0/B0/C0 calls, the shell)
pass within the first frames, before a TCP cyc_watch arm can land, so a
native-vs-Beetle boot comparison could not be run from the tree.
psx-runtime (debug_server.c) and psx-beetle (beetle_main.cpp) now read
PSX_CYC_WATCH ("<pc>" or "<pc>-<end>" for region mode) and PSX_CYC_WATCH_N
(default 16) at start-up and arm the watch before the first guest
instruction. The native arm shares the TCP handler's code path
(cyc_watch_arm); psx-beetle calls beetle_cyc_watch_arm. With neither
variable set nothing changes. tools/cycle_compare.py --no-arm then reads
both rings.
PSX_BEETLE_UNPACED=1 skips psx-beetle's 59.94 Hz wall-clock pacing for
headless oracle runs. Guest timing is unaffected.
docs/beetle-macos.md step 7 documents the variables and the anchor-key
difference for BIOS copy windows declared dispatch_key = "rom": the shell
is keyed natively at its ROM address (OpenBIOS 0xBFC0A500, SCPH-1001
0xBFC18000) while Beetle sees 0x80030000. Checked on macOS: the documented
example records the OpenBIOS shell entry at 5,015,670 in Beetle, the value
of the #429 land gate, and the native watch arms from boot.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Shokunin