docs/SEGMENT_AWARE_CODE.md §5.7 (decision 4). #417 made every KUSEG and
KSEG1 overlay dispatch fall to the interpreter, because every shard was
compiled for KSEG0. The loader now keys shards by segment, so a shard
compiled for KUSEG or KSEG1 runs for PCs of that segment.
Cache layout. KSEG0 shards keep their directory and {phys}_{crc} names, so
no existing cache is invalidated. KUSEG and KSEG1 shards live in the cache
tag's seg-kuseg/ and seg-kseg1/ subdirectories, under the same names; both
tiers scan them, and a shard's segment is its directory's
(cache_seg_from_path).
Manifest. A KUSEG or KSEG1 manifest starts with `S <segment>` (00000000 or
A0000000) before its first F. F entries are full VAs in that segment; the
parser no longer forces them to KSEG0. R records stay KSEG0-spelled byte
extents. Without S a manifest is KSEG0, as before. Refused, when indexing,
when loading and through live publication alike:
- a manifest whose segment differs from its directory's;
- an F entry outside its segment;
- S after an F, S twice, or S that is not a RAM segment base.
Dispatch. Each candidate records its segment. The physical indexes stay (the
bytes are one identity), and every lookup matches the PC's segment:
- the entry chain (idx_head_seg);
- CPS continuation ownership;
- the lazy manifest index and try_load_region;
- the interpreter's native-call contract.
The lazy-miss and range-owner memos are keyed by the full PC. A PC in a
segment that maps no RAM never runs a shard. Exports are resolved by their
full VA.
Counters. segment_alias_interp (from #417) now counts KUSEG/KSEG1 dispatches
the loader leaves to the interpreter because no shard of their segment is
cached or valid yet. The new segment_native counts those a shard of their
own segment runs. Both are in the TCP overlay_loader_status reply. A warm
cache takes the first to 0.
Full-VA compares:
- the overlay idle note's return test compares the resume PC with $ra
exactly, and counts it as internal only for a running shard of its segment;
- the shadow diff's two "returned to the caller" tests compare the exact PC;
- the diff mode's kernel-window exclusion tests the physical address, so
KSEG0 and KUSEG kernel candidates are treated alike. Before, it tested the
full PC, so only a KUSEG PC ever matched, and #417's gate never let one reach
it. This is opt-in (PSX_OVERLAY_DIFF).
Tests (loader harness, overlay_segment_gate):
- a KUSEG shard runs natively for its KUSEG PCs, entry and continuation,
while KSEG0/KSEG1 aliases are interpreted;
- one shard per segment for the same word, preloaded and lazy;
- a stale own-segment shard is interpreted rather than handed to another
segment's;
- a KUSEG continuation where KSEG0 has an entry at that word, and a lazy one
whose competing KSEG0 bundle is newer;
- refused manifests, including through live publication;
- the native-call contract;
- a source check that the interpreter records the segment from the full PC,
and that no site clears dispatch evidence without its segment bits.
The harness gains a segment-run scenario that drives real compiled shards
through a CPS dispatch loop. overlay_init_guard and overlay_retry_c11 follow
the per-segment lookup, and so do the disabled interpreter_perf_guards'
negative-miss and try_load_region checks (it still stops where it did on D).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docs/SEGMENT_AWARE_CODE.md §5.7. Overlay bytes are segment-free, but a PC
is not: a KUSEG, KSEG0 or KSEG1 entry of the same word links, traps, fetches
and stamps store PCs in its own segment, so each needs a shard compiled for
that segment. Capture kept only the physical word, so every shard was
compiled for KSEG0 and #417's gate interpreted the other segments forever.
dirty_ram_interp.h adds one sibling bitmap per segment that maps RAM
(g_dirty_ram_dispatch_seg_bitmap[3]; 768 KiB for the 8 MiB backing).
dirty_ram_dispatch_inner sets the bit for the full PC next to the existing
any-segment bit, once per interpreted dispatch. A PC in 0x20000000-0x7FFFFFFF
or KSEG2 addresses no RAM in Beetle and gets no bit.
dirty_ram_dispatch_evidence_clear() drops both together at every site that
dropped dispatch evidence: boot reset, a store into an executed page, DMA
preservation and the autocapture epoch. Snapshot jobs copy the siblings.
Capture schema v3 adds, per region,
"dispatch_entry_segments": {"kuseg": [...], "kseg0": [...], "kseg1": [...]}
with each PC spelled in its segment. dispatch_entry_pcs keeps its v2 meaning
(dispatched in any segment) and KSEG0 spelling, so existing readers are
unaffected. A record without the field is KSEG0 only.
Guest behaviour is unchanged: the bitmaps are host-only capture evidence and
are not saved in savestates. overlay_capture_retry_test checks the v3 record
through both writers (the background epoch and the synchronous final
capture), including that a cleared epoch's segment does not outlive it.
psx_ram_runtime_map_stubs.c defines the new bitmap, which memory.c's write
paths now reference, so psx_ram_runtime_map_test still links on GNU
toolchains. The disabled interpreter_perf_guards check follows the evidence
clear (it would otherwise contradict overlay_segment_gate's wiring check).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Three-layer faithful-core fix for the exception-return contract, root-caused
live over TCP on a deterministic dbg-build repro (vblank authority vetoed
forever while the guest spun in the kernel idle loop, in_exception stuck 1):
1. Interpreter RFE backend parity: dirty-RAM RFE now calls
psx_rfe_mark_escape() exactly like the recompiled opcode and every
overlay shard (ABI v12), and dirty_ram_dispatch_inner takes a pending
escape at the committed transfer before local dirty flow can keep
interpreting the resumed code inside psx_check_interrupts' window.
2. Legacy-sentinel deliveries arm the escape too: the old blanket decline
assumed a sentinel delivery always resumes AT the sentinel, but guest
TCBs never hold the sentinel (Fix B anti-leak), so a kernel
ReturnFromException restore resumes at an approximate real PC with no
exit path armed. The take site preserves the legacy contract: a resume
landing ON the sentinel still exits through the sentinel detector.
3. Same-thread completion of a sentinel delivery via explicit RFE /
ReturnFromException converts to the legacy exit (restore interrupted
GPRs, pc=0, continue the live host chain) keyed on the structural
entry_tcb == exit_tcb test - otherwise the approximate TCB PC surfaces
up the trampoline, severs the live chain, and the run dies one hop
later on an un-re-enterable mid-function PC ("execution completed").
Validated: production build passes the previously-fatal splash transition
(frames 4717+ at 60fps, was deterministic death at 1385 x3 runs); dbg build
advances past the wedge (was frozen at exactly 1385); all runtime tests +
structural guards pass (guards extended to pin the interp RFE contract).
Known follow-up: the PSX_DEBUG_TOOLS build still loses one kernel CD event
through a third timing path at the same site - tracked as an open
foundation item; production config unaffected.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Audit fixes for the async shard-publication pipeline (45a2dda):
- autocompile_shutdown(): stops the pipeline, kills the compiler tree via a
kill-on-close job object (assigned while the child is SUSPENDED so cmd.exe
cannot spawn grandchildren outside it), and JOINS the watcher + preparer —
never abandoning a thread inside the Windows loader lock at ExitProcess.
Wired first into shutdown_runtime(). Crash exits reap the tree via the job.
- Failed off-thread prepares retry (3 attempts, 100 ms) on the worker before
the artifact silently reverts to a synchronous emu-thread first-map; retry
and giveup counters exposed in autocompile_status_json.
- Capacity caps fail closed and LOUD: cand_register shouts once at CAND_CAP;
load_one_dll_prepared refuses at MAX_LOADED_DLLS before registering, so
dll_already_loaded can never lose track and double-register on rescan.
- Publication confinement is filesystem-resolved: published_path_valid
canonicalizes the cache root and the pipe path (GetFinalPathNameByHandle /
realpath), prefix-checks canonical forms, and the worker maps the canonical
string while holding a no-delete-share guard handle (TOCTOU). `..`/junction
spellings that passed the lexical check are now rejected — proven by the
new overlay_path_canon_test boundary test.
- test_autocompile_publication gains a shutdown-mid-run scenario asserting
prepared == committed + discarded with nothing queued; perf guards updated
to require validate -> guard-handle -> canonical first-map ordering.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Shokunin