docs/SEGMENT_AWARE_CODE.md §5.7. compile_overlays.py splits each capture
into one view per segment with entries (capture_segment_views). A v2
capture has no dispatch_entry_segments and comes back unchanged, KSEG0 only;
in v3, a dispatch entry that no segment list names is a KSEG0 entry. A view
spells load_addr and every PC in its segment, holds only that segment's
dispatch entries, and carries all the execution evidence, which has no
segment. A malformed segment record fails that capture loudly.
Two kinds of record also get a view where they saw no dispatch entry:
- a record with no dispatch entry in any segment (execution evidence only,
which the runtime writes) is one KSEG0 view, exactly as a v2 reader reads
it, so classification, forced interiors, declared entries and prior
manifests apply as before;
- declared_view_segments(): a --force-interior PC inside the record (KSEG0)
asks for the KSEG0 view, and a game.toml [[overlays]] table asks for the
segment its load address, or else each of its entries inside the record,
is spelled in.
Each view compiles like any capture, region and fragment passes alike:
- the image is compiled at `segment | phys`, so the recompiler names and
bakes it in that segment;
- helpers that spell PCs without an image address use the view's segment
(image_segment(); KSEG0 by default, so a KSEG0 compile is unchanged);
- the shard lands in segment_cache_dir(): the tag directory for KSEG0,
seg-kuseg/ and seg-kseg1/ otherwise.
The manifest writer adds `S <segment>` for KUSEG/KSEG1. The runtime-contract
parser keeps F entries' full VA, applies the S rules, and refuses a
manifest whose segment differs from its directory's. Exports are looked up
by their full VA.
Capacity accounting counts each leaf's segment directories in the leaf's
namespace, since the runtime's candidate table is shared, and tells shards
apart by segment and name. The static generator emits the exact PC per row,
so KUSEG, KSEG0 and KSEG1 rows of one word coexist, with a segment gate
over the segments it compiled. A KSEG0-only table emits the same text as
before. Non-KSEG0 static namespaces are ov_s0_ (KUSEG) and ov_s5_ (KSEG1).
The static isolated-fragment pass keys its images by segment too, so each
(entry, image) demand compiles against its own segment's view and one entry
dispatched in two segments gets a fragment in each. --force-interior PCs
stay KSEG0 and force KSEG0 views.
coverage_vault.py unions dispatch_entry_segments when it merges variants,
making a v2 record's entries explicit KSEG0 entries, and crops it when it
compacts regions.
Output for v2 captures is unchanged: the 52 R4 shard sources and manifests
compiled from D's race captures are byte-identical to D's.
Tests:
- new overlay_segment_keys (CTest). It runs capture views, manifests and the
cache layout, then the real compile_overlays.py driver, in process, on a v3
capture of an overlay entered at KUSEG and KSEG1, and checks:
- each shard lands in its directory with S and full-VA F records;
- every baked PC is in its segment, and KSEG1 charges a fetch per
instruction;
- a second run is served from each segment's cache;
- a KSEG0 view compiles byte-identically to the same bytes captured as v2;
- --static emits per-segment rows and namespaces, and an isolated
fragment per segment for an entry with no callable boundary;
- a dispatch-less v3 record with --force-interior compiles byte-identically
to its v2 reading, and a KUSEG-only record still gets its forced KSEG0
fragment;
- a KUSEG/KSEG1 view compiled into a cache holding a KSEG0 manifest of the
same bytes merges none of it, and the empty-primary reconciliation counts
only the view's own segment's fragments;
- the real loader runs the shards: links, store-PC stamps and fetch tags in
each segment, fetch cycles equal to Beetle's model, KSEG0 interpreted.
- coverage_vault: segment unions and crops.
- compile_overlays_static_split: a KUSEG row beside a KSEG0 row dispatches
by exact PC; unmapped-segment entries are refused. This replaces the
KSEG0-only rejection.
- aot_overlay_discovery: the capacity namespace lists the segment
directories.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
merge_cache used a flat listdir() against a nested cache
(<compiler>/<arch-abi>/cg<N>_<hash>/) so it matched nothing — the vault's
DLL mirror has been empty since the tool shipped. Walk the tree and keep
relative paths: the same content-keyed filename exists under different cg
dirs with different compiled bytes, so flattening would mix emitter
generations. stats() walks recursively to match.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Overlay coverage (overlay_captures.json + the gcc-compiled cache DLLs) normally
lives only in gitignored build dirs, so a `cmake` clean or fresh worktree wipes
it (re-derivable only by replaying those areas). coverage_vault.py maintains one
additive vault OUTSIDE any build dir: `merge` unions captures by variant
(load_addr + bytes hash; same variant unions executed/entry PCs, distinct
variants all kept) and copies content-keyed cache DLLs (copy-if-absent/newer).
Wired into Tomba's overlay_autocompile_cmd so every compile backs up additively;
also runnable by hand to consolidate scattered build dirs. The vault holds
game-derived bytes, so it stays private/gitignored (this script does not).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Shokunin