A PS-X EXE header may carry KUSEG addresses (Alien Resurrection SLUS-00633,
Kula World SCES-01000); the BIOS then runs the program at KUSEG PCs, so its
links, EPCs and I-cache tags are KUSEG. The parser folded such a header to
KSEG0, so the whole program compiled for a segment it never runs in, and
the dispatch lookup matched any segment by physical address
(docs/SEGMENT_AWARE_CODE.md §3.2, §5.3, §5.5).
- EXE parser: keeps the header's addresses. link_segment() is the load
address's segment; the load address and entry must share it and lie in
the 8 MiB RAM window of KUSEG, KSEG0 or KSEG1 (KSEG1 compiles as an
uncached home segment). phys_load_address()/contains_phys() do byte
addressing; the BSS overlap check and the text_size bound are physical.
- function_analysis: the three JAL scans take the target's top bits from
the delay slot's PC instead of assuming KSEG0 (unchanged for KSEG0).
- Seeds are range-checked by physical address. A seed in the link segment
is an entry (0x000100E4 was silently dropped before). A seed for the same
bytes in another segment asks for a segment variant (§5.4): it is listed
in a WARNING, never folded into the home body. Seeds outside the image are
counted. The producer_range, cross_call_allow and hosted_interior
directives have no variant form: a foreign spelling is refused with the
link-segment one instead of a bare "invalid" error.
- Config code sites that are matched exactly (widescreen and cull sites,
mod hooks, hot/data-shard funcs, the VSync-query function and its
event-horizon return PCs, persisted options) and name the image's bytes
in another segment are refused with the spelling that matches, since they
would now match nothing. Kinds matched by physical address (byte patches,
the full-word-guarded cull keep/angle/aspect-cone and signed-X-bound
sites, dome call sites) reach the same bytes in any spelling and are not
refused. Overlay compiles are exempt until PR E.
- tools/collect_game_misses.py wrote every dirty-RAM miss as a KSEG0 seed,
which a KUSEG-linked image now reports instead of compiling. It writes
them in the link segment (--segment, --game-toml, or a segmented text
range; it refuses to guess from a physical one) and adds the
segment_misses rows at their full PC.
- Dispatch source moves to game_dispatch_emitter.cpp. The lookup indexes
the physical word, then requires row.addr == addr; a second row for one
word is an error until variants share the table. psx_game_address_in_text
stays physical, so a segment miss takes the clean-text-miss path (the
previous commit records it).
For a KSEG0 title the game C is byte-identical; its dispatch file changes
only in psx_game_find_entry (the exact-PC compare) and its comment.
Tests:
- segment_aware_codegen closes link-segment, fetch-tag-segment,
irq-resume-segment, resume-pc-segment, store-pc-segment,
home-seed-accepted, alias-fetch-coherence and segment-miss; they stay as
regression guards, and resume-pc-segment now also checks the dispatch
rows' keys and resume PCs.
- kuseg_dispatch_lookup: KUSEG rows, names and baked PCs; aliases miss while
text stays physical; variant seeds reported; a function reached only by
jal and a SaveState-style continuation are discovered in a KUSEG image
(kills the two discovery scans reverted to KSEG0; the third, the
invalid-JAL data heuristic, reads the same bits either way); mixed-segment
headers rejected; a KSEG1 image charged per instruction; foreign config
sites refused; the RAM-mirror contract as before.
- kuseg_dispatch_lookup also compiles the lookup's binary-search form (a
table spanning 2 MiB or more) and queries every segment's alias of each
row; refuses a KSEG0 event-horizon site and KSEG0 seeds directives with
their KUSEG spelling; accepts a KSEG0 signed_x_bound site and applies it.
- emitter_runtime_pc_test: the dispatch rows follow runtime_pc() and keep
the compile identity in func_ names (kills B's two surviving mutants: 76
and 12 failing checks); two rows on one physical word are an error.
- collect_game_misses (new, tools/tests): link-segment seeds from each
source, segment misses at their full PC, no guessing, no duplicates,
against a fake server using the runtime's own summary row format.
- reachable_discovery_test: the parser keeps KUSEG, accepts KSEG1, rejects
mixed segments and non-RAM load addresses.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The bounded-jump-table resolver requires the guest's own bounds check
(`sltiu/beq`) and index scaling (`sll`), because those are what prove the
table's extent. A decoder that keeps its state as a pre-scaled byte offset
and dispatches through `lui/ori base; addu base,offset; lw; jr` has neither,
so the jr fell back to a bare CPS tail-transfer and every entry it reached
that was not already a dispatch key ran on the dirty-RAM interpreter. On
Breath of Fire III that is the file decoder's state dispatch at
`0x80164AB0` (five entries; the first entry is the word right after the
table), the last boot-EXE row a play session still interpreted after the
computed-stride resolver.
resolve_self_limited_jump_table recovers the extent from the table's own
layout instead: words are taken from the base while each is a 4-aligned
in-function code address outside every delay slot, and the table ends
where its lowest target begins, since a pointer table cannot overlap the
code it points at. As with the stride resolver the emitter switches on
the actual runtime target with the CPS tail-transfer as default, so the
recovered entries only add native cases; an index past the recovered
extent still takes the fallback. Wired as the third fallback at both
code-generator sites and in the exact-entry discovery walker.
On the title's boot EXE it matches exactly one site (five entries); every
other shard is byte-identical. The synthetic test gains a second function
with this shape.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A `jr R` with `R = base + index * stride` and no table in memory is how
unrolled copy loops enter their body in the middle (Duff's device). The
bounded-jump-table resolver only knows the `sltiu/beq; sll; addu; lw; jr`
chain, so the emitter fell back to a bare CPS tail-transfer; the trampoline
then found no dispatch key at the interior PC (straight-line code, no block
leader) and the dirty-RAM interpreter ran the rest of the copy on every
call. On Breath of Fire III that is the file decompressor's literal-copy
loop (`0x80164D1C`, 32 groups of `lbu/nop/sb`), which fires on every load
and was the whole non-kernel interpreted residual of a play session once
the kernel patch ranges (#346) and the call vectors (#381) ran native.
resolve_computed_stride_jump walks the chain back from the jr through
nearest definitions in straight-line code: `addu R, B, S` with S from
`sll S, I, k` or the two-sll sum `sll P,I,a; sll Q,I,b; addu S,P,Q`, and
then requires the code after the delay slot to be N >= 2 shape-identical
groups of `stride` bytes (same opcodes and register fields, immediates
free, no control flow inside a group). Targets are run_start + k * stride
for k = 0..N. The base register's value is never assumed: the emitter
switches on the actual runtime target with the CPS tail-transfer as the
default, so the recovered cases can only add native entries, never
redirect a jump that lands elsewhere. Wired into both code-generator
sites (interior labels + the switch) and the exact-entry discovery walker.
On the title's boot EXE it matches exactly one site, the decompressor loop
(33 entries); every other jr is untouched.
Adds recompiler/tests/test_computed_stride_jump_codegen.py (synthetic
PS-EXE with the idiom; asserts the switch, every case and the interior
labels), registered with CTest.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The overlay capture format DELIBERATELY appends one coherent guard instruction
past the end of a dirty-page run (overlay_capture.c write_json_window, `size
+= 4u`, pinned by runtime/tests/test_interpreter_perf_guards.py:47): a MIPS
branch sitting at the run's final word (...FFC) always executes its delay slot
in the next page (...000), and the shard must be able to emit and hash that
word. So a capture whose size is a page multiple PLUS 4 is a CORRECT capture --
N bytes of overlay plus one guard word. Measured: 36 of 36 Ape Escape
(SCUS-94423) vault records and 846 of 946 Tomba 2 (SCUS-94454) records are that
shape.
The recompiler was never taught the asymmetry: the guard word is a legal
DELAY-SLOT SOURCE but an ILLEGAL BLOCK LEADER, because the word after it does
not exist in the image. It had exactly one bound -- end_address() -- used both
to read words and to decide what is code.
MEASURED on Ape Escape, its own single-source vault at framework
v0.3.2-alpha-139-g1bf70960, 36 variants:
PSX_SHARD_RESULT ok=50 failed=17 skipped=32 capacity_fastpath=0
! [recompiler] overlay 0x80136000 crc 4BAB23C9: what(): cannot emit control
flow at 0x80138000: mandatory delay slot 0x80138004 is outside the input
image
+ 16 x [fragment] interior 0x801360.. @region 0x00136000, identical message
One root cause, seventeen shards. The whole region then falls back to the
dirty-RAM interpreter forever, and nothing at runtime says so.
Causal chain, every step read in source:
1. overlay_capture.c:349-361 appends the guard word; region 0x80136000 gets
size 0x2004, so the image is [0x80136000, 0x80138004).
2. compile_overlays.py make_psxexe wraps those bytes verbatim with text size
= len(data), so PS1Executable::end_address() = 0x80138004.
3. function_analysis.cpp in_exe / in_function admit 0x80138000 as ordinary
code, and the walk reaches it by fall-through from 0x80137FFC
(0x24022000, addiu -- not control flow), so func.end_addr becomes the
image end.
4. control_flow.cpp makes the final block end_addr = func.end_addr - 4 =
0x80138000: the block LEADER is the guard word.
5. control_flow.cpp's "Also check last instruction itself" promotion finds
0x80137FFC is not control flow and therefore promotes read_word(end_addr)
-- the guard word 0x1062005A, a beq -- to block.exit_instr, with
has_delay_slot = true and address == end_addr. THIS IS THE DEFECT.
6. code_generator.cpp computes delay_slot_addr = 0x80138004, read_word
returns nullopt, and it throws.
THE THROW AT STEP 6 IS CORRECT AND IS PRESERVED. Emitting a control transfer
without its architectural delay slot silently corrupts guest state, which is
what the code did before that throw was added. The gap is step 5.
Fix: give the image view two distinct bounds.
end_address() READ bound -- every byte the image supplies, so a
mandatory delay slot living at a guard word resolves.
analysis_end_address() ANALYSIS bound -- one past the last byte that may be
discovered, made a block leader, or made a block's
exit instruction.
read_word() keeps the READ bound. Discovery and block construction move to the
ANALYSIS bound: function_analysis.cpp in_exe / in_exe_p3 / add_forced_entry /
every hard_cap that used to be the image end / the jr-table producer_hi
defaults / the static jr-ra and JAL sweeps; control_flow.cpp gains
analysis_walk_hi(func) and uses it for cfg.function_end, alias entries,
add_boundary, the scan loop, the final block's end_addr and in_func;
code_generator.cpp's mid-function split pre-pass; main_psx.cpp seed admission
and the data-scan pointer promotion; analysis_db.cpp, main_toml.cpp,
main_cli.cpp target/seed admission. The clamp is applied only when it leaves a
non-empty walk, so a degenerate function starting at or past the bound keeps
failing closed rather than silently producing an empty CFG.
HOW THE BOUND IS DERIVED -- the producer STATES it; the recompiler is told.
The alternative was for the recompiler to infer the count from `size % 4096 ==
4`. That fits every capture the current writer has ever produced, but it is a
consumer guessing at a producer's intent, and this tree already has the
counter-example: 52 of the 946 SCUS-94454 records have residues from 112 to
3996 bytes (one not even 4-byte aligned) because they come from the STATIC
extractor, which records an exact file extent and appends no guard word. A rule
keyed to the page-run shape is a rule that shifts the moment a producer
changes.
So the count travels with the image, in a magic-tagged field in the otherwise
zero-filled tail of the 2048-byte PS-EXE header (exe_tag in ps1_exe_parser.h:
"PSXRGRD1" at 0x7E0, uint32 count at 0x7E8). Putting it in the header rather
than on the command line means every consumer gets it automatically and no
future invocation path can forget to pass it -- compile_overlays has three
make_psxexe sites and five recompiler spawn sites. An absent or malformed tag
means ZERO guard bytes, which is exactly what every genuine PS-X EXE is: the
BIOS and every retail main executable have no guard word, their analysis bound
equals their read bound, and their analysis is bit-for-bit unchanged.
Producer side:
overlay_capture.c writes "guard_bytes": 4|0 per region -- the writer
stating its own intent, additive to the v2 schema and
preserved through coverage_vault.py compaction.
compile_overlays.py capture_guard_bytes(): prefer that field; for captures
recorded before it existed, reconstruct the single
shape write_json_window can produce (page multiple + 4)
and return 0 for everything else, which is the
fail-closed answer -- an unrecognised guard word is
analysed as code and the step-6 throw still fires
loudly at generation time.
make_psxexe guard_bytes is keyword-only with NO default, so a new
call site is a TypeError, not a silently mis-analysed
shard. The count is threaded through
compile_fragment_batch / compile_interior_fragment /
generate_interior_fragment_static and into the fragment
fail-memo key (bumped to v3, so a memo recorded under
the old analysis cannot suppress a retry under the
corrected one).
SECOND-ORDER DEFECT FIXED IN THE SAME PASS. generate_ranges_manifest extends a
range by +4 for any block whose exit instruction IS its last word, so the
cloned delay slot joins the candidate CRC and the page-generation watch. That
extension was unclamped. Reachable TODAY, independent of the guard word: a
branch-likely opcode (0x14-0x17, RESERVED on the R3000A) as the image's very
last word is short-circuited by translate_basic_block into an inline RI raise
BEFORE any delay slot is read, so generation succeeds and the manifest claims 4
bytes past the image end. Measured on origin/master with a 28-byte synthetic
image: `R 80010000 20` for an image ending at 0x8001001C. The cache would then
validate the shard against bytes the shard never saw. Now clamped to the READ
bound -- not the analysis bound, because a guard word genuinely IS compiled in
when it serves as a delay slot and must participate in the CRC.
DEAD CODE, documented rather than trusted: basic_block.cpp's BasicBlockAnalyzer
bounds its walk correctly but nothing in the tree constructs it. It is still
compiled and still hashed into the codegen tag, so a change there invalidates
caches without moving an emitted byte. Its bounds move to the analysis bound
too -- wiring it up must not reintroduce this defect -- and a header comment now
says where the live walker actually is.
TESTS. Both registered, so ctest runs them and the configure-time
registration guard is satisfied.
recompiler/tests/test_overlay_guard_word_analysis_bound.py drives
psxrecomp-game over five synthetic images:
A tagged, 2 pages + guard word, last real word not control flow, guard word
IS a beq -- the measured Ape shape. Must generate; the guard word must not
be emitted as an instruction; the block must publish 0x80012000 for the
runtime to dispatch; no range may claim it.
B byte-for-byte the same payload with NO tag. Must still fail closed with
the mandatory-delay-slot diagnostic naming 0x80012000. A and B differ only
in the header tag, which proves the tag is what changed behaviour and that
the throw was not weakened.
B2 a misaligned declaration (6). Must be rejected and behave like B.
C tagged, with a real branch at the last analysable word. The guard word
must be emitted AS the delay slot and must be covered by the manifest.
D branch-likely as the final word. Generation must succeed and no range may
exceed the image end.
Against origin/master's psxrecomp-game the same file reports exactly the two
things this commit changes, and passes B, B2 and C:
FAIL: tagged guard word still fails generation: what(): cannot emit control
flow at 0x80012000: mandatory delay slot 0x80012004 is outside the
input image
FAIL: range [0x80010000,0x80010020) claims 4 byte(s) past the image end
0x8001001C
tools/tests/test_overlay_guard_bytes_declaration.py pins the producer half:
field precedence, the reconstruction table, rejection of nonsense
declarations, the tag's byte offsets, that make_psxexe with guard_bytes=0 is
byte-identical to the pre-fix wrapper, and that overlay_capture.c both appends
the guard word and now declares it.
MEASURED OUTCOME, Ape Escape, same vault, same flags, fresh cache:
before PSX_SHARD_RESULT ok=50 failed=17 skipped=32 capacity_fastpath=0
after PSX_SHARD_RESULT ok=31 failed=0 skipped=57 capacity_fastpath=0
ok falls because 19 isolated island fragment shards at 0x00136000 are no longer
needed once the region variant compiles as one shard. Coverage is what matters,
and it is measured from the DLL exports: 349 -> 361 unique func_XXXXXXXX
entries, ZERO lost, 12 gained -- 0x80136140, 0x8013614C, 0x80136154,
0x80136238, 0x801362D4, 0x801362DC, 0x801362F0, 0x80136324, 0x80136370,
0x80136400, 0x80136480, 0x80137950, all from the failure list. The other four
failing PCs were already served by a sibling variant's shard.
INERTNESS.
BIOS (SCPH1001, psxrecomp-bios --config bios/SCPH1001.toml): all four
generated artifacts BYTE-IDENTICAL, verified by sha256 rather than by exit
code, because tools/regen_bios.sh exits 0 on failure.
Tomba 2 main EXE (psxrecomp-game --config game.toml): all nine generated
artifacts BYTE-IDENTICAL, and the console log is identical too.
Tomba 2 overlays, 82 variants over 6 regions from its own vault: identical
result line (ok=98 failed=0 skipped=103) and identical shard identity set,
4901 unique func exports on both sides -- zero gained, zero lost. 97 of 98
range manifests are byte-identical. The one that differs is region
0x8010A000 variant A32DB188, size 0x2004, whose last function's range goes
from 0x218 to 0x214 bytes: its guard word (0x24840008, addiu) had been
emitted as an ordinary instruction and hashed, and now is not. That is
precisely the intended change and nothing else moved.
Bead beads-eio.3.100.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The legacy extent estimate bounded every function at the next discovered
start and ran the undefined-opcode density heuristic over that whole span.
Psy-Q/Capcom titles (Mega Man X4) place inline data / jump tables directly
after real functions; the trailing data inflated the measured extent and the
density check then misclassified the real function as a data section —
emitting psx_unknown_dispatch stubs for directly-called code (X4: the whole
0x800EE3A4.. text tail, ~7,373 entries, became stubs and boot fail-fasted).
Walk each function's control flow from entry to its real terminus (mirroring
the proven analyze_exact_entries overlay walk, incl. Psy-Q jr-table
resolution). Provably-clean bodies (return/tail exit reached, zero invalid
primary opcodes, all indirect jumps resolved in-function) are bounded at
their terminus and marked code; everything else falls back to the legacy
boundary + density heuristic, which still rejects genuine data tables minted
as starts.
CODEGEN-AFFECTING: function extents change emitted C. Land games via the
coordinated regen cascade (regenerate every title against this emitter before
shipping; stale main images / overlay caches must be rebuilt).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Ape Escape forward-port (e0c819c) added pre-prologue setup detection to
function_analysis (find_preprologue_setup_start / preprologue_window_is_valid +
is_bios_dispatch_thunk), extending function STARTS backward to absorb preceding
lui/lw global-load setup. On Tomba this mis-bounds a function in the Village-of-
All-Beginnings init path (function count 5562 -> 5492, ~70 functions absorbed),
producing a main-loop softlock: New Game / loading the Village save wedges with
the main loop spinning on the 0xCF0 SIO poll, never rendering the area (a "blue
screen"). Backend-independent (gcc + tcc), not overlay/interp related.
This worked in v0.5.x (pin 5007c46, pre-AE). Revert function_analysis.{cpp,h} to
that state. Confirmed: the 0xCF0 wedge signature disappears (repro cf0_frac
1.00 -> 0.02) and Tomba boots/plays again. Ape Escape is not a public release, so
rolling back its discovery changes is safe; MMX6's existing release is unaffected
(its binary was already built), but a future MMX6 rebuild should be re-verified
against this discovery. A follow-up can re-introduce the pre-prologue heuristic
correctly (game-gated or with a tighter boundary check) without the over-extension.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Coverage campaign step 2. Interior code addresses reached through
function-pointer tables (e.g. Tomba's entity event pumps: 0x8004DF00
family inside func_8004DD1C, table at 0x80013F18) previously dispatch-
missed into the dirty-RAM interpreter, and seeding them truncated the
host function (mid-function-seed softlock class).
Discovery (main_psx): iterative loop classifying every candidate entry
against the current function set — function start: covered; interior:
ALIAS entry (host never capped); gap: forced entry (explicit seeds
trusted; scanned pointers need jr-ra/prologue boundary evidence). The
generic scanner sweeps all data words outside code ranges for text
pointers with table evidence (an adjacent text-pointer word).
Emission (code_generator/control_flow): aliases grouped per host into
ONE shared static body — entry switch + the union of blocks reachable
from any entry (successors + jr jump-table edges) — plus a dispatchable
wrapper per entry. Sibling entries are injected as block leaders so all
group CFGs are identical. Aliases are excluded from the split pre-pass
containment so they never truncate hosts; the pre-pass cap is raised
16 -> 256 with a loud non-convergence warning (16 no longer converged
once promotions widened the function set).
Tomba (SCUS-94236): 1441 alias entries in 108 host groups, 25 scan-
promoted entries; dispatch 2855 -> 5562; generated C 12 MB -> 15.3 MB
(naive per-alias duplication was 80 MB). Verified on the overworld
route: 0x4Dxxx family gone from dirty_ram_stats per-PC (was ~0.5M
insns/session), 0x8004DF00 called natively once/frame from
ra=0x800462C4 (fntrace), dispatch_miss_total=0, aborts=0, user-visible
gameplay clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Restores entry-scoped overlay analysis (analyze_exact_entries +
split_mid_function_targets=false) as the MANDATORY mode for overlay inputs,
renamed --overlay-exact -> --overlay: the ingester's contract, set
unconditionally by compile_overlays.py for every overlay, never a human toggle.
Evidence-scoped discovery compiles only proven seeds + reachable code and keeps
branch/jump-table targets as in-parent labels — no whole-byte sweep, so it no
longer decodes embedded data tables as code.
Effect on the captured Tomba overlay: the data-as-code explosion collapses from
thousands of UNSUPPORTED to 122 clean functions with 42 residual unsupported
opcodes + 8 bad targets. Those residuals are now a per-function-audit concern
(Inc1-B), not a wholesale region failure. Main-EXE codegen is unchanged (Tomba
still regenerates 2854 functions; split pre-pass still runs in non-overlay mode).
Also adds docs/overlay-recompilation-design.md — the converged design of record
(precision-over-recall; compiled!=registered!=callable; write-invalidation over
CRC; internal-labels-in-parent; the RAM-write-path audit result; the three
implementation increments).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rips out the --overlay-exact recompiler path (analyze_exact_entries,
split_mid_function_targets, the CLI flag, and the compile_overlays.py flag),
returning the recompiler to master's naive whole-blob analysis. Overlay
compilation now splits mid-function branch targets into standalone functions —
the original behavior that produced black-on-load / blue-on-new-game — so we
can faithfully reproduce the failure before fixing it.
Pivot in approach: rather than excluding mid-function seeds at scan time, the
plan is to capture everything (the full overlay image is already dumped to
overlay_captures.json) and have the ingester detect each seed's containing
parent function at encode time and merge/ignore mid-function seeds, while
flagging entries reached only via computed/indirect jumps as jump-risk.
The runtime overlay loader/capture system (b2929aa) is unchanged. The 5
recompiler files are restored byte-exact to master (d22c9ce); config_loader's
overlay_cache flag is retained.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reinstates the runtime-loaded overlay DLL cache (Layer A) and the offline
overlay compilation pipeline that were built on the pruned codex/cache-discovery
branch (tip 2bbafc2) and never merged. Ported onto current master; rebuilt
rather than history-restored.
Runtime (off by default, opt in via [runtime] overlay_cache = true):
- overlay_loader.c/.h, overlay_api.h: CRC32-keyed DLL cache at
cache/<game_id>/<region>_<crc>.dll. On a dirty-RAM dispatch miss the loader
LoadLibrary's the matching DLL, wires runtime callbacks via overlay_init(),
enumerates func_XXXXXXXX exports, and dispatches them BEFORE the interpreter.
- overlay_capture.c/.h: write-once capture set; records overlay DMA bytes and
writes overlay_captures.json (execution-verified seeds) at clean exit.
- dma.c: capture + cd_dma_log hook at CD DMA completion (async model).
- cdrom.c: cdrom_get_setloc_lba() for disc-position correlation.
- dirty_ram_interp.c: g_dirty_ram_exec_pc_table (exec-verified PCs), a
decodability guard on JAL-into-dirty targets, and the overlay dispatch hooks.
- debug_server.c: cd_read_log + overlay_loader_status TCP commands.
- config_loader: [runtime] overlay_cache flag (default false).
Recompiler (compile_overlays.py / extract_overlays.py drive these):
- main_psx.cpp --overlay-exact: analyze only explicit entries + direct JAL
targets and set split_mid_function_targets=false, so overlay basic-block
targets are NOT split into bogus standalone functions (the mid-function-seed
failure mode that produced black-on-load / blue-on-new-game).
- function_analysis.cpp: analyze_exact_entries(); code_generator.cpp: gate the
mid-function split pre-pass behind split_mid_function_targets (default true,
so normal main-EXE codegen is unchanged - Tomba still regens to 2854 funcs).
Deltas from the original: save states dropped (never worked); overlay enable is
runtime-config-gated (off by default) instead of a build-time define; loader
stderr logging converted to the overlay_loader_status TCP path (CLAUDE.md Rule 3).
Recovered from _backups/2026-06-06-consolidation/psxrecomp-all-PRE.bundle.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Recompiler:
- function_analysis: pointer-table entry discovery (scan executable word tables
for function pointers, count as pointer_table_entry_count)
- function_analysis: generalize pre-prologue setup detection (multi-instruction
windows, any load opcode, zero-immediate loads, branch-in-window guard)
- full_function_emitter: use psx_check_interrupts() (not _at) at interrupt
safe points; fix g_psx_call_depth as extern (definition moved to interrupts.c)
- main_bios: add add_bios_vector_target_seeds() — augments seeds from ROM-
resident BIOS vector tables so handler functions are always discovered
- main_psx: print pointer_table_entry_count in analysis summary
Runtime:
- dirty_ram_interp: RFE in delay slot handled correctly (exec_rfe + psx_rfe_return,
not fall-through); exec_delay_slot returns int so callers can detect early-exit;
all branch/jump delay-slot callers updated
- dirty_ram_interp: RFE in exec_one calls psx_rfe_return and returns 1
- dirty_ram_interp: interrupt check after nonlocal calls when cpu->pc == 0
- interrupts: g_psx_call_depth definition moved here (was in dirty_ram_interp);
psx_check_interrupts handles pc==0/misaligned with sentinel 0x80000048
- cdrom: data_fifo_ready() helper; sector stream can refill buffer while IRQ is
asserted for in-callback DMA; data-ready status bit uses data_fifo_ready()
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Framework changes that get Ape Escape (USA) booting through to its
attract/demo sequence. Captures the working state at this milestone.
Runtime:
- cdrom: read/seek + data-path fixes for disc streaming
- dma / interrupts: channel + IRQ handling needed by the boot chain
- dirty_ram_interp: expanded coverage for self-modifying / RAM handlers
- sio, traps, crash_trace: supporting fixes
- debug_server: probe/inspection commands used during bring-up
Recompiler:
- code_generator / full_function_emitter / function_analysis / main_psx:
codegen fixes surfaced by the Ape Escape boot path
Committed as-is (no further edits) since the build is in a known-good,
demo-loading state. Branch: codex/ape-escape-support.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Uncommitted framework work on codex/ape-escape-support reaching the
post-boot DualShock-required screen. Recompiler codegen (code_generator,
full_function_emitter, function_analysis, config_loader) + runtime
(cdrom, sio, dirty_ram_interp, dma, iso_reader, mdec, gpu, memory,
traps, debug_server, fntrace, freeze_heartbeat, psx_cycles) and headers.
Committed as a restore point before rebuilding Tomba against this
framework. Build dirs (recompiler/build-codex*) intentionally excluded.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Framework work on codex/mmx6-support that makes Tomba build and play
(title screen, 0 dispatch misses) and MMX6 boot through BIOS to its
intro. Touches recompiler codegen (code_generator, function_analysis)
and runtime CD-ROM/DMA/MDEC/GPU/SIO timing (cdrom, dma, mdec, gpu,
psx_cycles, traps, fntrace, debug_server) + headers + runtime.cmake.
Milestone snapshot for collapsing onto master. Build dir excluded.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Recompiler — `recompiler/src/function_analysis.cpp` now stops a
function's end_addr at its mapped `jr $ra` + delay slot instead of
stretching through trailing data. Fixes Tomba's 0x80076DB0, a real
136-byte function that was being emitted as a psx_unknown_dispatch
stub because data after it poisoned the boundary heuristic.
`code_generator.cpp` emits `func_<next>(cpu); return;` as a tail
call when a mid-function seed split leaves the physical fall-through
in a different C function, plus a `debug_server_log_call_entry` at
every function entry. `full_function_emitter.cpp` splits dispatch
into `psx_dispatch_impl(cpu, addr, stop_addr)` with public
`psx_dispatch` and new `psx_dispatch_call(cpu, addr, return_addr)`
so JAL/JALR into RAM stubs can return cleanly without growing the
native stack.
Runtime — `runtime/src/dma.c`/`include/dma.h`/`src/memory.c` now
expose `dma_write_masked()`; the 16-bit and 8-bit mmio paths route
DICR/DPCR/CHCR partial writes through it so a partial DICR write no
longer accidentally clears unrelated channels' pending IRQ flags,
and CHCR only triggers DMA when the written mask actually includes
the start bit. Unified `complete_transfer(ch)` across all channels,
removed `prev_master_flag` edge detection.
MDEC (`runtime/src/mdec.c`, `include/mdec.h`) gains a full debug
surface: `MDECDebugState` snapshot, an always-on 4096-entry
`MDECDebugEvent` ring with 11 event kinds (reset, ctrl_write,
cmd_begin/done, decode_done, dma_in/out start/end, output_drained,
read_underflow), and `mdec_state` / `mdec_trace` / `mdec_trace_clear`
debug-server commands. DMA ch0/1 emit start/end events around MDEC
transfers.
CDROM (`runtime/src/cdrom.c`, `include/cdrom.h`) — read stream
separated from one-shot pending state (`read_cmd`, `read_delay`,
`start_read_stream`, `stop_read_stream`, `deliver_read_sector`);
`cdrom_state`/`cdrom_trace` report the new fields.
Thread/fiber tracing — `traps.c`'s Win32-fiber TCB save/restore now
emits 12 thread-event kinds; new `ThreadTraceEntry` ring (64 K) plus
`thread_trace` / `thread_trace_clear` debug-server commands report
both live CPU regs and the target TCB's full saved register block.
wtrace / mmio rings — `WriteTraceEntry` grew (cpu_pc, sp, v0/v1/a0-a3/
t0/t1); capacity 1 M -> 4 M. `MmioTraceEntry` grew (pc, cpu_pc, sp,
a0-a3, sr, epc, i_stat, i_mask). Both dump commands accept
`count`/`newest=1` for reverse iteration. `restore_trace` and
`dirty_block_log` emit extended register fields (a2/a3/sp).
Misc — `interrupts.c` polls debug-server every 16K checks so it
stays responsive without a separate thread. `psx_runtime.h`'s
`call_by_address` routes through `psx_dispatch_call`. Dirty-RAM
interp JAL/JALR consult `psx_dispatch_game_compiled` when the game
is wired in. iso_reader gains the entry points needed by the game
disc path. New `tools/_tomba_probe.py` for port-4470 health checks.
Generated — regenerated `SCPH1001_dispatch.c` against the new
`psx_dispatch_impl` shape.
ISSUES.md — opened Issue #3 (BIOS PS-logo glyph missing on the
disc-detected screen, cosmetic).
.gitignore — added `recompiler/build-asan*/`, `build-checked/`,
`__pycache__/`.
Validated end-to-end: BIOS boots, disc detected, Whoopee Camp FMV
plays with audio. Frame count advances through ~40K frames without
freezes. `unknown_dispatch_log.total = 0`. Remaining FMV cluster
issues (1/4-size render, FMV-2 audio missing, no skip) tracked
under TombaRecomp/ISSUES.md.
Co-Authored-By: Codex (OpenAI) <noreply@openai.com>
Add Pass 2.6 to FunctionAnalyzer: detect JAL targets whose callee starts
with `sw $ra, imm($a0)` (SaveState-style context-save helpers) and force
the post-delay-slot PC into the dispatch table. Recovers continuation
entries that aren't reached by JAL prologue or jr $ra discovery.
Extend find_function_start to recognize `lui rt; lw rt, off(rt)` 8 bytes
before jr $ra as a prologue marker. Rewrite analyze_function block-build
from std::set boundaries to a sorted std::vector + binary_search.
main_psx CLI: --seeds, --out-dir, --strict, --inspect; out_dir support.
Add psxrecomp-game executable target alongside PSXRecomp.
code_generator: reg_name() now emits cpu->gpr[N] consistently (matching
what the rest of the codegen already emitted — the old reg_names[] table
returning cpu->v0/cpu->ra was unreachable because CPUState has no named
fields, only gpr[32]). LWL/LWR/SWL/SWR moved from cpu->lwl(...) member
form to psx_lwl(cpu, ...) free-function form for game-EXE codegen reuse.
Remove a stray fmt::print left in build_basic_blocks that spammed
stdout on every regen — per the no-stdio-debug rule (memory v4).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Matt Stanley