mirror of
https://github.com/RetroPortingToolKit/psxrecomp
synced 2026-10-26 02:11:36 -04:00
feat(overlay): per-function native-disable for divergence bisection
Adds a phys-keyed blocklist in overlay_loader.c that forces a named overlay function through the sanctioned dirty-RAM interpreter (it still RUNS — not skip/stub/HLE), as if s_native_exec were off but for that one function only. Exposed as the `overlay_native_block` debug command (add/clear/report). This is the localization tool for native<->interp divergences (e.g. the Tomba 2 FMV freeze): binary-search which compiled overlay function's native execution flips the bug, without rebuilding or losing candidate visibility. Runtime/tool-only. Used it to localize the Tomba 2 end-of-Whoopee-Camp FMV freeze to func_80050B08's OWN native body (blocking it passes the freeze with callees still native; all 33 jal targets + continuations verified clean). Narrowed to the main-loop region with a strong interior-entry suspect at 0x80050CE8. Findings in docs/TOMBA2_NATIVE_FMV_FREEZE.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Matthew Stanley
@@ -121,19 +121,101 @@ shared) ranked, for this static-MIPS CPS recompiler:
|
||||
interpreter. No load-delay hazard detection currently exists (only a branch-delay
|
||||
scanner, `tools/scan_branch_delay_hazards.py`).
|
||||
|
||||
## Deeper investigation (2026-06-23, session 2) — codegen DOUBLY confirmed; red herrings ruled out
|
||||
|
||||
Raw RAM dumped + loaded into Ghidra (MIPS:LE:32 @ 0x80000000). Dumps are now
|
||||
KEYED by resident-overlay identity (tools dump_ram.py writes
|
||||
`<name>_f<frame>_<variant-fingerprint>.bin` + a .json manifest of game/frame/
|
||||
resident-fn CRCs) — a bare RAM image is ambiguous because the same addresses
|
||||
hold different overlay variants per scene.
|
||||
|
||||
Codegen-vs-race settled by TWO independent methods:
|
||||
1. Sandboxed `overlay_diff` shadow (identical state, no IRQ/timing) diverges.
|
||||
2. `overlay_irq_ratelimit n=4096` (coarse, interp-like IRQ cadence) set BEFORE
|
||||
the freeze → STILL freezes at ~1826. Not IRQ-cadence/timing.
|
||||
=> CODEGEN, definitively not a race.
|
||||
|
||||
Red herrings ruled OUT as the direct cause:
|
||||
- BREAK no-op: the spin-set's break is at 0x80089784. The dirty-RAM interpreter
|
||||
(`dirty_ram_interp.c:1041`, the engine under overlay_native_off) routes break
|
||||
to `psx_break` which CRASHES (traps.c:594 = trap_crash + exit(1)). Since
|
||||
overlay_native_off PROGRESSES (no crash), 0x89784 is NEVER reached in the
|
||||
working path => main (0x80050B08) does not return there normally. So the
|
||||
recompiler's break-no-op (code_generator.cpp:973) is a real latent bug (should
|
||||
fail-loud like strict_translator's `psx_break(...)`/the interp), but it is NOT
|
||||
what breaks the FMV. It only matters once something upstream wrongly reaches it.
|
||||
- Load-delay: path clean (see above).
|
||||
|
||||
Operative divergence (what actually happens): under native, control flow reaches
|
||||
the bootstrap re-loop — the native overlay ring shows a repeating cycle
|
||||
{0x89860 InitHeap-thunk → 0x89770 → 0x80050B08 "main" → 0x89788 init-array} with
|
||||
0x800896E0 (the crt0-style stack/heap/context init) re-entered each lap, calling
|
||||
InitHeap (→ A0 vector) every iteration — which is the `dispatch_tail` all-0xA0
|
||||
spin. Under interp this does not loop. i.e. native's 0x80050B08 ("main") RETURNS
|
||||
(native_ring returned:1) or the bootstrap otherwise re-enters, where interp keeps
|
||||
running. Root is a mis-emitted control transfer in the 0x80050B08 subtree (large
|
||||
fn, internal loops, many interior/alias entries) — NOT yet pinned to one insn.
|
||||
|
||||
Shadow caveat (rule 15): `overlay_shadow_dump` reports 13 divergences; the seq-1
|
||||
(0x896E0 t0/t1) is suspect as a SYSCALL-BOUNDARY ARTIFACT — native continues
|
||||
through the A0 dispatch (t1=0x39<<2=0xE4, t0=A0_table[0x39]) while the interp
|
||||
shadow stops at the overlay→kernel(0xA0) boundary (t1=0x39). Later records have
|
||||
real RAM divergences (e.g. fn 0x85D70: RAM native=0 vs interp=0xBFC050A4, a BIOS
|
||||
ROM pointer) suggesting a systemic BIOS-interaction/return-path codegen issue,
|
||||
not one function. Need finer (instruction-level) isolation.
|
||||
|
||||
## Session 3 (2026-06-23): per-function native-disable BUILT + bug localized to func_80050B08
|
||||
|
||||
Built the per-function native-disable (the tool ChatGPT recommended): a phys-keyed
|
||||
blocklist in overlay_loader.c that forces a named overlay function through the
|
||||
sanctioned dirty-RAM interpreter (NOT skip/stub/HLE — it still runs), exposed as the
|
||||
`overlay_native_block` debug command ({"addr":"0x.."} add / {"clear":1} clear /
|
||||
{} report). Runtime-only change (overlay_loader.c + debug_server.c), no regen.
|
||||
|
||||
BISECTION RESULT — culprit is **func_80050B08's own native body**:
|
||||
- `overlay_native_block addr=0x80050B08` set early → boot PASSES the freeze (frame
|
||||
2306+), hits=1. Because the blocked function runs via interp while its CALLEES stay
|
||||
native, this proves the bug is in 0x50B08's OWN compiled code, not a callee.
|
||||
- 0x80050B08 = the game main: a 33-`jal` subsystem-init sequence then an infinite main
|
||||
loop (no `jr $ra`; loops via goto). It is entered ONCE (hits=1); native exec of it
|
||||
causes the bootstrap re-loop / A0-spin; interp exec runs the game (frame advances).
|
||||
- Verified clean: all 33 CPS jal TARGETS match the RAM-decoded jal instructions; all
|
||||
jal return-continuations are present in the entry-switch (no missing/`default`
|
||||
fall-through-to-top). So it is NOT a wrong-target or missing-continuation bug.
|
||||
|
||||
NARROWED to the main-loop region (block_80050C6C..0x50DE0, the part with real
|
||||
conditionals). STRONG SUSPECT: an INTERIOR ENTRY in a loop — `block_80050CE8` is an
|
||||
entry-switch case landing MID-BLOCK, immediately AFTER `0x50CE4: lhu v0,-32612(a0)`,
|
||||
so entering at 0x50CE8 uses a STALE v0 (skips the load) before the `sltu v0,v0,v1` +
|
||||
`bne` scan-loop test. This matches ChatGPT's #1 suspect (continuation/interior-entry
|
||||
handling, the 47f3f15 zone). Need to confirm what transfers control to 0x50CE8 (a
|
||||
jalr-return / jr-table / branch alias) and whether native lands there with wrong
|
||||
state vs interp.
|
||||
|
||||
## Next steps
|
||||
|
||||
1. Drill to the exact diverging instruction inside the 0x800896E0 callee tree (it makes
|
||||
an A0:0x39 call the interpreter doesn't — find the branch whose condition native
|
||||
computes wrong). Disassemble 0x80089860 and the A0-call site; narrow with the diff
|
||||
infra.
|
||||
2. Confer with ChatGPT on the disasm to confirm the codegen construct.
|
||||
3. Class-fix in `recompiler/src` (never generated C, never a Tomba 2 workaround).
|
||||
4. Verify: regen overlay cache, `overlay_diff` shows 0 divergences for 0x896E0, and a
|
||||
live native run passes the freeze (frame > 2031) at 60fps with MDEC advancing.
|
||||
5. Adjudicate the exact divergent MIPS against the Beetle oracle + BIOS disasm + Ghidra;
|
||||
1. Build a PER-FUNCTION / per-entry-PC native-disable (ChatGPT's #1 tool; does not
|
||||
exist yet — only the global overlay_native_off). It must route the disabled
|
||||
HostUnit through the sanctioned dirty-RAM interpreter (NOT skip/stub/HLE). Then
|
||||
bisect: start by disabling native for 0x80050B08, then its callees, to find the
|
||||
single function whose native execution flips the freeze. This gives causal proof
|
||||
where the shadow (function-granular, with syscall-boundary artifacts) cannot.
|
||||
2. With the culprit function isolated, instruction-diff native-vs-interp within it
|
||||
(extend the shadow to record the first diverging guest PC, or step the dirty
|
||||
interp and the native side from the same entry and compare per-block) to pin the
|
||||
exact mis-emitted transfer.
|
||||
3. Confer with ChatGPT on that disasm to confirm the codegen construct (likely a
|
||||
jr/jalr/branch continuation-PC or interior/alias-entry transfer in 0x50B08's tree).
|
||||
4. Class-fix in `recompiler/src` (never generated C, never a Tomba 2 workaround).
|
||||
5. Verify: regen overlay cache, `overlay_diff` shows 0 divergences, live native run
|
||||
passes the freeze (frame > 2031) at 60fps with MDEC advancing.
|
||||
6. Adjudicate the exact divergent MIPS against the Beetle oracle + BIOS disasm + Ghidra;
|
||||
generated C is evidence only.
|
||||
|
||||
Separately (independent class fixes, lower priority): emit BREAK as fail-loud/exception
|
||||
in code_generator.cpp (match strict_translator/interp), and model the R3000A
|
||||
load-delay slot.
|
||||
|
||||
Diagnostic principle reminder: a shadow mismatch is "native differs from interp at
|
||||
fn X / insn Y", NOT automatically "native is wrong" — the interpreter can also be
|
||||
wrong; adjudicate against Beetle/disasm before committing a fix.
|
||||
|
||||
@@ -9045,6 +9045,36 @@ static void handle_overlay_native_off(int id, const char *json)
|
||||
send_fmt("{\"id\":%d,\"ok\":true,\"native_exec\":0}\n", id);
|
||||
}
|
||||
|
||||
/* overlay_native_block: per-function native-disable for bisection. Forces the
|
||||
* named overlay function(s) through the sanctioned dirty-RAM interpreter (the
|
||||
* function still runs — NOT skipped/stubbed), so you can binary-search which
|
||||
* compiled function's native execution causes a divergence without rebuilding.
|
||||
* {"cmd":"overlay_native_block","addr":"0x80050B08"} -> add (keyed by phys)
|
||||
* {"cmd":"overlay_native_block","clear":1} -> clear all
|
||||
* {"cmd":"overlay_native_block"} -> just report state */
|
||||
static void handle_overlay_native_block(int id, const char *json)
|
||||
{
|
||||
extern int overlay_loader_native_block_add(uint32_t addr);
|
||||
extern void overlay_loader_native_block_clear(void);
|
||||
extern int overlay_loader_native_block_list(uint32_t *out, int cap);
|
||||
extern uint64_t overlay_loader_native_block_hits(void);
|
||||
char abuf[32];
|
||||
if (json_get_int(json, "clear", 0)) overlay_loader_native_block_clear();
|
||||
if (json_get_str(json, "addr", abuf, sizeof(abuf)))
|
||||
overlay_loader_native_block_add(hex_to_u32(abuf));
|
||||
uint32_t list[64];
|
||||
int n = overlay_loader_native_block_list(list, 64);
|
||||
char buf[1200]; size_t pos = 0;
|
||||
pos += (size_t)snprintf(buf + pos, sizeof(buf) - pos,
|
||||
"{\"id\":%d,\"ok\":true,\"count\":%d,\"hits\":%llu,\"blocked\":[",
|
||||
id, n, (unsigned long long)overlay_loader_native_block_hits());
|
||||
for (int i = 0; i < n && i < 64; i++)
|
||||
pos += (size_t)snprintf(buf + pos, sizeof(buf) - pos,
|
||||
"%s\"0x%08X\"", i ? "," : "", list[i]);
|
||||
snprintf(buf + pos, sizeof(buf) - pos, "]}\n");
|
||||
send_fmt("%s", buf);
|
||||
}
|
||||
|
||||
/* overlay_dump: extract RAM regions that dirty_ram has marked executable
|
||||
* above a threshold physical address. Writes <crc32>.bin files to a
|
||||
* caller-supplied directory and returns a JSON manifest.
|
||||
@@ -9728,6 +9758,7 @@ static const CmdEntry s_commands[] = {
|
||||
{ "ra_load_watch", handle_ra_load_watch },
|
||||
{ "overlay_native_on", handle_overlay_native_on },
|
||||
{ "overlay_native_off", handle_overlay_native_off },
|
||||
{ "overlay_native_block", handle_overlay_native_block },
|
||||
{ "overlay_capture_dump", handle_overlay_capture_dump },
|
||||
{ "cdrom_instant_rate", handle_cdrom_instant_rate },
|
||||
{ "cd_overwrite", handle_cd_overwrite },
|
||||
|
||||
@@ -168,6 +168,39 @@ void overlay_loader_bad_entry_stats(uint32_t *owner, uint32_t *pc, uint64_t *cou
|
||||
if (pc) *pc = s_bad_entry_pc;
|
||||
if (count) *count = s_bad_entry_count;
|
||||
}
|
||||
|
||||
/* Per-function native-disable (bisection). Functions whose ENTRY phys address is
|
||||
* listed here are forced through the sanctioned dirty-RAM interpreter instead of
|
||||
* their native shard — exactly as if s_native_exec were off, but for that one
|
||||
* function only. This is a DIAGNOSTIC localization knob (not skip/stub/HLE): the
|
||||
* function still runs, just via the interpreter. Used to binary-search which
|
||||
* compiled overlay function's native execution causes a native-vs-interp
|
||||
* divergence. Keyed by phys (addr & 0x1FFFFFFF) so KSEG bits don't matter. */
|
||||
#define NATIVE_BLOCK_CAP 64
|
||||
static uint32_t s_native_block[NATIVE_BLOCK_CAP];
|
||||
static int s_native_block_n = 0;
|
||||
static uint64_t s_native_block_hits = 0;
|
||||
static int overlay_native_blocked(uint32_t addr) {
|
||||
if (s_native_block_n == 0) return 0;
|
||||
uint32_t p = addr & 0x1FFFFFFFu;
|
||||
for (int i = 0; i < s_native_block_n; i++)
|
||||
if (s_native_block[i] == p) { s_native_block_hits++; return 1; }
|
||||
return 0;
|
||||
}
|
||||
int overlay_loader_native_block_add(uint32_t addr) {
|
||||
uint32_t p = addr & 0x1FFFFFFFu;
|
||||
for (int i = 0; i < s_native_block_n; i++) if (s_native_block[i] == p) return s_native_block_n;
|
||||
if (s_native_block_n >= NATIVE_BLOCK_CAP) return -1;
|
||||
s_native_block[s_native_block_n++] = p;
|
||||
return s_native_block_n;
|
||||
}
|
||||
void overlay_loader_native_block_clear(void) { s_native_block_n = 0; s_native_block_hits = 0; }
|
||||
int overlay_loader_native_block_list(uint32_t *out, int cap) {
|
||||
int n = s_native_block_n < cap ? s_native_block_n : cap;
|
||||
for (int i = 0; i < n; i++) out[i] = s_native_block[i];
|
||||
return s_native_block_n;
|
||||
}
|
||||
uint64_t overlay_loader_native_block_hits(void) { return s_native_block_hits; }
|
||||
/* Address of the native overlay function currently executing (0 if none).
|
||||
* Used by the event-timeline ring to tag an event's execution mode. */
|
||||
uint32_t overlay_loader_get_inprogress(void) { return s_native_inprogress; }
|
||||
@@ -1268,7 +1301,8 @@ int overlay_loader_dispatch(CPUState *cpu, uint32_t addr) {
|
||||
if (matched) {
|
||||
if (c->state != ENTRY_VALID) { c->state = ENTRY_VALID; s_valid_count++; }
|
||||
if (c->device_touch) { s_disp_interp++; return 0; }
|
||||
if (!s_native_exec) { s_would_run_native++; s_disp_interp++; return 0; }
|
||||
if (!s_native_exec || overlay_native_blocked(c->addr))
|
||||
{ s_would_run_native++; s_disp_interp++; return 0; }
|
||||
uint32_t slot = s_nring_pos++ & (NRING_CAP - 1u);
|
||||
s_nring[slot].addr = addr;
|
||||
s_nring[slot].crc = c->crc_code;
|
||||
@@ -1358,8 +1392,10 @@ int overlay_loader_dispatch(CPUState *cpu, uint32_t addr) {
|
||||
|
||||
/* A/B: prove whether native EXECUTION is the cause. When off, the
|
||||
* candidate matched (byte-exact) but we DON'T run native — interp
|
||||
* handles it. */
|
||||
if (!s_native_exec) { s_would_run_native++; s_disp_interp++; return 0; }
|
||||
* handles it. The per-function blocklist forces the same interp
|
||||
* routing for one function only (bisection localization). */
|
||||
if (!s_native_exec || overlay_native_blocked(c->addr))
|
||||
{ s_would_run_native++; s_disp_interp++; return 0; }
|
||||
|
||||
/* Record into the always-on ring BEFORE the call; mark in-progress
|
||||
* so a freeze inside this fn is visible at dump time. */
|
||||
|
||||
Reference in New Issue
Block a user