From 7c5b766fd5ef53b24fc7810b7f366fd1cf18b8bf Mon Sep 17 00:00:00 2001 From: Matthew Stanley Date: Wed, 2 Sep 2026 00:37:44 -0700 Subject: [PATCH] overlay: ABI v22 game_text_native_ok shim (107 shards failed to link) The emitter emits a direct psx_game_text_native_ok() stale-static guard at block entry (code_generator.cpp:115) and before every direct jal (:2402). That function is DEFINED IN THE GENERATED DISPATCH SHARD (main_psx.cpp:1620), which links into the executable. An overlay DLL is a separate module and cannot resolve it, and overlay_dispatch_preamble.c.inc had no shim for it. Found by bumping Tomba 2 from a 254-commit-old pin to master and regenerating its shard cache against the identical vault: before bump : built OK 2799, FAILED 1 after bump : built OK 2233, FAILED 109 107 of the 109 were the same error: ld.exe: undefined reference to `psx_game_text_native_ok' collect2.exe: error: ld returned 1 exit status This is the fifth instance of one class, and overlay_api.h's own version history records the previous four: v9 faithful-timing callbacks overlay code failed to link v10 gte_read_ctrl GTE-heavy overlays failed to LINK without shims, silently blocking new coverage v12 rfe_mark_escape without the shim any rfe-containing overlay fails to LINK v14 gte_precision_store_word the rfe / gte_read_ctrl class again Every time the shape is identical: an emitter gains a runtime call, the overlay ABI does not gain the shim, and the only symptom is shards that quietly fail to build. The affected code then runs interpreted forever and nothing at runtime says so. Severity is coverage, not correctness -- but that is exactly the defect that let Ape Escape ship 100% interpreted without anyone noticing. Fix follows the v10/v12/v14 pattern exactly: - overlay_api.h: append game_text_native_ok to OverlayCallbacks, bump PSX_OVERLAY_ABI_VERSION 21 -> 22, add the version-history note - overlay_dispatch_preamble.c.inc: shim forwarding to g_cbs - overlay_loader.c init_callbacks: host populates it from the dispatch shard The NULL guard deliberately returns 0, not 1. Returning 0 routes the guest through the dispatcher -- slower, never wrong. Returning 1 would let an overlay run native code the guard was explicitly trying to reject. Follow-up worth doing, tracked as beads-eio.3.97: five instances of one class means the process is the bug, not the symbol. The emitter knows every helper name it can emit and the preamble enumerates every shim; a test comparing those two sets would have caught v9, v10, v12, v14 and this one at authoring time instead of at some future title's link step. Co-Authored-By: Claude Opus 5 (1M context) --- runtime/include/overlay_api.h | 20 ++++++++++++++++++- .../include/overlay_dispatch_preamble.c.inc | 10 ++++++++++ runtime/src/overlay_loader.c | 6 ++++++ 3 files changed, 35 insertions(+), 1 deletion(-) diff --git a/runtime/include/overlay_api.h b/runtime/include/overlay_api.h index 2ceda151..28a9f488 100644 --- a/runtime/include/overlay_api.h +++ b/runtime/include/overlay_api.h @@ -45,6 +45,18 @@ * masked, aliased, read-only, precision-sensitive, flag/IR, and derived * GTE regs; GTE-heavy overlays failed to LINK without shims (undefined * gte_read_ctrl), silently blocking new coverage. */ +/* v22: game_text_native_ok callback - the emitter emits a direct + * psx_game_text_native_ok() stale-static guard (code_generator.cpp, at + * block entry and before every direct jal). The function is DEFINED IN THE + * GENERATED DISPATCH SHARD, which lives in the executable, so an overlay + * DLL cannot see it: 107 of 109 shard failures on the 2026-09-02 Tomba 2 + * framework bump were `undefined reference to psx_game_text_native_ok`. + * Exactly the rfe / gte_read_ctrl / gte_precision_store_word class again - + * an emitter gained a runtime call without the overlay ABI gaining the + * shim, and the only symptom is shards that silently fail to build so the + * code runs interpreted forever. NULL-guarded: a host predating v22 makes + * the shim return 0, which routes the guest through the dispatcher instead + * of native code - slower, never wrong. */ /* v11 batches per-instruction cycle charges inside overlay DLLs and flushes at * block/device/store boundaries, removing a cross-DLL callback per instruction * without changing the shared guest-cycle timeline. */ @@ -93,7 +105,7 @@ * host/DLL flavor mix — base-flavour DLLs and hosts are untouched, so * the version stays. The emit-content change (PGXP_*() macros in all * generated C) is covered by the codegen hash + CODEGEN_VER below. */ -#define PSX_OVERLAY_ABI_VERSION 21 +#define PSX_OVERLAY_ABI_VERSION 22 /* Process-lifetime overlay candidate capacity. Every accepted manifest F * record consumes one slot, even when another DLL carries an identical @@ -309,6 +321,12 @@ typedef struct { /* Aspect-scaled terrain-frustum angle helper (ABI v21). */ uint32_t (*ws_angle_widen)(uint32_t vanilla); + /* Stale-static guard (ABI v22; see the version-history note above). + * Defined by the generated dispatch shard in the executable, so overlay + * DLLs must forward. Appended last; may be NULL on a host that predates + * it - the shim then returns 0 and the guest takes the dispatcher path. */ + int (*game_text_native_ok)(uint32_t addr); + /* PGXP dataflow-shadowing hook table (pgxp_hooks.h). Only pgxp-flavour * shards (PSX_OVERLAY_FLAVOR pgxp bit, compiled with -DPSX_PGXP=1) emit * calls to psx_pgxp_*; their preamble forwards through this table. Base- diff --git a/runtime/include/overlay_dispatch_preamble.c.inc b/runtime/include/overlay_dispatch_preamble.c.inc index ac0e9fc5..fefb8165 100644 --- a/runtime/include/overlay_dispatch_preamble.c.inc +++ b/runtime/include/overlay_dispatch_preamble.c.inc @@ -93,6 +93,16 @@ void psx_rfe_mark_escape(void) { overlay_flush_cycles(); g_cbs.rfe_mark_escape(); } +/* Stale-static guard (ABI v22). The emitter emits this at block entry and + * before every direct jal, but it is DEFINED IN THE GENERATED DISPATCH SHARD + * (executable side), so an overlay DLL cannot resolve it without this shim. + * Its absence is what made 107 shards fail to link on the 2026-09-02 bump. + * NULL-guarded: returning 0 sends the guest down the dispatcher path, which is + * slower but never wrong -- the opposite choice (returning 1) would let an + * overlay run native code the guard was trying to reject. */ +int psx_game_text_native_ok(uint32_t addr) { + return g_cbs.game_text_native_ok ? g_cbs.game_text_native_ok(addr) : 0; +} /* Faithful-timing functions (ABI v9): overlay code built with PSX_ENABLE_BLOCK_CYCLES * emits these; forward to the runtime's real impls so native overlays charge cycles on * the SAME timeline as the interp/BIOS (a local copy would diverge). NULL-guarded so a diff --git a/runtime/src/overlay_loader.c b/runtime/src/overlay_loader.c index 0bd26ccc..a2b32480 100644 --- a/runtime/src/overlay_loader.c +++ b/runtime/src/overlay_loader.c @@ -2260,6 +2260,12 @@ static void init_callbacks(void) { s_callbacks.psx_restore_state_escape = psx_restore_state_escape; /* Return-from-exception mark (ABI v12): overlay `rfe` ops forward here. */ s_callbacks.rfe_mark_escape = psx_rfe_mark_escape; + /* Stale-static guard (ABI v22): defined by the generated dispatch shard in + * this executable, so overlay DLLs forward here rather than link it. */ + { + extern int psx_game_text_native_ok(uint32_t addr); + s_callbacks.game_text_native_ok = psx_game_text_native_ok; + } /* Call-contract state (ABI v2): DLL code shares the runtime's bail * flag and counters through these pointers. */ s_callbacks.call_bail_flag = &g_psx_call_bail;