Files
psxrecomp/runtime/include/overlay_api.h
T
Matthew Stanley 7c5b766fd5 overlay: ABI v22 game_text_native_ok shim (107 shards failed to link)
The emitter emits a direct psx_game_text_native_ok() stale-static guard at block
entry (code_generator.cpp:115) and before every direct jal (:2402). That function
is DEFINED IN THE GENERATED DISPATCH SHARD (main_psx.cpp:1620), which links into
the executable. An overlay DLL is a separate module and cannot resolve it, and
overlay_dispatch_preamble.c.inc had no shim for it.

Found by bumping Tomba 2 from a 254-commit-old pin to master and regenerating its
shard cache against the identical vault:

  before bump : built OK 2799, FAILED 1
  after bump  : built OK 2233, FAILED 109

107 of the 109 were the same error:

  ld.exe: undefined reference to `psx_game_text_native_ok'
  collect2.exe: error: ld returned 1 exit status

This is the fifth instance of one class, and overlay_api.h's own version history
records the previous four:

  v9  faithful-timing callbacks  overlay code failed to link
  v10 gte_read_ctrl              GTE-heavy overlays failed to LINK without shims,
                                 silently blocking new coverage
  v12 rfe_mark_escape            without the shim any rfe-containing overlay
                                 fails to LINK
  v14 gte_precision_store_word   the rfe / gte_read_ctrl class again

Every time the shape is identical: an emitter gains a runtime call, the overlay
ABI does not gain the shim, and the only symptom is shards that quietly fail to
build. The affected code then runs interpreted forever and nothing at runtime
says so. Severity is coverage, not correctness -- but that is exactly the defect
that let Ape Escape ship 100% interpreted without anyone noticing.

Fix follows the v10/v12/v14 pattern exactly:
  - overlay_api.h: append game_text_native_ok to OverlayCallbacks, bump
    PSX_OVERLAY_ABI_VERSION 21 -> 22, add the version-history note
  - overlay_dispatch_preamble.c.inc: shim forwarding to g_cbs
  - overlay_loader.c init_callbacks: host populates it from the dispatch shard

The NULL guard deliberately returns 0, not 1. Returning 0 routes the guest
through the dispatcher -- slower, never wrong. Returning 1 would let an overlay
run native code the guard was explicitly trying to reject.

Follow-up worth doing, tracked as beads-eio.3.97: five instances of one class
means the process is the bug, not the symbol. The emitter knows every helper name
it can emit and the preamble enumerates every shim; a test comparing those two
sets would have caught v9, v10, v12, v14 and this one at authoring time instead
of at some future title's link step.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-02 00:37:44 -07:00

357 lines
21 KiB
C

#ifndef OVERLAY_API_H
#define OVERLAY_API_H
/* Shared ABI between psx-runtime.exe and overlay DLLs.
*
* After LoadLibrary, the runtime calls overlay_init() with this struct.
* The DLL stores the pointers and routes all cross-module calls through them.
*/
#include "cpu_state.h"
#include "pgxp_hooks.h"
#include <stdint.h>
/* ABI version exported by every overlay DLL as `overlay_abi()`. The loader
* rejects (and deletes, so autocompile regenerates) any DLL whose version
* doesn't match — including all pre-versioning DLLs, which lack the export.
*
* v2: dispatch call-contract (Bug D family) — DLL call sites carry (ra, sp)
* contract checks and share the runtime's bail state via the appended
* callback pointers below.
* v3: widescreen backdrop screenX squash ([widescreen.backdrop] x_sites) —
* overlay backdrop handlers now emit psx_ws_backdrop_x() at their screenX
* store. This is an EMIT-content change (not a struct-ABI change), but the
* version is the cache discriminator: bumping it forces the loader to
* reject pre-backdrop DLLs so autocompile regenerates them with the squash.
* (The deferred follow-up is to auto-derive this tag from a hash of the
* widescreen config + emit headers; until then it's a manual bump.)
* v4: widescreen backdrop column PRELOAD ([widescreen.cull] auto_backdrop) —
* overlay backdrop generators now emit psx_ws_backdrop_value() at the
* window START/END finalize, and the callback struct grows a
* ws_backdrop_value pointer (appended last). Bumping rejects pre-preload
* DLLs (which lack both the emit and a host that supplies the callback). */
/* v5: psx_syscall callback return type void->int (CPS, RECURSION_BUG.md §25);
* overlays compiled under CPS emit `if (psx_syscall(...)) return;`. */
/* v7: advance_cycles callback added; overlays now built with
* PSX_ENABLE_BLOCK_CYCLES and charge the shared host cycle/timer timeline.
* Bumping rejects stale DLLs that charged no cycles (Tomba2 Timer1 fork). */
/* v8: check_interrupts_at callback added so generated native overlay block
* checks can provide the real guest resume PC for async RFE recovery.
* v9: faithful-timing callbacks (cyc_load, icache_fetch, muldiv, mult_latency,
* gte, slice_block) so overlay code built with PSX_ENABLE_BLOCK_CYCLES links
* against the runtime's real timing impls (the cycle-accuracy rearchitecture).
* v10: GTE special-register accessors (gte_read_data/gte_read_ctrl/
* gte_write_data/gte_write_ctrl) — the emitter emits direct calls for
* masked, aliased, read-only, precision-sensitive, flag/IR, and derived
* GTE regs; GTE-heavy overlays failed to LINK without shims (undefined
* gte_read_ctrl), silently blocking new coverage. */
/* v22: game_text_native_ok callback - the emitter emits a direct
* psx_game_text_native_ok() stale-static guard (code_generator.cpp, at
* block entry and before every direct jal). The function is DEFINED IN THE
* GENERATED DISPATCH SHARD, which lives in the executable, so an overlay
* DLL cannot see it: 107 of 109 shard failures on the 2026-09-02 Tomba 2
* framework bump were `undefined reference to psx_game_text_native_ok`.
* Exactly the rfe / gte_read_ctrl / gte_precision_store_word class again -
* an emitter gained a runtime call without the overlay ABI gaining the
* shim, and the only symptom is shards that silently fail to build so the
* code runs interpreted forever. NULL-guarded: a host predating v22 makes
* the shim return 0, which routes the guest through the dispatcher instead
* of native code - slower, never wrong. */
/* v11 batches per-instruction cycle charges inside overlay DLLs and flushes at
* block/device/store boundaries, removing a cross-DLL callback per instruction
* without changing the shared guest-cycle timeline. */
/* v12: rfe_mark_escape callback — the emitter emits psx_rfe_mark_escape() at
* every `rfe` (code_generator.cpp COP0 case), including in overlay-resident
* exception-context code (Tomba kernel-window overlays hit this). It mutates
* the runtime's shared RFE-pending escape state (interrupts.c), so the DLL
* MUST forward — a local no-op would swallow exception returns. Without the
* shim any rfe-containing overlay fails to LINK (undefined psx_rfe_mark_escape)
* and that variant runs interpreted forever — the gte_read_ctrl class again. */
/* v13: ws_player_x_bound callback — the emitter rewrites a configured
* [widescreen] signed_x_bound LUI site into psx_ws_player_x_bound(vanilla)
* (code_generator.cpp; Einhander native-wide player bounds, 28165d6),
* INCLUDING in overlay-resident code (the site loop runs in overlay_mode
* too). The clamp computes against the host's live widescreen state (gpu.c),
* so the DLL must forward; without the shim any shard containing a bound
* site fails to LINK (undefined psx_ws_player_x_bound) and runs interpreted
* forever — the rfe/gte_read_ctrl class again. Identity fallback (vanilla)
* on a NULL host pointer keeps a v13 DLL correct (4:3) on an older host. */
/* v14: gte_precision_store_word forwarder — the emitter emits a direct call for
* every swc2 (GTE store-word); stateful runtime code (gte.cpp) so the DLL
* must forward or fail to LINK. Surfaced by full static overlay coverage
* (play-captured shards rarely compiled a swc2). Appended last; ABI bump
* to 14 rejects any pre-shim DLL. */
/* v15 histories existed independently on pre-integration branches:
* - generated stores call psx_store_cycle_barrier() before any host-visible
* write or SWL/SWR read-modify-write;
* - ws_depth_bound forwarder for configured aspect-scaled far-bound sites.
* v16: combines both ABI additions and rejects either stale v15 cache line. */
/* v17: ws_plane_nx forwarder for configured side frustum-plane normal-X
* cull sites. Appended last; NULL preserves the vanilla normal. */
/* v18: ws_xclip_bound forwarder for configured per-primitive X-reject
* bound-load cull sites. Appended last; NULL preserves the vanilla
* loaded value. */
/* v19: ws_cull_keep_result forwarder for full-word-guarded object/model
* participation comparisons. The verdict depends on the host's live
* widescreen margin; NULL preserves the vanilla comparison. */
/* v20: ws_aspect_cone_result forwarder for camera-horizontal, aspect-aware
* model participation with queue-safe guard/hysteresis policy. */
/* v21: ws_angle_widen forwarder for exact, aspect-scaled 12-bit terrain
* frustum half-angle constants. */
/* (PGXP, no version bump): OverlayCallbacks grew an appended-last PGXPHooks
* table pointer (pgxp_hooks.h). Only pgxp-flavour shards (compiled with
* -DPSX_PGXP=1, PSX_OVERLAY_FLAVOR_PGXP set) reference the psx_pgxp_*
* forwarders, and the flavor half of the ABI tag already rejects any
* host/DLL flavor mix — base-flavour DLLs and hosts are untouched, so
* the version stays. The emit-content change (PGXP_*() macros in all
* generated C) is covered by the codegen hash + CODEGEN_VER below. */
#define PSX_OVERLAY_ABI_VERSION 22
/* Process-lifetime overlay candidate capacity. Every accepted manifest F
* record consumes one slot, even when another DLL carries an identical
* (entry, CRC, ranges) identity. Offline publishers parse this same literal
* before adding a cache pair so compiler-side accounting cannot drift from
* the loader's fixed arrays. */
#define PSX_OVERLAY_CANDIDATE_CAP 32768
/* Codegen flavor of the recompiled output the overlays + runtime were built
* against. Overlays are keyed in the cache by guest-bytes CRC, which is
* flavor-BLIND — the same guest overlay yields the same filename whether it was
* compiled by the base recompiler or a variant (e.g. widescreen, which emits
* extra runtime symbols + GTE adjustments). Without a flavor tag a widescreen
* DLL and a base DLL for the same overlay collide. The flavor is folded into
* overlay_abi() (high 16 bits) so the loader's existing ABI gate rejects (and
* regenerates) any DLL whose flavor differs from the running build's, keeping
* caches separate even in a shared directory.
*
* Base/master MUST stay 0 so the tag == PSX_OVERLAY_ABI_VERSION (backward
* compatible with all existing base-flavor DLLs). A variant build overrides it
* (e.g. -DPSX_OVERLAY_FLAVOR=1 for widescreen) for BOTH the runtime and
* compile_overlays.py (its --flavor arg). */
#ifndef PSX_OVERLAY_FLAVOR
#define PSX_OVERLAY_FLAVOR 0
#endif
/* Flavor bits (compose bitwise; base = 0):
* 1 — widescreen variant codegen
* 2 — PGXP precision shadowing (-DPSX_PGXP=1 objects; pgxp_hooks.h) */
#define PSX_OVERLAY_FLAVOR_WIDESCREEN 1
#define PSX_OVERLAY_FLAVOR_PGXP 2
/* Combined tag exported by overlay_abi() and checked by the loader. */
#define PSX_OVERLAY_ABI_TAG \
((int)((PSX_OVERLAY_ABI_VERSION & 0xFFFF) | ((PSX_OVERLAY_FLAVOR & 0xFFFF) << 16)))
/* Codegen version — bumped whenever the recompiler emits DIFFERENT bytes for the
* same guest overlay WITHOUT an ABI change (the load-time overlay_abi() gate only
* catches ABI/flavor changes; a pure-emitter change produces a different DLL the
* gate would happily accept). The cache is namespaced by this version
* (gcc/<arch-abi>/cg<N>/), so a build with new codegen writes + reads a FRESH
* directory and never reuses a stale DLL; old versions coexist on disk (no
* auto-delete — a user who downgrades still finds their matching cache). Python
* (compile_overlays.py) parses this same constant from this header, so the two
* sides can never drift. BUMP THIS when you change code_generator.cpp emit in a
* way that alters overlay output (e.g. the auto_screen_x widescreen widening).
* 1: baseline + auto_screen_x render-funnel cull via psx_ws_cull_sltiu.
* 2: + auto_backdrop far-backdrop column preload via psx_ws_backdrop_value.
* 3: auto_backdrop switched to camera-tracked window WIDENING
* (psx_ws_backdrop_value gains a window_cols arg).
* 4: continuation-passing (PSX_CPS, RECURSION_BUG.md §25) — overlay funcs
* tail-transfer + carry an entry-switch; the loader routes continuations.
* Fresh namespace so CPS DLLs never reuse a legacy (unit-model) cg3 DLL.
* 6: + overlay store boundaries flush DLL-local pending cycles before host
* read/write side effects (SB/SH/SW/SWL/SWR/SWC2), configured negsub,
* masked-16-bit screen-X, and depth cull emits.
* 7: + configured side-plane normal-X (plane_nx) cull emit.
* 8: + configured per-primitive bound-load (xclip) cull emit.
* 9: + full-word-guarded object/model participation compare emit.
* 10: + PGXP dataflow-shadowing PGXP_*() macro emission at loads/stores/
* COP2 transfers/precision-carrying ALU ops (pgxp_hooks.h; no-ops in
* base-flavour objects, real hooks under -DPSX_PGXP=1). */
#define PSX_OVERLAY_CODEGEN_VER 10
/* Auto codegen hash (generated by hash_codegen.cmake from the recompiler codegen
* sources). Folded into the cache PATH next to cg<N> (gcc/<arch-abi>/cg<N>_<hash>/)
* so ANY emitter change auto-invalidates the cache, even within the same manual
* PSX_OVERLAY_CODEGEN_VER — closing the stale-but-cgN reuse that caused the v0.3.0
* black screen. The build writes overlay_codegen_hash.h (gitignored) into this
* dir; a fresh checkout without it falls back to 0 (== old single-cgN behaviour)
* until the first build. compile_overlays.py reads the same value, so the loader
* and the compiler always agree on the path. */
#if defined(__has_include)
# if __has_include("overlay_codegen_hash.h")
# include "overlay_codegen_hash.h"
# endif
#endif
#ifndef PSX_OVERLAY_CODEGEN_HASH
# define PSX_OVERLAY_CODEGEN_HASH 0u
#endif
typedef struct {
/* Core dispatch: routes call_by_address() and out-of-overlay jal */
void (*dispatch_call)(CPUState *cpu, uint32_t addr, uint32_t ra);
/* Interrupt check: called after every function return in overlay */
void (*check_interrupts)(CPUState *cpu);
/* Interrupt check with the guest PC that should resume if a game-installed
* handler later RFEs to the sentinel outside the synchronous host window. */
void (*check_interrupts_at)(CPUState *cpu, uint32_t resume_pc);
/* Guest-cycle accounting (ABI v7): block-cycle charge. Overlay code built
* with PSX_ENABLE_BLOCK_CYCLES must charge the SAME shared host cycle/timer
* timeline as the dirty-RAM interpreter and the BIOS, or timer-sensitive
* code reads different values per backend (Tomba2 logo Timer1 fork). */
void (*advance_cycles)(uint32_t cycles);
/* GTE coprocessor 2 execution */
void (*gte_execute)(CPUState *cpu, uint32_t cmd);
/* MIPS syscall (break/syscall instructions). Returns 1 if control
* transfers (cpu->pc set), 0 for a directly-handled void syscall. See
* cpu_state.h. The signature changed with CPS (RECURSION_BUG.md §25), so
* PSX_OVERLAY_ABI_VERSION was bumped to reject stale overlay caches. */
int (*psx_syscall)(CPUState *cpu, uint32_t code);
/* Unresolved dispatch target */
void (*psx_unknown_dispatch)(CPUState *cpu, uint32_t addr, uint32_t phys);
/* Debug instrumentation: called at every function entry (may be NULL) */
void (*log_call_entry)(uint32_t func_addr);
/* RestoreState/ReturnFromException longjmp escape (interrupts.c). The
* recompiler emits this at longjmp-return sites in exception-context
* kernel code (the install-slot / kernel-window class). Appended LAST:
* overlay_init copies the struct by value, so older DLLs built against
* the shorter struct simply never read this member. */
void (*psx_restore_state_escape)(void);
/* Call-contract state shared with the runtime (ABI v2; see the contract
* model in cpu_state.h). DLL code reads the bail flag and bumps the
* counters through these pointers. */
int *call_bail_flag;
uint64_t *bail_first;
uint64_t *bail_resolved;
/* Widescreen hooks (ABI v3). The recompiler can emit psx_ws_* calls into
* OVERLAY code (backdrop screenX squash; and, for other games, sprite-tag
* or cull sites that resolve into overlays). These compute against the
* host's live widescreen state (gpu.c), so the DLL forwards to the runtime
* through these. Appended LAST (struct grows back-compatibly); may be NULL
* on a host that predates them — the DLL glue falls back to identity. */
int (*ws_backdrop_x)(int x);
int (*ws_x_margin)(void);
void (*ws_sprite_tag)(CPUState *cpu);
/* Widescreen backdrop column-preload value substitution (ABI v4). Overlay
* backdrop generators call this at a detected window START/END bound; the
* runtime returns `orig` unless native-wide is engaged, in which case it
* widens the camera-tracked window by the 16:9 reveal (START -> orig-margin,
* END -> orig+margin; margin scales with window_cols). Appended LAST (struct
* grows back-compatibly); may be NULL on a host that predates it — the DLL
* glue falls back to orig. */
uint32_t (*ws_backdrop_value)(uint32_t orig, int is_end, int window_cols);
/* Fail-closed native entry guard (ABI v6). The generated CPS entry-switch
* calls this when the function is dispatched at a PC that is not one of its
* legal entries (a foreign interior PC from a range-ownership mismatch); the
* function returns without executing and the runtime routes the PC to the
* sanctioned dirty-RAM interpreter. Appended LAST (struct grows back-
* compatibly); may be NULL on a host that predates it. */
void (*psx_native_bad_entry)(CPUState *cpu, uint32_t owner, uint32_t pc);
/* Faithful-timing functions (ABI v9). The cycle-accuracy + due-cycle-scheduler
* rearchitecture made the recompiler emit these into OVERLAY code too (block-cycle
* mode). They touch shared runtime state (guest RAM, the global cycle counter, the
* I-cache tag array, the mul/div & GTE completion deadlines, the precise-slice
* interp), so the DLL MUST forward them to the runtime — a local copy would diverge
* from the interp/BIOS timeline (the exact class the cosim guards). Appended LAST
* (struct grows back-compatibly); the ABI bump to 9 rejects any pre-timing cache. */
uint32_t (*cyc_load_word)(CPUState *cpu, uint32_t addr, uint32_t rt, uint32_t reg_mask);
uint16_t (*cyc_load_half)(CPUState *cpu, uint32_t addr, uint32_t rt, uint32_t reg_mask);
uint8_t (*cyc_load_byte)(CPUState *cpu, uint32_t addr, uint32_t rt, uint32_t reg_mask);
uint32_t (*cyc_lwc2_read)(CPUState *cpu, uint32_t addr);
void (*icache_fetch)(CPUState *cpu, uint32_t addr);
void (*muldiv_set)(CPUState *cpu, uint32_t latency);
void (*muldiv_stall)(CPUState *cpu);
uint32_t (*mult_latency_s)(uint32_t rs);
uint32_t (*mult_latency_u)(uint32_t rs);
void (*gte_stall)(CPUState *cpu);
void (*gte_read)(CPUState *cpu, uint32_t rt);
int (*slice_block)(CPUState *cpu, uint32_t block_addr, uint32_t bcyc, int side_effects);
/* GTE special-register accessors (ABI v10). The emitter routes mfc2/cfc2/
* mtc2/ctc2 on flag/IR/derived registers through these (code_generator.cpp
* COP2 case) — a GTE-heavy overlay (Tomba2 lava/Trolley attract demos)
* emits direct calls and its DLL link fails without shims (undefined
* gte_read_ctrl — the class that silently blocked ALL new overlay coverage
* after the v9 caches rotated). Appended last; ABI bump to 10 rejects any
* pre-shim DLL. */
uint32_t (*gte_read_data)(CPUState *cpu, uint8_t reg);
uint32_t (*gte_read_ctrl)(CPUState *cpu, uint8_t reg);
void (*gte_write_data)(CPUState *cpu, uint8_t reg, uint32_t value);
void (*gte_write_ctrl)(CPUState *cpu, uint8_t reg, uint32_t value);
/* RFE escape marker (ABI v12; see the version-history note above).
* Appended last; forwards unconditionally — v12 hosts always supply it
* and the ABI gate rejects DLL/host mixes that don't. */
void (*rfe_mark_escape)(void);
/* Typed native-wide signed player-X bound (ABI v13; see the version-history
* note above). Computes against the host's live widescreen state (gpu.c
* psx_ws_player_x_bound). Appended last; may be NULL on a host that
* predates it — the DLL glue falls back to the vanilla constant (4:3). */
int32_t (*ws_player_x_bound)(int32_t vanilla);
/* Sub-pixel GTE precision-store tracker (ABI v14; see version-history note
* above). The emitter emits a direct gte_precision_store_word() call for
* every swc2 (GTE store-word); it is stateful runtime code (gte.cpp) so the
* DLL must forward. Without the shim any shard containing a swc2 fails to
* LINK (undefined gte_precision_store_word) and runs interpreted forever —
* the rfe/gte_read_ctrl class again. Surfaced by full static overlay
* coverage (play-captured shards rarely compiled a swc2). Appended last;
* may be NULL on a host that predates it — the shim no-ops (precision
* tracking is a widescreen enhancement, off by default). */
void (*gte_precision_store_word)(uint32_t addr, uint8_t reg);
/* Aspect-scaled far-bound helper (ABI v15). Runtime-state dependent, so
* overlay code forwards to the host; NULL preserves the vanilla bound. */
int32_t (*ws_depth_bound)(int32_t imm);
/* Side frustum-plane normal-X helper (ABI v17). Runtime-state dependent,
* so overlay code forwards to the host; NULL preserves the vanilla
* normal. */
int32_t (*ws_plane_nx)(int32_t nx);
/* Per-primitive X-reject bound-load helper (ABI v18). Runtime-state
* dependent, so overlay code forwards to the host; NULL preserves the
* vanilla loaded value. */
uint32_t (*ws_xclip_bound)(uint32_t vanilla);
/* Guarded object/model participation comparison helper (ABI v19).
* Runtime-state dependent; NULL preserves the vanilla comparison. */
uint32_t (*ws_cull_keep_result)(uint32_t vanilla, uint32_t forced);
/* Aspect-aware model-participation cone helper (ABI v20). */
uint32_t (*ws_aspect_cone_result)(uint32_t site, uint32_t vanilla,
uint32_t object, int32_t x,
int32_t z, int32_t y);
/* Aspect-scaled terrain-frustum angle helper (ABI v21). */
uint32_t (*ws_angle_widen)(uint32_t vanilla);
/* Stale-static guard (ABI v22; see the version-history note above).
* Defined by the generated dispatch shard in the executable, so overlay
* DLLs must forward. Appended last; may be NULL on a host that predates
* it - the shim then returns 0 and the guest takes the dispatcher path. */
int (*game_text_native_ok)(uint32_t addr);
/* PGXP dataflow-shadowing hook table (pgxp_hooks.h). Only pgxp-flavour
* shards (PSX_OVERLAY_FLAVOR pgxp bit, compiled with -DPSX_PGXP=1) emit
* calls to psx_pgxp_*; their preamble forwards through this table. Base-
* flavour shards never reference it. Appended LAST; NULL on an older host
* no-ops every hook — precision shadowing is visual-only, never
* load-bearing. The ABI version bump that arms this ships with the
* emitter change (Phase 2 of ENHANCEMENTS.md G1 value propagation). */
const PGXPHooks *pgxp;
} OverlayCallbacks;
#ifdef __cplusplus
extern "C" {
#endif
/* Exported by every overlay DLL. Call once after LoadLibrary. */
#ifdef _WIN32
__declspec(dllexport)
#else
__attribute__((visibility("default")))
#endif
void overlay_init(const OverlayCallbacks *cbs);
#ifdef __cplusplus
}
#endif
#endif /* OVERLAY_API_H */