Reconciled with #470 (PGXP): gpu.c precise vertices take PGXP only under geometry correction and count it, then native-wide edge recovery; the PGXP triangle stat stays PGXP-only. Stubs and tests from both kept; stamps recomputed.
A renderer that keeps a screen X as SX << 16 tests the right edge against
a lui-held W << 16 and the left edge with bltz on the shifted value. No
explicit [widescreen.cull] kind fits either: bltz_sites compares the whole
register with -margin, and a lui at a screen_x site was a build error.
screen_x_sites now also takes
lui rt, W -> rt = (W + margin) << 16
bltz x, outside -> taken while SX < -margin (x = SX << 16)
Both are the identity at 4:3. The helpers are defined for native code and
for overlay shards; their math is in ws_cull_edge.h. The two prototypes are
written only for a config that lists screen_x_sites, so the generated code
of every other title is unchanged.
Preserve current segmented dispatch, active-plan mod hooks and software scale limits. Extend function filters through the active plan, retain the last-store pointer with overlay ABI 25, and keep default config fingerprints stable. Refresh BIOS emitter stamps after byte-identical regeneration and reconcile regression fixtures.
The emitter writes both helpers at a listed bgez_sites or
clip_edge_x_load_sites address whose instruction matches in overlay code
too, but overlay_dispatch_preamble.c.inc did not define them, so such a
shard failed to link (macOS, Windows) or to load (Linux dlopen RTLD_NOW)
and its overlay stayed interpreted.
The preamble now defines both over psx_ws_x_margin(), with the same pure
math gpu.c uses (ws_cull_edge.h, stdint-only, staged with the other
runtime headers). ws_cull_edge.h joins the shard compile surface in
codegen_hash_sources.cmake. overlay_shim_compile_contract links a shim
that calls both with the emitter's extern declarations, and
overlay_widescreen_callbacks runs them in an executable: identity with no
margin callback, widened at margin 120, interior clip edges unchanged.
Both fail without the definitions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The emitter writes g_debug_last_store_pc before every guest store. The
overlay preamble gave each DLL a private copy on the theory that only debug
tooling reads it. The runtime actually reads it in every build:
- memory.c psx_write_word_raw keys its RAM 0x0-0xF store filters on it;
- the fingerprint pc/mmio/sp columns use it, and so do wtrace,
record_frame, the MMIO/GP1 rings, parity provenance and the freeze
heartbeat.
So after a native overlay store the host still named an older store, and
those consumers saw an instruction the interpreter would not have.
OverlayCallbacks gains last_store_pc, which the loader always points at
the runtime's copy. In PSX_OVERLAY_DLL_BUILD, cpu_state.h maps the
emitted name onto that pointer, the same pattern as the bail-state
pointers. Generated code is unchanged. A NULL host pointer keeps the old
private copy.
The ABI goes to 24, so the loader rejects and regenerates older shards.
The preamble change already moves the codegen-hash cache path.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The repository root carried seven markdown files that were not onboarding
material, plus two that had been superseded by the code they described. This
sorts them: reader-facing docs to docs/, working notes to docs/internal/, and
the two superseded ones merged into the docs that replaced them.
Moved to docs/: WIDESCREEN.md, ENHANCEMENTS.md, TCP_COMMANDS.md
Moved to docs/internal/: PRINCIPLES.md, ISSUES.md, DEBUG.md, GTE_LM_FIX.md,
and from docs/: bios-hardening.md, overlay-status.md,
overlay-plan.md, overlay-recompilation-design.md,
overlay-discovery.md, TOMBA2_EXC_CONSULT_RESPONSE.md,
CASE_A_AOT_GAP.md, audit_inventory.md
Moved to docs/internal/upstream/: UPSTREAM_2026-08-20.md
The root keeps only what belongs there: README.md, CONTRIBUTING.md, LICENSE,
CLAUDE.md, THIRD_PARTY_ATTRIBUTION.md, RELEASE_NOTES.md, the dotfiles and
psxrecomp_cli.py.
MERGES
SPLITGEN_MIGRATION.md folded into docs/BUILDING.md as "Split generated C". Its
migration instructions were stale in a way that mattered: it told every game
repo to hand-edit its CMakeLists and add a file(GLOB ...) over
<serial>_full*.c, but runtime/runtime.cmake now does that itself. GEN_FULL_GLOB
is a multi-value argument of psxrecomp_add_game_runtime(); the loop at
runtime.cmake:1985 makes each pattern absolute against CMAKE_CURRENT_SOURCE_DIR,
expands it with file(GLOB), and concatenates the hits into
GAME_GENERATED_FULL_C. So a title names the pattern and gets however many shards
a regeneration produced. The rewritten section describes that, the two-step
fallback when the globs match nothing (GEN_FULL_FALLBACK, else <serial>_full.c
derived from the dispatch marker by substituting _dispatch.c), and the one
caveat the code actually has: that file(GLOB) is plain, not CONFIGURE_DEPENDS,
so a regeneration which changes the shard COUNT needs a re-configure to be seen.
psxrecomp-toml-readme.md folded into docs/GAME_PROJECT_SETUP.md as "EXE-only
autofill (psxrecomp-toml)", placed after "Disc autofill (probe_disc.py)" and
before "Project Studio" so the two autofill paths sit together. Its documented
flags were checked against the usage text in recompiler/src/main_toml.cpp:28-42
and against the argument parser at :150-165 -- all seven options match with no
drift, so the flag list is reproduced as-is. Added a pointer that probe_disc.py
is preferable when a cue exists, since that tool does the same EXE-header work
plus disc identity and netplay gates.
FUNCTIONAL FIXES
tools/gen_tcp_commands.py wrote to REPO/"TCP_COMMANDS.md". After the move that
path does not exist, so the generator would have created a new stray root file
and --check would have failed against it forever. Retargeted to
REPO/"docs"/"TCP_COMMANDS.md", along with the seven docstring and message
strings that name the path.
Running it surfaced two further problems, both fixed here:
* The doc was genuinely stale -- pre-existing drift, not introduced by this
commit (the moved file is byte-identical to origin/master, and no
debug-server command table is touched here). Four commands registered since
the last regeneration were missing from the index -- cdrom_timing_dump,
savestate_status, ws_ui_groups, xprobe_watch -- and scanline had gained a
backend. Regenerated: 307 -> 311 commands, and the change is confined to the
AUTOGEN block (the hand-written head and tail are byte-identical to master).
tools/gen_tcp_commands.py --check now exits 0.
* DOC.write_text() has no newline argument, so Python translated every newline
to os.linesep. On Windows that rewrote all 585 lines of the file to CRLF and
turned a four-line index update into a whole-file diff -- measured: 589 CRLF
/ 0 LF after a run, against 0 CRLF / 585 LF in the repo. Now written through
an explicit newline="" handle, verified to leave the file at 0 CRLF.
runtime/include/debug_server.h:9 and runtime/src/debug_server.c:8 cited "TCP.md
and DEBUG.md". TCP.md has never existed in this repo under that name; both now
name docs/TCP_COMMANDS.md and docs/internal/DEBUG.md.
LINK AND REFERENCE FIXES
Every inbound reference to a moved file was located by grepping each basename
across the whole tracked tree, not just the markdown. Markdown hrefs in
README.md, CONTRIBUTING.md, docs/ARCHITECTURE.md, docs/COMPILING_OVERLAYS.md,
docs/AOT_OVERLAY_PLAN.md, docs/RELOCATION_MANIFEST_FORMAT.md,
docs/config_schema.md, docs/OVERLAY_XREF.md, docs/TCP_COMMANDS.md and several
docs/internal/ files now point at the new locations; where two files became
siblings under docs/internal/ the path was shortened rather than lengthened
(bios-hardening.md's reference to overlay-discovery.md, for instance).
docs/COMPILING_OVERLAYS.md's "in this folder" phrasing was wrong once two of the
three files it listed moved, so it now names internal/ explicitly.
24 source-comment mentions of ENHANCEMENTS.md sections G1.x across 19 files
(recompiler, runtime, cmake) were retargeted to docs/ENHANCEMENTS.md with the
section numbers untouched. docs/SHADOW_ENHANCEMENTS.md is a DIFFERENT file and
was deliberately excluded.
20 rationale strings citing docs/CASE_A_AOT_GAP.md across the four
recompiler/seeds/*.json files were updated. Done as a byte-level substring
replacement rather than a JSON round-trip: two of those files are stored with
CRLF endings and two with LF, and re-serializing them reformatted 5,700 lines.
The byte-level version changes exactly 20 lines and each file still parses.
References that look similar but point elsewhere were deliberately left alone,
and are worth naming so a future pass does not "fix" them:
* "recomp-template PRINCIPLES.md" / "NES recomp template PRINCIPLES.md" in
CLAUDE.md:43, docs/SHADOW_ENHANCEMENTS.md:28,
docs/internal/HLE_SCHEDULER_CARVEOUT_PLAN.md:5,
runtime/src/debug_server.c:4934 and tools/parity_diff.py:8 are a different
repository's file. Only the five that mean this repo's PRINCIPLES.md were
updated.
* Every remaining bare ISSUES.md mention is a GAME repo's tracker
(MegaManX6Recomp 2/7/8/9, TombaRecomp 6), not this repo's.
* docs/disc-speed.md:4,54,56 and docs/internal/bios-hardening.md:5 name
overlay-discovery as a BRANCH, not a file, so no path applies.
INDEXES
README.md's "New here?" table gains rows for docs/WIDESCREEN.md,
docs/ENHANCEMENTS.md and docs/TCP_COMMANDS.md. docs/internal/README.md gains
sections indexing all fourteen documents that moved in, plus an "Upstream
contributor reviews" section describing upstream/ and the new sweep file, plus a
"Removed" bullet for CYCLE_TIMING_ARCH.md (deleted in the previous commit)
pointing at FAITHFUL_TIMING_PLAN.md, CYCLE_MODEL_BEETLE.md and
PRECISE_IRQ_SLICE.md -- following the SLJIT_PERSIST_CACHE.md precedent already
in that file.
docs/ENHANCEMENTS.md:7 claimed framework bugs live in "game repos' ISSUES.md
until a framework tracker exists". A framework tracker does exist; it now says
so and names the Beads epic.
VERIFICATION
A link checker was written to scan every tracked *.md for relative links,
resolving each against the tree being scanned. Baseline at origin/master: 129
files, 137 relative links, 3 broken. After both commits: 127 files, 159 relative
links, 3 broken -- the SAME three, all pre-existing and untouched here:
docs/DATA_SHARDS.md -> INSTANT_LOADS_PARKED.md (never existed)
docs/DATA_SHARDS.md -> INSTANT_LOADS_SPIKE2.md (never existed)
docs/NETPLAY.md -> ../lib/recomp-net/README.md (inside a submodule)
Zero new broken links. The checker's own coverage was checked against a raw grep
for markdown link openers: the 3-link difference is C++ lambdas inside code
blocks, correctly not treated as links.
Also verified: py_compile over all 113 tracked tools/*.py plus psxrecomp_cli.py;
all four seed JSONs still parse; the test-registration guard still reports 123
test files all registered; recompiler/tests/test_coverage_vault.py still passes;
and no file in this commit had its line endings flipped (checked mechanically
across all 50 modified files, after two files were caught and redone).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The emitter emits a direct psx_game_text_native_ok() stale-static guard at block
entry (code_generator.cpp:115) and before every direct jal (:2402). That function
is DEFINED IN THE GENERATED DISPATCH SHARD (main_psx.cpp:1620), which links into
the executable. An overlay DLL is a separate module and cannot resolve it, and
overlay_dispatch_preamble.c.inc had no shim for it.
Found by bumping Tomba 2 from a 254-commit-old pin to master and regenerating its
shard cache against the identical vault:
before bump : built OK 2799, FAILED 1
after bump : built OK 2233, FAILED 109
107 of the 109 were the same error:
ld.exe: undefined reference to `psx_game_text_native_ok'
collect2.exe: error: ld returned 1 exit status
This is the fifth instance of one class, and overlay_api.h's own version history
records the previous four:
v9 faithful-timing callbacks overlay code failed to link
v10 gte_read_ctrl GTE-heavy overlays failed to LINK without shims,
silently blocking new coverage
v12 rfe_mark_escape without the shim any rfe-containing overlay
fails to LINK
v14 gte_precision_store_word the rfe / gte_read_ctrl class again
Every time the shape is identical: an emitter gains a runtime call, the overlay
ABI does not gain the shim, and the only symptom is shards that quietly fail to
build. The affected code then runs interpreted forever and nothing at runtime
says so. Severity is coverage, not correctness -- but that is exactly the defect
that let Ape Escape ship 100% interpreted without anyone noticing.
Fix follows the v10/v12/v14 pattern exactly:
- overlay_api.h: append game_text_native_ok to OverlayCallbacks, bump
PSX_OVERLAY_ABI_VERSION 21 -> 22, add the version-history note
- overlay_dispatch_preamble.c.inc: shim forwarding to g_cbs
- overlay_loader.c init_callbacks: host populates it from the dispatch shard
The NULL guard deliberately returns 0, not 1. Returning 0 routes the guest
through the dispatcher -- slower, never wrong. Returning 1 would let an overlay
run native code the guard was explicitly trying to reject.
Follow-up worth doing, tracked as beads-eio.3.97: five instances of one class
means the process is the bug, not the symbol. The emitter knows every helper name
it can emit and the preamble enumerates every shim; a test comparing those two
sets would have caught v9, v10, v12, v14 and this one at authoring time instead
of at some future title's link step.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Emitter side of the value-propagation engine (ENHANCEMENTS.md G1.10):
- recompiler/src/pgxp_hook_emitter.{h,cpp} (new, in the codegen-hash
surface): ONE hook grammar shared by BOTH emitters, wrapping loads/
stores/COP2 transfers and precision-carrying ALU ops (moves, add/sub,
or-merge, shifts, LUI, HI/LO, muldiv) with PGXP_*() macros. Clobberable
sources are captured into locals before the statement. Precision-
destroying ops (AND/XOR/SLT/...) are deliberately unhooked - validate-
on-read drops their stale shadows for free.
- code_generator.cpp: central append in translate_instruction; LWC2/SWC2
emissions capture the RAW loaded/stored word (a masked GTE register
write must validate against the word as loaded).
- strict_translator.cpp: translate() wraps translate_impl()'s c_code with
the shared grammar; deferred load-delay variants hook the psx_ldd temp.
- BIOS regenerated, fingerprints verified CHANGED (SCPH1001 + OpenBIOS;
42.6k/12.2k macro sites). Base runtime + 33/33 tests green against it.
- Gating: macros expand only under -DPSX_PGXP=1. runtime.cmake grows a
PGXP option on psxrecomp_add_runtime_target (defines PSX_PGXP=1 +
PSX_OVERLAY_FLAVOR=2, _pgxp exe suffix) and PSX_PGXP_VARIANT to build
the sibling target from the same generated tree (framework runtime +
game wrapper). No ABI version bump needed: the appended PGXPHooks
table is NULL-tolerant and the flavor half of the ABI tag rejects any
host/DLL flavor mix.
- Shard caches: flavor folded into the cache tag (cg<V>_<hash>_gc<hash>
_f<flavor>) in overlay_loader.c + compile_overlays.py so base and pgxp
caches never share a directory; compile_overlays.py passes
-DPSX_PGXP=1 when flavor bit 2 is set; overlay preamble forwards
psx_pgxp_* through the table. PSX_OVERLAY_CODEGEN_VER 9 -> 10.
- Env overrides PSX_GEOMETRY_CORRECTION / PSX_PERSPECTIVE_TEXTURING /
PSX_PGXP_CPU_MODE (validation path: free-running headless boots arm
from the first vertex); census grew produced/swc2_stores and the pgxp
verb reports suppress/active gate state.
Validated: PSXRecomp_pgxp.exe boots the recompiled BIOS with hooks armed
(engine active, per-vertex lookups flowing; the BIOS shell has no RTPS
content so produced=0 there is correct - coverage measurement moves to a
3D title in phase 3).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add opt-in [widescreen.cull] xclip_load_sites: a configured `lw rt,off(rs)`
loading a per-primitive X-reject bound (sltu-compared against ANDI-masked
u16 screen X) routes through psx_ws_xclip_bound() - INT32_MAX while the
margins are revealed (reject disabled; the wide-surface scissor clips the
overflow, wrapped off-left u16 coords pass), vanilla value at 4:3. Matched
native codegen emit and interpreter hook; overlay ABI v18 forwarder (NULL
preserves the vanilla loaded value); codegen ver 8. Parser, codegen and
overlay-mismatch regression coverage.
Split from #49: replaces the xclip_globals RAM poke with a recompile-time
instruction transform so native shards and the interpreter behave identically.
Add opt-in [widescreen.cull] plane_nx_sites: a configured `lw rt,off(rs)`
loading the X component of a side frustum-plane normal (sign-tested per
corner, dot = nx*px + nz*pz) routes through psx_ws_plane_nx() — scaled by
the inverse aspect factor (4*den)/(3*num) while the margins are revealed,
widening the plane cone by exactly atan((3*num)/(4*den)*tan(theta));
identity at 4:3. Matched native codegen emit and interpreter hook; overlay
ABI v17 forwarder (NULL preserves the vanilla normal); codegen ver 7.
Parser, codegen and overlay-mismatch regression coverage.
Split from #49: replaces the plane sign-test RAM pokes with a
recompile-time instruction transform so native shards and the interpreter
behave identically.
Add opt-in slti/sltiu far-bound scaling with matched native and interpreter semantics, correct immediate sign extension, and an overlay ABI forwarder for live aspect state.
Co-authored-by: Opok <novelasevilliousesp@gmail.com>
Two provably-correct framework fixes (independent of the AOT work) plus the
play-free overlay-extraction spike tooling and design doc.
Framework fixes (accurate, safe, complete):
- ABI v14: add gte_precision_store_word to OverlayCallbacks + preamble shim +
runtime population. The emitter emits a direct call for every swc2 (GTE
store-word); it is stateful runtime code (gte.cpp) so overlay DLLs must
forward or fail to LINK (undefined reference). Play-captured shards rarely
compiled a swc2; fuller static coverage surfaced the gap. Same pattern as
the v13 ws_player_x_bound / v10 gte_read_ctrl shims.
- Gate splitgen OFF in overlay mode (main_psx.cpp): the current recompiler's
unconditional split-TU output deletes <stem>_full.c and emits only shards,
which broke ALL overlay compilation (compile_overlays.py reads _full.c).
Overlays are single small TUs with no parallel-compile benefit; emit the
monolith. Restores overlay compile on the current tree.
AOT spike (docs/AOT_OVERLAY_PLAN.md + tools/aot_overlay_spike/, WIP):
- Play-free static overlay extraction from the disc image. Validated:
Tomba 1 reproduces 90% of played functions + ~1964 unplayed, live-verified
in-game (Dwarf Village runs native, zero runtime compile). Tomba 2 yields
661 functions byte-identical to the shipped/played vault (provably real),
all from the self-describing MAIN.EXE producer; the header-table BIN
producer needs loader-table RE (base recovery unreliable). Generality is
producer-dependent. Spike scripts are Tomba-specific proofs of concept
with hardcoded paths, not a finished tool.
- Correctness is guaranteed by the existing per-function code_crc dispatch
guard: a mis-positioned shard fails audit at compile or never fires at
runtime -> coverage loss, never wrong execution. Autocompile self-heals gaps.
CROSS-CUTTING: the ABI v14 change is part of the overlay codegen-hash source
set, so it ROLLS the cg tag (1f2ad026 -> e8484213). On master + a fw bump,
every title's overlay cache invalidates and needs regen+reshard. Branch only;
NOT pushed; other games' caches untouched. Needs user validation before merge.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
28165d6 (Einhander native-wide player bounds) rewrites configured
[widescreen] signed_x_bound LUI sites into psx_ws_player_x_bound(vanilla)
calls in code_generator.cpp — and that site loop runs in overlay_mode too, so
the call lands in overlay-resident code. But the shard link contract never
forwarded the symbol: any shard containing a bound site fails to LINK
(undefined psx_ws_player_x_bound) and that variant runs interpreted forever —
the same silent class as v10 gte_read_ctrl and v12 psx_rfe_mark_escape (now
loud under the shard observability suite, but still a coverage hole).
The clamp computes against the host's live widescreen state (gpu.c), so a
local copy can't work: add the ws_player_x_bound callback (appended last),
the preamble forwarder (identity fallback => correct 4:3 on a host that
predates it), and the loader init wiring. Bump the ABI to 13 so stale DLL
mixes are rejected and regenerate.
This also unblocks removing the sljit tier: sljit's emit_ws_player_x_bound
was the only other native path carrying the clamp; with the gcc shard path
forwarding it (and dirty_ram_interp already handling the site), the feature
no longer depends on sljit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Re-land of the shard-observability suite from 6da8c02 (stale branch), merged
against current master (keeps master's overlay_ext() platform suffixes,
recompiler_patch hash sources, and the unconditional v12 rfe forwarder).
- compile_overlays.py: ShardStats tallies every shard outcome — per-shard
"SHARD FAIL [class]" lines, a SHARD BUILD SUMMARY, a machine-readable
"PSX_SHARD_RESULT ok=N failed=M skipped=K" line, and a non-zero exit when
any shard that should have built failed (previously a run that built zero
shards exited 0, indistinguishable from a clean run).
- New --check preflight: build every shard into a throwaway dir and report
pass/fail — answers "would this game's shards compile against the current
recompiler + headers?" without a runtime.
- Retain audit-failed interior-fragment C (<phys>_<interior>_fragment_FAILED.c)
for triage, plus a persistent doomed-interior memo (interior_fail_memo.txt)
keyed (phys,interior,region_crc) so autocompile stops re-walking
data-as-code fragments every cycle (~90x off the V8 fragment pass).
- Extract DISPATCH_PREAMBLE to runtime/include/overlay_dispatch_preamble.c.inc
(single source of truth for the shard link contract, loaded by
load_dispatch_preamble(); missing file is a hard error).
- codegen_hash_sources.cmake: fold the shard COMPILE surface (overlay_api.h,
cpu_state.h, the .inc) into the cg tag so header/ABI/link-contract drift
reshards same-tree instead of silently breaking every new shard compile.
- autocompile.c: parse PSX_SHARD_RESULT from the child-output tail, expose
shard_ok/shard_fail/shard_skipped/shard_fail_total over autocompile_status;
ALWAYS rescan on completion so shards that did build load even when the
run also reported failures.
Live proof on the stale branch: Tomba's first autocompile surfaced 9 shard
link failures (undefined psx_rfe_mark_escape — since fixed by v12) and V8
surfaced 91 benign data-as-code fragment failures, all previously silent.
Codegen hash changes => every title reshards once on next build (expected;
loader rejects/regenerates stale shards).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Matthew Stanley