The repository root carried seven markdown files that were not onboarding
material, plus two that had been superseded by the code they described. This
sorts them: reader-facing docs to docs/, working notes to docs/internal/, and
the two superseded ones merged into the docs that replaced them.
Moved to docs/: WIDESCREEN.md, ENHANCEMENTS.md, TCP_COMMANDS.md
Moved to docs/internal/: PRINCIPLES.md, ISSUES.md, DEBUG.md, GTE_LM_FIX.md,
and from docs/: bios-hardening.md, overlay-status.md,
overlay-plan.md, overlay-recompilation-design.md,
overlay-discovery.md, TOMBA2_EXC_CONSULT_RESPONSE.md,
CASE_A_AOT_GAP.md, audit_inventory.md
Moved to docs/internal/upstream/: UPSTREAM_2026-08-20.md
The root keeps only what belongs there: README.md, CONTRIBUTING.md, LICENSE,
CLAUDE.md, THIRD_PARTY_ATTRIBUTION.md, RELEASE_NOTES.md, the dotfiles and
psxrecomp_cli.py.
MERGES
SPLITGEN_MIGRATION.md folded into docs/BUILDING.md as "Split generated C". Its
migration instructions were stale in a way that mattered: it told every game
repo to hand-edit its CMakeLists and add a file(GLOB ...) over
<serial>_full*.c, but runtime/runtime.cmake now does that itself. GEN_FULL_GLOB
is a multi-value argument of psxrecomp_add_game_runtime(); the loop at
runtime.cmake:1985 makes each pattern absolute against CMAKE_CURRENT_SOURCE_DIR,
expands it with file(GLOB), and concatenates the hits into
GAME_GENERATED_FULL_C. So a title names the pattern and gets however many shards
a regeneration produced. The rewritten section describes that, the two-step
fallback when the globs match nothing (GEN_FULL_FALLBACK, else <serial>_full.c
derived from the dispatch marker by substituting _dispatch.c), and the one
caveat the code actually has: that file(GLOB) is plain, not CONFIGURE_DEPENDS,
so a regeneration which changes the shard COUNT needs a re-configure to be seen.
psxrecomp-toml-readme.md folded into docs/GAME_PROJECT_SETUP.md as "EXE-only
autofill (psxrecomp-toml)", placed after "Disc autofill (probe_disc.py)" and
before "Project Studio" so the two autofill paths sit together. Its documented
flags were checked against the usage text in recompiler/src/main_toml.cpp:28-42
and against the argument parser at :150-165 -- all seven options match with no
drift, so the flag list is reproduced as-is. Added a pointer that probe_disc.py
is preferable when a cue exists, since that tool does the same EXE-header work
plus disc identity and netplay gates.
FUNCTIONAL FIXES
tools/gen_tcp_commands.py wrote to REPO/"TCP_COMMANDS.md". After the move that
path does not exist, so the generator would have created a new stray root file
and --check would have failed against it forever. Retargeted to
REPO/"docs"/"TCP_COMMANDS.md", along with the seven docstring and message
strings that name the path.
Running it surfaced two further problems, both fixed here:
* The doc was genuinely stale -- pre-existing drift, not introduced by this
commit (the moved file is byte-identical to origin/master, and no
debug-server command table is touched here). Four commands registered since
the last regeneration were missing from the index -- cdrom_timing_dump,
savestate_status, ws_ui_groups, xprobe_watch -- and scanline had gained a
backend. Regenerated: 307 -> 311 commands, and the change is confined to the
AUTOGEN block (the hand-written head and tail are byte-identical to master).
tools/gen_tcp_commands.py --check now exits 0.
* DOC.write_text() has no newline argument, so Python translated every newline
to os.linesep. On Windows that rewrote all 585 lines of the file to CRLF and
turned a four-line index update into a whole-file diff -- measured: 589 CRLF
/ 0 LF after a run, against 0 CRLF / 585 LF in the repo. Now written through
an explicit newline="" handle, verified to leave the file at 0 CRLF.
runtime/include/debug_server.h:9 and runtime/src/debug_server.c:8 cited "TCP.md
and DEBUG.md". TCP.md has never existed in this repo under that name; both now
name docs/TCP_COMMANDS.md and docs/internal/DEBUG.md.
LINK AND REFERENCE FIXES
Every inbound reference to a moved file was located by grepping each basename
across the whole tracked tree, not just the markdown. Markdown hrefs in
README.md, CONTRIBUTING.md, docs/ARCHITECTURE.md, docs/COMPILING_OVERLAYS.md,
docs/AOT_OVERLAY_PLAN.md, docs/RELOCATION_MANIFEST_FORMAT.md,
docs/config_schema.md, docs/OVERLAY_XREF.md, docs/TCP_COMMANDS.md and several
docs/internal/ files now point at the new locations; where two files became
siblings under docs/internal/ the path was shortened rather than lengthened
(bios-hardening.md's reference to overlay-discovery.md, for instance).
docs/COMPILING_OVERLAYS.md's "in this folder" phrasing was wrong once two of the
three files it listed moved, so it now names internal/ explicitly.
24 source-comment mentions of ENHANCEMENTS.md sections G1.x across 19 files
(recompiler, runtime, cmake) were retargeted to docs/ENHANCEMENTS.md with the
section numbers untouched. docs/SHADOW_ENHANCEMENTS.md is a DIFFERENT file and
was deliberately excluded.
20 rationale strings citing docs/CASE_A_AOT_GAP.md across the four
recompiler/seeds/*.json files were updated. Done as a byte-level substring
replacement rather than a JSON round-trip: two of those files are stored with
CRLF endings and two with LF, and re-serializing them reformatted 5,700 lines.
The byte-level version changes exactly 20 lines and each file still parses.
References that look similar but point elsewhere were deliberately left alone,
and are worth naming so a future pass does not "fix" them:
* "recomp-template PRINCIPLES.md" / "NES recomp template PRINCIPLES.md" in
CLAUDE.md:43, docs/SHADOW_ENHANCEMENTS.md:28,
docs/internal/HLE_SCHEDULER_CARVEOUT_PLAN.md:5,
runtime/src/debug_server.c:4934 and tools/parity_diff.py:8 are a different
repository's file. Only the five that mean this repo's PRINCIPLES.md were
updated.
* Every remaining bare ISSUES.md mention is a GAME repo's tracker
(MegaManX6Recomp 2/7/8/9, TombaRecomp 6), not this repo's.
* docs/disc-speed.md:4,54,56 and docs/internal/bios-hardening.md:5 name
overlay-discovery as a BRANCH, not a file, so no path applies.
INDEXES
README.md's "New here?" table gains rows for docs/WIDESCREEN.md,
docs/ENHANCEMENTS.md and docs/TCP_COMMANDS.md. docs/internal/README.md gains
sections indexing all fourteen documents that moved in, plus an "Upstream
contributor reviews" section describing upstream/ and the new sweep file, plus a
"Removed" bullet for CYCLE_TIMING_ARCH.md (deleted in the previous commit)
pointing at FAITHFUL_TIMING_PLAN.md, CYCLE_MODEL_BEETLE.md and
PRECISE_IRQ_SLICE.md -- following the SLJIT_PERSIST_CACHE.md precedent already
in that file.
docs/ENHANCEMENTS.md:7 claimed framework bugs live in "game repos' ISSUES.md
until a framework tracker exists". A framework tracker does exist; it now says
so and names the Beads epic.
VERIFICATION
A link checker was written to scan every tracked *.md for relative links,
resolving each against the tree being scanned. Baseline at origin/master: 129
files, 137 relative links, 3 broken. After both commits: 127 files, 159 relative
links, 3 broken -- the SAME three, all pre-existing and untouched here:
docs/DATA_SHARDS.md -> INSTANT_LOADS_PARKED.md (never existed)
docs/DATA_SHARDS.md -> INSTANT_LOADS_SPIKE2.md (never existed)
docs/NETPLAY.md -> ../lib/recomp-net/README.md (inside a submodule)
Zero new broken links. The checker's own coverage was checked against a raw grep
for markdown link openers: the 3-link difference is C++ lambdas inside code
blocks, correctly not treated as links.
Also verified: py_compile over all 113 tracked tools/*.py plus psxrecomp_cli.py;
all four seed JSONs still parse; the test-registration guard still reports 123
test files all registered; recompiler/tests/test_coverage_vault.py still passes;
and no file in this commit had its line endings flipped (checked mechanically
across all 50 modified files, after two files were caught and redone).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
5.9 KiB
BIOS Hardening
Date: 2026-05-30
Branch: bios-hardening
Depends on: overlay-discovery (merged to master)
What shipped in overlay-discovery
- Static A0/B0/C0 dispatch — BIOS vector trampolines pre-encoded from ROM tables; zero dirty_ram_interp at runtime for all syscall vectors.
- 0xBFC15F1C seed — PAD/SIO interrupt handler compiled to static C; ~50K dirty_ram blocks eliminated. One seed, CFG discovers all internal branch targets.
- SIO 0x0CF0 alias — trampoline at 0x0CF0 (lui+addiu+jalr to 0x641C) registered as a direct dispatch alias; 30K dirty_ram blocks eliminated.
- Disc speed 4x — post-BIOS switch via game entry_pc, XA streaming guard for FMVs.
- 75% dirty_ram reduction (119K → 29K blocks/session with Tomba save load).
Outstanding BIOS-side work
1. Disc speed "instant" hang — root cause unknown
disc_speed = "instant" (divisor=0, 1-cycle floor) hangs the game at the PlayStation
logo (post-Sony-logo BIOS screen, before game starts). disc_speed = "4x" is stable.
What we know:
- Sony logo passes cleanly (entry_pc trigger fires correctly post-BIOS-boot).
- Hang occurs during early game initialization, meaning the game's disc reads are involved.
- 500-cycle floor tried, still hangs. 4x (divisor=4, ~112K cycle floor) is stable.
Likely cause: game's early disc read loop has a tight IRQ poll with a retry counter that expires before the interrupt handler completes at sub-4x speeds.
Next step: binary search the minimum safe divisor. Try divisor=8 (2x faster than working 4x). If stable, try 16, 32, etc. until hang reproduced, then back off one step. The minimum safe value tells us the minimum inter-IRQ gap the game requires.
Longer-term fix: instrument the CD-ROM IRQ delivery path to log inter-IRQ intervals at runtime and compare against the game's retry counter depth.
2. Dispatch hit/miss counters
Discussed but not implemented. Currently no way to measure static dispatch hit rate (binary-search hit) vs dirty_ram fallback without adding counters.
Implementation: emit g_dispatch_static_hits (uint64_t) increment at the binary
search hit in emit_dispatch(), expose via existing dispatch_stats TCP command
(alongside dispatch_miss_total). Two lines in the generated dispatch + one TCP handler
extension.
Value: lets us verify static coverage quantitatively after any seed change. Should show near-100% static hits for BIOS functions after our work.
3. Exception chain continuations (0x0DF8, 0x0E08)
Measured: 7711 hits × 2 insns each = 15K interpreted instructions per session. Low priority but completable.
What they are: mid-function continuation points in the BIOS exception chain walker.
The walker calls chained handlers via jalr s1; these addresses are the continuations
after each call returns. They show up in dirty_ram because they're in the dirty kernel
copy pages and the recompiler never sees them as function starts.
Fix: find the containing function (the exception chain walker itself) and verify it's already seeded. If it is, these continuations should already be compiled as internal labels within that function. If not, seed the function start. Do not seed these continuation addresses directly (they're mid-function and will crash the recompiler).
4. Trampoline resolution logging gap
psx_unknown_dispatch in traps.c has 5 trampoline patterns (J, JR, lui+addiu+jr,
lui+ori+jr, A0/B0/C0 vector dispatch). When a pattern matches, it resolves silently and
returns — no counter, no ring buffer entry.
This means we have no visibility into how many dispatches go through trampoline resolution
vs the ring buffer vs dirty_ram. Add a g_trampoline_resolutions counter and expose it
on the dispatch_stats TCP command alongside dispatch_miss_total.
#3 — Overlay discovery (B+A) — next after Tomba seeding
Once Tomba seeding is complete, the remaining dirty_ram traffic will be genuinely overlay code (addresses > 0x98000, loaded from disc). That's when B+A slots in.
See overlay-discovery.md for the full architecture. Short version:
- Layer B (ship first): on CD DMA completion, log
(hash, load_addr, size)tologs/<SCUS_ID>/overlay_map.jsonl. At next build, recompile those overlays statically. No runtime linker needed. - Layer A (after B has real coverage): on DMA completion, check a per-game cache
of compiled DLLs. Cache hit →
LoadLibrary+ register. Cache miss → dirty-RAM fallback + background compile → cache populated for next encounter.
B+A starts after Tomba seeding because: (a) seeding eliminates the non-overlay dirty_ram noise so you know exactly what's left, and (b) Layer B's first real overlay log needs clean data to hash against.
Tomba game dispatch misses (game-side, seeding)
Captured 16 dirty_ram dispatch misses in the Tomba game text range (0x10000-0x98000).
Saved to TombaRecomp/seeds/dirty_ram_misses.txt. The 10 high-hit addresses (~38K
hits/session each) are the priority.
Before seeding: verify each address in Ghidra (SCUS_942.36_no_header) is a true function start (look for stack frame prologue or confirmed jal/jalr call site). Do NOT seed mid-function addresses — this caused a boot hang in this session.
High-hit addresses to verify:
0x8004DF00 38837 hits, 194185 insns
0x8004DF7C 38837 hits, 310696 insns
0x8002D4C8 38837 hits, 194185 insns
0x8002D4E0 38837 hits, 233022 insns
0x8002D524 38837 hits, 116511 insns
0x8004DF68 38837 hits, 53860 insns
0x8002D504 38837 hits, 38837 insns
0x8004DF28 38837 hits, 13465 insns
0x8004DF30 38837 hits, 13465 insns
0x8002D4FC 38837 hits, 13465 insns
All 10 have identical hit counts (38837) suggesting they are all called together — likely internal branch targets of 1-2 functions. Seed only the entry points; CFG will discover the rest.
Tools: tools/collect_game_misses.py accumulates new addresses each session.
Run after every gameplay session to grow the log before seeding.