mirror of
https://github.com/RetroPortingToolKit/psxrecomp
synced 2026-10-25 18:05:12 -04:00
e6ce5006b4
The repository root carried seven markdown files that were not onboarding
material, plus two that had been superseded by the code they described. This
sorts them: reader-facing docs to docs/, working notes to docs/internal/, and
the two superseded ones merged into the docs that replaced them.
Moved to docs/: WIDESCREEN.md, ENHANCEMENTS.md, TCP_COMMANDS.md
Moved to docs/internal/: PRINCIPLES.md, ISSUES.md, DEBUG.md, GTE_LM_FIX.md,
and from docs/: bios-hardening.md, overlay-status.md,
overlay-plan.md, overlay-recompilation-design.md,
overlay-discovery.md, TOMBA2_EXC_CONSULT_RESPONSE.md,
CASE_A_AOT_GAP.md, audit_inventory.md
Moved to docs/internal/upstream/: UPSTREAM_2026-08-20.md
The root keeps only what belongs there: README.md, CONTRIBUTING.md, LICENSE,
CLAUDE.md, THIRD_PARTY_ATTRIBUTION.md, RELEASE_NOTES.md, the dotfiles and
psxrecomp_cli.py.
MERGES
SPLITGEN_MIGRATION.md folded into docs/BUILDING.md as "Split generated C". Its
migration instructions were stale in a way that mattered: it told every game
repo to hand-edit its CMakeLists and add a file(GLOB ...) over
<serial>_full*.c, but runtime/runtime.cmake now does that itself. GEN_FULL_GLOB
is a multi-value argument of psxrecomp_add_game_runtime(); the loop at
runtime.cmake:1985 makes each pattern absolute against CMAKE_CURRENT_SOURCE_DIR,
expands it with file(GLOB), and concatenates the hits into
GAME_GENERATED_FULL_C. So a title names the pattern and gets however many shards
a regeneration produced. The rewritten section describes that, the two-step
fallback when the globs match nothing (GEN_FULL_FALLBACK, else <serial>_full.c
derived from the dispatch marker by substituting _dispatch.c), and the one
caveat the code actually has: that file(GLOB) is plain, not CONFIGURE_DEPENDS,
so a regeneration which changes the shard COUNT needs a re-configure to be seen.
psxrecomp-toml-readme.md folded into docs/GAME_PROJECT_SETUP.md as "EXE-only
autofill (psxrecomp-toml)", placed after "Disc autofill (probe_disc.py)" and
before "Project Studio" so the two autofill paths sit together. Its documented
flags were checked against the usage text in recompiler/src/main_toml.cpp:28-42
and against the argument parser at :150-165 -- all seven options match with no
drift, so the flag list is reproduced as-is. Added a pointer that probe_disc.py
is preferable when a cue exists, since that tool does the same EXE-header work
plus disc identity and netplay gates.
FUNCTIONAL FIXES
tools/gen_tcp_commands.py wrote to REPO/"TCP_COMMANDS.md". After the move that
path does not exist, so the generator would have created a new stray root file
and --check would have failed against it forever. Retargeted to
REPO/"docs"/"TCP_COMMANDS.md", along with the seven docstring and message
strings that name the path.
Running it surfaced two further problems, both fixed here:
* The doc was genuinely stale -- pre-existing drift, not introduced by this
commit (the moved file is byte-identical to origin/master, and no
debug-server command table is touched here). Four commands registered since
the last regeneration were missing from the index -- cdrom_timing_dump,
savestate_status, ws_ui_groups, xprobe_watch -- and scanline had gained a
backend. Regenerated: 307 -> 311 commands, and the change is confined to the
AUTOGEN block (the hand-written head and tail are byte-identical to master).
tools/gen_tcp_commands.py --check now exits 0.
* DOC.write_text() has no newline argument, so Python translated every newline
to os.linesep. On Windows that rewrote all 585 lines of the file to CRLF and
turned a four-line index update into a whole-file diff -- measured: 589 CRLF
/ 0 LF after a run, against 0 CRLF / 585 LF in the repo. Now written through
an explicit newline="" handle, verified to leave the file at 0 CRLF.
runtime/include/debug_server.h:9 and runtime/src/debug_server.c:8 cited "TCP.md
and DEBUG.md". TCP.md has never existed in this repo under that name; both now
name docs/TCP_COMMANDS.md and docs/internal/DEBUG.md.
LINK AND REFERENCE FIXES
Every inbound reference to a moved file was located by grepping each basename
across the whole tracked tree, not just the markdown. Markdown hrefs in
README.md, CONTRIBUTING.md, docs/ARCHITECTURE.md, docs/COMPILING_OVERLAYS.md,
docs/AOT_OVERLAY_PLAN.md, docs/RELOCATION_MANIFEST_FORMAT.md,
docs/config_schema.md, docs/OVERLAY_XREF.md, docs/TCP_COMMANDS.md and several
docs/internal/ files now point at the new locations; where two files became
siblings under docs/internal/ the path was shortened rather than lengthened
(bios-hardening.md's reference to overlay-discovery.md, for instance).
docs/COMPILING_OVERLAYS.md's "in this folder" phrasing was wrong once two of the
three files it listed moved, so it now names internal/ explicitly.
24 source-comment mentions of ENHANCEMENTS.md sections G1.x across 19 files
(recompiler, runtime, cmake) were retargeted to docs/ENHANCEMENTS.md with the
section numbers untouched. docs/SHADOW_ENHANCEMENTS.md is a DIFFERENT file and
was deliberately excluded.
20 rationale strings citing docs/CASE_A_AOT_GAP.md across the four
recompiler/seeds/*.json files were updated. Done as a byte-level substring
replacement rather than a JSON round-trip: two of those files are stored with
CRLF endings and two with LF, and re-serializing them reformatted 5,700 lines.
The byte-level version changes exactly 20 lines and each file still parses.
References that look similar but point elsewhere were deliberately left alone,
and are worth naming so a future pass does not "fix" them:
* "recomp-template PRINCIPLES.md" / "NES recomp template PRINCIPLES.md" in
CLAUDE.md:43, docs/SHADOW_ENHANCEMENTS.md:28,
docs/internal/HLE_SCHEDULER_CARVEOUT_PLAN.md:5,
runtime/src/debug_server.c:4934 and tools/parity_diff.py:8 are a different
repository's file. Only the five that mean this repo's PRINCIPLES.md were
updated.
* Every remaining bare ISSUES.md mention is a GAME repo's tracker
(MegaManX6Recomp 2/7/8/9, TombaRecomp 6), not this repo's.
* docs/disc-speed.md:4,54,56 and docs/internal/bios-hardening.md:5 name
overlay-discovery as a BRANCH, not a file, so no path applies.
INDEXES
README.md's "New here?" table gains rows for docs/WIDESCREEN.md,
docs/ENHANCEMENTS.md and docs/TCP_COMMANDS.md. docs/internal/README.md gains
sections indexing all fourteen documents that moved in, plus an "Upstream
contributor reviews" section describing upstream/ and the new sweep file, plus a
"Removed" bullet for CYCLE_TIMING_ARCH.md (deleted in the previous commit)
pointing at FAITHFUL_TIMING_PLAN.md, CYCLE_MODEL_BEETLE.md and
PRECISE_IRQ_SLICE.md -- following the SLJIT_PERSIST_CACHE.md precedent already
in that file.
docs/ENHANCEMENTS.md:7 claimed framework bugs live in "game repos' ISSUES.md
until a framework tracker exists". A framework tracker does exist; it now says
so and names the Beads epic.
VERIFICATION
A link checker was written to scan every tracked *.md for relative links,
resolving each against the tree being scanned. Baseline at origin/master: 129
files, 137 relative links, 3 broken. After both commits: 127 files, 159 relative
links, 3 broken -- the SAME three, all pre-existing and untouched here:
docs/DATA_SHARDS.md -> INSTANT_LOADS_PARKED.md (never existed)
docs/DATA_SHARDS.md -> INSTANT_LOADS_SPIKE2.md (never existed)
docs/NETPLAY.md -> ../lib/recomp-net/README.md (inside a submodule)
Zero new broken links. The checker's own coverage was checked against a raw grep
for markdown link openers: the 3-link difference is C++ lambdas inside code
blocks, correctly not treated as links.
Also verified: py_compile over all 113 tracked tools/*.py plus psxrecomp_cli.py;
all four seed JSONs still parse; the test-registration guard still reports 123
test files all registered; recompiler/tests/test_coverage_vault.py still passes;
and no file in this commit had its line endings flipped (checked mechanically
across all 50 modified files, after two files were caught and redone).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
182 lines
9.0 KiB
Python
182 lines
9.0 KiB
Python
"""codegen_audit_game — audit generated C from the game full-function-emitter.
|
|
|
|
Sibling of `codegen_audit.py` (which audits BIOS strict-translator emit).
|
|
The two emit conventions differ enough that the audit passes themselves
|
|
diverge — see `docs/internal/audit_inventory.md` for the rationale.
|
|
|
|
Tomba / game-emit characteristics:
|
|
- Direct function calls: `func_XXXXXXXX(cpu);`
|
|
- Indirect dispatch: `call_by_address(cpu, 0xXXXX);` or `call_by_address(cpu, cpu->gpr[N]);`
|
|
- Tail call (same as BIOS): `cpu->pc = 0xXXXX; return;`
|
|
- Branch cond: `int _bc_XXXXXXXX = (expr);` (declare-and-assign in one line)
|
|
- Block label: `block_XXXXXXXX:`
|
|
- Function def: `void func_XXXXXXXX(CPUState* cpu) { ... }`
|
|
|
|
Audit passes:
|
|
1. Direct-call resolution: every `func_XXXXXXXX(cpu)` call references a
|
|
defined function (forward-declared in dispatch_c, defined in full_c).
|
|
2. Literal `call_by_address(cpu, 0xXXXX)` target resolves to dispatch
|
|
table.
|
|
3. Indirect `call_by_address(cpu, cpu->gpr[N])` site distribution.
|
|
4. Tail-call (`cpu->pc=X; return;`) target resolves to dispatch table.
|
|
5. Branch-condition count & distribution per function.
|
|
6. `goto block_X` target resolves to a labelled block in the same
|
|
function (informational; cross-function gotos surface here).
|
|
|
|
Usage:
|
|
python tools/codegen_audit_game.py --config /path/to/game.toml
|
|
"""
|
|
import re
|
|
import sys
|
|
from collections import Counter, defaultdict
|
|
from pathlib import Path
|
|
|
|
sys.path.insert(0, str(Path(__file__).parent))
|
|
import audit_config
|
|
|
|
|
|
# ── Patterns shared with codegen_audit.py (BIOS emit) ──────────────────
|
|
RE_TAIL_CALL = re.compile(r'cpu->pc = 0x([0-9A-Fa-f]+)u?;\s*return;')
|
|
|
|
# Game emit uses a switch/case dispatch table:
|
|
# case 0xADDRu: func_ADDR(cpu); return 1;
|
|
# This is NOT the BIOS strict-translator's `{ 0xADDR, func_ADDR }` array.
|
|
RE_DISPATCH_TABLE_ENTRY = re.compile(
|
|
r'case\s+0x([0-9A-Fa-f]+)u?\s*:\s*func_[0-9A-Fa-f]+\s*\(cpu\)'
|
|
)
|
|
|
|
# ── Patterns specific to game full-function-emitter ────────────────────
|
|
RE_DIRECT_CALL = re.compile(r'\bfunc_([0-9A-Fa-f]+)\(cpu\);')
|
|
RE_CALL_BY_ADDR_LIT = re.compile(
|
|
r'call_by_address\(cpu,\s*0x([0-9A-Fa-f]+)u?\)\s*;'
|
|
)
|
|
RE_CALL_BY_ADDR_REG = re.compile(
|
|
r'call_by_address\(cpu,\s*cpu->gpr\[(\d+)\]\)\s*;'
|
|
)
|
|
RE_BC_DECL = re.compile(r'\bint\s+_bc_([0-9A-Fa-f]+)\s*=\s*\(')
|
|
RE_FUNC_DEF = re.compile(r'^void\s+func_([0-9A-Fa-f]+)\s*\(CPUState\s*\*\s*cpu\s*\)\s*$', re.M)
|
|
RE_FUNC_FWD = re.compile(r'^extern\s+void\s+func_([0-9A-Fa-f]+)\s*\(', re.M)
|
|
RE_BLOCK_LABEL = re.compile(r'^(block_[0-9A-Fa-f]+):', re.M)
|
|
RE_GOTO_BLOCK = re.compile(r'\bgoto\s+(block_[0-9A-Fa-f]+)\s*;')
|
|
|
|
|
|
def main():
|
|
cfg = audit_config.from_argv(sys.argv)
|
|
print(f"# codegen audit (game emit): {cfg.name}")
|
|
print(f" full_c = {cfg.full_c}")
|
|
print(f" dispatch_c = {cfg.dispatch_c}")
|
|
print()
|
|
|
|
if not cfg.full_c.exists():
|
|
print(f"ERROR: full_c not found: {cfg.full_c}", file=sys.stderr)
|
|
return 2
|
|
if not cfg.dispatch_c.exists():
|
|
print(f"ERROR: dispatch_c not found: {cfg.dispatch_c}", file=sys.stderr)
|
|
return 2
|
|
|
|
full_c = cfg.full_c.read_text(encoding="utf-8", errors="replace")
|
|
dispatch_c = cfg.dispatch_c.read_text(encoding="utf-8", errors="replace")
|
|
|
|
# ── 1. Direct-call resolution ──────────────────────────────────────
|
|
defined_funcs = set(RE_FUNC_DEF.findall(full_c))
|
|
forward_decls = set(RE_FUNC_FWD.findall(dispatch_c))
|
|
declared_funcs = defined_funcs | forward_decls
|
|
|
|
direct_calls = RE_DIRECT_CALL.findall(full_c)
|
|
direct_targets = Counter(c.upper() for c in direct_calls)
|
|
|
|
unresolved_calls = sorted(t for t in direct_targets if t not in
|
|
{f.upper() for f in declared_funcs})
|
|
print(f"[1] direct func_XXXX(cpu) calls: {sum(direct_targets.values())} total, "
|
|
f"{len(direct_targets)} unique targets")
|
|
print(f" defined functions in full_c: {len(defined_funcs)}")
|
|
print(f" forward decls in dispatch_c: {len(forward_decls)}")
|
|
print(f" unresolved direct-call targets: {len(unresolved_calls)}")
|
|
for t in unresolved_calls[:25]:
|
|
print(f" func_{t} ({direct_targets[t]} call site{'s' if direct_targets[t]!=1 else ''})")
|
|
|
|
# ── 2. Literal call_by_address targets ─────────────────────────────
|
|
# Targets in the declared code regions must resolve to the dispatch
|
|
# table. Targets outside (RAM-loaded overlays, install-at-runtime
|
|
# stubs) are handled by the runtime's dirty-RAM interpreter — expected
|
|
# to be absent from the static dispatch table.
|
|
def is_in_code(vaddr_normalized: int) -> bool:
|
|
for r in cfg.regions:
|
|
phys_lo = r.vaddr_base & cfg.kseg_mask
|
|
phys_hi = phys_lo + (r.rom_end - r.rom_start)
|
|
if phys_lo <= vaddr_normalized < phys_hi:
|
|
return True
|
|
return False
|
|
|
|
cba_lits = RE_CALL_BY_ADDR_LIT.findall(full_c)
|
|
cba_lit_set = set(cfg.normalize_addr(int(t, 16)) for t in cba_lits)
|
|
table_entries = RE_DISPATCH_TABLE_ENTRY.findall(dispatch_c)
|
|
table_set = set(cfg.normalize_addr(int(e, 16)) for e in table_entries)
|
|
|
|
cba_in_code = {t for t in cba_lit_set if is_in_code(t)}
|
|
cba_in_ram = cba_lit_set - cba_in_code
|
|
|
|
cba_missing_in_code = sorted(cba_in_code - table_set)
|
|
print(f"\n[2] literal call_by_address targets: {len(cba_lit_set)} unique, "
|
|
f"{sum(1 for _ in cba_lits)} sites")
|
|
print(f" dispatch table size: {len(table_set)} entries")
|
|
print(f" targets in declared code regions: {len(cba_in_code)}")
|
|
print(f" targets in RAM (dirty-RAM interpreter domain): {len(cba_in_ram)}")
|
|
print(f" in-code targets MISSING from dispatch: {len(cba_missing_in_code)}")
|
|
for m in cba_missing_in_code[:25]:
|
|
cnt = sum(1 for t in cba_lits if cfg.normalize_addr(int(t, 16)) == m)
|
|
print(f" 0x{m:08X} ({cnt} site{'s' if cnt!=1 else ''})")
|
|
|
|
# ── 3. Indirect call_by_address distribution ───────────────────────
|
|
cba_regs = RE_CALL_BY_ADDR_REG.findall(full_c)
|
|
print(f"\n[3] indirect call_by_address sites: {len(cba_regs)} total")
|
|
print(f" by register: {dict(Counter(cba_regs).most_common(8))}")
|
|
|
|
# ── 4. Tail-call targets ───────────────────────────────────────────
|
|
tail_calls = RE_TAIL_CALL.findall(full_c)
|
|
tail_set = set(cfg.normalize_addr(int(t, 16)) for t in tail_calls)
|
|
tail_missing = sorted(tail_set - table_set)
|
|
print(f"\n[4] tail-call (cpu->pc=...; return;) targets: {len(tail_set)} unique, "
|
|
f"{len(tail_calls)} sites")
|
|
print(f" tail-call targets MISSING from dispatch: {len(tail_missing)}")
|
|
for m in tail_missing[:25]:
|
|
cnt = sum(1 for t in tail_calls if cfg.normalize_addr(int(t, 16)) == m)
|
|
print(f" 0x{m:08X} ({cnt} site{'s' if cnt!=1 else ''})")
|
|
|
|
# ── 5. Branch-condition population ─────────────────────────────────
|
|
bc_decls = RE_BC_DECL.findall(full_c)
|
|
print(f"\n[5] branch-condition declarations (`int _bc_X = (...);`): {len(bc_decls)}")
|
|
|
|
# ── 6. goto block_X target resolution ──────────────────────────────
|
|
labels = set(RE_BLOCK_LABEL.findall(full_c))
|
|
gotos = RE_GOTO_BLOCK.findall(full_c)
|
|
unresolved_gotos = sorted(set(gotos) - labels)
|
|
print(f"\n[6] goto block_X targets: {len(gotos)} sites, {len(set(gotos))} unique")
|
|
print(f" block labels defined: {len(labels)}")
|
|
print(f" goto targets WITHOUT a matching block label: {len(unresolved_gotos)}")
|
|
for g in unresolved_gotos[:25]:
|
|
print(f" {g}")
|
|
|
|
# ── Summary ────────────────────────────────────────────────────────
|
|
real_bugs = (
|
|
len(unresolved_calls) # direct call to undefined function
|
|
+ len(cba_missing_in_code) # in-code literal call_by_address not in dispatch
|
|
+ len(tail_missing) # tail call not in dispatch
|
|
+ len(unresolved_gotos) # goto to undefined label
|
|
)
|
|
|
|
print(f"\n[summary] {cfg.name}")
|
|
print(f" unresolved direct calls : {len(unresolved_calls)}")
|
|
print(f" in-code call_by_address misses : {len(cba_missing_in_code)}")
|
|
print(f" RAM call_by_address (expected) : {len(cba_in_ram)}")
|
|
print(f" tail-call misses : {len(tail_missing)}")
|
|
print(f" unresolved goto labels : {len(unresolved_gotos)}")
|
|
print(f" total real-bug findings : {real_bugs}")
|
|
print(f" STATUS : {'CLEAN' if real_bugs == 0 else 'ISSUES FOUND'}")
|
|
|
|
return 0 if real_bugs == 0 else 1
|
|
|
|
|
|
if __name__ == "__main__":
|
|
sys.exit(main())
|