diff --git a/docs/internal/ACCURACY_BURNDOWN.md b/docs/internal/ACCURACY_BURNDOWN.md index 50372f05..7fbe727f 100644 --- a/docs/internal/ACCURACY_BURNDOWN.md +++ b/docs/internal/ACCURACY_BURNDOWN.md @@ -124,6 +124,18 @@ observe added to the recompiler so ANY block leader is anchorable on both backen + no-regression (no interp-path ruler yet — see below). - [x] Instruction-fetch / I-cache timing — DONE (see above; commits 958a928 + 0edb935). The ruler's 56→84 cold spread (I-cache line-refill transient) now reproduced natively. +- [x] **Uncached (KSEG1) fetch is charged per instruction in both emitters (2026-09-29).** + Beetle ReadInstruction charges +4 and clears the load give-back on EVERY fetch at + 0xA0000000 and above. The interp fetches at every PC, but both emitters charged only + at line leaders. In OpenBIOS, 5,477 of 9,592 in-place ROM (KSEG1) instruction sites + were uncharged: 21,908 cycles short per pass through that code. Both emitters now + emit a fetch before every instruction whose runtime PC is uncached + (`psx_fetch_uncached`, psx_instr_cost.h, shared with psx_icache.c). The A0/B0/C0 + call-vector stubs charge one fetch per executed word. Cached code keeps the leader + rule, and KSEG0 game output is byte-identical. Tests: ctest `uncached_fetch_charge` + (compiled == interp fetch path == Beetle transcription, per instruction, on + OpenBIOS) and `uncached_fetch_codegen_test` (game emitter). OPEN: full LLE boot + (`bios_hle = false`) parity against live Beetle (no oracle binary on the Mac). - [ ] **HW test-ROM ruler (#2)** — Amidog CPU/GTE timing ROMs for hand-crafted single-COMPONENT isolation (div-only, load-only loops) that organic BIOS code can't give (the prologue combines div+loads in one block). Strongest validator. @@ -143,6 +155,10 @@ Status: PARTIAL. Status: MODERATE-STRONG (regions games use). - [ ] KUSEG/KSEG0/KSEG1 mirroring, scratchpad, cache-isolation (IsC) — psx-spx. +- [ ] BIU bit 11 (I-cache disable, 0xFFFE0130) does not reach the fetch model: + memory.c stores it, psx_icache.c and the interp ignore it. Beetle charges +4 per + fetch while the cache is disabled (CPU_SetBIU). This matters only for RAM code run + with the cache off; `psx_fetch_uncached` is an address test and does not cover it. - [ ] I/O register semantics: read-to-clear, write-1-ack (I_STAT), masking, unmapped/garbage reads — psx-spx "I/O Map"; Beetle memory.cpp. diff --git a/docs/internal/FAITHFUL_TIMING_PLAN.md b/docs/internal/FAITHFUL_TIMING_PLAN.md index a7729273..d0bb6175 100644 --- a/docs/internal/FAITHFUL_TIMING_PLAN.md +++ b/docs/internal/FAITHFUL_TIMING_PLAN.md @@ -213,6 +213,35 @@ on a fixed region -> next. ## 5. Status / Log (update every session) +- **2026-09-29 (uncached KSEG1 fetch charged per instruction, both emitters):** + Beetle ReadInstruction never fills a line for a fetch at 0xA0000000 or above: + each one costs +4 and clears the load give-back. The interp fetches at every PC, + but both emitters emitted `psx_icache_fetch` only at line leaders, which is exact + for cached code only. In OpenBIOS, 5,477 of the 9,592 in-place ROM (KSEG1) + instruction sites were uncharged: 21,908 cycles per pass through that code. + Both emitters now charge a fetch before every instruction whose runtime PC is + uncached. The predicate is `psx_fetch_uncached` (psx_instr_cost.h), which + psx_icache.c also uses. The A0/B0/C0 call-vector stubs charge one fetch per + executed word. + - Cached code keeps the leader rule. R4's regenerated game C (50 shards and the + dispatch table) is byte-identical, as are the 26 R4 overlay shards compiled in + both runs. + - The OpenBIOS diff is insertions only: 5,477 fetches plus 5 stub fetches. + - The codegen hash changes because the emitter sources do, and + psx_instr_cost.h is now in the hash list. Every title reshards its overlay + cache once. + - Tests: ctest `uncached_fetch_charge` checks compiled == interp fetch path == + Beetle transcription per instruction on OpenBIOS, the stubs and synthetic + sequences. `uncached_fetch_codegen_test` checks the game emitter at KSEG1 + and KSEG0. Both fail on master. + - R4 (OpenBIOS: recompiled LLE kernel, HLE boot that skips the shell): the + game entry 0x8007D8F4 moves from guest cycle 97,718,389 to 100,583,391 + (+2,865,002, about 5.1 frames). The count was the same on two baseline + boots. R4 reached a race with 0 dispatch misses. + - Closes `bios-kseg1-fetch-charge` in the segment-aware ledger (PR #419). + - OPEN: full LLE boot (`bios_hle = false`) cycle parity against live Beetle. + There is no oracle binary on the Mac. + - **2026-09-29 (generic A/B identity tool, `feat/fp-identity-tool` on #420):** `tools/fp_identity.py` moves R4's warm/cold check into the framework for any title (launch template or `--runtime/--game/--disc`, `--seed` overlay diff --git a/recompiler/CMakeLists.txt b/recompiler/CMakeLists.txt index 1c627f1f..b302fa24 100644 --- a/recompiler/CMakeLists.txt +++ b/recompiler/CMakeLists.txt @@ -558,6 +558,35 @@ if(BUILD_TESTING) ${CMAKE_CURRENT_BINARY_DIR}/ws_cull_edge_interp_stderr.txt) endif() + # Uncached (KSEG1) code charges a fetch before every instruction; cached + # code keeps the line-leader rule (test_uncached_fetch_charge.py checks + # the BIOS emitter on OpenBIOS and the cycle model). + add_executable(uncached_fetch_codegen_test + tests/uncached_fetch_codegen_test.cpp + src/ps1_exe_parser.cpp + src/mips_decoder.cpp + src/control_flow.cpp + src/function_analysis.cpp + src/code_generator.cpp + src/pgxp_hook_emitter.cpp + src/annotations.cpp + src/bios_address_model.cpp + src/config_loader.cpp + src/recompiler_patch.cpp + ) + target_include_directories(uncached_fetch_codegen_test PRIVATE + include + src + lib/rabbitizer/include + lib/rabbitizer/cplusplus/include + lib/fmt/include + lib/toml11 + lib/ELFIO + ) + target_link_libraries(uncached_fetch_codegen_test PRIVATE fmt rabbitizer) + add_test(NAME uncached_fetch_codegen_test + COMMAND uncached_fetch_codegen_test) + add_executable(recompiler_patch_cxx17_header_test tests/recompiler_patch_cxx17_header_test.cpp ) @@ -791,6 +820,15 @@ if(BUILD_TESTING) ${CMAKE_CURRENT_SOURCE_DIR}/tests/test_bios_vector_stub_shapes.py --recompiler $ WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/..) + # Compiled BIOS fetch charges == the interpreter's (psx_icache.c) == + # Beetle, per instruction, on OpenBIOS: KSEG1 ROM code pays every fetch. + add_test( + NAME uncached_fetch_charge + COMMAND ${Python3_EXECUTABLE} + ${CMAKE_CURRENT_SOURCE_DIR}/tests/test_uncached_fetch_charge.py + --bios-recompiler $ + --compiler ${CMAKE_C_COMPILER} + WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/..) add_test( NAME stall_report COMMAND ${Python3_EXECUTABLE} diff --git a/recompiler/src/code_generator.cpp b/recompiler/src/code_generator.cpp index 6313c3ad..b2eb1a80 100644 --- a/recompiler/src/code_generator.cpp +++ b/recompiler/src/code_generator.cpp @@ -1885,17 +1885,23 @@ std::string CodeGenerator::translate_basic_block( }; // I-cache FETCH cost (faithful R3000A), emitted BEFORE the per-instruction // interlock/load — exactly like Beetle ReadInstruction precedes the base, and so a - // fetch MISS clears any pending load give-back before the next load arms one. Only - // emitted at cache-line LEADERS: a block leader / mid-block jump-table target (any - // address reachable other than by fall-through, i.e. a possibly-cold cache entry) OR - // a 16-byte-line start (addr&0xC==0, a sequential line crossing). Intra-line - // followers reached by fall-through are guaranteed hits — the leader's fetch - // refilled the line to its end — so they need no call (+0). Extra fetch points are - // harmless (a hit is +0); only UNDER-counting a cold entry would diverge, which the - // leader set prevents. The game runs at its KSEG0 load address, so `insn_addr` is - // already the runtime guest PC (matching the dirty-RAM interp's cpu->pc and Beetle). + // fetch MISS clears any pending load give-back before the next load arms one. The + // game runs at its load address, so `insn_addr` is already the runtime guest PC + // (matching the dirty-RAM interp's cpu->pc and Beetle). + // - CACHED PC: only emitted at cache-line LEADERS: a block leader / mid-block + // jump-table target (any address reachable other than by fall-through, i.e. a + // possibly-cold cache entry) OR a 16-byte-line start (addr&0xC==0, a sequential + // line crossing). Intra-line followers reached by fall-through are guaranteed + // hits — the leader's fetch refilled the line to its end — so they need no call + // (+0). Extra fetch points are harmless (a hit is +0); only UNDER-counting a cold + // entry would diverge, which the leader set prevents. + // - UNCACHED PC (psx_fetch_uncached, KSEG1): no fetch fills a line, so every + // instruction pays +4 and clears the load give-back — a fetch before EVERY + // instruction, exactly as the interpreter and Beetle charge it. KSEG0 code never + // takes this branch, so its output is unchanged. auto emit_pre_icache = [&](uint32_t insn_addr, const std::string& indent) { - if (!(insn_addr == block.start_addr || (insn_addr & 0xCu) == 0 || + if (!psx_fetch_uncached(insn_addr) && + !(insn_addr == block.start_addr || (insn_addr & 0xCu) == 0 || extra_labels_.count(insn_addr))) return; ss << "#ifdef PSX_ENABLE_BLOCK_CYCLES\n"; ss << indent << fmt::format("psx_icache_fetch(cpu, 0x{:08X}u);\n", insn_addr); diff --git a/recompiler/src/full_function_emitter.cpp b/recompiler/src/full_function_emitter.cpp index b6bf9d26..a64d4dab 100644 --- a/recompiler/src/full_function_emitter.cpp +++ b/recompiler/src/full_function_emitter.cpp @@ -765,22 +765,29 @@ bool FullFunctionEmitter::emit_function( // I-cache FETCH cost (faithful R3000A), emitted BEFORE the per-instruction // interlock/load — like Beetle ReadInstruction precedes the base, so a fetch MISS - // clears any pending load give-back before the next load arms one. Only emitted at - // cache-line LEADERS: a block leader (any branch/dispatch entry — a possibly-cold - // cache entry; cross-function targets are inserted into block_leaders above) OR a - // 16-byte-line start (addr&0xC==0, a sequential line crossing). Intra-line followers - // reached by fall-through are guaranteed hits (the leader refilled the line to its - // end) → no call (+0). `rom_addr` is the ROM/compile-time address; relocate_ra maps - // it to the RUNTIME guest PC the CPU actually fetches from (BIOS main stays in-place - // KSEG1 0xBFC..; relocated kernel Part 2 → 0x500+, shell → 0x80030000+), so the - // shared I-cache evolves identically to the dirty-RAM interp (cpu->pc) and Beetle — - // and the KSEG1 uncached test (>=0xA0000000) sees the true virtual address. The - // relocation preserves bits[3:0], so the line-leader test is space-independent. + // clears any pending load give-back before the next load arms one. `rom_addr` is + // the ROM/compile-time address; relocate_ra maps it to the RUNTIME guest PC the CPU + // actually fetches from (BIOS main stays in-place KSEG1 0xBFC..; relocated kernel + // Part 2 → 0x500+, shell → 0x80030000+), so the shared I-cache evolves identically + // to the dirty-RAM interp (cpu->pc) and Beetle. + // - CACHED runtime PC: emitted only at cache-line LEADERS: a block leader (any + // branch/dispatch entry — a possibly-cold cache entry; cross-function targets + // are inserted into block_leaders above) OR a 16-byte-line start (addr&0xC==0, + // a sequential line crossing). Intra-line followers reached by fall-through are + // guaranteed hits (the leader refilled the line to its end) → no call (+0). The + // relocation preserves bits[3:0], so the line-leader test is space-independent. + // - UNCACHED runtime PC (psx_fetch_uncached: the ROM run in place at KSEG1): no + // fetch ever fills a line, so there is no follower to elide. Every instruction + // pays +4 and clears the load give-back, exactly as the interpreter (a fetch at + // every cpu->pc) and Beetle charge it. Emitting only at leaders left 5,477 of + // OpenBIOS's 9,592 KSEG1 instruction sites 4 cycles short. auto emit_icache_fetch = [&](uint32_t rom_addr) { if (!per_insn_cycles) return; - if (!(block_leaders.count(rom_addr) || (rom_addr & 0xCu) == 0)) return; + const uint32_t pc = relocate_ra(rom_addr); + if (!psx_fetch_uncached(pc) && + !(block_leaders.count(rom_addr) || (rom_addr & 0xCu) == 0)) return; out += fmt::format("#ifdef PSX_ENABLE_BLOCK_CYCLES\n psx_icache_fetch(cpu, 0x{:08X}u);\n#endif\n", - relocate_ra(rom_addr)); + pc); }; auto emit_patch_range_guard = [&](uint32_t addr) { @@ -2073,7 +2080,11 @@ void FullFunctionEmitter::emit_dispatch( // that changes even one word fails closed to dirty_ram_interp below. The // cycle steps charge exactly the instructions the guest executes on that // shape (shape B's fourth word is never reached: the jr's delay slot is - // word 2). + // word 2). Each executed word gets its own fetch, in Beetle order (fetch, + // then the step), because `addr` keeps the caller's segment: at KUSEG/KSEG0 + // the first fetch fills the line and the rest are +0 hits, while a KSEG1 + // call (e.g. `jalr 0xA00000A0`) pays +4 and a give-back clear per word, + // as the interpreter does. out += fmt::format("static const PsxNativeStub {}psx_bios_native_stubs[3] = {{\n", g_sym_prefix); out += " { 0x000000A0u, 0x000000A0u, 0x000000B0u },\n"; @@ -2082,6 +2093,12 @@ void FullFunctionEmitter::emit_dispatch( out += "};\n"; out += fmt::format("static const uint32_t {}psx_bios_native_stub_count = 3u;\n\n", g_sym_prefix); + auto emit_stub_insn = [&](uint32_t offset, uint32_t word) { + out += offset ? fmt::format(" psx_icache_fetch(cpu, addr + {}u);\n", offset) + : std::string(" psx_icache_fetch(cpu, addr);\n"); + out += fmt::format(" psx_cyc_step(cpu, 0x{:X}u);\n", + psx_cyc_dep_res_mask(word)); + }; out += "static int psx_bios_try_native_call_stub(CPUState* cpu, " "uint32_t addr) {\n"; out += " uint32_t phys = addr & 0x1FFFFFFFu;\n"; @@ -2098,28 +2115,19 @@ void FullFunctionEmitter::emit_dispatch( out += " target = ((w0 & 0xFFFFu) << 16) +\n"; out += " (uint32_t)(int32_t)(int16_t)(w1 & 0xFFFFu);\n"; out += "#ifdef PSX_ENABLE_BLOCK_CYCLES\n"; - out += " psx_icache_fetch(cpu, addr);\n"; - out += fmt::format(" psx_cyc_step(cpu, 0x{:X}u);\n", - psx_cyc_dep_res_mask(0x3C080000u)); - out += fmt::format(" psx_cyc_step(cpu, 0x{:X}u);\n", - psx_cyc_dep_res_mask(0x25080000u)); - out += fmt::format(" psx_cyc_step(cpu, 0x{:X}u);\n", - psx_cyc_dep_res_mask(0x01000008u)); - out += fmt::format(" psx_cyc_step(cpu, 0x{:X}u);\n", - psx_cyc_dep_res_mask(0x00000000u)); + emit_stub_insn(0u, 0x3C080000u); + emit_stub_insn(4u, 0x25080000u); + emit_stub_insn(8u, 0x01000008u); + emit_stub_insn(12u, 0x00000000u); out += "#endif\n"; out += " } else if ((w0 & 0xFFFF0000u) == 0x24080000u &&\n"; out += " w1 == 0x01000008u && w2 == 0u && w3 == 0u) {\n"; out += " /* shape B: addiu $t0,$zero,lo; jr $t0; nop; nop */\n"; out += " target = (uint32_t)(int32_t)(int16_t)(w0 & 0xFFFFu);\n"; out += "#ifdef PSX_ENABLE_BLOCK_CYCLES\n"; - out += " psx_icache_fetch(cpu, addr);\n"; - out += fmt::format(" psx_cyc_step(cpu, 0x{:X}u);\n", - psx_cyc_dep_res_mask(0x24080000u)); - out += fmt::format(" psx_cyc_step(cpu, 0x{:X}u);\n", - psx_cyc_dep_res_mask(0x01000008u)); - out += fmt::format(" psx_cyc_step(cpu, 0x{:X}u);\n", - psx_cyc_dep_res_mask(0x00000000u)); + emit_stub_insn(0u, 0x24080000u); + emit_stub_insn(4u, 0x01000008u); + emit_stub_insn(8u, 0x00000000u); out += "#endif\n"; out += " } else {\n"; out += " return 0;\n"; diff --git a/recompiler/tests/test_uncached_fetch_charge.py b/recompiler/tests/test_uncached_fetch_charge.py new file mode 100644 index 00000000..476393f4 --- /dev/null +++ b/recompiler/tests/test_uncached_fetch_charge.py @@ -0,0 +1,428 @@ +#!/usr/bin/env python3 +"""Compiled BIOS code charges instruction fetch exactly as the interpreter does. + +Beetle's ReadInstruction, which runtime/src/psx_icache.c transcribes, never +fills an I-cache line for a fetch at 0xA0000000 or above. Every such fetch +costs +4 and clears the pending load give-back. The dirty-RAM interpreter +reaches that charge by calling psx_icache_fetch_interp at every PC. The BIOS +main ROM runs in place at KSEG1 (0xBFC0....), so compiled ROM code must charge +a fetch before EVERY instruction. The emitters used to charge only at block +leaders and 16-byte line starts, which is exact only for cached code. That +left 5,477 of OpenBIOS's 9,592 KSEG1 instruction sites 4 cycles short, each +also missing its give-back clear. + +The test compiles runtime/src/psx_icache.c into a small harness and drives it +two ways. "Compiled" runs only the fetches the emitter wrote +(psx_icache_fetch). "Interpreter" runs a fetch at every PC +(psx_icache_fetch_interp, the dirty-RAM interpreter's own entry point). Each +instruction's (cycles, give-back cleared) pair must be the same both ways: + + model psx_icache.c (both entry points) equals a transcription of Beetle + ReadInstruction on synthetic sequences: uncached repeat, cached cold + then warm, KSEG0 -> KUSEG alias refill, partial refill. + bios psxrecomp-bios on bios/OpenBIOS.toml. Every straight-line KSEG1 + run (the ROM in place) is replayed, and every KSEG1 instruction + must carry a fetch naming its own runtime PC. + stubs The A0/B0/C0 call-vector native stubs charge one fetch per + executed word, so a KSEG1 call pays like the interpreter. + +The game/overlay emitter's half is uncached_fetch_codegen_test.cpp. + +Usage: test_uncached_fetch_charge.py --bios-recompiler [--compiler cc] +Exit 0 = every compared instruction matches. +""" +import argparse +import os +import re +import shutil +import subprocess +import sys +import tempfile + +HERE = os.path.dirname(os.path.abspath(__file__)) +ROOT = os.path.normpath(os.path.join(HERE, "..", "..")) +RUNTIME = os.path.join(ROOT, "runtime") + +KUSEG, KSEG0, KSEG1 = 0x00000000, 0x80000000, 0xA0000000 + + +def uncached(pc): + """psx_fetch_uncached (runtime/include/psx_instr_cost.h).""" + return pc >= 0xA0000000 + + +# --------------------------------------------------------------------------- +# Beetle fetch model, transcribed from libretro/beetle-psx-libretro +# mednafen/psx/cpu.c @ a7f0811: ReadInstruction (738-835) with the I-cache on +# (BIU bit 11 set, as the BIOS leaves it). The tag compares the full virtual +# address. A miss clears the load give-back. KSEG1 then costs +4 and fills +# nothing. A cached miss costs 3, plus 1 for each word refilled from the +# missed word to the end of the line. (Same transcription as +# test_segment_aware_codegen.py in PR #419.) +# --------------------------------------------------------------------------- +class BeetleICache: + def __init__(self): + self.tv = [0x2] * 1024 + + def fetch(self, addr): + """(cycles, give-back cleared) for one fetch.""" + if self.tv[(addr & 0xFFC) >> 2] == addr: + return 0, 0 + if addr >= 0xA0000000: + return 4, 1 + line, base = addr & 0xFFFFFFF0, (addr & 0xFF0) >> 2 + for i in range(4): + self.tv[base + i] = line | (i << 2) | 0x2 + cost = 3 + for i in range((addr & 0xC) >> 2, 4): + self.tv[base + i] &= ~0x2 + cost += 1 + return cost, 1 + + +def beetle_per_insn(pcs): + model = BeetleICache() + return [model.fetch(pc) for pc in pcs] + + +# One line of input per instruction: +# R reset the cache (cold, as psx_icache_reset leaves it) +# E compiled instruction whose emitted fetch names +# N compiled instruction with no emitted fetch +# I interpreter instruction at (psx_icache_fetch_interp) +# Every E/N/I line prints " " for that instruction. +HARNESS = r""" +#include "cpu_state.h" +#include "psx_icache.h" +#include +#include +#include +int g_ls_replay_active = 0; +static unsigned long long cycles; +void psx_advance_cycles(uint32_t c) { cycles += c; } +int main(void) { + static CPUState cpu; + char line[64]; + while (fgets(line, sizeof line, stdin)) { + char op = line[0]; + if (op == 'R') { psx_icache_reset(); g_psx_icache_active = 1; continue; } + if (op != 'E' && op != 'N' && op != 'I') continue; + unsigned long long before = cycles; + cpu.read_absorb_which = 1u; + cpu.read_absorb[1] = 1u; + if (op == 'E') psx_icache_fetch(&cpu, (uint32_t)strtoul(line + 2, 0, 16)); + if (op == 'I') psx_icache_fetch_interp(&cpu, (uint32_t)strtoul(line + 2, 0, 16)); + printf("%llu %d\n", cycles - before, + (cpu.read_absorb_which == 0u && cpu.read_absorb[1] == 0u) ? 1 : 0); + } + return 0; +} +""" + + +def cc(compiler, sources, include, out, defines=()): + name = os.path.basename(compiler).lower() + if name in ("cl", "cl.exe", "clang-cl", "clang-cl.exe"): + cmd = [compiler, "/nologo", "/Od", "/I" + include] + ["/D" + d for d in defines] + cmd += sources + ["/Fe:" + out] + else: + cmd = [compiler, "-std=c11", "-O2", "-I", include] + ["-D" + d for d in defines] + cmd += sources + ["-o", out] + subprocess.run(cmd, check=True, cwd=os.path.dirname(out)) + + +class Harness: + def __init__(self, compiler, tmp): + src = os.path.join(tmp, "fetch_harness.c") + with open(src, "w") as f: + f.write(HARNESS) + self.exe = os.path.join(tmp, "fetch_harness" + (".exe" if os.name == "nt" else "")) + cc(compiler, [src, os.path.join(RUNTIME, "src", "psx_icache.c")], + os.path.join(RUNTIME, "include"), self.exe, + ("PSX_ENABLE_BLOCK_CYCLES=1", "PSX_OVERLAY_DLL_BUILD=1")) + + def run(self, programs): + """programs: lists of ('E', tag) / ('N', None) / ('I', pc), each run + from a cold cache. Returns one [(cycles, cleared)] list per program.""" + script = [] + for prog in programs: + script.append("R\n") + for op, pc in prog: + script.append("N\n" if op == "N" else "%s %08X\n" % (op, pc)) + out = subprocess.run([self.exe], input="".join(script), capture_output=True, + text=True, check=True).stdout.split("\n") + vals = [tuple(int(x) for x in ln.split()) for ln in out if ln.strip()] + res, i = [], 0 + for prog in programs: + res.append(vals[i:i + len(prog)]) + i += len(prog) + if i != len(vals): + raise SystemExit("harness output length mismatch") + return res + + +def interp_program(pcs): + return [("I", pc) for pc in pcs] + + +# --------------------------------------------------------------------------- +# BIOS emitter output parsing +# --------------------------------------------------------------------------- +FUNC_RE = re.compile(r"^(?:static )?void \w+\(CPUState") +LABEL_RE = re.compile(r"^label_([0-9A-F]{8}):") +FETCH_RE = re.compile(r"psx_icache_fetch\(cpu, 0x([0-9A-F]{8})u\)") +INSN_RE = re.compile(r"/\* (DELAY \(orphaned\) )?0x([0-9A-F]{8}): [0-9A-F]{8} ") + + +def bios_runs(text, problems): + """Straight-line runs of emitted instructions, split at labels, function + starts and address gaps. Each instruction is (runtime_pc, fetch tag or + None). Runtime PC = ROM address + the function's relocation delta, taken + from its fetches (relocate_ra moves a whole window, so one function has one + delta). + + An in-function instruction's fetch precedes its comment. An orphaned delay + slot (inlined after its jr/branch) prints its comment first and then its + fetch, interlock and body, so a fetch that directly follows an orphan's + comment (only preprocessor lines between) belongs to the orphan.""" + runs = [] + run = None + delta = None + pending = None + last_rom = None + orphan = None # [rom, run insns index] of an orphan still awaiting its fetch + + def close(): + nonlocal run + if run and run["insns"]: + runs.append(run) + run = None + + for line in text.splitlines(): + if FUNC_RE.match(line): + close() + delta = pending = last_rom = orphan = None + continue + m = LABEL_RE.match(line) + if m: + close() + orphan = None + run = {"from_label": True, "label": int(m.group(1), 16), "insns": []} + continue + m = FETCH_RE.search(line) + if m and orphan is not None: + rom, idx = orphan + orphan = None + tag = int(m.group(1), 16) + d = (tag - rom) & 0xFFFFFFFF + if delta is None or d != delta: + problems.append("orphan fetch 0x%08X names another window than ROM 0x%08X" + % (tag, rom)) + elif idx is not None: + run["insns"][idx] = (run["insns"][idx][0], tag) + continue + if orphan is not None and not line.lstrip().startswith("#"): + orphan = None + if m: + if pending is not None: + problems.append("two fetches with no instruction between them before 0x%08X" % + int(m.group(1), 16)) + pending = int(m.group(1), 16) + continue + m = INSN_RE.search(line) + if not m: + continue + rom = int(m.group(2), 16) + if run is None or (last_rom is not None and rom != last_rom + 4): + close() + run = {"from_label": False, "label": rom, "insns": []} + last_rom = rom + tag = pending + pending = None + if tag is not None: + d = (tag - rom) & 0xFFFFFFFF + if delta is None: + delta = d + elif d != delta: + problems.append("fetch 0x%08X names another window than ROM 0x%08X" % (tag, rom)) + if delta is None: + run.setdefault("unmapped", 0) + run["unmapped"] = run["unmapped"] + 1 + orphan = [rom, None] if m.group(1) else None + continue + run["insns"].append(((rom + delta) & 0xFFFFFFFF, tag)) + orphan = [rom, len(run["insns"]) - 1] if m.group(1) and tag is None else None + close() + return runs + + +STUB_FETCH_RE = re.compile(r"psx_icache_fetch\(cpu, addr(?: \+ (\d+)u)?\);") + + +def stub_shapes(dispatch_c): + """Fetch offsets and step count for each shape of the native call stub.""" + begin = dispatch_c.index("static int psx_bios_try_native_call_stub(") + end = dispatch_c.index("\n}\n", begin) + body = dispatch_c[begin:end] + shapes = {} + for name in ("shape A", "shape B"): + s = body.index(name) + e = body.find("} else", s) + seg = body[s:e if e >= 0 else len(body)] + events = re.findall(r"psx_icache_fetch\(cpu, addr(?: \+ \d+u)?\);|psx_cyc_step\(", seg) + offs = [int(m.group(1) or 0) for m in STUB_FETCH_RE.finditer(seg)] + shapes[name] = (offs, events) + return shapes + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--bios-recompiler", + default=os.path.join(ROOT, "recompiler", "build", "psxrecomp-bios")) + ap.add_argument("--compiler", + default=os.environ.get("CC") or shutil.which("cc") or shutil.which("gcc")) + args = ap.parse_args() + if not os.path.isfile(args.bios_recompiler): + raise SystemExit("recompiler not found: %s (build it first)" % args.bios_recompiler) + if not args.compiler: + raise SystemExit("a C compiler is required") + if os.environ.get("PSX_CODEGEN_CYCLE_PER_INSN", "1")[:1] == "0": + print("SKIP: PSX_CODEGEN_CYCLE_PER_INSN=0 emits no fetch charges") + return 0 + + failures = [] + with tempfile.TemporaryDirectory() as tmp: + harness = Harness(args.compiler, tmp) + + # -- model: psx_icache.c (compiled and interpreter entry points) == Beetle + straight = [0x00010100 + 4 * i for i in range(10)] + synthetic = { + "uncached repeat": [KSEG1 | a for a in straight] * 2, + "cached cold then warm": [KSEG0 | a for a in straight] * 2, + "KSEG0 then KUSEG alias": [KSEG0 | a for a in straight] + [KUSEG | a for a in straight], + "partial refill": [KSEG0 | (straight[0] + 8), KSEG0 | straight[0]], + } + names = list(synthetic) + got_i = harness.run([interp_program(synthetic[n]) for n in names]) + got_e = harness.run([[("E", pc) for pc in synthetic[n]] for n in names]) + for n, gi, ge in zip(names, got_i, got_e): + want = beetle_per_insn(synthetic[n]) + if gi != want or ge != want: + failures.append("model %s: psx_icache.c interp %s / compiled %s != Beetle %s" + % (n, gi, ge, want)) + # The old leader-only rule on an uncached run must be caught (the + # comparison below is sensitive to exactly the bug this fixes). + k1 = [KSEG1 | a for a in straight] + leader_only = [("E", pc) if pc == k1[0] or (pc & 0xC) == 0 else ("N", None) for pc in k1] + old, ref = harness.run([leader_only, interp_program(k1)]) + if old == ref: + failures.append("model: leader-only KSEG1 emission was not distinguishable from the interpreter") + + # -- bios: every emitted run vs the interpreter at every PC ------------ + out = os.path.join(tmp, "bios") + os.makedirs(out) + r = subprocess.run([args.bios_recompiler, "--config", + os.path.join(ROOT, "bios", "OpenBIOS.toml"), "--out-dir", out], + cwd=ROOT, capture_output=True, text=True) + if r.returncode != 0: + raise SystemExit("psxrecomp-bios failed:\n%s" % (r.stderr or r.stdout)) + with open(os.path.join(out, "OpenBIOS_full.c")) as f: + full_c = f.read() + with open(os.path.join(out, "OpenBIOS_dispatch.c")) as f: + dispatch_c = f.read() + + problems = [] + runs = bios_runs(full_c, problems) + failures += problems + unmapped = sum(r.get("unmapped", 0) for r in runs) + if unmapped: + failures.append("bios: %d instructions precede any fetch in their function" % unmapped) + # A run that does not start at a label is entered only by fall-through + # from code outside it (an orphaned delay slot's neighbour). Cached + # instructions there depend on earlier lines; compare them from their + # first fetch. Uncached instructions never depend on history. + compiled, interp, kinds, uncached_sites, uncached_missing = [], [], [], 0, 0 + for run in runs: + insns = run["insns"] + if not run["from_label"]: + first = next((i for i, (_, t) in enumerate(insns) if t is not None), len(insns)) + insns = [x for i, x in enumerate(insns) if i >= first or uncached(x[0])] + if not insns or not all(uncached(pc) for pc, _ in insns): + continue + for pc, tag in insns: + if uncached(pc): + uncached_sites += 1 + if tag != pc: + uncached_missing += 1 + compiled.append([("E", t) if t is not None else ("N", None) for _, t in insns]) + interp.append(interp_program([pc for pc, _ in insns])) + kinds.append(insns) + got_c = harness.run(compiled) + got_i = harness.run(interp) + mismatch = {"uncached": 0, "cached": 0} + cycles_short = 0 + beetle_bad = 0 + first_bad = None + total = 0 + for insns, gc, gi in zip(kinds, got_c, got_i): + want = beetle_per_insn([pc for pc, _ in insns]) + if gi != want: + beetle_bad += sum(1 for a, b in zip(gi, want) if a != b) + for (pc, _), c, i in zip(insns, gc, gi): + total += 1 + if c != i: + mismatch["uncached" if uncached(pc) else "cached"] += 1 + cycles_short += i[0] - c[0] + if first_bad is None: + first_bad = (pc, c, i) + print("bios: %d runs, %d instructions compared (%d at KSEG1)" + % (len(kinds), total, uncached_sites)) + if uncached_sites == 0: + failures.append("bios: no KSEG1 instructions found (parser drift?)") + if uncached_missing: + failures.append("bios: %d of %d KSEG1 instruction sites have no fetch naming their own PC" + % (uncached_missing, uncached_sites)) + if mismatch["uncached"] or mismatch["cached"]: + pc, c, i = first_bad + failures.append("bios: compiled != interpreter on %d KSEG1 and %d cached instructions " + "(%d cycles short over one pass), first at 0x%08X: compiled %s, " + "interpreter %s" % (mismatch["uncached"], mismatch["cached"], + cycles_short, pc, c, i)) + if beetle_bad: + failures.append("bios: psx_icache.c interpreter path != Beetle on %d instructions" + % beetle_bad) + + # -- stubs: one fetch per executed word, in fetch-then-step order ----- + want_offs = {"shape A": [0, 4, 8, 12], "shape B": [0, 4, 8]} + shapes = stub_shapes(dispatch_c) + progs, refs, labels = [], [], [] + for name, (offs, events) in shapes.items(): + if offs != want_offs[name]: + failures.append("stub %s: fetch offsets %s, executed words %s" + % (name, offs, want_offs[name])) + steps = [e for e in events if e.startswith("psx_cyc_step")] + order_ok = len(events) == 2 * len(steps) and all( + events[2 * k].startswith("psx_icache_fetch") and + events[2 * k + 1].startswith("psx_cyc_step") for k in range(len(steps))) + if not order_ok: + failures.append("stub %s: fetch/step order %s" % (name, events)) + for seg in (KUSEG, KSEG0, KSEG1): + base = seg | 0xA0 + progs.append([("E", base + o) if o in offs else ("N", None) + for o in want_offs[name]]) + refs.append(interp_program([base + o for o in want_offs[name]])) + labels.append("%s at 0x%08X" % (name, base)) + for label, gc, gi in zip(labels, harness.run(progs), harness.run(refs)): + if gc != gi: + failures.append("stub %s: compiled %s != interpreter %s" % (label, gc, gi)) + + for msg in failures: + print("FAIL: " + msg) + if failures: + return 1 + print("PASS: compiled fetch charges == interpreter == Beetle (OpenBIOS, stubs, model)") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/recompiler/tests/uncached_fetch_codegen_test.cpp b/recompiler/tests/uncached_fetch_codegen_test.cpp new file mode 100644 index 00000000..9d8cb745 --- /dev/null +++ b/recompiler/tests/uncached_fetch_codegen_test.cpp @@ -0,0 +1,141 @@ +// Game/overlay emitter: uncached (KSEG1) code charges an I-cache fetch before +// every instruction; cached code keeps the line-leader rule unchanged. +// +// Beetle's ReadInstruction (the model psx_icache.c transcribes) never fills a +// line for a fetch at 0xA0000000 or above, so every such fetch costs +4 and +// clears the load give-back. The dirty-RAM interpreter reaches the same charge +// by fetching at every PC. The emitter used to emit the fetch only at block +// leaders and 16-byte line starts, which is exact only for cached code. +// +// The same synthetic function is generated at KSEG0 and at KSEG1. Its entry +// starts mid-line and it has a loop (a mid-line branch target), a load, a +// branch with a delay slot and a jr/delay-slot return. The emitted fetch tags, +// in emission order, must be: +// KSEG1: every instruction PC, once each, in execution-layout order; +// KSEG0: exactly the leader-rule set (entry, line starts, block leaders). +// The fetch sequences' cycle cost is checked against psx_icache.c, the +// interpreter's fetch path and a Beetle transcription by +// test_uncached_fetch_charge.py (ctest uncached_fetch_charge). + +#include "code_generator.h" +#include "control_flow.h" + +#include +#include +#include +#include +#include + +namespace { + +int failures = 0; + +void check(bool condition, const std::string& message) { + if (!condition) { + std::fprintf(stderr, "FAIL: %s\n", message.c_str()); + ++failures; + } +} + +void append_word(std::vector& bytes, uint32_t word) { + bytes.push_back(static_cast(word)); + bytes.push_back(static_cast(word >> 8)); + bytes.push_back(static_cast(word >> 16)); + bytes.push_back(static_cast(word >> 24)); +} + +// Offsets from the entry, which sits at +8 inside a 16-byte line. +const uint32_t kProgram[] = { + 0x24080003u, // +00 addiu $t0, $zero, 3 + 0x8C890000u, // +04 lw $t1, 0($a0) + 0x01285021u, // +08 addu $t2, $t1, $t0 (line start) + 0x2508FFFFu, // +0C addiu $t0, $t0, -1 (loop top: mid-line leader) + 0x1500FFFEu, // +10 bne $t0, $zero, +0C + 0x01495021u, // +14 addu $t2, $t2, $t1 (delay slot) + 0x00000000u, // +18 nop (line start, leader after the loop) + 0x03E00008u, // +1C jr $ra + 0xAC8A0004u, // +20 sw $t2, 4($a0) (delay slot) +}; +constexpr uint32_t kCount = sizeof(kProgram) / sizeof(kProgram[0]); +constexpr uint32_t kEntryPhys = 0x00010008u; + +std::string generate_at(uint32_t segment) { + const uint32_t base = segment | kEntryPhys; + PSXRecomp::PS1Executable exe{}; + exe.header.load_address = base; + exe.header.initial_pc = base; + exe.header.file_size = kCount * 4u; + for (uint32_t w : kProgram) append_word(exe.code_data, w); + + PSXRecomp::Function function{}; + function.start_addr = base; + function.end_addr = base + kCount * 4u; + function.size = kCount * 4u; + function.name = "uncached_fetch_probe"; + + PSXRecomp::CodeGenConfig config{}; + config.emit_comments = true; + config.indent = " "; + PSXRecomp::ControlFlowAnalyzer analyzer(exe); + const auto cfg = analyzer.analyze_function(function); + PSXRecomp::CodeGenerator generator(exe, config); + return generator.generate_function(function, cfg).full_code; +} + +std::vector fetch_tags(const std::string& code) { + static const std::string needle = "psx_icache_fetch(cpu, 0x"; + std::vector tags; + for (size_t pos = code.find(needle); pos != std::string::npos; + pos = code.find(needle, pos + 1)) { + tags.push_back(static_cast( + std::strtoul(code.c_str() + pos + needle.size(), nullptr, 16))); + } + return tags; +} + +std::string hex_list(const std::vector& v) { + std::string s; + char buf[16]; + for (uint32_t x : v) { + std::snprintf(buf, sizeof buf, "%s%08X", s.empty() ? "" : " ", x); + s += buf; + } + return s; +} + +void check_segment(const char* name, uint32_t segment, + const std::vector& want_offsets) { + const std::string code = generate_at(segment); + const uint32_t base = segment | kEntryPhys; + std::vector want; + for (uint32_t off : want_offsets) want.push_back(base + off); + const std::vector got = fetch_tags(code); + check(got == want, std::string(name) + " fetch tags: got [" + hex_list(got) + + "], want [" + hex_list(want) + "]"); +} + +} // namespace + +int main() { + // PSX_CODEGEN_CYCLE_PER_INSN=0 (block-up-front mode) emits no fetches at + // all; this test is about the default per-instruction mode. + const char* mode = std::getenv("PSX_CODEGEN_CYCLE_PER_INSN"); + if (mode != nullptr && mode[0] == '0') { + std::puts("SKIP: PSX_CODEGEN_CYCLE_PER_INSN=0 emits no fetch charges"); + return 0; + } + + std::vector every; + for (uint32_t i = 0; i < kCount; ++i) every.push_back(i * 4u); + check_segment("KSEG1", 0xA0000000u, every); + // Entry (+00), line start (+08), loop leader (+0C), line start and + // post-loop leader (+18). Intra-line followers are guaranteed hits. + check_segment("KSEG0", 0x80000000u, {0x00u, 0x08u, 0x0Cu, 0x18u}); + + if (failures != 0) { + std::fprintf(stderr, "uncached_fetch_codegen_test: %d failure(s)\n", failures); + return 1; + } + std::puts("PASS: uncached fetch codegen (KSEG1 every instruction, KSEG0 leader rule)"); + return 0; +} diff --git a/runtime/codegen_hash_sources.cmake b/runtime/codegen_hash_sources.cmake index 37a9621e..074ad80f 100644 --- a/runtime/codegen_hash_sources.cmake +++ b/runtime/codegen_hash_sources.cmake @@ -35,6 +35,10 @@ set(PSXRECOMP_CODEGEN_HASH_SRCS ${PSXRECOMP_CODEGEN_HASH_ROOT}/recompiler/include/gte_register_classification.h # Shared instruction detector changes alter emitted overlay substitutions. ${PSXRECOMP_CODEGEN_HASH_ROOT}/runtime/include/ws_backdrop_detect.h + # The shared cycle-cost header decides emitted code too: both emitters fold + # psx_cyc_dep_res_mask into literals and gate fetch-charge emission on + # psx_fetch_uncached. + ${PSXRECOMP_CODEGEN_HASH_ROOT}/runtime/include/psx_instr_cost.h # --- image view + walker headers ------------------------------------------ # Same gap as the .cpp/.h note above, one level down. The image VIEW decides diff --git a/runtime/include/psx_instr_cost.h b/runtime/include/psx_instr_cost.h index f1fa904e..7686af85 100644 --- a/runtime/include/psx_instr_cost.h +++ b/runtime/include/psx_instr_cost.h @@ -154,6 +154,26 @@ static inline uint32_t psx_cyc_dep_res_mask(uint32_t insn) { ((0u - ((roles & PSX_CYC_ROLE_R31) != 0u)) & (1u << 31)); } +/* 1 iff an instruction fetch at virtual address `pc` bypasses the I-cache. + * Transcribed from Beetle PS_CPU::ReadInstruction (the in-tree cpu.cpp:534-601 + * that psx_icache.c follows): a fetch at 0xA0000000 or above (KSEG1, and KSEG2 + * above it) never hits and never fills a line. Each such fetch costs +4 and + * clears the pending load give-back, on EVERY instruction. + * + * One definition for three users: + * - psx_icache.c charges the uncached fetch with it; + * - the dirty-RAM interpreter reaches that charge by fetching at every PC; + * - both static emitters (code_generator.cpp, full_function_emitter.cpp) use it + * to emit a fetch charge before every instruction whose runtime PC is + * uncached. Cached code keeps the line-leader rule, where intra-line + * fall-through followers are guaranteed hits. + * BIU bit 11 (cache disable) is run-time state, not an address property. + * Neither psx_icache.c nor the interpreter models it yet + * (ACCURACY_BURNDOWN axis 4). */ +static inline int psx_fetch_uncached(uint32_t pc) { + return pc >= 0xA0000000u; +} + #ifdef __cplusplus } #endif diff --git a/runtime/src/psx_icache.c b/runtime/src/psx_icache.c index 5ba898a3..732e354c 100644 --- a/runtime/src/psx_icache.c +++ b/runtime/src/psx_icache.c @@ -8,6 +8,7 @@ #include "psx_icache.h" #include "cpu_state.h" #include "psx_cycles.h" +#include "psx_instr_cost.h" /* psx_fetch_uncached: shared with both emitters */ #include #include #include @@ -83,7 +84,7 @@ void psx_icache_fetch_miss(CPUState* cpu, uint32_t addr) { cpu->read_absorb[cpu->read_absorb_which] = 0u; cpu->read_absorb_which = 0u; - if (addr >= 0xA0000000u) { /* KSEG1 / uncached (BIOS ROM) */ + if (psx_fetch_uncached(addr)) { /* KSEG1 / uncached (BIOS ROM): every fetch */ psx_advance_cycles(4u); return; }