recompiler, runtime: compile BIOS segment variants; SCPH-1001 enters its kernel at 0xA0000500

SCPH-1001's reset code enters its relocated kernel through the uncached
alias (lui $a0,0xA000; addiu $a0,$a0,0x500; jr $a0 at ROM 0xBFC0044C).
Beetle charges those four instructions as uncached fetches, 20 cycles; the
BIOS dispatch normalized 0xA0000500 to the kernel body compiled for runtime
0x00000500, which charges cached fetches, 11. That is the -9 cycles left at
SCPH-1001's kernel entry since #429 (docs/SEGMENT_AWARE_CODE.md §3.3).

A BIOS seed that is a runtime PC inside a relocated copy window, in another
segment than the window runs in, now asks for a segment variant (§5.4): the
function at those ROM bytes and its direct-edge closure (branch and J/JAL
targets that leave it, the call return and fall-through past its end) are
emitted again with the seed's segment. Every PC a variant computes from its
own PCs is in that segment (links, fetch tags and the uncached test, IRQ
resume, branch, J and fallthrough targets, store and exception PCs), a KSEG1
variant charges a fetch per instruction, and a jump-table case in another
segment is dispatched instead of continuing locally. Variants and their
continuations get exact-PC rows in a segment_variant_table. A variant seed
that names no emitted function entry stops the build, and so does one in a
segment that does not map physical memory (0x20000000-0x7FFFFFFF, KSEG2).

The dispatch still finds the word by its normalized key (and applies the
kernel-bless guard); a new psx_bios_hit_body() then picks the body for the
exact PC. A variant compiled for that PC runs. Any other PC in another
segment than the one the word's home body was compiled for still runs the
home body, as before, but is now recorded as a BIOS segment miss in the
segment-miss ring (kind "bios", with the home PC), so a new alias entry is
loud instead of silent (§5.5). psx_bios_key_home_pc(), emitted from the
address model, gives each key's home runtime PC: its copy window's runtime
address, or the ROM run in place. The ring, TCP segment_misses and the exit
report carry each entry's kind; tools/collect_game_misses.py lists a BIOS
segment miss for the BIOS seeds instead of writing it into the game's.

SCPH-1001's seeds gain 0xA0000500 (kernel_entry_kseg1): a 4-instruction
KSEG1 variant of kernel_trampoline_0. Every BIOS's dispatch C gains the two
helpers and calls psx_bios_hit_body() where it called the home body. A BIOS
without variant seeds (OpenBIOS, SCPH-101, SCPH-5552) keeps its full C;
SCPH-1001's full C gains the variant body and its dispatch C the one-row
segment_variant_table.

New bios_segment_variants test: OpenBIOS with KSEG1 variants of kernel
0x12BC (closure 0x6F8 by jal and 0x70C by fall-through) and 0x13F4 (0x1424
by a conditional branch only) and a KSEG0 variant of kernel memset 0x540 (a
KUSEG jump table), plus a static check of SCPH-1001's seed. It compiles the
emitted lookup, with and without variants, and queries every variant row,
every home key's home PC (checked against the PC each function body was
compiled for) and every other segment's alias of it. Seeds in a segment
that does not map physical memory stop the build, and a runtime PC in its
window's own segment stays a discovery seed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Shokunin
2026-09-29 22:43:26 -07:00
parent 6a76eb452b
commit 94be8f6386
16 changed files with 896 additions and 72 deletions
+9
View File
@@ -869,6 +869,15 @@ if(BUILD_TESTING)
--bios-recompiler $<TARGET_FILE:psxrecomp-bios>
--compiler ${CMAKE_C_COMPILER}
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/..)
# BIOS segment variants (docs/SEGMENT_AWARE_CODE.md §5.4): OpenBIOS
# with segment-qualified seeds, and SCPH-1001's KSEG1 kernel entry.
add_test(
NAME bios_segment_variants
COMMAND ${Python3_EXECUTABLE}
${CMAKE_CURRENT_SOURCE_DIR}/tests/test_bios_segment_variants.py
--bios-recompiler $<TARGET_FILE:psxrecomp-bios>
--compiler ${CMAKE_C_COMPILER}
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/..)
add_test(
NAME overlay_shim_compile_contract
COMMAND ${Python3_EXECUTABLE}
@@ -5436,6 +5436,11 @@
{
"address": "0xBFC0D4F0",
"rationale": "kernel jump-table target: A0:B4 (GetSystemInfo-family) \u2014 ROM-resident entry of the post-init A0/B0/C0 tables (read from live kernel RAM 0x200/0x874/0x674); reachable only through the vector table, invisible to transitive discovery. Class found via MMX6 card-check: A0:2B memset dispatch-missed silently, killing the save-scan (2026-07-02)."
},
{
"address": "0xA0000500",
"label": "kernel_entry_kseg1",
"rationale": "segment-qualified seed (docs/SEGMENT_AWARE_CODE.md \u00a75.4): the reset code enters the relocated kernel through its uncached alias (lui $a0,0xA000; addiu $a0,$a0,0x500; jr $a0 at ROM 0xBFC0044C-0xBFC00454). Compiles a KSEG1 variant of kernel_trampoline_0 (ROM 0xBFC10000, runtime 0x00000500) that charges an uncached fetch per instruction, as Beetle does (20 cycles against the cached body's 11)."
}
],
"excluded": [
+24
View File
@@ -296,4 +296,28 @@ std::string BiosAddressModel::emit_normalize_c() const {
return out;
}
std::string BiosAddressModel::emit_key_home_pc_c(uint32_t rom_va) const {
std::string out;
out += "/* The runtime PC each dispatch key's home body was compiled for\n";
out += " * (docs/SEGMENT_AWARE_CODE.md §5.4): its copy window's runtime address,\n";
out += " * or the ROM run in place. 0 = in no window. */\n";
out += "static uint32_t psx_bios_key_home_pc(uint32_t key) {\n";
for (const BiosAddrCopy& c : copies_) {
const uint32_t lo = c.key_is_ram ? c.ram_lo : c.rom_lo;
out += fmt::format(" /* {}: {} key 0x{:X}+ runs at 0x{:08X}+ */\n", c.name,
c.key_is_ram ? "RAM" : "ROM", lo, c.runtime_base);
out += fmt::format(" if (key >= 0x{:08X}u && key <= 0x{:08X}u)\n", lo,
lo + c.len() - 1u);
out += fmt::format(" return key - 0x{:08X}u + 0x{:08X}u;\n", lo,
c.runtime_base);
}
out += " /* The ROM, run in place. */\n";
out += fmt::format(" if (key >= 0x{:08X}u && key <= 0x{:08X}u)\n", rom_base_phys_,
rom_base_phys_ + rom_size_ - 1u);
out += fmt::format(" return 0x{:08X}u | key;\n", rom_va & 0xE0000000u);
out += " return 0u;\n";
out += "}\n\n";
return out;
}
} // namespace PSXRecompV4
+9
View File
@@ -160,6 +160,15 @@ public:
// The generated dispatch's `static uint32_t normalize(uint32_t)`.
std::string emit_normalize_c() const;
// The generated dispatch's `static uint32_t psx_bios_key_home_pc(uint32_t)`:
// the runtime PC the home body of a dispatch key (a normalize() result)
// was compiled for, i.e. runtime_pc() of the key's ROM bytes. That is its
// copy window's runtime address, or the ROM run in place in `rom_va`'s
// segment; 0 for a key outside every window and the ROM.
// (docs/SEGMENT_AWARE_CODE.md §5.4: a PC in another segment than this one
// has no body of its own unless a segment variant was compiled.)
std::string emit_key_home_pc_c(uint32_t rom_va) const;
const std::vector<BiosAddrCopy>& copies() const { return copies_; }
private:
+228 -25
View File
@@ -106,8 +106,18 @@ static uint32_t ram_alias_to_rom(uint32_t addr) {
* BIOS copies kernel/shell code into RAM. Literal interrupt resume PCs must use
* this runtime address; otherwise psx_check_interrupts rejects ROM-shell EPCs
* and falls back to the legacy sentinel path. */
/* The segment of the variant being emitted (docs/SEGMENT_AWARE_CODE.md §5.4),
* or kNoVariant while emitting the home bodies. A variant is the same ROM
* bytes run at another segment's alias of their runtime window: every PC it
* computes from its own PCs (links, fetch tags and the uncached-fetch test,
* IRQ resume PCs, branch and fallthrough targets, EPCs, store PCs) is in that
* segment, and a J target keeps the segment's top bits. */
static constexpr uint32_t kNoVariant = 0xFFFFFFFFu;
static uint32_t g_variant_seg = kNoVariant;
static uint32_t bios_runtime_pc(uint32_t rom_pc) {
return addr_model().runtime_pc(rom_pc);
const uint32_t pc = addr_model().runtime_pc(rom_pc);
return g_variant_seg == kNoVariant ? pc : (g_variant_seg | (pc & 0x1FFFFFFFu));
}
uint32_t FullFunctionEmitter::read_u32_le(const std::vector<uint8_t>& rom, uint32_t offset) {
@@ -154,8 +164,17 @@ bool FullFunctionEmitter::emit_function(
std::vector<ContinuationLabel>& out_cross_targets,
std::string* out_interpreter_reason)
{
const uint32_t norm = func.normalized_addr;
// A home body is keyed (and named) by its normalized address; a segment
// variant by its exact runtime PC, which the dispatch compares in full.
const bool variant = g_variant_seg != kNoVariant;
const uint32_t norm = variant ? bios_runtime_pc(func.entry_addr) : func.normalized_addr;
if (out_interpreter_reason) out_interpreter_reason->clear();
// Dispatch key of a continuation label inside this body: its normalized
// address at home, its exact runtime PC in a variant. (Membership tests
// against the function-entry set stay normalized.)
auto cont_key = [&](uint32_t rom, uint32_t home_norm) -> uint32_t {
return variant ? bios_runtime_pc(rom) : home_norm;
};
// RECURSION_BUG.md §25 — continuation-passing call/return (the universal fix
// for the idle-freeze host-stack leak). When PSX_CPS is set at gen time,
@@ -241,7 +260,7 @@ bool FullFunctionEmitter::emit_function(
uint32_t cross_norm = normalize_address(cross);
if (all_function_entries_norm.count(cross_norm)) continue;
block_leaders.insert(cross);
local_continuations.push_back({cross, cross_norm, norm});
local_continuations.push_back({cross, cont_key(cross, cross_norm), norm});
}
// FAITHFUL_TIMING_PLAN P5 / game code_generator.cpp parity: every basic-block
@@ -266,7 +285,7 @@ bool FullFunctionEmitter::emit_function(
if (!addr_to_raw.count(leader)) continue;
uint32_t leader_norm = normalize_address(leader);
if (all_function_entries_norm.count(leader_norm)) continue;
local_continuations.push_back({leader, leader_norm, norm});
local_continuations.push_back({leader, cont_key(leader, leader_norm), norm});
}
// Helper: find which function (in all_function_entries_norm) contains a
@@ -333,7 +352,10 @@ bool FullFunctionEmitter::emit_function(
// on the upper 4 bits of the PC. The code runs at RAM addresses, not ROM
// addresses, so we must fix J/JAL targets in the branch resolution.
auto relocate_j_target = [](uint32_t rom_addr, uint32_t target) -> uint32_t {
return addr_model().relocate_j_target(rom_addr, target);
const uint32_t t = addr_model().relocate_j_target(rom_addr, target);
// A variant's J target keeps the variant PC's top bits (§2).
return g_variant_seg == kNoVariant
? t : ((bios_runtime_pc(rom_addr) & 0xF0000000u) | (t & 0x0FFFFFFFu));
};
/* Relocate a return address ($ra) from ROM PC to the runtime address.
@@ -843,7 +865,7 @@ bool FullFunctionEmitter::emit_function(
block_leaders.insert(resume_rom);
uint32_t resume_norm = normalize_address(resume_rom);
if (!all_function_entries_norm.count(resume_norm)) {
local_continuations.push_back({resume_rom, resume_norm, norm});
local_continuations.push_back({resume_rom, cont_key(resume_rom, resume_norm), norm});
}
}
}
@@ -1253,7 +1275,7 @@ bool FullFunctionEmitter::emit_function(
if (addr_to_raw.count(addr + 8)) {
uint32_t cn = normalize_address(addr + 8);
if (!all_function_entries_norm.count(cn))
local_continuations.push_back({addr + 8, cn, norm});
local_continuations.push_back({addr + 8, cont_key(addr + 8, cn), norm});
} else {
register_cross_function_target(addr + 8);
}
@@ -1309,7 +1331,7 @@ bool FullFunctionEmitter::emit_function(
if (addr_to_raw.count(addr + 8)) {
uint32_t cn = normalize_address(addr + 8);
if (!all_function_entries_norm.count(cn))
local_continuations.push_back({addr + 8, cn, norm});
local_continuations.push_back({addr + 8, cont_key(addr + 8, cn), norm});
} else {
register_cross_function_target(addr + 8);
}
@@ -1431,7 +1453,7 @@ bool FullFunctionEmitter::emit_function(
uint32_t cont_rom = pb.terminator_addr + 8;
uint32_t cont_norm = normalize_address(cont_rom);
if (!all_function_entries_norm.count(cont_norm)) {
local_continuations.push_back({cont_rom, cont_norm, norm});
local_continuations.push_back({cont_rom, cont_key(cont_rom, cont_norm), norm});
}
} else if (cps) {
register_cross_function_target(pb.terminator_addr + 8);
@@ -1477,7 +1499,7 @@ bool FullFunctionEmitter::emit_function(
uint32_t cont_rom = pb.terminator_addr + 8;
uint32_t cont_norm = normalize_address(cont_rom);
if (!all_function_entries_norm.count(cont_norm)) {
local_continuations.push_back({cont_rom, cont_norm, norm});
local_continuations.push_back({cont_rom, cont_key(cont_rom, cont_norm), norm});
}
} else if (cps) {
register_cross_function_target(pb.terminator_addr + 8);
@@ -1603,7 +1625,11 @@ bool FullFunctionEmitter::emit_function(
if (!seen_runtime.insert(rv).second) continue;
uint32_t rom_target = ram_alias_to_rom(rv);
register_cross_function_target(rom_target);
if (addr_to_raw.count(rom_target) && block_leaders.count(rom_target))
// A variant continues locally only for a case
// value in its own segment; another segment's
// value is dispatched with its full PC (§5.4).
if (addr_to_raw.count(rom_target) && block_leaders.count(rom_target) &&
(!variant || rv == bios_runtime_pc(rom_target)))
targets.push_back({rv, rom_target});
}
if (!targets.empty()) {
@@ -1748,7 +1774,8 @@ void FullFunctionEmitter::emit_dispatch(
const std::vector<uint8_t>& rom,
uint32_t base_addr,
const std::vector<BiosVectorTable>& bios_vectors,
const std::vector<BiosAlias>& bios_aliases)
const std::vector<BiosAlias>& bios_aliases,
const std::vector<std::pair<uint32_t, std::string>>& variant_rows)
{
// RECURSION_BUG.md §25 — continuation-passing. Under CPS the BIOS A0/B0/C0
// vector dispatch must TAIL-TRANSFER to the handler (set cpu->pc, return)
@@ -1786,6 +1813,7 @@ void FullFunctionEmitter::emit_dispatch(
// backend ABI come from the runtime headers rather than being
// re-declared here, so they cannot drift from what the runtime reads.
out += "#include \"psx_bios_backend.h\"\n";
out += "#include \"psx_segment_miss.h\"\n";
out += "#include <stdint.h>\n";
out += "#include <stdio.h>\n";
out += "#include <stdlib.h>\n\n";
@@ -1824,6 +1852,10 @@ void FullFunctionEmitter::emit_dispatch(
out += fmt::format("extern void {}(CPUState* cpu);\n", fn_sym(norm));
}
}
for (const auto& [pc, sym] : variant_rows) {
(void)pc;
out += fmt::format("extern void {}(CPUState* cpu);\n", sym);
}
out += "\n";
// --- BIOS vector switch handlers ---
@@ -1954,6 +1986,18 @@ void FullFunctionEmitter::emit_dispatch(
}
out += "};\n\n";
// Segment variants (docs/SEGMENT_AWARE_CODE.md §5.4): bodies compiled for
// one exact runtime PC in another segment than their window's. Keyed by
// the full PC; the normalized table above still finds the word (and its
// kernel-bless guard applies), then an exact match runs the variant.
if (!variant_rows.empty()) {
out += fmt::format("static const DispatchEntry segment_variant_table[{}] = {{\n",
variant_rows.size());
for (const auto& [pc, sym] : variant_rows)
out += fmt::format(" {{ 0x{:08X}u, {} }},\n", pc, sym);
out += "};\n\n";
}
// --- Kernel body-extent table (runtime kernel-image bless) ---
// The BIOS copies Kernel Part 2 from ROM [0x1FC10000,0x1FC18000) to RAM
// [0x500,0x8500) at boot; the functions above with keys in that window
@@ -2152,6 +2196,38 @@ void FullFunctionEmitter::emit_dispatch(
// C can never drift from the analysis that produced it.
out += addr_model().emit_normalize_c();
// The body a normalized hit runs (docs/SEGMENT_AWARE_CODE.md §5.4). The
// table is keyed by the normalized address, so it finds a word in any
// segment. A segment variant compiled for the exact PC runs instead of the
// word's home body. Any other PC in another segment than the home body's
// still runs the home body, which bakes its own segment's links, EPCs,
// fetch tags and store PCs: that is a BIOS segment miss, recorded in the
// segment-miss ring (kind BIOS) so the entry is loud and can be seeded.
out += addr_model().emit_key_home_pc_c(base_addr);
out += "extern uint64_t s_frame_count;\n";
out += "static PsxRecompFunc psx_bios_hit_body(CPUState* cpu, uint32_t addr, uint32_t key,\n";
out += " PsxRecompFunc home) {\n";
if (!variant_rows.empty()) {
out += " /* A segment variant compiled for this exact PC. */\n";
out += fmt::format(" for (int lo_ = 0, hi_ = {} - 1; lo_ <= hi_;) {{\n",
variant_rows.size());
out += " const int mid_ = (lo_ + hi_) / 2;\n";
out += " if (segment_variant_table[mid_].addr == addr)\n";
out += " return segment_variant_table[mid_].func;\n";
out += " if (segment_variant_table[mid_].addr < addr) lo_ = mid_ + 1;\n";
out += " else hi_ = mid_ - 1;\n";
out += " }\n";
}
out += " {\n";
out += " const uint32_t home_pc = psx_bios_key_home_pc(key);\n";
out += " if (home_pc != 0u && ((addr ^ home_pc) & 0xE0000000u) != 0u)\n";
out += " psx_segment_miss_record_kind(addr, home_pc, cpu->gpr[31], cpu->gpr[29],\n";
out += " (uint32_t)s_frame_count,\n";
out += " PSX_SEGMENT_MISS_BIOS);\n";
out += " }\n";
out += " return home;\n";
out += "}\n\n";
out += "extern int dirty_ram_dispatch(CPUState* cpu, uint32_t addr, uint32_t stop_addr);\n";
out += "extern int dirty_ram_is_dirty(uint32_t phys);\n";
out += "extern int psx_kernel_bless_dispatchable(uint32_t phys);\n";
@@ -2288,7 +2364,9 @@ void FullFunctionEmitter::emit_dispatch(
}
out += " g_debug_current_func_addr = phys;\n";
out += " debug_server_trace_dispatch(phys);\n";
out += " dispatch_table[mid].func(cpu);\n";
// The exact PC picks the body: a segment variant, or the home body
// (recorded when the PC is in another segment than it, §5.4).
out += " psx_bios_hit_body(cpu, addr, phys, dispatch_table[mid].func)(cpu);\n";
out += " g_dispatch_static_hits++;\n";
out += " found = 1;\n";
out += " break;\n";
@@ -2463,9 +2541,11 @@ EmitStats FullFunctionEmitter::emit(
const std::string& out_dir,
const std::string& out_stem,
const std::vector<BiosVectorTable>& bios_vectors,
const std::vector<BiosAlias>& bios_aliases)
const std::vector<BiosAlias>& bios_aliases,
const std::vector<BiosSegmentVariantSeed>& variant_seeds)
{
EmitStats stats;
g_variant_seg = kNoVariant;
// Setup / Retro zips omit generated/; create it before ofstream.
if (!out_dir.empty()) {
@@ -2566,19 +2646,20 @@ EmitStats FullFunctionEmitter::emit(
// ---- PASS 1: dry run to collect cross-function tail-call targets ----
std::map<uint32_t, std::set<uint32_t>> cross_targets_by_parent;
/* Instruction-address -> owning function (normalized). Hard caps
* partition the address space, so each walked instruction has exactly
* one owner. Used below to route cross-function targets to the
* function that actually CONTAINS them: the range-based parent guess
* (largest entry <= target) is wrong when seeds split a function into
* fragments and the preceding fragment ends before the target —
* emit_function's injection then silently dropped the continuation
* and the runtime FAIL-FASTed on dispatch (OpenBIOS exceptionHandler
* priority_loop 0xBFC208E8, split by the patch-slot code_ptr seeds).
* The segment-variant closure below uses it too. */
std::map<uint32_t, uint32_t> insn_owner;
{
std::vector<ContinuationLabel> dry_run_continuations;
std::vector<ContinuationLabel> dry_run_cross;
/* Instruction-address -> owning function (normalized). Hard caps
* partition the address space, so each walked instruction has exactly
* one owner. Used below to route cross-function targets to the
* function that actually CONTAINS them: the range-based parent guess
* (largest entry <= target) is wrong when seeds split a function into
* fragments and the preceding fragment ends before the target —
* emit_function's injection then silently dropped the continuation
* and the runtime FAIL-FASTed on dispatch (OpenBIOS exceptionHandler
* priority_loop 0xBFC208E8, split by the patch-slot code_ptr seeds). */
std::map<uint32_t, uint32_t> insn_owner;
for (const auto& fn : dr.functions) {
std::string lineage = fn.discovered_by;
uint32_t cap = hard_caps.count(fn.entry_addr)
@@ -2660,6 +2741,105 @@ EmitStats FullFunctionEmitter::emit(
stats.dispatch_entries = static_cast<uint32_t>(emitted_normalized.size());
// ---- Segment variants (docs/SEGMENT_AWARE_CODE.md §5.4) ----
// Each segment-qualified seed names a runtime PC in another segment than
// its copy window's. The function at its ROM bytes and that function's
// direct-edge closure (branch and J/JAL targets that leave it, the call
// return and fall-through past its end) are emitted again with the seed's
// segment. Dispatch runs a variant only for its exact PC.
std::vector<std::pair<uint32_t, std::string>> variant_rows; // runtime PC -> symbol
std::vector<ContinuationLabel> variant_continuations;
if (!variant_seeds.empty()) {
std::map<uint32_t, uint32_t> norm_to_entry;
for (const auto& fn : dr.functions) norm_to_entry[fn.normalized_addr] = fn.entry_addr;
auto owner_entry = [&](uint32_t rom) -> uint32_t {
if (entry_to_idx.count(rom)) return rom;
auto ow = insn_owner.find(rom);
if (ow == insn_owner.end()) return 0;
auto e = norm_to_entry.find(ow->second);
return e == norm_to_entry.end() ? 0 : e->second;
};
std::map<uint32_t, std::set<uint32_t>> closure; // segment -> ROM entries
for (const auto& vs : variant_seeds) {
if (!entry_to_idx.count(vs.rom) || !emitted_normalized.count(normalize_address(vs.rom))) {
throw std::runtime_error(fmt::format(
"segment-variant seed 0x{:08X} ({}) names ROM 0x{:08X}, which is no "
"emitted function entry (docs/SEGMENT_AWARE_CODE.md §5.4)",
vs.pc, vs.label, vs.rom));
}
const uint32_t seg = vs.pc & 0xE0000000u;
std::vector<uint32_t> work{vs.rom};
while (!work.empty()) {
const uint32_t entry = work.back();
work.pop_back();
if (!closure[seg].insert(entry).second) continue;
const auto& fn = dr.functions[entry_to_idx.at(entry)];
uint32_t cap = hard_caps.count(fn.entry_addr) ? hard_caps[fn.entry_addr] : rom_end + 1;
auto sfr = FunctionDiscovery::walk_function(rom, base_addr, rom_end, fn.entry_addr,
cap, fn.discovered_by);
std::set<uint32_t> own;
for (const auto& [a, w] : sfr.instructions) own.insert(a);
auto reach = [&](uint32_t target_rom) {
if (own.count(target_rom)) return;
const uint32_t e = owner_entry(target_rom);
if (e && emitted_normalized.count(normalize_address(e))) work.push_back(e);
};
uint32_t last = 0, prev_raw = 0, last_raw = 0;
for (const auto& [a, w] : sfr.instructions) {
const uint32_t op = w >> 26;
if (op == 0x02 || op == 0x03) { // j / jal: the runtime PC's top bits
const uint32_t rt_src = addr_model().runtime_pc(a);
const uint32_t t = ((rt_src + 4u) & 0xF0000000u) | ((w & 0x03FFFFFFu) << 2);
const uint32_t t_phys = t & 0x1FFFFFFFu;
reach(t_phys >= (base_addr & 0x1FFFFFFFu) &&
t_phys <= (rom_end & 0x1FFFFFFFu)
? (base_addr & 0xE0000000u) | t_phys
: addr_model().ram_alias_to_rom(t));
} else if (op == 0x01 || (op >= 0x04 && op <= 0x07)) { // branches
reach(a + 4u + (static_cast<uint32_t>(static_cast<int32_t>(
static_cast<int16_t>(w & 0xFFFFu))) << 2));
}
prev_raw = last_raw;
last_raw = w;
last = a;
}
// Past the end: a fall-through, or the return of a final call.
// Only an unconditional j / jr in the last pair ends the path.
auto ends = [](uint32_t w) {
return (w >> 26) == 0x02 || ((w >> 26) == 0 && (w & 0x3F) == 0x08);
};
if (last && !ends(prev_raw) && !ends(last_raw)) reach(last + 4u);
}
}
for (const auto& [seg, entries] : closure) {
for (uint32_t entry : entries) {
const auto& fn = dr.functions[entry_to_idx.at(entry)];
uint32_t cap = hard_caps.count(fn.entry_addr) ? hard_caps[fn.entry_addr] : rom_end + 1;
auto sfr = FunctionDiscovery::walk_function(rom, base_addr, rom_end, fn.entry_addr,
cap, fn.discovered_by);
if (!sfr.unsupported.empty()) continue;
std::set<uint32_t> injected;
if (cross_targets_by_parent.count(fn.normalized_addr))
injected = cross_targets_by_parent[fn.normalized_addr];
std::vector<ContinuationLabel> discard_cross;
std::string interpreter_reason;
g_variant_seg = seg;
const uint32_t key = bios_runtime_pc(fn.entry_addr);
const bool ok = emit_function(full_c, fn, sfr, all_function_entries_norm, rom,
base_addr, rom_end, variant_continuations, injected,
discard_cross, &interpreter_reason);
g_variant_seg = kNoVariant;
if (!ok) {
throw std::runtime_error(fmt::format(
"segment variant 0x{:08X} of ROM 0x{:08X} could not be emitted "
"though its home body was: {}", key, fn.entry_addr, interpreter_reason));
}
variant_rows.emplace_back(key, fn_sym(key));
stats.variant_functions++;
}
}
}
// Emit fatal stubs for skipped functions (e.g. FPU) so calls to them
// link but abort at runtime with a diagnostic.
for (const auto& [skip_addr, reason] : stats.skipped) {
@@ -2717,6 +2897,28 @@ EmitStats FullFunctionEmitter::emit(
}
}
// A variant's continuations are keyed by their exact runtime PC and
// re-enter the variant body.
{
std::map<uint32_t, ContinuationLabel> unique_variant;
std::set<uint32_t> variant_keys;
for (const auto& [pc, sym] : variant_rows) variant_keys.insert(pc);
for (const auto& cl : variant_continuations) {
if (addr_model().in_install_slot_range(normalize_address(cl.rom_addr))) continue;
if (!variant_keys.count(cl.norm_addr)) unique_variant[cl.norm_addr] = cl;
}
for (const auto& [key, cl] : unique_variant) {
full_c += fmt::format("void {}(CPUState* cpu) {{\n",
cont_sym(cl.parent_func_norm, cl.rom_addr));
full_c += fmt::format(" cpu->pc = 0x{:08X}u;\n", cl.rom_addr);
full_c += fmt::format(" {}(cpu);\n", fn_sym(cl.parent_func_norm));
full_c += "}\n\n";
variant_rows.emplace_back(key, cont_sym(cl.parent_func_norm, cl.rom_addr));
}
std::sort(variant_rows.begin(), variant_rows.end());
stats.variant_entries = static_cast<uint32_t>(variant_rows.size());
}
if (!unique_continuations.empty()) {
full_c += fmt::format("\n/* --- {} continuation wrappers for jal/jalr return routing --- */\n\n",
unique_continuations.size());
@@ -2744,7 +2946,8 @@ EmitStats FullFunctionEmitter::emit(
{
std::string dispatch_c;
emit_dispatch(dispatch_c, dr, emitted_normalized, unique_continuations,
bios_sha256, rom, base_addr, bios_vectors, bios_aliases);
bios_sha256, rom, base_addr, bios_vectors, bios_aliases,
variant_rows);
std::string path = out_dir + "/" + out_stem + "_dispatch.c";
write_file_if_changed(path, dispatch_c);
}
+19 -2
View File
@@ -40,6 +40,21 @@ struct EmitStats {
uint32_t continuation_entries = 0;
std::vector<std::pair<uint32_t, std::string>> skipped; // (addr, reason)
std::vector<std::pair<uint32_t, std::string>> interpreted; // (addr, reason)
uint32_t variant_functions = 0; // segment-variant bodies (§5.4)
uint32_t variant_entries = 0; // their exact-PC dispatch rows
};
// A segment-qualified BIOS seed (docs/SEGMENT_AWARE_CODE.md §5.4): a runtime
// PC inside a relocated copy window, in another segment than the window runs
// in. SCPH-1001 enters its kernel copy (runtime 0x00000500) through the
// uncached alias 0xA0000500. The function at `rom` and its direct-edge closure
// are emitted again with `pc`'s segment: every PC they bake is in it, a KSEG1
// variant charges a fetch per instruction, and the dispatch runs the variant
// for its exact PC (every other PC keeps the normalized key).
struct BiosSegmentVariantSeed {
uint32_t pc; // requested runtime PC, e.g. 0xA0000500
uint32_t rom; // ROM VA of the same bytes, e.g. 0xBFC10000
std::string label;
};
// A continuation label: the return point inside a calling function after
@@ -81,7 +96,8 @@ public:
const std::string& out_dir,
const std::string& out_stem,
const std::vector<BiosVectorTable>& bios_vectors = {},
const std::vector<BiosAlias>& bios_aliases = {});
const std::vector<BiosAlias>& bios_aliases = {},
const std::vector<BiosSegmentVariantSeed>& variant_seeds = {});
private:
// Emit a single function's C code to the output stream.
@@ -117,7 +133,8 @@ private:
const std::vector<uint8_t>& rom,
uint32_t base_addr,
const std::vector<BiosVectorTable>& bios_vectors,
const std::vector<BiosAlias>& bios_aliases);
const std::vector<BiosAlias>& bios_aliases,
const std::vector<std::pair<uint32_t, std::string>>& variant_rows);
static uint32_t normalize_address(uint32_t addr);
static uint32_t read_u32_le(const std::vector<uint8_t>& rom, uint32_t offset);
+61 -5
View File
@@ -46,6 +46,7 @@
#include "full_function_emitter.h"
#include "function_discovery.h"
#include "mips_decoder.h"
#include "ps1_exe_parser.h"
#include "write_if_changed.h"
namespace fs = std::filesystem;
@@ -763,6 +764,48 @@ int run_boot_slice(const fs::path& bios_path, const fs::path& out_dir,
return 0;
}
// ----- Segment-qualified seeds -----------------------------------------------
// A runtime PC inside a relocated copy window, in another segment than the
// window runs in, asks for a segment variant of the code at those ROM bytes
// (docs/SEGMENT_AWARE_CODE.md §5.4; SCPH-1001's kernel entry 0xA0000500). It
// is not a discovery seed, because discovery works on ROM addresses: move it
// out of `seeds` and return it with the ROM VA of its bytes. Only KUSEG below
// 0x20000000, KSEG0 and KSEG1 alias the window; a seed in any other segment
// names no code (it addresses no RAM) and stops the build.
std::vector<PSXRecompV4::BiosSegmentVariantSeed> split_segment_variant_seeds(
std::vector<PSXRecompV4::Seed>& seeds,
const PSXRecompV4::BiosAddressModel& model) {
std::vector<PSXRecompV4::BiosSegmentVariantSeed> variants;
std::vector<PSXRecompV4::Seed> rom_seeds;
for (const auto& sd : seeds) {
const uint32_t phys = sd.address & 0x1FFFFFFFu;
const uint32_t seg = sd.address & 0xE0000000u;
bool variant = false;
for (const auto& c : model.copies()) {
const uint32_t rt = c.runtime_base & 0x1FFFFFFFu;
if (phys >= rt && phys - rt < c.len() && seg != (c.runtime_base & 0xE0000000u)) {
if (!PSXRecomp::maps_physical(sd.address)) {
throw std::runtime_error(fmt::format(
"seed 0x{:08X} ({}) is in segment 0x{:08X}, which does not map "
"physical memory (only KUSEG below 0x20000000, KSEG0 and KSEG1 "
"do), so it cannot ask for a segment variant of the '{}' window "
"(docs/SEGMENT_AWARE_CODE.md §5.4); that window runs its bytes at "
"0x{:08X}", sd.address, sd.label, seg, c.name,
c.runtime_base + (phys - rt)));
}
variants.push_back({sd.address, 0xA0000000u | (c.rom_lo + (phys - rt)),
sd.label});
variant = true;
break;
}
}
if (!variant) rom_seeds.push_back(sd);
}
seeds = std::move(rom_seeds);
return variants;
}
// ----- Phase 2: full BIOS emission ------------------------------------------
int run_emit_full(const fs::path& bios_path, const fs::path& out_dir,
@@ -798,12 +841,18 @@ int run_emit_full(const fs::path& bios_path, const fs::path& out_dir,
// 2. Load seeds.
auto seeds = load_seeds(seed_path);
const auto variant_seeds = split_segment_variant_seeds(seeds, model);
const size_t vector_seed_count =
add_bios_vector_target_seeds(seeds, rom, kBiosBase,
kBiosBase + static_cast<uint32_t>(kBiosSize) - 1,
bios_vectors);
std::fprintf(stdout, "psxrecomp-bios: loaded %zu seeds from %s\n",
seeds.size(), seed_path.string().c_str());
seeds.size() + variant_seeds.size(), seed_path.string().c_str());
for (const auto& vs : variant_seeds) {
std::fprintf(stdout,
"psxrecomp-bios: segment-variant seed 0x%08X (%s): ROM 0x%08X compiled for "
"segment 0x%08X\n", vs.pc, vs.label.c_str(), vs.rom, vs.pc & 0xE0000000u);
}
if (vector_seed_count != 0) {
std::fprintf(stdout,
"psxrecomp-bios: added %zu ROM-resident BIOS vector target seeds\n",
@@ -850,13 +899,18 @@ int run_emit_full(const fs::path& bios_path, const fs::path& out_dir,
// 4. Emit full C.
const auto stats = PSXRecompV4::FullFunctionEmitter::emit(
rom, kBiosBase, kBiosBase + static_cast<uint32_t>(kBiosSize) - 1,
dr, sha, out_dir.string(), out_stem, bios_vectors, bios_aliases);
dr, sha, out_dir.string(), out_stem, bios_vectors, bios_aliases, variant_seeds);
std::fprintf(stdout,
"psxrecomp-bios: EMIT OK emitted=%u interpreted=%u skipped=%u instructions=%u "
"dispatch_entries=%u\n",
stats.functions_emitted, stats.functions_interpreted, stats.functions_skipped,
stats.total_instructions, stats.dispatch_entries);
if (stats.variant_functions != 0) {
std::fprintf(stdout,
"psxrecomp-bios: segment variants: %u function(s), %u exact-PC dispatch row(s)\n",
stats.variant_functions, stats.variant_entries);
}
if (stats.functions_interpreted > 0) {
std::fprintf(stdout, "psxrecomp-bios: interpreter fallbacks:\n");
@@ -890,10 +944,12 @@ int run_discover(const fs::path& bios_path, const fs::path& out_dir,
const auto rom = load_file_strict(bios_path, kBiosSize);
const std::string sha = sha256_hex(rom);
// 2. Load seeds.
const auto seeds = load_seeds(seed_path);
// 2. Load seeds. Segment-variant seeds are an emission request, not
// discovery input.
auto seeds = load_seeds(seed_path);
const size_t variant_seed_count = split_segment_variant_seeds(seeds, model).size();
std::fprintf(stdout, "psxrecomp-bios: loaded %zu seeds from %s\n",
seeds.size(), seed_path.string().c_str());
seeds.size() + variant_seed_count, seed_path.string().c_str());
// 3. Run discovery.
const auto dr = PSXRecompV4::FunctionDiscovery::discover(
@@ -0,0 +1,410 @@
#!/usr/bin/env python3
"""BIOS segment variants (docs/SEGMENT_AWARE_CODE.md §5.4, rollout PR D).
The BIOS copies code into RAM and runs it at one segment per copy window (the
OpenBIOS and SCPH-1001 kernels at KUSEG 0x500). Code entered through another
segment's alias of that window runs there on hardware: SCPH-1001's reset code
enters its kernel through the uncached alias 0xA0000500, so each of those
instructions is an uncached fetch. A segment-qualified seed in the BIOS seeds
file (a runtime PC in another segment than its window's) asks for a variant:
the function at those ROM bytes and its direct-edge closure are emitted again
with the seed's segment, and the dispatch runs a variant for its exact PC.
OpenBIOS is bundled, so this test runs the real psxrecomp-bios on it with three
extra seeds (KSEG1 variants of kernel 0x12BC and 0x13F4, and a KSEG0 variant
of kernel memset 0x540) and checks:
- the home output is unchanged: the variant build only adds lines;
- 0x12BC's closure also covers 0x6F8, which it calls with jal (the call
targets the KSEG1 PC 0xA00006F8), and 0x70C, which 0x6F8 falls through
into; 0x13F4's covers 0x1424, which only its conditional branch at
0x1408 reaches;
- every PC a variant bakes is in its segment: fetch tags, store PCs, links,
unaligned-access PCs, IRQ resume and fallthrough PCs (the kernel runs at
KUSEG, so no KUSEG PC of the window may appear);
- a KSEG1 variant charges a fetch before every instruction (§5.6), while a
KSEG0 variant keeps the cached line-leader rule;
- a jump-table case whose value is in another segment than the variant is
not a local case (memset's table holds KUSEG PCs), so it is dispatched;
- the dispatch gains an exact-PC variant table (entries and continuations,
sorted) consulted after the normalized row is found. The emitted lookup
(normalize(), the variant table, psx_bios_key_home_pc() and
psx_bios_hit_body()) is compiled and queried, with and without variants:
each function row's home PC is the PC its body was compiled for (its
first fetch tag), every variant row runs its own body, every home key's
home PC runs the
home body silently, and every other segment's alias of a home key runs
the home body and is recorded as a BIOS segment miss (kind BIOS, with the
home PC), so a new alias entry is loud;
- a variant seed that names no emitted function entry stops the build, and
so does one in a segment that does not map physical memory
(0x20000000-0x7FFFFFFF, KSEG2); a runtime PC in its window's own segment
is no variant request and stays a discovery seed;
- SCPH-1001's seeds ask for its KSEG1 kernel entry 0xA0000500 (the retail
image is not in the tree, so that part is a static check).
Usage: python test_bios_segment_variants.py --bios-recompiler <psxrecomp-bios>
[--compiler <cc>]
Exit 0 = PASS.
"""
import argparse
import json
import os
import re
import shutil
import subprocess
import sys
import tempfile
HERE = os.path.dirname(os.path.abspath(__file__))
ROOT = os.path.normpath(os.path.join(HERE, "..", ".."))
KSEG0, KSEG1 = 0x80000000, 0xA0000000
VARIANTS = {0xA00012BC: "variant_kseg1_12BC", 0x80000540: "variant_kseg0_540",
0xA00013F4: "variant_kseg1_13F4"}
def check(cond, msg):
if not cond:
raise SystemExit("FAIL: " + msg)
def profile(tmp, seeds_path):
with open(os.path.join(ROOT, "bios", "OpenBIOS.toml")) as f:
text = f.read()
# TOML literal strings, and function replacements: a Windows path's
# backslashes are neither TOML nor re.sub escapes.
rom = os.path.join(ROOT, "bios", "openbios.bin")
text = re.sub(r'^seeds\s*=.*$', lambda m: "seeds = '%s'" % seeds_path, text, flags=re.M)
text = re.sub(r'^rom\s*=.*$', lambda m: "rom = '%s'" % rom, text, flags=re.M)
path = os.path.join(tmp, "OpenBIOS_variants.toml")
with open(path, "w") as f:
f.write(text)
return path
def emit(binary, config, out):
os.makedirs(out, exist_ok=True)
r = subprocess.run([binary, "--config", config, "--out-dir", out], cwd=ROOT,
capture_output=True, text=True)
if r.returncode != 0:
return r, None, None
with open(os.path.join(out, "OpenBIOS_full.c")) as f:
full = f.read()
with open(os.path.join(out, "OpenBIOS_dispatch.c")) as f:
disp = f.read()
return r, full, disp
def body(full, name):
m = re.search(r"^void %s\(CPUState\* cpu\) \{\n(.*?)^\}" % name, full, re.M | re.S)
return m.group(1) if m else None
def emit_with(binary, tmp, name, seeds, extra):
"""Emit OpenBIOS with extra seed addresses; returns (result, full, dispatch)."""
data = dict(seeds, seeds=seeds["seeds"] + [
{"address": "0x%08X" % pc, "label": label, "rationale": "test"}
for pc, label in extra])
path = os.path.join(tmp, name + ".json")
with open(path, "w") as f:
json.dump(data, f)
return emit(binary, profile(tmp, path), os.path.join(tmp, name))
def fragment(src, begin, end):
i = src.index(begin)
return src[i:src.index(end, i) + len(end)]
# The emitted lookup a normalized dispatch hit runs, compiled and queried. The
# fragments are the emitted C itself; only the CPU state, the home body and
# the segment-miss recorder are doubles.
HARNESS_HEAD = r"""
#include <stdint.h>
#include <stdio.h>
typedef struct { uint32_t gpr[32]; uint32_t pc; } CPUState;
#define PSX_SEGMENT_MISS_BIOS 1u
static unsigned records;
static uint32_t rec_addr, rec_home, rec_kind;
static void psx_segment_miss_record_kind(uint32_t addr, uint32_t home, uint32_t ra,
uint32_t sp, uint32_t frame, uint32_t kind) {
(void)ra; (void)sp; (void)frame;
records++; rec_addr = addr; rec_home = home; rec_kind = kind;
}
uint64_t s_frame_count;
uint32_t g_psx_ram_mask = 0x001FFFFFu;
static void home_body(CPUState* cpu) { (void)cpu; }
"""
HARNESS_MAIN = r"""
static int failures;
#define FAIL(...) do { if (failures++ < 20) { printf(__VA_ARGS__); printf("\n"); } } while (0)
static int is_home_key(uint32_t k) {
for (unsigned i = 0; i < N_KEYS; ++i) if (home_keys[i] == k) return 1;
return 0;
}
static int variant_of(uint32_t pc) {
for (unsigned i = 0; i < N_VARIANTS; ++i) if (variant_pcs[i] == pc) return (int)i;
return -1;
}
int main(void) {
static const uint32_t segs[3] = {0x00000000u, 0x80000000u, 0xA0000000u};
CPUState cpu = {{0}, 0};
unsigned vectors = 0, homes = 0, aliases = 0, aliases_recorded = 0, variants_hit = 0;
for (unsigned i = 0; i < N_VARIANTS; ++i) {
const uint32_t pc = variant_pcs[i], key = normalize(pc);
const unsigned before = records;
if (!is_home_key(key)) FAIL("variant 0x%08X: its word 0x%08X has no home row", pc, key);
if (psx_bios_hit_body(&cpu, pc, key, home_body) != variant_fns[i] || records != before)
FAIL("variant 0x%08X does not run its own body silently", pc);
else variants_hit++;
}
for (unsigned i = 0; i < N_KEYS; ++i) {
const uint32_t key = home_keys[i], home = psx_bios_key_home_pc(key);
if (home == 0u) {
if (key != 0xA0u && key != 0xB0u && key != 0xC0u)
FAIL("key 0x%08X is in no window and is no call vector", key);
vectors++;
continue;
}
if (normalize(home) != key) FAIL("key 0x%08X: home PC 0x%08X normalizes elsewhere", key, home);
if (entry_pcs[i] != 0u && entry_pcs[i] != home)
FAIL("key 0x%08X: home PC 0x%08X, but its body was compiled for 0x%08X",
key, home, entry_pcs[i]);
if (variant_of(home) >= 0) FAIL("home PC 0x%08X has a variant row", home);
{
const unsigned before = records;
if (psx_bios_hit_body(&cpu, home, key, home_body) != home_body || records != before)
FAIL("home PC 0x%08X does not run its home body silently", home);
homes++;
}
for (unsigned s = 0; s < 3; ++s) {
const uint32_t alias = segs[s] | (home & 0x1FFFFFFFu);
const unsigned before = records;
if (alias == home || normalize(alias) != key) continue;
aliases++;
const int v = variant_of(alias);
PsxRecompFunc got = psx_bios_hit_body(&cpu, alias, key, home_body);
if (v >= 0) {
if (got != variant_fns[v] || records != before)
FAIL("alias 0x%08X does not run its variant silently", alias);
} else if (got != home_body || records != before + 1 || rec_addr != alias ||
rec_home != home || rec_kind != PSX_SEGMENT_MISS_BIOS) {
FAIL("alias 0x%08X of home 0x%08X is not a recorded BIOS segment miss", alias, home);
} else {
aliases_recorded++;
}
}
}
printf("%u %u %u %u %u %d\n", variants_hit, homes, aliases, aliases_recorded, vectors, failures);
return failures != 0;
}
"""
def run_lookup_harness(compiler, disp, full, tmp, name):
"""Compile the emitted hit lookup of `disp` and query it; returns
(variants hit, home PCs, aliases, aliases recorded, call vectors)."""
rows = re.search(r"static const DispatchEntry dispatch_table\[\d+\] = \{\n(.*?)\n\};",
disp, re.S)
check(rows, "the dispatch table is emitted")
keys = re.findall(r"\{ 0x([0-9A-F]{8})u, \w+ \}", rows.group(1))
# Ground truth for psx_bios_key_home_pc(): a function row's body charges
# its entry's fetch first, and that tag is the runtime PC it was compiled
# for.
entry_pc = {}
for key, sym in re.findall(r"\{ 0x([0-9A-F]{8})u, (\w+) \}", rows.group(1)):
text = None if "_cont_" in sym else body(full, sym)
m = text and re.search(r"psx_icache_fetch\(cpu, 0x([0-9A-F]{8})u\)", text)
if m:
entry_pc[key] = m.group(1)
check({int(pc, 16) & 0xE0000000 for pc in entry_pc.values()} == {0, KSEG0, KSEG1},
"function rows in all three windows (kernel, shell, ROM) give their compiled PC")
vm = re.search(r"static const DispatchEntry segment_variant_table\[\d+\] = \{\n.*?\n\};\n",
disp, re.S)
variants = re.findall(r"\{ 0x([0-9A-F]{8})u, (\w+) \}", vm.group(0)) if vm else []
src = HARNESS_HEAD
src += "".join("static void %s(CPUState* cpu) { (void)cpu; }\n" % sym for _, sym in variants)
src += fragment(disp, "typedef void (*PsxRecompFunc)(CPUState*);", "} DispatchEntry;\n")
if vm:
src += vm.group(0)
src += fragment(disp, "extern uint32_t g_psx_ram_mask;", " return phys;\n}\n")
src += fragment(disp, "/* The runtime PC each dispatch key's home body",
" return home;\n}\n")
src += "static const uint32_t home_keys[] = {%s};\n" % ", ".join("0x%su" % k for k in keys)
src += "static const uint32_t entry_pcs[] = {%s};\n" % ", ".join(
"0x%su" % entry_pc.get(k, "00000000") for k in keys)
src += "#define N_KEYS %du\n#define N_VARIANTS %du\n" % (len(keys), len(variants))
src += "static const uint32_t variant_pcs[] = {%s};\n" % (
", ".join("0x%su" % a for a, _ in variants) or "0u")
src += "static const PsxRecompFunc variant_fns[] = {%s};\n" % (
", ".join(sym for _, sym in variants) or "0")
src += HARNESS_MAIN
path = os.path.join(tmp, name + ".c")
exe = os.path.join(tmp, name + (".exe" if os.name == "nt" else ""))
with open(path, "w") as f:
f.write(src)
if os.path.basename(compiler).lower() in ("cl", "cl.exe", "clang-cl", "clang-cl.exe"):
cmd = [compiler, "/nologo", "/Od", path, "/Fe:" + exe]
else:
cmd = [compiler, "-std=c11", "-O1", path, "-o", exe]
r = subprocess.run(cmd, capture_output=True, text=True, cwd=tmp)
check(r.returncode == 0, "the emitted dispatch lookup compiles:\n" + (r.stdout + r.stderr)[-3000:])
r = subprocess.run([exe], capture_output=True, text=True)
check(r.returncode == 0, "the emitted dispatch lookup answers every query:\n" + r.stdout[-3000:])
return tuple(int(v) for v in r.stdout.split()[:5])
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--bios-recompiler",
default=os.path.join(ROOT, "recompiler", "build", "psxrecomp-bios"))
ap.add_argument("--compiler",
default=os.environ.get("CC") or shutil.which("cc") or shutil.which("gcc"))
args = ap.parse_args()
if not args.compiler:
raise SystemExit("a C compiler is required to query the emitted dispatch lookup")
if not os.path.isfile(args.bios_recompiler):
raise SystemExit("recompiler not found: %s (build it first)" % args.bios_recompiler)
with open(os.path.join(ROOT, "recompiler", "seeds", "openbios_elf_seeds.json")) as f:
seeds = json.load(f)
with tempfile.TemporaryDirectory() as tmp:
base_seeds = os.path.join(tmp, "base.json")
with open(base_seeds, "w") as f:
json.dump(seeds, f)
r, base_full, base_disp = emit(args.bios_recompiler, profile(tmp, base_seeds),
os.path.join(tmp, "base"))
check(base_full is not None, "baseline OpenBIOS emission failed:\n" + r.stderr[-2000:])
var = dict(seeds, seeds=seeds["seeds"] + [
{"address": "0x%08X" % pc, "label": label, "rationale": "test"}
for pc, label in VARIANTS.items()])
var_seeds = os.path.join(tmp, "variants.json")
with open(var_seeds, "w") as f:
json.dump(var, f)
r, full, disp = emit(args.bios_recompiler, profile(tmp, var_seeds),
os.path.join(tmp, "var"))
check(full is not None, "OpenBIOS with variant seeds failed:\n" + r.stderr[-2000:])
for pc, label in VARIANTS.items():
check("segment-variant seed 0x%08X (%s)" % (pc, label) in r.stdout,
"the variant seed 0x%08X is reported" % pc)
# -- the home output only gains lines -------------------------------
def only_additions(old, new):
it = iter(new.splitlines())
return all(any(line == n for n in it) for line in old.splitlines())
check(only_additions(base_full, full), "the variant build changes no home body")
check(only_additions(base_disp, disp), "the variant build changes no home dispatch line")
# -- closure and segment of every baked PC ---------------------------
names = set(re.findall(r"^void (OpenBIOS_func_[0-9A-F]{8})\(CPUState\* cpu\) \{",
full, re.M))
want = {"OpenBIOS_func_A00012BC", "OpenBIOS_func_A00006F8", "OpenBIOS_func_A000070C",
"OpenBIOS_func_80000540", "OpenBIOS_func_A00013F4", "OpenBIOS_func_A0001424"}
check(want <= names, "variant bodies, 0x6F8 by jal, 0x70C by fall-through and 0x1424 "
"by a conditional branch: %s" % sorted(n for n in names if n[14] in "8A"))
check("cpu->pc = 0xA00006F8u; return;" in body(full, "OpenBIOS_func_A00012BC"),
"a KSEG1 variant's jal keeps the KSEG1 top bits")
pcs = r"(?:psx_icache_fetch\(cpu, |g_debug_last_store_pc = |psx_unaligned_access" \
r"\(cpu, psx_addr, |psx_check_interrupts_at\(cpu, |cpu->gpr\[\d+\] = |" \
r"cpu->pc = )0x([0-9A-F]{8})u"
for name in sorted(want):
seg = int(name[14:], 16) & 0xE0000000
text = body(full, name)
check(text, "no body for " + name)
baked = [int(v, 16) for v in re.findall(pcs, text)]
window = [v for v in baked if 0x500 <= (v & 0x1FFFFFFF) < 0x8000]
check(window and all((v & 0xE0000000) == seg for v in window),
"%s bakes only its segment's kernel PCs, got %s" %
(name, sorted({"%08X" % v for v in window if (v & 0xE0000000) != seg})))
insns = re.findall(r"/\* 0x([0-9A-F]{8}): [0-9A-F]{8} ", text)
fetches = re.findall(r"psx_icache_fetch\(cpu, 0x([0-9A-F]{8})u\)", text)
if seg == KSEG1:
check(len(fetches) == len(insns),
"%s charges a fetch per instruction (%d fetches, %d instructions)"
% (name, len(fetches), len(insns)))
else:
check(0 < len(fetches) < len(insns),
"%s keeps the cached line-leader rule" % name)
memset = body(full, "OpenBIOS_func_80000540")
home_memset = body(full, "OpenBIOS_func_00000540")
check(re.search(r"case 0x000005[0-9A-F]{2}u:", home_memset) and
"jump table" in home_memset,
"the home memset switches locally on its KUSEG table values")
check(not re.search(r"case 0x000005[0-9A-F]{2}u:", memset) and
re.search(r"cpu->pc = (?:psx_jrt_BFC1E574|cpu->gpr\[9\]); return;", memset),
"the KSEG0 memset variant dispatches the KUSEG case values")
# -- dispatch -------------------------------------------------------
m = re.search(r"segment_variant_table\[(\d+)\] = \{\n(.*?)\n\};", disp, re.S)
check(m, "the dispatch has a segment-variant table")
rows = [(int(a, 16), sym) for a, sym in
re.findall(r"\{ 0x([0-9A-F]{8})u, (\w+) \}", m.group(2))]
check(int(m.group(1)) == len(rows) and [a for a, _ in rows] == sorted(a for a, _ in rows),
"the variant rows are counted and sorted")
keys = {a: sym for a, sym in rows}
check(keys.get(0xA00012BC) == "OpenBIOS_func_A00012BC" and
keys.get(0xA00006F8) == "OpenBIOS_func_A00006F8" and
keys.get(0xA000070C) == "OpenBIOS_func_A000070C" and
keys.get(0x80000540) == "OpenBIOS_func_80000540",
"each variant entry is keyed by its exact runtime PC")
check(any(sym.startswith("OpenBIOS_func_A000070C_cont_") and (a & 0xE0000000) == KSEG1
for a, sym in rows),
"a variant's call returns are exact-PC continuation rows")
check("psx_bios_hit_body(cpu, addr, phys, dispatch_table[mid].func)(cpu);" in disp and
"psx_bios_hit_body(cpu, addr, phys, dispatch_table[mid].func)(cpu);" in base_disp,
"a normalized hit runs the body psx_bios_hit_body picks for the exact PC")
got = run_lookup_harness(args.compiler, disp, full, tmp, "lookup_var")
# Every key has a home PC; the A0/B0/C0 vector handlers, when a
# profile emits them into the table, are the only keys in no window.
check(got[0] == len(rows) and got[1] > 1000 and got[3] > 1000 and got[4] in (0, 3),
"the variant lookup: %d of %d variant rows, %d home PCs, %d/%d aliases "
"recorded, %d call vectors" % (got[0], len(rows), got[1], got[3], got[2], got[4]))
base_got = run_lookup_harness(args.compiler, base_disp, base_full, tmp, "lookup_base")
check(base_got[0] == 0 and base_got[1] == got[1] and base_got[2] == base_got[3] ==
got[2] and got[3] == got[2] - len(rows),
"without variants every alias of a home key is a recorded BIOS segment miss; "
"with them, exactly the variant rows are not: base %s, variants %s"
% (base_got, got))
# -- a variant seed must name a function entry -----------------------
r, _, _ = emit_with(args.bios_recompiler, tmp, "bad", seeds, [(0xA0000504, "inside")])
check(r.returncode != 0 and "segment-variant seed 0xA0000504" in r.stderr,
"a variant seed inside a function body stops the build:\n" + r.stderr[-2000:])
# -- only KUSEG below 0x20000000, KSEG0 and KSEG1 map the window ------
for pc in (0x200012BC, 0xC00012BC):
r, _, _ = emit_with(args.bios_recompiler, tmp, "badseg", seeds, [(pc, "badseg")])
check(r.returncode != 0 and
("seed 0x%08X (badseg) is in segment 0x%08X, which does not map physical "
"memory" % (pc, pc & 0xE0000000)) in r.stderr and
"runs its bytes at 0x000012BC" in r.stderr,
"a seed in a segment that does not map physical memory stops the build:\n"
+ r.stderr[-2000:])
# -- a runtime PC in its window's own segment is no variant request ---
# It stays a discovery seed, and discovery takes ROM addresses only.
r, _, _ = emit_with(args.bios_recompiler, tmp, "same", seeds, [(0x000012BC, "same")])
check(r.returncode != 0 and "segment-variant seed 0x000012BC" not in r.stdout and
"seed address 0x000012BC (same) is outside ROM range" in r.stderr,
"a seed in its window's own segment stays a discovery seed:\n"
+ (r.stdout + r.stderr)[-2000:])
# -- SCPH-1001 enters its kernel at 0xA0000500 (static: no retail image) ---
with open(os.path.join(ROOT, "bios", "SCPH1001.toml")) as f:
m = re.search(r'^seeds\s*=\s*"([^"]+)"', f.read(), re.M)
with open(os.path.join(ROOT, m.group(1))) as f:
retail = json.load(f)
check(any(int(s["address"], 16) == 0xA0000500 for s in retail["seeds"]),
"SCPH-1001's seeds ask for the KSEG1 variant of its kernel entry 0xA0000500")
print("PASS: BIOS segment variants (closure, per-segment PCs, KSEG1 per-instruction "
"fetch, foreign jump-table cases dispatched, exact-PC dispatch rows compiled and "
"queried, aliases without a variant recorded, seeds outside the physical segments "
"refused, home output unchanged)")
return 0
if __name__ == "__main__":
sys.exit(main())
+24 -5
View File
@@ -2,7 +2,8 @@
#define PSX_SEGMENT_MISS_H
/* Segment misses in static game code (docs/SEGMENT_AWARE_CODE.md §5.5; §10
* decision 2: interpret loudly until the title is regenerated).
* decision 2: interpret loudly until the title is regenerated), and in the
* compiled BIOS (§5.4).
*
* A PC carries a segment (KUSEG 0x0..., KSEG0 0x8..., KSEG1 0xA...), and the
* game dispatch table is keyed by the full PC: each compiled body bakes the
@@ -27,7 +28,7 @@ typedef struct {
uint32_t ra;
uint32_t sp;
uint32_t frame;
uint32_t pad;
uint32_t kind; /* PSX_SEGMENT_MISS_GAME or PSX_SEGMENT_MISS_BIOS */
} PsxSegmentMissEntry;
#define PSX_SEGMENT_MISS_RING_CAP 4096u /* power of two */
@@ -39,10 +40,25 @@ typedef struct {
* psx_game_is_function_entry. */
uint32_t psx_segment_miss_home(uint32_t addr, int (*is_entry)(uint32_t));
/* Record one segment miss. */
/* What a segment miss ran. GAME: static game text, interpreted at its own
* PC (§5.5). BIOS: a relocated BIOS window (kernel, shell) or the ROM entered
* in another segment than the one its body was compiled for, with no segment
* variant of its own. The BIOS dispatch is keyed by the normalized address,
* so the window's home body ran, with its own segment's links, EPCs, fetch
* tags and store PCs; the fix is a segment-qualified seed in the BIOS seeds
* file (§5.4), not in the game's. */
#define PSX_SEGMENT_MISS_GAME 0u
#define PSX_SEGMENT_MISS_BIOS 1u
/* Record one segment miss of static game code (kind GAME). */
void psx_segment_miss_record(uint32_t addr, uint32_t home, uint32_t ra,
uint32_t sp, uint32_t frame);
/* Record one segment miss of either kind. A PC keeps the kind it was first
* recorded with in the per-PC counts. */
void psx_segment_miss_record_kind(uint32_t addr, uint32_t home, uint32_t ra,
uint32_t sp, uint32_t frame, uint32_t kind);
/* The dispatch hook: dirty_ram_dispatch_inner calls this for every clean
* game-text miss (a PC in the EXE's text that no compiled body ran). It
* records a segment miss when addr has no row of its own but another
@@ -60,9 +76,12 @@ PsxSegmentMissEntry psx_segment_miss_get(uint64_t seq);
/* Per-PC counts, highest first: up to max rows, returns the number written.
* Counts are exact for the first UNIQUE_CAP distinct PCs; later new PCs are
* counted in the total and the ring only. */
* counted in the total and the ring only. Any output array may be NULL. */
uint32_t psx_segment_miss_summary(uint32_t *addrs, uint32_t *homes,
uint64_t *counts, uint32_t max);
uint64_t *counts, uint32_t *kinds, uint32_t max);
/* "game" or "bios", for reports. */
const char *psx_segment_miss_kind_name(uint32_t kind);
void psx_segment_miss_reset(void);
+10 -6
View File
@@ -778,13 +778,15 @@ void psx_crash_trace_dump(const char *reason, void *seh_info) {
}
/* segment misses (docs/SEGMENT_AWARE_CODE.md §5.5): static game text
* entered at a PC whose compiled row is in another segment; each ran
* interpreted. Top 50 PCs by count. */
* entered at a PC whose compiled row is in another segment, each run
* interpreted (kind "game"), and compiled BIOS windows entered in another
* segment than their body's, each run through the home body (kind
* "bios", §5.4). Top 50 PCs by count. */
{
enum { kRows = 50 };
uint32_t addrs[kRows], homes[kRows];
uint32_t addrs[kRows], homes[kRows], kinds[kRows];
uint64_t counts[kRows];
uint32_t n = psx_segment_miss_summary(addrs, homes, counts, kRows);
uint32_t n = psx_segment_miss_summary(addrs, homes, counts, kinds, kRows);
append_fmt(buf, sizeof(buf), &pos,
" \"segment_misses\": {\n"
" \"total\": %llu,\n"
@@ -793,8 +795,10 @@ void psx_crash_trace_dump(const char *reason, void *seh_info) {
(unsigned long long)psx_segment_miss_total(), psx_segment_miss_unique());
for (uint32_t i = 0; i < n; i++) {
append_fmt(buf, sizeof(buf), &pos,
"%s{\"pc\":\"0x%08X\",\"home\":\"0x%08X\",\"count\":%llu}",
i == 0 ? "" : ",", addrs[i], homes[i], (unsigned long long)counts[i]);
"%s{\"pc\":\"0x%08X\",\"home\":\"0x%08X\",\"count\":%llu,"
"\"kind\":\"%s\"}",
i == 0 ? "" : ",", addrs[i], homes[i], (unsigned long long)counts[i],
psx_segment_miss_kind_name(kinds[i]));
}
append_str(buf, sizeof(buf), &pos, "]\n },\n");
}
+17 -11
View File
@@ -11701,10 +11701,13 @@ static void handle_dispatch_stats(int id, const char *json)
}
/* segment_misses: static game text entered at a PC whose physical word has a
* compiled row only in another segment (docs/SEGMENT_AWARE_CODE.md §5.5).
* Each ran interpreted; the fix is a segment-qualified seed (§5.4) and a
* regeneration. {"tail":N} returns the last N ring entries in order;
* otherwise a per-PC summary, highest count first (at most 200 rows). */
* compiled row only in another segment (docs/SEGMENT_AWARE_CODE.md §5.5),
* kind "game": each ran interpreted. Kind "bios": a compiled BIOS window
* entered in another segment than its body's, with no segment variant; the
* home body ran (§5.4). The fix is a segment-qualified seed (§5.4) and a
* regeneration, in the game's or the BIOS's seeds by kind. {"tail":N}
* returns the last N ring entries in order; otherwise a per-PC summary,
* highest count first (at most 200 rows). */
static void handle_segment_misses(int id, const char *json)
{
int tail = json_get_int(json, "tail", 0);
@@ -11724,20 +11727,23 @@ static void handle_segment_misses(int id, const char *json)
PsxSegmentMissEntry e = psx_segment_miss_get(total - (uint64_t)tail + (uint64_t)i);
pos += snprintf(out + pos, BUF_SZ - pos,
"%s{\"seq\":%llu,\"pc\":\"0x%08X\",\"home\":\"0x%08X\","
"\"ra\":\"0x%08X\",\"sp\":\"0x%08X\",\"frame\":%u}",
"\"ra\":\"0x%08X\",\"sp\":\"0x%08X\",\"frame\":%u,"
"\"kind\":\"%s\"}",
i ? "," : "", (unsigned long long)e.seq, e.addr, e.home,
e.ra, e.sp, e.frame);
e.ra, e.sp, e.frame, psx_segment_miss_kind_name(e.kind));
}
} else {
enum { kRows = 200 };
uint32_t addrs[kRows], homes[kRows];
uint32_t addrs[kRows], homes[kRows], kinds[kRows];
uint64_t counts[kRows];
uint32_t n = psx_segment_miss_summary(addrs, homes, counts, kRows);
uint32_t n = psx_segment_miss_summary(addrs, homes, counts, kinds, kRows);
pos += snprintf(out + pos, BUF_SZ - pos, "\"summary\":[");
for (uint32_t i = 0; i < n && pos < BUF_SZ - 96; i++) {
for (uint32_t i = 0; i < n && pos < BUF_SZ - 112; i++) {
pos += snprintf(out + pos, BUF_SZ - pos,
"%s{\"pc\":\"0x%08X\",\"home\":\"0x%08X\",\"count\":%llu}",
i ? "," : "", addrs[i], homes[i], (unsigned long long)counts[i]);
"%s{\"pc\":\"0x%08X\",\"home\":\"0x%08X\",\"count\":%llu,"
"\"kind\":\"%s\"}",
i ? "," : "", addrs[i], homes[i], (unsigned long long)counts[i],
psx_segment_miss_kind_name(kinds[i]));
}
}
pos += snprintf(out + pos, BUF_SZ - pos, "]}\n");
+18 -3
View File
@@ -1,5 +1,6 @@
/* psx_segment_miss.c — segment misses in static game code
* (docs/SEGMENT_AWARE_CODE.md §5.5). See psx_segment_miss.h.
* (docs/SEGMENT_AWARE_CODE.md §5.5) and in the compiled BIOS (§5.4). See
* psx_segment_miss.h.
*
* Always on, in every build: a segment miss runs interpreted, so it is the
* slow path already, and the record is what makes it loud (§10 decision 2).
@@ -18,6 +19,7 @@ static uint64_t s_seq = 0;
typedef struct {
uint32_t addr; /* 0 = empty slot (a segment miss is never at PC 0) */
uint32_t home;
uint32_t kind;
uint64_t count;
} UniqueSlot;
static PSX_BSS UniqueSlot s_unique[PSX_SEGMENT_MISS_UNIQUE_CAP];
@@ -37,6 +39,12 @@ uint32_t psx_segment_miss_home(uint32_t addr, int (*is_entry)(uint32_t))
void psx_segment_miss_record(uint32_t addr, uint32_t home, uint32_t ra,
uint32_t sp, uint32_t frame)
{
psx_segment_miss_record_kind(addr, home, ra, sp, frame, PSX_SEGMENT_MISS_GAME);
}
void psx_segment_miss_record_kind(uint32_t addr, uint32_t home, uint32_t ra,
uint32_t sp, uint32_t frame, uint32_t kind)
{
const uint64_t seq = s_seq++;
PsxSegmentMissEntry *e = &s_ring[seq & (PSX_SEGMENT_MISS_RING_CAP - 1u)];
@@ -46,7 +54,7 @@ void psx_segment_miss_record(uint32_t addr, uint32_t home, uint32_t ra,
e->ra = ra;
e->sp = sp;
e->frame = frame;
e->pad = 0;
e->kind = kind;
if (addr == 0) return;
{
const uint32_t start = (addr >> 2) % PSX_SEGMENT_MISS_UNIQUE_CAP;
@@ -59,6 +67,7 @@ void psx_segment_miss_record(uint32_t addr, uint32_t home, uint32_t ra,
if (slot->addr == 0) {
slot->addr = addr;
slot->home = home;
slot->kind = kind;
slot->count = 1;
s_unique_count++;
return;
@@ -85,8 +94,13 @@ PsxSegmentMissEntry psx_segment_miss_get(uint64_t seq)
return s_ring[seq & (PSX_SEGMENT_MISS_RING_CAP - 1u)];
}
const char *psx_segment_miss_kind_name(uint32_t kind)
{
return kind == PSX_SEGMENT_MISS_BIOS ? "bios" : "game";
}
uint32_t psx_segment_miss_summary(uint32_t *addrs, uint32_t *homes,
uint64_t *counts, uint32_t max)
uint64_t *counts, uint32_t *kinds, uint32_t max)
{
/* Selection by count, highest first, then by PC; the table is small. */
uint32_t written = 0;
@@ -109,6 +123,7 @@ uint32_t psx_segment_miss_summary(uint32_t *addrs, uint32_t *homes,
if (addrs) addrs[written] = s_unique[best].addr;
if (homes) homes[written] = s_unique[best].home;
if (counts) counts[written] = s_unique[best].count;
if (kinds) kinds[written] = s_unique[best].kind;
prev_count = s_unique[best].count;
prev_addr = s_unique[best].addr;
written++;
+28 -6
View File
@@ -13,6 +13,7 @@
#include "psx_segment_miss.h"
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
static int failures = 0;
@@ -69,17 +70,38 @@ int main(void)
{
PsxSegmentMissEntry e = psx_segment_miss_get(3);
CHECK(e.seq == 3 && e.addr == 0x800100E4u && e.home == 0x000100E4u &&
e.ra == 0x00010018u && e.frame == 20u, "the ring keeps the full PC in order");
e.ra == 0x00010018u && e.frame == 20u && e.kind == PSX_SEGMENT_MISS_GAME,
"the ring keeps the full PC in order, kind game");
}
{
uint32_t addrs[4], homes[4];
uint32_t addrs[4], homes[4], kinds[4];
uint64_t counts[4];
uint32_t n = psx_segment_miss_summary(addrs, homes, counts, 4);
uint32_t n = psx_segment_miss_summary(addrs, homes, counts, kinds, 4);
CHECK(n == 2, "summary has one row per PC");
CHECK(addrs[0] == 0xA00100F0u && homes[0] == 0x000100F0u && counts[0] == 3,
CHECK(addrs[0] == 0xA00100F0u && homes[0] == 0x000100F0u && counts[0] == 3 &&
kinds[0] == PSX_SEGMENT_MISS_GAME,
"summary is highest count first");
CHECK(addrs[1] == 0x800100E4u && counts[1] == 1, "summary second row");
CHECK(psx_segment_miss_summary(addrs, homes, counts, 1) == 1, "summary honours max");
CHECK(psx_segment_miss_summary(addrs, homes, counts, NULL, 1) == 1,
"summary honours max");
}
/* A BIOS segment miss (the compiled BIOS ran a window's home body for an
* alias PC, §5.4) keeps its kind in the ring and in the summary. */
psx_segment_miss_record_kind(0x80000500u, 0x00000500u, 0xBFC06F0Cu, 0x801FFF00u, 3u,
PSX_SEGMENT_MISS_BIOS);
{
PsxSegmentMissEntry e = psx_segment_miss_get(4);
uint32_t addrs[4], kinds[4];
uint32_t n = psx_segment_miss_summary(addrs, NULL, NULL, kinds, 4);
CHECK(e.addr == 0x80000500u && e.home == 0x00000500u && e.frame == 3u &&
e.kind == PSX_SEGMENT_MISS_BIOS, "a BIOS segment miss is recorded as kind bios");
/* Counts 3, 1, 1; the tie orders by PC. */
CHECK(n == 3 && addrs[1] == 0x80000500u && kinds[1] == PSX_SEGMENT_MISS_BIOS &&
kinds[0] == PSX_SEGMENT_MISS_GAME && kinds[2] == PSX_SEGMENT_MISS_GAME,
"the summary carries each PC's kind");
CHECK(strcmp(psx_segment_miss_kind_name(PSX_SEGMENT_MISS_BIOS), "bios") == 0 &&
strcmp(psx_segment_miss_kind_name(PSX_SEGMENT_MISS_GAME), "game") == 0,
"kind names");
}
/* Equal counts order by PC, and the summary visits each PC once. */
psx_segment_miss_reset();
@@ -87,7 +109,7 @@ int main(void)
psx_segment_miss_record(0x80010300u, 0x00010300u, 0, 0, 0);
{
uint32_t addrs[4];
uint32_t n = psx_segment_miss_summary(addrs, NULL, NULL, 4);
uint32_t n = psx_segment_miss_summary(addrs, NULL, NULL, NULL, 4);
CHECK(n == 2 && addrs[0] == 0x80010300u && addrs[1] == 0xA0010300u,
"ties order by PC");
}
+12 -3
View File
@@ -10,7 +10,9 @@ psx_segment_miss_note included). What a unit test cannot see is the wiring
inside the runtime, which only runs in a full build: deleting the dispatch
hook, the TCP command or a report field would leave every other test green
and make segment misses silent again. This guard pins that wiring and the
`segment_misses` summary row shape.
`segment_misses` row shape, whose `kind` tells a game segment miss from a
BIOS one (the compiled BIOS ran a window's home body for another segment's
alias PC; bios_segment_variants compiles and queries that emitted check).
"""
from pathlib import Path
import re
@@ -56,10 +58,15 @@ def main():
assert re.search(r'\{\s*"segment_misses",\s*handle_segment_misses\s*\}', DEBUG), \
'TCP segment_misses is not registered'
seg = body(DEBUG, 'static void handle_segment_misses(int id, const char *json)')
# The summary row shape (the seed tools read `pc`).
# The summary row shape (the seed tools read `pc` and `kind`: a "bios"
# row belongs in the BIOS seeds, not the game's).
assert '\\"summary\\":[' in seg and \
'{\\"pc\\":\\"0x%08X\\",\\"home\\":\\"0x%08X\\",\\"count\\":%llu}' in seg, \
('{\\"pc\\":\\"0x%08X\\",\\"home\\":\\"0x%08X\\",\\"count\\":%llu,"\n'
' "\\"kind\\":\\"%s\\"}"') in seg and \
'psx_segment_miss_kind_name(kinds[i])' in seg, \
'segment_misses summary rows changed shape'
assert '\\"kind\\":\\"%s\\"}' in seg and 'psx_segment_miss_kind_name(e.kind)' in seg, \
'segment_misses tail entries lost their kind'
stats = body(DEBUG, 'static void handle_dispatch_stats(int id, const char *json)')
assert 'segment_miss_total' in stats and 'psx_segment_miss_total()' in stats
assert 'segment_miss_unique' in stats and 'psx_segment_miss_unique()' in stats
@@ -71,6 +78,8 @@ def main():
dump = body(CRASH, 'void psx_crash_trace_dump(const char *reason, void *seh_info) {')
assert '\\"segment_misses\\"' in dump and 'psx_segment_miss_summary(' in dump, \
'the exit report lost its segment_misses section'
assert 'psx_segment_miss_kind_name(kinds[i])' in dump, \
'the exit report lost the segment-miss kind'
print('segment-miss wiring guards passed')
+9 -1
View File
@@ -10,7 +10,9 @@ Resurrection). The recompiler takes a seed in any other segment as a variant
request, not an entry: it compiles the home body's direct-edge closure again
in that segment (§5.4). Segment misses (TCP `segment_misses`: static text
entered in a segment with no body of its own) are written with their full PC,
so the next regeneration compiles a variant for each.
so the next regeneration compiles a variant for each. A `bios` segment miss
(a compiled BIOS window entered in another segment) is listed, not written:
its seed belongs in the BIOS profile's seeds.
The link segment comes from --segment, from --game-toml ([game].load_address,
or the header of its [game].exe), or from game_text_lo when that is written as
@@ -153,6 +155,12 @@ def main(argv=None):
if sm.get("ok"):
for row in sm.get("summary", []):
vaddr = int(row["pc"], 16)
if row.get("kind", "game") == "bios":
# The compiled BIOS ran a window's home body for this PC: its
# seed belongs in the BIOS seeds file, not the game's.
print("note: BIOS segment miss 0x%08X (home %s, %s hits): seed it in the "
"BIOS profile's seeds, not here" % (vaddr, row["home"], row["count"]))
continue
if not maps_physical(vaddr):
print("warning: segment miss 0x%08X is in a segment that does not map "
"physical memory; not a seed (the recompiler refuses it)" % vaddr)
+13 -5
View File
@@ -10,7 +10,8 @@ takes the segment from --segment, --game-toml or a segmented game_text_lo,
refuses to guess from a physical one, and writes `segment_misses` rows with
their full PC. A row in a segment that does not map physical memory
(0x20000000-0x7FFFFFFF, KSEG2) is reported and not written: the recompiler
refuses such a seed.
refuses such a seed. A `bios` row (the compiled BIOS ran a window's home
body for an alias PC) is listed for the BIOS seeds and not written.
A fake debug server answers the two commands the tool sends. Its
`segment_misses` rows use the format string of the runtime's handler, read
@@ -29,16 +30,19 @@ import threading
HERE = os.path.dirname(os.path.abspath(__file__))
ROOT = os.path.normpath(os.path.join(HERE, "..", ".."))
TOOL = os.path.join(ROOT, "tools", "collect_game_misses.py")
ROW_FMT = '{\\"pc\\":\\"0x%08X\\",\\"home\\":\\"0x%08X\\",\\"count\\":%llu}'
ROW_FMT = ('{\\"pc\\":\\"0x%08X\\",\\"home\\":\\"0x%08X\\",\\"count\\":%llu,"\n'
' "\\"kind\\":\\"%s\\"}"')
PER_PC = [ # physical, as dirty_ram_stats keys them
{"pc": "0x000100E4", "hits": 9, "insns": 40},
{"pc": "0x00010200", "hits": 3, "insns": 12},
{"pc": "0x000C0000", "hits": 5, "insns": 7}, # outside the text range
]
SEGMENT_MISSES = [(0x800100F0, 0x000100F0, 4), (0xA0010124, 0x00010124, 1),
SEGMENT_MISSES = [(0x800100F0, 0x000100F0, 4, "game"), (0xA0010124, 0x00010124, 1, "game"),
# 0x20000000-0x7FFFFFFF does not map physical memory: no seed.
(0x200100F0, 0x000100F0, 1)]
(0x200100F0, 0x000100F0, 1, "game"),
# A BIOS shell alias inside the text range: a BIOS seed, not a game one.
(0x00030000, 0x80030000, 2, "bios")]
class FakeDebugServer(threading.Thread):
@@ -56,7 +60,8 @@ class FakeDebugServer(threading.Thread):
return json.dumps({"id": 1, "ok": True, "per_pc": PER_PC})
if cmd == "segment_misses" and self.with_segment_misses:
# The runtime's own row format (C printf -> Python %).
rows = ",".join(ROW_FMT.replace('\\"', '"').replace("%llu", "%d") % r
fmt = "".join(p.strip().strip('"') for p in ROW_FMT.split("\n"))
rows = ",".join(fmt.replace('\\"', '"').replace("%llu", "%d") % r
for r in SEGMENT_MISSES)
return '{"id":1,"ok":true,"total":5,"unique":2,"summary":[%s]}' % rows
return json.dumps({"id": 1, "ok": False, "error": "unknown command"})
@@ -122,6 +127,9 @@ def main():
check("segment miss 0x200100F0 is in a segment that does not map" in r.stdout,
"a segment miss outside the physical segments is reported, not "
"written: %r" % r.stdout)
check("BIOS segment miss 0x00030000 (home 0x80030000, 2 hits)" in r.stdout,
"a BIOS segment miss is listed for the BIOS seeds, not written: %r"
% r.stdout)
# A second run appends nothing: known seeds are exact PCs.
r = run_tool(server.port, out, "--segment", "kuseg")
check(r.returncode == 0 and "No new addresses" in r.stdout and