mirror of
https://github.com/RetroPortingToolKit/psxrecomp
synced 2026-10-26 10:19:47 -04:00
recompiler: compile segment variants of game code from segment-qualified seeds
A seed that names the image's bytes in another segment than its link segment (0xA00100F0 for a KUSEG-linked EXE, 0x00012345 for a KSEG0 one) now compiles a segment variant (docs/SEGMENT_AWARE_CODE.md §5.4): the home functions the seed reaches through direct edges (calls, tail jumps, split pieces, fall-throughs, call returns; a seed at a CPS continuation roots its owner, an alias entry its alias group) are compiled again through a view of the image in that segment. The variant's names, rows and every PC it bakes are that segment's, and a KSEG1 variant charges a fetch per instruction (#429's rule through B's runtime_pc()). A cached variant is the home C with the segment swapped. Exact-match config sites move into the variant's segment, so a variant gets every hook the home body gets. A seed that names no home dispatch entry stops the build, and so does a seed in a segment that does not map physical memory (0x20000000-0x7FFFFFFF or KSEG2: Beetle's addr_mask leaves them unmasked, so they address no RAM and are no alias of the image's bytes). tools/collect_game_misses.py reports such a segment miss instead of writing it as a seed. The dispatch table holds a row per compiled segment of a word, adjacent and in PC order; when any word has more than one row the lookup (indexed and binary search) searches the word's rows for the exact PC. Two rows for one PC remain a build error. A title without variants gets the same dispatch source as before. A jump-table case whose value is in another segment than its body is no longer a local case: the default tail-transfer dispatches it with its full PC. This applies to every compile. In a variant it is the common case (a KSEG0 variant of KUSEG-linked code). A home or overlay compile has one only when its table holds another segment's PCs of its own bytes, for example a KSEG0-compiled overlay shard of code that runs at KUSEG. Such a case used to continue in the KSEG0 body with KSEG0 links and fetch tags. Now it dispatches: a game-text PC with no row in its segment is a recorded segment miss, and #417's gate sends an overlay PC to the interpreter; both run it at its own PC (PR E returns overlay code to native per-segment shards). R4's game C and its 26 overlay shard sources are unchanged. Ledger: segment-variants and kseg1-fetch-charge close (KNOWN_GAPS is empty) and become regression guards; resume-pc-segment now requires each row's key and resume PC to be in its body's segment, and segment-variants checks every PC a variant body bakes. New segment_variants_test; kuseg_dispatch_lookup and emitter_runtime_pc_test cover variant rows, closure, equality with the home C, per-instruction KSEG1 charging, foreign jump-table cases at home and in variants, data stubs (no row), alias groups declared for every shard and seeds outside the physical segments. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Shokunin
@@ -110,6 +110,7 @@ set(PSXRECOMP_GAME_SOURCES
|
||||
src/config_loader.cpp
|
||||
src/recompiler_patch.cpp
|
||||
src/game_dispatch_emitter.cpp
|
||||
src/segment_variants.cpp
|
||||
src/main_psx.cpp
|
||||
)
|
||||
|
||||
@@ -626,6 +627,35 @@ if(BUILD_TESTING)
|
||||
set_tests_properties(emitter_runtime_pc_test_legacy PROPERTIES
|
||||
ENVIRONMENT "PSX_CPS=0")
|
||||
|
||||
# Segment variants (docs/SEGMENT_AWARE_CODE.md §5.4): the planner's
|
||||
# direct-edge closure per segment, the segment view and the variant's
|
||||
# codegen config.
|
||||
add_executable(segment_variants_test
|
||||
tests/segment_variants_test.cpp
|
||||
src/segment_variants.cpp
|
||||
src/ps1_exe_parser.cpp
|
||||
src/mips_decoder.cpp
|
||||
src/control_flow.cpp
|
||||
src/function_analysis.cpp
|
||||
src/code_generator.cpp
|
||||
src/pgxp_hook_emitter.cpp
|
||||
src/annotations.cpp
|
||||
src/bios_address_model.cpp
|
||||
src/config_loader.cpp
|
||||
src/recompiler_patch.cpp
|
||||
)
|
||||
target_include_directories(segment_variants_test PRIVATE
|
||||
include
|
||||
src
|
||||
lib/rabbitizer/include
|
||||
lib/rabbitizer/cplusplus/include
|
||||
lib/fmt/include
|
||||
lib/toml11
|
||||
lib/ELFIO
|
||||
)
|
||||
target_link_libraries(segment_variants_test PRIVATE fmt rabbitizer)
|
||||
add_test(NAME segment_variants_test COMMAND segment_variants_test)
|
||||
|
||||
add_executable(recompiler_patch_cxx17_header_test
|
||||
tests/recompiler_patch_cxx17_header_test.cpp
|
||||
)
|
||||
|
||||
@@ -308,7 +308,14 @@ public:
|
||||
// and forward declarations for every recompiled function plus every
|
||||
// overlapping-alias shared body. Byte-for-byte the same declaration set
|
||||
// the monolith emits inline, just hoisted into a header.
|
||||
std::string build_shared_decls_header(const std::vector<GeneratedFunction>& gen_funcs) const;
|
||||
// `extra_decls` adds the alias-body prototypes of other generators whose
|
||||
// functions share these shards (segment variants, §5.4).
|
||||
std::string build_shared_decls_header(const std::vector<GeneratedFunction>& gen_funcs,
|
||||
const std::vector<std::string>& extra_decls = {}) const;
|
||||
|
||||
// Prototypes of the psx_alias_body_XXXXXXXX() bodies the most recent
|
||||
// generate_all_functions() emitted.
|
||||
const std::vector<std::string>& alias_body_decls() const { return alias_body_decls_; }
|
||||
|
||||
// Generate the per-function code-range manifest consumed by the overlay
|
||||
// loader's per-entry validity hash (design §8). For each function, emits its
|
||||
@@ -325,6 +332,12 @@ public:
|
||||
// generate_file() call, including functions synthesized by its split pass.
|
||||
const std::string& last_ranges_manifest() const { return last_ranges_manifest_; }
|
||||
|
||||
// The final function set and CFGs the most recent generate_file() call
|
||||
// emitted, after its split pass: what a segment variant's direct-edge
|
||||
// closure is planned over (docs/SEGMENT_AWARE_CODE.md §5.4).
|
||||
const std::vector<Function>& last_functions() const { return last_functions_; }
|
||||
const std::map<uint32_t, ControlFlowGraph>& last_cfgs() const { return last_cfgs_; }
|
||||
|
||||
// Set known functions for this compilation unit (for linking)
|
||||
void set_known_functions(const std::set<uint32_t>& functions) {
|
||||
known_functions_ = functions;
|
||||
@@ -352,6 +365,15 @@ public:
|
||||
uint32_t runtime_pc(uint32_t compile_addr) const {
|
||||
return code_seg_ | (compile_addr & kPhysMask);
|
||||
}
|
||||
// A jump-table case whose value (the PC the jr reaches) names the same
|
||||
// bytes as `image_addr` in another segment than this body's. Such a case
|
||||
// is not a local edge: the body bakes its own segment's PCs, so the value
|
||||
// is dispatched with its full PC instead (§5.4). Never true when the table
|
||||
// holds this body's own segment, which is every title's home compile.
|
||||
bool foreign_segment_case(uint32_t case_value, uint32_t image_addr) const {
|
||||
return (case_value & kPhysMask) == (image_addr & kPhysMask) &&
|
||||
case_value != runtime_pc(image_addr);
|
||||
}
|
||||
|
||||
private:
|
||||
const PS1Executable& exe_;
|
||||
@@ -359,6 +381,8 @@ private:
|
||||
uint32_t code_seg_; // set from the image in the constructor; see set_code_segment()
|
||||
std::set<uint32_t> known_functions_; // Addresses of functions in this compilation unit
|
||||
std::string last_ranges_manifest_;
|
||||
std::vector<Function> last_functions_;
|
||||
std::map<uint32_t, ControlFlowGraph> last_cfgs_;
|
||||
std::set<uint32_t> extra_labels_; // Mid-block addresses that need inline labels (jump table targets)
|
||||
const AnnotationTable* annotations_ = nullptr;
|
||||
|
||||
|
||||
@@ -18,6 +18,16 @@ inline constexpr uint32_t kSegKUSEG = 0x00000000u;
|
||||
inline constexpr uint32_t kSegKSEG0 = 0x80000000u;
|
||||
inline constexpr uint32_t kSegKSEG1 = 0xA0000000u;
|
||||
|
||||
// True when `pc` lies in one of the three windows that map the physical
|
||||
// address space: KUSEG below 0x20000000, KSEG0 and KSEG1. The rest of KUSEG
|
||||
// (0x20000000-0x7FFFFFFF) and KSEG2 are not mirrors of it: Beetle's addr_mask
|
||||
// leaves them unmasked, so they address no RAM or BIOS ROM, and a PC there
|
||||
// never names the bytes at `pc & kPhysMask`.
|
||||
inline constexpr bool maps_physical(uint32_t pc) {
|
||||
return pc < 0x20000000u || (pc & kSegmentMask) == kSegKSEG0 ||
|
||||
(pc & kSegmentMask) == kSegKSEG1;
|
||||
}
|
||||
|
||||
// PS-X EXE header structure (2048 bytes)
|
||||
#pragma pack(push, 1)
|
||||
struct PS1ExeHeader {
|
||||
|
||||
@@ -2449,6 +2449,13 @@ std::string CodeGenerator::translate_basic_block(
|
||||
std::vector<std::pair<uint32_t,uint32_t>> targets; // {runtime_addr, rom_addr}
|
||||
for (const auto& [runtime_addr, rom_addr] :
|
||||
exact_table.targets) {
|
||||
// A case value is the PC the jr reaches. A value
|
||||
// in another segment than this body's reaches the
|
||||
// same bytes as another code identity (§5.4): it
|
||||
// is left to the default tail-transfer, which
|
||||
// dispatches it with its full PC, instead of
|
||||
// continuing here with this segment's PCs.
|
||||
if (foreign_segment_case(runtime_addr, rom_addr)) continue;
|
||||
if (seen.insert(rom_addr).second &&
|
||||
(cfg.blocks.count(rom_addr) || extra_labels_.count(rom_addr))) {
|
||||
targets.push_back({runtime_addr, rom_addr});
|
||||
@@ -3042,6 +3049,9 @@ void CodeGenerator::scan_jr_tables(
|
||||
if (tb==0 || tc==0) continue;
|
||||
for (const auto& target_pair : exact_table.targets) {
|
||||
uint32_t t = target_pair.second;
|
||||
// Not a local edge: dispatched with its own segment (see the
|
||||
// switch emission in translate_basic_block).
|
||||
if (foreign_segment_case(target_pair.first, t)) continue;
|
||||
if (t >= cfg.function_start && t < cfg.function_end) {
|
||||
out_edges[baddr].push_back(t);
|
||||
if (!cfg.blocks.count(t))
|
||||
@@ -3537,7 +3547,8 @@ void CodeGenerator::emit_unaligned_helpers(std::ostream& ss, bool as_inline) con
|
||||
ss << "\n";
|
||||
}
|
||||
|
||||
std::string CodeGenerator::build_shared_decls_header(const std::vector<GeneratedFunction>& gen_funcs) const {
|
||||
std::string CodeGenerator::build_shared_decls_header(const std::vector<GeneratedFunction>& gen_funcs,
|
||||
const std::vector<std::string>& extra_decls) const {
|
||||
std::stringstream h;
|
||||
h << "/* Generated by PSXRecomp - shared declarations for split full.c shards. DO NOT EDIT. */\n";
|
||||
h << "#pragma once\n";
|
||||
@@ -3548,6 +3559,7 @@ std::string CodeGenerator::build_shared_decls_header(const std::vector<Generated
|
||||
h << "/* Forward declarations for all recompiled functions */\n";
|
||||
for (const auto& gf : gen_funcs) h << gf.signature << ";\n";
|
||||
for (const auto& decl : alias_body_decls_) h << decl << ";\n";
|
||||
for (const auto& decl : extra_decls) h << decl << ";\n";
|
||||
h << "\n";
|
||||
}
|
||||
return h.str();
|
||||
@@ -3801,6 +3813,8 @@ std::string CodeGenerator::generate_file(
|
||||
// dispatch generator consumes this so synthesized fallthrough functions
|
||||
// receive the same byte-precise native-validity ranges as ordinary entries.
|
||||
last_ranges_manifest_ = generate_ranges_manifest(functions_mut, cfgs_mut);
|
||||
last_functions_ = std::move(functions_mut);
|
||||
last_cfgs_ = std::move(cfgs_mut);
|
||||
|
||||
return ss.str();
|
||||
}
|
||||
|
||||
@@ -16,6 +16,20 @@ bool emit_game_dispatch(const CodeGenerator& codegen,
|
||||
const std::string& ranges_manifest,
|
||||
std::string& out,
|
||||
std::string& error) {
|
||||
return emit_game_dispatch(std::vector<GameDispatchUnit>{{&codegen, dispatch_addrs}},
|
||||
exe, ranges_manifest, out, error);
|
||||
}
|
||||
|
||||
bool emit_game_dispatch(const std::vector<GameDispatchUnit>& units,
|
||||
const PS1Executable& exe,
|
||||
const std::string& ranges_manifest,
|
||||
std::string& out,
|
||||
std::string& error) {
|
||||
if (units.empty() || !units.front().codegen) {
|
||||
error = "emit_game_dispatch: no compile to dispatch";
|
||||
return false;
|
||||
}
|
||||
const bool cps = units.front().codegen->cps_enabled();
|
||||
std::ostringstream ds;
|
||||
ds << "/* Generated by PSXRecomp - dynamic dispatch table */\n";
|
||||
ds << "#include \"psx_runtime.h\"\n\n";
|
||||
@@ -25,8 +39,10 @@ bool emit_game_dispatch(const CodeGenerator& codegen,
|
||||
|
||||
// Forward declarations
|
||||
ds << "/* Forward declarations for all recompiled functions */\n";
|
||||
for (uint32_t addr : dispatch_addrs) {
|
||||
ds << fmt::format("extern void func_{:08X}(CPUState* cpu);\n", addr);
|
||||
for (const auto& unit : units) {
|
||||
for (uint32_t addr : unit.dispatch_addrs) {
|
||||
ds << fmt::format("extern void func_{:08X}(CPUState* cpu);\n", addr);
|
||||
}
|
||||
}
|
||||
ds << "\n";
|
||||
|
||||
@@ -46,45 +62,57 @@ bool emit_game_dispatch(const CodeGenerator& codegen,
|
||||
// GCC lowers X4's roughly 60K cases to a multi-megabyte linear compare
|
||||
// chain. Binary search stays O(log N) at every optimizer level and
|
||||
// avoids emitting the same case set twice.
|
||||
// `key` and `resume` are the PCs the guest executes, routed through the
|
||||
// unit's runtime_pc() (docs/SEGMENT_AWARE_CODE.md §5.2); `addr` and
|
||||
// `owner` are compile identities (func_ names, .ranges owners). resume 0
|
||||
// means "entry at the prologue", not a PC.
|
||||
struct DispatchRecord {
|
||||
uint32_t addr;
|
||||
uint32_t key;
|
||||
uint32_t resume;
|
||||
uint32_t owner;
|
||||
uint32_t range_index;
|
||||
uint32_t range_count;
|
||||
};
|
||||
std::vector<DispatchRecord> records;
|
||||
records.reserve(dispatch_addrs.size() +
|
||||
(codegen.cps_enabled() ? codegen.cps_continuations().size() : 0));
|
||||
for (uint32_t addr : dispatch_addrs)
|
||||
records.push_back({addr, 0, addr, 0, 0});
|
||||
if (codegen.cps_enabled()) {
|
||||
const auto& conts = codegen.cps_continuations();
|
||||
for (const auto& [cont, owner] : conts) {
|
||||
if (dispatch_addrs.count(cont)) continue;
|
||||
// A continuation owned by a fail-closed stub is no more
|
||||
// executable than the stub entry itself.
|
||||
if (!dispatch_addrs.count(owner)) continue;
|
||||
records.push_back({cont, cont, owner, 0, 0});
|
||||
for (const auto& unit : units) {
|
||||
const CodeGenerator& codegen = *unit.codegen;
|
||||
for (uint32_t addr : unit.dispatch_addrs)
|
||||
records.push_back({addr, codegen.runtime_pc(addr), 0, addr, 0, 0});
|
||||
if (cps) {
|
||||
const auto& conts = codegen.cps_continuations();
|
||||
for (const auto& [cont, owner] : conts) {
|
||||
if (unit.dispatch_addrs.count(cont)) continue;
|
||||
// A continuation owned by a fail-closed stub is no more
|
||||
// executable than the stub entry itself.
|
||||
if (!unit.dispatch_addrs.count(owner)) continue;
|
||||
records.push_back({cont, codegen.runtime_pc(cont), codegen.runtime_pc(cont),
|
||||
owner, 0, 0});
|
||||
}
|
||||
}
|
||||
}
|
||||
// Sort by PHYSICAL address: the lookup indexes (or binary-searches) the
|
||||
// physical word and then compares the full PC. One compile has one code
|
||||
// segment, so the physical order is the raw order and every physical word
|
||||
// has at most one row; a second row for a word would be a variant of
|
||||
// another segment (§5.4), which this table does not hold yet.
|
||||
// Sort by PHYSICAL word, then by the full PC: the lookup indexes (or
|
||||
// binary-searches) the physical word and then requires the exact PC. A
|
||||
// word has one row per segment compiled for it, adjacent: the home
|
||||
// compile's and those of its segment variants (§5.4). Within one compile
|
||||
// the code segment is fixed, so two rows on one word are the same PC.
|
||||
std::sort(records.begin(), records.end(),
|
||||
[](const DispatchRecord& a, const DispatchRecord& b) {
|
||||
return (a.addr & 0x1FFFFFFFu) < (b.addr & 0x1FFFFFFFu);
|
||||
const uint32_t pa = a.key & 0x1FFFFFFFu, pb = b.key & 0x1FFFFFFFu;
|
||||
return pa != pb ? pa < pb : a.key < b.key;
|
||||
});
|
||||
bool shared_words = false;
|
||||
for (size_t i = 1; i < records.size(); ++i) {
|
||||
if ((records[i].addr & 0x1FFFFFFFu) == (records[i - 1].addr & 0x1FFFFFFFu)) {
|
||||
if (records[i].key == records[i - 1].key) {
|
||||
error = fmt::format(
|
||||
"dispatch rows 0x{:08X} and 0x{:08X} share a physical word; one "
|
||||
"compile has one code segment (docs/SEGMENT_AWARE_CODE.md §5.5)",
|
||||
records[i - 1].addr, records[i].addr);
|
||||
"dispatch rows 0x{:08X} and 0x{:08X} are the same PC 0x{:08X}: they "
|
||||
"share a physical word in one code segment "
|
||||
"(docs/SEGMENT_AWARE_CODE.md §5.5)",
|
||||
records[i - 1].addr, records[i].addr, records[i].key);
|
||||
return false;
|
||||
}
|
||||
if ((records[i].key & 0x1FFFFFFFu) == (records[i - 1].key & 0x1FFFFFFFu))
|
||||
shared_words = true;
|
||||
}
|
||||
|
||||
// Attach the exact CFG instruction ranges from the manifest to every
|
||||
@@ -159,15 +187,10 @@ bool emit_game_dispatch(const CodeGenerator& codegen,
|
||||
ds << " PsxGameDispatchFn fn;\n";
|
||||
ds << "} PsxGameDispatchEntry;\n\n";
|
||||
ds << "static const PsxGameDispatchEntry k_psx_game_dispatch[] = {\n";
|
||||
// Row keys and resume PCs are the PCs the guest executes, so they go
|
||||
// through the emitter's runtime_pc() (docs/SEGMENT_AWARE_CODE.md
|
||||
// §5.2); func_ names stay the compile identity. resume 0 means "entry
|
||||
// at the prologue", not a PC.
|
||||
for (const auto& rec : records) {
|
||||
ds << fmt::format(" {{0x{:08X}u, 0x{:08X}u, {}u, {}u, func_{:08X}}},\n",
|
||||
codegen.runtime_pc(rec.addr),
|
||||
rec.resume ? codegen.runtime_pc(rec.resume) : 0u,
|
||||
rec.range_index, rec.range_count, rec.owner);
|
||||
rec.key, rec.resume, rec.range_index, rec.range_count,
|
||||
rec.owner);
|
||||
}
|
||||
ds << "};\n";
|
||||
ds << fmt::format("#define PSX_GAME_DISPATCH_COUNT {}u\n\n", records.size());
|
||||
@@ -177,16 +200,17 @@ bool emit_game_dispatch(const CodeGenerator& codegen,
|
||||
// callers below still check live instruction ranges on every dispatch.
|
||||
// Bound the indexed span to one PS1 RAM image; wider tables retain the
|
||||
// binary search. No mutable cache or flag.
|
||||
const uint32_t lookup_lo = records.empty() ? 0u : (records.front().addr & 0x1FFFFFFFu);
|
||||
const uint32_t lookup_hi = records.empty() ? 0u : (records.back().addr & 0x1FFFFFFFu);
|
||||
const uint32_t lookup_lo = records.empty() ? 0u : (records.front().key & 0x1FFFFFFFu);
|
||||
const uint32_t lookup_hi = records.empty() ? 0u : (records.back().key & 0x1FFFFFFFu);
|
||||
bool indexed_lookup = !records.empty() && lookup_hi - lookup_lo < 0x200000u;
|
||||
for (size_t i = 0; i < records.size(); ++i) {
|
||||
if (records[i].addr & 3u) indexed_lookup = false;
|
||||
if (records[i].key & 3u) indexed_lookup = false;
|
||||
}
|
||||
if (indexed_lookup) {
|
||||
// Each word indexes its FIRST row; a word's other segments follow it.
|
||||
std::vector<uint32_t> index((lookup_hi - lookup_lo) / 4u + 1u, 0u);
|
||||
for (size_t i = 0; i < records.size(); ++i)
|
||||
index[((records[i].addr & 0x1FFFFFFFu) - lookup_lo) / 4u] = (uint32_t)i + 1u;
|
||||
for (size_t i = records.size(); i-- > 0;)
|
||||
index[((records[i].key & 0x1FFFFFFFu) - lookup_lo) / 4u] = (uint32_t)i + 1u;
|
||||
ds << "/* Immutable physical-word index; zero denotes a dispatch miss. */\n";
|
||||
ds << "static const " << (records.size() <= 65535u ? "uint16_t" : "uint32_t")
|
||||
<< " k_psx_game_dispatch_index[] = {\n";
|
||||
@@ -206,7 +230,30 @@ bool emit_game_dispatch(const CodeGenerator& codegen,
|
||||
ds << " * interprets it and records it, and never runs this body for it. */\n";
|
||||
ds << "static const PsxGameDispatchEntry* psx_game_find_entry(uint32_t addr) {\n";
|
||||
ds << " const uint32_t want = addr & 0x1FFFFFFFu;\n";
|
||||
if (indexed_lookup) {
|
||||
if (shared_words) {
|
||||
// Segment variants (§5.4): a word may have a row per compiled
|
||||
// segment, adjacent and in PC order. Find the word's first row, then
|
||||
// the exact PC among its rows.
|
||||
if (indexed_lookup) {
|
||||
ds << fmt::format(" const uint32_t offset = want - 0x{:08X}u;\n", lookup_lo);
|
||||
ds << fmt::format(" if ((want & 3u) || offset > 0x{:X}u) return 0;\n", lookup_hi - lookup_lo);
|
||||
ds << " const uint32_t index = k_psx_game_dispatch_index[offset >> 2];\n";
|
||||
ds << " if (!index) return 0;\n";
|
||||
ds << " uint32_t row = index - 1u;\n";
|
||||
} else {
|
||||
ds << " uint32_t lo = 0, hi = PSX_GAME_DISPATCH_COUNT;\n";
|
||||
ds << " while (lo < hi) { /* first row whose word is >= want */\n";
|
||||
ds << " uint32_t mid = lo + (hi - lo) / 2;\n";
|
||||
ds << " if ((k_psx_game_dispatch[mid].addr & 0x1FFFFFFFu) < want) lo = mid + 1;\n";
|
||||
ds << " else hi = mid;\n";
|
||||
ds << " }\n";
|
||||
ds << " uint32_t row = lo;\n";
|
||||
}
|
||||
ds << " for (; row < PSX_GAME_DISPATCH_COUNT &&\n";
|
||||
ds << " (k_psx_game_dispatch[row].addr & 0x1FFFFFFFu) == want; ++row)\n";
|
||||
ds << " if (k_psx_game_dispatch[row].addr == addr) return &k_psx_game_dispatch[row];\n";
|
||||
ds << " return 0;\n";
|
||||
} else if (indexed_lookup) {
|
||||
ds << fmt::format(" const uint32_t offset = want - 0x{:08X}u;\n", lookup_lo);
|
||||
ds << fmt::format(" if ((want & 3u) || offset > 0x{:X}u) return 0;\n", lookup_hi - lookup_lo);
|
||||
ds << " const uint32_t index = k_psx_game_dispatch_index[offset >> 2];\n";
|
||||
@@ -248,7 +295,7 @@ bool emit_game_dispatch(const CodeGenerator& codegen,
|
||||
ds << " if (entry->range_count == 0 || !dirty_ram_text_native_ok_ranges_from(\n";
|
||||
ds << " &k_psx_game_code_ranges[entry->range_index].lo, entry->range_count, addr)) return 0;\n";
|
||||
ds << " psx_check_interrupts_dispatch_entry(cpu, addr);\n";
|
||||
if (codegen.cps_enabled())
|
||||
if (cps)
|
||||
ds << " cpu->pc = entry->resume_pc;\n";
|
||||
ds << " /* load_accel: cycle-faithful VSync(-1) query (config-gated). */\n";
|
||||
ds << " {\n";
|
||||
@@ -274,7 +321,7 @@ bool emit_game_dispatch(const CodeGenerator& codegen,
|
||||
ds << " return psx_game_find_entry(addr) != 0;\n";
|
||||
ds << "}\n";
|
||||
|
||||
if (codegen.cps_enabled()) {
|
||||
if (cps) {
|
||||
// RECURSION_BUG.md §25 — mark CPS mode at startup so runtime code that
|
||||
// routes CPS continuations (overlay_loader.c) sees the contract.
|
||||
// Static ctor: no clash with the BIOS dispatch's marker.
|
||||
|
||||
@@ -17,21 +17,39 @@
|
||||
#include <cstdint>
|
||||
#include <set>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "code_generator.h"
|
||||
#include "ps1_exe_parser.h"
|
||||
|
||||
namespace PSXRecomp {
|
||||
|
||||
// Emits the dispatch source for `dispatch_addrs` (compile addresses of the
|
||||
// dispatchable func_ entries) and the generator's CPS continuations. Row keys
|
||||
// and resume PCs go through codegen.runtime_pc() (§5.2); func_ names keep the
|
||||
// compile identity. `ranges_manifest` is the emitted .ranges text, whose F/R
|
||||
// records give each row its exact instruction ranges.
|
||||
// One compile whose bodies the table dispatches: the home compile, or a
|
||||
// segment variant (§5.4). `dispatch_addrs` are the compile addresses of its
|
||||
// dispatchable func_ entries; its CPS continuations come from the generator.
|
||||
struct GameDispatchUnit {
|
||||
const CodeGenerator* codegen;
|
||||
std::set<uint32_t> dispatch_addrs;
|
||||
};
|
||||
|
||||
// Emits the dispatch source for every unit's entries and CPS continuations.
|
||||
// Row keys and resume PCs go through the unit's runtime_pc() (§5.2); func_
|
||||
// names keep the compile identity. `ranges_manifest` is the emitted .ranges
|
||||
// text of all units, whose F/R records give each row its exact instruction
|
||||
// ranges. Rows are sorted by physical word; one word may carry a row per
|
||||
// compiled segment (the home body and its variants), and the lookup requires
|
||||
// the exact PC. The first unit's CPS mode applies to the table.
|
||||
//
|
||||
// Returns false with `error` set when two rows would share a physical word:
|
||||
// one compile has one code segment, so that can only be a caller error until
|
||||
// per-segment variants (§5.4) share the table.
|
||||
// Returns false with `error` set when two rows name the same PC. Within one
|
||||
// compile that means two rows on one physical word: one compile has one code
|
||||
// segment.
|
||||
bool emit_game_dispatch(const std::vector<GameDispatchUnit>& units,
|
||||
const PS1Executable& exe,
|
||||
const std::string& ranges_manifest,
|
||||
std::string& out,
|
||||
std::string& error);
|
||||
|
||||
// The single-compile form (no segment variants).
|
||||
bool emit_game_dispatch(const CodeGenerator& codegen,
|
||||
const PS1Executable& exe,
|
||||
const std::set<uint32_t>& dispatch_addrs,
|
||||
|
||||
+112
-21
@@ -5,6 +5,7 @@
|
||||
#include <fstream>
|
||||
#include <initializer_list>
|
||||
#include <map>
|
||||
#include <memory>
|
||||
#include <sstream>
|
||||
#include <string>
|
||||
#include <algorithm>
|
||||
@@ -17,6 +18,7 @@
|
||||
#include "control_flow.h"
|
||||
#include "code_generator.h"
|
||||
#include "game_dispatch_emitter.h"
|
||||
#include "segment_variants.h"
|
||||
#include "annotations.hpp"
|
||||
#include "config_loader.h"
|
||||
#include "rabbitizer.hpp"
|
||||
@@ -773,10 +775,10 @@ static int psxrecomp_game_main(int argc, char** argv) {
|
||||
* §5.3, §5.4). The bounds are checked by physical address: a seed in the
|
||||
* image's link segment (0x000100E4 for a KUSEG-linked EXE, 0x800100E4
|
||||
* for a KSEG0 one) is an entry of the image. A seed for the same bytes in
|
||||
* another segment asks for a segment variant (§5.4), which is not
|
||||
* compiled yet: it is listed below and never folded into the home body,
|
||||
* so that PC stays a loud segment miss at run time. Seeds outside the
|
||||
* image are counted. */
|
||||
* another segment asks for a segment variant (§5.4): it is never folded
|
||||
* into the home body; after the home compile, the home functions it
|
||||
* reaches through direct edges are compiled again in its segment (see
|
||||
* plan_segment_variants below). Seeds outside the image are counted. */
|
||||
std::vector<uint32_t> file_seeds;
|
||||
std::vector<std::pair<uint32_t, std::string>> variant_seeds; // PC, line kind
|
||||
size_t outside_seeds = 0;
|
||||
@@ -942,6 +944,20 @@ static int psxrecomp_game_main(int argc, char** argv) {
|
||||
if (phys < (seed_lo & PSXRecomp::kPhysMask) ||
|
||||
phys >= (seed_lo & PSXRecomp::kPhysMask) + (seed_hi - seed_lo)) {
|
||||
++outside_seeds;
|
||||
} else if (!PSXRecomp::maps_physical(addr)) {
|
||||
// Only KUSEG below 0x20000000, KSEG0 and KSEG1 alias the
|
||||
// image's bytes; a variant in any other segment would
|
||||
// give dispatch rows to PCs hardware cannot fetch as RAM.
|
||||
fmt::print(stderr,
|
||||
"ERROR: seed 0x{:08X} is in segment 0x{:08X}, which does not "
|
||||
"map physical memory (only KUSEG below 0x20000000, KSEG0 and "
|
||||
"KSEG1 do), so it names no code of this image and cannot ask "
|
||||
"for a segment variant "
|
||||
"(docs/SEGMENT_AWARE_CODE.md §5.4). The image's bytes at that "
|
||||
"physical address are 0x{:08X} in its link segment.\n",
|
||||
addr, addr & PSXRecomp::kSegmentMask,
|
||||
exe->link_segment() | phys);
|
||||
return 1;
|
||||
} else if ((addr & PSXRecomp::kSegmentMask) != exe->link_segment()) {
|
||||
variant_seeds.emplace_back(
|
||||
addr, interior ? "interior" : trusted_root ? "dispatch_root"
|
||||
@@ -998,13 +1014,20 @@ static int psxrecomp_game_main(int argc, char** argv) {
|
||||
: seg == PSXRecomp::kSegKSEG0 ? "KSEG0"
|
||||
: seg == PSXRecomp::kSegKSEG1 ? "KSEG1" : "an unmapped segment";
|
||||
};
|
||||
fmt::print("WARNING: {} seed(s) name this image's bytes in another segment "
|
||||
"than its link segment {}. Each asks for a segment variant "
|
||||
"(docs/SEGMENT_AWARE_CODE.md §5.4), which is not compiled "
|
||||
"yet: that PC runs interpreted and is recorded as a segment "
|
||||
"miss. A seed for the image's own code is written in {}.\n",
|
||||
variant_seeds.size(), seg_name(exe->link_segment()),
|
||||
seg_name(exe->link_segment()));
|
||||
if (overlay_mode) {
|
||||
fmt::print("WARNING: {} seed(s) name this image's bytes in another "
|
||||
"segment than its link segment {}. Overlay compiles have "
|
||||
"no segment variants yet (docs/SEGMENT_AWARE_CODE.md §5.7): "
|
||||
"nothing is compiled for them.\n",
|
||||
variant_seeds.size(), seg_name(exe->link_segment()));
|
||||
} else {
|
||||
fmt::print("{} seed(s) name this image's bytes in another segment than "
|
||||
"its link segment {}. Each asks for a segment variant "
|
||||
"(docs/SEGMENT_AWARE_CODE.md §5.4), compiled after the home "
|
||||
"compile. A seed for the image's own code is written in {}.\n",
|
||||
variant_seeds.size(), seg_name(exe->link_segment()),
|
||||
seg_name(exe->link_segment()));
|
||||
}
|
||||
for (const auto& [addr, kind] : variant_seeds) {
|
||||
fmt::print(" variant seed 0x{:08X} ({}, {}): home PC would be 0x{:08X}\n",
|
||||
addr, kind, seg_name(addr & PSXRecomp::kSegmentMask),
|
||||
@@ -1563,6 +1586,70 @@ static int psxrecomp_game_main(int argc, char** argv) {
|
||||
}
|
||||
}
|
||||
|
||||
// Segment variants (docs/SEGMENT_AWARE_CODE.md §5.4). Each segment named
|
||||
// by a variant seed gets the direct-edge closure of the home functions
|
||||
// its seeds reach, compiled again through a view of the image in that
|
||||
// segment: its names, rows and every PC it bakes are that segment's, and
|
||||
// a KSEG1 variant charges a fetch per instruction (§5.6). The bodies join
|
||||
// the home shards, and their rows join the home dispatch table. Overlay
|
||||
// compiles have none yet (§5.7).
|
||||
std::vector<PSXRecomp::GeneratedFunction> all_gen_funcs = codegen.last_gen_funcs();
|
||||
std::vector<std::string> variant_alias_decls;
|
||||
std::string all_ranges_manifest = codegen.last_ranges_manifest().empty()
|
||||
? codegen.generate_ranges_manifest(analysis_result.functions, all_cfgs)
|
||||
: codegen.last_ranges_manifest();
|
||||
std::vector<PSXRecomp::GameDispatchUnit> dispatch_units{{&codegen, dispatch_addrs}};
|
||||
std::vector<std::unique_ptr<PSXRecomp::PS1Executable>> variant_views;
|
||||
std::vector<std::unique_ptr<PSXRecomp::CodeGenerator>> variant_codegens;
|
||||
if (!overlay_mode && !variant_seeds.empty()) {
|
||||
std::vector<uint32_t> requested;
|
||||
for (const auto& [addr, kind] : variant_seeds) {
|
||||
(void)kind;
|
||||
requested.push_back(addr);
|
||||
}
|
||||
const auto plans = PSXRecomp::plan_segment_variants(
|
||||
*exe, codegen.last_functions(), codegen.last_cfgs(),
|
||||
codegen.cps_continuations(), requested);
|
||||
for (const auto& plan : plans) {
|
||||
fmt::print("\n=== Segment variant 0x{:08X} (docs/SEGMENT_AWARE_CODE.md §5.4) ===\n",
|
||||
plan.segment);
|
||||
for (const auto& [pc, why] : plan.unplaced) {
|
||||
fmt::print(stderr,
|
||||
"ERROR: variant seed 0x{:08X} cannot be compiled: {}. Seed the home "
|
||||
"PC first, or drop the variant seed.\n", pc, why);
|
||||
}
|
||||
if (!plan.unplaced.empty()) return 1;
|
||||
fmt::print(" {} seed(s), direct-edge closure of {} function(s)\n",
|
||||
plan.seeds.size(), plan.functions.size());
|
||||
variant_views.push_back(std::make_unique<PSXRecomp::PS1Executable>(
|
||||
PSXRecomp::segment_view(*exe, plan.segment)));
|
||||
const PSXRecomp::PS1Executable& view = *variant_views.back();
|
||||
PSXRecomp::ControlFlowAnalyzer view_cfg(view);
|
||||
const auto variant_cfgs = view_cfg.analyze_all_functions(plan.functions);
|
||||
variant_codegens.push_back(std::make_unique<PSXRecomp::CodeGenerator>(
|
||||
view, PSXRecomp::rebase_codegen_config(codegen_config, *exe, plan.segment)));
|
||||
PSXRecomp::CodeGenerator& vgen = *variant_codegens.back();
|
||||
(void)vgen.generate_file(plan.functions, variant_cfgs);
|
||||
std::set<uint32_t> variant_addrs;
|
||||
for (const auto& gf : vgen.last_gen_funcs()) {
|
||||
all_gen_funcs.push_back(gf);
|
||||
if (gf.dispatchable && gf.function_name.rfind("func_", 0) == 0 &&
|
||||
gf.function_name.size() == 13)
|
||||
variant_addrs.insert(static_cast<uint32_t>(
|
||||
std::strtoul(gf.function_name.c_str() + 5, nullptr, 16)));
|
||||
}
|
||||
for (const auto& decl : vgen.alias_body_decls()) variant_alias_decls.push_back(decl);
|
||||
// The variant's F/R records, without the manifest's header comment.
|
||||
std::istringstream vr(vgen.last_ranges_manifest());
|
||||
for (std::string line; std::getline(vr, line);) {
|
||||
if (!line.empty() && line[0] != '#') all_ranges_manifest += line + "\n";
|
||||
}
|
||||
dispatch_units.push_back({&vgen, variant_addrs});
|
||||
fmt::print(" {} bodies, {} dispatch entries\n",
|
||||
vgen.last_gen_funcs().size(), variant_addrs.size());
|
||||
}
|
||||
}
|
||||
|
||||
// Split-TU output: instead of one monolithic <exe_stem>_full.c, write a
|
||||
// shared declarations header plus N shards bucketed by cumulative line
|
||||
// count, so the game build can compile the shards in parallel. The
|
||||
@@ -1587,7 +1674,7 @@ static int psxrecomp_game_main(int argc, char** argv) {
|
||||
std::filesystem::path decls_filename = out_dir / (exe_stem + "_decls.h");
|
||||
{
|
||||
const std::string decls =
|
||||
codegen.build_shared_decls_header(codegen.last_gen_funcs());
|
||||
codegen.build_shared_decls_header(all_gen_funcs, variant_alias_decls);
|
||||
if (write_file_if_changed(decls_filename, decls)) {
|
||||
fmt::print("✓ Saved shared decls header to {}\n", decls_filename.string());
|
||||
} else {
|
||||
@@ -1597,7 +1684,7 @@ static int psxrecomp_game_main(int argc, char** argv) {
|
||||
|
||||
// 2. Bucket generated functions into shards by cumulative line count.
|
||||
const int SHARD_LINE_BUDGET = 40000;
|
||||
const std::vector<PSXRecomp::GeneratedFunction>& gen_funcs_for_shards = codegen.last_gen_funcs();
|
||||
const std::vector<PSXRecomp::GeneratedFunction>& gen_funcs_for_shards = all_gen_funcs;
|
||||
std::string decls_basename = exe_stem + "_decls.h";
|
||||
|
||||
int shard_index = 0;
|
||||
@@ -1654,9 +1741,7 @@ static int psxrecomp_game_main(int argc, char** argv) {
|
||||
// Per-function code-range manifest (design §8): consumed by the overlay
|
||||
// loader's per-entry validity hash. Emitted alongside _full.c for every
|
||||
// build; only the overlay path actually loads it.
|
||||
const std::string ranges_manifest = codegen.last_ranges_manifest().empty()
|
||||
? codegen.generate_ranges_manifest(analysis_result.functions, all_cfgs)
|
||||
: codegen.last_ranges_manifest();
|
||||
const std::string& ranges_manifest = all_ranges_manifest;
|
||||
{
|
||||
std::filesystem::path ranges_filename = out_dir / (exe_stem + "_full.ranges");
|
||||
if (write_file_if_changed(ranges_filename, ranges_manifest)) {
|
||||
@@ -1676,18 +1761,24 @@ static int psxrecomp_game_main(int argc, char** argv) {
|
||||
fmt::print("Generating dispatch table: {}\n", dispatch_filename.string());
|
||||
|
||||
std::string dispatch_source, dispatch_error;
|
||||
if (!PSXRecomp::emit_game_dispatch(codegen, *exe, dispatch_addrs,
|
||||
if (!PSXRecomp::emit_game_dispatch(dispatch_units, *exe,
|
||||
ranges_manifest, dispatch_source,
|
||||
dispatch_error)) {
|
||||
fmt::print(stderr, "ERROR: {}\n", dispatch_error);
|
||||
return 1;
|
||||
}
|
||||
// Entries of the home compile, plus those of its segment variants.
|
||||
size_t variant_entries = 0;
|
||||
for (size_t u = 1; u < dispatch_units.size(); ++u)
|
||||
variant_entries += dispatch_units[u].dispatch_addrs.size();
|
||||
const std::string variant_note = variant_entries
|
||||
? fmt::format(", plus {} in segment variants", variant_entries) : std::string();
|
||||
if (write_file_if_changed(dispatch_filename, dispatch_source)) {
|
||||
fmt::print("✓ Dispatch table written ({} entries)\n\n",
|
||||
dispatch_addrs.size());
|
||||
fmt::print("✓ Dispatch table written ({} entries{})\n\n",
|
||||
dispatch_addrs.size(), variant_note);
|
||||
} else {
|
||||
fmt::print("✓ Dispatch table unchanged ({} entries)\n\n",
|
||||
dispatch_addrs.size());
|
||||
fmt::print("✓ Dispatch table unchanged ({} entries{})\n\n",
|
||||
dispatch_addrs.size(), variant_note);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,202 @@
|
||||
#include "segment_variants.h"
|
||||
|
||||
#include <algorithm>
|
||||
#include <set>
|
||||
|
||||
#include "fmt/format.h"
|
||||
|
||||
namespace PSXRecomp {
|
||||
|
||||
PS1Executable segment_view(const PS1Executable& exe, uint32_t seg) {
|
||||
PS1Executable view = exe;
|
||||
seg &= kSegmentMask;
|
||||
view.header.load_address = seg | (exe.header.load_address & kPhysMask);
|
||||
view.header.initial_pc = seg | (exe.header.initial_pc & kPhysMask);
|
||||
return view;
|
||||
}
|
||||
|
||||
std::vector<uint32_t> direct_successors(const ControlFlowGraph& cfg) {
|
||||
std::set<uint32_t> out;
|
||||
auto leave = [&](uint32_t target) {
|
||||
if (target != 0 && !cfg.blocks.count(target)) out.insert(target);
|
||||
};
|
||||
for (const auto& [addr, block] : cfg.blocks) {
|
||||
(void)addr;
|
||||
const ControlFlowInstr& x = block.exit_instr;
|
||||
switch (x.type) {
|
||||
case ControlFlowType::Branch:
|
||||
leave(x.target);
|
||||
leave(x.address + 8u);
|
||||
break;
|
||||
case ControlFlowType::Jump:
|
||||
leave(x.target);
|
||||
break;
|
||||
case ControlFlowType::JumpLink:
|
||||
leave(x.target);
|
||||
leave(x.address + 8u);
|
||||
break;
|
||||
case ControlFlowType::JumpLinkReg:
|
||||
leave(x.address + 8u);
|
||||
break;
|
||||
case ControlFlowType::None:
|
||||
if (block.successors.empty()) leave(block.end_addr + 4u);
|
||||
break;
|
||||
default: // jr / jr $ra: indirect, through dispatch
|
||||
break;
|
||||
}
|
||||
}
|
||||
return {out.begin(), out.end()};
|
||||
}
|
||||
|
||||
namespace {
|
||||
|
||||
Function rebase_function(const Function& f, uint32_t seg) {
|
||||
auto at = [seg](uint32_t a) { return a ? (seg | (a & kPhysMask)) : 0u; };
|
||||
Function v = f;
|
||||
v.start_addr = at(f.start_addr);
|
||||
v.end_addr = seg | (f.end_addr & kPhysMask);
|
||||
v.name = fmt::format("func_{:08X}", v.start_addr);
|
||||
v.alias_walk_lo = at(f.alias_walk_lo);
|
||||
for (uint32_t& e : v.alias_group_entries) e = at(e);
|
||||
v.producer_lo = at(f.producer_lo);
|
||||
v.producer_hi = f.producer_hi ? (seg | (f.producer_hi & kPhysMask)) : 0u;
|
||||
return v;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
std::vector<SegmentVariantPlan> plan_segment_variants(
|
||||
const PS1Executable& exe,
|
||||
const std::vector<Function>& home_functions,
|
||||
const std::map<uint32_t, ControlFlowGraph>& home_cfgs,
|
||||
const std::map<uint32_t, uint32_t>& home_continuations,
|
||||
const std::vector<uint32_t>& seeds) {
|
||||
const uint32_t link = exe.link_segment();
|
||||
std::map<uint32_t, const Function*> by_start;
|
||||
std::map<uint32_t, std::vector<uint32_t>> alias_groups; // host -> members
|
||||
for (const Function& f : home_functions) {
|
||||
by_start.emplace(f.start_addr, &f);
|
||||
if (f.alias_walk_lo != 0) alias_groups[f.alias_walk_lo].push_back(f.start_addr);
|
||||
}
|
||||
|
||||
std::map<uint32_t, SegmentVariantPlan> plans;
|
||||
std::map<uint32_t, std::set<uint32_t>> roots;
|
||||
for (uint32_t pc : seeds) {
|
||||
const uint32_t seg = pc & kSegmentMask;
|
||||
SegmentVariantPlan& plan = plans[seg];
|
||||
plan.segment = seg;
|
||||
plan.seeds.push_back(pc);
|
||||
const uint32_t home = link | (pc & kPhysMask);
|
||||
if (!maps_physical(pc)) {
|
||||
plan.unplaced.emplace_back(pc, fmt::format(
|
||||
"segment 0x{:08X} does not map physical memory (only KUSEG below "
|
||||
"0x20000000, KSEG0 and KSEG1 do), so it is no alias of the home PC "
|
||||
"0x{:08X}", seg, home));
|
||||
continue;
|
||||
}
|
||||
uint32_t root = 0;
|
||||
if (by_start.count(home)) {
|
||||
root = home;
|
||||
} else {
|
||||
auto cont = home_continuations.find(home);
|
||||
if (cont != home_continuations.end() && by_start.count(cont->second))
|
||||
root = cont->second;
|
||||
}
|
||||
if (root == 0) {
|
||||
plan.unplaced.emplace_back(pc, fmt::format(
|
||||
"its home PC 0x{:08X} is not a dispatch entry of the home compile "
|
||||
"(no function starts there and it is no CPS continuation)", home));
|
||||
continue;
|
||||
}
|
||||
roots[seg].insert(root);
|
||||
}
|
||||
|
||||
std::vector<SegmentVariantPlan> out;
|
||||
for (auto& [seg, plan] : plans) {
|
||||
std::set<uint32_t> included;
|
||||
std::vector<uint32_t> work(roots[seg].begin(), roots[seg].end());
|
||||
while (!work.empty()) {
|
||||
const uint32_t start = work.back();
|
||||
work.pop_back();
|
||||
if (!included.insert(start).second) continue;
|
||||
const Function* f = by_start.at(start);
|
||||
// Alias entries share one emitted body with their host's other
|
||||
// entries: compile the group together.
|
||||
if (f->alias_walk_lo != 0) {
|
||||
for (uint32_t member : alias_groups[f->alias_walk_lo]) work.push_back(member);
|
||||
}
|
||||
auto cfg = home_cfgs.find(start);
|
||||
if (cfg == home_cfgs.end()) continue;
|
||||
for (uint32_t succ : direct_successors(cfg->second)) {
|
||||
if (by_start.count(succ)) work.push_back(succ);
|
||||
}
|
||||
}
|
||||
for (uint32_t start : included) plan.functions.push_back(rebase_function(*by_start.at(start), seg));
|
||||
std::sort(plan.functions.begin(), plan.functions.end(),
|
||||
[](const Function& a, const Function& b) { return a.start_addr < b.start_addr; });
|
||||
out.push_back(std::move(plan));
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
CodeGenConfig rebase_codegen_config(const CodeGenConfig& cfg,
|
||||
const PS1Executable& exe, uint32_t seg) {
|
||||
seg &= kSegmentMask;
|
||||
const uint32_t link = exe.link_segment();
|
||||
auto at = [&](uint32_t a) {
|
||||
return (a != 0 && exe.contains_phys(a) && (a & kSegmentMask) == link)
|
||||
? (seg | (a & kPhysMask)) : a;
|
||||
};
|
||||
auto move_set = [&](std::set<uint32_t>& s) {
|
||||
std::set<uint32_t> moved;
|
||||
for (uint32_t a : s) moved.insert(at(a));
|
||||
s = std::move(moved);
|
||||
};
|
||||
CodeGenConfig v = cfg;
|
||||
move_set(v.ws_sprite_tag_funcs);
|
||||
move_set(v.data_shard_funcs);
|
||||
move_set(v.mod_function_entry_funcs);
|
||||
move_set(v.hot_funcs);
|
||||
move_set(v.load_charge_batch_funcs);
|
||||
move_set(v.ws_backdrop_unsquash_funcs);
|
||||
move_set(v.ws_cull_bias_sites);
|
||||
move_set(v.ws_cull_range_sites);
|
||||
move_set(v.ws_cull_a1_sites);
|
||||
move_set(v.ws_cull_screen_x_sites);
|
||||
move_set(v.ws_cull_slti_sites);
|
||||
move_set(v.ws_cull_slti_lower_sites);
|
||||
move_set(v.ws_cull_bltz_sites);
|
||||
move_set(v.ws_cull_bgez_sites);
|
||||
move_set(v.ws_cull_clip_edge_x_load_sites);
|
||||
move_set(v.ws_cull_negsub_sites);
|
||||
move_set(v.ws_cull_vxrange_sites);
|
||||
move_set(v.ws_cull_depth_sites);
|
||||
move_set(v.ws_cull_plane_nx_sites);
|
||||
move_set(v.ws_cull_xclip_load_sites);
|
||||
move_set(v.ws_cull_nclip_keep_sites);
|
||||
move_set(v.ws_cull_nclip_exact_sites);
|
||||
move_set(v.ws_cull_branch_keep_sites);
|
||||
move_set(v.ws_backdrop_x_sites);
|
||||
move_set(v.persist_init_store_sites);
|
||||
{
|
||||
std::map<uint32_t, std::array<uint32_t, 4>> moved;
|
||||
for (const auto& [func, data] : v.vsync_query_hle_funcs) moved[at(func)] = data;
|
||||
v.vsync_query_hle_funcs = std::move(moved);
|
||||
}
|
||||
v.ws_bg2d_init_func = at(v.ws_bg2d_init_func);
|
||||
v.ws_bg2d_count_site = at(v.ws_bg2d_count_site);
|
||||
v.ws_bg2d_startcol_site = at(v.ws_bg2d_startcol_site);
|
||||
v.ws_bg2d_startx_site = at(v.ws_bg2d_startx_site);
|
||||
v.ws_bg2d_stream_left_site = at(v.ws_bg2d_stream_left_site);
|
||||
v.ws_bg2d_stream_right_site = at(v.ws_bg2d_stream_right_site);
|
||||
v.ws_bg2d_bufbase_site = at(v.ws_bg2d_bufbase_site);
|
||||
v.ws_bg2d_cap_site = at(v.ws_bg2d_cap_site);
|
||||
// Physically matched, kept as written: ws_signed_x_bound_sites,
|
||||
// ws_cull_keep_sites, ws_cull_angle_sites, ws_aspect_cone.sites.
|
||||
// The split pre-pass ran on the home compile: the closure is already made
|
||||
// of its final pieces.
|
||||
v.split_mid_function_targets = false;
|
||||
return v;
|
||||
}
|
||||
|
||||
} // namespace PSXRecomp
|
||||
@@ -0,0 +1,80 @@
|
||||
#pragma once
|
||||
|
||||
// Per-segment compiled variants of static game code
|
||||
// (docs/SEGMENT_AWARE_CODE.md §5.4).
|
||||
//
|
||||
// A PC carries a segment, and a compiled body bakes the segment it was
|
||||
// compiled for into every link, EPC, fetch tag and store PC. The home compile
|
||||
// is the image's link segment. A seed that names the image's bytes in another
|
||||
// segment (a segment-qualified seed, §10 decision 3) asks for a variant: the
|
||||
// home functions that seed reaches through direct edges (calls, tail jumps,
|
||||
// split pieces, fallthroughs, CPS continuations), compiled again in that
|
||||
// segment. Direct control flow never changes segment (§2), so that closure is
|
||||
// everything the variant can reach without going through dispatch.
|
||||
//
|
||||
// A variant is compiled through a segment view of the image: the same bytes
|
||||
// with the header moved into the variant's segment. Its code identity is then
|
||||
// the variant VA (§5.1: func_A00100F0), the CodeGenerator's code segment is the
|
||||
// view's, and every PC it bakes is in that segment. A KSEG1 variant is charged
|
||||
// a fetch per instruction by #429's rule (§5.6) with no emitter change.
|
||||
|
||||
#include <cstdint>
|
||||
#include <map>
|
||||
#include <string>
|
||||
#include <utility>
|
||||
#include <vector>
|
||||
|
||||
#include "code_generator.h"
|
||||
#include "control_flow.h"
|
||||
#include "function_analysis.h"
|
||||
#include "ps1_exe_parser.h"
|
||||
|
||||
namespace PSXRecomp {
|
||||
|
||||
// The image seen from segment `seg`: the header's load address and entry move
|
||||
// into `seg`, the bytes do not change. Addresses in the view are variant VAs.
|
||||
PS1Executable segment_view(const PS1Executable& exe, uint32_t seg);
|
||||
|
||||
// One requested segment.
|
||||
struct SegmentVariantPlan {
|
||||
uint32_t segment = 0;
|
||||
// The closure, rebased into `segment` and sorted by start address.
|
||||
std::vector<Function> functions;
|
||||
// The requested PCs, as written.
|
||||
std::vector<uint32_t> seeds;
|
||||
// Seeds that name no home dispatch entry (a function entry or a CPS
|
||||
// continuation of the home compile), or that lie in a segment that does
|
||||
// not map physical memory (maps_physical()), with the reason. Nothing is
|
||||
// compiled for them.
|
||||
std::vector<std::pair<uint32_t, std::string>> unplaced;
|
||||
};
|
||||
|
||||
// The direct-edge successors of a home function that leave it: branch and
|
||||
// jump targets and not-taken paths outside its blocks, call targets, call
|
||||
// returns split into another piece, and a fall-through past its last block.
|
||||
// Indirect transfers are not edges: they go through dispatch with the full PC.
|
||||
std::vector<uint32_t> direct_successors(const ControlFlowGraph& cfg);
|
||||
|
||||
// Plans a variant per segment named by `seeds` (PCs outside the link segment
|
||||
// that name the image's bytes). `home_functions` and `home_cfgs` are the home
|
||||
// compile's final function set (CodeGenerator::last_functions / last_cfgs),
|
||||
// and `home_continuations` its CPS continuation -> owner map. A seed at a home
|
||||
// function entry roots that function; a seed at a CPS continuation roots its
|
||||
// owner (the variant has the continuation too); an alias entry brings its
|
||||
// whole alias group, which shares one body. Plans are in segment order.
|
||||
std::vector<SegmentVariantPlan> plan_segment_variants(
|
||||
const PS1Executable& exe,
|
||||
const std::vector<Function>& home_functions,
|
||||
const std::map<uint32_t, ControlFlowGraph>& home_cfgs,
|
||||
const std::map<uint32_t, uint32_t>& home_continuations,
|
||||
const std::vector<uint32_t>& seeds);
|
||||
|
||||
// The codegen config for a variant in `seg`. Config code sites that the
|
||||
// emitter matches exactly are written in the link segment (main_psx refuses
|
||||
// any other spelling, §5.3); each one that names the image's bytes moves into
|
||||
// `seg`, so the variant gets every hook and substitution the home body gets.
|
||||
// Kinds matched by physical address need no move and are kept as written.
|
||||
CodeGenConfig rebase_codegen_config(const CodeGenConfig& cfg,
|
||||
const PS1Executable& exe, uint32_t seg);
|
||||
|
||||
} // namespace PSXRecomp
|
||||
@@ -39,6 +39,9 @@
|
||||
// compiles in its own link segment, where runtime_pc() is the identity, so
|
||||
// only this test (code segment != compile segment) sees the routes; the
|
||||
// ledger and test_kuseg_dispatch_lookup.py check the emitted lookup end to end.
|
||||
// Two rows on one physical word are an error within one compile; a segment
|
||||
// variant's compile (PR D, §5.4) adds rows on its home rows' words, and the
|
||||
// lookup then searches the word's rows for the exact PC.
|
||||
|
||||
#include "code_generator.h"
|
||||
#include "control_flow.h"
|
||||
@@ -497,9 +500,11 @@ void check_dispatch_rows(Mode mode) {
|
||||
}
|
||||
}
|
||||
|
||||
// One compile has one code segment, so two rows on one physical word (a
|
||||
// variant of another segment, §5.4) are a build error until PR D's variants
|
||||
// share the table: never a row that silently shadows the other.
|
||||
// One compile has one code segment, so two rows on one physical word within
|
||||
// it are the same PC: a build error, never a row that silently shadows the
|
||||
// other. Segment variants (§5.4) are separate compiles, and their rows share
|
||||
// the home rows' words: adjacent, in PC order, looked up by exact PC. Two
|
||||
// compiles in one segment would name the same PCs, which is an error again.
|
||||
void check_dispatch_duplicate_word() {
|
||||
const uint32_t base = KSEG0 | kPhys;
|
||||
PSXRecomp::PS1Executable exe{};
|
||||
@@ -522,6 +527,37 @@ void check_dispatch_duplicate_word() {
|
||||
check(error.find("share a physical word") != std::string::npos,
|
||||
"dispatch: the duplicate-word error must say why, got: " + error);
|
||||
}
|
||||
|
||||
PSXRecomp::PS1Executable kseg1 = exe;
|
||||
kseg1.header.load_address = KSEG1 | kPhys;
|
||||
kseg1.header.initial_pc = KSEG1 | kPhys;
|
||||
PSXRecomp::CodeGenerator variant(kseg1, config);
|
||||
const std::vector<PSXRecomp::GameDispatchUnit> units{
|
||||
{&generator, {base, base + 0x20u}}, {&variant, {KSEG1 | kPhys}}};
|
||||
out.clear();
|
||||
error.clear();
|
||||
check(PSXRecomp::emit_game_dispatch(units, exe, "", out, error) && error.empty(),
|
||||
"dispatch: a variant compile's row may share its home row's word: " + error);
|
||||
const auto home_row = out.find("{0x" + hex(base) + "u, 0x00000000u");
|
||||
const auto variant_row = out.find("{0x" + hex(KSEG1 | kPhys) + "u, 0x00000000u");
|
||||
const auto variant_line =
|
||||
variant_row == std::string::npos
|
||||
? std::string()
|
||||
: out.substr(variant_row, out.find('\n', variant_row) - variant_row);
|
||||
check(home_row != std::string::npos && variant_row != std::string::npos &&
|
||||
home_row < variant_row &&
|
||||
variant_line.find("func_" + hex(KSEG1 | kPhys) + "}") != std::string::npos,
|
||||
"dispatch: the variant row follows the home row on their word and names its "
|
||||
"body: " + variant_line);
|
||||
check(out.find("k_psx_game_dispatch[row].addr == addr") != std::string::npos,
|
||||
"dispatch: a shared word is searched for the exact PC");
|
||||
const std::vector<PSXRecomp::GameDispatchUnit> twice{
|
||||
{&generator, {base}}, {&variant, {KSEG1 | kPhys}}, {&variant, {KSEG1 | kPhys}}};
|
||||
out.clear();
|
||||
error.clear();
|
||||
check(!PSXRecomp::emit_game_dispatch(twice, exe, "", out, error) &&
|
||||
error.find("are the same PC 0x" + hex(KSEG1 | kPhys)) != std::string::npos,
|
||||
"dispatch: two compiles in one segment name the same PC, an error: " + error);
|
||||
}
|
||||
|
||||
void check_strict_translator(bool strict_cps) {
|
||||
|
||||
@@ -0,0 +1,273 @@
|
||||
// Segment variants of static game code (docs/SEGMENT_AWARE_CODE.md §5.4,
|
||||
// rollout PR D): the planner that turns segment-qualified seeds into per-
|
||||
// segment direct-edge closures, the segment view of an image, and the codegen
|
||||
// config a variant compiles with.
|
||||
//
|
||||
// The planner works on a hand-built KSEG0 image whose functions are given
|
||||
// explicitly (so the test does not depend on discovery heuristics):
|
||||
// F0 0x00 jal F1 / j F2 a call and a tail jump
|
||||
// U 0x10 jr $ra reached by nothing
|
||||
// F1 0x18 jr $ra F0's callee
|
||||
// F2 0x20 beq -> F4, then runs a split branch out of the function and a
|
||||
// off its end into F3 fall-through past its last block
|
||||
// F3 0x2C jr $ra
|
||||
// F4 0x38 jr $ra
|
||||
// H 0x40 host with alias entries A1 0x44 and A2 0x48 (one shared body)
|
||||
// F5 0x50 beq -> F1 as its last pair: taken and not taken both leave
|
||||
// F6 0x58 jr $ra F5's not-taken path
|
||||
// It checks:
|
||||
// - a seed at a function entry roots that function; its closure follows
|
||||
// calls, tail jumps, out-of-function branch targets and fall-throughs,
|
||||
// and nothing else (U and H stay out);
|
||||
// - a seed at a CPS continuation roots its owner;
|
||||
// - an alias entry brings its whole alias group, not its host;
|
||||
// - a seed that names no home dispatch entry is reported, not guessed, and
|
||||
// so is a seed in a segment that does not map physical memory
|
||||
// (0x20000000-0x7FFFFFFF, KSEG2);
|
||||
// - plans come out per segment, in segment order, rebased: start/end,
|
||||
// name, alias host, alias group and producer range all move into the
|
||||
// segment;
|
||||
// - direct_successors lists exactly the edges that leave a function.
|
||||
// segment_view moves the header and keeps the bytes, and
|
||||
// rebase_codegen_config moves the exact-match code sites that name the image
|
||||
// into the variant's segment, keeps other addresses and the physically
|
||||
// matched kinds as written, and turns the split pre-pass off.
|
||||
|
||||
#include "segment_variants.h"
|
||||
|
||||
#include <cstdint>
|
||||
#include <cstdio>
|
||||
#include <map>
|
||||
#include <set>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
namespace {
|
||||
|
||||
int failures = 0;
|
||||
|
||||
void check(bool ok, const std::string& what) {
|
||||
if (!ok) {
|
||||
std::printf("FAIL: %s\n", what.c_str());
|
||||
++failures;
|
||||
}
|
||||
}
|
||||
|
||||
constexpr uint32_t B = 0x80010000u;
|
||||
constexpr uint32_t KUSEG = 0x00000000u, KSEG1 = 0xA0000000u;
|
||||
|
||||
uint32_t jal(uint32_t t) { return 0x0C000000u | ((t >> 2) & 0x03FFFFFFu); }
|
||||
uint32_t j(uint32_t t) { return 0x08000000u | ((t >> 2) & 0x03FFFFFFu); }
|
||||
uint32_t beqz(uint32_t pc, uint32_t t) { // beq a0, zero, t
|
||||
return 0x10800000u | (((t - (pc + 4u)) >> 2) & 0xFFFFu);
|
||||
}
|
||||
constexpr uint32_t JR_RA = 0x03E00008u, NOP = 0u, ADDIU_V0 = 0x24020001u;
|
||||
|
||||
PSXRecomp::PS1Executable image() {
|
||||
std::vector<uint32_t> w(0x60 / 4, NOP);
|
||||
auto at = [&](uint32_t off, uint32_t word) { w[off / 4] = word; };
|
||||
at(0x00, jal(B + 0x18)); at(0x08, j(B + 0x20)); // F0
|
||||
at(0x10, JR_RA); // U
|
||||
at(0x18, JR_RA); // F1
|
||||
at(0x20, beqz(B + 0x20, B + 0x38)); at(0x28, ADDIU_V0); // F2
|
||||
at(0x2C, JR_RA); // F3
|
||||
at(0x38, JR_RA); // F4
|
||||
at(0x40, ADDIU_V0); at(0x44, ADDIU_V0); at(0x48, JR_RA); // H, A1, A2
|
||||
at(0x50, beqz(B + 0x50, B + 0x18)); // F5
|
||||
at(0x58, JR_RA); // F6
|
||||
PSXRecomp::PS1Executable exe{};
|
||||
exe.header.load_address = B;
|
||||
exe.header.initial_pc = B;
|
||||
exe.header.file_size = static_cast<uint32_t>(w.size() * 4u);
|
||||
for (uint32_t word : w)
|
||||
for (int i = 0; i < 4; ++i) exe.code_data.push_back(static_cast<uint8_t>(word >> (8 * i)));
|
||||
return exe;
|
||||
}
|
||||
|
||||
PSXRecomp::Function fn(uint32_t lo, uint32_t hi, uint32_t alias_host = 0,
|
||||
std::vector<uint32_t> group = {}) {
|
||||
PSXRecomp::Function f;
|
||||
f.start_addr = lo;
|
||||
f.end_addr = hi;
|
||||
f.size = hi - lo;
|
||||
f.has_prologue = f.has_epilogue = false;
|
||||
f.stack_frame_size = 0;
|
||||
f.name = "func_" + std::to_string(lo);
|
||||
f.alias_walk_lo = alias_host;
|
||||
f.alias_group_entries = std::move(group);
|
||||
return f;
|
||||
}
|
||||
|
||||
std::string hex(uint32_t v) {
|
||||
char b[16];
|
||||
std::snprintf(b, sizeof b, "%08X", v);
|
||||
return b;
|
||||
}
|
||||
|
||||
std::set<uint32_t> starts(const PSXRecomp::SegmentVariantPlan& p) {
|
||||
std::set<uint32_t> out;
|
||||
for (const auto& f : p.functions) out.insert(f.start_addr);
|
||||
return out;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
int main() {
|
||||
const auto exe = image();
|
||||
const std::vector<uint32_t> group{B + 0x44, B + 0x48};
|
||||
const std::vector<PSXRecomp::Function> home{
|
||||
fn(B + 0x00, B + 0x10), fn(B + 0x10, B + 0x18), fn(B + 0x18, B + 0x20),
|
||||
fn(B + 0x20, B + 0x2C), fn(B + 0x2C, B + 0x34), fn(B + 0x38, B + 0x40),
|
||||
fn(B + 0x40, B + 0x50), fn(B + 0x44, B + 0x50, B + 0x40, group),
|
||||
fn(B + 0x48, B + 0x50, B + 0x40, group), fn(B + 0x50, B + 0x58),
|
||||
fn(B + 0x58, B + 0x60)};
|
||||
PSXRecomp::ControlFlowAnalyzer analyzer(exe);
|
||||
const auto cfgs = analyzer.analyze_all_functions(home);
|
||||
|
||||
// direct_successors: the edges that leave a function, and only those.
|
||||
{
|
||||
const auto f0 = PSXRecomp::direct_successors(cfgs.at(B));
|
||||
check(f0 == std::vector<uint32_t>{B + 0x18, B + 0x20},
|
||||
"F0 leaves by its jal and its j, got " + std::to_string(f0.size()));
|
||||
const auto f2 = PSXRecomp::direct_successors(cfgs.at(B + 0x20));
|
||||
check(f2 == std::vector<uint32_t>{B + 0x2C, B + 0x38},
|
||||
"F2 leaves by its branch target and by running off its end");
|
||||
check(PSXRecomp::direct_successors(cfgs.at(B + 0x18)).empty(),
|
||||
"a jr $ra is no direct edge");
|
||||
check(PSXRecomp::direct_successors(cfgs.at(B + 0x50)) ==
|
||||
std::vector<uint32_t>{B + 0x18, B + 0x58},
|
||||
"F5's final branch leaves both taken and not taken");
|
||||
}
|
||||
|
||||
const std::map<uint32_t, uint32_t> conts{{B + 0x08, B}};
|
||||
const std::vector<uint32_t> seeds{KSEG1 | B, KUSEG | ((B + 0x44) & 0x1FFFFFFFu),
|
||||
KSEG1 | ((B + 0x08) & 0x1FFFFFFFu),
|
||||
KSEG1 | ((B + 0x04) & 0x1FFFFFFFu)};
|
||||
const auto plans = PSXRecomp::plan_segment_variants(exe, home, cfgs, conts, seeds);
|
||||
check(plans.size() == 2 && plans[0].segment == KUSEG && plans[1].segment == KSEG1,
|
||||
"one plan per requested segment, in segment order");
|
||||
if (plans.size() == 2) {
|
||||
const auto& ku = plans[0];
|
||||
const auto& k1 = plans[1];
|
||||
const uint32_t p = B & 0x1FFFFFFFu;
|
||||
check(starts(ku) == std::set<uint32_t>{p + 0x44, p + 0x48},
|
||||
"an alias entry brings its alias group, not its host");
|
||||
check(starts(k1) == std::set<uint32_t>{KSEG1 | p, KSEG1 | (p + 0x18), KSEG1 | (p + 0x20),
|
||||
KSEG1 | (p + 0x2C), KSEG1 | (p + 0x38)},
|
||||
"the KSEG1 closure is F0 with its call, tail jump, branch target and "
|
||||
"fall-through, and neither U nor H");
|
||||
check(k1.seeds.size() == 3 && ku.seeds.size() == 1, "every seed is listed with its plan");
|
||||
check(k1.unplaced.size() == 1 && k1.unplaced[0].first == (KSEG1 | (p + 4)) &&
|
||||
k1.unplaced[0].second.find(hex(B + 4)) != std::string::npos,
|
||||
"a seed inside a function body names no dispatch entry and is reported "
|
||||
"with its home PC");
|
||||
bool sorted = true, rebased = true;
|
||||
for (size_t i = 0; i < k1.functions.size(); ++i) {
|
||||
const auto& f = k1.functions[i];
|
||||
if (i && k1.functions[i - 1].start_addr >= f.start_addr) sorted = false;
|
||||
if ((f.end_addr & 0xE0000000u) != KSEG1 || f.name != "func_" + hex(f.start_addr))
|
||||
rebased = false;
|
||||
}
|
||||
check(sorted && rebased, "KSEG1 functions are sorted, with rebased ends and names");
|
||||
for (const auto& f : ku.functions) {
|
||||
check(f.alias_walk_lo == p + 0x40 &&
|
||||
f.alias_group_entries == std::vector<uint32_t>{p + 0x44, p + 0x48} &&
|
||||
f.name == "func_" + hex(f.start_addr),
|
||||
"alias host and group move into the variant's segment");
|
||||
}
|
||||
}
|
||||
// A branch whose not-taken path runs past the function brings the next.
|
||||
{
|
||||
const auto f5 = PSXRecomp::plan_segment_variants(
|
||||
exe, home, cfgs, conts, {KSEG1 | ((B + 0x50) & 0x1FFFFFFFu)});
|
||||
const uint32_t p = B & 0x1FFFFFFFu;
|
||||
check(f5.size() == 1 &&
|
||||
starts(f5[0]) == std::set<uint32_t>{KSEG1 | (p + 0x18), KSEG1 | (p + 0x50),
|
||||
KSEG1 | (p + 0x58)},
|
||||
"F5's closure is F1 (taken) and F6 (not taken)");
|
||||
}
|
||||
// Only KUSEG below 0x20000000, KSEG0 and KSEG1 map physical memory: a seed
|
||||
// in 0x20000000-0x7FFFFFFF or in KSEG2 is no alias of the image's bytes.
|
||||
// It is reported with its home PC and compiles nothing.
|
||||
{
|
||||
const uint32_t p = B & 0x1FFFFFFFu;
|
||||
const auto bad = PSXRecomp::plan_segment_variants(
|
||||
exe, home, cfgs, conts, {0x20000000u | p, 0xC0000000u | p, 0x60000000u | p});
|
||||
bool refused = bad.size() == 3;
|
||||
for (const auto& plan : bad) {
|
||||
refused = refused && plan.functions.empty() && plan.unplaced.size() == 1 &&
|
||||
plan.unplaced[0].first == (plan.segment | p) &&
|
||||
plan.unplaced[0].second.find("does not map physical memory") !=
|
||||
std::string::npos &&
|
||||
plan.unplaced[0].second.find(hex(B)) != std::string::npos;
|
||||
}
|
||||
check(refused, "a seed in a segment that does not map physical memory is "
|
||||
"reported with its home PC and compiles nothing");
|
||||
check(PSXRecomp::maps_physical(0x1FFFFFFCu) && !PSXRecomp::maps_physical(0x20000000u) &&
|
||||
!PSXRecomp::maps_physical(0x7FFFFFFCu) && PSXRecomp::maps_physical(0x80000000u) &&
|
||||
PSXRecomp::maps_physical(0xBFFFFFFCu) && !PSXRecomp::maps_physical(0xC0000000u) &&
|
||||
!PSXRecomp::maps_physical(0xFFFE0130u),
|
||||
"maps_physical is KUSEG below 0x20000000, KSEG0 and KSEG1");
|
||||
}
|
||||
// A seed at a CPS continuation alone roots its owner.
|
||||
{
|
||||
const auto only = PSXRecomp::plan_segment_variants(
|
||||
exe, home, cfgs, conts, {KSEG1 | ((B + 0x08) & 0x1FFFFFFFu)});
|
||||
check(only.size() == 1 && only[0].unplaced.empty() &&
|
||||
starts(only[0]).count(KSEG1 | (B & 0x1FFFFFFFu)),
|
||||
"a continuation seed roots the function that owns it");
|
||||
}
|
||||
|
||||
// segment_view: the header moves, the bytes stay.
|
||||
{
|
||||
const auto view = PSXRecomp::segment_view(exe, KSEG1);
|
||||
check(view.load_address() == (KSEG1 | (B & 0x1FFFFFFFu)) &&
|
||||
view.entry_point() == (KSEG1 | (B & 0x1FFFFFFFu)) &&
|
||||
view.link_segment() == KSEG1 && view.code_data == exe.code_data,
|
||||
"the KSEG1 view moves the header and keeps the bytes");
|
||||
check(view.read_word(KSEG1 | ((B + 0x18) & 0x1FFFFFFFu)) == exe.read_word(B + 0x18),
|
||||
"the view reads the image's words at its own VAs");
|
||||
}
|
||||
|
||||
// rebase_codegen_config: exact-match sites naming the image move; the
|
||||
// rest is kept as written.
|
||||
{
|
||||
PSXRecomp::CodeGenConfig cfg;
|
||||
cfg.hot_funcs = {B + 0x18, 0x80090000u};
|
||||
cfg.mod_function_entry_funcs = {B};
|
||||
cfg.ws_cull_bias_sites = {B + 0x28};
|
||||
cfg.persist_init_store_sites = {B + 0x28};
|
||||
cfg.vsync_query_hle_funcs[B + 0x20] = {1u, 2u, 3u, 4u};
|
||||
cfg.ws_bg2d_count_site = B + 0x28;
|
||||
cfg.ws_bg2d_init_func = 0;
|
||||
PSXRecompV4::WidescreenSignedBoundSite bound{};
|
||||
bound.address = B + 0x28;
|
||||
cfg.ws_signed_x_bound_sites = {bound};
|
||||
const auto v = PSXRecomp::rebase_codegen_config(cfg, exe, KSEG1);
|
||||
const uint32_t p = B & 0x1FFFFFFFu;
|
||||
check(v.hot_funcs == std::set<uint32_t>{KSEG1 | (p + 0x18), 0x80090000u},
|
||||
"an in-image site moves into the segment; an address outside the image stays");
|
||||
check(v.mod_function_entry_funcs == std::set<uint32_t>{KSEG1 | p} &&
|
||||
v.ws_cull_bias_sites == std::set<uint32_t>{KSEG1 | (p + 0x28)} &&
|
||||
v.persist_init_store_sites == std::set<uint32_t>{KSEG1 | (p + 0x28)},
|
||||
"hook and site sets move");
|
||||
check(v.vsync_query_hle_funcs.size() == 1 &&
|
||||
v.vsync_query_hle_funcs.count(KSEG1 | (p + 0x20)) &&
|
||||
v.vsync_query_hle_funcs.at(KSEG1 | (p + 0x20))[3] == 4u,
|
||||
"the VSync-query function moves; its data addresses do not");
|
||||
check(v.ws_bg2d_count_site == (KSEG1 | (p + 0x28)) && v.ws_bg2d_init_func == 0,
|
||||
"scalar sites move; an unset one stays unset");
|
||||
check(v.ws_signed_x_bound_sites.size() == 1 &&
|
||||
v.ws_signed_x_bound_sites[0].address == B + 0x28,
|
||||
"a physically matched site is kept as written");
|
||||
check(!v.split_mid_function_targets && cfg.split_mid_function_targets,
|
||||
"a variant compiles the home compile's final pieces: no split pre-pass");
|
||||
}
|
||||
|
||||
if (failures) {
|
||||
std::printf("%d check(s) failed\n", failures);
|
||||
return 1;
|
||||
}
|
||||
std::printf("PASS: segment variants plan direct-edge closures per segment, rebased\n");
|
||||
return 0;
|
||||
}
|
||||
@@ -1,6 +1,7 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Recompiler codegen regression test: a KUSEG-linked EXE compiles for KUSEG,
|
||||
and dispatch is exact (docs/SEGMENT_AWARE_CODE.md §5.3, §5.5).
|
||||
segment-qualified seeds compile segment variants, and dispatch is exact
|
||||
(docs/SEGMENT_AWARE_CODE.md §5.3, §5.4, §5.5).
|
||||
|
||||
PS1 segments alias the same physical RAM, but a PC carries its segment into
|
||||
every link value, EPC and I-cache tag. Most PS-X EXE headers carry KSEG0
|
||||
@@ -16,8 +17,19 @@ KSEG0 or KSEG1 alias ran it too. Now:
|
||||
PC, so an alias PC with no body of its own misses; psx_game_address_in_text
|
||||
stays physical, which sends that segment miss to the interpreter;
|
||||
- a seed in the link segment is an entry; a seed for the same bytes in
|
||||
another segment is reported as a variant request (§5.4), never folded into
|
||||
the home body and never silently dropped;
|
||||
another segment is a variant request (§5.4), never folded into the home
|
||||
body: the home functions it reaches through direct edges are compiled
|
||||
again in its segment, with their own names, rows and baked PCs (a cached
|
||||
variant is the home C with the segment swapped; a KSEG1 one is charged a
|
||||
fetch per instruction), and several rows then share a physical word in
|
||||
both lookup forms. A jump-table case whose value is in another segment
|
||||
than its body tail-transfers instead of continuing locally, in a variant
|
||||
and in a home compile alike. A variant closure that reaches a data stub
|
||||
compiles it with no dispatch row, and one that reaches an alias entry
|
||||
compiles its whole alias group, whose shared body every shard can call
|
||||
through the declarations header. A variant seed that names no home
|
||||
dispatch entry, or that lies in a segment that does not map physical
|
||||
memory (0x20000000-0x7FFFFFFF, KSEG2), stops the build;
|
||||
- a header whose entry and load address are in different segments is
|
||||
rejected, a KSEG1-linked image compiles as uncached (a fetch charge per
|
||||
instruction), and a config code site in a foreign segment is refused.
|
||||
@@ -126,6 +138,18 @@ def compile_and_run(compiler, harness, sources, include):
|
||||
subprocess.run([binary], check=True, cwd=tmp)
|
||||
|
||||
|
||||
def compile_only(compiler, sources, includes):
|
||||
"""Compile `sources` without linking (syntax and declarations only)."""
|
||||
if os.path.basename(compiler).lower() in ("cl", "cl.exe", "clang-cl", "clang-cl.exe"):
|
||||
command = [compiler, "/nologo", "/Zs"] + ["/I" + i for i in includes] + sources
|
||||
else:
|
||||
command = [compiler, "-std=c11", "-fsyntax-only",
|
||||
"-Werror=implicit-function-declaration"] + \
|
||||
[a for i in includes for a in ("-I", i)] + sources
|
||||
r = subprocess.run(command, capture_output=True, text=True)
|
||||
return r.returncode == 0, r.stdout + r.stderr
|
||||
|
||||
|
||||
def run(recompiler, args, cwd=ROOT):
|
||||
return subprocess.run([recompiler] + args, capture_output=True, text=True, cwd=cwd)
|
||||
|
||||
@@ -155,6 +179,50 @@ def generate(recompiler, tmp, name, entry, load, seeds, body=None):
|
||||
return dsrc, full, r.stdout
|
||||
|
||||
|
||||
def bodies_of(full):
|
||||
"""{func_ name: body text} of the emitted C."""
|
||||
return {m.group(1): m.group(2) for m in re.finditer(
|
||||
r"^void (func_[0-9A-F]{8})\(CPUState\* cpu\)\s*\{(.*?)^\}", full, re.M | re.S)}
|
||||
|
||||
|
||||
def to_segment(text, seg):
|
||||
"""Every 8-digit hex token of the image moved into `seg` (constants,
|
||||
labels and names alike)."""
|
||||
return re.sub(r"(?<![0-9A-Fa-f])([0-9A-F])([0-9A-F]{7})(?![0-9A-Fa-f])",
|
||||
lambda m: "%08X" % (seg | (int(m.group(0), 16) & PHYS_MASK))
|
||||
if (int(m.group(0), 16) & PHYS_MASK) >> 16 == (LOAD >> 16) else m.group(0),
|
||||
text)
|
||||
|
||||
|
||||
def build_switch_body(load=LOAD, case_seg=None):
|
||||
"""A bounded jump table in an image at `load`: sltiu/beq guard, lw of a
|
||||
table of case PCs in `case_seg` (default: the image's own segment), jr.
|
||||
Each case branches to one shared jr $ra."""
|
||||
if case_seg is None:
|
||||
case_seg = load & SEG_MASK
|
||||
body = w([0x3C080000 | (load >> 16), 0x25100100, 0, 0x2C620003, 0x10400010, 0,
|
||||
0x00031080, 0x00501021, 0x8C420000, 0, 0x00400008, 0,
|
||||
0x24020001, 0x10000007, 0, 0x24020002, 0x10000004, 0,
|
||||
0x24020003, 0x10000001, 0, 0x03E00008, 0])
|
||||
body += b"\x00" * (0x100 - len(body))
|
||||
phys = load & PHYS_MASK
|
||||
return body + w([case_seg | (phys + 0x30), case_seg | (phys + 0x3C), case_seg | (phys + 0x48)])
|
||||
|
||||
|
||||
def build_stub_body():
|
||||
"""A function whose jal target decodes as data: the target is a data stub
|
||||
(psx_unknown_dispatch, no dispatch row), in the home compile and in a
|
||||
variant that reaches it."""
|
||||
return w([0x27BDFFF8, 0xAFBF0004, jal(LOAD + 0x20), 0, 0x8FBF0004, 0x27BD0008,
|
||||
0x03E00008, 0, 0xFFFFFFFF, 0xFFFFFFFF, 0xFFFFFFFF, 0xFFFFFFFF])
|
||||
|
||||
|
||||
def build_alias_body(n=40):
|
||||
"""One straight-line host function with interior seeds at +0x10 and +0x20:
|
||||
an alias group (one shared psx_alias_body_ body, a wrapper per entry)."""
|
||||
return w([0x24420001] * n + [0x03E00008, 0])
|
||||
|
||||
|
||||
def rows_of(src):
|
||||
tbl = re.search(r"k_psx_game_dispatch\[\] = \{(.*?)\n\};", src, re.DOTALL)
|
||||
if not tbl:
|
||||
@@ -185,24 +253,28 @@ def main():
|
||||
check("Loaded 2 extra function addresses" in log,
|
||||
"KUSEG seeds in the link segment must be loaded (log: %r)"
|
||||
% re.findall(r"Loaded .*", log))
|
||||
check("variant seed 0x80010020 (entry, KSEG0): home PC would be 0x00010020" in log,
|
||||
"a KSEG0 seed for a KUSEG image must be reported as a variant request")
|
||||
check("variant seed 0x80010020 (entry, KSEG0): home PC would be 0x00010020" in log
|
||||
and "Segment variant 0x80000000" in log,
|
||||
"a KSEG0 seed for a KUSEG image must be compiled as a variant request")
|
||||
rows = rows_of(src)
|
||||
check(len(rows) >= 2, "expected at least two dispatch entries, got %d" % len(rows))
|
||||
check(all((a & SEG_MASK) == KUSEG and (r == 0 or (r & SEG_MASK) == KUSEG)
|
||||
for a, r, _ in rows),
|
||||
"every row key and resume PC is a KUSEG PC: %s" % rows)
|
||||
check(all((int(fn[5:], 16) & SEG_MASK) == KUSEG and (r != 0 or fn == "func_%08X" % a)
|
||||
for a, r, fn in rows),
|
||||
"func_ names are the KUSEG identity: %s" % rows)
|
||||
check(len(rows) >= 3, "expected at least three dispatch entries, got %d" % len(rows))
|
||||
check(all((a & SEG_MASK) == (int(fn[5:], 16) & SEG_MASK) and
|
||||
(r == 0 or (r & SEG_MASK) == (a & SEG_MASK)) for a, r, fn in rows),
|
||||
"every row key and resume PC is in its body's segment: %s" % rows)
|
||||
check(all(r != 0 or fn == "func_%08X" % a for a, r, fn in rows),
|
||||
"entry rows name their own body: %s" % rows)
|
||||
check([(a, fn) for a, _, fn in rows if (a & SEG_MASK) != KUSEG] ==
|
||||
[(KSEG0 | (LOAD + 0x20), "func_80010020")],
|
||||
"the only row outside KUSEG is the requested KSEG0 variant: %s" % rows)
|
||||
check({a for a, r, _ in rows} >= {LOAD, LOAD + 0x20}, "both functions have rows")
|
||||
check("void func_00010000(CPUState* cpu)" in full and
|
||||
"void func_00010020(CPUState* cpu)" in full,
|
||||
"bodies are named by their KUSEG PC (§5.1)")
|
||||
check("cpu->gpr[31] = 0x00010010u;" in full, "the jal links a KUSEG PC")
|
||||
check("psx_icache_fetch(cpu, 0x00010000u)" in full, "fetch tags are KUSEG")
|
||||
check(not re.search(r"0x8001[0-9A-F]{4}u", full),
|
||||
"no KSEG0 PC of the image appears in the KUSEG body")
|
||||
bodies = bodies_of(full)
|
||||
check({"func_00010000", "func_00010020", "func_80010020"} <= set(bodies),
|
||||
"bodies are named by their PC, the variant's included (§5.1): %s" % sorted(bodies))
|
||||
home = bodies["func_00010000"] + bodies["func_00010020"]
|
||||
check("cpu->gpr[31] = 0x00010010u;" in home, "the jal links a KUSEG PC")
|
||||
check("psx_icache_fetch(cpu, 0x00010000u)" in home, "fetch tags are KUSEG")
|
||||
check(not re.search(r"0x8001[0-9A-F]{4}u", home),
|
||||
"no KSEG0 PC of the image appears in the KUSEG bodies")
|
||||
keys = [a & PHYS_MASK for a, _, _ in rows]
|
||||
check(keys == sorted(keys), "dispatch table is sorted by physical address")
|
||||
|
||||
@@ -212,20 +284,154 @@ def main():
|
||||
body = m.group(1)
|
||||
check("0x1FFFFFFFu" in body and "k_psx_game_dispatch_index" in body,
|
||||
"small resident tables index the physical word")
|
||||
check(".addr != addr" in body, "the lookup requires the exact PC")
|
||||
check("k_psx_game_dispatch[row].addr == addr" in body,
|
||||
"a word with rows in two segments is searched for the exact PC")
|
||||
|
||||
# -- segment variants: the direct-edge closure, per segment (§5.4) ----
|
||||
vsrc, vfull, vlog = generate(
|
||||
args.recompiler, tmp, "variants", LOAD, LOAD,
|
||||
[LOAD, LOAD + 0x20, KSEG0 | LOAD, KSEG1 | LOAD, KSEG0 | (LOAD + 0x10)])
|
||||
vrows = {a: (r, fn) for a, r, fn in rows_of(vsrc)}
|
||||
for seg in (KSEG0, KSEG1):
|
||||
name = "func_%08X" % (seg | LOAD)
|
||||
want = {seg | LOAD: (0, name), seg | (LOAD + 0x10): (seg | (LOAD + 0x10), name),
|
||||
seg | (LOAD + 0x20): (0, "func_%08X" % (seg | (LOAD + 0x20)))}
|
||||
got = {a: v for a, v in vrows.items() if (a & SEG_MASK) == seg}
|
||||
check(got == want, "the 0x%08X variant is A, its call return and its jal callee B "
|
||||
"(the closure), with rows in that segment: %s" % (seg, got))
|
||||
vbodies = bodies_of(vfull)
|
||||
for fn in ("func_00010000", "func_00010020"):
|
||||
v0 = "func_%08X" % (KSEG0 | int(fn[5:], 16))
|
||||
check(to_segment(vbodies[fn], KSEG0) == vbodies[v0],
|
||||
"a cached variant is the home C with the segment swapped (%s)" % v0)
|
||||
v1 = vbodies["func_%08X" % (KSEG1 | int(fn[5:], 16))]
|
||||
insns = re.findall(r"/\* 0x([0-9A-F]{8}): ", v1)
|
||||
fetched = set(re.findall(r"psx_icache_fetch\(cpu, 0x([0-9A-F]{8})u\)", v1))
|
||||
check(insns and all(i in fetched and i.startswith("A") for i in insns),
|
||||
"every instruction of a KSEG1 variant is charged its own fetch (§5.6)")
|
||||
check(not re.search(r"0x[08]001[0-9A-F]{4}u", v1),
|
||||
"a KSEG1 variant bakes no KUSEG or KSEG0 PC of the image")
|
||||
|
||||
# A variant seed must name a home dispatch entry.
|
||||
psx = os.path.join(tmp, "variants.psx")
|
||||
bad_seeds = os.path.join(tmp, "unplaced.seeds.txt")
|
||||
with open(bad_seeds, "w") as f:
|
||||
f.write("0x%08X\n0x%08X\n" % (LOAD, KSEG0 | (LOAD + 4)))
|
||||
r = run(args.recompiler, [psx, "--seeds", bad_seeds, "--out-dir",
|
||||
os.path.join(tmp, "unplaced")])
|
||||
check(r.returncode != 0 and "variant seed 0x80010004 cannot be compiled" in r.stderr,
|
||||
"a variant seed with no home dispatch entry stops the build:\n%s"
|
||||
% (r.stderr or r.stdout)[-2000:])
|
||||
|
||||
# A jump-table case is local only in its own segment: the KSEG0
|
||||
# variant of a KUSEG switch tail-transfers every KUSEG case value.
|
||||
_, sfull, _ = generate(args.recompiler, tmp, "switch", LOAD, LOAD,
|
||||
[LOAD, KSEG0 | LOAD], body=build_switch_body())
|
||||
sbodies = bodies_of(sfull)
|
||||
check("/* jump table 0x00010100" in sbodies["func_00010000"] and
|
||||
re.search(r"case 0x00010030u:\s*\n(.*\n)*?\s*goto block_00010030;",
|
||||
sbodies["func_00010000"]),
|
||||
"the home body switches locally on its own segment's case PCs")
|
||||
v = sbodies["func_80010000"]
|
||||
check("jump table" not in v and not re.search(r"case 0x0001[0-9A-F]{4}u", v) and
|
||||
"cpu->pc = _jt_80010028; return;" in v,
|
||||
"the KSEG0 variant tail-transfers a KUSEG case value to dispatch")
|
||||
|
||||
# A home compile follows the same rule. Its tables normally hold its
|
||||
# own segment's PCs and stay local switches; a table whose values are
|
||||
# another segment's PCs of the image's bytes (a KSEG0 image whose table
|
||||
# holds KUSEG PCs) dispatches those cases with their full PC, where
|
||||
# they are segment misses, instead of continuing in the KSEG0 body.
|
||||
for case_seg, local in ((KSEG0, True), (KUSEG, False)):
|
||||
name = "switch_k0_" + ("own" if local else "kuseg")
|
||||
_, hfull, _ = generate(args.recompiler, tmp, name, KSEG0 | LOAD, KSEG0 | LOAD,
|
||||
[KSEG0 | LOAD], body=build_switch_body(KSEG0 | LOAD, case_seg))
|
||||
h = bodies_of(hfull)["func_80010000"]
|
||||
if local:
|
||||
check("/* jump table 0x80010100" in h and
|
||||
re.search(r"case 0x80010030u:\s*\n(.*\n)*?\s*goto block_80010030;", h),
|
||||
"a KSEG0 home body switches locally on its own segment's case PCs")
|
||||
else:
|
||||
check("jump table" not in h and not re.search(r"case 0x0001[0-9A-F]{4}u", h) and
|
||||
"cpu->pc = _jt_80010028; return;" in h,
|
||||
"a KSEG0 home body tail-transfers a KUSEG case value to dispatch")
|
||||
|
||||
# Only KUSEG below 0x20000000, KSEG0 and KSEG1 map the image's bytes. A
|
||||
# seed in 0x20000000-0x7FFFFFFF or in KSEG2 names no code of the image:
|
||||
# it stops the build with the link-segment spelling of its bytes.
|
||||
for bad in (0x20000000 | LOAD, 0xC0000000 | LOAD):
|
||||
bad_seeds = os.path.join(tmp, "badseg.seeds.txt")
|
||||
with open(bad_seeds, "w") as f:
|
||||
f.write("0x%08X\n0x%08X\n" % (LOAD, bad))
|
||||
r = run(args.recompiler, [psx, "--seeds", bad_seeds, "--out-dir",
|
||||
os.path.join(tmp, "badseg")])
|
||||
check(r.returncode != 0 and
|
||||
("seed 0x%08X is in segment 0x%08X, which does not map physical memory"
|
||||
% (bad, bad & SEG_MASK)) in r.stderr and
|
||||
("are 0x%08X in its link segment" % LOAD) in r.stderr and
|
||||
"Segment variant" not in r.stdout,
|
||||
"a seed in a segment that does not map physical memory stops the build:\n%s"
|
||||
% (r.stderr or r.stdout)[-2000:])
|
||||
|
||||
# A variant closure that reaches a data stub compiles the stub in its
|
||||
# segment, and the stub gets no dispatch row, as at home.
|
||||
dsrc, dfull, _ = generate(args.recompiler, tmp, "stub", LOAD, LOAD, [LOAD, KSEG0 | LOAD],
|
||||
body=build_stub_body())
|
||||
drows = {a for a, _, _ in rows_of(dsrc)}
|
||||
check("psx_unknown_dispatch(cpu, 0x80010020u, 0x00010020u);" in
|
||||
bodies_of(dfull).get("func_80010020", ""),
|
||||
"the variant's jal target is compiled as its data stub")
|
||||
check((KSEG0 | LOAD) in drows and not drows & {LOAD + 0x20, KSEG0 | (LOAD + 0x20)},
|
||||
"a data stub has no dispatch row, at home or in a variant: %s"
|
||||
% sorted("%08X" % a for a in drows))
|
||||
|
||||
# A variant seed at an alias entry compiles its alias group: one shared
|
||||
# psx_alias_body_ in the variant's segment and a wrapper per entry.
|
||||
# The shards share one declarations header; every alias body a shard
|
||||
# defines is declared there, so a wrapper in another shard can call it
|
||||
# (compiled below against the header alone), and every emitted file
|
||||
# compiles.
|
||||
_, afull, _ = generate(args.recompiler, tmp, "alias", LOAD, LOAD,
|
||||
[LOAD, LOAD + 0x10, LOAD + 0x20, KSEG0 | (LOAD + 0x10)],
|
||||
body=build_alias_body())
|
||||
abodies = bodies_of(afull)
|
||||
check("psx_alias_body_80010000(cpu, 0x80010010u);" in abodies.get("func_80010010", "") and
|
||||
"psx_alias_body_80010000(cpu, 0x80010020u);" in abodies.get("func_80010020", ""),
|
||||
"an alias entry's variant brings its whole alias group: %s" % sorted(abodies))
|
||||
adir = os.path.join(tmp, "alias")
|
||||
hosts = sorted(set(re.findall(r"^void (psx_alias_body_[0-9A-F]{8})\(", afull, re.M)))
|
||||
check(hosts == ["psx_alias_body_00010000", "psx_alias_body_80010000"],
|
||||
"the home and the variant alias bodies are emitted: %s" % hosts)
|
||||
if args.compiler:
|
||||
probe = os.path.join(tmp, "alias_decls_probe.c")
|
||||
with open(probe, "w") as f:
|
||||
f.write('#include "alias.psx_decls.h"\n'
|
||||
"void (*const alias_bodies_probe[])(CPUState*, uint32_t) = {%s};\n"
|
||||
% ", ".join(hosts))
|
||||
sources = [probe] + sorted(os.path.join(adir, n) for n in os.listdir(adir)
|
||||
if n.endswith(".c"))
|
||||
ok, out = compile_only(args.compiler, sources, [adir, os.path.join(ROOT, "runtime", "include")])
|
||||
check(ok, "every alias body is declared in the shared header and the variant "
|
||||
"output compiles:\n" + out[-3000:])
|
||||
|
||||
# -- a table spanning 2 MiB or more takes the binary-search form ------
|
||||
wide_src, _, _ = generate(args.recompiler, tmp, "kuseg_wide", LOAD, LOAD,
|
||||
[LOAD, LOAD + WIDE_SPAN], body=build_wide_body())
|
||||
[LOAD, LOAD + WIDE_SPAN, KSEG1 | (LOAD + WIDE_SPAN)],
|
||||
body=build_wide_body())
|
||||
wide_rows = rows_of(wide_src)
|
||||
check({LOAD, LOAD + WIDE_SPAN} <= {a for a, _, _ in wide_rows} and
|
||||
all((a & SEG_MASK) == KUSEG for a, _, _ in wide_rows),
|
||||
"the wide KUSEG image has KUSEG rows at both ends: %s" % wide_rows)
|
||||
check({LOAD, LOAD + WIDE_SPAN, KSEG1 | (LOAD + WIDE_SPAN)} <=
|
||||
{a for a, _, _ in wide_rows} and
|
||||
all((a & SEG_MASK) == KUSEG for a, _, _ in wide_rows
|
||||
if a != KSEG1 | (LOAD + WIDE_SPAN)),
|
||||
"the wide KUSEG image has KUSEG rows at both ends and one KSEG1 variant: %s"
|
||||
% wide_rows)
|
||||
wm = re.search(r"static const PsxGameDispatchEntry\* psx_game_find_entry"
|
||||
r"\(uint32_t addr\) \{(.*?)\n\}", wide_src, re.DOTALL)
|
||||
check(wm and "k_psx_game_dispatch_index" not in wm.group(1) and
|
||||
"while (lo < hi)" in wm.group(1),
|
||||
"a table spanning 2 MiB or more binary-searches the physical word")
|
||||
"while (lo < hi)" in wm.group(1) and
|
||||
"k_psx_game_dispatch[row].addr == addr" in wm.group(1),
|
||||
"a table spanning 2 MiB or more binary-searches the physical word, "
|
||||
"then the exact PC among the word's rows")
|
||||
|
||||
# -- JAL-driven discovery takes the call's segment, not KSEG0 --------
|
||||
jsrc, _, jlog = generate(args.recompiler, tmp, "kuseg_jal", LOAD, LOAD, [],
|
||||
@@ -272,6 +478,16 @@ def main():
|
||||
if ok:
|
||||
check(r.returncode == 0, "a link-segment config site is accepted:\n%s"
|
||||
% (r.stderr or r.stdout)[-2000:])
|
||||
# The seeds ask for a KSEG0 variant of that function: a config
|
||||
# code site applies to every segment compiled for its bytes.
|
||||
hot = ""
|
||||
for n in os.listdir(os.path.join(tmp, "cfg_%08X" % site)):
|
||||
if n.endswith(".c"):
|
||||
with open(os.path.join(tmp, "cfg_%08X" % site, n)) as f:
|
||||
hot += f.read()
|
||||
for fn in ("func_00010020", "func_80010020"):
|
||||
check("__attribute__((hot))\n#endif\nvoid %s(CPUState* cpu)" % fn in hot,
|
||||
"the link-segment hot_funcs site reaches %s" % fn)
|
||||
else:
|
||||
check(r.returncode != 0 and
|
||||
"recompiler.hot_funcs: 0x80010020 -> 0x00010020" in r.stderr,
|
||||
@@ -402,12 +618,14 @@ int main(void) {
|
||||
expected = &k_psx_game_dispatch[i];
|
||||
assert(psx_game_find_entry(addr) == expected);
|
||||
assert(psx_game_is_function_entry(addr) == (expected != 0));
|
||||
if (segments[s] != 0x00000000u) assert(!expected);
|
||||
/* Outside KUSEG only the requested variant has a body. */
|
||||
if (segments[s] != 0x00000000u) assert(!expected || addr == 0x80010020u);
|
||||
if (addr == 0x80010020u) assert(expected && expected->addr == addr);
|
||||
int has_home = 0;
|
||||
for (unsigned i = 0; i < PSX_GAME_DISPATCH_COUNT; ++i)
|
||||
if ((k_psx_game_dispatch[i].addr & 0x1FFFFFFFu) == phys) has_home = 1;
|
||||
if (has_home) assert(psx_game_address_in_text(addr));
|
||||
if (has_home && segments[s] != 0x00000000u) {
|
||||
if (has_home && !expected) {
|
||||
int old_calls = calls, old_irqs = irqs;
|
||||
cpu.pc = 0xDEADBEEFu;
|
||||
assert(!psx_game_text_native_ok(addr));
|
||||
@@ -448,7 +666,7 @@ int main(void) {
|
||||
assert(!psx_game_find_entry(0xFFFFFFFFu));
|
||||
assert(!psx_game_find_entry(0));
|
||||
}
|
||||
assert(home_hits >= 4); /* both functions, both geometries */
|
||||
assert(home_hits >= 6); /* both functions and the variant, both geometries */
|
||||
return 0;
|
||||
}
|
||||
"""
|
||||
@@ -484,6 +702,8 @@ int main(void) {
|
||||
assert(PSX_GAME_DISPATCH_COUNT >= 2);
|
||||
for (unsigned i = 0; i < PSX_GAME_DISPATCH_COUNT; ++i) {
|
||||
const uint32_t phys = k_psx_game_dispatch[i].addr & 0x1FFFFFFFu;
|
||||
/* Each word once: a variant's row follows its home row's word. */
|
||||
if (i && (k_psx_game_dispatch[i - 1].addr & 0x1FFFFFFFu) == phys) continue;
|
||||
for (unsigned s = 0; s < 3; ++s) {
|
||||
for (int d = -4; d <= 4; d += 4) {
|
||||
const uint32_t addr = segments[s] | (phys + (uint32_t)d);
|
||||
@@ -492,7 +712,9 @@ int main(void) {
|
||||
if (k_psx_game_dispatch[j].addr == addr) expected = &k_psx_game_dispatch[j];
|
||||
assert(psx_game_find_entry(addr) == expected);
|
||||
assert(psx_game_is_function_entry(addr) == (expected != 0));
|
||||
if (segments[s] != 0x00000000u) assert(!expected);
|
||||
/* Outside KUSEG only the requested KSEG1 variant has a body. */
|
||||
if (segments[s] != 0x00000000u)
|
||||
assert(!expected || addr == (0xA0000000u | (0x00010000u + 0x200000u)));
|
||||
const int before = calls;
|
||||
assert(psx_dispatch_game_compiled(&cpu, addr) == (expected != 0));
|
||||
assert(calls == before + (expected != 0));
|
||||
@@ -506,11 +728,15 @@ int main(void) {
|
||||
"""
|
||||
compile_and_run(args.compiler, wide, [], include)
|
||||
|
||||
print("KUSEG-linked EXE test passed (%d KUSEG rows, exact lookup in the indexed and "
|
||||
"binary-search forms, aliases miss, variant seeds reported, KSEG1 image charged "
|
||||
"per instruction, foreign config sites and directives refused, physically "
|
||||
"matched sites accepted, mirror PCs never run compiled code in 2 or 8 MiB "
|
||||
"geometry)" % len(rows))
|
||||
print("KUSEG-linked EXE test passed (%d rows, exact lookup in the indexed and "
|
||||
"binary-search forms with variant rows, unrequested aliases miss, variants "
|
||||
"compiled as their direct-edge closure, cached variants equal the home C, KSEG1 "
|
||||
"variants and images charged per instruction, foreign jump-table cases "
|
||||
"dispatched at home and in variants, data stubs rowless, variant alias groups "
|
||||
"declared for every shard, seeds outside the physical segments refused, foreign "
|
||||
"config sites and directives refused, physically matched "
|
||||
"sites accepted, mirror PCs never run compiled code in 2 or 8 MiB geometry)"
|
||||
% len(rows))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -45,10 +45,7 @@ SEG_MASK, PHYS_MASK = 0xE0000000, 0x1FFFFFFF
|
||||
|
||||
# id -> (design section, what is missing). Remove an entry in the change
|
||||
# that closes it; the test fails until the ledger matches.
|
||||
KNOWN_GAPS = {
|
||||
"segment-variants": ("5.4", "a seeded KSEG1 entry gets no body of its own"),
|
||||
"kseg1-fetch-charge": ("5.6", "compiled KSEG1 code does not charge +4 per fetch"),
|
||||
}
|
||||
KNOWN_GAPS = {}
|
||||
|
||||
# Closed ids that stay as regression guards: each reports a new gap if the
|
||||
# property breaks again. id -> design section.
|
||||
@@ -66,6 +63,10 @@ REGRESSION_GUARDS = {
|
||||
"home-seed-accepted": "5.3",
|
||||
"alias-fetch-coherence": "5.3",
|
||||
"segment-miss": "5.5",
|
||||
# PR D: segment-qualified seeds compile per-segment variants, and KSEG1
|
||||
# variants charge a fetch per instruction.
|
||||
"segment-variants": "5.4",
|
||||
"kseg1-fetch-charge": "5.6",
|
||||
}
|
||||
|
||||
|
||||
@@ -431,6 +432,19 @@ def main():
|
||||
gaps.add("segment-variants")
|
||||
notes["segment-variants"] = ", ".join(
|
||||
"0x%08X->%s" % (q, "0x%08X" % found[q][0] if found[q] else "miss") for q in bad)
|
||||
else:
|
||||
# A variant body is its segment's code identity: every PC it bakes
|
||||
# (links, fetch tags, IRQ resume PCs, exits and continuation keys,
|
||||
# store PCs) is in that segment, and so is its name.
|
||||
foreign = sorted({pc for q in variant_q for gid, pattern in PC_SITES.items()
|
||||
for pc in pc_constants(bodies[found[q][2]], pattern)
|
||||
if (pc & SEG_MASK) != (q & SEG_MASK)} |
|
||||
{int(found[q][2][5:], 16) for q in variant_q
|
||||
if (int(found[q][2][5:], 16) & SEG_MASK) != (q & SEG_MASK)})
|
||||
if foreign:
|
||||
gaps.add("segment-variants")
|
||||
notes["segment-variants"] = "%d PCs baked in another segment, e.g. 0x%08X" % (
|
||||
len(foreign), foreign[0])
|
||||
wrong_home = [q for q in home_q if not found[q] or found[q][0] != q]
|
||||
stray = [q for q in miss_q if found[q]]
|
||||
if stray or wrong_home:
|
||||
@@ -445,9 +459,11 @@ def main():
|
||||
if (pc & SEG_MASK) != link})
|
||||
if gid == "resume-pc-segment":
|
||||
# The dispatch rows' keys and resume PCs are PCs too (§5.2):
|
||||
# dispatch sets cpu->pc = resume_pc before entering a body.
|
||||
off = sorted(set(off) | {pc for a, r, _ in rows for pc in (a, r)
|
||||
if pc and (pc & SEG_MASK) != link})
|
||||
# dispatch sets cpu->pc = resume_pc before entering a body, so
|
||||
# each row's key and resume PC are in the segment of the body
|
||||
# it enters (the home body's, or a variant's, §5.4).
|
||||
off = sorted(set(off) | {pc for a, r, fn in rows for pc in (a, r)
|
||||
if pc and (pc & SEG_MASK) != (int(fn[5:], 16) & SEG_MASK)})
|
||||
if off:
|
||||
gaps.add(gid)
|
||||
notes[gid] = "%d constants, e.g. 0x%08X" % (len(off), off[0])
|
||||
@@ -506,6 +522,7 @@ def main():
|
||||
run_recompiler(args.bios_recompiler, ["--config", openbios_toml, "--out-dir", tmp])
|
||||
with open(os.path.join(tmp, "OpenBIOS_full.c")) as f:
|
||||
bios_c = f.read()
|
||||
|
||||
total, uncharged = bios_uncharged_kseg1(bios_c)
|
||||
if total == 0:
|
||||
model_fail.append("OpenBIOS: no KSEG1 instructions found (parser drift?)")
|
||||
|
||||
@@ -6,10 +6,11 @@ A seed is a PC, so it carries a segment (docs/SEGMENT_AWARE_CODE.md §5.3,
|
||||
§5.4). dirty_ram_stats keys its rows by physical address, so the tool writes
|
||||
each one in the title's link segment: the segment of its EXE's load address
|
||||
(KSEG0 0x8001xxxx for most titles, KUSEG 0x0001xxxx for Kula World or Alien
|
||||
Resurrection). The recompiler reports a seed in any other segment as a
|
||||
variant request and does not compile it as an entry. Segment misses (TCP
|
||||
`segment_misses`: static text entered in a segment with no body of its own)
|
||||
are written with their full PC; each is a variant request (§5.4).
|
||||
Resurrection). The recompiler takes a seed in any other segment as a variant
|
||||
request, not an entry: it compiles the home body's direct-edge closure again
|
||||
in that segment (§5.4). Segment misses (TCP `segment_misses`: static text
|
||||
entered in a segment with no body of its own) are written with their full PC,
|
||||
so the next regeneration compiles a variant for each.
|
||||
|
||||
The link segment comes from --segment, from --game-toml ([game].load_address,
|
||||
or the header of its [game].exe), or from game_text_lo when that is written as
|
||||
@@ -41,6 +42,13 @@ SEG_MASK = 0xE0000000
|
||||
SEGMENTS = {"kuseg": 0x00000000, "kseg0": 0x80000000, "kseg1": 0xA0000000}
|
||||
|
||||
|
||||
def maps_physical(pc):
|
||||
"""KUSEG below 0x20000000, KSEG0 and KSEG1 map physical memory; a PC
|
||||
anywhere else addresses no RAM, so it names no code of the image, and the
|
||||
recompiler refuses it as a seed."""
|
||||
return pc < 0x20000000 or (pc & SEG_MASK) in (0x80000000, 0xA0000000)
|
||||
|
||||
|
||||
def parse_segment(text):
|
||||
key = text.strip().lower()
|
||||
if key in SEGMENTS:
|
||||
@@ -145,6 +153,10 @@ def main(argv=None):
|
||||
if sm.get("ok"):
|
||||
for row in sm.get("summary", []):
|
||||
vaddr = int(row["pc"], 16)
|
||||
if not maps_physical(vaddr):
|
||||
print("warning: segment miss 0x%08X is in a segment that does not map "
|
||||
"physical memory; not a seed (the recompiler refuses it)" % vaddr)
|
||||
continue
|
||||
if lo_phys <= (vaddr & PHYS_MASK) < hi_phys and vaddr not in known:
|
||||
new_entries.append((vaddr, row["count"],
|
||||
"segment miss: %d hits, home %s (variant request)"
|
||||
|
||||
@@ -8,7 +8,9 @@ every dirty_ram miss as KSEG0 could never close a KUSEG-linked title's misses
|
||||
(Kula World, Alien Resurrection). dirty_ram_stats rows are physical; the tool
|
||||
takes the segment from --segment, --game-toml or a segmented game_text_lo,
|
||||
refuses to guess from a physical one, and writes `segment_misses` rows with
|
||||
their full PC.
|
||||
their full PC. A row in a segment that does not map physical memory
|
||||
(0x20000000-0x7FFFFFFF, KSEG2) is reported and not written: the recompiler
|
||||
refuses such a seed.
|
||||
|
||||
A fake debug server answers the two commands the tool sends. Its
|
||||
`segment_misses` rows use the format string of the runtime's handler, read
|
||||
@@ -34,7 +36,9 @@ PER_PC = [ # physical, as dirty_ram_stats keys them
|
||||
{"pc": "0x00010200", "hits": 3, "insns": 12},
|
||||
{"pc": "0x000C0000", "hits": 5, "insns": 7}, # outside the text range
|
||||
]
|
||||
SEGMENT_MISSES = [(0x800100F0, 0x000100F0, 4), (0xA0010124, 0x00010124, 1)]
|
||||
SEGMENT_MISSES = [(0x800100F0, 0x000100F0, 4), (0xA0010124, 0x00010124, 1),
|
||||
# 0x20000000-0x7FFFFFFF does not map physical memory: no seed.
|
||||
(0x200100F0, 0x000100F0, 1)]
|
||||
|
||||
|
||||
class FakeDebugServer(threading.Thread):
|
||||
@@ -115,6 +119,9 @@ def main():
|
||||
text = f.read()
|
||||
check("home 0x000100F0 (variant request)" in text,
|
||||
"a segment miss is written as a variant request: %r" % text)
|
||||
check("segment miss 0x200100F0 is in a segment that does not map" in r.stdout,
|
||||
"a segment miss outside the physical segments is reported, not "
|
||||
"written: %r" % r.stdout)
|
||||
# A second run appends nothing: known seeds are exact PCs.
|
||||
r = run_tool(server.port, out, "--segment", "kuseg")
|
||||
check(r.returncode == 0 and "No new addresses" in r.stdout and
|
||||
|
||||
Reference in New Issue
Block a user