runtime: [runtime] overlay_region_floor config key; bundled toolchain drives gcc when available

Two shipping gaps left Gran Turismo's gameplay and FMV code on the
interpreter (0.55-0.64x real time) even though the recompiled boot EXE was
fine:

1. The overlay region floor defaults to the boot EXE text end. GT's secondary
   EXEs (GTOS/GTMENU/GTMAIN/GTEND) all load at 0x80010000, inside that range,
   so they were never overlay-cache eligible. The only way to lower the floor
   was the PSX_OVERLAY_REGION_FLOOR env var, which a shipped package cannot
   carry. Add `[runtime] overlay_region_floor` (hex string or integer), applied
   before the env override with the same kernel-window clamp.

2. The bundled overlay_toolchain/ was only used to build the tcc command.
   A dev/production box with gcc on PATH but no game.toml autocompile command
   got no compiler at all. Treat the bundle as a gcc toolchain provider too:
   gcc counts as available when the bundle is present and a gcc/cc/clang is
   on PATH, and the same bundle then drives compile_overlays.py with
   --compiler gcc. One package yields gcc shards on a dev box and tcc shards
   on a toolchain-less player box.

Measured on GT1 (SCUS-94194), headless boot -> intro FMV -> attract, with the
floor at 0x10000 and gcc autocompile: 1.27-1.38x real time vs 0.55-0.64x;
SPU production 27 kHz -> 59 kHz (the FMV audio distortion was starvation).

(cherry picked from commit 0d226e5069bdf1395c819110beec40367a44a699)
This commit is contained in:
Alexandros Mandravillis
2026-08-26 15:26:28 +03:00
parent 01c647e8a2
commit d596b0ec46
6 changed files with 107 additions and 31 deletions
+5
View File
@@ -330,6 +330,11 @@ The other load-time accelerators are likewise opt-in:
[runtime]
idle_skip = true
turbo_audio_sink = true
overlay_region_floor = "0x10000" # optional: lowest RAM address treated as overlay region.
# Default = boot EXE text end. Lower it for titles whose gameplay
# code loads at/inside the boot text range (GT1 secondary EXEs at
# 0x80010000, Driver 2 mission pages) so it is overlay-cache
# eligible. Clamped >= 0x10000; PSX_OVERLAY_REGION_FLOOR overrides.
```
### `turbo_loads` / `offer_turbo_loads` — deprecated and ignored
+7
View File
@@ -501,6 +501,13 @@ static RuntimeConfig parse_runtime_block(const toml::value& cfg, const fs::path&
}
if (runtime.contains("overlay_backend")) {
rt.overlay_backend = toml::find<std::string>(runtime, "overlay_backend");
}
if (runtime.contains("overlay_region_floor")) {
const auto& v = toml::find(runtime, "overlay_region_floor");
rt.overlay_region_floor = v.is_string()
? parse_hex(v.as_string(), "runtime.overlay_region_floor")
: (uint32_t)v.as_integer();
rt.has_overlay_region_floor = true;
}
if (runtime.contains("overlay_native_block")) {
for (const auto& a : toml::find<std::vector<std::string>>(runtime, "overlay_native_block")) {
+10
View File
@@ -347,6 +347,16 @@ struct RuntimeConfig {
// gcc shards still load). The env var PSX_OVERLAY_BACKEND overrides at runtime.
std::string overlay_backend;
// overlay_region_floor: per-title override of the overlay region floor
// (defaults to the boot EXE text end). Titles whose gameplay code loads
// at/inside the boot text range (Gran Turismo's secondary EXEs all load
// at 0x80010000; Driver 2 streams mission code over its text pages) need
// it lowered so that code is overlay-cache eligible instead of falling to
// single-instruction interpretation. Same semantics and clamp as the
// PSX_OVERLAY_REGION_FLOOR env override, which still takes precedence.
bool has_overlay_region_floor = false;
uint32_t overlay_region_floor = 0;
// overlay_native_block: per-game overlay function entries that must stay on
// the dirty-RAM interpreter even when a matching native DLL exists. Intended
// for small timing-sensitive setup/task routines while the rest of the
+3
View File
@@ -458,6 +458,9 @@ if(BUILD_TESTING)
add_test(NAME savestate_status_protocol_test
COMMAND ${Python3_EXECUTABLE}
${CMAKE_CURRENT_SOURCE_DIR}/tests/test_savestate_status_protocol.py)
add_test(NAME overlay_region_floor_config_test
COMMAND ${Python3_EXECUTABLE}
${CMAKE_CURRENT_SOURCE_DIR}/tests/test_overlay_region_floor_config.py)
add_test(NAME screenshot_hires_pitch_test
COMMAND ${Python3_EXECUTABLE}
${CMAKE_CURRENT_SOURCE_DIR}/tests/test_screenshot_hires_pitch.py)
+54 -31
View File
@@ -11085,6 +11085,10 @@ int main(int argc, char** argv) {
* chaining and makes them overlay-cache candidates, so live-byte
* closures can own them instead of single-instruction dispatch
* thrash. Clamped to stay above the kernel window. */
if (gc.runtime.has_overlay_region_floor) {
uint32_t v = gc.runtime.overlay_region_floor & 0x1FFFFFFFu;
if (v >= 0x00010000u) g_overlay_region_floor = v;
}
{
const char* fenv = std::getenv("PSX_OVERLAY_REGION_FLOOR");
if (fenv && fenv[0]) {
@@ -11501,49 +11505,68 @@ int main(int argc, char** argv) {
overlay_autocapture_set_enabled(0);
const char *cfg_backend = deferred_overlay_backend.empty()
? nullptr : deferred_overlay_backend.c_str();
int gcc_avail = deferred_has_overlay_ac
/* The bundled overlay_toolchain/ (embedded Python + compile_overlays.py +
* recompiler + headers + tcc) can drive EITHER compiler: tcc from the
* bundle, or a real gcc when one is on PATH. So a gcc toolchain counts
* as available when there is a configured gcc command OR the bundle is
* present to build one - the same package then yields gcc shards on a
* dev box and tcc shards on a toolchain-less player box. */
extern int g_psx_cps_mode;
const std::filesystem::path tk_xd = exe_dir_from_argv(argv[0]);
const std::filesystem::path tk_dir = tk_xd / "overlay_toolchain";
const std::filesystem::path tk_py = tk_dir / "python" / "python.exe";
const bool tk_present = std::filesystem::exists(tk_py);
auto build_toolchain_cmd = [&](const char *compiler) {
auto cmd_quote = [](const std::string& s) {
return std::string("\"") + s + "\"";
};
std::string c =
cmd_quote(tk_py.string()) + " " +
cmd_quote((tk_dir / "compile_overlays.py").string()) +
" --captures " + cmd_quote(captures_path.string()) +
" --game-toml " + cmd_quote(std::string(
game_config_path ? game_config_path : "game.toml")) +
" --recompiler " + cmd_quote((tk_dir / "psxrecomp-game.exe").string()) +
" --runtime-include " + cmd_quote((tk_dir / "include").string()) +
" --out-dir " + cmd_quote((tk_xd / "cache").string()) +
(g_psx_cps_mode ? " --cps" : "") +
" --compiler " + compiler;
if (std::string(compiler) == "tcc")
c += " --tcc " + cmd_quote((tk_dir / "tcc" / "tcc.exe").string());
return c;
};
int gcc_avail = (deferred_has_overlay_ac || tk_present)
&& autocompile_toolchain_available();
OverlayBackend eff = overlay_backend_resolve(cfg_backend, gcc_avail);
std::string built_tcc_cmd;
std::string built_gcc_cmd;
std::string env_ac_cmd;
const std::string *ac_cmd = nullptr;
if (eff == OVERLAY_BACKEND_TCC) {
if (deferred_has_overlay_ac_tcc) {
ac_cmd = &deferred_overlay_ac_tcc;
} else if (tk_present) {
built_tcc_cmd = build_toolchain_cmd("tcc");
ac_cmd = &built_tcc_cmd;
std::fprintf(stdout,
"psxrecomp: tcc tier using bundled toolchain (%s)\n",
tk_dir.string().c_str());
} else {
extern int g_psx_cps_mode;
std::filesystem::path xd = exe_dir_from_argv(argv[0]);
std::filesystem::path tk = xd / "overlay_toolchain";
std::filesystem::path py = tk / "python" / "python.exe";
if (std::filesystem::exists(py)) {
auto cmd_quote = [](const std::string& s) {
return std::string("\"") + s + "\"";
};
built_tcc_cmd =
cmd_quote(py.string()) + " " +
cmd_quote((tk / "compile_overlays.py").string()) +
" --captures " + cmd_quote(captures_path.string()) +
" --game-toml " + cmd_quote(std::string(
game_config_path ? game_config_path : "game.toml")) +
" --recompiler " + cmd_quote((tk / "psxrecomp-game.exe").string()) +
" --runtime-include " + cmd_quote((tk / "include").string()) +
" --out-dir " + cmd_quote((xd / "cache").string()) +
(g_psx_cps_mode ? " --cps" : "") +
" --compiler tcc --tcc " +
cmd_quote((tk / "tcc" / "tcc.exe").string());
ac_cmd = &built_tcc_cmd;
std::fprintf(stdout,
"psxrecomp: tcc tier using bundled toolchain (%s)\n",
tk.string().c_str());
} else {
std::fprintf(stdout,
"psxrecomp: tcc tier active but no bundled toolchain at %s "
"(overlay gaps -> interpreter)\n", tk.string().c_str());
}
std::fprintf(stdout,
"psxrecomp: tcc tier active but no bundled toolchain at %s "
"(overlay gaps -> interpreter)\n", tk_dir.string().c_str());
}
} else {
if (deferred_has_overlay_ac)
if (deferred_has_overlay_ac) {
ac_cmd = &deferred_overlay_ac;
} else if (tk_present) {
/* gcc on PATH + bundled toolchain: gcc shards from the same bundle. */
built_gcc_cmd = build_toolchain_cmd("gcc");
ac_cmd = &built_gcc_cmd;
std::fprintf(stdout,
"psxrecomp: gcc tier using bundled toolchain (%s) with gcc from PATH\n",
tk_dir.string().c_str());
}
}
if (const char *e = std::getenv("PSX_OVERLAY_AUTOCOMPILE_CMD")) {
if (e[0]) {
@@ -0,0 +1,28 @@
#!/usr/bin/env python3
"""Structural guards for the [runtime] overlay_region_floor key and the bundled-toolchain gcc path.
- config_loader parses `overlay_region_floor` (hex string or integer) into the runtime config;
- main.cpp applies it before the PSX_OVERLAY_REGION_FLOOR env override, with the same kernel-window clamp;
- the bundled overlay_toolchain/ can drive gcc (not only tcc) when a gcc toolchain is on PATH.
"""
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
LOADER_H = (ROOT / "recompiler/src/config_loader.h").read_text(encoding="utf-8")
LOADER_CPP = (ROOT / "recompiler/src/config_loader.cpp").read_text(encoding="utf-8")
MAIN = (ROOT / "runtime/src/main.cpp").read_text(encoding="utf-8")
assert "has_overlay_region_floor" in LOADER_H and "overlay_region_floor" in LOADER_H
assert 'runtime.contains("overlay_region_floor")' in LOADER_CPP
assert 'parse_hex(v.as_string(), "runtime.overlay_region_floor")' in LOADER_CPP, "hex string form must be accepted"
cfg = MAIN.index("gc.runtime.has_overlay_region_floor")
env = MAIN.index('std::getenv("PSX_OVERLAY_REGION_FLOOR")')
assert cfg < env, "config key must be applied before the env override so env still wins"
window = MAIN[cfg:cfg + 300]
assert "0x00010000u" in window, "config floor must keep the kernel-window clamp"
assert "build_toolchain_cmd(\"gcc\")" in MAIN, "bundled toolchain must be able to drive gcc"
assert "build_toolchain_cmd(\"tcc\")" in MAIN
assert "(deferred_has_overlay_ac || tk_present)" in MAIN, "gcc availability must count the bundle"
print("overlay_region_floor config + bundled-gcc guards: OK")