Conflict: recompiler/CMakeLists.txt. Master gained #134's test registrations
in the same block where #133 added stall_report. Resolved as the union of both.
Also RE-ENABLES aot_overlay_discovery, which is not part of the conflict region
and so would otherwise have survived the merge untouched. #134 registered it
DISABLED with this reason:
"fails in check_real_batched_fragment_publication: that check reaches
MOD.compile_fragment_batch, which spawns psxrecomp-game with cwd set to its
own TemporaryDirectory, so BIOS-profile resolution finds nothing ... it
needs an owner, and DISABLED keeps it counted until then."
This branch is that owner -- resolving BIOS-profile lookup against
--project-root instead of the CWD is exactly that fix. Landing it while leaving
the test disabled would ship the fix with its own proof switched off, which is
the silent-rot class #134 exists to prevent.
Measured on this tree, msys64 toolchain driven from PowerShell:
- pristine origin/master: test_aot_overlay_discovery.py exits 1 at
check_real_batched_fragment_publication, "no BIOS profile found"
- this merge: ALL PASS, and ctest -R aot_overlay_discovery passes in 7.64s
with the DISABLED property removed
- baked codegen hash 0xfbb44d49u on BOTH arms -- the cache tag does not move,
so existing overlay caches keep loading
- full ctest failure set identical to pristine master, no new failures
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Load acceleration moved to the Mods catalog in 125d84d
(psx.enhancement.fast-loading "Fast Loading (host pacing)" and
psx.enhancement.cd-speed, both `game_id = "*"` so they ship with every
title, both default-off), and recomp-ui deliberately stopped drawing a
generic Turbo loads row. But the legacy switch was left live here with
`offer_turbo_loads` defaulting TRUE, so any title that had not explicitly
migrated still got turbo forced on — with no UI to turn it back off.
That shipped to players. MegaManX6Recomp v1.0.4 and v1.0.5 both carry
`[runtime] turbo_loads = true` in the game.toml inside the release zip
(verified by extracting the artifacts). Two paths forced it on:
* game.toml [runtime] turbo_loads set g_turbo_loads_enabled directly;
* a stale [video] turbo_loads in settings.toml was restored, and the
seed write persisted it — so one run latched the value permanently
and no later game.toml change could undo it.
Reported as MegaManX6Recomp#14: the WARNING screen self-advances and load
spinners run fast, with Turbo confirmed off in Mods. Tomba 2 hit the same
defect (fixed per-title), and MMX5 is also unmigrated. Third occurrence of
one config default, so fix the class instead of the titles.
Both keys are still PARSED, so existing configs load without error, but
neither is honoured:
* offer_turbo_loads defaults false and turbo_loads_offered becomes a
constexpr false, matching how ws_offered / skip_fmv_offered /
frame_interpolation_offered already model mod-owned features. No title
can offer the generic switch, so forgetting to migrate is no longer
punished.
* game.toml turbo_loads logs a deprecation line naming the Fast Loading
mod and enables nothing.
* settings.toml [video] turbo_loads is neither restored nor written back
out, so the stale row is dropped on the first save — same write-only
latch reasoning already documented for fast_boot, which bit harder here
because there was no UI control at all.
Also fixes a related hole found while auditing: the in-game Settings apply
path wrote both g_turbo_loads_enabled and g_auto_skip_fmv from a launcher
snapshot taken BEFORE mod_runtime_activate_plugins() ran, so the first
in-game Apply silently turned a player's enabled Fast Loading or Tweaks FMV
feature back off. Both are now gated on their offered flags.
The supported surfaces are unchanged: the Fast Loading / CD Speed mod
plugins still enable acceleration (verified: "mod selected 8x load
acceleration"), and the `turbo_loads` TCP debug command still toggles it
for development.
Verified against MMX6 built on this framework: with game.toml
`turbo_loads = true` AND `offer_turbo_loads = true` AND a stale
settings.toml `turbo_loads = true`, both deprecation lines fire and the
debug server reports enabled=0, turbo_frames=0. With the keys removed and
no mods enabled, it boots silently with enabled=0.
test_mod_load_acceleration.py is updated to pin the new contract; it also
had a stale assertion on the old 2..16 multiplier range, which is now
pinned to the symbolic PSX_MOD_LOAD_ACCEL_MAX bound. The guard-test suite
goes from 9 pre-existing failures to 8 (no new failures; this one fixed).
Refs: beads-eio.3.26, MegaManX6Recomp#14
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Diagnosing "why is this title spending its time in the interpreter" currently
means issuing eight TCP commands by hand and correlating their JSON. That is
enough friction that it does not get done, and when it is done by someone
without the runtime's internals in their head the numbers get misread -- #72
is a live example: an empty phase_hot was read as a possible bad query when it
in fact meant nothing native was executing at all.
stall_report.py collapses that into one command and one pasteable report,
usable against any title on any port:
[1] Is the overlay cache actually BEING USED, or just present on disk?
dispatch_native vs dispatch_interp_fallback, stale_blocked, the
autocompile degraded channel, and shard ok/fail/skipped counts.
[2] When code IS interpreted, is it chaining or thrashing?
dirty_ram insns_run/blocks_run = instructions per dispatcher
round-trip, with the kernel-window-is-per-block-by-design caveat
stated inline so a low ratio there is not misread as a bug.
[3] Where did the wall-clock actually go?
The largest host_us gaps in the always-on PC-sample ring, each with
the guest cycles that advanced across it. That ratio separates three
failure shapes that look identical in a frame-time graph: CPU kept up
(gap is GPU/present/vsync), CPU-bound but executing, and emu thread
not running at all.
[4]-[6] phase shares, hottest interpreted PCs tagged kernel vs
above-floor, and phase_hot deltas for both the native and static sets.
PASSIVE RING CONSUMER by construction, per the project's observability rule:
it never arms a trace, never pauses, never single-steps. Everything is a
cumulative counter (two snapshots subtract to a window) or a read of a ring
that has been recording since process start, so attaching late still sees the
event. `snap` reports cumulative; `run --secs N` reports deltas.
Two caveats are printed in the output rather than left to be rediscovered:
current_func is a STATIC dispatch stamp and localizes without proving
interpretation, and an empty native phase_hot means nothing native ran rather
than a malformed query.
Validated against a live Tomba 2 debug build (Release + PSX_DEBUG_TOOLS=ON,
port 4515), both modes and the --out JSON artifact. Representative real
output: dispatch_native 3,253,132 (97.16%) vs interp_fallback 95,047 (2.84%),
16,329 instructions per dispatcher round-trip, and one 68 ms gap at 0.46x
realtime flagged in-exception. Development of the tool caught two defects in
itself that way -- snap mode subtracted a snapshot from itself and reported
zeros for everything, and non-ASCII output mojibaked on a cp1252 console --
both fixed before commit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
compile_overlays.py spawns psxrecomp-game with cwd = dirname(game.toml),
because the seeds and capture paths in the config are relative to it. That
cwd is also where the recompiler probes for bios/SCPH1001.toml when the
config carries no explicit [recompiler] bios_config -- and --ws-config, the
only config channel overlay compiles use, deliberately does not adopt the
config's paths, so bios_config is never consulted for them.
For a PACKAGED game.toml (packaging/release/game.toml) that directory holds
neither bios/SCPH1001.toml nor a vendored framework one level down. Every
shard then died with "FATAL: no BIOS profile found", ok=0, and the overlay
cache silently failed to build. The run still exits, so the only symptom is
that the title ships with no cache and every player's first session runs
interpreted. Reported as issue #72, and it is what the recipe printed by the
game repos' package_release.ps1 told people to run.
Fix:
- psxrecomp-game takes --project-root <dir> and resolves the profile against
it: the direct bios/SCPH1001.toml probe, the one-level vendored-framework
probe, and a relative [recompiler] bios_config (documented as relative to
the game project root, previously resolved against the CWD). Absent the
flag, behaviour is unchanged.
- Every "no BIOS profile found" / bios_config failure now names the absolute
directory it searched. Without it the message reads as a config error when
it is a CWD error, which is what cost two vault parses on the Tomba 2
v0.0.8 cut.
- compile_overlays.py forwards --project-root to all three recompiler spawn
sites, defaulting it to the framework root derived from the required
--runtime-include. The derivation is used ONLY when it actually locates a
profile, so it can repair a broken invocation but never alter one that
already worked -- and the recipe the packager prints now works as written,
with no game-repo change. Every attribute lookup degrades to "pass no
flag" so the in-process callers that build a partial args object keep
working.
This also un-breaks the framework's own overlay test suite, which was RED on
master for the same reason: test_aot_overlay_discovery.py compiles against
tools/cycle_testrom/game.toml, so the recompiler probed
tools/cycle_testrom/ -- which has no BIOS profile -- and every shard failed.
Measured on this tree with one recompiler binary, changing only the tool and
tests: master exits 1 at check_real_batched_fragment_publication; this commit
reaches ALL PASS.
Two test repairs were needed to get there, both pre-existing and both masked
by the profile failure that stopped the suite before it could reach them:
- check_full_candidate_cli_fastpath stages a SYNTHETIC runtime_include in a
temp dir, so the framework root is not derivable from it and the root must
be named explicitly.
- the same test staged its cache prefill under cg<ver>_<hash>, missing the
_gc<config-hash> component the cache path gained. A prefill one directory
off is not found, so the run compiled for real instead of taking the fast
path the test exists to assert.
The cache tag is unaffected: main_psx.cpp is not in codegen_hash_sources.cmake
(verified -- the baked hash is fbb44d49 before and after), and
--overlay-config-hash returns before any of this code runs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Resolve stash conflict in psx_lobby_client by keeping getpeername-based
relay rewrite with the RFC1918-aware path, and land pending rollback
hookup/runtime changes.
Master fixed the embed_spirv.py concurrency race independently and better
(tempfile.TemporaryDirectory cleans up on early-return error paths too), so the
rebase conflict was resolved in master's favour and our mkdtemp version
removed. This branch stays SPU-only.
Records the trap that invalidated the first Tomba 2 measurement: game.toml's
overlay_autocompile_cmd names a recompiler path, and when it is absent
autocompile fails every attempt, no shard is ever built, and the runtime
silently interprets everything. Tomba 2 expects a worktree-relative path the
documented build recipe does not produce; MMX6 hardcodes an absolute path into
the main repo that works only by luck of local layout. Check autocompile_status
before trusting any perf number from a fresh worktree.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The first FMV A/B was run on a fresh worktree whose overlay shard cache was
cold AND whose autocompile could never work: game.toml points at
psxrecomp-v4/recompiler/build-t2/psxrecomp-game.exe while the documented build
recipe builds the recompiler at build-recompiler/ in the worktree root, so it
failed 8/8 with WinError 2 and the runtime silently ran interpreted
(dispatch_native = 0). The '40 fps / ~19 underruns per second' attributed to
master in the previous commit was that broken setup, not master.
Re-measured against the fully warm cache with the pre-SPU exe as a control in
the same worktree: interp_share 0.7813 vs 0.7775, exc_share 0.2386 vs 0.2381,
39.1 vs 38.0-38.4 fps. Identical within noise, so the DSP work is free.
Separately characterises the real pre-existing defect: the FMV is 96% native by
dispatch count but 78% interpreted by wall clock with 24% in exception
handling, even warm.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Measured, not asserted:
- BIOS LLE boot proves the reverb engine runs and rings down after the dry
signal stops; noise LFSR observed shifting with correct parity feedback.
- Tomba 2 intro FMV A/B against the exact parent commit shows the added
per-frame DSP work is performance-neutral (min p50 40.0 vs 40.3 fps) and
does not starve the CD bus or the host sink. The FMV's 40 fps and its
~19/s host underruns are PRE-EXISTING on master.
- Records all ten places the documentation ran out as an oracle-comparison
work list, and the two merge prerequisites (rebase onto moved master; bump
recomp-ui to 99558ee or no title compiles).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Closes the four gaps named in issue #103 and lands the separable, verified
parts of PR #102. See docs/internal/SPU_FIDELITY_103.md for the per-piece
disposition and the oracle-verification queue.
SPU DSP (issue #103), clean-room from hardware documentation:
- Reverb: the documented 22050 Hz engine — vLIN/vRIN input, same-side and
crossed different-side IIR reflections, 4-tap comb early echo, both
all-pass filters, vLOUT/vROUT output, and the work-area address advance.
Gated on SPUCNT bit 7 for WRITES only; reads, IRQ checks, address advance
and output always run, as on hardware. Addressing is confined to
[mBASE, 0x80000). Per-voice sends from EON, CD send from SPUCNT bits 0+2.
- Noise: 16-bit LFSR with the documented parity feedback, clocked at the
SPUCNT bits 8-13 rate; NON voices substitute it for their ADPCM sample.
- Volume sweeps: all 24 voice L/R plus main L/R. Bit 15 selects sweep vs
direct; sweep is a live envelope on the same rate machinery as ADSR, and a
guest read of a sweeping register returns the live level.
- Capture buffers: CD L/R and voice 1/3 output written to SPU RAM, so a game
that parks the IRQ address there still gets its interrupt.
- SPU IRQ (I_STAT bit 9): checked at every SPU RAM access class — FIFO, DMA
both directions, ADPCM block fetch, capture writes, reverb work-area
accesses — gated on SPUCNT bit 6, mirrored in SPUSTAT bit 6, acked by
clearing SPUCNT bit 6.
The CD-only fast path in spu_render was removed rather than extended: the
SPU now has per-frame work that must run in every path, and an FMV (XA audio
with zero active voices) is exactly the CD-with-reverb case.
This is a clean-room implementation from the psx-spx/nocash register map and
documented algorithm. Beetle's source was deliberately not consulted: it is
GPL-2.0-or-later and this project is PolyForm Noncommercial. PR #16 was
parked for exactly that reason and PR #13's reverb was separately wrong (it
gated on bit 15 of dAPF1, an address offset, instead of SPUCNT bit 7). The
one place this knowingly deviates from hardware is the 22.05 -> 44.1 kHz
reconstruction filter, which the documentation does not specify; it is
isolated in a single function and flagged for oracle comparison.
CAUSE.IP2 (from PR #102):
IP2 is combinational on R3000A — it mirrors the interrupt line and falls the
instant the guest acks I_STAT or masks the source. It was being OR'd in at
delivery and never cleared, in TWO places (the compiled path and
psx_interpreter.c, the latter missed by PR #102), leaving a phantom pending
interrupt that can spin a kernel dispatcher looping on CAUSE.IP & SR.IM.
psx_irq_refresh_cause_ip2() is now the sole writer of bit 10, refreshed at
raise, I_STAT ack, I_MASK write, HLE context restore and power-on. Verified
against the Beetle oracle's irq.cpp/cpu.cpp rather than asserted, and
covered by test_cause_ip2_combinational.
Mid-dispatch audio pump (from PR #102), with a fix: routed through a gate
that mirrors sdl_audio_update's mute/sink decision. Pumping unconditionally
from the VBlank edge, as submitted, would have pushed real audio during every
turbo-load hard mute and defeated the freeze-in-place mute model.
SPU RAM DMA readback (from PR #102): DMA4 in the SPU->RAM direction wrote
literal zeros; SPU RAM is readable memory and titles carry state through it
across an Exec boundary.
Also fixes an unrelated build race found on the way: tools/embed_spirv.py
wrote its intermediate SPIR-V to <source>.spv, so the two runtime targets
that embed the same shaders raced on one path and one deleted the file the
other was reading.
Co-Authored-By: Alexandros Mandravillis <Alexbeav@live.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A contributor bringing up a new title hit four dead-ends in a row, three of
which were the same missing step wearing different masks: `regen_bios.sh`
builds the BIOS emitter but never configures it, and nothing said so.
- "no recompiler build dir found (set PSXRECOMP_BIOS_BUILD)" pointed at an
env var when a fresh clone has no CMake tree to point it at. Both discovery
failure and an explicit override now name the actual missing step and print
the configure command.
- Setting PSXRECOMP_BIOS_BUILD skipped the CMakeCache.txt check that discovery
applies, so an unconfigured directory reached `cmake --build` and died with
cmake's bare "Error: could not load cache". Validate the override too.
- A CMake configure that FAILS still writes CMakeCache.txt — the cache lands
before the generate step, which never runs after a FATAL_ERROR. So "has a
cache" did not mean "is buildable": the directory left behind by his failed
SDL and BIOS-backend configures had a cache and no build.ninja, and the
build reported only `ninja: error: loading 'build.ninja': GetLastError() =
2`. Read the generator out of the cache, check for its build files, and say
what happened. Discovery skips such a directory and keeps looking rather
than selecting it and dying; an unrecognized generator falls through to
cmake instead of inventing a failure.
All errors now go to stderr (the old one went to stdout).
Docs: the README's framework build block configured the recompiler and jumped
straight to the runtime build, omitting the required BIOS regen between them —
exactly the hole he fell into. It is now the same numbered 1/2/3 sequence
docs/BUILDING.md already used. BUILDING.md gains troubleshooting entries for
the two CMake half-configured-directory errors, the fresh-clone "No recompiled
BIOS backend available", and PSXRECOMP_BIOS_BUILD being resolved relative to
the framework root rather than the shell's cwd.
Verified: both new failure paths and the half-configured-directory case
reproduce and now self-explain; discovery precedence (build-t2 over build),
explicit-override precedence, Ninja/Makefiles/unknown generators, and a CRLF
cache all behave; end-to-end `regen_bios.sh --config bios/OpenBIOS.toml`
still emits (Dispatch entries: 3911).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Alex Vanderveen