Emitter side of the value-propagation engine (ENHANCEMENTS.md G1.10):
- recompiler/src/pgxp_hook_emitter.{h,cpp} (new, in the codegen-hash
surface): ONE hook grammar shared by BOTH emitters, wrapping loads/
stores/COP2 transfers and precision-carrying ALU ops (moves, add/sub,
or-merge, shifts, LUI, HI/LO, muldiv) with PGXP_*() macros. Clobberable
sources are captured into locals before the statement. Precision-
destroying ops (AND/XOR/SLT/...) are deliberately unhooked - validate-
on-read drops their stale shadows for free.
- code_generator.cpp: central append in translate_instruction; LWC2/SWC2
emissions capture the RAW loaded/stored word (a masked GTE register
write must validate against the word as loaded).
- strict_translator.cpp: translate() wraps translate_impl()'s c_code with
the shared grammar; deferred load-delay variants hook the psx_ldd temp.
- BIOS regenerated, fingerprints verified CHANGED (SCPH1001 + OpenBIOS;
42.6k/12.2k macro sites). Base runtime + 33/33 tests green against it.
- Gating: macros expand only under -DPSX_PGXP=1. runtime.cmake grows a
PGXP option on psxrecomp_add_runtime_target (defines PSX_PGXP=1 +
PSX_OVERLAY_FLAVOR=2, _pgxp exe suffix) and PSX_PGXP_VARIANT to build
the sibling target from the same generated tree (framework runtime +
game wrapper). No ABI version bump needed: the appended PGXPHooks
table is NULL-tolerant and the flavor half of the ABI tag rejects any
host/DLL flavor mix.
- Shard caches: flavor folded into the cache tag (cg<V>_<hash>_gc<hash>
_f<flavor>) in overlay_loader.c + compile_overlays.py so base and pgxp
caches never share a directory; compile_overlays.py passes
-DPSX_PGXP=1 when flavor bit 2 is set; overlay preamble forwards
psx_pgxp_* through the table. PSX_OVERLAY_CODEGEN_VER 9 -> 10.
- Env overrides PSX_GEOMETRY_CORRECTION / PSX_PERSPECTIVE_TEXTURING /
PSX_PGXP_CPU_MODE (validation path: free-running headless boots arm
from the first vertex); census grew produced/swc2_stores and the pgxp
verb reports suppress/active gate state.
Validated: PSXRecomp_pgxp.exe boots the recompiled BIOS with hooks armed
(engine active, per-vertex lookups flowing; the BIOS shell has no RTPS
content so produced=0 there is correct - coverage measurement moves to a
3D title in phase 3).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Conflict: recompiler/CMakeLists.txt. Master gained #134's test registrations
in the same block where #133 added stall_report. Resolved as the union of both.
Also RE-ENABLES aot_overlay_discovery, which is not part of the conflict region
and so would otherwise have survived the merge untouched. #134 registered it
DISABLED with this reason:
"fails in check_real_batched_fragment_publication: that check reaches
MOD.compile_fragment_batch, which spawns psxrecomp-game with cwd set to its
own TemporaryDirectory, so BIOS-profile resolution finds nothing ... it
needs an owner, and DISABLED keeps it counted until then."
This branch is that owner -- resolving BIOS-profile lookup against
--project-root instead of the CWD is exactly that fix. Landing it while leaving
the test disabled would ship the fix with its own proof switched off, which is
the silent-rot class #134 exists to prevent.
Measured on this tree, msys64 toolchain driven from PowerShell:
- pristine origin/master: test_aot_overlay_discovery.py exits 1 at
check_real_batched_fragment_publication, "no BIOS profile found"
- this merge: ALL PASS, and ctest -R aot_overlay_discovery passes in 7.64s
with the DISABLED property removed
- baked codegen hash 0xfbb44d49u on BOTH arms -- the cache tag does not move,
so existing overlay caches keep loading
- full ctest failure set identical to pristine master, no new failures
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Wire lib/retcomm-rbengine beside recomp-net and replace MotK in-tree sched,
hash_confirm, input hist, snap ring, and rb_post with thin PSX shims so the
ecosystem can share invent/pacing policy outside each title facade.
Co-authored-by: Cursor <cursoragent@cursor.com>
Keep dispatch counters as raw event counts, fail closed on malformed or reset evidence, label rolling and censored scopes, retain bounded-ring provenance, and write hash-bound diagnostic artifacts atomically. Add focused protocol tests and register them with CTest.
phase window as rolling
Both reported by @Alexbeav on PR #131, both confirmed against the runtime
source and a live warm-cache instance.
1. autocompile_status nests every compile field under "compile" (see
handle_autocompile_status in debug_server.c). The report read runs, fails,
the shard counts and the degraded flag off the OUTER object, where none of
them exist, so it always printed zeros and never printed a degradation
reason.
This was the worst possible placement for the bug: section [1] exists to
catch a silently-failing autocompile, and it reported a healthy-looking
row of zeros in exactly that case. It was visible in the validation output
of the original commit -- "autocompile: runs=0 fails=0 shard_ok=0" -- and
was not questioned, which is the same unexamined-quiet-number failure the
tool is meant to prevent.
Measured on a live instance during a 15s window: the fixed version reports
runs=3 fails=3 consecutive_fails=3 last_exit=1 and the full DEGRADED
reason string. The old version reported runs=0 fails=0 and no warning at
all for the same window.
The section now also reports `configured`, `state`, and
`shard_result_seen`, because those are what make a zero interpretable:
not-configured zeros are expected, whereas configured-but-never-saw-a-
PSX_SHARD_RESULT zeros are meaningless rather than healthy, and the report
now says which it is. A missing `compile` object is called out as a
protocol mismatch instead of being silently treated as zeros.
2. phase_profile sums a ring of recent seconds and reports its own window_s;
it is a ROLLING WINDOW, not a since-boot total, and it does not align with
this report's window. It was labelled "cumulative since boot". It now
prints the runtime's reported window and states plainly that the scope
differs from the report's own.
Left deliberately for the follow-up @Alexbeav offered: fail-closed handling
of missing/non-numeric/decreasing counters, the dispatch-counter percentages
being dispatch-event shares rather than coverage or time-share, top-set
censoring in the phase_hot and per-PC deltas, ring metadata for
reproducibility, and artifact schema/tool-hash/atomicity.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Diagnosing "why is this title spending its time in the interpreter" currently
means issuing eight TCP commands by hand and correlating their JSON. That is
enough friction that it does not get done, and when it is done by someone
without the runtime's internals in their head the numbers get misread -- #72
is a live example: an empty phase_hot was read as a possible bad query when it
in fact meant nothing native was executing at all.
stall_report.py collapses that into one command and one pasteable report,
usable against any title on any port:
[1] Is the overlay cache actually BEING USED, or just present on disk?
dispatch_native vs dispatch_interp_fallback, stale_blocked, the
autocompile degraded channel, and shard ok/fail/skipped counts.
[2] When code IS interpreted, is it chaining or thrashing?
dirty_ram insns_run/blocks_run = instructions per dispatcher
round-trip, with the kernel-window-is-per-block-by-design caveat
stated inline so a low ratio there is not misread as a bug.
[3] Where did the wall-clock actually go?
The largest host_us gaps in the always-on PC-sample ring, each with
the guest cycles that advanced across it. That ratio separates three
failure shapes that look identical in a frame-time graph: CPU kept up
(gap is GPU/present/vsync), CPU-bound but executing, and emu thread
not running at all.
[4]-[6] phase shares, hottest interpreted PCs tagged kernel vs
above-floor, and phase_hot deltas for both the native and static sets.
PASSIVE RING CONSUMER by construction, per the project's observability rule:
it never arms a trace, never pauses, never single-steps. Everything is a
cumulative counter (two snapshots subtract to a window) or a read of a ring
that has been recording since process start, so attaching late still sees the
event. `snap` reports cumulative; `run --secs N` reports deltas.
Two caveats are printed in the output rather than left to be rediscovered:
current_func is a STATIC dispatch stamp and localizes without proving
interpretation, and an empty native phase_hot means nothing native ran rather
than a malformed query.
Validated against a live Tomba 2 debug build (Release + PSX_DEBUG_TOOLS=ON,
port 4515), both modes and the --out JSON artifact. Representative real
output: dispatch_native 3,253,132 (97.16%) vs interp_fallback 95,047 (2.84%),
16,329 instructions per dispatcher round-trip, and one 68 ms gap at 0.46x
realtime flagged in-exception. Development of the tool caught two defects in
itself that way -- snap mode subtracted a snapshot from itself and reported
zeros for everything, and non-ASCII output mojibaked on a cp1252 console --
both fixed before commit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
compile_overlays.py spawns psxrecomp-game with cwd = dirname(game.toml),
because the seeds and capture paths in the config are relative to it. That
cwd is also where the recompiler probes for bios/SCPH1001.toml when the
config carries no explicit [recompiler] bios_config -- and --ws-config, the
only config channel overlay compiles use, deliberately does not adopt the
config's paths, so bios_config is never consulted for them.
For a PACKAGED game.toml (packaging/release/game.toml) that directory holds
neither bios/SCPH1001.toml nor a vendored framework one level down. Every
shard then died with "FATAL: no BIOS profile found", ok=0, and the overlay
cache silently failed to build. The run still exits, so the only symptom is
that the title ships with no cache and every player's first session runs
interpreted. Reported as issue #72, and it is what the recipe printed by the
game repos' package_release.ps1 told people to run.
Fix:
- psxrecomp-game takes --project-root <dir> and resolves the profile against
it: the direct bios/SCPH1001.toml probe, the one-level vendored-framework
probe, and a relative [recompiler] bios_config (documented as relative to
the game project root, previously resolved against the CWD). Absent the
flag, behaviour is unchanged.
- Every "no BIOS profile found" / bios_config failure now names the absolute
directory it searched. Without it the message reads as a config error when
it is a CWD error, which is what cost two vault parses on the Tomba 2
v0.0.8 cut.
- compile_overlays.py forwards --project-root to all three recompiler spawn
sites, defaulting it to the framework root derived from the required
--runtime-include. The derivation is used ONLY when it actually locates a
profile, so it can repair a broken invocation but never alter one that
already worked -- and the recipe the packager prints now works as written,
with no game-repo change. Every attribute lookup degrades to "pass no
flag" so the in-process callers that build a partial args object keep
working.
This also un-breaks the framework's own overlay test suite, which was RED on
master for the same reason: test_aot_overlay_discovery.py compiles against
tools/cycle_testrom/game.toml, so the recompiler probed
tools/cycle_testrom/ -- which has no BIOS profile -- and every shard failed.
Measured on this tree with one recompiler binary, changing only the tool and
tests: master exits 1 at check_real_batched_fragment_publication; this commit
reaches ALL PASS.
Two test repairs were needed to get there, both pre-existing and both masked
by the profile failure that stopped the suite before it could reach them:
- check_full_candidate_cli_fastpath stages a SYNTHETIC runtime_include in a
temp dir, so the framework root is not derivable from it and the root must
be named explicitly.
- the same test staged its cache prefill under cg<ver>_<hash>, missing the
_gc<config-hash> component the cache path gained. A prefill one directory
off is not found, so the run compiled for real instead of taking the fast
path the test exists to assert.
The cache tag is unaffected: main_psx.cpp is not in codegen_hash_sources.cmake
(verified -- the baked hash is fbb44d49 before and after), and
--overlay-config-hash returns before any of this code runs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Hybrid kept reappearing in titles that never meant to ship it, because it was
the ZERO value of PadMode and therefore the default everywhere: PlayerInput,
p1_mode/p2_mode, ctrl_locked_*, and default_p1/p2_mode all initialised to it.
A game then had to opt OUT with [controller] allow_hybrid = false, and every
shipping title did exactly that or pinned a mode - so no title actually ran it.
Hybrid now has exactly one route: a trusted game plugin calling
psx_mod_set_controller_mode_override(), which is how Tomba's hybrid controller
mod already requests it. Everything that made it reachable by accident is gone:
- defaults are ANALOG (PlayerInput, p1/p2, ctrl_locked_*, default_p1/p2)
- [controller] allow_hybrid is removed, along with the startup clamp that
existed only to undo the bad default
- game.toml default_mode/p1_mode/p2_mode = hybrid now THROWS, naming the
mod API as the only route, rather than being silently coerced
- a stale settings.toml hybrid migrates to analog via
pad_mode_from_settings_string(); a user should not fail to launch over a
value they never typed
- pad_mode_to_string() never writes hybrid back
Deliberately KEPT, because the mod runs on it: PAD_MODE_HYBRID,
PSX_MOD_CONTROLLER_HYBRID, psx_mod_set_controller_mode_override(),
hybrid_stick_active/hybrid_dpad_active, PlayerInput::hybrid_analog and
psx_keybinds_dpad_active.
Paired with the recomp-ui change that drops the Hybrid segment and the
allow_hybrid field from RecompLauncherCGameInfo - the two must land together.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Probe cmake --version; if the cache or latest pointer is unusable, clear
the bad pointer and stamp, then re-download (or install from zip) so
Generate & rebuild can recover without manual cleanup.
Co-authored-by: Cursor <cursoragent@cursor.com>
Install into versioned <tag>/ with latest as a pointer (not extract-into-
latest), honor RETCOMM_DATA_HOME/TOOLCHAIN_CACHE, and search both XDG and
~/.local/share so install.sh packs resolve during ensure/rebuild.
Co-authored-by: Cursor <cursoragent@cursor.com>
Matthew Stanley