Keep dispatch counters as raw event counts, fail closed on malformed or reset evidence, label rolling and censored scopes, retain bounded-ring provenance, and write hash-bound diagnostic artifacts atomically. Add focused protocol tests and register them with CTest.
phase window as rolling
Both reported by @Alexbeav on PR #131, both confirmed against the runtime
source and a live warm-cache instance.
1. autocompile_status nests every compile field under "compile" (see
handle_autocompile_status in debug_server.c). The report read runs, fails,
the shard counts and the degraded flag off the OUTER object, where none of
them exist, so it always printed zeros and never printed a degradation
reason.
This was the worst possible placement for the bug: section [1] exists to
catch a silently-failing autocompile, and it reported a healthy-looking
row of zeros in exactly that case. It was visible in the validation output
of the original commit -- "autocompile: runs=0 fails=0 shard_ok=0" -- and
was not questioned, which is the same unexamined-quiet-number failure the
tool is meant to prevent.
Measured on a live instance during a 15s window: the fixed version reports
runs=3 fails=3 consecutive_fails=3 last_exit=1 and the full DEGRADED
reason string. The old version reported runs=0 fails=0 and no warning at
all for the same window.
The section now also reports `configured`, `state`, and
`shard_result_seen`, because those are what make a zero interpretable:
not-configured zeros are expected, whereas configured-but-never-saw-a-
PSX_SHARD_RESULT zeros are meaningless rather than healthy, and the report
now says which it is. A missing `compile` object is called out as a
protocol mismatch instead of being silently treated as zeros.
2. phase_profile sums a ring of recent seconds and reports its own window_s;
it is a ROLLING WINDOW, not a since-boot total, and it does not align with
this report's window. It was labelled "cumulative since boot". It now
prints the runtime's reported window and states plainly that the scope
differs from the report's own.
Left deliberately for the follow-up @Alexbeav offered: fail-closed handling
of missing/non-numeric/decreasing counters, the dispatch-counter percentages
being dispatch-event shares rather than coverage or time-share, top-set
censoring in the phase_hot and per-PC deltas, ring metadata for
reproducibility, and artifact schema/tool-hash/atomicity.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Diagnosing "why is this title spending its time in the interpreter" currently
means issuing eight TCP commands by hand and correlating their JSON. That is
enough friction that it does not get done, and when it is done by someone
without the runtime's internals in their head the numbers get misread -- #72
is a live example: an empty phase_hot was read as a possible bad query when it
in fact meant nothing native was executing at all.
stall_report.py collapses that into one command and one pasteable report,
usable against any title on any port:
[1] Is the overlay cache actually BEING USED, or just present on disk?
dispatch_native vs dispatch_interp_fallback, stale_blocked, the
autocompile degraded channel, and shard ok/fail/skipped counts.
[2] When code IS interpreted, is it chaining or thrashing?
dirty_ram insns_run/blocks_run = instructions per dispatcher
round-trip, with the kernel-window-is-per-block-by-design caveat
stated inline so a low ratio there is not misread as a bug.
[3] Where did the wall-clock actually go?
The largest host_us gaps in the always-on PC-sample ring, each with
the guest cycles that advanced across it. That ratio separates three
failure shapes that look identical in a frame-time graph: CPU kept up
(gap is GPU/present/vsync), CPU-bound but executing, and emu thread
not running at all.
[4]-[6] phase shares, hottest interpreted PCs tagged kernel vs
above-floor, and phase_hot deltas for both the native and static sets.
PASSIVE RING CONSUMER by construction, per the project's observability rule:
it never arms a trace, never pauses, never single-steps. Everything is a
cumulative counter (two snapshots subtract to a window) or a read of a ring
that has been recording since process start, so attaching late still sees the
event. `snap` reports cumulative; `run --secs N` reports deltas.
Two caveats are printed in the output rather than left to be rediscovered:
current_func is a STATIC dispatch stamp and localizes without proving
interpretation, and an empty native phase_hot means nothing native ran rather
than a malformed query.
Validated against a live Tomba 2 debug build (Release + PSX_DEBUG_TOOLS=ON,
port 4515), both modes and the --out JSON artifact. Representative real
output: dispatch_native 3,253,132 (97.16%) vs interp_fallback 95,047 (2.84%),
16,329 instructions per dispatcher round-trip, and one 68 ms gap at 0.46x
realtime flagged in-exception. Development of the tool caught two defects in
itself that way -- snap mode subtracted a snapshot from itself and reported
zeros for everything, and non-ASCII output mojibaked on a cp1252 console --
both fixed before commit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Alexandros Mandravillis