mirror of
https://github.com/RetroPortingToolKit/psxrecomp
synced 2026-10-28 02:56:36 -04:00
fcd577b02ca79646dec51d9ac69bf0fdd8a6977d
1716 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Alex Vanderveen
|
fcd577b02c |
scale_trace: always say HOW the numbers were taken; trust adjacent frames
First clean run — both sides sampled, no errors, no fallback: psx-runtime 72, 74, 76, 78 steps of 2 oracle 12, 16, 20, 24 steps of 4 Both sweep smoothly; neither jumps. The ranges are disjoint (72-78 against 12-24), which is the two sides sitting at different points in the fade, not a fault. But the report could not say whether psx-runtime's series was per-frame or free-running, because native_sampling was only set on the fallback path. Those support completely different claims: steps of 2 across CONSECUTIVE frames is a smooth fade, while steps of 2 across samples ninety frames apart is nothing at all. The field is now set before either path runs and overwritten by whichever one did. And the eight-sample floor was applied to per-frame data, where it does not belong. Consecutive frames measure the increment DIRECTLY rather than sampling a distribution — four adjacent frames stepping by 2 IS the step. The floor still applies to free-running samples, which say nothing however many there are. Applying it to per-frame discarded a measurement that was already conclusive. A per-frame result now also states what the increment means: at a median step of 2 the scale crosses its whole range in about 64 frames, roughly a second, which does not produce hard bands. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
fdf8a76274 |
scale_trace: actually apply the aliasing guard (the last commit did not)
The previous commit claimed two fixes and shipped one. Its patch script hit a
failed assertion on the third replacement and exited BEFORE writing the file,
so the two edits it had already made in memory were discarded — the same shape
as the probe fill that was written in memory and never saved. The commit
message described work that was not in the tree, which is worse than not doing
it, and only the tests I wrote alongside caught it.
Both edits are now in:
* every per-frame claim checks native_sampling, so "these steps are the
animation increment" cannot be printed over data the fallback produced
* a fallback run gets its own granularity label saying outright that its
steps are aliased and compare nothing
The reported document had native_sampling "free-running-fallback" AND
granularity "native-measured-per-frame" at once. That combination is now
unreachable.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Alex Vanderveen
|
498a0fe45f |
scale_trace: never call aliased data an animation increment
The report contradicted itself: native_sampling "free-running-fallback" AND granularity "native-measured-per-frame" in the same document — the data is aliased, and its steps are the animation increment. Both cannot be true. The guard existed and sat on a different branch than the one that fired, so the claim I said last commit could not be made was made on the very next run. Free-running samples land about ninety frames apart, so their differences are aliasing. Presenting them as an increment invites precisely the wrong conclusion — that our fade jumps by 44 — when nothing measured that. Every per-frame claim now checks the sampling method, a test asserts that for each one by reading the source, and a fallback run gets its own label saying the steps mean nothing. The other half is why the fallback ran at all. Per-frame now reports "a block leader fired but no register sample came with it", which was accurate and narrow: it insisted on the SINGLE nearest leader and threw away frames where a leader fired but the recorded sample belonged to a different one. It now takes a sample from whichever fired leader has one, nearest first — true of the frame rather than of one address. Data so far, with the caveat attached: psx-runtime 26..128, oracle 76..86, both aliased. Neither says how either one moves. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
1def9faeae |
packet_writers: hand the emulator back on EVERY path
"Find colour writers keeps pausing the native and leaving it paused, causing it to fail to find the writer." The class-absent path returned from the middle of the parked section without resuming. So the game stopped advancing, the effect never came round again, every retry walked the same stale display list and reported the same "no PolyG4+semi in the current list" — and the cause was invisible from outside, because the message describes the list rather than the emulator being stopped. It is self-perpetuating: the first failure guarantees the next one. Leaving it parked is worse than any error this tool can report, so it is now structurally impossible rather than remembered. The park spans the walk AND the trace by design -- the layout has to still describe the buffer when the writes happen -- so a try/finally around either half alone was never enough. The parked section is a separate function wrapped in a guard, and all seven of its exits resume. The guard resumes once even when the trace block has already handed back explicitly, and does not swallow the exception it is unwinding: a failed resume must not replace the error that caused it. Audited the other tools for the same shape. range_writers, scale_trace, gpu_parity and gpu_display_list all resume in a finally; colour_inputs and probe_regs never park psx-runtime at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
b609424177 |
gpu_parity: wait for the effect, because the image is the untested half
Pooling every scale observation taken so far: psx-runtime 28 36 42 48 60 64 66 68 72 94 128 (28..128) oracle 52 64 68 72 76 96 112 116 ... 128 (52..128) The ranges overlap almost completely, and every sample is aliased. The scale data cannot diagnose anything, and saying otherwise would be reading a pattern into noise for the third time. What the accumulated evidence does say is narrower. Both display lists carry the SAME subtractive primitives — PolyF4+semi|B-F x184 and PolyG3+semi|B-F x176 on each side — and the flat ones were byte-identical at (64,64,64). Subtractive blending is how the outer darkening is drawn, and "the vignette is missing" is half the reported symptom. So the guest is submitting the darkening on both sides. Geometry identical, GTE unused, translation clean over 5.7M blocks, base colour table identical at rest, subtractive primitives present and matching. Every guest-side explanation has been measured and ruled out, and the one thing never tested at this effect is the IMAGE — because the oracle could not reach the scene until the memory card was shared. So gpu_parity gains the same wait every other tool has, on both sides. An image diff of a scene without the effect answers a question nobody asked. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
1c1281e455 |
wait_for_class: make the wait affordable, and un-park before polling
The fallback worked — psx-runtime produced 12 samples free-running. The oracle still reported "PolyG4+semi never appeared within 120s. Drawing instead: nothing", and the cost of the check is why. class_on_screen() snapshots all 2 MB of RAM: 128 read_ram calls. Against a PAUSED oracle, whose socket falls back to a ~1 Hz idle timer, that is 128 seconds for ONE poll. A 120-second wait therefore completes a single partial read and reports "nothing" — not a statement about what is drawing, a statement about a read that never finished. The oracle was very likely parked from an earlier breakpoint run, which nothing in the wait undid. So the wait now resumes first, pays for the full scan once, and then re-reads only the span the display list occupies — a couple of KB, one or two reads instead of 128. A cached root that stops walking means the list moved, and the next poll pays for a rescan rather than sticking to a dead pointer. A `check` seam lets a caller supply its own probe, which is also how this is tested without standing up a synthetic display list in fake RAM. Recorded for the next run: psx-runtime's series came from the free-running fallback, so its steps (median 12, max 64) are ALIASED — samples ~90 frames apart — and are not an animation increment. native_sampling says so in the report, and the per-frame verdict is suppressed when the fallback ran. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
ea41b4d28a |
scale_trace: fall back to the probe that works, and stop misreporting the scene
Repeated runs capturing nothing from psx-runtime while the effect was being replayed, and the reported reason was wrong twice over. The scene check ran AFTER the finally block resumed psx-runtime, so it reported what was on screen seconds later rather than during sampling — and for a brief effect that reliably says "not on screen" when it was there the whole time. It now checks while still parked. And per-frame mode capturing nothing is not the same as the code being unreachable. The free-running probe demonstrably works here: it returned 4800 hits and real register values ($s4=0x800E2634, $sp=0x1F800234, scratchpad, as the oracle also reports). So per-frame returning empty is a property of stepping, not of the block. It now retries free-running automatically, which produces data with a stated caveat instead of a fourth empty run — and records which method produced the numbers, so aliased steps are never presented as an animation increment. The failure message also separates two causes that need different fixes: no candidate fired at all (the block starts further back, or this overlay is not running through COMPILED blocks — the probe only fires at compiled block leaders), versus a leader fired but no register sample came with it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
0f70aacccc |
tools: wait for the effect instead of racing it
"the effect is not on screen on psx-runtime, so this code never ran. Currently drawing: PolyFT4 x584, PolyF4+semi x184, PolyG3+semi x176, PolyFT3 x151" — the diagnostic added last commit, doing its job on the first run. The effect simply was not up when the tool looked. That is the real problem, and it is not a diagnostic problem. Every tool here needs the effect RUNNING, and requiring it to be on screen at the instant a button is clicked means racing a transient animation. Losing that race produces a different-looking failure in each tool — no writes recorded, no candidate fired, no table found, the block was not reached — four symptoms, one cause, none of them naming it. Running lockstep beforehand appeared to help only because it takes long enough for the effect to come round again. So the tools wait. Press the button first, then trigger the effect; each polls the display list until its class appears, up to two minutes, and each side is waited for INDEPENDENTLY in its own thread. That last part matters: it removes the need to hold two emulators inside the same effect simultaneously, which was never realistic by hand. Wired into scale_trace, packet_writers, range_writers and colour_inputs — every tool that needs the effect. Timing out reports what WAS drawing, because "wrong scene" and "tool is broken" look identical otherwise. A transient DebugError mid-poll does not abort the wait: the emulator can be mid-pause, and one failed poll is not a reason to stop waiting for something that has not happened yet. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
7fc10ffa67 |
scale_trace: sweep across frames, and stop needing the oracle for this question
Reported from a live session: DuckStation being paused repeatedly through the animation while psx-runtime was not. Two things behind it. Per-frame sampling never swept. It flattened every search window and took the first sixteen candidates, which is one window — the nearest 0x3C bytes — so a block starting earlier than that was never found. The failure then read "the block was not reached on any of 12 stepped frames", which is true and points the reader at the effect not drawing rather than at a search that barely looked. Each stepped frame now arms the NEXT window, so the sweep advances with the animation instead of being spent before it starts. And on failure it ASKED whether the effect was drawing instead of checking. class_on_screen answers that directly, and the two causes need opposite responses: nothing on screen means get the game to the scene, a covered range means the block starts further back. It now says which, and how far it looked. The larger point is the one the observation exposes. Stepping psx-runtime frame by frame while the oracle free-runs measures two different things, and for "does OUR fade sweep or jump" the oracle is not needed at all — consecutive frames give the real increment with no alignment, no shared phase and no second emulator. Requiring both sides forced the operator to hold two emulators inside the same effect at once, which is the hardest part of this whole exercise. A per-frame series of five or more samples now stands on its own, and says plainly whether our fade is smooth or coarse. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
1a567d21ce |
scale_trace: sample both sides at once, and step before reading
Two bugs, both reported from a live session: psx-runtime visibly parked while the oracle ran on untouched, and the oracle then said it had no samples. The sides were sampled SEQUENTIALLY. psx-runtime was sampled to completion first and the oracle asked afterwards, by which time the effect had finished over there and the breakpoint could never fire. Replaying the animation does not help, because the two halves of the run are minutes apart. They now run on threads and are watched during the same pass. And per-frame sampling could only ever capture one value. It paused psx-runtime, then called probe_registers — which arms the probe and SLEEPS waiting for it to fire. On a paused emulator no code executes, so the block leader is never reached and every sample after the first comes back empty. That is where the "1 sample" run came from, and the verdict drawn from it. The order has to be arm -> STEP -> read, so the block executes between arming and reading, and sample_per_frame now does that itself rather than delegating to a helper whose whole method is waiting for a running emulator. Both are pinned by tests that read the source order directly, because both failures produced plausible output rather than an error: one sample looks like a sample, and an empty oracle looks like an oracle that was not there. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
13b86d8853 |
scale_trace: sample CONSECUTIVE frames, because the steps were aliasing
psx-runtime finally produced a real series — 7 samples, 48/94/128, apparent steps of 40-46 — against the oracle's 122/124/126/128 stepping by 2. Twenty times coarser, and it would explain hard edges instead of a gradient. It is not evidence. Both series are sampled about 1.5s apart, roughly ninety frames, so the difference between consecutive readings is aliasing rather than the animation increment. A fade stepping by 2 every frame, sampled every ninety frames, produces arbitrary-looking jumps too — which is exactly what 48/94/128 could be. --per-frame steps ONE frame between reads, so the deltas ARE the increment. That needs one emulator rather than two, which is the property that has made every successful measurement in this investigation work: no frame alignment, no matching buffer half, no shared phase. If psx-runtime's per-frame step is 2-4 the fade is smooth and the fault is elsewhere; if it is 40 the fade is coarse and that is the picture. The oracle's series stays non-frame-adjacent, so its apparent step is an upper bound only, and the report says so rather than comparing the two as if they were measured the same way. Also fixes the Studio passing a bare "--gap" as the off case for a boolean flag, which would have failed in argparse before the tool ran. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
5de4822b6e |
scale_trace: a verdict needs a series; caching must not replace the search
The tool reported "native-not-animating" from ONE psx-runtime sample, with the note "never moved from 128 across 1 samples". That is not a claim anyone can make, and it agreed with a hypothesis already disproved twice here — the most dangerous shape a bug can take, because it confirms what the reader suspects. The cause was the previous commit. Caching the block leader to make sampling affordable REPLACED the sweep instead of preceding it, so the moment the cached address stopped firing — an overlay reloading elsewhere, or that block simply not being reached on a pass — the run ended. A 24-sample request became one sample, and then a conclusion was drawn from it. Both halves fixed. The cached leader is now tried first and the full sweep still follows it, and a failed sample drops the cache and continues rather than ending the series. Any "constant" verdict requires at least five samples from that side; below it the result is too-few-samples, stated as such. Variation needs no floor, because two different values prove movement however few samples there are. The oracle's own series is worth recording: 112, 116, 120, 124, 128 — five distinct values in even steps of 4. Whatever psx-runtime does, the oracle sweeps its fade smoothly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
72ceb274e8 |
scale_trace: compare HOW each side animates, and stop resampling the search
First complete both-sides comparison, and the verdict is both-animate: the fade runs on psx-runtime too. That is the second time the pinned-scale idea has been disproved by measurement, and it stays disproved. The shapes differ though, and min/max cannot express it: oracle 122, 124, 126, 128 steps of 2, a 6-wide band psx-runtime 28, 128 nothing between, a 100-wide jump A fade that sweeps smoothly and one that jumps between extremes produce very different pictures from identical geometry — which is the reported symptom. So describe() now reports step size alongside range, and both-animate compares granularity rather than stopping at "both vary". It refuses to compare granularity below eight samples per side, because two distinct values out of three is ALSO exactly what sparse sampling of a smooth ramp looks like. The run above had three and four; that is suggestive and nothing more. Which is why sampling was the real problem: every psx-runtime sample re-ran the whole block-leader sweep, so a run capped out at three. The leader is found once and reused, and the default sample count is up to 24. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
2cd0a93c9a |
probe: sweep for the block leader instead of guessing one window
The memory card worked — the oracle reached the effect and produced samples for the first time: $s6 at 116, 120, 124, 128, a smooth descending ramp. That is the fade animating, and it is the first cross-emulator measurement in this investigation that was not blocked by the two sides being in different places. psx-runtime then failed, at a different colour writer: none of its 16 candidates was a basic-block leader. The message was accurate and useless. A single fixed window (step 0x10 over 0x100) is a guess about where the enclosing block starts, and it found a leader for 0x8006844C while finding none at all for 0x800684C0 — two writers sixteen bytes of code apart. Now it sweeps: sixteen candidates at 4-byte steps immediately before the target, then successive contiguous windows further back until something fires. Every instruction is a possible leader, so the step has to be 4 — a coarser one can walk straight over the only address that would have fired, which is what happened. Windows are contiguous by construction, because a gap between them hides exactly the same thing. Failing now reports how far back it actually searched, rather than naming a step and a range the operator has to reason about. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
712bd5a5a6 |
oracle: share the game's memory card, so it can reach the scene
Both failure reasons are now recorded, and together they name the blocker that
has stopped every cross-emulator measurement at this effect:
oracle: did not reach 0x8006844C while sampling. That PC is overlay code, so
it only exists while the right overlay is resident and the oracle has
to be at the same point in the game.
psx-runtime gets driven to the effect by hand; the oracle sits wherever a fresh
boot leaves it, and a breakpoint on an overlay that is not loaded can never
fire. Nothing about the tooling fixes that — the oracle has to actually be
there.
The practical route is the game's own save. Native keeps its card at
saves/card1.mcd; the oracle defaults to PerGameTitle in its own directory, so
it boots with an empty one. `start --memcard` copies the card in and switches
Card1Type to Shared, and the Studio passes it automatically when a card is
present next to the project. The oracle can then load the player's save slot
and arrive at the same place — far more practical than replaying input from
boot or moving a savestate between two emulators that do not share a format.
Copied rather than shared in place: two emulators writing one card is a good
way to lose a save, and the reference copy only needs to be read. A missing
card is reported, because booting with an empty one looks exactly like a
working oracle that simply never reaches the scene.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Alex Vanderveen
|
be8a3666b1 |
scale_trace: the pinned-scale hypothesis is WRONG, and empty sides must explain
psx-runtime's $s6 read 66, 72 and 128 across three samples. It varies. The fade animates on our side too, so "the scale is pinned at the neutral value" — which the last commit put forward on the strength of three separate single-sample runs all happening to catch 128 — is disproved. That is worth stating rather than quietly moving on. Three single readings taken for other purposes looked like a pattern; the first tool that sampled repeatedly killed it immediately. The lesson is the same one this investigation keeps teaching: single samples of an animating quantity are not evidence about it, and the fix is always to measure variation rather than values. Second problem, in the tool rather than the theory: the oracle produced no samples and the report said only "oracle": null, verdict "incomplete". That states there is no comparison without saying whether the emulator was unreachable, never reached the PC, or refused the breakpoint — three causes needing three different responses, and the reason is known at the point of failure and nowhere else. Both samplers now return their reason alongside their values. It also threw away the half that DID answer, which in this run was the half that disproved the hypothesis. A one-sided result is not a comparison, but it is not nothing either, and the Studio now says so explicitly instead of showing an empty panel. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
3791f1e803 |
scale_trace: does the fade animate at all, on each side separately
The phase test came back tested: false — psx-runtime's scale never left 128 during the sampling window — and the tool then reported "region-differs" anyway, stating as established the exact thing it had failed to check. Fixed: an untested phase question now downgrades the verdict instead of being ignored. But a scale that refuses to move is a finding, not a failed setup step. Across every run the oracle's has come back different — 52, 64, 68, 76, 96, 126, 128 — while psx-runtime's has read 128 each time colour_inputs sampled it. And 128 is the NEUTRAL scale: x*128>>7 leaves x unchanged, so at that value this routine applies no fade at all. Bright unfaded polygons are exactly what the code would then produce, which is the reported symptom. scale_trace samples $s6 on both emulators and compares VARIATION rather than values. That is the point: it needs no frame alignment, no matching buffer half, and no shared animation phase. Those three have each produced a confident wrong answer in this investigation, and a register that sweeps on one side while sitting still on the other is a difference none of them can manufacture. Four outcomes, kept apart because they point four different ways: ours pinned and theirs sweeping (the suspected fault, called out specially when the pinned value is the neutral one), both sweeping (the fade runs, so the difference is in the values), both constant (most likely nothing is animating right now, and that says nothing), and theirs pinned while ours sweeps (the opposite of the expectation, not to be reported as if it confirmed it). This is a HYPOTHESIS with supporting observations, not a conclusion. One probe_regs run did read 72 on psx-runtime, so the register is not always 128 and a single reading either way settles nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
c12b65dcce |
colour_inputs: ask ONE emulator whether the region is phase-driven
The characterised difference is not conclusive, and the shape says why. At 0x000E4E00 the two hold the SAME palette entries offset by exactly four bytes; elsewhere psx-runtime holds f8 88 08 where the oracle holds f8 f8 b0; and in places psx-runtime holds 16-bit negatives (-100, -98, -19 — coordinates) where the oracle holds colours. That is consistent with a real layout divergence AND with the two being at different points in the animation, which they are: scale 128 against 72. The churn check does not settle it. It re-reads after 0.4s, so it catches a region being written continuously but not one rewritten only when the animation STEPS — which is exactly how a region can report static: true and still be phase-driven. Every "region-differs" result so far rests on that gap. So: sample the region on psx-runtime at two different values of $s6. One emulator, no alignment, no oracle. If its own copy moves with the scale, the region is animation-driven and comparing it across two emulators at different scales proves nothing — including every earlier result. If it does not move, the difference is real. A scale that never changes within the window reports "untested" rather than "not phase dependent". Those are different, and treating the second as the first would license the comparison on no evidence at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
751a2bf2aa |
colour_inputs: report WHAT differs in the region, not just how much
The run now says region_static: true with region_churn_bytes: 0 and 2747 of
23088 bytes differing. By the tool's own criterion that is a real divergence —
the region is not mid-write — and the report still only carried a number.
A count is compatible with several completely different causes: psx-runtime has
not written the data at all, psx-runtime is writing where the oracle does not,
or both hold values that disagree. Those point in different directions, and
"2747" cannot be read to tell them apart.
The report now carries the differing runs with both sides' bytes side by side,
coalesced so that a 4-byte entry differing in three of its bytes reads as one
finding rather than three, plus a plain statement of the shape:
* zeros on one side -> that side has not written this
* data on both -> a genuine disagreement, and whether it falls on
4-byte boundaries (whole entries) or not (stray bytes)
Also worth recording against the earlier reading: with the effect OFF the same
region is byte-identical and completely static across eight samples. So this
difference appears with the effect, which is what makes it interesting rather
than ambient.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Alex Vanderveen
|
5f212964d1 |
range_writers: attribute writes over an arbitrary span, as a Studio button
The colour investigation reached a REGION rather than a structure. With the effect off, 0x000E0BF8..0x000E6628 is byte-identical between the two emulators and completely static; with it running they differ. So the base colour data agrees and the divergence is in what the effect WRITES there — which makes "what writes this range" the next question, and packet_writers could not ask it because it assumes a packet layout to classify fields. Same instrument without that assumption: point it at any span and it reports which instructions store there, how many times, over what address span, and with what values. The span matters because a store walking an array is a different thing from one hitting a single field, and the values matter because a writer storing the same constant everywhere is clearing while one storing varied values is computing. Same two disciplines as packet_writers, for the same reasons. Tracing happens over a bounded number of STEPPED frames, since the game rebuilds its buffers constantly and a free-running trace attributes writes to whatever the addresses meant at some other moment. Code is captured while parked, because these are overlay addresses and the same PC decodes differently once another overlay loads. It also refuses to report an empty trace as a result: if the class that drives this code is not on screen it says so and names what is drawing instead, rather than returning no writers and letting that read as "nothing writes here". The Studio prefills the range from whatever Compare colour inputs last measured, so it does not have to be copied by hand between panes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
18c140badf |
colour_inputs: check the region is STABLE before calling a difference real
The region comparison reported 2747 of 23088 bytes differing and called it
phase-independent. Reading the same region directly, minutes later, gave ZERO
differing bytes — eight samples over ten seconds, byte-identical between the two
emulators, and psx-runtime's copy unchanged throughout.
The difference between the two readings is that the effect was running for one
and not the other. Both emulators are on the same scene right now (PolyFT4 x576,
PolyF4+semi x184, PolyFT3 x1 on each) with the effect off.
So "compare the whole region instead" only escapes phase if the region is not
itself being written, and this one is — while the effect runs. That is worth
knowing rather than papering over, because it splits the finding cleanly:
* with the effect OFF the region is byte-identical, so the base colour data
agrees between the emulators
* with it ON the two differ, so the divergence is in what the effect WRITES
there, not in the table it starts from
The tool now re-reads the region after a short delay and reports the churn. A
difference in a region that is changing gets its own verdict saying the two
sides may simply be caught at different moments, and pointing at the write
trace as the next instrument. A difference in a region that is NOT changing is
reported as real, which is the only case where that claim is earned.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Alex Vanderveen
|
af114bee4a |
colour_inputs: compare the region, because $s4 tracks the animation
The capture works now — $sp reads 0x1F800234, scratchpad, matching the oracle — and the run finally compared through both emulators' own pointers. It reported "source-differs", and that is still not trustworthy. $s4 MOVES. Across two samples psx-runtime's went 0x800E2634 -> 0x800E4634 while $s6 went 72 -> 128; the oracle's moved the same way. The pointer tracks the animation, so a window at each side's own $s4 compares different ENTRIES of the same array whenever the two are at different moments — which they always are. The 0x1A30 gap between the two source addresses is not the 0x8000 double buffer and not a divergence; it is elapsed time. This is the third time a phase difference has been reported as a data difference (frame numbers, buffer halves, now array offsets), so the fix is the same shape as before: measure something phase cannot move. When the two scales disagree, the whole enclosing region is compared instead. If the arrays match, the data is fine wherever each pointer sits; if they do not, that is real and independent of where either side happened to be. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
287d9a1c22 |
probe: actually FILL the register array, and refuse a capture that did not take
Every register came back 0x00000000 — including $sp, which cannot be zero in
running code. The struct had gpr[32] and the JSON emitted it; nothing ever
wrote to it. The patch that added the fill did the replacement in memory and
never wrote the file, and the next edit re-read from disk without it. Two
thirds of a change, compiling and running and reporting 32 zeros.
Zero then flowed all the way downstream without anything objecting: 0 minus the
load offset masks to 0x1FFFFFF4, which looks like an address, so colour_inputs
read "psx-runtime's table" from unmapped memory and reported a difference
against the oracle. The same document carried "differing_bytes": 0 and
"source_identical": false — the read had come back short, so zip() compared
nothing while the buffers still differed.
Three guards, because one bad value crossed four steps unchallenged:
* probe_registers refuses a capture where every register is zero and says so,
rather than returning zeros as data
* plausible_pointer() rejects anything that is not a RAM address before it is
used as one, including the masked underflow that got through
* the comparison refuses unequal read lengths instead of reporting a
difference between a buffer and nothing
Scratchpad is deliberately not a plausible table pointer: $sp lives there, and
accepting it would compare the wrong region while looking entirely reasonable.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Alex Vanderveen
|
af7b6f379e |
read the whole reply, and tell "not running" apart from "not a block leader"
The multi-line fix did not work, because I fixed the read loop and left the parse. raw() kept reading until the JSON document was complete and then still did split(b"\n", 1)[0] before parsing it — so pc_probe_dump was truncated in exactly the same place, and the error still named the server. Verified against the live runtime this time rather than reasoned about: 843 bytes, 6 newlines, parses. The whole buffer is parsed now (JSON treats the embedded newlines as whitespace), with the first line kept as a fallback for a peer that sends more than one document down a connection. Underneath that, the probe was reporting the wrong cause. Every candidate came back with count 0, and the message offered two possibilities: the addresses are mid-block, or the code did not run. It was the second — the effect was not on screen at all — and that is checkable rather than guessable. class_on_screen() walks the display list and says which it is: "PolyG4+semi is not being drawn, so this code never ran and no candidate could fire. Currently drawing: PolyFT4 x576, PolyF4+semi x184, PolyFT3 x1." Several tools depend on a routine actually running — a write trace over its packets, a block probe at one of its instructions — and each reports its own symptom when it is not: no writes recorded, no candidate fired, no table found. All three are now one plain statement. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
e0a5709877 |
fix: a Studio crash on null JSON, and a truncated multi-line reply
Clicking a colour-only row aborted the Studio:
type_error.302: type must be string, but is null
nlohmann's value<T>() THROWS when a key exists but holds null, rather than
returning the default. colour_inputs wrote "block_leader": null for a probe
that had not fired, and reading it took the process down. A viewer must never
die on the content of a file it is shown, so this is fixed on both sides:
* the report omits absent keys instead of writing nulls — "missing" and
"present but null" should not be different things for it to express
* every string and integer read in the newer loaders goes through null-safe
accessors, so a malformed or unexpected document degrades to defaults
Underneath it was a real failure the crash was hiding:
pc_probe_dump: malformed reply (Expecting value: line 1 column 80)
That handler builds its JSON with repeated send_line() calls — header, one line
per slot, one per sample — so the first newline lands in the MIDDLE of the
document. DebugConn stopped at the first newline and got a truncated object,
and the error named the server rather than the reader. It now stops at a
newline only when what has arrived is a complete JSON document, counted by
braces with quotes and escapes tracked (a parse attempt per chunk would
dominate the read on a megabyte of hex).
So the probe never actually reported $s4; the whole run fell back to the
content search without that being visible as a failure.
frames_load_test now loads packet_writers/gte_check/colour_inputs/lockstep from
a real directory too. That mode would have caught this before it reached a
click: the crashing file loads cleanly under it now.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Alex Vanderveen
|
e82927a408 |
colour_inputs: read psx-runtime's OWN pointer instead of assuming the oracle's
The last three runs were inconclusive for the same reason: the tool had the oracle's $s4 and no way to get psx-runtime's, so it read the oracle's ADDRESS on psx-runtime. That address holds an index list there, not colours — a fact about memory layout, not about the colour data, and the tool kept reporting it as though it were about the colour data. Now both sides are read through their own pointers. psx-runtime's comes from the block-leader probe: $s4 is callee-saved and set outside the loop, so its value at block entry is its value at the store. The comparison is finally between the table each routine actually reads. The content search stays as a fallback for when the probe cannot fire, but it is now visibly a fallback -- the report records `compared_by`, and the Studio says outright that a content search cannot know which table psx-runtime's code reads. Two answers of different quality should not look alike. A ±0x8000 address delta is called out as the double buffer rather than as a difference, which is what it was the whole time. probe_registers() is shared between the two tools rather than duplicated. Two copies of "find the enclosing block" would drift, and the failure would be silent: both would return real register values, from different blocks. Also adds a bare "Read psx-runtime registers" button, since psx-runtime has no PC breakpoint and this is otherwise the only way to see what its code held. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
ed96761c2a |
probe: capture every GPR at a block leader, and find the enclosing block
The $s4 gap blocked the same question twice, so it is closed with the cheaper of the two options rather than by teaching the recompiler to emit per-instruction hooks. pc_probe captured seven hand-picked registers -- t0, fp, v0 and four draw-specific fields -- chosen for an earlier investigation. The next question always wants a different one, and this time it was $s4 and $s6, the source pointer and scale for the packet colour routine. It now captures all 32, named exactly as the DuckStation oracle names them in pc_hit_last so the two sides compare without a translation table. 64 samples x 32 registers is 8 KB. The other half is that the probe fires at basic-block LEADERS, and a colour store at 0x8006844C is mid-block, so arming it directly never matches. The block is found by arming a spread of candidates BACKWARDS from the target -- a leader is at or before the instruction it contains, and searching forward would find the next block, whose registers describe what happens afterwards. Whatever fires is a real leader; the nearest one below the target encloses it. The accuracy cost is stated in the output rather than left to be remembered. Registers are read at block ENTRY: for callee-saved registers set outside the loop ($s4, $s6) that is the same value as at the target, and for one the block computes ($v0, $t6, $a0 there) it is not. Each requested register is flagged by class, because a value that is real, plausible and taken from the wrong instant is the hardest kind to catch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
3fe8a5a5b9 |
colour_inputs: an exact miss is not absence — report how much IS there
The tool reported "table-absent-on-native" for every colour writer, and that
claim was too strong. Checking psx-runtime's RAM by hand:
* the oracle's full 32-byte table is indeed absent as a contiguous run
* but its first 4 bytes appear SIX times, on a regular 0x18 stride
* and the region at 0x0E4600 is BYTE-IDENTICAL between the two emulators —
same palette (f85050, f8f8b0, 080808, f8f8f8), same order
So the data exists. What differs is the arrangement, and that points somewhere
completely different from "this was never built": upstream data generation
versus a layout or record-size difference.
At the oracle's own pointer address (0x0E4BF8) psx-runtime holds 16-bit values
where 0x38 recurs every five halfwords followed by four ascending numbers --
an index list, not colours. The two emulators keep different structures at that
address, which is itself the finding.
The search is now graduated (32/16/12/8/4 bytes), reports the longest prefix
present, how many times, and on what stride if it is regular. "Absent" is
reserved for genuinely finding nothing.
Both verdicts now also state what the tool CANNOT determine: which table
psx-runtime's own code reads. That needs its $s4, and psx-runtime has no PC
breakpoint. Leaving that unsaid is what let the earlier verdict read as more
conclusive than it was.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Alex Vanderveen
|
651acb42e5 |
lockstep: say what a skip reason MEANS, not just how many
The segment-level run came back with 10,619 checked and 74,111 skipped, and the weak-coverage guard correctly refused to call it a result. But "74,111" on its own does not distinguish the two situations that produce it, and they call for opposite responses: something is wrong, or this mode structurally cannot cover this game. Here it is the second. 99% of the skips are segments containing an interrupt, which cannot be replayed — so in an interrupt-heavy game segment granularity skips almost everything by construction. The tool now names the dominant reason when one accounts for 60% or more, explains it, and in this case says outright to use block granularity instead. That is not hypothetical: the block-level run over the same game covered 5,679,898 blocks with ZERO skips. Same instrument, same question, and the two modes differ by four orders of magnitude in what they actually compared. No single reason above 60% names no cause at all, since inventing one would be worse than staying quiet. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
499deb0c87 |
lockstep: keep the document type out of the divergence type's way
The first real run came back clean over 5,679,898 blocks with zero skips — and the Studio would have rejected its own output. The engine calls the DIVERGENCE type "kind", every report document in this toolchain calls the DOCUMENT type "kind", and merging the reply as-is overwrote one with the other. On a clean run that wrote "kind": "none", and the loader's type check then failed on a file the tool had just produced. The C++ loader carried a comment warning about exactly this collision. Writing the warning and not preventing it is worse than not noticing at all, since the comment reads as though it had been handled. The divergence type is now div_kind and the document type is restored after the merge. Also sharpened what "clean" is allowed to claim. It said recompilation was not the fault; that overstates it. Lockstep compares the compiled path against the interpreted one INSIDE one process, so it exonerates the translation and nothing else — the memory subsystem, HLE, DMA and device emulation are common to both paths, and a fault living there produces identical results on both sides and is invisible here. A clean lockstep narrows the search; it does not close it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
762a62e95f |
studio: lockstep — compare compiled code against our own interpreter
The code bisect, as a button. It runs the same guest code twice inside psx-runtime -- once compiled, once interpreted -- and reports the first place they disagree, with the register or memory value each produced. This is the right instrument for a recompilation bug because it needs none of what every cross-emulator comparison in this investigation has had to fight first: frame alignment, address correspondence between two allocators, and getting both emulators to the same point in the game. Each of those has produced a confident wrong answer at least once here -- most recently "source-differs", which turned out to be the two sides sitting on opposite halves of a double buffer. It also has a target rather than being a fishing expedition. The colour routine is built from lwl/lwr/swl/swr and stages entirely through scratchpad, and unaligned access is the classic recompiler weak spot. The load-bearing detail is the verdict, not the flag. "found: false" is produced by two completely different situations -- the compiled code matched everywhere it was compared, and nothing was compared at all -- because the segment comparator skips whatever it cannot replay. A run that skipped everything is indistinguishable from a clean one unless the counters are read alongside it, so: nothing checked -> INCONCLUSIVE, said in those words more skipped than checked-> WEAK, with the suggestion to use block granularity checked and quiet -> clean, naming how many units backed the claim divergence -> kind, pc, both values, and the memory ops before it Reporting an unearned pass here would be worse than reporting nothing, since this is the one instrument whose result does not need caveating. Divergence kinds that mean the comparator gave up (path-cap, unsupported) are called out as limits rather than findings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
d40eebd3e5 |
colour_inputs: find the table by content — addresses do not carry over
The tool ran clean and reported "source-differs". The report was wrong, and the
data it printed is what shows why.
The oracle's half is solid and self-confirming: its table at 0x000E4BF8 holds
(248,80,80) and (8,8,8), and scaled by the recorded factor of 126 those give
(244,78,78) and (7,7,7) -- the colours in its own display list. That is the
colour table, verified by reproducing its own output rather than by assumption.
psx-runtime's bytes at that same address are not a colour table at all: 24 of
24 odd bytes are zero, so it is an array of 16-bit values. Two different
structures, not one structure with different contents.
The assumption underneath was that the table sits at the same address on both,
and it does not. Measured in the same report: the oracle's packet pointer was
0x8010D9A0 while psx-runtime's packets were at 0x00115078..0x00115974. The two
allocators place the same structures at different addresses, so reading the
oracle's pointer on psx-runtime reads unrelated memory -- and the difference
reported was entirely the tool's own doing.
The table is now LOCATED rather than assumed: search psx-runtime's RAM for the
oracle's actual bytes. Three outcomes instead of two, and the new one is the
most informative:
* found, identical -> input fine; scale or arithmetic is at fault
* found, differing -> the table itself diverged
* not found anywhere -> psx-runtime never built this table at all, so the
divergence is upstream of the colour routine
A structural check refuses to read a 16-bit array as a colour table, and a
needle shorter than 12 bytes is refused because a short match in 2 MB is
meaningless and would return an arbitrary address.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Alex Vanderveen
|
ae38ab87d7 |
colour_inputs: pace for a PAUSED oracle, and always un-park it
"pc_hit_last: server closed without replying", and every retry after it worse. Two bugs, neither of which failed in a way that pointed at its cause. A DuckStation breakpoint PAUSES the emulator when it fires. A paused oracle pumps its debug socket only from a Qt idle timer -- 100 ms with a gamepad attached, 1000 ms without -- so a headless oracle is a ~1 Hz server the moment the breakpoint hits. Polling it every 100 ms stacks up connections it cannot accept, and that surfaces as "server closed without replying": a network-shaped error with a pacing-shaped cause. Polling is now 1.5 s, the wait has headroom, and one dropped reply around the hit is survived rather than treated as failure, because the hit parks the emulator mid-exchange. Worse, the tool cleared the breakpoint and exited WITHOUT resuming. So the first run left the oracle parked at 1 Hz permanently and every attempt after it failed for a different reason than the first -- which is a miserable thing to debug from the outside. It now resumes in the finally block, and resumes BEFORE arming too, so a previously stranded oracle recovers on its own instead of needing a restart. ORACLE_PAUSED_POLL_S and oracle_resume() live in psx_gpu_frame because this is the third time this behaviour has cost time: it also explains the 2 MB read that could not finish while parked, and the earlier decision never to pause the oracle for a snapshot. Studio gets a Resume button next to the oracle's status for the same reason -- the symptom (everything times out) does not resemble the cause (it is parked). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
b5807378fb |
colour: compare the INPUTS, not the outputs — a Studio button
packet_writers found the code; this says which of its two inputs is wrong. Every vertex colour is source_rgb * $s6 >> 7, so a wrong colour is either a wrong SOURCE or a wrong SCALE, and those point in opposite directions: one means look upstream at whatever fills the table, the other means look at this routine. Guessing between them is how an investigation goes sideways. The obvious approach — break at the PC on both emulators and diff registers — is not available. The oracle has pc_break and captures all 32 GPRs on hit; psx-runtime has no equivalent, and adding one means emitting per-instruction hooks from the recompiler, which is a different project. Its pc_probe fires at basic-block leaders with a fixed register set. The asymmetry turns out not to matter, because the register we need from the oracle is a POINTER. Once it reports $s4, the source table's ADDRESS is known, and the same address can be read on both sides with read_ram, which both support. So the comparison that settles it is of memory, not registers, and it needs the breakpoint on one side only. Reports the scale it saw as well, since a neutral scale means the code takes the branch that skips modulation entirely — a different path with different consequences. The oracle failing to reach the PC is treated as a normal outcome and explained, not as a crash: these are overlay addresses, so they only exist while the right overlay is resident and the oracle has to be at the same point in the game. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
2771968757 |
disasm: name the unaligned and cop2 forms the colour path is built from
The captured listings made the colour computation readable, except for the
instructions that actually carry the data. LWL/LWR/SWL/SWR printed as
"op22"/"op26"/"op2a"/"op2e" and GTE ops as bare cop2 words -- in the exact
window under investigation, and covering the exact question (is the GTE
involved?) the investigation was trying to answer.
With them decoded, the per-vertex colour path reads end to end:
lwl $t6,-9($s4) unaligned load of the source RGB
lwr $t6,-12($s4)
swl $t6,67($sp) staged unaligned into stack scratch
swr $t6,64($sp)
beq $s6,$t3,skip neutral scale passes through unmodulated
lbu $v0,64($sp) per channel:
mult $v0,$s6 value * scale
sra $v0,$t6,7 >> 7
sb $v0,48($sp)
lw $v0,48($sp) reassemble the three bytes
sw $v0,4($a3) -> packet colour word
So every vertex colour is source_rgb * $s6 >> 7, and the source reaches the
packet only through LWL/LWR -> SWL/SWR -> lbu. Geometry does not go through
that path, which fits the evidence exactly: identical vertices, wrong colours.
cop2 needed splitting too -- bit 25 separates register moves from GTE commands,
and rt in lwc2/swc2 indexes a GTE register rather than a GPR, so printing "$t4"
there would send the reader after the wrong thing.
Checked the recompiler's LWL/LWR by hand while here; the keep-mask formulas are
correct in all four byte-offset cases. Noted rather than concluded: reading
code is how the last four hypotheses in this investigation went wrong.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Alex Vanderveen
|
88fedc3694 |
parity: align BOTH emulators, and stop inventing display-state differences
Two bugs, both visible in every parity run so far.
Alignment only ever steered DuckStation. It read psx-runtime's frame, drove the
oracle at it, then took screenshots -- while the native run kept going. So the
two images were of different frames essentially always, and the tool printed
that as a warning to read past rather than as the reason the comparison was
void. It also failed outright whenever the oracle had already passed the target,
because run_to_frame cannot go backwards ("target frame already passed").
Neither side can rewind, but both can move FORWARD: psx-runtime by stepping
(pause/step are real again -- the "native run cannot be paused" in that warning
had been stale since they were restored), DuckStation by run_to_frame. The
target is therefore whichever emulator is further ahead, and the one behind
catches up. Both are parked for the screenshots, and the game is handed back
afterwards rather than left relying on the park's silence timeout.
The display-state block was pure noise. The two servers name the same facts
differently -- display_x/width/height against display_area_x/disp_w/disp_h --
so comparing raw keys reported native=None for every field. That reads as the
runtime failing to report its display state and is really a vocabulary
mismatch: a difference invented by the tool, printed under a heading saying the
emulators diverged. Names are now normalised, display_disable is inverted
rather than merely renamed onto display_enable (or every frame would differ),
and a field only one side reports is marked one_sided instead of counted as
disagreement.
Studio leads with the alignment verdict, because every other number in the
report is conditional on it.
Also: the resume-on-error patch first landed in align_frames() instead of
main(), referencing an `args` that does not exist there -- caught by extending
the undefined-global bytecode guard to gpu_parity, packet_writers and gte_check.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Alex Vanderveen
|
c7717aecec |
writers: capture the code while parked, and show it in the pane
The trace now works — a clean run separates six instructions that write ONLY colour words (64 each) from four that write ONLY vertex words (64 each, which is exactly 4 vertices x 64 packets), with no mixed rows. Those six are the answer to "what computes these colours". But reading them was still a second, separate step, and that is not safe here. The PCs are at 0x80068xxx, above the main EXE at 0x8003F000, so they are OVERLAY code: the instruction at a given address depends on which overlay is resident. Disassembling after the scene moves on decodes whatever replaced it and looks entirely plausible. So the listing is captured during the trace, while the emulator is already parked and the right overlay is loaded, and travels in the report. Includes a small MIPS decoder — enough to read a store and the arithmetic feeding it — with tests on the forms that appear around a packet write. Store operand order is pinned because rt is the source and rs the base, and swapping them yields a different instruction that still reads sensibly. Shifts are pinned because taking the shift amount from rs instead of shamt is a classic decoder bug that produces plausible nonsense. cop2 is pinned so GTE ops cannot be silently decoded as something else. Those tests immediately earned themselves: disasm_around mixed a virtual PC into the offset arithmetic and compared it against a masked one, so no line was ever marked as the target. The listing would have come back with nothing highlighted, reading as "the instruction is not in this window". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
d690776fb3 |
step: count FRAMES, not calls — and never present an empty trace as a result
`step n=1` could park before a single frame completed, so the write trace ran over nothing and the Studio showed an empty table with no indication anything had gone wrong. Measured before the fix: step n=10 advanced 2 frames, step n=1 advanced 0. debug_server_wait_if_paused() is called from three places -- the vblank path in main.cpp and two in dirty_ram_interp.c -- and each one decremented the step counter, so it drained up to 3x per frame. It now only counts a decrement when s_frame_count has actually changed, and handle_step stamps the frame it is leaving so that one does not count. Verified live: n=1 advances 1, n=5 advances 5, n=10 advances 10. This also fixes the Studio's Step button, which had the same defect. packet_writers.py no longer trusts the paused flag to mean a frame ran -- it compares frame NUMBERS across the step and errors out if nothing advanced. And an empty trace is now an explicit error rather than a report with zero rows: the packets had just been walked, so "no writes" means the trace window was wrong, not that nothing writes them. An empty table is indistinguishable from a finished search, which is the worst way for a tool to fail. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
9e0401cbef |
oracle: "running" must mean OUR oracle, not "something owns the port"
state() probed port 4371 and reported whatever answered as the oracle. The port is shared -- another emulator, a stale build, an unrelated service -- and a bare probe cannot tell whose it is. An install that did not exist would report "answering", which tells a GUI to offer Stop for something it never started. It also made this suite pass or fail depending on what else happened to be running: two duckstation_oracle tests failed three separate times during this work, and each time I read them as environmental noise rather than as the defect they were reporting. They were right. Ownership is the pidfile the tool writes when it starts one, checked with signal 0. Anything else on the port is a conflict and gets its own field (port_conflict) rather than being folded into the state ladder -- that ladder describes how far the INSTALL has got (absent -> fetched -> built -> installed -> answering) and a GUI picks its button off it, so it has to stay monotonic in installation, not in what happens to hold a socket. The suite now passes with a live oracle answering on 4371. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
2685b3836d |
writers: trace one stepped frame; flag aliased attribution
The first real run produced a table that looked like an answer and was not. Ten instructions, every one of them splitting about evenly between colour and vertex words (93:89, 93:88, 92:90, ...). A single MIPS store writes ONE field, so an even split cannot be what the code does -- it is what stale addresses look like. 11% of writes also landed outside the mapped layout, and the buffer base moved between two walks minutes apart (0x0010D1C4 -> 0x0010D078). The packet buffer is rebuilt at a different address most frames, so a map built from one walk describes different fields a frame later. A free-running trace therefore cannot attribute anything, and it fails silently: the smear reads as "this instruction writes both", which is a plausible-sounding and completely wrong conclusion. Now the game is parked, walked, stepped a bounded number of frames with the trace armed, then walked AGAIN to confirm the addresses still mean what they meant. Any that changed role are reported. This is what the restored step machinery is for. Two independent guards, because the operator should not have to notice this: the second walk catches a moved buffer directly, and an even colour:vertex split across most instructions is detected on its own. Either one marks the report unreliable, and the Studio then greys the table out and disables disassembly rather than inviting someone to read an artefact. --watch is kept but documented as the legacy free-running mode that produces exactly this failure. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
9d341a434c |
studio: a Shading pane — colour writers and GTE check, no CLI
The two instruments that narrow a colour-only render bug were CLI-only, which is useless while juggling an emulator to reach the scene the bug needs. Both are now buttons. Packet colour writers: watches the packet's colour words and reports which instructions store to them, split colour vs vertex, with the colour-only rows flagged — those are the ones to read when geometry is right and colour is not. Attribution is by ADDRESS because an ordering-table game reports the same submit PC for every packet on screen, so "which function drew this" has one answer per frame. Select a row and disassemble around it in place. GTE self-check: re-derives the depth-cue interpolation from the GTE's own recorded inputs against the hardware description, and reports three distinct outcomes rather than pass/fail. "Not used" is its own verdict: this game issues zero INTPL, and an operation that never executes cannot be the cause, which points somewhere different from "executed and correct". Both tools gained --json. disasm_ram.py gained a real CLI (--pc/--count/ --before, marking the named instruction with >>) while keeping its positional form; it previously had no --help and took bare positional args, which is not something to build a GUI button on. --before exists because the interesting address is a STORE the write trace named, and reading it needs the instructions BEFORE it, where the stored value was computed. The pane says plainly when the class is not on screen, and lists what IS drawing, rather than reporting on whatever happens to be in the buffer. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
296981c9bc |
gte: point the instruments at the colour bug; they clear the GTE
Two new instruments, and a result that narrows the search a long way. gte_check.py re-derives the GTE's depth-cue interpolation from its own recorded inputs and compares against the hardware description, written from the spec rather than copied from runtime/src/gte.cpp -- a reference copied from the implementation under test agrees with it by construction and proves nothing. It needs no oracle: a mismatch is a bug in our GTE, a match means the inputs were already wrong when they arrived and the fault is upstream. That distinction is the actionable part. Pointed at the live runtime it reports something better than a match. Legend of Mana issues ZERO INTPL operations (nintpl=0 across every recorded frame), and nsat=0 -- nothing saturates in projection either, across 1081 projections a frame. The depth-cue path is not merely correct here, it is not used. Vertex colours are written by plain CPU stores from overlay code (dma=-1, PCs above 0x8003F000, invisible to psxrecomp-analyze), not computed by GTE lighting. So the colour divergence is recompiled CPU arithmetic in an overlay, not the GTE library. That is worth having ruled out; it also means the fingerprint's pc_hash is the right next instrument rather than any GPU-side tool. packet_writers.py finds which instruction writes a primitive's COLOUR words, by address rather than by function -- an ordering-table game reports one submit PC for every packet, so attribution cannot answer it. Field addresses are derived from each decoded packet's own layout. A first attempt assumed a fixed 0x24 stride and classified every write in the buffer against it; when the scene changed the assumption slipped and it produced a confident table naming the wrong instructions, with nothing in the output to say so. Stride depends on the opcode and a buffer holds several, so the layout is now read off the packet. The tool says plainly when the effect is not on screen instead of reporting on whatever happens to be in the buffer. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
1cbf5d99b1 |
dlist: compare colour STRUCTURE, which phase cannot fake
With both walks coherent, the native-vs-oracle pair says something clear. Geometry matches: 761/800 paired primitives carry identical vertices, and all 64 additive quads of the effect match exactly. The guest builds the same picture on both. Colour does not, and the measurement that shows it is distinct-colour COUNT. Every other colour comparison needs the two emulators on the same frame, which nothing short of deterministic replay achieves. But a procedural effect reuses a handful of colours across all its primitives, and that stays true at every phase -- an effect at a different moment has DIFFERENT colours, not more of them. So an order-of-magnitude gap is evidence about the computation rather than about when each side was sampled. For PolyG4+semi|B+F, the same 64 primitives: oracle 3 distinct colours, 8 per-prim patterns psx-runtime 153 distinct colours, 54 per-prim patterns 51x. And the clearest case needs no phase argument at all: PolyFT4|opaque, 584 primitives on both, two distinct colours on both, different VALUES -- (64,32,32)/(24,24,24) against (80,64,64)/(40,40,40). Those are static background quads; they are not animating between the two samples. Meanwhile PolyF4+semi|B-F matches exactly at (64,64,64) on both. Flat colour agrees, shaded colour does not. That places the divergence in the guest's colour computation, upstream of the renderer -- not in geometry, not in blending, and not in the rasteriser. compare() also tolerates a report with no per-primitive detail now, so an older dlist.json still yields a class delta instead of raising. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
4f8a2edf86 |
dlist: re-read the list parked; refuse lopsided colour verdicts
Both changes come from reading a real native-vs-oracle pair. The good news in that pair: all 64 additive quads carry BIT-IDENTICAL vertices on the two emulators, and every (opcode, blend) class count matches. Whatever the render bug is, the geometry the guest builds is not it. The bad news was in how the oracle's half was captured. Reading 2 MB from a running emulator takes ~128 round trips, and the game rebuilds its ordering table underneath -- that walk spanned 6 frames and came back with a whole frame of additive quads carrying just 4 distinct colour values across 768 channels. A torn read that looks exactly like data, and I nearly read it as evidence that the runtime's colours were corrupt. The fix is cheap now that the scan reports the list's span: the first pass finds WHERE the list is, then the emulator is parked and only that span (a few KB) is re-read, so the decoded list comes from ONE instant. A few KB is affordable even at the 1 Hz the oracle drops to while parked, which is what made this impossible when the read was the whole of RAM. Walks now report whether they are coherent, and say THIS WALK MAY BE TORN when they are not. gpu_colour_parity refuses a verdict past a 4x sample imbalance. It had reported "differ" from 3328 vertices against 256 -- roughly thirteen frames of an animating effect against a single one. Those differ by construction and say nothing about the emulators. Also fixes two tests that encoded the old ping-based frame(): one mock answered every command with a frame field, so it answered `frame` too and quietly robbed the test of the ping fallback it existed to check. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
cf4add4ea6 |
fingerprint: per-column coverage gate, and don't name the window edge
Both fixes come from the first real run against a live pair. A column where one side records thousands of writes and the other records ZERO sailed through the global coverage check, which only compares total write counts. It then reported a confident "FIRST DIVERGENCE" for scratchpad, where the truth was that the oracle had never observed a single scratchpad write (0 against psx-runtime's 719264). That is a coverage hole, and calling it a divergence would send someone looking for a guest fault that does not exist. Each column now checks its own counts and says NOT COMPARABLE instead. Divergence on the very first frame of the window is not a finding either: it means the fork happened BEFORE the window, and the frame named is just wherever the window happened to start. It now says so and suggests widening. The global ratio band was 0.5x..2.0x, and a live pair sat at 1.98x -- the oracle's CPU-store hook does not see DMA-written RAM -- squeaking through as "comparable". Tightened to 0.67x..1.5x. Working, verified live: the oracle records fingerprints in interpreter mode, and the guest cycle clocks agree to 0.31% drift across a 150-frame window, which is what makes cycle-based alignment a sharp test rather than a guess. Still open: the oracle sees no scratchpad writes at all, and its main-RAM write count runs well under psx-runtime's. Until that coverage gap closes the tool will keep refusing to name a divergence frame, which is the correct behaviour. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
769c35a01d |
fingerprint: fix frame(), page the pull, gate on guest cycles
Three things stood between the fingerprint and being usable, all found by
running it against a live pair rather than by reading it.
DebugConn.frame() asked `ping` for the frame number. psx-runtime intercepts
ping in a lock-free fast path that never reaches the command dispatcher, so its
reply carries only {ok, pong, io_thread} -- no frame. Every native run
therefore reported frame 0, silently, and anything aligning two emulators by
frame number was comparing against zero. This is the "native frame 0" that has
been showing up in gpu_parity output all along. The oracle DOES put a frame in
its ping, which is what made it look like an oracle quirk rather than a
native-side blind spot. frame() now asks the `frame` command, which both
answer, and falls back to ping.
frame_fingerprint replies are now paged (64 frames per call). A whole ring in
one request is over a megabyte, and the oracle's socket truncates anything past
~16 KB, after which the read blocks forever waiting for a newline that never
comes. Paging here is not an optimisation; it is the difference between working
and hanging.
Added an alignment gate ahead of the coverage gate. A frame number means
"frames since THIS emulator booted", so overlapping frame numbers prove
nothing: start one thirty seconds after the other and both report a frame 500
that are half a minute apart. A diff across that names a divergence at the
first frame compared, every time, and it looks exactly like a real finding.
The gate compares guest cycle counts and refuses when they disagree. Measured
on a live pair, the two agree on cycles-per-frame to 0.3% (564480 vs 566204),
so this is a sharp test, and it correctly refused a mismatched pair on its
first live run.
The oracle now seeds ExecutionMode = Interpreter. The store hook lives in the
interpreter's write path; a recompiler oracle inlines stores, never reaches it,
and would compare clean against anything. A false "no divergence" is worse than
a slow reference.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Alex Vanderveen
|
49e30e9d13 |
dlist: walk both emulators concurrently and compare
Walking the two sides one after the other lets the effect animate in between, so the class delta ends up describing elapsed time as much as any real divergence. --both runs each side in its own thread (DebugConn opens a fresh socket per command and holds no shared state) so the two snapshots land as close together as the reads allow, then prints the per-class delta and writes dlist-native.json and dlist-oracle.json. The two sides are deliberately NOT symmetric: psx-runtime is parked for its snapshot, because its park loop keeps serving commands, while the oracle is read running -- pausing DuckStation drops its debug socket to the Qt idle timer (1 Hz with no gamepad) and the read cannot finish. Since the oracle is read running, each side reports the frame number either side of its snapshot and warns when it advanced more than two frames during the read: such a walk may straddle a rebuild of the list, and that should be visibly suspect rather than quietly reported as fact. The summary line now names only the files actually written. It previously claimed to have written both even when neither side could be reached, which would leave a stale file from an earlier run looking like fresh output. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
ebe0757cbc |
tools: cover the whole ordering table; stop deciding colour on peak
Two wrong answers the results made visible. find_display_lists walked ONE BUCKET and called it the display list. An ordering table is an array of link-only entries, each chaining to a run of primitives; those links were excluded from the graph, so every bucket's first primitive looked like an unpointed head. The output looked complete and was not: a real walk came back with 946 primitives and no PolyG4+semi|B+F at all, while the GP0 ring on the same runtime held 3008 vertices of exactly that class. The bucket it picked simply did not contain the effect under investigation. Link entries now join the graph (guarded so a zeroed word, which reads as length 0 pointing at address 0, cannot flood it), so the scan finds the table head and covers every bucket. The colour verdict rested on `peak`, which is a maximum and therefore grows with the number of samples drawn. Comparing 3008 vertices against 1024 it reported "differ" for two distributions whose medians (13 vs 17), p90s (144 vs 126) and means (49.8 vs 42.0) all agreed -- a confident wrong answer, which is worse than none. The verdict is now histogram intersection over normalised bins, invariant to both sample count and animation phase, with an explicit "inconclusive" band. On the reported data the old rule says differ and the new one says inconclusive, which is what the evidence supports. Studio: the Display list pane kept one dlist.json, so walking the oracle destroyed the runtime's walk and the two could never be compared -- the question worth asking. Each side now writes its own file, both are held at once, and a class-count table shows the delta. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Alex Vanderveen
|
c7ca393fa7 |
tools: fix undefined import, stop the oracle path following DMA
Two bugs the GUI surfaced, both from the same root: an edit that was assumed
to have applied rather than checked.
* gpu_display_list.py raised "NameError: find_display_lists is not defined".
A patch script added that import with an unasserted str.replace(); the
anchor text had drifted, the replace silently did nothing, and the file
still parsed and imported cleanly. It only failed when a user clicked the
button. Every other edit in that script asserted its anchor.
* gpu_colour_parity.py sampled nothing from the oracle ("no matching
primitives sampled") because its oracle path still resolved the display
list through DMA channel 2's MADR, which now correctly returns None -- a
finished linked-list transfer leaves the 0xFFFFFF terminator there. The
fix for that landed in the inspector and was never carried across. It now
scans for the list, locks onto the root, and re-reads only the span the
list occupies instead of 2 MB per sample. It also no longer pauses the
oracle: DuckStation pumps its debug socket from the emulation loop, so a
paused oracle drops to the Qt idle timer (1 Hz with no gamepad) and the
reads cannot finish.
test_tool_imports.py guards the class rather than the instance. It walks each
tool's BYTECODE for LOAD_GLOBAL and checks every name resolves, which reaches
branches no test executes -- an undefined global is only an error on the line
that runs it, so importing and parsing prove nothing. Verified by
reintroducing the exact bug: it fails with the missing name.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Alex Vanderveen
|
76a43de2ee |
oracle: port frame_fingerprint, add first-divergence diff
Gives the DuckStation oracle the same per-frame write fingerprint psxrecomp
already had, so the two can be compared by "which frame did execution fork"
rather than by pictures. Comparing pictures needs the two aligned to the same
instant, which nothing short of deterministic replay achieves; the fingerprint
is cumulative, so frame N's hash answers whether the whole write history up to
frame N matched, and phase does not enter into it.
Ported byte for byte from debug_server.c -- same FNV-1a constants, same JSON
field names, same decision to keep main RAM, store-PC, device registers, and
scratchpad in SEPARATE columns. Folding device churn into the RAM hash is what
hid a real first divergence before: the RAM hash sees only the aftermath of a
device-interaction fork, so it points frames too late.
Two traps are closed rather than left for the reader:
* The store hook lives in the interpreter's write path (hooked in
WriteMemoryByte/HalfWord/Word so swl/swr/sc are covered, not at the
sb/sh/sw cases). A recompiler build inlines stores, never reaches it, and
would record almost nothing -- so the reply carries "interpreter" and
fingerprint_diff.py refuses to report on a build that is not in that mode.
* Values are truncated to the width actually written before hashing, matching
what memory.c hands fp_record_write. Hashing the untruncated register would
make every byte store disagree and every frame "diverge" for no guest
reason.
fingerprint_diff.py compares write COUNTS before it compares hashes and refuses
to name a divergence frame when they do not track. The oracle's hook does not
see DMA-written RAM; a mismatch from that is indistinguishable from a real fork,
and a confident wrong answer would send you hunting one that is not there.
Unverified: hash equality between the two has not been demonstrated on a live
pair yet -- the oracle needs a rebuild first.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|