5 Commits

Author SHA1 Message Date
Matthew Stanley e6ce5006b4 docs: move root and stale docs into docs/ and docs/internal/
The repository root carried seven markdown files that were not onboarding
material, plus two that had been superseded by the code they described. This
sorts them: reader-facing docs to docs/, working notes to docs/internal/, and
the two superseded ones merged into the docs that replaced them.

Moved to docs/:            WIDESCREEN.md, ENHANCEMENTS.md, TCP_COMMANDS.md
Moved to docs/internal/:   PRINCIPLES.md, ISSUES.md, DEBUG.md, GTE_LM_FIX.md,
                           and from docs/: bios-hardening.md, overlay-status.md,
                           overlay-plan.md, overlay-recompilation-design.md,
                           overlay-discovery.md, TOMBA2_EXC_CONSULT_RESPONSE.md,
                           CASE_A_AOT_GAP.md, audit_inventory.md
Moved to docs/internal/upstream/: UPSTREAM_2026-08-20.md

The root keeps only what belongs there: README.md, CONTRIBUTING.md, LICENSE,
CLAUDE.md, THIRD_PARTY_ATTRIBUTION.md, RELEASE_NOTES.md, the dotfiles and
psxrecomp_cli.py.

MERGES

SPLITGEN_MIGRATION.md folded into docs/BUILDING.md as "Split generated C". Its
migration instructions were stale in a way that mattered: it told every game
repo to hand-edit its CMakeLists and add a file(GLOB ...) over
<serial>_full*.c, but runtime/runtime.cmake now does that itself. GEN_FULL_GLOB
is a multi-value argument of psxrecomp_add_game_runtime(); the loop at
runtime.cmake:1985 makes each pattern absolute against CMAKE_CURRENT_SOURCE_DIR,
expands it with file(GLOB), and concatenates the hits into
GAME_GENERATED_FULL_C. So a title names the pattern and gets however many shards
a regeneration produced. The rewritten section describes that, the two-step
fallback when the globs match nothing (GEN_FULL_FALLBACK, else <serial>_full.c
derived from the dispatch marker by substituting _dispatch.c), and the one
caveat the code actually has: that file(GLOB) is plain, not CONFIGURE_DEPENDS,
so a regeneration which changes the shard COUNT needs a re-configure to be seen.

psxrecomp-toml-readme.md folded into docs/GAME_PROJECT_SETUP.md as "EXE-only
autofill (psxrecomp-toml)", placed after "Disc autofill (probe_disc.py)" and
before "Project Studio" so the two autofill paths sit together. Its documented
flags were checked against the usage text in recompiler/src/main_toml.cpp:28-42
and against the argument parser at :150-165 -- all seven options match with no
drift, so the flag list is reproduced as-is. Added a pointer that probe_disc.py
is preferable when a cue exists, since that tool does the same EXE-header work
plus disc identity and netplay gates.

FUNCTIONAL FIXES

tools/gen_tcp_commands.py wrote to REPO/"TCP_COMMANDS.md". After the move that
path does not exist, so the generator would have created a new stray root file
and --check would have failed against it forever. Retargeted to
REPO/"docs"/"TCP_COMMANDS.md", along with the seven docstring and message
strings that name the path.

Running it surfaced two further problems, both fixed here:

  * The doc was genuinely stale -- pre-existing drift, not introduced by this
    commit (the moved file is byte-identical to origin/master, and no
    debug-server command table is touched here). Four commands registered since
    the last regeneration were missing from the index -- cdrom_timing_dump,
    savestate_status, ws_ui_groups, xprobe_watch -- and scanline had gained a
    backend. Regenerated: 307 -> 311 commands, and the change is confined to the
    AUTOGEN block (the hand-written head and tail are byte-identical to master).
    tools/gen_tcp_commands.py --check now exits 0.
  * DOC.write_text() has no newline argument, so Python translated every newline
    to os.linesep. On Windows that rewrote all 585 lines of the file to CRLF and
    turned a four-line index update into a whole-file diff -- measured: 589 CRLF
    / 0 LF after a run, against 0 CRLF / 585 LF in the repo. Now written through
    an explicit newline="" handle, verified to leave the file at 0 CRLF.

runtime/include/debug_server.h:9 and runtime/src/debug_server.c:8 cited "TCP.md
and DEBUG.md". TCP.md has never existed in this repo under that name; both now
name docs/TCP_COMMANDS.md and docs/internal/DEBUG.md.

LINK AND REFERENCE FIXES

Every inbound reference to a moved file was located by grepping each basename
across the whole tracked tree, not just the markdown. Markdown hrefs in
README.md, CONTRIBUTING.md, docs/ARCHITECTURE.md, docs/COMPILING_OVERLAYS.md,
docs/AOT_OVERLAY_PLAN.md, docs/RELOCATION_MANIFEST_FORMAT.md,
docs/config_schema.md, docs/OVERLAY_XREF.md, docs/TCP_COMMANDS.md and several
docs/internal/ files now point at the new locations; where two files became
siblings under docs/internal/ the path was shortened rather than lengthened
(bios-hardening.md's reference to overlay-discovery.md, for instance).
docs/COMPILING_OVERLAYS.md's "in this folder" phrasing was wrong once two of the
three files it listed moved, so it now names internal/ explicitly.

24 source-comment mentions of ENHANCEMENTS.md sections G1.x across 19 files
(recompiler, runtime, cmake) were retargeted to docs/ENHANCEMENTS.md with the
section numbers untouched. docs/SHADOW_ENHANCEMENTS.md is a DIFFERENT file and
was deliberately excluded.

20 rationale strings citing docs/CASE_A_AOT_GAP.md across the four
recompiler/seeds/*.json files were updated. Done as a byte-level substring
replacement rather than a JSON round-trip: two of those files are stored with
CRLF endings and two with LF, and re-serializing them reformatted 5,700 lines.
The byte-level version changes exactly 20 lines and each file still parses.

References that look similar but point elsewhere were deliberately left alone,
and are worth naming so a future pass does not "fix" them:

  * "recomp-template PRINCIPLES.md" / "NES recomp template PRINCIPLES.md" in
    CLAUDE.md:43, docs/SHADOW_ENHANCEMENTS.md:28,
    docs/internal/HLE_SCHEDULER_CARVEOUT_PLAN.md:5,
    runtime/src/debug_server.c:4934 and tools/parity_diff.py:8 are a different
    repository's file. Only the five that mean this repo's PRINCIPLES.md were
    updated.
  * Every remaining bare ISSUES.md mention is a GAME repo's tracker
    (MegaManX6Recomp 2/7/8/9, TombaRecomp 6), not this repo's.
  * docs/disc-speed.md:4,54,56 and docs/internal/bios-hardening.md:5 name
    overlay-discovery as a BRANCH, not a file, so no path applies.

INDEXES

README.md's "New here?" table gains rows for docs/WIDESCREEN.md,
docs/ENHANCEMENTS.md and docs/TCP_COMMANDS.md. docs/internal/README.md gains
sections indexing all fourteen documents that moved in, plus an "Upstream
contributor reviews" section describing upstream/ and the new sweep file, plus a
"Removed" bullet for CYCLE_TIMING_ARCH.md (deleted in the previous commit)
pointing at FAITHFUL_TIMING_PLAN.md, CYCLE_MODEL_BEETLE.md and
PRECISE_IRQ_SLICE.md -- following the SLJIT_PERSIST_CACHE.md precedent already
in that file.

docs/ENHANCEMENTS.md:7 claimed framework bugs live in "game repos' ISSUES.md
until a framework tracker exists". A framework tracker does exist; it now says
so and names the Beads epic.

VERIFICATION

A link checker was written to scan every tracked *.md for relative links,
resolving each against the tree being scanned. Baseline at origin/master: 129
files, 137 relative links, 3 broken. After both commits: 127 files, 159 relative
links, 3 broken -- the SAME three, all pre-existing and untouched here:

    docs/DATA_SHARDS.md -> INSTANT_LOADS_PARKED.md   (never existed)
    docs/DATA_SHARDS.md -> INSTANT_LOADS_SPIKE2.md   (never existed)
    docs/NETPLAY.md     -> ../lib/recomp-net/README.md  (inside a submodule)

Zero new broken links. The checker's own coverage was checked against a raw grep
for markdown link openers: the 3-link difference is C++ lambdas inside code
blocks, correctly not treated as links.

Also verified: py_compile over all 113 tracked tools/*.py plus psxrecomp_cli.py;
all four seed JSONs still parse; the test-registration guard still reports 123
test files all registered; recompiler/tests/test_coverage_vault.py still passes;
and no file in this commit had its line endings flipped (checked mechanically
across all 50 modified files, after two files were caught and redone).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-03 07:39:07 -07:00
Matthew Stanley 6e2a36c4c8 observability: always-on production rings + halt-and-serve on opted-in listener
Landed while root-causing the Tomba 2 warm-cache boot death; each of these
was a hole that blocked attribution on the production binary:

- psx_fatal_halt: halt-and-serve is now keyed on the TCP listener being
  live (debug_server_get_status) instead of the build flavor. A production
  run with PSX_DEBUG_SERVER=1 + PSX_EXIT_HALT=1 now stays inspectable at
  the fatal instead of exit(1)ing the evidence away.
- debug_server_get_status: was declared in debug_server.h but never
  implemented; implemented (listener state, port, saved socket error).
- event_ring: EVENT_RING_ENABLED=1 unconditionally. The ring records
  edges only; its production disable predated the production listener and
  contradicted its own ALWAYS-ON design note.
- sp_ring (new): stack-domain (64 KB) transition ring at fntrace_record --
  one entry per dispatch whose guest SP crossed a domain, i.e. the
  provenance record for 'who installed this stack pointer'.
- disp_ring (new): 128-entry unconditional dispatch tail (target/ra/sp/
  cycle) so the final control-flow sequence before a death is always
  reconstructable post-mortem.
- overlay_loader: the native-call ring (overlay_native_ring recent[]) is
  now recorded in production too -- each entry names the requested PC and
  the claiming candidate CRC, which is what finally attributed the
  wrong-variant native run.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 00:06:51 -07:00
Matthew Stanley 70c1eb8a51 Merge master (Bug B pacing fix + overlay cache campaign) into dev/ape-escape
Brings the ape-escape framework (real-EPC interrupt delivery,
g_psx_call_depth safe-point gating, continuation-entry codegen, DualShock
controller config, exception/entryint/irq-check rings, interp
dispatch_until + exit logging + static-text fallback) up to master
035a9fa (frame-pacing TOCTOU fix, vsync self-heal, watchdog classifier,
overlay JIT cache + autocapture/autocompile, Bug A interp call contract,
prod build hygiene).

Key reconciliations (ambiguity resolved toward master per plan):
- interrupts.c: real-EPC delivery (psx_check_interrupts_at) is the
  primary entry; the legacy psx_check_interrupts wrapper keeps the
  0x80000048 sentinel fallback (ape tip restored it), so old-flavor
  generated code keeps working interrupt delivery. Master's bottom
  compat shim removed (would have recursed). Both observability paths
  kept (event-ring gate outcomes + ape irq-check ring; new
  GATE_CALL_DEPTH/GATE_BAD_RESUME kinds).
- Claimed-IRQ cooldown: master's per-source CLAIMED_PROGRESS_QUANTUM=8
  (SIO exempt) over ape's unconditional 0; ape's 2026-06-05 starvation
  note kept as a MERGE NOTE — first knob to A/B if title-LOAD regresses.
  (User-validated: Tomba title-LOAD slot1/slot2 renders and plays on
  this merge — the historical ape-fw blackout is gone.)
- dirty_ram_interp.c: ape's dispatch_until_impl structure (stop_addr,
  static-text fallback, exit-log ring) with master's Bug A call contract
  (psx_dispatch_game_compiled + overlay_loader_call_native at interp
  jal/jalr), overlay loader/fingerprint entry checks, freeze-gated flow
  ring, chain-target attribution, and interp-active bracketing on the
  public wrappers. Ape tip's final local-flow rule kept: dirty game-text
  NEVER chains locally (the tip reversed its own earlier widening).
- freeze_heartbeat.c: classifiers composed — per-frame storm threshold
  (master) + progress-counter gating and 6s window (ape) + slow_frames
  kind (master), thresholds rescaled to the window.
- code_generator.cpp: ape's continuation-entry switch + master's
  refactored scan_jr_tables.
- debug_server.c: ape originals win over master's ported copies
  (bioscall ring, card trace with the richer address set); master-only
  tools kept (dirty_*_dump_file, card_trace filter handler).
- dma.c/cdrom.c: master's overlay-capture path + ape's DMA/command
  history diagnostics, both wired.

Builds: recompiler + Tomba runtime (legacy gen x merged runtime).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-09 22:05:13 -07:00
Matthew Stanley 035a9fa968 release hygiene: explicit two-step file pickers, prod event-ring gate, decouple capture from compile
- First-run pickers now state exactly which user-supplied file is being
  requested: an info box + step-numbered dialog title for the BIOS
  (Step 1 of 2, SCPH1001.BIN) and the game disc (Step 2 of 2, named
  game + accepted formats), each noting what it is NOT. Game name comes
  from the loaded game config.
- event_ring recording is now compiled out under PSX_NO_DEBUG_TOOLS:
  it is only readable via the TCP debug server (stripped in production)
  and is not part of the freeze dump, so recording was pure hot-path
  cost. All other rings were already no-op gated.
- Overlay autocapture (which writes the player-shareable
  overlay_captures.json) is now enabled whenever overlay_cache is on,
  no longer coupled to the dev-only autocompile command — player builds
  produce the contribution file the README promises; the background
  compile still requires a configured command and no-ops without one.

Production surface after this: no console (-mwindows), no TCP server,
no ring recording; keeps overlay cache + captures, freeze watchdog JSON,
and crash report.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-09 20:28:10 -07:00
Matthew Stanley 5041712a05 runtime: phase-5 checkpoint — event-timeline ring, instant-disc rate fix, overlay diagnostics
Accumulated working-tree state for the overlay-JIT-cache phase, committed as a
checkpoint before the Bug A fix (next commit):

- event_ring.{c,h}: always-on unified event-timeline ring (IRQ deliver/gate/
  raise, I_STAT changes, DMA kick/sched/done, ENQ/DEQ per source, exec-mode
  tagged). Edge-recorded, eviction-bounded, survives a blue screen; dump via
  event_ring_dump/_tail/_clear.
- cdrom.c: disc_speed="instant" no longer collapses every sector read to the
  500cy floor (which flooded ~1100 DMA3 completions/frame and starved main
  code). CDROM_INSTANT_PERIOD bounds sector IRQs to CDROM_INSTANT_MAX_PER_FRAME
  (32)/frame. Known caveat: a transient ~1-2s reentry storm at boot frame ~210
  still trips the watchdog once, then recovers.
- interrupts.c: claimed-IRQ cooldown is now per-source — SIO stays 0 (card
  reads need back-to-back IRQs); DMA/VBLANK/timer get CLAIMED_PROGRESS_QUANTUM
  (8) blocks of guaranteed main-code progress between deliveries. Root cause of
  the instant-disc freeze: cooldown=0 let the block leader re-take the
  exception before its body ran (pinned at the 0x8005A1D8 field loop).
- gpu/dma/memory/sio/timers/psx_cycles: event-ring instrumentation at raise/
  kick/complete/schedule sites; g_event_step_conservative (single-cycle event
  stepping A/B used to refute the event-granularity hypothesis for Bug A).
- overlay_capture.c, overlay_loader.h, crc32: capture/manifest plumbing for
  per-entry validity (Inc3).
- recompiler: overlay-mode emission updates (code_generator, main_psx).
- tools: compile_overlays.py audit/patch updates; tomba_loadtest.py.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-09 12:24:40 -07:00