5 Commits

Author SHA1 Message Date
Matthew Stanley 5ded80e61e Merge master (AOT overlay generation) into perf/interpreter-hotpath
Integration branch: the interpreter-perf series (interpreter fallback +
additive shard autocompile + always-on production observability + the
fail-closed alias-body emitter fix) rebased-by-merge onto master's AOT
overlay generation (play-free extraction, bounded indirect discovery,
AOT recall, shadow-diff redesign).

Semantic-union highlights:
- gte.cpp: HEAD's direct-field marshaling fast paths kept; master's
  s_gte_replay_sandbox guards grafted onto every affected site.
- psx_icache.c: HEAD's global tag array + fetch_miss split kept;
  master's transactional shadow record/replay retargeted onto it.
- overlay_capture.c: HEAD's commit/retry/wait-pending pipeline kept;
  master's durable history + autocapture APIs wired into it
  (persist_history_snapshot fed from the immutable .d contribution).
- overlay_loader.c: master's interp-record/native-replay shadow-diff
  redesign kept as base with HEAD's scheduler-bail hardening grafted
  (new SHDIFF_SCHED divergence kind); strict manifest v2 parsing keeps
  master's P pair-id lines; always-on native-call ring preserved.
- main.cpp shutdown calls BOTH overlay_autocapture_shutdown() and
  overlay_capture_wait_pending() (the latter drains the preserve-write
  FIFO the former does not).
- recompiler: master's generate_alias_group refactor retains the
  fail-closed alias-body re-entry defaults from c85f1d1.

Validated on this tree: recompiler + runtime + tests build; all six
runtime unit tests PASS; interpreter perf guards PASS; recompiler
patch/structural tests PASS (l1 decoder oracle noise = 376 pre-existing
on both parents).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 00:43:41 -07:00
Matthew Stanley 3949854981 Optimize interpreter fallback and additive overlay autocompile 2026-07-18 08:32:42 -07:00
Matthew Stanley a544809edf aot: persist overlay capture history atomically 2026-07-17 23:56:58 -07:00
Matthew Stanley 739994cfcd Step 2.8: variant-capture automation (detect -> capture -> compile -> rescan)
Runtime detects an uncovered overlay/kernel variant by sustained dirty-RAM
interp pressure inside the capture windows (g_dirty_window_dispatches,
sampled every 600 vblanks; fires only at a coherent non-loading moment via
cdrom_load_in_progress, 60s cooldown, 16/session cap), writes the capture
JSON, and spawns the game.toml-configured compile command
([runtime] overlay_autocompile_cmd, cmd.exe /C, cwd=project root, child
output kept in an in-memory ring — no log files). On child exit 0 the emu
thread rescans the cache (overlay_loader_rescan: idempotent dir re-scan +
checked-regions reset) so the new DLL loads on the next dispatch and the
variant goes native in-session. Dedup is tool-side (existing-CRC skip);
pressure drops to zero once code is native, so the loop self-limits.

New TCP: autocompile_status / autocompile_run / overlay_rescan.
compile_dll now uses absolute native paths and reports exit codes.

VERIFIED live (user-played route): 2 auto-triggers fired, captures written,
child ran recompiler+audit (new variants 1E7DDE2C, A62D0A7C discovered and
audit-clean), rescan picked the compiled DLL up in-session. OPEN ISSUE: gcc
exits 1 with no output inside the spawned cmd.exe child (same absolute-path
command succeeds from an interactive shell) — compile step currently
completed manually; suspect child-env difference, to be isolated next
session.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-09 16:16:36 -07:00
Matthew Stanley b2929aa504 overlay: restore overlay DLL cache + capture (faithful port of codex/cache-discovery)
Reinstates the runtime-loaded overlay DLL cache (Layer A) and the offline
overlay compilation pipeline that were built on the pruned codex/cache-discovery
branch (tip 2bbafc2) and never merged. Ported onto current master; rebuilt
rather than history-restored.

Runtime (off by default, opt in via [runtime] overlay_cache = true):
- overlay_loader.c/.h, overlay_api.h: CRC32-keyed DLL cache at
  cache/<game_id>/<region>_<crc>.dll. On a dirty-RAM dispatch miss the loader
  LoadLibrary's the matching DLL, wires runtime callbacks via overlay_init(),
  enumerates func_XXXXXXXX exports, and dispatches them BEFORE the interpreter.
- overlay_capture.c/.h: write-once capture set; records overlay DMA bytes and
  writes overlay_captures.json (execution-verified seeds) at clean exit.
- dma.c: capture + cd_dma_log hook at CD DMA completion (async model).
- cdrom.c: cdrom_get_setloc_lba() for disc-position correlation.
- dirty_ram_interp.c: g_dirty_ram_exec_pc_table (exec-verified PCs), a
  decodability guard on JAL-into-dirty targets, and the overlay dispatch hooks.
- debug_server.c: cd_read_log + overlay_loader_status TCP commands.
- config_loader: [runtime] overlay_cache flag (default false).

Recompiler (compile_overlays.py / extract_overlays.py drive these):
- main_psx.cpp --overlay-exact: analyze only explicit entries + direct JAL
  targets and set split_mid_function_targets=false, so overlay basic-block
  targets are NOT split into bogus standalone functions (the mid-function-seed
  failure mode that produced black-on-load / blue-on-new-game).
- function_analysis.cpp: analyze_exact_entries(); code_generator.cpp: gate the
  mid-function split pre-pass behind split_mid_function_targets (default true,
  so normal main-EXE codegen is unchanged - Tomba still regens to 2854 funcs).

Deltas from the original: save states dropped (never worked); overlay enable is
runtime-config-gated (off by default) instead of a build-time define; loader
stderr logging converted to the overlay_loader_status TCP path (CLAUDE.md Rule 3).

Recovered from _backups/2026-06-06-consolidation/psxrecomp-all-PRE.bundle.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-07 12:32:19 -07:00