data-shards spike: memoized pure-function replay (capture/verify/replay + cycle-faithful credit)

- runtime/src/data_shards.c + include: read-before-write recorder fed from
  the memory chokepoints; byte-verified replay (kernel-bless pattern);
  recorded-cycle credit in slices through psx_advance_cycles +
  psx_check_interrupts_at; purity poisons (MMIO/DMA/thread-switch/budget);
  .dss persistence under the unified cache dir; TCP data_shards cmd.
- emitter: [data_shards] funcs -> psx_datashard_enter at entry +
  psx_datashard_ret at jr-$ra sites (rolls cg-hash).
- obs: phase_hot set=static keyed on new g_psx_last_fn_entry entry stamp
  (current_func dispatch stamp survives post-IRQ resumption and pooled 55%
  of static samples on kernel 0xF40); TCP starv_ring query for the
  always-on PC-sample ring.
- docs/DATA_SHARDS.md: design, soundness model, gates, FMV exclusion.

docs/DATA_SHARDS.md

@@ -0,0 +1,145 @@
+# Post-decompression data shards (memoized pure-function replay)
+
+Branch: `spike/tomba-load-shards` (framework) + `spike/load-shards` (TombaRecomp).
+Status: **spike in progress** — feasibility experiment.
+
+## The idea
+
+We are a RECOMP, not a DECOMP: we cannot AOT-transform game data the way a
+decompilation project can (an .o2r/asset-archive pipeline needs source-level
+knowledge of every asset). But we CAN do what we already do for *code*
+overlays: **capture on first encounter, replay ever after**. Native code
+shards memoize "what does this RAM-resident code compile to"; data shards
+memoize "what does this pure function compute".
+
+Target case: Tomba's NOW-LOADING decompression. First run: the decompressor
+executes faithfully (native/interp) and we record its effect. Every later
+run with byte-identical inputs: skip the execution, apply the recorded
+effect. The shard store persists on disk (o2r-like), so a player's second
+session has fast loads everywhere they've been.
+
+## Prior art this builds on (and why it dodges the parked spikes)
+
+- [instant-loads round 1](INSTANT_LOADS_PARKED.md): pumping the loader's
+  cooperative yields starves the streamer → hardlock. **We don't touch the
+  yield structure at all.**
+- [instant-loads round 2](INSTANT_LOADS_SPIKE2.md): loads are
+  emulation-throughput-bound under turbo; "instant requires making the WORK
+  fast". **This is exactly that** — the work becomes a memcmp + memcpy.
+- CD adaptive-instant (parked): delivering sectors early desyncs guest
+  timing. **We deliver nothing early** — see cycle-faithful replay below.
+- Kernel bless (fw 8f23892): byte-verify-then-trust is the established
+  soundness pattern. Data shards are the same pattern applied to data
+  transforms: **verify the read-set bytes, then trust the recorded
+  write-set.**
+
+## Soundness model
+
+A call is memoizable iff it is a deterministic transform of RAM inputs to
+RAM outputs. Capture proves this per-shard; replay verifies it per-hit.
+
+**Capture (first encounter, or shard-miss):** execute the call in the
+dirty-RAM interpreter with a load/store trace armed (non-exception context
+only), from function entry until control returns to the entry `ra` at the
+entry `sp` depth. Record:
+
+- **read-set**: every address the call reads *before* writing it
+  (read-before-write = the true input), with the bytes observed. Ordered,
+  merged into ranges.
+- **write-set**: final bytes of every address written (post-state ranges).
+- **reg-out**: `v0/v1` (+ anything the ABI lets it define).
+- **cycle-cost**: non-exception guest cycles consumed by the call.
+- **purity verdict**: any MMIO/scratchpad-IO read, COP0 access, syscall,
+  exception-return into the traced frame, or unresolved control transfer
+  out of the frame → **not memoizable**, shard poisoned (negative-cached
+  for the session, never re-captured this run).
+
+**Replay (subsequent encounter):** look up shards by
+`(func_addr, a0..a3, read_set_hash_prefilter)`; byte-compare the full
+recorded read-set against live RAM (kernel-bless-style — the compare IS the
+proof); on match: apply write-set, set reg-out, **advance the recorded
+cycle-cost through the normal psx_advance_cycles/IRQ machinery in slices**,
+return to `ra`. On mismatch: fall through to real execution (optionally
+capture a new variant — the store is additive per input, like overlay scene
+variants).
+
+## Cycle-faithful by default
+
+Replay credits the *recorded* guest cycles through the existing cycle
+advance + interrupt-check path. Guest-visible timing is identical to real
+execution: the loader thread still spends the same guest time per queue
+step, still yields identically, CD cadence untouched, VBlank/IRQ phase
+preserved to the slice. All the wedge classes that killed the earlier
+spikes were guest-*timing* perturbations; this mechanism perturbs none.
+
+The win is host wall time under `turbo_loads`: the pig window fast-forwards
+guest time at host speed, and the dominant host cost inside the window (if
+profiling confirms: decompression execution) becomes ~zero. An "instant"
+mode (credit ~0 cycles) is a later, opt-in enhancement — NOT the default,
+and not needed for the primary win.
+
+Known fidelity delta (documented, accepted for the spike): an IRQ handler
+that fires mid-credit observes the write-set already complete (memory state
+"from the future" within the call window). Handlers reading an in-flight
+decompression buffer mid-call would misbehave — no known case; capture can
+optionally record IRQ arrivals to flag suspicious shards.
+
+## Mechanism
+
+1. **Gen-time entry hook** (same emission point as round-1 loader_pump):
+   functions listed in `game.toml [data_shards] funcs = [0x...]` get
+   `if (psx_datashard_enter(cpu, 0xADDR)) return;` emitted at entry.
+   Config-listed for the spike; auto-detection (hot, pure, hook-worthy) is
+   a later phase. Rolls the cg-hash → regen + reshard (worktree owns its
+   whole chain).
+2. **Runtime store** (`runtime/src/data_shards.c`): in-memory index +
+   on-disk persistence under the unified cache dir
+   (`cache/<game-id>/datashards/`), one file per shard + a manifest.
+   Loaded lazily; saved on capture. Guarded by the same codegen-tag stamp
+   discipline as code shards (a shard records the emitter tag it was
+   captured under only for bookkeeping — shards are codegen-independent by
+   construction, they describe *guest* semantics).
+3. **Capture executor**: the dirty-RAM interpreter run-to-return with a
+   tracing callback. Trace only non-exception context. Abort (poison) on
+   purity violations.
+4. **Observability**: TCP `data_shards` — hits, misses, captures, poisons,
+   bytes replayed, cycles credited, per-func table. Always-on counters;
+   ring of recent events.
+
+## Exclusions and spike environment (2026-07-10, user directives)
+
+- **FMVs are excluded from data-sharding.** MDEC playback is a timed
+  stream (XA audio interlock, per-frame DMA), not a pure transform —
+  memoizing it is semantically wrong. User-reported FMV lag turned out to
+  be the **OpenGL renderer** (~0.70x pace during FMV, audio warble;
+  software renderer plays the same FMV at ~0.91x+). GL FMV perf is a
+  separate investigation — this spike runs on the **software renderer**.
+- Software renderer at Tomba's shipped 2x SSAA is rasterizer-bound
+  in-game (measured 0.51x, 60.8% of wall in GP0) — the spike's
+  settings.toml pins `renderer=software, supersampling=1`.
+
+## Feasibility gates (measure first)
+
+Gate 1 — **profile the pig window** (phase_profile + phot + dirty_ram_stats
+across a real area transition on a healthy overlay cache): what fraction of
+wall time is decompressor execution? If the window is CD-cadence/idle
+dominated, data shards buy little → report honestly, park.
+
+Gate 2 — **purity**: does the Tomba decompressor trace clean (no MMIO, no
+syscalls, bounded frame)? If it reads CD/DMA registers directly, the
+memoization boundary must move (e.g. per-asset unpack leaf, not the
+streaming wrapper).
+
+Gate 3 — **UX delta**: wall-clock NOW-LOADING duration, cold (capture run)
+vs warm (replay run), same transition. The number the user feels.
+
+## Open questions
+
+- Hook auto-selection (profile-guided) vs config list — config list for the
+  spike.
+- Cross-session invalidation: shards keyed by full read-set bytes are
+  self-validating; disc/game version changes wash out naturally.
+- Shard size policy: decompressed area assets are O(100KB–1MB) each;
+  eviction/compaction if the store grows unreasonably (compress shard
+  payloads with zstd/deflate on disk — ironic, but host-side decompress is
+  free relative to interp).

recompiler/include/code_generator.h

@@ -50,6 +50,12 @@ struct CodeGenConfig {
     // proportion correction at GP0 submission. Empty = no hooks (default).
     std::set<uint32_t> ws_sprite_tag_funcs;
 
+    // Data-shard hooks ([data_shards] funcs): memoized pure-function replay.
+    // Entry gets `if (psx_datashard_enter(cpu, KEY)) return;` (replay path),
+    // every jr-$ra return gets `psx_datashard_ret(cpu)` (capture finalize).
+    // See docs/DATA_SHARDS.md. Empty = no hooks (default).
+    std::set<uint32_t> data_shard_funcs;
+
     // MMX6-class 2D tile-ring stage initializer. Emit a reveal-invalidation
     // callback at entry so host-only wide pixels cannot survive a stage swap.
     uint32_t ws_bg2d_init_func = 0;

recompiler/src/code_generator.cpp

@@ -1700,6 +1700,12 @@ std::string CodeGenerator::translate_basic_block(
                         ss << config_.indent
                            << "gte_ws_set_suppress(0);  /* widescreen: end far-backdrop un-squash (8C) */\n";
                     }
+                    if (config_.data_shard_funcs.count(cfg.function_start)) {
+                        // Capture finalize: fires only when this is the armed
+                        // function's own return (sp/ra checked runtime-side).
+                        ss << config_.indent
+                           << "psx_datashard_ret(cpu);  /* data-shard: finalize capture */\n";
+                    }
                     if (ra_loaded_from_non_sp) {
                         ss << emit_interrupt_check_expr(delay_saved_target, config_.indent);
                         ss << config_.indent
@@ -2190,6 +2196,16 @@ GeneratedFunction CodeGenerator::generate_function(
     body_ss << config_.indent
             << fmt::format("debug_server_log_call_entry(0x{:08X}u);\n",
                           func.start_addr);
+    if (config_.data_shard_funcs.count(func.start_addr)) {
+        // Data-shard hook (docs/DATA_SHARDS.md): on a verified shard hit the
+        // runtime applies the recorded effect, credits the recorded cycles,
+        // publishes pc=$ra and we return without executing the body. Otherwise
+        // it arms the capture recorder and the body runs natively.
+        body_ss << config_.indent
+                << fmt::format("if (psx_datashard_enter(cpu, 0x{:08X}u)) return;"
+                               "  /* data-shard: replay or arm capture */\n",
+                               func.start_addr);
+    }
     if (config_.ws_sprite_tag_funcs.count(func.start_addr)) {
         body_ss << config_.indent
                 << "psx_ws_sprite_tag(cpu);  /* widescreen: record prim ($a0) + anchor */\n";
@@ -2632,6 +2648,8 @@ std::string CodeGenerator::generate_file(
     ss << "extern void cosim_block(uint32_t block_leader_phys);\n";
     ss << "extern void cosim_instr(uint32_t pc);\n";
     ss << "#endif\n";
+    ss << "extern int  psx_datashard_enter(CPUState* cpu, uint32_t key);  /* data-shard replay/capture (data_shards.c) */\n";
+    ss << "extern void psx_datashard_ret(CPUState* cpu);                  /* data-shard capture finalize */\n";
     ss << "extern void psx_ws_sprite_tag(CPUState* cpu);  /* widescreen prim tag (gpu.c) */\n";
     ss << "extern void psx_ws_mmx6_bg_stage_init(void);    /* ws 2D stage reveal invalidation (gpu.c) */\n";
     ss << "extern int  psx_ws_x_margin(void);  /* widescreen cull-margin term (gpu.c) */\n";

recompiler/src/config_loader.cpp

@@ -725,6 +725,16 @@ GameConfig load_game_config(const fs::path& config_path_in) {
     bool ws_nw_phase_backdrop = false;
     bool ws_offered = true;
     bool ws_ultrawide_offered = false;
+    // Optional [data_shards] block — memoized pure-function replay hooks.
+    std::vector<uint32_t> data_shard_funcs;
+    if (cfg.contains("data_shards")) {
+        const toml::value& dsv = toml::find(cfg, "data_shards");
+        if (dsv.contains("funcs")) {
+            const auto& arr = toml::find<std::vector<std::string>>(dsv, "funcs");
+            for (const auto& a : arr)
+                data_shard_funcs.push_back(parse_hex(a, "data_shards.funcs"));
+        }
+    }
     if (cfg.contains("widescreen")) {
         const toml::value& ws = toml::find(cfg, "widescreen");
         if (ws.contains("sprite_tag_funcs")) {
@@ -944,6 +954,7 @@ GameConfig load_game_config(const fs::path& config_path_in) {
         /*ws_sprite_tag_funcs*/   ws_sprite_tag_funcs,
         /*ws_sprite_anchor_addr*/ ws_sprite_anchor_addr,
         /*ws_hud_sprt_squash*/    ws_hud_sprt_squash,
+        /*data_shard_funcs*/      data_shard_funcs,
         /*ws_cull_bias_sites*/    ws_cull_bias_sites,
         /*ws_cull_range_sites*/   ws_cull_range_sites,
         /*ws_cull_a1_sites*/      ws_cull_a1_sites,

recompiler/src/config_loader.h

@@ -410,6 +410,13 @@ struct GameConfig {
     uint32_t              ws_sprite_anchor_addr = 0;
     bool                  ws_hud_sprt_squash = false;
 
+    // [data_shards] funcs: functions that get the memoized pure-function
+    // replay entry/return hooks (psx_datashard_enter/psx_datashard_ret).
+    // Enhancement-phase load-time work; see docs/DATA_SHARDS.md. Capture is
+    // self-proving (byte-verified read-set), so listing a function that turns
+    // out to be impure only costs a poisoned capture, never a wrong replay.
+    std::vector<uint32_t> data_shard_funcs;
+
     // Cull-margin widening. The game's per-object draw classifier compares
     // (objX - camX + BIAS) against a RANGE derived from the 4:3 screen width;
     // the GTE squash shows ~33% more world, so the fixed margin collapses and

recompiler/src/main_psx.cpp

@@ -120,6 +120,7 @@ int main(int argc, char** argv) {
     bool                  inspect_mode = false;
     bool                  overlay_mode = false;
     std::set<uint32_t>    ws_tag_funcs;         // [widescreen] sprite_tag_funcs
+    std::set<uint32_t>    ds_funcs;             // [data_shards] funcs
     std::set<uint32_t>    ws_cull_bias, ws_cull_range, ws_cull_a1; // [widescreen.cull]
     std::set<uint32_t>    ws_cull_screen_x;    // [widescreen.cull] screen_x_sites
     std::set<uint32_t>    ws_cull_slti;         // [widescreen.cull] slti_sites
@@ -147,6 +148,7 @@ int main(int argc, char** argv) {
         out_dir              = cfg.out_dir;
         ws_tag_funcs.insert(cfg.ws_sprite_tag_funcs.begin(),
                             cfg.ws_sprite_tag_funcs.end());
+        ds_funcs.insert(cfg.data_shard_funcs.begin(), cfg.data_shard_funcs.end());
         ws_cull_bias.insert(cfg.ws_cull_bias_sites.begin(), cfg.ws_cull_bias_sites.end());
         ws_cull_range.insert(cfg.ws_cull_range_sites.begin(), cfg.ws_cull_range_sites.end());
         ws_cull_a1.insert(cfg.ws_cull_a1_sites.begin(), cfg.ws_cull_a1_sites.end());
@@ -777,6 +779,7 @@ int main(int argc, char** argv) {
     codegen_config.split_mid_function_targets = !overlay_mode;
     codegen_config.overlay_mode = overlay_mode;
     codegen_config.ws_sprite_tag_funcs = ws_tag_funcs;
+    codegen_config.data_shard_funcs = ds_funcs;
     codegen_config.ws_bg2d_init_func = ws_bg2d_init_func;
     codegen_config.ws_cull_bias_sites  = ws_cull_bias;
     codegen_config.ws_cull_range_sites = ws_cull_range;

runtime/include/data_shards.h

@@ -0,0 +1,50 @@
+/* data_shards.h — memoized pure-function replay ("post-decompression shards").
+ *
+ * Design: docs/DATA_SHARDS.md. Functions listed in game.toml [data_shards]
+ * funcs get a gen-time entry hook (psx_datashard_enter) and return hooks
+ * (psx_datashard_ret). First encounter of a (function, inputs) pair runs
+ * NATIVELY under a read/write recorder fed from the memory chokepoints;
+ * the finalized shard stores the read-set (input proof), write-set (effect),
+ * caller-visible register results, and the guest cycle cost. Subsequent
+ * encounters byte-verify the read-set against live RAM (kernel-bless style)
+ * and, on match, apply the write-set, restore registers, and credit the
+ * recorded cycles through the normal advance/IRQ machinery — guest timing
+ * preserved, host executes ~a memcmp+memcpy instead of the function.
+ *
+ * Purity is proven per-capture: any MMIO access, DMA write into RAM, or
+ * abnormal exit during the recording window poisons the capture. FMV/MDEC
+ * streaming is out of scope by design (user directive 2026-07-10).
+ */
+#ifndef PSXRECOMP_DATA_SHARDS_H
+#define PSXRECOMP_DATA_SHARDS_H
+
+#include <stdint.h>
+#include "cpu_state.h"
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/* Gen-time hooks (emitted by code_generator for [data_shards] funcs). */
+int  psx_datashard_enter(CPUState* cpu, uint32_t key);  /* 1 = replayed, return now */
+void psx_datashard_ret(CPUState* cpu);                  /* at the function's jr $ra */
+
+/* Memory-chokepoint feeds (memory.c). Called only while g_ds_recording != 0,
+ * outside exception context. addr is the VIRTUAL address. */
+extern volatile int g_ds_recording;
+void ds_note_read(uint32_t addr, uint32_t size);
+void ds_note_write(uint32_t addr, uint32_t size);
+void ds_note_mmio(uint32_t addr, int is_read);   /* purity poison */
+void ds_note_dma_write(void);                    /* DMA wrote RAM mid-window: poison */
+
+/* Runtime init (cache dir for persistence; NULL disables persistence). */
+void ds_init(const char* cache_dir, const char* game_id);
+
+/* Debug/TCP surface. */
+void ds_stats_json(char* buf, int cap);
+void ds_set_enabled(int on);
+
+#ifdef __cplusplus
+}
+#endif
+#endif /* PSXRECOMP_DATA_SHARDS_H */

runtime/include/starvation_ring.h

@@ -101,6 +101,14 @@ void starvation_ring_dump(const char *path);
 /* Periodic PC sample. Caller responsible for throttling. */
 void starvation_ring_pc_sample(void);
 
+/* Ring query (TCP `starv_ring`): total events recorded so far, and a copy
+ * of the entry with sequence number `seq` (0 = first ever). Returns 1 and
+ * fills *out when `seq` is still resident in the ring; 0 when evicted or
+ * not yet written. Same-thread with the writers (emu/main thread), so the
+ * copy is tear-free. */
+uint64_t starvation_ring_total(void);
+int      starvation_ring_get(uint64_t seq, StarvationEntry *out);
+
 #ifdef __cplusplus
 }
 #endif

runtime/runtime.cmake

@@ -190,6 +190,7 @@ set(PSXRECOMP_RUNTIME_SOURCES
     ${PSXRECOMP_ROOT}/runtime/src/psx_icache.c
     ${PSXRECOMP_ROOT}/runtime/src/starvation_ring.c
     ${PSXRECOMP_ROOT}/runtime/src/latency_ring.c
+    ${PSXRECOMP_ROOT}/runtime/src/data_shards.c
     ${PSXRECOMP_ROOT}/runtime/src/card_read_summary.c
     ${PSXRECOMP_ROOT}/runtime/src/card_data_writes.c
     ${PSXRECOMP_ROOT}/runtime/src/overlay_capture.c

runtime/src/data_shards.c

@@ -0,0 +1,494 @@
+/* data_shards.c — memoized pure-function replay ("post-decompression shards").
+ * See data_shards.h and docs/DATA_SHARDS.md for the design and soundness
+ * model. Summary of the lifecycle:
+ *
+ *   enter(key):
+ *     - shard whose read-set byte-matches live RAM?  -> REPLAY:
+ *         apply write-set (through psx_write_* so dirty-code tracking, text
+ *         guard, card traps etc. all fire exactly as a real run would),
+ *         mark executable ranges like the CD-DMA path does, restore the
+ *         caller-visible registers, credit the recorded guest cycles in
+ *         slices through psx_advance_cycles + psx_check_interrupts_at
+ *         (events and IRQs land at their authentic guest cycle), publish
+ *         pc = $ra, return 1.
+ *     - no shard and recorder idle -> ARM capture, return 0 (body runs
+ *         natively; the memory chokepoints feed the recorder).
+ *   ret(): at the hooked function's own jr $ra with the armed sp -> finalize:
+ *     build read/write range lists from the byte maps, snapshot the
+ *     caller-visible registers and cycle cost, store + persist.
+ *
+ * Purity poisons (capture aborted, never replayed): MMIO access outside
+ * exception context, DMA write to RAM inside the window, a green-thread
+ * switch inside the window (other threads' work would leak into the trace),
+ * host-frame escape (longjmp/RestoreState), window byte budget exceeded.
+ *
+ * Known v1 fidelity notes (documented in DATA_SHARDS.md):
+ *  - cycle cost includes any ISR cycles that ran inside the capture window
+ *    (replay re-runs ISRs live, so those cycles are credited ~twice; small).
+ *  - an IRQ handler that fires mid-replay observes the write-set complete.
+ *  - GTE/COP0 state changes by the function are not captured (target
+ *    functions are data transforms; a GTE-using function must not be listed).
+ */
+#include "data_shards.h"
+#include "psx_cycles.h"
+#include "interrupts.h"
+#include "crc32.h"
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#ifdef _WIN32
+#include <io.h>
+#include <direct.h>
+#endif
+
+extern uint8_t *memory_get_ram_ptr(void);
+extern uint8_t *memory_get_scratchpad_ptr(void);
+extern void psx_write_word(uint32_t addr, uint32_t val);
+extern void psx_write_byte(uint32_t addr, uint8_t val);
+extern void dirty_ram_mark_executable_range(uint32_t addr, uint32_t len);
+
+#define DS_RAM_BYTES   (2u * 1024u * 1024u)
+#define DS_SP_BYTES    1024u
+#define DS_SPACE       (DS_RAM_BYTES + DS_SP_BYTES)   /* unified index space */
+#define DS_PAGE_SHIFT  12
+#define DS_NPAGES      ((DS_SPACE + (1u << DS_PAGE_SHIFT) - 1) >> DS_PAGE_SHIFT)
+#define DS_MAX_BYTES   (4u * 1024u * 1024u)  /* window write+read budget */
+#define DS_REG_COUNT   19                    /* at,v0,v1,a0-a3,t0-t7,t8,t9,hi,lo */
+
+/* caller-visible register set restored on replay */
+static const uint8_t k_ds_gprs[17] = {1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,24,25};
+
+typedef struct { uint32_t idx, len; } DsRange;
+
+typedef struct DsShard {
+    struct DsShard* next;
+    uint32_t key;
+    uint32_t args[4];          /* a0-a3 at entry (prefilter) */
+    uint32_t rd_crc;           /* crc of read-set bytes (prefilter) */
+    uint32_t nrd, nwr;
+    DsRange* rd;               /* read-set ranges (unified idx) */
+    uint8_t* rd_bytes;         /* concatenated expected bytes */
+    uint32_t rd_total;
+    DsRange* wr;
+    uint8_t* wr_bytes;
+    uint32_t wr_total;
+    uint32_t regs[DS_REG_COUNT];
+    uint64_t cyc_cost;
+    uint32_t hits;
+} DsShard;
+
+#define DS_KEY_BUCKETS 64
+static DsShard* s_buckets[DS_KEY_BUCKETS];
+static int s_loaded_keys[256]; static int s_loaded_n = 0;  /* keys with disk load done */
+
+volatile int g_ds_recording = 0;
+static int   s_enabled = 1;
+
+/* ---- recorder state ---- */
+static CPUState* s_arm_cpu = 0;
+static uint32_t  s_arm_key = 0, s_arm_sp = 0, s_arm_ra = 0;
+static uint32_t  s_arm_args[4];
+static uint64_t  s_arm_cyc0 = 0;
+static int       s_poisoned = 0;
+static uint32_t  s_bytes_touched = 0;
+
+static uint8_t* s_rd_flag = 0;   /* 1 = read before written */
+static uint8_t* s_wr_flag = 0;   /* 1 = written */
+static uint8_t* s_rd_orig = 0;   /* byte value at first (pre-write) read */
+static uint8_t  s_page_touched[DS_NPAGES];
+static uint32_t s_pages[DS_NPAGES]; static uint32_t s_npages = 0;
+
+/* ---- stats (TCP `data_shards`) ---- */
+static struct {
+    uint64_t enters, replays, captures_armed, captures_ok, captures_poisoned;
+    uint64_t verify_fail, bytes_replayed;
+    uint64_t cyc_credited;
+    uint32_t last_key, last_poison_key, shards;
+    uint32_t poison_mmio, poison_dma, poison_thread, poison_budget, poison_stale;
+} s_st;
+
+static const char* s_cache_dir = 0;
+static char s_dir_buf[1024];
+
+void ds_set_enabled(int on) { s_enabled = on; }
+
+void ds_init(const char* cache_dir, const char* game_id) {
+    if (!cache_dir || !game_id) return;
+    snprintf(s_dir_buf, sizeof(s_dir_buf), "%s/%s/datashards", cache_dir, game_id);
+    s_cache_dir = s_dir_buf;
+#ifdef _WIN32
+    /* best-effort mkdir -p */
+    char acc[1100]; int n = 0;
+    for (const char* p = s_dir_buf; ; p++) {
+        if (*p == '/' || *p == '\\' || *p == 0) {
+            acc[n] = 0;
+            if (n > 2) _mkdir(acc);
+            if (*p == 0) break;
+        }
+        if (n < (int)sizeof(acc) - 1) acc[n++] = (*p == '/') ? '\\' : *p;
+    }
+#endif
+}
+
+/* ---------- unified index helpers ---------- */
+
+/* virtual addr -> unified index, or -1 (untracked: ROM/open bus), or -2 (MMIO) */
+static int64_t ds_index_of(uint32_t addr) {
+    uint32_t phys = addr & 0x1FFFFFFFu;
+    if (phys < 0x00800000u) return (int64_t)(phys & (DS_RAM_BYTES - 1u));
+    if (phys >= 0x1F800000u && phys < 0x1F800000u + DS_SP_BYTES)
+        return (int64_t)(DS_RAM_BYTES + (phys - 0x1F800000u));
+    if (phys >= 0x1F801000u && phys < 0x1F803000u) return -2;   /* MMIO */
+    if (phys >= 0x1FC00000u) return -1;                          /* BIOS ROM: constant */
+    return -2;  /* anything else device-ish: treat as impure */
+}
+
+static uint8_t ds_live_byte(uint32_t idx) {
+    return idx < DS_RAM_BYTES ? memory_get_ram_ptr()[idx]
+                              : memory_get_scratchpad_ptr()[idx - DS_RAM_BYTES];
+}
+
+static void ds_touch_page(uint32_t idx) {
+    uint32_t pg = idx >> DS_PAGE_SHIFT;
+    if (!s_page_touched[pg]) { s_page_touched[pg] = 1; s_pages[s_npages++] = pg; }
+}
+
+static void ds_poison(uint32_t which) {
+    if (!g_ds_recording || s_poisoned) return;
+    s_poisoned = 1;
+    s_st.captures_poisoned++;
+    s_st.last_poison_key = s_arm_key;
+    switch (which) {
+    case 1: s_st.poison_mmio++;   break;
+    case 2: s_st.poison_dma++;    break;
+    case 3: s_st.poison_thread++; break;
+    case 4: s_st.poison_budget++; break;
+    default: s_st.poison_stale++; break;
+    }
+}
+
+/* ---------- chokepoint feeds ---------- */
+
+void ds_note_read(uint32_t addr, uint32_t size) {
+    if (!g_ds_recording || s_poisoned) return;
+    int64_t ix = ds_index_of(addr);
+    if (ix == -1) return;
+    if (ix == -2) { ds_poison(1); return; }
+    uint32_t idx = (uint32_t)ix;
+    for (uint32_t i = 0; i < size; i++, idx++) {
+        if (s_wr_flag[idx] || s_rd_flag[idx]) continue;
+        s_rd_flag[idx] = 1;
+        s_rd_orig[idx] = ds_live_byte(idx);
+        ds_touch_page(idx);
+        if (++s_bytes_touched > DS_MAX_BYTES) { ds_poison(4); return; }
+    }
+}
+
+void ds_note_write(uint32_t addr, uint32_t size) {
+    if (!g_ds_recording || s_poisoned) return;
+    int64_t ix = ds_index_of(addr);
+    if (ix == -1) return;
+    if (ix == -2) { ds_poison(1); return; }
+    uint32_t idx = (uint32_t)ix;
+    for (uint32_t i = 0; i < size; i++, idx++) {
+        if (!s_wr_flag[idx]) {
+            s_wr_flag[idx] = 1;
+            ds_touch_page(idx);
+            if (++s_bytes_touched > DS_MAX_BYTES) { ds_poison(4); return; }
+        }
+    }
+}
+
+void ds_note_mmio(uint32_t addr, int is_read) {
+    (void)addr; (void)is_read;
+    ds_poison(1);
+}
+
+void ds_note_dma_write(void) { ds_poison(2); }
+
+/* traps.c calls this on every guest thread switch */
+void ds_note_thread_switch(void) { if (g_ds_recording) ds_poison(3); }
+
+/* ---------- capture arm / finalize ---------- */
+
+static void ds_recorder_reset(void) {
+    for (uint32_t i = 0; i < s_npages; i++) {
+        uint32_t base = s_pages[i] << DS_PAGE_SHIFT;
+        uint32_t len = (1u << DS_PAGE_SHIFT);
+        if (base + len > DS_SPACE) len = DS_SPACE - base;
+        memset(s_rd_flag + base, 0, len);
+        memset(s_wr_flag + base, 0, len);
+        s_page_touched[s_pages[i]] = 0;
+    }
+    s_npages = 0;
+    s_bytes_touched = 0;
+    s_poisoned = 0;
+}
+
+static int ds_alloc_maps(void) {
+    if (s_rd_flag) return 1;
+    s_rd_flag = (uint8_t*)calloc(1, DS_SPACE);
+    s_wr_flag = (uint8_t*)calloc(1, DS_SPACE);
+    s_rd_orig = (uint8_t*)malloc(DS_SPACE);
+    return s_rd_flag && s_wr_flag && s_rd_orig;
+}
+
+static DsShard** ds_bucket(uint32_t key) {
+    return &s_buckets[(key >> 2) & (DS_KEY_BUCKETS - 1)];
+}
+
+/* Build ranges from a flag map over the touched pages. Returns count. */
+static uint32_t ds_build_ranges(const uint8_t* flag, DsRange** out_ranges,
+                                uint8_t** out_bytes, const uint8_t* byte_src,
+                                int bytes_from_live, uint32_t* out_total) {
+    /* sort touched pages so ranges come out ordered and mergeable */
+    for (uint32_t i = 1; i < s_npages; i++) {         /* insertion sort: n <= 1025 */
+        uint32_t v = s_pages[i]; uint32_t j = i;
+        while (j > 0 && s_pages[j-1] > v) { s_pages[j] = s_pages[j-1]; j--; }
+        s_pages[j] = v;
+    }
+    uint32_t nr = 0, total = 0;
+    DsRange* ranges = 0; uint32_t cap = 0;
+    int64_t open = -1; uint32_t open_start = 0;
+    uint32_t prev_end = 0; int have_prev = 0;
+    for (uint32_t p = 0; p < s_npages; p++) {
+        uint32_t base = s_pages[p] << DS_PAGE_SHIFT;
+        uint32_t end = base + (1u << DS_PAGE_SHIFT);
+        if (end > DS_SPACE) end = DS_SPACE;
+        if (have_prev && base != prev_end && open >= 0) {
+            /* page gap closes any open run */
+            if (nr == cap) { cap = cap ? cap * 2 : 64; ranges = (DsRange*)realloc(ranges, cap * sizeof(DsRange)); }
+            ranges[nr].idx = open_start; ranges[nr].len = prev_end - open_start;
+            total += ranges[nr].len; nr++;
+            open = -1;
+        }
+        for (uint32_t i = base; i < end; i++) {
+            if (flag[i]) {
+                if (open < 0) { open = 1; open_start = i; }
+            } else if (open >= 0) {
+                if (nr == cap) { cap = cap ? cap * 2 : 64; ranges = (DsRange*)realloc(ranges, cap * sizeof(DsRange)); }
+                ranges[nr].idx = open_start; ranges[nr].len = i - open_start;
+                total += ranges[nr].len; nr++;
+                open = -1;
+            }
+        }
+        prev_end = end; have_prev = 1;
+    }
+    if (open >= 0) {
+        if (nr == cap) { cap = cap + 1; ranges = (DsRange*)realloc(ranges, cap * sizeof(DsRange)); }
+        ranges[nr].idx = open_start; ranges[nr].len = prev_end - open_start;
+        total += ranges[nr].len; nr++;
+    }
+    uint8_t* blob = (uint8_t*)malloc(total ? total : 1);
+    uint32_t off = 0;
+    for (uint32_t r = 0; r < nr; r++)
+        for (uint32_t i = 0; i < ranges[r].len; i++, off++)
+            blob[off] = bytes_from_live ? ds_live_byte(ranges[r].idx + i)
+                                        : byte_src[ranges[r].idx + i];
+    *out_ranges = ranges; *out_bytes = blob; *out_total = total;
+    return nr;
+}
+
+static void ds_persist(const DsShard* s);
+
+static void ds_finalize(CPUState* cpu) {
+    DsShard* s = (DsShard*)calloc(1, sizeof(DsShard));
+    s->key = s_arm_key;
+    memcpy(s->args, s_arm_args, sizeof(s->args));
+    s->nrd = ds_build_ranges(s_rd_flag, &s->rd, &s->rd_bytes, s_rd_orig, 0, &s->rd_total);
+    s->nwr = ds_build_ranges(s_wr_flag, &s->wr, &s->wr_bytes, 0, 1, &s->wr_total);
+    s->rd_crc = crc32_compute(s->rd_bytes, s->rd_total);
+    for (int i = 0; i < 17; i++) s->regs[i] = cpu->gpr[k_ds_gprs[i]];
+    s->regs[17] = cpu->hi; s->regs[18] = cpu->lo;
+    s->cyc_cost = psx_get_cycle_count() - s_arm_cyc0;
+    DsShard** b = ds_bucket(s->key);
+    s->next = *b; *b = s;
+    s_st.captures_ok++; s_st.shards++;
+    ds_persist(s);
+}
+
+/* ---------- persistence ---------- */
+
+static void ds_persist(const DsShard* s) {
+    if (!s_cache_dir) return;
+    char path[1200];
+    snprintf(path, sizeof(path), "%s/%08X_%08X.dss", s_cache_dir, s->key, s->rd_crc);
+    FILE* f = fopen(path, "wb");
+    if (!f) return;
+    uint32_t hdr[8] = { 0x31534450u /* 'PDS1' */, s->key,
+                        (uint32_t)s->cyc_cost, (uint32_t)(s->cyc_cost >> 32),
+                        s->nrd, s->nwr, s->rd_total, s->wr_total };
+    fwrite(hdr, 4, 8, f);
+    fwrite(s->args, 4, 4, f);
+    fwrite(s->regs, 4, DS_REG_COUNT, f);
+    fwrite(s->rd, sizeof(DsRange), s->nrd, f);
+    fwrite(s->rd_bytes, 1, s->rd_total, f);
+    fwrite(s->wr, sizeof(DsRange), s->nwr, f);
+    fwrite(s->wr_bytes, 1, s->wr_total, f);
+    fclose(f);
+}
+
+static void ds_load_key(uint32_t key) {
+    for (int i = 0; i < s_loaded_n; i++) if (s_loaded_keys[i] == (int)key) return;
+    if (s_loaded_n < 256) s_loaded_keys[s_loaded_n++] = (int)key;
+    if (!s_cache_dir) return;
+#ifdef _WIN32
+    char pat[1200];
+    snprintf(pat, sizeof(pat), "%s/%08X_*.dss", s_cache_dir, key);
+    struct _finddata_t fd; intptr_t h = _findfirst(pat, &fd);
+    if (h == -1) return;
+    do {
+        char path[1400];
+        snprintf(path, sizeof(path), "%s/%s", s_cache_dir, fd.name);
+        FILE* f = fopen(path, "rb");
+        if (!f) continue;
+        uint32_t hdr[8];
+        if (fread(hdr, 4, 8, f) == 8 && hdr[0] == 0x31534450u && hdr[1] == key) {
+            DsShard* s = (DsShard*)calloc(1, sizeof(DsShard));
+            s->key = key;
+            s->cyc_cost = (uint64_t)hdr[2] | ((uint64_t)hdr[3] << 32);
+            s->nrd = hdr[4]; s->nwr = hdr[5]; s->rd_total = hdr[6]; s->wr_total = hdr[7];
+            s->rd = (DsRange*)malloc(s->nrd * sizeof(DsRange));
+            s->wr = (DsRange*)malloc(s->nwr * sizeof(DsRange));
+            s->rd_bytes = (uint8_t*)malloc(s->rd_total ? s->rd_total : 1);
+            s->wr_bytes = (uint8_t*)malloc(s->wr_total ? s->wr_total : 1);
+            int ok = fread(s->args, 4, 4, f) == 4
+                  && fread(s->regs, 4, DS_REG_COUNT, f) == DS_REG_COUNT
+                  && fread(s->rd, sizeof(DsRange), s->nrd, f) == s->nrd
+                  && fread(s->rd_bytes, 1, s->rd_total, f) == s->rd_total
+                  && fread(s->wr, sizeof(DsRange), s->nwr, f) == s->nwr
+                  && fread(s->wr_bytes, 1, s->wr_total, f) == s->wr_total;
+            if (ok) {
+                s->rd_crc = crc32_compute(s->rd_bytes, s->rd_total);
+                DsShard** b = ds_bucket(key);
+                s->next = *b; *b = s;
+                s_st.shards++;
+            } else {
+                free(s->rd); free(s->wr); free(s->rd_bytes); free(s->wr_bytes); free(s);
+            }
+        }
+        fclose(f);
+    } while (_findnext(h, &fd) == 0);
+    _findclose(h);
+#endif
+}
+
+/* ---------- replay ---------- */
+
+static int ds_verify(const DsShard* s) {
+    uint32_t off = 0;
+    for (uint32_t r = 0; r < s->nrd; r++) {
+        uint32_t idx = s->rd[r].idx, len = s->rd[r].len;
+        const uint8_t* live = idx < DS_RAM_BYTES
+            ? memory_get_ram_ptr() + idx
+            : memory_get_scratchpad_ptr() + (idx - DS_RAM_BYTES);
+        /* a range never straddles the RAM/scratch boundary by construction */
+        if (memcmp(live, s->rd_bytes + off, len) != 0) return 0;
+        off += len;
+    }
+    return 1;
+}
+
+static void ds_apply(CPUState* cpu, DsShard* s) {
+    uint32_t off = 0;
+    for (uint32_t r = 0; r < s->nwr; r++) {
+        uint32_t idx = s->wr[r].idx, len = s->wr[r].len;
+        if (idx < DS_RAM_BYTES) {
+            /* write through the chokepointed byte path so dirty-code tracking,
+             * text guard, card traps and cosim all see the stores; then mark
+             * the executable range exactly like the CD-DMA delivery path. */
+            uint32_t va = 0x80000000u + idx;
+            for (uint32_t i = 0; i < len; i++)
+                psx_write_byte(va + i, s->wr_bytes[off + i]);
+            dirty_ram_mark_executable_range(va, len);
+        } else {
+            uint32_t va = 0x1F800000u + (idx - DS_RAM_BYTES);
+            for (uint32_t i = 0; i < len; i++)
+                psx_write_byte(va + i, s->wr_bytes[off + i]);
+        }
+        off += len;
+    }
+    for (int i = 0; i < 17; i++) cpu->gpr[k_ds_gprs[i]] = s->regs[i];
+    cpu->hi = s->regs[17]; cpu->lo = s->regs[18];
+    s_st.bytes_replayed += s->wr_total;
+    /* cycle credit: slices through the normal machinery so timers/VBlank/CD
+     * events and IRQ delivery land at their authentic guest cycles. */
+    uint32_t ra = cpu->gpr[31];
+    uint64_t remaining = s->cyc_cost;
+    while (remaining) {
+        uint32_t chunk = remaining > 2048u ? 2048u : (uint32_t)remaining;
+        psx_advance_cycles(chunk);
+        remaining -= chunk;
+        psx_check_interrupts_at(cpu, ra);
+    }
+    s_st.cyc_credited += s->cyc_cost;
+}
+
+/* ---------- gen-time hooks ---------- */
+
+int psx_datashard_enter(CPUState* cpu, uint32_t key) {
+    if (!s_enabled) return 0;
+    s_st.enters++; s_st.last_key = key;
+
+    if (g_ds_recording) {
+        /* nested/stale window: a hooked function re-entered (recursion or an
+         * abnormal exit skipped ret). Abort the outer capture, run natively. */
+        if (key == s_arm_key || cpu != s_arm_cpu) ds_poison(5);
+        return 0;
+    }
+
+    ds_load_key(key);
+    for (DsShard* s = *ds_bucket(key); s; s = s->next) {
+        if (s->key != key) continue;
+        if (s->args[0] != cpu->gpr[4] || s->args[1] != cpu->gpr[5] ||
+            s->args[2] != cpu->gpr[6] || s->args[3] != cpu->gpr[7]) continue;
+        if (ds_verify(s)) {
+            s->hits++; s_st.replays++;
+            ds_apply(cpu, s);
+            cpu->pc = cpu->gpr[31];   /* CPS: publish $ra, caller resumes */
+            return 1;
+        }
+        s_st.verify_fail++;
+    }
+
+    if (!ds_alloc_maps()) return 0;
+    ds_recorder_reset();
+    s_arm_cpu = cpu; s_arm_key = key;
+    s_arm_sp = cpu->gpr[29]; s_arm_ra = cpu->gpr[31];
+    s_arm_args[0] = cpu->gpr[4]; s_arm_args[1] = cpu->gpr[5];
+    s_arm_args[2] = cpu->gpr[6]; s_arm_args[3] = cpu->gpr[7];
+    s_arm_cyc0 = psx_get_cycle_count();
+    g_ds_recording = 1;
+    s_st.captures_armed++;
+    return 0;
+}
+
+void psx_datashard_ret(CPUState* cpu) {
+    if (!g_ds_recording || cpu != s_arm_cpu) return;
+    if (cpu->gpr[29] != s_arm_sp || cpu->gpr[31] != s_arm_ra) return; /* callee's ret */
+    g_ds_recording = 0;
+    if (!s_poisoned) ds_finalize(cpu);
+    ds_recorder_reset();
+}
+
+/* ---------- debug surface ---------- */
+
+void ds_stats_json(char* buf, int cap) {
+    snprintf(buf, (size_t)cap,
+        "\"enabled\":%d,\"enters\":%llu,\"replays\":%llu,"
+        "\"captures_armed\":%llu,\"captures_ok\":%llu,\"captures_poisoned\":%llu,"
+        "\"verify_fail\":%llu,\"shards\":%u,\"bytes_replayed\":%llu,"
+        "\"cyc_credited\":%llu,\"recording\":%d,\"last_key\":\"0x%08X\","
+        "\"last_poison_key\":\"0x%08X\",\"poison_mmio\":%u,\"poison_dma\":%u,"
+        "\"poison_thread\":%u,\"poison_budget\":%u,\"poison_stale\":%u",
+        s_enabled,
+        (unsigned long long)s_st.enters, (unsigned long long)s_st.replays,
+        (unsigned long long)s_st.captures_armed, (unsigned long long)s_st.captures_ok,
+        (unsigned long long)s_st.captures_poisoned,
+        (unsigned long long)s_st.verify_fail, s_st.shards,
+        (unsigned long long)s_st.bytes_replayed,
+        (unsigned long long)s_st.cyc_credited,
+        g_ds_recording, s_st.last_key, s_st.last_poison_key,
+        s_st.poison_mmio, s_st.poison_dma, s_st.poison_thread,
+        s_st.poison_budget, s_st.poison_stale);
+}

runtime/src/debug_server.c

@@ -1999,7 +1999,17 @@ void debug_server_synth_recurse_arm(void) { s_synth_recurse_armed = 1; }
 #endif
 
 static inline void cyc_watch_observe(uint32_t block_leader_phys);  /* defined below; used at fn-entry */
+
+/* Last guest function ENTERED, fed unconditionally at every compiled entry
+ * (one u32 store). The wall-time sampler's static histogram keys on THIS, not
+ * g_debug_current_func_addr: the dispatch stamp survives across post-exception
+ * resumption, so under any IRQ cadence it pools most static samples on the
+ * kernel exception-exit body (measured 55% on 0xF40) instead of the code that
+ * actually ran. Entry-stamp attribution is leaf-biased but honest. */
+volatile uint32_t g_psx_last_fn_entry = 0;
+
 void debug_server_log_call_entry(uint32_t func_addr) {
+    g_psx_last_fn_entry = func_addr;
 #ifdef PSX_STACK_GUARD
     g_psx_recent_fn[g_psx_recent_fn_i++ & (PSX_RECENT_FN_CAP - 1u)] = func_addr;
     psx_native_stack_guard(func_addr);   /* runs in debug AND release (before the early-return) */
@@ -2723,6 +2733,7 @@ static void handle_dirty_insn_dump_file(int id, const char *json)
  * argument-passing chains, and answer "who called X with what args"
  * across processes (psx-runtime port 4370, psx-beetle port 4380). */
 #include "fntrace.h"
+#include "starvation_ring.h"
 #include "bios_hle.h"
 #include "parity_trace.h"
 #include "device_trace.h"
@@ -11750,17 +11761,32 @@ static volatile uint32_t s_phot_addr[PHOT_SLOTS];
 static volatile uint64_t s_phot_cnt [PHOT_SLOTS];
 static volatile uint64_t s_phot_native_total = 0, s_phot_drops = 0;
 
-static void phot_add(uint32_t addr)
+/* Same histogram for STATIC-phase samples, keyed on the static-dispatch
+ * stamp (g_debug_current_func_addr). Coarser than the native histogram —
+ * the stamp is per static dispatch, not per C frame — but under CPS the
+ * dispatch cadence is dense enough to attribute wall time to main-EXE
+ * functions (the load-window decompressor question). */
+static volatile uint32_t s_phots_addr[PHOT_SLOTS];
+static volatile uint64_t s_phots_cnt [PHOT_SLOTS];
+static volatile uint64_t s_phot_static_total = 0, s_phots_drops = 0;
+
+static void phot_add_to(volatile uint32_t *ha, volatile uint64_t *hc,
+                        volatile uint64_t *drops, uint32_t addr)
 {
     if (!addr) return;
     uint32_t h = (addr >> 2) * 2654435761u;
     for (uint32_t p = 0; p < 16; p++) {
         uint32_t i = (h + p) & (PHOT_SLOTS - 1u);
-        uint32_t a = s_phot_addr[i];
-        if (a == addr) { s_phot_cnt[i]++; return; }
-        if (a == 0)    { s_phot_addr[i] = addr; s_phot_cnt[i] = 1; return; }
+        uint32_t a = ha[i];
+        if (a == addr) { hc[i]++; return; }
+        if (a == 0)    { ha[i] = addr; hc[i] = 1; return; }
     }
-    s_phot_drops++;
+    (*drops)++;
+}
+
+static void phot_add(uint32_t addr)
+{
+    phot_add_to(s_phot_addr, s_phot_cnt, &s_phot_drops, addr);
 }
 
 #ifdef _WIN32
@@ -11796,7 +11822,14 @@ static void *phase_sampler_main(void *arg)
                 s_phot_native_total++;
                 phot_add(overlay_loader_native_inprogress());
                 break;
-        case 3: s_phase_static[slot]++; break;
+        case 3: s_phase_static[slot]++;
+                s_phot_static_total++;
+                {
+                    extern volatile uint32_t g_psx_last_fn_entry;
+                    phot_add_to(s_phots_addr, s_phots_cnt, &s_phots_drops,
+                                g_psx_last_fn_entry);
+                }
+                break;
         case 4: s_phase_gpu[slot]++; break;
         default: break;                          /* 0 = host/other */
         }
@@ -11859,19 +11892,26 @@ static void handle_phase_profile(int id, const char *json)
 }
 
 /* phase_hot: top guest functions by native wall-time samples (cumulative
- * since boot — diff two snapshots for a window). {"cmd":"phase_hot","top":N} */
+ * since boot — diff two snapshots for a window). {"cmd":"phase_hot","top":N}
+ * Optional {"set":"static"} ranks the STATIC-phase histogram instead
+ * (main-EXE functions via the static-dispatch stamp). */
 static void handle_phase_hot(int id, const char *json)
 {
     int top = json_get_int(json, "top", 20);
     if (top < 1)  top = 1;
     if (top > 64) top = 64;
+    char set[16] = "native";
+    json_get_str(json, "set", set, sizeof(set));
+    int is_static = (set[0] == 's');
+    volatile uint32_t *ha = is_static ? s_phots_addr : s_phot_addr;
+    volatile uint64_t *hc = is_static ? s_phots_cnt  : s_phot_cnt;
     uint32_t best_addr[64];
     uint64_t best_cnt[64];
     int n = 0;
     for (int i = 0; i < PHOT_SLOTS; i++) {
-        uint32_t a = s_phot_addr[i];
+        uint32_t a = ha[i];
         if (!a) continue;
-        uint64_t c = s_phot_cnt[i];
+        uint64_t c = hc[i];
         int j = n < top ? n : top - 1;
         if (n < top) n++;
         else if (c <= best_cnt[j]) continue;
@@ -11883,13 +11923,16 @@ static void handle_phase_hot(int id, const char *json)
         best_addr[j] = a;
         best_cnt[j]  = c;
     }
-    uint64_t tot = s_phot_native_total;
+    uint64_t tot = is_static ? s_phot_static_total : s_phot_native_total;
     char buf[4096];
     int len = snprintf(buf, sizeof(buf),
-                       "{\"id\":%d,\"ok\":true,\"native_samples_total\":%llu,"
+                       "{\"id\":%d,\"ok\":true,\"set\":\"%s\","
+                       "\"phase_samples_total\":%llu,"
                        "\"hash_drops\":%llu,\"top\":[",
-                       id, (unsigned long long)tot,
-                       (unsigned long long)s_phot_drops);
+                       id, is_static ? "static" : "native",
+                       (unsigned long long)tot,
+                       (unsigned long long)(is_static ? s_phots_drops
+                                                      : s_phot_drops));
     for (int i = 0; i < n && len < (int)sizeof(buf) - 96; i++) {
         len += snprintf(buf + len, sizeof(buf) - (size_t)len,
                         "%s{\"addr\":\"0x%08X\",\"samples\":%llu,\"share\":%.4f}",
@@ -11921,8 +11964,74 @@ static void handle_idle_skip(int id, const char *json)
              g_idle_skip_last_pc, g_idle_skip_last_quantum);
 }
 
+/* starv_ring: query the always-on starvation/PC-sample ring (16K entries,
+ * continuous since boot). {"cmd":"starv_ring","count":N,"kind":K} -> last N
+ * entries (newest last), K = StarvationEventKind filter (15 = PC samples;
+ * -1/omitted = all kinds). PC samples carry (cyc, host_us, current_func):
+ * consecutive deltas give guest-throughput over wall time, and current_func
+ * localizes where the emu thread was — the ring-buffer answer to "where did
+ * the last N seconds go" without arming anything. */
+static void handle_starv_ring(int id, const char *json)
+{
+    int count = json_get_int(json, "count", 128);
+    int kind  = json_get_int(json, "kind", -1);
+    if (count < 1)    count = 1;
+    if (count > 2048) count = 2048;
+    uint64_t total = starvation_ring_total();
+    /* Walk backward collecting up to `count` matches, then emit oldest-first. */
+    static uint64_t match_seq[2048];
+    int n = 0;
+    uint64_t seq = total;
+    StarvationEntry e;
+    while (n < count && seq > 0) {
+        seq--;
+        if (!starvation_ring_get(seq, &e)) break;   /* fell off the ring */
+        if (kind >= 0 && e.kind != (uint8_t)kind) continue;
+        match_seq[n++] = seq;
+    }
+    size_t cap = 160u * (size_t)(n + 2);
+    char *buf = (char *)malloc(cap);
+    if (!buf) { send_err(id, "oom"); return; }
+    int len = snprintf(buf, cap,
+                       "{\"id\":%d,\"ok\":true,\"total\":%llu,\"returned\":%d,"
+                       "\"entries\":[", id, (unsigned long long)total, n);
+    int emitted = 0;
+    for (int i = n - 1; i >= 0; i--) {              /* oldest first */
+        if (!starvation_ring_get(match_seq[i], &e)) continue;
+        len += snprintf(buf + len, cap - (size_t)len,
+                        "%s{\"seq\":%llu,\"kind\":%u,\"cyc\":%llu,\"us\":%llu,"
+                        "\"func\":\"0x%08X\",\"store_pc\":\"0x%08X\",\"in_exc\":%u}",
+                        emitted++ ? "," : "",
+                        (unsigned long long)e.seq, e.kind,
+                        (unsigned long long)e.psx_cycle_count,
+                        (unsigned long long)e.host_us,
+                        e.current_func, e.last_store_pc, e.in_exception);
+        if ((size_t)len + 192 > cap) break;         /* never overrun */
+    }
+    len += snprintf(buf + len, cap - (size_t)len, "]}");
+    (void)len;
+    send_fmt("%s", buf);
+    free(buf);
+}
+
+/* data_shards: memoized pure-function replay counters (data_shards.c).
+ *   {"cmd":"data_shards"}              -> counters
+ *   {"cmd":"data_shards","enable":0|1} -> toggle, then counters */
+static void handle_data_shards(int id, const char *json)
+{
+    extern void ds_stats_json(char* buf, int cap);
+    extern void ds_set_enabled(int on);
+    int en = json_get_int(json, "enable", -1);
+    if (en == 0 || en == 1) ds_set_enabled(en);
+    char body[1024];
+    ds_stats_json(body, sizeof(body));
+    send_fmt("{\"id\":%d,\"ok\":true,%s}", id, body);
+}
+
 static const CmdEntry s_commands[] = {
     { "phase_profile",     handle_phase_profile },
+    { "starv_ring",        handle_starv_ring },
+    { "data_shards",       handle_data_shards },
     { "phase_hot",         handle_phase_hot },
     { "idle_skip",         handle_idle_skip },
     { "lockstep",          handle_lockstep },

runtime/src/memory.c

@@ -19,6 +19,7 @@
 #include "spu.h"
 #include "timers.h"
 #include "lockstep.h"
+#include "data_shards.h"
 #include "psx_cycles.h"
 #include <stdint.h>
 #include <stdio.h>
@@ -1200,8 +1201,14 @@ extern int g_ls_suppress_record;
 extern int g_dma_exec_depth;
 extern int psx_get_in_exception(void);
 static uint32_t psx_read_word_raw(uint32_t addr);
+/* data-shard capture feed (data_shards.c): record guest reads/writes while a
+ * capture window is armed. Same exclusions as the lockstep hooks: never in
+ * exception context (ISR work replays live), never under DMA (DMA writes
+ * poison the window via ds_note_dma_write instead). */
 uint32_t psx_read_word(uint32_t addr) {
     if (g_ls_mode == 2) return ls_read_hook(addr, 4, 0u);
+    if (g_ds_recording && g_dma_exec_depth == 0 && !psx_get_in_exception())
+        ds_note_read(addr, 4);
     if (g_ls_mode != 1 || s_ls_op_active || g_ls_suppress_record || g_dma_exec_depth > 0) return psx_read_word_raw(addr);
     s_ls_op_active = 1;
     uint32_t v = psx_read_word_raw(addr);
@@ -1308,6 +1315,10 @@ void psx_write_word(uint32_t addr, uint32_t val) {
     extern void (*g_overlay_flush_pending_cycles)(void);
     if (g_overlay_flush_pending_cycles) g_overlay_flush_pending_cycles();
     if (g_ls_mode == 2) { ls_write_hook(addr, 4, val); return; }
+    if (g_ds_recording) {
+        if (g_dma_exec_depth > 0) ds_note_dma_write();
+        else if (!psx_get_in_exception()) ds_note_write(addr, 4);
+    }
     if (g_ls_mode != 1 || s_ls_op_active || g_ls_suppress_record || g_dma_exec_depth > 0) { psx_write_word_raw(addr, val); return; }
     if (!psx_get_in_exception()) ls_write_hook(addr, 4, val);
     s_ls_op_active = 1;
@@ -1426,6 +1437,8 @@ static void psx_write_word_raw(uint32_t addr, uint32_t val) {
 static uint16_t psx_read_half_raw(uint32_t addr);
 uint16_t psx_read_half(uint32_t addr) {
     if (g_ls_mode == 2) return (uint16_t)ls_read_hook(addr, 2, 0u);
+    if (g_ds_recording && g_dma_exec_depth == 0 && !psx_get_in_exception())
+        ds_note_read(addr, 2);
     if (g_ls_mode != 1 || s_ls_op_active || g_ls_suppress_record || g_dma_exec_depth > 0) return psx_read_half_raw(addr);
     s_ls_op_active = 1;
     uint16_t v = psx_read_half_raw(addr);
@@ -1462,6 +1475,10 @@ void psx_write_half(uint32_t addr, uint16_t val) {
     extern void (*g_overlay_flush_pending_cycles)(void);
     if (g_overlay_flush_pending_cycles) g_overlay_flush_pending_cycles();
     if (g_ls_mode == 2) { ls_write_hook(addr, 2, val); return; }
+    if (g_ds_recording) {
+        if (g_dma_exec_depth > 0) ds_note_dma_write();
+        else if (!psx_get_in_exception()) ds_note_write(addr, 2);
+    }
     if (g_ls_mode != 1 || s_ls_op_active || g_ls_suppress_record || g_dma_exec_depth > 0) { psx_write_half_raw(addr, val); return; }
     if (!psx_get_in_exception()) ls_write_hook(addr, 2, val);
     s_ls_op_active = 1;

runtime/src/overlay_loader.c

@@ -1884,6 +1884,8 @@ void overlay_loader_init(const char *cache_dir, const char *game_id) {
         s_range_pc_cache[i].cand = -1;
     strncpy(s_cache_dir, cache_dir, sizeof(s_cache_dir) - 1);
     strncpy(s_game_id,   game_id,   sizeof(s_game_id)   - 1);
+    /* data shards persist under the same unified cache root (data_shards.c) */
+    { extern void ds_init(const char*, const char*); ds_init(cache_dir, game_id); }
     init_callbacks();
     scan_cache_dir();
     overlay_image_warm_init();

runtime/src/starvation_ring.c

@@ -165,6 +165,16 @@ void starvation_ring_pc_sample(void) {
     e->i_mask               = i_mask;
 }
 
+uint64_t starvation_ring_total(void) { return s_seq; }
+
+int starvation_ring_get(uint64_t seq, StarvationEntry *out) {
+    if (seq >= s_seq) return 0;                                /* not written yet */
+    if (s_seq - seq > STARVATION_RING_CAP) return 0;           /* evicted */
+    *out = s_ring[seq & (STARVATION_RING_CAP - 1)];
+    if (out->seq != seq) return 0;                             /* raced an eviction */
+    return 1;
+}
+
 void starvation_ring_dump(const char *path) {
     if (s_dump_done) return;  /* dump exactly once */
     s_dump_done = 1;
@@ -255,5 +265,7 @@ void starvation_watchdog_heartbeat(void) {}
 void starvation_watchdog_check(void) {}
 void starvation_ring_dump(const char *path) { (void)path; }
 void starvation_ring_pc_sample(void) {}
+uint64_t starvation_ring_total(void) { return 0; }
+int starvation_ring_get(uint64_t seq, StarvationEntry *out) { (void)seq; (void)out; return 0; }
 
 #endif /* STARVATION_RING_ENABLED */

runtime/src/traps.c

@@ -535,6 +535,9 @@ static HostThreadFiber* psx_get_or_create_host_thread(CPUState* cpu, uint32_t tc
 static int psx_change_thread_fiber(CPUState* cpu, uint32_t target_tcb)
 {
     uint32_t current_tcb = psx_current_tcb_ptr(cpu);
+    /* data-shard purity: a green-thread switch inside a capture window would
+     * record OTHER threads' work into the shard — poison the capture. */
+    { extern void ds_note_thread_switch(void); ds_note_thread_switch(); }
     debug_server_log_thread_event(3, cpu, current_tcb, target_tcb, 0);
     if (!psx_is_valid_tcb(cpu, current_tcb) || !psx_is_valid_tcb(cpu, target_tcb)) {
         debug_server_log_thread_event(4, cpu, current_tcb, target_tcb, 0);