Commit Graph

10 Commits

Author SHA1 Message Date
Drew T 788f33d523 feat(phase-30 S45 L3-p3): SC02/9 = the Steam Knight boss module — decoded, captured, retro-verified, onboarded; parked = 5
- the gate DECODED from matched C (func_8012832C case 0x300E -> func_80128998 -> streaming
  API with &cdFileLocTable[144]) -> scene arithmetic named the 1ST-BOSS arena -> ONE targeted
  load captured it at 0x801E4C60
- RETRO-VERIFIED: Phase-3's dumps/ram_castle.bin (2026-06-14) holds it at the SAME address,
  same 6,764-B exact prefix — R10 two independent datapoints two months apart;
  bossHp_SteamKnight (0x801E4398) lives inside this module's image
- onboarded md_SC02_009 (id 0x3E, TLO 0x4): BYTE-IDENTICAL first build; fleet 213;
  R22 213/213; tools-health OK; audit-disc UNCLAIMED 6 -> 5, residue 0
- the last 5 (MAIN/7, MAIN/9, SC03/53/54/56) reclassified emulator->STATIC-RE targets with
  decoded leads (memory-map §S45 p3); loc-id map appended to docs/debug-menu-list.txt
- negatives banked: pause menu, memory-box prompt, new-game intro, high/low game, Minku
  spawn (slot-A actor 0x15 = md_MAIN_015 candidate naming)
2026-08-07 15:07:55 -06:00
Drew T fa7b9d4c71 feat(phase-30 S45 L3): the emulator tour — all 28 script modules + MAIN/3 onboarded; fleet 212, R22 212/212
- THE TOUR (Drew driving the retail debug menu; mode-7 hammer over the Redux web API):
  all 28 script modules captured live at four byte-verified per-chapter slots
  (SC03/73-79 @0x801EF468 ch2-period, SC03/132-138 @0x801E25E8 ch3, SC04/24-30
  @0x801E7B28, SC05/23-29 @0x801ED988); the routing law: debug-menu AREA selects the
  chapter, each CITY interior streams its own module (member k <-> interior k).
  md_MAIN_011/DISELECT byte-proven 24,236/24,240 in RAM; slots A/B/boot R34-verified live.
- MAIN/3 DISCOVERED: the main-menu module (id 0x39, 121,884 B), mis-bucketed as data by
  BOTH audit oracles; live byte-proven @0x800CEDF8 (42,632-B exact prefix); onboarded.
- 29 onboardings BYTE-IDENTICAL on first build -> fleet 212; R22 212/212 after three
  md_MAIN_003 catches: the A4 DsMix leak; an extract-order-sensitive splat boundary
  (bytes: a 1-word data sentinel in .text + fn at +4 -> pinned in symbols file);
  corpus.stubs now treats D_*/jtbl_* INCLUDE_ASM as blob includes (mirrors progress.py)
- module-id census (offline, disc-wide): 77 id-law code payloads, 0 further misses;
  SC03/55 = confirmed DATA. audit-disc: UNCLAIMED 34 -> 6, residue 0 — the 6 carry
  byte-checked negative evidence; next tier = the CD-read tracer
- docs: memory-map §S45 (slots + routing + debug-menu ops), disc-completeness S45
  addendum, decision-log R31 entry, docs/debug-menu-list.txt (Drew's transcription)
- .run/s45 evidence allowlisted (tour logs/scripts/rosters); 104 ram dumps LOCAL-ONLY
- new baseline: 93.8% instr / 95.68% fn / 87.2% distinct over 212
2026-08-07 14:03:51 -06:00
Drew T 4cadac4e11 feat(phase-30 S45 II.1c): module batch dedup-banked + verified — 408 banks, R22 183/183, audit-disc 75->34 (parked-only)
- dedup measure (R37 probe): 69/1,113 module fns h_exact-match matched corpus (~6%, LOW as
  planned — modules are novel frontier); dedup_extend inapplicable (same-vram group model) ->
  family_sweep --hseq --band all over the 57 matched-exemplar families: 408 member-matches
  banked (182 into modules, 226 into the big 3 — families Part I's --only scoping missed),
  169 failed + 77 STRUCT = genuine per-member frontier
- R22 clean-fleet 183/183 BYTE-IDENTICAL; audit-disc UNCLAIMED 75->34 residue 0 (34 = 31
  parked-for-L3 + SC03/53,54,56 — 3 rows Discovery-3 never tiered, now parked with evidence)
- three instrument fixes, each negative-control-proven:
  - family_sweep --hseq stub map derives ov_*+md_*+resident (was sig.ov_* glob -> module
    members silently 'not-stub', R32 class) [committed earlier as commit:1506]
  - sig-modules seeds from the built ELF's func_* symbols (bootstrap GLUES adjacent fns
    around jtbl dispatch -> 24 false TRUNCATED; perturbed-sig control still bites)
  - corpus.audit counts CODE lines only (module .s carries its header jtbl as .word lines);
    progress.py buckets INCLUDE_RODATA symbols as blobs (unbucketed R32 hole)
- NEW HONEST BASELINE (183 binaries): 94.0% instr / 95.96% fn-count / 87.6% distinct;
  tools-health OK, audit-digest OK
2026-08-06 13:14:03 -06:00
Drew T 41ff2ba127 docs(phase-30 S44 I.3): checkpoint — Part I complete; fresh session resumes at Part II
- R22 clean-fleet 143/143; fleet 96.13% fn / 94.4% instr (honest grown denominator; pre-expansion
  line 95.00% on 140 kept for continuity) / 88.3% distinct. audit-binaries OK over 143.
- make audit-disc: UNCLAIMED 78 -> 75 payloads (3,564,021 -> 2,038,104 B), residue 0 — the three
  claims flipped automatically via check.sha, exactly as the ledger was designed.
- S44 session total: 5,126 member-functions banked into the 3 new overlays; the ~2,051 remaining
  stubs are the new frontier, visible to every tool via citizenship (no separate ledger rows —
  recorded as a deviation from plan I.2e, redundant by construction).
- Part II handoff live in the plan file + the S44 checkpoint block.
2026-08-06 12:10:25 -06:00
Drew T e6e05d727a fix(phase-30 S39/S7): regenerate main's 7-week-stale Ghidra sig — honest denominator, metrics re-baselined
main's sig was dated 2026-06-14 and covered only 1,525 of its 2,002 INCLUDE_ASM stubs: 477
stubs were INVISIBLE to the weighted metrics, so main was being graded against two-thirds of
itself and the fleet denominator was understated.

  tools/ghidra_mcp_stop.sh  (R23 - clean stop, "Save succeeded", releases the .rep lock)
  make sig-refresh BINARY=main  -> DumpFunctionSignatures.java, read-only, -noanalysis

  signatures      1,729 -> 2,205  (+476)
  stubs covered   1,525 -> 2,001  of 2,002   (1 still uncovered, named below)
  main .text ins  60,201 -> 79,510  (+19,309)

METRICS RE-BASELINED (a DENOMINATOR CORRECTION, not a regression -- the Phase-27 precedent,
where onboarding 4 hidden overlays honestly moved 68.9% -> 67.0%):

  instr-weighted   94.5% -> 94.4%   (12,419,169 / 13,160,961; numerator UNCHANGED)
  distinct-code    89.3% -> 89.0%   (5,029,454 / 5,654,184;   numerator UNCHANGED)
  fn-count         96.51% (unchanged -- it never read the sig)
  MAIN game-code   0.72% -> 0.55%   (436 / 79,510)

Both numerators are identical across the change; only the denominator grew. audit-digest OK.

CAVEAT UNCHANGED (R34): main still has no INDEPENDENT boundary oracle -- sig_image cannot sign a
PS-X EXE (header offset, interleaved islands), so main's function list rests on Ghidra alone and
audit-corpus cannot cross-check it. That is the open item in docs/second-oracle.md and part of
task #9's denominator work.

NOTE the sig itself is gitignored/regenerable, so this commit lands only the digests; a fresh
clone reproduces it with `make sig-refresh BINARY=main` (Ghidra required, MCP stopped).

MCP IS NOW DOWN (I stopped it for the project lock). The SessionStart hook restarts it next
session; run /mcp before any Ghidra work (R29).
2026-08-05 00:41:54 -06:00
Drew T 76db32455c feat(phase-28 T0): fix img_path (derive, don't guess) — the SC07 pool was doubly hidden
R35 sequencing: fix the instrument before the probe that scopes the phase.

- family_remap.img_path: DERIVE the payload from config/splat.<bin>.yaml's target_path
  (R33 — the file the BUILD reads, so it cannot drift from the bytes) instead of
  reconstructing `.../FILE_{nnn}.dir/0.4.dec` from the alias. RAISES on a missing
  payload (R32) — the silent None WAS the defect.
  Negative control (the fix must change an answer the old tool gave):
    ov_SC01_001 -> 0.4.dec   UNCHANGED (no regression)
    ov_SC07_006 -> None      ->  .../1.4.dec
    resident    -> (n/a)     ->  MAIN.CD.dir/FILE_010.dir/1.1   (free; feeds T5)
    ov_SC99_999 -> None      ->  raises
  Downstream: all 233 shared substantial fns between ov_SC07_006 and ov_SC01_001
  classify PURE (reloc-only). Under the old tool every one returned LEN = "not
  templatable" AND poisoned its family's diff_class to MIXED (family_hseq.py:141-143).
  Same bug class as new_overlay.sh's hardcoded 0.4.dec glob (which hid these four
  overlays for a month) — left uncorrected in a second tool. Fourth instance of the
  project's dominant defect class, sitting directly under the number P28 must measure.

- .run/family_hseq.json regenerated: 134 -> 138 overlays (the 4 P27 SC07 overlays newly
  visible); metrics re-baselined 68.9 -> 67.0% instr (now agreeing with the committed
  progress.fleet.md); LEN across the whole frontier = 0 (a phantom-LEN from a missing
  image is now structurally impossible). Proven consistent against the post-tools-health
  sigs by a second run (byte-identical) rather than assuming sig_image is deterministic.

- FINDING — a large, doubly-hidden target pool: 1255 families / 6268 members / 230,612 ins
  whose ONLY unmatched members are in the 4 new SC07 overlays (0 elsewhere — a clean
  partition), each behind an ALREADY-MATCHED, byte-proven ov_SC01_077 exemplar. Classes
  PURE 5575 (89%) / IMM 633 (10%) / STRUCT 60 (1%). Hidden twice: P27's disc audit created
  it by onboarding the overlays but never regenerated the map — and had it, img_path would
  have classified every member LEN. Corroborated independently by tools-health: the 4 new
  overlays are ~97% unmatched (stubs ~2,400, matched ~80) vs ~85% matched for their
  siblings. PREDICTION, not a bank — h_seq predicts, the whole-binary gate decides (G3/P9).
  -> T3's headline stratum, and a better probe than planned: the exemplar is already
  byte-proven, so a failure isolates the templating mechanism with no drafting variable.

- SELF-CORRECTION (R14): the approved plan's own population figures (163 families /
  13,232 members) came from the STALE map — my numbers were an instance of the defect this
  phase is about. Honest: 1418 matched-exemplar families / 21,889 unmatched members
  (PURE 17,024 = 78% / IMM 4,473 = 20% / STRUCT 392 = 1.8% — the roadmap's "register-drift"
  swing class stays ~2% of the input, so that framing is unchanged). Legacy pool unchanged
  at 163 families (the fix + the 4 overlays are purely additive).

- T3 strata (honest): SC07-only 1255 fam / 6268 mem / 230,612 ins · legacy PURE non-jr
  95 / 7993 / 478,379 · legacy IMM 36 / 6644 / 212,707 · legacy MIXED 30 / 968 / 10,462 ·
  legacy PURE w/ jr 2 / 16 / 5,088. Total addressable 937,248 ins = 21.7% of all remaining
  weight = 7.16pp of fleet instr if it all banked — the prize the roadmap declared dead.

- tools-health GREEN: sigs fresh; corpus(+resident) 0 PHANTOM + 0 TRUNCATED; cdecl;
  report(lint + dedup 1840 validated / 0 failed, C1 coverage 227211/227211). No source or
  build input touched (analysis tooling + regenerated digests only) -> no byte claim, no
  R22 cycle owed. docs/duplicates.cross.md regenerated: overlays 134x -> 138x, h_exact
  cross-binary 9366 -> 9484 groups; resident sig now the sig_image one (P27 T10 intent).
2026-07-15 21:44:13 -06:00
Drew T c716eb15e1 feat(phase-15): T4 — fleet progress roll-up + fix cross-report double-count
- progress.py: refactor to report(binary) + set_binary() + a per-binary .s index
  (fleet run 6m38s -> 7s); add --fleet -> docs/progress.fleet.md (deterministic,
  source-derived per-binary table + fleet totals). Single-binary output byte-unchanged.
- fleet now: 947 REAL / 959 LINKED / 13132 byte-identical / 344010 matchable = 3.82%
  across 136 binaries (main + resident + 134 overlays)
- dup_report --cross: HONESTY FIX (R14) — an onboarded overlay is both a named BINARIES
  entry AND a .run/sig.ov_* glob hit; it was counted twice, inflating collapsible bytes
  to 58.7M once the fleet was onboarded. Dedupe by alias -> accurate 9366 h_exact groups
  / 28.6M collapsible (matches the Phase-11 figure)
- Makefile: wire progress.py --fleet into make report (main-only block)
- docs: progress.fleet.md (new), duplicates.cross.md regenerated, SETUP inventory
- make report green; dedup-check 8 validated / 0 failed
- ghidra/ churn NOT staged (R23)
2026-06-16 20:00:19 -06:00
Drew T 52514b5f45 docs(phase-13): T6 docs — cookbook §13 overlay runbook + SETUP/gen2-roadmap + digests
- cookbook §13: the canonical add-an-overlay runbook (new_overlay.sh, the flat-blob
  config, the non-4-aligned bin-carve/asset/trim gotcha, A->B->C workflow, dedup-credit
  two shapes, build-all/check-all) — the Phase-15 fleet recipe
- SETUP tooling inventory: tools/new_overlay.sh (R21)
- gen2-roadmap Phase-13 milestone: ACHIEVED amendment (6 binaries byte-identical,
  704 harvested, dup-pair collapse, T7 effort finding)
- regenerated per-overlay progress/difficulty/duplicates digests + cross report
2026-06-16 17:56:38 -06:00
Drew T f672c709c1 feat(phase-11): T6 — sign the 134 overlays; cross-report spans 136 binaries (milestone report half)
- sig_image overlay discovery: linear partition (split contiguous code at jr+delay boundaries) +
  detect_code_end (first run of invalid instrs = the code->data transition; overlay code decodes
  ~100% valid, data drops to 43-95%). BFS dead-ended — overlays dispatch via function-pointer
  tables, not jal — so call-graph BFS found ~2 fns; linear partition recovers the whole code prefix
- Makefile: sig-overlays: signed 134 overlays -> .run/sig.ov_*.jsonl signs all 134 SCxx 0.4.dec @ 0x80128158 -> .run/sig.ov_*.jsonl (27s)
- dup_report --cross: ingest the overlay sigs (sig.ov_* convention) + main/resident; condensed
  source header (overlay set-sha for reproducibility); cap top-200 per subsection + state totals;
  compact member sample + #bin column
- RESULT (docs/duplicates.cross.md): h_exact 9366 cross-binary groups / 28.5 MB collapsible; h_norm
  8957 / 38.3 MB. Top group = a 770-instruction function BYTE-IDENTICAL in all 134 overlays (one
  match credits the whole fleet) — 'one match unlocks many' quantified; the Phase-12/13 work queue
- dedup-check green; committed per-binary digests byte-stable; report deterministic (sorted glob,
  no Date/random); the EXE<->resident pair still shares nothing (overlay<->overlay dominates)
2026-06-16 01:26:49 -06:00
Drew T 7cc2911456 feat(phase-11): T1 — cross-binary dup_report --cross → duplicates.cross.md
- dup_report.py: additive --cross mode ingests all BINARIES sigs, tags each row
  with its binary, buckets by h_exact then h_norm across binaries, splits
  cross-binary (binaries>1, the Phase-12/13 work queue) vs intra-binary, ranks by
  collapsible bytes (count-1)*nins*4, one row per member -> docs/duplicates.cross.md
- Makefile: make report emits the cross report once (gated BINARY=main)
- per-binary duplicates.md/.resident.md stay byte-stable (per-binary path unchanged)
- EXE<->resident cross-binary groups = 0 (4.0 vs 4.7 + different roles, as the
  byte-finding predicted); real cross groups arrive with overlays (T6)
- phase-ends/CURRENT_PHASE.md: Phase 11 plan + per-task log
2026-06-16 00:37:18 -06:00