Commit Graph

26 Commits

Author SHA1 Message Date
Drew T fa7b9d4c71 feat(phase-30 S45 L3): the emulator tour — all 28 script modules + MAIN/3 onboarded; fleet 212, R22 212/212
- THE TOUR (Drew driving the retail debug menu; mode-7 hammer over the Redux web API):
  all 28 script modules captured live at four byte-verified per-chapter slots
  (SC03/73-79 @0x801EF468 ch2-period, SC03/132-138 @0x801E25E8 ch3, SC04/24-30
  @0x801E7B28, SC05/23-29 @0x801ED988); the routing law: debug-menu AREA selects the
  chapter, each CITY interior streams its own module (member k <-> interior k).
  md_MAIN_011/DISELECT byte-proven 24,236/24,240 in RAM; slots A/B/boot R34-verified live.
- MAIN/3 DISCOVERED: the main-menu module (id 0x39, 121,884 B), mis-bucketed as data by
  BOTH audit oracles; live byte-proven @0x800CEDF8 (42,632-B exact prefix); onboarded.
- 29 onboardings BYTE-IDENTICAL on first build -> fleet 212; R22 212/212 after three
  md_MAIN_003 catches: the A4 DsMix leak; an extract-order-sensitive splat boundary
  (bytes: a 1-word data sentinel in .text + fn at +4 -> pinned in symbols file);
  corpus.stubs now treats D_*/jtbl_* INCLUDE_ASM as blob includes (mirrors progress.py)
- module-id census (offline, disc-wide): 77 id-law code payloads, 0 further misses;
  SC03/55 = confirmed DATA. audit-disc: UNCLAIMED 34 -> 6, residue 0 — the 6 carry
  byte-checked negative evidence; next tier = the CD-read tracer
- docs: memory-map §S45 (slots + routing + debug-menu ops), disc-completeness S45
  addendum, decision-log R31 entry, docs/debug-menu-list.txt (Drew's transcription)
- .run/s45 evidence allowlisted (tour logs/scripts/rosters); 104 ram dumps LOCAL-ONLY
- new baseline: 93.8% instr / 95.68% fn / 87.2% distinct over 212
2026-08-07 14:03:51 -06:00
Drew T e75ed7adcc docs(phase-30 S43): checkpoint — the permuter takes 63->41 and plateaus; evidence preserved
- func_8017C6F4 FINAL for this session: hand 63 -> ILS 42 (pin-free seed, masked 44, flat over 8
  warm restarts) -> ILS 41 (pin-t5 seed, masked 43, flat over 5). Best draft
  .run/s43/func_8017C6F4.ils43-pin.c (closeness 41), logged + allowlisted. Both basins are now
  MEASURED FLAT — do not re-run the ILS on these seeds; next levers are §148-C by hand, then Fable5.
- .gitignore: allowlist .run/s43/*.py + *.json so the refutation evidence (probe_leftovers.py,
  leftover_probe.json) is preserved, not one `git clean` from gone (R20, the S42 lesson).
- S43 checkpoint block refreshed at the top of the file: the four instrument defects as one table,
  the one number that moved, the resume list (with "26 unpropagated members" struck as refuted),
  the harvest_verify import hazard, and my four process errors.
2026-08-05 16:18:12 -06:00
Drew T fa122cf62f fix(phase-30 S43): permuter takes func_8017C6F4 63->42; the "rumour row" was an ADDRESS COLLISION
- THE FLOOR MOVED: permuter_ils on the S42 draft -> masked 65->44 (cycle 1, flat over 5 warm
  restarts); re-measured in match_one terms 63 -> 42 mismatched, 947/947 ins. First movement
  after ~40 hand probes, and it came from repairing an instrument (S43-1), not from new C.
  Draft preserved + allowlisted: .run/s43/func_8017C6F4.ils44.c; logged at closeness 42.
- THE S42 "rumour" CLAIM WAS WRONG (R14): the 2026-07-01 row HAS an artifact, it IS on disk,
  and it reproduces exactly (14 mismatched of 15 target ins, SIZE-MISMATCH/redraft). It is a
  near-worthless draft on a DIFFERENT BODY: 0x8017C6F4 is 15 ins in ov_SC03_010/011/013 and
  948 ins in ov_SC03_126/003 + ov_SC04_021 + ov_SC05_019 (§148-E, ledger side).
- THREE ledger defects fixed: (1) load_best keyed on ADDRESS ALONE -> the two bodies merged and
  the lower ABSOLUTE closeness won, so 14-of-15-wrong (7% correct) masked 63-of-947 (93%);
  now sub-keyed by known nins, legacy rows unchanged. (2) binary=null defaulted to ov_SC01_077,
  where the fn does not exist AT ALL, and "not an open stub" was read as "banked" -> today's
  result was invisible to render/grinder/target-selection (absent != done, R32/R34); now derive
  binary from the draft path + only drop when closed everywhere it exists. (3) `log` had NO
  --binary flag -- the root cause of every null; added + derived in append_record.
- IMPACT DERIVED, NOT ASSERTED (R37): replaying the pre-fix selection = 836 -> 837, 1 appeared
  (func_8017C6F4 nins=947), 0 vanished. One row today; the mechanism would eat every future one.
- PROBED AND NOT BUILT: relative-closeness ranking (only 24/836 rows carry closeness+nins, and
  the two orderings agree 14/15 on those). Documented in the log instead.
2026-08-05 16:06:16 -06:00
Drew T 35d00fe3ec chore(phase-30 S42): PRESERVE the two serial NEAR drafts + log them; flag a draft-less ledger row
Answering "did you bank the results": the two serial functions did NOT match, so there was nothing
to bank (G3 -- NEAR is not a match). Everything that DID match this session is already banked and
committed (7 from the S4 redo, 24 wave exemplars + propagations, both giants x138).

But the drafts were about to be LOST, which is worse than not banking them:

  .run/s42/ov_SC01_077/func_8017C294.c        NEAR(12) of 246   ~245k subagent tokens
  .run/s42/ov_SC03_126/func_8017C6F4.c        NEAR(63) of 947   ~434k subagent tokens
  .run/s42/ov_SC03_126/func_8017C6F4.pin-t5.c NEAR(47), pinned variant

All three were gitignored -- one `git clean` from gone (R20: commit irreplaceable work). Added a
curated /.run/s42/ allowlist and committed them. They are the best base any future attempt has:
func_8017C6F4 has frame 0x120 + vars=232 EXACT with only a register rotation left, and its permuter
has never been aimed at it (make_base_c fails on the gte_ macro block -- demacroize first).

Both logged to the backlog with today's MEASURED values, class, reach and draft path.

⚠️ LEDGER INTEGRITY, flagged not silently fixed: the backlog already held
`func_8017C6F4 closeness=14` (2026-07-01, ov_SC03_010, source=bulk-harvest) -- BETTER than today's
63, but with **draft: None, klass: None, nins: None, reach: None**. There is no artifact behind it
and no draft of it survives on disk (today's agent scanned every stored draft and found two, both
junk). `load_best` takes the LOWEST closeness per address, so this unverifiable row will out-rank
today's real, reproducible 63 in every future target selection.

This is the Phase-28 defect class (`func_80178004` recorded close=0 when it was 91). It is left in
place rather than deleted because deciding between "a lost good draft" and "a bad number" needs
evidence I do not have. **Whoever picks this up: treat the 14 as UNVERIFIED, start from the
committed 63/47 drafts, and if the 14 cannot be reproduced, purge the row.**

The general rule this argues for: a backlog row with no draft artifact is a rumour, not a result --
`backlog.py log` should require a draft path (or mark the row unverifiable) so an artifact-less
number cannot outrank a reproducible one.
2026-08-05 14:54:59 -06:00
Drew T e6e059a286 docs(phase-29): T31 — the 6 wave NEAR measured; 2 improved, 0 banked; the ADDRESSING route is the finding
NOTHING BANKED — no draft reached closeness 0. Recorded as such (P9). No src/ or config/ change,
so the fleet is untouched at HEAD's verified 140/140; R22 deliberately NOT re-run and NOT claimed.

- VERIFIED the checkpoint's six closeness numbers against the bytes (R14/R35): 217/10/11/6/7/9 all
  reproduce EXACTLY through match_one --json, with asm_subdir/-O0 DERIVED from wave22_targets.json
  rather than guessed. All six are reach-138 cores = ~135,516 templated ins (~1.04pp) if cracked.
  Reproducer .run/near6_measure.py.
- PERMUTER (only 2 of 6 are §60a-admissible): func_80177B5C 11->7, func_80140958 10->6, both
  re-measured with match_one — an oracle INDEPENDENT of the permuter's scorer (R34) — agreeing
  exactly. Both plateaued after cycle 1. Seeds preserved + allowlisted.
- THE FINDING: residual_class routes ADDRESSING -> permuter, but gcc-2.7.2-map/cse_expr.md §2
  documents that exact class (hoist-vs-remat) as STEERABLE by a byte-proven C recipe. The tool
  spends CPU searching for what the map says has a deterministic fix, and both ADDRESSING targets
  plateaued exactly as that predicts. R35-shaped instrument defect, not a compiler wall.
- NEGATIVE RESULT, byte-tested on func_80132F40 (6 variants): the §83d CSE fork is REAL (s32 fixes
  the min-block opcodes but hoists &v[0] into a 5th callee-saved reg, 47 mism), and the §2 kill
  moves it (47->40) but does NOT dissolve it in 3 placements. §2 has an unstated boundary: proven
  where the address's only uses are call arguments; a STRUCT-COPY source address survives the kill.
  close=6 (s16) remains the best known state — the wave agent's verdict, independently re-earned.
- NEW DIAGNOSIS: func_80140958's post-permuter residual is not scheduling — my draft CONSTANT-FOLDED
  a loop value the target keeps live (li v0,3 / li a3,3 / li t3,12 vs addu/sll from $v1). Untried.
- NOT SPENT, deliberately: func_80176734 (already a no-bank Fable5 pass, §H), func_80140D68 (~200
  compiles already), func_8012E364 (~2500 variants already).
- tools-health exit 0 (green, fail-closed) at preflight.
2026-07-28 10:23:27 -06:00
Drew T 72e6a096ca chore(phase-29): preserve the wave22 unbanked drafts + the regenerated family map
CAUGHT BY VERIFYING THE CHECKPOINT INSTEAD OF ASSERTING IT. `.run/wave22/` was covered by the blanket
`/.run/*` ignore, so the 13 UNBANKED drafts — 6 NEAR with precise residual diagnoses and 7 that
reached match_one MATCH but did not bank whole-binary — existed only on disk. My own checkpoint's
"START HERE" list names them as next-session fuel, and they cost ~2.59M subagent tokens to produce.
One `git clean -fdx` would have destroyed them: exactly the exposure the .run/giants Fable5 cracks
had before Phase 27 curated them (R20).

Now allowlisted: .run/wave22/*.c + .run/wave22_targets.json (the per-target canonical callee/data
declarations resolved from the real TU scope — the §17a-1 lever the wave was built on).
Also commits docs/family-hseq.md, regenerated by this session's family_hseq runs.
2026-07-28 10:05:29 -06:00
Drew T a8872fa169 docs(phase-29): cookbook §69 — behemoth recon (func_80183814, 5122 ins): map it, don't draft it
First attempt on the game's largest unmatched function. No match (never the goal); the deliverable
is the map, and every claim is byte-verified against the target .s.

- STRUCTURE: an actor state machine, not a straight-line giant. 21-case switch via jtbl_801F4CE4
  (sltiu 0x15); 359 jals to only 37 DISTINCT callees (verified); 48-ins preamble + 19-ins shared tail.
- THE FINDING: it decomposes into repeated templates, not 5122 unique instructions —
  35 instances of one "spawn-effect" packet (~1400 ins, crack one -> 34 free),
  7 "wait/countdown" (already reproduced at 0 skeleton diffs), 12 "HUD/text",
  plus twin cases (0≈3, 1≈4). Only 3 cross-jump edges couple anything.
- TWO GENERAL LAWS FOR GIANTS, measured: (1) register pressure is GLOBAL, so a partial draft gets
  10 callee-saved regs instead of 8 and a matching PREFIX is structurally unavailable — write all
  cases coarsely first, then refine; (2) match_one's global number is meaningless on a partial giant
  (666 vs 5122) — measure REGION-ALIGNED instead.
- NEW REUSABLE TOOL: .run/giants/s18_regions_comparator.py (region-aligned skeleton comparator, works
  on any giant). Caveat travels with it: masks register numbers + jal targets, so it proves STRUCTURE,
  never closeness; finish on the whole-binary gate (G3/P9).
- 2 idioms cracked in passing (the D_x[t+K] constant-fold needing a separate index statement; the
  (s16)*(u16*)p + /455 magic-0x90090091 form).
- VERDICT: tractable but a ~2000-line WRITE, not a hard puzzle — no scheduler wall, no unsteerable
  regalloc. Recipe for the next attempt recorded.
- artifacts preserved under the tracked .run/giants/ path (.gitignore now allowlists *.py there).
2026-07-25 00:33:06 -06:00
Drew T 5e78dccf07 feat(phase-29 §8e): jtbl 8-align pad-spec filter — the 4-giant unblock tooling (fleet-neutral)
- ROOT CAUSE PINNED (the session-2 half-pin was INVERTED; both probes were vacuous, R35):
  cc1 emits .align 3 before EVERY jump table; maspsx passes it VERBATIM (the :435 'drop' is
  an inventory-only pass); as bakes the pad SECTION-RELATIVE; link placement was never guilty
  (SUBALIGN(2) + ALIGN(.,4) place 4-mod-8 carve starts tight). Merging originally-separate
  TUs fires an intra-TU align where the original packed tight -> +4 at rodata 0xCC ->
  image-wide %lo shift. Honest probes persisted: .run/probe_jtbl/ (verdict.md + objdumps).
- NEW tools/jtbl_rodata_pads.py: post-maspsx filter replaces each rodata .align 3 with the
  ORIGINAL's exact pad bytes per a JTBL_PADS spec; fail-loud on table-count drift /
  non-align-3 / non-jtbl rodata content. Byte-proven: verbatim 0xE4 pad-at-0xCC ->
  filtered 0xE0 tight (= the merged carve span).
- jtbl_carve.py: spec-aware same-subseg merge (gap 0 or 4-with-zero-payload-word; else
  NON-CONTIGUOUS -> isolate), interval-arithmetic pad specs (committed values CARRIED,
  never re-derived), JTBL_PADS target-var emission into overlays.mk + revert() restore +
  stale-.o invalidation; the false 'maspsx drops .align' docstring corrected (H5).
- Makefile: $(if $(JTBL_PADS),| jtbl_rodata_pads.py ...) stage in build/src/%.o + file-scope
  empty default (env-shield). jtbl_family_bank.stub_file: duplicate-stub fail-loud (the
  earlier 'ladder failure' was a wrong-TU splice into a stale _a.c stub, byte-witnessed).
- R22 clean-fleet WITH the fix wired: 140/140 byte-identical, tools-health green
  (dedup 1846/0, C1 234205/234205), ZERO new banks -- fleet-neutral by construction.
- cookbook §8e (the jtbl alignment law) + §8a/§8a-pad corrections; decision-log R31 entry;
  SETUP.md tool row; .gitignore allowlist for the probe verdict artifacts.
2026-07-18 02:03:48 -06:00
Drew T 2bcd1344e3 chore(phase-27): preserve the Fable5 wave-1/2 recon (R20) — 112K, not 260M
The Fable5 discovery sprint's irreplaceable output, banked before the distillation (that's the
pending Max task). Each is the product of a ~250k-token Fable5 pass; the bulk beside them (260M of
RTL dumps + bisection .s under pincrash/) is regenerable and stays ignored (R33).

- wave-1 crack recon: func_8014D820 (block-0 cracked pin-free 261->110, "reused-load-temp
  serialization" lever), func_801670E4 (RC-6-not-S3 reclassification + the reg_renumber-swap oracle),
  func_8016CBC0 (root-A cracked byte-zero, "coalescing knife-edge" refuted, density-dial lever) +
  the workhorse variant.
- pin_crash_sigabrt.md + pincrash/{minimal_repro.c, *.gdb}: the §42e pin-crash wall CHARACTERIZED and
  REFUTED as a compiler wall — it's the extract_unit macro-drop (fixed in T5); sched.c:2725
  create_reg_dead_note abort; pinned families stage 133/133 clean once macros ride along.

All three wave-1 seeds produced oracle-proven reclassifications refuting cookbook §44-Lever-5 wall
names + new pin-free levers (Fable5 discovers, cheap-Opus applies). Distillation -> cookbook is next.
2026-07-15 19:22:21 -06:00
Drew T 6e99157bcf chore(phase-27 T3 addendum): widen the .run/giants allowlist — cookbook §45 cited untracked files
Found while reading the seeds for T1: cookbook §45 names
.run/giants/func_80133CD4.fable.c as its worked example and .run/giants/fable_cd4/
as the flagship's gdb oracle — BOTH were untracked. The docs cite artifacts that
were not in the repo.

- .gitignore: widen by FILE TYPE, not directory — .run/giants/*.{c,md,sh} +
  fable_cd4/*.{c,md,sh,gdb,txt}. +49 files / 460K.
- Now preserved: the flagship func_80133CD4 crack + its gdb oracle (§45's cited
  worked example); the byte-verified pf*.c regression ladder (the seeds' own
  Method/reproducibility section cites it: pf2 78, pf_c2 30, pf_d1 35, pf_h1 280);
  the dump.sh/mon*.sh RTL harnesses; the banked giants' drafts (80135480, 80163EC8,
  80166994).
- Still ignored (regenerable via dump.sh, R33): d_pf*.i.*, *.s, dumps_m*/, and the
  ILS/permuter .log files. Negative control re-verified: all 5 probes IGNORED, no
  db.*.gbf staged (R23).

Lesson (R31 candidate): a doc that cites a path is an untested claim about the repo.
The §45 citation and its file were 4 days out of sync; only reading the seed for an
unrelated reason caught it. Candidate lint: cookbook path citations must resolve to
tracked files.
2026-07-15 17:35:29 -06:00
Drew T 2351c43e66 chore(phase-27 T3): preserve the irreplaceable .run/ recon (R20) — 2.2M, not 12.3M
Pulled ahead of T1: the Fable5 sprint's agents work inside .run/, and its Phase-25
seed recons were untracked — an agent overwriting .run/giants/*.opus.c would have
destroyed irreplaceable input. 5 minutes to remove that risk.

- .gitignore: /.run/ -> contents-exclude form (/.run/* + ! exceptions), following the
  /tools/bin/*.sha256 precedent. Resolves R20 (commit irreplaceable RE work) vs R12
  (.run/ is scratch) by splitting the directory on the real axis: what a rerun CANNOT
  reproduce.
- PRESERVED (~2.2M / 31 files): the 6 Phase-25 *.opus.{c,md} giant seed recons (49K);
  the func_80178004 gdb-on-cc1 harness + ORACLE_PROOF.md + the v00-v07 draft ladder +
  the sched/combine .lst evidence (~110K — the distilled output of a 477k-token Fable5
  pass, and the method §52 lever 6 depends on); backlog.jsonl (1.9M) + fuel_manifest.json.
- STILL IGNORED (regenerable, R33): dumps_v00..v07/ and d_pf*.i.* gcc RTL scratch —
  12.3M reproducible via runorc.sh + the .gdb scripts; the MCP log; draft scratch.
  The plan said "track the dirs"; the bytes said the dirs are 96% regenerable.
- VERIFIED both directions: git add --dry-run stages exactly the 30 intended files and
  0 bulk; negative control — ghidra-mcp.log / dumps_v00 / d_pf.i.sched / d_pf.s all
  still IGNORED. No db.*.gbf staged (R23 restart-noise).
2026-07-15 17:32:59 -06:00
Drew T 80b89d8a2d feat(phase-23): LLM matching tier + the Fable5 wall-breaker + the gcc-2.7.2 codegen map (v1.22.0)
- LLM TIER (T1-T10.9): tools/serve_local + api_draft + lora_grind + bulk_harvest + the LoRA
  pipeline (v3 model). Local v3 saturated <=15 (~1300 banks, $0). Frontier GLM-5.2 (~$4)
  confirmed the def-side wall INTRINSIC + the idiom well DRY at the leaf (both angles).
- THE PIVOT (07-02): Fable5Max reading the gcc-2.7.2 SOURCE matched func_8014EE14, a §20/§10
  store-vs-load giant "CONFIRMED unsteerable" for 22 phases -> the "wall is intrinsic" verdict
  is MODEL-RELATIVE. Generalized to Opus agents applying §30 (2 more giants, 3-5x cheaper).
- §31 THE CODEGEN MAP: 4 Fable5 agents read the whole gcc-2.7.2 source -> docs/gcc-2.7.2-map/
  {sched,regalloc,loop,cse_expr}.md (935 lines, byte-proven residual->lever catalogs) +
  cookbook §31 (index+triage). Broke hoist-vs-remat/delay-slot/coalescing walls. Correction:
  gcc-papermario is 2.8.1 not 2.7.2 -> tools/reference/gcc-2.7.2/ (SETUP §5.6).
- THE WAVE (8 reach-134 near-misses, Opus + §31): 3 leaf-MATCH by lookup (47-137k tok vs
  200-250k walls) + 5 tight permuter seeds (func_80132784 240->4-off!), zero dead-ends.
- 5 giants/hard-fns banked x134 (func_8014EE14/F2E0/150528/149374/144090); fleet 63.66->64.86%,
  136/136 byte-identical, 0 NON_MATCHING. cookbook §30/§30a/§31. No new governance rules.
- FINDING: matching is SOLVED by the map; bottlenecks are now the permuter (S11 seeds) +
  whole-binary integration (auto-bank leaf-MATCHes) -> Phase 24. worklog -> logs/Phase23.md.
2026-07-02 21:51:25 -06:00
Drew T 76054ef383 chore(phase-23): gitignore the unused .venv-serve (llama-cpp SIGILLs; serve via .venv-train) 2026-06-30 07:37:10 -06:00
Drew T bff03dbdb5 fix(phase-23): T7 — lora_grind broad-rotation banking (two gate bugs); ov_SC01_000 7/15
The 500-fn calibration banked 0/222 across the binary rotation. Root-caused (R14, by
reading the code + the run's own backlog — resolving a flat contradiction between two
scout agents) to TWO independent bugs in lora_grind's use of gate_stage.run_gate, NOT
model quality:
- Bug A: good_sha() passed the sha1sum line "<sha>  <name>" vs harvest_verify's bare
  sha1() -> 0 banks for EVERY binary incl. 077 (so the "0/12" was a bug artifact, not
  an exhausted tail)
- Bug B: the gate call left src/asm/out at the hardcoded ov_SC01_077 defaults -> non-077
  drafts dropped at the 077 stub-filter, silently (and the asm mis-resolution contaminated
  the backlog near-miss classification)

Fix (tools/gate_stage.py): run_gate resolves src/asm/out/good_sha from `binary` when unset
(binary-agnostic, no silent ov_SC01_077 default an overlay inherits; good_sha bare-hash
normalized) + a loud negative-control guard (0-overlap binary/src mismatch warns, so a 0
can never again masquerade as 'nothing matched'). tools/lora_grind.good_sha fixed at source.
Byte-neutral: make check-all 136/136.

Proof: ov_SC01_000 spot-run banked 7/15 (47%) byte-identical (@commit:0322); reach-2
func_8017CE24 propagated x2. ROI finding: 6/7 banks are reach-1 (overlay-unique) -> broad
rotation is high bank-RATE / low fleet-% ROI; the fleet lever is reach>=2 targeting (T9) +
corpus-v3 (T8). Backlog now correctly classified (4x close=1 = grinder fuel).

- docs/gen2-mips-matching-model.md: T7 RESULT section
- phase-ends/CURRENT_PHASE.md: T7 done; next = T8 corpus-v3 / T9 reach>=2 selection
2026-06-29 23:44:12 -06:00
Drew T ad535ad1ab feat(phase-22): gen2 matching-model design + export_pairs.py corpus miner
docs/gen2-mips-matching-model.md: the BFM/gcc-2.7.2 matching-specialist idea (LoRA on our own
gate-verified pairs — the corpus off-the-shelf RE LLMs lack). export_pairs.py mines 1307 banked
(asm<->C) pairs from build objects (asm/ is gitignored, so disasm the ROM-identical build, splat-like
format) + src defs -> datasets/match_pairs/{pairs,train,test}.jsonl (gitignored, 1174/133 split).
api_draft.py: TEMP env-tunable. .gitignore: datasets/ models/ weights.
2026-06-29 14:27:19 -06:00
Drew T d20b20c64b feat(phase-18): T0 — setup + exemplar set; match_one validated as the §10 oracle
- clone gcc-papermario (a6afc2af) + xenogears-decomp (f27c0768) -> tools/reference/
  (gitignored research refs; SHAs pinned in docs/SETUP.md §5.6, R20/R21)
- apparatus byte-validated: match_one cleanly measures the §10 residual on both quirk
  exemplars (the floor-free oracle Phase 17 failed to use; permuter score was polluted):
  - func_8012B8E4: 75=75 ins, 24 mismatched -> $s0/$s1 regalloc swap + reassociation (§10 A)
  - func_8012B4B8: 94 vs 84 ins, 88 mismatched -> extra-reg spill + struct-copy/reload (§10 B)
- CURRENT_PHASE.md crash-recovery log; harness task list (R28); no build-affecting change
2026-06-20 11:08:21 -06:00
Drew T 144839f24b feat(phase-13): T1a/T1b — SC01/077 onboarded + all-asm byte-match (d19c9580)
- ov_SC01_077 scaffolded: config/overlays.mk (OVERLAY_BINARIES + var block),
  config/splat.ov_SC01_077.yaml (from template), check.sha, empty symbols, +
  ov_SC01_077 entry & sentinel anchor in the 4 Python BINARIES dicts
- GATE T1b: make check BINARY=ov_SC01_077 -> d19c9580 BYTE-IDENTICAL @ 100%
  INCLUDE_ASM; clean rebuild leaves main 143dbb89 + resident 8e17e02f unregressed
- reusable non-word-aligned-overlay handling (~75% of fleet) added to the
  template + Makefile: (1) [word_floor, bin, trailing] carve for the final 1-3
  bytes spimdisasm drops; (2) build/assets/%.o incbin rule (+ .data align=1) for
  splat bin assets (asset_path scoped per-alias); (3) objcopy end-align TRIM
  (shrink-only, <=3 B) removing the .ld's segment-end ALIGN(.,4) pad
- .gitignore /assets/ (regenerable splat output); clean removes assets/
2026-06-16 15:39:52 -06:00
Drew T d380b13fc5 feat(phase-9): close-out — PhaseEnd_Phase9 + binary-agnostic toolchain (v1.9.0)
- PhaseEnd_Phase9.md written; CURRENT_PHASE.md archived -> phase-ends/logs/Phase9.md (R19)
- MILESTONE (gate-2 confirmed): the toolchain is binary-agnostic — EXE rebuilds
  143dbb89 through the parameterized path WITH and WITHOUT SDK objects; make report
  52/959/7/50.24%; every binary-specific value a required param (no EXE default);
  wrong --vram-base -> cae22f7e (negative control). Landed as 11 per-tool checkpoints.
- R23: stop the Ghidra MCP + commit the DB at phase-end/RE-checkpoint (lock won't
  release until MCP closes; SessionEnd is too late for a mid-session commit). This
  commit reconciles the Ghidra program DB (db.12 -> db.15, 99% identical: Phase 9 did
  zero RE writes) after a clean ghidra_mcp_stop.sh save.
- .gitignore: ignore /ghidra/**/*.lock~ (the lock-backup that churned every session)
- bumps project version 1.8.0 -> 1.9.0
2026-06-15 22:01:17 -06:00
Drew T 6f248462dd chore(gen2-prep): Gen2 roadmap + private-repo backup hardening + SETUP/submodule fixes
- docs/gen2-roadmap.md: approved Gen2 plan (phases 8-15; finish-EXE-first,
  two-repo LATE public flip, substantial exit) + Backup & disaster-recovery policy
- back up irreplaceable / hard-to-re-source artifacts to the private remote (R20):
  - track ghidra/ (the RE database; *.lock/tmp*.ps transients excluded)
  - tools/psyq/ working libs/tools (minus the two >100MB raw source archives)
  - tools/bin/*.tar.gz (old-gcc cc1 compiler sources; extracted binaries stay regenerable)
  - tools/ghidra-ext/ (GhidrAssistMCP 2.8.0 + ghidra_psx_ldr 2026.06.04 installer zips)
- .claude/settings.json: SessionStart + SessionEnd hooks (auto-start / clean-save the
  headless Ghidra MCP); moved out of the gitignored settings.local.json so they are backed up
- docs/SETUP.md: document the MCP server lifecycle + persistence model + session hooks (§2.8),
  a tooling inventory, .run/ runtime scratch (§1a), and the backup posture (§ Backup) — R21
- .gitmodules: ignore=dirty on the 4 submodules (maspsx/asm-differ/m2c/decomp-permuter) —
  silences the phantom "modified" state from generated pycache + cross-filesystem filemode;
  gitlinks unchanged (still at the pinned commits)
- rules R20 (back up irreplaceable work + gathered tooling; loosens R8) and R21 (keep
  SETUP.md current) recorded in memory; to be formalized at the next PhaseEnd
2026-06-15 17:15:31 -06:00
Drew T d55882b4b8 feat(phase-5): splat config + all-asm byte-identical build of SLUS_007.26
- config/splat.us.exe.yaml: splat PSX config (platform psx, compiler PSYQ,
  subalign 2, gp_value 0x80074750, base_path .., main segment align 4); one
  asm text seg [0x800] + data seg [0x531DC], header via splat's header segtype
- Makefile: implement extract (splat split) / build (as -> ld(splat .ld +
  undefined_*_auto) -> objcopy) / check (SHA1 vs config/check.us.sha, auto-run)
  / expected (asm-differ baseline) / clean; check-env untouched
- config/check.us.sha: committed build-check hash (validation stage 2)
- config/symbols.us.txt: now hand-curated — 15 2D_* -> gfx2D_* (illegal leading
  digit broke as), +D_800629D4 label; header flags re-merge + Ghidra-mirror (G6)
- .gitignore: ignore regenerable splat outputs (asm/, include macros,
  undefined_*_auto.txt); refine the earlier commit-asm/ note
- docs: SETUP.md §4.5 binutils 2.42 = byte-clean (no 2.35 downgrade), §5.3 +
  ledger #8 RESOLVED -G0, §6.3 as-built; memory-map.md gp0 -> -G0 + text/data split
- MILESTONE: make extract && make build && make check -> build/us/SLUS_007.26
  SHA1 143dbb89... == original (BYTE-IDENTICAL), 100% asm, check fails on corruption
- rule R15 (symbol file = curated, valid-identifier build input, Ghidra-mirrored)
- bumps project version 1.4.0 -> 1.5.0
2026-06-14 12:05:38 -06:00
Drew T ffe2352db2 feat(phase-4): matching-build toolchain (WSL) — make check-env green
- apt: build-essential, binutils-mipsel-linux-gnu 2.42 (+gcc-mipsel 12.x), ninja
  1.11.1, python3-pip/venv, libelf-dev, clang-format, p7zip, bchunk, bsdmainutils
- .venv + splat64 0.41.0 ([mips]: spimdisasm 1.41.0, rabbitizer 1.16.2, PyYAML 6.0.3)
- 4 pinned submodules under tools/: maspsx@874855c5, decomp-permuter@b44b0622,
  asm-differ@2ad4a4a4, m2c@4266cc28 (.gitmodules)
- old-gcc 0.17 cc1 candidates -> tools/bin/gcc-2.7.2-{psx,cdk}/ (32-bit i386 static;
  tarballs are flat + same-named -> per-subdir extraction); CHECKSUMS.sha256 committed
  (gitignore exception !/tools/bin/*.sha256); psx cc1 self-id "GNU C 2.7.2 [AL 1.1,
  MM 40] Sony Playstation"
- Makefile: check-env (.ONESHELL bash) asserts python>=3.12, venv import splat, both
  cc1 (stdin smoke), maspsx, mipsel as/ld/objcopy, sha1(EXE)==EXPECTED_EXE_SHA1;
  binutils>=2.38 -> WARN (verdict deferred to Phase 5, §4.5); extract/build/check/
  expected/clean are loud-failing Phase-5 stubs (names per §6.3)
- docs/SETUP.md: as-built versions; §4.7 corrected (i386, flat-tarball fix); §4.8
  deferred to Phase 6; ledger #5 confirmed (Ubuntu 24.04.4), #6 (2.42), #7 (sha256)
- MILESTONE: make check-env exits 0; zero ROM-derived bytes staged
- bumps project version 1.3.1 -> 1.4.0
2026-06-14 03:56:07 -06:00
Drew T 3207d52491 feat(phase-3.5): prototype research spike — go/no-go (protos don't accelerate Gen1 matching)
- extract_proto_exe.py: subdir-aware proto main-EXE extraction (reuses iso9660); extracted
  sep8 SLUS_007.26 (413,696 B) + aug31 MUSASHI/USA_DEMO.EXE (415,744 B, base 0x80018000)
- ghidra_import.sh: reusable headless import; both protos imported into the bfm project
  (sep8 1726 funcs, aug31 1737 funcs, both PsyQ 4.0.0), R9-verified
- DumpFunctionSignatures.java + match_protos.py: 3-tier (exact/norm/seq) cross-binary
  correspondence; MIPS normalizer validated by anchors
- FINDINGS: (T2) no debug symbols in either proto — Hidden Palace "no symbols" VERIFIED;
  recon "Sep-8 less-stripped" REFUTED by per-file byte compare. (T3) Sep-8 99.6% byte-identical
  to retail (only 3 funcs differ: DebugMenuHandler, CdReadSectorReadyCB, SaveLoadRoutine);
  Aug-31 66% norm-identical, 862 1:1 correspondences. (T4) demo shares retail's 18-entry
  game-mode dispatch architecture but the handler code diverged — Q#10 resolved
- VERDICT (docs/proto-correspondence.md): NO-GO as a Gen1 label/symbol accelerator; GO to retain
  Sep-8 (Phase-6 compiler corroborant) + both (Gen2 assets); proto-side labels in
  config/symbols.proto-{sep8,demo}.txt (R13-tagged, never feed symbols.us.txt)
- reorg extracted/ into per-ROM subfolders (extracted/retail/, extracted/proto/); nested
  gitignore allowlist; extractor defaults updated; manifest --verify PASS
- memory-map.md: Phase-3.5 block + demo dispatch anchors; Q#10 RESOLVED; §5 proto notes VERIFIED
- ghidra_mcp_start.sh: PROG arg to serve a prototype
- rules R13 (proto-provenance/confidence tagging), R14 (verify recon counts vs bytes)
- bumps project version 1.3.0 -> 1.3.1
2026-06-14 03:19:23 -06:00
Drew T bd5e21b366 feat(phase-3): file-loader & overlay-map RE + PCSX-Redux runtime oracle
- RE'd the full file-loader/overlay subsystem in Ghidra via MCP: CdReadRequest ->
  CdReadStateMachine (hand-rolled CdControl SeekL/ReadN reader, NOT PsyQ CdRead);
  LIST.CD RAM cache cdFileLocTable + ResourceGetCdLoc/resourceIdMap lookup; LZSS
  staging buffer 0x80079A70; loadDestPtrTable; ~50 symbols named (T1-T5,T8)
- T6b MILESTONE: resident blob @0x800CEDF8 + location overlay @0x80128158 proven
  byte-identical (sha1-equal contiguous prefixes) vs live PCSX-Redux RAM; blobs
  imported into Ghidra (ImportOverlay.java); overlay-map table -> memory-map.md §4.3
- PCSX-Redux runtime oracle: Windows-native build, WSL reads RAM via the web API
  at 172.17.208.1:8081; ledger #4 resolved
- live-verified player/time state: gold(/10) HP BP day hour tiredness boss-HP;
  save-header template (Q#5) + debug-menu dispatch (Q#9: DEBUG.BIN path is dead)
- overlay map: 13 area-states across SC01/02/03, 2 duplicate pairs, location x
  chapter keying, free-roam vs menu-hub granularity
- resolves Open Q#1/#3/#4/#5/#6/#8/#9/#12; corrects SLUS-90029 (FF8 demo, not BFM)
- 2 US prototypes acquired + CRC-verified (Aug-31 1369DE07, Sep-8 5C24728E)
- rules R9 (verify persistence), R10 (>=3-datapoint live verification),
  R11 (runtime oracle Windows-native, bridged to WSL)
- bumps project version 1.2.0 -> 1.3.0
2026-06-14 02:04:28 -06:00
Drew T b6ea3a2747 feat(phase-2): deterministic disc/.CD/PAC/LZSS extraction pipeline
- extract.py: one-command full-disc extraction (27 files: 24 Track-1 verbatim
  + 3 .DA as raw CD-DA audio from tracks 2-4) over the frozen iso9660.py
- cd_archive.py + pac.py: .CD TOC walker + PAC splitter, {index}.{type} naming
  (no collisions; 447 sub-files, 1189 entries), SQV detection (98)
- lzss.py: game-semantics decoder (pos==0 terminator, -1 bias) + 5 unit tests;
  138 type-4 decoded clean (0 warnings)
- manifest.py: deterministic JSON-Lines manifest (1801 artifacts) + --verify
- crosscheck.py + tools/brave-CUE/posix_shim.h: build CUE's brave from source
  (GPL untouched), byte-compare; 1484 raw identical, 138/138 type-4 F6
- docs/formats.md: F5 counts (SC02=43), F2 refuted, F6 resolved, F1 type-7=139
- docs/effort-map.md + CLAUDE.md: Effort-map check rule (R7), Ultracode fix
- .gitignore: /extracted/* + !manifest/EXE exceptions; brave _build/
- install gcc 13.3.0 for the oracle
- milestone: one command extracts the full disc; MAIN.CD=49 sub-files; decoder
  agrees byte-for-byte with CUE up to the game pos==0 terminator; zero ROM
  staged; project version 1.1.0 -> 1.2.0
2026-06-13 19:47:42 -06:00
Drew T ca51930605 docs: convert to all-in-WSL architecture; relax H1 ROM policy while private
Surgically correct PROJECT_CONTEXT.md env sections (H1/H2, Quick Ref, Architecture diagram, Assumptions/Risks, Phase 1 & 4 roadmap) to Linux-first single-clone; all rules/roadmap content otherwise preserved. SETUP.md rewritten Linux-first with all facts intact + redump disc-provenance added. .gitignore permits ROM-derived content (asm/assets/extracted) while private; raw dump + build churn stay ignored.
2026-06-10 23:43:31 -06:00
Drew T d4334fc1ca plan files created 2026-06-10 22:02:07 -06:00