Commit Graph

73 Commits

Author SHA1 Message Date
Drew T 66d1af0e09 docs(phase-21): automation runbook + status tool (grinder running; worker = remote-driven)
- docs/automation-runbook.md: launch/monitor/stop for the grinder (live) + worker /loop;
  safety invariants (byte-gate sole arbiter, git crash-safe, auto_stop kill switch);
  remote-management note (Drew can remote in -> no bulletproof keep-alive needed).
- auto_status.sh: show grinder + worker heartbeats, backlog, phase-21 checkpoints, daemons.
2026-06-21 13:12:04 -06:00
Drew T 8de18e617d feat(phase-21): T4/T5/T6 — worker validated (func_80164530 x134), grinder, orchestrator
- T4 worker Workflow validated: 3-target wave, all self-MATCH; whole-binary gate banked
  func_80164530 x134 (5 register-pins + offset-fold + scheduling). fleet 58.90->58.98%.
  §20 holds (1/3 survived the whole-binary gate); the 2 near/failed -> backlog.
- T5 grinder: token-free permuter of backlog near-misses -> gate_stage; gate correctly
  rejected a permuter over-prediction (integrity intact). auto_supervisor DRIVER-param.
- T6 orchestrator.py: ROI pool rotation (prep/finish/status); gate_stage flock serializes
  grinder+orchestrator builds. Sandbox: detached daemons build fine, foreground needs override.
2026-06-21 13:06:34 -06:00
Drew T 2d7926e9f6 feat(phase-21): T4/T5 tooling — worker Workflow + wave selector + permuter grinder
- tools/workflows/worker_wave.js: parallel toolkit-aware drafter agents (xHigh) per target,
  each writes its best matching C to the drafts dir (gate run by the orchestrator after).
  args parsed robustly (harness may serialize to a JSON string — §20 gotcha).
- tools/wave_targets.py: select a wave batch from the fuel manifest (pool/region/N, skips
  backlog walls). tools/grinder.py: token-free permuter grinder — permute backlog near-misses
  -> gate_stage -> bank; STOP/heartbeat, idles for new worker near-misses, supervisor-friendly.
2026-06-21 12:41:14 -06:00
Drew T c2e93e50cd feat(phase-21): T3 — backlog ledger + shared gate stage (validated, +0.08%)
- tools/backlog.py: near-miss ledger (.run/backlog.jsonl + docs/backlog.md ranked by
  reach->closeness->size; drop-now-matched filter, P9). tools/gate_stage.py: the shared
  deterministic spine both worker+grinder call (canon_resident_calls -> cast_call_sites ->
  sig_unify -> harvest_verify --chunk 1 -> dedup_propagate --auto-from -> backlog -> summary).
- validated end-to-end on a real 30-draft m2c sample: banked func_801710DC + propagated x134
  (dedup 1506->1508), 29 near-misses logged+ranked. make check-all 136/136 byte-identical
  (R22); fleet 58.82%->58.90%; dedup-check 1508 validated/0 failed (G3/P9).
- capped-fns reframe (R14): they're matched-inline (propagation-blocked), not draftable ->
  deferred to the worker; not claiming the unbanked +0.3% (P9).
2026-06-21 12:32:21 -06:00
Drew T 5674587bcb feat(phase-21): T2 — fuel prefetch, Ghidra-C cache complete for the ROI pool
- headless DecompileFunctions over 263 uncached ROI-pool fns (MCP stopped, R23):
  263 ok / 0 fail / 0 no-func. cache 300 -> 563. ROI-pool fuel now 325/325 cached
  (28/28 giants, 310/310 reach-134, 7/7 capped) -> the unattended run never needs MCP.
- FIX DecompileFunctions.java: key output by entry address func_<UPPERHEX>.c (Ghidra's
  raw-import default is FUN_<lowerhex>, which every consumer + the existing 300 files
  miss). Renamed the 263; the tool fix makes re-prefetch reproducible.
- build_fuel_manifest.py --emit-prefetch: reproducible uncached-ROI-pool addr-list
  (for the T7 runbook). 349 reach-1 x1-leverage fns intentionally excluded (P9).
- ghidra/ db churn is the MCP-stop no-op save (R23 restart-noise, not staged).
2026-06-21 12:00:47 -06:00
Drew T 971f95f7c8 feat(phase-21): T1 — fuel manifest + R14 giants byte-verification
- tools/build_fuel_manifest.py -> .run/fuel_manifest.json: the automation target
  pool. reach RECOMPUTED from the current 134 overlay sigs (distinct-overlay
  h_exact), not trusted from cached fields (R14). Classes GIANT/WAVE/STRUCT/PINS/
  STUB/O0/O1 + the CAPPED recovery pool; cached? flags Ghidra-C gaps for T2.
- 803 live func_ stubs; 310 reach-134, 486 reach-1. R14: 28/28 giants verified
  genuinely byte-identical x134 (label honest, unlike the -O0 cluster). -O0
  nuance: 8/9 overlay-local but func_8013C08C is reach-134 (real exception).
- T2 workload scoped: 253 reach-134 uncached; 171/223 tractable reach-134 uncached.
- CURRENT_PHASE.md: T1 logged. No MCP, no build mutation.
2026-06-21 11:50:31 -06:00
Drew T 6c5bd1c4a9 feat(phase-20): cast_call_sites.py — the §17a-1 callee-cast cap tool; 6 recovered, fleet 58.63%->58.82%
- tools/cast_call_sites.py: per draft, callee whose canonical TU sig != draft's
  intended sig -> decl->canonical (kills in-TU conflicting types, keeps symbol in
  scope) + call-site cast ((ret(*)(args))func_X)(args) (codegen-neutral: gcc folds
  the cast of a known symbol -> direct jal). Pure --in/--out; whole-binary gate is
  the arbiter (G3/P9). Pipeline: canon_resident_calls -> cast_call_sites ->
  sig_unify -> harvest_verify --chunk 1
- recovered 6 of T6 batch-1's 33 gate-fails byte-identical (func_80153C44/8015CF58/
  801711FC/80161BE0/801683D8/8015F948); 5 propagated x134 + 1 local; check-all 136/136
- R14 CORRECTION of cookbook §20's '~33' projection (byte-proven): the batch is NOT
  mostly callee-conflicts — ~6 callee (cast-recovered), ~18 DEF-side loose-typing wall
  (def's matchable sig conflicts with the unchangeable caller-canonical; caller-side
  fix BLOCKED because INCLUDE_ASM declares no symbol -> a shared macro's internal
  extern is load-bearing in the 133 stub overlays), + a small DATA tail. The def-side
  needs re-drafting under the caller-canonical sig (future wave), not a text transform
- cookbook §20 RESOLUTION + SETUP tool inventory (R16/R21/R30); CURRENT_PHASE log (R28)
- fleet 58.63% -> 58.82%, dedup 1501 -> 1508, 0 NON_MATCHING (G4)
2026-06-21 02:04:39 -06:00
Drew T ba6cb87d30 feat(phase-20): T2 — exemplar_miner routes 835 residuals (WAVE 472 / PINS 45 reach-134 / STUB 90)
- tools/exemplar_miner.py: consume wall_taxonomy.json + per-overlay reach (dedup_propagate's
  computation) -> route each residual to a lever (WAVE/STRUCT/PINS/STUB); emit
  docs/exemplar_curriculum.md + .run/exemplar_routing.json (wave-gen input)
- routing: WAVE 472 (218 reach-134, T6 fuel) / STRUCT 159 / PINS 114 (45 reach-134) / STUB 90
  (74 ARITY_WALL loose-typing + 16 NONFAITHFUL, honest P9)
- KEY T3 input: 17 reach-134 fns at m2c-mismatch=1 = cleanest regalloc/schedule isolates
  (likely SS17-pins wins + the genuine class residuals); 3 named exemplars confirmed reach-134
- note: census is the Jun-19 snapshot (pre-Phase-19 waves); T5/T6 regen = authoritative WAVE list
2026-06-20 23:55:51 -06:00
Drew T 4f5f73c480 feat(phase-20): T1 — typedef lift closes the type-blocked propagation cap (9 fns ×134)
- build_engine_types.py: add find_typedefs() (brace-aware) + typedef_name() (anon-struct,
  fn-ptr, alias) + same-name/overlap collision guards; lift 14 typedefs to engine_types.h
  in source order after the named structs (A801593E4 -> S801593E4 dep preserved)
- engine_types.h: 48 named types + 14 typedefs; ov_SC01_077 byte-neutral (still d19c9580)
- dedup_propagate --auto-from: 9 reach-134 fns propagated x134 (8013E4B4 80148C9C 8014A238
  8014C118 80153978 801593E4 80168664 8016B964 8017AD0C); engine_core.h +9 DEFINE macros
- FINDING (R14/P9): cap was 16; only 9 were TYPE-blocked (all freed). The other 7 fail on
  undeclared callees/data (80142B2C 801535F4 80155E30 80157580 801576A8 80168F40 80170B48)
  -> the SS19 declaration-plumbing class, a recovery-tooling lever, NOT the type-lift
- check-all 136/136 byte-identical; fleet 58.00%% -> 58.35%%; dedup 1485 -> 1494 / 0 failed; 0 NON_MATCHING (G4)
2026-06-20 23:49:58 -06:00
Drew T 3e99d2836d feat(phase-19): T3 batch-2 wave — 25 matched, 8 propagated ×134, fleet 57.62%->57.93%
- Ultracode wave (38 targets, clean hints after the gen_harvest_targets fix): 35/38 match_one
  MATCH (92%, up from 88%); 2 of 3 garble-retries now match; 3 DIFF = genuine (delay-slot
  schedule, call-crossing hoist-vs-remat, narrow-param STOP)
- gen_harvest_targets.py: fixed INLINE_DEF_RE (was matching indented `if(func_X(...)){`
  call-exprs as defs -> garbled callee hints agents pasted as externs); 0 garbled of 749
- whole-binary gate: 25 banked. ALL 18 stage failures were the shared-caller ARITY class ->
  NEW tools/fix_arity_callers.py automates the no-proto caller-extern recovery (+8); 10 remain
  = genuine loose-typing conflicts (caller sig != def sig), Phase-16 documented wall
- pipeline finding folded in: gate canon-only FIRST, sig_unify FALLBACK (.run/t3_gate2.sh)
- propagated 8 ×134 (17 stayed ov_SC01_077-local: inline named structs need engine_types lifting)
- check-all 136/136 BYTE-IDENTICAL (R22); dedup 1475->1483 validated/0 failed; 0 NON_MATCHING (G4)
- fleet byte-identical 57.62% -> 57.93% (+0.31%)
2026-06-20 19:51:50 -06:00
Drew T 85afa0dc81 feat(phase-19): T2 recovery tooling — canon_resident_calls + no-proto caller fix (3 matches)
- R14 finding: the 3 Phase-18 "implicit-int / propagate-first" near-misses are NOT
  implicit-int (reproduced each through the byte-gate). Real classes:
  (a) resident-callee LINK-miss: draft calls func_<ADDR> but the addr has a curated
      name (0x8004CFEC = ratan2) -> "undefined reference to func_8004CFEC". NEW
      tools/canon_resident_calls.py rewrites func_<ADDR> -> curated name (pure draft
      text, body bytes unchanged) -> run FIRST in the recovery pipeline.
  (b) shared-caller ARITY conflict: an engine_core.h caller macro declares the callee
      file-scope `extern void func_X(void)` then calls func_X(); the real def takes an
      arg -> "conflicting types". FIX = caller extern -> no-proto `extern void func_X();`
      (byte-neutral for the caller; compatible with promotion-safe params int/long/ptr).
- 3 matches banked in ov_SC01_077: func_8017209C (ratan2 rewrite), func_80147514 (s32),
  func_80168F40 (void*); 2 no-proto lines in src/shared/engine_core.h
- recovery pipeline: draft -> canon_resident_calls -> sig_unify -> harvest_verify --chunk 1
- make check-all 136/136 BYTE-IDENTICAL (full extract, R22); dedup 1450 validated/0 failed;
  0 NON_MATCHING in any default build (G4); fleet 56.64% (the 3 are ov_SC01_077-local,
  x134 propagation rides T3/Phase-20)
- cookbook §17a-3 corrected (R14/R16); SETUP tool inventory (R21); CURRENT_PHASE T2 done
2026-06-20 17:12:00 -06:00
Drew T c4b0cef86a feat(phase-17): close — canonical-sig layer built; the wall is the compiler, not sigs; pivot to gcc research (v1.16.0)
- canonical-sig layer (session 4): tools/census_conflict_callees.py + derive_canonical_sigs.py
  -> a 20-extern byte-neutral block atop ov_SC01_077.c (LOCAL, not engine_core.h); census
  conflict callees 20->0, blocked targets 24->0; gate pipeline now draft -> sig_unify (MANDATORY)
  -> harvest_verify --chunk 1; fleet 136/136 byte-identical (R22), 55.51% (no regression)
- FINDING (R14/P9): the conflict wall is 7%-reach not ~2x; the 4 reach-134 circular targets are
  ALL gcc-quirk/regalloc/layout-bound (0 banked); the high-reach core IS the quirk tail; struct
  types are byte-neutral for matching (the wall is gcc codegen, not knowledge)
- leverage analysis: fleet % is function-count-weighted (size adds no %); "unblock many" = the
  layer (declaration, not matching); reach is the lever (already reach-sorted); 247 tractable
  reach-134 stubs ~ +3-4% projected
- GO/NO-GO: NO-GO on brute waves at the current ceiling; GO on a compiler-quirk research phase
  (read gcc-2.7.2 source + Xenogears + the §10/regalloc classes, R17) -> then resume the wave
- docs: cookbook §16 corrected + hand-matching-process.md §8 (the layer + the finding + handoff)
- worklog archived -> phase-ends/logs/Phase17.md (R19); bumps 1.15.0 -> 1.16.0
2026-06-20 09:35:18 -06:00
Drew T 0dd9bf6bac feat(phase-17): calibration harvest wave — fleet 55.04%->55.51% (+0.47%)
- Ultracode wave: 30 high-reach tractable residuals, parallel guided-hand-matching
  agents (m2c + Ghidra-C cache + asm + actor-struct + S1/S2/S3a playbook), self-
  validate match_one. 18/30 match_one MATCH (60%); 10/30 whole-binary verified (33%)
  after sig_unify recovered 2 -> all 10 propagated x134; 2 demo fns caught up
- 136/136 byte-identical (R22 clean check-all); REAL +1,610; 1421 dedup groups
- KEY FINDING: the match_one->whole-binary gap (60%->33%) is 100% SIG CONFLICTS
  (shared callees like func_80131CA8 declared inconsistently across parallel agents),
  NOT codegen. The canonical-sig layer (deferred as low-value) is the ESSENTIAL
  enabler -> would lift 33% toward 60%. 12/30 are the genuine gcc-quirk tail (DIFF).
- Ghidra pre-pass: tools/ghidra_scripts/DecompileFunctions.java (headless batch
  decompile -> .run/ghidra_c cache; no /mcp needed); 300 tractable targets sized
- harness: .run/harvest_wave_s3.js (embedded-targets Workflow; args channel doesn't
  transit arrays). cost ~1.89M tokens / 30 fns
2026-06-19 20:54:45 -06:00
Drew T 1431b523a4 feat(phase-17): T4+T5 — actor struct recovered (comprehension) but byte-neutral for matching
- recovered the engine actor/player struct cold via PCSX-Redux (R10/R11): base 0x80078E00,
  ~154 fields over 0x24C bytes, 32 pointer fields; HP/BP/gold/day/hour/position/BP-drain all
  live-verified and anchored to the Phase-3 player state + the walk diff. -> docs/actor-struct.md
- tools/ram_probe.py: PCSX-Redux RAM capture/diff/read/ptr-scan workhorse for field-typing
- FINDING (the matching verdict): feeding the struct to m2c as --context is BYTE-NEUTRAL.
  m2c adopts it (arg0->f_X) but structural-miss sample scores 0 better / 10 same / 2 worse vs
  m2c-alone (full code+RAM pointer set). The misses are 50-98% mismatched = m2c decompiler
  limits, not missing types. T4 fn-ptr context unblocks compilation but functions stay structural.
- conclusion: rich context (struct/type recovery) is a comprehension / decomp.me win, NOT a
  byte-match lever -> Phase-16 wall re-confirmed from the type angle. struct-core-pivot.md updated.
- emulator field-naming pass deferred (comprehension, off the matching path); NEXT: T6 permuter
  validation on the 146 genuine near-misses, then T7 go/no-go.
2026-06-19 13:03:14 -06:00
Drew T 070cf375c9 feat(phase-17): T3 — bank 42 sig_unify matches + propagate x134 (fleet 54.48%->55.00%); fix taxonomy classifier
- banked 42 gate-proven SIG_FIXABLE_KR matches in ov_SC01_077 (of 48 predicted; 6 honestly
  reverted by the whole-binary byte-gate, P9) + propagated 13 h_exact-shared x134 via
  dedup_propagate (one body -> 134 overlays, each byte-gated, fail-closed)
- fleet byte-identical-from-source 54.48% -> 55.00% (+0.52%); REAL +1,778; dedup groups
  1395 -> 1408; make check-all 136/136 byte-identical (R22); dedup-check 1408 validated/0 failed
- wall probe (R14): raw drafts 0/99 + sig_unify fails -> the K&R/sig_unify avenue is exhausted;
  the residual conflict-class is ARITY_WALL (the §14e param-count dead-end) or structural (T4/T5)
- FIX a T1 classifier bug surfaced by verifying the wall vs the bytes: tu_compile_error captured
  the baseline's pre-existing byte-NEUTRAL warnings ("makes integer from pointer", cc1 rc=0) and
  mislabeled 99 fns LOOSE_TYPING_WALL; now keeps only non-warning errors -> there is NO loose-typing
  wall, the real residual wall is ARITY_WALL=74. docs/wall-taxonomy.md amended with the correction.
- 134 overlay .c files carry the propagated DEFINE_func instantiations; a one-time full-fleet
  make extract was required for the per-overlay propagation gate + check-all
2026-06-19 12:13:38 -06:00
Drew T edb0360d30 feat(phase-17): T1 wall taxonomy — census the 957 residuals, size every avenue (skip T2)
- tools/wall_taxonomy.py: full-census classifier (m2c --valid-syntax -> match_one ->
  sig_unify + full-overlay-TU cc1) labels every unmatched ov_SC01_077 stub by blocker
  class; reproducible (~22s, --jobs 16); read-only (throwaway _wt_*.c, auto-removed)
- docs/wall-taxonomy.md: ranked taxonomy + per-lever aggregation + sequencing recommendation
  (957 residuals, 99.2% carry an actionable label)
- findings (byte-gate-validated, R14: SIG_FIXABLE_KR = exactly the 4/4 gate-passing set):
  - T4+T5 rich context (struct/jump-table/fn-ptr-table types) = 67% reach, incl. all 30 jtbl giants
  - T6 permuter = 146 candidates (101 <=4-mismatch near-certain)
  - T3 sig_unify = 48 gate-proven immediate wins
  - loose-typing WALL = 99 (10%); non-faithful defer = 16
  - T2 DATA-symbol byte-array header = 0 functions -> SKIP (the cookbook §14c-c lever is empty)
- recommended resequence: T3 -> T4+T5 (built together) -> T6 ; drop T2 (pending Drew's P5d nod)
- struct-core-pivot.md: Phase-17 quantification note; CURRENT_PHASE.md: T1 done + resequence
- all binaries byte-identical (T1 read-only; ov_SC01_077 d19c9580 clean rebuild)
2026-06-19 11:31:41 -06:00
Drew T 9062f0fcc0 docs(phase-16): PIVOT — m2c+permuter won't crack the loose-typed core; new plan + harness fixes
- docs/struct-core-pivot.md: findings + decision + new research directions. Root cause = the
  original engine is LOOSELY TYPED (K&R; same fn called with int/ptr, arg/no-arg across sites),
  so no single canonical signature exists -> m2c guesses inconsistently, permuter can't fix
  semantics, byte-gate (correctly) rejects. Yields ~3%, not the crack. New plan: emulator-recover
  the actor struct/types -> Ghidra global type propagation -> Ghidra-C -> permuter+gate.
- harness bug-fixes (REAL, kept): p16_permute output-0-only match (killed the false '42%'),
  base.c keeps callee externs, winner_to_draft line-strip; sig_unify canonicalizes m2c's
  no-extern prototypes; gen_engine_decls.py (documents why a global canonical header breaks
  loose-typed matches).
- a few byte-gated leaf matches banked in ov_SC01_077.c.
2026-06-19 10:05:58 -06:00
Drew T ed6314f090 feat(phase-16): wire the permuter into auto_driver (the near-miss yield lever)
- driver per-fn flow: m2c -> match_one prefilter -> if near-miss & --permute-secs>0, run
  decomp-permuter (time-boxed, reuses p16_permute setup/run) -> use the score-0 winner as the
  draft -> whole-binary byte-gate. The permuter is closing near-misses well in the overnight
  batch, so it's the driver's core yield step for the unattended run.
- +--permute-j (RAM-bound on the 15GiB box).
2026-06-18 23:41:40 -06:00
Drew T 5dd42ad703 feat(phase-16): permuter batch tester + known-answer improvement-loop workhorse
- tools/p16_permute.py: measure decomp-permuter close-rate on m2c near-misses (nested
  M2C_FIELD->cast expander + permuter setup/run/parse). Overnight batch validated the
  permuter CLOSES near-misses (func_8012A988 in 34s).
- tools/p16_improve.py: known-answer-guided improvement loop (Drew's method) — graduated
  difficulty bands, runs the real pipeline, CATEGORIZES every fail into an actionable class
  (compile:NULL / stack-var / undeclared / conflicting / near-miss), reports re-derivation
  ceiling. Friday's workhorse: fix classes until dry, track the ceiling.
- src/ov_SC01_077.c: +~6 byte-gated leaf matches from the macro+sig_unify+NULL gate.
2026-06-18 23:40:17 -06:00
Drew T c8c9fd20c1 feat(phase-16): S5 driver+supervisor+safe-exit + NULL fix + sig_unify pipeline
- common.h: +NULL (byte-neutral; recovers 54/106 m2c CC1-fails that were just NULL-undeclared)
- tools/auto_driver.py: never-stop worklist loop (m2c -> sig_unify -> byte-gate -> propagate ->
  commit), STOP-sentinel safe-exit at every fn boundary, heartbeat.json
- tools/auto_supervisor.sh (pure-bash babysitter, relaunch-on-crash, reap permuters, MCP-off),
  auto_stop.sh (touch .run/auto/STOP), auto_status.sh (remote check-in)
- p16_known_answer.py: +--gate (whole-binary capability test)
- YIELD REALITY (honest, P9): macro+sig_unify whole-binary = 4/16 (25%) on KNOWN-matchable,
  ~1% on the unmatched hard tail. The permuter is the untested differentiator -> overnight test.
- CURRENT_PHASE: full night's findings + Drew's never-stop/safe-exit/graduated-ladder refinements
2026-06-18 23:15:31 -06:00
Drew T 381b0652a1 feat(phase-16): pipeline proven — known-answer ladder 67% m2c-direct re-derivation
- tools/p16_known_answer.py: graduated known-answer validation (Drew's method) —
  stub already-matched fns -> re-extract .s -> run pipeline -> measure re-derivation,
  restore safely (never git-checkout mid-harvest, §14c). Auto-picks a difficulty ladder.
- RESULT: 8/12 known-answer fns re-derived DIRECTLY by m2c+macros (no permuter, no struct
  types). Remainder = permuter/struct/typing candidates. Overlay byte-restored (d19c9580).
- Infra proven end-to-end: m2c --valid-syntax + common.h macros -> compile -> byte-gate.
- CURRENT_PHASE: Drew refinements (graduated ladder, never-stop run, safe-exit sentinel).
2026-06-18 22:53:19 -06:00
Drew T a10f7f8a1a feat(phase-15): sig-unify recovery + find_site brace-fix — fleet 52.08% -> 54.44%
- tools/sig_unify.py (NEW): unifies a draft's FULL signature set to the banked-canonical
  decls — both callee externs AND the draft's OWN definition signature (return + param
  types, body param-names preserved). Recovery for the standalone-MATCH residual: a probe
  found 30/30 sampled failures were type conflicts, 0 false-positives. Gated 191 -> 32 verified.
- tools/dedup_propagate.py find_site FIX: the inline-def detector required the opening brace
  ON THE SAME LINE as the signature, silently dropping every next-line-brace def from
  propagation (sig_unify/permuter outputs + ~stragglers all session). Now accepts brace on
  the same OR next line -> unlocked a 61-function propagation backlog.
- propagated 61 fns fleet-wide (incl. 32 sig-unify + the permuter win func_801508F8 +
  earlier-dropped next-line-brace matches), all 134 byte-identical; dedup-check 1394/0
- R22 CLEAN fleet rebuild: 136/136 byte-identical. REAL 177501 -> 185646. Zero agent tokens.
2026-06-18 20:00:50 -06:00
Drew T 4b0f4d1003 feat(phase-15): #3 types-header propagation — +8 struct-using shared fns; fleet 51.77% -> 52.08%
- fix tools/build_engine_types.py to be ADDITIVE: merge new source types with the
  types already in engine_types.h (a re-run after a prior --strip was destructive —
  it dropped the ~34 already-migrated types and broke the build; caught + reverted)
- engine_types.h 34 -> 42 named types (the 8 from wave-1's struct-using matches);
  inline defs stripped from ov_SC01_077.c (byte-neutral)
- dedup_propagate --auto-from: 8 of 15 struct-blocked matches now self-contained ->
  propagated across all 134 overlays (byte-gated); 7 remain (anonymous/typedef'd types)
- R22 CLEAN fleet rebuild: 136/136 byte-identical; dedup-check 1333/0
- zero agent tokens (deterministic)
2026-06-18 18:03:44 -06:00
Drew T 5c5e9fb973 feat(phase-15): T6 struct follow-up — shared engine_types.h unblocks 37 struct-using fns; fleet 46.27% -> 47.70%
- tools/build_engine_types.py -> src/shared/engine_types.h: extract the 34 named struct/union
  types from ov_SC01_077.c (forward decls + defs in source order; consumes trailing
  __attribute__((packed)) + aliases through the ';'). 0 same-name-different-layout collisions.
- engine_core.h includes engine_types.h; the 34 inline defs stripped from ov_SC01_077.c
  (byte-neutral — type defs emit no code; verified byte-identical).
- dedup_propagate.py: relax the struct guard to skip only inline named-struct/typedef
  DEFINITIONS (header-type USAGES + anonymous local structs now propagate); add engine_types.h
  to the compile pre-filter so struct-using bodies resolve.
- 37 struct-using shared fns propagated fleet-wide (skip 41 -> 4 overlay-local typedef'd types),
  134 overlays byte-identical; dedup 1183 -> 1220 groups, 1220 validated / 0 failed.
- fleet REAL substantive 157,541 -> 162,462; byte-identical 46.27% -> 47.70%.
- R22 clean rebuild: 136/136 byte-identical, 0 extract failures.
2026-06-17 15:25:44 -06:00
Drew T bdfb86dc4f feat(phase-15): T6 v2 — data-sig-aware harvest + struct-guarded propagate; fleet 31.22% -> 42.14%
- gen_harvest_targets.py: now also resolves canonical DATA-symbol declarations per target
  (scan the .s for D_XXXX refs -> `extern <type> D_XXXX;` from banked code) in addition to
  callee function sigs — the §14c(c) fix at the source, so agents declare both functions and
  data correctly and avoid the dominant remaining conflict class.
- v2 harvest (callee+data-sig-aware) on the 591 small still-stub call-heavy fns + a gap-fill
  agent for 6 targets dropped by a connection-closed-mid-response failure (the retry-wave
  misses truncated-non-null results). Gate: 306 verified (ov_SC01_077 1554 -> 1248 stubs).
- dedup_propagate.py: skip any struct/union-touching body — agents named structs with colliding
  generic names (`struct S`/`struct vec`) inline, so two macros' types redefine/conflict when
  instantiated in one overlay (ov_SC01_000 abort). 34 skip; struct-using shared fns stay banked
  in 077 (shared-types-header follow-up).
- 281 functions propagated fleet-wide (134 overlays each, every overlay byte-gated); dedup
  795 -> 1076 groups, dedup_integrate --check 1076 validated / 0 failed.
- fleet REAL substantive 105,764 -> 143,322; byte-identical 31.22% -> 42.14%.
- R22 clean rebuild: 136/136 byte-identical, 0 extract failures.
- cookbook §12: gap-fill is now a MANDATORY post-step for every multi-agent run (reconcile
  produced-vs-expected: missing + truncated drafts) before gating.
2026-06-17 11:59:33 -06:00
Drew T 1d94f9ec96 feat(phase-15): T6d — deterministic decl-canonicalization recovers 19 more shared fns; fleet -> 31.22%
- tools/canon_draft_decls.py: rewrites a quarantined harvest draft's extern/data
  declarations to the banked-consistent set (functions from engine_core.h DEFINE
  bodies / extern decls / inline defs; data from existing `extern <type> D_XXXX;`),
  so a draft whose BODY is byte-correct (passes match_one) but whose DECLARATION
  collided with already-banked code can re-gate cleanly. The whole-binary
  harvest_verify byte-gate stays the sole arbiter — a semantics-changing
  canonicalization just fails and reverts.
- recovered 19 of the 90 match_one-MATCH-but-conflicting quarantined failures
  (no API — immune to the rate limiting), propagated fleet-wide, all 134 overlays
  byte-identical; dedup 776 -> 795 groups, 0 failed.
- fleet byte-identical 30.48% -> 31.22%; ov_SC01_077 1574 -> 1535 stubs.
- R22 clean rebuild: 136/136 byte-identical, 0 extract failures.
- the remaining ~70 match_one-MATCH residual needs definition-widening (§14c(d),
  narrow-return) or true data-type resolution — queued for the next pass, with the
  un-started medium band (430) and the 54 genuine instruction-misses (permuter/§3a).
2026-06-17 00:30:45 -06:00
Drew T ef1cf9f05e feat(phase-15): T6 pass-1 — callee-sig-aware harvest of the call-heavy shared core; fleet 25.38% -> 30.48%
- tools/gen_harvest_targets.py: callee-signature-aware target manifest (the §14b
  extern-conflict wall-breaker). Pre-resolves each callee's EXACT signature from
  engine_core.h DEFINE bodies + extern declarations + inline defs, so drafts never
  emit a conflicting extern. Lifted call-heavy gate yield from the §14b-naive 16%
  to 60-67%.
- ov_SC01_077: 142 functions banked (1716 -> 1574 INCLUDE_ASM stubs) via the
  Ultracode parallel-draft + harvest_verify byte-gate loop (cookbook §12). The
  draft Workflow was throttled by server-side API rate limiting (stopped at
  ~246/616); the incremental gate -> dedup_propagate loop is deterministic and
  throttle-immune, so the wins banked steadily regardless.
- dedup_propagate --auto-from ov_SC01_077: ~132 functions propagated fleet-wide
  (134 overlays each), every round per-overlay byte-gated; dedup 644 -> 776 groups,
  dedup_integrate --check 776 validated / 0 failed.
- fleet REAL 85,651 -> 103,217; byte-identical 25.38% -> 30.48%.
- R22 clean rebuild: make clean + extract all 136 + check-all = 136/136 byte-
  identical, 0 extract failures.
- cookbook §14c: the wall-breaker + the 4 conflict flavors (defined / declared /
  DATA-symbol / narrow-return-widening) + the never-git-checkout-mid-harvest
  hazard + the rate-limit-resilient incremental loop. Residual = the data-symbol
  conflict class (structural canonical-decls fix queued for next pass) + the
  un-started medium band (430).
2026-06-17 00:12:23 -06:00
Drew T aefe2eac18 feat(phase-15): T5a — bulk-propagate 553 shared functions fleet-wide (3.82% -> 22.14%)
- dedup_propagate --auto-from ov_SC01_077: propagated 553 already-matched, self-contained
  shared engine functions across all 134 overlays, each byte-gated. 9 functions skipped
  (use 077-local struct types -> not mechanically liftable; honest, P9). One R22 CLEAN
  rebuild: 136/136 byte-identical.
- fleet REAL 947 -> 74,527; byte-identical 13,132 -> 76,174 / 344,010 = 22.14%; 561 dedup
  groups, dedup-check 561 validated / 0 failed
- tool hardening for fleet scale: registry SHORTHAND (vram + binaries; dedup.us.yaml ~4k
  lines vs ~77k verbose); per-overlay apply (one read/write per file); gate-only-changed;
  skip already-registered (additive/resumable); compile pre-filter (body must build with
  common.h alone); group_members() reader (both forms) in dedup_integrate + progress;
  progress _DEDUP_CACHE (fleet 6m -> 7s)
- the first bulk attempt fail-closed-reverted on a local-typed body (byte-gate working) ->
  added the compile filter; cookbook 14a documents the 5 fleet-bulk lessons
- ghidra/ churn NOT staged (R23)
2026-06-16 20:35:39 -06:00
Drew T c716eb15e1 feat(phase-15): T4 — fleet progress roll-up + fix cross-report double-count
- progress.py: refactor to report(binary) + set_binary() + a per-binary .s index
  (fleet run 6m38s -> 7s); add --fleet -> docs/progress.fleet.md (deterministic,
  source-derived per-binary table + fleet totals). Single-binary output byte-unchanged.
- fleet now: 947 REAL / 959 LINKED / 13132 byte-identical / 344010 matchable = 3.82%
  across 136 binaries (main + resident + 134 overlays)
- dup_report --cross: HONESTY FIX (R14) — an onboarded overlay is both a named BINARIES
  entry AND a .run/sig.ov_* glob hit; it was counted twice, inflating collapsible bytes
  to 58.7M once the fleet was onboarded. Dedupe by alias -> accurate 9366 h_exact groups
  / 28.6M collapsible (matches the Phase-11 figure)
- Makefile: wire progress.py --fleet into make report (main-only block)
- docs: progress.fleet.md (new), duplicates.cross.md regenerated, SETUP inventory
- make report green; dedup-check 8 validated / 0 failed
- ghidra/ churn NOT staged (R23)
2026-06-16 20:00:19 -06:00
Drew T 87136a024e feat(phase-15): T3 — onboard the remaining 118 overlays; full fleet 136/136 byte-identical
- onboarded all 118 not-yet-onboarded overlays via tools/new_overlay.sh (0 failed)
  -> fleet = 134 location overlays + main + resident = 136 binaries
- R22 CLEAN rebuild (make clean -> extract all 136 -> make check-all):
  136 passed, 0 failed of 136 -- every overlay byte-identical at 100% INCLUDE_ASM
  from a fully clean tree, 0 extract failures
- per-overlay config (splat.ov_*.yaml / check.ov_*.sha / symbols.ov_*.txt) +
  config/overlays.mk var blocks + the 4 report/diff BINARIES dicts auto-registered
  (sentinel-anchored, idempotent); src/ov_*/ stub records committed (harvest_verify
  reverts via git checkout, so the records must be tracked before any harvest)
- ghidra/ churn NOT staged (R23); build/asm/assets/.run gitignored
- report-digest regen (fleet-wide duplicates.cross.md + progress.fleet.md) is T4
2026-06-16 19:41:37 -06:00
Drew T 0c7619231d feat(phase-15): T2 — dedup_propagate.py (match-once -> propagate-many) proven on wave
- tools/dedup_propagate.py: lift a matched body -> DEFINE_func_<ADDR>() macro in
  src/shared/engine_core.h -> instantiate in place at every onboarded overlay sharing
  that h_exact -> snapshot + per-overlay byte-gate (fail-closed revert) -> register in
  config/dedup.us.yaml. Keyed by addr-int (sig lowercase vs splat func_%08X).
- proof: 4 medium funcs propagated across all 16 onboarded overlays; clean check-all
  18/18 byte-identical; dedup_integrate 8 validated/0 failed; idempotent; negative test
  (corrupt shared body) -> make check FAILS then restores
- Makefile: header-dependency tracking (cpp -MMD -MP + -include ) so editing a
  shared header recompiles dependents — closes a stale-build/false-pass gap the negative
  test exposed; output-neutral (R22 clean 18/18)
- structural self-check: a leftover INCLUDE_ASM stub is byte-identical, so the byte-gate
  can't catch under-application -> assert DEFINE present + stub gone
- FINDING: 577 of ov_SC01_077's 785 matches are h_exact across all 134 overlays
  (2.19 MB collapsible, already matched) -> T5 = bulk-propagate these + harvest the rest
- cookbook §14 (the propagation runbook + gotchas), SETUP tool inventory
- ghidra/ churn NOT staged (R23); build/asm/.run gitignored
2026-06-16 19:24:08 -06:00
Drew T e5a2bd4083 feat(phase-15): T1 — onboard diverse wave-1 (12 overlays); fleet 18/18 byte-identical
- onboarded ov_{SC01_000,SC01_001,SC02_000,SC02_003,SC03_002,SC03_006,
  SC04_000,SC04_018,SC04_019,SC05_000,SC06_000,SC07_000} via tools/new_overlay.sh
  — spans all 7 SC areas + 2 full duplicate pairs (SC02_000≡003, SC04_018≡019)
- each byte-identical at 100% INCLUDE_ASM (non-4-aligned auto-handled); no
  position-lock anomalies — the cross-shape risk is retired before the full onboard
- make check-all = 18/18 passed (main + resident + 4 prior + 12 new)
- config/overlays.mk + 4 report dicts auto-registered (sentinel, idempotent)
- ghidra/ DB churn NOT staged (R23); asm/build/assets/.run gitignored
2026-06-16 19:02:14 -06:00
Drew T 67592993ea feat(phase-13): T3+T4 — new_overlay.sh + dup-pair & free-roam onboarded byte-identical
- tools/new_overlay.sh: one-command overlay onboarding (template instantiate +
  @TRAILING@ carve + overlays.mk + 4-dict sentinel-insert + extract/build/check);
  idempotent (re-run = no-op, no dupe entries)
- T3: ov_SC03_001 (free-roam, 815,887 B) byte-identical f8fd92f5
- T2 (onboard): ov_SC01_005 + ov_SC01_006 (byte-identical pair) both 56760dbe
- template tokens unquoted (correct YAML int types); SETUP §2.8 MCP-reconnect note (R21)
- 6 binaries now build byte-identical (main, resident, 4 overlays)
2026-06-16 16:44:01 -06:00
Drew T 144839f24b feat(phase-13): T1a/T1b — SC01/077 onboarded + all-asm byte-match (d19c9580)
- ov_SC01_077 scaffolded: config/overlays.mk (OVERLAY_BINARIES + var block),
  config/splat.ov_SC01_077.yaml (from template), check.sha, empty symbols, +
  ov_SC01_077 entry & sentinel anchor in the 4 Python BINARIES dicts
- GATE T1b: make check BINARY=ov_SC01_077 -> d19c9580 BYTE-IDENTICAL @ 100%
  INCLUDE_ASM; clean rebuild leaves main 143dbb89 + resident 8e17e02f unregressed
- reusable non-word-aligned-overlay handling (~75% of fleet) added to the
  template + Makefile: (1) [word_floor, bin, trailing] carve for the final 1-3
  bytes spimdisasm drops; (2) build/assets/%.o incbin rule (+ .data align=1) for
  splat bin assets (asset_path scoped per-alias); (3) objcopy end-align TRIM
  (shrink-only, <=3 B) removing the .ld's segment-end ALIGN(.,4) pad
- .gitignore /assets/ (regenerable splat output); clean removes assets/
2026-06-16 15:39:52 -06:00
Drew T 4c7cf48ebf feat(phase-12): T3 — +2 (typedef-strip recovery) + per-function match tool (REAL 87->89)
- tools/match_one.py (new): compile ONE function's C standalone (pinned triple), mask
  relocations (jal/HI16/LO16 like psyq_identify), compare to the target bytes in its .s.
  Isolated temp dir -> parallelizable; a real per-function asm-differ loop for agents to
  iterate against without a shared build. Validated: a matched fn -> MATCH, residuals ->
  informative diffs.
- harvest_verify.py + match_one.py: strip inline scalar-typedef redefinitions (some drafts
  inlined `typedef unsigned char u8;` -> C89 redefinition error vs common.h = a COMPILE
  fail, not a byte mismatch). Re-gate recovered func_800CFBCC + func_800D1938.
- REAL 89 (62.76% byte-identical); resident 8e17e02f, main 143dbb89 (R22 clean).
- 54 stubs remain (scheduling/regalloc residuals + big GPU/dispatch fns) -> match_one-driven
  iteration pass next.
2026-06-16 12:43:59 -06:00
Drew T c37332fa19 feat(phase-12): T3 batch 2 — 62 resident functions matched via parallel-draft + byte-gate (REAL 8->70)
Ultracode harvest: a 16-agent workflow (resident-harvest-draft) drafted matching C for
the 135 unmatched resident functions (each agent: asm + cookbook + the 8 worked examples,
writing .run/drafts/<fn>.c, no builds/no Ghidra). Then tools/harvest_verify.py (new) byte-
gated every draft against 8e17e02f one at a time (chunk+bisect), keeping only the byte-
perfect ones and reverting the rest -- the build is the sole arbiter (G3/P9).

- 62 / 127 drafts verified byte-identical in one pass; 65 near/hard misses reverted to stubs.
- REAL 8 -> 70 (48.28%); byte-identical 49.66%. resident 8e17e02f, main 143dbb89 (R22 clean).
- cleaned 35 redundant '#include "common.h"' lines the drafts inlined (still byte-identical).
- remaining: 65 failed drafts (close-misses + the hard dispatch/GPU/big-fn tail) + 8 undrafted
  (2 agent batches hit transient API 500s) -> second-pass workflow + manual hard tail.
2026-06-16 11:59:02 -06:00
Drew T f672c709c1 feat(phase-11): T6 — sign the 134 overlays; cross-report spans 136 binaries (milestone report half)
- sig_image overlay discovery: linear partition (split contiguous code at jr+delay boundaries) +
  detect_code_end (first run of invalid instrs = the code->data transition; overlay code decodes
  ~100% valid, data drops to 43-95%). BFS dead-ended — overlays dispatch via function-pointer
  tables, not jal — so call-graph BFS found ~2 fns; linear partition recovers the whole code prefix
- Makefile: sig-overlays: signed 134 overlays -> .run/sig.ov_*.jsonl signs all 134 SCxx 0.4.dec @ 0x80128158 -> .run/sig.ov_*.jsonl (27s)
- dup_report --cross: ingest the overlay sigs (sig.ov_* convention) + main/resident; condensed
  source header (overlay set-sha for reproducibility); cap top-200 per subsection + state totals;
  compact member sample + #bin column
- RESULT (docs/duplicates.cross.md): h_exact 9366 cross-binary groups / 28.5 MB collapsible; h_norm
  8957 / 38.3 MB. Top group = a 770-instruction function BYTE-IDENTICAL in all 134 overlays (one
  match credits the whole fleet) — 'one match unlocks many' quantified; the Phase-12/13 work queue
- dedup-check green; committed per-binary digests byte-stable; report deterministic (sorted glob,
  no Date/random); the EXE<->resident pair still shares nothing (overlay<->overlay dominates)
2026-06-16 01:26:49 -06:00
Drew T 81e33c7a1e feat(phase-11): T5 — sig_image self-consistent h_norm + h_seq (scope-guard D2)
- norm_stream(): mask address-sensitive fields so structurally-identical functions at different
  addresses normalize equal — j/jal 26-bit targets, lui highs, and hi/lo-paired I-type address-los
  (a consistent lui->reg pending-hi tracker); KEEP registers, true constants, PC-relative branch
  offsets. h_seq = mnemonic sequence. Self-consistent within sig_image (the overlay fleet).
- validated on the resident vs the Ghidra oracle: h_exact equivalence classes 6/6 == Ghidra;
  h_norm reproduces 7/9 Ghidra structural groups — the 2 disagreements are BENIGN (differ only in
  masked address-low / jump targets; Ghidra's own reference analysis is inconsistent there). Any
  h_norm candidate is byte-gated (its sole acceptance, per the roadmap).
- DEVIATION D2: self-consistent h_norm, NOT a Ghidra-byte-exact normToken replica. Byte-reality
  makes cross-tool structural matching low-value (EXE<->resident share nothing; overlays share with
  each other via h_exact at the same vram, and CALL not embed the resident). h_exact is the
  format-independent cross-tool workhorse; full normToken replica is a documented future refinement.
  Scope-guard pre-approved in the plan.
2026-06-16 01:15:07 -06:00
Drew T 685e35fdc6 feat(phase-11): T4 — sig_image.py, Ghidra-free h_exact signer + boundary detection
- tools/sig_image.py: linear rabbitizer disassembler that signs a flat image (overlay 0.4.dec /
  resident 1.1) at a vram base, JSONL field-identical to DumpFunctionSignatures.java. h_exact =
  SHA1 of raw bytes (format-independent — the cross-binary workhorse; overlays share the same vram
  so same-offset dups are byte-identical). Seeded + jal-closure bootstrap entry discovery.
- boundary rule: a function ends at the first 'jr $ra'(+delay) at/after every forward branch/jump
  target — handles early-return jr AND ignores the trailing orphan jr;nop (double-epilogue)
- VALIDATED vs the resident Ghidra oracle: 100% h_exact on the contiguous/non-GTE subset (140/140),
  ZERO UNEXPLAINED (whenever nbytes agrees, h_exact agrees) — the byte pipeline is exact; 98.6%
  overall. The 2 misses are non-contiguous Ghidra bodies (D5, inherent to a linear sweep)
- h_norm/h_seq emitted as conservative placeholders (= h_exact -> zero false structural matches)
  until T5 calibrates the normToken/mnemonic replica; overlays aren't signed until T6 (post-T5)
2026-06-16 01:07:28 -06:00
Drew T a8ae190fa3 feat(phase-11): T3 — intra-binary collapse PROOF (one body, two addresses, byte-identical)
- src/shared/clearTbl40.h: CLEAR_TBL40 macro = the matched byte-clear loop body, authored ONCE;
  instantiated at both func_80037004 and func_80037334 in src/800.c (one source -> two vrams)
- matched first try; clean rebuild main -> 143dbb89 BYTE-IDENTICAL WITH the shared C AND WITHOUT
  it (INCLUDE_ASM stub fallback) -> dual invariant proven (R22 clean rebuilds, both states)
- config/dedup.us.yaml: I0_clearTbl40 group registered (h_exact a0744d60…); dedup_integrate
  --check validates it; negative tests (corrupt hash / wrong vram) fail-closed (exit 1)
- tools/progress.py: count dedup-shared members as REAL via the registry (the macro form isn't a
  parseable function def); REAL 52 -> 54, byte-identical 50.24% -> 50.33%; honest measurement (P9)
- tools/dedup_integrate.py: display vram in hex in diagnostics
- the machinery half of the Phase-11 milestone is proven on the byte-verified EXE
2026-06-16 00:56:35 -06:00
Drew T b06d74c1be feat(phase-11): T2 — dedup.us.yaml registry + dedup_integrate byte-honesty validator
- config/dedup.us.yaml: cross-binary code-share registry (group -> {id,tier,hash,
  source,func,members[{binary,vram,name}]}); empty for now (T3 adds the first group)
- tools/dedup_integrate.py: --check validator. Fail-closed if a shared function's
  current sig hash drifted from the recorded hash (a stale share can never silently
  mislead, P9/G3); unsigned binary -> WARN (unvalidated), not a hard fail
- Makefile: dedup-check is the fail-closed last line of `make report` (gated BINARY=main)
- DEVIATION D1: game-code dedup is SOURCE-LEVEL (shared body in src/shared/<fn>.h,
  instantiated per site), NOT a psyq_integrate object-swap — game funcs are interior to
  one object per binary so the linker can't swap them; the byte-gate is the existing
  per-binary make check. .ld interpose stays the library mechanism (Phase 8)
- verified: --check passes on empty registry; make report runs it; clean rebuild main
  -> 143dbb89 BYTE-IDENTICAL (T2 is a proven build no-op, R22)
2026-06-16 00:47:20 -06:00
Drew T 7cc2911456 feat(phase-11): T1 — cross-binary dup_report --cross → duplicates.cross.md
- dup_report.py: additive --cross mode ingests all BINARIES sigs, tags each row
  with its binary, buckets by h_exact then h_norm across binaries, splits
  cross-binary (binaries>1, the Phase-12/13 work queue) vs intra-binary, ranks by
  collapsible bytes (count-1)*nins*4, one row per member -> docs/duplicates.cross.md
- Makefile: make report emits the cross report once (gated BINARY=main)
- per-binary duplicates.md/.resident.md stay byte-stable (per-binary path unchanged)
- EXE<->resident cross-binary groups = 0 (4.0 vs 4.7 + different roles, as the
  byte-finding predicted); real cross groups arrive with overlays (T6)
- phase-ends/CURRENT_PHASE.md: Phase 11 plan + per-task log
2026-06-16 00:37:18 -06:00
Drew T 03978ec4ba feat(phase-10): T4 — resident Ghidra program (2nd binary) + symbol seed + sig
- tools/ghidra_import_raw.sh (NEW): raw-blob importer (BinaryLoader + --loader-baseAddr +
  PSX:LE:32:default) — the Gen2 counterpart to ghidra_import.sh (PS-X-EXE only); reusable for
  Phase-13 location overlays. Imports the resident blob as program 'resident' @0x800CEDF8.
- tools/ghidra_scripts/DefineFunctions.java (NEW): seed splat's validated entry points
  (.run/<prog>_funcs.txt) — raw-binary auto-analysis finds only the reachable subset (23/143);
  this defines all 143 (created=120/existed=23/failed=0). R9-verified 143 funcs persisted.
- FINDING: DetectPsyQ reports the resident is PsyQ 4.7.0 (the EXE is 4.0.0); the lone in-range
  PsyQ-signature hit is DsMix (libsnd -> resident holds the sound driver). 4.7 .LIBs DEFERRED to
  Phase 11 start (Drew); carried to PhaseEnd Notes. Phase 10 needs nothing from 4.7.
- config/symbols.resident.txt: seed DsMix (R13 candidate, 4.0-sig vs 4.7 blob — confirm in Phase 11);
  stacked under symbols.us.txt, applied on re-extract -> resident still 8e17e02f BYTE-IDENTICAL (R22),
  main still 143dbb89 (no regression)
- Makefile: per-binary GHIDRA_PROG -> 'make sig-refresh BINARY=resident' (.run/sig.resident.jsonl)
- dup_report.resident now real (6 byte-identical intra-resident groups — Phase-11 dedup fodder)
- ghidra DB committed (R23, MCP stopped): new resident program 00000003.* (+ main db.15->16 no-op)
2026-06-15 23:25:00 -06:00
Drew T cc73951128 feat(phase-10): T3 — per-binary reports/diff/expected for the resident binary
- diff_settings.py + progress.py/difficulty.py/dup_report.py: add the `resident` BINARIES entry
  (build/resident/, config/check.resident.sha, src/resident, asm/resident/nonmatchings, per-binary docs)
- dup_report.py: degrade gracefully when the Ghidra sig (.run/sig.resident.jsonl) is absent —
  write a placeholder + exit 0 instead of crashing the whole `make report` (sig is a T4 export)
- progress.py: scope linked_subsegs() to BINARY==main — PsyQ library linking is the EXE's layout
  (Phase 8, gated ifeq BINARY,main), so a second binary has 0 LINKED (was: listed main's libs)
- Makefile expected: per-binary-safe — drop `rm -rf expected/build` (clobbered every sibling
  baseline); refresh only the active binary's image dir + merge-copy (cp never deletes siblings)
- reports: resident = 0 REAL / 143 INCLUDE_ASM stubs / 100% stub (correct); main UNCHANGED
  (52 REAL / 959 LINKED / 50.24%); both expected/ baselines coexist (verified)
2026-06-15 23:06:25 -06:00
Drew T 1b9397c462 refactor(phase-9): T8b — remove transitional defaults across link pipeline (no-EXE-default end state)
Every binary-specific value is now a REQUIRED parameter — the phase's #1-risk
mitigation (no EXE default an overlay could silently inherit):
- psyq_link.py: removed module EXE/VRAM_BASE; recover_sym_addrs/unique_byte_vram
  vram_base now positional-required; link_object vram_base keyword-required (*,...);
  CLI --vram-base/--exe required
- psyq_identify.py: removed EXE/VRAM_BASE globals; --vram-base/--exe required
- psyq_link_region.py + psyq_link_lib.py: dropped 'VRAM_BASE' import + EXE; defaults
  removed (placement/classify/build_region required); CLI required
- psyq_integrate.py: dropped VRAM_BASE import + EXE; integrate() vram_base/exe_path/
  symbols_path keyword-required; CLI --vram-base/--exe/--symbols required
- PROOFS: psyq_link bare -> 'required: --vram-base, --exe' (loud); build with AND
  without SDK -> 143dbb89; wrong base -> cae22f7e; curated dirs regenerate identical
2026-06-15 21:00:36 -06:00
Drew T cb7a280994 refactor(phase-9): T8a — de-default the EXE-curation helpers (thread --exe/--vram-base)
- make_snd_used.py + make_apicard_used.py: argparse --exe/--vram-base (default the
  EXE's); thread into psyq_identify subprocess + link_object (required post-T8b).
  RLO/RHI/EXCLUDE_ADDR kept as the EXE's region data (EXE-specific, documented)
- gen_lib_subsegs.py: --vram-base/--exe (default EXE); threaded into psyq_identify
  and the pos->fileoff arithmetic (module VRAM_BASE constant removed)
- split_src_region.py: --symbols overrides SYMS_PATH (default the EXE's symbols.us.txt)
- checks: snd_used (60) + apicard_used (22) regenerate BYTE-IDENTICAL; gen_lib_subsegs
  libgte -> 22 blocks/53 objs; clean build -> 143dbb89
2026-06-15 20:55:28 -06:00
Drew T 5e0d80a887 refactor(phase-9): T7 — ld_interleave.py --front/--tail; gate extract call to BINARY=main
- ld_interleave.py: argparse [ld] + --front/--tail (repeatable; default to the EXE's
  sandwich objects, transitional); the .data->.rodata->.data island is EXE-specific
- Makefile extract: ld_interleave call wrapped in ifeq(BINARY,main) (overlays have no
  rodata island) and passes --front 53198.data.o --tail 6324C.data.o explicitly
- GATES: positive extract+build -> 143dbb89 (tail_data=1 from the threaded flag);
  negative --tail BOGUS.data.o -> 'tail data object not found' exit=1 (threaded);
  restore -> 143dbb89
2026-06-15 20:49:45 -06:00
Drew T 22b20e8f31 refactor(phase-9): T6 — --binary selector for dup_report + difficulty; report threads BINARY
- dup_report.py + difficulty.py: argparse --binary (default main) + BINARIES table
  (main = current sig/src/asm/md/csv paths); overlay src/asm subtree deferred to P10
- Makefile report target threads --binary $(BINARY) to progress/difficulty/dup_report
- no-op proof: docs/progress.md + docs/duplicates.md UNCHANGED; difficulty.py old-code
  vs new-code output IDENTICAL (pure path-table refactor)
- docs/difficulty.md regenerated (the Phase-8-deferred regen: 9 harvested funcs left
  the unmatched queue — 12/12 row shift); build path untouched (still 143dbb89)
2026-06-15 20:46:48 -06:00
Drew T 06676e6ab1 refactor(phase-9): T5 — psyq_integrate flags + progress regex + 9 Makefile call sites
The build-path linchpin (one commit — integrate signature and the progress.py
Makefile regex must move in lockstep):
- psyq_integrate.py: argparse (--vram-base/--exe/--symbols flags BEFORE the 4
  positionals + nargs=* window); threads vram_base/exe into placement/classify
  and symbols_path into the stub-name->addr resolution (was hardcoded symbols.us.txt)
- Makefile: all 9 psyq_integrate calls pass --vram-base $(main_VRAM_BASE) --exe
  $(main_EXE) --symbols $(main_SYMBOLS) (flags before positionals)
- progress.py: stub-list regex gains (?:\s+--\S+\s+\S+)* to consume the leading
  flags; + a --binary selector (BINARIES table, default main = current paths)
- GATES: positive build -> 143dbb89; report 52/959/7/50.24% (regex still finds the
  9 stub lists); negative main_VRAM_BASE=0x8000F804 -> cae22f7e (NOT target);
  without-SDK fresh-clone fallback -> 143dbb89
2026-06-15 20:42:32 -06:00
Drew T 0a59e96ce4 refactor(phase-9): T4 — thread vram_base/exe through psyq_link_region + psyq_link_lib
- build_region/placement/classify (+ psyq_link_lib) take vram_base/exe, default to
  the kept globals (transitional, removed T8); pass --vram-base/--exe to the
  psyq_identify subprocess argv; argparse mains (window nargs=*)
- in-process callers from psyq_integrate (classify/placement) stay green via the
  defaults until T5 passes them explicitly; VRAM_BASE import kept as the default
  source (dropped at T8 with the rest)
- checks: link_lib libcd 18/18 byte-identical; wrong --vram-base 0x8000F900 -> 6/18
  (threaded); link_region libcd per-object byte-identical True; build -> 143dbb89
2026-06-15 20:36:32 -06:00