Commit Graph

12 Commits

Author SHA1 Message Date
Drew T 876dc7f053 feat(phase-28 T5): resident 21 -> 14 stubs (7 banked, 90.34%) + fix match_one's fake isolation
Ultracode wave: 16 isolated drafters over the resident's non-jtbl stubs (the 5 jtbl deferred —
they need the rodata-island carve, §53). Drafts only; the whole-binary byte-gate arbitrated after.

- BANKED 7/16, byte-gated: func_800CEFD0(77) func_800D0D7C(45) func_800D1B80(22) func_800D1E28(37)
  func_800D1FC8(62) func_800D29F8(172) func_800D2D10(39).
  Resident REAL 122 -> 129, stubs 21 -> 14, byte-ident 124/145 (85.52%) -> 131/145 (90.34%).
  FLEET instr 9017152 -> 9017606 (+454 ins). R22 make clean && extract-all && check-all ->
  140 passed, 0 failed of 140 (the first R22 was killed by a terminal crash and RE-RUN, not assumed).
  Ground truth on 14 agrees 3 ways: source grep, splat-emitted stub .s count, progress.py.

- §52b's LAW, MEASURED AGAIN INDEPENDENTLY: the agents self-reported 11 match_one MATCH; the
  whole-binary gate banked 7 (64%). All 4 blocked MATCHes died on `conflicting types`
  (D_8010EDEC / D_80115110 / func_800D1984 / cdFileLocTable) — the loose-typing def-side wall, NOT
  codegen. gate_stage's recovery banked 0/5 on them. A match_one MATCH is a CANDIDATE (G3/P9).

- FIX — match_one's isolation was FAKE, and its own docstring was the false spec. It promises
  "Fully isolated (own temp dir) so many run in PARALLEL with no shared build -- a real asm-differ
  loop for an agent to iterate against", while `--work` defaulted to the SHARED '.run/match': every
  concurrent caller compiled into the same t.c/t.o. FOUND BY AN AGENT MID-WAVE, the only way it can
  be found — it read another agent's function out of its own scratch ("found another agent's
  func_800D2650 in my t.c") and reported it. Every other agent steered by a loop that could hand it
  someone else's compile: a CONFIDENT WRONG verdict, worse than a crash. Default is now a private
  .run/match/<fn>.<pid>; the default IS the promise. (Some agents had already worked around it by
  passing --work themselves.) The byte-gate was never at risk — it is the sole arbiter — but the
  iteration loop the agents steer by absolutely was.

- The 14 remaining: 5 PLUMBING (loose-typing) + 4 DIFF (genuine codegen: func_800D2650 close=4,
  func_800CFAD0 close=5, func_800D0E30 close=12, func_800D27DC close=48) + 5 jtbl deferred.
  Dossier next (T5b) — the agents' per-function residual analyses are the durable asset (R30).
2026-07-16 01:41:57 -06:00
Drew T 2c4e2344da fix(phase-28 T5): progress.py — #if 0 blindness + the len()-sum; resident 123/146 -> 122/145
R35: fix the instrument before planting a flag on its denominator.

- #if 0 BLINDNESS: classify() knew `#ifdef NON_MATCHING` (:425) but not `#if 0`, so a dead
  analysis body was read as a live definition AND its real INCLUDE_ASM stub counted separately —
  the SAME function in BOTH `real` and `stubs`. Live case: resident.c:868-925 wraps a full
  void func_800D00E4(s32){...} in #if 0 (its jtbl dossier) and re-declares the stub at :926.
  Now the block is skipped entirely: dead code is neither matched nor stubbed.

- THE len()-SUM (the dual defect): `placed` was a set union, so it caught a function in NO bucket
  — but `matchable` SUMMED len()s, so a function in TWO buckets counted twice and nothing
  complained. matchable/byteident are now SET unions, plus a new OVER-coverage assertion that
  fails loudly if any fn lands in multiple buckets. R32 means both directions: nothing missing,
  nothing double-counted.

- NEGATIVE CONTROL (the fix must change an answer the old tool gave):
    resident REAL 123 -> 122 | matchable 146 -> 145 | 85.62% -> 85.52% | func_800D00E4 no longer
    double-counted. FLEET instr 68.9% UNCHANGED (no #if 0 in the overlays) — the fix is scoped.

- FINDING (logged for T7's audit-binaries, does NOT block the flag-plant): the two INDEPENDENT
  oracles now agree exactly at 144 — corpus (21 stubs + 123 matched, derived from the tree) and
  sig_image (the 2nd oracle) — with EMPTY set difference both ways. progress.py still reports 145
  because it counts func_800CEDFC and func_800D33E0, which are DEFINED in resident.c but absent
  from sig_image. 0x800CEDFC is the resident's vram base +4 (the first function, code starts at
  file offset 0x4 after the leading data word), yet make audit-corpus reports 0 PHANTOM +
  0 TRUNCATED. Either sig_image has a boundary blind spot or those defs are not image functions.
  progress.py's text-scanning classify() is exactly the re-parsing R33 says should be DERIVED from
  corpus instead — a real refactor, logged not rushed.
  The flag-plant claim is unaffected: it rests on corpus.stubs('resident') == 21 (tree-derived,
  verified 5 ways), not on the contested denominator.
2026-07-16 00:17:57 -06:00
Drew T 52514b5f45 docs(phase-13): T6 docs — cookbook §13 overlay runbook + SETUP/gen2-roadmap + digests
- cookbook §13: the canonical add-an-overlay runbook (new_overlay.sh, the flat-blob
  config, the non-4-aligned bin-carve/asset/trim gotcha, A->B->C workflow, dedup-credit
  two shapes, build-all/check-all) — the Phase-15 fleet recipe
- SETUP tooling inventory: tools/new_overlay.sh (R21)
- gen2-roadmap Phase-13 milestone: ACHIEVED amendment (6 binaries byte-identical,
  704 harvested, dup-pair collapse, T7 effort finding)
- regenerated per-overlay progress/difficulty/duplicates digests + cross report
2026-06-16 17:56:38 -06:00
Drew T a480d83bee feat(phase-12): T3a — extern-conflict recovery, +20 (REAL 102->123, 85.6%)
The pass-4 functions that match_one matched standalone but the single-TU build rejected
on conflicting shared-symbol extern types are recovered by UNIFYING those types in
resident.c (widening definitions' return types where byte-identical — u16 f(){return
u16g} ≡ s32 f(){...} both lhu;jr; choosing asm-correct widths for globals; per-site
*(u16*)&g tricks where a neighbor pins the type). ~20 symbols reconciled
(func_800291B4/CFD68/D1714/D1078/801465C0/... + D_800AE6A8/80078E50/...).

20 of 21 recovered; func_800D00E4 stays an honest INCLUDE_ASM stub (its switch needs the
external jtbl_80113ED8 relocation, not a C-source fix — analysis preserved under #if 0).
REAL 102 -> 123 (85.62% byte-identical); resident 8e17e02f, main 143dbb89 (R22 clean).
22 stubs remain: ~20 documented compiler-internal residuals (permuter/§3a) + D00E4 + dedup.
2026-06-16 14:05:20 -06:00
Drew T 393d34604b feat(phase-12): T3 pass 4 — match_one iteration loop, +13 (REAL 89->102, 71.7%)
resident-harvest-iterate workflow: 13 agents iterated the 54 hard functions using
tools/match_one.py (per-function compile+diff loop), self-correcting against real
feedback. Agents reported 34 match_one-MATCHes; the whole-build byte-gate confirmed 13
(func_800D23D0 D1754 CEF5C CF584 D17B8 D2E6C D0588 D2CA8 CFC5C CF3B8 CF104 D20C0 D1BF8).
The ~21 standalone-MATCH-but-gate-FAIL are TU-level extern-type conflicts (a draft
declares a shared symbol's type to suit its own call site, conflicting with another
function's declaration) -> recoverable via a unified extern header (next).
Expert idioms found: struct-copy prologue, void-dispatcher-is-actually-s32-returning,
*10 -> 2x+8x decomposition, polarity flips. REAL 102 (71.72% byte-identical); both
binaries clean. 41 stubs remain (extern-conflict casualties + documented cross-jump/
regalloc residuals = decomp-permuter / cookbook §3a territory, honest P9 deferrals).
2026-06-16 13:15:50 -06:00
Drew T 4c7cf48ebf feat(phase-12): T3 — +2 (typedef-strip recovery) + per-function match tool (REAL 87->89)
- tools/match_one.py (new): compile ONE function's C standalone (pinned triple), mask
  relocations (jal/HI16/LO16 like psyq_identify), compare to the target bytes in its .s.
  Isolated temp dir -> parallelizable; a real per-function asm-differ loop for agents to
  iterate against without a shared build. Validated: a matched fn -> MATCH, residuals ->
  informative diffs.
- harvest_verify.py + match_one.py: strip inline scalar-typedef redefinitions (some drafts
  inlined `typedef unsigned char u8;` -> C89 redefinition error vs common.h = a COMPILE
  fail, not a byte mismatch). Re-gate recovered func_800CFBCC + func_800D1938.
- REAL 89 (62.76% byte-identical); resident 8e17e02f, main 143dbb89 (R22 clean).
- 54 stubs remain (scheduling/regalloc residuals + big GPU/dispatch fns) -> match_one-driven
  iteration pass next.
2026-06-16 12:43:59 -06:00
Drew T 8dbaa489f7 feat(phase-12): T3 batch 3b — +9 via retry-resilient redraft pass (REAL 78->87, 61.4%)
Pass 3 (resident-harvest-redraft3): retry-wave workflow re-drafted all 65 remaining
stubs seeded by BOTH prior failed drafts + a sharpened debugging checklist (callee
return-type-forces-andi, signed-vs-unsigned sltu, store order, QImode decrement, the
register-pointer far-load idiom). All 13 batches completed (retry waves defeated the
transient server rate-limiting). tools/harvest_verify.py byte-gated -> +9:
func_800D2F8C func_800D1F90 func_800D2DFC func_800D222C func_800D10C8 func_800CFBF8
func_800D3120 func_800D1B10 func_800D18DC.

Resident harvest this session: REAL 1 -> 87 (61.38% byte-identical), all hand-matched
engine C, via 3 parallel-draft workflow passes (79 fns) + a deterministic byte-gate that
makes a wrong match impossible (G3/P9). resident 8e17e02f, main 143dbb89 (R22 clean).
56 stubs remain (scheduling/regalloc residuals + the big GPU/dispatch fns) -> per-function
asm-differ/permuter loop + serial Ghidra next.
2026-06-16 12:35:21 -06:00
Drew T 69227e4f6a feat(phase-12): T3 batch 3 — +8 resident fns via redraft pass + byte-gate (REAL 70->78)
Second-pass workflow (resident-harvest-redraft): agents re-drafted the failed/undrafted
functions seeded by their prior draft + a debugging checklist (load width/sign, branch
polarity, arg count, store order). Transient server rate-limiting killed 10/14 agents, so
only ~25 re-drafts landed; tools/harvest_verify.py byte-gated them -> 8 more verified:
func_800D33B8 func_800D2EE8 func_800D0C74 func_800D0CA0 func_800D2DAC func_800CF5D4
func_800D2FEC func_800CFDE8. The recovered bugs: twin-fn u16-return (D2EE8 mirrors the
matched D319C), missing call arg (CF5D4), proven index idiom (CFDE8).
REAL 70 -> 78 (53.79%); resident 8e17e02f, main 143dbb89 (R22 clean).
Remaining: ~65 stubs (17 redraft-resisters + ~48 never-redrafted [rate-limited]) -> pass 3
+ serial asm-differ on the hard GPU/dispatch tail.
2026-06-16 12:15:27 -06:00
Drew T c37332fa19 feat(phase-12): T3 batch 2 — 62 resident functions matched via parallel-draft + byte-gate (REAL 8->70)
Ultracode harvest: a 16-agent workflow (resident-harvest-draft) drafted matching C for
the 135 unmatched resident functions (each agent: asm + cookbook + the 8 worked examples,
writing .run/drafts/<fn>.c, no builds/no Ghidra). Then tools/harvest_verify.py (new) byte-
gated every draft against 8e17e02f one at a time (chunk+bisect), keeping only the byte-
perfect ones and reverting the rest -- the build is the sole arbiter (G3/P9).

- 62 / 127 drafts verified byte-identical in one pass; 65 near/hard misses reverted to stubs.
- REAL 8 -> 70 (48.28%); byte-identical 49.66%. resident 8e17e02f, main 143dbb89 (R22 clean).
- cleaned 35 redundant '#include "common.h"' lines the drafts inlined (still byte-identical).
- remaining: 65 failed drafts (close-misses + the hard dispatch/GPU/big-fn tail) + 8 undrafted
  (2 agent batches hit transient API 500s) -> second-pass workflow + manual hard tail.
2026-06-16 11:59:02 -06:00
Drew T 6d1b0c051e feat(phase-12): T3 batch 1 — 6 trivial resident accessors matched (REAL 2->8)
Resident engine harvest underway (triple confirmed in T2). Batch 1, all byte-identical:
- func_800CF804 / func_800CF818 — set D_80127084 / D_80114E78 = 1 (s32)
- func_800CF82C / func_800CF840 — set D_800B9A0E = 1 / 2 (s16, sh)
- func_800D1714 — get D_80078E50 (u16; player-state region ~0x80078Exx)
- func_800D1724 — set D_800AE6B0 = arg (s32)

make report BINARY=resident: REAL 2 -> 8 (5.52%); byte-identical 10/145 (6.90%).
R22 clean rebuild green: resident 8e17e02f, main 143dbb89.
2026-06-16 11:17:54 -06:00
Drew T b8457a8636 feat(phase-12): T2 — resident compiler triple CONFIRMED = the EXE's (2 byte-exact matches)
- opt level: resident is -O2 (0x21f0a003 frame-ptr sig count = 0, vs the EXE's 54;
  the EXE's 800 game module is also -O2). No per-file override needed.
- triple (Makefile default, no override): gcc-2.7.2-psx -O2 -G0 -mips1 -mcpu=3000
  -mgas -msoft-float -fgnu-linker + maspsx --aspsx-version=2.56 --expand-div.
- 2 byte-exact resident matches confirm it across idiom classes:
  * func_800CF854 — lbu D_800AE6BF; sltu $zero (unsigned-!=0 accessor)
  * func_800CEE84 — 8-call init: branch-store (D_80127084=0; if(D_800B9A00) =1),
    delay-slot-filled func_80010AE0(0x3E0), early-scheduled load
- R22 clean rebuild: resident 8e17e02f, main 143dbb89 — both byte-identical.
  make report BINARY=resident: REAL 0 -> 2.
- R24: the "resident is 4.7" DetectPsyQ hit was a coincidental DsMix signature; the
  resident's actual game-code triple == the EXE's (byte-confirmed). No candidate
  ladder; the STOP-condition did not fire. Harvest (T3) de-risked.
2026-06-16 11:14:25 -06:00
Drew T cc73951128 feat(phase-10): T3 — per-binary reports/diff/expected for the resident binary
- diff_settings.py + progress.py/difficulty.py/dup_report.py: add the `resident` BINARIES entry
  (build/resident/, config/check.resident.sha, src/resident, asm/resident/nonmatchings, per-binary docs)
- dup_report.py: degrade gracefully when the Ghidra sig (.run/sig.resident.jsonl) is absent —
  write a placeholder + exit 0 instead of crashing the whole `make report` (sig is a T4 export)
- progress.py: scope linked_subsegs() to BINARY==main — PsyQ library linking is the EXE's layout
  (Phase 8, gated ifeq BINARY,main), so a second binary has 0 LINKED (was: listed main's libs)
- Makefile expected: per-binary-safe — drop `rm -rf expected/build` (clobbered every sibling
  baseline); refresh only the active binary's image dir + merge-copy (cp never deletes siblings)
- reports: resident = 0 REAL / 143 INCLUDE_ASM stubs / 100% stub (correct); main UNCHANGED
  (52 REAL / 959 LINKED / 50.24%); both expected/ baselines coexist (verified)
2026-06-15 23:06:25 -06:00