Commit Graph

778 Commits

Author SHA1 Message Date
Drew T 97d9eb43e8 feat(decomp): grinder gate — +3 fns x0 propagated (fleet None%) 2026-07-21 13:31:45 -06:00
Drew T ca7369b0f9 feat(decomp): grinder gate — +3 fns x0 propagated (fleet None%) 2026-07-21 13:31:11 -06:00
Drew T 1feba77a92 feat(decomp): grinder gate — +2 fns x0 propagated (fleet None%) 2026-07-21 13:30:38 -06:00
Drew T 3b7399ea88 feat(decomp): grinder gate — +1 fns x0 propagated (fleet None%) 2026-07-21 13:30:04 -06:00
Drew T dfb17fc495 feat(decomp): grinder gate — +2 fns x0 propagated (fleet None%) 2026-07-21 13:29:30 -06:00
Drew T 4fb4d96d54 docs(phase-29): SESSION-6 CHECKPOINT — Task 13A/13B done; the permuter's problem was targeting 2026-07-21 11:25:19 -06:00
Drew T b8ee9fa9d2 feat(phase-29 Task-13B): directed permuter wins first try; 3 latent gate defects fixed at root
First run of the Task-13A-targeted grinder (--once --batch 8 --permute-secs 90):
  targeting: ON — 1665 classified; only bucket=permuter is admitted
  targeting skipped 1601 non-permuter candidates
    {redraft 707, structural 583, integration 305, unclassified 4, unknown 2}
  permuter WON func_80181F78 @ ov_SC03_014 (close was 1)   [~6 min]

BANKED 2 (both whole-binary byte-gated, R22 clean-fleet 140/140):
- func_80181F78 (8 ins)  — classified DELAY-SLOT / schedule
- func_80141B90 (29 ins) — classified IMM-VALUE / cse
Both were classified bucket=permuter by residual_class BEFORE the run: the classifier
predicted a search-closer could reach them, and one did. First end-to-end validation of
the targeting thesis.

THREE latent defects, all pre-existing, all unreachable while the grinder banked nothing
since Phase 21 — the fix made it win and every one fired at once (cookbook §60a):

1. gate_stage commit path crashed on src=None. `src` is DELIBERATELY never defaulted
   (Phase 26-A: a default silently PINS the gate to the main .c), but the commit did
   `git add src …` unconditionally -> every caller that omits it (grinder, orchestrator,
   idiom_hunt) crashes THE MOMENT IT BANKS. Now `git add -u src/`, which also retires the
   `src/ov_*/*.c` filename glob that once omitted 4 R22-verified banks from a commit.
2. _xform ladder dirs (-cn/-cast/-rc/-uni) ACCUMULATE across runs: they held 34 stale
   drafts, so the gate processed 34 when the grinder submitted 1, and banked one function
   it was never asked to try — which would have been committed under a message naming a
   different one. Nothing wrong entered the tree (G3/P9 held; the gate banks only
   byte-identical output) but report and work had diverged. Now cleared per run.
3. grinder called gate_stage with the default propagate=True = `dedup_propagate
   --auto-from`, the §55b fleet-wide path that timed out at 3600s and left 90/140 overlays
   broken — and being INSIDE the gate it takes the banks down with it. Now propagate=False;
   banks commit cheap, propagation is its own targeted --addr batch (§55b law).

R22: make clean && extract-all && check-all -> 140 passed, 0 failed of 140.
tools-health OK (dedup 1847/0, C1 234343/234343); 0 NON_MATCHING (G4).
2026-07-21 11:24:48 -06:00
Drew T b3ab5c2253 feat(phase-29 Task-13A): deterministic residual classifier — the permuter's problem is TARGETING
The autopsy (hindsight-study §7) assumed the permuter loses for want of a mutation.
Measured over the whole open backlog, it loses because it is aimed at work a
search-closer provably cannot close.

- NEW tools/residual_class.py: decide a near-miss's class FROM THE BYTES. Decodes each
  mismatching MIPS word -> (op-skeleton, register-fields, immediate); drift FIRST (one
  inserted insn inflates `closeness` by the tail length), then consistent-injective
  register map -> REGALLOC-PERM (§31 S11/RC-3), same-multiset-reorder -> SCHEDULE-REORDER,
  DELAY-SLOT, WIDTH/BRANCH-POLARITY/STRENGTH/ADDRESSING/IMM-OFFSET/IMM-VALUE. Every class
  routes to a BUCKET = which tool the failure wants. Uncovered opcode -> UNKNOWN, COUNTED
  (R32). 16 synthetic unit tests (test_residual_class.py).
- NEW tools/autopsy.py: `collect` materialises the corpus Task-12's telemetry never filled
  (1 of 6,169 records had a residual) by recompiling every open draft through the EXISTING
  match_one path (R33) — 1,752 drafts in 21s at -j12. `report` -> docs/autopsy.md.
- NEW corpus.o0_sources()/is_o0(): the opt-level oracle DERIVED from the Makefile's own -O0
  rules, coverage-asserted. Scoring an -O0 target at -O2 makes the residual 100% artefact
  (the trap this phase hit four times).
- R34 cross-check baked in: residual_class's closeness vs masked_diff.structured_diff's,
  asserted per row; 1,673/1,673 agree, 0 classifier errors.

FINDING: of the 972 records the grinder's own filter admits, only 75 (7.7%) are
permuter-shaped; 547 are structural and 348 are drafts that are not the function at all.
~92% of the daemon's CPU went where it could not win — the byte-grounded explanation of
"7 banks all-time, all Phase 21, 0 since" (Phase-22 audit). grinder.candidates() now
filters on the measured bucket (1,303 -> 78) and takes its directed profile from the
measured class, not the logged label (91% carry none -> it ran on gcc defaults).
Degrades to undirected if uncollected and says so; --no-targeting A/Bs it.

Two measured corollaries (R14, not projections):
- 699 records rank as near-misses at closeness up to 278 purely from a length artefact:
  un-attempted work misfiled as a backlog of hard functions -> new `redraft` bucket.
- a 12-draft gate probe of the `integration` bucket banked 1/12 (11 PLUMBING), so the 306
  prices Task 14's reconcile ladder rather than promising free banks. func_80167714
  (104 ins, reach-134) banked x1, un-propagated by design (§55b).

Two defects fixed forward:
- masked_diff._common_typedefs() used ONE shared probe path, so parallel match_one
  processes clobbered each other: 14 of 1,752 drafts lost in a single 12-way run (0.8%),
  silently, in every parallel wave ever run. Now per-PID.
- gate_stage.match_one_closeness never passed --o0 -> phantom residuals for every -O0
  function, written straight into the backlog this autopsy reads.

R22 clean-fleet: check-all 140 passed, 0 failed of 140; tools-health OK (dedup 1847/0,
C1 234343/234343); 0 NON_MATCHING (G4). Flywheel captured in-session (R30/R31):
cookbook §60, decision-log entry, SETUP.md inventory.
2026-07-21 11:03:59 -06:00
Drew T e4e8ec5ca9 docs(phase-29): SESSION-5 CHECKPOINT — full session-checkpoint block for a fresh session
Session-5 (crack-waves 1-4 + 2 tooling fixes + burndown + hindsight-study + Task-12
telemetry). 140/140 byte-identical, fleet 78.0% instr. NEXT: Task 13 (classifier +
autopsy) then Task 14 (gate_stage ladder + auto-R22). Written BEFORE the pause per
the new checkpoint-before-pause rule.
2026-07-20 23:52:06 -06:00
Drew T f61952bfde docs(phase-29): Task-12 done (structured telemetry flows) + Task 13/14 pickup for a fresh session 2026-07-20 19:50:46 -06:00
Drew T eec96f76a4 feat(phase-29 Task-12): structured failure telemetry + durable permuter-winner save
The permuter-autopsy prerequisite (Drew-directed). Captures the WHAT/WHERE of every
non-match so the classifier/autopsy (Task-13) can mine it, not just a scalar closeness:
- masked_diff.structured_diff(): the per-instruction masked residual [(idx,mine,tgt),...]
- match_one --json: emits {status,closeness,nins,residual} (refactored to share the primitive)
- backlog FIELDS += residual, passes_tried
- gate_stage.match_one_closeness returns + logs the residual on every near/fail (verified
  end-to-end: a near-miss's structured residual now lands in backlog.jsonl)
- grinder: durably save the winning C to .run/permuter-winners/ BEFORE gate_stage banks
  (this session's lesson — 3 permuter wins were lost to a mid-flight revert)
2026-07-20 19:50:14 -06:00
Drew T 30fa4fb34d docs(phase-29): crack-wave 4 checkpoint + crack-track floor; pivot to permuter-automation
6 -O2 cores banked x1 (commit:0766, R22 140/140); -O2 x138 propagation + 4 -O0 deferred
(integration/-O0 walls). Cached crack-fuel drained (waves 1-4). Grinder-concurrency
lesson logged. Next: Tasks 12-14 (the permuter-autopsy program).
2026-07-20 19:43:11 -06:00
Drew T b6549a5f63 feat(phase-29 crack-wave4): bank x1 the 6 -O2 exemplars in ov_SC01_000
func_801749C8, func_80165CA0, func_8012C658, func_801681FC, func_80142B2C,
func_80168430 (crack-wave4). func_8012C658 + func_80142B2C reconciled via the
proven ov_SC01_077 exemplars (S8012C658 struct + void* self-decl / 0-arg
func_80142C84 cast + func_80143994 s32-cast). Whole-binary gate byte-identical.
2026-07-20 19:33:45 -06:00
Drew T 5c936c078c docs(phase-29): hindsight study — optimal decomp ordering + offline-automatic endgame (§7)
A study substrate (UNIVERSAL/COMPILER-CLASS/BFM-SPECIFIC tagged) for a future Fable
pass to refine into a general roadmap for any-platform/any-compiler decomps. §7 adds
the permuter-failure-autopsy loop (mine failures -> extend the ILS wrapper -> shrink
LLM to genuine humps) per Drew's 2026-07-20 direction. Companion to Tasks 12-14.
2026-07-20 19:24:17 -06:00
Drew T a5bad7f48d feat(phase-29 Task-11): burn-down + velocity tracker (the ROI-close instrument)
tools/burndown.py snapshots the 3 fleet metrics + remaining substantial frontier
each run (docs/burndown.jsonl, git-HEAD-keyed), computes velocity vs the previous
commit-snapshot, and flags the yield floor (mean instr-pp over last 3 snapshots <
threshold => 'consider closing P29'). Seeded with the session-4-close baseline.
Reads progress.fleet.md + family-hseq.md (regen those first for a true reading).
2026-07-20 14:55:49 -06:00
Drew T d50c2f6bb9 feat(phase-29 tooling): func_8017AE2C family ×137 via the span-fit jtbl fix (R22 140/140)
The jtbl_family_bank span-fit auto-isolate fix (this session) recovers the wave-3
exemplar-only core func_8017AE2C: all 137 members banked whole-binary byte-identical.
func_8014032C stays blocked (distinct table-count-drift wall, excluded from the fix).
~+24k ins. R22 clean-fleet 140/140.
2026-07-20 14:50:43 -06:00
Drew T 09a561e3e7 fix(phase-29): jtbl_family_bank auto-isolates on the span-fit wall too
Previously only auto-isolated on NON-CONTIGUOUS carve collisions; the span-fit
wall ('table starts do not fit the span', when --like's merged span doesn't match
a sibling's jtbl layout) fell through to carve-fail. jr_isolate unblocks it too
(byte-proven: func_8017AE2C ov_SC01_000 member BANKED). The distinct table-count-
drift error ('more rodata .align than pad specs') stays excluded (not isolate-fixable).
2026-07-20 14:03:51 -06:00
Drew T 9591d61f78 fix(phase-29): family_sweep --hseq --source now overrides the manifest exemplar
The --hseq path templated from pick_exemplar's choice (hard-prefers ov_SC01_077),
silently ignoring --source (wave-2 finding). Now when --source names a different
overlay carrying a MATCHED member of a family, template from IT (self-correcting:
only a non-stub source member is used; whole-binary gate stays the arbiter). The
ov077-default path is unchanged (override skipped).
2026-07-20 14:00:52 -06:00
Drew T 3add702615 docs(phase-29 crack-wave3): session close — wave-3 +~1378 instances, R22 140/140, 77.8% instr
12/16 fresh cached families cracked + propagated (9 full 138/138, 80169BBC 134,
2 jtbl-wall exemplar-only). 4 permuter seeds preserved (R20). Weekly limit hit.
2026-07-20 12:32:40 -06:00
Drew T 5dc984f942 docs(phase-29 crack-wave3): regen family-hseq frontier after wave-3 banks (88.1% fn / 77.8% instr) 2026-07-20 12:29:11 -06:00
Drew T 7d22e8d258 feat(phase-29 crack-wave3): func_8017AE2C jtbl exemplar banked ×1 (174 ins; §8 span-fit carve needs per-overlay isolation, members blocked) 2026-07-20 12:17:31 -06:00
Drew T edf2a3a5e6 chore(phase-29 crack-wave3): isolate func_8017AE2C into own jr region in ov_SC01_077 (byte-neutral; unblocks §8 span-fit carve) 2026-07-20 12:14:39 -06:00
Drew T 0872dc3762 feat(phase-29 crack-wave3): func_80140608 jtbl family bank 138/138 (§8 jtbl_carve; earlier isolate-fail resolved as overlay jr-structure evolved) 2026-07-20 12:12:47 -06:00
Drew T bd2afeb51c feat(phase-29 crack-wave3): func_801555F4 jtbl family bank 138/138 (§18 fix: drop redundant D_8011F730 extern; TU s32 decl + sign-agnostic |= is byte-neutral where reconcile_tu's &-cast dropped 1 ins) 2026-07-20 11:33:52 -06:00
Drew T 3f3a4df5ce feat(phase-29 crack-wave3): func_801594E8 jtbl family bank 138/138 (§8 jtbl_carve + jr_isolate) 2026-07-20 11:12:11 -06:00
Drew T cd63e31487 feat(phase-29 crack-wave3): func_801588CC jtbl family bank 138/138 (§8 jtbl_carve + jr_isolate) 2026-07-20 10:54:14 -06:00
Drew T d2b5f34039 feat(phase-29 crack-wave3): func_80154C24 jtbl family bank 138/138 (§8 jtbl_carve + jr_isolate) 2026-07-20 10:15:46 -06:00
Drew T 5a3f6900bd feat(phase-29 crack-wave3): func_8014032C jtbl exemplar banked ×1 (§59(3) two-table merged-span wall blocks member propagation) 2026-07-20 09:37:42 -06:00
Drew T 364b1a07e0 feat(phase-29 crack-wave3): func_8015C32C jtbl family bank +18 final members (138/138 complete) 2026-07-20 09:28:28 -06:00
Drew T 432396ca64 feat(phase-29 crack-wave3): func_8015C32C jtbl bank ov_SC06_018 (transient crash recovery) 2026-07-19 13:33:26 -06:00
Drew T 135213704e feat(phase-29 crack-wave3): func_8015C32C jtbl family bank +110 members (§8 jtbl_carve)
Propagated the func_8015C32C jump-table exemplar to 110 more overlays via
jtbl_family_bank (carve + ld_interleave + remap, per-sibling whole-binary gated).
Run halted by an uncaught exception at member 111 (ov_SC06_018); 110/119 attempted
banked clean. Remaining ~19 members to follow.
2026-07-19 13:32:36 -06:00
Drew T e160080904 feat(phase-29 crack-wave3): func_8015C32C jtbl family bank ×9 (exemplar + 8 siblings, §8 jtbl_carve)
First jtbl core of crack-wave3: gcc emits the switch jump table into .rodata, so
each overlay needs jtbl_carve + ld_interleave. Path validated 9/9 BANKED.
2026-07-19 12:46:09 -06:00
Drew T 257b7d69ad feat(phase-26): h_seq family sweep — 137 member-matches banked via remap_hseq 2026-07-19 12:28:46 -06:00
Drew T 925dbfa14f fix(phase-29 crack-wave3): rename Prim->Prim_B234 in func_8016B234 exemplar (collides with engine_types.h Prim in member TUs, byte-neutral) 2026-07-19 12:16:15 -06:00
Drew T 48e5c716f2 fix(phase-29 crack-wave3): collapse struct Prim to single-line typedef in func_8016B234 exemplar (§59(1) remap carries typedefs not multiline structs, byte-neutral) 2026-07-19 12:04:36 -06:00
Drew T dcd9bc49a1 docs(phase-29 crack-wave3): regen family-hseq manifest after func_8016B234/C998 banks 2026-07-19 12:02:54 -06:00
Drew T 2ac8070f3d feat(phase-26): h_seq family sweep — 137 member-matches banked via remap_hseq 2026-07-19 11:56:19 -06:00
Drew T bffbb8b9b7 feat(phase-29 crack-wave3): func_8016B234 + func_8016C998 banked ×1 in ov_SC01_077
- func_8016B234: §58b self-def — engine_core.h func_8016B234 (void)->() no-proto
  (byte-neutral: only 0-arg callers fleet-wide); typedef preamble stripped.
- func_8016C998: §59(1) local struct rename Slot/Blk32->Slot_998/Blk32_998
  (collide with TU's Slot) + D_801D9CA0 extern moved to block scope (TU's other
  decls of it are all block-scope; a file-scope one collided).
R22 clean-fleet pending (engine_core.h touched).
2026-07-19 11:47:54 -06:00
Drew T 5513530a5b feat(phase-29 crack-wave3): func_80160534 family sweep — 137 members banked ×137 across fleet
Exemplar cast (§56b) unblocked propagation: the bare func_80161208(p) call
relied on ov_SC01_077's no-proto decl, but member TUs carry a (void) prototype.
Casting the call member-proofs the templated body. All banks whole-binary byte-gated.
2026-07-19 11:46:56 -06:00
Drew T e47e680b18 fix(phase-29 crack-wave3): cast func_80161208 call in func_80160534 exemplar (§56b member-proof, byte-neutral) 2026-07-19 11:43:34 -06:00
Drew T 32ab6e5496 feat(phase-26): h_seq family sweep — 133 member-matches banked via remap_hseq 2026-07-19 11:40:19 -06:00
Drew T d22de5d0e8 feat(phase-29 crack-wave3): func_80160534 banked ×1 in ov_SC01_077 (§58 reconcile_tu D_801152A0) 2026-07-19 11:34:01 -06:00
Drew T 6a367f4246 feat(phase-29 crack-wave3): func_80169BBC banked ×1 in ov_SC01_077 (162 ins)
First clean bank of crack-wave 3 (8 fresh cached ov077 substantial families,
8/8 MATCH). Other 7 pending §58 reconcile; ×138 propagation to follow.
2026-07-19 10:28:57 -06:00
Drew T 1ad5cc359a docs(phase-29 crack-wave2): session-4 checkpoint — +819 members (76.3% instr), frontier honest
Crack-wave 2 (absent-from-ov077 pool) + propagation: 6 families banked ~815
members fleet-wide (R22 140/140). Session-4 total: 75.2->76.3% instr /
60.6->62.8% distinct / 87.04->87.47% fn-count across 2 crack-waves (~1500 banks).
Tooling finding: family_sweep --source is ignored (uses manifest exemplar).
2 preserved permuter seeds. Non-MCP substantial frontier worked out; next fresh
cracks need /mcp prefetch.
2026-07-19 02:48:11 -06:00
Drew T dd184d3ee3 docs(phase-29 crack-wave2): regen family-manifest after +814 member banks (frontier snapshot) 2026-07-19 02:40:11 -06:00
Drew T 8e00adb0a7 feat(phase-29 crack-wave2): func_8013D9B0 family sweep +135 (fully banked)
Propagated from the ov_SC01_000 exemplar (void func_8013D9B0(int param_1), a 141-ins
handwritten GTE color-interp loop w/ $s0-$s4 register pins). Each sibling TU carries
extern-void-(void) self-decls of func_8013D9B0 that conflict with the (int) def; NSD is
unsafe here (it reads the else-func_8013D9B0(N) call statements as decls and mis-casts,
§57a), so the self-decls were mechanically reconciled to K&R empty-args (byte-neutral,
compatible with the (int) def + every 0-arg/1-arg/fn-ptr-cast call site) across 132 TUs
before the sweep. 135/135 members banked, 0 failed (pins templated clean across the
structurally-identical engine siblings).
2026-07-19 02:37:50 -06:00
Drew T b00461774a feat(phase-29 crack-wave2): func_80141A60 family sweep +135 (135/135, fully banked)
Propagated from the ov_SC01_000 exemplar. remap_hseq's gather_externs synthesizes a
non-volatile `extern u16 D_8011511A;` per sibling, which defeats the §18 volatile-strip
CSE (cast reads reload instead of folding to one load) -> byte-DIFF. Every sibling TU
(jr_8013FFD8) already declares D_8011511A volatile at file scope via the banked
func_801418F8, so the fix is to strip the draft's own decl and let the cast-at-use over
the TU's volatile decl produce the single-load match. 135/135 banked, 0 failed.
2026-07-19 02:35:33 -06:00
Drew T c48a485ff9 feat(phase-29 crack-wave2): +2 residual exemplars banked x1 in ov_SC01_000
func_80141A60 (76): §18 cast-at-use *(u16*)&D_8011511A defeats the TU's file-scope
  volatile decl of D_8011511A so gcc CSEs the 3 reads into one load (the draft matched
  standalone but DIFFd in-TU due to the volatile reload).
func_8013D9B0 (141): reconciled the TU's two extern-void-(void) self-decls to K&R
  empty-args (compatible with the int-param def; byte-neutral for the fn-ptr-cast and
  0-arg call sites). NSD was unsafe here (it mis-parses the else-func_8013D9B0(N) call
  statements as decls and mis-casts to void(*)(void), the §57a hazard). Both re-gate
  BYTE-IDENTICAL.
2026-07-19 02:26:05 -06:00
Drew T f72ad4a85d feat(phase-29 crack-wave2): func_8016DC20 +3 SC07 residuals (carried-extern reconcile)
The jr_8013F350 mega-TU declares func_80128ED8(s32,s32*) and DEFINE-macro func_80146C3C,
conflicting with the draft's carried externs. Dropped both externs + fn-ptr-cast the
func_80128ED8 call at use (byte-identical). Banked ov_SC07_006/007/011. Remaining: 1
o2b member (ov_SC07_010) — -O2 reconstruction TU hits an as --fatal-warnings $at wall.
2026-07-19 02:09:23 -06:00
Drew T 3eb6b019ff feat(phase-29 crack-wave2): func_801418F8 +10 residuals via D_8011512E u16 canonical
The 10 jr_8013F350-split residuals failed on 'conflicting types for D_8011512E'
(exemplar decl s16 vs the jr_8013F350 TUs' canonical u16). Reconciled the exemplar
decl to u16 (byte-neutral — sole use is store-0); re-swept. func_801418F8 now 0 stubs.
Remaining: 4 func_8016DC20 SC07 members (byte-DIFF in the o2b/jr_8013F350 split).
2026-07-19 02:04:50 -06:00