Commit Graph

7 Commits

Author SHA1 Message Date
Drew T b3ab5c2253 feat(phase-29 Task-13A): deterministic residual classifier — the permuter's problem is TARGETING
The autopsy (hindsight-study §7) assumed the permuter loses for want of a mutation.
Measured over the whole open backlog, it loses because it is aimed at work a
search-closer provably cannot close.

- NEW tools/residual_class.py: decide a near-miss's class FROM THE BYTES. Decodes each
  mismatching MIPS word -> (op-skeleton, register-fields, immediate); drift FIRST (one
  inserted insn inflates `closeness` by the tail length), then consistent-injective
  register map -> REGALLOC-PERM (§31 S11/RC-3), same-multiset-reorder -> SCHEDULE-REORDER,
  DELAY-SLOT, WIDTH/BRANCH-POLARITY/STRENGTH/ADDRESSING/IMM-OFFSET/IMM-VALUE. Every class
  routes to a BUCKET = which tool the failure wants. Uncovered opcode -> UNKNOWN, COUNTED
  (R32). 16 synthetic unit tests (test_residual_class.py).
- NEW tools/autopsy.py: `collect` materialises the corpus Task-12's telemetry never filled
  (1 of 6,169 records had a residual) by recompiling every open draft through the EXISTING
  match_one path (R33) — 1,752 drafts in 21s at -j12. `report` -> docs/autopsy.md.
- NEW corpus.o0_sources()/is_o0(): the opt-level oracle DERIVED from the Makefile's own -O0
  rules, coverage-asserted. Scoring an -O0 target at -O2 makes the residual 100% artefact
  (the trap this phase hit four times).
- R34 cross-check baked in: residual_class's closeness vs masked_diff.structured_diff's,
  asserted per row; 1,673/1,673 agree, 0 classifier errors.

FINDING: of the 972 records the grinder's own filter admits, only 75 (7.7%) are
permuter-shaped; 547 are structural and 348 are drafts that are not the function at all.
~92% of the daemon's CPU went where it could not win — the byte-grounded explanation of
"7 banks all-time, all Phase 21, 0 since" (Phase-22 audit). grinder.candidates() now
filters on the measured bucket (1,303 -> 78) and takes its directed profile from the
measured class, not the logged label (91% carry none -> it ran on gcc defaults).
Degrades to undirected if uncollected and says so; --no-targeting A/Bs it.

Two measured corollaries (R14, not projections):
- 699 records rank as near-misses at closeness up to 278 purely from a length artefact:
  un-attempted work misfiled as a backlog of hard functions -> new `redraft` bucket.
- a 12-draft gate probe of the `integration` bucket banked 1/12 (11 PLUMBING), so the 306
  prices Task 14's reconcile ladder rather than promising free banks. func_80167714
  (104 ins, reach-134) banked x1, un-propagated by design (§55b).

Two defects fixed forward:
- masked_diff._common_typedefs() used ONE shared probe path, so parallel match_one
  processes clobbered each other: 14 of 1,752 drafts lost in a single 12-way run (0.8%),
  silently, in every parallel wave ever run. Now per-PID.
- gate_stage.match_one_closeness never passed --o0 -> phantom residuals for every -O0
  function, written straight into the backlog this autopsy reads.

R22 clean-fleet: check-all 140 passed, 0 failed of 140; tools-health OK (dedup 1847/0,
C1 234343/234343); 0 NON_MATCHING (G4). Flywheel captured in-session (R30/R31):
cookbook §60, decision-log entry, SETUP.md inventory.
2026-07-21 11:03:59 -06:00
Drew T eec96f76a4 feat(phase-29 Task-12): structured failure telemetry + durable permuter-winner save
The permuter-autopsy prerequisite (Drew-directed). Captures the WHAT/WHERE of every
non-match so the classifier/autopsy (Task-13) can mine it, not just a scalar closeness:
- masked_diff.structured_diff(): the per-instruction masked residual [(idx,mine,tgt),...]
- match_one --json: emits {status,closeness,nins,residual} (refactored to share the primitive)
- backlog FIELDS += residual, passes_tried
- gate_stage.match_one_closeness returns + logs the residual on every near/fail (verified
  end-to-end: a near-miss's structured residual now lands in backlog.jsonl)
- grinder: durably save the winning C to .run/permuter-winners/ BEFORE gate_stage banks
  (this session's lesson — 3 permuter wins were lost to a mid-flight revert)
2026-07-20 19:50:14 -06:00
Drew T 876dc7f053 feat(phase-28 T5): resident 21 -> 14 stubs (7 banked, 90.34%) + fix match_one's fake isolation
Ultracode wave: 16 isolated drafters over the resident's non-jtbl stubs (the 5 jtbl deferred —
they need the rodata-island carve, §53). Drafts only; the whole-binary byte-gate arbitrated after.

- BANKED 7/16, byte-gated: func_800CEFD0(77) func_800D0D7C(45) func_800D1B80(22) func_800D1E28(37)
  func_800D1FC8(62) func_800D29F8(172) func_800D2D10(39).
  Resident REAL 122 -> 129, stubs 21 -> 14, byte-ident 124/145 (85.52%) -> 131/145 (90.34%).
  FLEET instr 9017152 -> 9017606 (+454 ins). R22 make clean && extract-all && check-all ->
  140 passed, 0 failed of 140 (the first R22 was killed by a terminal crash and RE-RUN, not assumed).
  Ground truth on 14 agrees 3 ways: source grep, splat-emitted stub .s count, progress.py.

- §52b's LAW, MEASURED AGAIN INDEPENDENTLY: the agents self-reported 11 match_one MATCH; the
  whole-binary gate banked 7 (64%). All 4 blocked MATCHes died on `conflicting types`
  (D_8010EDEC / D_80115110 / func_800D1984 / cdFileLocTable) — the loose-typing def-side wall, NOT
  codegen. gate_stage's recovery banked 0/5 on them. A match_one MATCH is a CANDIDATE (G3/P9).

- FIX — match_one's isolation was FAKE, and its own docstring was the false spec. It promises
  "Fully isolated (own temp dir) so many run in PARALLEL with no shared build -- a real asm-differ
  loop for an agent to iterate against", while `--work` defaulted to the SHARED '.run/match': every
  concurrent caller compiled into the same t.c/t.o. FOUND BY AN AGENT MID-WAVE, the only way it can
  be found — it read another agent's function out of its own scratch ("found another agent's
  func_800D2650 in my t.c") and reported it. Every other agent steered by a loop that could hand it
  someone else's compile: a CONFIDENT WRONG verdict, worse than a crash. Default is now a private
  .run/match/<fn>.<pid>; the default IS the promise. (Some agents had already worked around it by
  passing --work themselves.) The byte-gate was never at risk — it is the sole arbiter — but the
  iteration loop the agents steer by absolutely was.

- The 14 remaining: 5 PLUMBING (loose-typing) + 4 DIFF (genuine codegen: func_800D2650 close=4,
  func_800CFAD0 close=5, func_800D0E30 close=12, func_800D27DC close=48) + 5 jtbl deferred.
  Dossier next (T5b) — the agents' per-function residual analyses are the durable asset (R30).
2026-07-16 01:41:57 -06:00
Drew T e0a0becfaa feat(phase-27 T4): one cdecl typedef-strip primitive (was six regexes) + surface cc1 stderr
The plan named two defective regexes; the tree had SIX with complementary holes, each
silently recording the resulting compile failure as "not a match" — a plumbing error
wearing a compiler wall's clothes, the exact class the 26-A audit exists to end (R32).

- cdecl.py: the canonical primitive — typedef_names(tu_path) + strip_provided_typedefs
  (draft, provided). Built on tu_statements (robust) NOT tu_scope (which coverage-asserts
  -> would crash the byte-gate on any unrelated unparseable file-scope statement). Splits
  multi-typedef lines (split_statements, depth-aware); covers scalar AND struct typedefs;
  keeps draft-local types. lru_cached.
- harvest_verify.py: strips PER-TU (cdecl.typedef_names of the draft's real target TU) ->
  unblocks the 39 struct-typedef drafts the scalar-only _TD dropped. And SURFACES cc1
  stderr: build() stashes it; a single-draft failure is classified DIFF / PLUMBING:… /
  CC1-FAIL / SKIP -> .run/harvest_failed.classified.txt. A `redefinition` is no longer
  recorded byte-identically to a codegen miss.
- masked_diff.py: strip_scalar_typedefs() (common.h set derived from the header once, R33,
  cached) replaces SCALAR_TYPEDEF_RE.sub for the ISOLATED compile; wired into match_one +
  p16_permute. Fixes the multi-typedef-LINE skip that discarded 42 masked-MATCH drafts over
  whitespace. Unblocks B4's func_8015C32C (redefinition of 's16').
- canon_sig_reconcile / eval_lora / format_finetune keep their own copies — migrate
  per-bank, byte-gated (the audit-prescribed cadence, not a big-bang swap).

VERIFIED:
- HEADLINE known-answer: func_8015C030 -> MATCH (23 ins) UNEDITED via match_one (was
  CC1-FAIL; the multi-line typedef split alone fixes it — a live x134-family draft that
  was being discarded over whitespace).
- unit: 7/7 scalars stripped; a local struct KEPT; a TU-provided Blk16 stripped.
- classifier unit: DIFF / PLUMBING:… / CC1-FAIL / SKIP all label correctly.
- all 5 edited tools import + AST-parse clean.
- R22 clean-fleet: check-all 136/136; main clean-rebuild 143dbb89. (A mid-test c4546248
  "mismatch" was a stale-incremental artifact from concurrent compiles, cleared by a clean
  rebuild — the R22 lesson; edits touch only tools/, src/ stayed git-clean.)
- SAFETY: a strip bug can only fail-to-bank, never falsely bank (INCLUDE_ASM pastes the
  original asm; a wrong draft always changes bytes -> always fails SHA1).
2026-07-15 18:11:43 -06:00
Drew T d572cc4d2b chore(phase-25): T5b batch-3 (_o0 measured) + T6 Fable5 brief + match_one --o0 (fleet 72.29%, handoff)
- batch-3: the 3 _o0 giants 3/3 isolation-MATCH at -O0 (new tools/match_one.py --o0 flag);
  0 banked — an -O0 in-context byte-diff (canon_sig_reconcile's void->s32 is NOT byte-neutral
  at -O0) -> deferred to T7. Frontier 125/127 draftable exemplars measured.
- R14 correction: the mechanical reconcile-sweep is NOT the clean 51-target x134 win first
  hoped. The frontier "match" status carried un-verified agent claims (5-sample spot-check
  = 3/5 genuine); ~19/51 have clean engine_core.h canonicals (the reliable canon_sig_reconcile
  tier), the rest hit VARIED walls (callee-sig conflicts, non-identical types Vec3/SVEC,
  macro-local data) -> this is the genuine Fable5/T6 residual, not a mechanical sweep.
- docs/phase25-t6-fable-brief.md: the grounded Step-B input package for the fresh Fable5Max
  session to author the crack curriculum (95-stub worklist by wall-class + tools + 5 swing
  questions). CURRENT_PHASE: T6 hand-off (fresh session, Fable5Max, read the brief). Phase OPEN.
2026-07-09 22:45:56 -06:00
Drew T 5984749421 feat(phase-24): T2 — floor-free relocation-masked permuter scorer (-drz)
- tools/masked_diff.py: shared objdump -drz masking oracle (jal/j 26-bit + HI16/LO16
  immediate mask; -z keeps nop runs -> no GTE under-count). object-vs-object (permuter,
  +reloc-operand equality) and object-vs-.s (match_one) modes.
- match_one.py refactored onto masked_diff (-dr -> -drz); regression-clean on count-exact
  seeds (35/52/72/110, leaf-MATCHes), output format unchanged (gate_stage/grinder compatible)
- tools/masked_scorer.py MaskedScorer: drop-in for decomp-permuter's Scorer; scores masked
  .text closeness (bottoms out at 0) not the stock mnemonic-diff floor
- tools/permuter/run_masked.py: rebinds src.main.Scorer -> MaskedScorer (NO submodule edit,
  R3/R20); p16_permute.run_permuter wired to it
- VALIDATED: masked_self=0; masked_cand~match_one; STOCK floor 1750-2100 vs masked 36-77
  (the wander cause); live permuter base score = masked 77 (not stock 1930), descends to 74
- R14 SELF-CORRECTION: -drz confirms func_80132784 is 4/400 count-exact (T1's -dr 204 was a
  16-nop-collapse artifact); backlog re-logged close 4. The flagship IS 4 ins away.
- no build-input changed (136/136 untouched); 2 compile FAILs surfaced -> T3 (GTE asm, typedefs)
2026-07-02 23:10:54 -06:00
Drew T 4c7cf48ebf feat(phase-12): T3 — +2 (typedef-strip recovery) + per-function match tool (REAL 87->89)
- tools/match_one.py (new): compile ONE function's C standalone (pinned triple), mask
  relocations (jal/HI16/LO16 like psyq_identify), compare to the target bytes in its .s.
  Isolated temp dir -> parallelizable; a real per-function asm-differ loop for agents to
  iterate against without a shared build. Validated: a matched fn -> MATCH, residuals ->
  informative diffs.
- harvest_verify.py + match_one.py: strip inline scalar-typedef redefinitions (some drafts
  inlined `typedef unsigned char u8;` -> C89 redefinition error vs common.h = a COMPILE
  fail, not a byte mismatch). Re-gate recovered func_800CFBCC + func_800D1938.
- REAL 89 (62.76% byte-identical); resident 8e17e02f, main 143dbb89 (R22 clean).
- 54 stubs remain (scheduling/regalloc residuals + big GPU/dispatch fns) -> match_one-driven
  iteration pass next.
2026-06-16 12:43:59 -06:00