Commit Graph

11 Commits

Author SHA1 Message Date
Drew T feb32ed23e feat(phase-30): T4 — grinder ILS warm-restart wired in; SS119 posture audited + a stale doc hazard struck
(1) --fix-def-sig POSTURE: AUDITED CLEAN. `action="store_true"` (defaults False), one
consumer via getattr(a,"fix_def_sig",False), and NO caller anywhere passes it — checked
tools/, .run/ scripts, docs recipes and the Makefile. The flag help already carries the
SS119 warning.

    BUT the audit surfaced a live hazard the earlier pass missed: docs/decision-log.md
    still recommended "--fix-def-sig should likely be default-on for the h_seq path".
    That was byte-REFUTED by T84/SS119 — the flag is a REPAIR, not a default; on 0x80161c98
    it imposed a signedness-wrong `s32 a1` over the true `u32`, turned a byte-correct draft
    into a 1-instruction DIFF (slti vs sltiu), and held 137 members at 0 until DROPPED.
    Struck through in place with a superseding note rather than deleted, so the original
    reasoning stays legible (R31) — but a forward-looking "should be default-on" sitting in
    a doc a fresh session reads FOR DIRECTION is a hazard, not a historical note.

(2) GRINDER WARM-START: tools/permuter_ils.py has sat beside grinder.py since Phase 24 and
was never wired in, so every grind was a COLD search that burned its whole time box
re-descending ground the previous run had already covered. grinder.py now runs `--cycles`
(default 4) timeboxed permutes, each warm-restarted from the previous cycle's best byte
waypoint, stopping early on no gain. `--cycles 1` reproduces the old cold behaviour exactly,
so it is opt-out. --permute-secs is now documented as the PER-CYCLE box.

JUSTIFIED BY MEASUREMENT, not by the task list: the lane looked dead (Phase-22 audit: 7
all-time banks, all Phase 21, 0 since), so I checked for live fuel before building. The
backlog holds 665 open near-misses in the permuter-tractable band (close 1-20), 157 of them
close 1-4, including func_8016BA68 at close=1 with reach=134.

HONEST LIMIT: this is a WIRING change whose yield is UNPROVEN. The Phase-24 evidence for ILS
is one function (func_80148094, 72 -> 36 over ~8 restarts); I have not run it on this
backlog. A winner remains a CANDIDATE — the whole-binary byte-gate is still the sole arbiter
(G3/P9), and an intermediate waypoint is only ever re-seeded, never banked.
2026-07-31 12:26:03 -06:00
Drew T 4d5bee0be8 fix(phase-30): T4 — 2 'Phase-22 grinder bugs' verified STALE (already fixed); real fixes: asm_subdir_for derives from corpus (R33), --fix-def-sig help carries the §119 warning
Verified against code rather than the ledger: the split-blind lookup globs */ correctly, and the
churn was fixed by T5's input-signature gating. Both struck. asm_subdir_for was still a parallel
oracle (silent g[0] on multi-match) -> now corpus.asm_path. --fix-def-sig defaults off correctly
but advertised 'Byte-neutral; gate arbitrates' — the claim T84 refuted (signedness-wrong header
decl over a byte-correct draft; 137 members held at 0 until the flag was dropped).

A defect ledger nobody re-verifies decays into busywork — verify before scheduling.
2026-07-30 20:17:15 -06:00
Drew T 5c894c1e62 feat(phase-29 Task-13B): func_80141B90 x138 + the reach repricing + the length profile
PROPAGATION (§55b, its own targeted batch): dedup_propagate --addr 0x80141B90 --recover
-> "138 overlays byte-identical after propagation"; 117 remaining stubs -> 0; 1 new
dedup group. This was the ONLY one of the 21 directed-run banks worth propagating.

THE REPRICING (R14 — measure a bucket's VALUE, not just its conversion rate):
the directed run converted 27% (21/77) but moved the fleet ~0.03pp, because h_exact
reach of the 21 is: func_80141B90=138, TEN at reach-1 (nothing to propagate), rest 2-10.
Instruction-weighted, the ENTIRE permuter bucket is worth ~0.36pp at 100% conversion.
The mechanism is validated; the fuel was small. Priced frontier (ins-weighted / 13.08M):
  LENGTH-DRIFT |d|<=2   472,178  ~3.6pp  (339 fns)   <- the real permuter-adjacent lever
  integration           419,162  ~3.2pp  (305 fns)   <- Task 14's ladder
  WIDTH                  71,593  ~0.55pp (45)
  permuter (current)     46,571  ~0.36pp (74)
  BRANCH-POLARITY         9,462  ~0.07pp (22)
So WIDTH/BRANCH-POLARITY are NOT worth prioritizing; my earlier "~200 candidates"
framing undersold LENGTH-DRIFT 10x and oversold WIDTH.

NEW: permuter_weights._LENGTH profile (perm_temp_for_expr/perm_expand_expr are the only
passes that change instruction COUNT; the reorder/decl-order levers that dominate the
regalloc+schedule profiles cannot, so they are down-weighted here) + residual_class
._drift_route (|d|<=2 -> permuter/`length`, larger stays structural — same class,
opposite tool) + classify() accepts a PROFILE NAME directly (the measured profile beats
re-parsing a free-text label). 17 unit tests green.

grinder: --profile filter (probe ONE residual class's conversion) + a PERSISTENT attempt
ledger. `tried` was in-process only, so every fresh --once run re-permuted the previous
run's losers — the permuter is deterministic given (base.c, target.o), so that CPU can
never produce a new win. Measured: a 20-target probe drew 19 already-tried targets.
Keyed by draft_sig so an improved draft legitimately re-opens the function.
2026-07-21 15:05:56 -06:00
Drew T b8ee9fa9d2 feat(phase-29 Task-13B): directed permuter wins first try; 3 latent gate defects fixed at root
First run of the Task-13A-targeted grinder (--once --batch 8 --permute-secs 90):
  targeting: ON — 1665 classified; only bucket=permuter is admitted
  targeting skipped 1601 non-permuter candidates
    {redraft 707, structural 583, integration 305, unclassified 4, unknown 2}
  permuter WON func_80181F78 @ ov_SC03_014 (close was 1)   [~6 min]

BANKED 2 (both whole-binary byte-gated, R22 clean-fleet 140/140):
- func_80181F78 (8 ins)  — classified DELAY-SLOT / schedule
- func_80141B90 (29 ins) — classified IMM-VALUE / cse
Both were classified bucket=permuter by residual_class BEFORE the run: the classifier
predicted a search-closer could reach them, and one did. First end-to-end validation of
the targeting thesis.

THREE latent defects, all pre-existing, all unreachable while the grinder banked nothing
since Phase 21 — the fix made it win and every one fired at once (cookbook §60a):

1. gate_stage commit path crashed on src=None. `src` is DELIBERATELY never defaulted
   (Phase 26-A: a default silently PINS the gate to the main .c), but the commit did
   `git add src …` unconditionally -> every caller that omits it (grinder, orchestrator,
   idiom_hunt) crashes THE MOMENT IT BANKS. Now `git add -u src/`, which also retires the
   `src/ov_*/*.c` filename glob that once omitted 4 R22-verified banks from a commit.
2. _xform ladder dirs (-cn/-cast/-rc/-uni) ACCUMULATE across runs: they held 34 stale
   drafts, so the gate processed 34 when the grinder submitted 1, and banked one function
   it was never asked to try — which would have been committed under a message naming a
   different one. Nothing wrong entered the tree (G3/P9 held; the gate banks only
   byte-identical output) but report and work had diverged. Now cleared per run.
3. grinder called gate_stage with the default propagate=True = `dedup_propagate
   --auto-from`, the §55b fleet-wide path that timed out at 3600s and left 90/140 overlays
   broken — and being INSIDE the gate it takes the banks down with it. Now propagate=False;
   banks commit cheap, propagation is its own targeted --addr batch (§55b law).

R22: make clean && extract-all && check-all -> 140 passed, 0 failed of 140.
tools-health OK (dedup 1847/0, C1 234343/234343); 0 NON_MATCHING (G4).
2026-07-21 11:24:48 -06:00
Drew T b3ab5c2253 feat(phase-29 Task-13A): deterministic residual classifier — the permuter's problem is TARGETING
The autopsy (hindsight-study §7) assumed the permuter loses for want of a mutation.
Measured over the whole open backlog, it loses because it is aimed at work a
search-closer provably cannot close.

- NEW tools/residual_class.py: decide a near-miss's class FROM THE BYTES. Decodes each
  mismatching MIPS word -> (op-skeleton, register-fields, immediate); drift FIRST (one
  inserted insn inflates `closeness` by the tail length), then consistent-injective
  register map -> REGALLOC-PERM (§31 S11/RC-3), same-multiset-reorder -> SCHEDULE-REORDER,
  DELAY-SLOT, WIDTH/BRANCH-POLARITY/STRENGTH/ADDRESSING/IMM-OFFSET/IMM-VALUE. Every class
  routes to a BUCKET = which tool the failure wants. Uncovered opcode -> UNKNOWN, COUNTED
  (R32). 16 synthetic unit tests (test_residual_class.py).
- NEW tools/autopsy.py: `collect` materialises the corpus Task-12's telemetry never filled
  (1 of 6,169 records had a residual) by recompiling every open draft through the EXISTING
  match_one path (R33) — 1,752 drafts in 21s at -j12. `report` -> docs/autopsy.md.
- NEW corpus.o0_sources()/is_o0(): the opt-level oracle DERIVED from the Makefile's own -O0
  rules, coverage-asserted. Scoring an -O0 target at -O2 makes the residual 100% artefact
  (the trap this phase hit four times).
- R34 cross-check baked in: residual_class's closeness vs masked_diff.structured_diff's,
  asserted per row; 1,673/1,673 agree, 0 classifier errors.

FINDING: of the 972 records the grinder's own filter admits, only 75 (7.7%) are
permuter-shaped; 547 are structural and 348 are drafts that are not the function at all.
~92% of the daemon's CPU went where it could not win — the byte-grounded explanation of
"7 banks all-time, all Phase 21, 0 since" (Phase-22 audit). grinder.candidates() now
filters on the measured bucket (1,303 -> 78) and takes its directed profile from the
measured class, not the logged label (91% carry none -> it ran on gcc defaults).
Degrades to undirected if uncollected and says so; --no-targeting A/Bs it.

Two measured corollaries (R14, not projections):
- 699 records rank as near-misses at closeness up to 278 purely from a length artefact:
  un-attempted work misfiled as a backlog of hard functions -> new `redraft` bucket.
- a 12-draft gate probe of the `integration` bucket banked 1/12 (11 PLUMBING), so the 306
  prices Task 14's reconcile ladder rather than promising free banks. func_80167714
  (104 ins, reach-134) banked x1, un-propagated by design (§55b).

Two defects fixed forward:
- masked_diff._common_typedefs() used ONE shared probe path, so parallel match_one
  processes clobbered each other: 14 of 1,752 drafts lost in a single 12-way run (0.8%),
  silently, in every parallel wave ever run. Now per-PID.
- gate_stage.match_one_closeness never passed --o0 -> phantom residuals for every -O0
  function, written straight into the backlog this autopsy reads.

R22 clean-fleet: check-all 140 passed, 0 failed of 140; tools-health OK (dedup 1847/0,
C1 234343/234343); 0 NON_MATCHING (G4). Flywheel captured in-session (R30/R31):
cookbook §60, decision-log entry, SETUP.md inventory.
2026-07-21 11:03:59 -06:00
Drew T eec96f76a4 feat(phase-29 Task-12): structured failure telemetry + durable permuter-winner save
The permuter-autopsy prerequisite (Drew-directed). Captures the WHAT/WHERE of every
non-match so the classifier/autopsy (Task-13) can mine it, not just a scalar closeness:
- masked_diff.structured_diff(): the per-instruction masked residual [(idx,mine,tgt),...]
- match_one --json: emits {status,closeness,nins,residual} (refactored to share the primitive)
- backlog FIELDS += residual, passes_tried
- gate_stage.match_one_closeness returns + logs the residual on every near/fail (verified
  end-to-end: a near-miss's structured residual now lands in backlog.jsonl)
- grinder: durably save the winning C to .run/permuter-winners/ BEFORE gate_stage banks
  (this session's lesson — 3 permuter wins were lost to a mid-flight revert)
2026-07-20 19:50:14 -06:00
Drew T 65e96e5d5a feat(phase-28 T2): purge the poisoned grinder blacklist — 8 of its 22 had MATCHED anyway
The blacklist recorded "permuter won, byte-gate rejected => plumbing-bound, never
re-permute" for 22 fns. It was manufactured by a gate that no longer exists.

- R14 ON THE PREMISE, TWICE (before touching anything):
  * The roadmap's two named grinder bugs are ALREADY FIXED (commit:0327, commit:0200) — stale line.
  * docs/tooling-audit.md:933-937 DOWNGRADED ITS OWN FINDING with three corrections: the
    prescribed fix is a NO-OP (deleting the scanner banks zero fns — harvest_verify was
    single-TU BY CONSTRUCTION, the defect was mislocalized to it); nothing is being discarded
    now (grinder STOPPED since 2026-07-02 — confirmed via .run/auto/STOP); queue magnitude
    inflated (1252, not 1298). T2's only real content was the persisted artifact.

- AND THE AUDIT'S SNAPSHOT IS ITSELF STALE (verified in code): harvest_verify is now fully
  multi-TU — _stubs derives every stub across every TU from the corpus oracle (:122), render()
  splices "each draft into the TU that actually holds its stub", _write() writes multiple
  paths, un-stubbed drafts are REPORTED not silently dropped (R32). The gate that manufactured
  the blacklist is gone. No harvest_verify change was needed or made.

- THE PROOF IT WAS MANUFACTURED, NOT OBSERVED: of the 22 entries, 8 have since MATCHED anyway
  (func_80131D68/80149374/8014FE60/80150528/8016BBE0/80171C64/80174684/8017F290); the other 14
  are still stubs and would have been skipped FOREVER on a dead gate's verdict. (16 of 22 were
  split-hosted, so harvest_verify never compiled them — the permuter's byte-matches were
  discarded UNBUILT. The audit predicted 5 matched-anyway; it is 8.)

- DONE: blacklist -> [] (poisoned copy preserved at grinder_blacklist.json.poisoned-pre-T2);
  grinder.py now carries the RULE (R35): a blacklist entry is a verdict from a SPECIFIC GATE
  and EXPIRES when that gate changes — purge and re-derive from the fixed gate, never inherit.
  A persisted negative verdict is only as good as the instrument that produced it.

- No byte claim (analysis tooling + a scratch artifact only) -> no R22 cycle owed.
2026-07-15 22:37:19 -06:00
Drew T 0eadf313ee feat(phase-24): T5 — §31-directed permuter mutation + grinder input-changed gating
- tools/permuter_weights.py (NEW): classify(klass,where) -> regalloc|schedule|cse|None
  + render_settings_toml emitting the [weight_overrides] table decomp-permuter merges over
  the gcc defaults (main.py:336 / helpers.py per-key replace). Biases pass-selection toward
  each class's §31 levers (perm_reorder_decls RC-1/3, perm_reorder_stmts RC-2/S1,
  perm_temp_for_expr S2, perm_commutative cse); value/type noise -> ~0.1. NO submodule edit.
- p16_permute.setup(..., klass=, where=) writes it (+--klass, backlog auto-lookup);
  grinder.py auto-threads klass/where_stuck. klass=None -> gcc defaults (undirected superset).
- grinder fix (R14): replace the blind tried.clear() idle churn with input-changed gating
  (draft_sig=(best_draft mtime, closeness) — re-open a fn only when the worker improved it).
- VALIDATED: regalloc profile drove func_8014E048 masked 36->29 (match_one -drz 35->28) where
  the undirected search stalled; re-logged (28, improved draft promoted). Flagship S11/RC-6
  count-exact seeds improve but do not gate (intrinsic, §31 "two probes dont grind") -> T5b fuel.
- distill: cookbook §3b + SETUP inventory (R16/R30). No build-input changed -> 136/136 untouched.
2026-07-03 00:14:29 -06:00
Drew T e91859fb4a feat(phase-23): grinder per-binary fix (5-layer) — unlock non-077 near-miss grinding
The grinder/backlog pipeline was ov_SC01_077-hardcoded 5 layers deep (same class as the
T7 lora_grind bug). Fixed all so the permuter grinder can process a non-077 near-miss:
1. gate_stage.append_record stores the source "binary"
2. backlog.FIELDS keeps it (else append_record dropped it)
3. backlog.load_best/_open_stubs is fleet-aware: a fn matched in ov_SC01_077 but
   propagation-stuck stays OPEN in its overlay, so it surfaces via that record instead
   of being dropped as "matched" (the grinder must SEE it to grind it)
4. p16_permute.setup takes the target binary's asm-subdir (was hardcoded 077)
5. grinder resolves per-binary asm + gates grouped by binary + allows unknown nins
Backward-compatible: legacy records (no binary) default ov_SC01_077.

Validated end-to-end: the 3 fresh reach-134 close=1 ov_SC01_000 fns now surface, resolve
to ov_SC01_000's asm, and gate via ov_SC01_000.

TWO byte-evidenced findings (redirect the fuel strategy):
- the reach>=2 close=1 fuel is MODEL semantic-misses, not permuter fuel: func_8012E27C's
  target is "return 1" but the 7B drafted an empty "void f(void){}" (corpus overfit
  empty-leaf); func_8012BF4C/AD64 are trivial sw/sh setters drafted empty. A corrected
  draft banks them (+3 byte-identical via the fixed gate, @commit:0326); the permuter cannot
  add a missing return/store. Lever = corpus-v3 leaf variety, not the permuter.
- x reach is propagation-capped: the 3 are inline-matched in ov_SC01_077_a.c (the stuck-
  local cap) -> dedup_propagate "nothing to propagate" -> banked x1. Lever = dedup-collapse.

check-all 136/136 throughout. docs/gen2-mips-matching-model.md + CURRENT_PHASE updated.
2026-06-30 00:58:13 -06:00
Drew T 4927f38c46 feat(phase-21): option C — grinder blacklist + sig_unify DEF-side arity-extend recovery
- grinder.py: persistent blacklist for permuter-won/gate-rejected (plumbing-bound) fns;
  stops the tried.clear()-after-idle churn (was 0 banks in 8h on func_8014F3E8/func_8014FE60)
- sig_unify.py rewrite_def: adopt the canonical param list on arity mismatch (DEF-side
  loose-typing wall) — unused extras sit in $a0-$a3, free at -O2; banked func_8016EDEC/EE40
- cookbook §22 (DEF-side recovery + grinder blacklist); CURRENT_PHASE option-C log
- compounds: sig_unify is in the gate pipeline -> every future wave + the grinder auto-recover
2026-06-21 22:55:40 -06:00
Drew T 2d7926e9f6 feat(phase-21): T4/T5 tooling — worker Workflow + wave selector + permuter grinder
- tools/workflows/worker_wave.js: parallel toolkit-aware drafter agents (xHigh) per target,
  each writes its best matching C to the drafts dir (gate run by the orchestrator after).
  args parsed robustly (harness may serialize to a JSON string — §20 gotcha).
- tools/wave_targets.py: select a wave batch from the fuel manifest (pool/region/N, skips
  backlog walls). tools/grinder.py: token-free permuter grinder — permute backlog near-misses
  -> gate_stage -> bank; STOP/heartbeat, idles for new worker near-misses, supervisor-friendly.
2026-06-21 12:41:14 -06:00