Commit Graph

217 Commits

Author SHA1 Message Date
Drew T 72e71811d3 feat(phase-21): grinder gate — +1 fns x1 propagated (fleet 61.04%) 2026-06-22 08:54:00 -06:00
Drew T b1272109c0 feat(phase-21): worker gate — +3 fns x2 propagated (fleet 61.0%) 2026-06-22 08:51:03 -06:00
Drew T 501d112758 docs(phase-21): session handoff + queued 10h run (fleet 60.93%) — crash-recovery for fresh session
- CURRENT_PHASE ★ CURRENT STATE: overnight loop summary (59.05->60.93%, ~55 fns, 12 waves), the 3
  gate-safe tooling upgrades + 6 idioms, the tractable-pool-depleting finding, exact resume cycle
- ▶ QUEUED 10-HOUR RUN: fuel verified (80 tractable+28 giants+9 o0 cached, grinder 21 eligible),
  staged first batch, paste-ready fresh-session start (grinder + worker loop, auto-rotate to giants)
- T6 marked done (the loop ran); T7 (close) pending Drew's gate-2. NOT STARTED (STOP present).
2026-06-22 07:47:14 -06:00
Drew T 9e9278d4a8 feat(phase-21): grinder gate — +1 fns x1 propagated (fleet 60.93%) 2026-06-22 05:31:06 -06:00
Drew T c63a41b1a8 feat(phase-21): worker gate — +1 fns x1 propagated (fleet 60.89%) 2026-06-22 05:07:05 -06:00
Drew T 43a356b3fb feat(phase-21): worker gate — +6 fns x5 propagated (fleet 60.85%) 2026-06-22 04:25:30 -06:00
Drew T 0b77956d54 docs(phase-21): drafter-distilled idioms (non-volatile reload-clobber, phantom-frame induce) + backlog/progress regen
func_801758FC's drafter added two byte-verified idioms to cookbook §21 (additive, H5-safe).
2026-06-22 03:31:30 -06:00
Drew T 06496b5238 feat(phase-21): worker gate — +4 fns x4 propagated (fleet 60.65%) 2026-06-22 03:30:16 -06:00
Drew T 33e21d3050 feat(phase-21): worker gate — +3 fns x3 propagated (fleet 60.5%) 2026-06-22 02:50:42 -06:00
Drew T 7b477c82b9 feat(phase-21): wave_targets — exclude self-MATCH-but-gate-rejected via drafter verdict too
close=0 alone missed fns sig_unify regresses to close>0 post-transform (e.g. func_8014B944,
self-MATCH every wave, never banks). Also match the drafter's 'none — MATCH' where_stuck verdict.
Excluded set 7 -> 16; pool waves now draft more FRESH targets (wave-8 close-rate already up to 0.21).
2026-06-22 02:13:17 -06:00
Drew T 77644039b8 feat(phase-21): worker gate — +6 fns x6 propagated (fleet 60.38%) 2026-06-22 02:10:22 -06:00
Drew T 7d46a6056e feat(phase-21): worker gate — +4 fns x4 propagated (fleet 60.15%) 2026-06-22 01:34:03 -06:00
Drew T 40fa76284e feat(phase-21): wave_targets skips close=0 plumbing-blocked near-misses from the pool
- self-MATCH-but-gate-rejected fns can never bank via re-drafting (the drafter reproduces the
  byte-correct body the gate rejects again) -> exclude from POOL selection so agents draft FRESH
  targets; they stay in the backlog as recovery-tooling/hand-finish fuel
- CURRENT_PHASE: sustained-loop state + pool-mode override policy (compaction-recovery anchor)
2026-06-22 00:59:06 -06:00
Drew T 8a6db99b85 feat(phase-21): worker gate — +3 fns x3 propagated (fleet 59.99%) 2026-06-22 00:53:33 -06:00
Drew T 29858df0b4 docs(phase-21): distill — register-short-truthiness sll-16 idiom from func_8013F244 2026-06-22 00:08:05 -06:00
Drew T 1332f43d55 feat(phase-21): worker gate — +4 fns x4 propagated (fleet 59.87%) 2026-06-22 00:05:35 -06:00
Drew T 28767a6a07 feat(phase-21): worker gate — +1 fns x1 propagated (fleet 59.72%) 2026-06-21 23:26:10 -06:00
Drew T 4927f38c46 feat(phase-21): option C — grinder blacklist + sig_unify DEF-side arity-extend recovery
- grinder.py: persistent blacklist for permuter-won/gate-rejected (plumbing-bound) fns;
  stops the tried.clear()-after-idle churn (was 0 banks in 8h on func_8014F3E8/func_8014FE60)
- sig_unify.py rewrite_def: adopt the canonical param list on arity mismatch (DEF-side
  loose-typing wall) — unused extras sit in $a0-$a3, free at -O2; banked func_8016EDEC/EE40
- cookbook §22 (DEF-side recovery + grinder blacklist); CURRENT_PHASE option-C log
- compounds: sig_unify is in the gate pipeline -> every future wave + the grinder auto-recover
2026-06-21 22:55:40 -06:00
Drew T a1b29eac8a feat(phase-21): recovery gate — +2 fns x2 propagated (fleet 59.68%) 2026-06-21 22:51:51 -06:00
Drew T 9dcabfbdc7 docs(phase-21): distill — giv-anchor store-order idiom from func_80178298 2026-06-21 22:39:22 -06:00
Drew T 125222d14e feat(phase-21): worker gate — +7 fns x6 propagated (fleet 59.6%) 2026-06-21 22:35:16 -06:00
Drew T 0744e5ad89 docs(phase-21): distill — disjoint-bits add->ori barrier idiom from func_80169058 2026-06-21 22:05:23 -06:00
Drew T 7f272eb6b8 feat(phase-21): worker gate — +5 fns x4 propagated (fleet 59.37%) 2026-06-21 22:02:17 -06:00
Drew T fc3b7d25e1 docs(phase-21): distill — memcpy-unaligned + 16-byte struct block-copy + out-param-buffer idioms from func_80153800/func_80163A94/func_801749C8 2026-06-21 15:22:16 -06:00
Drew T 1d7c31393c feat(phase-21): worker gate — +7 fns x4 propagated (fleet 59.21%) 2026-06-21 15:17:27 -06:00
Drew T 0a79cf63cd docs(phase-21): resume log — flywheel built, fleet 59.05%, ready for fresh-session /loop 2026-06-21 13:58:44 -06:00
Drew T 5fc5d64445 feat(phase-21): R2/R3 — the learning flywheel (class-grouped waves + distill)
- residual class travels with the draft: drafter stamps // @class / // @stuck; gate_stage logs
  the worker's gcc-quirk class to the backlog (not the coarse manifest class).
- wave_targets.py --class <C> / --list-classes: class-grouped re-attempt waves over the backlog's
  classified near-misses (REGALLOC/SCHEDULE/REMAT/STRUCT/IV/LOOPGUARD/LOOSE/PLUMBING).
- orchestrator prep: auto-picks a class-focused wave when a class accrues >= threshold near-misses,
  else a fresh pool harvest; clears the draft dir per wave.
- tools/workflows/distill.js: post-wave agent extracts NEW byte-verified gcc idioms -> cookbook §21
  (R16, conservative); drafters now read the LIVE cookbook so distilled idioms feed forward.
- runbook: the full prep->worker->gate->distill cycle (the Phase-18 close-rate loop, automated).
2026-06-21 13:56:59 -06:00
Drew T aa010db9d4 feat(phase-21): R1 — gate pre-reboot wave2 drafts (+2 fns x134)
- gate_stage on the 15 worker drafts saved before the reboot: banked 2 byte-matches,
  propagated x134 (dedup 1508->1510); the gate process was killed pre-commit (empty log),
  banks recovered from the working tree (dedup_propagate fail-closed per-overlay = verified).
  Remaining 13 -> backlog near-misses.
2026-06-21 13:50:53 -06:00
Drew T 4d00a40609 feat(phase-21): worker gate — +4 fns x2 propagated (fleet 59.05%) 2026-06-21 13:50:39 -06:00
Drew T 7bd3fa2ee9 docs(phase-21): backlog snapshot at reboot stopping point 2026-06-21 13:30:01 -06:00
Drew T 1b3fc8b20a docs(phase-21): trip hand-off — grinder running, worker staged for remote drive
- automation manager built + validated (fleet 58.82->58.98%, 136/136 byte-identical, 3 real
  matches banked x134). Grinder running autonomous/token-free; worker /loop = Drew remote-driven.
- runbook docs/automation-runbook.md; CURRENT_PHASE hand-off; formal PhaseEnd awaits gate-2.
2026-06-21 13:13:17 -06:00
Drew T 66d1af0e09 docs(phase-21): automation runbook + status tool (grinder running; worker = remote-driven)
- docs/automation-runbook.md: launch/monitor/stop for the grinder (live) + worker /loop;
  safety invariants (byte-gate sole arbiter, git crash-safe, auto_stop kill switch);
  remote-management note (Drew can remote in -> no bulletproof keep-alive needed).
- auto_status.sh: show grinder + worker heartbeats, backlog, phase-21 checkpoints, daemons.
2026-06-21 13:12:04 -06:00
Drew T 8de18e617d feat(phase-21): T4/T5/T6 — worker validated (func_80164530 x134), grinder, orchestrator
- T4 worker Workflow validated: 3-target wave, all self-MATCH; whole-binary gate banked
  func_80164530 x134 (5 register-pins + offset-fold + scheduling). fleet 58.90->58.98%.
  §20 holds (1/3 survived the whole-binary gate); the 2 near/failed -> backlog.
- T5 grinder: token-free permuter of backlog near-misses -> gate_stage; gate correctly
  rejected a permuter over-prediction (integrity intact). auto_supervisor DRIVER-param.
- T6 orchestrator.py: ROI pool rotation (prep/finish/status); gate_stage flock serializes
  grinder+orchestrator builds. Sandbox: detached daemons build fine, foreground needs override.
2026-06-21 13:06:34 -06:00
Drew T bee543722d feat(phase-21): worker gate — +1 fns x1 propagated (fleet 58.98%) 2026-06-21 13:04:40 -06:00
Drew T 76c5feb116 feat(phase-21): worker gate — +1 fns x1 propagated (fleet 58.94%) 2026-06-21 12:50:16 -06:00
Drew T 2d7926e9f6 feat(phase-21): T4/T5 tooling — worker Workflow + wave selector + permuter grinder
- tools/workflows/worker_wave.js: parallel toolkit-aware drafter agents (xHigh) per target,
  each writes its best matching C to the drafts dir (gate run by the orchestrator after).
  args parsed robustly (harness may serialize to a JSON string — §20 gotcha).
- tools/wave_targets.py: select a wave batch from the fuel manifest (pool/region/N, skips
  backlog walls). tools/grinder.py: token-free permuter grinder — permute backlog near-misses
  -> gate_stage -> bank; STOP/heartbeat, idles for new worker near-misses, supervisor-friendly.
2026-06-21 12:41:14 -06:00
Drew T c2e93e50cd feat(phase-21): T3 — backlog ledger + shared gate stage (validated, +0.08%)
- tools/backlog.py: near-miss ledger (.run/backlog.jsonl + docs/backlog.md ranked by
  reach->closeness->size; drop-now-matched filter, P9). tools/gate_stage.py: the shared
  deterministic spine both worker+grinder call (canon_resident_calls -> cast_call_sites ->
  sig_unify -> harvest_verify --chunk 1 -> dedup_propagate --auto-from -> backlog -> summary).
- validated end-to-end on a real 30-draft m2c sample: banked func_801710DC + propagated x134
  (dedup 1506->1508), 29 near-misses logged+ranked. make check-all 136/136 byte-identical
  (R22); fleet 58.82%->58.90%; dedup-check 1508 validated/0 failed (G3/P9).
- capped-fns reframe (R14): they're matched-inline (propagation-blocked), not draftable ->
  deferred to the worker; not claiming the unbanked +0.3% (P9).
2026-06-21 12:32:21 -06:00
Drew T 5674587bcb feat(phase-21): T2 — fuel prefetch, Ghidra-C cache complete for the ROI pool
- headless DecompileFunctions over 263 uncached ROI-pool fns (MCP stopped, R23):
  263 ok / 0 fail / 0 no-func. cache 300 -> 563. ROI-pool fuel now 325/325 cached
  (28/28 giants, 310/310 reach-134, 7/7 capped) -> the unattended run never needs MCP.
- FIX DecompileFunctions.java: key output by entry address func_<UPPERHEX>.c (Ghidra's
  raw-import default is FUN_<lowerhex>, which every consumer + the existing 300 files
  miss). Renamed the 263; the tool fix makes re-prefetch reproducible.
- build_fuel_manifest.py --emit-prefetch: reproducible uncached-ROI-pool addr-list
  (for the T7 runbook). 349 reach-1 x1-leverage fns intentionally excluded (P9).
- ghidra/ db churn is the MCP-stop no-op save (R23 restart-noise, not staged).
2026-06-21 12:00:47 -06:00
Drew T 971f95f7c8 feat(phase-21): T1 — fuel manifest + R14 giants byte-verification
- tools/build_fuel_manifest.py -> .run/fuel_manifest.json: the automation target
  pool. reach RECOMPUTED from the current 134 overlay sigs (distinct-overlay
  h_exact), not trusted from cached fields (R14). Classes GIANT/WAVE/STRUCT/PINS/
  STUB/O0/O1 + the CAPPED recovery pool; cached? flags Ghidra-C gaps for T2.
- 803 live func_ stubs; 310 reach-134, 486 reach-1. R14: 28/28 giants verified
  genuinely byte-identical x134 (label honest, unlike the -O0 cluster). -O0
  nuance: 8/9 overlay-local but func_8013C08C is reach-134 (real exception).
- T2 workload scoped: 253 reach-134 uncached; 171/223 tractable reach-134 uncached.
- CURRENT_PHASE.md: T1 logged. No MCP, no build mutation.
2026-06-21 11:50:31 -06:00
Drew T e803651f0c feat(phase-20): close — cap tool + T3a %lo crack + 3 byte-proven findings; fleet 58.00%->58.82% (v1.19.0)
- THE CAP TOOL tools/cast_call_sites.py (§17a-1 callee-cast): per draft, callee whose canonical
  TU sig != draft's intended sig -> decl->canonical + call-site cast ((ret(*)(args))func_X)(args)
  (codegen-neutral; gcc folds the cast of a known symbol -> direct jal). Recovered 6 of T6 batch-1's
  33 gate-fails byte-identical (5 x134 + 1 local). Pipeline canon_resident_calls -> cast_call_sites
  -> sig_unify -> harvest_verify --chunk 1.
- T3a CRACKED (not cited): the %lo-folding indexed-global -O0 residual falls to the array-of-STRUCT
  idiom (extern Struct base[], sizeof==stride, base[i].field; NOT *(T*)(&sym+i*stride) which
  materializes &sym). func_8013B7AC banked. Reusable idiom fleet-wide. cookbook §18/§20.
- THREE byte-proven findings (R14): (1) the wave cap is the DEF-side loose-typing wall, not
  callee-conflicts (the §20 "~33" projection was wrong); (2) the -O0 cluster is OVERLAY-LOCAL,
  not reach-134 -> the ×134 rollout is INVALID; (3) data-cast is MOOT (0 real data conflicts).
- T1 typedef type-lift (+0.35%, 9 fns x134); T2 exemplar_miner router; T4 codegen map (cookbook §20);
  T6 batch-1 (+0.28%, 8). dedup 1485->1513/0 failed. check-all 136/136 BYTE-IDENTICAL (R22);
  0 NON_MATCHING (G4). fleet 58.00% -> 58.82%.
- rule R30 (capture context-dependent knowledge before a fresh session). worklog -> logs/Phase20.md.
- bumps 1.18.0 -> 1.19.0
2026-06-21 10:26:34 -06:00
Drew T f1a7e518f5 feat(phase-20): T3a CRACKED — %lo-folding -O0 falls to the array-of-struct idiom
- the %lo-folding indexed-global -O0 residual (§18, long "irreducible") is CRACKED: declare
  the global as "extern Struct base[]" (sizeof == stride) and access "base[i].field" -> gcc
  keeps sym a symbol_ref and folds %lo(sym+off) into the store, byte-matching. The failing
  forms "*(T*)(&sym + i*stride)" materialize &sym as a value first, blocking the fold.
- func_8013B7AC byte-gated + banked (x1 — the -O0 cluster is overlay-local, R14 finding above);
  the idiom is reusable fleet-wide for ANY indexed-global access (-O0 or -O2).
- cookbook §18 ("%lo-folding indexed global — CRACKED") + §20 placeholder closed.
- the other 9 -O0 stubs: idiom applies but need per-fn -O0 drafting (3 quick ones tried, failed
  on -O0 codegen detail; 4 are large) -> backlog/autonomy fuel. clean rebuild d19c9580 (R22).
2026-06-21 09:58:01 -06:00
Drew T d5ddbc587d docs(phase-20): log consolidated state — cached ≤90-ins tail byte-exhausted; T5 is the lever 2026-06-21 09:31:20 -06:00
Drew T a978081b2d docs(phase-20): data-cast is moot (R14) — 0 data-type conflicts on the tail
- byte-proven: 0 drafts declare an extern struct/union data conflict; the lone apparent
  DATA-conflict (func_8016A8FC/D_800AE620) was a typedef-REDEFINITION (Blk20 inline vs
  engine_types.h, same type) masking the def-sig loose-typing wall. Stripping the inline
  typedef compiles but still byte-mismatches. data-cast = code for 0 cases -> not built.
- cookbook §20 DATA-conflict note corrected; fleet unchanged 58.82%.
2026-06-21 09:30:22 -06:00
Drew T 7f2f314896 docs(phase-20): -O0 ×134 rollout is INVALID — cluster is overlay-local, not reach-134 (R14)
- byte-proven: only 1/134 overlays shares ov_SC01_077's -O0 cluster bytes; the 6 matched
  -O0 fns reference per-overlay data addresses (func_8013B568 -> D_80187270 in ov_SC01_077
  but 0x80182B04 in ov_SC01_005), so they're overlay-LOCAL, not ×134-shareable. The
  Phase-18/19 'reach-134' label conflated function-present with byte-identical (R14).
- the §18 split infra (rollout tool + Makefile %_o0.o pattern + PROVIDE .ld) was built and
  byte-validated to compile/link a 2nd overlay; the whole-binary gate then exposed the
  per-overlay-data wall -> reverted (the finding is the deliverable, P9).
- -O0 rollout = ×1 per-overlay-unique re-matching (~0.2-0.3%), NOT +0.6% free -> SKIP.
  cookbook §18 corrected; fleet unchanged 58.82%, 136/136 byte-identical.
2026-06-21 09:26:44 -06:00
Drew T 6c5bd1c4a9 feat(phase-20): cast_call_sites.py — the §17a-1 callee-cast cap tool; 6 recovered, fleet 58.63%->58.82%
- tools/cast_call_sites.py: per draft, callee whose canonical TU sig != draft's
  intended sig -> decl->canonical (kills in-TU conflicting types, keeps symbol in
  scope) + call-site cast ((ret(*)(args))func_X)(args) (codegen-neutral: gcc folds
  the cast of a known symbol -> direct jal). Pure --in/--out; whole-binary gate is
  the arbiter (G3/P9). Pipeline: canon_resident_calls -> cast_call_sites ->
  sig_unify -> harvest_verify --chunk 1
- recovered 6 of T6 batch-1's 33 gate-fails byte-identical (func_80153C44/8015CF58/
  801711FC/80161BE0/801683D8/8015F948); 5 propagated x134 + 1 local; check-all 136/136
- R14 CORRECTION of cookbook §20's '~33' projection (byte-proven): the batch is NOT
  mostly callee-conflicts — ~6 callee (cast-recovered), ~18 DEF-side loose-typing wall
  (def's matchable sig conflicts with the unchangeable caller-canonical; caller-side
  fix BLOCKED because INCLUDE_ASM declares no symbol -> a shared macro's internal
  extern is load-bearing in the 133 stub overlays), + a small DATA tail. The def-side
  needs re-drafting under the caller-canonical sig (future wave), not a text transform
- cookbook §20 RESOLUTION + SETUP tool inventory (R16/R21/R30); CURRENT_PHASE log (R28)
- fleet 58.63% -> 58.82%, dedup 1501 -> 1508, 0 NON_MATCHING (G4)
2026-06-21 02:04:39 -06:00
Drew T ea3ef7fb80 docs(phase-20): record candidate R30 — capture context-dependent knowledge before a fresh session
- Drew (2026-06-21): cookbook entries / byte-verified findings / distillations / doc corrections
  must be written DURING the producing session (fresh sessions get only summaries, lose the detail).
  Defer only mechanical/continuable work. Extends R16. Memory: capture-knowledge-before-fresh-session.
2026-06-21 01:25:46 -06:00
Drew T 2070cdb000 docs(phase-20): T4 — distill the codegen map (cookbook §20) + R14 loop-guard correction
- cookbook §20: the wave-at-scale GATE CAP (match_one over-predicts; the in-TU loose-typing
  call-graph wall; all recovery levers fail; the only fix is automating §17a-1 call-site casts =
  Phase-21 cap lever) + the stale-.o diagnostic lesson
- CONFIRMED-irreducible classes w/ byte-evidence: §10 store-vs-load (8014F2E0/80150528), §10
  hoist-vs-remat (80149374/801493D0); NEW classes: IV-combine combine_givs (80177AD4),
  hoisted-invariant prologue order (80177F84), the -O1 build-infra class (80161A90)
- operational gotchas: Workflow args-as-array, harvest_verify --chunk 1
- R14 correction to §17a: loop-guard (8012C2D0) is STRENGTH-REDUCTION/IV-final-value addressing,
  NOT operand-order
- SETUP §tooling: exemplar_miner.py (T2) + build_engine_types.py typedef-lift (T1) (R21)
- T3c cited (written up), T4 done; CURRENT_PHASE handoff refreshed
2026-06-21 01:24:01 -06:00
Drew T 187f7200d4 docs(phase-20): multi-session handoff — resume point for T3a/T4/T5 + the cap-recovery (Drew: continue, don't close)
- phase NOT closing; CURRENT_PHASE.md is the fresh-session resume (plan approved -> resume autonomously)
- remaining prioritized: (1) the SS17a auto-call-site-cast recovery tool (the diagnosed cap;
  recovers T6 batch-1's 33 reach-134 ~+1% + lifts every wave), (2) T4 distill, (3) T3a %lo,
  (4) T5 Ghidra-C regen + more waves, (5) -O1/-O0 split-file build-infra
- drafts preserved under .run/drafts-t6-cn/; reusable wave recipe recorded (incl. the args-as-array
  + harvest_verify --chunk 1 gotchas)
2026-06-21 01:17:48 -06:00
Drew T dab6ccb96f docs(phase-20): T6 batch-1 recovery — diagnose the loose-typing call-graph wall (gate cap)
- byte-verified: the 33 wave drafts that match_one-MATCH but fail the whole-binary gate are the
  loose-typing wall in the CALL GRAPH (a callee declared with conflicting types across overlay
  sites -> draft extern hits in-TU 'conflicting types'); match_one over-predicts (standalone +
  masks jal/hi/lo). All recovery levers fail (sig_unify/canon_draft_decls/fix_arity/no-proto/
  strip-externs); the only fix is per-site fn-ptr casts (SS17a) = Phase-21 recovery tooling
- diagnostic lesson: a failed in-TU build leaves a STALE .o (trust the whole-binary SHA, not objdump)
2026-06-21 01:12:18 -06:00
Drew T 9991e4d6be feat(phase-20): T6 batch-1 — harvest wave 41/48 match_one, 8 banked (7 x134), fleet 58.35->58.63%
- Ultracode wave on 48 cached reach-134 <=90-ins targets: 41/48 match_one MATCH (85%)
- whole-binary byte-gate: 8/41 banked (canon 3 + sig_unify 5); 7 propagated x134 + 1 local;
  dedup 1494->1501, check-all 136/136, +0.28%
- FINDING (R14/P9): match_one->gate gap ~80% this batch = EXTERN-CODEGEN-COUPLING (a callee's
  declared sig changes the call codegen; match_one uses the draft externs, the gate uses the
  overlay decls). The 33 lost are byte-correct vs the wrong externs -> need a callee-extern-
  canonicalize pass (NOT sig_unify, which rewrites the def-sig and regresses byte-correct drafts)
- the 7 DIFFs = gcc-class diagnoses: T3c store-vs-load (8014F2E0/80150528) CONFIRMED unsteerable
  (sched.c); SS10 hoist-vs-remat (80149374/801493D0); IV-combine (80177AD4); hoisted-invariant
  order (80177F84); 80161A90 = -O1 (build-infra)
2026-06-21 00:57:04 -06:00