Commit Graph

13 Commits

Author SHA1 Message Date
Drew T 4b0f4d1003 feat(phase-15): #3 types-header propagation — +8 struct-using shared fns; fleet 51.77% -> 52.08%
- fix tools/build_engine_types.py to be ADDITIVE: merge new source types with the
  types already in engine_types.h (a re-run after a prior --strip was destructive —
  it dropped the ~34 already-migrated types and broke the build; caught + reverted)
- engine_types.h 34 -> 42 named types (the 8 from wave-1's struct-using matches);
  inline defs stripped from ov_SC01_077.c (byte-neutral)
- dedup_propagate --auto-from: 8 of 15 struct-blocked matches now self-contained ->
  propagated across all 134 overlays (byte-gated); 7 remain (anonymous/typedef'd types)
- R22 CLEAN fleet rebuild: 136/136 byte-identical; dedup-check 1333/0
- zero agent tokens (deterministic)
2026-06-18 18:03:44 -06:00
Drew T be0ceae55d feat(phase-15): T6 wave-1 (50-agent) harvest — 27 shared fns matched, 9 propagated fleet-wide; fleet 51.41% -> 51.77%
- 50-agent Ultracode callee-sig-aware harvest of 300 reach-134 shared stubs in
  ov_SC01_077; whole-binary byte-gate verified 27 (26 + 1 canon-recovery), 273
  genuine compiler-internal residual (recovery recovered only 1 -> confirmed permuter-class)
- dedup_propagate --auto-from: 9 self-contained matches propagated across all 134
  overlays (byte-gated); 15 verified use overlay-local types -> pending engine_types.h
- R22 CLEAN fleet rebuild: 136/136 byte-identical, 0 extract fails; dedup-check 1325/0
- NOTE: blind-draft shared-core harvest has hit diminishing returns (~9% gate yield);
  next work pivots to giants (match_one) + permuter + deterministic types-header propagation
2026-06-18 17:21:54 -06:00
Drew T 85b3599e5b feat(phase-15): T6 groups-of-10 missing harvest — fleet 48.02% -> 51.41% (over half)
- Drew's "groups of 10 instead of 100+": rebuilt the harvest workflow to process the 375
  never-drafted shared stubs (leaf-first) in SEQUENTIAL WAVES of <=10 agents (intra-wave
  retry). Result: 0 dead batches (vs 33 from a 48-batch burst), ~42 drafts/60s (vs ~1/45s)
  under the sustained server-side throttle; 374/375 drafted (+1 gap-fill found).
- gate: 85 verified -> 79 propagated fleet-wide (7 struct/local skipped), 134 overlays
  byte-identical; dedup 1237 -> 1316 groups, dedup_integrate --check 1316 validated / 0 failed.
- fleet REAL substantive ~164k -> 175,212; byte-identical 48.02% -> 51.41% (OVER HALF the
  matchable code across all 136 binaries rebuilds byte-identical from source).
- R22 clean rebuild: 136/136 byte-identical, 0 extract failures.
- cookbook §12: the rate-limit-gentle sequential-waves-of-10 pattern + leaf-first ordering.
2026-06-17 18:38:03 -06:00
Drew T 5c5e9fb973 feat(phase-15): T6 struct follow-up — shared engine_types.h unblocks 37 struct-using fns; fleet 46.27% -> 47.70%
- tools/build_engine_types.py -> src/shared/engine_types.h: extract the 34 named struct/union
  types from ov_SC01_077.c (forward decls + defs in source order; consumes trailing
  __attribute__((packed)) + aliases through the ';'). 0 same-name-different-layout collisions.
- engine_core.h includes engine_types.h; the 34 inline defs stripped from ov_SC01_077.c
  (byte-neutral — type defs emit no code; verified byte-identical).
- dedup_propagate.py: relax the struct guard to skip only inline named-struct/typedef
  DEFINITIONS (header-type USAGES + anonymous local structs now propagate); add engine_types.h
  to the compile pre-filter so struct-using bodies resolve.
- 37 struct-using shared fns propagated fleet-wide (skip 41 -> 4 overlay-local typedef'd types),
  134 overlays byte-identical; dedup 1183 -> 1220 groups, 1220 validated / 0 failed.
- fleet REAL substantive 157,541 -> 162,462; byte-identical 46.27% -> 47.70%.
- R22 clean rebuild: 136/136 byte-identical, 0 extract failures.
2026-06-17 15:25:44 -06:00
Drew T 57a0a88f37 feat(phase-15): T6 tail — quarantine decl-recovery propagates 42 more; fleet 44.64% -> 46.27%
- canon_draft_decls recovery on the combined 505 quarantined drafts (285 small + 220 medium),
  now canonicalizing against the much-larger banked symbol set: 47 verified (no API), 42
  propagated fleet-wide (struct/local-type skipped), 134 overlays byte-identical; dedup
  1141 -> 1183 groups, dedup_integrate --check 1183 validated / 0 failed.
- fleet REAL substantive 151,908 -> 157,541; byte-identical 44.64% -> 46.27%.
- R22 clean rebuild: 136/136 byte-identical, 0 extract failures.
- scoped the struct-using-shared-fn lever (read-only): 34 named struct/union types in 077,
  ZERO same-name-different-layout collisions -> a shared-types-header is a clean path to
  propagate the ~100 struct-using fns (next).
2026-06-17 15:06:29 -06:00
Drew T db4fddc44e feat(phase-15): T6g medium band — harvest+propagate 65 shared fns; fleet 42.14% -> 44.64%
- medium harvest (callee+data-sig-aware) on the 293 still-stub call-heavy fns (nins 41-150):
  30-agent Workflow + a 3-target gap-fill (mandatory produced-vs-expected reconcile). Gate:
  73 verified (25% — these are the harder GTE/matrix/regalloc functions; 220 quarantined to
  .run/drafts-T6c-fail for a later recovery pass).
- dedup_propagate.py: broaden the struct/union guard to skip ANY struct/union-touching body
  (not just definitions) — agents used colliding generic struct names in params/casts/externs
  too. 37 skipped; 65 propagated fleet-wide, 134 overlays byte-identical; dedup 1076 -> 1141
  groups, dedup_integrate --check 1141 validated / 0 failed.
- fleet REAL substantive 143,322 -> 151,908; byte-identical 42.14% -> 44.64%.
- R22 clean rebuild: 136/136 byte-identical, 0 extract failures.
- open tail (paused here): 285+220 quarantined drafts (regalloc/struct/§10 residuals), the
  struct-using shared fns (shared-types-header follow-up), 077-unique remainder, T7 giants.
2026-06-17 13:21:36 -06:00
Drew T bdfb86dc4f feat(phase-15): T6 v2 — data-sig-aware harvest + struct-guarded propagate; fleet 31.22% -> 42.14%
- gen_harvest_targets.py: now also resolves canonical DATA-symbol declarations per target
  (scan the .s for D_XXXX refs -> `extern <type> D_XXXX;` from banked code) in addition to
  callee function sigs — the §14c(c) fix at the source, so agents declare both functions and
  data correctly and avoid the dominant remaining conflict class.
- v2 harvest (callee+data-sig-aware) on the 591 small still-stub call-heavy fns + a gap-fill
  agent for 6 targets dropped by a connection-closed-mid-response failure (the retry-wave
  misses truncated-non-null results). Gate: 306 verified (ov_SC01_077 1554 -> 1248 stubs).
- dedup_propagate.py: skip any struct/union-touching body — agents named structs with colliding
  generic names (`struct S`/`struct vec`) inline, so two macros' types redefine/conflict when
  instantiated in one overlay (ov_SC01_000 abort). 34 skip; struct-using shared fns stay banked
  in 077 (shared-types-header follow-up).
- 281 functions propagated fleet-wide (134 overlays each, every overlay byte-gated); dedup
  795 -> 1076 groups, dedup_integrate --check 1076 validated / 0 failed.
- fleet REAL substantive 105,764 -> 143,322; byte-identical 31.22% -> 42.14%.
- R22 clean rebuild: 136/136 byte-identical, 0 extract failures.
- cookbook §12: gap-fill is now a MANDATORY post-step for every multi-agent run (reconcile
  produced-vs-expected: missing + truncated drafts) before gating.
2026-06-17 11:59:33 -06:00
Drew T 1d94f9ec96 feat(phase-15): T6d — deterministic decl-canonicalization recovers 19 more shared fns; fleet -> 31.22%
- tools/canon_draft_decls.py: rewrites a quarantined harvest draft's extern/data
  declarations to the banked-consistent set (functions from engine_core.h DEFINE
  bodies / extern decls / inline defs; data from existing `extern <type> D_XXXX;`),
  so a draft whose BODY is byte-correct (passes match_one) but whose DECLARATION
  collided with already-banked code can re-gate cleanly. The whole-binary
  harvest_verify byte-gate stays the sole arbiter — a semantics-changing
  canonicalization just fails and reverts.
- recovered 19 of the 90 match_one-MATCH-but-conflicting quarantined failures
  (no API — immune to the rate limiting), propagated fleet-wide, all 134 overlays
  byte-identical; dedup 776 -> 795 groups, 0 failed.
- fleet byte-identical 30.48% -> 31.22%; ov_SC01_077 1574 -> 1535 stubs.
- R22 clean rebuild: 136/136 byte-identical, 0 extract failures.
- the remaining ~70 match_one-MATCH residual needs definition-widening (§14c(d),
  narrow-return) or true data-type resolution — queued for the next pass, with the
  un-started medium band (430) and the 54 genuine instruction-misses (permuter/§3a).
2026-06-17 00:30:45 -06:00
Drew T ef1cf9f05e feat(phase-15): T6 pass-1 — callee-sig-aware harvest of the call-heavy shared core; fleet 25.38% -> 30.48%
- tools/gen_harvest_targets.py: callee-signature-aware target manifest (the §14b
  extern-conflict wall-breaker). Pre-resolves each callee's EXACT signature from
  engine_core.h DEFINE bodies + extern declarations + inline defs, so drafts never
  emit a conflicting extern. Lifted call-heavy gate yield from the §14b-naive 16%
  to 60-67%.
- ov_SC01_077: 142 functions banked (1716 -> 1574 INCLUDE_ASM stubs) via the
  Ultracode parallel-draft + harvest_verify byte-gate loop (cookbook §12). The
  draft Workflow was throttled by server-side API rate limiting (stopped at
  ~246/616); the incremental gate -> dedup_propagate loop is deterministic and
  throttle-immune, so the wins banked steadily regardless.
- dedup_propagate --auto-from ov_SC01_077: ~132 functions propagated fleet-wide
  (134 overlays each), every round per-overlay byte-gated; dedup 644 -> 776 groups,
  dedup_integrate --check 776 validated / 0 failed.
- fleet REAL 85,651 -> 103,217; byte-identical 25.38% -> 30.48%.
- R22 clean rebuild: make clean + extract all 136 + check-all = 136/136 byte-
  identical, 0 extract failures.
- cookbook §14c: the wall-breaker + the 4 conflict flavors (defined / declared /
  DATA-symbol / narrow-return-widening) + the never-git-checkout-mid-harvest
  hazard + the rate-limit-resilient incremental loop. Residual = the data-symbol
  conflict class (structural canonical-decls fix queued for next pass) + the
  un-started medium band (430).
2026-06-17 00:12:23 -06:00
Drew T 9d5c087054 feat(phase-15): T5b pass-2 leaf-first harvest (+34 shared fns); fleet 25.38%
- leaf-first pass: 52 pure-LEAF unmatched-shared funcs (no jal/%hi/%lo -> match_one is an
  exact gate); 5-agent Ultracode Workflow -> 36/52 verified (69% yield, vs 16% mixed) ->
  34 propagated fleet-wide (644 groups, dedup-check 0 failed); R22 clean 136/136
- fleet REAL 81,093 -> 85,651; byte-identical 24.05% -> 25.38%
- pure leaves now ~exhausted; remaining shared core is call-heavy (extern-conflict wall)
- CURRENT_PHASE.md: RESUME HANDOFF for a fresh session (T6 callee-aware harvest, T7 giants
  + per-overlay unique, then T8 PhaseEnd) — no PhaseEnd this session (Drew)
- ghidra/ churn NOT staged (R23)
2026-06-16 22:18:06 -06:00
Drew T 2ac97878b2 feat(phase-15): T5b — Ultracode harvest pass 1 (+49 shared fns); fleet 24.05%
- 25-agent Workflow drafted + match_one-verified 300 small unmatched-shared functions;
  harvest_verify byte-gate in ov_SC01_077 verified 49 (byte-identical d19c9580);
  dedup_propagate spread them fleet-wide (610 groups, dedup-check 0 failed)
- fleet REAL 74,527 -> 81,093; byte-identical 22.14% -> 24.05%; R22 clean 136/136
- yield 16% = the call-heavy hard tail: match_one masks jal/HI16/LO16 (false MATCH on
  call-heavy fns); real misses are the extern-type-conflict wall (draft callee extern
  != the callee real macro signature). Leaf fns matched; call-heavy = open-ended tail.
- cookbook 14b documents the wall + the path forward (leaf-first, callee-signature-aware
  drafting, link-gated per-fn check)
- ghidra/ churn NOT staged (R23)
2026-06-16 21:16:46 -06:00
Drew T aefe2eac18 feat(phase-15): T5a — bulk-propagate 553 shared functions fleet-wide (3.82% -> 22.14%)
- dedup_propagate --auto-from ov_SC01_077: propagated 553 already-matched, self-contained
  shared engine functions across all 134 overlays, each byte-gated. 9 functions skipped
  (use 077-local struct types -> not mechanically liftable; honest, P9). One R22 CLEAN
  rebuild: 136/136 byte-identical.
- fleet REAL 947 -> 74,527; byte-identical 13,132 -> 76,174 / 344,010 = 22.14%; 561 dedup
  groups, dedup-check 561 validated / 0 failed
- tool hardening for fleet scale: registry SHORTHAND (vram + binaries; dedup.us.yaml ~4k
  lines vs ~77k verbose); per-overlay apply (one read/write per file); gate-only-changed;
  skip already-registered (additive/resumable); compile pre-filter (body must build with
  common.h alone); group_members() reader (both forms) in dedup_integrate + progress;
  progress _DEDUP_CACHE (fleet 6m -> 7s)
- the first bulk attempt fail-closed-reverted on a local-typed body (byte-gate working) ->
  added the compile filter; cookbook 14a documents the 5 fleet-bulk lessons
- ghidra/ churn NOT staged (R23)
2026-06-16 20:35:39 -06:00
Drew T c716eb15e1 feat(phase-15): T4 — fleet progress roll-up + fix cross-report double-count
- progress.py: refactor to report(binary) + set_binary() + a per-binary .s index
  (fleet run 6m38s -> 7s); add --fleet -> docs/progress.fleet.md (deterministic,
  source-derived per-binary table + fleet totals). Single-binary output byte-unchanged.
- fleet now: 947 REAL / 959 LINKED / 13132 byte-identical / 344010 matchable = 3.82%
  across 136 binaries (main + resident + 134 overlays)
- dup_report --cross: HONESTY FIX (R14) — an onboarded overlay is both a named BINARIES
  entry AND a .run/sig.ov_* glob hit; it was counted twice, inflating collapsible bytes
  to 58.7M once the fleet was onboarded. Dedupe by alias -> accurate 9366 h_exact groups
  / 28.6M collapsible (matches the Phase-11 figure)
- Makefile: wire progress.py --fleet into make report (main-only block)
- docs: progress.fleet.md (new), duplicates.cross.md regenerated, SETUP inventory
- make report green; dedup-check 8 validated / 0 failed
- ghidra/ churn NOT staged (R23)
2026-06-16 20:00:19 -06:00