11 Commits

Author SHA1 Message Date
Drew T b23300fc21 feat(phase-33): A2 reporting instruments — make sig-main rewritten (tools/main_seed_ends.py: main's game-code boundaries DERIVED from the link map + objects, tiling asserted; 809 fns / 45,150 ins), progress.py weighs main by the build-derived sig and EXITS with no sig (R32), the digest's oracle clause derived live (0 phantom / 0 truncated / 0 pad-tail), backlog.linked_closed retires the LINKED-range legacy row (0 open), dup_report on the derived sig; CORRECTED main denominator 45,150 (Ghidra's flow boundaries left 3,628 words of game code unowned) — fleet instr 13,492,113 / distinct 5,820,205, all 100%; make report BINARY=main EXIT=0, audit-digest OK; SETUP §6.8 + P33 A2 rows (R21) 2026-09-06 18:46:52 -06:00
Drew T b6fd3fff28 docs(phase-32): T4b hand pass — func_80032A74 PROVED at 1 by producer census (§501-M); ghost_census.py + cc1_dumps.sh repair; backlog tie-break repair
- main:func_80032A74 (422 ins, closeness 1): the residual is ONE reload-time slot at sp+0x48 (u16 draft = 422/422 code, DIFF 22 frame
  rows; s16 draft = DIFF 1 at idx 244 lh vs lhu). Every post-parameter slot producer enumerated from gcc-2.7.2 (reload1.c:658 ghost
  alter_reg / caller-save.c:249 area / reload1.c:879 invalid-equiv / reload1.c:3499 spill_stack_slot) and refuted on the bytes: combine's
  newi2pat ghosts re-derive a narrow load and the site is lhu (no lb, no double load); a save area without sw/lw needs sched.c:4962
  staleness and no register-only insn shares a block with a call; $t0 holds no pseudo; LO mult results retry into GR_REGS. Verdict
  PROVED at 1 (pin kept with the verdict; ledger WALL-PROVED).
- NEW mechanism measured: local-alloc.c optimize_reg_copy_2 (tmp = x; tmp op= c; x = tmp) mints a ghost with stale refs, but after
  regclass -> GR_REGS, allocated, no slot (P13/P14). 18 isolated reproducers, 0 draft variants; cookbook §501-M; accelerators (12).
- tools/ghost_census.py (new): ghosts in a .lreg dump with their class (ST_REGS => slot). tools/cc1_dumps.sh: prints the .frame line,
  ins count, spill lines and the census; the under-counting standalone-(use) grep is gone. SETUP rows (R21).
- tools/backlog.py: load_best kept the EARLIEST record at equal closeness (docstring said latest) — the S84 row never rendered; fixed.
- CURRENT_PHASE.md: S84 log + refreshed 🛑 checkpoint (rows (b)(c)(d) next, then T5). No src/config/carve change; fleet check-all
  218/218 rc 0 at the S84 preflight.
2026-09-06 11:44:40 -06:00
Drew T 369dd14f4f fix(phase-30 S44 I.1d): the module class reaches every enumerating consumer
- family_hseq: widened from src/ov_*+sig.ov_* to every non-main binary (resident + md_*); the map
  now carries 139 binaries incl. resident (was overlays-only — which is exactly why the R36 gate's
  CHECK 4 could never see them). Self-count uses the SAME widened globs (cannot drift).
- progress --weighted :647 + audit_frontier :57: + sig.md_* globs.
- corpus.sig_is_independent: md_* sigs are sig_image-signed => independent (R34 trust).
- backlog alias regex + prefetch_fleet (md_* derived from splat configs) + dedup_propagate
  (reads modules.mk alongside overlays.mk — excluding modules would re-create the SC07
  invisible-work bug one class over).
- VERIFIED: family map regenerated with resident (139 binaries); audit-binaries OK over 140;
  all six tools parse.
2026-08-06 10:57:17 -06:00
Drew T fa122cf62f fix(phase-30 S43): permuter takes func_8017C6F4 63->42; the "rumour row" was an ADDRESS COLLISION
- THE FLOOR MOVED: permuter_ils on the S42 draft -> masked 65->44 (cycle 1, flat over 5 warm
  restarts); re-measured in match_one terms 63 -> 42 mismatched, 947/947 ins. First movement
  after ~40 hand probes, and it came from repairing an instrument (S43-1), not from new C.
  Draft preserved + allowlisted: .run/s43/func_8017C6F4.ils44.c; logged at closeness 42.
- THE S42 "rumour" CLAIM WAS WRONG (R14): the 2026-07-01 row HAS an artifact, it IS on disk,
  and it reproduces exactly (14 mismatched of 15 target ins, SIZE-MISMATCH/redraft). It is a
  near-worthless draft on a DIFFERENT BODY: 0x8017C6F4 is 15 ins in ov_SC03_010/011/013 and
  948 ins in ov_SC03_126/003 + ov_SC04_021 + ov_SC05_019 (§148-E, ledger side).
- THREE ledger defects fixed: (1) load_best keyed on ADDRESS ALONE -> the two bodies merged and
  the lower ABSOLUTE closeness won, so 14-of-15-wrong (7% correct) masked 63-of-947 (93%);
  now sub-keyed by known nins, legacy rows unchanged. (2) binary=null defaulted to ov_SC01_077,
  where the fn does not exist AT ALL, and "not an open stub" was read as "banked" -> today's
  result was invisible to render/grinder/target-selection (absent != done, R32/R34); now derive
  binary from the draft path + only drop when closed everywhere it exists. (3) `log` had NO
  --binary flag -- the root cause of every null; added + derived in append_record.
- IMPACT DERIVED, NOT ASSERTED (R37): replaying the pre-fix selection = 836 -> 837, 1 appeared
  (func_8017C6F4 nins=947), 0 vanished. One row today; the mechanism would eat every future one.
- PROBED AND NOT BUILT: relative-closeness ranking (only 24/836 rows carry closeness+nins, and
  the two orderings agree 14/15 on those). Documented in the log instead.
2026-08-05 16:06:16 -06:00
Drew T e03494dc1f feat(phase-30): T0d — backlog _open_stubs derives from corpus.stubs (STUB_RE deleted, R33); ledger verified already-clean
- Filter + prune existed since 07-24; jsonl already compacted (the tree's uncommitted edit was
  S25's un-committed prune output; prune today 1350->1350/0 dropped). Stale rows were worklist.md's.
- Oracle agreement: 0 divergence across 131 ledger binaries (my first probe compared names vs int
  addrs — R35 on my own instrument). Hex-case canonicalized in the membership test (R32).
- Parity proven post-change: load_best 1350 == 1350. §83 doctrinal caveat stands.
2026-07-30 16:39:05 -06:00
Drew T 2b38c68333 feat(phase-29): SESSION-21 T1-T3 — the frontier measured, the family-exemplar wave, 3 tool fixes
- T1 FRONTIER MEASURED (zero-token, R35: family map regenerated on fresh sigs first — it was
  stale by ~657 banked members): 36,020 stubs / 2,345,599 weighted ins remain, and only
  9.0% are h_exact-FREE. PROPAGATION IS TAPPED (238 distinct classes / 3,245 instances);
  22,498 distinct classes / 1,680,097 distinct ins is what is actually left. The mass is FLAT
  across all 139 binaries (~300-550 sub-500 stubs each) -> "pick the best overlay" is not a
  strategy. .run/s21_frontier.py + .run/s21_frontier.json
- T2 THE AXIS IS THE FAMILY, NOT THE LOCATION: 1,342 substantial h_seq families /
  1,298,135 templatable ins = 55% of ALL remaining weighted instructions. Routed by blocker:
  jr/§81 181 fams (33.6%) · DRAFT-with-cached-Ghidra-C 91 (28.6%) · DRAFT-modal 1,023 (27.5%)
  · zero-crack 45 (6.5%) · permanent walls 2 (3.9%). Live+cached+non-wall in ov_SC01_077 = 54
  families / 589,502 ins, value steeply concentrated (top 24 = 96%).
  .run/s21_targets.py + .run/s21_targets.json + .run/s21_draft_pool.json
- T3 WAVE 1 LAUNCHED: tools/workflows/family_core_wave.js (NEW) — 24 xHigh drafters, one per
  family exemplar, stake 575,488 templatable ins (24% of remaining). Supersedes worker_wave.js
  for family work: carries each target's family STAKE, encodes the four §58/§87 integration
  rules at source (splat D_<UPPERHEX> not Ghidra DAT_; never invent a symbol; canonical callee
  sigs; leave decl plumbing to the ladder), and requires symcheck.py on any claimed MATCH.
- T3b LADDER HYGIENE, both SESSION-20 carry items fixed — one defect, two masks: a byte-NEUTRAL
  transform was left in the tree when it banked nothing. family_sweep's --normalize-self-decls
  backstop only fired on MISMATCH (left 123 files of dead diff on a 0/123 run); gate_stage's
  ARITY undo narrowed to src/shared/ and left ~40 TUs. Both now restore the full snapshot when
  NOTHING banked (no banks to preserve => the splice hazard cannot apply). §61 on the success path.
- T3c BACKLOG addr DEFECT fixed (R32/R33): new addr_of() derives the address from `name`,
  assert_addr_coverage() fails loud on an unkeyable row, append_record fills both directions.
  Found a latent bug doing it: load_best() keyed on `addr or name`, splitting one function into
  two "best" records. Keyable rows 128/1,701 (7.5%) -> 1,701/1,701 (100%).
2026-07-26 23:37:35 -06:00
Drew T ef85803b1f feat(phase-29): backlog prune — compact the append-only near-miss log + wire into make report
The near-miss ledger (.run/backlog.jsonl) is append-only, so it filled with already-banked noise:
6,867 rows, ~98% banked. load_best()/render() already filtered on READ (docs/backlog.md was correct),
but the raw log drifted stale and every render re-scanned all 6,867 rows against the stub oracle.

- backlog.py: new `prune` subcommand — atomic rewrite (temp + os.replace) to load_best()'s output
  (drop-now-matched P9 + best-per-addr collapse). Idempotent. 6,867 -> 1,704 open near-misses.
- Makefile: `backlog.py prune` wired into `make report` (BINARY=main block) so the ledger tracks
  reality every cycle instead of drifting.
- Finding (Drew's question): crack waves DO log every non-byte-match to the backlog durably
  (gate_stage copies best_draft -> .run/backlog_drafts/). BUT the `closeness` field is UNRELIABLE —
  byte-correct drafts (match_one MATCH) are logged with closeness>0 (e.g. func_8012F49C logged 29,
  actually MATCH). And a reach-N function's draft is overlay-SPECIFIC (per-location symbols), so the
  backlog is a messy recovery source vs the fresh per-wave stranded drafts. Integration-recovery
  should consume the fresh wave-dir strandeds, not re-derive from the backlog.
2026-07-24 11:36:54 -06:00
Drew T eec96f76a4 feat(phase-29 Task-12): structured failure telemetry + durable permuter-winner save
The permuter-autopsy prerequisite (Drew-directed). Captures the WHAT/WHERE of every
non-match so the classifier/autopsy (Task-13) can mine it, not just a scalar closeness:
- masked_diff.structured_diff(): the per-instruction masked residual [(idx,mine,tgt),...]
- match_one --json: emits {status,closeness,nins,residual} (refactored to share the primitive)
- backlog FIELDS += residual, passes_tried
- gate_stage.match_one_closeness returns + logs the residual on every near/fail (verified
  end-to-end: a near-miss's structured residual now lands in backlog.jsonl)
- grinder: durably save the winning C to .run/permuter-winners/ BEFORE gate_stage banks
  (this session's lesson — 3 permuter wins were lost to a mid-flight revert)
2026-07-20 19:50:14 -06:00
Drew T b1c58d7668 docs(phase-26a): A10 wave 1 — closeness-0 wall re-test CONFIRMED REAL (0/958 bank)
Re-gated all 958 closeness-0 open-stub backlog drafts through the FIXED gate
across 135 binaries in parallel: banked=0, near=957, failed=71. The closeness-0
backlog is genuine whole-binary near-misses, NOT tooling misses -- match_one's
isolated closeness==0 systematically overstates whole-binary bankability, and the
repaired gate recovers none. P9: a re-confirmed wall is as valuable as a dissolved
one. (The audit's tooling-walls were already banked by A3f/g/h + A9b, +2.1% instr.)

backlog.py: env-gated BACKLOG_NO_RENDER so parallel workers skip the render race
(append is atomic) -- backward-compatible parallel-safety. backlog.md refreshed
with the re-test's whole-binary-informed scores.
2026-07-14 23:54:16 -06:00
Drew T e91859fb4a feat(phase-23): grinder per-binary fix (5-layer) — unlock non-077 near-miss grinding
The grinder/backlog pipeline was ov_SC01_077-hardcoded 5 layers deep (same class as the
T7 lora_grind bug). Fixed all so the permuter grinder can process a non-077 near-miss:
1. gate_stage.append_record stores the source "binary"
2. backlog.FIELDS keeps it (else append_record dropped it)
3. backlog.load_best/_open_stubs is fleet-aware: a fn matched in ov_SC01_077 but
   propagation-stuck stays OPEN in its overlay, so it surfaces via that record instead
   of being dropped as "matched" (the grinder must SEE it to grind it)
4. p16_permute.setup takes the target binary's asm-subdir (was hardcoded 077)
5. grinder resolves per-binary asm + gates grouped by binary + allows unknown nins
Backward-compatible: legacy records (no binary) default ov_SC01_077.

Validated end-to-end: the 3 fresh reach-134 close=1 ov_SC01_000 fns now surface, resolve
to ov_SC01_000's asm, and gate via ov_SC01_000.

TWO byte-evidenced findings (redirect the fuel strategy):
- the reach>=2 close=1 fuel is MODEL semantic-misses, not permuter fuel: func_8012E27C's
  target is "return 1" but the 7B drafted an empty "void f(void){}" (corpus overfit
  empty-leaf); func_8012BF4C/AD64 are trivial sw/sh setters drafted empty. A corrected
  draft banks them (+3 byte-identical via the fixed gate, @commit:0326); the permuter cannot
  add a missing return/store. Lever = corpus-v3 leaf variety, not the permuter.
- x reach is propagation-capped: the 3 are inline-matched in ov_SC01_077_a.c (the stuck-
  local cap) -> dedup_propagate "nothing to propagate" -> banked x1. Lever = dedup-collapse.

check-all 136/136 throughout. docs/gen2-mips-matching-model.md + CURRENT_PHASE updated.
2026-06-30 00:58:13 -06:00
Drew T c2e93e50cd feat(phase-21): T3 — backlog ledger + shared gate stage (validated, +0.08%)
- tools/backlog.py: near-miss ledger (.run/backlog.jsonl + docs/backlog.md ranked by
  reach->closeness->size; drop-now-matched filter, P9). tools/gate_stage.py: the shared
  deterministic spine both worker+grinder call (canon_resident_calls -> cast_call_sites ->
  sig_unify -> harvest_verify --chunk 1 -> dedup_propagate --auto-from -> backlog -> summary).
- validated end-to-end on a real 30-draft m2c sample: banked func_801710DC + propagated x134
  (dedup 1506->1508), 29 near-misses logged+ranked. make check-all 136/136 byte-identical
  (R22); fleet 58.82%->58.90%; dedup-check 1508 validated/0 failed (G3/P9).
- capped-fns reframe (R14): they're matched-inline (propagation-blocked), not draftable ->
  deferred to the worker; not claiming the unbanked +0.3% (P9).
2026-06-21 12:32:21 -06:00