BFM-decomp/docs/wave-metrics.md

Crack-wave performance metrics

Evolvable reference (docs/ layer). Started 2026-08-04 (Phase 30, sessions S33–S37). Append one row per wave. The point of this file is that the wave harness has knobs — the prompt, the concurrency shape, the model tier — and their effects are only visible if the numbers are recorded in one place. Two of them were already measured to matter a lot (§ below).

Sources, all derivable — do not hand-transcribe (R33):

• counts/tokens/duration: the Workflow completion notification (agent_count, subagent_tokens, duration_ms)
• per-agent wall-clock + parallelism: min/max(timestamp) per agent-*.jsonl under ~/.claude/projects/.../subagents/workflows/<runId>/
• banked: the whole-binary gate log (.run/s*_gate.log) — the gate is the arbiter, never the agents' claim (G3/P9)

The table

wave	script	targets	templ ins	agents	tokens	wall	parallelism	median / slowest agent	claimed	banked	+members
1 (S33)	s33w.js	17	26,227	25	6.87M	169 min	4.8×	33 / 63 min	15/17	15 (13+2 rec)	65
2 (S34)	s34w.js	13	37,943	19	4.51M	208 min	2.7×	20 / 79 min	11/13	10 (9+1 rec)	18
3 (S35)	s35w.js	13	17,644	14	2.73M	140 min	2.1×	9 / 51 min	13/13	13 (12+1 rec)	21
4 (S36)	s36w.js	14	16,844	17	2.99M	136 min	2.5×	14 / 48 min	14/14	14 (13+1 rec)	26
5 (S37)	s37w.js	16	16,884	19	2.71M	82 min	3.8×	9 / 50 min	16/16	16 (16+0 rec)	26
6 (S38)	w6.js	16	50,596	24	5.20M	71 min	7.4×	25 / 40 min	14/16	7	25

All waves: Sonnet drafters with an Opus escalation rung (§136i), whole-binary byte-gate as the sole arbiter, R22 clean-fleet 140/140 after every banked batch.

Finding 1 — the PROMPT is the lever, and the agents write it

Bank rate 76% → 77% → 100% → 100% → 100%, with the models and the gate held constant. The only variable was the prompt's extra block.

The jump came from STEP 0: grep -rn "<MAGIC>" src/ using a distinctive literal from the target .s, placed ahead of engine_core.h in §136c's search order — because §136c's first two steps are same-TU/shared-header scoped and structurally cannot reach a banked twin in a different overlay's TU, which is where the big template classes live.

That step came from a wave-2 agent's index_gap report. The wave schema asks every agent for index_hit / index_gap; harvesting those into the next wave's prompt is what compounds. By wave 4 most agents cited step 0 by name and reported index_gap: none.

Honesty caveat: wave 3–5 targets also trended easier (smaller families, more banked twins available), so the prompt is not solely responsible for 100%. The claim that holds without qualification is the direction and the mechanism, not the exact percentage.

Finding 2 — pipeline() vs batched parallel(): 40% faster on 14% more targets

Waves 1–4 split each wave into two parallel() batches, which is a hard barrier — batch 2 cannot start until batch 1's slowest agent finishes. Visible directly in the launch timestamps as 37–50 minute dead gaps at each boundary.

	wave 4 parallel()	wave 5 pipeline()
targets	14	16
wall-clock	136 min	82 min
parallelism	2.5×	3.8×
median agent	14 min	9 min

The batching existed to dodge S10's 30-wide server throttle, but the harness already caps workflow agents at min(16, cores-2) and these waves are 13–16 targets — so it bought nothing and cost a barrier. Use .run/s37w.js's execution block for every future wave.

The remaining floor is real: the slowest single agent is still ~50 min at 200–300 turns. That is genuine match_one iteration (one agent tested 470+ statement orderings on a 793-instruction function). Wall-clock cannot go below the slowest chain, so the lever there is target selection, not concurrency.

Finding 3 — economics

Roughly 170k–300k tokens per banked head across waves 3–5 (the stable regime). But a head is not the unit of value: the unit is the head plus its propagated members, and the sweep rate is a property of the FAMILY, not the wave — wave 3 swept 21/21 while wave 2 swept 18/165, because wave 2's big families are per-location variants that do not template (settled by probe: remapped member is BUILD OK and byte-different ⇒ genuine per-member codegen). Rank targets by open templatable instructions derived from corpus.stubs, and expect the sweep yield to be bimodal, not average.

Finding 4 — a perfect gate is a signal that the prompt rules landed

Wave 5 is the first 16/16 banked with ZERO reconcile. Waves 1–4 each needed 1–2 declaration reconciles after the gate; wave 5 needed none. The difference is that by then the prompt carried both §138 rules (the (macro-shape, TU-shape) pair, and the reconcile-direction line-number check) plus the wave-4 lesson that two targets sharing a TU can create each other's conflicts. The reconcile lane is the fallback, not the plan — when it goes quiet, the prompt is doing its job. Lifetime reconcile record across the session: 21/22.

How to add a row

After a wave's gate + sweep land, before the commit:

# tokens / agents / duration: from the Workflow completion notification
# parallelism + per-agent spread:
python3 - <<'PY'
import json, glob, datetime
d = "<runId dir under ~/.claude/projects/.../subagents/workflows/>"
T = lambda s: datetime.datetime.fromisoformat(s.replace("Z","+00:00"))
ag=[]
for f in glob.glob(f"{d}/agent-*.jsonl"):
    ts=[T(r["timestamp"]) for r in map(json.loads, open(f)) if r.get("timestamp")]
    if ts: ag.append(((max(ts)-min(ts)).total_seconds()/60, min(ts), max(ts)))
wall=(max(a[2] for a in ag)-min(a[1] for a in ag)).total_seconds()/60
tot=sum(a[0] for a in ag); ag.sort()
print(f"{len(ag)} agents · wall {wall:.0f}m · agent-time {tot:.0f}m · {tot/wall:.1f}x · "
      f"median {ag[len(ag)//2][0]:.0f}m · slowest {ag[-1][0]:.0f}m")
PY

Related: cookbook §138 (the reconcile/search rules these waves produced), memory wave-prompt-seed-step0-and-gaps (what to seed the next prompt with), docs/effort-map.md (Ultracode/breadth policy), docs/calibration.md (templatability measurements).

Finding 5 — rank waves by INSTRUCTIONS BANKED, not heads; and the knob that moved in wave 6

Wave 6 deliberately changed the difficulty knob: 50,596 templatable instructions offered vs ~16,800, median target 438 instructions vs 143, max 947 vs 397, and most targets in the has_mid_jr (jump-table-carve) class. Its bank rate — 7/16 heads — therefore is not comparable to waves 3–5's 100%, and should never be read next to them as if the knob were fixed.

By instructions, the bigger pool won:

	wave 5	wave 6
pool offered	16,884	50,596
heads banked	16/16	7/16
ins in banked heads	2,753	2,204
propagated members	26	25
total instructions banked	~7,200 (est.)	10,616 (exact)
pool realised	~43% (est.)	21%

Wave 6's 10,616 is exact — make report moved 12,368,236 → 12,378,852, matching the hand-derivation to the instruction. Wave 5's figure is an estimate (its per-family member breakdown wasn't recorded — record it from now on).

So wave 6 banked ~45% more instructions while banking less than half as many heads, because a bigger head carries more instructions and its family propagates at the same cost per sibling — func_8017FEE0, one 299-instruction head, became 4,485 instructions across 15 siblings for ~0 agent tokens. Head count flatters small-target waves; instruction weight is what moves the fleet.

The metric to beat is POOL REALISATION (21%), not bank rate. A wave banking 16/16 of a 17k pool is worth less than one banking 7/16 of a 50k pool. And wave 6's 21% is a floor, not a ceiling: it lost an entire gate cycle to the _reload_corpus defect (§139), and 9 of its 16 targets are still open and routable rather than refuted.

Finding 6 — a wave's tally is a COVERAGE claim, and needs its own assertion

Wave 6's first gate printed BANKED 5 / FAILED 1 over 16 drafts. Ten produced no verdict at all because the child crashed and the driver only grepped stdout for two line-prefixes. Nine of the ten were claiming MATCH, and one of them (func_8017EA84, 579 ins) banks byte-identical under the fixed path — a real match reported as nothing.

Any row in this table is a coverage claim. Before recording one, confirm the gate accounted for every draft (banked + failed + no-verdict == drafts, now asserted in .run/s6f_gate.py). Full post-mortem: cookbook §139.

---

Wave S40-1 (2026-08-05) — 8 targets, the first wave ever aimed at the open-only h_norm clusters

Pool verified BEFORE the wave (R14). The frontier report's cluster pool was carried with an explicit "not verified" caveat, and its other headline claim (the whale open in 4 SC07 overlays) had already proved 3/4 wrong. Measured from the sigs instead:

	clusters	fns	ins	multiplier
claimed	1,689	5,956	326,261	2.7×
measured	1,677	5,795	319,755	3.68×

Within 2–4% on size, and the multiplier is better than claimed. Verify each claim separately: the same document was right here and wrong about the whale.

Result — and the two numbers say different things.

stage	result
match_one close=0	8 / 8
whole-binary gate, first pass	5 / 8
after deterministic recovery	8 / 8 — ~0 agent tokens

All three first-pass failures were integration plumbing, each a different known lever, ZERO codegen walls:

fn	gate error	lever
func_801802EC	redefinition of morph_lerp	strip the §77 probe layer — the draft carried types + a static inline helper so match_one could compile standalone; the real TU already defines them. Scaffolding is not part of the bank.
func_8018B238	conflicting types for D_80115158	recover_giant — the draft declared it file-scope as a struct array while the TU declares u8[] block-scope inside other functions; block-scoping the draft's externs removes the collision.
func_8017EF54	conflicting types for func_8017EF54	§37/§124 def-side asm-label alias — TU declares void f(void) for no-arg callers, byte-true def takes s32 in $a0; no-prototype escape illegal once a param promotes, so the definition takes a private C identifier + __asm__("func_8017EF54").

The lesson for reading any future wave row: the gate number is not the close-rate. 5/8 measured integration, not matching. Run the recovery ladder before recording a wave's yield, or the table will under-report the drafters and send the next wave hunting compiler walls that are not there.

Cost: 1.31M subagent tokens, 8 agents, 0 errors. Idioms harvested: cookbook §144 (literal spelling picks the immediate encoding). Defect found: .run/ghidra_c/func_8017EF54.c is a stale decompile of the WRONG function — the prefetch cache is not trustworthy per-entry.

---

Wave T (P31 S54, 2026-08-17) — the first wave to bank EVERY byte-verified draft

	wave S (S53)	wave T (S54)
cards	71	71
drafted instructions	6,532	6,557
gate groups	2	5
agents / tokens / wall	82 / — / —	87 / 8.71M / 70 min
standalone MATCH	70 (99%)	70 (99%)
banked at the gate	63 of 71	70 of 70 — zero drops
mechanical siblings banked behind it	—	49 (4,954 ins)

What changed, and it is the only thing that changed: the wave script grew a Reconcile phase. After a gate group's drafts finish, ONE agent per group runs the deterministic ladder over that group's own slate — reconcile_slate --apply → resolve the refusals with the §183 playbook → pregate_check — re-verifying every edit with match_one, before any rebuild. Wave R had banked 18 of 45 on its first slate and needed a separate recovery lane afterwards (§181: 26 of the 27 rejections were byte-perfect drafts). Wave T dropped nothing, in five groups, on first contact.

Two selector changes made the pool worth more per agent (§193-A, --one-per-gid + --rank total): the naive fleet-wide draw had put 22 of 44 skeletons in the wave TWICE (sibling overlays at two addresses), and ranking gate groups by face mass ignored the sibling mass hanging behind each card. Deduped and re-ranked, the same 71 agent-slots carried 19,266 instructions of potential — 6,557 drafted plus 12,709 staged for the free remap — of which 4,954 sibling instructions banked mechanically the same session.

Cost line for the ledger: 8.71M subagent tokens over 87 agents, 0 errors, ~124k tokens per banked head — but the head is not the unit: 70 heads + 49 mechanical siblings = 11,511 instructions banked for one wave's tokens, against wave 6's 10,616 and wave 5's ~7,200.

Harvest attached (19 more agents, 1.97M tokens): 71 index_gap reports → 9 CONFIRMED laws (cookbook §193-A…I), 5 REJECTED, and 61 answered by sections that already existed. That last number is the one to act on: the knowledge base knew, and the agents did not find it. §193-A is the mechanical cause and its fix — the card's exemplar/sibs pointers are drawn from the atlas's OPEN set by construction (0/34 banked, and 0% at any maturity), while the atlas's own MATCHED-pool twin (seed.ref, 4/4 banked on a control draw) was being discarded by the card builder.

Waves U and V (P31 S54) — the Reconcile phase holds, and the card's pointers get fixed

	wave T	wave U	wave V
cards	71	73	70
drafted instructions	6,557	6,567	6,503
gate groups	5	7	9
standalone MATCH	70 (99%)	73 (100%)	69 (99%)
banked at the gate	70 of 70	73 of 73	68 of 69
agents / tokens / wall	87 / 8.71M / 70 min	86 / 7.42M / 45 min	85 / 8.88M / 71 min
mechanical siblings banked	49	34	25
index_gaps -> confirmed / rejected / already-covered	71 -> 9 / 5 / 61	64 -> 14 / 5 / 44	67 -> 14 / 9 / 76

Three data points, one steady result and one non-trend. The steady result is BANKING: the Reconcile phase has now delivered 70/70, 73/73 and 68/69 against wave R's 18-of-45 before it existed. The non-trend is the 100%: wave U's perfect draft rate did not repeat, and 99% -> 100% -> 99% is noise on 70-card samples. Do not report the 100% as a new floor.

Wave U's real signal is the COST, not the rate. Same models, same gate, same laws — but the cards carried seed_ref, the atlas's banked twin (§193-A), for the first time. 15% fewer tokens and 64% of the wall-clock for one more card. Several agents filed their class as "none (direct twin transcription)". Reading a banked body beats deriving from the .s, and the atlas had been computing that pointer all along while the card builder discarded it.

THE NUMBER TO WATCH IS GATE GROUPS: 5 -> 7 -> 9 for the same ~6,500 instructions. That is the fleet-wide 60-200 mass band thinning under --rank total. Each group is a whole-binary rebuild, so the gate cost per banked instruction is rising even while the bank rate holds. At ~12 groups, re-open the band question against the velocity ledger rather than drawing another wave on reflex.

Wave W (P31 S54) — the UNKNOWN lane, and the selector that was overriding the prompt

	wave V	wave W
cards / drafted ins	70 / 6,503	71 / 6,500
gate groups	9	3
drafts per rebuild	7.8	24.3
standalone MATCH	69/70	71/71 of the agents that ran (2 lost to server rate-limiting)
banked	68	68
mechanical siblings	25	0
index_gaps -> confirmed / rejected / already-covered	67 -> 14 / 9 / 76	68 -> 4 / 3 / 41

The draw is the story. Wave W's first draw came up SHORT — 63 cards, 5,647 ins, 10 gate groups — continuing 5 -> 7 -> 9. Probing the atlas rather than accepting it: UNKNOWN held 607 of the 1,179 in-band members, more than every other lever lane combined, and build_wave_atlas's default --levers excluded it — while the wave PROMPT tells every agent that UNKNOWN drafts like any other lane. The tool was silently overriding the doctrine, and the "thinning frontier" was a default flag.

Two costs, both real, recorded so the next draw is a choice and not a reflex:

• Zero siblings. UNKNOWN groups are mostly singletons: waves T/U/V got 49/34/25 functions free from the remap, W got none. Face mass and delivered mass are different numbers (§193-A/--rank total), and the UNKNOWN lane maximises the first at the expense of the second.
• Harvest yield fell to 4 from 14. Partly the seeding (three sessions of laws off the table), partly that this lane's functions are ordinary — the atlas could not name a lever precisely because there was no distinctive one to name.

Gate-group concentration is the wall-clock lever overnight: 3 rebuilds instead of 9 is roughly an hour back per wave, and wall-clock — not tokens — is what bounds a session that runs waves in series.

Wave Y (P31 S55) — the recovery lane earns its place on its first outing

	wave X	wave Y
cards / drafted ins	65 / 6,549	70 / 6,550
gate groups	4	5
standalone MATCH	64/65	69/70
banked at the gate	62	64
recovered afterwards	—	+4 (§200, zero agent tokens)
mechanical siblings	1	+4
wave total	63	72

The recovery lane changed the wave's yield by 6%, and it ran during a total API outage. Twelve agents died across two rounds of 529 Overloaded without starting, so the five gate drops were fixed by hand — and the five refusal classes collapsed into one move (§200: bind a private C identifier to the link symbol instead of negotiating over the shared one). Four re-gated and banked; the fifth had already banked. The lane cost nothing and returned four functions.

Read that number carefully before generalising it. Recovery is cheap here because the drops were DECLARATION refusals on already-byte-verified drafts — §181's class, where the work is done and only the plumbing failed. A recovery lane over genuine near-misses is a different economy: wave Y's one near-miss (func_8017FCFC, closeness 3) was NOT recovered, and its previous agent had already attributed the residual to sched1 by elimination. Budget recovery against gate drops; treat near-misses as permuter fuel unless a specific new law applies.

Session S54/S55 (2026-08-17/18) — six waves, one table

wave	cards	gate groups	drafted	banked	siblings	wave total	harvest (conf/rej/covered)
T	71	5	70 (99%)	70/70	49	119	9 / 5 / 61
U	73	7	73 (100%)	73/73	34	107	14 / 5 / 44
V	70	9	69 (99%)	68/69	25	93	14 / 9 / 76
W	71	3	71/71 run	68/70	0	68	4 / 3 / 41
X	65	4	64 (98%)	62/63	1	63	7 / 2 / 56
Y	70	5	69 (99%)	64 + 4 recovered	4	72	5 / 8 / 53

Banking is the stable result; drafting was never the problem. Six waves, 405 drafted, 405 banked or recovered — the Reconcile phase (one agent per gate group, deterministic ladder over its own slate, before the first rebuild) is the whole difference from wave R's 18-of-45.

Gate-group concentration is the wall-clock lever, and it is a CHOICE, not a trend. 5 -> 7 -> 9 looked like a thinning frontier; it was --levers excluding UNKNOWN. Adding it gave wave W three groups at 24.3 drafts per rebuild. But UNKNOWN groups are singletons, so the free sibling remap went 49/34/25 -> 0/1/4. Face mass and delivered mass are different numbers; pick per wave.

Harvest yield falls as it should. Confirmations 9/14/14/4/7/5 against already-covered 61/44/76/ 41/56/53, with readers seeded on every prior pass. Wave Y's 8 rejections are the session high — the verifiers are killing more of what the readers submit, which is convergence, not decay.