tools/rtu_shadow.py: --wave X records rtu_match's verdict for every draft of a not-yet-gated wave
(mapped to its binary through the SHARD's targets file, never by bare name — 54 of wave fa's 470
cards share a name across binaries); --join X after the gate commit prints rtu-verdict x outcome,
P(bank | rtu MATCH), the false-negative rate and current-vs-inverted build counts (R41 denominators).
build_wave_atlas: a (binary, fn) whose latest resolver verdict is STAGED/BANKED/GATE-REJECTED is not
drawn — its body already matches at the real TU and on symbols; DIFF/CC1 verdicts stay drawable
with the CURRENT closeness the resolver demoted them to.
A tell-lever card named a lever and then made the agent go find its sites: it said
"extend-tell" and nothing about where or how many. atlas_features already counts the
detectors per function into .run/feat.<bin>.jsonl at atlas time, so this is a JOIN, not a
computation — one dict load per binary in the wave.
build_wave_atlas attaches 'tells' {extpair, dupselect, magic_div, sign_lh, sign_lb} to every
card that has a nonzero one — not just tell-lever cards, because a sll/sra pair site or a
repeated select is worth knowing whatever lever drew the card. Measured on a live draw: 85 of
185 cards carry counts.
api_agent._fuel renders them as a CHECKLIST rather than a hint, which is the point: the
counts come from the TARGET's own bytes, so a draft emitting fewer has provably missed sites
and should go looking before spending a turn elsewhere.
Verified: a card with {extpair 3, dupselect 2, sign_lh 1} renders all three with the zero
fields omitted; NEGATIVE CONTROL — a card with no tells renders no TELLS line at all.
Takes effect on the next draw + the next shard (api_agent is spawned per draft).
Neither extreme was right. Ignoring generations lets a card that has failed five waves
compete with one nobody has ever drafted; filtering to a single generation starved the fleet
to 46 cards. So generation becomes the PRIMARY ORDERING and the existing mass/count criterion
breaks ties, at both levels of the assembly:
* gate groups holding never-drafted cards rank ahead of all-retry groups (before the
--max-bins truncation, so an untouched group is never cut for a fat retry group);
* within a group, untouched cards are taken before retries.
Wave SIZE is untouched — only the order changes — so the fleet stays full while the scarce
never-drafted work always goes out first.
Verified on a live draw: gen0 2 · gen1 2 · gen2 14 · gen3 2 · gen4 3 · gen5+ 340. It took
EVERY card below generation 5 (all 23 available) and filled the remaining 340 slots from the
5+ pile, which is the whole point.
The gen0 count is 2 because wave dw drew the last 51 untouched skeletons an hour ago. That is
the campaign's real state: essentially everything drawable has now been drafted at least
once, and ~635 distinct skeletons have refused. The remaining work is levers, not draws.
--generational (or BFM_GENERATIONAL=1) draws ONLY the lowest generation present: no function
gets a 2nd draft while any drawable function still lacks a 1st. draw_count is derived from
the prior wave card files already being read for the already-waved filter.
MEASURED BEFORE SHIPPING, and it changed the plan. With every cap opened — no band, no
--max-bins, all levers, whole fleet minus main — generation 0 is:
232 candidates -> 46 distinct skeletons (186 are same-gid siblings the remap banks free)
against ~681 drawable skeletons in total. So "3,926 never-drafted stubs" was three illusions
stacked: ~961 are main's LINKED PsyQ stubs and data blobs (not decomp targets at all), most of
the rest are same-gid siblings that one exemplar banks mechanically, and 2,119 were already
drawn in earlier waves. Making generational the DEFAULT starved the fleet from ~640 cards to
46 — so it is opt-in, for a priority pass over the untouched population, not standing policy.
The real shape of the endgame, stated plainly: of ~681 distinct drawable skeletons, only 46
have never been drafted. The other ~635 refused at least one draft each. That is a LEVER
problem — distillation, A-prop, the o0/jtbl carves, the permuter — not a resampling problem,
and no amount of drafting throughput addresses it.
Fired as wave dw (51 cards / 1,990 ins across 36 binaries) so the untouched population is
drafted today rather than left as a policy.
1. MAIN LANE — the largest single block of unfinished work was drawing 32 cards a wave.
main_lane.draw() never passed --max-bins, so it inherited build_wave_atlas's default of
12 gate groups — a cap that exists because each group costs a whole-binary rebuild, and
main's own --only-bins docstring says the opposite applies to it: "main is gated ONCE per
SLATE, so main has no per-TU gate cost and --max-bins can be large". Nobody passed it.
Measured cost: main banked ~19 stubs/hour against 1,291 remaining while the overlay lane
ran 650-card waves beside it. Now --max-bins 400 (MAIN_MAX_BINS overrides), and the lane
shell draws 600 cards with 600 workers instead of 200/150.
2. TWO LANES GATE, SO READ BOTH LOGS — a defect I introduced this session. The in-flight
exclusion derived "this wave has been gated" from .run/gater.log only, but the main lane
gates its own waves into .run/main_lane.log. Every m## wave therefore looked permanently
in flight and main's draw lost 425 cards to an exclusion meant for work in progress.
3. TAIL_DONE_FRAC 0.80 -> 0.65. At 0.80 the fleet runs 2-3 overlapping waves at ~250
req/min; the residual troughs are the gap between one wave draining and the next ramping.
65% keeps 3-4 waves overlapping. Stragglers keep their full 700s grace in the finisher
thread — this changes when the NEXT wave starts, never what lands.
4. ATOMIC ATLAS WRITE. The lanes read .run/atlas.json at every draw and atlas.py dumped
straight onto it, leaving a truncated file readable for the length of the write. Now
written to .tmp and os.replace'd.
Context for 1-3: the atlas both lanes draw from is dated 08-23 01:13 — two days stale,
predating ~4,600 banks — and its regen chain is running now (its own R32 assertion caught a
stale family map first and named the fix).
They were held out because each has a cheaper deterministic owner: the family remap banks a
`remap` card for zero tokens, `plumbing` belongs to recover_integration, `needs-autopsy`
wants a look before a draft. That reasoning priced AGENT TOKENS as the scarce resource. On a
free model the scarce resource is CARDS — holding 1,219 instances out of every wave to
protect a budget that does not bind starves a 2,000-agent fleet.
lever-not-in-lane drops 1,383 -> 164 in a live draw. The deterministic lanes still run and
still reach these first; the byte gate refuses a duplicate, so a card a remap already banked
costs one wasted shard, never a wrong bank.
MEASURED over 18 consecutive waves. Consecutive card sets: ck->cl 239/239 shared, co->cp
238/238, cv->cw 222/222, db->dc 208/209 — and the "different" pairs still shared 50-90%.
Yield alternated in lockstep: 47.6% / 3.8% / 35.3% / 3.6% / 29.9% / 3.7% / 43.4% / 14.6%,
because the duplicate wave gates AFTER the original banked its cards. Half of all drafting
went to work already in flight, and it read as campaign decay.
ROOT CAUSE: --retry-unbanked returns "previously waved but still an OPEN STUB" cards to the
pool — right in principle, unfinished work is not spent work. But the pre-draw for wave N+1
runs WHILE wave N drafts, when none of wave N's cards have been gated, so every one of them
is still an open stub and the filter hands the whole wave back. The ranking then rebuilds it
card for card. The filter knew about "banked" and "not banked" and had no notion of "in
flight".
FIX: a wave is finished when its GATE has run, and the gater already says so in its own log
(R33 — derive from the artifact that exists). Tags with no GATE line stay excluded; a tag
with no gate line whose cards are older than 6 h was killed, and is released so nothing is
locked out forever.
THROUGHPUT, same commit — the draw was setting the campaign's request rate:
* --max-bins 24 -> 160. Concentrating a wave into 24 gate groups was a CPU-economy choice
made when CPU was scarce. It is not: a live gate runs at load 2.7 of 32 cores (8%), one
harvest_verify at --chunk 1. Meanwhile the drafting fleet — the resource actually bounded
by the free-model clock — got 196 cards out of 699 available. Re-drawn with 160 bins:
644 drafts / 46,590 ins across 136 binaries, 3.3x the wave for the same gate economics.
* TAIL_DONE_FRAC 0.95 -> 0.80. Overlapping at 95% still left 25% of minutes under 20 req/min,
because a wave's last 5% is its SLOWEST 5% and 12 stragglers cannot fill a fleet. Handing
off at 80% starts the next ramp with ~40 agents still working. Stragglers keep their full
700s grace in the finisher thread; nothing is cut short.
* --queue-depth 2 -> 4, so a bigger wave's longer gate never parks the drafter.
Arithmetic this is aimed at: req/min = agents-in-flight x ~0.8 (a 16k-token turn at ~30
tok/s emits few requests). 644 cards x two overlapping waves puts the fleet where the
endpoint has already been measured to sustain it — 764 req/min for 15 min at 8% 429s, peak
2,755 in one minute.
corpus.stubs() misreports for a binary while a gate has draft bodies substituted into
its sources (R35) — but only for THAT binary. The blanket refusal cost far more than
it saved: the gater runs almost continuously, so nearly every fresh draw was refused
and the drafter fell back to PRE-DRAWN waves. Measured at 15:20 — wave br refused,
wave bj (drawn hours earlier) drafted instead.
That is worse than idle time: a pre-drawn wave carries the OLD draw-time defaults, so
every feature landed today — the tells quota, the jtbl quota, the -O0 filter, the
oversize filter — was silently not reaching the fleet, while the logs showed healthy
410-shard waves.
Which binaries are mid-gate is not a guess: gate_stage and gate_main hold
.run/auto/gate.<bin>.lock for exactly that window, so a non-blocking test-lock answers
it per binary. Those are added to EXCLUDE for that draw; the refusal survives only for
--only-bins draws where every requested binary is busy.
Verified live against a running gate: the draw that would have been refused now
returns 40 cards.
Three defects found by running it, all of the same family — a check that is true about
the wrong thing:
1. FALSE BANKS. The first accounting asked corpus.stubs('main') whether each name was
still a stub; that returns {addr: Stub(symbol=...)} — a dict keyed by INT. Comparing
a NAME against a set of ints is always True, so the lane reported '12 banked of 12'
from a gate that banked nothing and committed nothing. Credit now requires BOTH the
INCLUDE_ASM line gone from the working tree AND main re-checked byte-identical.
2. INNOCENT DRAFTS DYING WITH A DECL CLASH. gate_main deliberately refuses to bisect a
COMPILE conflict (right for a human caller, wrong for an unattended lane): the first
live batch hit a conflict on a symbol that was in the TU and in NO draft, so there
was nothing to drop and 40 innocent drafts died with it. The lane now halves the
slate — a main rebuild measures ~15 s, so bisecting is cheaper than discarding.
3. THE DRAW GUARD REFUSED EVERYTHING. build_wave_atlas refused to draw whenever any
gate was in flight (R35: corpus.stubs misreports mid-gate) — but only for the
binaries being gated. Main's sources are touched by gate_main alone, so a main-only
draw now watches gate_main and every other draw keeps the blanket refusal.
Failed drafts are parked with a try count instead of discarded (a failed draft is
evidence), capped at 2 so an unbankable body cannot spin the lane.
Live: 13 main functions banked, main byte-identical at 143dbb89, stubs 1713 -> 1700.
TELLS QUOTA (Drew approved): a dedicated tells wave drew only 70-87 cards — a full
40-minute drafting slot at a quarter of a default wave — because the 5-80 size cap and
the tells pool cannot fill more. Tells now ride inside ordinary waves with a 60-card
quota, same as jtbl. The size cap moved into the draw itself: tell-lever members above
--tells-max-ins (80) are not drawn at all, because the measured bank rate is 27-40%
at 5-80, 10% at 81-120, 1% at 121-200 and 0% above — those 383 members / 51,941 ins
are idiom_serial's work, and the skip counter names it (R45).
A QUOTA IS A FLOOR UNLESS IT IS ALSO A CEILING. First test: putting the tell levers in
the default list let them win the ranked fill too, and a 300-card wave came back 122
tells (41%). The size cap held; the mix did not. Tells now enter through the quota or
not at all.
GATE JOBS 24. The quotas deliberately pull cards from binaries outside the ranked gate
groups, so a measured draw went from ~24 groups to 63 — 63 whole-binary rebuilds per
wave, five serial batches at 12 jobs. The box is 32 cores at ~6% (load 3.1) with 39 GB
free. Lands via restart_gater_when_idle.sh so no sweep is killed mid-flight.
Correction to the record: the live draw already passed --max-bins 24 (plus
--one-per-gid and --exclude-bins main). An earlier measurement of mine used the tool's
default of 12 without those flags and read as 'max-bins is the cap' — it is not;
--one-per-gid is, and deliberately: it defers same-skeleton siblings to the free
deterministic remap instead of paying an agent twice.
Adding the lever alone was inert: a jtbl card is one-per-binary by construction, so
jtbl cards are maximally UN-concentrated, and the gate-group ranking exists precisely
to pack many drafts behind one rebuild. Measured — a mixed draw offered 71 jtbl
candidates and selected ZERO. It would have done that every wave, forever, while
looking enabled.
--jtbl-quota (default 6) seeds the wave with the largest eligible jtbl cards before
the ranked fill, so the lane advances inside ordinary waves instead of needing a
dedicated one (a jtbl-only draw is ~12 cards and would idle a 2,000-agent fleet).
Cost is exactly 6 extra whole-binary rebuilds per wave.
Verified on a live-shaped draw: 6 jtbl-carve cards in a 200-card wave, 13,470 ins
across 7 gate groups (was 11,451 ins in 1 group), with the probe filter refusing what
the gate cannot reach — main 29, island-blocked 10, island-pads 6, one-per-binary 71.
Its entire code subseg is the -O0 run and the .c is stub-only, so this is the boot
precedent: a whole-object CC1FLAGS override, no splat change, no carve, and none of
the 18-P29 re-disassembly risk. Proven byte-neutral by a CLEAN per-binary rebuild —
build dirs deleted, re-extract, rebuild:
sha1 80731bac0ddd6b3e354f43b2c179582b12590752 == config/check.md_MAIN_011.sha
Landed with the coupling fix it requires, or the 21 would have stayed invisible.
Three tools decided -O0-ness from the subseg NAME ('_o0' in it, or 'boot'); this
object keeps its plain name, so match_one would have warned 'cannot bank' about
functions that now bank and the wave draw would have kept refusing to draw them. All
three now ask corpus.o0_subseg(), which derives the answer from the Makefile itself
(R33: a name is a convention, the Makefile is ground truth).
Verified end to end: match_one compiles md_MAIN_011 targets at -O0 with the
cannot-bank warning correctly gone, the wave draw emits cards for them, and
test_o0_detect still passes 167/167 coverage with 0 false positives.
The carve stays INSIDE the byte-gate (harvest_verify._jtbl_prep_one, the §61b-proven
order); everything new routes work to it:
* jtbl_carve: island_probe (read-only classifier: tail/covered/island-end/island-blocked/
island-pads/main-manual), --island-split (the one-line §260 insert, end-adjacent only),
and apply() now recognizes a completed island split as a no-op success instead of the
historical refusal.
* harvest_verify: _ISLAND_WALLS branch — on the §154-A refusal the gate isolates (body
still spliced), inserts the split line, re-extracts, re-carves. Snapshot-restore covers it.
* jtbl_lane.py (new): probe → draft (--draft-dir or api_agent) → gate via the exact
sweep_parallel worker call, HOLDING the campaign draw lock across gate+commit → commit
named per-binary paths. One jtbl target per gate invocation (§61c).
* build_wave_atlas: probe filter + one-jtbl-card-per-binary cap (inert unless
--levers jtbl-carve). idiom_serial: refuses the jtbl-carve lever (R43; its pre-carve ran
the refuted order — S58: 8 attempts, 0 banks).
Proven end to end with the live campaign running (banks committed separately):
commit:2661 ov_SC03_014/func_8017DCC0 tail: §8b adjacent merge + §8e pad recovery,
jr_8017AE2C.o .rodata 0x14→0x28 TIGHT, sha d84b01a2 green
commit:2663 md_SC03_076/func_801F218C covered: §260 STAGE 2 — sha 9a165e36 identical
with the table COMPILER-EMITTED; the first md_* jr bank ever
commit:2664 md_SC03_135/func_801E5358 island-end: the FULL split done BY THE GATE on a
virgin module (~1.1s, R40-checked), sha b901fda5, md.o 0x27c→0x268 + jr.o 0x14
Census (245/245 members probed, R32): 181 members / 26,445 ins reachable unattended;
main 47 parked (gate_main cadence), island-blocked 10 (stack order), island-pads 6
(needs §8e pads for modules.mk), no-jtbl 1 (atlas mislabel).
Design + failure semantics + campaign hook: docs/tool-designs/jtbl-automation-s59.md;
cookbook §260-A.
The build decides the opt level per OBJECT and the Makefile's -O0 rules cover boot,
ov_SC01_077_o0 and src/ov_*/ov_*_o0?.c — nothing else, src/md_*/ least of all. A
function whose target bytes carry the -O0 frame-pointer prologue while its subseg is
compiled -O2 is unbankable however good the draft (R43, cookbook 261).
Measured before the filter existed: 11 such functions had been drawn 79 times across
19 waves — 6 per wave recently — under head-crack/UNKNOWN/tells/len-vein/redraft
labels, because the atlas o0-lane label catches only a fraction of the real -O0
population. Not one could ever have banked. A live draw now reports
'o0-in-an-O2-object': 54 in the skip census.
The first version of the helper was itself the defect it exists to prevent: it took
corpus.asm_path() to be a directory when it is the .s FILE path, so every open()
raised, it returned False for every card, and it filtered nothing while appearing to
run. Its unit test passed because the test fed it the directory the code expected
instead of the value the caller passes.
jtbl and o0 are held out of the default lane because a draft cannot BANK without
tooling first (a rodata carve; an -O0 object, and no Makefile glob matches md_* at
all). 'cc1' is not that: it records that some earlier draft failed to compile. A
compiling draft banks through the normal splice->build->SHA path regardless of its
history, and a non-compiling one fails fast at cc1 for the price of one build.
Holding 14 groups / 105 instances / 6,511 ins out of every wave bought nothing.
Effective on the next wave draw (build_wave_atlas is a fresh subprocess per draw —
no lane restart needed).
CURRENT_PHASE.md gains a CRASH-RECOVERY checkpoint (not a fresh-session handoff): what is
running, restart order, the measured fleet/scaling facts, the fixes that must not regress,
and the ordered work queue.
Lanes: drafter (never stop it), gater (restartable), maintenance (free A-prop sibling lane),
stallguard (60s auto-repair). Drafting holds no lock; one narrow draw-vs-gate lock exists
because build_wave_atlas reads corpus.stubs and misreads substituted drafts mid-gate.
main is off the wave critical path — 157 drafts parked to .run/main_queue/ rather than
stalling the gater for another hour on a bisecting whole-EXE rebuild.
api_agent: 5xx retried like 429 (a 502 was abandoning functions at near-19), HTTP_TIMEOUT
420s not 1800 (a hung request parked an agent 30 min), EXTRA_READABLE for tooling briefs,
and bare-directory paths no longer refused against their own granted root.
Wave Z: 74/75 MATCH, 72 banked first gate, 2 recovered for zero agent tokens
(§203 address-order typedef hoist; §202 alias on a DEFINITION for the §183.3
DEF-side return wall). Sibling sweep re-scoped after --only was keyed on the
atlas gid rather than family_hseq's exemplar: 3 -> 50 banked.
Harvest: 82 gap reports -> 5 laws, 16 rejected, 30 already-covered; the
adversarial verifier killed 7 of 12 claims. §204 written (510 lines).
Tooling, all negative-controlled:
- the S46 validity gate was off the path for waves T-Z; wired into
build_wave_atlas where cards are born, with the PRE-DRAFT-ONLY constraint
documented (it condemns banked work if run later)
- wave_snapshot honors the card's `sub` (9/9 legacy byte-identical, 66/66
split-TU recovered)
- §204-E: decl_prior's %hi/%lo arm had never fired (jal 306->306 zero
regressions, data 0->299)
Wave Y's harvest found a defect in the card field shipped EARLIER THE SAME SESSION (§196). Overlay
functions are named by VRAM address and 134 overlays load at the same window, so build()'s bare
defs[name] counter mixed N unrelated functions -- and §196 ranked that row ABOVE the destination TU.
Measured over the tree: 3,911 of 9,861 symbols with a definition are defined in >1 binary; 1,219
disagree on ARITY, 1,204 of them in the overlay window; 818 of the disagreements were a top-two TIE
that Counter.most_common broke by sorted-file order, so the lowest-numbered defining overlay
silently won. On wave Y's five binaries, 26 of 65 overlay-window DEF rows (40%) named another
overlay's function. Byte-proven cost: applying one row's arity to func_8017E83C took it from
MATCH (114 ins) to 113 ins / 83 mismatched.
Fix: defs are keyed by BINARY; a DEF row is emitted only from the target's own binary for
overlay-window symbols, and otherwise withheld with a stated reason ('N other binaries define this
address; none is yours') rather than silently omitted. Resident/shared/main symbols are fleet-unique
and were always correct (0 of 43 wrong). build_wave_atlas passes the target binary through.
Wave Z was ALREADY STAGED with 24 poisoned rows across its 292; redrawn with the fixed tool -> 0
poisoned, 24 withheld with a reason. This is the fourth same-session self-correction (§194-E ->
§193-A, §199-A -> §189-A, §197-A -> §136-9, §201-A -> §196), and the first one caught before the
bad data reached agents.
HARVEST (10 agents, 68 gap reports): 4 CONFIRMED, 3 REJECTED, 41 already-covered. Yield fell from
14 because the readers were seeded with §193 AND §194 AND §195 -- three sessions of laws off the
table before they started, which is the flywheel working.
§197-A IS BANKED AS ONE ENTRY WITH ITS ATTRIBUTION FLAGGED CONTESTED. Two verifiers independently
confirmed the same phenomenon (lhu;sll 16;sra 16+K where the target has lh;sra K) and attributed it
to DIFFERENT passes -- one to cse's fold_rtx associative block with -da dumps showing the middle insn
already gone in x.i.cse, one to combine preferring the count-merge over forming lh. R34 forbids
treating agreement between disagreeing oracles as corroboration, so the observable, the tell and the
cure (a zero-byte asm re-tie; a fresh temp works with no second SET at all) are banked, and the pass
question is recorded as open. Both verifiers independently byte-refuted §136 type-form rule 9's cure:
u16 v[4] and SVECTOR v compile BYTE-IDENTICALLY in rule 9's own context. Banner added at rule 9.
§198 (in the tool's own help text): UNKNOWN was excluded from the default --levers while the wave
PROMPT told every agent "UNKNOWN does NOT mean hard -- it means the atlas could not name a lever."
The tool was silently overriding the doctrine. Measured: UNKNOWN held 607 of 1,179 in-band members,
more than every other lane combined; wave W drew 73 cards from it into 3 gate groups (24.3 drafts per
rebuild vs wave V's 7.8) for 71/71 drafted, 68 banked. Recorded cost: UNKNOWN groups are mostly
singletons, so the free sibling remap yielded ZERO where waves T/U/V got 49/34/25.
Also releases func_80185480 and func_8017EC98 from wave W's spent-card file -- their agents died on
server rate-limiting and no draft exists, so they must not be marked already-waved.
Wave-V token accounting picked this lever: drafting is 88.3% of a wave's output tokens (repair 8.4%,
reconcile 3.2%) at 9.3 match_one compiles per agent, so the leverage is in what a drafter must GUESS
before its first compile. Two such guesses are answered in the tree and were on nobody's card:
callee arity/return type (§195-A: NO positive tell exists in the asm, the procedure is a two-arity
A/B) and global type (every CONFLICTING-EXTERN drop). tools/decl_prior.py indexes 4,162 files ->
67,094 symbols (9,739 with a banked DEFINITION); cards now carry the rows for their own target's
symbols, ranked DEF > TU (authoritative, wave law 2) > FLEET modal + rivals with counts.
Control draw: 63 rows over 10 cards, 83% already settled by the destination TU (still a saving --
the agent was grepping for them by hand), 17% with no local answer at all.
Third instance in one session of the same law (§193-A seed_ref, §194-E tu_ref, §196): the answer was
already computed and the card did not carry it.
26 agents over wave U's 64 index_gap reports (7 cluster readers, one adversarial verifier per
candidate defaulting to REJECT, seeded with §193 so it could not be re-derived): 14 CONFIRMED,
5 REJECTED, 44 already answered by an existing section (wave T: 9/5/61).
TWO OF THE 14 CORRECT WORK BANKED THE SAME DAY, and both are now cross-banner'd:
* §194-E — `exemplar` is not merely un-banked (§193-A): it names the card's OWN target on 42/73
wave-U and 36/71 wave-T cards, and the `seed_ref` §193-A shipped is same-binary 0/51, so the card
still carried ZERO destination-TU locality. Fixed both ways: a self-pointing exemplar is now
emitted as null, and cards carry `tu_ref` — banked functions in the card's OWN .c ranked by
symbols shared with the TARGET's .s relocations (62% of wave-T targets had such a neighbour vs
19% for the cross-overlay literal grep). Operand-only extraction: a naive uppercase-word regex
read the .s comment column's hex words as symbol names (34 "symbols", 31 of them hex).
* §194-N — §193-D's C dial is misstated: the lever is a SURVIVING CODE_LABEL, not "a label between
the block and the call". jump_optimize deletes any label with LABEL_NUSES == 0 long before sched1
and rewrites a C user label into NOTE_INSN_DELETED_LABEL, which is not a basic-block boundary.
Highlights of the rest: §194-A a zero-byte fence is a one-way wall RELATIVE to the statement being
steered (after = emit-first), and the barrier predicate is volatile-or-colon-less, not the "memory"
clobber; §194-J back-to-back identical stores are deleted by flow.c's last_mem_set unless volatile;
§194-K blinding sched1's alias oracle with a second SET is the first zero-byte dependence-CREATING
lever; §194-M a store in a conditional branch's delay slot proves its C statement DOMINATES the
branch.
19 agents over wave T's 71 index_gap reports: 5 cluster readers, then one adversarial verifier per
candidate defaulting to REJECT. Result: 9 CONFIRMED, 5 REJECTED, and 61 of 71 gaps answered by a
section that ALREADY EXISTS — a retrieval problem, not a knowledge problem.
§193-A is the fix for that retrieval problem, and it is a construction invariant, not drainage: a
wave card's "exemplar"/"sibs" come from the atlas's OPEN set (atlas.py:96 load_open -> corpus.stubs,
:657 max over open members), so they are stubs 0/34 and 0/146 measured on wave T — and 0% at any
project maturity. Meanwhile atlas.py:505-536 already computes a MATCHED-pool twin (pool = sig -
stubs) and build_wave_atlas kept only its similarity score while discarding the identity. Cards now
carry seed_ref + the "M:" matched kNN neighbours; verified on a fresh draw, seed_ref resolves BANKED
4/4 where exemplar resolves banked 0/12.
Two of the nine refute existing sections in place: §193-B byte-refutes §43's "the (s16)param_of_s32
cast form CANNOT reproduce this" (the decider is combine.c:929's cross-call guard, not regalloc),
and §193-G shows §164-54's "scope to >=4 arms" bound is byte-wrong (balance_case_nodes splits at
i > 2). §193-C bounds §8/§48-A1's cross-jump refund to TAILS only — gcc-2.7.2 has no prefix merge.
build_wave_atlas: --one-per-gid collapses same-skeleton siblings to one card and defers
them to <out>.siblings.json for the post-bank family_sweep remap (R32 accounting asserted);
--rank total ranks gate groups by DELIVERED mass (card + deferred siblings). Measured on the
wave-T draw: 6,557 drafted ins carrying 12,709 sibling ins behind 69 of 71 gids = 19,266
instructions of potential for 71 agents, vs 9,985 behind 57 under --rank mass. R39 NC: the
flag is byte-inert on a pool whose gids are unique.
gate_main/pregate_check (§192): three defects that made the pre-gate ladder main-only while
reporting "clean" on overlay slates — (1) resolve_conflicts/substitute hardcoded
corpus.stubs('main') -> per-binary _stubs_for(); (2) sym_of returned the keyword `void` for
every `extern void (*D_x[])(...)`, manufacturing 192 phantom CONFLICTING-EXTERNs (NC over
5,526,100 declarations: 189,301 changed verdicts, 0 regressions); (3) `void f()` and
`void f(void)` were normalized together, costing 40 more phantoms — C89's unspecified-
parameter rule is now gate_main.sig_conflict. §192b: the tool refuses when it substituted 0
files, and prints the per-draft [DROP] reasons it used to compute and discard.
Same overlay slate now reports 2 failures, both real (duplicate typedef; memcpy declared two
ways). Cookbook §192/§192b + index regenerated (585 sections).
The 68%->95% lever from §176h.C2, mechanized. Reconciliation belongs INSIDE the wave: a banked
draft's declarations become the TU's, so a sibling clash hardens into a file clash and post-bank
recovery is measurably worse (18 parked drafts still MATCH, only 1 survived after their wave banked
vs 5 before).
AUTO-FIXES, each re-verified with match_one and REVERTED if a byte moves (a declaration change is
a codegen change, §176f):
* COSMETIC-TYPEDEF two names for a structurally identical struct -> adopt the other. Compared by
BODY, never by name (OtBlk_80015498 == OtBlk_80016450; Elem12 != B12). This body comparison is
also the answer to §176h.C's spelled-name limit.
* SIGNEDNESS / ALIAS / ARRAY-VS-SCALAR -> adopt the TU's spelling, fixing the use site.
* DEFPARAMS (NEW LEVER) -> adopt the TU's parameter types on the DEFINITION and re-narrow with a
shadowing local: `void f(s32 a0_p) { s16 a0 = (s16)a0_p; <body unchanged> }`. One textual
insertion instead of rewriting every use site, and the cast emits the same sll/sra pair.
Byte-identical on both cases tried.
REFUSES, with named reasons, because these are decisions and not edits: DIFFERENT-STRUCT (two real
layouts for one symbol), IMMOVABLE-TU-DECL (gate_main reverts src/, so it needs its own commit +
rebuild + R22), DEF-SIDE-RETURN (adopting the TU's return type usually costs the match -- measured
on func_8001ABBC), and BROKE-MATCH for anything its own verification rejects.
Measured on wave P's leftover slate: 6 -> 9 compatible, 3 auto-reconciled, 2 repairs reverted by
the tool's own byte check, 7 named for a human.
build_wave_atlas: --rank mass (main's gate cost is per SLATE, so ranking groups by member count
silently collapses a wide band to the smallest functions -- measured 60 cards/2,604 ins where 46
cards/4,829 ins were available), and the selector no longer counts ITS OWN OUTPUT as already-waved
(re-running with identical filters had been shrinking the pool 60 -> 46).
A wave is now sized by INSTRUCTION MASS, not card count. The metric is instruction-weighted, so a
wave is worth what its instructions are worth: the 12-42-ins card lanes carried ~1,400 ins/wave
(~0.011pp, ~440 waves to finish) while wave O carried 6,266 ins at the same gate cost and the same
draft rate.
build_wave_atlas --target-ins draws cards until the instruction budget is met (still capped by n)
and refuses to under-fill silently. Standard recipe: --target-ins 6500 --min-ins 60 --max-ins 200
--max-bins 4, levers now including UNKNOWN.
THE MEASUREMENT BEHIND IT: draft rate barely decays with size -- wave M 98% at avg 51 ins, wave N
92% at avg 65, wave O 96% at avg 128. Mass is nearly free.
THE UNKNOWN UNLOCK: UNKNOWN is not a difficulty label, it means the atlas could not name a lever,
and it had been routed as needing its own bespoke lane. Wave O's 22-card R37 probe drafted it like
any other lane -- reclassifying ~138k ins (a quarter of everything open) as ordinary wave fuel.
With UNKNOWN in, 9,224 fns / 417,325 ins = 70% of all open instructions are agent-draftable; the
60-200-ins mass band alone is 164,357 ins ~= 27 waves, and is the band to work first.
Also banks the five-step PRE-GATE PROTOCOL (independent re-verify -> reloc_identity -> dry-run to
0-dropped -> reconcile declarations toward the form the match needs -> gate), cookbook §176g.
build_wave_atlas.py:
- taken-set was a hardcoded 'abcdefghijkl' wave-letter literal -> glob('.run/wave_*_cards.json')
(R33 derive, don't re-derive). NC: 634 -> 726 taken, strict superset, +92 cards from waves m/n
that the literal would have silently re-issued.
- --exclude-bins defaulted to 'main' on the REFUTED link-defect rationale; default now empty and
the help states the real reason (main cannot be gated incrementally; use tools/gate_main.py).
- new --only-bins allow-list: main waves need it, since gate_main rebuilds once per SLATE.
gate_lane.py: an unhandled gate_stage exception produced no JSON line -> v={} -> '0 banked',
indistinguishable from an honest empty gate (cost 2 cycles in P31). Now a non-zero rc or a missing
JSON line is labelled CRASH with the stderr tail, recorded in the results JSON, listed in the
summary as NEVER GATED, and exits non-zero. R39 NC both directions.
- wave D (48 adapt cards): 47/48 standalone, 45 BANKED across two gates (40 + 5 late-repair).
Operational lesson: build the gate slate AFTER the repair stage lands — the first slate was
built early and 5 rescued drafts needed a second gate.
- wave C (35: 11 tell + 24 weak): 32 banked, 91% gate. Weak lane proven 24/24 on haiku.
- MAIN BLOCKER DIAGNOSED (the night's most valuable finding): main drafts are byte-correct yet
gate 0/4. Byte-diff of the built EXE = exactly 2 bytes in 413,696, NOT in the drafted fn: a
jal at 0x80060E74 retargets func_80061FA8 (game code, 800c2.o) -> firstfile (PsyQ libapi
A66.o). Adding one C fn perturbs symbol resolution between game code and the LINKED PsyQ
archives. main is an INTEGRATION wall, not a matching wall -> its own lane; excluded from
build_wave_atlas by default.
- build_wave_atlas.py now packs by (binary, TU) — the REAL gate-group key, since each group is
one whole-binary rebuild (wave D: 42 drafts / 23 groups = the throughput ceiling).
- 3 NEARs enqueued as grinder fuel incl. func_80183578 at close=1 DELAY-SLOT (§60a precedent).
- the old glob('src/<binary>/*.c') found nothing for main (sources live at src/*.c), so every
main draft grouped under src=None: an R36 consumer-blindness, latent because main has never
been wave-gated (main is 0.5% matched, the largest coherent mass left)
- derive from corpus.stubs()[..].path instead; NC'd 3 ways: still-open wave-C drafts 3/3 agree,
overlay sample 96/96 agree (no regression), main now resolves None -> src/800.c
- tools/build_wave_atlas.py: wave selection off the frontier atlas, optimized for GATE
THROUGHPUT (gate cost scales with (binary,TU) groups, not drafts: wave C was 1.3 drafts per
rebuild; atlas selection concentrates to ~96) and weighted toward instruction mass
Drew T