Neither extreme was right. Ignoring generations lets a card that has failed five waves
compete with one nobody has ever drafted; filtering to a single generation starved the fleet
to 46 cards. So generation becomes the PRIMARY ORDERING and the existing mass/count criterion
breaks ties, at both levels of the assembly:
* gate groups holding never-drafted cards rank ahead of all-retry groups (before the
--max-bins truncation, so an untouched group is never cut for a fat retry group);
* within a group, untouched cards are taken before retries.
Wave SIZE is untouched — only the order changes — so the fleet stays full while the scarce
never-drafted work always goes out first.
Verified on a live draw: gen0 2 · gen1 2 · gen2 14 · gen3 2 · gen4 3 · gen5+ 340. It took
EVERY card below generation 5 (all 23 available) and filled the remaining 340 slots from the
5+ pile, which is the whole point.
The gen0 count is 2 because wave dw drew the last 51 untouched skeletons an hour ago. That is
the campaign's real state: essentially everything drawable has now been drafted at least
once, and ~635 distinct skeletons have refused. The remaining work is levers, not draws.
Uncommitted src/ changes found at gate entry. These are banked functions from a lane that gates with commit=False, not residue — preserved, not reverted. Top-level src/*.c (main TUs) are excluded by construction (S59).
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
func_8002823C
func_80031F14
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
GetDispEnv
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
func_80014128
func_8001E378
Uncommitted src/ changes found at gate entry. These are banked functions from a lane that gates with commit=False, not residue — preserved, not reverted. Top-level src/*.c (main TUs) are excluded by construction (S59).
--generational (or BFM_GENERATIONAL=1) draws ONLY the lowest generation present: no function
gets a 2nd draft while any drawable function still lacks a 1st. draw_count is derived from
the prior wave card files already being read for the already-waved filter.
MEASURED BEFORE SHIPPING, and it changed the plan. With every cap opened — no band, no
--max-bins, all levers, whole fleet minus main — generation 0 is:
232 candidates -> 46 distinct skeletons (186 are same-gid siblings the remap banks free)
against ~681 drawable skeletons in total. So "3,926 never-drafted stubs" was three illusions
stacked: ~961 are main's LINKED PsyQ stubs and data blobs (not decomp targets at all), most of
the rest are same-gid siblings that one exemplar banks mechanically, and 2,119 were already
drawn in earlier waves. Making generational the DEFAULT starved the fleet from ~640 cards to
46 — so it is opt-in, for a priority pass over the untouched population, not standing policy.
The real shape of the endgame, stated plainly: of ~681 distinct drawable skeletons, only 46
have never been drafted. The other ~635 refused at least one draft each. That is a LEVER
problem — distillation, A-prop, the o0/jtbl carves, the permuter — not a resampling problem,
and no amount of drafting throughput addresses it.
Fired as wave dw (51 cards / 1,990 ins across 36 binaries) so the untouched population is
drafted today rather than left as a policy.
Uncommitted src/ changes found at gate entry. These are banked functions from a lane that gates with commit=False, not residue — preserved, not reverted. Top-level src/*.c (main TUs) are excluded by construction (S59).
Uncommitted src/ changes found at gate entry. These are banked functions from a lane that gates with commit=False, not residue — preserved, not reverted. Top-level src/*.c (main TUs) are excluded by construction (S59).
The gate's dirty-tree committer swept an empty config/overlays.mk into commit:2863 and took
the whole fleet down with it. R42 says commit a dirty tree rather than revert — true for
src/, where a per-binary gate leaves PROVEN banks uncommitted and reverting destroys them.
A config file is the opposite case: it holds no proven state that exists only in the
worktree, and a collapsed one is never intended.
config_sane() runs at all three commit sites: if config/overlays.mk or config/dedup.us.yaml
has fewer than 80% of HEAD's lines, it is restored from HEAD, NOT committed, and the refusal
is logged loudly. Controls both ways — positive (min_ratio=1.5 makes the healthy registry
trip the same branch: detected, restore path runs, file intact) and negative (normal
threshold: silent, returns True). P28's registry died this way too (H5); now it is enforced
rather than remembered.
commit:2863 ("ox wave dk overlays — 2 banked") committed config/overlays.mk as a ZERO-LINE
file, deleting the registry that defines all 141 overlay binaries: EXE paths, VRAM bases,
ASM_DIR/SRC_DIR roots, symbol files, and every JTBL_PADS spec.
BLAST RADIUS while it was empty:
* main could not build AT ALL. Its object glob prunes sibling binaries via
$($(b)_ASM_DIR); with no binaries registered, every overlay's asm/<ov>/nonmatchings/*.s
fell into MAIN's OBJS and was assembled standalone, where glabel/endlabel/nonmatching
are undefined ("unrecognized opcode `glabel func_801831C8'").
* the main lane found its committed baseline RED and REFUSED to draft or gate (correctly,
R43) — 1,288 open main stubs idle behind it.
* overlay gates collapsed: GATE do banked 0 of 236 gated, GATE dk 2 of 445 drafts.
* every overlay left BINARIES, so check-all's fleet scope collapsed with it.
HOW IT HAPPENED — two failures, neither sufficient alone:
1. A NON-ATOMIC WRITE. The version at commit:2863^ was ALREADY damaged: it ends with a stray
partial line ("uto.txt") after the proper final line — one writer's tail landing after
another's. This file is rewritten in place with no tmp+rename while lanes edit it
concurrently (jtbl carves, o0 subsplits, pad specs).
2. A BLANKET COMMITTER. "tree dirty at gate entry — committing it rather than reverting"
swept the truncated file into a commit. R42 says commit rather than revert and that
stands for src/ — but a tool cannot tell a truncated config from an intended one, and
this is the second time that reasoning has destroyed a registry (P28's yaml.safe_dump,
H5: never silently drop content on a rewrite).
Restored from commit:2863^ with the stray fragment removed; make parses it again. Atomic
writes and a sanity guard on the committer come next.
THE CLASS. JTBL_PADS is a per-object spec written by jtbl_carve at CARVE time — one entry
per rodata `.align 3`, each 0 or 4 — describing how many jump tables the object emits. That
is a DERIVED property of the current source stored as static config, so any bank carrying a
`switch` (or any bank being reverted) invalidates it and nothing re-derives it. Three of the
five REDs on 08-25 were this one design choice: ov_SC02_005 and ov_SC07_006 from wave dd
banking switch-bearing functions, ov_SC04_018 from the identical symptom with the opposite
cause — a reverted bank taking its table with it.
WHY NOT DERIVE IT. The COUNT is derivable from the assembly stream; the VALUES are not — a
pad records where the ORIGINAL image has an inter-table pad, which lives in the retail
layout, not in our source. Guessing shifts every downstream data symbol: silent corruption,
the worst outcome available. So tools/jtbl_pads_fix.py does not derive. It ENUMERATES the
2^(N-1) candidate specs (first entry 0, rest in {0,4}) and accepts one ONLY if it is the
UNIQUE candidate that rebuilds the binary byte-identical to config/check.<bin>.sha; zero or
two matches restore the original and refuse. R39 negative control: on a healthy binary it
reports "no pad-count drift" and changes nothing.
TWO INSTRUMENT BUGS THIS TOOL FOUND IN ITSELF:
* JTBL_PADS is a target-specific MAKE VARIABLE, so changing it does NOT make the .o out of
date. The first run reported "no drift" against a spec I had deliberately broken. It now
deletes the armed objects before every build — R22's incremental trap in config costume.
* A failed object build leaves the PREVIOUS binary in build/<bin>/<bin>, so
`make build; sha1sum build/<bin>/<bin>` reports the OLD artifact as if it were this
build's — a FALSE GREEN over a build that never linked, which briefly convinced me two
binaries were fixed. build_sha now deletes the output too and requires make to exit 0.
Same family as R49: an error inside something shaped like success.
CADENCE: the fleet sweep runs EVERY maintenance pass, not every 4th. A RED fails at BUILD,
so every draft gated against it is rejected regardless of quality and the wave reads as a
drafting failure — detection latency is the whole cost. Gates now finish in ~35 min rather
than 60, so the sweep is affordable each pass. It still FIXES NOTHING by design, with this
single exception, admissible only because it proves itself against the byte gate first.
Uncommitted src/ changes found at gate entry. These are banked functions from a lane that gates with commit=False, not residue — preserved, not reverted. Top-level src/*.c (main TUs) are excluded by construction (S59).
ov_SC02_005 and ov_SC07_006 both failed to assemble with "jtbl_rodata_pads: consumed 1
rodata .align(s) but 2 pad spec(s) given", both from wave dd (commit:2847).
CAUSE, not guessed: dd banked a function CARRYING A SWITCH into each object
(func_8018A150 / func_80183524). A switch changes how many .rdata jump tables the object
emits, and JTBL_PADS is a static per-object spec written by jtbl_carve at CARVE time — it
describes the table population as it was, and nothing re-derives it when banking changes it.
Verified rather than assumed, the same experiment both times: set the spec to the count now
emitted and rebuild. Both are BYTE-IDENTICAL, so the banks are correct and the spec was
stale — the opposite of ov_SC04_018 this morning, where the identical symptom came from a
LOST bank and the spec was right. The symptom does not tell you which; the byte gate does.
ov_SC02_005 9c233988b061... GREEN with 0
ov_SC07_006 7ca772be5656... GREEN with 0
Both binaries were RED from 11:00 (dd's commit) until now — every draft gated against them
in that window was rejected for a reason that had nothing to do with the draft.
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
GetDrawEnv
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
func_80014148
func_80028FBC
1. MAIN LANE — the largest single block of unfinished work was drawing 32 cards a wave.
main_lane.draw() never passed --max-bins, so it inherited build_wave_atlas's default of
12 gate groups — a cap that exists because each group costs a whole-binary rebuild, and
main's own --only-bins docstring says the opposite applies to it: "main is gated ONCE per
SLATE, so main has no per-TU gate cost and --max-bins can be large". Nobody passed it.
Measured cost: main banked ~19 stubs/hour against 1,291 remaining while the overlay lane
ran 650-card waves beside it. Now --max-bins 400 (MAIN_MAX_BINS overrides), and the lane
shell draws 600 cards with 600 workers instead of 200/150.
2. TWO LANES GATE, SO READ BOTH LOGS — a defect I introduced this session. The in-flight
exclusion derived "this wave has been gated" from .run/gater.log only, but the main lane
gates its own waves into .run/main_lane.log. Every m## wave therefore looked permanently
in flight and main's draw lost 425 cards to an exclusion meant for work in progress.
3. TAIL_DONE_FRAC 0.80 -> 0.65. At 0.80 the fleet runs 2-3 overlapping waves at ~250
req/min; the residual troughs are the gap between one wave draining and the next ramping.
65% keeps 3-4 waves overlapping. Stragglers keep their full 700s grace in the finisher
thread — this changes when the NEXT wave starts, never what lands.
4. ATOMIC ATLAS WRITE. The lanes read .run/atlas.json at every draw and atlas.py dumped
straight onto it, leaving a truncated file readable for the length of the write. Now
written to .tmp and os.replace'd.
Context for 1-3: the atlas both lanes draw from is dated 08-23 01:13 — two days stale,
predating ~4,600 banks — and its regen chain is running now (its own R32 assertion caught a
stale family map first and named the fix).
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
SYS_OBJ_2C6C
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
func_80016A5C
func_80029000
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
SetTexWindow
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
func_80017274
func_80029104
MEASURED on a live gate: 24 workers, 32 cores, and 0-2 concurrent builds at load 2.2.
gate_stage takes the fleet-shared lock EXCLUSIVE whenever it might write shared state, and
`_writes_shared = propagate or not GATE_NO_ARITY`. sweep_parallel passes propagate=False but
never set GATE_NO_ARITY, so the arity pre-pass (default on) made EVERY worker a writer and
all 24 queued on one lock. The gate has been effectively serial for the whole campaign,
while the CPU it was supposedly rationing sat at 7% — and that gate time is what recycles
cards back into the draw, so it throttled the drafting fleet too.
bulk_harvest has documented the contract since P30 — "SET GATE_NO_ARITY=1 FOR THIS PHASE ...
route arity-needing drafts to the serial phase" — and this driver, the one the campaign
gater actually calls, was the one that did not.
PHASE A: parallel, GATE_NO_ARITY=1, workers are READERS and actually run concurrently.
ASSERT: `git status --porcelain src/shared config` must be empty afterwards — the only
cheap detector for a shared-state write escaping a worker (bulk_harvest's rule).
PHASE B: serial with the pre-pass on, for binaries whose drafts failed to COMPILE — the
backlog separates "won't compile standalone (loose-typing / missing decl)" from
"residual: N mismatch", and only the former is what fix_arity_callers fixes.
Recent rows are ~13% failed, so phase B stays small instead of handing back the
parallelism. The lever is kept, not traded away.
Tested on ov_SC07_009 with two deliberately wrong drafts: one that compiles and mismatches
(stays in phase A), one that cannot compile (routes to phase B). Both phases ran, neither
banked, shared state clean, tree clean.
Takes effect on the gater's next wave — sweep_parallel is a subprocess, no restart needed.
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
func_800173BC
They were held out because each has a cheaper deterministic owner: the family remap banks a
`remap` card for zero tokens, `plumbing` belongs to recover_integration, `needs-autopsy`
wants a look before a draft. That reasoning priced AGENT TOKENS as the scarce resource. On a
free model the scarce resource is CARDS — holding 1,219 instances out of every wave to
protect a budget that does not bind starves a 2,000-agent fleet.
lever-not-in-lane drops 1,383 -> 164 in a live draw. The deterministic lanes still run and
still reach these first; the byte gate refuses a duplicate, so a card a remap already banked
costs one wasted shard, never a wrong bank.
MEASURED over 18 consecutive waves. Consecutive card sets: ck->cl 239/239 shared, co->cp
238/238, cv->cw 222/222, db->dc 208/209 — and the "different" pairs still shared 50-90%.
Yield alternated in lockstep: 47.6% / 3.8% / 35.3% / 3.6% / 29.9% / 3.7% / 43.4% / 14.6%,
because the duplicate wave gates AFTER the original banked its cards. Half of all drafting
went to work already in flight, and it read as campaign decay.
ROOT CAUSE: --retry-unbanked returns "previously waved but still an OPEN STUB" cards to the
pool — right in principle, unfinished work is not spent work. But the pre-draw for wave N+1
runs WHILE wave N drafts, when none of wave N's cards have been gated, so every one of them
is still an open stub and the filter hands the whole wave back. The ranking then rebuilds it
card for card. The filter knew about "banked" and "not banked" and had no notion of "in
flight".
FIX: a wave is finished when its GATE has run, and the gater already says so in its own log
(R33 — derive from the artifact that exists). Tags with no GATE line stay excluded; a tag
with no gate line whose cards are older than 6 h was killed, and is released so nothing is
locked out forever.
THROUGHPUT, same commit — the draw was setting the campaign's request rate:
* --max-bins 24 -> 160. Concentrating a wave into 24 gate groups was a CPU-economy choice
made when CPU was scarce. It is not: a live gate runs at load 2.7 of 32 cores (8%), one
harvest_verify at --chunk 1. Meanwhile the drafting fleet — the resource actually bounded
by the free-model clock — got 196 cards out of 699 available. Re-drawn with 160 bins:
644 drafts / 46,590 ins across 136 binaries, 3.3x the wave for the same gate economics.
* TAIL_DONE_FRAC 0.95 -> 0.80. Overlapping at 95% still left 25% of minutes under 20 req/min,
because a wave's last 5% is its SLOWEST 5% and 12 stragglers cannot fill a fleet. Handing
off at 80% starts the next ramp with ~40 agents still working. Stragglers keep their full
700s grace in the finisher thread; nothing is cut short.
* --queue-depth 2 -> 4, so a bigger wave's longer gate never parks the drafter.
Arithmetic this is aimed at: req/min = agents-in-flight x ~0.8 (a 16k-token turn at ~30
tok/s emits few requests). 644 cards x two overlapping waves puts the fleet where the
endpoint has already been measured to sustain it — 764 req/min for 15 min at 8% 429s, peak
2,755 in one minute.
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
func_800174FC
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
func_80010DA0
Uncommitted src/ changes found at gate entry. These are banked functions from a lane that gates with commit=False, not residue — preserved, not reverted. Top-level src/*.c (main TUs) are excluded by construction (S59).
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
func_8001A9D8
func_8005D588
The object failed to assemble: "jtbl_rodata_pads: consumed 3 rodata .align(s) but 4 pad
spec(s) given". The tempting fix is to relax the spec to 3 — it builds and it is
BYTE-IDENTICAL, which is exactly what makes it dangerous.
The spec was right and the SOURCE was wrong. func_8017E7CC was banked as of commit:2628 (68
INCLUDE_ASM in the TU) and back to a stub at commit:2629 (69) — the free A-prop maintenance
pass reverted a byte-proven function, and the object's 4th table went with it. Restoring
the body from commit:2628 with the ORIGINAL 0,0,0,0 spec builds BYTE-IDENTICAL
(fe9b413fc48ba615ccf81cde49d7a1efad481bc9), which is the proof the spec was never drifted.
R42 again, from the other direction: the destroyed bank did not merely vanish, it took a
build with it, and the failure wore a config-drift costume. A byte-identical build is not
evidence that the change you made was the right one — both fixes are byte-identical here
and only one keeps the function.
`extern void func_8018D088(void);` at line 3810 against `s32 func_8018D088(void *a0)`
at 3862: 'conflicting types', the object never compiled, and the whole overlay had been
RED. The decl now matches the definition; ov_SC03_001 builds BYTE-IDENTICAL
(f8fd92f59c6871577c61cb626cb9ddb0db5ba884).
Two edits into one TU each verified alone and conflicted together — the shared-TU race a
per-binary gate cannot see when the second writer never rebuilds the first writer's file.
Caught by the maintenance lane's periodic fleet R22 at 06:39 (R50), which is the only
oracle that looks at binaries no lane is touching.
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
func_8001BADC
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
func_8001125C
Drew T