THE CLASS. JTBL_PADS is a per-object spec written by jtbl_carve at CARVE time — one entry
per rodata `.align 3`, each 0 or 4 — describing how many jump tables the object emits. That
is a DERIVED property of the current source stored as static config, so any bank carrying a
`switch` (or any bank being reverted) invalidates it and nothing re-derives it. Three of the
five REDs on 08-25 were this one design choice: ov_SC02_005 and ov_SC07_006 from wave dd
banking switch-bearing functions, ov_SC04_018 from the identical symptom with the opposite
cause — a reverted bank taking its table with it.
WHY NOT DERIVE IT. The COUNT is derivable from the assembly stream; the VALUES are not — a
pad records where the ORIGINAL image has an inter-table pad, which lives in the retail
layout, not in our source. Guessing shifts every downstream data symbol: silent corruption,
the worst outcome available. So tools/jtbl_pads_fix.py does not derive. It ENUMERATES the
2^(N-1) candidate specs (first entry 0, rest in {0,4}) and accepts one ONLY if it is the
UNIQUE candidate that rebuilds the binary byte-identical to config/check.<bin>.sha; zero or
two matches restore the original and refuse. R39 negative control: on a healthy binary it
reports "no pad-count drift" and changes nothing.
TWO INSTRUMENT BUGS THIS TOOL FOUND IN ITSELF:
* JTBL_PADS is a target-specific MAKE VARIABLE, so changing it does NOT make the .o out of
date. The first run reported "no drift" against a spec I had deliberately broken. It now
deletes the armed objects before every build — R22's incremental trap in config costume.
* A failed object build leaves the PREVIOUS binary in build/<bin>/<bin>, so
`make build; sha1sum build/<bin>/<bin>` reports the OLD artifact as if it were this
build's — a FALSE GREEN over a build that never linked, which briefly convinced me two
binaries were fixed. build_sha now deletes the output too and requires make to exit 0.
Same family as R49: an error inside something shaped like success.
CADENCE: the fleet sweep runs EVERY maintenance pass, not every 4th. A RED fails at BUILD,
so every draft gated against it is rejected regardless of quality and the wave reads as a
drafting failure — detection latency is the whole cost. Gates now finish in ~35 min rather
than 60, so the sweep is affordable each pass. It still FIXES NOTHING by design, with this
single exception, admissible only because it proves itself against the byte gate first.
Uncommitted src/ changes found at gate entry. These are banked functions from a lane that gates with commit=False, not residue — preserved, not reverted. Top-level src/*.c (main TUs) are excluded by construction (S59).
ov_SC02_005 and ov_SC07_006 both failed to assemble with "jtbl_rodata_pads: consumed 1
rodata .align(s) but 2 pad spec(s) given", both from wave dd (commit:2847).
CAUSE, not guessed: dd banked a function CARRYING A SWITCH into each object
(func_8018A150 / func_80183524). A switch changes how many .rdata jump tables the object
emits, and JTBL_PADS is a static per-object spec written by jtbl_carve at CARVE time — it
describes the table population as it was, and nothing re-derives it when banking changes it.
Verified rather than assumed, the same experiment both times: set the spec to the count now
emitted and rebuild. Both are BYTE-IDENTICAL, so the banks are correct and the spec was
stale — the opposite of ov_SC04_018 this morning, where the identical symptom came from a
LOST bank and the spec was right. The symptom does not tell you which; the byte gate does.
ov_SC02_005 9c233988b061... GREEN with 0
ov_SC07_006 7ca772be5656... GREEN with 0
Both binaries were RED from 11:00 (dd's commit) until now — every draft gated against them
in that window was rejected for a reason that had nothing to do with the draft.
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
GetDrawEnv
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
func_80014148
func_80028FBC
1. MAIN LANE — the largest single block of unfinished work was drawing 32 cards a wave.
main_lane.draw() never passed --max-bins, so it inherited build_wave_atlas's default of
12 gate groups — a cap that exists because each group costs a whole-binary rebuild, and
main's own --only-bins docstring says the opposite applies to it: "main is gated ONCE per
SLATE, so main has no per-TU gate cost and --max-bins can be large". Nobody passed it.
Measured cost: main banked ~19 stubs/hour against 1,291 remaining while the overlay lane
ran 650-card waves beside it. Now --max-bins 400 (MAIN_MAX_BINS overrides), and the lane
shell draws 600 cards with 600 workers instead of 200/150.
2. TWO LANES GATE, SO READ BOTH LOGS — a defect I introduced this session. The in-flight
exclusion derived "this wave has been gated" from .run/gater.log only, but the main lane
gates its own waves into .run/main_lane.log. Every m## wave therefore looked permanently
in flight and main's draw lost 425 cards to an exclusion meant for work in progress.
3. TAIL_DONE_FRAC 0.80 -> 0.65. At 0.80 the fleet runs 2-3 overlapping waves at ~250
req/min; the residual troughs are the gap between one wave draining and the next ramping.
65% keeps 3-4 waves overlapping. Stragglers keep their full 700s grace in the finisher
thread — this changes when the NEXT wave starts, never what lands.
4. ATOMIC ATLAS WRITE. The lanes read .run/atlas.json at every draw and atlas.py dumped
straight onto it, leaving a truncated file readable for the length of the write. Now
written to .tmp and os.replace'd.
Context for 1-3: the atlas both lanes draw from is dated 08-23 01:13 — two days stale,
predating ~4,600 banks — and its regen chain is running now (its own R32 assertion caught a
stale family map first and named the fix).
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
SYS_OBJ_2C6C
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
func_80016A5C
func_80029000
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
SetTexWindow
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
func_80017274
func_80029104
MEASURED on a live gate: 24 workers, 32 cores, and 0-2 concurrent builds at load 2.2.
gate_stage takes the fleet-shared lock EXCLUSIVE whenever it might write shared state, and
`_writes_shared = propagate or not GATE_NO_ARITY`. sweep_parallel passes propagate=False but
never set GATE_NO_ARITY, so the arity pre-pass (default on) made EVERY worker a writer and
all 24 queued on one lock. The gate has been effectively serial for the whole campaign,
while the CPU it was supposedly rationing sat at 7% — and that gate time is what recycles
cards back into the draw, so it throttled the drafting fleet too.
bulk_harvest has documented the contract since P30 — "SET GATE_NO_ARITY=1 FOR THIS PHASE ...
route arity-needing drafts to the serial phase" — and this driver, the one the campaign
gater actually calls, was the one that did not.
PHASE A: parallel, GATE_NO_ARITY=1, workers are READERS and actually run concurrently.
ASSERT: `git status --porcelain src/shared config` must be empty afterwards — the only
cheap detector for a shared-state write escaping a worker (bulk_harvest's rule).
PHASE B: serial with the pre-pass on, for binaries whose drafts failed to COMPILE — the
backlog separates "won't compile standalone (loose-typing / missing decl)" from
"residual: N mismatch", and only the former is what fix_arity_callers fixes.
Recent rows are ~13% failed, so phase B stays small instead of handing back the
parallelism. The lever is kept, not traded away.
Tested on ov_SC07_009 with two deliberately wrong drafts: one that compiles and mismatches
(stays in phase A), one that cannot compile (routes to phase B). Both phases ran, neither
banked, shared state clean, tree clean.
Takes effect on the gater's next wave — sweep_parallel is a subprocess, no restart needed.
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
func_800173BC
They were held out because each has a cheaper deterministic owner: the family remap banks a
`remap` card for zero tokens, `plumbing` belongs to recover_integration, `needs-autopsy`
wants a look before a draft. That reasoning priced AGENT TOKENS as the scarce resource. On a
free model the scarce resource is CARDS — holding 1,219 instances out of every wave to
protect a budget that does not bind starves a 2,000-agent fleet.
lever-not-in-lane drops 1,383 -> 164 in a live draw. The deterministic lanes still run and
still reach these first; the byte gate refuses a duplicate, so a card a remap already banked
costs one wasted shard, never a wrong bank.
MEASURED over 18 consecutive waves. Consecutive card sets: ck->cl 239/239 shared, co->cp
238/238, cv->cw 222/222, db->dc 208/209 — and the "different" pairs still shared 50-90%.
Yield alternated in lockstep: 47.6% / 3.8% / 35.3% / 3.6% / 29.9% / 3.7% / 43.4% / 14.6%,
because the duplicate wave gates AFTER the original banked its cards. Half of all drafting
went to work already in flight, and it read as campaign decay.
ROOT CAUSE: --retry-unbanked returns "previously waved but still an OPEN STUB" cards to the
pool — right in principle, unfinished work is not spent work. But the pre-draw for wave N+1
runs WHILE wave N drafts, when none of wave N's cards have been gated, so every one of them
is still an open stub and the filter hands the whole wave back. The ranking then rebuilds it
card for card. The filter knew about "banked" and "not banked" and had no notion of "in
flight".
FIX: a wave is finished when its GATE has run, and the gater already says so in its own log
(R33 — derive from the artifact that exists). Tags with no GATE line stay excluded; a tag
with no gate line whose cards are older than 6 h was killed, and is released so nothing is
locked out forever.
THROUGHPUT, same commit — the draw was setting the campaign's request rate:
* --max-bins 24 -> 160. Concentrating a wave into 24 gate groups was a CPU-economy choice
made when CPU was scarce. It is not: a live gate runs at load 2.7 of 32 cores (8%), one
harvest_verify at --chunk 1. Meanwhile the drafting fleet — the resource actually bounded
by the free-model clock — got 196 cards out of 699 available. Re-drawn with 160 bins:
644 drafts / 46,590 ins across 136 binaries, 3.3x the wave for the same gate economics.
* TAIL_DONE_FRAC 0.95 -> 0.80. Overlapping at 95% still left 25% of minutes under 20 req/min,
because a wave's last 5% is its SLOWEST 5% and 12 stragglers cannot fill a fleet. Handing
off at 80% starts the next ramp with ~40 agents still working. Stragglers keep their full
700s grace in the finisher thread; nothing is cut short.
* --queue-depth 2 -> 4, so a bigger wave's longer gate never parks the drafter.
Arithmetic this is aimed at: req/min = agents-in-flight x ~0.8 (a 16k-token turn at ~30
tok/s emits few requests). 644 cards x two overlapping waves puts the fleet where the
endpoint has already been measured to sustain it — 764 req/min for 15 min at 8% 429s, peak
2,755 in one minute.
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
func_800174FC
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
func_80010DA0
Uncommitted src/ changes found at gate entry. These are banked functions from a lane that gates with commit=False, not residue — preserved, not reverted. Top-level src/*.c (main TUs) are excluded by construction (S59).
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
func_8001A9D8
func_8005D588
The object failed to assemble: "jtbl_rodata_pads: consumed 3 rodata .align(s) but 4 pad
spec(s) given". The tempting fix is to relax the spec to 3 — it builds and it is
BYTE-IDENTICAL, which is exactly what makes it dangerous.
The spec was right and the SOURCE was wrong. func_8017E7CC was banked as of commit:2628 (68
INCLUDE_ASM in the TU) and back to a stub at commit:2629 (69) — the free A-prop maintenance
pass reverted a byte-proven function, and the object's 4th table went with it. Restoring
the body from commit:2628 with the ORIGINAL 0,0,0,0 spec builds BYTE-IDENTICAL
(fe9b413fc48ba615ccf81cde49d7a1efad481bc9), which is the proof the spec was never drifted.
R42 again, from the other direction: the destroyed bank did not merely vanish, it took a
build with it, and the failure wore a config-drift costume. A byte-identical build is not
evidence that the change you made was the right one — both fixes are byte-identical here
and only one keeps the function.
`extern void func_8018D088(void);` at line 3810 against `s32 func_8018D088(void *a0)`
at 3862: 'conflicting types', the object never compiled, and the whole overlay had been
RED. The decl now matches the definition; ov_SC03_001 builds BYTE-IDENTICAL
(f8fd92f59c6871577c61cb626cb9ddb0db5ba884).
Two edits into one TU each verified alone and conflicted together — the shared-TU race a
per-binary gate cannot see when the second writer never rebuilds the first writer's file.
Caught by the maintenance lane's periodic fleet R22 at 06:39 (R50), which is the only
oracle that looks at binaries no lane is touching.
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
func_8001BADC
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
func_8001125C
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
func_80027BF0
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
func_8002EEB8
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
func_8001CFDC
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
func_80010E7C
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
func_8002FDC8
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
func_80011680
vmNoiseOn
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
func_80028D78
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
func_800111BC
func_80028DBC
func_800495EC
One clean whole-EXE rebuild verified the batch (gate_main), and main re-checked
BYTE-IDENTICAL against config/check.us.sha before anything was credited.
DsReadyCallback
SetIR123
func_8001D074
func_80028E00
func_800538BC
func_8005FBA8
Drew T