MEASURED on a live gate: 24 workers, 32 cores, and 0-2 concurrent builds at load 2.2.
gate_stage takes the fleet-shared lock EXCLUSIVE whenever it might write shared state, and
`_writes_shared = propagate or not GATE_NO_ARITY`. sweep_parallel passes propagate=False but
never set GATE_NO_ARITY, so the arity pre-pass (default on) made EVERY worker a writer and
all 24 queued on one lock. The gate has been effectively serial for the whole campaign,
while the CPU it was supposedly rationing sat at 7% — and that gate time is what recycles
cards back into the draw, so it throttled the drafting fleet too.
bulk_harvest has documented the contract since P30 — "SET GATE_NO_ARITY=1 FOR THIS PHASE ...
route arity-needing drafts to the serial phase" — and this driver, the one the campaign
gater actually calls, was the one that did not.
PHASE A: parallel, GATE_NO_ARITY=1, workers are READERS and actually run concurrently.
ASSERT: `git status --porcelain src/shared config` must be empty afterwards — the only
cheap detector for a shared-state write escaping a worker (bulk_harvest's rule).
PHASE B: serial with the pre-pass on, for binaries whose drafts failed to COMPILE — the
backlog separates "won't compile standalone (loose-typing / missing decl)" from
"residual: N mismatch", and only the former is what fix_arity_callers fixes.
Recent rows are ~13% failed, so phase B stays small instead of handing back the
parallelism. The lever is kept, not traded away.
Tested on ov_SC07_009 with two deliberately wrong drafts: one that compiles and mismatches
(stays in phase A), one that cannot compile (routes to phase B). Both phases ran, neither
banked, shared state clean, tree clean.
Takes effect on the gater's next wave — sweep_parallel is a subprocess, no restart needed.
R42: gate_main reverted 61 byte-proven overlay banks it could not distinguish from its own
substitution (sweep_parallel gates commit=False by design). Fixed by committing overlay banks
before the main batch, chunking main at 8 to bound bisect cost, and replacing every blind
'git checkout -- src/ config/' with commit-or-refuse in ox_campaign and idiom_serial.
R43: sweep_parallel had an explicit branch admitting main, which cannot be gated incrementally
— wave ab banked 0/105 main cards while its non-main cards banked 94/115 (82%), and the wave
read as a drafting failure. sweep_parallel now refuses main and names gate_main.py.
Also: validate_targets now prefers the card's own addr field (named symbols like SYS_OBJ_F00
were MALFORMED and discarded whole 220-card waves); ox_campaign deals model lanes by
smallest-ratio scheduling (a 73-card wave had put 73 shards on ox and 0 on deepseek);
docs/accelerators.md gains the four vacuous-check defects.
bulk_harvest's Phase B has been a ProcessPoolExecutor over DISTINCT binaries (per-binary flock,
per-worker result files, compute_fleet=False) since Phase 23 — but welded to Phase A's LLM drafting.
Family sweeps stage drafts differently (family_sweep --stage-only), so the farm was UNREACHABLE from
that path, and SESSION-20 gated 389 + 268 + 104 members SERIALLY for no architectural reason (~8-16x
throughput loss on a 32-thread box). This is a thin adapter: same gate_stage.run_gate, same
per-binary lock, NO new gate logic.
Also fixes the phantom-dir bug at source: a bare .run/sweep/*/ glob matches gate_stage's own
intermediate ladder dirs (-cn/-cast/-rc/-s2in/-uni) and calls them as binaries — 24 phantom
PARTIAL 0/1 lines that inflated one run's notbanked from 0 to 56. Requires config/splat.<bin>.yaml
to exist (R33/R36: derive the binary set, never glob it). Smoke-tested: the phantom is skipped and
named, real binaries kept.
§89 records both throughput rules the project already had and was not following.
Drew T