mirror of
https://github.com/Druthulu/BFM-decomp
synced 2026-10-04 00:20:10 -04:00
Drew T
f78ea55b3e
THE GATE WAS A BLACK BOX. sweep_parallel's stdout was captured and dropped, so a gate logged "reloc_identity -> gating 216" and then THIRTY MINUTES OF SILENCE before its bank line — no worker count, no per-binary progress, no phase-A/phase-B split. Gate times went 31 -> 37 -> 50 -> 67 min across ej/ek/en/eo with nothing to diagnose from, and I twice asserted things about phase B that the log could not support (its absence measured LOG CAPTURE, not behaviour). A lane that must run unattended has to leave evidence. MEASURED WHILE DIAGNOSING, and it rules out the obvious suspects: load average 2.6 on 32 cores with 1-3 concurrent builds during a gate — the gate is NOT CPU-bound and is not saturating its own -j 24. Raising to 32 is cheap given ~8% utilisation, but the real answer will come from the log this change adds. TAIL_DONE_FRAC 0.85 -> 0.80. 0.85 overcorrected: the fleet fell to 15 agents / 11 req/min because the drafter parks between waves while the gater drains a deep queue. 0.75 was too deep (26% 429s, draft completion sliding 94->91->73->47% across eq/er/es/et). Neither number is really the lever: the drafter cannot start a wave the gater has no room for, so the gate throughput is what bounds the campaign now. Generational tiering confirmed already correct: the top-off orders by generation at both assembly levels (group ranking and within-group) without FILTERING any tier out, so every generation stays eligible and the scarce never-drafted work simply goes first.
15 lines
830 B
Bash
15 lines
830 B
Bash
#!/usr/bin/env bash
|
|
# The restartable lane. Kill and relaunch this freely — drafting is unaffected.
|
|
set -u
|
|
cd /home/musashi/bfm-decomp
|
|
while [ ! -e .run/ox_campaign.stop ]; do
|
|
# GATE JOBS 24 (P31 S59, Drew approved the CPU). Each job is one whole-binary rebuild and the box
|
|
# is 32 cores running at ~6% (load 3.1, 211% of 3200%) with 39 GB free — 12 was leaving the machine
|
|
# idle. It matters now because the tells and jtbl QUOTAS deliberately pull cards from binaries
|
|
# outside the ranked gate groups: a measured draw went from ~24 groups to 63, i.e. 63 rebuilds per
|
|
# wave, and at 12 jobs that is five serial batches. Raise this with the quota sizes, not on its own.
|
|
.venv/bin/python tools/ox_campaign.py --gater --gate-jobs 32 2>&1
|
|
echo "[$(date +%H:%M:%S)] [gater] exited; restarting in 20s"
|
|
sleep 20
|
|
done
|