From db3fe3490bba76469ec7f0e1efbcd3d9542b830b Mon Sep 17 00:00:00 2001 From: Drew T <50529377+Druthulu@users.noreply.github.com> Date: Mon, 24 Aug 2026 16:36:47 -0600 Subject: [PATCH] =?UTF-8?q?fix(lanes):=20raise=20HTTP=5FTIMEOUT=20with=20M?= =?UTF-8?q?AXTOK=20=E2=80=94=20they=20are=20one=20setting,=20not=20two?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Probed ox-alpha directly on a real MIPS derivation: no reasoning cap 265.2s finish=stop completion=8,067 reasoning=0 30 tok/s reasoning cap 2000 22.3s finish=stop completion= 672 reasoning=0 reasoning cap 6000 41.4s finish=stop completion= 618 reasoning=0 Three findings. (1) ox reports reasoning_tokens=0 — its thinking is IN the content stream, so the output cap was capping the reasoning; that is exactly why turns ended in 'no tool call (finish=length)'. (2) The uncapped hard prompt wanted 8,067 tokens — it was finishing precisely where the old 8k cap cut it off. (3) It generates at ~30 tok/s, not the ~54 I estimated from turn gaps, so a full 16k generation needs ~530s and the 420s socket would have killed the very turns the bigger budget exists to allow. A timeout wastes the whole turn; truncation at least leaves a partial. HTTP_TIMEOUT=700 on both drafting lanes. The ordering that must hold is generation < HTTP_TIMEOUT (700) < stallguard's wedged-agent kill (1200s). 420 was itself deliberate — 1800 once parked a hung agent for thirty minutes — and 700 keeps a hang under 12 minutes without strangling legitimate deep reasoning. Also recorded: a reasoning cap DOES work on ox, but it shortens the ANSWER too (618-672 total tokens), so it is a quality knob, not a fix for truncation. --- tools/lanes/drafter.sh | 10 ++++++++++ tools/lanes/main.sh | 10 ++++++++++ 2 files changed, 20 insertions(+) diff --git a/tools/lanes/drafter.sh b/tools/lanes/drafter.sh index 02dfe7770..01ed9befe 100644 --- a/tools/lanes/drafter.sh +++ b/tools/lanes/drafter.sh @@ -79,6 +79,16 @@ # larger output budget costs latency and nothing else. set -u cd /home/musashi/bfm-decomp +# HTTP_TIMEOUT 700 (P31 S59) — MUST be raised WITH MAXTOK; they are one setting, not two. +# Measured on ox-alpha with a real MIPS derivation: 30 tok/s, and an UNCAPPED hard prompt ran +# 265 s for 8,067 completion tokens — i.e. it wanted to finish exactly where the old 8k cap cut it +# off, which is the nudge storm we were seeing. At 30 tok/s a full 16k generation needs ~530 s, so +# leaving the socket at 420 s would have killed the very turns the bigger budget exists to allow — +# and a timeout wastes the whole turn, where truncation at least leaves a partial. +# The ordering that must hold: generation < HTTP_TIMEOUT (700) < stallguard's wedged-agent kill +# (1200 s). 420 was itself a deliberate choice after 1800 parked a hung agent for THIRTY minutes; +# this keeps that concern (a hang costs <12 min) without strangling legitimate deep reasoning. +export HTTP_TIMEOUT=700 export MAX_429=10 while [ ! -e .run/ox_campaign.stop ]; do .venv/bin/python tools/ox_campaign.py --drafter \ diff --git a/tools/lanes/main.sh b/tools/lanes/main.sh index e5da9d67b..fedd00427 100644 --- a/tools/lanes/main.sh +++ b/tools/lanes/main.sh @@ -13,6 +13,16 @@ # excluded — already drafted, never gated, free), then draws and drafts its own main cards. set -u cd /home/musashi/bfm-decomp +# HTTP_TIMEOUT 700 (P31 S59) — MUST be raised WITH MAXTOK; they are one setting, not two. +# Measured on ox-alpha with a real MIPS derivation: 30 tok/s, and an UNCAPPED hard prompt ran +# 265 s for 8,067 completion tokens — i.e. it wanted to finish exactly where the old 8k cap cut it +# off, which is the nudge storm we were seeing. At 30 tok/s a full 16k generation needs ~530 s, so +# leaving the socket at 420 s would have killed the very turns the bigger budget exists to allow — +# and a timeout wastes the whole turn, where truncation at least leaves a partial. +# The ordering that must hold: generation < HTTP_TIMEOUT (700) < stallguard's wedged-agent kill +# (1200 s). 420 was itself a deliberate choice after 1800 parked a hung agent for THIRTY minutes; +# this keeps that concern (a hang costs <12 min) without strangling legitimate deep reasoning. +export HTTP_TIMEOUT=700 while [ ! -e .run/ox_campaign.stop ]; do .venv/bin/python tools/main_lane.py --workers 150 --batch 40 --cards 200 --max-ins 200 --maxtok 16000 2>&1 echo "[$(date +%H:%M:%S)] [main-lane] exited; restarting in 20s"