Commit Graph

4 Commits

Author SHA1 Message Date
Drew T e76a55abdf fix(distill): a stale READY marker no longer blocks the lane forever
A marker is a CLAIM on work, not a record of it. axbm.json sat in .run/distill_ready
for 10.5 hours AFTER its waves were distilled into cookbook §269 — the reviewer landed
the sections, updated the mined state, and never removed the marker — and distill_scan's
one-pending-marker-at-a-time rule (correct, it stopped eight overlapping batches) then
refused to raise anything while 18 waves / 315 novel candidates accumulated behind it.

The marker's own waves are checkable against the mined state, so check them: a marker
whose every wave is already mined clears itself and says so. Negative control (R39): a
marker naming any still-unmined wave survives untouched.

Same family as R47 — a stage that consumes work must also consume the token that
represents it.
2026-08-25 00:49:24 -06:00
Drew T 86ebc95a81 feat(ops): one status view for ALL lanes; straggler grace = one full turn; distill dedupe
STATUS BLINDNESS (Drew): status checks kept reporting the overlay drafter and the
gater — the lanes whose logs scroll — while the main, maintenance and distill lanes
went unmentioned for hours. A lane you do not report is a lane you do not notice
failing: the main lane spent an afternoon on an old config and bisected a whole batch
to zero banks without that ever reaching a status line. tools/campaign_status.py
prints every lane with ITS OWN metrics, read from artefacts rather than memory.

STRAGGLER GRACE 120 -> 700, tied to HTTP_TIMEOUT so they cannot drift. collect_drafts
queues a wave once 95% of shards finish, then waits this long for the rest — and 120s
is shorter than a single turn (~530s for a 16k generation at ~30 tok/s). So raising
the token budget converted truncated turns into agents guillotined mid-thought with NO
draft: overlay draft completion fell from 84-89% at 8k to 41% (bt) and 69% (bu).

DISTILL DEDUPE: each pass re-offers everything unmined, so the lane wrote a fresh
overlapping marker every five minutes — eight queued, each a superset of the last, and
a reviewer cannot tell which one is the work. One pending marker at a time.
2026-08-24 18:27:05 -06:00
Drew T 6c580a28a0 docs(distill): name the review tier — Opus/Sonnet, not Fable
Drew, 2026-08-24: distillation is judgement over an existing corpus (read harvested
notes, decide covered / addendum / new against 760+ sections), not a new wall class.
Fable is for the walls — an unsolved tooling problem, an adversarial design review, a
residual no documented lever reaches. I routed a distill batch to Fable; recorded here
so the next session reads the tier off the lane rather than guessing it.
2026-08-24 13:57:31 -06:00
Drew T 333a169458 fix(distill): state is {tag: novel-count}, never a done-list — it nearly buried 82 candidates
A re-gated wave rewrites its candidate file with NEW rows under the SAME tag, so
"have I seen this tag" answers the wrong question. Two instances in one hour:

  * wave `at` was re-gated hours after its first harvest, so an mtime-keyed seed
    called it new and 52 mostly-re-derived rows went to a reviewer;
  * my own hand-edit of the state folded "queued for review" into "reviewed", which
    marked `ax` and `bm` — 82 candidates, gated minutes earlier — as mined by nobody.
    Caught only because their files were newer than the edit.

The scan now compares COUNTS: a tag re-opens the moment its file grows past what was
mined from it. Extracted to tools/distill_scan.py so the logic is testable rather than
living inside a heredoc inside a lane loop (the heredoc-in-heredoc edit is also what
produced a syntax-broken lane script a minute earlier).

Verified: the lane now raises exactly the true pending batch — ax + bm, 82 novel.
2026-08-24 13:38:52 -06:00