The first version of this parsed 'failed by class:' from the worker's stdout and
was INERT: the worker is gate_stage, which never prints that line (harvest_verify
does, one level down). classes came back empty for all 17 binaries of a batch and
the retry gate that consumed it fired ZERO times — a field that is always empty
makes its consumer a silent no-op (R54). Verified the claim only after re-reading
the log; correcting it here.
Now parallel_gate copies harvest_verify's <stem>.classified.txt out of the
worktree before teardown (it lives in the worktree's own .run/, which is not
symlinked and dies with it) and derives the class summary from those rows. That
also PRESERVES the verdict layer, which until now survived only as a side effect
of gater_lane re-running the whole binary in-tree afterwards (R47).
gater_lane judges the retry on the rows: a class with no per-function diagnostic
is the blind-worktree signature; anything cc1 named is a real compile error and
the serial rebuild would only reproduce it.
Verified live on ov_SC07_000: 'NOT retrying in-tree' fired, and the verdict row
landed at .run/gate_lane/ov_SC07_000.pgate.classified.txt.
tools/triage_ladder.py — the zero-token pre-agent pass, split PRE (target-side:
BANKED/WALL-332/PARKED, no build) from POST (residual_rules_b, needs a draft).
--escalate refuses a walled or banked target; --acceptance is the R39/R32 harness.
Refuses on a non-quiescent tree: a merging gate makes the stub oracle wrong in
both directions (measured, ov_SC01_004:func_8017EB30).
Acceptance, on the whole corpus: false-skip 0/1367 open stubs, recall 426/426
matched, wall tier fires on exactly the 10 enumerated walls (0 extra, 0 missing).
The first wall control asked for evidence that CANNOT exist — it scanned banked
functions' .s, which splat never writes — and printed '0 scanned / 0 tripped',
indistinguishable from a pass. The R32 empty-denominator assertion caught it on
its first run; replaced with a two-sided sweep over all open stubs.
tools/cast_self_callers.py — the §378 lever + --sync-decls for the narrow-param
case C89 forbids no-proto from reaching (§378a).
Wiring: wave_args drops walled/parked targets at draw time via pre_classify (one
implementation, R33); escalate_fable.js refuses any target without triage:'DRAFT'.
Tool fixes found by measurement:
* fix_arity_callers was blind to main entirely (globbed src/main/main*.c; main is
src/*.c) — reported success over an empty file set through three gates. Now
refuses when --binary selects no files.
* parallel_gate records each worker's 'failed by class' line (was truncated out of
the 200-char tail); gater_lane retries in-tree ONLY on the diagnostic-free
blind-worktree signature — S69 ran 22 serial retries against real cc1 errors.
docs: cookbook §376/§377/§378 (index 1033), SETUP.md, wave-playbook §4b.
I shipped a guard that used drafting-scratch mtimes as a liveness proxy. It failed
in both possible directions within minutes:
* FALSE PASS: the find included '.run/*wave*', which expanded past ARG_MAX
('Argument list too long'). find then matched nothing, the guard PASSED, and I
ran clean: removed build/, expected/, and the regenerated splat tree (asm/, assets/, include macros, undefined_*_auto.txt). on a live lane — deleting asm/ under five drafting agents. I
restored it immediately (extract-all 212/212) but that is damage control, not a
design.
* FALSE PASS, structurally: even with the glob fixed, an agent that THINKS longer
than the window is indistinguishable from a finished one — the exact flaw I had
already written into gater_lane's docstring for the verdicts file ('a quiet file
mtime is deliberately NOT accepted as one') and then rebuilt here anyway.
tools/lane_inflight.py is the fix: liveness is RECORDED, not inferred. The
orchestrator adds a target when it launches the workflow and removes it when the
verdict returns — both actions it already performs, so the ledger cannot drift
without skipping a step that is taken anyway. exits non-zero when any agent
is live, which IS the guard, and both r22_verify.sh and parallel_gate --r22 now use
it instead of touching the filesystem.
Negative-controlled both directions: refuses with 5 live agents named and their
start times; passes when the ledger is drained.
The lesson worth more than the fix: I had already identified 'a quiet mtime is not
a completion signal' as a defect class, documented it, and then re-implemented it
in a different file. Writing a rule down does not stop you applying its opposite
somewhere else.
The exclusivity guard I added to tools/r22_verify.sh left the path actually used
most -- parallel_gate --r22 -- unguarded, because the destructive 'make clean'
lives in BOTH. Four times this session a drafting agent reported 'asm/<binary> is
MISSING from the tree' mid-draft; one survived only by finding an old snapshot and
still returned MATCH, which is luck, not safety. Drafters never write src/, which
is precisely why a dirty-tree check cannot see them: they DEPEND on state the
operation destroys. A guard belongs where the operation is (R54).
Skipping R22 DEFERS a fleet check rather than performing one, and a deferred check
nobody tracks reads as 'verified' at session close -- the same failure mode as a
loud error nobody counts (R32). So the skip now appends to .run/R22_DEBT with the
commit it deferred after, and a green clean-fleet run DELETES that file. The
session checkpoint must quote it.
stage_generated hard-coded build/<bin>/{<bin>.ld,undefined_*_auto.txt}. That is the
OVERLAY convention. main's Makefile variables put its linker script at
build/us/SLUS_007.26.ld and BOTH undefined_*_auto.txt at the REPO ROOT, so a
worktree got none of them, could not link, and every main draft came back rejected
-- indistinguishable from a wave of bad drafts. Measured this session: main banked
0 of 3 while the same drafts were match_one MATCH.
The tell was already being recorded and thrown away: the results JSON carried
missing_generated: [main.ld, undefined_syms_auto.txt, undefined_funcs_auto.txt]
and nothing consumed it -- R32's corrected form, a loud failure nobody counts is
exactly as invisible as a silent one. Same shape as R43's 'sweep_parallel accepted
main and banked 0/105'.
Now: paths come from the Makefile's own <b>_LD_SCRIPT / <b>_UNDEF_SYMS /
<b>_UNDEF_FUNCS (R33 -- derive from the invariant), are mirrored at the same
repo-relative location in the worktree, and a missing one REFUSES the binary with
the reason instead of gating it anyway (R43).
Negative control (R39): resolved and existence-checked across all 213 binaries --
0 would be refused, so the previously-succeeding population is untouched.
The S67 first attempt banked jtbl bodies in worktrees and left their carve config behind, so 13 of
213 went red (reverted commit:3396). A carve writes THREE things and the merge must carry all or none:
1. src/<bin>/*.c per-binary, adopted like any bank
2. config/splat.<bin>.yaml per-binary, adopted whole, baseline-checked
3. config/overlays.mk SHARED — adopt ONLY this binary's BLOCK
ovl_block()/splice_ovl_block() cut on the headers, so two workers carving
different binaries edit disjoint regions and cannot clobber each other. Same pinned-baseline refusal
as the per-file adopt, at block granularity — never a blanket file add
(the carve-state-files-never-blanket-add rule).
Verified: block round-trips byte-identically and leaves other binaries' blocks untouched.
Drew: "we need to parallel the jtbl stuff too. nothing should be serial."
THE BLOCKER: harvest_verify's jtbl carve runs `make extract`, and a worktree's asm/ is a SYMLINK to
the main tree (parallel_gate.py:77) — so a carving worker would rewrite the MAIN tree's asm while
other workers read it. That is the only reason jtbl drafts had a serial lane, and it cost ~1 hour to
gate 16 binaries in order to protect ONE jtbl draft this session.
THE FIX IS CHEAP, and the measurement is why: asm/ is 448 MB but ONE binary's subtree is 3.6-5.0 MB.
isolate_asm() replaces the blanket symlink with a real directory that SYMLINKS every other binary
(read-only, free) and holds a real COPY of just the binary being carved. `make extract BINARY=<b>`
then writes only inside the worktree. ~5 MB per worker on a box with 32 GB free.
Applied per JOB, not per worktree, because worker slots are reused across binaries — _drafts_carry_jtbl
uses the SAME predicate harvest_verify carves on (a jtbl_ reference in the target .s), so the router
and the gate cannot disagree (R33/R34).
NEGATIVE CONTROL: _drafts_carry_jtbl agrees with gate_wave.split()'s independent classification on
all 37 binaries of the S67 draft set, both directions.
parallel_gate.py — the per-binary byte gate was serial BY HARNESS, not by nature. Each binary already
compiles into its own build/<bin>/, links its own .ld and checks its own SHA; what serialized it was
shared mutable state in the ONE checkout (the splice, assert_write_set's GLOBAL git status, and the
deliberately-broad `git add -u src/` that must stay broad). Measured: a 109-binary sweep ran ~1
min/binary on a 32-core box at load 1.4 (~4% utilisation), and an xargs -P 4 attempt over the shared
tree CORRUPTED it earlier today.
Fix is ISOLATION, not locking: one git worktree per worker (own index, own src/, own build/). Workers
gate and NEVER commit; the orchestrator adopts only drafts the gate ACCEPTED, and only where the main
tree still matches the pinned baseline (otherwise REFUSED, never clobbered), then ONE commit and ONE
R22 clean-fleet sweep verifies the merged whole.
Measured on 85 binaries / 234 candidates: 178 banked in 12m19s wall for 127m40s CPU = 10.4x
parallelism, ~7x end-to-end vs serial, 99 files merged, 0 refused, check-all 213/213.
Five things a fresh worktree does NOT have, each found by measurement and each first appearing as
"the draft failed": splat-generated include/*.inc, the EMPTY tools/maspsx submodule, gitignored
tools/bin (cc1) + tools/psyq, build/{<bin>,assets/<bin>} outputs, and extracted/retail. Dirs mixing
tracked and untracked content cannot be symlinked wholesale (ln -s nests INSIDE them) — hence
link_missing(). The negative control that catches all of it: an UNMODIFIED binary must build
BYTE-IDENTICAL in the worktree. Before that control, the first parallel run reported a clean,
plausible "0 banked across 4 binaries" that was pure environment artifact.
twin_sweep.py — enumerate every open stub that has an ALREADY-BANKED structural twin, remap it
mechanically, gate it. Yields measured today: h_exact 139/157 = 88.5%, h_norm 36/45 then 178/234
= 76-80%. h_seq stays refused (Phase-26) and is not used. THE POOL REFILLS: each bank becomes an
exemplar for its siblings — 165 fresh candidates existed immediately after banking 178. Run it
BEFORE drawing any wave; t5_cards does not build seed_ref, so cards assert "no banked twin" for
these and agents redraft answers we already hold (a t5u Opus slot ground ov_SC03_023:func_8017BEBC
to closeness 45 while ov_SC02_004 held a byte-identical banked copy).
Drew T