THE TRUE IDENTITY OF THE LONG-STANDING 'main link defect' (2026-08-15). The extra C
function never broke the link; the RELINK it forced did.
integrate() derives each *_externals.ld from trial_undefined() against the CURRENT
ld_path, so its answer depends on how much of the linker script has ALREADY been
rewritten. On a virgin splat .ld the apicard region is still the stub object
(defining only firstfile2), so at the libmcrd stage 'firstfile' is undefined and
gets an entry. On an already-rewritten .ld, A66.o is present and defines
'firstfile' at 0x80062248, the trial no longer reports it undefined, and the entry
'firstfile = 0x80061FA8;' is DROPPED -- after which LIBMCRD's jal binds to A66.o
and main comes out 2 of 413,696 bytes different from retail (file 0x51674,
VA 0x80060E74, retail jal 0x80061FA8 vs built jal 0x80062248).
That is why main was green ONLY on the first build after a fresh extract, and it
is why NO main draft could ever bank through an incremental gate: the baseline was
already red before any draft was spliced.
integrate()'s own comment already CLAIMED this operation was idempotent ('a re-run
on an already-rewritten .ld only redoes syms'). This makes it true: the externals
map is merged with the file's prior contents, newly-derived values winning on a
name collision, names the new derivation no longer sees kept at their previous
address. The file becomes a function of the tree, not of how many times this ran.
It reports what it kept rather than doing it silently.
VERIFIED, three builds:
fresh extract + build ...... GREEN (unchanged)
INCREMENTAL relink ......... GREEN (was RED -- the failing case)
third relink ............... GREEN (monotonic across repeats)
and the merge is observed firing: 'kept 6/15/2 extern(s) this re-run no longer saw
as undefined' across the integrate stages.
Root-caused by a Fable agent, verified here against the bytes.
The third instance of the overlay-layout assumption, and the worst of them.
gate_stage synthesised --out 'build/<bin>/<bin>' and --good-sha from
'config/check.<bin>.sha'. For main BOTH are wrong: its image is
build/us/SLUS_007.26 (Makefile main_OUT) and its locked hash is
config/check.us.sha. So sha1(out) was None, _check_sha('main') found nothing, and
good_sha fell through to DEF_SHA -- ov_SC01_077's hash. EVERY main draft was
compared against a DIFFERENT BINARY'S SHA, auto-failed, reverted regardless of the
build, and reported as 'near' -- indistinguishable from a real codegen residual.
harvest_verify already owns these facts (its own comment: 'the Makefile and
config/check.<bin>.sha already state these facts; do not keep a second copy') and
refuses loudly when it cannot derive them. gate_stage's synthesised flags bypassed
both. Now they are passed through ONLY when a caller explicitly sets them. Same
defect the 2026-07-22 comment fixed on the CLI path for good_sha and left alive one
argument over, and in run_gate's API path.
Measured: three main drafts proven byte-perfect in the REAL link (whole image
differs from retail by 2 of 413,696 bytes, both a pre-existing baseline defect
unrelated to the drafts) reported {"banked": 0, "near": 3}.
NEGATIVE CONTROL (R39), zero-build, all 213 binaries: the (out, good_sha) pair
reaching harvest_verify is UNCHANGED for 212 of 213; main is the only one that
moves, from ('build/main/main', DEF_SHA=ov_SC01_077) to
('build/us/SLUS_007.26', 143dbb89...). 0 binaries have no derivable sha. The
derivation agrees with the Makefile's own $(BINARY)_OUT / $(BINARY)_CHECK_SHA for
main, resident and an overlay.
Three defects, all found by the tool's own zeros rather than by reading it.
1. VERDICTS KEYED BY ARM. An escalation is BY DEFINITION launched while the lower
tier's verdict already exists, so keying completion by (binary, fn) let the
in-flight FABLE draft be staged on the strength of the OPUS verdict -- the same
in-flight bug the verdict gate exists to prevent, one level up. Caught in a dry
run before it gated anything. An arm-less row still counts for every arm so a
hand-written backfill keeps working.
2. LEDGER KEYED BY ARM. Gating the opus draft of a function currently being
escalated used to ledger away the fable draft that follows it -- silently
discarding the escalation's product. The already-banked check is what stops a
genuine duplicate: once a function banks its stub is gone and every arm's draft
is skipped as banked-elsewhere. Legacy binary:fn entries for still-OPEN
functions were dropped so they get re-judged (9 of 14); banked ones kept.
3. --skip-binary. A gate that races a lane writing that binary's src/ produces a
FALSE verdict on a draft that is fine. Measured this session, by me: a
clean-fleet R22 raced an authorised src/800.c splice and reported '212 passed,
1 failed of 213' on a tree that rebuilt byte-identical minutes later. Being
clean RIGHT NOW is not the test; nothing being able to dirty it during the run
is -- and that is not something timing can be trusted to arrange.
The class banked 5 functions today (func_801457A4 x3 at the whale's end boundary,
func_80183830 x2 one region lower) so it deserved a sweep rather than a third
hand-derivation. It reads every splat yaml's _o0<letter> 'c' subsegs, takes the
START of the NEXT subseg as the boundary vaddr, and reports an open stub sitting
exactly there whose target carries the -O0 prologue tell.
RESULT: 141 binaries with an _o0 subseg, 288 boundaries examined, 0 candidates.
THE CLASS IS EXHAUSTED -- today's five were the last of it.
A sweep returning 0 must prove it CAN return non-zero, so that null is
negative-controlled: the 288 computed boundaries include 0x801457A4 in 138
binaries and 0x80183830 in exactly ov_SC03_118 + ov_SC03_119 -- i.e. it does find
the addresses it banked, they simply have no open stub any more.
Every rejected boundary is printed WITH ITS REASON and the denominator is printed
(R32): a sweep that reports only its hits cannot be told from one that scanned
nothing. It deliberately does not consult the family map -- rollout_o0 refuses this
recipe for a bookkeeping reason ('family with exemplar ... not found in the map'),
not a structural one, and is separately blind to any _o0 basename.
A draft file appears at <wave>/<arm>/<fn>.c long before its agent is finished --
agents iterate in place and the wave brief tells them to write the file, not to
write it last. Gating one mid-flight spends a build on unfinished work, records an
honest-looking rejection, and then LEDGERS it, so the FINISHED draft is skipped as
'already-gated' when it lands. That is a silent loss of the whole draft.
Measured this session: ov_SC01_000:func_8017E594 was gated at 0 banked while its
workflow was still running, and its ledger entry had to be cleared by hand.
Completion is now an explicit signal -- .run/gate_lane/verdicts.jsonl, one object
per RETURNED verdict, appended by the orchestrator. A quiet file mtime is
deliberately NOT accepted as one: an agent thinking for four minutes between edits
looks identical to a finished agent. --any-draft opts out, and says what it costs.
[gater] skipped 1 (IN-FLIGHT (no verdict yet)): ov_SC01_000:func_8017E594
rtu_match built its TU as src/<source>/<split>.c and its asm dir as
asm/<source>/nonmatchings/<split>. That is the OVERLAY layout. main keeps its
sources as LOOSE FILES in src/ (src/800.c) with asm at asm/nonmatchings/800, so
blocker_probe's 'stub.path.split("/")[1]' handed rtu_match '800.c' as the source
dir and it looked for src/800.c/800.c, then asm/src/nonmatchings/800/<fn>.s.
Every main draft came back ERR with an EMPTY detail -- indistinguishable from a
bad draft. The corpus Stub already carries both facts (.path and .asm_dir);
reconstructing them was the whole bug. rtu_match now takes --tu and refuses a
nonexistent TU with the reason instead of handing it to cpp (R43).
Proof it was the instrument, not the drafts: the same 4 main drafts, unchanged,
now probe MATCH 69 / DIFF 69-36-mismatched / MATCH 68 / MATCH 71.
3 of 4 are real-TU MATCH. Before this they were 4 of 4 ERR.
stage_generated hard-coded build/<bin>/{<bin>.ld,undefined_*_auto.txt}. That is the
OVERLAY convention. main's Makefile variables put its linker script at
build/us/SLUS_007.26.ld and BOTH undefined_*_auto.txt at the REPO ROOT, so a
worktree got none of them, could not link, and every main draft came back rejected
-- indistinguishable from a wave of bad drafts. Measured this session: main banked
0 of 3 while the same drafts were match_one MATCH.
The tell was already being recorded and thrown away: the results JSON carried
missing_generated: [main.ld, undefined_syms_auto.txt, undefined_funcs_auto.txt]
and nothing consumed it -- R32's corrected form, a loud failure nobody counts is
exactly as invisible as a silent one. Same shape as R43's 'sweep_parallel accepted
main and banked 0/105'.
Now: paths come from the Makefile's own <b>_LD_SCRIPT / <b>_UNDEF_SYMS /
<b>_UNDEF_FUNCS (R33 -- derive from the invariant), are mirrored at the same
repo-relative location in the worktree, and a missing one REFUSES the binary with
the reason instead of gating it anyway (R43).
Negative control (R39): resolved and existence-checked across all 213 binaries --
0 would be refused, so the previously-succeeding population is untouched.
Arm dirs are walked alphabetically, so 'fable' < 'opus' < 'sonnet' and the staging
copy silently OVERWROTE: a sonnet NEAR would have replaced the fable MATCH that was
escalated to rescue it. Measured live on main/func_800241C0 (sonnet closeness 19,
fable MATCH) -- the escalation's entire product would have been lost to a directory
listing order, and the gate would have reported an honest failure on the wrong draft.
Now ranked fable > opus > sonnet > v3 > haiku, and a collision is REPORTED, never
resolved silently:
[gater] main:func_800241C0 drafted by fable/sonnet — staging the fable draft
An escalation that re-derives what the cheaper tier already closed pays twice for
the same instructions. This passes the prior draft, its measured closeness and its
full residual report into the prompt, tells the agent to reproduce that closeness
first (and to STOP and report if it cannot -- R40), and forbids re-trying the
levers the prior agent already ruled out.
Points the agent at the escalation path the project actually has for a
compiler-internal residual (R17): the pinned gcc-2.7.2 source in-repo,
docs/gcc-2.7.2-map, and pass-disabling as a DIAGNOSTIC only.
Requires a 'new_idiom' field in the verdict: a Fable run that closes a function
but names no reusable lever has bought one function; one that names the lever
buys the class.
Drains a drafting wave's finished drafts into parallel_gate, grouped by binary,
while drafting keeps streaming. Accumulates to --min-drafts because same-binary
drafts must share a build (S67 had ov_SC05_010 x3 in one batch). --r22 by
default: it re-verifies the whole fleet from make clean after the merge and
aborts instead of committing a red binary -- the guard that would have caught
S67's '13 of 213 red, every one a jtbl binary' at once, for ~2.5 min.
Ledger keyed 'binary:fn' (R48 -- func_8017BEBC is a different function in
different overlays). A draft absent from its wave's targets.json is REFUSED
LOUDLY, never guessed at (R43); negative-controlled both directions: synthetic
unresolvable draft -> exit 1, clean tree -> exit 0, normal path unchanged.
Propagation, twin_sweep and harvest stay periodic and operator-driven: they need
aggregate, and cookbook 330 existed only because four instances landed in one wave.
The S67 FINAL-3 OPEN item, plus the two defects found while doing it.
* fix(dedup_propagate): the tool could not run AT ALL. S67's -j patch wrote
`os.environ` at module level in the one module that imports `os as _os`, so
every invocation died with NameError before doing any work. Propagation was
not deferred, it was impossible. Import-checked the other 7 -j-patched tools.
* propagation, honestly scoped: the real closable set is 11, not 32, derived two
independent ways that agree (seed_ref exact+same_addr, and a direct corpus
derivation). The 3,161-entry --auto-from plan over 53 overlays is dedup
hygiene over already-matched code and closes almost no open stub.
Applied: 2 banked byte-green (ov_SC04_018 func_80181270, func_80182AF8);
3 gate-refused and cleanly reverted; 6 blocked with named blockers
(3 CARRY-FIXABLE, 3 func_80144B9C not-inline-def -> needs the o0 whale carve).
R22 clean fleet: extract 212/212, check 213 passed 0 failed of 213, rc 0/0/0.
Frontier 453 -> 451.
* fix(seed_ref): REFUSE targets in LINKED subsegs. The playbook calls this tool
"the fleet-wide answer" and it reported 82 open stubs with a banked twin --
43 of them main stubs whose TUs the linker script never references. Any C
written there compiles, links and leaves the SHA1 green WHETHER OR NOT IT IS
CORRECT, so a mechanical twin lane fed from that list could have minted up to
43 gate-green FALSE matches the byte gate cannot see. draw_waves has refused
these since S66; this oracle did not. The refusal is counted and printed, not
silent. NC: guarded 39 subset of raw 82, all 43 dropped are main, the non-main
population is identical.
* wave drawn: .run/S68o1 (24 opus 187-770 ins) + .run/S68m1 (30 main), cards +
packs + wave_args asserted, queue of 53. Drafting opened at concurrency 5.
The S67 first attempt banked jtbl bodies in worktrees and left their carve config behind, so 13 of
213 went red (reverted commit:3396). A carve writes THREE things and the merge must carry all or none:
1. src/<bin>/*.c per-binary, adopted like any bank
2. config/splat.<bin>.yaml per-binary, adopted whole, baseline-checked
3. config/overlays.mk SHARED — adopt ONLY this binary's BLOCK
ovl_block()/splice_ovl_block() cut on the headers, so two workers carving
different binaries edit disjoint regions and cannot clobber each other. Same pinned-baseline refusal
as the per-file adopt, at block granularity — never a blanket file add
(the carve-state-files-never-blanket-add rule).
Verified: block round-trips byte-identically and leaves other binaries' blocks untouched.
Drew: "we need to parallel the jtbl stuff too. nothing should be serial."
THE BLOCKER: harvest_verify's jtbl carve runs `make extract`, and a worktree's asm/ is a SYMLINK to
the main tree (parallel_gate.py:77) — so a carving worker would rewrite the MAIN tree's asm while
other workers read it. That is the only reason jtbl drafts had a serial lane, and it cost ~1 hour to
gate 16 binaries in order to protect ONE jtbl draft this session.
THE FIX IS CHEAP, and the measurement is why: asm/ is 448 MB but ONE binary's subtree is 3.6-5.0 MB.
isolate_asm() replaces the blanket symlink with a real directory that SYMLINKS every other binary
(read-only, free) and holds a real COPY of just the binary being carved. `make extract BINARY=<b>`
then writes only inside the worktree. ~5 MB per worker on a box with 32 GB free.
Applied per JOB, not per worktree, because worker slots are reused across binaries — _drafts_carry_jtbl
uses the SAME predicate harvest_verify carves on (a jtbl_ reference in the target .s), so the router
and the gate cannot disagree (R33/R34).
NEGATIVE CONTROL: _drafts_carry_jtbl agrees with gate_wave.split()'s independent classification on
all 37 binaries of the S67 draft set, both directions.
Completes the S67 audit: 0 remaining 'make build' call sites without -j. family_sweep is the one
that matters most of the three — twin_sweep delegates to it, so every sibling remap now gets the
6.1x build too.
NOT parallelizable, checked: 'make extract' is a single `splat split` process, so -j cannot help
it; its cost is splat's own runtime. The 4 extract calls in harvest_verify's jtbl carve path stay
as they are.
MEASURED on ov_SC03_010 (35 objects), clean each time, byte-verified against the locked SHA:
make build 7.18 s real / 6.84 s user <- SERIAL, one core, on a 32-thread box
make -j16 build 1.18 s real / 11.3 s user <- 6.1x, IDENTICAL bytes
Negative control at -j32 over ov_SC03_010 + ov_SC01_004 + md_MAIN_031: all rc=0, all byte-identical
to config/check.<bin>.sha.
WHY IT WAS MISSED: the Makefile's `JOBS ?= 16` is parallelism ACROSS binaries (`xargs -P`), which
parallel_gate already uses for extract-all/check-all. Parallelism WITHIN one binary's ~35 objects was
never passed by any tool, though docs/SETUP.md:416 documents `make -j$(nproc) build` as the form.
PATCHED the two hot sites:
* harvest_verify.py — the gate's build, run ONCE PER DRAFT (--chunk 1). Every gate in the project.
* dedup_propagate.py byte_gate — run once per propagation candidate, which is why a wide
propagation dominated a 33-minute gate this session.
Both honour BFM_BUILD_JOBS, else os.cpu_count().
SAFE BY CONSTRUCTION: these builds feed a locked-SHA comparison, so a bad parallel build FAILS the
gate rather than banking wrong bytes. The error direction is a false NEGATIVE, never a false bank;
G3/P9 remains the sole arbiter.
Correction recorded: I earlier extrapolated "9 serial binaries x 30 min" from ONE 33-minute
measurement. That was unfounded — propagation time scales with how many sites a body reaches, and
other gates today propagated x19/x8/x7 quickly. One slow binary is not a rate (R41).
THE DOC GAP, and it cost tokens this session. `docs/automation-runbook.md` was titled "the
autonomous campaign, as it actually runs" while documenting the RETIRED OpenRouter/ox-alpha system
whose lanes are all deliberately DEAD. The current Claude-wave pipeline existed only as two dense
tooling-inventory rows in SETUP.md — reference, not procedure. Three of this session's costliest
mistakes were procedural and a playbook prevents each:
* hand-typed a refill target -> invented func_80184F60 (2nd instruction of a matched function), 58k
* hand-rolled a serial gate loop when parallel_gate existed -> ~1h for what took 103s
* re-derived a function banked verbatim in ~20 overlays -> 102k
NEW docs/wave-playbook.md — start to finish, each guard paired with the MEASUREMENT that produced it
(that pairing is the part a generic decomp guide cannot have, and the seed of the future template).
automation-runbook.md retitled HISTORICAL with a pointer; SETUP.md §6.9 links the playbook.
NEW tools/seed_ref.py — the cross-TU banked twin, joined on corpus signature hashes (no atlas knn,
~2s fleet-wide), wired into t5_cards.py. FLEET: 87 open stubs have a banked twin; 41 of them sit in
twin_sweep's refusal ledger, invisible to BOTH tools at once. Documents twin_sweep's two holes:
load_sigs covers 141/213 binaries (main, resident, all md_MAIN_* absent), and one curated symbol
name silently disables an entire binary via a bare `except Exception: pass`.
Schema note: seed_ref's binary/fn are the EXEMPLAR's, because api_agent greps src/{binary} for {fn};
naming them after the target would send every agent grepping for itself — caught pre-ship.
HARVEST §333-§338 from the s67o2_1/pool_1 waves:
§333 frame size is set by DECLARED aggregates, not used ones — an unreferenced trailing local is a
dial (3 instances; one worth 30 of 32 residual rows)
§334 a reload spill slot rounds to BIGGEST_ALIGNMENT for align AND size: one 4-byte pseudo grew a
frame by 16 (82->53)
§335 `extern u16 A[]` at a variable subscript allocates ~8B/access of dead stack temps that inflate
the frame with ZERO extra instructions — invisible in a body diff (141->20)
§336 the §5a barrier goes at the BOTTOM of the twin; find_cross_jump walks BACKWARD
§337 the CC1-ONLY blocker class: blocker_probe's static oracle says "none" and cc1 still fails
§338 _sltiu_bounds misreads a non-switch sltiu as a bounds check, over-spanning the table
gate_wave.py now STREAMS both lanes (R55) — it captured output and printed at the end, leaving a
zero-byte log indistinguishable from a hang.
HARVEST — the s67o1/s67m1 wave banked 7 cookbook sections:
* §325 a shared small constant stored twice in the pre-loop block is a LOCAL-ALLOC $s-occupant that
steals the argument allocno's register — pin the ARGUMENT-derived local, not the constant
(pinning the constant reached only closeness 15). byte-proven func_80184F18.
* §326 spelling two reads of the same halfword differently (sym[i] vs *(s16*)(base+i*4+2)) yields
different address rtx and DEFEATS address-CSE, restoring separate %hi/%lo groups. func_8017FAAC.
* §327 a range test must be HImode: with s32 + a (u16) cast gcc PROVES the mask redundant and drops
the andi — a real -1 length drift that reads as a schedule. +3 levers. func_8017EC34.
* §328 NEW LAW: the volatile alias must be an aliased OBJECT; `*(volatile s32*)&sym` unfolds %lo
into a separate addiu (+1 ins). func_80181B8C.
* §329 fold-const narrows `(int)s16 & 0xFFF` onto the RAW HImode pseudo, breaking the
sign-extend/mask register tie; a zero-byte `s32 e = t;` widening temp restores it (30 rows -> 0).
* §330 the NEIGHBOUR-SHAPE lever, four independent instances in one wave — copy an already-banked
in-TU function's SPELLING before any codegen reasoning (one dissolved 18 REGALLOC-PERM rows in a
single compile). Corollary: a warm start from another binary is often worth LESS than the
neighbour 20 lines away.
* §331 OPEN GAP, recorded as unsolved: no lever eliminates an UNWANTED DUPLICATE copy at a
branch-target block head (main/func_80013154, closeness 12, ~16 iterations, 5 approaches refuted).
TOOLIFY — tools/gate_wave.py: split the batch on the per-draft jtbl predicate, run parallel_gate
and the serial jtbl lane CONCURRENTLY. Measured this session: 4 binaries in 103s wall through
parallel_gate (87/87/88/102s each) vs ~6 min serially; I had gated all 16 serially to protect ONE
jtbl draft, ~1 hour. The split precedes the run because a jtbl worker does NOT fail cleanly — it
re-extracts through the worktree's asm/ symlink and writes the MAIN tree while other workers read it.
Its own negative control found two defects in it before first use:
* listdir counted gate_stage's _xform output dirs (-cn/-cast/-rc/-sd, written as SIBLINGS inside
the drafts root) as binaries: 20 "binaries" for a 16-binary wave. Now validated against
progress.BINARIES and refused loudly (R32/R43).
* a post-hoc control over BANKED functions cannot reproduce a split (has_jtbl has no stub to read);
re-controlled against a live draft set, where it correctly routes the two functions the gate had
independently reported CARVE-REFUSED.
R22 caught it: 212/213 after the S67-cc1 gate run. `ov_SC04_018` was RED.
ROOT CAUSE (from the diff, not inferred). Commit commit:3354's propagation replaced three bodies in
`ov_SC04_018_jr_80135D20.c` with DEFINE_func_*() instantiations and deleted the 981 lines they
occupied — INCLUDING the TU's file-scope declaration layer, which the two surviving non-deduped
bodies still referenced. A duplicate copy of those decls survived at line 225, BELOW the function
that uses them at line 42, so C89 ordering made it fatal (`D_8018D7A4' undeclared).
THE STRUCTURAL GAP: gate_stage byte-gates the SOURCE binary, then propagation writes to N OTHER
binaries and nothing re-verifies them. "fleet 99.2%" in the commit subject is a metric, not a gate.
This is the blind spot R50 exists for, and only the periodic whole-fleet R22 could see it.
REPAIR: restored src/ov_SC04_018 to commit:3354^, re-extracted (banking had pruned the .s stubs the
restored INCLUDE_ASM lines need), rebuilt rc=0 at the locked SHA fe9b413f. dedup-check clean
(2193 validated, 0 failed, C1 255302/255302). Cost: the 2 banks in that binary.
NEW tools/restore_dropped_decls.py — compiler-driven recovery for this failure mode: build, read
which identifiers cc1 calls undeclared, look each one up in the pre-deletion git ref, insert it
above the leading #include block, repeat. Two defects found and fixed in it while using it:
* anchoring after "the last extern in the first 400 lines" inserts BELOW the point of use, so the
build fails identically and the loop re-inserts forever (measured: 25 rounds, 100 dead decls).
The only safe anchor is the top of the file.
* a no-progress guard now REFUSES when a round asks for what the last round already inserted.
It also correctly refused when the failure changed class (link-level undefined references), which
is how the wider damage was found rather than papered over.
NOT a defect of the S67 §8d rung: scope_demote_drafts only ever writes draft dirs under .run/.
FLEET: make clean + extract-all + check-all = 213 passed, 0 failed of 213.
FRONTIER: 530 -> 526 (4 functions closed this session, measured from corpus.stubs).
MEASURED (denominators in .run/S67_findings.md):
* 193 of the 530 open functions ALREADY have a draft on disk (1,885 wave targets seen,
1,521 banked, 171 open-no-draft, 166 never drawn). Classified in their real TUs:
37 MATCH / 67 NEAR / 89 CC1-FAIL.
* 159 open functions (30% of the frontier) reference a jump table; 96 are PLAN-REFUSED
by build_carve (non-contiguous same-subseg .rodata), 75 non-main across 38 subsegs.
Not a codegen wall and not a decl wall — carve plumbing.
NEW
* tools/strand_census.py — coverage-asserted census + rtu_match classifier + draft staging.
Keys binary:fn (R48); classifies each pair once after merging every manifest's view.
* tools/o0_detect.py — the -O0 prologue tell extracted from match_one (which parses argv at
import and therefore cannot be imported). match_one re-exports it; ONE definition (R33).
Wiring it into the classifier turned md_MAIN_003 from 8 NEAR (7 of them >20) into 6 MATCH.
Negative-controlled both directions.
* tools/scope_demote_drafts.py — §8d as an _xform-contract gate rung. NOT yet exercised.
FIXED
* jtbl_carve --probe now runs build_carve (a pure planner) and reports plan-refused. It
previously called only island_probe, which answers a necessary-not-sufficient question —
every blocked function probed "carveable", and S66 priced 32 of them as free on that.
* blocker_probe.macro_scope selects the LAST #define per macro name, matching cpp.
engine_core.h has 1,037 duplicate DEFINE_func_ names and 4 with DIFFERENT bodies.
NOT VALIDATED — DO NOT SCALE
* jr_isolate_all: two real defects fixed (carried types deduped by name; header-provided
types no longer re-emitted) but ov_SC02_000 STILL fails the byte gate after them.
Open lead: file_scope_types carries a block without its enclosing #if guard. 20 of 35
blocked overlays dry-run clean and that number means nothing until one round-trips.
0 functions banked this session. tools-health has ONE pre-existing cdecl defect
(1 of 74,749 declarations, func_8017EE08_p55352/struct ZnRec) — cdecl.py and its inputs
are byte-identical to HEAD, so it is not from this change.
Knowledge banked: cookbook §322/§323/§323a/§323b, decision-log pivot, accelerators #13/#14.
draw_waves.py gains --only-main (the main lane draws main and nothing else; implies --main so the
LINKED refusal still applies). Progress/backlog regenerated: fleet 99.2% instruction-weighted,
98.1% distinct.
Round-3 harvest measured a RETRIEVAL defect, not a knowledge gap: func_800CDBA8 and func_800CB00C
both re-derived laws the book already states (§165-19/§162d1/§30#3 birthing-boost; the
§135-1/§165-28 unsigned-switch-selector rule) by grinding match_one, with an EMPTY
cookbook_refs_used — they never grepped the index. SYS.md already tells them to ('START AT
docs/cookbook-index.md — it is symptom-keyed'); the instruction is there and gets skipped.
So stop asking. When the pack has ALREADY MEASURED the prior draft's residual and therefore knows
its klass, paste the matching symptom bucket from docs/cookbook-index.md straight into the pack,
capped at 40 rows with a pointer to the rest. Retrieval becomes push, not pull.
Mapping is klass -> bucket (REGALLOC-PERM/LOCAL -> register allocation & pins; SCHEDULE-REORDER,
OPCODE-MIXED, LENGTH-DRIFT -> instruction scheduling; DELAY-SLOT/BRANCH-POLARITY -> delay slots &
branches; WIDTH/IMM-VALUE/IMM-OFFSET -> types, signedness & width; ADDRESSING -> CSE; CC1-FAIL ->
declarations). An unmapped klass yields no bucket rather than a wrong one.
Negative control on the O21 slate, both directions: 12 packs carry a measured residual, 10 of those
get a bucket (2 have an unmapped klass and correctly get none); packs with no residual get none.
The loop closes visibly — the func_8017EDE8 pack now carries this round's OWN addendum about
§137's barrier failing in a cross-jump shared tail, which is exactly that function's residual class.
Measured against corpus.stubs at gate time on this session's own waves:
w2 80 drawn / 12 still open -> 85% of its agents re-derived already-banked functions
w3 80 drawn / 39 still open -> 51%
x1 80 drawn / 71 still open -> 11%
~109 of 240 agents in w2+w3 spent full budgets on work that was already banked, by the concurrent
parallel_gate commit or by sibling propagation from a twin remap. The agents DID notice ('stale
pack', 'target .s no longer exists') but only after reading the pack, and several reconstructed the
banked body just to have something to submit.
draw_waves filters against the DRAWN ledger, which stops drawing a target twice but says nothing
about whether it is still OPEN. In a campaign where gates land continuously, a wave drawn at T and
launched at T+2h is stale by construction. Openness is the same class of assertion wave_args already
makes about the .s and the pack, so it belongs here, where it costs nothing.
Negative control (both extremes): x1 drops exactly its 9 closed targets and keeps 71; w2 drops 68
and keeps 12 — matching the independent corpus.stubs measurement.
- match_one.py: a CPP/CC1/MASPSX/AS failure printed bare text and exited 1 even under --json, so
every programmatic caller got json.loads of a non-JSON line. claude_wave_packs swallowed 4 of 19
prior drafts as 'residual not measured: Expecting value' — the most actionable datum a pack can
carry (the draft does not COMPILE, here is the error) was the one it dropped. Negative control:
a near draft still measures identically (closeness 2, same residual rows); human mode unchanged.
- claude_wave_packs.py: renders that verdict, pointing the agent at the card's decl_prior block.
Coverage on wave r1 went 15/19 -> 19/19 packs carrying a measured verdict.
- wave_args.py (new): emits the claude_wave_draft.js args from <wave>/targets.json, asserting the
.s exists, that sub is exactly its parent dir, and that the pack exists. Written because I
hand-typed sub as 'ov_SC03_112/jr_80181D08' when the truth is
'asm/ov_SC03_112/nonmatchings/ov_SC03_112_jr_801817E0' (a stub's asm dir is named for its jr-carve
block, not itself) — all 19 agent oracles would have failed identically and read as a model
failure. Negative control: that exact string is REFUSED.
- draw_waves.py (new): draws waves off corpus.stubs cheapest-first, ledger-filtered, with
name-collision deferral (packs are name-keyed and refuse a duplicate). Re-measured the frontier:
the S65 tier map's '~557 cheap singletons (3-17 ins)' conflated one-member FAMILIES with small
functions — only 28 undrawn non-main stubs are <=17 ins; the bulk is 51-120 (258) and >120 (245).
The lock I added earlier today guards a real hazard (the driver mutates the shared tree and is not
parallel-safe), but I scoped it to the whole tool instead of the mutating path. --probe-only execs
blocker_probe, which compiles in its own scratch dir and touches nothing — excluding it buys no
safety and costs a free diagnostic.
Measured cost: a t7b drafting agent (func_801832E0) tried the probe TWICE, was refused both times by
a concurrent sweep holding the lock, and submitted with its blocker unconfirmed — exactly the $0
diagnostic the pack tells agents to run first.
Control: with the lock held, --probe-only now returns rc 0 and its verdict table; the mutating path
still returns rc 1 REFUSED.
The seam decays hard as it is worked out. Measured across four rounds today: 139/157 = 88.5%, then
178/234 = 76%, then 71/165 = 43%, then 1/95 = 1%. By the last round almost every candidate was one a
previous round had already tried and the gate had refused, so the sweep spent ~7 minutes of builds to
bank one function.
A refusal is deterministic for a given (target, EXEMPLAR) pair — the same exemplar remaps to the same
text — so the ledger keys on both and skips those by default. A NEW exemplar for the same target is a
different question and is retried automatically, which matters because the pool refills as banking
mints exemplars. --retry-refused overrides.
The ledger starts empty (today's rounds predate it) and populates from .run/pgate_results.json.
The distill prompt asserted 'the whole-binary byte-gate ACCEPTED the final draft, so the final body is
ground truth' for EVERY target. With --with-unbanked now feeding it drafts the gate REFUSED, that
sentence would have laundered an unproven body into a byte-proven cookbook entry (R14/G3). A target
with banked=False now gets an explicit PROVENANCE WARNING telling the distiller to extract the lever
anyway (cookbook 52: a model that FAILS still distils the idiom that cracks its siblings) but to mark
every claim UNPROVEN and never assert byte-equality; the verifier is told the same so its
entry_markdown carries the label. Exercised this session: the one surviving ADDENDUM came from an
unbanked draft and is filed UNPROVEN.
Also regenerates the scoreboard after the day's banks.
parallel_gate.py — the per-binary byte gate was serial BY HARNESS, not by nature. Each binary already
compiles into its own build/<bin>/, links its own .ld and checks its own SHA; what serialized it was
shared mutable state in the ONE checkout (the splice, assert_write_set's GLOBAL git status, and the
deliberately-broad `git add -u src/` that must stay broad). Measured: a 109-binary sweep ran ~1
min/binary on a 32-core box at load 1.4 (~4% utilisation), and an xargs -P 4 attempt over the shared
tree CORRUPTED it earlier today.
Fix is ISOLATION, not locking: one git worktree per worker (own index, own src/, own build/). Workers
gate and NEVER commit; the orchestrator adopts only drafts the gate ACCEPTED, and only where the main
tree still matches the pinned baseline (otherwise REFUSED, never clobbered), then ONE commit and ONE
R22 clean-fleet sweep verifies the merged whole.
Measured on 85 binaries / 234 candidates: 178 banked in 12m19s wall for 127m40s CPU = 10.4x
parallelism, ~7x end-to-end vs serial, 99 files merged, 0 refused, check-all 213/213.
Five things a fresh worktree does NOT have, each found by measurement and each first appearing as
"the draft failed": splat-generated include/*.inc, the EMPTY tools/maspsx submodule, gitignored
tools/bin (cc1) + tools/psyq, build/{<bin>,assets/<bin>} outputs, and extracted/retail. Dirs mixing
tracked and untracked content cannot be symlinked wholesale (ln -s nests INSIDE them) — hence
link_missing(). The negative control that catches all of it: an UNMODIFIED binary must build
BYTE-IDENTICAL in the worktree. Before that control, the first parallel run reported a clean,
plausible "0 banked across 4 binaries" that was pure environment artifact.
twin_sweep.py — enumerate every open stub that has an ALREADY-BANKED structural twin, remap it
mechanically, gate it. Yields measured today: h_exact 139/157 = 88.5%, h_norm 36/45 then 178/234
= 76-80%. h_seq stays refused (Phase-26) and is not used. THE POOL REFILLS: each bank becomes an
exemplar for its siblings — 165 fresh candidates existed immediately after banking 178. Run it
BEFORE drawing any wave; t5_cards does not build seed_ref, so cards assert "no banked twin" for
these and agents redraft answers we already hold (a t5u Opus slot ground ov_SC03_023:func_8017BEBC
to closeness 45 while ov_SC02_004 held a byte-identical banked copy).
A pack carrying a previous attempt said only 'the gate did not accept it, so it is wrong somewhere'.
That discards the one datum that decides how the agent spends its budget. Now the builder runs
match_one on that draft and embeds the verdict:
* near -> the closeness, the verdict sig, and the residual rows (idx / mine / tgt, capped at 16
with an honest '... N more'), plus how to READ them: two adjacent rows with the same
instructions in the opposite order = a SCHEDULE swap; a register-only difference = the
value came from the wrong place (often the copy, not the pre-copy value); a beqz/bnez
row = invert the test and swap the arms, constants included.
* match -> NOT a drafting job. The body is byte-correct in isolation and the gate refused it for an
INTEGRATION reason, so the pack names the $0 recover_integration --probe-only instead of
letting an agent burn a wave slot redrafting a correct body. If the probe also says
MATCH the residual is outside the function (the section-8e JTBL_PADS class).
Measured on the two t5u seeds, which I injected BY HAND this session before automating it:
func_8017F234 = 3 mismatched of 202 (a schedule swap + an "andi" reading the copy instead of the
pre-copy value); func_8017E7E8 = 11 of 66 (inverted branch + a cast written back into the variable
instead of a temp). Told that, an agent edits one use site; told "wrong somewhere", it re-derives 202
instructions.
Cost: one compile per target that HAS a prior draft; --no-residual opts out. Failures are swallowed
into a "(residual not measured: ...)" line — measuring must never break pack generation.
Control: rebuilt t5u's 15 packs into a scratch dir — both seeds gained the block automatically with
the same numbers I measured by hand, and a target with no prior draft is byte-identical to before.
Three properties compose into tree corruption under concurrency:
(a) assert_write_set measures a GLOBAL git status, so a concurrent run's writes read as THIS
run's blast-radius violation and abort it;
(b) an abort does NOT restore the stage edits already on disk;
(c) gate_stage's commit is a deliberately broad 'git add -u src/' — and it must be, since
propagation touches many overlays and a narrower filename glob once DROPPED four R22-verified
banks — so a concurrent --commit sweeps the aborted run's half-applied edits into its commit.
Measured today: xargs -P 4 over 33 binaries put 696 broken lines of ov_MAIN_012 into md_MAIN_026's
+1 bank commit; check-all went 212/213 and the wave bank was blocked behind it (R59).
Narrowing the gate's git add was the WRONG fix (it would restore defect (c)'s predecessor). Instead
the driver enforces its own contract: flock on .run/recover/.driver.lock, refuse loudly (R43).
Control: with the lock held -> rc 1 REFUSED; lock free -> rc 0 and the probe runs normally.
Two defects in one filter, both measured on the t5s wave (24 banked / 29 transcripts):
1. FALSE POSITIVE. The keyword 'no cookbook lever' matched "MATCH on first compile, no cookbook
lever needed" — a note reporting a TRIVIAL function — and that was the ONLY selection out of 24,
while three genuine multi-lever notes went unpicked. A selector whose single hit is the one note
saying 'nothing to learn here' is inverted, not merely noisy. Keyword removed, NOT_NOVEL guard
added, and the phrasings agents actually use ('cookbook lacks', 'new lever', 'worth banking',
'levers not in') added. Same 24-transcript scope now selects func_8017E044 instead.
2. STRUCTURAL BLINDNESS. The distiller only ever considered BANKED functions — but the richest
idiom notes come from the HARDEST functions, which are the least likely to bank. func_8017EB30
(279 ins, four levers written up) and func_8017C014 (246 ins, two) both say 'NOT in the cookbook
and worth banking' and were never candidates. That defeats cookbook §52 — a model that FAILS to
crack a wall still distills the idiom that cracks its siblings — using the flywheel's own tool.
New --with-unbanked includes them, each carrying banked=False so the distilling agent knows the
lever is UNPROVEN by the byte gate (R14/G3).
R39 control: the previously-selected note is no longer selected (it was the false positive) and
nothing legitimately selected was dropped.
os.execv'd tools/blocker_probe.py with --drafts <run_dir>/drafts while that directory was still
created further down, so every non---draft-dir probe died with FileNotFoundError. Only --draft-dir
worked, because stage_drafts() had already populated the dir. Staging now happens first.
Control: the --draft-dir path returns the same verdict as before the move (ov_SC06_029
func_80185214 -> DIFF 52/52 ins, identical to the pre-edit run). --funcs now works: 10 backlog
candidates classified in one pass (2 real-TU MATCH, 3 conflicting-types, 2 too-few-arguments,
1 parse error).
Worth recording (R40): my own probe loop grepped for result rows and swallowed the traceback, so the
crash read as 'no blockers found' — a silently narrowed scope in the harness, not the tool.
mask_for(reloc_kind='26') returned 0, i.e. 'compare NOTHING at this position'. Both comparers pick
the mask from ONE side (diff_object_s from mine, diff_object_object from the target's), so a j/jal
there masked the OTHER side's instruction entirely. Reproduced on synthetic pairs of real encodings:
my 'j 8017e248' (0805f892) vs target 'bne v0,v1' (14430002) -> 0; vs 'nop' (00000000) -> 0; my 'jal'
vs target 'bne' -> 0; while the mirror (my 'bne' vs target 'j') -> 1. That asymmetry is the bug.
_j_mismatch cannot cover it: it fires only when BOTH sides carry an internal-j target, which a
j-vs-bne pair by definition does not.
Fix: return 0xFC000000 — the 26-bit target field stays masked (it IS link-time), the opcode never
is. diff_object_object's masked-slot test updated to match so the reloc symbol+addend check still
fires there.
R39 negative control (tools/stub_invariant_audit.py, the INCLUDE_ASM invariant): 2554 stubs, nonzero
3 before and 3 after — the same three known main length-delta survivors, same values. Zero new false
positives, over a population that exercises the changed path (812 stubs carry internal-j .text
relocs, 3164 such instructions).
Found by a t5s drafting agent on func_8017EB30 (reported as a one-sided internal-j check); verified
here to be broader than reported. No bank was ever at risk — the whole-binary gate is independent
(G3) — but every crack agent and the permuter scorer read this number. R35/R14.
recover_integration's macro-externs stage rewrote a draft's callee extern to the FLEET macro's
signature and then gated only the rewrite. func_ADDR names are per-address, not per-function, so
another overlay's 'extern void func_8017C338(void)' replaced this overlay's correct 4-arg decl and
manufactured the CC1-FAIL it reported as the draft's failure. The untouched draft banks
byte-identical (ov_SC03_012:func_8017BEBC, 246 ins, banked in the previous commit).
reconcile_and_gate(draft_rewrite=) now gates raw (pass 1a) then rewrites only what raw refused
(pass 1b), records the winning variant per fn, and re-gates that variant in pass 2.
harvest_verify.classify_fail kept the 'note:' half of a benign warning pair and labelled a built
draft CC1-FAIL with it; notes now drop with their warnings. Negative-controlled over 5 diagnostic
shapes — only warning+note-only changed (to the honest no-diagnostic label). R39/R57/R32.
Cookbook 920 -> 921 sections, index green.
Drew T