Commit Graph

3420 Commits

Author SHA1 Message Date
Drew T f7fee7fcde feat(decomp): main banks again — 3 fns (func_80012B58, func_800241C0, func_800242D0)
The first main banks since the two harness defects were fixed. All three were
proven byte-perfect in the real link by the Fable investigation BEFORE any fix,
and reported {banked:0, near:3} purely because:
  * gate_stage compared main against ov_SC01_077's SHA (build/main/main never
    exists, config/check.main.sha never exists, DEF_SHA took over), and
  * psyq_integrate dropped 'firstfile = 0x80061FA8;' on every incremental relink,
    so main's BASELINE was already 2 bytes red before a draft was spliced.

func_800242D0 additionally needed one reconcile: it declared 'extern u16
D_80063870' while the just-banked func_800241C0 declares 'extern s16
D_80063870[]' at file scope. gcc-2.7.2 rejects the conflicting redeclaration at
file scope AND at block scope (the block-scope shadow was tried and also
rejected), so the draft now matches the banked spelling and takes the address by
array decay. Only the address is used (t4 is a 'register s16 *'), so the element
type never reaches codegen -- and the whole-binary SHA proves it.

harvest_verify in the main tree: verified 3 / failed 0, final SHA
143dbb89f34491258bbc27810d0a12ec8b43a8dd BYTE-IDENTICAL. main 1048 -> 1045 stubs.

NOTE for the next session: parallel_gate's WORKTREE still cannot gate main (its
generated-input staging covers the 3 Makefile-named files but main's link needs
more). main is one binary, so gate it in the main tree with harvest_verify --
there is no parallelism to lose.
2026-08-31 17:08:32 -06:00
Drew T 3ebbec9426 fix(psyq_integrate): main was RED on every incremental relink — make the externals file monotonic
THE TRUE IDENTITY OF THE LONG-STANDING 'main link defect' (2026-08-15). The extra C
function never broke the link; the RELINK it forced did.

integrate() derives each *_externals.ld from trial_undefined() against the CURRENT
ld_path, so its answer depends on how much of the linker script has ALREADY been
rewritten. On a virgin splat .ld the apicard region is still the stub object
(defining only firstfile2), so at the libmcrd stage 'firstfile' is undefined and
gets an entry. On an already-rewritten .ld, A66.o is present and defines
'firstfile' at 0x80062248, the trial no longer reports it undefined, and the entry
'firstfile = 0x80061FA8;' is DROPPED -- after which LIBMCRD's jal binds to A66.o
and main comes out 2 of 413,696 bytes different from retail (file 0x51674,
VA 0x80060E74, retail jal 0x80061FA8 vs built jal 0x80062248).

That is why main was green ONLY on the first build after a fresh extract, and it
is why NO main draft could ever bank through an incremental gate: the baseline was
already red before any draft was spliced.

integrate()'s own comment already CLAIMED this operation was idempotent ('a re-run
on an already-rewritten .ld only redoes syms'). This makes it true: the externals
map is merged with the file's prior contents, newly-derived values winning on a
name collision, names the new derivation no longer sees kept at their previous
address. The file becomes a function of the tree, not of how many times this ran.
It reports what it kept rather than doing it silently.

VERIFIED, three builds:
  fresh extract + build ...... GREEN (unchanged)
  INCREMENTAL relink ......... GREEN (was RED -- the failing case)
  third relink ............... GREEN (monotonic across repeats)
and the merge is observed firing: 'kept 6/15/2 extern(s) this re-run no longer saw
as undefined' across the integrate stages.

Root-caused by a Fable agent, verified here against the bytes.
2026-08-31 17:03:50 -06:00
Drew T 14a72240d2 feat(decomp): parallel gate — 2 fns across 2 binaries (5 workers)
ov_SC01_000    func_8017E594
  ov_SC07_001    func_8017F834
2026-08-31 16:59:49 -06:00
Drew T fd28dd714d fix(gate_stage): main was gated against ov_SC01_077's SHA — stop synthesising out/good_sha
The third instance of the overlay-layout assumption, and the worst of them.

gate_stage synthesised --out 'build/<bin>/<bin>' and --good-sha from
'config/check.<bin>.sha'. For main BOTH are wrong: its image is
build/us/SLUS_007.26 (Makefile main_OUT) and its locked hash is
config/check.us.sha. So sha1(out) was None, _check_sha('main') found nothing, and
good_sha fell through to DEF_SHA -- ov_SC01_077's hash. EVERY main draft was
compared against a DIFFERENT BINARY'S SHA, auto-failed, reverted regardless of the
build, and reported as 'near' -- indistinguishable from a real codegen residual.

harvest_verify already owns these facts (its own comment: 'the Makefile and
config/check.<bin>.sha already state these facts; do not keep a second copy') and
refuses loudly when it cannot derive them. gate_stage's synthesised flags bypassed
both. Now they are passed through ONLY when a caller explicitly sets them. Same
defect the 2026-07-22 comment fixed on the CLI path for good_sha and left alive one
argument over, and in run_gate's API path.

Measured: three main drafts proven byte-perfect in the REAL link (whole image
differs from retail by 2 of 413,696 bytes, both a pre-existing baseline defect
unrelated to the drafts) reported {"banked": 0, "near": 3}.

NEGATIVE CONTROL (R39), zero-build, all 213 binaries: the (out, good_sha) pair
reaching harvest_verify is UNCHANGED for 212 of 213; main is the only one that
moves, from ('build/main/main', DEF_SHA=ov_SC01_077) to
('build/us/SLUS_007.26', 143dbb89...). 0 binaries have no derivable sha. The
derivation agrees with the Makefile's own $(BINARY)_OUT / $(BINARY)_CHECK_SHA for
main, resident and an overlay.
2026-08-31 16:59:17 -06:00
Drew T 3c534853f2 fix(gater): key verdicts AND the ledger by ARM; add --skip-binary for lanes that may be writing
Three defects, all found by the tool's own zeros rather than by reading it.

1. VERDICTS KEYED BY ARM. An escalation is BY DEFINITION launched while the lower
   tier's verdict already exists, so keying completion by (binary, fn) let the
   in-flight FABLE draft be staged on the strength of the OPUS verdict -- the same
   in-flight bug the verdict gate exists to prevent, one level up. Caught in a dry
   run before it gated anything. An arm-less row still counts for every arm so a
   hand-written backfill keeps working.

2. LEDGER KEYED BY ARM. Gating the opus draft of a function currently being
   escalated used to ledger away the fable draft that follows it -- silently
   discarding the escalation's product. The already-banked check is what stops a
   genuine duplicate: once a function banks its stub is gone and every arm's draft
   is skipped as banked-elsewhere. Legacy binary:fn entries for still-OPEN
   functions were dropped so they get re-judged (9 of 14); banked ones kept.

3. --skip-binary. A gate that races a lane writing that binary's src/ produces a
   FALSE verdict on a draft that is fine. Measured this session, by me: a
   clean-fleet R22 raced an authorised src/800.c splice and reported '212 passed,
   1 failed of 213' on a tree that rebuilt byte-identical minutes later. Being
   clean RIGHT NOW is not the test; nothing being able to dirty it during the run
   is -- and that is not something timing can be trusted to arrange.
2026-08-31 16:56:32 -06:00
Drew T 7153f881c9 feat(o0): o0_boundary.py — the stranded-boundary -O0 sweep, and its honest null
The class banked 5 functions today (func_801457A4 x3 at the whale's end boundary,
func_80183830 x2 one region lower) so it deserved a sweep rather than a third
hand-derivation. It reads every splat yaml's _o0<letter> 'c' subsegs, takes the
START of the NEXT subseg as the boundary vaddr, and reports an open stub sitting
exactly there whose target carries the -O0 prologue tell.

RESULT: 141 binaries with an _o0 subseg, 288 boundaries examined, 0 candidates.
THE CLASS IS EXHAUSTED -- today's five were the last of it.

A sweep returning 0 must prove it CAN return non-zero, so that null is
negative-controlled: the 288 computed boundaries include 0x801457A4 in 138
binaries and 0x80183830 in exactly ov_SC03_118 + ov_SC03_119 -- i.e. it does find
the addresses it banked, they simply have no open stub any more.

Every rejected boundary is printed WITH ITS REASON and the denominator is printed
(R32): a sweep that reports only its hits cannot be told from one that scanned
nothing. It deliberately does not consult the family map -- rollout_o0 refuses this
recipe for a bookkeeping reason ('family with exemplar ... not found in the map'),
not a structural one, and is separately blind to any _o0 basename.
2026-08-31 16:50:09 -06:00
Drew T c746204c32 feat(decomp): -O0 stranded-boundary bank — func_80183830 in ov_SC03_118 + ov_SC03_119, zero agent tokens
The same shape as func_801457A4 at the whale's end boundary (cookbook §362), one
region lower: _o0d spans 0x80183178..0x80183830 and func_80183830 is the FIRST
function of the -O2 jr_80183830 object immediately after it. Its target carries the
-O0 prologue tell, so it can only bank in an -O0 object -- and _o0d's .text ends
exactly at its address, so the def lands correctly with NO splat change.

ATOMIC ACROSS TWO FILES: append the def to <ov>_o0d.c AND drop the INCLUDE_ASM from
<ov>_jr_80183830.c in one edit, so the two object sizes cancel and no address moves.

Body mechanically remapped from the banked twin ov_SC03_014:0x801842E0 (2
per-overlay symbols substituted); match_one closeness 0 on both before gating.
Both BYTE-IDENTICAL against their check.sha.

rollout_o0 could not drive this: 'family with exemplar func_80183830 not found in
the map' -- the family map has no entry, so the driver refuses. The recipe is the
tool; the map is not a precondition for it.
2026-08-31 16:48:34 -06:00
Drew T 9c96b47ce0 docs(phase-31): S68 progress — whale carve (6 fns/2,547 ins), the main-layout bug class, gater hardening 2026-08-31 16:46:57 -06:00
Drew T ff99f4acec fix(gater): never gate a draft whose workflow has not returned a verdict
A draft file appears at <wave>/<arm>/<fn>.c long before its agent is finished --
agents iterate in place and the wave brief tells them to write the file, not to
write it last. Gating one mid-flight spends a build on unfinished work, records an
honest-looking rejection, and then LEDGERS it, so the FINISHED draft is skipped as
'already-gated' when it lands. That is a silent loss of the whole draft.

Measured this session: ov_SC01_000:func_8017E594 was gated at 0 banked while its
workflow was still running, and its ledger entry had to be cleared by hand.

Completion is now an explicit signal -- .run/gate_lane/verdicts.jsonl, one object
per RETURNED verdict, appended by the orchestrator. A quiet file mtime is
deliberately NOT accepted as one: an agent thinking for four minutes between edits
looks identical to a finished agent. --any-draft opts out, and says what it costs.

  [gater] skipped 1 (IN-FLIGHT (no verdict yet)): ov_SC01_000:func_8017E594
2026-08-31 16:45:58 -06:00
Drew T c55cebea9b docs(cookbook): S68 harvest — §354-§362, nine sections from the wave and the whale carve
§354 the giv worth-while test as a dial (re-associate the addend into the index
     term; strength_reduce declines and $fp is freed) - ov_SC03_105/func_801824CC
§355 a remapped sibling's SOURCE bias is not its EMITTED bias; gcc re-anchors
     reduced givs, so do NOT hand-shift offsets to match the asm
§356 measure a draft in the TU it will live in: 39 of 43 'undeclared' cc1-fails
     were the standalone probe's environment, not the draft (R35)
§357 one struct pointer, not two - a second source variable builds a THIRD iv
§358 sharpens §333: an UNREFERENCED fixed-size aggregate local is load-bearing;
     expand_decl slots every aggregate, so an unused decl is a frame-layout knob
§359 spell a sign-widen as an explicit two-step function-scoped temp; a single
     (s16) cast and a register pin both measured FAILED
§360 the 'compiler found a shorter equivalent' pair - with its third lever marked
     REFUTED rather than deleted, so nobody re-derives it
§361 ★ a loop-tail byte signature that names its source shape, and the law that a
     'scheduling tie' may be an artifact of your own earlier lever. The prior
     agent's sched1 diagnosis was WRONG and its own hack was the cause.
     Escalation economics: sonnet 229k tokens no bank, fable 74k tokens MATCH.
§362 two traps when a carve moves a stub into the -O0 TU (rollout_o0 goes blind;
     the §8b decl layer conflicts with the shared header on 7 symbols)

Index regenerated: 1013 sections, 14 symptom buckets.
2026-08-31 16:37:35 -06:00
Drew T d3b72e8f64 fix(rtu_match/blocker_probe): main was structurally unprobeable — pass the TU path and asm subdir, never reconstruct them
rtu_match built its TU as src/<source>/<split>.c and its asm dir as
asm/<source>/nonmatchings/<split>. That is the OVERLAY layout. main keeps its
sources as LOOSE FILES in src/ (src/800.c) with asm at asm/nonmatchings/800, so
blocker_probe's 'stub.path.split("/")[1]' handed rtu_match '800.c' as the source
dir and it looked for src/800.c/800.c, then asm/src/nonmatchings/800/<fn>.s.

Every main draft came back ERR with an EMPTY detail -- indistinguishable from a
bad draft. The corpus Stub already carries both facts (.path and .asm_dir);
reconstructing them was the whole bug. rtu_match now takes --tu and refuses a
nonexistent TU with the reason instead of handing it to cpp (R43).

Proof it was the instrument, not the drafts: the same 4 main drafts, unchanged,
now probe MATCH 69 / DIFF 69-36-mismatched / MATCH 68 / MATCH 71.
3 of 4 are real-TU MATCH. Before this they were 4 of 4 ERR.
2026-08-31 16:34:01 -06:00
Drew T 8f89cbcccd fix(pgate): main could never bank in a worktree — derive link inputs from the Makefile, and REFUSE when they are absent
stage_generated hard-coded build/<bin>/{<bin>.ld,undefined_*_auto.txt}. That is the
OVERLAY convention. main's Makefile variables put its linker script at
build/us/SLUS_007.26.ld and BOTH undefined_*_auto.txt at the REPO ROOT, so a
worktree got none of them, could not link, and every main draft came back rejected
-- indistinguishable from a wave of bad drafts. Measured this session: main banked
0 of 3 while the same drafts were match_one MATCH.

The tell was already being recorded and thrown away: the results JSON carried
missing_generated: [main.ld, undefined_syms_auto.txt, undefined_funcs_auto.txt]
and nothing consumed it -- R32's corrected form, a loud failure nobody counts is
exactly as invisible as a silent one. Same shape as R43's 'sweep_parallel accepted
main and banked 0/105'.

Now: paths come from the Makefile's own <b>_LD_SCRIPT / <b>_UNDEF_SYMS /
<b>_UNDEF_FUNCS (R33 -- derive from the invariant), are mirrored at the same
repo-relative location in the worktree, and a missing one REFUSES the binary with
the reason instead of gating it anyway (R43).

Negative control (R39): resolved and existence-checked across all 213 binaries --
0 would be refused, so the previously-succeeding population is untouched.
2026-08-31 16:29:45 -06:00
Drew T c1200003ac fix(gater): escalation supersedes — highest-tier arm wins a (binary,fn) collision
Arm dirs are walked alphabetically, so 'fable' < 'opus' < 'sonnet' and the staging
copy silently OVERWROTE: a sonnet NEAR would have replaced the fable MATCH that was
escalated to rescue it. Measured live on main/func_800241C0 (sonnet closeness 19,
fable MATCH) -- the escalation's entire product would have been lost to a directory
listing order, and the gate would have reported an honest failure on the wrong draft.

Now ranked fable > opus > sonnet > v3 > haiku, and a collision is REPORTED, never
resolved silently:
  [gater] main:func_800241C0 drafted by fable/sonnet — staging the fable draft
2026-08-31 16:26:23 -06:00
Drew T f03deb6c99 feat(decomp): -O0 whale bank — ov_SC02_037 + ov_SC03_107, 4 fns / 1,698 ins, zero agent tokens
func_80144B9C (770) + func_801457A4 (79) in each. Both BYTE-IDENTICAL against
their check.sha. ov_SC02_037 now has 0 open stubs (BINARY COMPLETE);
ov_SC03_107 has 1 left.

Per-overlay the whale's 770 instructions are byte-identical (sha1 74186b97e5d9
across all six overlays sampled) so the shared header is exact; func_801457A4
differs by exactly one data symbol, remapped per overlay:
  ov_MAIN_012 D_8017E338 | ov_SC02_037 D_80183BC0 | ov_SC03_107 D_8018245C

This closes the LAST 6 of 6 route-ready -O0 whale members fleet-wide
(rollout_o0 --all-o0 reported TOTAL {'no-o0b': 6} before this).
2026-08-31 16:23:01 -06:00
Drew T 7a89bc4fb4 feat(config): -O0 whale carve step 1 — ov_SC02_037 + ov_SC03_107
Same split as ov_MAIN_012: 0x80144B9C..0x801458E0 out of <ov>_jr_8013F350.
2 unmatched stubs, 0 already-matched islands, carve repoints (none),
config/overlays.mk UNCHANGED. Both byte-neutral against their check.sha and both
interleave_check ALIGNED (33/33 and 30/30). Derived and carved under
.run/auto/gate.<ov>.lock.
2026-08-31 16:22:41 -06:00
Drew T 61335ad18e feat(decomp): -O0 whale bank — ov_MAIN_012 func_80144B9C (770) + func_801457A4 (79), BINARY COMPLETE
Replaced the carve's generated _o0d.c wholesale with the minimal fleet-standard
whale TU. Keeping the generated §8b carried decl layer was NOT an option: it
conflicts with shared/func_80144B9C.h on 7 symbols (func_80015978 void*/s32,
func_800CF854 void/s32, func_801336E8, D_801274C8/CC/D0). Legitimate to drop it
here because the region holds ONLY these two functions (0xD44 = 0xC08 + 0x13C
exactly), so nothing in the TU needs the carried decls.

func_801457A4 remapped from ov_SC01_077_o0b.c: its 79 instructions differ across
overlays by exactly ONE data symbol (D_80186AD0 -> D_8017E338).

Byte-identical: d6b3e8b971cdd6c53aea8c4f265afb82b363283c == config/check.ov_MAIN_012.sha.
corpus.stubs(ov_MAIN_012) = 0 -- the binary has no open stubs left.

NOT via rollout_o0: its stub_file_of() skips any basename containing _o0, and the
carve moved BOTH stubs into _o0d.c, so the driver is structurally blind to them
and would have reported 'no-stub / already banked?' and banked nothing.
2026-08-31 16:21:13 -06:00
Drew T ff524cd07e feat(config): -O0 whale carve step 1 — split 0x80144B9C..0x801458E0 out of ov_MAIN_012_jr_8013F350
o0_subsplit: 2 unmatched stubs (func_80144B9C 770 ins + func_801457A4 79 ins), 0
already-matched islands interleaved, so K=0 and one -O0 region is correct. carve
repoints (none); config/overlays.mk UNCHANGED (the whale object owns no .rodata
carve anywhere in the fleet: 0 of 213 splat yamls carve .rodata to an _o0b).

BYTE-NEUTRAL, which is the whole claim of a carve:
  build d6b3e8b971cdd6c53aea8c4f265afb82b363283c == config/check.ov_MAIN_012.sha
interleave_check ALIGNED. Derived AND carved under .run/auto/gate.ov_MAIN_012.lock
(the commit:2791 rule: a plan derived outside the lock can describe a tree state that
never existed).
2026-08-31 16:20:38 -06:00
Drew T dd608ce5fd feat(decomp): parallel gate — 5 fns across 5 binaries (6 workers)
md_SC04_027    func_801EB014
  ov_SC07_007    func_8013DD68
  md_MAIN_003    func_800D1E9C
  ov_SC03_105    func_801824CC
  ov_SC06_029    func_80181DF8
2026-08-31 16:19:19 -06:00
Drew T e57deb0a8b feat(wave): escalate_fable.js — a Fable tier that is warm-started, not restarted
An escalation that re-derives what the cheaper tier already closed pays twice for
the same instructions. This passes the prior draft, its measured closeness and its
full residual report into the prompt, tells the agent to reproduce that closeness
first (and to STOP and report if it cannot -- R40), and forbids re-trying the
levers the prior agent already ruled out.

Points the agent at the escalation path the project actually has for a
compiler-internal residual (R17): the pinned gcc-2.7.2 source in-repo,
docs/gcc-2.7.2-map, and pass-disabling as a DIAGNOSTIC only.

Requires a 'new_idiom' field in the verdict: a Fable run that closes a function
but names no reusable lever has bought one function; one that names the lever
buys the class.
2026-08-31 16:18:00 -06:00
Drew T 380308b954 feat(gater): tools/gater_lane.py — the continuous gater lane
Drains a drafting wave's finished drafts into parallel_gate, grouped by binary,
while drafting keeps streaming. Accumulates to --min-drafts because same-binary
drafts must share a build (S67 had ov_SC05_010 x3 in one batch). --r22 by
default: it re-verifies the whole fleet from make clean after the merge and
aborts instead of committing a red binary -- the guard that would have caught
S67's '13 of 213 red, every one a jtbl binary' at once, for ~2.5 min.

Ledger keyed 'binary:fn' (R48 -- func_8017BEBC is a different function in
different overlays). A draft absent from its wave's targets.json is REFUSED
LOUDLY, never guessed at (R43); negative-controlled both directions: synthetic
unresolvable draft -> exit 1, clean tree -> exit 0, normal path unchanged.
Propagation, twin_sweep and harvest stay periodic and operator-driven: they need
aggregate, and cookbook 330 existed only because four instances landed in one wave.
2026-08-31 16:02:32 -06:00
Drew T ed53a68f18 feat(p31 s68): deferred propagation done honestly (2 banked) + seed_ref was offering DEAD TEXT
The S67 FINAL-3 OPEN item, plus the two defects found while doing it.

* fix(dedup_propagate): the tool could not run AT ALL. S67's -j patch wrote
  `os.environ` at module level in the one module that imports `os as _os`, so
  every invocation died with NameError before doing any work. Propagation was
  not deferred, it was impossible. Import-checked the other 7 -j-patched tools.

* propagation, honestly scoped: the real closable set is 11, not 32, derived two
  independent ways that agree (seed_ref exact+same_addr, and a direct corpus
  derivation). The 3,161-entry --auto-from plan over 53 overlays is dedup
  hygiene over already-matched code and closes almost no open stub.
  Applied: 2 banked byte-green (ov_SC04_018 func_80181270, func_80182AF8);
  3 gate-refused and cleanly reverted; 6 blocked with named blockers
  (3 CARRY-FIXABLE, 3 func_80144B9C not-inline-def -> needs the o0 whale carve).
  R22 clean fleet: extract 212/212, check 213 passed 0 failed of 213, rc 0/0/0.
  Frontier 453 -> 451.

* fix(seed_ref): REFUSE targets in LINKED subsegs. The playbook calls this tool
  "the fleet-wide answer" and it reported 82 open stubs with a banked twin --
  43 of them main stubs whose TUs the linker script never references. Any C
  written there compiles, links and leaves the SHA1 green WHETHER OR NOT IT IS
  CORRECT, so a mechanical twin lane fed from that list could have minted up to
  43 gate-green FALSE matches the byte gate cannot see. draw_waves has refused
  these since S66; this oracle did not. The refusal is counted and printed, not
  silent. NC: guarded 39 subset of raw 82, all 43 dropped are main, the non-main
  population is identical.

* wave drawn: .run/S68o1 (24 opus 187-770 ins) + .run/S68m1 (30 main), cards +
  packs + wave_args asserted, queue of 53. Drafting opened at concurrency 5.
2026-08-31 15:59:01 -06:00
Drew T 746a7cf656 docs(phase-31): S67 FINAL-3 — 77 closed (530 -> 453), fleet 213/213, jtbl parallelised (both halves), propagation deferred 2026-08-31 15:23:25 -06:00
Drew T 51abe2d174 feat(decomp): parallel gate — 19 fns across 14 binaries (8 workers)
ov_SC03_110    func_8018035C
  ov_SC04_000    func_8017F98C
  ov_SC04_015    func_8017E520
  ov_SC03_121    func_8017D8D4 func_80180E64
  ov_SC04_011    func_801827DC
  ov_SC03_007    func_801810E4 func_80184AFC
  ov_SC03_105    func_8017F55C func_801831E0
  ov_SC02_003    func_8017F62C
  ov_SC06_024    func_8017E1F0
  ov_SC05_001    func_80183C9C
  ov_SC06_030    func_8017F268 func_80181164
  ov_SC05_010    func_8018340C func_8018388C
  ov_SC04_016    func_8017D118
  ov_SC06_022    func_8017EBE8
2026-08-31 15:19:41 -06:00
Drew T 93ea53217e docs(playbook): a carve writes THREE outputs — merge all three, splice overlays.mk per block, gate jtbl with --r22 2026-08-31 15:17:27 -06:00
Drew T 101dec757a fix(pgate): merge carve STATE too — jtbl parallelizes for real now
The S67 first attempt banked jtbl bodies in worktrees and left their carve config behind, so 13 of
213 went red (reverted commit:3396). A carve writes THREE things and the merge must carry all or none:
  1. src/<bin>/*.c            per-binary, adopted like any bank
  2. config/splat.<bin>.yaml  per-binary, adopted whole, baseline-checked
  3. config/overlays.mk       SHARED — adopt ONLY this binary's BLOCK

ovl_block()/splice_ovl_block() cut on the  headers, so two workers carving
different binaries edit disjoint regions and cannot clobber each other. Same pinned-baseline refusal
as the per-file adopt, at block granularity — never a blanket file add
(the carve-state-files-never-blanket-add rule).

Verified: block round-trips byte-identically and leaves other binaries' blocks untouched.
2026-08-31 15:14:10 -06:00
Drew T a732097c37 Revert "feat(decomp): parallel gate — 19 fns across 14 binaries (8 workers)"
This reverts commit commit:3394.
2026-08-31 15:09:32 -06:00
Drew T 5bcb322283 docs: gating is fully parallel — no serial lane; jtbl unlocked via isolate_asm; launch detached with setsid 2026-08-31 14:55:39 -06:00
Drew T c26ddf136d feat(decomp): parallel gate — 19 fns across 14 binaries (8 workers)
ov_SC04_000    func_8017F98C
  ov_SC04_015    func_8017E520
  ov_SC03_121    func_8017D8D4 func_80180E64
  ov_SC03_110    func_8018035C
  ov_SC04_011    func_801827DC
  ov_SC03_007    func_801810E4 func_80184AFC
  ov_SC03_105    func_8017F55C func_801831E0
  ov_SC02_003    func_8017F62C
  ov_SC05_001    func_80183C9C
  ov_SC06_024    func_8017E1F0
  ov_SC06_030    func_8017F268 func_80181164
  ov_SC05_010    func_8018340C func_8018388C
  ov_SC04_016    func_8017D118
  ov_SC06_022    func_8017EBE8
2026-08-31 14:53:41 -06:00
Drew T 2b618caf6b perf(pgate): jtbl drafts now parallelize — isolate ONE binary's asm instead of symlinking all 448MB
Drew: "we need to parallel the jtbl stuff too. nothing should be serial."

THE BLOCKER: harvest_verify's jtbl carve runs `make extract`, and a worktree's asm/ is a SYMLINK to
the main tree (parallel_gate.py:77) — so a carving worker would rewrite the MAIN tree's asm while
other workers read it. That is the only reason jtbl drafts had a serial lane, and it cost ~1 hour to
gate 16 binaries in order to protect ONE jtbl draft this session.

THE FIX IS CHEAP, and the measurement is why: asm/ is 448 MB but ONE binary's subtree is 3.6-5.0 MB.
isolate_asm() replaces the blanket symlink with a real directory that SYMLINKS every other binary
(read-only, free) and holds a real COPY of just the binary being carved. `make extract BINARY=<b>`
then writes only inside the worktree. ~5 MB per worker on a box with 32 GB free.

Applied per JOB, not per worktree, because worker slots are reused across binaries — _drafts_carry_jtbl
uses the SAME predicate harvest_verify carves on (a jtbl_ reference in the target .s), so the router
and the gate cannot disagree (R33/R34).

NEGATIVE CONTROL: _drafts_carry_jtbl agrees with gate_wave.split()'s independent classification on
all 37 binaries of the S67 draft set, both directions.
2026-08-31 14:46:23 -06:00
Drew T dfc5d5b3c5 feat(decomp): parallel gate — 13 fns across 13 binaries (12 workers)
md_SC07_004    func_801A3180
  ov_SC01_080    func_801809E4
  ov_SC03_006    func_801823B8
  ov_SC01_001    func_801810A4
  ov_SC02_005    func_80190538
  ov_SC02_031    func_80183458
  ov_SC01_084    func_80181A7C
  ov_SC03_092    func_8017E780
  ov_SC04_008    func_8017D3F8
  ov_SC03_102    func_80180C38
  ov_SC03_107    func_8017CF48
  ov_SC07_006    func_8013DD68
  ov_SC07_002    func_80181394
2026-08-31 14:45:51 -06:00
Drew T 389d7d6197 feat(decomp): wave gate — +1 fns x0 propagated (fleet 99.2%) 2026-08-31 14:41:41 -06:00
Drew T 3a4eac272e docs(cookbook): §353 -fno-thread-jumps as a pass-identification oracle; launder the value to keep a dead re-test 2026-08-31 14:37:16 -06:00
Drew T 91a622b4c5 docs(cookbook): §352 CRITICAL — two identical zero-byte barriers merge with EACH OTHER, defeating their purpose 2026-08-31 14:34:16 -06:00
Drew T b39db34b56 docs(cookbook): §351 /s is a per-access dial; the base-split spelling; a pin-induced sched1 residual with the remaining door named 2026-08-31 14:33:55 -06:00
Drew T bbddb26e68 docs(cookbook): §350 the zero-byte re-tie also kills sched1's birthing_insn_p boost — attribute WHICH pass it moved 2026-08-31 14:32:30 -06:00
Drew T fcada63ada docs(cookbook): §349 n_times_set>1 on a base pointer defeats both the invariant hoist and the address giv 2026-08-31 14:27:27 -06:00
Drew T 932977cc5a docs(cookbook): §348 base spelling picks the addressing mode (3-insn lui/%lo vs 2-insn addu/lw); §137 refuted on constants 2026-08-31 14:27:10 -06:00
Drew T 6bd3d6ba42 perf: -j on the last three per-binary make build sites (family_sweep/twin_sweep, rollout_o0, restore_dropped_decls)
Completes the S67 audit: 0 remaining 'make build' call sites without -j. family_sweep is the one
that matters most of the three — twin_sweep delegates to it, so every sibling remap now gets the
6.1x build too.

NOT parallelizable, checked: 'make extract' is a single `splat split` process, so -j cannot help
it; its cost is splat's own runtime. The 4 extract calls in harvest_verify's jtbl carve path stay
as they are.
2026-08-31 14:24:03 -06:00
Drew T 6ad89305b8 docs(cookbook): §347/§343 addenda — one variable per purpose (3rd instance); sltiu proves an unsigned return 2026-08-31 14:21:45 -06:00
Drew T 096da8e4ff docs(cookbook): §347 loop regalloc is a declaration-order/live-range dial (no live-range splitting in gcc-2.7.2) 2026-08-31 14:21:25 -06:00
Drew T 675b4702b3 perf(gate): pass -j to the per-binary build — 6.1x on the inner loop of every gate
MEASURED on ov_SC03_010 (35 objects), clean each time, byte-verified against the locked SHA:
    make build            7.18 s real / 6.84 s user   <- SERIAL, one core, on a 32-thread box
    make -j16 build       1.18 s real / 11.3 s user   <- 6.1x, IDENTICAL bytes
Negative control at -j32 over ov_SC03_010 + ov_SC01_004 + md_MAIN_031: all rc=0, all byte-identical
to config/check.<bin>.sha.

WHY IT WAS MISSED: the Makefile's `JOBS ?= 16` is parallelism ACROSS binaries (`xargs -P`), which
parallel_gate already uses for extract-all/check-all. Parallelism WITHIN one binary's ~35 objects was
never passed by any tool, though docs/SETUP.md:416 documents `make -j$(nproc) build` as the form.

PATCHED the two hot sites:
* harvest_verify.py — the gate's build, run ONCE PER DRAFT (--chunk 1). Every gate in the project.
* dedup_propagate.py byte_gate — run once per propagation candidate, which is why a wide
  propagation dominated a 33-minute gate this session.
Both honour BFM_BUILD_JOBS, else os.cpu_count().

SAFE BY CONSTRUCTION: these builds feed a locked-SHA comparison, so a bad parallel build FAILS the
gate rather than banking wrong bytes. The error direction is a false NEGATIVE, never a false bank;
G3/P9 remains the sole arbiter.

Correction recorded: I earlier extrapolated "9 serial binaries x 30 min" from ONE 33-minute
measurement. That was unfounded — propagation time scales with how many sites a body reaches, and
other gates today propagated x19/x8/x7 quickly. One slow binary is not a rate (R41).
2026-08-31 14:18:05 -06:00
Drew T bd9b9f827d docs(cookbook): §346 COND_EXPR singleton path — c?X:-X negates in place, if/else does not 2026-08-31 14:14:53 -06:00
Drew T 34da91b45e docs(p31 s67): harvest §339-§345 from the 30-workflow streaming burst
Seven sections from 30 single-function opus workflows on 187-297 instruction targets (30/30 MATCH):
 §339 a 2-case switch OMITS gcc's low-bound range test (stmt.c emit_case_nodes) — so slti/bnez
      between two beqs is a COUNT TELL that a case node is missing from your draft
 §340 §194-K corollary: a 'scheduler' residual can be sched.c's ALIAS ORACLE inventing a false
      true-dependence; source order picks the edge's DIRECTION, so reverse it into an anti-dep
      rather than fighting it (10->0, zero bytes; 3 alternatives refuted with reasons)
 §341 an HImode store temp reweights a sched2 tie no statement order can reach
 §342 NEW LAW: a twin's  param cast in a local is NOT byte-neutral when a later param also
      needs a callee-saved reg — and the §333 converse does NOT hold (gcc may already pad the gap)
 §343 decl_prior's fleet MAJORITY can be wrong about the true signature — read the RIVALS.
      Measured: void(s32) x1374 vs the truth s32(s32) x163. The tool is honest, the corpus is wrong.
 §344 raise a biv's global_alloc priority with a zero-byte REFERENCE; a register pin kills LSR
 §345 volatile STORE evicts the MEM from cse and keeps sh; volatile LOAD blocks combine and
      degrades lh into lhu+sll+sra — the qualifier is not symmetric

Also: seed_ref validated on a live A/B. The same 187-ins body cost 102,193 tokens / 476 s in
ov_SC03_107 when the card said 'no banked twin', and 72,077 tokens / 135 s in ov_SC07_006 once the
card carried the twin — 30% fewer tokens, 3.5x faster. A second instance (func_8017F62C) went
63,595 vs 118,485 tokens. others_open=137 on that one exemplar, so it compounds.
2026-08-31 14:14:26 -06:00
Drew T 022d50325a docs(playbook): the pgrep bracket is NOT enough when launch and wait share a shell
Measured a SECOND time in S67, and the first fix was incomplete. A waiter using the bracketed
pattern still matched itself and spun 1h35m, because the same shell command had LAUNCHED the job —
so its own command line carried the unbracketed 'gate_stage.py --binary ov_SC07_007' from the nohup
half. The regex gate_[s]tage.py does not match the literal bracketed text, but it happily matches
the plain text sitting earlier on the same line.

Rule is now: launch and wait in SEPARATE shell invocations, or better, wait on a completion MARKER
the job writes to its own log rather than on process liveness.
2026-08-31 13:51:48 -06:00
Drew T 00812fc62f docs(p31 s67): wave-playbook (the CURRENT pipeline) + seed_ref fix + harvest §333-§338
THE DOC GAP, and it cost tokens this session. `docs/automation-runbook.md` was titled "the
autonomous campaign, as it actually runs" while documenting the RETIRED OpenRouter/ox-alpha system
whose lanes are all deliberately DEAD. The current Claude-wave pipeline existed only as two dense
tooling-inventory rows in SETUP.md — reference, not procedure. Three of this session's costliest
mistakes were procedural and a playbook prevents each:
  * hand-typed a refill target -> invented func_80184F60 (2nd instruction of a matched function), 58k
  * hand-rolled a serial gate loop when parallel_gate existed -> ~1h for what took 103s
  * re-derived a function banked verbatim in ~20 overlays -> 102k

NEW docs/wave-playbook.md — start to finish, each guard paired with the MEASUREMENT that produced it
(that pairing is the part a generic decomp guide cannot have, and the seed of the future template).
automation-runbook.md retitled HISTORICAL with a pointer; SETUP.md §6.9 links the playbook.

NEW tools/seed_ref.py — the cross-TU banked twin, joined on corpus signature hashes (no atlas knn,
~2s fleet-wide), wired into t5_cards.py. FLEET: 87 open stubs have a banked twin; 41 of them sit in
twin_sweep's refusal ledger, invisible to BOTH tools at once. Documents twin_sweep's two holes:
load_sigs covers 141/213 binaries (main, resident, all md_MAIN_* absent), and one curated symbol
name silently disables an entire binary via a bare `except Exception: pass`.
Schema note: seed_ref's binary/fn are the EXEMPLAR's, because api_agent greps src/{binary} for {fn};
naming them after the target would send every agent grepping for itself — caught pre-ship.

HARVEST §333-§338 from the s67o2_1/pool_1 waves:
 §333 frame size is set by DECLARED aggregates, not used ones — an unreferenced trailing local is a
      dial (3 instances; one worth 30 of 32 residual rows)
 §334 a reload spill slot rounds to BIGGEST_ALIGNMENT for align AND size: one 4-byte pseudo grew a
      frame by 16 (82->53)
 §335 `extern u16 A[]` at a variable subscript allocates ~8B/access of dead stack temps that inflate
      the frame with ZERO extra instructions — invisible in a body diff (141->20)
 §336 the §5a barrier goes at the BOTTOM of the twin; find_cross_jump walks BACKWARD
 §337 the CC1-ONLY blocker class: blocker_probe's static oracle says "none" and cc1 still fails
 §338 _sltiu_bounds misreads a non-switch sltiu as a bounds check, over-spanning the table

gate_wave.py now STREAMS both lanes (R55) — it captured output and printed at the end, leaving a
zero-byte log indistinguishable from a hang.
2026-08-31 13:48:56 -06:00
Drew T d989ed08ae feat(decomp): parallel gate — 14 fns across 11 binaries (6 workers)
md_SC07_003    func_801A38E4
  md_MAIN_027    func_800CB4A4
  ov_SC02_037    func_8013DD68
  ov_SC02_003    func_8018046C
  ov_SC01_084    func_8017EA98 func_80181F80 func_80184138
  ov_SC03_107    func_8013DD68
  ov_SC05_003    func_8017E380
  ov_SC05_008    func_8017DF68 func_8017EBB8
  ov_SC05_010    func_8017FAF4
  ov_SC06_032    func_80190D70
  ov_SC06_025    func_8017EA74
2026-08-31 13:43:44 -06:00
Drew T 21a2212ae3 feat(decomp): S67 main r2 — 1 banked (main 89 -> 88) + §332/§332a toolchain-wall findings
Wave s67m2_1: 7 sonnet agents, 1 MATCH banked, 6 NEAR — but 4 of the 7 are NOT drafting failures:
* func_8005FA94 / func_8005D244 — oracle_reorder.py bypass gives 0/55 and 0/62 diffs: the C is
  byte-correct, the pinned as -O1 cannot emit the §188 epilogue. func_8005D244 is additionally
  libpad pdent3.o, an SDK object owned by psyq_integrate.py — it should never have been drawn.
* func_80062144 / func_8005DBD8 — §332, traced to the compiler sources: gcc-2.7.2 emits a symbolic
  la as ONE atomic length-2 insn (no HIGH/LO_SUM split in this backend), eligible_for_delay requires
  length==1, so it can never fill a jump delay slot; the retail split is ASPSX macro-hopping that
  maspsx does not replicate. Byte-verified by running maspsx over cc1's raw -dS output.
  6 such functions fleet-wide, NONE banked.

§332a records the draw-policy consequence: main's cheap population is spent and the residual is
ENRICHED in toolchain walls, so main's apparent match rate is contamination, not a model signal.
Wall ledger at .run/S67_walls.txt for the --exclude mechanism.
2026-08-31 13:21:01 -06:00
Drew T b8a4eae41a docs(phase-31): S67 FINAL-2 — 29 closed (530 -> 501), fleet 213/213, wave 20/20 MATCH, gate_wave.py replaces the serial loop 2026-08-31 12:48:46 -06:00
Drew T 6bb3d240fa feat(p31 s67): harvest §325-§331 + gate_wave.py (split jtbl/parallel, both lanes concurrent)
HARVEST — the s67o1/s67m1 wave banked 7 cookbook sections:
* §325 a shared small constant stored twice in the pre-loop block is a LOCAL-ALLOC $s-occupant that
  steals the argument allocno's register — pin the ARGUMENT-derived local, not the constant
  (pinning the constant reached only closeness 15). byte-proven func_80184F18.
* §326 spelling two reads of the same halfword differently (sym[i] vs *(s16*)(base+i*4+2)) yields
  different address rtx and DEFEATS address-CSE, restoring separate %hi/%lo groups. func_8017FAAC.
* §327 a range test must be HImode: with s32 + a (u16) cast gcc PROVES the mask redundant and drops
  the andi — a real -1 length drift that reads as a schedule. +3 levers. func_8017EC34.
* §328 NEW LAW: the volatile alias must be an aliased OBJECT; `*(volatile s32*)&sym` unfolds %lo
  into a separate addiu (+1 ins). func_80181B8C.
* §329 fold-const narrows `(int)s16 & 0xFFF` onto the RAW HImode pseudo, breaking the
  sign-extend/mask register tie; a zero-byte `s32 e = t;` widening temp restores it (30 rows -> 0).
* §330 the NEIGHBOUR-SHAPE lever, four independent instances in one wave — copy an already-banked
  in-TU function's SPELLING before any codegen reasoning (one dissolved 18 REGALLOC-PERM rows in a
  single compile). Corollary: a warm start from another binary is often worth LESS than the
  neighbour 20 lines away.
* §331 OPEN GAP, recorded as unsolved: no lever eliminates an UNWANTED DUPLICATE copy at a
  branch-target block head (main/func_80013154, closeness 12, ~16 iterations, 5 approaches refuted).

TOOLIFY — tools/gate_wave.py: split the batch on the per-draft jtbl predicate, run parallel_gate
and the serial jtbl lane CONCURRENTLY. Measured this session: 4 binaries in 103s wall through
parallel_gate (87/87/88/102s each) vs ~6 min serially; I had gated all 16 serially to protect ONE
jtbl draft, ~1 hour. The split precedes the run because a jtbl worker does NOT fail cleanly — it
re-extracts through the worktree's asm/ symlink and writes the MAIN tree while other workers read it.

Its own negative control found two defects in it before first use:
  * listdir counted gate_stage's _xform output dirs (-cn/-cast/-rc/-sd, written as SIBLINGS inside
    the drafts root) as binaries: 20 "binaries" for a 16-binary wave. Now validated against
    progress.BINARIES and refused loudly (R32/R43).
  * a post-hoc control over BANKED functions cannot reproduce a split (has_jtbl has no stub to read);
    re-controlled against a live draft set, where it correctly routes the two functions the gate had
    independently reported CARVE-REFUSED.
2026-08-31 12:44:13 -06:00
Drew T b94af95740 feat(decomp): parallel gate — 1 fns across 1 binaries (2 workers)
ov_SC03_118    func_80184F18
2026-08-31 12:40:21 -06:00