gate_stage._jtbl_prepare carried the SAME config-only undo as harvest_verify's did,
and ate the tree again on the first ladder run: 5 orphan region files, truncated
TUs, `undefined reference to func_80192F64`. That INVALIDATED the run's 0/10, so it
was re-measured rather than reported (R35 — a probe from a broken tool is not
evidence). Tree restored from HEAD and re-verified byte-identical first.
DELETED, not patched (R33 — the best outcome is a deleted stage). It was wrong on
two independent axes:
1. §61b already byte-proved THE CARVE MUST FOLLOW THE SPLICE. A batch pre-pass
carving unspliced functions reports "prepared" and yields a spec that fails
once the body lands — which is why it banked nothing.
2. Its undo snapshotted only config/, while jr_isolate_all rewrites region 0 back
over the ORIGINAL src/<ov>/<nm>.c truncated.
harvest_verify's per-draft prep is the correct mechanism, snapshots the full source
set, and undoes per function. Two implementations of one capability, the outer one
ineffective AND destructive.
THE HONEST RE-MEASUREMENT (clean tree; tree verified clean after):
- 0/10 bank, but 9/10 now COMPILE and land as whole-binary byte-DIFF; 1/10 plumbing.
- match_one close=0 on several (the function's own bytes exact) and rtu_match says
MATCH-in-real-TU for func_80135888 — while func_801299C8's transformed draft does
not compile in its real TU at all. The residual is MIXED, not uniform; at least one
is an IMAGE-level effect rather than the draft or its TU decl context (prime
suspect: jtbl/rodata carve placement). NOT generalized from one data point.
- This PRICES Task 14 stages 2-3 by measurement: the existing ladder converts 0 of
10, so they are not "wire in normalize_self_decls + the type-lift and collect ten
banks" — the projection error §57a already caught once this phase.
- R22 clean-fleet 140/140 BYTE-IDENTICAL with the giant func_8018F694 banked and the
func_80135A4C family swept 138/138
- cookbook §61d (the tree-eating undo in two tools; the constant-label defect; the
re-probe + ladder measurements; the general rule: an undo whose scope is narrower
than its write scope destroys work no byte-gate can see)
- decision-log + CURRENT_PHASE updated (R30/R31)
THE DEFECT (byte-witnessed, and it is the §61c mechanism). `jr_isolate_all`
repartitions a code object by writing region 0 back over the ORIGINAL
src/<ov>/<nm>.c — TRUNCATED to just that region — and emitting the rest as new
_jr_<lo>.c files. `_jtbl_restore` undid only config/ + the new region files, so
every gate-REJECTED draft left the original TU permanently truncated and its
stubs gone. Nothing regenerates them (splat does not rewrite a committed overlay
.c). Measured across an 11-draft re-probe: live stubs 419 -> 414 -> 406 -> 395,
ending in `undefined reference to func_80191C50`.
It is INVISIBLE to the gate that causes it: the incremental build keeps linking
stale objects (§42b) so `make build` stays green while a CLEAN rebuild fails.
That is exactly the "139/140, twice" signature that became the §61c blocker —
the tree was being eaten by the undo meant to protect it. The §61c attribution
(batch _jtbl_prep residue) is now a demonstrated defect, not an inference.
FIXES (both negative-control-validated on a draft that fails):
- _jtbl_snapshot captures every src/<binary>/*.c; _jtbl_restore restores them and
removes exactly the files the attempt created (derived from the snapshot's file
set, not re-guessed from the _jr_* name shape, R33). Undo by SNAPSHOT-RESTORE,
never an inverse transform — §61's law one level deeper. Same failing draft that
previously broke the tree now leaves it byte-identical, git status clean.
- classify_fail ignores `warning:` lines. The benign `conflicting types for
built-in function 'memcpy'` warning was winning the match on 8 of 8 failures
across four different real causes — a label identical for every input, which the
cookbook had to work around by hand ("the gate label is useless here, splice
individually and read real cc1 stderr"). Now reports the real error, and falls
through to CC1-FAIL:<last error line> rather than guessing.
THE RE-PROBE THIS ENABLED (11 preserved t5wave cracks, one invocation each):
- BANKED: func_8018F694 (478 ins) — one of the wave's three giants, previously
recorded as part of "the gate banked ZERO".
- The other 10 now carry TEN DISTINCT diagnoses: 4 data-decl conflicts (D_80193B64
x2, D_8011D030, D_80126B5C), 3 callee-decl (func_80135480 x2, func_8012F14C),
3 self-decl/own-sig (§57). ZERO jtbl-drift, ZERO local-type redefinition, ZERO
codegen DIFF.
- So §61a's "§8e-2 jtbl table-count drift blocks 10 of 12" does NOT survive the
carve-follows-splice prep: the carve now succeeds and what is left is ordinary
decl plumbing the existing ladder already handles (cast_call_sites / reconcile_tu
/ normalize_self_decls / fix_arity_callers) = Task 14 stages 2-3, no new tooling.
- ov_SC06_018 BYTE-IDENTICAL cbbc4f44 with the giant banked; nothing committed broken
at any point (tree was restored from HEAD and re-verified before these fixes)
- cookbook: the `git add -u` complementary hole (an isolation's NEW region file is
untracked, so a carve/isolation bank needs `git add -A src/ config/`)
The session-7 checkpoint gated the entire jtbl track behind one finding: the
carve+isolation path yields a bank that is incrementally valid and clean-invalid
(139/140, [FAIL] ov_SC06_018, "twice, identically"). The prescribed diagnosis
(diff the incremental vs clean object set) never ran, because the failure does
not reproduce.
MEASURED, with the bank applied through the single-function automated path
(harvest_verify --chunk 1 -> [jtbl] carved -> + chunk(1) -> BYTE-IDENTICAL):
per-binary clean (rm asm+build; extract; build) -> BYTE-IDENTICAL cbbc4f44
make clean && extract-all && check-all (run 1) -> 140 passed, 0 failed of 140
make clean && extract-all && check-all (run 2) -> 140 passed, 0 failed of 140
ATTRIBUTION (best-supported; the failing tree is gone): the 139/140 runs were
taken on the tree left by the BATCH _jtbl_prep (6 table-bearing -> 1 carved,
4 isolate-FAILED, 1 stale-asm carve fail) — five failed preps' residue of
stranded carves + half-applied isolations. The per-function snapshot-restore
that removes exactly that residue landed AFTER those runs, in commit:0803, the
same commit that named the blocker.
THE LESSON (R35 on ourselves, -> decision-log): "twice, identically" was not a
replication — two reads of the SAME contaminated state is one observation. A
replication must RE-CREATE the state, not re-run the check. Standing guard:
re-apply a fault from a known-clean tree before writing it down as a property
of the mechanism. Sixth "structural wall" to resolve to our own tree/tooling.
- BANKED: func_80135A4C (181 ins) x1 in ov_SC06_018 — isolated into its own
code subseg + .rodata carve (single-table, no JTBL_PADS; tail3..tail18 renumber)
- §61c faults 1-2 STAND: a stranded carve poisons the overlay; per-function undo
is unsound in a batch -> ONE jtbl draft per harvest_verify invocation.
jr_inventory's 1:1 ownership assertion was right and is unchanged.
- UNFROZEN: this family = 138 members / PURE / 24,978 ins ~ +0.19pp (jtbl_family_bank,
§53 carve law); the 9 preserved t5wave cracks (Task 14 stages 2-3, §57 plumbing)
- R22 clean-fleet 140/140 x2; tools-health OK (dedup 1848/0, C1 234481/234481,
cdecl 53189/53189, audit-binaries 140); 0 NON_MATCHING (G4)
- fleet 78.0% instr / 66.5% distinct / 87.95% fn-count
- also: preserve the 4 untracked wave-4 .o0 drafts (R20); killed an orphaned cc1
from the Jul-21 session burning a full core for 13h23m
THE BLOCKING FINDING (§61c): the jtbl carve+isolation path yields a state that is
INCREMENTALLY valid and CLEAN-INVALID. func_80135A4C banks every time through the automated
path (BYTE-IDENTICAL at harvest_verify's gate) and fails `make clean && extract-all &&
check-all` twice, identically (139/140, [FAIL] ov_SC06_018). The bank is therefore NOT
reproducible from committed config + source, and the gate that authorises it cannot see the
defect because the gate IS the incremental build (§42b, most expensive form).
=> NO jtbl core can be banked until that divergence is diagnosed. Next step is to diff the
incremental vs clean build/ov_SC06_018/** object set + generated .ld/asm for the carved
subseg — NOT to bank more. All 12 wave cracks stay preserved at .run/giants/t5wave_*.
TWO REAL DESIGN FAULTS FOUND AND FIXED IN harvest_verify:
1. A stranded carve poisons the overlay. _jtbl_prep carved a draft the gate then REJECTED;
the carve remained with NO owner (the fn is still INCLUDE_ASM), and jr_inventory's 1:1
ownership assertion then refused every later isolation in that overlay
([('UNOWNED','0x801d288c')] = func_801299C8's table). THE ASSERTION WAS RIGHT AND CAUGHT IT
— R32/R33 working as designed; the defect was mine for leaving the carve behind. Fixed:
per-function carve + exact snapshot-restore (config text + only this attempt's region files)
on gate rejection.
2. Per-function undo is UNSOUND IN A BATCH: isolation REPARTITIONS shared source, so restoring
one draft's snapshot deletes region files now hosting OTHER pending drafts, and their stubs
vanish (KeyError in render). jtbl drafts must run one per harvest_verify invocation, or the
undo must be region-aware.
MEASURED so it is not re-derived: of 11 preserved cracks exactly ONE (func_80135A4C) reaches
byte-identical through the carve path; the other 4 table-bearing ones fail one-at-a-time too,
on PLUMBING (§57 self-decl et al), not the carve.
Tree reverted; clean-fleet 140/140; nothing banked. cookbook §61c.
THE LAW IMPLEMENTED: the carve must follow the splice. harvest_verify._jtbl_prep() splices each
table-bearing draft TEMPORARILY, asks jtbl_carve, isolates on the §8b walls, un-splices, re-extracts,
and RE-DERIVES the stub map + baseline (isolation MOVES a stub's TU, so both are keyed on stale
paths otherwise). gate_stage's batch pre-pass could not work: while a fn is still INCLUDE_ASM the
non-contiguity is undetectable, so jtbl_carve reports success and yields a spec that fails when the
body lands.
BYTE-PROVEN AUTOMATED: func_80135A4C (181 ins, 138 members) ->
[jtbl] carved 1/1 table-bearing draft(s): func_80135A4C
+ chunk(1): func_80135A4C verified 1 / failed 0 BYTE-IDENTICAL
NEW BLOCKER, PRECISELY NAMED — banking a jtbl core makes its own carve UNOWNED to jr_inventory,
which then refuses every subsequent isolation in that overlay:
"committed .rodata carve ownership is not 1:1 (R32/R33) — a stranded/duplicated carve
(§8b func_801734BC class): [('UNOWNED', '0x801d288c')]"
Byte-proven both ways: on the COMMITTED tree `jr_isolate_all --only func_80135260 --dry-run`
succeeds; with func_80135A4C banked it fails the assertion. The assertion is RIGHT (a banked fn's
stub .s is pruned, so the owner lookup finds nobody) but its conclusion is wrong — the carve IS
owned, by C rather than a stub. SO TODAY JTBL CORES BANK ONE PER OVERLAY. 10-draft batch: 6
table-bearing -> 1 carved, 4 isolate-FAILED on this assertion, 1 stale-asm carve failure.
NEXT INCREMENT (precise): resolve jr_inventory's carve owners from corpus.matched U stubs, not
stubs alone (R33 — the same derive-don't-reparse move that fixed the corpus oracle).
BANK NOT KEPT: R22 clean-fleet showed 139/140 (ov_SC06_018 FAILS from a clean tree) even though the
INCREMENTAL build read BYTE-IDENTICAL — the stale-incremental false pass R22 exists to catch (§42b).
Reverted; clean-fleet re-verified 140/140, tools-health OK (dedup 1848/0). All 12 wave cracks remain
preserved at .run/giants/t5wave_*. cookbook §61b updated.
DIAGNOSIS CORRECTED: the wave's 10/12 blocker is NOT "§8e-2 table-count drift" (the symptom
the filter reports) but a NON-CONTIGUOUS .rodata carve — the new function's table is separated
from the TU's existing carve by an UNMATCHED function's table, and one object cannot straddle
that gap. jtbl_carve names its own remedy in the refusal message.
RECIPE BYTE-PROVEN (func_80135A4C, 181 ins / 138 members):
jr_isolate_all --only <fn> ; make extract ; <splice> ; jtbl_carve --func <fn> ;
make extract ; make build -> BYTE-IDENTICAL
(isolation verified byte-neutral on its own first; a config change needs extract, not just build.)
BUILT: gate_stage._jtbl_prepare — per-draft carve + auto-isolate on the §8b walls, logic LIFTED
from jtbl_family_bank (R33: one implementation, two callers — their divergence IS this bug),
snapshot-restore undo, GATE_NO_ARITY A/B guard. Ladder: canon -> cast -> reconcile_tu -> jtbl
-> arity -> gate -> sig_unify -> gate.
IT DOES NOT YET BANK, and that is the finding: THE CARVE MUST FOLLOW THE SPLICE. The
non-contiguity is only DETECTABLE once the body is in the object; while the fn is still
INCLUDE_ASM, jtbl_carve reports SUCCESS and produces a spec that fails when the body lands.
Byte-witnessed both ways (spliced -> NON-CONTIGUOUS 0xaa810/0xaa920; unspliced -> "prepared 1/1"
then byte-DIFF). Innocent suspects A/B'd out: the draft is IDENTICAL through canon/cast/
reconcile_tu, and GATE_NO_ARITY=1 changes nothing. FIX = per-draft prep inside harvest_verify's
splice loop (it owns the splice), not a batch pre-pass in gate_stage.
SUB-FINDINGS: (a) a wholesale `git checkout -- config/` undo is WRONG in a batch gate — it
discarded a previously-banked-but-UNCOMMITTED carve, leaving that bank's source with no subseg
(undefined reference to func_80136C90 at link). Now snapshot-restore + drop only this run's
region files (§61's constraint, which I had written and then not applied here). (b) being in a
_jr_* TU != having a table: only 4 of 8 wave drafts reference a jtbl_.
Tree restored byte-identical; nothing banked. cookbook §61a corrected + §61b.
Ultracode wave, 12 agents (~2M tokens), over freshly-prefetched ov_SC06_018 exemplars.
11 MATCH / 1 near, INCLUDING ALL THREE GIANTS (710/673/478 ins). Whole-binary gate: ZERO.
Splicing each failure individually (the gate's own label is §58's memcpy red-herring) gave
THREE DISTINCT blockers, none of which the ladder clears:
(1) §8e-2 jtbl table-count drift -- 10 of 12. "more rodata .align directives than pad specs".
STRUCTURAL FINDING: fresh crack fuel in a well-matched overlay CONCENTRATES in jtbl-carved
TUs (the non-carved ones were harvested first), so §8e-2 GATES the next tranche of
substantial cracking rather than being a straggler.
(2) §57 self-decl conflict -- the 2 plain-TU drafts ("argument 'arg2' doesn't match prototype").
normalize_self_decls exists, is wired into family_sweep, and is NOT in gate_stage -- the
same gap the arity pre-pass had.
(3) local-type redefinition (from the Task-14 diagnosis set) -- wants the type-lift.
So gate_stage needs THREE stages; only the arity pre-pass landed today.
All 12 drafts PRESERVED at .run/giants/t5wave_* (R20): genuine cracks with per-function lever
notes (cross-jump barrier placement, MEM_IN_STRUCT_P store/load ordering, §43 K&R s16 params,
$s-pins, CSE-break barriers). Do NOT re-draft -- they bank the moment the stages exist.
METHOD NOTE: `make build | grep -i error` missed the real failure TWICE (the jtbl_rodata_pads
line contains no "error" token; and the build failed at a later stage than the warnings I read).
Check rc, read the tail unfiltered -- a filtered build log is a selection tool, and every
selection tool here has eventually lied (R32/R35).
Tree reverted clean; nothing banked. cookbook §61a.
DIAGNOSED, not assumed. The 12-draft integration probe banked 1/12 and reported the SAME
label for 10 of the 11 failures: `conflicting types for built-in function 'memcpy'` — the
§58 red-herring (a WARNING, from an unrelated TU position). Splicing three top-reach
failures individually and reading real cc1 stderr gave the actual causes:
conflicting types for `func_XXXX' 3/3 <- loose-typing ARITY conflict
redefinition of `struct V8' <- a SECOND class (type-lift), stage 2
A banked shared caller macro in engine_core.h declares the function with FEWER params than
its byte-true definition takes (the original calls K&R-style with fewer args than the callee
reads); a C89 prototype makes that a hard error. tools/fix_arity_callers.py --any-proto
already fixes it and was simply NEVER WIRED into gate_stage's ladder (only family_sweep
carried §57). Now wired as a TU-side pre-pass.
MEASURED: 2 of 7 top integration candidates banked (func_8016EFC8, func_80164418, both
reach-138) vs the 1/12 old-ladder baseline. R22 140/140; tools-health OK (dedup 1848/0).
INCIDENT — this stage BROKE 138/140 AND R22 CAUGHT IT (nothing was ever committed):
pairing `--apply --any-proto` with `--revert` for the unbanked drafts corrupted declarations
fleet-wide. `--revert` rewrites ()->(void), which inverts a PLAIN apply but NOT --any-proto,
so an unbanked fn whose real decl was `extern void func_801708B0(void *a0)` came back as
`(void)` — in engine_core.h (included by all 138 overlays) and 6 sites in ov_SC01_077's own
sources. harvest_verify --binary ov_SC01_077 reported BYTE-IDENTICAL and was RIGHT about that
binary; the other 137 were structurally invisible to it. Repaired to the exact lines.
ROOT CAUSE FIXED: the ladder now snapshots every file the pre-pass touches and undoes by
RESTORE + re-apply-for-the-banked-set-only — exact by construction, cannot invent a signature.
NEW HARD CONSTRAINT (cookbook §61): any ladder stage mutating SHARED state must be undone by
snapshot restore, never an inverse transform, and validated FLEET-WIDE (R22) rather than by
the per-binary gate that authorised it. §55b's propagation law, one level down. The planned
type-lift stage edits engine_types.h and inherits it by default.
ALSO FIXED: the first wiring passed only --drafts (the narrow-param FILTER) without the
required --funcs, so the stage exited `no funcs given` as a SILENT NO-OP and the gate reported
0/6 as though diagnosed. sh() does not raise on non-zero exit -> explicit rc check added.
The hindsight-study §7 taxonomy predicts plateaus decompose into missing-transform (the
"highest-value bucket and the whole point"), seed-structural, and genuine-wall. Run against
real plateaus this class produced NO missing-transforms, and the answer needed no LLM.
MEASURED: `length` probe, 20 targets, 1 win. tail 1/6; partial 0/12.
AUTOPSY (read directly from the bytes, 3 partial plateaus):
- func_8017F0C0 / func_801806C8: target has `sltiu $v0,$v0,1` = gcc's codegen for `!x`/`x==0`;
the drafts wrote `(u32)(D_x ^ 1)` which emits `xori`. No local mutation crosses that.
- func_8017FF90: draft stores to arg0+8, target stores to a GLOBAL. Different function.
=> these are WRONG DRAFTS wearing a small closeness, i.e. seed-structural, not a mutation gap.
THE FIX IS THE OPPOSITE OF "ADD TRANSFORMS" — a tighter ADMISSION rule:
- _drift_route: permuter only when |d|<=2 AND explains=="tail" (the shape that measurably
converts). length pool 339 -> 34; permuter bucket 389 -> 84.
- SIZE-MISMATCH: added a PROPORTIONAL test (|d| >= 0.5*nt). max(2,0.15*nt) is far too
permissive on a tiny target — a 2-ins draft vs a 4-ins target read as a near-miss.
permuter_weights needs NO extension for this class.
Transferable (cookbook §60b): raising a search-closer's yield is at least as often about
refusing it unreachable work as widening its mutation set. Same knife as Task-13A's
targeting fix, one cut finer. Drafter idiom recorded: `sltiu rd,rs,1` => `!x`, never `x^1`.
17 unit tests green; corpus re-collected (1654 rows, closeness cross-check clean).
PROPAGATION (§55b, its own targeted batch): dedup_propagate --addr 0x80141B90 --recover
-> "138 overlays byte-identical after propagation"; 117 remaining stubs -> 0; 1 new
dedup group. This was the ONLY one of the 21 directed-run banks worth propagating.
THE REPRICING (R14 — measure a bucket's VALUE, not just its conversion rate):
the directed run converted 27% (21/77) but moved the fleet ~0.03pp, because h_exact
reach of the 21 is: func_80141B90=138, TEN at reach-1 (nothing to propagate), rest 2-10.
Instruction-weighted, the ENTIRE permuter bucket is worth ~0.36pp at 100% conversion.
The mechanism is validated; the fuel was small. Priced frontier (ins-weighted / 13.08M):
LENGTH-DRIFT |d|<=2 472,178 ~3.6pp (339 fns) <- the real permuter-adjacent lever
integration 419,162 ~3.2pp (305 fns) <- Task 14's ladder
WIDTH 71,593 ~0.55pp (45)
permuter (current) 46,571 ~0.36pp (74)
BRANCH-POLARITY 9,462 ~0.07pp (22)
So WIDTH/BRANCH-POLARITY are NOT worth prioritizing; my earlier "~200 candidates"
framing undersold LENGTH-DRIFT 10x and oversold WIDTH.
NEW: permuter_weights._LENGTH profile (perm_temp_for_expr/perm_expand_expr are the only
passes that change instruction COUNT; the reorder/decl-order levers that dominate the
regalloc+schedule profiles cannot, so they are down-weighted here) + residual_class
._drift_route (|d|<=2 -> permuter/`length`, larger stays structural — same class,
opposite tool) + classify() accepts a PROFILE NAME directly (the measured profile beats
re-parsing a free-text label). 17 unit tests green.
grinder: --profile filter (probe ONE residual class's conversion) + a PERSISTENT attempt
ledger. `tried` was in-process only, so every fresh --once run re-permuted the previous
run's losers — the permuter is deterministic given (base.c, target.o), so that CPU can
never produce a new win. Measured: a 20-target probe drew 19 already-tried targets.
Keyed by draft_sig so an improved draft legitimately re-opens the function.
First run of the Task-13A-targeted grinder (--once --batch 8 --permute-secs 90):
targeting: ON — 1665 classified; only bucket=permuter is admitted
targeting skipped 1601 non-permuter candidates
{redraft 707, structural 583, integration 305, unclassified 4, unknown 2}
permuter WON func_80181F78 @ ov_SC03_014 (close was 1) [~6 min]
BANKED 2 (both whole-binary byte-gated, R22 clean-fleet 140/140):
- func_80181F78 (8 ins) — classified DELAY-SLOT / schedule
- func_80141B90 (29 ins) — classified IMM-VALUE / cse
Both were classified bucket=permuter by residual_class BEFORE the run: the classifier
predicted a search-closer could reach them, and one did. First end-to-end validation of
the targeting thesis.
THREE latent defects, all pre-existing, all unreachable while the grinder banked nothing
since Phase 21 — the fix made it win and every one fired at once (cookbook §60a):
1. gate_stage commit path crashed on src=None. `src` is DELIBERATELY never defaulted
(Phase 26-A: a default silently PINS the gate to the main .c), but the commit did
`git add src …` unconditionally -> every caller that omits it (grinder, orchestrator,
idiom_hunt) crashes THE MOMENT IT BANKS. Now `git add -u src/`, which also retires the
`src/ov_*/*.c` filename glob that once omitted 4 R22-verified banks from a commit.
2. _xform ladder dirs (-cn/-cast/-rc/-uni) ACCUMULATE across runs: they held 34 stale
drafts, so the gate processed 34 when the grinder submitted 1, and banked one function
it was never asked to try — which would have been committed under a message naming a
different one. Nothing wrong entered the tree (G3/P9 held; the gate banks only
byte-identical output) but report and work had diverged. Now cleared per run.
3. grinder called gate_stage with the default propagate=True = `dedup_propagate
--auto-from`, the §55b fleet-wide path that timed out at 3600s and left 90/140 overlays
broken — and being INSIDE the gate it takes the banks down with it. Now propagate=False;
banks commit cheap, propagation is its own targeted --addr batch (§55b law).
R22: make clean && extract-all && check-all -> 140 passed, 0 failed of 140.
tools-health OK (dedup 1847/0, C1 234343/234343); 0 NON_MATCHING (G4).
The autopsy (hindsight-study §7) assumed the permuter loses for want of a mutation.
Measured over the whole open backlog, it loses because it is aimed at work a
search-closer provably cannot close.
- NEW tools/residual_class.py: decide a near-miss's class FROM THE BYTES. Decodes each
mismatching MIPS word -> (op-skeleton, register-fields, immediate); drift FIRST (one
inserted insn inflates `closeness` by the tail length), then consistent-injective
register map -> REGALLOC-PERM (§31 S11/RC-3), same-multiset-reorder -> SCHEDULE-REORDER,
DELAY-SLOT, WIDTH/BRANCH-POLARITY/STRENGTH/ADDRESSING/IMM-OFFSET/IMM-VALUE. Every class
routes to a BUCKET = which tool the failure wants. Uncovered opcode -> UNKNOWN, COUNTED
(R32). 16 synthetic unit tests (test_residual_class.py).
- NEW tools/autopsy.py: `collect` materialises the corpus Task-12's telemetry never filled
(1 of 6,169 records had a residual) by recompiling every open draft through the EXISTING
match_one path (R33) — 1,752 drafts in 21s at -j12. `report` -> docs/autopsy.md.
- NEW corpus.o0_sources()/is_o0(): the opt-level oracle DERIVED from the Makefile's own -O0
rules, coverage-asserted. Scoring an -O0 target at -O2 makes the residual 100% artefact
(the trap this phase hit four times).
- R34 cross-check baked in: residual_class's closeness vs masked_diff.structured_diff's,
asserted per row; 1,673/1,673 agree, 0 classifier errors.
FINDING: of the 972 records the grinder's own filter admits, only 75 (7.7%) are
permuter-shaped; 547 are structural and 348 are drafts that are not the function at all.
~92% of the daemon's CPU went where it could not win — the byte-grounded explanation of
"7 banks all-time, all Phase 21, 0 since" (Phase-22 audit). grinder.candidates() now
filters on the measured bucket (1,303 -> 78) and takes its directed profile from the
measured class, not the logged label (91% carry none -> it ran on gcc defaults).
Degrades to undirected if uncollected and says so; --no-targeting A/Bs it.
Two measured corollaries (R14, not projections):
- 699 records rank as near-misses at closeness up to 278 purely from a length artefact:
un-attempted work misfiled as a backlog of hard functions -> new `redraft` bucket.
- a 12-draft gate probe of the `integration` bucket banked 1/12 (11 PLUMBING), so the 306
prices Task 14's reconcile ladder rather than promising free banks. func_80167714
(104 ins, reach-134) banked x1, un-propagated by design (§55b).
Two defects fixed forward:
- masked_diff._common_typedefs() used ONE shared probe path, so parallel match_one
processes clobbered each other: 14 of 1,752 drafts lost in a single 12-way run (0.8%),
silently, in every parallel wave ever run. Now per-PID.
- gate_stage.match_one_closeness never passed --o0 -> phantom residuals for every -O0
function, written straight into the backlog this autopsy reads.
R22 clean-fleet: check-all 140 passed, 0 failed of 140; tools-health OK (dedup 1847/0,
C1 234343/234343); 0 NON_MATCHING (G4). Flywheel captured in-session (R30/R31):
cookbook §60, decision-log entry, SETUP.md inventory.
A study substrate (UNIVERSAL/COMPILER-CLASS/BFM-SPECIFIC tagged) for a future Fable
pass to refine into a general roadmap for any-platform/any-compiler decomps. §7 adds
the permuter-failure-autopsy loop (mine failures -> extend the ILS wrapper -> shrink
LLM to genuine humps) per Drew's 2026-07-20 direction. Companion to Tasks 12-14.
tools/burndown.py snapshots the 3 fleet metrics + remaining substantial frontier
each run (docs/burndown.jsonl, git-HEAD-keyed), computes velocity vs the previous
commit-snapshot, and flags the yield floor (mean instr-pp over last 3 snapshots <
threshold => 'consider closing P29'). Seeded with the session-4-close baseline.
Reads progress.fleet.md + family-hseq.md (regen those first for a true reading).
Exemplar cast (§56b) unblocked propagation: the bare func_80161208(p) call
relied on ov_SC01_077's no-proto decl, but member TUs carry a (void) prototype.
Casting the call member-proofs the templated body. All banks whole-binary byte-gated.
- func_8016CBC0's 137 members were NOT self-decl-blocked (§57 survey: 0 divergent self-decls) but blocked
on the exemplar's 6 LOCAL typedefs (Rec_/Blk_/SVEC_/CLR_/Poly_/Mtx8_8016CBC0). Lifted all 6 verbatim into
src/shared/engine_types.h (fleet-visible via engine_core.h) + removed the locals from the exemplar TU.
No fleet name-collision; exemplar rebuilds byte-identical d19c9580 (the type-lift is byte-neutral).
- family_sweep --hseq --only 0x8016CBC0 --no-preclassify --normalize-self-decls -> 137/137 banked, 0 failed
(the decl-normalize stage a correct no-op here — 0 self-decls). 0 func_8016CBC0 stubs remain fleet-wide.
- R22 clean-fleet 140/140 byte-identical; tools-health OK (dedup 1846/0, C1 234205/234205); 0 NON_MATCHING
linked (G4). fleet 74.9->75.1% instr / 60.1->60.6% distinct / 86.96->87.00% fn-count.
- R14 lesson: two families that both looked like a 'stuck 137-family' needed DIFFERENT levers
(decl-normalize §57 vs type-lift) — route by the real cc1 error, not the shape. session-total +270 members.
- new tools/normalize_self_decls.py — the SAME-FUNCTION decl-normalize: drop each decl of the templated
fn F that the sibling TU's OWN already-banked callers declare divergently (block-scope, a different C
form than the exemplar's fn-ptr cast) + cast its in-scope calls (byte-neutral §17a-1). cast_call_sites
does the callee direction, reconcile_tu the data direction; this is the third: F itself.
- family_sweep --normalize-self-decls: new per-sibling stage after reconcile_def_sig (edits the sibling
TU file = harvest_verify's baseline, like edit_remap_sweep) + snapshot/final-SHA-MISMATCH revert backstop.
- byte-proven the sole blocker was the caller-decl conflict, NOT --fix-def-sig (which renamed F's def
params a0..a3 while the body used arg0..arg3 -> arg0 undeclared; R14 confound removed). 133/133 banked,
0 failed, 0 backstop fires; 0 func_801670E4 stubs remain fleet-wide.
- R22 clean-fleet 140/140 byte-identical; tools-health OK (dedup 1846/0, C1 234205/234205); 0 NON_MATCHING
linked (G4). fleet 74.6->74.9% instr / 59.4->60.1% distinct / 86.92->86.96% fn-count. cookbook §57.
- R14: func_8016CBC0 (also stuck 137-family) has NO divergent self-decl -> type-lift-blocked, a SEPARATE
lever (6-typedef cluster, like 8012956c), not this pass. Route by the real cc1 error.
Regen family map post-bank -> exemplar promoted draft-ov077->matched-ov077 -> standard
family_sweep --hseq templates the committed reconciled body (the --reconcile-raw path
mishandles per-overlay data externs; standard path is correct). +249x138 ~= +34k ins.
- ROOT CAUSE PINNED (the session-2 half-pin was INVERTED; both probes were vacuous, R35):
cc1 emits .align 3 before EVERY jump table; maspsx passes it VERBATIM (the :435 'drop' is
an inventory-only pass); as bakes the pad SECTION-RELATIVE; link placement was never guilty
(SUBALIGN(2) + ALIGN(.,4) place 4-mod-8 carve starts tight). Merging originally-separate
TUs fires an intra-TU align where the original packed tight -> +4 at rodata 0xCC ->
image-wide %lo shift. Honest probes persisted: .run/probe_jtbl/ (verdict.md + objdumps).
- NEW tools/jtbl_rodata_pads.py: post-maspsx filter replaces each rodata .align 3 with the
ORIGINAL's exact pad bytes per a JTBL_PADS spec; fail-loud on table-count drift /
non-align-3 / non-jtbl rodata content. Byte-proven: verbatim 0xE4 pad-at-0xCC ->
filtered 0xE0 tight (= the merged carve span).
- jtbl_carve.py: spec-aware same-subseg merge (gap 0 or 4-with-zero-payload-word; else
NON-CONTIGUOUS -> isolate), interval-arithmetic pad specs (committed values CARRIED,
never re-derived), JTBL_PADS target-var emission into overlays.mk + revert() restore +
stale-.o invalidation; the false 'maspsx drops .align' docstring corrected (H5).
- Makefile: $(if $(JTBL_PADS),| jtbl_rodata_pads.py ...) stage in build/src/%.o + file-scope
empty default (env-shield). jtbl_family_bank.stub_file: duplicate-stub fail-loud (the
earlier 'ladder failure' was a wrong-TU splice into a stale _a.c stub, byte-witnessed).
- R22 clean-fleet WITH the fix wired: 140/140 byte-identical, tools-health green
(dedup 1846/0, C1 234205/234205), ZERO new banks -- fleet-neutral by construction.
- cookbook §8e (the jtbl alignment law) + §8a/§8a-pad corrections; decision-log R31 entry;
SETUP.md tool row; .gitignore allowlist for the probe verdict artifacts.
Task-4: the non-jtbl giant func_8013FAF8 (312 ins) is now DONE fleet-wide. It is an h_seq family
(masked-identical body, per-overlay symbols/relocs) so dedup_propagate refused it (reach<2); the
path is family_sweep --hseq --only 0x8013FAF8 --allow-pins (per-overlay symbol remap).
First sweep banked 0/137: family_sweep copies the exemplar's extern block VERBATIM, and the
hand-crafted exemplar had 4 callee externs diverging from the fleet-canonical (func_8005A600
void->s32, func_80024054 s32->void*, func_80137D08 s32*->int, func_8013AB54 s32*->s32) -> every
member threw `conflicting types`. Aligning those 4 in the COMMITTED exemplar .c to the
member-consensus form (byte-neutral; ov_SC01_077 stays byte-identical) -> re-sweep 137/137 banked,
0 failed. +312 ins x137 ~= +42.7k ins.
R22 clean-fleet 140/140 byte-identical; tools-health OK. Cookbook §56b (exemplar externs must be
fleet-canonical for an h_seq sweep; contrast dedup_propagate --recover which auto-reconciles CALLER
externs -- family_sweep does not, so pre-align the CALLEE externs by hand).
Both non-jtbl giants now fully banked+propagated (func_8013FAF8 x137, func_8014F4C0 x134); the 4
jtbl giants remain deferred on the byte-proven 8-align jtbl-carve gap.
Task-4 giant-bank #1 of the 12 preserved p29t3 drafts. func_8013FAF8 (312 ins, a
menu/HUD prim builder stubbed in 138 overlays) BANKED x1 in ov_SC01_077, whole-binary
byte-identical; R22 clean-fleet 140/140; tools-health OK (dedup 1843/0, audit-binaries OK).
The README billed it "pure def-sig plumbing" but it was a 5-conflict multi-symbol
reconciliation (~5 gate iterations), all byte-neutral + gate-arbitrated (G3/P9):
- def-sig s16/s16 vs canonical s32/s32 -> NARROWED the extern fleet-wide (404 decls/265
files). NOT --fix-def-sig: the s32 variant diverges at insn 22 (match_one, R35). Neutral
because every fleet caller passes (s16)-cast or small-const args (verified).
- 3 data-symbol conflicts (D_80115128 lh / D_800B9A02 lhu / D_80187AC0 s32[]) declared
BEFORE the splice (block-scope §55a-blocked) -> the TU's §18 cast-at-use-site convention
(*(s16*)&, *(u16*)&, ((s32*)&sym)[i]) forces the load width regardless of decl signedness,
keeps the TU decl untouched, propagation-safe. No CSE-hoist across 5 uses.
- 2 fn-extern conflicts (func_8013FFD8 s16 arg0, func_80141100 int(int)) -> reconciled the
draft decl to the TU def + byte-neutral call-site cast. Still MATCH 312/312.
Stubs 138->137, 1 def, 0 NON_MATCHING (G4). Fleet holds 71.4/53.3/86.42 (the ×1 exemplar is
negligible until propagation ×137, batched per §55b). Technique -> cookbook §56.
TASK 3 COMPLETE (both deliverables):
(a) jr cores: a 2nd ultracode wave cracked all four giants at match_one — func_8013F350 (490),
func_80131340 (424, verified TWICE: standalone AND in-TU cc1 rc=0), func_80159C84 (337),
func_8013C414 (329, -O0). ALL FOUR gate-rejected on INTEGRATION (def-sig / §8a rodata island /
decl conflict), not codegen; each blocker NAMED. func_8013C414 matched ONLY because of the new
worker_wave --o0 fix (a -O0 target self-checked at -O2 can never match — the §53/§54 wrong-build-step
trap that produced Phase-28's "~3%" artifact).
(b) §53 has_mid_jr INTERLOCK: built + negative-control-proven (commit:0670).
PRESERVED (R20 — ~2.6M agent tokens to reproduce; bank them, do NOT re-draft):
.run/giants/p29t3_*.c (12 drafts) + p29t3_README.md — every draft is match_one-MATCH or a
characterized near-miss, with its exact blocker + a cheapest-first bank order. Best value:
func_8013FAF8 (312, MATCH, pure def-sig plumbing); then func_80131340 (in-TU-proven);
func_8013F350 (§30#2 macro-widen void->s32); func_8013C414 (§8a rodata-island carve).
Permuter fuel: func_80177940 (close=5), func_801670E4 (16 — strip pins first, permuter can't
ingest them), func_8014D820 (33), func_80140958 (116).
SESSION CHECKPOINT — safe to open a fresh session:
140/140 byte-identical; tools-health OK; dedup 1843/0; 0 NON_MATCHING (G4); main 143dbb89.
Fleet 68.9 -> 71.4% instr / 49.5 -> 53.3% distinct-code / 83.94 -> 86.42% fn-count.
Tasks 1,2,3,6 DONE. Remaining: T4 (member harvest + grinder warm-start/bug-fixes), T5 (Ghidra-C
prefetch, needs /mcp), T7 (burn-down + ROI close). Carried: the 2 local-type-blocked cores
(func_8014E284, func_80137DD4) need the build_engine_types type-lift to propagate x137.
- targeted dedup_propagate --addr per core (NOT --auto-from), --recover for stragglers:
0x8014ADE0 -> 138 overlays byte-identical
0x801325B8 -> 134 (ov_SC07_011 byte-diverges -> auto-excluded, kept x1 — what --recover is for)
0x801387B8 -> 138 overlays byte-identical
= ~410 member-instances; 3 new dedup groups (1840 -> 1843), C1 coverage 233795/233795.
- 2 of the 5 banked cores (func_8014E284, func_80137DD4) stay ×1: "not self-contained (local types)"
-> blocked on the build_engine_types type-lift (the §19/§20 propagation cap). Carried.
- R22 clean-fleet 140/140 BYTE-IDENTICAL; audit-binaries OK; dedup 1843/0; 0 NON_MATCHING (G4).
Fleet instr 71.0 -> 71.4% / fn-count 86.30 -> 86.42% / distinct-code 53.3%.
- SELF-CORRECTION (R14/R35), now fixed in cookbook §55c + CURRENT_PHASE: my earlier claim that this
propagate "needs ~2h+" was WRONG. That timing was taken while the tree still carried the partial
damage of a killed --auto-from (90/140 overlays broken), so every member-gate was failing/retrying.
On a HEALTHY tree a targeted --addr propagate is ~233s/core (all 3 = ~27 min) — ~20x faster. Only
--auto-from is genuinely fleet-slow. A timing taken on a broken tree measures the breakage, not the
tool — recover the tree FIRST, then measure.
- cookbook §55: the wave's new byte-proven levers (§49-variant birthing-boost suppression via
reg_n_sets 1->2; sched1 birthing/LUID + "cc1 -dL" movable introspection; switch-tree vs jtbl
CASE_VALUES_THRESHOLD=5; block-scope-extern beats *(T*)&sym) + the GATE-ORCHESTRATION law
(--no-propagate per group then ONE targeted --addr; commit banks BEFORE propagating; a reverted src
needs a re-extract; gate_stage's default harvest_verified.txt accumulates -> phantom banks).
Acting on Task-1's verdict (the legacy-PURE "~3%" is tooling, not a wall): the -O2
type-lift arm. family_sweep --hseq --no-preclassify --band all --allow-pins templates
each matched exemplar's C onto its unbanked same-family members and whole-binary
byte-gates every one (the sole arbiter, G3/P9). The pin-crash wall being dissolved
(Phase-27/28 _carry_macros) let --allow-pins retry the pinned exemplars.
- BANKED 3,407 member-matches across 136 overlays (INCLUDE_ASM stubs -> matched C).
- R22 clean-fleet verify: 136/137 modified overlays byte-identical from a clean rebuild;
the 1 FAIL (ov_SC07_010) was the SIGTERM mid-gate partial -> reverted, byte-identical.
- make check-all: 140 passed / 140 BYTE-IDENTICAL; dedup-check 1840/0; audit-binaries OK;
audit-cdecl green; 0 NON_MATCHING in any default build (G4).
- Fleet: instr-weighted 68.9 -> 70.2% (+1.3pp) . distinct-code 49.5 -> 51.9% (+2.4pp) .
fn-count 83.94 -> 84.73% (+0.79pp).
- Process lesson (CURRENT_PHASE.md): --band all sweeps are too long for one background
pass (got SIGTERM'd); future Arm B runs go band-bounded + committed-per-batch (resumable).
Residual FAILED members (pin/drift/plumbing) + the -O0 Arm A carve remain.
Phase-29 opens by running the Phase-28 disambiguating probe before scaling any
"(cores)x(reach)" arithmetic on the legacy-PURE-non-jr "~3% as-tooled" swing number.
- NEW tools/diff_regions.py (the deferred roadmap tool): remap the exemplar exactly
as family_sweep --hseq stages it, compile at the EXEMPLAR's real opt level
(auto-detected from the Makefile -O0 rules), masked-diff vs target, classify each
member O0-FLAG / TEMPLATES / PLUMBING-ISO / REGALLOC / NO-TARGET. Composes
family_remap + match_one + masked_diff (R33).
- VERDICT (byte-proven): the 274 DIFF is dominated (~272) by two -O0 families
(0x8013c964, 0x8013c938) that family_sweep compiled -O2 (member stub files are -O2);
an -O2 compile of an -O0 target can never match. Compiled -O0 they masked-MATCH
(func_8013C964->MATCH(10), func_8013C938->MATCH(11)). 106-member sample across
nins 2..133: O0-FLAG 45 / already-banked 29 / TEMPLATES 17 / type-lift-plumbing 15 /
REGALLOC 0. ZERO codegen walls.
- The "~3% ceiling" is RETIRED (a tooling artifact — the 3rd structural wall to resolve
to tooling after B2 and SC07). The member track is a mechanical -O0 split rollout +
type-lift sweep (Task 2a), NOT a per-member grind; member_adapt.py not needed here.
- Honest scope (§52b, R14/R35): masked-MATCH is a CANDIDATE; whole-binary banking is
Task 2a's gate. Verdict + byte evidence -> docs/calibration.md + docs/decision-log.md (R31).
- RE-SCOPED at plan time (R35): the roadmap's swing number rested on a broken-tool probe. B2's
"structural families bank ~0%" (0/8, which reshaped 2 phases of strategy) was a MISSING CARVE ->
8/8, then 102/115 (88.7%). The ~0% doctrine has NO surviving post-fix evidence.
- T0 img_path derive-not-guess -> un-hid a 230,612-ins SC07 pool doubly hidden (P27 onboarded 4
overlays, never regenerated the map; the hardcode would have called every member "LEN").
- T3-A: the SC07 pool is h_exact + UNWIRED, not h_seq. T4 dedup_extend (NEW) wired 6174/6457 (95.6%)
-> groups 134->138 binaries, C1 coverage +6174. NOT member_adapt (the number said build nothing).
- T5: resident 21->14 (90.34%, 7 banked via an Ultracode wave) + honest dossier for the 14; fixed
progress.py (#if 0 + len()-sum) and match_one's FAKE isolation (found by an agent mid-wave).
- T2 purged the poisoned grinder blacklist (8/22 matched anyway). T6 killed the --chunk 1 double-build.
- T7: make audit-binaries (the R36 citizenship gate, negative-control-proven) + fixed disc_code_sweep's
BLINDNESS to compressed code (the type-4 row was vacuous for 138 known binaries) + the worklist key-bug.
- T3b: the legacy h_seq swing number = ~3% AS-TOOLED, CLASSIFIED (274 DIFF / 37 PLUMBING), ceiling
UNKNOWN -- the 274 DIFF is byte-PURE members whose remapped bodies don't reproduce (the same
tooling-vs-wall ambiguity that resolved to TOOLING twice this phase). P29 disambiguates before scaling.
- THREE SELF-INFLICTED DEFECTS fixed forward: the registry yaml.safe_dump (H5, destroyed 47 comments +
1832 hex fields, invisible to every byte-gate), two DIFF mis-reports (R14), the match_one shared scratch.
- rule R36 (a newly-discovered binary is not real until every consumer knows it; enforced by
audit-binaries). cookbook §53. R22 140/140 throughout; tools-health OK; dedup 1840/0.
The roadmap's actual swing number (the LEGACY h_seq templatability rate; T3-A's SC07 pool was a
different question — h_exact + unwired, banked 95.6%). family_sweep --hseq --chunk 1 over 6 legacy
PURE non-jr families (has_mid_jr excluded per §53):
- 9 BANKED / 37 PLUMBING / 274 DIFF (173 skipped not-stub/pinned) = ~3% (9/320). R22 140/140.
UNLIKE Phase 26, the failures are CLASSIFIED — 274 genuine gate-DIFF, not an unclassified 0%.
- THE LOAD-BEARING NUANCE (R14/R35 on my own probe): the 274 DIFF is NOT structural variance. The
members are byte-level PURE (classify_member = reloc-only, 20/20 sampled), genuine h_seq (all
DIFF_BYTES vs the exemplar, so family_sweep is the RIGHT tool not dedup_extend), at the SAME vram.
A PURE family should reproduce once its relocs are remapped -> 274 non-reproducing members means the
remapped body FAILS TO RECOMPILE to the member's bytes: either (a) an incomplete symbol_map (the
recurring jtbl/prefix bug -- B2's 0/8 and T4's 12 DIFFs BOTH resolved to tooling THIS phase) or
(b) genuine TU-context regalloc divergence (a real wall).
- HONEST VERDICT: ~3% as-tooled, CEILING UNKNOWN. Provisionally consistent with "legacy h_seq doesn't
mechanically template" but on a probe whose dominant failure mode is the exact tooling-vs-wall
ambiguity that keeps resolving to TOOLING. Did NOT rush-resolve it at ~40% context (that is how
Phase 26 manufactured a wrong 0%). P29 MUST run the disambiguating probe (diff one PURE DIFF
member's staged bytes region-by-region: reloc-position mismatch = fixable remap; regalloc-away-
from-relocs = TU wall) BEFORE scaling "(cores)x(reach)" on 3%.
- calibration.md + decision-log R31 record the measurement + the named next probe. 9 real banks.
Drew T