Commit Graph

308 Commits

Author SHA1 Message Date
Drew T a33c6f85f5 docs(phase-26a): A11 — distill + close the tooling-integrity audit (26-A COMPLETE)
Closed the inserted half-phase. tooling-audit.md: DIAGNOSIS -> AUDIT-CLOSE LEDGER
(A1-A10 outcomes + the payoff 66.5->68.6% instr + remaining/handoff); the "two
rules" -> R32/R33/R34 crisp for P10 ratification at the Phase-26 PhaseEnd.
decision-log: the A10 wall-re-test verdict (R31 -- the broken tools WERE the walls;
the payoff was banked by the fixes; the closeness-0 residual is genuine; the real
deliverable is the 3 rules + the derived-oracle pattern). SETUP: the A9d-A10 tool
changes (R21). Cookbook §51 verified complete; LAW 3 tagged R34.

Observables green: final R22 clean-fleet 136/136 BYTE-IDENTICAL; make report EXIT 0
(dedup 1840/0, C1 227211/227211 signed, lint_symbol_refs wired + passing);
audit-corpus 0 slices; audit-cdecl green. Zero src/config changes this session.

Phase 26 resumes at Task 7 (fresh session).
2026-07-15 00:23:31 -06:00
Drew T e9038a04ea docs(phase-26a): A10 COMPLETE — wall re-test verdict on all 5 walls
The audit thesis is CONFIRMED: the tooling-walls were dissolved by the FIXES and
the payoff banked there (A3f/g/h + A9a/b, 66.5->68.6% instr), while the re-tests
confirm the residual walls are real.
- fuel/closeness-0: CONFIRMED REAL (0/958 bank at fleet scale).
- arity (Phase-15 dead-end): was tooling (A3c order-dependent rule); 13/18 banked.
- def-side loose-typing: was partly tooling; A9b banked func_8017A4AC x134.
- type-heavy: blocking tool build_engine_types was broken (A7 fixed it, now runs);
  the ~1,200-member family harvest is Task-8 integration, not a pure re-test.
- 780 h_seq callee-oracle rejections: was tooling; A3h banked +2,675.
A re-confirmed wall is as valuable as a dissolved one (P9). Next: A11 close.
2026-07-15 00:01:42 -06:00
Drew T b1c58d7668 docs(phase-26a): A10 wave 1 — closeness-0 wall re-test CONFIRMED REAL (0/958 bank)
Re-gated all 958 closeness-0 open-stub backlog drafts through the FIXED gate
across 135 binaries in parallel: banked=0, near=957, failed=71. The closeness-0
backlog is genuine whole-binary near-misses, NOT tooling misses -- match_one's
isolated closeness==0 systematically overstates whole-binary bankability, and the
repaired gate recovers none. P9: a re-confirmed wall is as valuable as a dissolved
one. (The audit's tooling-walls were already banked by A3f/g/h + A9b, +2.1% instr.)

backlog.py: env-gated BACKLOG_NO_RENDER so parallel workers skip the render race
(append is atomic) -- backward-compatible parallel-safety. backlog.md refreshed
with the re-test's whole-binary-informed scores.
2026-07-14 23:54:16 -06:00
Drew T f2b2a9d778 docs(phase-26a): A10 Max-phase — wall re-test scoping + arity measurement (fan-out pending R27)
Scoped the 5 walls. The audit fixes already dissolved the easy wall (A3f/g/h +
A9b, 66.5->68.6% instr). Remaining re-test pool = 2,218 open-stub backlog drafts
(958 at closeness 0). #2 arity: 13/18 Phase-15 arity-conflict fns already banked
(wall largely fell). Confirmed the remaining re-test is breadth (serial gate_stage
timed out). R27 boundary reached: prompting Drew for /effort ultracode before the
parallel re-gate fan-out. Tree clean (HEAD commit:0618 before this).
2026-07-14 22:27:19 -06:00
Drew T ea20bdf9f3 fix(phase-26a): A9g — jr_inventory: retire the ephemeral roster, derive banked from the image (R33)
jr_inventory's `banked` set was filtered by an EPHEMERAL, gitignored
.run/banked_func_*.json roster: a `rm -rf .run` / fresh clone would blind ALL
banked jr at once, cross-address siblings (roster named after the exemplar) were
structurally invisible, and non-leader banked jr were missed. "The purest R33
case in the group" (audit).

FIX (the audit's exact prescription): delete the roster glob + `cand` filter;
`banked` is DERIVED FROM THE IMAGE — a real-C def/define fn is a banked jr iff
family_remap.reloc_targets shows it references a committed .rodata carve offset
(config + image, both durable; cross-address- and non-leader-immune). R32
assertion: every committed carve must resolve to EXACTLY ONE owner or abort (a
stranded/duplicated carve is the §8b func_801734BC incident, never silent).

Also fixed the adjacent finding: the asm_jr scan's func_-fullmatch dropped the
curated-name listCdBuffer jr; now resolved via oss.addr_of(). (The --only path's
own fullmatch is left — it parses user input, not the corpus.)

Perf: read the overlay image ONCE and pass it to reloc_targets(..., data=) — a
new backward-compatible param on family_remap (regression: 0/80 mismatch vs the
re-read path).

Verified: data-param behavior-identical; the R33 win — ov_SC02_000 now finds the
cross-address sibling func_8017FCB0 the roster missed; full-fleet parallel run =
134/134 OK, 0 false aborts, 1336 banked jr == 1336 carves -> 1:1 ownership holds
fleet-wide. Byte-safe: jr_isolate_all is not in the make build/extract path
(R22-neutral); the change makes future isolations strictly more correct.
2026-07-14 22:03:43 -06:00
Drew T 96e025a324 fix(phase-26a): A9f — overlay_src_split swallowed 2 real definitions; the selftest was blind
scan_construct's force_decl latched from the FIRST token and returned at the
first depth-0 `;`, so a definition sharing a physical line with leading externs
(`extern A; extern B; void f(){...}`) was never anchored — absorbed into the
next anchor's preamble. The parser jr_isolate_all rewrites source from was short
two functions in the exemplar overlay. The round-trip selftest is a SERIALISATION
check (a miss lands in a preamble -> round-trip still exact BY CONSTRUCTION), so
it was structurally incapable of seeing this.

FIX (byte-safe): force_decl no longer survives a same-line `;` with trailing
code — re-classify from the remainder and keep scanning so the def anchors (its
leading externs stay in its whole-line item text -> round-trip byte-identical).
Rejected the audit's "split into 3 constructs": round-trip joins whole-line
chunks with `\n`, so sub-line splitting would insert a newline where a space was.
def_name now names the LAST top-level header before `{` (the definition, not the
first same-line extern; byte-identical on every single-def construct).

R32: hidden_definitions() coverage oracle wired into selftest — an independent
detector of `func_XXXX(...){` bodies not anchored. The selftest is now a coverage
check, not just serialisation.

Verified: 2 swallowed -> 0; regression over 1738 overlay .c = 0 round-trip fails,
0 non-monotonic, 0 non-additive changes, +2 anchored defs. Byte-safe: tool not in
the build path (R22-neutral); ov_SC01_077 rebuilds d19c9580; neither def straddles
a committed subseg boundary. Audit ledger line refs were stale (src rewritten);
real cases are ov_SC01_077_after.c:2020 + ov_SC01_077_jr_8015444C.c:1495.
2026-07-14 21:43:52 -06:00
Drew T 68d29ba8af fix(phase-26a): A9e — reconcile_tu already wired into bank_exemplar (A3d); document the ladder
NULL RESULT (P9/R14): the session-12 "wire reconcile_tu into bank_exemplar"
handoff item was stale — A3d (commit:0601) already wired reconcile_tu into
jtbl_family_bank.recover() ("on BOTH banking paths"), and bank_exemplar's
`recovered` stage delegates to fb.recover = cast_call_sites + reconcile_tu.
Proven working by A9b (func_8017A4AC banked at the recovered stage, reconcile_tu
resolving its struct + fn-ptr conflicts). No live tool references the RETIRED
reconcile_decls (only docstrings + the audit-cdecl differential harness).

No code change warranted. Byte-neutral hardening only: document the
raw/scoped/recovered/reconciled fallback-ladder composition inline in
bank_exemplar so a future session does not re-run this "is it wired?" trace.
2026-07-14 21:25:19 -06:00
Drew T 40477281ce fix(phase-26a): A9d — retire the dead Phase-17 canonical-sig chain (R33)
DELETE tools/census_conflict_callees.py + tools/derive_canonical_sigs.py.

- census_conflict_callees: audit-CONFIRMED marked-for-deletion (commit:0593;
  decision-log 836). It re-derives from C text the per-TU "defined/declared/
  stubbed/external?" question that reconcile_tu (Phase 26) answers FROM THE
  BUILD — and does it WRONG in the unsafe direction (unknown -> conflict-free).
- derive_canonical_sigs (census's ONLY consumer): genuinely dead — last touched
  Phase-17 (commit:0140), output .run/canonical_sigs.json read by nothing (no
  Makefile/workflow/import), no-ops on the 2-byte [] input, asm-arity heuristic
  36% wrong vs byte-exact banked C. Its purpose was retired in A3d
  (fleet-majority oracle -> reconcile_tu's per-TU oracle). Deleting census
  orphans it, so the whole dead chain ceases to exist (R33: the best outcome is
  a DELETED SCANNER, not a fixed regex).

Byte-neutral by construction (neither tool is in any build/report path):
module-import smoke over the 13 importable harvest/bank/report/reconcile tools
= all clean; bank_exemplar is a run-only script (indexes sys.argv at module
scope), imports neither deleted module. No src/config change -> no byte moves.

Doc-pointer hygiene: hand-matching-process.md 8a, matching-cookbook.md
(canonical-sig-layer entry), tooling-audit.md (ledger row + derive entry) all
annotated DELETED/historical so nothing points at a nonexistent tool.
2026-07-14 21:21:02 -06:00
Drew T 181191b6af docs(phase-26a): log A9c (lint_symbol_refs green + wired into make report) 2026-07-14 20:46:53 -06:00
Drew T 49004f5b06 docs(phase-26a): log A9a (canon_sig_reconcile fn-ptr fix) + A9b (func_8017A4AC ×134 wall re-test) 2026-07-14 20:39:23 -06:00
Drew T 959cc9b7c1 docs(phase-26a): log A3h — standing-lead harvest (measured, banked +1.3% instr, Bucket X honest-stop) 2026-07-14 19:04:13 -06:00
Drew T 7a3921a3f3 docs(phase-26a): log A3f+A3g (harvest + propagation) + disk-hygiene note 2026-07-14 16:48:23 -06:00
Drew T b89fcc2edc fix(phase-26a): A3e — gate_stage pinned the byte-gate back to 4.9%, OF A3'S OWN FIX
THE WORST DEFECT IN THE AUDIT IS NOT IN A SCANNER. It is one default argument in the CALLER of a
scanner we had already fixed.

    # tools/gate_stage.py:315
    summary = run_gate(a.drafts, binary=b, src=a.src or f"src/{b}/{b}.c", ...)   # ALWAYS the main .c

`src` RESTRICTS the byte-gate to ONE translation unit, and _gate1 does `if src: cmd += ["--src", src]`
-- always truthy. A3 had just taught harvest_verify to DERIVE each draft's home TU *when --src is
omitted*, lifting the byte-gate's reach from 4.9% to 100%. gate_stage NEVER OMITS IT. The fix was
neutralised by its own caller's default, and the PRIMARY BANKING PATH -- every wave, the grinder, the
orchestrator, bulk_harvest -- remained structurally unable to bank 250 of ov_SC01_077's 263 stubs.

WHY IT SURVIVED 26 PHASES: harvest_verify cannot splice a draft whose stub is not in the TU it was
pointed at, so the draft never verifies -- and is then logged as near/failed, i.e. AS A MATCHING
PROBLEM. The wave reports a poor close-rate; the function goes to the backlog as a compiler residual.

    A tool that CANNOT bank a function is indistinguishable, in every log this project keeps,
    from a function that CANNOT BE banked.

PROOF, same draft / same gate / same second: gate_stage rejected func_80129C40; harvest_verify run
directly (no --src) VERIFIED it byte-identical and banked it.

AND A COUNTING BUG THAT HID THE HIDING (gate_stage:261): when match_one says MATCH but the whole-binary
gate rejects, the record is logged status="near" and THE COUNTER IS NEVER INCREMENTED. A 63-draft run
printed `banked 0, near 0, failed 0` -- three zeros that do not sum to 63 -- for phases. Nobody ever
added them up. (The number was not wrong. It was ABSENT.)

ALSO FIXED, sig_unify (the same disease, one level down): it SILENTLY DROPPED 190 of 196 drafts (97%).
`cur_stubs` was read from the main .c (13 of 263 stubs), so any draft whose stub lives in a _jr_ carve
hit `if fn not in cur_stubs: continue` -- dropped BEFORE THE WRITE: never copied to --out, never gated,
never logged, while the summary printed "drafts unified: 6" and read like success. THIS IS GATE_STAGE'S
STAGE-2 RECOVERY -- the pass whose whole job is to rescue the stage-1 failures -- and it has been a
no-op for nearly every draft it was meant to save. Now: TU derived per draft (corpus.stubs), canon
derived from cdecl.tu_scope (cpp -- macro-injected decls finally visible), and _keep() so an
already-acceptable decl is left alone (the §19 "sig_unify regresses canonical drafts" failure mode).
Reach: 6 -> 196 drafts; callee-externs rewritten 2 -> 90; own def-sig 2 -> 86.

MEASURED, all three consumers migrated (196 never-banked drafts):
    near   5 -> 116        failed  190 -> 17
=> 173 of 190 "failures" were PLUMBING, not codegen: now compiling and SCORED instead of invisible.

THE PRIZE (measured, not claimed): the backlog holds 1,588 entries at closeness==0 -- body byte-exact
per match_one, whole-binary gate rejected. 1,215 have been banked since by other paths. 373 ARE STILL
OPEN STUBS WHOSE BODIES ARE ALREADY BYTE-EXACT, sitting in a ledger that calls them unrecoverable.

⚠ THE HARVEST ITSELF IS NOT IN THIS COMMIT, AND IS NOT CLAIMED (P9). Gating the 63 ov_SC01_077 ones
dragged `dedup_propagate --auto-from --recover` behind it; it ran >1h and hit its timeout -- its
first-ever run over the FULL corpus (A6/A7 unblocked the 407 files it could never see). It MUTATES THE
TREE BEFORE IT GATES, so the kill left 859 files + engine_core.h (+544 lines) written and UN-GATED with
the registry never updated. R22 on that tree: 44 passed / 92 FAILED -> `git checkout -- src/ config/`,
fleet restored to 136/136. Nothing lost (H4: the tree was clean, so the revert was one command).
Two real lessons, recorded: dedup_propagate is NOT crash-safe and must never run under a timeout it can
hit; and a 63-draft experiment must not drag an unbounded fleet-wide propagation behind it.

  R22 clean-fleet after revert: 136 passed, 0 failed of 136.  src/ and config/ clean.
  cookbook §51g LAW 11: A FIX IS NOT LANDED UNTIL ITS CALLER STOPS OVERRIDING IT. After fixing a
  scanner, grep every call site and ask whether a caller's default re-disables it. An audit that stops
  at the callee is half an audit.
2026-07-14 15:39:07 -06:00
Drew T 4aae5e7589 fix(phase-26a): A3d — retire the fleet-majority oracle: it was WRONG for the TU 16% of the time, on both banking paths
R33 applied to the worst finding in the audit: this oracle was not fixed, it was RETIRED.

    reconcile_decls asks "what does the FLEET call this symbol?"
    C asks           "what does THIS TRANSLATION UNIT declare?"

The engine is loosely typed -- the same address is legitimately declared with incompatible types in
different overlays -- so a single fleet-wide answer is WRONG FOR SOME TU BY CONSTRUCTION. And it is
worse than a silent skip: it writes an ACTIVELY WRONG declaration into the draft, which then
collides with the very TU it was meant to conform to.

MEASURED across ov_SC01_077's 12 TUs, against what cpp says each TU really declares:

    the fleet oracle AGREES with the TU ................ 2883
    the fleet oracle CONFLICTS with it (cc1 REJECTS)  ..  548    <- 16%
    the TU declares it, the oracle has NO answer ......   357

and it was LIVE ON BOTH BANKING PATHS:
  * gate_stage      -- rewrote 60 of 196 drafts in the current batch
  * jtbl_family_bank -- EVERY SIBLING of the ×134 family sweep, the project's economic engine.
    A poisoned decl means that sibling silently does not bank, and the loss is invisible: the sweep
    simply reports a smaller number. The irony is exact -- that function's own docstring already
    knew the conflicting symbols are PER-OVERLAY, which is precisely why a FLEET oracle could never
    have been right.

reconcile_tu.py (written in Phase 26 but NEVER WIRED) now supersedes it, rebuilt on cdecl:
  * ask cpp what the TU declares (macro-injected DEFINE_func_* externs included -- a raw scan
    cannot see them, §8c / §51g LAW 7);
  * ask cc1 whether the draft's decl can coexist (cdecl.compatible, validated against the real
    gcc-2.7.2 front end on 1,485 live pairs -- NOT the C standard, NOT modern gcc; §51g LAW 9);
  * NOT declared -> leave the draft alone (its extern types are load-bearing: %lo-folding, access
    width, alignment); compatible -> nothing; CONFLICTING -> the TU wins + cast at every USE so the
    draft's intended access survives byte-for-byte;
  * derives WHICH TU from corpus.stubs() rather than a hand-passed --src-file (§51g LAW 10).
  * handles the fn-ptr kind NATIVELY -- which is why it supersedes rather than patches: teaching
    reconcile_decls' parser to see `extern void (*D_x[])(void);` would have ARMED its fn-ptr-blind
    data_access_subs to rewrite a call-through `D_x[i]()` into `((u8 *)D_x)[i]()`. Fixing the regex
    would have detonated a dormant bug.

AND THE NULL RESULT, AGAIN, REPORTED AS SUCH (P9/R14): on the 196 never-banked historical drafts the
new oracle banks EXACTLY AS MANY AS THE OLD ONE -- zero. That tail fails on CODEGEN, not on decl
plumbing. The two disagree on 45 of 196 drafts and the outcome does not move. This is a CORRECTNESS
fix (548 wrong declarations removed from two live pipelines, protecting all FUTURE drafts and every
future family sweep), not a banking win, and it is not being sold as one. Three nulls in one session.

reconcile_decls.py is kept as EVIDENCE, marked RETIRED, with no live caller.

  R22 clean-fleet: make clean + extract-all + check-all -> 136 passed, 0 failed of 136
  src/ untouched (0 changes)   reconcile_tu: 0 coverage defects over 196 drafts
  NOTE: the family-sweep path gets its real exercise at Task 8 -- watch the per-sibling bank rate.
2026-07-14 12:53:14 -06:00
Drew T f4502f11bb fix(phase-26a): A3c — the recovery passes were reconciling 95% of drafts against the WRONG TU
FIRST CONSUMER MIGRATION onto the cdecl oracle — and the compiler taught me two things I had
wrong, one of which reopens a wall that has been closed since Phase 15.

1. cdecl.compatible() — "will cc1 accept these two declarations of one name?"
   The predicate four tools each half-implement and get wrong: norm_sig / _norm_type collapse the
   int family to ONE token, so a SIGNEDNESS change reads as "already compatible" and gets no
   rewrite -- while cc1 REJECTS that redeclaration. Right about codegen, wrong about the front end,
   which never reaches codegen.

2. THE ADJUDICATOR MUST BE THE COMPILER THAT COMPILES YOUR CODE (cookbook §51g LAW 9).
   I wrote the rules from the C standard, then let a compiler judge. It contradicted me -- and then
   the RIGHT compiler contradicted the first one. Three different answers:

       declarations in one TU        | standard | modern gcc | gcc-2.7.2 cc1
       typedef int X;  twice         | error    | ACCEPTS    | ERROR
       extern u16 X; + volatile u16 X| error    | error      | ACCEPTS
       void X(s16);  then  void X(); | error    | error      | ACCEPTS
       void X();     then  void X(s16)| error   | error      | ERROR

   --compat now adjudicates with tools/bin/gcc-2.7.2-psx/cc1, the front end that actually
   arbitrates the build: 1,485/1,485 live corpus pairs agree, 0 disagree, 0 skipped.

3. THE PRIZE: the Phase-15 narrow-param wall rests on a false premise.
   The no-prototype rule is ORDER-DEPENDENT. `void X(s16); void X();` COMPILES; only the reverse
   fails. Phase 15 closed "the 159 arity/narrow-param conflicts" as "no clean deterministic fix --
   it is simply C's default-promotion rule". cc1 does not enforce that rule in the direction the
   wall assumed. Four three-line probes, 90 seconds, zero tokens. -> A10 RE-TEST TARGET.
   Probe the compiler for FACTS; read its source only for LEVERS; byte-validate both. (We read
   gcc-papermario for five phases believing it was 2.7.2. It was 2.8.1.)

4. THE MIGRATION: cast_call_sites canonicalized 95.1% of drafts against a TU that would never
   compile them. `--src-file` is an OPTIONAL HAND-PASSED flag defaulting to src/<ov>/<ov>.c, and no
   caller knows about the Phase-26 _jr_<ADDR> carves: ov_SC01_077 has 263 open stubs across 12 TUs
   and only 13 are in the main .c -- while harvest_verify (A3) correctly splices into the real one.
   Now DERIVED from corpus.stubs() (the INCLUDE_ASM line is self-describing), with the canonical map
   derived from cdecl.tu_scope() (cpp -- so macro-injected DEFINE_func_* decls are finally visible).
   Callee-conflict repair reach: 8 -> 58 of 196 drafts (7x).

5. AND THE NULL RESULT, REPORTED AS SUCH (P9/R14). Those 58 banked ZERO functions. The historical
   draft tail fails on CODEGEN, not plumbing -- func_801387B8, which the audit blames on a single
   unparsed `[4]`, is really 67/100 instructions off with a $s0/$s1 swap (that claim does not
   reproduce on today's tree). The real gain is narrower and still worth having: 52 drafts moved
   from "won't compile" to "compiles, N instructions off" -- from an INVISIBLE failure that reads as
   a compiler wall into a SCORED near-miss the permuter and the §47/§48 dials can act on. That is
   the audit's thesis, not a bank. THREE times in one session a confirmed mechanism produced a null
   consequence.

Also: my own new audit printed "ALL ORACLES GREEN" while silently skipping 100% of its corpus (a
missing -Isrc). The exact bug class, in the tool written to hunt it. An unadjudicable check is not
a passed check.

  R22 clean-fleet: make clean + extract-all + check-all -> 136 passed, 0 failed of 136
  src/ untouched (0 changes)   make audit-cdecl: green   --compat: 1485/1485
  NEXT: sig_unify + reconcile_decls carry the SAME wrong-TU bug (same --src-file flag).
2026-07-14 12:26:01 -06:00
Drew T f9742cf9c0 feat(phase-26a): A3b — cdecl.py, THE C-declaration oracle: one grammar, fifteen deleted models
Fifteen tools each carried their own regex model of "what is a C declaration", and they
disagreed — two tools in ONE pipeline disagree today about whether `extern s32 D_a, D_b;`
is a declaration at all. All fifteen shared one character class,
    extern\s+([A-Za-z_][\w\s\*]*?\bD_[0-9A-Fa-f]+\s*(?:\[\s*\])?)\s*;
which cannot hold '(', ',', or a non-empty [N] — so three whole shapes were invisible to
every one of them: fn-ptr/jump-table arrays, sized arrays (one unparsed `[4]` has blocked
func_801387B8 in 134 TUs), and multi-declarators (the WHOLE line dropped, not just #2..N).

REJECTED the audit's own prescription (a shape-aware alternation per tool, ~15 coordinated
regex edits) on R33 grounds: fifteen hand-maintained models are exactly what diverged, and
an alternation only ever covers the shapes somebody remembered. The thing being scanned HAS
A GRAMMAR. C's declarator grammar is small, closed and TOTAL — it describes fn-ptr arrays,
sized/2-D arrays, multi-declarators, fn-ptr params and K&R identifier-lists without being
told they exist. ~250 lines of recursive descent: LESS code than the regexes it deletes, and
exhaustive by construction rather than by memory. (decision-log 2026-07-14.)

Two statement paths, because the inputs genuinely differ:
  * tu_statements()    - a TU's file scope, derived from cpp. A decl inside a DEFINE_func_*
                         macro body declares NOTHING until the macro is invoked (the §8c law);
                         a raw scan is wrong in both directions. cpp answers it exactly, in
                         54 ms/TU (~20 s for the fleet, cacheable).
  * split_statements() - span-preserving raw split, for drafts (which get rewritten).

THREE ORACLES, whole corpus — a measurement, not a belief:
  * coverage      2,952,246 depth-0 statements -> 2,731,521 declarators, 0 PARSER DEFECTS
  * the real gcc  50,405 distinct declarations compiled beside this parser's reconstruction
                  of each one -> 0 REJECTED
  * differential  0 file-scope symbols the incumbents see that cdecl misses; 26 in
                  engine_core.h they cannot see; 6 they wrongly promote from BLOCK scope

Two ideas worth keeping (cookbook §51g, LAWS 4-8):
  * THE CANDIDATE SET IS DERIVED TOO (R33 applied to R32). At file scope C admits nothing but
    declarations, so R32's over-approximating detector is *every depth-0 statement* — supplied
    by the grammar, with no hand-maintained candidate regex to rot.
  * GCC ADJUDICATES MY OWN COVERAGE GAP. Deciding for myself which failures "don't count" is
    grading my own homework — the habit that wrote the fifteen bugs. A statement gcc ALSO
    rejects is not C (my rejection is correct, the INPUT is corrupt); one gcc ACCEPTS and I do
    not is MY defect. All 33 residual: NOT-C, all dead .run/drafts* scratch, none in src/.

NEW findings (docs/tooling-audit.md):
  * reconcile_decls.DATA_DECL_LINE_RE finds ZERO decls in engine_core.h — it is line-anchored
    and every decl there ends in a '\'. Its "authoritative tier" has ALWAYS been empty.
  * gen_harvest_targets + sig_unify count BLOCK-SCOPE externs (6, byte-proven inside a macro's
    function body) as file-scope canonicals — the §8d `conflicting types` confusion.
  * tu_ambient's func regex ([^()]* params) drops ANY callee with a fn-ptr parameter.
  * R14 near-miss: 33 drafts contain `extern if ((func_80029178(0x119) & 0xFF) != 0);`, written
    by a RECOVERY TOOL — but the source bug was already fixed in Phase 19 (0 garbage / 300 sigs
    today). Mechanism confirmed, consequence nil. Note what it cost while live: a draft that
    cannot compile fails the byte-gate and reads downstream as an INTRINSIC COMPILER WALL.

Bugs the oracles caught in ME (and would otherwise have shipped): `extern s32 (*D_801274D0)(s32);`
parsed the BASE TYPE as the name; a K&R declaration-list flushes as SEVERAL spans, so the body
attached to the wrong one and leaked the K&R parameter names into file scope as fake globals.

SCOPE, deliberate: NO consumer is migrated here, so this cannot move a byte. The audit warns
that making the parser see more ARMS dormant transforms (reconcile_decls.data_access_subs would
mangle `D_1[i]()` -> `((u8 *)D_1)[i]()` the moment fn-ptr decls become visible to it). Migration
is one tool at a time, each byte-gated.

  R22 clean-fleet: make clean + extract-all + check-all -> 136 passed, 0 failed of 136
  make audit-corpus: 0 PHANTOM + 0 TRUNCATED    make audit-cdecl: ALL ORACLES GREEN (new gate)
2026-07-14 11:37:13 -06:00
Drew T c7772bc452 docs(phase-26a): SESSION-9 CLOSE — cookbook §51 (the tooling-integrity laws) + handoff
R30/R16: the context-dependent artifacts, written while the context is live.

cookbook §51 — the SILENT SKIP: the bug class, why the byte-gate cannot see it, the
over-approximating-detector method, and FOUR LAWS:
  1. Derive, don't re-derive — the best outcome is a DELETED SCANNER (28 findings -> one
     derived oracle + ~10 deleted scanners). A derived fact cannot rot; a hand-maintained
     copy of it is a liability that grows with every structural change.
  2. Assert your COVERAGE, not merely your correctness. *** A LOUD FAILURE THAT NOBODY
     COUNTS IS EXACTLY AS INVISIBLE AS A SILENT ONE *** — build_engine_types printed
     '[overlap] handle manually' every single time for four phases while dead on 81% of its
     own corpus. This CORRECTS the first draft of R32 ('fail loud'), which was not enough.
  3. When an oracle is structurally blind to a class of error, add a SECOND ORACLE THAT CAN
     DISAGREE WITH IT — not a better assertion inside it. We had two all along and never made
     them argue. (And scope the comparison to where the second oracle is genuinely independent:
     the same check run outside its domain reports 914 slices when the truth is 193.)
  4. A rule that needs a human to remember it is not a gate. Make it structural.
  + the FALSE-WALL PIPELINE (a silent skip -> a wasted draft -> a backlog 'matching failure'
    -> reserved_walls() PERMANENTLY blacklists a function that was never attempted), and a
    checklist for any new corpus-scanning tool.

CURRENT_PHASE: session-9 handoff — what is done, what remains (each with its spec on disk),
and the R32-corrected / R33 / R34-new rule candidates for P10 ratification.
2026-07-14 10:40:04 -06:00
Drew T a2a507d079 docs(phase-26a): A4/A5 + the stale-object false-pass hole recorded 2026-07-14 10:12:59 -06:00
Drew T 3be734bde8 docs(phase-26a): A3 progress — the oracle is built; 4 of ~10 scanners deleted
corpus.py + build_fuel_manifest + wave_targets + harvest_verify/gate_stage landed.
Remaining: family_manifest/family_hseq (62% of the endgame plan is phantom targets),
DELETE census_conflict_callees (R33), exemplar_miner/difficulty, jr_isolate_all.
2026-07-14 09:30:38 -06:00
Drew T ffb6f1a40f docs(phase-26a): A2 — the full audit; 28 findings survive; the endgame plan was majority-fiction
38 agents / 2.24M tok / 0 err. 32 findings raised -> 28 SURVIVED adversarial verification
(4 REFUTED, 16 downgraded). 40 scanners measured CLEAN. Full write-up: docs/tooling-audit.md ROUND 2.

THE ROOT CAUSE — one bug, ~10 times: a hand-maintained model of the corpus layout (a file
allowlist, a single-.c assumption, a func_-only regex, a REGION_SUB dict) sitting on top of a
filesystem that already answers the question. Every TU split silently widened it.
DECAY PROVEN: .run/fuel_manifest.json (Jul 8) recorded 130 stubs; the same tool today returns 30.
The Phase-26 splits moved ~100 stubs out from under a dict literal last edited in Phase 22 — and
nobody noticed, because an un-nominated target produces SILENCE, not an error.

MEASURED: 91.6% of ALL remaining project gain is invisible to target selection (true 994,633 ins;
the manifest sees 83,305). 117 of 127 reach-134 fns never nominated. harvest_verify cannot see
56,742 of 58,717 (96.6%) open stubs. wave_targets hands 78 of 87 targets a nonexistent asm path.

THREE RESULTS OVERTURN SETTLED CONCLUSIONS:
 1. Phase-22's 'the permuter's fuel is exhausted' is UNSAFE. grinder banks through harvest_verify,
    which sees ONE TU — 1,290 of its own 1,298 queued fns live in another. 99% could never have
    banked. '0 banks since Phase 21' is equally consistent with 'the tool could not bank'.
 2. The Phase-25/26 endgame plan is MAJORITY-FICTION. family-manifest.md advertises 2,758
    multi-member families / 11.0 MB; 1,071 of them / 6.80 MB (62% of the byte-weight) are ALREADY
    FULLY MATCHED. The ranking — the file's whole purpose — is sorted mostly on dead work.
 3. A CORPUS defect the byte-gate is structurally blind to: symbols.us.txt:981 puts a main-EXE DATA
    symbol (listCdBuffer = 0x80180000) into every overlay's symbol stack, but in overlay space that
    address is CODE. splat cuts 97 real functions in half and invents 96 phantom ones = 193 slices
    NOBODY CAN EVER MATCH, in 97 of 134 overlays — and the build stays byte-identical and green,
    because the .s halves are pasted back verbatim. A perfect correctness oracle, a null coverage
    oracle. What saved us: sig_image was RIGHT (58,524/58,621 vs spimdisasm; correct on all 97
    disagreements). A SECOND INDEPENDENT ORACLE is the only reason it was visible at all.

FIX RESTRUCTURED around the root cause: ONE derived corpus oracle (A3) + ~10 DELETED scanners —
not ten fixed regexes. Plus the listCdBuffer corpus fix (A4) and the closeness oracle (A5, which
lies on 155 functions, feeding false walls into reserved_walls()).

decision-log (R31): the why, and the design lesson — a derived fact cannot rot; a hand-maintained
copy of it is a liability that grows with every structural change. We had no instrument that could
report ABSENCE: every gate we owned answered 'is this right?', none answered 'is this all?'
2026-07-14 03:41:43 -06:00
Drew T 92a695ef93 docs(phase-26a): ORDER CORRECTED — the full 18-tool audit runs BEFORE the fix campaign
Drew, mid-session: 'I thought the last session said there were some 15 tools we need to audit.'
He was right, and my ordering was wrong.

I had put the 18-tool audit near the END (as A9). docs/tooling-audit.md prescribes the opposite:
dedup_integrate -> jtbl_family_bank -> the SELECTION tools -> masked_diff/match_one -> THEN the
40 measured findings. The reason is the one that matters:

  A hole in a SELECTION tool makes work invisible to PLANNING — the worst kind, because you
  never know to look.

Fixing on top of unaudited selection tooling means re-running every fix when the audit later
finds the hole. So: A2 is now the full audit; A3-A9 (the fix campaign) are blocked on it.

Tool coverage, stated plainly: A1 (1) + A2 (18) + the fix campaign (~17 already-measured) = ~36
tools — not 82. The filter, from the audit doc: does it PARSE something, and does it GATE or
SELECT work? The remaining ~46 are dead LLM-tier scripts.

A1's result recorded in-file (the three false greens, the causal chain, the null-result blast
radius that confirms R33).
2026-07-14 02:51:59 -06:00
Drew T 978ef703ac docs(phase-26a): A0 — the tooling-integrity audit, as an INSERTED HALF-PHASE (Drew's call)
- Drew (2026-07-14, gate 1): run the audit inside Phase 26, then resume at Task 7.
  Declined the alternative (close Phase 26 early on an unmet milestone -> Phase 27):
  the audit is a PREREQUISITE to structural completion, not a successor to it — the
  tooling that MEASURES the milestone is the thing at fault. Phase-3.5 precedent.
- CURRENT_PHASE.md: the Phase 26-A block (A0-A11), built FROM docs/tooling-audit.md
  (40 measured findings), R33-before-R32 ordering — the best outcome is a DELETED
  scanner, not a fixed regex.
- decision-log (R31): the why, the structural blind spot (a scanner extracts N, the
  true count is M > N, and nobody ever compared N to M — the byte-gate is a perfect
  CORRECTNESS oracle and a NULL COVERAGE oracle), and A1's first finding.
- harness task list built (R28).
2026-07-14 02:39:38 -06:00
Drew T b757de73a8 docs(phase-26): checkpoint now POINTS AT docs/tooling-audit.md as the audit phase's input document
The 40 measured findings were living only in an ephemeral workflow journal outside the repo; the
checkpoint carried my SUMMARY of the audit, not the audit. Now the fresh session is routed to the
evidence, with the priority order (dedup_integrate FIRST — it can print a false green), the R33-before-R32
method (the best outcome is a DELETED scanner), and the real prize: re-test the walls that were diagnosed
on top of the broken 10% callee oracle (the def-side loose-typing wall, the 159 arity conflicts, the
type-heavy tail).
2026-07-14 02:26:14 -06:00
Drew T 6e8c459ade docs(phase-26): SESSION-8 CLOSE — checkpoint for a fresh session; the tooling-integrity audit gates what comes next
RESULTS. Fleet instr-weighted 63.0 -> 66.5%, distinct-code 39.1 -> 46.8%, fn-count 82.61%.
FINAL R22: make clean + extract-all + check-all -> 136/136 BYTE-IDENTICAL, 0 coverage defects.
dedup 1813/0. 0 NON_MATCHING (G4). 31 commits.

13 CORES CRACKED incl. the four heaviest functions in the game (952/890/562/536 ins). The 12-agent
Ultracode wave returned 11/12 first-pass MATCH, each adversarially verified (a skeptic re-ran match_one
+ the §8a jump-table check). Banked x134 this session: func_8015AE2C, func_80178D40, func_8015A3C8,
func_8013FFD8, func_8016AB6C, func_8015444C, func_801380E0 (+ func_8017BEBC x1).

THE TOOLKIT CROSSED A LINE — three ZERO-BYTE DIALS now cover the three passes that produce essentially
every "irreducible" residual, each with a diagnostic signature a cheap agent recognises on sight:
  registers rotated            -> global.c allocno priority -> §47 slider / §48-A pricing dials
  two insns swapped, SAME regs -> sched.c rank_for_schedule LUID tiebreak -> §49 LUID dial
  structure right, count wrong -> loop peel / cross-jump -> §46 / §48-D
That is why 9/12 fell first-pass to ORDINARY agents. Fable5 DISCOVERS a class; everyone else APPLIES it.
New: §46 §47 §48(+A4) §49 §50. Read §50-B before using §48-A1/A4 — it BOUNDS them (the "cross_jump
refunds the bytes" claim is FALSE for a 1-insn tail reached by two jumps; jump.c:1993 minimum=2).

DREW'S DIRECTIVE (binding): the TOOLING-INTEGRITY AUDIT comes BEFORE any further matching work, and is
NOT part of this phase. First act of the fresh session is a Tier-1 phase-boundary call (close Phase 26
early, or run the audit as an inserted phase — Drew decides).

WHY: seven silent-skip tool bugs in one session, and they are a STRUCTURAL blind spot — a scanner
extracts N items, the truth is M > N, and nobody ever compared N to M. The byte-gate is a perfect
CORRECTNESS oracle and a NULL COVERAGE oracle: it has been green since Phase 5 at 0% decompiled (
INCLUDE_ASM pastes the ORIGINAL asm), so a green gate is compatible with ANY decomp %. One 10% hole in
the callee oracle made NINE byte-exact functions look like an intrinsic compiler wall. The real question
the audit answers: how many walls we have already "byte-proven" across 26 phases were lookup misses
wearing a wall's clothes? (The def-side loose-typing wall, the 159 arity conflicts, the type-heavy tail
were ALL diagnosed on top of that hole.) Audit scope so far is 19 of 82 tools (23%), by risk — NOT
comprehensive; dedup_integrate.py is unaudited and can print a FALSE GREEN.

RULE CANDIDATES (P10, Drew ratifies at PhaseEnd):
  R32 Coverage assertion — a corpus scanner must assert its own coverage and fail loud on unparsed input.
  R33 Derive, don't re-derive — where a proven invariant answers the question, derive from it. The best
      audit outcome is not a fixed regex; it is a DELETED scanner.

SELF-CORRECTION ON THE RECORD (P9/R14): I told Drew the headline numbers under-reported by ~190k
instructions. WRONG. weighted_metrics() never calls classify(), so it was structurally immune; the
published numbers were correct all along. I verified the DEFECT but not its BLAST RADIUS. A null result
against a strong prediction is a refutation — chase it.
2026-07-14 02:21:08 -06:00
Drew T 91fd1257ba docs(phase-26): session-8 checkpoint — 3 heaviest cores cracked, 12-core wave 9/12 MATCH, 6 silent-skip bugs fixed 2026-07-14 00:13:50 -06:00
Drew T 1ab9905368 feat(phase-26): §8d scope_data_externs — the ×133 sweep blocker fixed; func_8015AE2C banked ×134
- ROOT CAUSE (R14 — the session-7 diagnosis was half right): the isolated region builds [ OK ]
  WITHOUT the body, so §8b isolation was never implicated. `family_remap.gather_externs` prepends
  carried decls at FILE scope; D_801812A4 is a fn-ptr dispatch table the sibling declares FOUR
  incompatible ways at BLOCK scope inside its own later functions, so the carried file-scope decl
  ESTABLISHES A GLOBAL THE TU NEVER HAD and every later block-scope extern must now agree with it.
  Byte-proven asymmetry: BLOCK(int)->BLOCK(struct*)->FILE(void*) builds; FILE(void*)->BLOCK(int)
  errors. It was the ONLY hard error in the build — all 27 carried function externs were fine raw.

- THE FIX (demote, don't reconcile): tools/scope_data_externs.py emits a carried D_ extern at BLOCK
  scope inside the function body when the TU has no file-scope decl of it above the insertion point.
  Byte-neutral (an extern emits no code; type + access opcodes unchanged) and never worse than raw,
  so it needs no oracle, no type comparator, no fn-ptr parser. Restores fidelity — the original
  declares these symbols at block scope in exactly this way. Wired into jtbl_family_bank as the
  `scoped` stage: raw -> scoped -> recovered -> reconciled (scoped is the base for the later stages).

- reconcile_decls is the WRONG instrument for this class, twice: its oracle answers "what does the
  FLEET call this symbol" when the question is "what can THIS TU see", and its DATA_DECL_LINE_RE
  cannot parse `extern void (*D_x[])(void *);` — silently skipping the very symbols that were
  failing (the phase's third silent-skip bug, after find_site braces + overlay_files splits).

- R17 TRIAGE RULE, first real test, held: `conflicting types` = the compiler REFUSED TO COMPILE =
  a C front-end diagnostic = our Python. Reading cse.c/global.c would have taught nothing.

- RESULT: func_8015AE2C (562 ins, reach 134) swept 133/133 siblings, 0 failures. R22 clean-fleet
  136/136 BYTE-IDENTICAL (534 changed src files); dedup-check 1813 validated / 0 failed; 0
  NON_MATCHING (G4). instr-weighted 63.0 -> 63.6%; distinct-code 39.1 -> 40.5% (+256 unique fns /
  +79,957 ins) — one core, ~0 agent tokens.

- knowledge captured during the producing session (R30/R31/R21): cookbook §8d, decision-log
  2026-07-13 session 8, SETUP tool-inventory row; CURRENT_PHASE session-8 checkpoint.
2026-07-13 20:45:15 -06:00
Drew T 12631df74a docs(phase-26): R17 triage rule — 'wrong bytes' -> read gcc; 'won't compile' -> read our Python
Drew asked whether the x133 sweep blocker warrants a gcc-2.7.2 source read. It does not,
and the distinction is worth pinning down because it routes every future residual:

- The sweep blocker is a C FRONT-END diagnostic (conflicting types: two incompatible
  file-scope decls of one identifier in one TU). gcc is correctly rejecting plain C89.
  The bug is in reconcile_decls (fleet-majority oracle vs the TU's visible decl).
  Reading cse.c/loop.c/global.c would tell you nothing.
- func_8017BEBC (close=2) is the opposite: it compiles fine and emits the wrong bytes, and
  the cause is localized to global.c's allocno-priority tie. THAT is the R17/§45-B target
  (gdb-on-cc1 read of allocno_live_length) — 2 instructions from a 107K-ins bank.

Rule: 'wrong BYTES' -> read the compiler (R17). 'won't COMPILE' -> read our Python.
cookbook §31-triage + the CURRENT_PHASE NEXT block annotated with the routing.
2026-07-13 19:50:51 -06:00
Drew T 9bbfff9b8c docs(phase-26): session-7 checkpoint — heavy-jr waves run; 1 core banked x1, 2 cracks in hand, sweep blocker diagnosed 2026-07-13 18:35:38 -06:00
Drew T f89fd50afb docs(phase-26): session-6 checkpoint — de-risk complete, 3 gate-cap bugs fixed; §41d 2026-07-13 14:44:07 -06:00
Drew T 38ac5659aa feat(phase-26): §8b scoping wall BROKEN — decl-environment reconstruction + lazy per-core isolation
The full 54-jr isolate-all on ov_SC01_077 now builds d19c9580 BYTE-IDENTICAL
(R22 clean-fleet 136/136) — the configuration session 5 could not build. The
heavy-jr harvest (191 cores / 5.53M templatable ins) is unblocked.

- R14 CORRECTION: session-5's "gcc-2.7.2 block-scope-extern TU-persistence" root
  cause was WRONG. There is no gcc quirk — DEFINE_func_* macros expand at FILE
  scope, so their leading externs are genuine file-scope decls that merely live in
  engine_core.h, invisible to any col-0 .c scan (1377 macros / 3929 lines / 1462 syms).
- REJECTED the approved "global symbol->type map + shadow set" design: the engine is
  loosely typed (func_80173544 is DEFINED `s32 f(void*)` yet declared `extern void
  f(void);` inside func_801734BC's body), so declaring every USED symbol hoists that
  block-scope shadow to file scope and CREATES the conflict a shadow-set then dodges.
  Instead reconstruct the original TU's file-scope decl environment and carry it
  strictly FORWARD — conflict-free by construction (every carried decl already
  coexisted with every definition in the one original TU; compatibility is
  order-symmetric; shadows stay in bodies and travel with their item).
- The byte-gate found two MORE lost decl sources, not predicted: (a) a definition is
  itself a declaration for everything below it in its TU (func_8012B2CC undeclared);
  (b) file-local typedefs used by a carried prototype (parse error, Vec3s). K&R defs
  must render `extern T f();` (unprototyped), never f(void).
- LAZY per-core isolation wired into jtbl_family_bank (Drew's call — upfront-x134 =
  ~7,200 region files): jtbl_carve NON-CONTIGUOUS fail-loud -> jr_isolate_all --only
  <core> -> re-extract -> re-carve. Proven on func_80178D40 (890x134, heaviest core):
  carve blocked -> isolated (byte-neutral d19c9580) -> carve in its own subseg.
- TWO LATENT BUGS fixed (both would have corrupted the heavy sweeps):
  * jtbl_carve.func_subseg derived the owning subseg from the ASM TREE, which `make
    extract` never prunes -> after an isolation it returned the STALE owner and
    silently re-created the very collision the isolation removed. Now config-derived.
  * jtbl_family_bank/jtbl_carve revert() DELETED the shared overlays.mk carve var
    unconditionally -> would destroy a COMMITTED carve (all 134 overlays have one) on
    any failed sibling. Now restored to its committed value; only region files created
    by this attempt are removed; dirty-tree preflight refuses to start a sweep.
- docs: cookbook §8b RESOLVED + new §8c "splitting a TU means rebuilding its
  DECLARATION ENVIRONMENT, not moving text"; decision-log 2026-07-13 (R30/R31).
- parser selftest 404/404; R22 clean-fleet 136/136; 0 NON_MATCHING (G4).
2026-07-13 13:25:39 -06:00
Drew T 234788dfc4 feat(phase-26): §8b overlay-src parser (404/404) + jr isolation tool + the gcc-scoping wall finding
- tools/overlay_src_split.py: overlay-.c-aware partition (header = includes + Phase-17
  canonical-sig layer; per-address items = preamble + body; robust def/decl/K&R/DEFINE_func/
  SETTER/RETCONST classification). Fleet-validated 404/404 overlay .c, 341,902 items —
  round-trip exact / 0 unresolved / 0 non-monotonic. The Stage-2 isolation unblock.
- tools/jr_isolate_all.py: multi-cut jr resegment (config split at jr boundaries, source
  repartition + INCLUDE_ASM path repoint, banked-jr carve repoint, -O0 skip, ambient decl
  carry). SINGLE-cut isolation byte-identical (func_8013FFD8 -> d19c9580, R22).
- FINDING (decision-log 2026-07-13): full 54-jr isolation of the dense _after object hits
  gcc-2.7.2 block-scope-extern TU-persistence (func_801734BC/D_80126B3E declared only in
  engine_core.h DEFINE_func macros); mechanical TU-split breaks it. Fix = declaration-
  completion from a global symbol->type map (Drew-approved next step; lazy per-core).
- baseline intact (ov_SC01_077 rebuilds d19c9580); no config/src/binary change committed.
  CURRENT_PHASE session-5 checkpoint + decision-log R31. db.*.gbf = R23 noise, not staged.
2026-07-13 12:00:26 -06:00
Drew T d1dd29d815 feat(phase-26): §8b multi-jtbl same-subseg — contiguous MERGE (built) + isolation scaffold
Session-4 same-subseg handling (the de-risk preamble's harder half; byte-proof of
the merged build + isolation deferred to Stage 2 with concrete cores):

- jtbl_carve.py: MERGE adjacent same-subseg carves into one spanning .rodata piece
  (a code object emits its jtbls contiguous, so two matched jr-fns in one subseg are
  byte-correct iff their jtbls abut). BOUND-FIX: a new jtbl's end is bounded by the
  next raw dlabel OR the next existing carve start (an already-carved adjacent jtbl
  is gone from the data asm -> raw dlabels over-extend it -> false "non-contiguous").
  Config-proven (func_80171B4C 801D8C48 merges with func_801734BC 801D8C68). NO-OP
  for family-1/cross-subseg (single carve per subseg) -> committed configs unaffected.
- jr_isolate.py (scaffold, NOT yet functional): the non-contiguous case — split a fn
  into its own code subseg (whale _o0b precedent) so its jtbl carves independently.
  BLOCKED on split_src_region, which can't partition the overlay .c (global canonical-
  sig extern layer + per-fn callee-externs + DEFINE_func macros + @class annotations,
  ~922 non-address items). Stage-2 build item (overlay-.c-aware source split).
- cookbook §8b (the --order sandwich + the two same-subseg cases + the blocker);
  CURRENT_PHASE session-4 checkpoint updated with the Stage-2 unblock decision.
2026-07-12 22:06:17 -06:00
Drew T ca50ee6978 feat(phase-26): §8 multi-jtbl --order carve + family-1 (func_801734BC ×134)
- ld_interleave.py --order: address-ordered N-piece data->rodata->data sandwich
  for overlays with 2+ matched jr-functions; legacy --front/--tail path is byte-
  untouched (main EXE + the 133 single-carve func_8012ACE0 siblings unaffected)
- jtbl_carve.py rewritten additive/regenerate-from-config: parse the tail data
  region + existing .rodata carves, split the containing data piece for the new
  jtbl, re-emit the address-ordered pieces + the --order arg; same-subseg carve
  collision fails loud (-> jr isolation); idempotent
- jtbl_family_bank.py: `make extract` BEFORE the carve (asm must match the reverted
  committed config; the old error-string retry was fragile) + revert-on-carve-fail
- family-1: func_801734BC (34-ins PURE jr, ov_SC01_077_after) matched in ov077
  (shared-tail switch idiom) + banked 133/133 siblings = x134 — CROSS-subseg
  multi-jtbl (func_8012ACE0 in _a + func_801734BC in _after)
- R22 clean-fleet 136/136 byte-identical (~52s); 0 NON_MATCHING (G4)
2026-07-12 21:51:27 -06:00
Drew T 79ca6b4975 docs(phase-26): refine session-3 checkpoint — small-jr-first as de-risk preamble, then heavy 191
- Drew's sequencing (agreed): do the 45 small jr families FIRST — not for byte-weight (~+1% instr,
  129K ins) but to de-risk + harden the §8 x134 pipeline before the heavy Fable5 cores bet on it.
- decisive technical reason: jtbl_carve only built the single-jtbl carve; func_8012ACE0 is now
  matched in all 133 siblings, so family #2 forces the multi-jtbl address-ordered `ld_interleave
  --order` carve -> build & prove it on cheap 30-ins targets first. Also needs no Fable5.
- guardrail kept explicit: small tier = MEANS (harden pipeline + build multi-jtbl), NOT the
  objective; the 191 heavy jr families (5.53M ins) remain THE byte-weight target -> pivot after.
- CURRENT_PHASE.md SESSION-3 checkpoint updated to Stage 1 (small + build multi-jtbl) -> Stage 2
  (heavy 191, Fable5 un-paused). decision-log addendum with the forcing-function wiki lesson.
2026-07-12 20:24:21 -06:00
Drew T 4e17a7e77b docs(phase-26): session-3 checkpoint — heavy-byte-weight reframe (§8 unlocked the switch cores)
- decision-log (R31): §8 unblocked the SINGLE heaviest byte-weight chunk of the game — 9 of
  the 10 heaviest unmatched family cores are switch (jr) functions (func_80178D40 890x134 =
  477K ins alone); jr substantial = 191 fams / 5.53M templatable ins. My "45 small jr families"
  recommendation (129K ins) was a light-tail trap — Drew caught it against the endgame plan
  (heaviest-byte-weight-first). Corrected next play: Fable5 crack the heavy jr cores -> §8 x134
  bank -> parallel R22 verify; needs Task 7 (Fable5) un-paused (§8 makes that worth it now).
- CURRENT_PHASE.md: SESSION-3 checkpoint as the fresh-session resume point (4 commits this
  session: tiny-band commit:0531, §8 PoC commit:0532, §8 x134 commit:0533, R22 parallel commit:0534;
  distinct-code 30.3->39.1%, instr-weighted 58.2->63.0%, R22 now ~50s)
2026-07-12 19:20:58 -06:00
Drew T 6fb8d323bd perf(phase-26): parallelize R22 clean-fleet — ~9min -> ~50s (10.5x), 136/136 byte-identical
- profiled the clean-fleet R22: extract-all ~6m11s (136 serial `splat split`) + check-all
  ~2m58s (136 serial builds) = ~9 min, all serial on a 32-core box
- the only shared WRITE is the 4 generated include/*.inc macros at extract time (identical
  content per binary); everything else is per-binary-disjoint and include/ is read-only during
  a build -> concurrent builds/extracts don't race
- Makefile: JOBS ?= 16 + `make extract-all` (seed main serial for the macros, then parallel-
  extract the rest via xargs -P) + parallel `make check-all` (xargs -P), correctness-gated
- MEASURED: `make clean && make extract-all && make check-all` = ~50s, check-all 136/136
  BYTE-IDENTICAL (== the serial result) -> ~10.5x. Compounds across the endgame (R22 per commit)
- new R22 recipe: `make clean && make extract-all && make check-all` (was the serial for-loop)
2026-07-12 19:12:33 -06:00
Drew T 5a08180617 feat(phase-26): §8 ×134 automation — func_8012ACE0 banked fleet-wide (133/133, R22 136/136)
- the jr-function ×134 harvest pipeline, proven end-to-end: per family sibling,
  jtbl_carve (per-sibling jtbl-rodata carve, computed from THAT sibling's own jtbl
  address — the fn is at the same vram across overlays but its jtbl floats) -> make
  extract (auto ld_interleave) -> remap_hseq + canon_sig_reconcile -> whole-binary gate
- tools/jtbl_carve.py: per-overlay §8 carve generator (config data-tail split +
  <ov>_JTBL_INTERLEAVE var)
- tools/jtbl_family_bank.py: the sibling sweep driver (idempotent, revert-on-fail, byte-gated)
- tools/family_remap.py: extract_unit now carries single-line typedefs (jr-function bodies
  define local `typedef struct{} Foo_<addr>;` that must template with the body — the
  propagation cap for these; additive, byte-gate-protected)
- func_8012ACE0 family: 133/133 siblings BANKED, 0 failures; R22 clean-fleet 136/136
  byte-identical; 0 NON_MATCHING (G4)
- metrics: distinct-code 39.1% (50,698 unique fns), instr-weighted 63.0%
- opportunity (has_mid_jr families): 237 total (5,805 members) = 46 small mid/tiny
  (771 members, same mechanical pipeline) + 191 substantial (the Fable5 cores, Task 7 paused)
- NEXT: R22 profiling/parallelization; then the other 45 small jr families
2026-07-12 19:02:25 -06:00
Drew T 095a611e75 feat(phase-26): §8 jtbl-rodata tooling — overlay PoC proven (func_8012ACE0, R22 136/136)
- overlay jr-functions can now bank as C: gcc switch jump tables form a .rodata island at
  the overlay TAIL; carve a matched fn's jtbl into a dotted [.rodata, <code-subseg>] subseg
  + ld_interleave (data->rodata->data sandwich) places it byte-exact. cookbook §8a + SETUP.
- tools/ld_interleave.py: --section .<binary> param (derives the <binary>_TEXT/DATA/RODATA/
  DATA2/BSS symbol prefix); default .main = the EXE, byte-identical (backward-compat proven)
- Makefile + config/overlays.mk: <bin>_JTBL_INTERLEAVE hook + a $(strip)-guarded extract
  branch (gotcha caught: a trailing #comment on the := left whitespace -> non-empty -> the
  branch misfired on resident with the EXE defaults)
- PoC: func_8012ACE0 (25-ins jr-fn in ov_SC01_077) reconciled (canon_sig_reconcile) + banked
  BYTE-IDENTICAL d19c9580 -- the first overlay jr-function matched through the C pipeline
- R22 FULL-FLEET clean rebuild: 136 passed, 0 failed (main 143dbb89 unaffected by the
  ld_interleave change); 0 NON_MATCHING in any default build (G4)
- P9 findings: func_80159C84/func_8015444C (the 2 carried Fable5 jr bodies) are rtu_match
  FALSE-matches (incomplete jtbls: 52B vs 56B -> never bank); the maspsx "hang" scare was a
  truncated experimental-file artifact (real pipeline builds in ~1s)
- metrics: distinct-code 39.1% (50,572 unique fns), instr-weighted 62.9%
- NEXT: the ×134 automation (generate the per-overlay carve + template the reconciled body)
2026-07-12 16:37:37 -06:00
Drew T 025cc03f69 feat(phase-26): tiny-band mechanical harvest — 17,975 member-matches, R22 136/136
- family_sweep --hseq --band tiny: 180 tiny matched-exemplar families ->
  17,975 member-matches BANKED / 5,617 gate-rejected (h_seq-collision
  false-templates — the whole-binary byte-gate refused every one; G3/P9),
  266 overlay .c files touched (~76% bank rate)
- R22 clean-fleet (make clean + extract-all-136 + check-all) -> 136/136,
  0 failed; dedup-check 1813 validated / 0 failed; 0 NON_MATCHING in any
  default build (G4)
- metrics: distinct-code 35.2 -> 39.1% (50,571/84,996 unique fns),
  instr-weighted 60.9 -> 62.9%. mechanical size-bands (substantial/mid/tiny)
  now harvested; remaining levers = the two harvest gaps (§8 jtbl-rodata,
  reconcile fn-ptr-extern)
- regen docs/family-hseq.md + docs/progress.fleet.md; CURRENT_PHASE.md log
2026-07-12 15:38:40 -06:00
Drew T 3074ceb485 feat(phase-26): mid-band clean harvest — 6853 member-matches (320 distinct fns), R22 136/136
- plain --hseq --band mid: 6,853 whole-binary member-matches banked = 320 distinct engine functions
  templated across the overlays (matched-exemplar families, plain templating; the ×N leverage of the
  h_seq per-location reframe). 3,098 type-heavy tail + 1,743 pinned + 873 no-ov077-body skipped/failed
  (expected — the reconcile/§8/pin tail, not machinery failures).
- R22 clean-fleet: make clean + extract-all-136 + check-all = 136 passed, 0 failed. 0 NON_MATCHING.
- METRICS: distinct-code 31.1->35.2% (+6,437 byte-distinct classes), instr-weighted 58.9->60.9%.
  Phase-26 total: distinct 30.3->35.2%, instr 58.2->60.9%.
- session-2 checkpoint (CURRENT_PHASE 'SESSION-2 CHECKPOINT') current: next = tiny band, then the §8
  jtbl tooling (Drew-approved) + reconcile fn-ptr fix. Fable5 held until re-approval.
2026-07-12 11:28:48 -06:00
Drew T b94e417edf docs(phase-26): session-2 checkpoint — full resume context (mid-harvest in flight, §8 approved, 2 gaps)
Comprehensive CURRENT_PHASE handoff for a fresh session: committed baseline commit:0528 (729 banks, R22
136/136); the h_seq engine + reconcile-raw tooling map; the mid-band harvest in flight (~2477+ banked,
uncommitted -> R22 + commit next); the two harvest gaps (§8 jtbl-rodata [Drew-approved] + reconcile
fn-ptr-extern); the Fable5 rtu_match-vs-whole-binary finding (no more Fable5 until re-approval); and the
mechanical next-steps priority order. R30 knowledge capture before context handoff.
2026-07-12 10:22:31 -06:00
Drew T 62f9533024 feat(phase-26): +266 reconcile-class banks (2 no-jtbl cracks x133) + Fable5 batch-1 whole-binary findings
- +266 member-matches: func_8015CD20/func_8015C128 templated x133 via --reconcile-raw (each SHA-gated
  per-overlay vs config/check.<ov>.sha = byte-identical, G3). Full R22 deferred until func_80176218
  releases asm/ (established per-overlay-gate + deferred-R22 pattern, as the committed 463 which R22'd 136/136).
- family_sweep: --reconcile-raw now also covers draft-ov077 (unbanked) cracks (template from the RAW seed).
- P9 CORRECTION + decision-log 2026-07-12: the 2 Fable5 cracks rtu_match-MATCH but FAIL the whole-binary
  gate (both jr-functions; rtu_match masks relocs + excludes neutralized INCLUDE_ASM rodata, so it never
  verifies the §8 jtbl rodata). TWO harvest gaps: §8 jtbl-rodata (blocks all jr cracks) + reconcile
  data-extern (D_801891B8-class, blocks ~15/21 no-jtbl triage cracks). 6 no-jtbl reconcile-clean cracks
  bank whole-binary (729 members). rtu_match is NOT a sufficient arbiter for jr-functions.
2026-07-12 01:38:49 -06:00
Drew T 7ccf48f2f4 feat(phase-26): Task-8 reconcile wiring (§41c h_seq) + 463 reconcile-class banks
- BUILT the per-sibling reconcile: family_remap.remap_hseq_body (h_seq-remap a RAW crack draft: symbol +
  immediate + cross-address self-rename) + family_sweep.reconcile_remap_hseq + --reconcile-raw. Per sibling,
  remap the RAW crack then canon_sig_reconcile against that sibling's own TU (the h_seq port of the h_norm
  M2 path) — because a reconciled body is TU-specific and can't template plainly (validation: 0/4).
- HARVEST: the 4 triage isolation-cracks (func_80155800/80167540/801506A4/8016A73C) templated 463/0 x~133
  via --reconcile-raw (0 failures). Metrics: instr 58.5->58.9%, distinct 30.9->31.1%.
- each overlay SHA-gated by harvest_verify vs config/check.<ov>.sha (byte-identical = the match def, G3).
  FULL R22 clean-fleet DEFERRED until the concurrent Fable5 crack agents release asm/ (their m2c needs it);
  R22 fleet-confirm to follow post-window.
- cookbook §40c (the h_seq per-sibling reconcile technique, R30).
2026-07-12 00:03:47 -06:00
Drew T 8dbde10752 feat(phase-26): Task-8 validation slice — reconcile→bank proven (4 cracks into ov077, R22 136/136)
- pre-Fable5-window de-risk (Drew): validate reconcile→gate→template on the triage cracks before the window.
- reconcile→bank WORKS: raw 0/23 (§41 def-side wall) -> canon_sig_reconcile v3.2 -> 4/15 banked into ov077
  (func_801506A4/8016A73C/80167540/80155800), byte-identical, R22 clean-fleet 136/136.
- templating a RECONCILED body x133 FAILS 0/4: reconciled bodies are ov077-TU-specific (canonical-sig casts
  + collision-renames) -> need per-sibling re-reconcile (§41c). Task-8 prerequisite: port the h_norm
  --reconcile M2 path into hseq_sweep so the type-using families (triage cracks + the 61 Fable5 cores) can
  template x134. PURE families already template plainly (Task 5: 399 banked).
- decision-log 2026-07-11: the slice paid for itself — found the templating gap BEFORE spending the window.
  Paused before building the wiring per Drew.
2026-07-11 23:40:01 -06:00
Drew T 5f07d099e6 docs(phase-26): finalize Task-6 triage (119 agents) + banking caveat; pause before Task 8
- full triage complete: cheap 29 (23 closeness-0 isolation-MATCH) / permuter 29 / fable5 61 (1.71M ins).
- attempted to bank the 23 cracked wins into ov077 -> 0/23: the match_one isolation-MATCHes are genuine
  function matches but carry standalone struct/scalar typedefs + Ghidra-typed sigs that conflict with the
  real ov077 TU (redefinition of struct Obj / conflicting types) = the §41 def-side wall. Banking needs
  the Task-8 --reconcile / canon_sig_reconcile pass (not run — paused before Task 8 per Drew).
- docs/phase26-triage.md carries the crack curriculum + the caveat; seeds in .run/phase26-seeds/.
- src pristine, ov077 byte-identical.
2026-07-11 23:12:53 -06:00
Drew T 80536efbea feat(phase-26): task 6 — Ultracode triage of 119 draftable substantial families
- .run/wf_triage*.js Workflow: per-family m2c draft (+§8 jtbl handling) -> match_one reloc-masked
  closeness -> classify cheap/permuter/fable5 + §31/§45 lever + seed to .run/phase26-seeds/.
- 119 families triaged (2.38M templatable ins): cheap 22 (324k ins, 20 already isolation-MATCH) /
  permuter 23 (340k) / fable5 74 (1.72M). The cheap-Opus triage flywheel cracked 20 families to
  closeness-0 as a side effect (each templates x134) -> Task-8 gate+template candidates.
- docs/phase26-triage.md = the crack curriculum (already-cracked seeds + top-30 Fable5 targets +
  permuter band). Seeds in .run/phase26-seeds/. match_one closeness is an INDICATOR; the whole-binary
  byte-gate (Task 8) is the sole arbiter (G3/P9).
2026-07-11 23:00:26 -06:00
Drew T d05203b9a0 feat(phase-26): task 5 — h_seq zero-crack GO/NO-GO = GO; 532 members banked (R22 136/136)
- remap_hseq.gather_externs: carry file-scope externs for body-referenced symbols (extract_unit only
  grabbed adjacent ones) — the decl class that blocked per-location bodies indexing a global. func_8015F118
  gate-fail -> BYTE-IDENTICAL; the 3 tracker-miss PURE families then bank 133/133 each.
- ran the real whole-binary byte-gate on the 29 substantial matched-exemplar families:
  532 members BANKED (byte-gated). Per-family: 3 tracker-miss PURE (0x8015d5e8/0x8015f118/0x801407f4)
  bank 100% x133 = 399 byte-perfect (the tracker-fix free win); 1 cross-addr family 50%; 9 zero-bank
  families are type-using (Work8016/Prim/...) -> the existing --reconcile/type-lift follow-on (Task 8);
  16 families pinned -> Task 7 pin-free re-crack.
- VERDICT: the h_seq machinery (tracker + imm + cross-address + extern-carry) is byte-proven 100% correct
  on clean families. GO to scale.
- R22 clean-fleet: make clean + extract-all-136 + check-all = 136 passed, 0 failed. 0 NON_MATCHING (G4).
  Metrics: distinct-code 30.3->30.9% (+375 fns), instr-weighted 58.2->58.5%.
- decision-log 2026-07-11 (R31: stratify a mechanical-harvest rate by family/class before judging it).
2026-07-11 22:15:30 -06:00
Drew T 3ce81cbf56 feat(phase-26): task 4 — family_sweep --hseq mode (cross-address + imm templating)
- additive hseq_sweep(): consumes .run/family_hseq.json, templates each matched-exemplar family's
  still-stubbed members via remap_hseq (reloc remap + imm subst + cross-address self-rename), stages
  by (overlay,split), gates each group once via harvest_verify. h_norm path byte-UNTOUCHED (new --hseq
  branch routes before it). Unique per-group verified-out fixes a latent multi-split overwrite.
- member pre-filter = remap_hseq refusal (STRUCT regalloc-drift / unresolved immediates).
- static pin guard: skip families whose matched exemplar carries a hard-reg pin __asm__("$N") — the
  x1-only cracks (decision-log 2026-07-11: func_8016DF5C/8013D9B0/80133AB0) that cc1-crash sibling TUs;
  they route to Task 7 pin-free re-crack instead of bisection-storming the gate.
- flags: --hseq[=manifest] --band --min-members --stage-only --only.
- VERIFIED (--stage-only): substantial band 29 families -> 1507 clean members staged / 267 groups,
  1643 correctly skipped pinned (16/29 pinned); h_norm --only path intact; drafts byte-correct by
  construction (V3 0-DIFF), type-using -> whole-TU gate (Task 5).
2026-07-11 21:26:00 -06:00
Drew T 462734edb2 feat(phase-26): task 3 — T2a immediate engine (Tier 1) + T2b cross-address, 0-DIFF verified
- family_remap: imm_value (signed/unsigned field-aware) + imm_map_tier1 (diff-driven literal swap:
  asm-side ambiguity guard defers values that also appear at a non-differing position; C-literal
  swap preserves sign + hex-case) + remap_hseq (symbol remap + imm subst + cross-address self-rename
  in one pass; refuses STRUCT/unresolved members -> caller skips, byte-gate would reject anyway).
- Tier 2 (targeted probe) intentionally DEFERRED — build-if-needed per Task-5 measurement; only ~8
  low-weight IMM families, and the whole-binary byte-gate arbitrates (plan: simpler where gate arbitrates).
- VERIFIED (.run/v3_imm.py, match_one reloc-masked so immediates are checked exactly): 0 DIFF on every
  compilable derived draft — 3 IMM + 2 PURE cross-address MATCH; 4 remap-fails were correctly-deferred
  asm-ambiguous values (not wrong output); 45 compile-fail are match_one isolation limits -> whole-TU
  gate in Task 5. Cookbook §40b imm-engine forward-ref fixed.
2026-07-11 21:14:09 -06:00
Drew T faedd103e4 feat(phase-26): task 2 — tools/family_hseq.py full-frontier survey + shared word-diff classifier
- family_remap.py: shared classifier (stream_words / reloc_indices / reg_fields / classify_member)
  — per-instruction diff class RELOC/IMM/IMM_SA/STRUCT, register-drift aware (h_seq ignores registers,
  so regalloc-drift members are STRUCT-excluded, not templatable). Reused by T1/T2a/T3.
- tools/family_hseq.py: cluster ALL unmatched overlay instances by h_seq; per family classify every
  member vs exemplar (PURE/IMM/MIXED), tag per-location/cross-address/scattered, matched-sibling count,
  has_mid_jr (§8), exemplar pick (matched-ov077 > matched > draft-ov077 > modal), size band.
  -> .run/family_hseq.json + docs/family-hseq.md (committed digest).
- VERIFIED: fleet 74.8/58.2/30.3 (= PhaseEnd_25 + progress.py exact); tail cross-check 663 families /
  186 substantial / 1.847M ins (exact); classification vs Plan-agent PURE-same 62 & IMM 8 exact;
  890x134/562x134 PURE per-location + 952x113 #addr21 IMM confirmed. Full frontier: 581 substantial
  families / 3.22M templatable ins; 345 matched-sibling PURE/IMM families / 1.14M ins = V2/V3 corpus.
2026-07-11 21:05:29 -06:00