mirror of
https://github.com/Druthulu/BFM-decomp
synced 2026-09-28 06:49:47 -04:00
8ffcf9646eb65ed70a42a8d666daa244f8cfb99f
603 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Drew T
|
8ffcf9646e |
feat(phase-26a): A3f — 33 functions banked that the project had written off as compiler walls
The payoff of A3e, byte-verified. These 33 sat in the backlog at closeness==0 -- match_one said
their bodies were BYTE-EXACT -- and the whole-binary gate rejected them, so they were logged as
`near`/`failed`, i.e. AS MATCHING PROBLEMS, and filed as intrinsic compiler residuals.
They were not hard. They were UNREACHABLE. gate_stage passed `--src src/<ov>/<ov>.c`
unconditionally, which restricts the byte-gate to ONE translation unit -- and every one of these
functions has its stub in a SPLIT TU. Look at where they landed:
src/ov_SC01_077/ov_SC01_077_a.c
src/ov_SC01_077/ov_SC01_077_after.c
src/ov_SC01_077/ov_SC01_077_jr_8012ACE0.c
src/ov_SC01_077/ov_SC01_077_jr_8015AE2C.c
src/ov_SC01_077/ov_SC01_077_jr_8016AB6C.c
src/ov_SC01_077/ov_SC01_077_jr_801734BC.c
src/ov_SC01_077/ov_SC01_077_jr_80178D40.c
src/ov_SC01_077/ov_SC01_077_jr_80182268.c
8 files. SEVEN of them are _jr_/_a/_after carves. NOT ONE is the main .c -- the only file the gate
was ever allowed to look at.
gate: 63 drafts -> banked 33, near 30, FAILED 0 (--no-propagate; the gate and the propagation
are different jobs, and letting an experiment tow an unbounded fleet-wide propagation is
what broke the tree an hour ago)
R22 CLEAN-FLEET: make clean + extract-all + check-all -> 136 passed, 0 failed of 136
dedup-check: 1823 validated, 0 failed | C1 coverage 224933/224933
METRICS, ×1, HONESTLY (no propagation yet -- the multiplier is still ahead):
functions byte-identical 284,526 -> 284,559 (+33)
instructions 8,470,381 -> 8,471,912 (+1,531)
fn-count % 82.79% -> 82.80%
instr-weighted % 66.7% -> 66.7% (flat: ×1 banks do not move the headline)
WHAT THIS MEASURES, beyond the 33: the backlog holds 1,588 entries at closeness==0. 1,215 have been
banked since by other paths. 373 ARE STILL OPEN STUBS WITH BYTE-EXACT BODIES. 63 of them were in
ov_SC01_077 and 33 banked -- a 52% rate on functions the ledger calls unrecoverable. The other 310
are spread across the remaining overlays: same class, same fix, not yet attempted.
Cookbook §51g LAW 11 -- a fix is not landed until its caller stops overriding it. And the reason
this hid for 26 phases, which belongs in the posterity doc: A TOOL THAT CANNOT BANK A FUNCTION IS
INDISTINGUISHABLE, IN EVERY LOG THIS PROJECT KEEPS, FROM A FUNCTION THAT CANNOT BE BANKED.
|
||
|
Drew T
|
b89fcc2edc |
fix(phase-26a): A3e — gate_stage pinned the byte-gate back to 4.9%, OF A3'S OWN FIX
THE WORST DEFECT IN THE AUDIT IS NOT IN A SCANNER. It is one default argument in the CALLER of a
scanner we had already fixed.
# tools/gate_stage.py:315
summary = run_gate(a.drafts, binary=b, src=a.src or f"src/{b}/{b}.c", ...) # ALWAYS the main .c
`src` RESTRICTS the byte-gate to ONE translation unit, and _gate1 does `if src: cmd += ["--src", src]`
-- always truthy. A3 had just taught harvest_verify to DERIVE each draft's home TU *when --src is
omitted*, lifting the byte-gate's reach from 4.9% to 100%. gate_stage NEVER OMITS IT. The fix was
neutralised by its own caller's default, and the PRIMARY BANKING PATH -- every wave, the grinder, the
orchestrator, bulk_harvest -- remained structurally unable to bank 250 of ov_SC01_077's 263 stubs.
WHY IT SURVIVED 26 PHASES: harvest_verify cannot splice a draft whose stub is not in the TU it was
pointed at, so the draft never verifies -- and is then logged as near/failed, i.e. AS A MATCHING
PROBLEM. The wave reports a poor close-rate; the function goes to the backlog as a compiler residual.
A tool that CANNOT bank a function is indistinguishable, in every log this project keeps,
from a function that CANNOT BE banked.
PROOF, same draft / same gate / same second: gate_stage rejected func_80129C40; harvest_verify run
directly (no --src) VERIFIED it byte-identical and banked it.
AND A COUNTING BUG THAT HID THE HIDING (gate_stage:261): when match_one says MATCH but the whole-binary
gate rejects, the record is logged status="near" and THE COUNTER IS NEVER INCREMENTED. A 63-draft run
printed `banked 0, near 0, failed 0` -- three zeros that do not sum to 63 -- for phases. Nobody ever
added them up. (The number was not wrong. It was ABSENT.)
ALSO FIXED, sig_unify (the same disease, one level down): it SILENTLY DROPPED 190 of 196 drafts (97%).
`cur_stubs` was read from the main .c (13 of 263 stubs), so any draft whose stub lives in a _jr_ carve
hit `if fn not in cur_stubs: continue` -- dropped BEFORE THE WRITE: never copied to --out, never gated,
never logged, while the summary printed "drafts unified: 6" and read like success. THIS IS GATE_STAGE'S
STAGE-2 RECOVERY -- the pass whose whole job is to rescue the stage-1 failures -- and it has been a
no-op for nearly every draft it was meant to save. Now: TU derived per draft (corpus.stubs), canon
derived from cdecl.tu_scope (cpp -- macro-injected decls finally visible), and _keep() so an
already-acceptable decl is left alone (the §19 "sig_unify regresses canonical drafts" failure mode).
Reach: 6 -> 196 drafts; callee-externs rewritten 2 -> 90; own def-sig 2 -> 86.
MEASURED, all three consumers migrated (196 never-banked drafts):
near 5 -> 116 failed 190 -> 17
=> 173 of 190 "failures" were PLUMBING, not codegen: now compiling and SCORED instead of invisible.
THE PRIZE (measured, not claimed): the backlog holds 1,588 entries at closeness==0 -- body byte-exact
per match_one, whole-binary gate rejected. 1,215 have been banked since by other paths. 373 ARE STILL
OPEN STUBS WHOSE BODIES ARE ALREADY BYTE-EXACT, sitting in a ledger that calls them unrecoverable.
⚠ THE HARVEST ITSELF IS NOT IN THIS COMMIT, AND IS NOT CLAIMED (P9). Gating the 63 ov_SC01_077 ones
dragged `dedup_propagate --auto-from --recover` behind it; it ran >1h and hit its timeout -- its
first-ever run over the FULL corpus (A6/A7 unblocked the 407 files it could never see). It MUTATES THE
TREE BEFORE IT GATES, so the kill left 859 files + engine_core.h (+544 lines) written and UN-GATED with
the registry never updated. R22 on that tree: 44 passed / 92 FAILED -> `git checkout -- src/ config/`,
fleet restored to 136/136. Nothing lost (H4: the tree was clean, so the revert was one command).
Two real lessons, recorded: dedup_propagate is NOT crash-safe and must never run under a timeout it can
hit; and a 63-draft experiment must not drag an unbounded fleet-wide propagation behind it.
R22 clean-fleet after revert: 136 passed, 0 failed of 136. src/ and config/ clean.
cookbook §51g LAW 11: A FIX IS NOT LANDED UNTIL ITS CALLER STOPS OVERRIDING IT. After fixing a
scanner, grep every call site and ask whether a caller's default re-disables it. An audit that stops
at the callee is half an audit.
|
||
|
Drew T
|
4aae5e7589 |
fix(phase-26a): A3d — retire the fleet-majority oracle: it was WRONG for the TU 16% of the time, on both banking paths
R33 applied to the worst finding in the audit: this oracle was not fixed, it was RETIRED.
reconcile_decls asks "what does the FLEET call this symbol?"
C asks "what does THIS TRANSLATION UNIT declare?"
The engine is loosely typed -- the same address is legitimately declared with incompatible types in
different overlays -- so a single fleet-wide answer is WRONG FOR SOME TU BY CONSTRUCTION. And it is
worse than a silent skip: it writes an ACTIVELY WRONG declaration into the draft, which then
collides with the very TU it was meant to conform to.
MEASURED across ov_SC01_077's 12 TUs, against what cpp says each TU really declares:
the fleet oracle AGREES with the TU ................ 2883
the fleet oracle CONFLICTS with it (cc1 REJECTS) .. 548 <- 16%
the TU declares it, the oracle has NO answer ...... 357
and it was LIVE ON BOTH BANKING PATHS:
* gate_stage -- rewrote 60 of 196 drafts in the current batch
* jtbl_family_bank -- EVERY SIBLING of the ×134 family sweep, the project's economic engine.
A poisoned decl means that sibling silently does not bank, and the loss is invisible: the sweep
simply reports a smaller number. The irony is exact -- that function's own docstring already
knew the conflicting symbols are PER-OVERLAY, which is precisely why a FLEET oracle could never
have been right.
reconcile_tu.py (written in Phase 26 but NEVER WIRED) now supersedes it, rebuilt on cdecl:
* ask cpp what the TU declares (macro-injected DEFINE_func_* externs included -- a raw scan
cannot see them, §8c / §51g LAW 7);
* ask cc1 whether the draft's decl can coexist (cdecl.compatible, validated against the real
gcc-2.7.2 front end on 1,485 live pairs -- NOT the C standard, NOT modern gcc; §51g LAW 9);
* NOT declared -> leave the draft alone (its extern types are load-bearing: %lo-folding, access
width, alignment); compatible -> nothing; CONFLICTING -> the TU wins + cast at every USE so the
draft's intended access survives byte-for-byte;
* derives WHICH TU from corpus.stubs() rather than a hand-passed --src-file (§51g LAW 10).
* handles the fn-ptr kind NATIVELY -- which is why it supersedes rather than patches: teaching
reconcile_decls' parser to see `extern void (*D_x[])(void);` would have ARMED its fn-ptr-blind
data_access_subs to rewrite a call-through `D_x[i]()` into `((u8 *)D_x)[i]()`. Fixing the regex
would have detonated a dormant bug.
AND THE NULL RESULT, AGAIN, REPORTED AS SUCH (P9/R14): on the 196 never-banked historical drafts the
new oracle banks EXACTLY AS MANY AS THE OLD ONE -- zero. That tail fails on CODEGEN, not on decl
plumbing. The two disagree on 45 of 196 drafts and the outcome does not move. This is a CORRECTNESS
fix (548 wrong declarations removed from two live pipelines, protecting all FUTURE drafts and every
future family sweep), not a banking win, and it is not being sold as one. Three nulls in one session.
reconcile_decls.py is kept as EVIDENCE, marked RETIRED, with no live caller.
R22 clean-fleet: make clean + extract-all + check-all -> 136 passed, 0 failed of 136
src/ untouched (0 changes) reconcile_tu: 0 coverage defects over 196 drafts
NOTE: the family-sweep path gets its real exercise at Task 8 -- watch the per-sibling bank rate.
|
||
|
Drew T
|
f4502f11bb |
fix(phase-26a): A3c — the recovery passes were reconciling 95% of drafts against the WRONG TU
FIRST CONSUMER MIGRATION onto the cdecl oracle — and the compiler taught me two things I had
wrong, one of which reopens a wall that has been closed since Phase 15.
1. cdecl.compatible() — "will cc1 accept these two declarations of one name?"
The predicate four tools each half-implement and get wrong: norm_sig / _norm_type collapse the
int family to ONE token, so a SIGNEDNESS change reads as "already compatible" and gets no
rewrite -- while cc1 REJECTS that redeclaration. Right about codegen, wrong about the front end,
which never reaches codegen.
2. THE ADJUDICATOR MUST BE THE COMPILER THAT COMPILES YOUR CODE (cookbook §51g LAW 9).
I wrote the rules from the C standard, then let a compiler judge. It contradicted me -- and then
the RIGHT compiler contradicted the first one. Three different answers:
declarations in one TU | standard | modern gcc | gcc-2.7.2 cc1
typedef int X; twice | error | ACCEPTS | ERROR
extern u16 X; + volatile u16 X| error | error | ACCEPTS
void X(s16); then void X(); | error | error | ACCEPTS
void X(); then void X(s16)| error | error | ERROR
--compat now adjudicates with tools/bin/gcc-2.7.2-psx/cc1, the front end that actually
arbitrates the build: 1,485/1,485 live corpus pairs agree, 0 disagree, 0 skipped.
3. THE PRIZE: the Phase-15 narrow-param wall rests on a false premise.
The no-prototype rule is ORDER-DEPENDENT. `void X(s16); void X();` COMPILES; only the reverse
fails. Phase 15 closed "the 159 arity/narrow-param conflicts" as "no clean deterministic fix --
it is simply C's default-promotion rule". cc1 does not enforce that rule in the direction the
wall assumed. Four three-line probes, 90 seconds, zero tokens. -> A10 RE-TEST TARGET.
Probe the compiler for FACTS; read its source only for LEVERS; byte-validate both. (We read
gcc-papermario for five phases believing it was 2.7.2. It was 2.8.1.)
4. THE MIGRATION: cast_call_sites canonicalized 95.1% of drafts against a TU that would never
compile them. `--src-file` is an OPTIONAL HAND-PASSED flag defaulting to src/<ov>/<ov>.c, and no
caller knows about the Phase-26 _jr_<ADDR> carves: ov_SC01_077 has 263 open stubs across 12 TUs
and only 13 are in the main .c -- while harvest_verify (A3) correctly splices into the real one.
Now DERIVED from corpus.stubs() (the INCLUDE_ASM line is self-describing), with the canonical map
derived from cdecl.tu_scope() (cpp -- so macro-injected DEFINE_func_* decls are finally visible).
Callee-conflict repair reach: 8 -> 58 of 196 drafts (7x).
5. AND THE NULL RESULT, REPORTED AS SUCH (P9/R14). Those 58 banked ZERO functions. The historical
draft tail fails on CODEGEN, not plumbing -- func_801387B8, which the audit blames on a single
unparsed `[4]`, is really 67/100 instructions off with a $s0/$s1 swap (that claim does not
reproduce on today's tree). The real gain is narrower and still worth having: 52 drafts moved
from "won't compile" to "compiles, N instructions off" -- from an INVISIBLE failure that reads as
a compiler wall into a SCORED near-miss the permuter and the §47/§48 dials can act on. That is
the audit's thesis, not a bank. THREE times in one session a confirmed mechanism produced a null
consequence.
Also: my own new audit printed "ALL ORACLES GREEN" while silently skipping 100% of its corpus (a
missing -Isrc). The exact bug class, in the tool written to hunt it. An unadjudicable check is not
a passed check.
R22 clean-fleet: make clean + extract-all + check-all -> 136 passed, 0 failed of 136
src/ untouched (0 changes) make audit-cdecl: green --compat: 1485/1485
NEXT: sig_unify + reconcile_decls carry the SAME wrong-TU bug (same --src-file flag).
|
||
|
Drew T
|
f9742cf9c0 |
feat(phase-26a): A3b — cdecl.py, THE C-declaration oracle: one grammar, fifteen deleted models
Fifteen tools each carried their own regex model of "what is a C declaration", and they
disagreed — two tools in ONE pipeline disagree today about whether `extern s32 D_a, D_b;`
is a declaration at all. All fifteen shared one character class,
extern\s+([A-Za-z_][\w\s\*]*?\bD_[0-9A-Fa-f]+\s*(?:\[\s*\])?)\s*;
which cannot hold '(', ',', or a non-empty [N] — so three whole shapes were invisible to
every one of them: fn-ptr/jump-table arrays, sized arrays (one unparsed `[4]` has blocked
func_801387B8 in 134 TUs), and multi-declarators (the WHOLE line dropped, not just #2..N).
REJECTED the audit's own prescription (a shape-aware alternation per tool, ~15 coordinated
regex edits) on R33 grounds: fifteen hand-maintained models are exactly what diverged, and
an alternation only ever covers the shapes somebody remembered. The thing being scanned HAS
A GRAMMAR. C's declarator grammar is small, closed and TOTAL — it describes fn-ptr arrays,
sized/2-D arrays, multi-declarators, fn-ptr params and K&R identifier-lists without being
told they exist. ~250 lines of recursive descent: LESS code than the regexes it deletes, and
exhaustive by construction rather than by memory. (decision-log 2026-07-14.)
Two statement paths, because the inputs genuinely differ:
* tu_statements() - a TU's file scope, derived from cpp. A decl inside a DEFINE_func_*
macro body declares NOTHING until the macro is invoked (the §8c law);
a raw scan is wrong in both directions. cpp answers it exactly, in
54 ms/TU (~20 s for the fleet, cacheable).
* split_statements() - span-preserving raw split, for drafts (which get rewritten).
THREE ORACLES, whole corpus — a measurement, not a belief:
* coverage 2,952,246 depth-0 statements -> 2,731,521 declarators, 0 PARSER DEFECTS
* the real gcc 50,405 distinct declarations compiled beside this parser's reconstruction
of each one -> 0 REJECTED
* differential 0 file-scope symbols the incumbents see that cdecl misses; 26 in
engine_core.h they cannot see; 6 they wrongly promote from BLOCK scope
Two ideas worth keeping (cookbook §51g, LAWS 4-8):
* THE CANDIDATE SET IS DERIVED TOO (R33 applied to R32). At file scope C admits nothing but
declarations, so R32's over-approximating detector is *every depth-0 statement* — supplied
by the grammar, with no hand-maintained candidate regex to rot.
* GCC ADJUDICATES MY OWN COVERAGE GAP. Deciding for myself which failures "don't count" is
grading my own homework — the habit that wrote the fifteen bugs. A statement gcc ALSO
rejects is not C (my rejection is correct, the INPUT is corrupt); one gcc ACCEPTS and I do
not is MY defect. All 33 residual: NOT-C, all dead .run/drafts* scratch, none in src/.
NEW findings (docs/tooling-audit.md):
* reconcile_decls.DATA_DECL_LINE_RE finds ZERO decls in engine_core.h — it is line-anchored
and every decl there ends in a '\'. Its "authoritative tier" has ALWAYS been empty.
* gen_harvest_targets + sig_unify count BLOCK-SCOPE externs (6, byte-proven inside a macro's
function body) as file-scope canonicals — the §8d `conflicting types` confusion.
* tu_ambient's func regex ([^()]* params) drops ANY callee with a fn-ptr parameter.
* R14 near-miss: 33 drafts contain `extern if ((func_80029178(0x119) & 0xFF) != 0);`, written
by a RECOVERY TOOL — but the source bug was already fixed in Phase 19 (0 garbage / 300 sigs
today). Mechanism confirmed, consequence nil. Note what it cost while live: a draft that
cannot compile fails the byte-gate and reads downstream as an INTRINSIC COMPILER WALL.
Bugs the oracles caught in ME (and would otherwise have shipped): `extern s32 (*D_801274D0)(s32);`
parsed the BASE TYPE as the name; a K&R declaration-list flushes as SEVERAL spans, so the body
attached to the wrong one and leaked the K&R parameter names into file scope as fake globals.
SCOPE, deliberate: NO consumer is migrated here, so this cannot move a byte. The audit warns
that making the parser see more ARMS dormant transforms (reconcile_decls.data_access_subs would
mangle `D_1[i]()` -> `((u8 *)D_1)[i]()` the moment fn-ptr decls become visible to it). Migration
is one tool at a time, each byte-gated.
R22 clean-fleet: make clean + extract-all + check-all -> 136 passed, 0 failed of 136
make audit-corpus: 0 PHANTOM + 0 TRUNCATED make audit-cdecl: ALL ORACLES GREEN (new gate)
|
||
|
Drew T
|
c7772bc452 |
docs(phase-26a): SESSION-9 CLOSE — cookbook §51 (the tooling-integrity laws) + handoff
R30/R16: the context-dependent artifacts, written while the context is live.
cookbook §51 — the SILENT SKIP: the bug class, why the byte-gate cannot see it, the
over-approximating-detector method, and FOUR LAWS:
1. Derive, don't re-derive — the best outcome is a DELETED SCANNER (28 findings -> one
derived oracle + ~10 deleted scanners). A derived fact cannot rot; a hand-maintained
copy of it is a liability that grows with every structural change.
2. Assert your COVERAGE, not merely your correctness. *** A LOUD FAILURE THAT NOBODY
COUNTS IS EXACTLY AS INVISIBLE AS A SILENT ONE *** — build_engine_types printed
'[overlap] handle manually' every single time for four phases while dead on 81% of its
own corpus. This CORRECTS the first draft of R32 ('fail loud'), which was not enough.
3. When an oracle is structurally blind to a class of error, add a SECOND ORACLE THAT CAN
DISAGREE WITH IT — not a better assertion inside it. We had two all along and never made
them argue. (And scope the comparison to where the second oracle is genuinely independent:
the same check run outside its domain reports 914 slices when the truth is 193.)
4. A rule that needs a human to remember it is not a gate. Make it structural.
+ the FALSE-WALL PIPELINE (a silent skip -> a wasted draft -> a backlog 'matching failure'
-> reserved_walls() PERMANENTLY blacklists a function that was never attempted), and a
checklist for any new corpus-scanning tool.
CURRENT_PHASE: session-9 handoff — what is done, what remains (each with its spec on disk),
and the R32-corrected / R33 / R34-new rule candidates for P10 ratification.
|
||
|
Drew T
|
2086b15b48 |
fix(phase-26a): A8 — jr_isolate_all silently dropped 683 prototypes: a LATENT BYTE-CHANGER
_file_scope_decls() hoists a region's file-scope decls into the carried layer. Its _SAFE_TYPE guard
only ever whitelisted BUILTIN base types — so a decl naming a carried file-local type was recognised
by _HOIST_RE as hoistable and then SILENTLY DROPPED.
THE COMMENT ON _SAFE_TYPE DESCRIBED A FIX THAT WAS NEVER APPLIED TO THE CODE. Verbatim: "one naming a
FILE-LOCAL type is only safe once that type is carried too — which file_scope_types() now does, so such
decls ride along after their typedef." The predicate never implemented it.
MEASURED (audit: 4,040 fleet-wide; independently re-measured here over 4 overlays: 189 drops, 32 of
them function PROTOTYPES):
* The 3,357 dropped DATA externs are LOUD — an undeclared identifier is a compile error.
* The 683 dropped function PROTOTYPES are NOT. In C89 an undeclared function is implicitly `int f()`,
so the TU still COMPILES — with the wrong return type and lost pointer-ness. And this project has
BYTE-PROVEN that the return type drives codegen (cookbook: "schedule — delay-slot fill via void
return type"; ov_SC01_077_after.c carries an `extern int`->`extern void` flip described as
byte-neutral precisely because the return type moves the delay slot).
=> A DROPPED PROTOTYPE IS A SILENT BYTE-CHANGER, armed to fire on the next carve.
Today's split is green only because the source redundantly re-declares externs per fn-group, so most
items happen to carry their own decl. That is luck, not design.
Two of the rejected base types were not even file-local: `uint` (139 drops) and `code_fn` (21) are
DEFINED IN src/shared/engine_types.h, which engine_core.h pulls into every region — the predicate was
rejecting INCLUDE-PROVIDED types it had no reason to reject. `volatile` (3) fell off because the
qualifier group had `const` but not `volatile`.
FIX: implement what the comment promised. A decl is hoistable if its base type is a builtin, OR is
carried by this layer's own file_scope_types, OR is provided by the shared headers (108 type names
parsed from engine_types.h + common.h — including FN-PTR typedefs, whose name sits inside the parens
and which every name-before-';' pattern misses; those were exactly the 5 residual drops).
COVERAGE ASSERTION (R32): a line _HOIST_RE recognises as hoistable but that cannot be placed is now a
HARD FAILURE with the base-type histogram printed, not a silent no-op. Verified safe: 0 residual drops
across 5 overlays. This one check would have surfaced all 4,040 the day the first split shipped.
No build impact (the isolator runs only when carving); --dry-run clean; tree unchanged.
|
||
|
Drew T
|
82d79e7a32 |
fix(phase-26a): A6/A7 — the family engine could not see half its corpus; 17 fns banked x134 free
R22: check-all 136 PASSED / 0 FAILED. dedup-check 1823 validated / 0 failed (C1 coverage 224,933/224,933).
Fleet instr-weighted 66.5% -> 66.7%.
=== dedup_propagate: it was blind to HALF the corpus ===
overlay_files() used a hardcoded suffix allowlist ("_a","_o0","_o0b","_after") that predated the
Phase-26 jr carves -> 404 of the fleet's 811 overlay .c. The 407-file gap held 36,135 INCLUDE_ASM stubs
and ~32,000 inline defs, and overlay_files gates ALL of dedup_propagate (source_text / find_site /
apply_plan / struct_check / reconcile_caller_extern). Now a GLOB — never an allowlist, because the NEXT
split family would re-open it. The asm_subdir is always the file stem, an invariant the old four entries
already satisfied.
find_site's def-detector required the signature line to END in ')' and the next non-blank line to START
with '{'. It therefore silently dropped THREE shapes: K&R definitions (`s32 f(arg0)` / `s32 arg0;` / `{`),
multi-line signatures, and single-line bodies. K&R is the project's house style for exactly the biggest,
highest-reach functions — func_8015AE2C (562 ins), func_80166994, func_80133CD4, func_8015A3C8 — and they
live in the _jr_* files overlay_files could not even open. Fixing either alone would have been useless:
the glob exposes the files, and find_site would still drop their biggest prizes. Both fixed together.
* The signature's closing paren is now found by a real paren-walk, not line.count() or split(')')[-1]:
a single-line body containing a call (`void f(int a){ g(a); }`) has balanced parens of its own, so
both shortcuts land on the WRONG paren and then misread the body's ';' as a prototype terminator.
* AGREEMENT ASSERTION (the audit's): find_site vs family_remap.extract_unit -> 701 agree / 0 disagree.
Negative controls hold (a prototype+call is rejected; a 1-line body with a call is a def).
=== THE HARVEST (free work, byte-gated) ===
--auto-from ov_SC01_077 now nominates what it could never see: 20 planned, 17 propagated x134, 3 dropped
as cross-overlay stragglers. 134 overlays rebuilt BYTE-IDENTICAL; 17 new dedup groups.
Includes ALL FOUR functions A1 caught the registry lying about (func_80128ED8 / 8012C098 / 8012C0EC /
8012C750): 0 stubs remaining, real shared macros. THE LOOP CLOSES — A1 found the lie, and THIS is the
bug that had made it true (3 of the 4 are defined in ov_SC01_077_jr_8012ACE0.c, which the allowlist could
not open, so the propagation never ran and dedup_integrate greenlit the result).
=== family_remap: 96 PHANTOM exemplars -> 0 ===
extract_unit globbed only src/<ov>/<ov>*.c, so a function matched via a SHARED body had no source form
and read as NOT MATCHED. 93-96 of 218 h_seq "matched" exemplars were phantom, carrying 2,157 candidate
members of which 1,834 are still-stubbed, PURE/IMM-clean, symbol_map-clean and unpinned — staged and
gated today, dropped before the first build then. It is now TOTAL over BOTH shared-body mechanisms:
(1) the DEFINE_func_<ADDR>() macro — reconstructed as the exact INVERSE of dedup_propagate.make_macro
(derived from the generator, not re-guessed from the text);
(2) a DIRECT definition in a shared header, #included per overlay — the whale (func_80144B9C, 770 ins,
-O0), which the registry explicitly records as "NOT a DEFINE_ macro".
CENSUS: 216 matched exemplars, 216 real, 0 PHANTOM.
symbol_map named the symbol by HOW IT WAS LOADED, not by WHAT IT IS: reloc_targets labels every lui/%lo
pair "data", and a FUNCTION's address taken via lui/%lo (an address-taken callback) is exactly that shape
(splat's own .s: %lo(func_8017E1D4), 7 occurrences). The map got a D_<ADDR> key while the C writes
func_<ADDR>, so the word-bounded substitution matched NOTHING and silently no-op'd — the sibling kept the
EXEMPLAR's function pointer and the loss was booked as a BYTE failure, indistinguishable from a compiler
wall. Now emits both keys (addresses are unique; the pass is simultaneous, so the extra key is free).
gather_externs was line-oriented, so a WRAPPED comma extern was invisible in both directions (the first
line has no ';', the continuation has no `extern`). ov_SC01_077.c:271-272 declares NINE symbols that way,
and the exemplar referencing them (func_8013D178) is a 133-member family — every sibling was staged with
NO declaration, failed to compile, and bisect-stormed its whole gate group. Now statement-oriented, and
an unresolved symbol is REPORTED, never silently dropped.
=== family_sweep.stub_map / build_engine_types ===
stub_map: func_-only -> a curated-name stub read as "already matched" -> phantom exemplar. Now corpus-derived.
build_engine_types hard-exited on 1,070 of 1,470 type-bearing overlay .c (73%; the audit measured 573/709
= 81% on its narrower set) because 1,929 TAGGED-struct typedefs tripped a guard whose own comment asserts
"our source has only ANONYMOUS-struct typedefs" — true in Phase 20, false since the harvest agents started
writing tagged structs. inject_capped_externs routes every type-bearing body HERE as the type-heavy tail's
ONLY sanctioned unblocker, so the tail's unblocker could not run on the corpus the tail lives in.
A contained def (the typedef's span encloses the body) is liftable — it just must not be counted twice;
only a PARTIAL overlap is malformed. Verified on a file that used to hard-exit: 5 tagged typedefs folded +
forward-declared, 46 types written, exit 0.
** AND THE SHARPEST LESSON IN THE AUDIT: this one was never silent. It printed "[overlap] ... handle
manually" every single time. But the message reads like a rare edge case rather than a four-fifths
coverage failure, so nobody ever COUNTED it. A loud failure that nobody counts is exactly as
invisible as a silent one. R32 must be "assert your coverage", not merely "fail loud". **
R14 self-catches, recorded because I hit both while fixing them: my first shared-header scan read a macro
body's `extern void f(void); \` as a DEFINITION (the trailing continuation means the line does not end in
';', so the decl guard never fired) — the exact bug fixed at commit:0552, reintroduced by me and caught only
because the whale resolved from the WRONG file. Column-0 anchoring fixes it by construction. And my
phantom census returned 0/0 twice because I guessed the manifest schema instead of reading it.
|
||
|
Drew T
|
a2a507d079 | docs(phase-26a): A4/A5 + the stale-object false-pass hole recorded | ||
|
Drew T
|
af2f40d153 |
fix(phase-26a): A4/A5 — 193 unmatchable slices dissolved; the closeness oracle stops lying
R22 CLEAN-FLEET: make clean -> extract 136 -> build 136 -> check-all = 136 PASSED, 0 FAILED.
make audit-corpus: 0 PHANTOM + 0 TRUNCATED (was 193).
=== A4: a CORPUS defect the byte-gate could never have caught ===
config/symbols.us.txt:981 declared `listCdBuffer = 0x80180000` — a correct Phase-3 name for MAIN's
LIST.CD RAM buffer. But that address is OUTSIDE main's image and INSIDE the overlay slot, and every
overlay's splat config stacks symbols.us.txt. High RAM is REUSED: an address that is a buffer to main
is live CODE to an overlay. So splat saw a symbol boundary mid-code and, across 97 of 134 overlays:
* CUT 97 REAL FUNCTIONS IN HALF (a head ending on a `lui`, no return), and
* INVENTED 96 PHANTOM ONES (a tail beginning by reading the assembler temp $at).
193 slices NOBODY COULD EVER MATCH — not "hard", not "a compiler wall": unmatchable by construction.
They sat in the harvest queue as ordinary work, so agents would burn on them forever and the failures
would be filed as intrinsic compiler residuals.
The phantom listCdBuffer.s in ov_SC01_005 literally begins:
lw $ra, 0x10($sp) / addiu $sp, $sp, 0x18 / jr $ra
splat cut a function immediately before its EPILOGUE and called the epilogue a function.
AND IT HAD ALREADY CONTAMINATED REAL WORK: in ov_SC03_031 the cut landed where the epilogue was
exactly `jr $ra; nop`, so the Phase-26 x134 sweep innocently BANKED the phantom as
`void listCdBuffer(void) {}` — byte-correct, gate-green, entirely fictitious — while leaving
func_8017FFC4 permanently unmatchable. Removed.
WHY NO GATE CAUGHT IT, AND WHY THAT IS THE POINT: INCLUDE_ASM pastes the two .s halves back VERBATIM
in original order, so the image is byte-identical either way. The byte-gate was green the whole time
and always would have been. It is a perfect CORRECTNESS oracle and a NULL COVERAGE oracle. No
assertion added INSIDE it could ever have found this. What found it was a SECOND, INDEPENDENT oracle:
tools/sig_image.py derives boundaries from the ORIGINAL bytes without splat, and DISAGREED with the
corpus (58,524/58,621 agreement with spimdisasm; correct on all 97 disagreements).
=> When one oracle is structurally blind to a class of error, the answer is not a better assertion
inside it. It is a SECOND ORACLE THAT CAN DISAGREE WITH IT. (`make audit-corpus` is now that.)
THE RULE (the mirror of R13/R15, never written down): a symbol whose address falls inside ANOTHER
binary's vram window must never enter that binary's symbol stack.
FIX: config/symbols.us.ram.txt — main-scoped symbols outside main's image — stacked ONLY by
config/splat.us.exe.yaml. Main keeps the name it needs (10 %hi / 11 %lo refs; 143dbb89 byte-identical);
the overlays never see it. Exactly one symbol was in scope fleet-wide; the resident window was clean.
AND A REAL FUNCTION THE ACCIDENT WAS HIDING: in ov_SC01_084 / ov_SC02_041 / ov_SC03_094 / ov_SC06_008
there IS a genuine function at 0x80180000 (111 / 35 / 28 / 74 ins), reachable ONLY via a fn-pointer
table (.word func_80180000) and never by `jal` — so splat cannot find it and needs the boundary
DECLARED. listCdBuffer had been supplying it by luck. Now declared honestly, per-overlay, in
config/symbols.<ov>.txt — exactly where R13/R15 says an overlay-scoped symbol belongs.
=== A5: the closeness oracle every crack agent trusts was lying on 155 functions ===
masked_diff._reloc_kind() knew 26/HI16/LO16. An over-approximating sweep of every reloc objdump emits
across all 3,367 build objects found FOUR: R_MIPS_26, HI16, LO16 — and R_MIPS_PC16 (211). PC16 fell
through to a FULL-WORD compare, but the object holds an UNRESOLVED PLACEHOLDER in the branch
displacement, so that compare can NEVER succeed.
DECISIVE TEST (derived from the invariant, not from reading the regex): INCLUDE_ASM pastes the
ORIGINAL asm, so for every stub diff_object_s() MUST be 0. Measured, coverage-asserted:
2,741 functions scored — old mask: 150 LIES; PC16 masked: 4 LIES.
(The 4 survivors are the separate length-delta defect.) A phantom non-zero sends an agent to grind at
a wall that is not there, and the wasted attempt is then booked as a MATCHING failure, feeding
reserved_walls() and PERMANENTLY BLACKLISTING a function that was never broken.
=== NEW FINDING (found by cutting the R22 corner): a STALE OBJECT CAN PRODUCE A FALSE PASS ===
`.o <- .s` is not a dependency make can see: assembly arrives via INCLUDE_ASM, expanded to a `.include`
consumed by maspsx/as AFTER cpp, while -MMD tracks headers only. Re-extract, build incrementally, and
make links a STALE object. This is not merely slow — INCLUDE_ASM pastes the ORIGINAL bytes, so a stale
object still yields the original image: SHA1 GOES GREEN while the split just changed is never exercised.
A broken config change can be "verified" by an incremental build. Live proof: 8 of 136 binaries linked
stale objects here; they failed LOUDLY ONLY BY LUCK (the dead symbol was an undefined reference) — a
merely-different-but-valid split would have gone green on all 136.
R22/H3 already legislate this, and I broke them. But a rule that needs a human to remember it is not a
gate. FIX: `extract` now invalidates the objects that include what it just rewrote (main's are top-level,
so -maxdepth 1 — verified it cannot clobber the other 1,605 objects). Structural, not advisory.
R14 self-catch, recorded: my first A5 test passed `fn=` to diff_object_s(), which takes two args; the
TypeError was swallowed by my own `except Exception: continue` and it reported 0 scored / 0 lies. I
wrote the exact bug I was auditing, inside the test for it. Caught only because 0 looked wrong. The
test now asserts its own coverage.
|
||
|
Drew T
|
9794b13ed2 |
fix(phase-26a): A3 — the endgame plan was 2.8x too big; the matched set is now DERIVED
docs/family-manifest.md is the document the whole Phase-25/26 structural-family endgame was planned
from. Its matched-set oracle asked ov_SC01_077 ALONE: matched := {h_exact of that one overlay's
non-stub fns} | dedup hashes. So a function ABSENT from that overlay — or stubbed there but matched
in the other 133 — came out "unmatched" and was ranked as live work.
advertised real (derived)
multi-member families 2,758 -> 1,495
"hidden leverage" 11.0 MB -> 3.9 MB
matched-free lever 235/5.9 MB -> 57/1.0 MB
7.1 MB of the advertised leverage was DEAD WORK. And because `instances` counted every overlay
carrying a function — including the ones where it was already banked — the byte-weight RANKING (the
file's entire purpose: "draft these first") was sorted mostly on already-finished code, with the
real targets buried underneath. The A2 audit predicted "true frontier: 1,475 families / 3.9 MB";
derived independently here it is 1,495 / 3.9 MB.
R33: the invariant answers this with no oracle at all —
an h_exact class is WORK iff at least ONE of its instances is still an INCLUDE_ASM stub.
That also makes the dedup-hash union redundant (a dedup-shared member is by definition not a stub),
so the `hash:` regex over config/dedup.us.yaml is DELETED. `instances` now counts only the members
still to bank, so the leverage is the real x-N.
family_hseq: stub scan -> corpus (+100 curated-name stubs the func_-only regex could not see; they
had made 3 still-stubbed functions look like MATCHED exemplars, which every sweep then re-nominates,
produces nothing from, and books as a silent skip). Its hardcoded "expect ~663/~186/~1.85M" self-check
was a stale 2026-07-11 snapshot — 38 banking commits have landed since — and is now labelled a
point-in-time reference, not an invariant. (Verified my change can only GROW the frontier: +100 stubs.)
census_conflict_callees: scoped to src/<ov>/<ov>.c alone, so it saw 13 of 264 stubs and reported
"wave scope: 2 still-stub" when the truth is 57 — every downstream percentage computed against a
denominator 96% too small. Now 0/57 (the audit's exact figure). Its 0-conflict answer was right BY
LUCK; it is now right for a reason. MARKED FOR DELETION (R33): it re-derives from C text what
reconcile_tu.py answers from the build, and its parse holes fail in the UNSAFE direction (an unknown
callee is silently bucketed "conflict-free"). Delete once reconcile_tu is wired into its only consumer.
R14 near-miss, recorded: my first census patch handed collect_stubs() a set of NAMES where it wanted
ADDRESSES, so the membership test was always false and it printed 0/0. Caught only because 0
contradicted the audit's expected 57. A scanner that returns 0 is indistinguishable from a scanner
that found nothing — which is the entire thesis of this audit, and it very nearly bit me while
fixing it.
|
||
|
Drew T
|
3be734bde8 |
docs(phase-26a): A3 progress — the oracle is built; 4 of ~10 scanners deleted
corpus.py + build_fuel_manifest + wave_targets + harvest_verify/gate_stage landed. Remaining: family_manifest/family_hseq (62% of the endgame plan is phantom targets), DELETE census_conflict_callees (R33), exemplar_miner/difficulty, jr_isolate_all. |
||
|
Drew T
|
70a99aa9f7 |
fix(phase-26a): A3 — the byte-gate could only see ONE translation unit (96.6% of stubs unreachable)
harvest_verify is the sole arbiter (G3/P9) and has never accepted a wrong match. It also could not
REACH most of the work: it scanned the single file the caller passed as --src and silently dropped
every draft whose stub lived elsewhere. An overlay's source spans up to 14 .c files, so:
open overlay stubs it could not see : 56,742 of 58,717 (96.6%)
ov_SC01_077 reachable by the gate : 13 of 264 (4.9%) -> 264 of 264 (100%)
THREE of the six callers passed no --src at all (orchestrator.py, grinder.py, idiom_hunt.py) and so
inherited gate_stage's `src = src or f"src/{binary}/{binary}.c"` default. For grinder.py that means
1,290 of its own 1,298 QUEUED FUNCTIONS COULD NEVER BANK, however good the permuter's output was.
=> Phase-22's "the permuter's fuel is exhausted" was never a safe conclusion. Re-test (A12).
gate_stage knew the right answer and then handed the gate the wrong file: its negative control
ALREADY globs every split .c to build bin_stubs. The default is now removed; --src is passed only
when a caller deliberately restricts the gate to one TU.
WHAT CHANGED, PRECISELY: only the SPLICE LOCATION. Each draft is now spliced into whichever TU
actually holds its stub, derived from tools/corpus.py. Every TU links into the same image, so ONE
`make build BINARY=<bin>` still gates them all — correct AND strictly fewer builds than the
per-split re-gate it replaces.
SAFETY (this is the byte-gate, so the argument is explicit): the VERDICT is untouched — `make build`
+ SHA1 == the locked hash. INCLUDE_ASM pastes the ORIGINAL assembly, so a wrong draft always changes
the bytes and always fails SHA1. A bug in the splice can therefore make the tool FAIL TO BANK; it
CANNOT make it falsely bank. The failure mode is conservative by construction.
VERIFIED end-to-end (2 real builds, tree clean before and after):
* discovery: 264 live stubs across 12 TUs (was: only those in the single --src file)
* IDENTITY known-answer test: 3 drafts whose stubs live in THREE DIFFERENT split TUs
(_jr_801734BC, _after, _jr_8012ACE0) — all discovered, spliced into their own files, built,
SHA-matched, committed, restored. Final SHA d19c9580 BYTE-IDENTICAL. Under the old code all
three were silently dropped as "not stubbed".
* `git checkout -- src/` recovers, exactly as the docstring promises.
Also derived rather than defaulted: --good-sha now reads config/check.<bin>.sha (a caller that
passed --binary but forgot --good-sha used to gate an overlay against RESIDENT's SHA), and
match_one_closeness resolves the asm subdir PER FUNCTION — one subdir for a whole batch is the same
single-TU bug, and pointing match_one at the wrong one scores a draft against a DIFFERENT function's
asm, producing a phantom non-zero closeness that lands in the backlog and feeds reserved_walls().
No committed source or config changed, so no build artifact can have moved; the byte-gate was
exercised twice and returned BYTE-IDENTICAL both times.
|
||
|
Drew T
|
a302908f24 |
fix(phase-26a): A3 — target selection was blind to 91.6% of the remaining work; now derived
The audit's CRITICAL finding, fixed at the root. Both tools now derive the corpus from
tools/corpus.py instead of keeping their own decaying copy of the tree layout.
build_fuel_manifest.live_stubs() — a hardcoded 3-file dict {<ov>.c, _a.c, _o0.c}. ov_SC01_077 has
FOURTEEN .c files, so it saw 30 of 264 stubs AND REPORTED SUCCESS. Everything downstream consumes
this manifest — worklist.py (100% of its rows), wave_targets.py (100% of its pools) — so:
targets 30 -> 263
reach-134 targets 10 -> 127 (the ENTIRE high-ROI band was invisible)
remaining gain 83,305 -> 994,633 instructions
994,633 is the A2 audit's predicted figure TO THE UNIT — a fourth independent confirmation
(auditor -> skeptic -> corpus.py -> this). Four of the five highest-leverage functions in the whole
project sit in split regions no tool could see; the top one, func_80178004 (165 ins x reach 134 =
22,110), had never been nominated by anything.
It rotted SILENTLY: .run/fuel_manifest.json (Jul 8) recorded 130 stubs; the same code today returns
30, because the Phase-26 jr splits moved ~100 stubs out from under a dict literal last edited in
Phase 22. Nobody noticed, because a target that is never nominated produces SILENCE, not an error.
wave_targets.REGION_SUB / asm_for() — a 3-entry dict with a silent fallback to the main subdir.
ov_SC01_077 has TWELVE asm subdirs, so 78 of the 87 targets any --class wave emitted handed a
drafter an asm path THAT DOES NOT EXIST. The drafter then drafts against nothing, and the wasted
attempt is booked in the backlog as a *matching* failure — which feeds reserved_walls() and
PERMANENTLY BLACKLISTS a function that was never actually attempted. A silent skip compounding into
a false wall. Now 263/263 asm paths resolve, 0 missing; asm_for() raises rather than guess.
Also: --region's 3-value whitelist defaulted to 'main', which sees 13 of 264 stubs even with a
correct manifest -> default 'any', free-form.
R33 throughout: the INCLUDE_ASM line is SELF-DESCRIBING (its first argument IS the asm subdir,
because splat wrote it there), so both dicts were second copies of a fact the tree already states.
A dict literal is strictly worse than the filesystem AND it fails OPEN. Never re-introduce one.
No build impact (selection/report tools only); docs/worklist.md regenerated with the honest numbers.
|
||
|
Drew T
|
f7b7399ebe |
feat(phase-26a): A3 — tools/corpus.py, ONE derived corpus oracle (+ a second oracle that can disagree)
The 28 surviving audit findings collapse to ONE bug repeated ~10 times: a hand-maintained model of
the corpus layout (a file allowlist, a single-.c assumption, a func_-only symbol regex, a REGION_SUB
dict) sitting on top of a filesystem that already answers the question. The fix is not ten repaired
regexes — it is one DERIVED oracle and ten deleted scanners (R33).
WHAT IT DERIVES FROM
1. THE FILESYSTEM. Which .c files make up a binary, and where a function's .s lives, are FACTS OF
THE TREE THAT SPLAT ITSELF WROTE. The INCLUDE_ASM line is SELF-DESCRIBING — its first argument
IS the asm subdir — so there is nothing to guess and no dict to rot. A dict literal is strictly
worse than the filesystem AND it fails OPEN (silently yields a wrong path) instead of closed.
2. THE PROVEN INVARIANT. INCLUDE_ASM pastes the ORIGINAL asm and the build is byte-identical, so a
function NOT wrapped in it is byte-exact. `matched` is DERIVED as sig - stubs, never re-parsed
from C text. (progress.py learned this the hard way: weighted_metrics() derived and was right;
classify() re-parsed C and inherited a bug.)
VALIDATED against the real corpus:
* ov_SC01_077: 264 stubs across 14 files. The old 3-file allowlist saw 30.
* Fleet: 58,717 stubs vs the allowlist's 1,992 — 56,725 (96.6%) were INVISIBLE.
* Coverage-asserted (R32): every INCLUDE_ASM line must parse, every symbol must resolve (ANY C
identifier — a func_-only regex silently misses the 100 curated listCdBuffer stubs), every stub
must have a .s. A silent skip is a DEFECT, not a no-op.
THE SECOND ORACLE (`make audit-corpus`) — the real lesson of this audit.
The byte-gate is structurally BLIND to a bad function boundary: the .s halves are pasted back
verbatim in original order, so the image stays byte-identical and green. Only an oracle that can
DISAGREE can see it. sig_image is that oracle — Ghidra-free, derived from the ORIGINAL bytes,
independent of splat. corpus.audit() cross-checks the two and reports:
PHANTOM — a stub address the sig does not know: splat INVENTED a function.
TRUNCATED — a stub whose .s length != the sig's: splat MIS-SLICED one.
It reports 193 (96 + 97) — reproducing the A2 audit's number EXACTLY, from an independently written
tool. That is a third confirmation of the listCdBuffer defect (auditor -> skeptic -> this).
AND AN R14 SELF-CATCH, recorded because the near-miss is the lesson.
Run naively over all 136 binaries the same check reports 914 slices — 4.7x the truth. It is noise:
main/resident are signed by the GHIDRA dumper, whose boundaries are shorter than splat's by design
(and which never analysed the linked PsyQ subsegs at all), so the comparison measures GHIDRA'S limits,
not splat's errors. Only the overlays are signed by sig_image, the oracle actually validated at
58,524/58,621. sig_is_independent() now encodes that domain, with the reasoning, so nobody repeats it.
A check applied outside its valid domain does not become more thorough — it becomes noise.
`make audit-corpus` is RED by design until A4 removes the bad symbol line; then it becomes a gate.
|
||
|
Drew T
|
ffb6f1a40f |
docs(phase-26a): A2 — the full audit; 28 findings survive; the endgame plan was majority-fiction
38 agents / 2.24M tok / 0 err. 32 findings raised -> 28 SURVIVED adversarial verification
(4 REFUTED, 16 downgraded). 40 scanners measured CLEAN. Full write-up: docs/tooling-audit.md ROUND 2.
THE ROOT CAUSE — one bug, ~10 times: a hand-maintained model of the corpus layout (a file
allowlist, a single-.c assumption, a func_-only regex, a REGION_SUB dict) sitting on top of a
filesystem that already answers the question. Every TU split silently widened it.
DECAY PROVEN: .run/fuel_manifest.json (Jul 8) recorded 130 stubs; the same tool today returns 30.
The Phase-26 splits moved ~100 stubs out from under a dict literal last edited in Phase 22 — and
nobody noticed, because an un-nominated target produces SILENCE, not an error.
MEASURED: 91.6% of ALL remaining project gain is invisible to target selection (true 994,633 ins;
the manifest sees 83,305). 117 of 127 reach-134 fns never nominated. harvest_verify cannot see
56,742 of 58,717 (96.6%) open stubs. wave_targets hands 78 of 87 targets a nonexistent asm path.
THREE RESULTS OVERTURN SETTLED CONCLUSIONS:
1. Phase-22's 'the permuter's fuel is exhausted' is UNSAFE. grinder banks through harvest_verify,
which sees ONE TU — 1,290 of its own 1,298 queued fns live in another. 99% could never have
banked. '0 banks since Phase 21' is equally consistent with 'the tool could not bank'.
2. The Phase-25/26 endgame plan is MAJORITY-FICTION. family-manifest.md advertises 2,758
multi-member families / 11.0 MB; 1,071 of them / 6.80 MB (62% of the byte-weight) are ALREADY
FULLY MATCHED. The ranking — the file's whole purpose — is sorted mostly on dead work.
3. A CORPUS defect the byte-gate is structurally blind to: symbols.us.txt:981 puts a main-EXE DATA
symbol (listCdBuffer = 0x80180000) into every overlay's symbol stack, but in overlay space that
address is CODE. splat cuts 97 real functions in half and invents 96 phantom ones = 193 slices
NOBODY CAN EVER MATCH, in 97 of 134 overlays — and the build stays byte-identical and green,
because the .s halves are pasted back verbatim. A perfect correctness oracle, a null coverage
oracle. What saved us: sig_image was RIGHT (58,524/58,621 vs spimdisasm; correct on all 97
disagreements). A SECOND INDEPENDENT ORACLE is the only reason it was visible at all.
FIX RESTRUCTURED around the root cause: ONE derived corpus oracle (A3) + ~10 DELETED scanners —
not ten fixed regexes. Plus the listCdBuffer corpus fix (A4) and the closeness oracle (A5, which
lies on 155 functions, feeding false walls into reserved_walls()).
decision-log (R31): the why, and the design lesson — a derived fact cannot rot; a hand-maintained
copy of it is a liability that grows with every structural change. We had no instrument that could
report ABSENCE: every gate we owned answered 'is this right?', none answered 'is this all?'
|
||
|
Drew T
|
92a695ef93 |
docs(phase-26a): ORDER CORRECTED — the full 18-tool audit runs BEFORE the fix campaign
Drew, mid-session: 'I thought the last session said there were some 15 tools we need to audit.' He was right, and my ordering was wrong. I had put the 18-tool audit near the END (as A9). docs/tooling-audit.md prescribes the opposite: dedup_integrate -> jtbl_family_bank -> the SELECTION tools -> masked_diff/match_one -> THEN the 40 measured findings. The reason is the one that matters: A hole in a SELECTION tool makes work invisible to PLANNING — the worst kind, because you never know to look. Fixing on top of unaudited selection tooling means re-running every fix when the audit later finds the hole. So: A2 is now the full audit; A3-A9 (the fix campaign) are blocked on it. Tool coverage, stated plainly: A1 (1) + A2 (18) + the fix campaign (~17 already-measured) = ~36 tools — not 82. The filter, from the audit doc: does it PARSE something, and does it GATE or SELECT work? The remaining ~46 are dead LLM-tier scripts. A1's result recorded in-file (the three false greens, the causal chain, the null-result blast radius that confirms R33). |
||
|
Drew T
|
bb65d36341 |
fix(phase-26a): A1 — dedup_integrate was a gate that could print a FALSE GREEN
The audit's priority #1: a fail-closed byte-honesty validator whose silent skips nothing downstream can catch. Three false-green paths, all measured, all now fail-closed with negative controls. R33 FIRST (derive, don't re-derive). The registry makes two claims; the tool only ever checked one, and mis-described that one: C1 EQUIVALENCE ("these vrams hold the same code in the ORIGINAL") — checked against the sigs, which sign the ORIGINAL bytes. KEPT. But the docstring claimed it also caught SOURCE drift: it cannot. A sig is a property of the ROM, immutable w.r.t. src/. Source drift is caught by the BUILD. Docstring corrected (P9). C2 BANK ("matched once in the source header, instantiated at every member") — NEVER CHECKED. Now DERIVED from the build invariant: INCLUDE_ASM pastes the ORIGINAL asm, so a member NOT wrapped in it is byte-exact, and one that IS wrapped is not banked — whatever the registry says. C2a: the group's macro token must occur in its source file. C2b: no member may still be an INCLUDE_ASM stub. THE THREE FALSE GREENS 1. 1808 groups claimed a DEFINE_func_* macro; only 1801 exist. The 7 ghosts printed [ OK ] — hiding 532 member-instances / 22,344 instructions of REAL, UNBANKED work (4 fns matched in ov_SC01_077, still INCLUDE_ASM in the other 133 overlays). 2. An absent .run/sig.<bin>.jsonl degraded to "0 validated, 0 failed" and EXIT 0. On a fresh clone the gate validated NOTHING and passed. Now fails; --allow-unsigned is the escape. 3. The bank claim was never checked at all. THE CAUSAL CHAIN (the audit's thesis in one example). 3 of the 4 hidden fns are defined in ov_SC01_077_jr_8012ACE0.c — a _jr_* split file. dedup_propagate.overlay_files allowlists only ("_a","_o0","_o0b","_after"), so the propagator could not SEE them; the group was registered anyway; dedup_integrate greenlit the lie. TWO silent-skip bugs compounding: one created the hole, the other hid it. Harvest fuel -> .run/audit/a1_harvest_fuel.json, banked in A5. BLAST RADIUS, MEASURED NOT PREDICTED (R14). Headline metrics UNCHANGED to the decimal (instr-weighted 66.5%, distinct-code 46.8%) — weighted_metrics() derives from the invariant and was structurally immune to the lying registry. FLEET REAL substantive unchanged (282,466): progress.py had already been taught to distrust it (commit:0574). Only dedup_integrate still believed it. A null result that CONFIRMS R33: the tool that refused to re-derive was the one that was right. - registry repaired: 1813 -> 1806 groups (7 ghosts removed; instances 223,725 -> 222,787) - make report GREEN end-to-end: 1806 validated, 0 failed | C1 coverage 222,787/222,787 signed - negative controls: stubbed member -> exit 1; missing sig -> exit 1; --allow-unsigned -> exit 0 - report-only tool: no compiled artifact depends on it, so no R22 clean-fleet is owed here |
||
|
Drew T
|
978ef703ac |
docs(phase-26a): A0 — the tooling-integrity audit, as an INSERTED HALF-PHASE (Drew's call)
- Drew (2026-07-14, gate 1): run the audit inside Phase 26, then resume at Task 7. Declined the alternative (close Phase 26 early on an unmet milestone -> Phase 27): the audit is a PREREQUISITE to structural completion, not a successor to it — the tooling that MEASURES the milestone is the thing at fault. Phase-3.5 precedent. - CURRENT_PHASE.md: the Phase 26-A block (A0-A11), built FROM docs/tooling-audit.md (40 measured findings), R33-before-R32 ordering — the best outcome is a DELETED scanner, not a fixed regex. - decision-log (R31): the why, the structural blind spot (a scanner extracts N, the true count is M > N, and nobody ever compared N to M — the byte-gate is a perfect CORRECTNESS oracle and a NULL COVERAGE oracle), and A1's first finding. - harness task list built (R28). |
||
|
Drew T
|
b757de73a8 |
docs(phase-26): checkpoint now POINTS AT docs/tooling-audit.md as the audit phase's input document
The 40 measured findings were living only in an ephemeral workflow journal outside the repo; the checkpoint carried my SUMMARY of the audit, not the audit. Now the fresh session is routed to the evidence, with the priority order (dedup_integrate FIRST — it can print a false green), the R33-before-R32 method (the best outcome is a DELETED scanner), and the real prize: re-test the walls that were diagnosed on top of the broken 10% callee oracle (the def-side loose-typing wall, the 159 arity conflicts, the type-heavy tail). |
||
|
Drew T
|
9a94e9ba46 |
docs(phase-26): docs/tooling-audit.md — the 40 measured findings, made DURABLE (R30)
The audit's evidence (6 agents + 6 skeptics, 1.2M tokens, 40 findings with file:line proof and measured candidate/parsed/skip counts) existed ONLY in a workflow journal OUTSIDE the repo. A fresh session would have inherited my SUMMARY of the audit, not the audit — exactly the R30 failure mode (capture context-dependent artifacts DURING the session that produced them). Drew caught it. Now committed as the plannable input to the audit phase, with: - the method (measure found-vs-candidates against an OVER-approximating detector; never "review the regex" — that is the failure mode that wrote these bugs); - why it gates the matching work (the byte-gate is a perfect CORRECTNESS oracle and a NULL COVERAGE oracle: green since Phase 5 at 0% decompiled, so it is compatible with ANY decomp %); - THE QUESTION IT ANSWERS: how many walls we have already "byte-proven" across 26 phases were lookup misses wearing a wall's clothes? (the def-side loose-typing wall, the 159 arity conflicts, the type-heavy tail were ALL diagnosed on top of the 10% callee-oracle hole); - R32 (coverage assertion) + R33 (derive, don't re-derive — apply FIRST: the best outcome is a DELETED scanner, not a fixed regex); - the priority order (dedup_integrate FIRST — a fail-closed validator that can print a FALSE GREEN); - the 7 bugs already fixed (do not redo) and the 63 tools not yet audited, with the filter for which matter. |
||
|
Drew T
|
6e8c459ade |
docs(phase-26): SESSION-8 CLOSE — checkpoint for a fresh session; the tooling-integrity audit gates what comes next
RESULTS. Fleet instr-weighted 63.0 -> 66.5%, distinct-code 39.1 -> 46.8%, fn-count 82.61%.
FINAL R22: make clean + extract-all + check-all -> 136/136 BYTE-IDENTICAL, 0 coverage defects.
dedup 1813/0. 0 NON_MATCHING (G4). 31 commits.
13 CORES CRACKED incl. the four heaviest functions in the game (952/890/562/536 ins). The 12-agent
Ultracode wave returned 11/12 first-pass MATCH, each adversarially verified (a skeptic re-ran match_one
+ the §8a jump-table check). Banked x134 this session: func_8015AE2C, func_80178D40, func_8015A3C8,
func_8013FFD8, func_8016AB6C, func_8015444C, func_801380E0 (+ func_8017BEBC x1).
THE TOOLKIT CROSSED A LINE — three ZERO-BYTE DIALS now cover the three passes that produce essentially
every "irreducible" residual, each with a diagnostic signature a cheap agent recognises on sight:
registers rotated -> global.c allocno priority -> §47 slider / §48-A pricing dials
two insns swapped, SAME regs -> sched.c rank_for_schedule LUID tiebreak -> §49 LUID dial
structure right, count wrong -> loop peel / cross-jump -> §46 / §48-D
That is why 9/12 fell first-pass to ORDINARY agents. Fable5 DISCOVERS a class; everyone else APPLIES it.
New: §46 §47 §48(+A4) §49 §50. Read §50-B before using §48-A1/A4 — it BOUNDS them (the "cross_jump
refunds the bytes" claim is FALSE for a 1-insn tail reached by two jumps; jump.c:1993 minimum=2).
DREW'S DIRECTIVE (binding): the TOOLING-INTEGRITY AUDIT comes BEFORE any further matching work, and is
NOT part of this phase. First act of the fresh session is a Tier-1 phase-boundary call (close Phase 26
early, or run the audit as an inserted phase — Drew decides).
WHY: seven silent-skip tool bugs in one session, and they are a STRUCTURAL blind spot — a scanner
extracts N items, the truth is M > N, and nobody ever compared N to M. The byte-gate is a perfect
CORRECTNESS oracle and a NULL COVERAGE oracle: it has been green since Phase 5 at 0% decompiled (
INCLUDE_ASM pastes the ORIGINAL asm), so a green gate is compatible with ANY decomp %. One 10% hole in
the callee oracle made NINE byte-exact functions look like an intrinsic compiler wall. The real question
the audit answers: how many walls we have already "byte-proven" across 26 phases were lookup misses
wearing a wall's clothes? (The def-side loose-typing wall, the 159 arity conflicts, the type-heavy tail
were ALL diagnosed on top of that hole.) Audit scope so far is 19 of 82 tools (23%), by risk — NOT
comprehensive; dedup_integrate.py is unaudited and can print a FALSE GREEN.
RULE CANDIDATES (P10, Drew ratifies at PhaseEnd):
R32 Coverage assertion — a corpus scanner must assert its own coverage and fail loud on unparsed input.
R33 Derive, don't re-derive — where a proven invariant answers the question, derive from it. The best
audit outcome is not a fixed regex; it is a DELETED scanner.
SELF-CORRECTION ON THE RECORD (P9/R14): I told Drew the headline numbers under-reported by ~190k
instructions. WRONG. weighted_metrics() never calls classify(), so it was structurally immune; the
published numbers were correct all along. I verified the DEFECT but not its BLAST RADIUS. A null result
against a strong prediction is a refutation — chase it.
|
||
|
Drew T
|
cc7ee23d03 | feat(phase-26): func_801380E0 swept ×134 siblings — R22 136/136 byte-identical | ||
|
Drew T
|
5b1de7acaa |
feat(phase-26): tools/reconcile_tu.py — ask "what can THIS TU see", not "what does the fleet call it"
WRITTEN + VALIDATED, DELIBERATELY NOT WIRED IN (inert; nothing imports it). Wiring + byte-gating is the
first item of the integration fix pass, AFTER the tooling-integrity audit Drew gated it behind.
The successor to reconcile_decls.py for the templating/banking path. Two things are wrong with that tool,
and the second is structural, not a typo (Phase-26 scanner audit):
1. BLIND TO FUNCTION POINTERS. DATA_DECL_LINE_RE wants `extern <type-words> D_x[];`, so the `(` in
extern void (*D_801DA75C)(void); <- fn-ptr scalar
extern void (*D_801812A4[])(void *); <- fn-ptr array (a dispatch table)
breaks its type run; the line never matches; the tool SILENTLY SKIPS exactly the symbols that are
failing and reports success. Blocking func_8017A4AC (536 ins x134 = 287 KB) today.
2. ITS ORACLE ASKS THE WRONG QUESTION. It elects a canonical decl by FLEET MAJORITY. But 34.4% of fleet
symbols carry >=2 mutually incompatible spellings, so a single fleet-wide answer is PROVABLY WRONG FOR
SOME TU BY CONSTRUCTION — and it is worse than a skip: it returns an ACTIVELY WRONG decl (measured:
3,717 symbols) that then collides with the very macro it was meant to conform to.
The only question that matters is what gcc compares the draft against: WHAT CAN THIS TU SEE. So reconcile_tu
reconstructs the TU's visible file-scope decl environment from BOTH §8c sources — col-0 decls AND the externs
INJECTED BY engine_core.h MACRO INVOCATIONS (a DEFINE_func_*() expands at file scope, so its leading externs
are genuine file-scope decls of the invoking TU, invisible to any col-0 scan: 544 visible syms from 1801
macros) — then conforms the draft's decl to it and CASTS AT EVERY USE (gcc folds a compile-time cast of a
known symbol, so the emitted bytes are unchanged; the whole-binary byte-gate remains the sole arbiter).
Carries a COVERAGE ASSERTION (R32): every line that LOOKS like an extern of a D_ symbol must parse, or it is
reported LOUDLY (--strict exits non-zero). A silent skip is a defect, not a no-op.
Validated read-only on func_8017A4AC: resolves D_801DA75C (fn-ptr -> `extern s32` + call-site cast),
D_80126B58 (struct), D_801DA734 (ptr).
|
||
|
Drew T
|
4e1ec84028 | docs(phase-26): decision-log — Drew: the tooling-integrity audit GATES further matching work and gets its own phase (fix the instrument before taking more readings) | ||
|
Drew T
|
8079240216 |
docs(phase-26): cookbook §50 — refinements that BOUND §47/§48 (from the func_80135EB0 wall)
The one wave core that did not close still paid for itself: - §50-A the exact §47 priority encoding: pri = floor_log2(refs)*refs*size/(death-birth), birth/death = 2*insn_number, DEATH IS 2*M NOT 2*M+1; ties break by ascending qty = BIRTH ORDER. A tie you can compute is a tie you can break. - §50-B ** BOUNDS §48-A1/A4 **: 'cross_jump refunds the bytes' is only true for tails >= 2 insns, or when one path FALLS THROUGH. jump.c:1993 calls find_cross_jump(minimum=2) and does not count the jumps themselves, so two j's with a 1-insn common tail will NOT merge. Check the tail length before using A1/A4. - §50-C an s16 param + 'x|1' manufactures a poison temp (ior->T; sll; sra); s32 does not (combine reuses i2dest). Widening a parameter can DELETE an allocno. - §50-D copy preferences beat plain preferences and need a BLOCK BOUNDARY (combine's LOG_LINKS never cross blocks). - §50-E maspsx/gas MERGES lui $at for two stores to the same 64K page — which is why the original interleaves its global stores. Never 'tidy up' the store order of a matched function. - §50-F the documented wall: a local-alloc qty_compare_1 race needing a reload-deleted no-op copy in a specific range. |
||
|
Drew T
|
c12c497e10 | feat(phase-26): func_801380E0 banked ×1 (crack wave) — whole-binary gate, R22 136/136 | ||
|
Drew T
|
8a3f227ac1 | feat(phase-26): func_8015444C swept ×134 siblings — R22 136/136 byte-identical | ||
|
Drew T
|
7e7c80659b | docs(phase-26): decision-log — the audit's finding was real, my reading of it was wrong; derive metrics from invariants, don't re-parse the world (R14/P9 self-correction) | ||
|
Drew T
|
329ab4cbe7 |
fix(phase-26): progress.py — K&R definitions were invisible; phantom dedup members; + a COVERAGE ASSERTION
TWO REAL BUGS in classify(), and an HONEST CORRECTION of their blast radius (P9/R14).
- BUG 1 (under-count). classify() decides definition-vs-declaration by scanning to the first `{` or
`;`. A K&R definition puts its parameter declarations BEFORE the brace:
s32 func_8015AE2C(arg0)
s32 arg0; <- a `;` before the `{`
{ ... }
so it was read as a forward declaration and dropped into NO bucket — not REAL, not a stub,
invisible. And a K&R def is MANDATORY whenever a zero-arg engine_core.h thunk calls the function,
i.e. exactly the heavy-jr cores our own banking recipe produces: func_8015AE2C (562x134),
func_8015A3C8 (493x132), func_80166994 (369x134) were all compiled, linked and BYTE-IDENTICAL in
the shipped build while counting as zero. Fix: skip over K&R parameter declarations (a bare
`<type> <name>;` carrying no parens — that is what distinguishes it from a wrapped ANSI
prototype's continuation line, which always carries the `)`).
- BUG 2 (over-count). `real |= dedup_members(BINARY)` folded in EVERY registered dedup member without
checking it is actually instantiated. A member still sitting as an INCLUDE_ASM stub was counted
REAL *and* stayed in `stubs` — double-counting into `matchable` and inflating `byteident`
(532 phantom instances, per the scanner audit). Fix: subtract `stubs`. The registry is advisory;
the source tree is authoritative.
- COVERAGE ASSERTION (the rule ratified 2026-07-14): ground truth = every function splat emitted a
.s for. Anything classify() cannot place in ANY bucket is now reported LOUDLY (stderr + the .md),
because a silent skip is a defect, not a no-op. Currently: 0 unplaced.
- CORRECTION (this is the part that matters — I over-claimed and the bytes refuted me). The scanner
audit reported ~243k instructions "counted as nothing", and I repeated it. WRONG. weighted_metrics()
— which produces the HEADLINE instr-weighted and distinct-code numbers — does NOT call classify()
at all. It tests `func not in src_stubs(binary)`: since the fleet is 136/136 byte-identical,
anything not wrapped in INCLUDE_ASM must be compiled C emitting the exact original bytes. That test
never parses a definition, so it is IMMUNE to this bug. Verified: old-vs-new on the same tree gives
identical weighted numbers. The published 65.6% / 44.9% were CORRECT ALL ALONG; only the secondary
REAL count and fn-count % were wrong.
THE LESSON, sharper than the one we started with: a metric DERIVED FROM A PROVEN INVARIANT beats a
metric that RE-PARSES THE WORLD. weighted_metrics() leans on the byte-gate and inherits its
correctness; classify() re-derives the same fact by parsing C and inherited a bug instead. Prefer
the former wherever an invariant exists.
|
||
|
Drew T
|
9caea60142 | feat(phase-26): func_8015444C banked ×1 (crack wave) — whole-binary gate, R22 136/136 | ||
|
Drew T
|
cffbdbe88e | feat(phase-26): func_8016AB6C swept ×134 siblings — R22 136/136 byte-identical | ||
|
Drew T
|
f5f3c44693 | feat(phase-26): func_8016AB6C banked ×1 (crack wave) — whole-binary gate, R22 136/136 | ||
|
Drew T
|
55a63fae95 |
fix(phase-26): remove the duplicate code-subseg line from ov_SC01_077's committed config
Residue of the same isolation-revert bug fixed for ov_SC01_000 in commit:0558: a failed bank left its isolation's config in place, the retry re-isolated on top, and a duplicate - [0x4b364, c, ov_SC01_077_jr_801734BC] line rode into a commit. It is HARMLESS to splat (a zero-length subseg), so R22 stayed green and the correctness gate never saw it — but it BLOCKED every subsequent isolation, which is what failed 5 of the 9 crack-wave banks. Caught only by the fail-loud validation added in commit:0558 (a tool that refuses to proceed on input it does not understand), never by the byte-gate. Fleet audit: ov_SC01_077 was the ONLY affected config of 137. ov_SC01_077 rebuilds d19c9580 BYTE-IDENTICAL. |
||
|
Drew T
|
ffcd914ed5 | feat(phase-26): func_8013FFD8 swept ×128 siblings — R22 136/136 byte-identical | ||
|
Drew T
|
a30500eecf |
docs(phase-26): cookbook §49 — the LUID DIAL (func_8017A4AC MATCH, 536 ins ×134, the biggest remaining fn)
The sched.c analogue of §47's live-length slider. A close=2 with IDENTICAL registers is not a regalloc residual — it is sched2's rank_for_schedule falling through to its final tiebreak, 'return INSN_LUID(tmp) - INSN_LUID(tmp2)', i.e. position in the .greg stream. Root cause is upstream in sched1: adjust_priority/birthing_insn_p gives every register-DEFINING insn LAUNCH_PRIORITY 0x7f000001 (sched.c:2574), which sinks the un-boosted insn past its rivals and inverts the LUID order. THE DIAL: materialize a call argument's sign-extension into an explicit s32 temp, placed AFTER the intervening statement (adjacent to the load, combine fuses lhu+sll+sra into one lh and you LOSE 3 insns; the intervening store blocks the fusion). Widen the prototype to (s32,s32) so the call adds no conversion. Same instructions emitted, earlier INSN_LUID -> the tie flips. Zero-byte dial family is now three: §47 live-length (global.c), §48-A1/A4 sink-init/sink-call (global.c/local-alloc), §49 LUID (sched.c). Method: -dS -dR dumps the ready lists + priorities; equal priorities => you are on a LUID tiebreak => the fix is PLACEMENT, not registers. 11 of 12 wave cores now MATCH. |
||
|
Drew T
|
b813432b1b | feat(phase-26): func_8013FFD8 banked ×1 (crack wave) — whole-binary gate, R22 136/136 | ||
|
Drew T
|
7a5657e83c | feat(phase-26): func_8015A3C8 swept x132 siblings — R22 136/136 | ||
|
Drew T
|
0bf0313a6d |
docs(phase-26): cookbook §48-A4 — SINK THE CONSUMER CALL INTO THE ARMS (func_8016AB6C MATCH)
The inverse of A1: A1 sinks an INIT to shorten a live range; this sinks the CONSUMER to delete the allocno outright. A value defined in both if/else arms and consumed only by a call at the join becomes a cross-block global allocno whose copy-prefs include the ARG register — and find_reg's copy-pref override scans ascending regno (NOT reg_alloc_order), so $5 deterministically beats $16. The only escape is allocno_calls_crossed>0 (global.c:906 strips caller-saved prefs), which a pseudo defined after one call and dead before the next does not get. Duplicating the consumer call into the arms demotes it to a call-crossing BLOCK-LOCAL -> local-alloc gives it a callee-saved reg, preserving the §48-A2 $s0 occupant; the identical post-reload tails are re-merged by cross_jump, so the duplication costs ZERO bytes. 10 of the 12 wave cores now MATCH. |
||
|
Drew T
|
5aea9b1112 | docs(phase-26): decision-log — promote the silent-skip lesson from a rule to a MECHANISM (coverage oracles); flag the verdicts reached on top of the broken oracle (Drew approved) | ||
|
Drew T
|
91fd1257ba | docs(phase-26): session-8 checkpoint — 3 heaviest cores cracked, 12-core wave 9/12 MATCH, 6 silent-skip bugs fixed | ||
|
Drew T
|
29478ab9b0 |
docs(phase-26): cookbook §48 — the 12-core jr crack wave (9/12 MATCH): allocno-pricing dials, the EBB rule, type-driven codegen, the cross-jump ratchet
- §48-A allocno-PRICING DIALS (global.c:594): sink an init into the if/else arms to collapse a live-range and RAISE priority byte-neutrally (cross_jump re-merges the tails after regalloc); the local-alloc $s0 occupant that pushes arg0 to $s1; block-scoped per-case temps as a local-alloc tie gate (local-alloc.c:1765 refuses to tie a multi-block pseudo). - §48-B THE EBB RULE, the general form of §46-L2: anything that must survive cse needs its def and uses in different extended basic blocks — reg-reg copies, held global addresses (la $sN in a loop preheader, def at loop top + use inside a jtbl-reached case), pointers-to-global across calls. Corollary: a pointer-to-global survives only if EVERY use is at offset 0 (fold_rtx folds sym+k). - §48-C the C TYPE selects the addressing mode: scalar global -> lui/%lo, struct global -> la+offset; lwl/lwr block copy == a plain assign of a 2-byte-aligned struct (mips.c:output_block_move needs align>=4 for the lw/sw arm); the dead-sibling-scalar trap (cost 108 ins — use a real array). - §48-D the CROSS-JUMP RATCHET: two cases needing opposite branch senses cannot be a mirrored if/else — cross_jump + jump.c's invert-over-uncond-jump collapse them into one. Use gotos into labels inside the other case. - 9/12 first-pass MATCH by ordinary agents applying the map. Fable5 discovers a class; everyone else applies it. |
||
|
Drew T
|
6f3441d261 |
fix(phase-26): 10% of the canonical-callee oracle was silently missing (own-line-brace DEFINE macros)
- BUG: gen_harvest_targets.SIG_IN_BODY_RE required `)\s*{` between a DEFINE_func_* macro's signature
and its opening brace. When the brace sits on its OWN continuation line there is a line-continuation
BACKSLASH between them:
s32 func_80148824(void *arg0) \
{ \
and `\s` does not match `\`. So the regex silently dropped every own-line-brace macro.
- BLAST RADIUS (measured): 186 of 1801 engine_core.h shared signatures — 10% of the oracle — were
MISSING from the canonical-callee map that cast_call_sites / sig_unify / gen_harvest_targets resolve
against. A draft calling one of them kept its own guessed signature, hit `conflicting types` against
the TU's real definition, and the recovery pass reported nothing to fix — the failure looked like a
hard wall. This is why the crack wave's byte-exact cores would not bank.
- FIX: `[\s\\]*` instead of `\s*`. Oracle 2122 -> 2308 entries.
- PROOF: func_8015A3C8 (493 ins, MATCH standalone) went from "28 conflicting types, unbankable" to
BANKED ×1 BYTE-IDENTICAL at the `recovered` stage, with zero hand edits. R22 clean-fleet 136/136.
- This is the phase's SIXTH silent-skip bug and the THIRD of the same brace-placement class (§19
find_site; scope_data_externs' own-line brace; now this). Cookbook §40's standing lesson applies:
a tool that silently no-ops on input it cannot parse is indistinguishable from one that had nothing
to do — prefer fail-loud on unparsed input.
|
||
|
Drew T
|
cc08601ae7 |
feat(phase-26): func_80178D40 swept ×134 — the heaviest core in the game, fleet-wide
- 132/132 siblings banked (0 failures) via jtbl_family_bank --raw + the lazy-isolation chain. Each sibling: isolate -> jtbl carve -> remap from the raw crack -> stage ladder (raw -> scoped §8d -> recovered -> reconciled) -> WHOLE-BINARY byte-gate. - R22 clean-fleet 136/136 BYTE-IDENTICAL from `make clean`; 0 NON_MATCHING (G4). - METRICS: instr-weighted 63.8 -> 64.7%; distinct-code 40.7 -> 42.8% (+2.1 points from ONE core — 890 ins x 133 overlays = ~118K instructions of unique engine code); fn-count 82.43%. - tools/bank_exemplar.py promoted from scratch: bank a cracked EXEMPLAR ×1 through the same stage ladder jtbl_family_bank uses for siblings (carve/lazy-isolate -> raw/scoped/recovered/reconciled -> whole-binary gate). The exemplar path was previously hand-run each time. |
||
|
Drew T
|
07ebb5658d |
fix(phase-26): jr_isolate_all empty-region0 skip — cutting an already-isolated region's non-leader works
Cutting func_80178D40 out of ov_SC01_000_jr_801734BC adds the region's banked LEADER (0x801734BC) as a cut too (the one-carve-per-object rule), making region 0 EMPTY (the object's first item IS the first cut) — and region 1's derived name equals the object name, so emitting region 0 duplicated the line exactly -> splat "segments out of order". Skip an empty region 0; region 1 rightly claims the object's offset and name. First sibling then banks through the full chain (isolation validation green -> carve -> --raw remap -> stage ladder -> whole-binary gate): ov_SC01_000 BANKED, included here. The remaining 132 siblings sweep next. |
||
|
Drew T
|
7e4165676d |
fix(phase-26): isolation-residue corruption chain — config cleanup + revert() restores config + fail-loud validation + --raw sweep mode
Three-layer fix for the func_80178D40 ×133 sweep failures: - LAYER 1 (the residue): jtbl_family_bank.revert() restored carve pieces + src/ but NOT the isolation's CODE-subseg lines in the splat config. A failed bank attempt (BEBC's first try) left its isolation config in place; the successful retry re-isolated on top and a DUPLICATE `- [0x4b364, c, ov_SC01_000_jr_801734BC]` line rode into the commit (harmless to splat — zero-length — so R22 stayed green). revert() now also restores config/splat.<ov>.yaml. The committed duplicate is removed (ov_SC01_000 rebuilt BYTE-IDENTICAL 9052dc0e). - LAYER 2 (the detonation): jr_isolate_all walked the duplicated object TWICE -> two replacements -> a reversed duplicate block -> splat "segments out of order". It now VALIDATES the generated config (code subsegs strictly ascending, names unique) and refuses to write on violation, naming the likely cause — a corrupt input dies at the tool, not three tools later. - LAYER 3 (the sweep template): jtbl_family_bank gains --raw <crack.c> — template from the RAW crack via remap_hseq_body instead of the exemplar's banked source unit. REQUIRED when the exemplar banked at the `reconciled` stage: a reconciled body is TU-SPECIFIC (§41c — uniquified type names, TU-targeted casts), so extract_unit hands the sweep a polluted template and every sibling gate-fails (byte-proven: 178D40 banked reconciled -> sweep 0/4; 8015AE2C banked raw -> sweep 133/133). Same law as family_sweep --reconcile-raw. |
||
|
Drew T
|
660aa9f215 |
feat(phase-26): func_80178D40 (890 ins, ×134 — the heaviest core) banked ×1 in ov_SC01_077
The §46 crack (MATCH 890/890, pin-free) banked through the whole-binary gate: lazy isolation -> new region ov_SC01_077_jr_80178D40 + jtbl carve -> the FULL stage ladder (raw 36 conflicts -> scoped -> recovered 5 -> RECONCILED banked; canon_sig_reconcile's type-name uniquification resolved the SV3/Obj20/Blk typedef collisions) -> BYTE-IDENTICAL d19c9580. R22 clean-fleet 136/136. The ×133 sibling sweep (PURE per-location, members staged) runs next. |
||
|
Drew T
|
3a67dd609f |
feat(phase-26): func_8017BEBC (952 ins, ×113) CLOSED + banked ×1 — the §47 live-length slider
The largest unmatched core in the game, walled at close=2 for the permuter (25 min, no close) and
queued for a gdb-on-cc1 read. Closed WITHOUT gdb — the RTL dumps were the oracle:
- THE TIE, byte-measured (.lreg): &g.sz1 pseudo 228 refs 13 / live_length 783; &g.sz2 pseudo 230
refs 13 / 782 -> pri = int(390000/L) = 498 == 498, an exact int-truncation tie in global.c:594
allocno_compare. Tie-break = creation order -> allocation follows emission; the target needs them
to DIFFER (allocation sz2-first, emission sz1-first). The shipped operand-permutation workaround
could only pick one (close=2 vs close=10).
- THE FIX (§47): restore NATURAL operand order (emission correct) + ONE zero-byte
`__asm__ volatile ("")` placed BETWEEN two existing GTE volatile asms (no new cse/sched barrier —
one is already there) -> +1 static insn at global-alloc time -> L 784/783 -> pri 497 vs 498 ->
the tie SPLITS toward the shorter-lived (later-created) pseudo, which is ALWAYS the direction
"allocation != creation" requires. All 10 grants cascade; MATCH 952/952 first try; the slider
emits only #APP/#NO_APP (zero bytes). PIN-FREE, ×113 template-safe.
- BANKED ×1 in ov_SC01_000 through the WHOLE-BINARY gate (jr fn — match_one is not the arbiter,
§8a): lazy isolation -> new region ov_SC01_000_jr_8017BEBC + 9-piece jtbl interleave -> splice ->
BYTE-IDENTICAL. One TU-visible decl reconcile en route (D_800B9A02: declare the TU's `short`,
force the unsigned halfword at use `(*(u16*)&D_800B9A02)` — §8d sub-class (b)).
- R22 clean-fleet 136/136 BYTE-IDENTICAL; 0 NON_MATCHING (G4). The ×113 sibling sweep is IMM-class
(scattered addresses) -> Task-8 mechanical work via the imm engine.
- cookbook §47 (the slider method + the placement rule + the direction law); decision-log (R31).
|
||
|
Drew T
|
a0e7ff7f6f |
fix(phase-26): scope_data_externs ANSI-brace fix + wire §8d into family_sweep --hseq; 780-class diagnosed
- _body_open_brace only matched a `{` on its own line (the K&R shape), so fix() SILENTLY NO-OP'D on
every ANSI draft — the same silent-skip disease as the four catalogued in §40/§8d, caught because
the h_seq re-sweep banked 0/780. Now brace-scans forward from the signature (ANSI same-line,
ANSI own-line, and K&R all work). Load-bearing for func_80178D40's upcoming ×134 bank.
- family_sweep --hseq now applies the §8d scoped stage at staging time.
- HONEST RESULT: still 0/780 — the substantial-band h_seq rejections are a DIFFERENT (sibling) class,
now fully diagnosed against the bytes:
gcc-2.7.2 decl-conflict semantics: a VISIBLE file-scope decl + a conflicting later decl (file OR
block) is a HARD ERROR; a limbo-only block decl (scope closed) + a conflicting later decl is a
warning. The h_seq drafts carry the EXEMPLAR TU's spellings; sibling TUs legitimately spell the
same symbol differently (loose typing), and the visible decl is often MACRO-INJECTED — a
DEFINE_func_* leading extern (§8c), invisible to any col-0 scan (e.g. D_80115158's `short` decl
enters ov_SC01_000.c via DEFINE_func_8014168C() @4637; the draft carries ov077's
`unsigned short` -> conflicting types at ANY scope).
Sub-class (a) no-visible-decl -> §8d demotion (the jr class, proven x133). Sub-class (b) visible
decl, different spelling -> needs reconcile-to-TU-VISIBLE (rewrite the draft decl to the TU-visible
spelling + byte-neutral access cast; oracle = col-0 decls above the stub + engine_core.h macro
externs for the DEFINE_ invocations above). Parked as a designed follow-up task; the 780 members
are mechanical-recovery fodder once the tool exists.
|
||
|
Drew T
|
b8a525bb98 |
feat(phase-26): h_seq substantial-band re-sweep — 266 free member-matches (~0 agent tokens)
The extract_unit fix (commit:0552) revealed 82 families with a genuinely-matched exemplar and UNSWEPT siblings (~2.03M templatable bytes) — mostly exemplars cracked AFTER the session-2/3 mechanical band sweeps ran (the giant campaign + recent cores), so the sweep had simply never seen them. - re-ran `family_sweep --hseq --band substantial` on a regenerated manifest: 29 matched-exemplar families, 1046 member drafts staged, 1643 correctly skipped as pinned-exemplar. - BANKED 266 member-matches / 780 gate-rejected. The whole-binary byte-gate (G3/P9) arbitrated every one; R22 clean-fleet 136/136 BYTE-IDENTICAL from `make clean`. - metrics: instr-weighted 63.6 -> 63.8%; distinct-code 40.5 -> 40.7%; fn-count 82.39%. The 780 gate-rejections are the next lever: family_sweep's h_seq path does NOT yet carry the §8d `scoped` stage (it prepends carried data externs at FILE scope, the exact class that blocked the jr sweeps), so a large share are expected to be the same decl-environment conflict. Investigated next. |