mirror of
https://github.com/Druthulu/BFM-decomp
synced 2026-09-27 05:56:00 -04:00
8f3e4a12ba0e30ad59cdae37f77fbce3f32290db
195 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Drew T
|
b1c58d7668 |
docs(phase-26a): A10 wave 1 — closeness-0 wall re-test CONFIRMED REAL (0/958 bank)
Re-gated all 958 closeness-0 open-stub backlog drafts through the FIXED gate across 135 binaries in parallel: banked=0, near=957, failed=71. The closeness-0 backlog is genuine whole-binary near-misses, NOT tooling misses -- match_one's isolated closeness==0 systematically overstates whole-binary bankability, and the repaired gate recovers none. P9: a re-confirmed wall is as valuable as a dissolved one. (The audit's tooling-walls were already banked by A3f/g/h + A9b, +2.1% instr.) backlog.py: env-gated BACKLOG_NO_RENDER so parallel workers skip the render race (append is atomic) -- backward-compatible parallel-safety. backlog.md refreshed with the re-test's whole-binary-informed scores. |
||
|
Drew T
|
ea20bdf9f3 |
fix(phase-26a): A9g — jr_inventory: retire the ephemeral roster, derive banked from the image (R33)
jr_inventory's `banked` set was filtered by an EPHEMERAL, gitignored .run/banked_func_*.json roster: a `rm -rf .run` / fresh clone would blind ALL banked jr at once, cross-address siblings (roster named after the exemplar) were structurally invisible, and non-leader banked jr were missed. "The purest R33 case in the group" (audit). FIX (the audit's exact prescription): delete the roster glob + `cand` filter; `banked` is DERIVED FROM THE IMAGE — a real-C def/define fn is a banked jr iff family_remap.reloc_targets shows it references a committed .rodata carve offset (config + image, both durable; cross-address- and non-leader-immune). R32 assertion: every committed carve must resolve to EXACTLY ONE owner or abort (a stranded/duplicated carve is the §8b func_801734BC incident, never silent). Also fixed the adjacent finding: the asm_jr scan's func_-fullmatch dropped the curated-name listCdBuffer jr; now resolved via oss.addr_of(). (The --only path's own fullmatch is left — it parses user input, not the corpus.) Perf: read the overlay image ONCE and pass it to reloc_targets(..., data=) — a new backward-compatible param on family_remap (regression: 0/80 mismatch vs the re-read path). Verified: data-param behavior-identical; the R33 win — ov_SC02_000 now finds the cross-address sibling func_8017FCB0 the roster missed; full-fleet parallel run = 134/134 OK, 0 false aborts, 1336 banked jr == 1336 carves -> 1:1 ownership holds fleet-wide. Byte-safe: jr_isolate_all is not in the make build/extract path (R22-neutral); the change makes future isolations strictly more correct. |
||
|
Drew T
|
96e025a324 |
fix(phase-26a): A9f — overlay_src_split swallowed 2 real definitions; the selftest was blind
scan_construct's force_decl latched from the FIRST token and returned at the
first depth-0 `;`, so a definition sharing a physical line with leading externs
(`extern A; extern B; void f(){...}`) was never anchored — absorbed into the
next anchor's preamble. The parser jr_isolate_all rewrites source from was short
two functions in the exemplar overlay. The round-trip selftest is a SERIALISATION
check (a miss lands in a preamble -> round-trip still exact BY CONSTRUCTION), so
it was structurally incapable of seeing this.
FIX (byte-safe): force_decl no longer survives a same-line `;` with trailing
code — re-classify from the remainder and keep scanning so the def anchors (its
leading externs stay in its whole-line item text -> round-trip byte-identical).
Rejected the audit's "split into 3 constructs": round-trip joins whole-line
chunks with `\n`, so sub-line splitting would insert a newline where a space was.
def_name now names the LAST top-level header before `{` (the definition, not the
first same-line extern; byte-identical on every single-def construct).
R32: hidden_definitions() coverage oracle wired into selftest — an independent
detector of `func_XXXX(...){` bodies not anchored. The selftest is now a coverage
check, not just serialisation.
Verified: 2 swallowed -> 0; regression over 1738 overlay .c = 0 round-trip fails,
0 non-monotonic, 0 non-additive changes, +2 anchored defs. Byte-safe: tool not in
the build path (R22-neutral); ov_SC01_077 rebuilds d19c9580; neither def straddles
a committed subseg boundary. Audit ledger line refs were stale (src rewritten);
real cases are ov_SC01_077_after.c:2020 + ov_SC01_077_jr_8015444C.c:1495.
|
||
|
Drew T
|
68d29ba8af |
fix(phase-26a): A9e — reconcile_tu already wired into bank_exemplar (A3d); document the ladder
NULL RESULT (P9/R14): the session-12 "wire reconcile_tu into bank_exemplar"
handoff item was stale — A3d (commit:0601) already wired reconcile_tu into
jtbl_family_bank.recover() ("on BOTH banking paths"), and bank_exemplar's
`recovered` stage delegates to fb.recover = cast_call_sites + reconcile_tu.
Proven working by A9b (func_8017A4AC banked at the recovered stage, reconcile_tu
resolving its struct + fn-ptr conflicts). No live tool references the RETIRED
reconcile_decls (only docstrings + the audit-cdecl differential harness).
No code change warranted. Byte-neutral hardening only: document the
raw/scoped/recovered/reconciled fallback-ladder composition inline in
bank_exemplar so a future session does not re-run this "is it wired?" trace.
|
||
|
Drew T
|
40477281ce |
fix(phase-26a): A9d — retire the dead Phase-17 canonical-sig chain (R33)
DELETE tools/census_conflict_callees.py + tools/derive_canonical_sigs.py. - census_conflict_callees: audit-CONFIRMED marked-for-deletion (commit:0593; decision-log 836). It re-derives from C text the per-TU "defined/declared/ stubbed/external?" question that reconcile_tu (Phase 26) answers FROM THE BUILD — and does it WRONG in the unsafe direction (unknown -> conflict-free). - derive_canonical_sigs (census's ONLY consumer): genuinely dead — last touched Phase-17 (commit:0140), output .run/canonical_sigs.json read by nothing (no Makefile/workflow/import), no-ops on the 2-byte [] input, asm-arity heuristic 36% wrong vs byte-exact banked C. Its purpose was retired in A3d (fleet-majority oracle -> reconcile_tu's per-TU oracle). Deleting census orphans it, so the whole dead chain ceases to exist (R33: the best outcome is a DELETED SCANNER, not a fixed regex). Byte-neutral by construction (neither tool is in any build/report path): module-import smoke over the 13 importable harvest/bank/report/reconcile tools = all clean; bank_exemplar is a run-only script (indexes sys.argv at module scope), imports neither deleted module. No src/config change -> no byte moves. Doc-pointer hygiene: hand-matching-process.md 8a, matching-cookbook.md (canonical-sig-layer entry), tooling-audit.md (ledger row + derive entry) all annotated DELETED/historical so nothing points at a nonexistent tool. |
||
|
Drew T
|
d7d2613ae6 |
fix(phase-26a): A9c — lint_symbol_refs green + wired into make report
The ONLY detector for the R22 rename-drift failure mode (a symbols.us.txt rename leaves a func_<ADDR> ref dangling in committed src; a clean rebuild fails, an incremental build masks it with a stale .o — undetected Phase 21→23). It was RED (262 false positives) and UNWIRED. The audit's 3 blind spots, all fixed: #1 globbed src/**/*.c only -> now ALSO scans src/shared/*.h, where engine_core.h's 10k+ func_/D_ tokens live and one dangling ref breaks EVERY clean build at once (negative-control-proven: an injected bare ref in a shared header IS flagged). #2 read 2 of 138 symbol files -> now reads every REAL stacked file (config/symbols*.txt), and correctly EXCLUDES the R13 proto files (never stacked into a build; would invent phantoms). #3 no __asm__("label") model -> ALL 262 false positives were this class: engine_core.h binds func_8005C324 to memcpy via `... __asm__("memcpy")`, so the ref emits the LABEL and resolves at link regardless of the rename. asm_labeled_addrs() now recognizes it. VERIFIED: green on HEAD (exit 0); negative control (inject a bare func_800d1bd8 -> DsMix in a shared header) FLAGS it (exit 1) then clears on removal — detection intact, not trivially green. Wired as a fail-closed gate in `make report` (make -n confirms), beside dedup_integrate --check. No src/build change (report-time integrity gate) -> the byte-identical build is unaffected. |
||
|
Drew T
|
abbedcee8b |
fix(phase-26a): A9a — canon_sig_reconcile sees fn-ptr dispatch tables (cdecl supplement)
The def-side-wall recovery tool, live on the ×134 economic-engine paths (jtbl_family_bank,
family_sweep --reconcile-raw, bank_exemplar, t7_bank, scope_data_externs, family_remap). The
audit's two HIGH findings: its tu_ambient/visible_above classifier regexes and _reconcile_data's
_DATA_EXTERN_RE have a type class `[\w \*]` that cannot hold a `(`, so they are 100% blind to
fn-ptr DATA decls — `extern void (*D_x[])(void);`, the per-overlay jump-table DISPATCH arrays
jtbl_family_bank exists to bank. A fn-ptr symbol thus landed in NO bucket, fell out of `visible`,
and _reconcile_data block-moved the draft's extern into a guaranteed `conflicting types`.
FIX (R33-adjacent, but SUPPLEMENT not wholesale-replace — the proven regex stays byte-identical):
- tu_ambient / visible_above: keep the v3.2 regex classification UNCHANGED, then supplement from
cdecl.tu_scope with ONLY the fn-ptr symbols the regex drops (_fnptr_data). A full delegation was
tried and rejected — cdecl normalizes funcs formatting (void*→void *, param names, and a real
return-ptr hazard u8*→u8) which rippled 728k value-changes into the callee string-compare; the
audit flagged fn-ptr DATA, not funcs, so the blast radius is held to exactly that.
- _reconcile_data: a fn-ptr pre-pass (_FNPTR_DATA_RE) — visible ⇒ STRIP (ambient serves; a
call-through `D_x[i]()` is decl-INDEPENDENT indirect codegen, so NO access-cast: casting a
fn-ptr would mangle it into `((u8*)D_x)[i]()`, the dormant transform reconcile_tu documents);
not visible ⇒ block-move verbatim. Plus the F2 fix (bail on a `(` in the CODE, not a comment).
- R32 coverage assertion: a fn-ptr data extern may never survive the pass.
PROVEN ADDITIVE (regression baseline .run/audit/a9a_*):
- classifier snapshot over 1683 TUs: REGRESSIONS 0, ADDITIONS 69,798 (all data fn-ptr).
- reconcile() OLD-vs-NEW over 434 real drafts against home TUs: 0 regressions.
- src/ UNTOUCHED (a tool change moves no bytes); the whole-binary byte-gate (G3/P9) remains the
sole arbiter — a wrong reconcile fails to bank, never falsely banks.
|
||
|
Drew T
|
b89fcc2edc |
fix(phase-26a): A3e — gate_stage pinned the byte-gate back to 4.9%, OF A3'S OWN FIX
THE WORST DEFECT IN THE AUDIT IS NOT IN A SCANNER. It is one default argument in the CALLER of a
scanner we had already fixed.
# tools/gate_stage.py:315
summary = run_gate(a.drafts, binary=b, src=a.src or f"src/{b}/{b}.c", ...) # ALWAYS the main .c
`src` RESTRICTS the byte-gate to ONE translation unit, and _gate1 does `if src: cmd += ["--src", src]`
-- always truthy. A3 had just taught harvest_verify to DERIVE each draft's home TU *when --src is
omitted*, lifting the byte-gate's reach from 4.9% to 100%. gate_stage NEVER OMITS IT. The fix was
neutralised by its own caller's default, and the PRIMARY BANKING PATH -- every wave, the grinder, the
orchestrator, bulk_harvest -- remained structurally unable to bank 250 of ov_SC01_077's 263 stubs.
WHY IT SURVIVED 26 PHASES: harvest_verify cannot splice a draft whose stub is not in the TU it was
pointed at, so the draft never verifies -- and is then logged as near/failed, i.e. AS A MATCHING
PROBLEM. The wave reports a poor close-rate; the function goes to the backlog as a compiler residual.
A tool that CANNOT bank a function is indistinguishable, in every log this project keeps,
from a function that CANNOT BE banked.
PROOF, same draft / same gate / same second: gate_stage rejected func_80129C40; harvest_verify run
directly (no --src) VERIFIED it byte-identical and banked it.
AND A COUNTING BUG THAT HID THE HIDING (gate_stage:261): when match_one says MATCH but the whole-binary
gate rejects, the record is logged status="near" and THE COUNTER IS NEVER INCREMENTED. A 63-draft run
printed `banked 0, near 0, failed 0` -- three zeros that do not sum to 63 -- for phases. Nobody ever
added them up. (The number was not wrong. It was ABSENT.)
ALSO FIXED, sig_unify (the same disease, one level down): it SILENTLY DROPPED 190 of 196 drafts (97%).
`cur_stubs` was read from the main .c (13 of 263 stubs), so any draft whose stub lives in a _jr_ carve
hit `if fn not in cur_stubs: continue` -- dropped BEFORE THE WRITE: never copied to --out, never gated,
never logged, while the summary printed "drafts unified: 6" and read like success. THIS IS GATE_STAGE'S
STAGE-2 RECOVERY -- the pass whose whole job is to rescue the stage-1 failures -- and it has been a
no-op for nearly every draft it was meant to save. Now: TU derived per draft (corpus.stubs), canon
derived from cdecl.tu_scope (cpp -- macro-injected decls finally visible), and _keep() so an
already-acceptable decl is left alone (the §19 "sig_unify regresses canonical drafts" failure mode).
Reach: 6 -> 196 drafts; callee-externs rewritten 2 -> 90; own def-sig 2 -> 86.
MEASURED, all three consumers migrated (196 never-banked drafts):
near 5 -> 116 failed 190 -> 17
=> 173 of 190 "failures" were PLUMBING, not codegen: now compiling and SCORED instead of invisible.
THE PRIZE (measured, not claimed): the backlog holds 1,588 entries at closeness==0 -- body byte-exact
per match_one, whole-binary gate rejected. 1,215 have been banked since by other paths. 373 ARE STILL
OPEN STUBS WHOSE BODIES ARE ALREADY BYTE-EXACT, sitting in a ledger that calls them unrecoverable.
⚠ THE HARVEST ITSELF IS NOT IN THIS COMMIT, AND IS NOT CLAIMED (P9). Gating the 63 ov_SC01_077 ones
dragged `dedup_propagate --auto-from --recover` behind it; it ran >1h and hit its timeout -- its
first-ever run over the FULL corpus (A6/A7 unblocked the 407 files it could never see). It MUTATES THE
TREE BEFORE IT GATES, so the kill left 859 files + engine_core.h (+544 lines) written and UN-GATED with
the registry never updated. R22 on that tree: 44 passed / 92 FAILED -> `git checkout -- src/ config/`,
fleet restored to 136/136. Nothing lost (H4: the tree was clean, so the revert was one command).
Two real lessons, recorded: dedup_propagate is NOT crash-safe and must never run under a timeout it can
hit; and a 63-draft experiment must not drag an unbounded fleet-wide propagation behind it.
R22 clean-fleet after revert: 136 passed, 0 failed of 136. src/ and config/ clean.
cookbook §51g LAW 11: A FIX IS NOT LANDED UNTIL ITS CALLER STOPS OVERRIDING IT. After fixing a
scanner, grep every call site and ask whether a caller's default re-disables it. An audit that stops
at the callee is half an audit.
|
||
|
Drew T
|
4aae5e7589 |
fix(phase-26a): A3d — retire the fleet-majority oracle: it was WRONG for the TU 16% of the time, on both banking paths
R33 applied to the worst finding in the audit: this oracle was not fixed, it was RETIRED.
reconcile_decls asks "what does the FLEET call this symbol?"
C asks "what does THIS TRANSLATION UNIT declare?"
The engine is loosely typed -- the same address is legitimately declared with incompatible types in
different overlays -- so a single fleet-wide answer is WRONG FOR SOME TU BY CONSTRUCTION. And it is
worse than a silent skip: it writes an ACTIVELY WRONG declaration into the draft, which then
collides with the very TU it was meant to conform to.
MEASURED across ov_SC01_077's 12 TUs, against what cpp says each TU really declares:
the fleet oracle AGREES with the TU ................ 2883
the fleet oracle CONFLICTS with it (cc1 REJECTS) .. 548 <- 16%
the TU declares it, the oracle has NO answer ...... 357
and it was LIVE ON BOTH BANKING PATHS:
* gate_stage -- rewrote 60 of 196 drafts in the current batch
* jtbl_family_bank -- EVERY SIBLING of the ×134 family sweep, the project's economic engine.
A poisoned decl means that sibling silently does not bank, and the loss is invisible: the sweep
simply reports a smaller number. The irony is exact -- that function's own docstring already
knew the conflicting symbols are PER-OVERLAY, which is precisely why a FLEET oracle could never
have been right.
reconcile_tu.py (written in Phase 26 but NEVER WIRED) now supersedes it, rebuilt on cdecl:
* ask cpp what the TU declares (macro-injected DEFINE_func_* externs included -- a raw scan
cannot see them, §8c / §51g LAW 7);
* ask cc1 whether the draft's decl can coexist (cdecl.compatible, validated against the real
gcc-2.7.2 front end on 1,485 live pairs -- NOT the C standard, NOT modern gcc; §51g LAW 9);
* NOT declared -> leave the draft alone (its extern types are load-bearing: %lo-folding, access
width, alignment); compatible -> nothing; CONFLICTING -> the TU wins + cast at every USE so the
draft's intended access survives byte-for-byte;
* derives WHICH TU from corpus.stubs() rather than a hand-passed --src-file (§51g LAW 10).
* handles the fn-ptr kind NATIVELY -- which is why it supersedes rather than patches: teaching
reconcile_decls' parser to see `extern void (*D_x[])(void);` would have ARMED its fn-ptr-blind
data_access_subs to rewrite a call-through `D_x[i]()` into `((u8 *)D_x)[i]()`. Fixing the regex
would have detonated a dormant bug.
AND THE NULL RESULT, AGAIN, REPORTED AS SUCH (P9/R14): on the 196 never-banked historical drafts the
new oracle banks EXACTLY AS MANY AS THE OLD ONE -- zero. That tail fails on CODEGEN, not on decl
plumbing. The two disagree on 45 of 196 drafts and the outcome does not move. This is a CORRECTNESS
fix (548 wrong declarations removed from two live pipelines, protecting all FUTURE drafts and every
future family sweep), not a banking win, and it is not being sold as one. Three nulls in one session.
reconcile_decls.py is kept as EVIDENCE, marked RETIRED, with no live caller.
R22 clean-fleet: make clean + extract-all + check-all -> 136 passed, 0 failed of 136
src/ untouched (0 changes) reconcile_tu: 0 coverage defects over 196 drafts
NOTE: the family-sweep path gets its real exercise at Task 8 -- watch the per-sibling bank rate.
|
||
|
Drew T
|
f4502f11bb |
fix(phase-26a): A3c — the recovery passes were reconciling 95% of drafts against the WRONG TU
FIRST CONSUMER MIGRATION onto the cdecl oracle — and the compiler taught me two things I had
wrong, one of which reopens a wall that has been closed since Phase 15.
1. cdecl.compatible() — "will cc1 accept these two declarations of one name?"
The predicate four tools each half-implement and get wrong: norm_sig / _norm_type collapse the
int family to ONE token, so a SIGNEDNESS change reads as "already compatible" and gets no
rewrite -- while cc1 REJECTS that redeclaration. Right about codegen, wrong about the front end,
which never reaches codegen.
2. THE ADJUDICATOR MUST BE THE COMPILER THAT COMPILES YOUR CODE (cookbook §51g LAW 9).
I wrote the rules from the C standard, then let a compiler judge. It contradicted me -- and then
the RIGHT compiler contradicted the first one. Three different answers:
declarations in one TU | standard | modern gcc | gcc-2.7.2 cc1
typedef int X; twice | error | ACCEPTS | ERROR
extern u16 X; + volatile u16 X| error | error | ACCEPTS
void X(s16); then void X(); | error | error | ACCEPTS
void X(); then void X(s16)| error | error | ERROR
--compat now adjudicates with tools/bin/gcc-2.7.2-psx/cc1, the front end that actually
arbitrates the build: 1,485/1,485 live corpus pairs agree, 0 disagree, 0 skipped.
3. THE PRIZE: the Phase-15 narrow-param wall rests on a false premise.
The no-prototype rule is ORDER-DEPENDENT. `void X(s16); void X();` COMPILES; only the reverse
fails. Phase 15 closed "the 159 arity/narrow-param conflicts" as "no clean deterministic fix --
it is simply C's default-promotion rule". cc1 does not enforce that rule in the direction the
wall assumed. Four three-line probes, 90 seconds, zero tokens. -> A10 RE-TEST TARGET.
Probe the compiler for FACTS; read its source only for LEVERS; byte-validate both. (We read
gcc-papermario for five phases believing it was 2.7.2. It was 2.8.1.)
4. THE MIGRATION: cast_call_sites canonicalized 95.1% of drafts against a TU that would never
compile them. `--src-file` is an OPTIONAL HAND-PASSED flag defaulting to src/<ov>/<ov>.c, and no
caller knows about the Phase-26 _jr_<ADDR> carves: ov_SC01_077 has 263 open stubs across 12 TUs
and only 13 are in the main .c -- while harvest_verify (A3) correctly splices into the real one.
Now DERIVED from corpus.stubs() (the INCLUDE_ASM line is self-describing), with the canonical map
derived from cdecl.tu_scope() (cpp -- so macro-injected DEFINE_func_* decls are finally visible).
Callee-conflict repair reach: 8 -> 58 of 196 drafts (7x).
5. AND THE NULL RESULT, REPORTED AS SUCH (P9/R14). Those 58 banked ZERO functions. The historical
draft tail fails on CODEGEN, not plumbing -- func_801387B8, which the audit blames on a single
unparsed `[4]`, is really 67/100 instructions off with a $s0/$s1 swap (that claim does not
reproduce on today's tree). The real gain is narrower and still worth having: 52 drafts moved
from "won't compile" to "compiles, N instructions off" -- from an INVISIBLE failure that reads as
a compiler wall into a SCORED near-miss the permuter and the §47/§48 dials can act on. That is
the audit's thesis, not a bank. THREE times in one session a confirmed mechanism produced a null
consequence.
Also: my own new audit printed "ALL ORACLES GREEN" while silently skipping 100% of its corpus (a
missing -Isrc). The exact bug class, in the tool written to hunt it. An unadjudicable check is not
a passed check.
R22 clean-fleet: make clean + extract-all + check-all -> 136 passed, 0 failed of 136
src/ untouched (0 changes) make audit-cdecl: green --compat: 1485/1485
NEXT: sig_unify + reconcile_decls carry the SAME wrong-TU bug (same --src-file flag).
|
||
|
Drew T
|
f9742cf9c0 |
feat(phase-26a): A3b — cdecl.py, THE C-declaration oracle: one grammar, fifteen deleted models
Fifteen tools each carried their own regex model of "what is a C declaration", and they
disagreed — two tools in ONE pipeline disagree today about whether `extern s32 D_a, D_b;`
is a declaration at all. All fifteen shared one character class,
extern\s+([A-Za-z_][\w\s\*]*?\bD_[0-9A-Fa-f]+\s*(?:\[\s*\])?)\s*;
which cannot hold '(', ',', or a non-empty [N] — so three whole shapes were invisible to
every one of them: fn-ptr/jump-table arrays, sized arrays (one unparsed `[4]` has blocked
func_801387B8 in 134 TUs), and multi-declarators (the WHOLE line dropped, not just #2..N).
REJECTED the audit's own prescription (a shape-aware alternation per tool, ~15 coordinated
regex edits) on R33 grounds: fifteen hand-maintained models are exactly what diverged, and
an alternation only ever covers the shapes somebody remembered. The thing being scanned HAS
A GRAMMAR. C's declarator grammar is small, closed and TOTAL — it describes fn-ptr arrays,
sized/2-D arrays, multi-declarators, fn-ptr params and K&R identifier-lists without being
told they exist. ~250 lines of recursive descent: LESS code than the regexes it deletes, and
exhaustive by construction rather than by memory. (decision-log 2026-07-14.)
Two statement paths, because the inputs genuinely differ:
* tu_statements() - a TU's file scope, derived from cpp. A decl inside a DEFINE_func_*
macro body declares NOTHING until the macro is invoked (the §8c law);
a raw scan is wrong in both directions. cpp answers it exactly, in
54 ms/TU (~20 s for the fleet, cacheable).
* split_statements() - span-preserving raw split, for drafts (which get rewritten).
THREE ORACLES, whole corpus — a measurement, not a belief:
* coverage 2,952,246 depth-0 statements -> 2,731,521 declarators, 0 PARSER DEFECTS
* the real gcc 50,405 distinct declarations compiled beside this parser's reconstruction
of each one -> 0 REJECTED
* differential 0 file-scope symbols the incumbents see that cdecl misses; 26 in
engine_core.h they cannot see; 6 they wrongly promote from BLOCK scope
Two ideas worth keeping (cookbook §51g, LAWS 4-8):
* THE CANDIDATE SET IS DERIVED TOO (R33 applied to R32). At file scope C admits nothing but
declarations, so R32's over-approximating detector is *every depth-0 statement* — supplied
by the grammar, with no hand-maintained candidate regex to rot.
* GCC ADJUDICATES MY OWN COVERAGE GAP. Deciding for myself which failures "don't count" is
grading my own homework — the habit that wrote the fifteen bugs. A statement gcc ALSO
rejects is not C (my rejection is correct, the INPUT is corrupt); one gcc ACCEPTS and I do
not is MY defect. All 33 residual: NOT-C, all dead .run/drafts* scratch, none in src/.
NEW findings (docs/tooling-audit.md):
* reconcile_decls.DATA_DECL_LINE_RE finds ZERO decls in engine_core.h — it is line-anchored
and every decl there ends in a '\'. Its "authoritative tier" has ALWAYS been empty.
* gen_harvest_targets + sig_unify count BLOCK-SCOPE externs (6, byte-proven inside a macro's
function body) as file-scope canonicals — the §8d `conflicting types` confusion.
* tu_ambient's func regex ([^()]* params) drops ANY callee with a fn-ptr parameter.
* R14 near-miss: 33 drafts contain `extern if ((func_80029178(0x119) & 0xFF) != 0);`, written
by a RECOVERY TOOL — but the source bug was already fixed in Phase 19 (0 garbage / 300 sigs
today). Mechanism confirmed, consequence nil. Note what it cost while live: a draft that
cannot compile fails the byte-gate and reads downstream as an INTRINSIC COMPILER WALL.
Bugs the oracles caught in ME (and would otherwise have shipped): `extern s32 (*D_801274D0)(s32);`
parsed the BASE TYPE as the name; a K&R declaration-list flushes as SEVERAL spans, so the body
attached to the wrong one and leaked the K&R parameter names into file scope as fake globals.
SCOPE, deliberate: NO consumer is migrated here, so this cannot move a byte. The audit warns
that making the parser see more ARMS dormant transforms (reconcile_decls.data_access_subs would
mangle `D_1[i]()` -> `((u8 *)D_1)[i]()` the moment fn-ptr decls become visible to it). Migration
is one tool at a time, each byte-gated.
R22 clean-fleet: make clean + extract-all + check-all -> 136 passed, 0 failed of 136
make audit-corpus: 0 PHANTOM + 0 TRUNCATED make audit-cdecl: ALL ORACLES GREEN (new gate)
|
||
|
Drew T
|
2086b15b48 |
fix(phase-26a): A8 — jr_isolate_all silently dropped 683 prototypes: a LATENT BYTE-CHANGER
_file_scope_decls() hoists a region's file-scope decls into the carried layer. Its _SAFE_TYPE guard
only ever whitelisted BUILTIN base types — so a decl naming a carried file-local type was recognised
by _HOIST_RE as hoistable and then SILENTLY DROPPED.
THE COMMENT ON _SAFE_TYPE DESCRIBED A FIX THAT WAS NEVER APPLIED TO THE CODE. Verbatim: "one naming a
FILE-LOCAL type is only safe once that type is carried too — which file_scope_types() now does, so such
decls ride along after their typedef." The predicate never implemented it.
MEASURED (audit: 4,040 fleet-wide; independently re-measured here over 4 overlays: 189 drops, 32 of
them function PROTOTYPES):
* The 3,357 dropped DATA externs are LOUD — an undeclared identifier is a compile error.
* The 683 dropped function PROTOTYPES are NOT. In C89 an undeclared function is implicitly `int f()`,
so the TU still COMPILES — with the wrong return type and lost pointer-ness. And this project has
BYTE-PROVEN that the return type drives codegen (cookbook: "schedule — delay-slot fill via void
return type"; ov_SC01_077_after.c carries an `extern int`->`extern void` flip described as
byte-neutral precisely because the return type moves the delay slot).
=> A DROPPED PROTOTYPE IS A SILENT BYTE-CHANGER, armed to fire on the next carve.
Today's split is green only because the source redundantly re-declares externs per fn-group, so most
items happen to carry their own decl. That is luck, not design.
Two of the rejected base types were not even file-local: `uint` (139 drops) and `code_fn` (21) are
DEFINED IN src/shared/engine_types.h, which engine_core.h pulls into every region — the predicate was
rejecting INCLUDE-PROVIDED types it had no reason to reject. `volatile` (3) fell off because the
qualifier group had `const` but not `volatile`.
FIX: implement what the comment promised. A decl is hoistable if its base type is a builtin, OR is
carried by this layer's own file_scope_types, OR is provided by the shared headers (108 type names
parsed from engine_types.h + common.h — including FN-PTR typedefs, whose name sits inside the parens
and which every name-before-';' pattern misses; those were exactly the 5 residual drops).
COVERAGE ASSERTION (R32): a line _HOIST_RE recognises as hoistable but that cannot be placed is now a
HARD FAILURE with the base-type histogram printed, not a silent no-op. Verified safe: 0 residual drops
across 5 overlays. This one check would have surfaced all 4,040 the day the first split shipped.
No build impact (the isolator runs only when carving); --dry-run clean; tree unchanged.
|
||
|
Drew T
|
82d79e7a32 |
fix(phase-26a): A6/A7 — the family engine could not see half its corpus; 17 fns banked x134 free
R22: check-all 136 PASSED / 0 FAILED. dedup-check 1823 validated / 0 failed (C1 coverage 224,933/224,933).
Fleet instr-weighted 66.5% -> 66.7%.
=== dedup_propagate: it was blind to HALF the corpus ===
overlay_files() used a hardcoded suffix allowlist ("_a","_o0","_o0b","_after") that predated the
Phase-26 jr carves -> 404 of the fleet's 811 overlay .c. The 407-file gap held 36,135 INCLUDE_ASM stubs
and ~32,000 inline defs, and overlay_files gates ALL of dedup_propagate (source_text / find_site /
apply_plan / struct_check / reconcile_caller_extern). Now a GLOB — never an allowlist, because the NEXT
split family would re-open it. The asm_subdir is always the file stem, an invariant the old four entries
already satisfied.
find_site's def-detector required the signature line to END in ')' and the next non-blank line to START
with '{'. It therefore silently dropped THREE shapes: K&R definitions (`s32 f(arg0)` / `s32 arg0;` / `{`),
multi-line signatures, and single-line bodies. K&R is the project's house style for exactly the biggest,
highest-reach functions — func_8015AE2C (562 ins), func_80166994, func_80133CD4, func_8015A3C8 — and they
live in the _jr_* files overlay_files could not even open. Fixing either alone would have been useless:
the glob exposes the files, and find_site would still drop their biggest prizes. Both fixed together.
* The signature's closing paren is now found by a real paren-walk, not line.count() or split(')')[-1]:
a single-line body containing a call (`void f(int a){ g(a); }`) has balanced parens of its own, so
both shortcuts land on the WRONG paren and then misread the body's ';' as a prototype terminator.
* AGREEMENT ASSERTION (the audit's): find_site vs family_remap.extract_unit -> 701 agree / 0 disagree.
Negative controls hold (a prototype+call is rejected; a 1-line body with a call is a def).
=== THE HARVEST (free work, byte-gated) ===
--auto-from ov_SC01_077 now nominates what it could never see: 20 planned, 17 propagated x134, 3 dropped
as cross-overlay stragglers. 134 overlays rebuilt BYTE-IDENTICAL; 17 new dedup groups.
Includes ALL FOUR functions A1 caught the registry lying about (func_80128ED8 / 8012C098 / 8012C0EC /
8012C750): 0 stubs remaining, real shared macros. THE LOOP CLOSES — A1 found the lie, and THIS is the
bug that had made it true (3 of the 4 are defined in ov_SC01_077_jr_8012ACE0.c, which the allowlist could
not open, so the propagation never ran and dedup_integrate greenlit the result).
=== family_remap: 96 PHANTOM exemplars -> 0 ===
extract_unit globbed only src/<ov>/<ov>*.c, so a function matched via a SHARED body had no source form
and read as NOT MATCHED. 93-96 of 218 h_seq "matched" exemplars were phantom, carrying 2,157 candidate
members of which 1,834 are still-stubbed, PURE/IMM-clean, symbol_map-clean and unpinned — staged and
gated today, dropped before the first build then. It is now TOTAL over BOTH shared-body mechanisms:
(1) the DEFINE_func_<ADDR>() macro — reconstructed as the exact INVERSE of dedup_propagate.make_macro
(derived from the generator, not re-guessed from the text);
(2) a DIRECT definition in a shared header, #included per overlay — the whale (func_80144B9C, 770 ins,
-O0), which the registry explicitly records as "NOT a DEFINE_ macro".
CENSUS: 216 matched exemplars, 216 real, 0 PHANTOM.
symbol_map named the symbol by HOW IT WAS LOADED, not by WHAT IT IS: reloc_targets labels every lui/%lo
pair "data", and a FUNCTION's address taken via lui/%lo (an address-taken callback) is exactly that shape
(splat's own .s: %lo(func_8017E1D4), 7 occurrences). The map got a D_<ADDR> key while the C writes
func_<ADDR>, so the word-bounded substitution matched NOTHING and silently no-op'd — the sibling kept the
EXEMPLAR's function pointer and the loss was booked as a BYTE failure, indistinguishable from a compiler
wall. Now emits both keys (addresses are unique; the pass is simultaneous, so the extra key is free).
gather_externs was line-oriented, so a WRAPPED comma extern was invisible in both directions (the first
line has no ';', the continuation has no `extern`). ov_SC01_077.c:271-272 declares NINE symbols that way,
and the exemplar referencing them (func_8013D178) is a 133-member family — every sibling was staged with
NO declaration, failed to compile, and bisect-stormed its whole gate group. Now statement-oriented, and
an unresolved symbol is REPORTED, never silently dropped.
=== family_sweep.stub_map / build_engine_types ===
stub_map: func_-only -> a curated-name stub read as "already matched" -> phantom exemplar. Now corpus-derived.
build_engine_types hard-exited on 1,070 of 1,470 type-bearing overlay .c (73%; the audit measured 573/709
= 81% on its narrower set) because 1,929 TAGGED-struct typedefs tripped a guard whose own comment asserts
"our source has only ANONYMOUS-struct typedefs" — true in Phase 20, false since the harvest agents started
writing tagged structs. inject_capped_externs routes every type-bearing body HERE as the type-heavy tail's
ONLY sanctioned unblocker, so the tail's unblocker could not run on the corpus the tail lives in.
A contained def (the typedef's span encloses the body) is liftable — it just must not be counted twice;
only a PARTIAL overlap is malformed. Verified on a file that used to hard-exit: 5 tagged typedefs folded +
forward-declared, 46 types written, exit 0.
** AND THE SHARPEST LESSON IN THE AUDIT: this one was never silent. It printed "[overlap] ... handle
manually" every single time. But the message reads like a rare edge case rather than a four-fifths
coverage failure, so nobody ever COUNTED it. A loud failure that nobody counts is exactly as
invisible as a silent one. R32 must be "assert your coverage", not merely "fail loud". **
R14 self-catches, recorded because I hit both while fixing them: my first shared-header scan read a macro
body's `extern void f(void); \` as a DEFINITION (the trailing continuation means the line does not end in
';', so the decl guard never fired) — the exact bug fixed at commit:0552, reintroduced by me and caught only
because the whale resolved from the WRONG file. Column-0 anchoring fixes it by construction. And my
phantom census returned 0/0 twice because I guessed the manifest schema instead of reading it.
|
||
|
Drew T
|
af2f40d153 |
fix(phase-26a): A4/A5 — 193 unmatchable slices dissolved; the closeness oracle stops lying
R22 CLEAN-FLEET: make clean -> extract 136 -> build 136 -> check-all = 136 PASSED, 0 FAILED.
make audit-corpus: 0 PHANTOM + 0 TRUNCATED (was 193).
=== A4: a CORPUS defect the byte-gate could never have caught ===
config/symbols.us.txt:981 declared `listCdBuffer = 0x80180000` — a correct Phase-3 name for MAIN's
LIST.CD RAM buffer. But that address is OUTSIDE main's image and INSIDE the overlay slot, and every
overlay's splat config stacks symbols.us.txt. High RAM is REUSED: an address that is a buffer to main
is live CODE to an overlay. So splat saw a symbol boundary mid-code and, across 97 of 134 overlays:
* CUT 97 REAL FUNCTIONS IN HALF (a head ending on a `lui`, no return), and
* INVENTED 96 PHANTOM ONES (a tail beginning by reading the assembler temp $at).
193 slices NOBODY COULD EVER MATCH — not "hard", not "a compiler wall": unmatchable by construction.
They sat in the harvest queue as ordinary work, so agents would burn on them forever and the failures
would be filed as intrinsic compiler residuals.
The phantom listCdBuffer.s in ov_SC01_005 literally begins:
lw $ra, 0x10($sp) / addiu $sp, $sp, 0x18 / jr $ra
splat cut a function immediately before its EPILOGUE and called the epilogue a function.
AND IT HAD ALREADY CONTAMINATED REAL WORK: in ov_SC03_031 the cut landed where the epilogue was
exactly `jr $ra; nop`, so the Phase-26 x134 sweep innocently BANKED the phantom as
`void listCdBuffer(void) {}` — byte-correct, gate-green, entirely fictitious — while leaving
func_8017FFC4 permanently unmatchable. Removed.
WHY NO GATE CAUGHT IT, AND WHY THAT IS THE POINT: INCLUDE_ASM pastes the two .s halves back VERBATIM
in original order, so the image is byte-identical either way. The byte-gate was green the whole time
and always would have been. It is a perfect CORRECTNESS oracle and a NULL COVERAGE oracle. No
assertion added INSIDE it could ever have found this. What found it was a SECOND, INDEPENDENT oracle:
tools/sig_image.py derives boundaries from the ORIGINAL bytes without splat, and DISAGREED with the
corpus (58,524/58,621 agreement with spimdisasm; correct on all 97 disagreements).
=> When one oracle is structurally blind to a class of error, the answer is not a better assertion
inside it. It is a SECOND ORACLE THAT CAN DISAGREE WITH IT. (`make audit-corpus` is now that.)
THE RULE (the mirror of R13/R15, never written down): a symbol whose address falls inside ANOTHER
binary's vram window must never enter that binary's symbol stack.
FIX: config/symbols.us.ram.txt — main-scoped symbols outside main's image — stacked ONLY by
config/splat.us.exe.yaml. Main keeps the name it needs (10 %hi / 11 %lo refs; 143dbb89 byte-identical);
the overlays never see it. Exactly one symbol was in scope fleet-wide; the resident window was clean.
AND A REAL FUNCTION THE ACCIDENT WAS HIDING: in ov_SC01_084 / ov_SC02_041 / ov_SC03_094 / ov_SC06_008
there IS a genuine function at 0x80180000 (111 / 35 / 28 / 74 ins), reachable ONLY via a fn-pointer
table (.word func_80180000) and never by `jal` — so splat cannot find it and needs the boundary
DECLARED. listCdBuffer had been supplying it by luck. Now declared honestly, per-overlay, in
config/symbols.<ov>.txt — exactly where R13/R15 says an overlay-scoped symbol belongs.
=== A5: the closeness oracle every crack agent trusts was lying on 155 functions ===
masked_diff._reloc_kind() knew 26/HI16/LO16. An over-approximating sweep of every reloc objdump emits
across all 3,367 build objects found FOUR: R_MIPS_26, HI16, LO16 — and R_MIPS_PC16 (211). PC16 fell
through to a FULL-WORD compare, but the object holds an UNRESOLVED PLACEHOLDER in the branch
displacement, so that compare can NEVER succeed.
DECISIVE TEST (derived from the invariant, not from reading the regex): INCLUDE_ASM pastes the
ORIGINAL asm, so for every stub diff_object_s() MUST be 0. Measured, coverage-asserted:
2,741 functions scored — old mask: 150 LIES; PC16 masked: 4 LIES.
(The 4 survivors are the separate length-delta defect.) A phantom non-zero sends an agent to grind at
a wall that is not there, and the wasted attempt is then booked as a MATCHING failure, feeding
reserved_walls() and PERMANENTLY BLACKLISTING a function that was never broken.
=== NEW FINDING (found by cutting the R22 corner): a STALE OBJECT CAN PRODUCE A FALSE PASS ===
`.o <- .s` is not a dependency make can see: assembly arrives via INCLUDE_ASM, expanded to a `.include`
consumed by maspsx/as AFTER cpp, while -MMD tracks headers only. Re-extract, build incrementally, and
make links a STALE object. This is not merely slow — INCLUDE_ASM pastes the ORIGINAL bytes, so a stale
object still yields the original image: SHA1 GOES GREEN while the split just changed is never exercised.
A broken config change can be "verified" by an incremental build. Live proof: 8 of 136 binaries linked
stale objects here; they failed LOUDLY ONLY BY LUCK (the dead symbol was an undefined reference) — a
merely-different-but-valid split would have gone green on all 136.
R22/H3 already legislate this, and I broke them. But a rule that needs a human to remember it is not a
gate. FIX: `extract` now invalidates the objects that include what it just rewrote (main's are top-level,
so -maxdepth 1 — verified it cannot clobber the other 1,605 objects). Structural, not advisory.
R14 self-catch, recorded: my first A5 test passed `fn=` to diff_object_s(), which takes two args; the
TypeError was swallowed by my own `except Exception: continue` and it reported 0 scored / 0 lies. I
wrote the exact bug I was auditing, inside the test for it. Caught only because 0 looked wrong. The
test now asserts its own coverage.
|
||
|
Drew T
|
9794b13ed2 |
fix(phase-26a): A3 — the endgame plan was 2.8x too big; the matched set is now DERIVED
docs/family-manifest.md is the document the whole Phase-25/26 structural-family endgame was planned
from. Its matched-set oracle asked ov_SC01_077 ALONE: matched := {h_exact of that one overlay's
non-stub fns} | dedup hashes. So a function ABSENT from that overlay — or stubbed there but matched
in the other 133 — came out "unmatched" and was ranked as live work.
advertised real (derived)
multi-member families 2,758 -> 1,495
"hidden leverage" 11.0 MB -> 3.9 MB
matched-free lever 235/5.9 MB -> 57/1.0 MB
7.1 MB of the advertised leverage was DEAD WORK. And because `instances` counted every overlay
carrying a function — including the ones where it was already banked — the byte-weight RANKING (the
file's entire purpose: "draft these first") was sorted mostly on already-finished code, with the
real targets buried underneath. The A2 audit predicted "true frontier: 1,475 families / 3.9 MB";
derived independently here it is 1,495 / 3.9 MB.
R33: the invariant answers this with no oracle at all —
an h_exact class is WORK iff at least ONE of its instances is still an INCLUDE_ASM stub.
That also makes the dedup-hash union redundant (a dedup-shared member is by definition not a stub),
so the `hash:` regex over config/dedup.us.yaml is DELETED. `instances` now counts only the members
still to bank, so the leverage is the real x-N.
family_hseq: stub scan -> corpus (+100 curated-name stubs the func_-only regex could not see; they
had made 3 still-stubbed functions look like MATCHED exemplars, which every sweep then re-nominates,
produces nothing from, and books as a silent skip). Its hardcoded "expect ~663/~186/~1.85M" self-check
was a stale 2026-07-11 snapshot — 38 banking commits have landed since — and is now labelled a
point-in-time reference, not an invariant. (Verified my change can only GROW the frontier: +100 stubs.)
census_conflict_callees: scoped to src/<ov>/<ov>.c alone, so it saw 13 of 264 stubs and reported
"wave scope: 2 still-stub" when the truth is 57 — every downstream percentage computed against a
denominator 96% too small. Now 0/57 (the audit's exact figure). Its 0-conflict answer was right BY
LUCK; it is now right for a reason. MARKED FOR DELETION (R33): it re-derives from C text what
reconcile_tu.py answers from the build, and its parse holes fail in the UNSAFE direction (an unknown
callee is silently bucketed "conflict-free"). Delete once reconcile_tu is wired into its only consumer.
R14 near-miss, recorded: my first census patch handed collect_stubs() a set of NAMES where it wanted
ADDRESSES, so the membership test was always false and it printed 0/0. Caught only because 0
contradicted the audit's expected 57. A scanner that returns 0 is indistinguishable from a scanner
that found nothing — which is the entire thesis of this audit, and it very nearly bit me while
fixing it.
|
||
|
Drew T
|
70a99aa9f7 |
fix(phase-26a): A3 — the byte-gate could only see ONE translation unit (96.6% of stubs unreachable)
harvest_verify is the sole arbiter (G3/P9) and has never accepted a wrong match. It also could not
REACH most of the work: it scanned the single file the caller passed as --src and silently dropped
every draft whose stub lived elsewhere. An overlay's source spans up to 14 .c files, so:
open overlay stubs it could not see : 56,742 of 58,717 (96.6%)
ov_SC01_077 reachable by the gate : 13 of 264 (4.9%) -> 264 of 264 (100%)
THREE of the six callers passed no --src at all (orchestrator.py, grinder.py, idiom_hunt.py) and so
inherited gate_stage's `src = src or f"src/{binary}/{binary}.c"` default. For grinder.py that means
1,290 of its own 1,298 QUEUED FUNCTIONS COULD NEVER BANK, however good the permuter's output was.
=> Phase-22's "the permuter's fuel is exhausted" was never a safe conclusion. Re-test (A12).
gate_stage knew the right answer and then handed the gate the wrong file: its negative control
ALREADY globs every split .c to build bin_stubs. The default is now removed; --src is passed only
when a caller deliberately restricts the gate to one TU.
WHAT CHANGED, PRECISELY: only the SPLICE LOCATION. Each draft is now spliced into whichever TU
actually holds its stub, derived from tools/corpus.py. Every TU links into the same image, so ONE
`make build BINARY=<bin>` still gates them all — correct AND strictly fewer builds than the
per-split re-gate it replaces.
SAFETY (this is the byte-gate, so the argument is explicit): the VERDICT is untouched — `make build`
+ SHA1 == the locked hash. INCLUDE_ASM pastes the ORIGINAL assembly, so a wrong draft always changes
the bytes and always fails SHA1. A bug in the splice can therefore make the tool FAIL TO BANK; it
CANNOT make it falsely bank. The failure mode is conservative by construction.
VERIFIED end-to-end (2 real builds, tree clean before and after):
* discovery: 264 live stubs across 12 TUs (was: only those in the single --src file)
* IDENTITY known-answer test: 3 drafts whose stubs live in THREE DIFFERENT split TUs
(_jr_801734BC, _after, _jr_8012ACE0) — all discovered, spliced into their own files, built,
SHA-matched, committed, restored. Final SHA d19c9580 BYTE-IDENTICAL. Under the old code all
three were silently dropped as "not stubbed".
* `git checkout -- src/` recovers, exactly as the docstring promises.
Also derived rather than defaulted: --good-sha now reads config/check.<bin>.sha (a caller that
passed --binary but forgot --good-sha used to gate an overlay against RESIDENT's SHA), and
match_one_closeness resolves the asm subdir PER FUNCTION — one subdir for a whole batch is the same
single-TU bug, and pointing match_one at the wrong one scores a draft against a DIFFERENT function's
asm, producing a phantom non-zero closeness that lands in the backlog and feeds reserved_walls().
No committed source or config changed, so no build artifact can have moved; the byte-gate was
exercised twice and returned BYTE-IDENTICAL both times.
|
||
|
Drew T
|
a302908f24 |
fix(phase-26a): A3 — target selection was blind to 91.6% of the remaining work; now derived
The audit's CRITICAL finding, fixed at the root. Both tools now derive the corpus from
tools/corpus.py instead of keeping their own decaying copy of the tree layout.
build_fuel_manifest.live_stubs() — a hardcoded 3-file dict {<ov>.c, _a.c, _o0.c}. ov_SC01_077 has
FOURTEEN .c files, so it saw 30 of 264 stubs AND REPORTED SUCCESS. Everything downstream consumes
this manifest — worklist.py (100% of its rows), wave_targets.py (100% of its pools) — so:
targets 30 -> 263
reach-134 targets 10 -> 127 (the ENTIRE high-ROI band was invisible)
remaining gain 83,305 -> 994,633 instructions
994,633 is the A2 audit's predicted figure TO THE UNIT — a fourth independent confirmation
(auditor -> skeptic -> corpus.py -> this). Four of the five highest-leverage functions in the whole
project sit in split regions no tool could see; the top one, func_80178004 (165 ins x reach 134 =
22,110), had never been nominated by anything.
It rotted SILENTLY: .run/fuel_manifest.json (Jul 8) recorded 130 stubs; the same code today returns
30, because the Phase-26 jr splits moved ~100 stubs out from under a dict literal last edited in
Phase 22. Nobody noticed, because a target that is never nominated produces SILENCE, not an error.
wave_targets.REGION_SUB / asm_for() — a 3-entry dict with a silent fallback to the main subdir.
ov_SC01_077 has TWELVE asm subdirs, so 78 of the 87 targets any --class wave emitted handed a
drafter an asm path THAT DOES NOT EXIST. The drafter then drafts against nothing, and the wasted
attempt is booked in the backlog as a *matching* failure — which feeds reserved_walls() and
PERMANENTLY BLACKLISTS a function that was never actually attempted. A silent skip compounding into
a false wall. Now 263/263 asm paths resolve, 0 missing; asm_for() raises rather than guess.
Also: --region's 3-value whitelist defaulted to 'main', which sees 13 of 264 stubs even with a
correct manifest -> default 'any', free-form.
R33 throughout: the INCLUDE_ASM line is SELF-DESCRIBING (its first argument IS the asm subdir,
because splat wrote it there), so both dicts were second copies of a fact the tree already states.
A dict literal is strictly worse than the filesystem AND it fails OPEN. Never re-introduce one.
No build impact (selection/report tools only); docs/worklist.md regenerated with the honest numbers.
|
||
|
Drew T
|
f7b7399ebe |
feat(phase-26a): A3 — tools/corpus.py, ONE derived corpus oracle (+ a second oracle that can disagree)
The 28 surviving audit findings collapse to ONE bug repeated ~10 times: a hand-maintained model of
the corpus layout (a file allowlist, a single-.c assumption, a func_-only symbol regex, a REGION_SUB
dict) sitting on top of a filesystem that already answers the question. The fix is not ten repaired
regexes — it is one DERIVED oracle and ten deleted scanners (R33).
WHAT IT DERIVES FROM
1. THE FILESYSTEM. Which .c files make up a binary, and where a function's .s lives, are FACTS OF
THE TREE THAT SPLAT ITSELF WROTE. The INCLUDE_ASM line is SELF-DESCRIBING — its first argument
IS the asm subdir — so there is nothing to guess and no dict to rot. A dict literal is strictly
worse than the filesystem AND it fails OPEN (silently yields a wrong path) instead of closed.
2. THE PROVEN INVARIANT. INCLUDE_ASM pastes the ORIGINAL asm and the build is byte-identical, so a
function NOT wrapped in it is byte-exact. `matched` is DERIVED as sig - stubs, never re-parsed
from C text. (progress.py learned this the hard way: weighted_metrics() derived and was right;
classify() re-parsed C and inherited a bug.)
VALIDATED against the real corpus:
* ov_SC01_077: 264 stubs across 14 files. The old 3-file allowlist saw 30.
* Fleet: 58,717 stubs vs the allowlist's 1,992 — 56,725 (96.6%) were INVISIBLE.
* Coverage-asserted (R32): every INCLUDE_ASM line must parse, every symbol must resolve (ANY C
identifier — a func_-only regex silently misses the 100 curated listCdBuffer stubs), every stub
must have a .s. A silent skip is a DEFECT, not a no-op.
THE SECOND ORACLE (`make audit-corpus`) — the real lesson of this audit.
The byte-gate is structurally BLIND to a bad function boundary: the .s halves are pasted back
verbatim in original order, so the image stays byte-identical and green. Only an oracle that can
DISAGREE can see it. sig_image is that oracle — Ghidra-free, derived from the ORIGINAL bytes,
independent of splat. corpus.audit() cross-checks the two and reports:
PHANTOM — a stub address the sig does not know: splat INVENTED a function.
TRUNCATED — a stub whose .s length != the sig's: splat MIS-SLICED one.
It reports 193 (96 + 97) — reproducing the A2 audit's number EXACTLY, from an independently written
tool. That is a third confirmation of the listCdBuffer defect (auditor -> skeptic -> this).
AND AN R14 SELF-CATCH, recorded because the near-miss is the lesson.
Run naively over all 136 binaries the same check reports 914 slices — 4.7x the truth. It is noise:
main/resident are signed by the GHIDRA dumper, whose boundaries are shorter than splat's by design
(and which never analysed the linked PsyQ subsegs at all), so the comparison measures GHIDRA'S limits,
not splat's errors. Only the overlays are signed by sig_image, the oracle actually validated at
58,524/58,621. sig_is_independent() now encodes that domain, with the reasoning, so nobody repeats it.
A check applied outside its valid domain does not become more thorough — it becomes noise.
`make audit-corpus` is RED by design until A4 removes the bad symbol line; then it becomes a gate.
|
||
|
Drew T
|
bb65d36341 |
fix(phase-26a): A1 — dedup_integrate was a gate that could print a FALSE GREEN
The audit's priority #1: a fail-closed byte-honesty validator whose silent skips nothing downstream can catch. Three false-green paths, all measured, all now fail-closed with negative controls. R33 FIRST (derive, don't re-derive). The registry makes two claims; the tool only ever checked one, and mis-described that one: C1 EQUIVALENCE ("these vrams hold the same code in the ORIGINAL") — checked against the sigs, which sign the ORIGINAL bytes. KEPT. But the docstring claimed it also caught SOURCE drift: it cannot. A sig is a property of the ROM, immutable w.r.t. src/. Source drift is caught by the BUILD. Docstring corrected (P9). C2 BANK ("matched once in the source header, instantiated at every member") — NEVER CHECKED. Now DERIVED from the build invariant: INCLUDE_ASM pastes the ORIGINAL asm, so a member NOT wrapped in it is byte-exact, and one that IS wrapped is not banked — whatever the registry says. C2a: the group's macro token must occur in its source file. C2b: no member may still be an INCLUDE_ASM stub. THE THREE FALSE GREENS 1. 1808 groups claimed a DEFINE_func_* macro; only 1801 exist. The 7 ghosts printed [ OK ] — hiding 532 member-instances / 22,344 instructions of REAL, UNBANKED work (4 fns matched in ov_SC01_077, still INCLUDE_ASM in the other 133 overlays). 2. An absent .run/sig.<bin>.jsonl degraded to "0 validated, 0 failed" and EXIT 0. On a fresh clone the gate validated NOTHING and passed. Now fails; --allow-unsigned is the escape. 3. The bank claim was never checked at all. THE CAUSAL CHAIN (the audit's thesis in one example). 3 of the 4 hidden fns are defined in ov_SC01_077_jr_8012ACE0.c — a _jr_* split file. dedup_propagate.overlay_files allowlists only ("_a","_o0","_o0b","_after"), so the propagator could not SEE them; the group was registered anyway; dedup_integrate greenlit the lie. TWO silent-skip bugs compounding: one created the hole, the other hid it. Harvest fuel -> .run/audit/a1_harvest_fuel.json, banked in A5. BLAST RADIUS, MEASURED NOT PREDICTED (R14). Headline metrics UNCHANGED to the decimal (instr-weighted 66.5%, distinct-code 46.8%) — weighted_metrics() derives from the invariant and was structurally immune to the lying registry. FLEET REAL substantive unchanged (282,466): progress.py had already been taught to distrust it (commit:0574). Only dedup_integrate still believed it. A null result that CONFIRMS R33: the tool that refused to re-derive was the one that was right. - registry repaired: 1813 -> 1806 groups (7 ghosts removed; instances 223,725 -> 222,787) - make report GREEN end-to-end: 1806 validated, 0 failed | C1 coverage 222,787/222,787 signed - negative controls: stubbed member -> exit 1; missing sig -> exit 1; --allow-unsigned -> exit 0 - report-only tool: no compiled artifact depends on it, so no R22 clean-fleet is owed here |
||
|
Drew T
|
5b1de7acaa |
feat(phase-26): tools/reconcile_tu.py — ask "what can THIS TU see", not "what does the fleet call it"
WRITTEN + VALIDATED, DELIBERATELY NOT WIRED IN (inert; nothing imports it). Wiring + byte-gating is the
first item of the integration fix pass, AFTER the tooling-integrity audit Drew gated it behind.
The successor to reconcile_decls.py for the templating/banking path. Two things are wrong with that tool,
and the second is structural, not a typo (Phase-26 scanner audit):
1. BLIND TO FUNCTION POINTERS. DATA_DECL_LINE_RE wants `extern <type-words> D_x[];`, so the `(` in
extern void (*D_801DA75C)(void); <- fn-ptr scalar
extern void (*D_801812A4[])(void *); <- fn-ptr array (a dispatch table)
breaks its type run; the line never matches; the tool SILENTLY SKIPS exactly the symbols that are
failing and reports success. Blocking func_8017A4AC (536 ins x134 = 287 KB) today.
2. ITS ORACLE ASKS THE WRONG QUESTION. It elects a canonical decl by FLEET MAJORITY. But 34.4% of fleet
symbols carry >=2 mutually incompatible spellings, so a single fleet-wide answer is PROVABLY WRONG FOR
SOME TU BY CONSTRUCTION — and it is worse than a skip: it returns an ACTIVELY WRONG decl (measured:
3,717 symbols) that then collides with the very macro it was meant to conform to.
The only question that matters is what gcc compares the draft against: WHAT CAN THIS TU SEE. So reconcile_tu
reconstructs the TU's visible file-scope decl environment from BOTH §8c sources — col-0 decls AND the externs
INJECTED BY engine_core.h MACRO INVOCATIONS (a DEFINE_func_*() expands at file scope, so its leading externs
are genuine file-scope decls of the invoking TU, invisible to any col-0 scan: 544 visible syms from 1801
macros) — then conforms the draft's decl to it and CASTS AT EVERY USE (gcc folds a compile-time cast of a
known symbol, so the emitted bytes are unchanged; the whole-binary byte-gate remains the sole arbiter).
Carries a COVERAGE ASSERTION (R32): every line that LOOKS like an extern of a D_ symbol must parse, or it is
reported LOUDLY (--strict exits non-zero). A silent skip is a defect, not a no-op.
Validated read-only on func_8017A4AC: resolves D_801DA75C (fn-ptr -> `extern s32` + call-site cast),
D_80126B58 (struct), D_801DA734 (ptr).
|
||
|
Drew T
|
329ab4cbe7 |
fix(phase-26): progress.py — K&R definitions were invisible; phantom dedup members; + a COVERAGE ASSERTION
TWO REAL BUGS in classify(), and an HONEST CORRECTION of their blast radius (P9/R14).
- BUG 1 (under-count). classify() decides definition-vs-declaration by scanning to the first `{` or
`;`. A K&R definition puts its parameter declarations BEFORE the brace:
s32 func_8015AE2C(arg0)
s32 arg0; <- a `;` before the `{`
{ ... }
so it was read as a forward declaration and dropped into NO bucket — not REAL, not a stub,
invisible. And a K&R def is MANDATORY whenever a zero-arg engine_core.h thunk calls the function,
i.e. exactly the heavy-jr cores our own banking recipe produces: func_8015AE2C (562x134),
func_8015A3C8 (493x132), func_80166994 (369x134) were all compiled, linked and BYTE-IDENTICAL in
the shipped build while counting as zero. Fix: skip over K&R parameter declarations (a bare
`<type> <name>;` carrying no parens — that is what distinguishes it from a wrapped ANSI
prototype's continuation line, which always carries the `)`).
- BUG 2 (over-count). `real |= dedup_members(BINARY)` folded in EVERY registered dedup member without
checking it is actually instantiated. A member still sitting as an INCLUDE_ASM stub was counted
REAL *and* stayed in `stubs` — double-counting into `matchable` and inflating `byteident`
(532 phantom instances, per the scanner audit). Fix: subtract `stubs`. The registry is advisory;
the source tree is authoritative.
- COVERAGE ASSERTION (the rule ratified 2026-07-14): ground truth = every function splat emitted a
.s for. Anything classify() cannot place in ANY bucket is now reported LOUDLY (stderr + the .md),
because a silent skip is a defect, not a no-op. Currently: 0 unplaced.
- CORRECTION (this is the part that matters — I over-claimed and the bytes refuted me). The scanner
audit reported ~243k instructions "counted as nothing", and I repeated it. WRONG. weighted_metrics()
— which produces the HEADLINE instr-weighted and distinct-code numbers — does NOT call classify()
at all. It tests `func not in src_stubs(binary)`: since the fleet is 136/136 byte-identical,
anything not wrapped in INCLUDE_ASM must be compiled C emitting the exact original bytes. That test
never parses a definition, so it is IMMUNE to this bug. Verified: old-vs-new on the same tree gives
identical weighted numbers. The published 65.6% / 44.9% were CORRECT ALL ALONG; only the secondary
REAL count and fn-count % were wrong.
THE LESSON, sharper than the one we started with: a metric DERIVED FROM A PROVEN INVARIANT beats a
metric that RE-PARSES THE WORLD. weighted_metrics() leans on the byte-gate and inherits its
correctness; classify() re-derives the same fact by parsing C and inherited a bug instead. Prefer
the former wherever an invariant exists.
|
||
|
Drew T
|
6f3441d261 |
fix(phase-26): 10% of the canonical-callee oracle was silently missing (own-line-brace DEFINE macros)
- BUG: gen_harvest_targets.SIG_IN_BODY_RE required `)\s*{` between a DEFINE_func_* macro's signature
and its opening brace. When the brace sits on its OWN continuation line there is a line-continuation
BACKSLASH between them:
s32 func_80148824(void *arg0) \
{ \
and `\s` does not match `\`. So the regex silently dropped every own-line-brace macro.
- BLAST RADIUS (measured): 186 of 1801 engine_core.h shared signatures — 10% of the oracle — were
MISSING from the canonical-callee map that cast_call_sites / sig_unify / gen_harvest_targets resolve
against. A draft calling one of them kept its own guessed signature, hit `conflicting types` against
the TU's real definition, and the recovery pass reported nothing to fix — the failure looked like a
hard wall. This is why the crack wave's byte-exact cores would not bank.
- FIX: `[\s\\]*` instead of `\s*`. Oracle 2122 -> 2308 entries.
- PROOF: func_8015A3C8 (493 ins, MATCH standalone) went from "28 conflicting types, unbankable" to
BANKED ×1 BYTE-IDENTICAL at the `recovered` stage, with zero hand edits. R22 clean-fleet 136/136.
- This is the phase's SIXTH silent-skip bug and the THIRD of the same brace-placement class (§19
find_site; scope_data_externs' own-line brace; now this). Cookbook §40's standing lesson applies:
a tool that silently no-ops on input it cannot parse is indistinguishable from one that had nothing
to do — prefer fail-loud on unparsed input.
|
||
|
Drew T
|
cc08601ae7 |
feat(phase-26): func_80178D40 swept ×134 — the heaviest core in the game, fleet-wide
- 132/132 siblings banked (0 failures) via jtbl_family_bank --raw + the lazy-isolation chain. Each sibling: isolate -> jtbl carve -> remap from the raw crack -> stage ladder (raw -> scoped §8d -> recovered -> reconciled) -> WHOLE-BINARY byte-gate. - R22 clean-fleet 136/136 BYTE-IDENTICAL from `make clean`; 0 NON_MATCHING (G4). - METRICS: instr-weighted 63.8 -> 64.7%; distinct-code 40.7 -> 42.8% (+2.1 points from ONE core — 890 ins x 133 overlays = ~118K instructions of unique engine code); fn-count 82.43%. - tools/bank_exemplar.py promoted from scratch: bank a cracked EXEMPLAR ×1 through the same stage ladder jtbl_family_bank uses for siblings (carve/lazy-isolate -> raw/scoped/recovered/reconciled -> whole-binary gate). The exemplar path was previously hand-run each time. |
||
|
Drew T
|
07ebb5658d |
fix(phase-26): jr_isolate_all empty-region0 skip — cutting an already-isolated region's non-leader works
Cutting func_80178D40 out of ov_SC01_000_jr_801734BC adds the region's banked LEADER (0x801734BC) as a cut too (the one-carve-per-object rule), making region 0 EMPTY (the object's first item IS the first cut) — and region 1's derived name equals the object name, so emitting region 0 duplicated the line exactly -> splat "segments out of order". Skip an empty region 0; region 1 rightly claims the object's offset and name. First sibling then banks through the full chain (isolation validation green -> carve -> --raw remap -> stage ladder -> whole-binary gate): ov_SC01_000 BANKED, included here. The remaining 132 siblings sweep next. |
||
|
Drew T
|
7e4165676d |
fix(phase-26): isolation-residue corruption chain — config cleanup + revert() restores config + fail-loud validation + --raw sweep mode
Three-layer fix for the func_80178D40 ×133 sweep failures: - LAYER 1 (the residue): jtbl_family_bank.revert() restored carve pieces + src/ but NOT the isolation's CODE-subseg lines in the splat config. A failed bank attempt (BEBC's first try) left its isolation config in place; the successful retry re-isolated on top and a DUPLICATE `- [0x4b364, c, ov_SC01_000_jr_801734BC]` line rode into the commit (harmless to splat — zero-length — so R22 stayed green). revert() now also restores config/splat.<ov>.yaml. The committed duplicate is removed (ov_SC01_000 rebuilt BYTE-IDENTICAL 9052dc0e). - LAYER 2 (the detonation): jr_isolate_all walked the duplicated object TWICE -> two replacements -> a reversed duplicate block -> splat "segments out of order". It now VALIDATES the generated config (code subsegs strictly ascending, names unique) and refuses to write on violation, naming the likely cause — a corrupt input dies at the tool, not three tools later. - LAYER 3 (the sweep template): jtbl_family_bank gains --raw <crack.c> — template from the RAW crack via remap_hseq_body instead of the exemplar's banked source unit. REQUIRED when the exemplar banked at the `reconciled` stage: a reconciled body is TU-SPECIFIC (§41c — uniquified type names, TU-targeted casts), so extract_unit hands the sweep a polluted template and every sibling gate-fails (byte-proven: 178D40 banked reconciled -> sweep 0/4; 8015AE2C banked raw -> sweep 133/133). Same law as family_sweep --reconcile-raw. |
||
|
Drew T
|
a0e7ff7f6f |
fix(phase-26): scope_data_externs ANSI-brace fix + wire §8d into family_sweep --hseq; 780-class diagnosed
- _body_open_brace only matched a `{` on its own line (the K&R shape), so fix() SILENTLY NO-OP'D on
every ANSI draft — the same silent-skip disease as the four catalogued in §40/§8d, caught because
the h_seq re-sweep banked 0/780. Now brace-scans forward from the signature (ANSI same-line,
ANSI own-line, and K&R all work). Load-bearing for func_80178D40's upcoming ×134 bank.
- family_sweep --hseq now applies the §8d scoped stage at staging time.
- HONEST RESULT: still 0/780 — the substantial-band h_seq rejections are a DIFFERENT (sibling) class,
now fully diagnosed against the bytes:
gcc-2.7.2 decl-conflict semantics: a VISIBLE file-scope decl + a conflicting later decl (file OR
block) is a HARD ERROR; a limbo-only block decl (scope closed) + a conflicting later decl is a
warning. The h_seq drafts carry the EXEMPLAR TU's spellings; sibling TUs legitimately spell the
same symbol differently (loose typing), and the visible decl is often MACRO-INJECTED — a
DEFINE_func_* leading extern (§8c), invisible to any col-0 scan (e.g. D_80115158's `short` decl
enters ov_SC01_000.c via DEFINE_func_8014168C() @4637; the draft carries ov077's
`unsigned short` -> conflicting types at ANY scope).
Sub-class (a) no-visible-decl -> §8d demotion (the jr class, proven x133). Sub-class (b) visible
decl, different spelling -> needs reconcile-to-TU-VISIBLE (rewrite the draft decl to the TU-visible
spelling + byte-neutral access cast; oracle = col-0 decls above the stub + engine_core.h macro
externs for the DEFINE_ invocations above). Parked as a designed follow-up task; the 780 members
are mechanical-recovery fodder once the tool exists.
|
||
|
Drew T
|
cc6220eba4 |
fix(phase-26): extract_unit mistook m2c declarations for definitions (15 phantom exemplars)
- BUG: the guard `not ln.rstrip().endswith(";")` misses m2c's declaration form
`M2C_UNK func_80178D40(s32, s32); /* extern */` — the raw line ends in `*/`, not `;`, so a
DECLARATION was accepted as a DEFINITION and the forward brace-scan swallowed the NEXT
function's body, handing remap_hseq a garbage unit.
- BLAST RADIUS (measured): 15 of 35 substantial-family exemplars were phantom "matches" — all
still INCLUDE_ASM stubs (incl. func_80178D40 and the carried-queue func_801670E4); 3 more
anchored on the Phase-17 canonical-sig layer's `extern … /* match-first, arity N */` decls and
templated garbage, leaving those families SILENTLY UNBANKABLE. The whole-binary byte-gate
rejected every one — no wrong match was ever banked (G3/P9 held) — but the engine burned a
build per sibling and every extract_unit-based readiness analysis was wrong.
- FIX: strip trailing comments before the `;` test.
- REGRESSION-GATED over the whole corpus (6,286 family exemplars, .run/_eu_before.json):
15 phantom exemplars now correctly refused; 3 garbage units corrected to the REAL definition
(found in the right region file); 0 real definitions lost; 0 unit contents otherwise changed.
src/ and config/ untouched, so the committed build is unaffected.
- cookbook §40: the trap + the general lesson — this is the phase's FOURTH silent-skip bug
(find_site braces, overlay_files splits, reconcile_decls fn-ptr regex, now this). A tool that
silently no-ops on input it cannot parse is indistinguishable from one that had nothing to do.
|
||
|
Drew T
|
1ab9905368 |
feat(phase-26): §8d scope_data_externs — the ×133 sweep blocker fixed; func_8015AE2C banked ×134
- ROOT CAUSE (R14 — the session-7 diagnosis was half right): the isolated region builds [ OK ] WITHOUT the body, so §8b isolation was never implicated. `family_remap.gather_externs` prepends carried decls at FILE scope; D_801812A4 is a fn-ptr dispatch table the sibling declares FOUR incompatible ways at BLOCK scope inside its own later functions, so the carried file-scope decl ESTABLISHES A GLOBAL THE TU NEVER HAD and every later block-scope extern must now agree with it. Byte-proven asymmetry: BLOCK(int)->BLOCK(struct*)->FILE(void*) builds; FILE(void*)->BLOCK(int) errors. It was the ONLY hard error in the build — all 27 carried function externs were fine raw. - THE FIX (demote, don't reconcile): tools/scope_data_externs.py emits a carried D_ extern at BLOCK scope inside the function body when the TU has no file-scope decl of it above the insertion point. Byte-neutral (an extern emits no code; type + access opcodes unchanged) and never worse than raw, so it needs no oracle, no type comparator, no fn-ptr parser. Restores fidelity — the original declares these symbols at block scope in exactly this way. Wired into jtbl_family_bank as the `scoped` stage: raw -> scoped -> recovered -> reconciled (scoped is the base for the later stages). - reconcile_decls is the WRONG instrument for this class, twice: its oracle answers "what does the FLEET call this symbol" when the question is "what can THIS TU see", and its DATA_DECL_LINE_RE cannot parse `extern void (*D_x[])(void *);` — silently skipping the very symbols that were failing (the phase's third silent-skip bug, after find_site braces + overlay_files splits). - R17 TRIAGE RULE, first real test, held: `conflicting types` = the compiler REFUSED TO COMPILE = a C front-end diagnostic = our Python. Reading cse.c/global.c would have taught nothing. - RESULT: func_8015AE2C (562 ins, reach 134) swept 133/133 siblings, 0 failures. R22 clean-fleet 136/136 BYTE-IDENTICAL (534 changed src files); dedup-check 1813 validated / 0 failed; 0 NON_MATCHING (G4). instr-weighted 63.0 -> 63.6%; distinct-code 39.1 -> 40.5% (+256 unique fns / +79,957 ins) — one core, ~0 agent tokens. - knowledge captured during the producing session (R30/R31/R21): cookbook §8d, decision-log 2026-07-13 session 8, SETUP tool-inventory row; CURRENT_PHASE session-8 checkpoint. |
||
|
Drew T
|
f206659dcf |
feat(phase-26): jtbl_family_bank recovery stage (per-sibling cast+reconcile) + tolerant stage loop
- NEW "recovered" stage between raw and reconciled: cast_call_sites + reconcile_decls run against THIS sibling's TU. The recovery must be redone per sibling because the conflicting symbols are largely PER-OVERLAY (D_801812A4 in ov_SC01_000 vs D_800D4F8C in ov_SC01_077), so the exemplar's recovered decls do not transfer through the remap. - The stage loop no longer aborts a sibling when a stage cannot PRODUCE a candidate. It skips to the next one. canon_sig_reconcile raises on a K&R definition (it expects an ANSI signature), and a K&R def is MANDATORY whenever a zero-arg engine_core.h thunk calls the function (func_8015AE2C) — so that must not kill the bank. - gate-fail now reports the last stage error instead of an empty string. STATUS (P9): the func_8015AE2C x133 sweep is still BLOCKED and this commit does not close it. Remaining blocker, precisely diagnosed: the remapped body's DATA externs conflict with the sibling's §8b carried decl layer (which carries the macro externs of earlier regions and faithfully reproduces the original TU's declaration environment). reconcile_decls resolves against a FLEET-MAJORITY canonical oracle, not against the TU's actually-visible decl, so it picks a type that still conflicts. Fix direction: reconcile the body's externs against the TU's carried layer (authoritative) — or drop body externs the layer already provides and cast at use. The exemplar itself is banked and green. |
||
|
Drew T
|
9b93c254c2 |
feat(phase-26): func_8015AE2C (562 ins, x134) banked — Fable5 MATCH + 3 isolation bugs fixed
Exemplar banked byte-identical (d19c9580); R22 clean-fleet 136/136. Fable5 crack: MATCH 562/562, pin-free, jump table verified. THREE REAL BUGS the bank exposed in jr_isolate_all (each byte-proven; each would have silently corrupted every future heavy-core bank): 1. --only filtered `banked` as well as the cut set, so already-banked jr went untracked and their carves were never followed. --only selects what to CUT; it must not erase the record of what is already banked. 2. carve ownership was read from splat .s — but splat emits NO .s for a MATCHED function (its .c holds real C), so the lookup found nothing. Now resolved from the extracted IMAGE via family_remap.reloc_targets (byte-exact: func_801734BC -> 0x801d8c68 etc). 3. THE STRUCTURAL ONE: a region may host at most ONE .rodata carve, because an object's .rodata is a single CONTIGUOUS section. Cutting at func_8015AE2C (jtbl 0x801D8B54) left the banked func_801734BC (jtbl 0x801D8C68) inside the same region, so the object emitted a 0x34 .rodata spanning BOTH tables (image +33 B). Every already-banked jr in a cut object is now cut too -> exactly one carve per object. Cookbook 8b's "bank same-subseg families ASCENDING" note warned about this; it is now enforced by construction instead of left to discipline. Also required (per the crack's own analysis, all byte-verified): - engine_core.h: DEFINE_func_8015BEC4's zero-arg thunk returns func_8015AE2C(), so the extern must drop its (void) prototype and the def must stay K&R/unprototyped. Byte-neutral across all 136 (R22 green). - recovery chain: cast_call_sites (27 callees) + reconcile_decls (3 data syms). The raw body declares callees with types that conflict with their real engine_core.h defs; the original never redeclares them, it CASTS at the call site (cookbook 20). Layout now exact: .rodata 0x801d8b54/0x1c (7 entries, pad trimmed) + 0x801d8c68/0x14 + 0x801d92a0/0x20 — one table per object, each at its true address. |
||
|
Drew T
|
754ac3428f |
fix(phase-26): permuter silently no-op'd on every GTE draft (+ --asm-subdir)
Second silent no-op of the §G class, found while permuting func_8017BEBC (close=2):
- hide_asm() is built for __asm__ STATEMENTS and `register __asm__("$sN")` pins inside a
function body (it scans back to the previous ;{} and forward to the next top-level ;).
A draft whose GTE ops are #defines CONTAINING __asm__ (the PsyQ inline_c.h convention,
i.e. most renderer code) therefore had its macro DEFINITIONS chewed up, swallowing the
function itself -> pycparser 'Function <fn> not found in base.c' -> decomp-permuter
no-op'd in 0s. base.c contained ZERO occurrences of the target function.
FIX: cpp_expand_macros() pre-expands with `cpp -P` so each GTE op becomes an inline
__asm__ statement hide_asm can carry via the b64 pragma. Applied ONLY when a
'#define ... __asm__' is present -> macro-free drafts byte-untouched.
- p16_permute was hardcoded to OV=ov_SC01_077's MAIN object, so no core in another overlay
or split object could be permuted at all. Added --asm-subdir (threaded explicitly: a
def-time default arg cannot see a mutated global).
LESSON (cookbook): permuter 'no match (0s)' is a TOOLING failure signature, never a real
search result. Verify workers actually ran.
Verified: base.c now holds the function; 16 workers searching on func_8017BEBC.
|
||
|
Drew T
|
b0691f4bd9 |
fix(phase-26): jtbl_carve trims trailing .align pad words (§8a-pad)
A trailing `.word 0x00000000` under a jtbl dlabel is the ORIGINAL TU's intra-rdata .align 3 padding, NOT a table entry (0x00000000 is not a jump target). The true entry count is the fn's `sltiu <n>` bound: func_8015AE2C has sltiu 0x7 = 7 entries yet its raw dlabel spans 8 words. maspsx drops all .align, so a C-emitted jump table can never reproduce the pad. Carving to the next dlabel would reserve 8 words while the compiled object supplies 7 -> .rodata under-fills by 4 B -> every later symbol shifts +4 (the same image-corruption class as §41d). jtbl_range now trims trailing zero words, leaving the pad in the raw post-carve data piece. Retroactively explains the §8a func_80159C84 '5 words vs the real 6' false-MATCH. Existing carves are parsed from CONFIG, not re-derived, so committed banks are unaffected (verified: the 3 carved jtbls are absent from the raw data asm). The build never invokes jtbl_carve, so the fleet is inert to this change until the next bank. Found by the Fable5 crack of func_8015AE2C (562 ins x134, MATCH, pin-free). |
||
|
Drew T
|
c66b530f71 |
fix(phase-26): bound the §8b carried decl layer + cross-address sibling naming
Two bugs the func_80182268 sibling sweep exposed (both would have silently capped every future jr family bank): - extract_unit walks BACKWARD from a definition absorbing preceding extern/comment lines as the fn's preamble. The §8b carried decl layer sits directly above the FIRST item of an isolated region, so the unit swallowed the whole layer -> the template dragged ~140 unrelated externs into each sibling (some naming types the sibling TU lacks) -> gate-fail. jr_isolate_all now emits an explicit end-marker and extract_unit stops at it (also guards the Phase-17 canonical-sig layer). - jtbl_family_bank passed the EXEMPLAR's name to the sibling's carve/isolate/stub lookup. Cross-address families (same engine fn at a different vram per overlay) therefore never resolved: ov_SC01_077 @0x80182268 -> ov_SC02_000/003 @0x8017FCB0. The sibling's name is now derived from to_addr. The first two banked jr families were same-address, so this had never surfaced. ov_SC01_077 d19c9580 byte-identical; R22 clean-fleet 136/136. |
||
|
Drew T
|
e79d030499 |
feat(phase-26): func_80182268 banked via the LAZY isolation path + the void->s32 gate-cap fix
End-to-end proof of the §8b lazy bank composition on a real cracked jr core: lazy isolate -> jtbl_carve into the isolated subseg -> C body -> whole-binary gate -> d19c9580 BYTE-IDENTICAL; R22 clean-fleet 136/136. - func_80182268 (31-ins jr, ov_SC01_077_after) MATCHED first try: shared-tail fallthrough (jtbl cases 3+7 enter case 4's tail) + the u16-shift sign-extend idiom ((s8)(*(u16*)(p+0x70) >> 8) -> lhu/sll16/sra24). Carve collided with the committed func_801734BC carve -> lazy isolation fired exactly as designed. - R14 FINDING (cookbook §41d): the Phase-17 canonical convention "void->s32 return is byte-neutral (§3a-1)" is FALSE for a void body with no `return` — it costs ONE extra instruction. canon_sig_reconcile applies it unconditionally, so it turned a perfect 31-ins MATCH into 32 ins. That extra word made the isolated object's .text 4 B long, shifting EVERY data symbol +4 -> ~271k differing bytes, image +5 B. match_one said MATCH; only the whole-binary gate caught it (G3/P9). - FIX (generalizes the §19 sig_unify lesson): every recovery pass is a FALLBACK, never unconditional. jtbl_family_bank now gates RAW first, reconciled only on failure. - 136/136 byte-identical from a clean tree (R22); 0 NON_MATCHING (G4). |
||
|
Drew T
|
38ac5659aa |
feat(phase-26): §8b scoping wall BROKEN — decl-environment reconstruction + lazy per-core isolation
The full 54-jr isolate-all on ov_SC01_077 now builds d19c9580 BYTE-IDENTICAL
(R22 clean-fleet 136/136) — the configuration session 5 could not build. The
heavy-jr harvest (191 cores / 5.53M templatable ins) is unblocked.
- R14 CORRECTION: session-5's "gcc-2.7.2 block-scope-extern TU-persistence" root
cause was WRONG. There is no gcc quirk — DEFINE_func_* macros expand at FILE
scope, so their leading externs are genuine file-scope decls that merely live in
engine_core.h, invisible to any col-0 .c scan (1377 macros / 3929 lines / 1462 syms).
- REJECTED the approved "global symbol->type map + shadow set" design: the engine is
loosely typed (func_80173544 is DEFINED `s32 f(void*)` yet declared `extern void
f(void);` inside func_801734BC's body), so declaring every USED symbol hoists that
block-scope shadow to file scope and CREATES the conflict a shadow-set then dodges.
Instead reconstruct the original TU's file-scope decl environment and carry it
strictly FORWARD — conflict-free by construction (every carried decl already
coexisted with every definition in the one original TU; compatibility is
order-symmetric; shadows stay in bodies and travel with their item).
- The byte-gate found two MORE lost decl sources, not predicted: (a) a definition is
itself a declaration for everything below it in its TU (func_8012B2CC undeclared);
(b) file-local typedefs used by a carried prototype (parse error, Vec3s). K&R defs
must render `extern T f();` (unprototyped), never f(void).
- LAZY per-core isolation wired into jtbl_family_bank (Drew's call — upfront-x134 =
~7,200 region files): jtbl_carve NON-CONTIGUOUS fail-loud -> jr_isolate_all --only
<core> -> re-extract -> re-carve. Proven on func_80178D40 (890x134, heaviest core):
carve blocked -> isolated (byte-neutral d19c9580) -> carve in its own subseg.
- TWO LATENT BUGS fixed (both would have corrupted the heavy sweeps):
* jtbl_carve.func_subseg derived the owning subseg from the ASM TREE, which `make
extract` never prunes -> after an isolation it returned the STALE owner and
silently re-created the very collision the isolation removed. Now config-derived.
* jtbl_family_bank/jtbl_carve revert() DELETED the shared overlays.mk carve var
unconditionally -> would destroy a COMMITTED carve (all 134 overlays have one) on
any failed sibling. Now restored to its committed value; only region files created
by this attempt are removed; dirty-tree preflight refuses to start a sweep.
- docs: cookbook §8b RESOLVED + new §8c "splitting a TU means rebuilding its
DECLARATION ENVIRONMENT, not moving text"; decision-log 2026-07-13 (R30/R31).
- parser selftest 404/404; R22 clean-fleet 136/136; 0 NON_MATCHING (G4).
|
||
|
Drew T
|
234788dfc4 |
feat(phase-26): §8b overlay-src parser (404/404) + jr isolation tool + the gcc-scoping wall finding
- tools/overlay_src_split.py: overlay-.c-aware partition (header = includes + Phase-17 canonical-sig layer; per-address items = preamble + body; robust def/decl/K&R/DEFINE_func/ SETTER/RETCONST classification). Fleet-validated 404/404 overlay .c, 341,902 items — round-trip exact / 0 unresolved / 0 non-monotonic. The Stage-2 isolation unblock. - tools/jr_isolate_all.py: multi-cut jr resegment (config split at jr boundaries, source repartition + INCLUDE_ASM path repoint, banked-jr carve repoint, -O0 skip, ambient decl carry). SINGLE-cut isolation byte-identical (func_8013FFD8 -> d19c9580, R22). - FINDING (decision-log 2026-07-13): full 54-jr isolation of the dense _after object hits gcc-2.7.2 block-scope-extern TU-persistence (func_801734BC/D_80126B3E declared only in engine_core.h DEFINE_func macros); mechanical TU-split breaks it. Fix = declaration- completion from a global symbol->type map (Drew-approved next step; lazy per-core). - baseline intact (ov_SC01_077 rebuilds d19c9580); no config/src/binary change committed. CURRENT_PHASE session-5 checkpoint + decision-log R31. db.*.gbf = R23 noise, not staged. |
||
|
Drew T
|
d1dd29d815 |
feat(phase-26): §8b multi-jtbl same-subseg — contiguous MERGE (built) + isolation scaffold
Session-4 same-subseg handling (the de-risk preamble's harder half; byte-proof of the merged build + isolation deferred to Stage 2 with concrete cores): - jtbl_carve.py: MERGE adjacent same-subseg carves into one spanning .rodata piece (a code object emits its jtbls contiguous, so two matched jr-fns in one subseg are byte-correct iff their jtbls abut). BOUND-FIX: a new jtbl's end is bounded by the next raw dlabel OR the next existing carve start (an already-carved adjacent jtbl is gone from the data asm -> raw dlabels over-extend it -> false "non-contiguous"). Config-proven (func_80171B4C 801D8C48 merges with func_801734BC 801D8C68). NO-OP for family-1/cross-subseg (single carve per subseg) -> committed configs unaffected. - jr_isolate.py (scaffold, NOT yet functional): the non-contiguous case — split a fn into its own code subseg (whale _o0b precedent) so its jtbl carves independently. BLOCKED on split_src_region, which can't partition the overlay .c (global canonical- sig extern layer + per-fn callee-externs + DEFINE_func macros + @class annotations, ~922 non-address items). Stage-2 build item (overlay-.c-aware source split). - cookbook §8b (the --order sandwich + the two same-subseg cases + the blocker); CURRENT_PHASE session-4 checkpoint updated with the Stage-2 unblock decision. |
||
|
Drew T
|
ca50ee6978 |
feat(phase-26): §8 multi-jtbl --order carve + family-1 (func_801734BC ×134)
- ld_interleave.py --order: address-ordered N-piece data->rodata->data sandwich for overlays with 2+ matched jr-functions; legacy --front/--tail path is byte- untouched (main EXE + the 133 single-carve func_8012ACE0 siblings unaffected) - jtbl_carve.py rewritten additive/regenerate-from-config: parse the tail data region + existing .rodata carves, split the containing data piece for the new jtbl, re-emit the address-ordered pieces + the --order arg; same-subseg carve collision fails loud (-> jr isolation); idempotent - jtbl_family_bank.py: `make extract` BEFORE the carve (asm must match the reverted committed config; the old error-string retry was fragile) + revert-on-carve-fail - family-1: func_801734BC (34-ins PURE jr, ov_SC01_077_after) matched in ov077 (shared-tail switch idiom) + banked 133/133 siblings = x134 — CROSS-subseg multi-jtbl (func_8012ACE0 in _a + func_801734BC in _after) - R22 clean-fleet 136/136 byte-identical (~52s); 0 NON_MATCHING (G4) |
||
|
Drew T
|
5a08180617 |
feat(phase-26): §8 ×134 automation — func_8012ACE0 banked fleet-wide (133/133, R22 136/136)
- the jr-function ×134 harvest pipeline, proven end-to-end: per family sibling,
jtbl_carve (per-sibling jtbl-rodata carve, computed from THAT sibling's own jtbl
address — the fn is at the same vram across overlays but its jtbl floats) -> make
extract (auto ld_interleave) -> remap_hseq + canon_sig_reconcile -> whole-binary gate
- tools/jtbl_carve.py: per-overlay §8 carve generator (config data-tail split +
<ov>_JTBL_INTERLEAVE var)
- tools/jtbl_family_bank.py: the sibling sweep driver (idempotent, revert-on-fail, byte-gated)
- tools/family_remap.py: extract_unit now carries single-line typedefs (jr-function bodies
define local `typedef struct{} Foo_<addr>;` that must template with the body — the
propagation cap for these; additive, byte-gate-protected)
- func_8012ACE0 family: 133/133 siblings BANKED, 0 failures; R22 clean-fleet 136/136
byte-identical; 0 NON_MATCHING (G4)
- metrics: distinct-code 39.1% (50,698 unique fns), instr-weighted 63.0%
- opportunity (has_mid_jr families): 237 total (5,805 members) = 46 small mid/tiny
(771 members, same mechanical pipeline) + 191 substantial (the Fable5 cores, Task 7 paused)
- NEXT: R22 profiling/parallelization; then the other 45 small jr families
|
||
|
Drew T
|
095a611e75 |
feat(phase-26): §8 jtbl-rodata tooling — overlay PoC proven (func_8012ACE0, R22 136/136)
- overlay jr-functions can now bank as C: gcc switch jump tables form a .rodata island at the overlay TAIL; carve a matched fn's jtbl into a dotted [.rodata, <code-subseg>] subseg + ld_interleave (data->rodata->data sandwich) places it byte-exact. cookbook §8a + SETUP. - tools/ld_interleave.py: --section .<binary> param (derives the <binary>_TEXT/DATA/RODATA/ DATA2/BSS symbol prefix); default .main = the EXE, byte-identical (backward-compat proven) - Makefile + config/overlays.mk: <bin>_JTBL_INTERLEAVE hook + a $(strip)-guarded extract branch (gotcha caught: a trailing #comment on the := left whitespace -> non-empty -> the branch misfired on resident with the EXE defaults) - PoC: func_8012ACE0 (25-ins jr-fn in ov_SC01_077) reconciled (canon_sig_reconcile) + banked BYTE-IDENTICAL d19c9580 -- the first overlay jr-function matched through the C pipeline - R22 FULL-FLEET clean rebuild: 136 passed, 0 failed (main 143dbb89 unaffected by the ld_interleave change); 0 NON_MATCHING in any default build (G4) - P9 findings: func_80159C84/func_8015444C (the 2 carried Fable5 jr bodies) are rtu_match FALSE-matches (incomplete jtbls: 52B vs 56B -> never bank); the maspsx "hang" scare was a truncated experimental-file artifact (real pipeline builds in ~1s) - metrics: distinct-code 39.1% (50,572 unique fns), instr-weighted 62.9% - NEXT: the ×134 automation (generate the per-overlay carve + template the reconciled body) |
||
|
Drew T
|
62f9533024 |
feat(phase-26): +266 reconcile-class banks (2 no-jtbl cracks x133) + Fable5 batch-1 whole-binary findings
- +266 member-matches: func_8015CD20/func_8015C128 templated x133 via --reconcile-raw (each SHA-gated per-overlay vs config/check.<ov>.sha = byte-identical, G3). Full R22 deferred until func_80176218 releases asm/ (established per-overlay-gate + deferred-R22 pattern, as the committed 463 which R22'd 136/136). - family_sweep: --reconcile-raw now also covers draft-ov077 (unbanked) cracks (template from the RAW seed). - P9 CORRECTION + decision-log 2026-07-12: the 2 Fable5 cracks rtu_match-MATCH but FAIL the whole-binary gate (both jr-functions; rtu_match masks relocs + excludes neutralized INCLUDE_ASM rodata, so it never verifies the §8 jtbl rodata). TWO harvest gaps: §8 jtbl-rodata (blocks all jr cracks) + reconcile data-extern (D_801891B8-class, blocks ~15/21 no-jtbl triage cracks). 6 no-jtbl reconcile-clean cracks bank whole-binary (729 members). rtu_match is NOT a sufficient arbiter for jr-functions. |
||
|
Drew T
|
7ccf48f2f4 |
feat(phase-26): Task-8 reconcile wiring (§41c h_seq) + 463 reconcile-class banks
- BUILT the per-sibling reconcile: family_remap.remap_hseq_body (h_seq-remap a RAW crack draft: symbol + immediate + cross-address self-rename) + family_sweep.reconcile_remap_hseq + --reconcile-raw. Per sibling, remap the RAW crack then canon_sig_reconcile against that sibling's own TU (the h_seq port of the h_norm M2 path) — because a reconciled body is TU-specific and can't template plainly (validation: 0/4). - HARVEST: the 4 triage isolation-cracks (func_80155800/80167540/801506A4/8016A73C) templated 463/0 x~133 via --reconcile-raw (0 failures). Metrics: instr 58.5->58.9%, distinct 30.9->31.1%. - each overlay SHA-gated by harvest_verify vs config/check.<ov>.sha (byte-identical = the match def, G3). FULL R22 clean-fleet DEFERRED until the concurrent Fable5 crack agents release asm/ (their m2c needs it); R22 fleet-confirm to follow post-window. - cookbook §40c (the h_seq per-sibling reconcile technique, R30). |
||
|
Drew T
|
d05203b9a0 |
feat(phase-26): task 5 — h_seq zero-crack GO/NO-GO = GO; 532 members banked (R22 136/136)
- remap_hseq.gather_externs: carry file-scope externs for body-referenced symbols (extract_unit only grabbed adjacent ones) — the decl class that blocked per-location bodies indexing a global. func_8015F118 gate-fail -> BYTE-IDENTICAL; the 3 tracker-miss PURE families then bank 133/133 each. - ran the real whole-binary byte-gate on the 29 substantial matched-exemplar families: 532 members BANKED (byte-gated). Per-family: 3 tracker-miss PURE (0x8015d5e8/0x8015f118/0x801407f4) bank 100% x133 = 399 byte-perfect (the tracker-fix free win); 1 cross-addr family 50%; 9 zero-bank families are type-using (Work8016/Prim/...) -> the existing --reconcile/type-lift follow-on (Task 8); 16 families pinned -> Task 7 pin-free re-crack. - VERDICT: the h_seq machinery (tracker + imm + cross-address + extern-carry) is byte-proven 100% correct on clean families. GO to scale. - R22 clean-fleet: make clean + extract-all-136 + check-all = 136 passed, 0 failed. 0 NON_MATCHING (G4). Metrics: distinct-code 30.3->30.9% (+375 fns), instr-weighted 58.2->58.5%. - decision-log 2026-07-11 (R31: stratify a mechanical-harvest rate by family/class before judging it). |
||
|
Drew T
|
3ce81cbf56 |
feat(phase-26): task 4 — family_sweep --hseq mode (cross-address + imm templating)
- additive hseq_sweep(): consumes .run/family_hseq.json, templates each matched-exemplar family's
still-stubbed members via remap_hseq (reloc remap + imm subst + cross-address self-rename), stages
by (overlay,split), gates each group once via harvest_verify. h_norm path byte-UNTOUCHED (new --hseq
branch routes before it). Unique per-group verified-out fixes a latent multi-split overwrite.
- member pre-filter = remap_hseq refusal (STRUCT regalloc-drift / unresolved immediates).
- static pin guard: skip families whose matched exemplar carries a hard-reg pin __asm__("$N") — the
x1-only cracks (decision-log 2026-07-11: func_8016DF5C/8013D9B0/80133AB0) that cc1-crash sibling TUs;
they route to Task 7 pin-free re-crack instead of bisection-storming the gate.
- flags: --hseq[=manifest] --band --min-members --stage-only --only.
- VERIFIED (--stage-only): substantial band 29 families -> 1507 clean members staged / 267 groups,
1643 correctly skipped pinned (16/29 pinned); h_norm --only path intact; drafts byte-correct by
construction (V3 0-DIFF), type-using -> whole-TU gate (Task 5).
|
||
|
Drew T
|
462734edb2 |
feat(phase-26): task 3 — T2a immediate engine (Tier 1) + T2b cross-address, 0-DIFF verified
- family_remap: imm_value (signed/unsigned field-aware) + imm_map_tier1 (diff-driven literal swap: asm-side ambiguity guard defers values that also appear at a non-differing position; C-literal swap preserves sign + hex-case) + remap_hseq (symbol remap + imm subst + cross-address self-rename in one pass; refuses STRUCT/unresolved members -> caller skips, byte-gate would reject anyway). - Tier 2 (targeted probe) intentionally DEFERRED — build-if-needed per Task-5 measurement; only ~8 low-weight IMM families, and the whole-binary byte-gate arbitrates (plan: simpler where gate arbitrates). - VERIFIED (.run/v3_imm.py, match_one reloc-masked so immediates are checked exactly): 0 DIFF on every compilable derived draft — 3 IMM + 2 PURE cross-address MATCH; 4 remap-fails were correctly-deferred asm-ambiguous values (not wrong output); 45 compile-fail are match_one isolation limits -> whole-TU gate in Task 5. Cookbook §40b imm-engine forward-ref fixed. |
||
|
Drew T
|
faedd103e4 |
feat(phase-26): task 2 — tools/family_hseq.py full-frontier survey + shared word-diff classifier
- family_remap.py: shared classifier (stream_words / reloc_indices / reg_fields / classify_member) — per-instruction diff class RELOC/IMM/IMM_SA/STRUCT, register-drift aware (h_seq ignores registers, so regalloc-drift members are STRUCT-excluded, not templatable). Reused by T1/T2a/T3. - tools/family_hseq.py: cluster ALL unmatched overlay instances by h_seq; per family classify every member vs exemplar (PURE/IMM/MIXED), tag per-location/cross-address/scattered, matched-sibling count, has_mid_jr (§8), exemplar pick (matched-ov077 > matched > draft-ov077 > modal), size band. -> .run/family_hseq.json + docs/family-hseq.md (committed digest). - VERIFIED: fleet 74.8/58.2/30.3 (= PhaseEnd_25 + progress.py exact); tail cross-check 663 families / 186 substantial / 1.847M ins (exact); classification vs Plan-agent PURE-same 62 & IMM 8 exact; 890x134/562x134 PURE per-location + 952x113 #addr21 IMM confirmed. Full frontier: 581 substantial families / 3.22M templatable ins; 345 matched-sibling PURE/IMM families / 1.14M ins = V2/V3 corpus. |
||
|
Drew T
|
5b7e366648 |
feat(phase-26): task 1 — extended reloc tracker (addu-hi) + single-pass remap; V0/V1 green
- reloc_targets: propagate lui-hi through add/addu index arithmetic (gcc-2.7.2 indexed-global idiom lui;addu $idx;lw %lo($at)). The pre-26 tracker dropped it -> D[i] functions mis-normalized per overlay (inflated the 'h_norm reach-1 tail') AND lost their indexed D_ symbols in remap. norm_stream/h_norm deliberately UNTOUCHED (fleet metrics + proven sweep depend on it). - remap/symbol_map: backward-compatible to_addr=None (cross-address T2b sibling + self-rename) + imm_map hook (T2a); sequential re.sub -> single-pass simultaneous substitution (fixes latent chained-rename/value-permutation corruption). All 5 family_sweep call sites unchanged. - V0 (.run/v0_reloc.py): func_80141100 22/22 NEW==OLD (zero regression); func_801407F4 15/15 vs splat .s (pre-fix 10), recovers indexed D_80187B88/90/B0; cross-addr symbol_map clean. - V1 (.run/v1_regression.py): 160 real h_norm sibling pairs, 96 SAME, 0 lost — differences are strict indexed-reloc improvements only. - docs: cookbook §40b (the technique, R30) + decision-log 2026-07-11 (the strategic why, R31). |
||
|
Drew T
|
c62fe7f7ea |
feat(phase-25): task A giant #1 — func_80166994 (369 ins) cracked ×134 via Fable5 + the K&R s16-param idiom (§43)
- Fable5 subagent cracked func_80166994 (trail/afterimage ring recorder, 369 ins) — FULLY STRUCTURAL, zero register pins -> swept ×134 CLEAN (exemplar + 133 siblings byte-identical). R22 clean-fleet 136/136; instr-weighted 56.8% -> 57.2%; distinct-code 27.3% -> 28.2% - NEW IDIOM cookbook §43: a K&R s16-param DEFINITION dissolves the §17/§29 "narrow-param wall". On MIPS K&R promotes s16->int (ABI-identical to the canon-sig s32), body keeps the in-place sll aN,16 narrow/extend the (s16)cast form can't reproduce. void->s32 return-flip pair: split //@EDIT (self-fn, ov077-specific) + engine_core.h ec_edit ×5 (byte-neutral, callers discard) - family_sweep --edit-remap: split-edits now OPTIONAL (apply where present, never skip; the whole-binary byte-gate is the sole arbiter, G3/P9) — a sibling lacking the ov077 canon-sig decl still banks via ec_edit + body. edit-absent tracked, not skipped - R14: the prior wave's "@stuck: none — MATCH" note on func_80166994 was STALE/FALSE (re-ran DIFF 366/369). Verify a MATCH claim vs the bytes, never a stale note - structural cracks are the ×134-SAFE ones (contrast §42e pin-heavy families that cc1-SIGABRT in sibling TUs). Other 6 giants -> cheap-Opus applying §43+§31, Fable5 only on new-class evidence |
||
|
Drew T
|
c0379f0738 |
feat(phase-25): progress.py --weighted — byte/instruction-weighted metrics (the honest headline numbers)
- weighted_metrics() from .run/sig.*.jsonl + src stubs (executable code only, resident + 134 overlays; main EXE excluded). Two framings: fleet instr-weighted (per-overlay, the decomp.dev -display number) + dedup distinct-code (each unique h_exact once, the distinct-RE number) - --fleet now emits THREE labeled metrics into docs/progress.fleet.md: fn-count 74.48% (×134- inflated), instr-weighted 56.8% (shipped .text), distinct-code 27.3% (of 84,996 unique fns) - --weighted prints the two weighted numbers standalone; degrades gracefully if sigs absent - corrects the stale "~30-35% byte-weighted" estimate: the giant campaign since Phase 19 raised the fleet instr-weighted number to 56.8%; the distinct-code 27.3% is the unique-monster-tail truth - SETUP §tooling row updated (R21) |
||
|
Drew T
|
5fcb040dc3 |
feat(phase-25): task B — family_sweep --edit-remap; 2 array-decay families ×134 (+266 fns), 4 cc1-crash-walled
- family_sweep.py: new --edit-remap MANIFEST mode (§42e) — per family, symbol-remap the split-scope //@EDIT old||new per sibling + apply once-global engine_core.h ec_edits (byte-neutral), stage the family_remap body, gate via harvest_verify (the sole arbiter) - BANKED 266/266 (0 failed): func_80136824 + func_80136334 (array-decay ptr-flip) ×133 siblings each — full ×134. R22 clean-fleet 136/136, fleet 74.40% -> 74.48%, dedup 1813/0 - R14 FINDING (cookbook §42e addendum + decision-log): the other 4 byte-drift families (func_80133AB0 zero-reg pin, func_8016DF5C/8013D9B0 GTE-pin, func_80156044 trampoline) cc1-SIGABRT (Error 134) in the SIBLING TU — hand pins are ov077-TU-context-specific, NOT mechanically ×134-recoverable; backlogged as ×1/permuter fuel. rtu_match/match_one are blind here (neutralized/isolation compiles crash too); only make build is truth - 0 NON_MATCHING in any default build (G4) |