Files
BFM-decomp/cookbook/C0149.md
T

13 KiB
Raw Blame History

§138 — The propagation lanes: a gate refusal is a DECLARATION, and which lever you owe depends on blast radius

Symptom key: dedup_extend or dedup_propagate plans N members and banks 0, or reports PLUMBING: conflicting types for X / an undiagnosed DIFF. Measured this session: a 0/36 lane went to 31/36, and every single blocker was a declaration — not one was compiler codegen.

The triage, cheapest first

Capture each failure's own compiler error (§136a — classify on the build's OUTPUT, never its exit status) and bucket by which symbol is named. The named symbols repeat: 9 failures per binary reduced to 4 distinct symbols shared across all four binaries. Then pick the lowest-blast-radius lever that is byte-neutral by construction:

conflict lever radius
macro declares (void), TU declares (ptr), use is cast relax the macro decl to () T2, 1 token
TU decl is unused boilerplate (zero uncast uses) conform the TU decl to the fleet canon T1
TU declares the symbol volatile, or any type the macro can't match asm-label alias on the DATA T2
the fn's own signature (return/arity/param) asm-label alias on the DEFINITION T2

Before relaxing to (), MEASURE the whole fleet's decl shapes for that symbol. () is illegal against a prototype carrying a default-promotion param (s8/s16/u8/u16/char/short/float) — the documented gcc-2.7.2 dead-end. It is legal against pointers and s32. One grep -rhoE over src/ answers it; 4,020 decls of func_80146C3C were all (void)/()/(u8*), so the relax was safe and bought 8 of 36 for one token.

One conflict HIDES the next. A declaration conflict aborts the compile, so the error you see says nothing about what is behind it (the S29 law). Fixing func_8012E5CC immediately revealed func_80147364 at the same site. Re-run after every fix; do not price the lane off the first error.

Bucket by the (macro-shape, TU-shape) PAIR — NOT by the symbol. The same symbol conflicts in BOTH directions across the fleet, and the lever differs. func_80146C3C cost this lesson twice in one session: in the EXTEND lane the macro said (void) and the TU (u8*); in the PROPAGATE lane the mirror — macro (u8*), TU (void). Relaxing only the (void) form (42 decls, byte-neutral, R22-clean) fixed the first and did nothing for the second, which then banked 0/1 in all 134 overlays. One awk over the macro you are ACTUALLY fixing — not its sibling — shows the pair before you spend a 134-build run. Read the declaration of the macro in front of you.

volatile in the host TU is a SCHEDULING BARRIER — and it looks exactly like a codegen wall

The four "undiagnosed DIFF"s in dedup_extend's own header were this. Its correctness argument says an h_exact match guarantees byte-identity including relocs, so a DIFF should be impossible. Both halves resolved against the bytes:

  1. The contract HELD — func_80162FF4's original bytes are sha1-identical in the failing and the working overlay (af1aceb2…). Check this first: it splits "the registry is lying" from "the TU is different" in one command, with no build.
  2. The TU differed — the host TU declared extern volatile s32 D_80127090/94/98 at FILE scope, which no working overlay's copy of that TU does. Volatile makes the macro's three stores a barrier, so addu $a0,$s2,$zero could not sink into the jal's delay slot: the build emitted it early plus a nop, one instruction longer.

The tell: a diff that is a positional shift with a nop appearing at a delay slot is an ordering constraint, not a wrong body. Look for a qualifier (volatile, const) on a symbol the body touches before reaching for a codegen idiom. The fix is the data asm-label alias (extern s32 aD_80127090 __asm__("D_80127090")): a distinct C identifier is immune to any TU's declaration of that symbol, and is byte-neutral wherever the macro already worked.

The DEFINITION-side alias is the only escape when the fleet canon disagrees on a promoting param

func_80147364's byte-true definition is (u16, u16); 4,046 fleet decls say (u16, s32). () is illegal (u16 promotes) and conforming the decls would change caller codegen. Author the macro as void aF80147364(u16, u16) __asm__("func_80147364"); + a definition of the aliased name: the real symbol is emitted, every caller keeps its own declaration, blast radius is zero. In-tree precedent: 1,725 files already use this form. Banked ×137 first try.

Rank the lane by measured concentration, not by class count

A "45 classes / 20,837 ins" queue was really 5 classes carrying 89% of it. Re-split the ledger by nins × n_stub before scheduling anything — and note the per-class outcomes diverge wildly (4,110 banked ×137 · 3,288 banked ×138 · 3,973 dropped · 3,886 at 4/138 · 3,288 tool-gapped), so a lane average predicts nothing. --recover is not a retry: the caller-extern reconcile that is 16/16 lifetime on drafts banked 4 of 138 on a propagation. Probe one excluded member's build output before re-running any lever that already returned a bad number.

THREE carry variants hide in one "CARRY-FIXABLE" bucket — and they need different fixes

dedup_propagate reports missing file-scope extern (CARRY-FIXABLE) for all of them, but the response differs and two are NOT tool bugs:

variant what extract_unit cannot carry response
a multi-line comment halts the preamble backscan the externs above it fix the tool (cdecl._mask)
a draft-local struct Tag {…} in the preamble the type (refused by design: two macros defining one tag redefine it in a TU) switch the exemplar to the shared engine_types.h type if the layout already exists there — byte-neutral
a file-scope static inline helper the helper hand-author the macro with the helper inlined, and exclude the source overlay

The third is the sneakiest, because gcc-2.7.2 accepts implicit function declarations: the extracted body passes compiles_standalone with the helper undeclared, so nothing complains until a whole-binary byte DIFF 137 gates later. And the macro cannot be instantiated in the SOURCE overlay — its file-scope helper is still there, so the macro's copy is a duplicate definition. The invocation shape is --source-overlay X --binaries <all-but-X>; passing --binaries alone removes the source from the scan pool and errors with "no source overlay has it matched".

A function defined under an asm-label alias is invisible to a name-anchored head regex. The def is named aF<ADDR> in C and only BINDS the real symbol via __asm__("func_<ADDR>"), so any matcher anchored on the literal func_<ADDR> returns None and every caller reads that as not matched. family_remap._alias_decl_for resolves the form; make other tools reuse it rather than grow a second matcher (R33). Two live consequences found together: dedup_propagate.find_site was blind to the form entirely, and _alias_decl_for itself was blind to the wrapped (multi-line) declaration — the §134 shape again. Measured before fixing (R37): 91 distinct alias decls fleet-wide, the per-line matcher resolved 90, and the single miss was func_801466F0. That one function then took all THREE fixes plus a type-lift — alias-aware find_site, wrapped-alias matching, and hoisting its record typedef to engine_types.h — which is exactly why it survived four phases of being written off: each blocker on its own looked sufficient to explain the failure.

Tool boundary worth knowing: once a group's members are DEFINE_func_*() sites, dedup_propagate can no longer extend it — find_site never returns a def, so the auto-source scan errors. dedup_extend is the tool for an already-macro-ized group. (And running dedup_extend --check-only across ordinary overlays is a cheap census of how much wiring is outstanding fleet-wide: measured here as exactly 1 group per overlay, i.e. no hidden backlog.)

§134 again, in a second tool — and the waiter rule corrected

dedup_propagate.find_site's preamble backscan had the SESSION-18 fix for blank / // / single-line /* … */ lines, and still halted on a multi-line block comment (middle lines start *; the last ends */ without starting /*). Same class S6b fixed three times in family_remap. Decide skippability on cdecl._mask, not on line syntax — one oracle (R33), every comment form, immune to a /* inside a string, with an R32 assertion that the mask is length-preserving. Also: a body declaring a draft-local struct Tag {…} is unextractable by design; if the identical layout already exists in src/shared/engine_types.h, switching the exemplar to the shared tag is byte-neutral and unblocks propagation.

Waiter rule, corrected (three failures, one mechanism — the signal sampled is not the thing waited for): pgrep -x make is right for ONE make and wrong for a campaign — dedup_propagate runs a sequence of make build BINARY=<ov>, so a poll lands in the gap between two and reports a live campaign finished. And pgrep -f <pattern> self-matches its own command line, so that waiter can never exit. Wait on the campaign process by its real argv (ps -eo args | grep 'python3 tools/…') or on tools/treelock.sh --status, which is a statement of intent spanning the gaps. Likewise never wrap a campaign in nohup … & inside a backgrounded call: the harness then signals completion of the wrapper — a fleet check "finished" at 63/140.

STEP 0 of sibling-first: grep src/ for a distinctive LITERAL from the .s

§136c's search order (engine_core.h near-twin -> same-TU banked sibling -> the .s) has a hole: both of its first two steps are same-TU or shared-header scoped, so neither can reach a banked twin that lives in a different overlay's TU — and the large template classes live cross-overlay by construction. Measured: func_80188C04 (328 ins) was byte-identical to an already-banked func_801833F0 in ov_SC02_028, and one command found it —

grep -rn "E100000A" src/          # a magic word lifted straight out of the target .s

— after which the body was reused verbatim with only the file-local type/macro suffixes renamed. Put this ahead of engine_core.h. Pick a distinctive constant from the target: a magic word, an unusual mask, an odd immediate. Corollary for the family map: it carries an in-family exemplar pointer only, so a family whose twin is banked elsewhere looks un-cracked — and, separately, that pointer can name an instance that is already banked, which hides the whole family from any ranking built on it. Derive open sites from corpus.stubs over the member list. Measured on one wave: ranking off the map's exemplar yielded 16,696 templatable ins; deriving from corpus.stubs yielded 41,023 — including a 55-ins family open in 138 overlays and a 46-ins one open in 133.

Reconciling a gate-refused draft: which way you edit depends on WHERE the TU's decl is

A draft that match_one-MATCHes but the whole-binary gate refuses is declaration plumbing (the agents cannot run the gate, so a TU-level conflict is invisible to them). Two opposite fixes, and picking the wrong one creates the next error:

the TU's decl is… symptom fix
ABOVE the splice point redefinition of struct X / conflicting types for a TYPE DELETE the draft's duplicate — the TU already provides it (§100)
BELOW the splice point 'X' undeclared (first use…) after you deleted yours KEEP a decl, in the TU's exact shape, and cast at the use (§17a-1 D2)

Measured both in one session: func_8017D318 needed the deletions (the TU defines the type trio above it — and a struct PW tag that a first pass missed, so it took two rounds), while func_80181EE0 needed the opposite — I removed its decl, and the TU's own turned out to be ~180 lines BELOW the splice, leaving the identifier undeclared. Grep the TU for the symbol and compare line numbers with the stub line before editing either way.

And apply §136a to your own capture tooling. My blocker-capture filtered the build log for error|conflicting|undefined reference and so reported "no compile error" on a build that was failing with redefinition of struct PW8017C290 and 'func_80143C74' undeclared — neither phrase matched. A narrow keyword filter is exactly how a real error goes unseen. Keep any line naming a source position (\.[ch]:\d+), minus the known SHB macro noise.

Symptom line for the index: "a propagation/extend lane plans N and banks 0" — read each failure's compiler error, bucket by named symbol, and apply the lowest-radius byte-neutral alias; a volatile in the host TU and a fleet decl carrying a promoting param are the two that masquerade as codegen.