fix(phase-31): S54 — four measured over-refusals in the overlay pre-gate path

Each one refused byte-verified work; each fix is probed, not reasoned:
* built-in redeclaration: a cc1 probe shows two conflicting "memcpy" declarations give
  "warning: conflicting types for built-in function" + exit 0, while the same pair on a
  non-builtin name errors. Every overlay TU in the fleet declares memcpy twice and compiles
  today -> CONFLICTING-EXTERN on a builtin is now WARN.
* driver mismatch: overlays bank via gate_lane -> gate_stage -> harvest_verify, which strips
  every typedef the target TU provides; pregate_check modelled gate_main's hoist/strip instead
  and reported DUPLICATE-TYPEDEF for exactly the duplicates the real gate removes.
  substitute() now takes an optional per-draft transform; pregate passes the overlay one.
* block-scope typedefs: two functions may each declare their own typedef inside their bodies
  (that is how a draft stays self-contained for match_one). _typedefs now honours the brace
  depth map the caller already computed.
* project scalar aliases: include/common.h's "typedef s32 M2C_UNK;" makes "extern s32 D_x" and
  "extern M2C_UNK D_x" the same declaration; _ALIASES now DERIVES those from common.h (R33).

Measured on the 5 leftover slates: 28 drafts, all re-verified MATCH by match_one, went from
"0 kept / phantom FAILs" to main 2 clean, ov_SC04_011 15 clean, ov_SC03_028 1 clean,
ov_SC06_029 4 + 1 named TU edit, ov_SC02_005 2 real TYPEDEF-USED-ABOVE-DEFINITION.
This commit is contained in:
Drew T
2026-08-17 11:20:19 -06:00
parent 85671bc1f7
commit bb34eac42f
2 changed files with 79 additions and 13 deletions
+40 -8
View File
@@ -34,7 +34,7 @@ Usage:
default is a DRY RUN that reports what would be substituted and any conflicts.
--apply performs the substitution + clean rebuild and leaves banked drafts in the tree.
"""
import argparse, collections, functools, json, re, subprocess, sys
import argparse, collections, functools, json, os, re, subprocess, sys
sys.path.insert(0, 'tools')
import corpus
@@ -82,6 +82,27 @@ def sym_of(d):
# symbol both ways do not conflict -- but a textual comparison calls them different and drops a
# good draft (R39 over-refusal; wave O hit it with `extern short D_800B9A02`). Signedness is NOT
# normalized away: u16 vs s16 is a genuine conflict and must stay one.
def _common_h_aliases():
"""The project's OWN scalar typedefs, read from include/common.h rather than restated here (R33).
`typedef s32 M2C_UNK;` makes `extern s32 D_x;` and `extern M2C_UNK D_x;` the SAME declaration,
but a textual comparison calls them a CONFLICTING-EXTERN and refuses a byte-verified draft --
measured on ov_SC03_028, where both spellings already coexist in a TU that compiles today
(R39 over-refusal). Only aliases of a scalar we already canonicalize are folded in; anything
else (struct typedefs) is deliberately left alone."""
out, scalars = {}, {'s8', 'u8', 's16', 'u16', 's32', 'u32', 's64', 'u64', 'f32', 'f64'}
try:
txt = open(os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))),
'include', 'common.h')).read()
except OSError:
return out
for m in re.finditer(r'^\s*typedef\s+([A-Za-z_]\w*)\s+([A-Za-z_]\w*)\s*;', txt, re.M):
base, name = m.group(1), m.group(2)
if base in scalars and name not in scalars:
out[name] = base
return out
_ALIASES = {
'char': 's8', 'signed char': 's8', 'unsigned char': 'u8',
'short': 's16', 'signed short': 's16', 'short int': 's16',
@@ -89,6 +110,7 @@ _ALIASES = {
'int': 's32', 'signed int': 's32', 'long': 's32', 'long int': 's32', 'signed long': 's32',
'unsigned': 'u32', 'unsigned int': 'u32', 'unsigned long': 'u32', 'float': 'f32',
}
_ALIASES.update(_common_h_aliases()) # M2C_UNK -> s32, M2C_UNK16 -> s16, ... (derived, not restated)
def _alias(t):
@@ -399,7 +421,7 @@ def hoist_typedefs(t, wanted):
[n for n in seen if defs[n][0] > anchor])
def substitute(entries, write=True):
def substitute(entries, write=True, transform=None):
"""Replace each INCLUDE_ASM stub line with its draft body.
write=False produces the substituted text WITHOUT touching the tree, which is what
@@ -409,13 +431,21 @@ def substitute(entries, write=True):
Per-binary since P31 S54 (see `_stubs_for`): an entry's own 'binary' selects its stub map, and
an entry whose symbol is in NO stub map is reported loudly instead of being dropped on the floor
(R32 -- a silent skip here is what made pregate_check green on overlay slates)."""
(R32 -- a silent skip here is what made pregate_check green on overlay slates).
`transform(body, tu_path, binary) -> body` is an optional per-draft rewrite applied BEFORE the
typedef strip. It exists because main and the overlays are banked by different drivers with
different draft transforms: gate_main uses the hoist/strip logic below, while gate_lane ->
gate_stage -> harvest_verify strips every typedef the target TU already provides
(`cdecl.strip_provided_typedefs`). A checker that models the wrong driver reports failures the
real gate would never see -- pregate_check passes the overlay transform for non-main entries."""
byfile = collections.defaultdict(list)
unresolved = []
for e in entries:
st = _stubs_for(e.get('binary')).get(e['fn'])
if st: byfile[st.path].append((st.addr, e['fn'], st.asm_dir, e['draft']))
else: unresolved.append((e.get('binary') or 'main', e['fn']))
b = e.get('binary') or 'main'
st = _stubs_for(b).get(e['fn'])
if st: byfile[st.path].append((st.addr, e['fn'], st.asm_dir, e['draft'], b))
else: unresolved.append((b, e['fn']))
if unresolved:
print(" substitute: %d entr%s resolved to NO stub (not open in that binary?): %s"
% (len(unresolved), 'y' if len(unresolved) == 1 else 'ies',
@@ -427,7 +457,7 @@ def substitute(entries, write=True):
# Only names the incoming drafts actually define are candidates -- we never reorganize a
# file for types nobody in this slate needs.
want = set()
for _addr, _fn, _asmdir, draft in items:
for _addr, _fn, _asmdir, draft, _b in items:
body_txt = open(draft).read()
for p in (TYPEDEF_BLOCK, TYPEDEF_PLAIN):
want |= {m.group(1) for m in p.finditer(body_txt)}
@@ -462,9 +492,11 @@ def substitute(entries, write=True):
# slate order the surviving typedef can end up BELOW a draft that uses it -> "syntax error
# before D_800A651C" at the earlier draft's line. Each stub is substituted at its own
# position in the .c, so the walk must follow those positions. (S52, cost 2 rebuilds.)
for _addr, fn, asmdir, draft in sorted(items):
for _addr, fn, asmdir, draft, binary in sorted(items):
body = "\n".join(l for l in open(draft).read().splitlines()
if not l.strip().startswith('#include'))
if transform:
body = transform(body, path, binary)
old = f'INCLUDE_ASM("{asmdir}", {fn});'
if old not in t:
continue
+39 -5
View File
@@ -40,6 +40,14 @@ sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import cdecl
import gate_main as gm
# gcc-2.7.2 built-ins: a conflicting redeclaration of one of these is a WARNING (verified against
# the pinned cc1, P31 S54), so it must not block a rebuild. Anything else is a hard error.
_BUILTINS = {
'memcpy', 'memset', 'memcmp', 'strcpy', 'strncpy', 'strcmp', 'strncmp', 'strlen', 'strcat',
'strncat', 'strchr', 'strrchr', 'abs', 'labs', 'fabs', 'alloca', 'sqrt', 'sin', 'cos',
'printf', 'sprintf', 'fprintf', 'putchar', 'puts', 'exit',
}
IDENT = r'[A-Za-z_]\w*'
@@ -79,11 +87,20 @@ def _norm_sig(sig):
return gm.norm_sig(sig)
def _typedefs(text):
"""name -> (offset, normalized_body) for every typedef the text defines."""
def _typedefs(text, depth=None):
"""name -> [(offset, normalized_body)] for every FILE-SCOPE typedef the text defines.
BLOCK SCOPE IS A SCOPE (P31 S54, R39). Two functions may each declare their own
`typedef struct {...} Ent_801DFBF8;` INSIDE their bodies -- that is legal C and the project's
drafts do it constantly (it is how a draft stays self-contained for match_one without touching
the TU's namespace). Counting those as definitions produced a DUPLICATE-TYPEDEF FAIL against
a slate whose drafts were each independently byte-verified. `depth` is the brace-depth map the
caller already computed; passing it restricts the scan to depth 0."""
out = {}
for pat in (gm.TYPEDEF_BLOCK, gm.TYPEDEF_PLAIN):
for m in pat.finditer(text):
if depth is not None and depth[m.start()]:
continue
out.setdefault(m.group(1), []).append((m.start(), ' '.join(m.group(0).split())))
return out
@@ -112,7 +129,7 @@ def check_text(path, text):
# byte-identically for weeks. Seven of nine FAILs on a real wave-R slate were comment-borne.
# The masking oracle was already computed one line above and simply was not used here; because
# _mask is length-preserving, every reported offset stays valid.
tds = _typedefs(masked)
tds = _typedefs(masked, depth)
# 3. duplicate typedef -- ANY redefinition, identical body or not.
# C89 has no "compatible redefinition" allowance for typedefs: `typedef struct {...} T;` twice
@@ -183,7 +200,14 @@ def check_text(path, text):
continue
sig = _norm_sig(gm.typesig(d))
if s in decls and gm.sig_conflict(decls[s][1], sig):
findings.append(('FAIL', 'CONFLICTING-EXTERN',
# A BUILT-IN IS A WARNING, NOT AN ERROR -- measured, not assumed (P31 S54). Every
# overlay TU in the fleet declares memcpy twice (`(void*, const void*, u32)` near the
# top, `(void*, void*, s32)` further down) and every one of them COMPILES TODAY. Probed
# against the pinned cc1: two conflicting declarations of `memcpy` give
# "warning: conflicting types for built-in function `memcpy'" and exit 0, while the same
# pair on a non-builtin name gives "conflicting types for `myfun'". Reporting these as
# FAIL sent the reconcile lane hunting a defect the compiler does not have (R39).
findings.append(('WARN' if s in _BUILTINS else 'FAIL', 'CONFLICTING-EXTERN',
f'{path}: `{s}` declared {decls[s][1]} at offset {decls[s][0]} and '
f'{sig} at offset {m.start()}'))
else:
@@ -222,7 +246,17 @@ def main():
slate = json.load(open(a.slate))
kept, dropped = gm.resolve_conflicts(slate)
_n, texts = gm.substitute(kept, write=False)
# MODEL THE DRIVER THAT WILL ACTUALLY BANK THIS SLATE (P31 S54). main goes through gate_main's
# hoist/strip; an overlay goes gate_lane -> gate_stage -> harvest_verify, which strips every
# typedef its target TU already provides. Without this, the first overlay slates this tool could
# see reported DUPLICATE-TYPEDEF for exactly the duplicates the real gate removes.
def _driver_transform(body, tu_path, binary):
if binary == 'main':
return body
return cdecl.strip_provided_typedefs(body, cdecl.typedef_names(tu_path))
_n, texts = gm.substitute(kept, write=False, transform=_driver_transform)
# R32 COVERAGE ASSERTION (P31 S54). Until `gate_main` learned per-binary stub maps, an OVERLAY
# slate resolved to zero stubs and this tool printed "checking 0 substituted file(s) ... clean"