mirror of
https://github.com/Druthulu/BFM-decomp
synced 2026-09-29 15:18:24 -04:00
feat(phase-29): T16.2 — GATE A passed on measured numbers; the §20 DEF-conflict doctrine refuted per-overlay; func_8012CC88 banked
S0 measured all 36 stranded drafts in 9.2s, two oracles agreeing 36/36. - POPULATION CORRECTED (R14): the audit's "~92% byte-correct" is a whole-wave figure; among the STRANDED residue match_one says 24/36 MATCH / 11 near / 1 ERR = 67%. The 11 near are unfinished drafts, not integration problems -- and they are exactly the ones that compile and DIFF. - BLOCKERS (they STACK; cc1 reveals only the first): self_decl_hdr=21, callee_decl=19, data_decl=16, self_decl_tu=5, local_type=5. Per function by MAX tier: T0=6, T1=26, not-integration=4. - T3 BASELINE, measured free: the existing draft-side ladder clears callee_decl 19->3 and data_decl 16->0 yet converts 1 of 36 to compiling, which then DIFFs -- §61d verbatim. The wave's gate orchestration was NOT broken; the ladder simply cannot reach this population. - NEW tools/demacroize.py: the conflicting extern lives INSIDE a DEFINE_func_* macro BODY, so it exists only where instantiated. Expanding those instantiations in the overlay's OWN TU, correcting only the conflicting decl to the draft's byte-true sig (never dropping it, §57a-1), dissolves the conflict with nothing written outside src/ov_SC07_006/. This is §63's own unexplored "per-overlay-local decl" -- and it refutes §20's "the DEF-conflict class is byte-proven unrecoverable by text transform" for the per-overlay case. - MEASURED on the 14 clean candidates: 13 MATCH / 1 DIFF in the real TU. - END-TO-END: func_8012CC88 banked whole-binary BYTE-IDENTICAL (stub gone per grep, not per report); R22 clean-fleet 140 passed / 0 failed of 140 -> the T1 blast-radius claim validated empirically (the difference from fix_header_decl, which broke 139/140 from the same per-binary green light). - GATE A: 32/36 real decl errors (>=12) and 13/36 rtu-MATCH simulated (>=12). PASS. Price stated honestly: a de-macroized bank is x1 -- full distinct-code credit, ~1/138 of instr.
This commit is contained in:
Drew T
@@ -2132,4 +2132,40 @@ conditional) · main-EXE/B9 + GLM/B6 + resident's 14 walls (P30) · behemoths B7
|
||||
**Smoke test (3 fns, `-v`): both oracles AGREE 3/3, 0 static-only, 0 cc1-only.** Not yet evidence
|
||||
about the population — that is T2.
|
||||
|
||||
- **✅ 2026-07-24 (SESSION-16, Max) — Task 16 / T2 = S0: KILL GATE A PASSED ON MEASURED NUMBERS, and
|
||||
the §20 "DEF-conflict is unrecoverable by text transform" doctrine is REFUTED for the per-overlay case.**
|
||||
Whole probe over all 36 stranded drafts: **9.2 s**. Both oracles agree 36/36 (0 static-only, 0 cc1-only).
|
||||
**THE POPULATION, corrected (R14).** The SESSION-15 audit's "~92% of drafts are byte-correct" is a
|
||||
WHOLE-WAVE figure. Among the STRANDED residue, `match_one` says **24/36 MATCH · 11 near · 1 ERR = 67%**.
|
||||
The 11 `near` are unfinished drafts, NOT integration problems — and they are exactly the 4 that
|
||||
compile in their real TU and DIFF. So "we strand ~16 paid-for CORRECT functions per wave" is ~11,
|
||||
not 16, and one third of the "stranded correct" premise was never correct.
|
||||
**BLOCKERS (per-blocker counts; they STACK — cc1 reveals only the first):** `self_decl_hdr` **21** ·
|
||||
`callee_decl` 19 · `data_decl` 16 · `self_decl_tu` 5 · `local_type` 5. **Per function by MAX tier:
|
||||
T0=6 · T1=26 · not-an-integration-problem=4.** The dominant blocker is the SHARED-HEADER self-decl —
|
||||
far bigger than the pre-probe static inference (6–7 of 16) predicted.
|
||||
**T3's baseline, measured early and for free.** Running the EXISTING draft-side ladder transforms
|
||||
(`cast_call_sites` + `reconcile_tu`) over all 36 clears what they target — `callee_decl` 19→3,
|
||||
`data_decl` 16→**0** — and converts exactly **1 of 36** from CC1-FAIL to compiling, which then
|
||||
**DIFFs**. That is §61d reproducing verbatim: dissolving the plumbing reveals what the plumbing hid.
|
||||
The existing ladder cannot bank this population; the wave's gate orchestration was NOT broken.
|
||||
**THE ESCAPE, BUILT AND BYTE-PROVEN — `tools/demacroize.py` (NEW).** The conflicting `extern` lives
|
||||
INSIDE a `DEFINE_func_*` macro BODY in `engine_core.h`, so it exists only where the macro is
|
||||
INSTANTIATED. Replacing those instantiations, in this overlay's own TU, with the macro's own
|
||||
expansion — correcting ONLY the conflicting decl to the draft's byte-true sig, never dropping it
|
||||
(§57a-1) — dissolves the conflict while writing nothing outside `src/ov_SC07_006/`. This is the
|
||||
"per-overlay-local decl" §63's own note named as the unexplored alternative.
|
||||
**Measured on the 14 clean candidates** (match_one MATCH, `self_decl_hdr` their only remaining
|
||||
blocker): **13 MATCH / 1 DIFF** in the REAL TU (1–12 macro instantiations each; `func_8014F3E8`
|
||||
DIFFs at 28-vs-32 ins — a short draft, not an integration failure).
|
||||
**END-TO-END PROOF (the arbiter, not the proxy):** `func_8012CC88` (6 instantiations)
|
||||
→ `harvest_verify --chunk 1` → **verified 1 / failed 0, BYTE-IDENTICAL**; stub gone, confirmed by
|
||||
`grep INCLUDE_ASM` on the source, never the report (§55b trap 4).
|
||||
**THE TIER CLAIM, VALIDATED EMPIRICALLY:** full R22 `make clean && extract-all && check-all` →
|
||||
**140 passed, 0 failed of 140** (80 s). A `src/<binary>/**`-confined edit does not perturb the
|
||||
fleet — which is precisely the difference from `fix_header_decl`, whose `src/shared/` write broke
|
||||
139/140 under the same per-binary green light (§63 UPDATE).
|
||||
**GATE A:** cond-1 ≥12/36 with a real decl/type error → **32/36 PASS**; cond-2 ≥12/36 rtu-MATCH
|
||||
once the blocker is simulated → **13/36 PASS**. Proceed.
|
||||
|
||||
|
||||
|
||||
@@ -1206,25 +1206,101 @@ DEFINE_func_8012CAE4() /* dedup: shared engine-core @0x8012cae4 (src/shared) */
|
||||
DEFINE_func_8012CB64() /* dedup: shared engine-core @0x8012cb64 (src/shared) */
|
||||
|
||||
|
||||
DEFINE_func_8012CBA4() /* dedup: shared engine-core @0x8012cba4 (src/shared) */
|
||||
/* de-macroized: per-overlay-local decl for func_8012CC88 (byte-true sig); do NOT re-macroize */
|
||||
extern s32 func_8012CC88(s32, s32, s32);
|
||||
extern u8 D_800D3918[];
|
||||
void func_8012CBA4(s32 a0) {
|
||||
func_8012CC88(a0, 0, (s32)D_800D3918);
|
||||
} /* dedup: shared engine-core @0x8012cba4 (src/shared) */
|
||||
|
||||
|
||||
DEFINE_func_8012CBCC() /* dedup: shared engine-core @0x8012cbcc (src/shared) */
|
||||
/* de-macroized: per-overlay-local decl for func_8012CC88 (byte-true sig); do NOT re-macroize */
|
||||
extern s32 func_8012CC88(s32, s32, s32);
|
||||
extern u8 D_800D3918[];
|
||||
void func_8012CBCC(s32 a0) {
|
||||
func_8012CC88(a0, 1, (s32)D_800D3918);
|
||||
} /* dedup: shared engine-core @0x8012cbcc (src/shared) */
|
||||
|
||||
|
||||
DEFINE_func_8012CBF4() /* dedup: shared engine-core @0x8012cbf4 (src/shared) */
|
||||
/* de-macroized: per-overlay-local decl for func_8012CC88 (byte-true sig); do NOT re-macroize */
|
||||
extern u8 D_800D3918[];
|
||||
extern s32 func_8012CC88(s32, s32, s32);
|
||||
void func_8012CBF4(s32 a0) {
|
||||
func_8012CC88(a0, 0x11, (s32)D_800D3918);
|
||||
} /* dedup: shared engine-core @0x8012cbf4 (src/shared) */
|
||||
|
||||
|
||||
DEFINE_func_8012CC1C() /* dedup: shared engine-core @0x8012cc1c (src/shared) */
|
||||
/* de-macroized: per-overlay-local decl for func_8012CC88 (byte-true sig); do NOT re-macroize */
|
||||
extern s32 func_8012CC88(s32, s32, s32);
|
||||
void func_8012CC1C(s32 arg0, s32 arg1) {
|
||||
func_8012CC88(arg0, 0, arg1);
|
||||
} /* dedup: shared engine-core @0x8012cc1c (src/shared) */
|
||||
|
||||
|
||||
DEFINE_func_8012CC40() /* dedup: shared engine-core @0x8012cc40 (src/shared) */
|
||||
/* de-macroized: per-overlay-local decl for func_8012CC88 (byte-true sig); do NOT re-macroize */
|
||||
extern s32 func_8012CC88(s32, s32, s32);
|
||||
void func_8012CC40(s32 arg0, s32 arg1) {
|
||||
func_8012CC88(arg0, 1, arg1);
|
||||
} /* dedup: shared engine-core @0x8012cc40 (src/shared) */
|
||||
|
||||
|
||||
DEFINE_func_8012CC64() /* dedup: shared engine-core @0x8012cc64 (src/shared) */
|
||||
/* de-macroized: per-overlay-local decl for func_8012CC88 (byte-true sig); do NOT re-macroize */
|
||||
extern s32 func_8012CC88(s32, s32, s32);
|
||||
void func_8012CC64(s32 a0, s32 a1) {
|
||||
func_8012CC88(a0, 0x11, a1);
|
||||
} /* dedup: shared engine-core @0x8012cc64 (src/shared) */
|
||||
|
||||
|
||||
INCLUDE_ASM("asm/ov_SC07_006/nonmatchings/ov_SC07_006", func_8012CC88);
|
||||
// @class: other
|
||||
// @stuck: none — MATCH (105 ins). BANK NOTE: engine_core.h declares `extern void func_8012CC88(s32,s32,s32)` at 5 sites; the return value is REAL (void-return probe = 102 ins / 13 mismatch, LENGTH-DRIFT -3), so the HEADER decl must be lifted void->s32 (§63 fix_header_decl.py), not the draft.
|
||||
#include "common.h"
|
||||
|
||||
extern s32 func_8012CEB0(s32 a0, s32 a1, s32 a2);
|
||||
|
||||
/* a0 = actor/entity base, a1 = mode passed through to func_8012CEB0, a2 = s16[3] offset vector.
|
||||
* Two 8-byte s16 vectors live at sp+0x10 and sp+0x18 and are passed by address; the second one
|
||||
* is written back to the entity's hi-16 fixed-point position after the offset is removed. */
|
||||
s32 func_8012CC88(s32 a0, s32 a1, s32 a2) {
|
||||
typedef struct { s16 vx, vy, vz, pad; } SV3_8012CC88;
|
||||
SV3_8012CC88 sp10;
|
||||
SV3_8012CC88 sp18;
|
||||
s32 v0;
|
||||
|
||||
sp10.vx = *(u16*)(a0 + 0x3A);
|
||||
sp10.vy = *(u16*)(a0 + 0x3E);
|
||||
sp10.vz = *(u16*)(a0 + 0x42);
|
||||
sp10.vx += *(u16*)(a2 + 0);
|
||||
sp10.vy += *(u16*)(a2 + 2);
|
||||
sp10.vz += *(u16*)(a2 + 4);
|
||||
|
||||
*(s32*)(a0 + 0x10) += *(s32*)(a0 + 0x44);
|
||||
*(s32*)(a0 + 0x14) += *(s32*)(a0 + 0x48);
|
||||
*(s32*)(a0 + 0x18) += *(s32*)(a0 + 0x4C);
|
||||
*(s32*)(a0 + 0x04) += *(s32*)(a0 + 0x10);
|
||||
*(s32*)(a0 + 0x08) += *(s32*)(a0 + 0x14);
|
||||
*(s32*)(a0 + 0x0C) += *(s32*)(a0 + 0x18);
|
||||
|
||||
sp18.vx = *(u16*)(a0 + 0x06);
|
||||
sp18.vy = *(u16*)(a0 + 0x0A);
|
||||
sp18.vz = *(u16*)(a0 + 0x0E);
|
||||
sp18.vx += *(u16*)(a2 + 0);
|
||||
sp18.vy += *(u16*)(a2 + 2);
|
||||
sp18.vz += *(u16*)(a2 + 4);
|
||||
|
||||
v0 = func_8012CEB0((s32)&sp10, (s32)&sp18, a1);
|
||||
|
||||
sp18.vx -= *(u16*)(a2 + 0);
|
||||
sp18.vy -= *(u16*)(a2 + 2);
|
||||
sp18.vz -= *(u16*)(a2 + 4);
|
||||
*(s16*)(a0 + 0x06) = sp18.vx;
|
||||
*(s16*)(a0 + 0x0A) = sp18.vy;
|
||||
*(s16*)(a0 + 0x0E) = sp18.vz;
|
||||
|
||||
if (v0 & 0x6000) {
|
||||
*(s32*)(a0 + 0x14) = 0;
|
||||
}
|
||||
return v0;
|
||||
}
|
||||
|
||||
DEFINE_func_8012CE2C() /* dedup: shared engine-core @0x8012ce2c (src/shared) */
|
||||
|
||||
|
||||
@@ -138,6 +138,11 @@ def static_verdict(fn, stub, draft_text):
|
||||
if d.is_definition and d.name == fn:
|
||||
out.append(('self_decl_hdr' if src.startswith('hdr') else 'self_decl_tu',
|
||||
'%s declares %s | def is %s' % (src, other.type, d.type)))
|
||||
elif 'typedef' in (d.storage, other.storage):
|
||||
# cc1 forbids redeclaring a typedef AT ALL, even identically (`compatible` encodes it).
|
||||
# This is a TYPE conflict, not a data conflict: the fix is to rename/drop the draft's
|
||||
# copy, which is draft-only — routing it as `data_decl` would send it at the wrong tool.
|
||||
out.append(('local_type', '%s: %s declares %s | draft %s' % (d.name, src, other.type, d.type)))
|
||||
else:
|
||||
klass = 'callee_decl' if d.kind == 'func' else 'data_decl'
|
||||
out.append((klass, '%s: %s declares %s | draft %s' % (d.name, src, other.type, d.type)))
|
||||
|
||||
@@ -0,0 +1,150 @@
|
||||
#!/usr/bin/env python3
|
||||
"""demacroize.py — the per-overlay-local escape from a shared-header self-decl conflict.
|
||||
|
||||
THE BLOCKER. A byte-correct draft for `func_X` fails the whole-binary gate with
|
||||
`conflicting types for func_X`, because a `DEFINE_func_*` macro in `src/shared/engine_core.h`
|
||||
forward-declares `func_X` with a caller-derived signature the byte-true definition cannot
|
||||
satisfy (typically `void` where the real return is live). This is the single largest stranded
|
||||
class.
|
||||
|
||||
WHY THE OBVIOUS FIXES ARE WRONG.
|
||||
* Rewriting the DRAFT to the header's signature compiles but byte-MISMATCHes — the body
|
||||
genuinely needs its own signature (§20; §58b: forcing a `void` return dropped the return
|
||||
computation). The draft is byte-truth; the header decl is a stub-era guess.
|
||||
* Rewriting the SHARED HEADER decl (`fix_header_decl`) is fleet-blind: it passed the
|
||||
per-binary gate on 3 of 3 and then failed R22 139/140, because that decl reaches ~137 other
|
||||
overlays where those functions are ALREADY matched and a caller's codegen shifts under the
|
||||
widened declaration (§63 UPDATE). Off-limits.
|
||||
|
||||
THE ESCAPE. The conflicting `extern` lives INSIDE the macro BODY, so it only exists where the
|
||||
macro is INSTANTIATED. Replacing that instantiation, in this overlay's own TU, with the macro's
|
||||
own expansion — with only the conflicting declaration corrected to the draft's byte-true
|
||||
signature — dissolves the conflict while writing nothing outside `src/<binary>/`. Every other
|
||||
overlay is textually untouched, so no R22 risk is created by construction. This is the
|
||||
"per-overlay-local decl" §63's own note named as the unexplored alternative.
|
||||
|
||||
THE PRICE, STATED HONESTLY. The de-macroized function can no longer be propagated ×138 from the
|
||||
shared macro (`dedup_propagate` would re-macroize it). A bank recovered this way is ×1: it
|
||||
credits the FULL distinct-code unit (progress.py marks an h_exact class matched if ANY instance
|
||||
is) but only ~1/138 of the instruction-weighted headline.
|
||||
|
||||
CAVEAT (why --apply must still be byte-gated). The macro's own function is ALREADY matched in
|
||||
this binary. Correcting the declaration it sees can shift ITS codegen — the §63 failure mode,
|
||||
relocated somewhere the per-binary gate can actually see it. The whole-binary byte-gate is the
|
||||
arbiter (G3/P9); this tool only prepares the edit.
|
||||
|
||||
tools/demacroize.py --binary ov_SC07_006 --fn func_8012CC88 --draft <draft.c> --emit-edits
|
||||
tools/demacroize.py --binary ov_SC07_006 --fn func_8012CC88 --draft <draft.c> --apply
|
||||
"""
|
||||
import argparse, os, re, sys
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
REPO = os.path.dirname(HERE)
|
||||
sys.path.insert(0, HERE)
|
||||
import cdecl
|
||||
import corpus
|
||||
|
||||
ENGINE_CORE = 'src/shared/engine_core.h'
|
||||
|
||||
|
||||
def macro_bodies(header=ENGINE_CORE):
|
||||
"""{macro: body_text} for every DEFINE_func_* in the shared header (continuations joined)."""
|
||||
text = open(os.path.join(REPO, header), errors='replace').read()
|
||||
out = {}
|
||||
for m in re.finditer(r'^#define\s+(DEFINE_func_[0-9A-Fa-f]+)\s*\(\s*\)', text, re.M):
|
||||
i, body = text.find('\n', m.end()), []
|
||||
if i < 0:
|
||||
continue
|
||||
i += 1
|
||||
while i < len(text):
|
||||
nl = text.find('\n', i)
|
||||
line = text[i:nl] if nl >= 0 else text[i:]
|
||||
body.append(line.rstrip().rstrip('\\').rstrip())
|
||||
if not line.rstrip().endswith('\\') or nl < 0:
|
||||
break
|
||||
i = nl + 1
|
||||
out[m.group(1)] = '\n'.join(body)
|
||||
return out
|
||||
|
||||
|
||||
def draft_decl(fn, draft_text):
|
||||
"""`extern <byte-true sig>;` reconstructed from the draft's own definition (cdecl renders it,
|
||||
so the spelling is canonical rather than copied source text)."""
|
||||
for d in cdecl.declarations(cdecl.split_statements(draft_text), path='<draft>'):
|
||||
if d.name == fn and d.is_definition:
|
||||
return d.declaration(storage='extern')
|
||||
raise SystemExit('demacroize: no definition of %s in the draft' % fn)
|
||||
|
||||
|
||||
def plan(binary, fn, draft_text):
|
||||
"""[(macro, old_instantiation, new_expansion)] — every macro instantiated in fn's home TU
|
||||
whose body declares fn incompatibly with the draft's definition."""
|
||||
stub = {s.symbol: s for s in corpus.stubs(binary).values()}.get(fn)
|
||||
if stub is None:
|
||||
raise SystemExit('demacroize: %s is not an INCLUDE_ASM stub in %s (already banked?)' % (fn, binary))
|
||||
tu_path = os.path.join(REPO, stub.path)
|
||||
tu_text = open(tu_path, errors='replace').read()
|
||||
want = draft_decl(fn, draft_text)
|
||||
want_d = cdecl.parse(want)[0]
|
||||
|
||||
out = []
|
||||
for macro, body in sorted(macro_bodies().items()):
|
||||
if not re.search(r'\b%s\s*\(\s*\)' % re.escape(macro), tu_text):
|
||||
continue # not instantiated in THIS TU
|
||||
hit = None
|
||||
for stmt in re.findall(r'\bextern\b[^;{}]*;', body):
|
||||
try:
|
||||
ds = cdecl.parse(stmt)
|
||||
except cdecl.CDeclError:
|
||||
continue
|
||||
for d in ds:
|
||||
if d.name == fn and not (cdecl.compatible(d, want_d) and cdecl.compatible(want_d, d)):
|
||||
hit = stmt
|
||||
if hit is None:
|
||||
continue
|
||||
# Correct ONLY the conflicting declaration; never DROP it (§57a correction 1 — dropping a
|
||||
# decl breaks the def-after-caller order the caller relies on).
|
||||
out.append((macro, stub, hit, body.replace(hit, want, 1)))
|
||||
return tu_path, out
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument('--binary', required=True)
|
||||
ap.add_argument('--fn', required=True)
|
||||
ap.add_argument('--draft', required=True)
|
||||
g = ap.add_mutually_exclusive_group(required=True)
|
||||
g.add_argument('--emit-edits', action='store_true', help='print //@EDIT lines for rtu_match (read-only)')
|
||||
g.add_argument('--apply', action='store_true', help='rewrite the TU in place (T1; byte-gate after)')
|
||||
a = ap.parse_args()
|
||||
|
||||
draft_text = open(a.draft, errors='replace').read()
|
||||
tu_path, items = plan(a.binary, a.fn, draft_text)
|
||||
if not items:
|
||||
print('demacroize: nothing to do for %s (no instantiated macro declares it incompatibly)' % a.fn)
|
||||
return 1
|
||||
|
||||
if a.emit_edits:
|
||||
for macro, _stub, _old, new in items:
|
||||
# rtu_match applies these to the TU before splicing; \n is unescaped there.
|
||||
print('//@EDIT %s()||%s' % (macro, new.replace('\\', '\\\\').replace('\n', '\\n')))
|
||||
print('// %d macro instantiation(s) in %s' % (len(items), os.path.relpath(tu_path, REPO)),
|
||||
file=sys.stderr)
|
||||
return 0
|
||||
|
||||
text = open(tu_path, errors='replace').read()
|
||||
for macro, _stub, _old, new in items:
|
||||
pat = re.compile(r'^([ \t]*)%s\(\s*\)' % re.escape(macro), re.M)
|
||||
if not pat.search(text):
|
||||
raise SystemExit('demacroize: instantiation of %s not found for in-place edit' % macro)
|
||||
text = pat.sub(lambda m: '%s/* de-macroized: per-overlay-local decl for %s (byte-true sig); '
|
||||
'do NOT re-macroize */\n%s' % (m.group(1), a.fn, new), text, count=1)
|
||||
open(tu_path, 'w').write(text)
|
||||
print('demacroize: expanded %d macro(s) in %s for %s'
|
||||
% (len(items), os.path.relpath(tu_path, REPO), a.fn))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user