mirror of
https://github.com/Druthulu/BFM-decomp
synced 2026-09-26 13:33:34 -04:00
fix(phase-30 S47): three splitter defects blocking the monolithic-overlay carve
Found while scoping the jr carve for the 3 newly-onboarded binaries. My scoping said "one
unplaceable construct" — it was the first of four layers. Three are fixed here; the fourth is out
of this tool's scope and leaves the carve blocked.
1. asm_label_aliases: the scan could START inside a #define. `#define gte_SetRotMatrix(r0)
__asm__ volatile ("lw $12, 0( %0 );" ...)` is textually `ident(...) __asm__(...)`, and
cdecl._mask blanks string CONTENT — deleting the `;`s that would stop the greedy [^;{}]*.
The match ran 116 lines and swallowed the real `aF8012EFB8 ... __asm__("func_8012EFB8");`,
so the alias never entered the map and addr_of returned None. jr_isolate_all then refused to
carve (R32, correctly), which presented as 112 isolate-fails that looked like a per-binary wall.
Fixed: _mask_cpp_directives() — a preprocessor directive is the other place a match must not
start. Masking comments/strings fixed the comment case and left this one.
2. _split_macro_body returned a `static inline` internal HELPER as the macro's definition, so
_proto_from_lines hoisted `extern static inline void tail_8012F274(...);` into all 41 regions:
invalid C (multiple storage classes) AND the wrong function — the exported definition sits
below the helper and lost its implied declaration. Fixed: skip static definitions
brace-balanced on the masked body. A static helper needs no hoisted declaration at all.
3. A declaration that WRAPS across continuation lines was taken as one line, so half became a
`;`-less extern and the continuation was read as the definition header, producing
`extern __asm__(""); void aF801466F0(...);` in 22 regions. Fixed: accumulate until the
statement terminates, tested on the masked text. Same wrapped-declaration blindness
family_remap._alias_decl_for records fixing at S33 — never propagated here (§134/§139).
Not fixed, and why: jtbl_rodata_pads reports "consumed 0 rodata .align(s) but 4 pad spec(s) given
— table-count drift vs the carve". The carve moves jtbl-owning functions into _jr_ regions but
leaves the pad specs on the residual gap object. jr_isolate_all's docstring states this class is
NOT isolate-fixable; it needs JTBL_PADS repointing in overlays.mk. 122 jr member-slots stay blocked.
Regression-checked: 1,948 macros parse with 0 malformed externs; alias maps unchanged on three
already-carved overlays. No build impact (splitters run offline). Carve reverted, tree clean.
This commit is contained in:
@@ -334,6 +334,45 @@ gcc-2.7.2 never emits on hard errors, so every hard error reads `CC1-FAIL(no-dia
|
||||
blindness: it is how cheap declaration faults get filed as codegen walls. `rtu_match` was fixed for
|
||||
this at T0(b); this classifier was not. **Fix it before reading any future 0% as a wall.**
|
||||
|
||||
## ✅ TASK A3 (jr set) — **56 siblings banked / 11 families**; R22 213/213. Carve BLOCKED at layer 4.
|
||||
`jtbl_family_bank` over 50 matched-exemplar jr families (183 member-slots), one commit per family
|
||||
(its per-sibling revert restores from HEAD, so an uncommitted prior family would be destroyed).
|
||||
**Outcome: BANKED 56 · isolate-fail 112 · gate-fail 10 · carve-fail 4.** R22 **213 passed / 0 of 213**.
|
||||
|
||||
**The 112 isolate-fails were NOT a wall and NOT the reconciled-body trap I predicted.** They fall in
|
||||
exactly THREE overlays — `ov_SC02_037` (39), `ov_SC03_107` (38), `ov_MAIN_012` (36), zero elsewhere —
|
||||
the newly-onboarded binaries from task C, which are **monolithic (0 jr splits vs ~27 elsewhere)**.
|
||||
`jr_isolate_all` cannot carve them, so every member located there fails before the byte-gate runs.
|
||||
**⚠ I recommended `--raw` retries on the reconciled-body hypothesis. That was WRONG** — 39 families
|
||||
scored 0 because their entire member set is those 3 binaries (3 members, 3 failures). Corrected
|
||||
before spending the run, but it was my second bad call of the session (cf. the A1 over-claim).
|
||||
|
||||
### The carve: 3 REAL defects found+fixed, then a 4th that is out of scope
|
||||
My scoping said "one unplaceable construct". **That was wrong — it was the first of four layers.**
|
||||
1. **`asm_label_aliases` scan starts inside a `#define`.** `#define gte_SetRotMatrix(r0) __asm__
|
||||
volatile ("lw $12, 0( %0 );" …)` is textually `ident(…) __asm__(…)`, and `cdecl._mask` blanks
|
||||
string CONTENT — deleting the `;`s that would stop the greedy `[^;{}]*`. The match ran **116
|
||||
lines**, swallowing the real `aF8012EFB8 … __asm__("func_8012EFB8");`, so the alias never entered
|
||||
the map, `addr_of` returned None, and the carve refused (R32, correctly). **Fixed:
|
||||
`_mask_cpp_directives()`** — a preprocessor directive is the other place a match must not START.
|
||||
2. **`_split_macro_body` returned a `static inline` HELPER as the macro's definition**, so
|
||||
`_proto_from_lines` hoisted `extern static inline void tail_8012F274(…);` into all 41 regions —
|
||||
invalid C (`multiple storage classes`) AND the wrong function (the exported def sits below it and
|
||||
lost its implied declaration). **Fixed:** skip `static` definitions brace-balanced on the masked
|
||||
body; a static helper needs no hoisted decl (internal linkage, each region gets its own copy).
|
||||
3. **A declaration that WRAPS across continuation lines** (`extern void aF801466F0(…) \` +
|
||||
`__asm__("func_801466F0"); \`) was taken as one line, so half became a `;`-less extern and the
|
||||
continuation was read as THE DEFINITION HEADER → `extern __asm__(""); void aF801466F0(…);` in 22
|
||||
regions. **Fixed:** accumulate until the statement terminates, tested on the MASKED text.
|
||||
*This is the SAME wrapped-declaration blindness `family_remap._alias_decl_for` records fixing at
|
||||
S33 — never propagated here. Third instance of one class in one file (cookbook §134/§139).*
|
||||
4. **BLOCKED — `jtbl_rodata_pads: consumed 0 rodata .align(s) but 4 pad spec(s) given — table-count
|
||||
drift vs the carve`.** The carve moves jtbl-owning fns into `_jr_` regions but leaves the pad
|
||||
specs on the residual gap object. **`jr_isolate_all`'s own docstring says this class is NOT
|
||||
isolate-fixable.** It needs JTBL_PADS repointing in `overlays.mk` — a separate, larger job.
|
||||
**Carve reverted; tree clean. Fixes 1-3 kept** (regression-checked: 1,948 macros parse, 0 malformed
|
||||
externs; alias maps unchanged on 3 already-carved overlays). **122 jr member-slots remain blocked.**
|
||||
|
||||
## ▶ RESUME HERE — D (and the leftovers below)
|
||||
**C is DONE.** For reference, the command was:
|
||||
`.venv/bin/python tools/dedup_extend.py --binaries ov_SC03_107,ov_MAIN_012,ov_SC02_037`
|
||||
|
||||
@@ -79,6 +79,23 @@ _ALIAS_SCAN = re.compile(
|
||||
r'\b([A-Za-z_]\w*)\s*\([^;{}]*\)\s*__asm__\s*\([^;{}]*\)\s*;', re.S)
|
||||
|
||||
|
||||
def _mask_cpp_directives(text):
|
||||
"""Length-preserving blank of every preprocessor directive, including `\\` continuations.
|
||||
|
||||
A `#define` body is not C declaration text, but it can look exactly like one. Blanking it is
|
||||
the same move `cdecl._mask` makes for comments and strings, for the same reason: a scanner must
|
||||
not be able to START a match somewhere the C grammar does not apply. Newlines are preserved so
|
||||
offsets and line numbers into the original text remain valid."""
|
||||
out, cont = [], False
|
||||
for ln in text.split('\n'):
|
||||
if cont or ln.lstrip().startswith('#'):
|
||||
cont = ln.rstrip().endswith('\\')
|
||||
out.append(' ' * len(ln))
|
||||
else:
|
||||
out.append(ln)
|
||||
return '\n'.join(out)
|
||||
|
||||
|
||||
def asm_label_aliases(src):
|
||||
"""{C identifier -> emitted symbol} for every definition-side asm-label alias in `src`.
|
||||
|
||||
@@ -102,7 +119,21 @@ def asm_label_aliases(src):
|
||||
# `_mask` also blanks string CONTENT *and its quotes*, so the scan pattern cannot require them —
|
||||
# `_ALIAS_SCAN` accepts any `__asm__( … )`, and the real symbol is then read back out of `src`
|
||||
# at the same offsets (the mask is length-preserving, which is what makes that legal).
|
||||
masked = cdecl._mask(src)
|
||||
# ...AND A PREPROCESSOR DIRECTIVE IS THE OTHER PLACE A MATCH MUST NOT START (P30 S47).
|
||||
# Masking comments/strings fixed the comment case and left this one. A macro definition
|
||||
# #define gte_SetRotMatrix(r0) __asm__ volatile ( "lw $12, 0( %0 );" … )
|
||||
# is textually `ident( … ) __asm__( … )` — indistinguishable from an alias declaration to
|
||||
# `_ALIAS_SCAN` — and `_mask` blanks string CONTENT, which DELETES the `;` characters inside
|
||||
# those asm strings that would otherwise have stopped the greedy `[^;{}]*`. So the scan starts
|
||||
# in the macro body and runs until the next real `;` **116 lines later**, swallowing the genuine
|
||||
# `s32 aF8012EFB8(…) __asm__("func_8012EFB8");` on the way; `finditer` resumes past the end, so
|
||||
# the alias never enters the map and `addr_of` returns None for its definition.
|
||||
# Consequence measured: `jr_isolate_all` refused to carve ov_SC02_037 / ov_SC03_107 / ov_MAIN_012
|
||||
# (R32, correctly — it will not rewrite a file it cannot fully place), which left 121 jr
|
||||
# functions uncarved and 122 jr member-slots unreachable, presenting as 112 `isolate-fail`s that
|
||||
# looked like a per-binary wall.
|
||||
# The mask is length-preserving, so offsets into `src` stay valid.
|
||||
masked = _mask_cpp_directives(cdecl._mask(src))
|
||||
out = {}
|
||||
for m in _ALIAS_SCAN.finditer(masked):
|
||||
real = _ALIAS_DECL.search(src[m.start():m.end()])
|
||||
@@ -413,12 +444,48 @@ def _split_macro_body(body):
|
||||
if len(mbody) != len(body):
|
||||
mbody = body
|
||||
out = []
|
||||
for k, ln in enumerate(body):
|
||||
s = ln.strip()
|
||||
k, n = 0, len(body)
|
||||
while k < n:
|
||||
s = body[k].strip()
|
||||
if mbody[k].strip() == "":
|
||||
k += 1
|
||||
continue
|
||||
if s.startswith("extern"):
|
||||
out.append(s)
|
||||
# A DECLARATION MAY WRAP ACROSS CONTINUATION LINES (P30 S47, byte-witnessed):
|
||||
# extern void aF801466F0(u16 a0, …) \
|
||||
# __asm__("func_801466F0"); \
|
||||
# Taking one line as one declaration emitted the first half as a `;`-less extern AND
|
||||
# then treated `__asm__("func_801466F0");` as THE DEFINITION HEADER — so the proto
|
||||
# generator produced `extern __asm__(""); void aF801466F0(…);` in all 22 regions that
|
||||
# hoist it. Accumulate until the statement actually terminates, testing the MASKED text
|
||||
# so a `;` inside a string or comment cannot end it early.
|
||||
# Same line-oriented blindness `family_remap._alias_decl_for` records fixing at S33 for
|
||||
# this exact wrapped-alias shape; it was never propagated here (cookbook §134/§139).
|
||||
acc = [s]
|
||||
while ";" not in mbody[k] and k + 1 < n:
|
||||
k += 1
|
||||
acc.append(body[k].strip())
|
||||
out.append(" ".join(acc))
|
||||
k += 1
|
||||
continue
|
||||
# A `static` DEFINITION inside a macro body is an INTERNAL HELPER, not the macro's
|
||||
# principal definition (P30 S47, byte-witnessed). Stopping here returned the helper as
|
||||
# "the definition", so `_proto_from_lines` hoisted `extern static inline void
|
||||
# tail_8012F274(s32 *in, s32 e);` into all 41 carved regions of ov_SC02_037 — which is
|
||||
# BOTH invalid C (`multiple storage classes`) and the wrong function: the macro's exported
|
||||
# definition sits BELOW the helper and lost its implied declaration entirely.
|
||||
# A static helper needs no hoisted declaration at all: it has internal linkage, and every
|
||||
# region that instantiates the macro gets its own copy from the expansion.
|
||||
# Skip it brace-balanced (on the MASKED body, so a brace in a string or comment cannot
|
||||
# unbalance the scan) and keep looking for the exported definition.
|
||||
if re.match(r'static\b', s):
|
||||
depth, seen = 0, False
|
||||
while k < n:
|
||||
depth += mbody[k].count("{") - mbody[k].count("}")
|
||||
seen = seen or "{" in mbody[k]
|
||||
k += 1
|
||||
if seen and depth <= 0:
|
||||
break
|
||||
continue
|
||||
return out, body[k:] # the definition header — the body starts here
|
||||
return out, []
|
||||
|
||||
Reference in New Issue
Block a user