mirror of
https://github.com/Druthulu/BFM-decomp
synced 2026-09-27 22:45:39 -04:00
85fb289db582d842fc41dc059fa187bb992e76ea
11 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Drew T
|
ec749fa584 |
fix(phase-30 S48-0b): JTBL_PADS follows its span through jr isolation
The 0b blocker was not "the pads line is left behind" alone — it fails two different ways, and the second one is silent: * bare isolate + `make build`: the stale line arms the pads filter on the RESIDUAL object, which emits no jump table -> `jtbl_rodata_pads: consumed 0 rodata .align(s) but 4 pad spec(s) given` (S47). * isolate -> jtbl_carve (the jtbl_family_bank path): `set_pads_vars` regenerates the block keyed by the CURRENT subseg names, finds no prior spec under the new `_jr_<addr>` name, and DROPS the line. cc1's natural `.align 3` then pads the span's non-8-aligned interior tables and the image shifts — reported only as `built, bytes differ`. - jr_isolate_all.repoint_overlays_mk: repoint the `build/src/<ov>/<sub>.o: JTBL_PADS` target with the `--order` leaf whenever a carve moves; refuse loud if the old object still hosts a .rodata piece (R32). - jtbl_carve.set_pads_vars: second, disagreeing oracle (R34) — refuse when a spec would vanish for a subseg no longer in the carve set (rename/merge drift), instead of silently emitting a padless object. R37 probe: func_801789AC -> ov_SC02_037 went `built, bytes differ` -> BANKED on the whole-binary byte gate. ov_SC02_037's spec is 0,0,0,0 over tables +0x0,+0x14,+0x34,+0x4c — load-bearing (span start is 4 mod 8). |
||
|
Drew T
|
2483fc902a |
fix(tools): jr_isolate_all was SILENTLY DELETING asm-label-alias definitions during a repartition
ROOT CAUSE (byte-witnessed, P30 S38 — the fifth tool with this same blindness).
A function banked under the §37/§73 DEFINITION-SIDE ASM-LABEL ALIAS form is spelled with a private
C identifier and bound to its real symbol by a GNU asm label:
void aF8018A860(s32, s16 *, u8 *, u8 *) __asm__("func_80183AF8"); <- decl, stays in preamble
void aF8018A860(s32, s16 *, u8 *, u8 *) { ... } <- THIS emits func_80183AF8
overlay_src_split.addr_of() resolves `func_<hex>` arithmetically and everything else through `syms`.
`aF8018A860` matches NEITHER, so it returned None — and partition() keeps only items with a
resolved address, so the definition was dropped from EVERY region. The file was then rewritten
without it and nothing said so. One carve of ov_SC02_028 deleted the definitions emitting BOTH
func_80183AF8 and func_80184268; the overlay stopped linking with `undefined reference`, and six
wave-6 drafts were written off against that as a plumbing/compiler wall.
TWO FIXES:
- CAUSE: overlay_src_split now builds an asm-label alias map from the source and resolves a
definition through its EMITTED SYMBOL rather than its C name (verified: aF8018A860 -> 0x80183AF8,
aF8018AFD0 -> 0x80184268 — exactly the two symbols the link was missing).
- SILENCE: partition() and jr_isolate_all._partition() now REFUSE to rewrite a file when any
construct's address does not resolve (R32), instead of discarding it. That guard alone would
have surfaced this the first time it happened.
RESULT: 3 of the 6 alias-class wave-6 drafts bank immediately, for ZERO agent tokens —
func_801884D8 (137 ins) · func_80180B04 (251) · func_801380E0 (438). R22 clean-fleet 140/140.
The other 3 (the three LARGEST: 557/513/710 ins) have a second, size-correlated cause — open.
NOTE FOR THE FLYWHEEL: family_remap._alias_decl_for ALREADY handled this exact form, and its
docstring records the identical lesson ("that blindness was the WHOLE of the h_seq sweep's 137 'no
matched unit' skips. The tool, not the compiler (R35)"). The fix was never propagated. The alias
form needs ONE shared oracle, the way §134 comment-masking ended up on cdecl._mask — five tools
have now independently rediscovered it.
|
||
|
Drew T
|
faf4547345 |
feat(phase-29): func_8017C954 BANKED — jr carve chain cleared; a shared type was PRESENT but INVISIBLE
- BANKED (1,194 ins, ×1 distinct-code). Chain cleared, each step byte-gated before the next was
built on it: one-line fix to jr_isolate_all._engine_types() -> jr_isolate_all --only
func_8017C954 (2 fns / 1 object, NOT the bare 47-fn / 21-object resegment) -> BYTE-IDENTICAL
b7b0d4ae -> jtbl_carve --func func_8017C954 (44-piece carve set + interleave order) ->
BYTE-IDENTICAL -> harvest_verify VERIFIED BYTE-IDENTICAL -> R22 clean-fleet 140/140,
tools-health OK. instr 80.5 -> 80.6%; distinct-code 3,842,906 -> 3,844,100.
- THE DEFECT (tools/jr_isolate_all.py): _engine_types() harvested shared type names with four
patterns -- `typedef ... X;`, `} X;`, forward-decl `struct X;`, fn-ptr typedef -- and a TAGGED
DEFINITION WITH A BODY matches NONE of them. So `struct PW8017E6D8 { int w; }
__attribute__((packed));` at engine_types.h:658 was present in the shared header yet invisible
to the carried-type check, and `extern struct PW8017E6D8 D_801E1EC4;` could not be placed.
MEASURED BLAST RADIUS: 77 such tags in engine_types.h were invisible. One added pattern fixes
all 77.
- WHY THIS COST 20 MINUTES INSTEAD OF A MYSTERY BYTE-DIFF THREE PHASES LATER: the Phase-26 audit
had already turned this predicate's SILENT DROP into a LOUD REFUSAL. The original bug dropped
4,040 col-0 decls, 683 of them function PROTOTYPES -- and a dropped prototype is a SILENT
BYTE-CHANGER (C89 implicit `int f()`; return type drives delay-slot fill in this codebase). The
refusal named the exact symbols and the exact remedy. A loud "I cannot place this" is worth far
more than a green build -- the audit paying for itself, live.
- §81: the 3-step jr-carve chain + why match_one CANNOT see the problem (it masks jal/HI16/LO16,
so a jump-table function reports MATCH while the whole-binary gate reports DIFF, correctly).
Detect with `grep -cE 'jr \$(v0|v1|a0|t[0-9])'` on the target .s + a jtbl_ in asm/<ov>/data/.
ALWAYS use --only: bare would have resegmented 47 jr-functions across 21 objects.
|
||
|
Drew T
|
ea20bdf9f3 |
fix(phase-26a): A9g — jr_inventory: retire the ephemeral roster, derive banked from the image (R33)
jr_inventory's `banked` set was filtered by an EPHEMERAL, gitignored .run/banked_func_*.json roster: a `rm -rf .run` / fresh clone would blind ALL banked jr at once, cross-address siblings (roster named after the exemplar) were structurally invisible, and non-leader banked jr were missed. "The purest R33 case in the group" (audit). FIX (the audit's exact prescription): delete the roster glob + `cand` filter; `banked` is DERIVED FROM THE IMAGE — a real-C def/define fn is a banked jr iff family_remap.reloc_targets shows it references a committed .rodata carve offset (config + image, both durable; cross-address- and non-leader-immune). R32 assertion: every committed carve must resolve to EXACTLY ONE owner or abort (a stranded/duplicated carve is the §8b func_801734BC incident, never silent). Also fixed the adjacent finding: the asm_jr scan's func_-fullmatch dropped the curated-name listCdBuffer jr; now resolved via oss.addr_of(). (The --only path's own fullmatch is left — it parses user input, not the corpus.) Perf: read the overlay image ONCE and pass it to reloc_targets(..., data=) — a new backward-compatible param on family_remap (regression: 0/80 mismatch vs the re-read path). Verified: data-param behavior-identical; the R33 win — ov_SC02_000 now finds the cross-address sibling func_8017FCB0 the roster missed; full-fleet parallel run = 134/134 OK, 0 false aborts, 1336 banked jr == 1336 carves -> 1:1 ownership holds fleet-wide. Byte-safe: jr_isolate_all is not in the make build/extract path (R22-neutral); the change makes future isolations strictly more correct. |
||
|
Drew T
|
2086b15b48 |
fix(phase-26a): A8 — jr_isolate_all silently dropped 683 prototypes: a LATENT BYTE-CHANGER
_file_scope_decls() hoists a region's file-scope decls into the carried layer. Its _SAFE_TYPE guard
only ever whitelisted BUILTIN base types — so a decl naming a carried file-local type was recognised
by _HOIST_RE as hoistable and then SILENTLY DROPPED.
THE COMMENT ON _SAFE_TYPE DESCRIBED A FIX THAT WAS NEVER APPLIED TO THE CODE. Verbatim: "one naming a
FILE-LOCAL type is only safe once that type is carried too — which file_scope_types() now does, so such
decls ride along after their typedef." The predicate never implemented it.
MEASURED (audit: 4,040 fleet-wide; independently re-measured here over 4 overlays: 189 drops, 32 of
them function PROTOTYPES):
* The 3,357 dropped DATA externs are LOUD — an undeclared identifier is a compile error.
* The 683 dropped function PROTOTYPES are NOT. In C89 an undeclared function is implicitly `int f()`,
so the TU still COMPILES — with the wrong return type and lost pointer-ness. And this project has
BYTE-PROVEN that the return type drives codegen (cookbook: "schedule — delay-slot fill via void
return type"; ov_SC01_077_after.c carries an `extern int`->`extern void` flip described as
byte-neutral precisely because the return type moves the delay slot).
=> A DROPPED PROTOTYPE IS A SILENT BYTE-CHANGER, armed to fire on the next carve.
Today's split is green only because the source redundantly re-declares externs per fn-group, so most
items happen to carry their own decl. That is luck, not design.
Two of the rejected base types were not even file-local: `uint` (139 drops) and `code_fn` (21) are
DEFINED IN src/shared/engine_types.h, which engine_core.h pulls into every region — the predicate was
rejecting INCLUDE-PROVIDED types it had no reason to reject. `volatile` (3) fell off because the
qualifier group had `const` but not `volatile`.
FIX: implement what the comment promised. A decl is hoistable if its base type is a builtin, OR is
carried by this layer's own file_scope_types, OR is provided by the shared headers (108 type names
parsed from engine_types.h + common.h — including FN-PTR typedefs, whose name sits inside the parens
and which every name-before-';' pattern misses; those were exactly the 5 residual drops).
COVERAGE ASSERTION (R32): a line _HOIST_RE recognises as hoistable but that cannot be placed is now a
HARD FAILURE with the base-type histogram printed, not a silent no-op. Verified safe: 0 residual drops
across 5 overlays. This one check would have surfaced all 4,040 the day the first split shipped.
No build impact (the isolator runs only when carving); --dry-run clean; tree unchanged.
|
||
|
Drew T
|
07ebb5658d |
fix(phase-26): jr_isolate_all empty-region0 skip — cutting an already-isolated region's non-leader works
Cutting func_80178D40 out of ov_SC01_000_jr_801734BC adds the region's banked LEADER (0x801734BC) as a cut too (the one-carve-per-object rule), making region 0 EMPTY (the object's first item IS the first cut) — and region 1's derived name equals the object name, so emitting region 0 duplicated the line exactly -> splat "segments out of order". Skip an empty region 0; region 1 rightly claims the object's offset and name. First sibling then banks through the full chain (isolation validation green -> carve -> --raw remap -> stage ladder -> whole-binary gate): ov_SC01_000 BANKED, included here. The remaining 132 siblings sweep next. |
||
|
Drew T
|
7e4165676d |
fix(phase-26): isolation-residue corruption chain — config cleanup + revert() restores config + fail-loud validation + --raw sweep mode
Three-layer fix for the func_80178D40 ×133 sweep failures: - LAYER 1 (the residue): jtbl_family_bank.revert() restored carve pieces + src/ but NOT the isolation's CODE-subseg lines in the splat config. A failed bank attempt (BEBC's first try) left its isolation config in place; the successful retry re-isolated on top and a DUPLICATE `- [0x4b364, c, ov_SC01_000_jr_801734BC]` line rode into the commit (harmless to splat — zero-length — so R22 stayed green). revert() now also restores config/splat.<ov>.yaml. The committed duplicate is removed (ov_SC01_000 rebuilt BYTE-IDENTICAL 9052dc0e). - LAYER 2 (the detonation): jr_isolate_all walked the duplicated object TWICE -> two replacements -> a reversed duplicate block -> splat "segments out of order". It now VALIDATES the generated config (code subsegs strictly ascending, names unique) and refuses to write on violation, naming the likely cause — a corrupt input dies at the tool, not three tools later. - LAYER 3 (the sweep template): jtbl_family_bank gains --raw <crack.c> — template from the RAW crack via remap_hseq_body instead of the exemplar's banked source unit. REQUIRED when the exemplar banked at the `reconciled` stage: a reconciled body is TU-SPECIFIC (§41c — uniquified type names, TU-targeted casts), so extract_unit hands the sweep a polluted template and every sibling gate-fails (byte-proven: 178D40 banked reconciled -> sweep 0/4; 8015AE2C banked raw -> sweep 133/133). Same law as family_sweep --reconcile-raw. |
||
|
Drew T
|
9b93c254c2 |
feat(phase-26): func_8015AE2C (562 ins, x134) banked — Fable5 MATCH + 3 isolation bugs fixed
Exemplar banked byte-identical (d19c9580); R22 clean-fleet 136/136. Fable5 crack: MATCH 562/562, pin-free, jump table verified. THREE REAL BUGS the bank exposed in jr_isolate_all (each byte-proven; each would have silently corrupted every future heavy-core bank): 1. --only filtered `banked` as well as the cut set, so already-banked jr went untracked and their carves were never followed. --only selects what to CUT; it must not erase the record of what is already banked. 2. carve ownership was read from splat .s — but splat emits NO .s for a MATCHED function (its .c holds real C), so the lookup found nothing. Now resolved from the extracted IMAGE via family_remap.reloc_targets (byte-exact: func_801734BC -> 0x801d8c68 etc). 3. THE STRUCTURAL ONE: a region may host at most ONE .rodata carve, because an object's .rodata is a single CONTIGUOUS section. Cutting at func_8015AE2C (jtbl 0x801D8B54) left the banked func_801734BC (jtbl 0x801D8C68) inside the same region, so the object emitted a 0x34 .rodata spanning BOTH tables (image +33 B). Every already-banked jr in a cut object is now cut too -> exactly one carve per object. Cookbook 8b's "bank same-subseg families ASCENDING" note warned about this; it is now enforced by construction instead of left to discipline. Also required (per the crack's own analysis, all byte-verified): - engine_core.h: DEFINE_func_8015BEC4's zero-arg thunk returns func_8015AE2C(), so the extern must drop its (void) prototype and the def must stay K&R/unprototyped. Byte-neutral across all 136 (R22 green). - recovery chain: cast_call_sites (27 callees) + reconcile_decls (3 data syms). The raw body declares callees with types that conflict with their real engine_core.h defs; the original never redeclares them, it CASTS at the call site (cookbook 20). Layout now exact: .rodata 0x801d8b54/0x1c (7 entries, pad trimmed) + 0x801d8c68/0x14 + 0x801d92a0/0x20 — one table per object, each at its true address. |
||
|
Drew T
|
c66b530f71 |
fix(phase-26): bound the §8b carried decl layer + cross-address sibling naming
Two bugs the func_80182268 sibling sweep exposed (both would have silently capped every future jr family bank): - extract_unit walks BACKWARD from a definition absorbing preceding extern/comment lines as the fn's preamble. The §8b carried decl layer sits directly above the FIRST item of an isolated region, so the unit swallowed the whole layer -> the template dragged ~140 unrelated externs into each sibling (some naming types the sibling TU lacks) -> gate-fail. jr_isolate_all now emits an explicit end-marker and extract_unit stops at it (also guards the Phase-17 canonical-sig layer). - jtbl_family_bank passed the EXEMPLAR's name to the sibling's carve/isolate/stub lookup. Cross-address families (same engine fn at a different vram per overlay) therefore never resolved: ov_SC01_077 @0x80182268 -> ov_SC02_000/003 @0x8017FCB0. The sibling's name is now derived from to_addr. The first two banked jr families were same-address, so this had never surfaced. ov_SC01_077 d19c9580 byte-identical; R22 clean-fleet 136/136. |
||
|
Drew T
|
38ac5659aa |
feat(phase-26): §8b scoping wall BROKEN — decl-environment reconstruction + lazy per-core isolation
The full 54-jr isolate-all on ov_SC01_077 now builds d19c9580 BYTE-IDENTICAL
(R22 clean-fleet 136/136) — the configuration session 5 could not build. The
heavy-jr harvest (191 cores / 5.53M templatable ins) is unblocked.
- R14 CORRECTION: session-5's "gcc-2.7.2 block-scope-extern TU-persistence" root
cause was WRONG. There is no gcc quirk — DEFINE_func_* macros expand at FILE
scope, so their leading externs are genuine file-scope decls that merely live in
engine_core.h, invisible to any col-0 .c scan (1377 macros / 3929 lines / 1462 syms).
- REJECTED the approved "global symbol->type map + shadow set" design: the engine is
loosely typed (func_80173544 is DEFINED `s32 f(void*)` yet declared `extern void
f(void);` inside func_801734BC's body), so declaring every USED symbol hoists that
block-scope shadow to file scope and CREATES the conflict a shadow-set then dodges.
Instead reconstruct the original TU's file-scope decl environment and carry it
strictly FORWARD — conflict-free by construction (every carried decl already
coexisted with every definition in the one original TU; compatibility is
order-symmetric; shadows stay in bodies and travel with their item).
- The byte-gate found two MORE lost decl sources, not predicted: (a) a definition is
itself a declaration for everything below it in its TU (func_8012B2CC undeclared);
(b) file-local typedefs used by a carried prototype (parse error, Vec3s). K&R defs
must render `extern T f();` (unprototyped), never f(void).
- LAZY per-core isolation wired into jtbl_family_bank (Drew's call — upfront-x134 =
~7,200 region files): jtbl_carve NON-CONTIGUOUS fail-loud -> jr_isolate_all --only
<core> -> re-extract -> re-carve. Proven on func_80178D40 (890x134, heaviest core):
carve blocked -> isolated (byte-neutral d19c9580) -> carve in its own subseg.
- TWO LATENT BUGS fixed (both would have corrupted the heavy sweeps):
* jtbl_carve.func_subseg derived the owning subseg from the ASM TREE, which `make
extract` never prunes -> after an isolation it returned the STALE owner and
silently re-created the very collision the isolation removed. Now config-derived.
* jtbl_family_bank/jtbl_carve revert() DELETED the shared overlays.mk carve var
unconditionally -> would destroy a COMMITTED carve (all 134 overlays have one) on
any failed sibling. Now restored to its committed value; only region files created
by this attempt are removed; dirty-tree preflight refuses to start a sweep.
- docs: cookbook §8b RESOLVED + new §8c "splitting a TU means rebuilding its
DECLARATION ENVIRONMENT, not moving text"; decision-log 2026-07-13 (R30/R31).
- parser selftest 404/404; R22 clean-fleet 136/136; 0 NON_MATCHING (G4).
|
||
|
Drew T
|
234788dfc4 |
feat(phase-26): §8b overlay-src parser (404/404) + jr isolation tool + the gcc-scoping wall finding
- tools/overlay_src_split.py: overlay-.c-aware partition (header = includes + Phase-17 canonical-sig layer; per-address items = preamble + body; robust def/decl/K&R/DEFINE_func/ SETTER/RETCONST classification). Fleet-validated 404/404 overlay .c, 341,902 items — round-trip exact / 0 unresolved / 0 non-monotonic. The Stage-2 isolation unblock. - tools/jr_isolate_all.py: multi-cut jr resegment (config split at jr boundaries, source repartition + INCLUDE_ASM path repoint, banked-jr carve repoint, -O0 skip, ambient decl carry). SINGLE-cut isolation byte-identical (func_8013FFD8 -> d19c9580, R22). - FINDING (decision-log 2026-07-13): full 54-jr isolation of the dense _after object hits gcc-2.7.2 block-scope-extern TU-persistence (func_801734BC/D_80126B3E declared only in engine_core.h DEFINE_func macros); mechanical TU-split breaks it. Fix = declaration- completion from a global symbol->type map (Drew-approved next step; lazy per-core). - baseline intact (ov_SC01_077 rebuilds d19c9580); no config/src/binary change committed. CURRENT_PHASE session-5 checkpoint + decision-log R31. db.*.gbf = R23 noise, not staged. |