fix(psyq_integrate): main was RED on every incremental relink — make the externals file monotonic

THE TRUE IDENTITY OF THE LONG-STANDING 'main link defect' (2026-08-15). The extra C
function never broke the link; the RELINK it forced did.

integrate() derives each *_externals.ld from trial_undefined() against the CURRENT
ld_path, so its answer depends on how much of the linker script has ALREADY been
rewritten. On a virgin splat .ld the apicard region is still the stub object
(defining only firstfile2), so at the libmcrd stage 'firstfile' is undefined and
gets an entry. On an already-rewritten .ld, A66.o is present and defines
'firstfile' at 0x80062248, the trial no longer reports it undefined, and the entry
'firstfile = 0x80061FA8;' is DROPPED -- after which LIBMCRD's jal binds to A66.o
and main comes out 2 of 413,696 bytes different from retail (file 0x51674,
VA 0x80060E74, retail jal 0x80061FA8 vs built jal 0x80062248).

That is why main was green ONLY on the first build after a fresh extract, and it
is why NO main draft could ever bank through an incremental gate: the baseline was
already red before any draft was spliced.

integrate()'s own comment already CLAIMED this operation was idempotent ('a re-run
on an already-rewritten .ld only redoes syms'). This makes it true: the externals
map is merged with the file's prior contents, newly-derived values winning on a
name collision, names the new derivation no longer sees kept at their previous
address. The file becomes a function of the tree, not of how many times this ran.
It reports what it kept rather than doing it silently.

VERIFIED, three builds:
  fresh extract + build ...... GREEN (unchanged)
  INCREMENTAL relink ......... GREEN (was RED -- the failing case)
  third relink ............... GREEN (monotonic across repeats)
and the merge is observed firing: 'kept 6/15/2 extern(s) this re-run no longer saw
as undefined' across the integrate stages.

Root-caused by a Fable agent, verified here against the bytes.
This commit is contained in:
Drew T
2026-08-31 17:03:50 -06:00
parent 14a72240d2
commit 3ebbec9426
+33 -1
View File
@@ -160,9 +160,41 @@ def integrate(elf_dir, ld_path, objdir, syms_path, stubs, lo=None, hi=None,
externals[s] = recovered[s]
else:
missing.append(s)
# MONOTONIC MERGE — the externals file may only GROW for a given tree state.
#
# `trial_undefined` above is evaluated against the CURRENT ld_path, so its answer depends on how
# much of the linker script has ALREADY been rewritten. On a virgin splat .ld the apicard region
# is still the stub object (which defines only `firstfile2`), so at the libmcrd stage `firstfile`
# is undefined and gets an entry. On an ALREADY-REWRITTEN .ld, A66.o is present and defines
# `firstfile` at 0x80062248, the trial no longer reports it undefined, and the entry
# `firstfile = 0x80061FA8;` was DROPPED — after which LIBMCRD's `jal` binds to A66.o and main
# comes out 2 bytes different from retail (file 0x51674, VA 0x80060E74).
#
# That is why main was green only on the first build after a fresh extract and red on every
# incremental relink, and it is the true identity of the long-standing "main link defect"
# (2026-08-15): the extra C function did not break the link, the RELINK it forced did.
# integrate()'s own comment at the top already claimed this operation was idempotent
# ("a re-run on an already-rewritten .ld only redoes syms"); this makes that true.
#
# Newly-derived values WIN on a name collision (an address that legitimately moved must move);
# names the new derivation no longer sees are KEPT at their previous address. So the file is a
# function of the tree, not of the number of times this ran. Diagnosed P31 S68 (fable).
prior = {}
if os.path.exists(syms_path):
for ln in open(syms_path):
m = re.match(r"(\w+)\s*=\s*0x([0-9A-Fa-f]+)", ln)
if m:
prior[m.group(1)] = int(m.group(2), 16)
readded = sorted(set(prior) - set(externals))
merged = dict(prior)
merged.update(externals) # new derivation wins where both have the name
with open(syms_path, "w") as f:
for s, a in sorted(externals.items(), key=lambda kv: kv[1]):
for s, a in sorted(merged.items(), key=lambda kv: kv[1]):
f.write(f"{s} = 0x{a:08X};\n")
if readded:
print(f" (kept {len(readded)} extern(s) this re-run no longer saw as undefined: "
f"{readded[:8]}{'...' if len(readded) > 8 else ''})")
externals = merged
print(f"integrate {os.path.basename(elf_dir)}: {len(order)} objects in {len(blocks)} block(s) "
f"-> stubs {stubs}; {len(nol)} NOLOAD sections; {len(externals)} externals -> {syms_path}")
if missing: