The 'unsteerable' regalloc-order class is NOT unsteerable. Drew's directive (hand-match
everything) exposed the lever I'd skipped: explicit register pinning.
- func_8012B8E4 (the flagship circular reach-134 'unsteerable' fn): 21 -> MATCH via
* register pins: register s32 d __asm__("$16"); register s32 s1ang __asm__("$17")
-> FORCES the $s0/$s1 global allocation gcc otherwise density-orders the other way
* branch-polarity invert (§3-T4): 24->21
* explicit clamp temps (t=u6+0x1000; iVar4=u5-t): fixes the reassociation, 7->3
* scheduling barrier __asm__ __volatile__("" : : "r"(u5)): anchors the last sll, 3->MATCH
- byte-gated (harvest_verify, G3/P9) + propagated x134 (per-overlay gate); 134 byte-identical
- recipe = §10-family hand technique (pins + barriers); labor-intensive (per-fn register map
from the asm) but it WORKS -> the circular tail IS hand-matchable, not asm-only
- supersedes the T1-T5 'unsteerable' verdict; cookbook §17 + PhaseEnd to be revised accordingly
- canonical-sig layer (session 4): tools/census_conflict_callees.py + derive_canonical_sigs.py
-> a 20-extern byte-neutral block atop ov_SC01_077.c (LOCAL, not engine_core.h); census
conflict callees 20->0, blocked targets 24->0; gate pipeline now draft -> sig_unify (MANDATORY)
-> harvest_verify --chunk 1; fleet 136/136 byte-identical (R22), 55.51% (no regression)
- FINDING (R14/P9): the conflict wall is 7%-reach not ~2x; the 4 reach-134 circular targets are
ALL gcc-quirk/regalloc/layout-bound (0 banked); the high-reach core IS the quirk tail; struct
types are byte-neutral for matching (the wall is gcc codegen, not knowledge)
- leverage analysis: fleet % is function-count-weighted (size adds no %); "unblock many" = the
layer (declaration, not matching); reach is the lever (already reach-sorted); 247 tractable
reach-134 stubs ~ +3-4% projected
- GO/NO-GO: NO-GO on brute waves at the current ceiling; GO on a compiler-quirk research phase
(read gcc-2.7.2 source + Xenogears + the §10/regalloc classes, R17) -> then resume the wave
- docs: cookbook §16 corrected + hand-matching-process.md §8 (the layer + the finding + handoff)
- worklog archived -> phase-ends/logs/Phase17.md (R19); bumps 1.15.0 -> 1.16.0
- banked 42 gate-proven SIG_FIXABLE_KR matches in ov_SC01_077 (of 48 predicted; 6 honestly
reverted by the whole-binary byte-gate, P9) + propagated 13 h_exact-shared x134 via
dedup_propagate (one body -> 134 overlays, each byte-gated, fail-closed)
- fleet byte-identical-from-source 54.48% -> 55.00% (+0.52%); REAL +1,778; dedup groups
1395 -> 1408; make check-all 136/136 byte-identical (R22); dedup-check 1408 validated/0 failed
- wall probe (R14): raw drafts 0/99 + sig_unify fails -> the K&R/sig_unify avenue is exhausted;
the residual conflict-class is ARITY_WALL (the §14e param-count dead-end) or structural (T4/T5)
- FIX a T1 classifier bug surfaced by verifying the wall vs the bytes: tu_compile_error captured
the baseline's pre-existing byte-NEUTRAL warnings ("makes integer from pointer", cc1 rc=0) and
mislabeled 99 fns LOOSE_TYPING_WALL; now keeps only non-warning errors -> there is NO loose-typing
wall, the real residual wall is ARITY_WALL=74. docs/wall-taxonomy.md amended with the correction.
- 134 overlay .c files carry the propagated DEFINE_func instantiations; a one-time full-fleet
make extract was required for the per-overlay propagation gate + check-all
- docs/struct-core-pivot.md: findings + decision + new research directions. Root cause = the
original engine is LOOSELY TYPED (K&R; same fn called with int/ptr, arg/no-arg across sites),
so no single canonical signature exists -> m2c guesses inconsistently, permuter can't fix
semantics, byte-gate (correctly) rejects. Yields ~3%, not the crack. New plan: emulator-recover
the actor struct/types -> Ghidra global type propagation -> Ghidra-C -> permuter+gate.
- harness bug-fixes (REAL, kept): p16_permute output-0-only match (killed the false '42%'),
base.c keeps callee externs, winner_to_draft line-strip; sig_unify canonicalizes m2c's
no-extern prototypes; gen_engine_decls.py (documents why a global canonical header breaks
loose-typed matches).
- a few byte-gated leaf matches banked in ov_SC01_077.c.
- tools/sig_unify.py (NEW): unifies a draft's FULL signature set to the banked-canonical
decls — both callee externs AND the draft's OWN definition signature (return + param
types, body param-names preserved). Recovery for the standalone-MATCH residual: a probe
found 30/30 sampled failures were type conflicts, 0 false-positives. Gated 191 -> 32 verified.
- tools/dedup_propagate.py find_site FIX: the inline-def detector required the opening brace
ON THE SAME LINE as the signature, silently dropping every next-line-brace def from
propagation (sig_unify/permuter outputs + ~stragglers all session). Now accepts brace on
the same OR next line -> unlocked a 61-function propagation backlog.
- propagated 61 fns fleet-wide (incl. 32 sig-unify + the permuter win func_801508F8 +
earlier-dropped next-line-brace matches), all 134 byte-identical; dedup-check 1394/0
- R22 CLEAN fleet rebuild: 136/136 byte-identical. REAL 177501 -> 185646. Zero agent tokens.
- fix tools/build_engine_types.py to be ADDITIVE: merge new source types with the
types already in engine_types.h (a re-run after a prior --strip was destructive —
it dropped the ~34 already-migrated types and broke the build; caught + reverted)
- engine_types.h 34 -> 42 named types (the 8 from wave-1's struct-using matches);
inline defs stripped from ov_SC01_077.c (byte-neutral)
- dedup_propagate --auto-from: 8 of 15 struct-blocked matches now self-contained ->
propagated across all 134 overlays (byte-gated); 7 remain (anonymous/typedef'd types)
- R22 CLEAN fleet rebuild: 136/136 byte-identical; dedup-check 1333/0
- zero agent tokens (deterministic)
- canon_draft_decls recovery on the combined 505 quarantined drafts (285 small + 220 medium),
now canonicalizing against the much-larger banked symbol set: 47 verified (no API), 42
propagated fleet-wide (struct/local-type skipped), 134 overlays byte-identical; dedup
1141 -> 1183 groups, dedup_integrate --check 1183 validated / 0 failed.
- fleet REAL substantive 151,908 -> 157,541; byte-identical 44.64% -> 46.27%.
- R22 clean rebuild: 136/136 byte-identical, 0 extract failures.
- scoped the struct-using-shared-fn lever (read-only): 34 named struct/union types in 077,
ZERO same-name-different-layout collisions -> a shared-types-header is a clean path to
propagate the ~100 struct-using fns (next).
- gen_harvest_targets.py: now also resolves canonical DATA-symbol declarations per target
(scan the .s for D_XXXX refs -> `extern <type> D_XXXX;` from banked code) in addition to
callee function sigs — the §14c(c) fix at the source, so agents declare both functions and
data correctly and avoid the dominant remaining conflict class.
- v2 harvest (callee+data-sig-aware) on the 591 small still-stub call-heavy fns + a gap-fill
agent for 6 targets dropped by a connection-closed-mid-response failure (the retry-wave
misses truncated-non-null results). Gate: 306 verified (ov_SC01_077 1554 -> 1248 stubs).
- dedup_propagate.py: skip any struct/union-touching body — agents named structs with colliding
generic names (`struct S`/`struct vec`) inline, so two macros' types redefine/conflict when
instantiated in one overlay (ov_SC01_000 abort). 34 skip; struct-using shared fns stay banked
in 077 (shared-types-header follow-up).
- 281 functions propagated fleet-wide (134 overlays each, every overlay byte-gated); dedup
795 -> 1076 groups, dedup_integrate --check 1076 validated / 0 failed.
- fleet REAL substantive 105,764 -> 143,322; byte-identical 31.22% -> 42.14%.
- R22 clean rebuild: 136/136 byte-identical, 0 extract failures.
- cookbook §12: gap-fill is now a MANDATORY post-step for every multi-agent run (reconcile
produced-vs-expected: missing + truncated drafts) before gating.
- tools/canon_draft_decls.py: rewrites a quarantined harvest draft's extern/data
declarations to the banked-consistent set (functions from engine_core.h DEFINE
bodies / extern decls / inline defs; data from existing `extern <type> D_XXXX;`),
so a draft whose BODY is byte-correct (passes match_one) but whose DECLARATION
collided with already-banked code can re-gate cleanly. The whole-binary
harvest_verify byte-gate stays the sole arbiter — a semantics-changing
canonicalization just fails and reverts.
- recovered 19 of the 90 match_one-MATCH-but-conflicting quarantined failures
(no API — immune to the rate limiting), propagated fleet-wide, all 134 overlays
byte-identical; dedup 776 -> 795 groups, 0 failed.
- fleet byte-identical 30.48% -> 31.22%; ov_SC01_077 1574 -> 1535 stubs.
- R22 clean rebuild: 136/136 byte-identical, 0 extract failures.
- the remaining ~70 match_one-MATCH residual needs definition-widening (§14c(d),
narrow-return) or true data-type resolution — queued for the next pass, with the
un-started medium band (430) and the 54 genuine instruction-misses (permuter/§3a).
- onboarded all 118 not-yet-onboarded overlays via tools/new_overlay.sh (0 failed)
-> fleet = 134 location overlays + main + resident = 136 binaries
- R22 CLEAN rebuild (make clean -> extract all 136 -> make check-all):
136 passed, 0 failed of 136 -- every overlay byte-identical at 100% INCLUDE_ASM
from a fully clean tree, 0 extract failures
- per-overlay config (splat.ov_*.yaml / check.ov_*.sha / symbols.ov_*.txt) +
config/overlays.mk var blocks + the 4 report/diff BINARIES dicts auto-registered
(sentinel-anchored, idempotent); src/ov_*/ stub records committed (harvest_verify
reverts via git checkout, so the records must be tracked before any harvest)
- ghidra/ churn NOT staged (R23); build/asm/assets/.run gitignored
- report-digest regen (fleet-wide duplicates.cross.md + progress.fleet.md) is T4
- tools/dedup_propagate.py: lift a matched body -> DEFINE_func_<ADDR>() macro in
src/shared/engine_core.h -> instantiate in place at every onboarded overlay sharing
that h_exact -> snapshot + per-overlay byte-gate (fail-closed revert) -> register in
config/dedup.us.yaml. Keyed by addr-int (sig lowercase vs splat func_%08X).
- proof: 4 medium funcs propagated across all 16 onboarded overlays; clean check-all
18/18 byte-identical; dedup_integrate 8 validated/0 failed; idempotent; negative test
(corrupt shared body) -> make check FAILS then restores
- Makefile: header-dependency tracking (cpp -MMD -MP + -include ) so editing a
shared header recompiles dependents — closes a stale-build/false-pass gap the negative
test exposed; output-neutral (R22 clean 18/18)
- structural self-check: a leftover INCLUDE_ASM stub is byte-identical, so the byte-gate
can't catch under-application -> assert DEFINE present + stub gone
- FINDING: 577 of ov_SC01_077's 785 matches are h_exact across all 134 overlays
(2.19 MB collapsible, already matched) -> T5 = bulk-propagate these + harvest the rest
- cookbook §14 (the propagation runbook + gotchas), SETUP tool inventory
- ghidra/ churn NOT staged (R23); build/asm/.run gitignored
- onboarded ov_{SC01_000,SC01_001,SC02_000,SC02_003,SC03_002,SC03_006,
SC04_000,SC04_018,SC04_019,SC05_000,SC06_000,SC07_000} via tools/new_overlay.sh
— spans all 7 SC areas + 2 full duplicate pairs (SC02_000≡003, SC04_018≡019)
- each byte-identical at 100% INCLUDE_ASM (non-4-aligned auto-handled); no
position-lock anomalies — the cross-shape risk is retired before the full onboard
- make check-all = 18/18 passed (main + resident + 4 prior + 12 new)
- config/overlays.mk + 4 report dicts auto-registered (sentinel, idempotent)
- ghidra/ DB churn NOT staged (R23); asm/build/assets/.run gitignored
Controlled: 80 fixed SC01/077 fns (nins 8-15), identical blind-draft Workflow,
only session effort varied. Arm A (Ultracode/xHigh) 13/80, Arm B (Max) 14/80;
A∩B=12. 13-vs-14 = noise -> per-agent depth does NOT change blind-draft harvest
yield; the 'pass 1 Max did better' observation was difficulty-confounded.
DECISION: prefer Ultracode for breadth harvests (same yield + orchestration),
Max for deep synthesis. Banked Arm B's 14 -> ov_SC01_077 REAL 704 (byte-identical).
(scriptPath re-invoke didn't forward args -> Arm B overwrote Arm A's drafts dir;
Arm A was already gated/recorded, so the comparison stands; 1 A-only fn unbanked.)
finding -> docs/effort-map.md (Evidence section)
- src/shared/ov_setters.h: generic SETTER/RETCONST macro bodies (cookbook §11
source-level share), instantiated IN PLACE (address order preserved) in both
ov_SC01_005.c and ov_SC01_006.c via ../shared/ relative include
- config/dedup.us.yaml: D1/D2/D3 cross-overlay groups (members in BOTH 005 & 006)
- both make check BINARY=ov_SC01_005 AND =ov_SC01_006 -> 56760dbe BYTE-IDENTICAL
from the one shared source; dedup_integrate --check: 4 validated, 0 failed;
REAL=3 credited to both members
match_one iterate pass throttled by a transient server rate-limit (20/36 batches
lost); banked 52 from the 16 completed. Build d19c9580. Resume with paced blind
draft + retry-wave (§12 resilience).
splat carved 2505 functions cleanly (func_* names; the top cross-report group
func_80144B9C carved correctly at 770 ins). The C record is durable (splat
preserves it across re-extract); matches land here in T1d/T1e.
The pass-4 functions that match_one matched standalone but the single-TU build rejected
on conflicting shared-symbol extern types are recovered by UNIFYING those types in
resident.c (widening definitions' return types where byte-identical — u16 f(){return
u16g} ≡ s32 f(){...} both lhu;jr; choosing asm-correct widths for globals; per-site
*(u16*)&g tricks where a neighbor pins the type). ~20 symbols reconciled
(func_800291B4/CFD68/D1714/D1078/801465C0/... + D_800AE6A8/80078E50/...).
20 of 21 recovered; func_800D00E4 stays an honest INCLUDE_ASM stub (its switch needs the
external jtbl_80113ED8 relocation, not a C-source fix — analysis preserved under #if 0).
REAL 102 -> 123 (85.62% byte-identical); resident 8e17e02f, main 143dbb89 (R22 clean).
22 stubs remain: ~20 documented compiler-internal residuals (permuter/§3a) + D00E4 + dedup.
- tools/match_one.py (new): compile ONE function's C standalone (pinned triple), mask
relocations (jal/HI16/LO16 like psyq_identify), compare to the target bytes in its .s.
Isolated temp dir -> parallelizable; a real per-function asm-differ loop for agents to
iterate against without a shared build. Validated: a matched fn -> MATCH, residuals ->
informative diffs.
- harvest_verify.py + match_one.py: strip inline scalar-typedef redefinitions (some drafts
inlined `typedef unsigned char u8;` -> C89 redefinition error vs common.h = a COMPILE
fail, not a byte mismatch). Re-gate recovered func_800CFBCC + func_800D1938.
- REAL 89 (62.76% byte-identical); resident 8e17e02f, main 143dbb89 (R22 clean).
- 54 stubs remain (scheduling/regalloc residuals + big GPU/dispatch fns) -> match_one-driven
iteration pass next.
Ultracode harvest: a 16-agent workflow (resident-harvest-draft) drafted matching C for
the 135 unmatched resident functions (each agent: asm + cookbook + the 8 worked examples,
writing .run/drafts/<fn>.c, no builds/no Ghidra). Then tools/harvest_verify.py (new) byte-
gated every draft against 8e17e02f one at a time (chunk+bisect), keeping only the byte-
perfect ones and reverting the rest -- the build is the sole arbiter (G3/P9).
- 62 / 127 drafts verified byte-identical in one pass; 65 near/hard misses reverted to stubs.
- REAL 8 -> 70 (48.28%); byte-identical 49.66%. resident 8e17e02f, main 143dbb89 (R22 clean).
- cleaned 35 redundant '#include "common.h"' lines the drafts inlined (still byte-identical).
- remaining: 65 failed drafts (close-misses + the hard dispatch/GPU/big-fn tail) + 8 undrafted
(2 agent batches hit transient API 500s) -> second-pass workflow + manual hard tail.
- opt level: resident is -O2 (0x21f0a003 frame-ptr sig count = 0, vs the EXE's 54;
the EXE's 800 game module is also -O2). No per-file override needed.
- triple (Makefile default, no override): gcc-2.7.2-psx -O2 -G0 -mips1 -mcpu=3000
-mgas -msoft-float -fgnu-linker + maspsx --aspsx-version=2.56 --expand-div.
- 2 byte-exact resident matches confirm it across idiom classes:
* func_800CF854 — lbu D_800AE6BF; sltu $zero (unsigned-!=0 accessor)
* func_800CEE84 — 8-call init: branch-store (D_80127084=0; if(D_800B9A00) =1),
delay-slot-filled func_80010AE0(0x3E0), early-scheduled load
- R22 clean rebuild: resident 8e17e02f, main 143dbb89 — both byte-identical.
make report BINARY=resident: REAL 0 -> 2.
- R24: the "resident is 4.7" DetectPsyQ hit was a coincidental DsMix signature; the
resident's actual game-code triple == the EXE's (byte-confirmed). No candidate
ladder; the STOP-condition did not fire. Harvest (T3) de-risked.
- src/shared/clearTbl40.h: CLEAR_TBL40 macro = the matched byte-clear loop body, authored ONCE;
instantiated at both func_80037004 and func_80037334 in src/800.c (one source -> two vrams)
- matched first try; clean rebuild main -> 143dbb89 BYTE-IDENTICAL WITH the shared C AND WITHOUT
it (INCLUDE_ASM stub fallback) -> dual invariant proven (R22 clean rebuilds, both states)
- config/dedup.us.yaml: I0_clearTbl40 group registered (h_exact a0744d60…); dedup_integrate
--check validates it; negative tests (corrupt hash / wrong vram) fail-closed (exit 1)
- tools/progress.py: count dedup-shared members as REAL via the registry (the macro form isn't a
parseable function def); REAL 52 -> 54, byte-identical 50.24% -> 50.33%; honest measurement (P9)
- tools/dedup_integrate.py: display vram in hex in diagnostics
- the machinery half of the Phase-11 milestone is proven on the byte-verified EXE
- tools/ghidra_import_raw.sh (NEW): raw-blob importer (BinaryLoader + --loader-baseAddr +
PSX:LE:32:default) — the Gen2 counterpart to ghidra_import.sh (PS-X-EXE only); reusable for
Phase-13 location overlays. Imports the resident blob as program 'resident' @0x800CEDF8.
- tools/ghidra_scripts/DefineFunctions.java (NEW): seed splat's validated entry points
(.run/<prog>_funcs.txt) — raw-binary auto-analysis finds only the reachable subset (23/143);
this defines all 143 (created=120/existed=23/failed=0). R9-verified 143 funcs persisted.
- FINDING: DetectPsyQ reports the resident is PsyQ 4.7.0 (the EXE is 4.0.0); the lone in-range
PsyQ-signature hit is DsMix (libsnd -> resident holds the sound driver). 4.7 .LIBs DEFERRED to
Phase 11 start (Drew); carried to PhaseEnd Notes. Phase 10 needs nothing from 4.7.
- config/symbols.resident.txt: seed DsMix (R13 candidate, 4.0-sig vs 4.7 blob — confirm in Phase 11);
stacked under symbols.us.txt, applied on re-extract -> resident still 8e17e02f BYTE-IDENTICAL (R22),
main still 143dbb89 (no regression)
- Makefile: per-binary GHIDRA_PROG -> 'make sig-refresh BINARY=resident' (.run/sig.resident.jsonl)
- dup_report.resident now real (6 byte-identical intra-resident groups — Phase-11 dedup fodder)
- ghidra DB committed (R23, MCP stopped): new resident program 00000003.* (+ main db.15->16 no-op)
- mapped the blob vs bytes: word0 data header (0x36); clean code 0x4..0x4610 (145 funcs,
zero embedded jump-tables/trap-ops); data tail 0x4610..EOF (pointer tables + 78.5KB zeros)
- split: [0x0, rodata, hdr] + [0x4, c, resident] + [0x4610, data, tail]
- fix 1: the leading data word sits BEFORE code, which fights section_order; emit it as
rodata (no-dot) so section_order [.rodata,.text,.data,.bss] places it first @0x800CEDF8 —
a 1-word analogue of main's rodata-island, no ld_interleave needed
- fix 2: build_path=build (not build/resident) so splat's .ld object paths match the
Makefile's build/asm/** + build/src/** pattern rules (only elf/ld/output in build/resident/)
- MILESTONE: make build BINARY=resident -> 8e17e02ff8954d07c979449198f7e1645046b353
BYTE-IDENTICAL (365,404 B, end vram 0x80128154); -G0 confirmed (0 gp refs);
R22 clean-rebuild green; main still 143dbb89… (no regression)
- src/resident/resident.c (143 INCLUDE_ASM stubs) committed; asm/resident/** regenerated
- libspu and libsnd interleave in 0x3A444..0x4239C (the 800-subseg tail), so they link
as ONE combined region rather than two tangled passes
- NEW tools/make_snd_used.py: build the curated combined dir — merge both libs by vram,
pick the byte-matching object per aliased address (link_object), exclude 4 addresses
that don't reconcile in-region (kept as byte-identical stubs):
0x3C438 S_R/S_W, 0x3D424 S_GRMDT/FB/T — scattered-.bss commons, cross-object (§9.1)
0x3D94C S_IH/UT_RON — false placement (inside libsnd SSSTART.o)
0x3FA64 VM_F (237 ins) — scattered-.bss commons (the one real loss)
- subsegs via gen_lib_subsegs.py (9 snd blocks + 8 sgap game/excluded-stub gaps);
integrate window 0x3A444..0x4239C; region byte-verified 60/60
- src/800.c trimmed at [0x3A444,0x4239C): kept 732 items < 0x3A444 (ALL matched C +
game + the deferred SSGM stub preserved), 0 real-C moved
- dual byte-gate PASS: 143dbb89 with and without the sound objects
- LINKED 710 -> 935 (+225); REAL still 43; byte-identical 48.66% (nearly half the EXE)
- deferred: SSGM.o @0x1BD80 (isolated, matched-C region, 8 ins)
Drew T