- 🚨 top banner: T0-T5 'unsteerable' framing SUPERSEDED; resume 3->1->2 at Step 3b
with /effort ultracode ON; read cookbook §17 (the corrected TOOLKIT) first
- ▶ STEP-3b RUNBOOK: the §17 toolkit summary (reconstruct/array-decay/PINS/barrier +
the MANDATORY canonical-retype gate gotcha + stub-only narrow-param), the exact gate
pipeline + commands (match_one/harvest_verify/dedup_propagate, ov_SC01_077 good-sha
d19c9580), the durable worked examples (engine_core.h DEFINE_func_8012B8E4/801399A8),
the staged assets (harvest_wave_s4.js, probe_targets_s4.json, .run/ghidra_c/), fleet
baseline 55.58%/136-of-136, and the Step 1->2 close path
- everything a fresh full-context session needs to author the upgraded wave + run it
- §17 rewritten: the call-crossing register-ORDER class is MATCHABLE with register
__asm__ pins + a scheduling barrier (byte-proven func_8012B8E4), not unsteerable.
The wall was a missing lever (the pins), not an impossibility. Recipe documented as
a TOOLKIT: triage with match_one -> reconstruct / array-decay / PINS / barrier.
- §16 cross-refs + hand-matching §8e corrected to match. The ONLY genuine stub-and-skip
left is the narrow-param loose-typing conflict (func_80146A6C).
- self-correction (P9/R14): I concluded 'unsteerable' before trying the pins; Drew's
hand-match-everything directive exposed the gap. Recorded honestly.
The 'unsteerable' regalloc-order class is NOT unsteerable. Drew's directive (hand-match
everything) exposed the lever I'd skipped: explicit register pinning.
- func_8012B8E4 (the flagship circular reach-134 'unsteerable' fn): 21 -> MATCH via
* register pins: register s32 d __asm__("$16"); register s32 s1ang __asm__("$17")
-> FORCES the $s0/$s1 global allocation gcc otherwise density-orders the other way
* branch-polarity invert (§3-T4): 24->21
* explicit clamp temps (t=u6+0x1000; iVar4=u5-t): fixes the reassociation, 7->3
* scheduling barrier __asm__ __volatile__("" : : "r"(u5)): anchors the last sll, 3->MATCH
- byte-gated (harvest_verify, G3/P9) + propagated x134 (per-overlay gate); 134 byte-identical
- recipe = §10-family hand technique (pins + barriers); labor-intensive (per-fn register map
from the asm) but it WORKS -> the circular tail IS hand-matchable, not asm-only
- supersedes the T1-T5 'unsteerable' verdict; cookbook §17 + PhaseEnd to be revised accordingly
- the build-divergence escape is closed without the heavy Wine/CC1PSX run:
* gcc-2.7.2-psx byte-matches ~700 fns (incl. call-crossing callee-saved values)
-> its global allocation IS faithful to the original compiler -> real CC1PSX
would emit the SAME unsteerable swap
* cdk cc1 tested directly: diverges but WORSE (32 vs 21) -> cc1-switch doesn't crack it
* Xenogears (real-era toolchain) ships the class as asm
- Wine is a heavy install here (106 pkgs + i386 + wineprefix); low marginal value ->
deferred per the plan escape clause; binaries staged at tools/psyq/psyq4.0/ if wanted
- R22: clean-rebuild of ov_SC01_077 with the match = d19c9580 BYTE-IDENTICAL
- all Phase-18 tasks complete; ready for gate-2 milestone confirmation
- T6 2nd attempt (func_80146A6C): 18=18 ins, 8 mismatched, blocked by the Phase-16
LOOSE-TYPING wall (target loads a4 as s16/lhu; canonical engine_core.h sig declares
a4 s32/lw -> no consistent type). So STRUCTURAL_MISS is mixed: some close, some walled.
- T4: per-class idiom verdicts set (for-loop+statement-order+sig_unify byte-gated to a
full match; branch-polarity + array-decay-remat = real codegen fixes via match_one;
regalloc-order byte-proven unsteerable)
- CURRENT_PHASE updated (crash-recovery log)
- T2 (Xenogears mine, 2 bg agents): independent decomp on IDENTICAL gcc-2.7.2-psx -O2
has NO C lever for the call-crossing $s0/$s1 ORDER class; ships it as INCLUDE_ASM
(1174 nonmatch). Their idioms: variable-split, width/cast, per-file compiler-switch.
- flag/cc1 sweep (func_8012B8E4): -funsigned-char/-fpcc-struct-return/-fpeephole/
-fcaller-saves all 21 (no change); cdk worse (32); -fno-schedule-insns worse (71)
- T3 verdict: regalloc-ORDER class = UNSTEERABLE from C (global.c density ordering;
exhaustive empirics + independent corroboration)
- NEW IDIOM found on func_8012B4B8 (remat class): array-decay forces rematerialization
(s32 mtx[8] passed as decay -> remat not hoist; fixes 88->52 incl. the hard regalloc).
&struct/.w/*(M8*) all hoist. Bankable; this fn has a struct-copy conflict so partial.
- synthesis: .run/p18/T2_T3_synthesis.md; reconciles §10-vs-§16 for T5
- strategic redirect: high-reach circular tail largely unmatchable from C -> match-%
lever is the tractable-247 wave (Phase 19) + accept the tail as asm (Xenogears policy)
- replayed §10 levers on func_8012B8E4 under match_one (the floor-free oracle):
- §3-T4 branch-polarity invert: 24 -> 21 mismatched (a real §10-family win the
permuter's floor-polluted score could not measure in Phase 17)
- statement-order: no effect; coupling s1ang+u6: regressed to 71; -O3: identical alloc
- ROOT pinned via the cloned gcc-2.7.2 source: the residual $s0<->$s1 swap is two
ratan2-call-crossing pseudos allocated by global.c:allocno_compare (density =
log2(n_refs)*n_refs/live_length), NOT local-alloc; short-lived s1ang wins $s0
- GATE = NO branch (§10 helps but doesn't close) -> T3 sharply scoped + mechanism-pinned;
func_8012B4B8 (hoist-remat+struct) baselined (88 mismatched), deferred to T3/T4
- CURRENT_PHASE updated; findings in .run/p18/ (gitignored scratch)
- canonical-sig layer (session 4): tools/census_conflict_callees.py + derive_canonical_sigs.py
-> a 20-extern byte-neutral block atop ov_SC01_077.c (LOCAL, not engine_core.h); census
conflict callees 20->0, blocked targets 24->0; gate pipeline now draft -> sig_unify (MANDATORY)
-> harvest_verify --chunk 1; fleet 136/136 byte-identical (R22), 55.51% (no regression)
- FINDING (R14/P9): the conflict wall is 7%-reach not ~2x; the 4 reach-134 circular targets are
ALL gcc-quirk/regalloc/layout-bound (0 banked); the high-reach core IS the quirk tail; struct
types are byte-neutral for matching (the wall is gcc codegen, not knowledge)
- leverage analysis: fleet % is function-count-weighted (size adds no %); "unblock many" = the
layer (declaration, not matching); reach is the lever (already reach-sorted); 247 tractable
reach-134 stubs ~ +3-4% projected
- GO/NO-GO: NO-GO on brute waves at the current ceiling; GO on a compiler-quirk research phase
(read gcc-2.7.2 source + Xenogears + the §10/regalloc classes, R17) -> then resume the wave
- docs: cookbook §16 corrected + hand-matching-process.md §8 (the layer + the finding + handoff)
- worklog archived -> phase-ends/logs/Phase17.md (R19); bumps 1.15.0 -> 1.16.0
- CURRENT_PHASE.md: new session-4 START HERE (build the canonical-sig layer, then
scale the wave); old demo-protocol START HERE marked superseded
- hand-matching-process.md S7: the Ultracode harvest wave (pipeline, calibration
result 60% match_one / 33% whole-binary / +0.47%) + THE CANONICAL-SIG WALL (the
60->33 gap is 100% sig conflicts; surgical per-callee canonicalization = ~2x lever)
- matching-cookbook.md S16: guided-hand-matching breakthrough + new idioms (mask-local,
shared-ret0 goto, v0/v1 coalescing) + the wave/canonical-sig finding (R16 flywheel)
- SETUP.md: DecompileFunctions.java (headless batch Ghidra-C pre-pass) in the inventory (R21)
Capture the strategic pivot (Drew, 2026-06-19) and everything the fresh session needs:
- NO-GO on the 5-day brute force; GO on guided hand-matching (whole-binary-gated, ×134/match)
- the demo protocol (prove the loop on a medium struct-using fn, then a giant)
- THE CRUX: gate whole-binary, not match_one (isolated masks relocations -> doesn't transfer)
- the actor struct (base 0x80078E00 + confirmed fields + files) as the hand-matching asset
- tooling map, candidate targets, fleet-extract note
- deferred & decided (CUDA/ML = research wildcard not the plan; emulator naming paused)
- effort=Max for hand-matching (re-set, session-only); /mcp after the SessionStart restart
After the demo + PhaseEnd: plan-mode a whole-binary-gated permuter driven by Ghidra-C.
The permuter closes isolated near-misses (3-4/8 in <=75s; converted winners match_one
MATCH 17/53/54 ins) but 0/4 survive the whole-binary gate. Root cause (R17): the isolated
compile context differs from the whole-binary TU (callee inlining in the one-big-file
077.c), so match_one's "isolated == whole-binary" assumption is false for these and the
permuter optimizes the wrong target. This is the mechanism behind Phase 16's "42% mirage".
All 5 avenues tested: T2=0, T3=+0.52% (banked), T4/T5 byte-neutral, T6=0. The harness
ceiling did not rise to eureka. NEXT: T7 go/no-go (Tier-1/Max).
- recovered the engine actor/player struct cold via PCSX-Redux (R10/R11): base 0x80078E00,
~154 fields over 0x24C bytes, 32 pointer fields; HP/BP/gold/day/hour/position/BP-drain all
live-verified and anchored to the Phase-3 player state + the walk diff. -> docs/actor-struct.md
- tools/ram_probe.py: PCSX-Redux RAM capture/diff/read/ptr-scan workhorse for field-typing
- FINDING (the matching verdict): feeding the struct to m2c as --context is BYTE-NEUTRAL.
m2c adopts it (arg0->f_X) but structural-miss sample scores 0 better / 10 same / 2 worse vs
m2c-alone (full code+RAM pointer set). The misses are 50-98% mismatched = m2c decompiler
limits, not missing types. T4 fn-ptr context unblocks compilation but functions stay structural.
- conclusion: rich context (struct/type recovery) is a comprehension / decomp.me win, NOT a
byte-match lever -> Phase-16 wall re-confirmed from the type angle. struct-core-pivot.md updated.
- emulator field-naming pass deferred (comprehension, off the matching path); NEXT: T6 permuter
validation on the 146 genuine near-misses, then T7 go/no-go.
T4 in progress: proved the m2c --context fn-ptr-table mechanism, mapped byte-safe
shared-header integration (163 ground-truth + 186 used tables), found the fn-ptr fix
unblocks compilation but the byte-lever is T5's actor struct -> build T4 generator +
T5 struct together. No code artifact yet (probes in .run scratch). Paused for Drew's
PCSX-Redux emulator field-typing (T5) scheduling. Tree clean; all binaries byte-identical.
- banked 42 gate-proven SIG_FIXABLE_KR matches in ov_SC01_077 (of 48 predicted; 6 honestly
reverted by the whole-binary byte-gate, P9) + propagated 13 h_exact-shared x134 via
dedup_propagate (one body -> 134 overlays, each byte-gated, fail-closed)
- fleet byte-identical-from-source 54.48% -> 55.00% (+0.52%); REAL +1,778; dedup groups
1395 -> 1408; make check-all 136/136 byte-identical (R22); dedup-check 1408 validated/0 failed
- wall probe (R14): raw drafts 0/99 + sig_unify fails -> the K&R/sig_unify avenue is exhausted;
the residual conflict-class is ARITY_WALL (the §14e param-count dead-end) or structural (T4/T5)
- FIX a T1 classifier bug surfaced by verifying the wall vs the bytes: tu_compile_error captured
the baseline's pre-existing byte-NEUTRAL warnings ("makes integer from pointer", cc1 rc=0) and
mislabeled 99 fns LOOSE_TYPING_WALL; now keeps only non-warning errors -> there is NO loose-typing
wall, the real residual wall is ARITY_WALL=74. docs/wall-taxonomy.md amended with the correction.
- 134 overlay .c files carry the propagated DEFINE_func instantiations; a one-time full-fleet
make extract was required for the per-overlay propagation gate + check-all
- PhaseEnd_Phase16.md: the loose-typing wall (byte-proven), the kept harness fixes, the unattended
driver (built/not-deployed), and the new research directions (emulator -> Ghidra type propagation).
- CURRENT_PHASE.md archived -> logs/Phase16.md (R19).
- Negative-but-decisive result (Phase-3.5 precedent); bumps 1.14.0 -> 1.15.0.
- docs/struct-core-pivot.md: findings + decision + new research directions. Root cause = the
original engine is LOOSELY TYPED (K&R; same fn called with int/ptr, arg/no-arg across sites),
so no single canonical signature exists -> m2c guesses inconsistently, permuter can't fix
semantics, byte-gate (correctly) rejects. Yields ~3%, not the crack. New plan: emulator-recover
the actor struct/types -> Ghidra global type propagation -> Ghidra-C -> permuter+gate.
- harness bug-fixes (REAL, kept): p16_permute output-0-only match (killed the false '42%'),
base.c keeps callee externs, winner_to_draft line-strip; sig_unify canonicalizes m2c's
no-extern prototypes; gen_engine_decls.py (documents why a global canonical header breaks
loose-typed matches).
- a few byte-gated leaf matches banked in ov_SC01_077.c.
- driver per-fn flow: m2c -> match_one prefilter -> if near-miss & --permute-secs>0, run
decomp-permuter (time-boxed, reuses p16_permute setup/run) -> use the score-0 winner as the
draft -> whole-binary byte-gate. The permuter is closing near-misses well in the overnight
batch, so it's the driver's core yield step for the unattended run.
- +--permute-j (RAM-bound on the 15GiB box).
- S0 (cookbook §15): ran m2c on real ov_SC01_077 stubs (R14). Finding: m2c
--valid-syntax output COMPILES via m2c_macros.h (M2C_FIELD = byte-faithful cast);
30/30 sampled targets use only byte-faithful macros. Compiling != matching: the
residual is regalloc/schedule -> decomp-permuter is the byte-closer; struct types
are an enhancer, not the sole gate (S3/GATE-B measures the lift). ML parked (owner).
- common.h: add s64/u64/f64 + M2C_UNK* typedefs + byte-faithful M2C_FIELD/M2C_BITWISE
macros (non-faithful m2c macros left UNDEFINED = early 'defer' signal). Byte-neutral:
main 143dbb89 (clean rebuild, R22), resident 8e17e02f, ov_SC01_077 d19c9580 all OK.
- CURRENT_PHASE.md: Phase 16 plan + S0-S9 gated tasks + Sun-afternoon timeline +
known-answer oracle test method.
- tools/sig_unify.py (NEW): unifies a draft's FULL signature set to the banked-canonical
decls — both callee externs AND the draft's OWN definition signature (return + param
types, body param-names preserved). Recovery for the standalone-MATCH residual: a probe
found 30/30 sampled failures were type conflicts, 0 false-positives. Gated 191 -> 32 verified.
- tools/dedup_propagate.py find_site FIX: the inline-def detector required the opening brace
ON THE SAME LINE as the signature, silently dropping every next-line-brace def from
propagation (sig_unify/permuter outputs + ~stragglers all session). Now accepts brace on
the same OR next line -> unlocked a 61-function propagation backlog.
- propagated 61 fns fleet-wide (incl. 32 sig-unify + the permuter win func_801508F8 +
earlier-dropped next-line-brace matches), all 134 byte-identical; dedup-check 1394/0
- R22 CLEAN fleet rebuild: 136/136 byte-identical. REAL 177501 -> 185646. Zero agent tokens.
- fix tools/build_engine_types.py to be ADDITIVE: merge new source types with the
types already in engine_types.h (a re-run after a prior --strip was destructive —
it dropped the ~34 already-migrated types and broke the build; caught + reverted)
- engine_types.h 34 -> 42 named types (the 8 from wave-1's struct-using matches);
inline defs stripped from ov_SC01_077.c (byte-neutral)
- dedup_propagate --auto-from: 8 of 15 struct-blocked matches now self-contained ->
propagated across all 134 overlays (byte-gated); 7 remain (anonymous/typedef'd types)
- R22 CLEAN fleet rebuild: 136/136 byte-identical; dedup-check 1333/0
- zero agent tokens (deterministic)
- canon_draft_decls recovery on the combined 505 quarantined drafts (285 small + 220 medium),
now canonicalizing against the much-larger banked symbol set: 47 verified (no API), 42
propagated fleet-wide (struct/local-type skipped), 134 overlays byte-identical; dedup
1141 -> 1183 groups, dedup_integrate --check 1183 validated / 0 failed.
- fleet REAL substantive 151,908 -> 157,541; byte-identical 44.64% -> 46.27%.
- R22 clean rebuild: 136/136 byte-identical, 0 extract failures.
- scoped the struct-using-shared-fn lever (read-only): 34 named struct/union types in 077,
ZERO same-name-different-layout collisions -> a shared-types-header is a clean path to
propagate the ~100 struct-using fns (next).
- gen_harvest_targets.py: now also resolves canonical DATA-symbol declarations per target
(scan the .s for D_XXXX refs -> `extern <type> D_XXXX;` from banked code) in addition to
callee function sigs — the §14c(c) fix at the source, so agents declare both functions and
data correctly and avoid the dominant remaining conflict class.
- v2 harvest (callee+data-sig-aware) on the 591 small still-stub call-heavy fns + a gap-fill
agent for 6 targets dropped by a connection-closed-mid-response failure (the retry-wave
misses truncated-non-null results). Gate: 306 verified (ov_SC01_077 1554 -> 1248 stubs).
- dedup_propagate.py: skip any struct/union-touching body — agents named structs with colliding
generic names (`struct S`/`struct vec`) inline, so two macros' types redefine/conflict when
instantiated in one overlay (ov_SC01_000 abort). 34 skip; struct-using shared fns stay banked
in 077 (shared-types-header follow-up).
- 281 functions propagated fleet-wide (134 overlays each, every overlay byte-gated); dedup
795 -> 1076 groups, dedup_integrate --check 1076 validated / 0 failed.
- fleet REAL substantive 105,764 -> 143,322; byte-identical 31.22% -> 42.14%.
- R22 clean rebuild: 136/136 byte-identical, 0 extract failures.
- cookbook §12: gap-fill is now a MANDATORY post-step for every multi-agent run (reconcile
produced-vs-expected: missing + truncated drafts) before gating.
- tools/canon_draft_decls.py: rewrites a quarantined harvest draft's extern/data
declarations to the banked-consistent set (functions from engine_core.h DEFINE
bodies / extern decls / inline defs; data from existing `extern <type> D_XXXX;`),
so a draft whose BODY is byte-correct (passes match_one) but whose DECLARATION
collided with already-banked code can re-gate cleanly. The whole-binary
harvest_verify byte-gate stays the sole arbiter — a semantics-changing
canonicalization just fails and reverts.
- recovered 19 of the 90 match_one-MATCH-but-conflicting quarantined failures
(no API — immune to the rate limiting), propagated fleet-wide, all 134 overlays
byte-identical; dedup 776 -> 795 groups, 0 failed.
- fleet byte-identical 30.48% -> 31.22%; ov_SC01_077 1574 -> 1535 stubs.
- R22 clean rebuild: 136/136 byte-identical, 0 extract failures.
- the remaining ~70 match_one-MATCH residual needs definition-widening (§14c(d),
narrow-return) or true data-type resolution — queued for the next pass, with the
un-started medium band (430) and the 54 genuine instruction-misses (permuter/§3a).
- progress.py: refactor to report(binary) + set_binary() + a per-binary .s index
(fleet run 6m38s -> 7s); add --fleet -> docs/progress.fleet.md (deterministic,
source-derived per-binary table + fleet totals). Single-binary output byte-unchanged.
- fleet now: 947 REAL / 959 LINKED / 13132 byte-identical / 344010 matchable = 3.82%
across 136 binaries (main + resident + 134 overlays)
- dup_report --cross: HONESTY FIX (R14) — an onboarded overlay is both a named BINARIES
entry AND a .run/sig.ov_* glob hit; it was counted twice, inflating collapsible bytes
to 58.7M once the fleet was onboarded. Dedupe by alias -> accurate 9366 h_exact groups
/ 28.6M collapsible (matches the Phase-11 figure)
- Makefile: wire progress.py --fleet into make report (main-only block)
- docs: progress.fleet.md (new), duplicates.cross.md regenerated, SETUP inventory
- make report green; dedup-check 8 validated / 0 failed
- ghidra/ churn NOT staged (R23)
- onboarded all 118 not-yet-onboarded overlays via tools/new_overlay.sh (0 failed)
-> fleet = 134 location overlays + main + resident = 136 binaries
- R22 CLEAN rebuild (make clean -> extract all 136 -> make check-all):
136 passed, 0 failed of 136 -- every overlay byte-identical at 100% INCLUDE_ASM
from a fully clean tree, 0 extract failures
- per-overlay config (splat.ov_*.yaml / check.ov_*.sha / symbols.ov_*.txt) +
config/overlays.mk var blocks + the 4 report/diff BINARIES dicts auto-registered
(sentinel-anchored, idempotent); src/ov_*/ stub records committed (harvest_verify
reverts via git checkout, so the records must be tracked before any harvest)
- ghidra/ churn NOT staged (R23); build/asm/assets/.run gitignored
- report-digest regen (fleet-wide duplicates.cross.md + progress.fleet.md) is T4
- tools/dedup_propagate.py: lift a matched body -> DEFINE_func_<ADDR>() macro in
src/shared/engine_core.h -> instantiate in place at every onboarded overlay sharing
that h_exact -> snapshot + per-overlay byte-gate (fail-closed revert) -> register in
config/dedup.us.yaml. Keyed by addr-int (sig lowercase vs splat func_%08X).
- proof: 4 medium funcs propagated across all 16 onboarded overlays; clean check-all
18/18 byte-identical; dedup_integrate 8 validated/0 failed; idempotent; negative test
(corrupt shared body) -> make check FAILS then restores
- Makefile: header-dependency tracking (cpp -MMD -MP + -include ) so editing a
shared header recompiles dependents — closes a stale-build/false-pass gap the negative
test exposed; output-neutral (R22 clean 18/18)
- structural self-check: a leftover INCLUDE_ASM stub is byte-identical, so the byte-gate
can't catch under-application -> assert DEFINE present + stub gone
- FINDING: 577 of ov_SC01_077's 785 matches are h_exact across all 134 overlays
(2.19 MB collapsible, already matched) -> T5 = bulk-propagate these + harvest the rest
- cookbook §14 (the propagation runbook + gotchas), SETUP tool inventory
- ghidra/ churn NOT staged (R23); build/asm/.run gitignored
Drew T