- grinder.py: persistent blacklist for permuter-won/gate-rejected (plumbing-bound) fns;
stops the tried.clear()-after-idle churn (was 0 banks in 8h on func_8014F3E8/func_8014FE60)
- sig_unify.py rewrite_def: adopt the canonical param list on arity mismatch (DEF-side
loose-typing wall) — unused extras sit in $a0-$a3, free at -O2; banked func_8016EDEC/EE40
- cookbook §22 (DEF-side recovery + grinder blacklist); CURRENT_PHASE option-C log
- compounds: sig_unify is in the gate pipeline -> every future wave + the grinder auto-recover
- residual class travels with the draft: drafter stamps // @class / // @stuck; gate_stage logs
the worker's gcc-quirk class to the backlog (not the coarse manifest class).
- wave_targets.py --class <C> / --list-classes: class-grouped re-attempt waves over the backlog's
classified near-misses (REGALLOC/SCHEDULE/REMAT/STRUCT/IV/LOOPGUARD/LOOSE/PLUMBING).
- orchestrator prep: auto-picks a class-focused wave when a class accrues >= threshold near-misses,
else a fresh pool harvest; clears the draft dir per wave.
- tools/workflows/distill.js: post-wave agent extracts NEW byte-verified gcc idioms -> cookbook §21
(R16, conservative); drafters now read the LIVE cookbook so distilled idioms feed forward.
- runbook: the full prep->worker->gate->distill cycle (the Phase-18 close-rate loop, automated).
- gate_stage on the 15 worker drafts saved before the reboot: banked 2 byte-matches,
propagated x134 (dedup 1508->1510); the gate process was killed pre-commit (empty log),
banks recovered from the working tree (dedup_propagate fail-closed per-overlay = verified).
Remaining 13 -> backlog near-misses.
- the %lo-folding indexed-global -O0 residual (§18, long "irreducible") is CRACKED: declare
the global as "extern Struct base[]" (sizeof == stride) and access "base[i].field" -> gcc
keeps sym a symbol_ref and folds %lo(sym+off) into the store, byte-matching. The failing
forms "*(T*)(&sym + i*stride)" materialize &sym as a value first, blocking the fold.
- func_8013B7AC byte-gated + banked (x1 — the -O0 cluster is overlay-local, R14 finding above);
the idiom is reusable fleet-wide for ANY indexed-global access (-O0 or -O2).
- cookbook §18 ("%lo-folding indexed global — CRACKED") + §20 placeholder closed.
- the other 9 -O0 stubs: idiom applies but need per-fn -O0 drafting (3 quick ones tried, failed
on -O0 codegen detail; 4 are large) -> backlog/autonomy fuel. clean rebuild d19c9580 (R22).
- byte-proven: 0 drafts declare an extern struct/union data conflict; the lone apparent
DATA-conflict (func_8016A8FC/D_800AE620) was a typedef-REDEFINITION (Blk20 inline vs
engine_types.h, same type) masking the def-sig loose-typing wall. Stripping the inline
typedef compiles but still byte-mismatches. data-cast = code for 0 cases -> not built.
- cookbook §20 DATA-conflict note corrected; fleet unchanged 58.82%.
- byte-proven: only 1/134 overlays shares ov_SC01_077's -O0 cluster bytes; the 6 matched
-O0 fns reference per-overlay data addresses (func_8013B568 -> D_80187270 in ov_SC01_077
but 0x80182B04 in ov_SC01_005), so they're overlay-LOCAL, not ×134-shareable. The
Phase-18/19 'reach-134' label conflated function-present with byte-identical (R14).
- the §18 split infra (rollout tool + Makefile %_o0.o pattern + PROVIDE .ld) was built and
byte-validated to compile/link a 2nd overlay; the whole-binary gate then exposed the
per-overlay-data wall -> reverted (the finding is the deliverable, P9).
- -O0 rollout = ×1 per-overlay-unique re-matching (~0.2-0.3%), NOT +0.6% free -> SKIP.
cookbook §18 corrected; fleet unchanged 58.82%, 136/136 byte-identical.
- tools/cast_call_sites.py: per draft, callee whose canonical TU sig != draft's
intended sig -> decl->canonical (kills in-TU conflicting types, keeps symbol in
scope) + call-site cast ((ret(*)(args))func_X)(args) (codegen-neutral: gcc folds
the cast of a known symbol -> direct jal). Pure --in/--out; whole-binary gate is
the arbiter (G3/P9). Pipeline: canon_resident_calls -> cast_call_sites ->
sig_unify -> harvest_verify --chunk 1
- recovered 6 of T6 batch-1's 33 gate-fails byte-identical (func_80153C44/8015CF58/
801711FC/80161BE0/801683D8/8015F948); 5 propagated x134 + 1 local; check-all 136/136
- R14 CORRECTION of cookbook §20's '~33' projection (byte-proven): the batch is NOT
mostly callee-conflicts — ~6 callee (cast-recovered), ~18 DEF-side loose-typing wall
(def's matchable sig conflicts with the unchangeable caller-canonical; caller-side
fix BLOCKED because INCLUDE_ASM declares no symbol -> a shared macro's internal
extern is load-bearing in the 133 stub overlays), + a small DATA tail. The def-side
needs re-drafting under the caller-canonical sig (future wave), not a text transform
- cookbook §20 RESOLUTION + SETUP tool inventory (R16/R21/R30); CURRENT_PHASE log (R28)
- fleet 58.63% -> 58.82%, dedup 1501 -> 1508, 0 NON_MATCHING (G4)
- build_engine_types.py --strip: lifted 48 named types -> src/shared/engine_types.h
(byte-neutral, ov_SC01_077 d19c9580); unblocked +4 struct-using matches -> propagated ×134
(incl. func_80147514, a T2 match freed by the lift)
- check-all 136/136 BYTE-IDENTICAL (R22); fleet 57.93% -> 58.00%; phase total 56.64% -> 58.00%
- KEY FINDING: propagation (not matching) now caps the fleet-% yield — ~16 banked matches stay
ov_SC01_077-local because their bodies use typedef'd/anonymous/sibling local types that the
named-struct-only build_engine_types can't lift (dedup_propagate compiles_standalone rejects).
Phase-20 lever: extend the type-lift to typedefs -> recovers those ×134 for ~0 agent tokens
- R14 finding: the 3 Phase-18 "implicit-int / propagate-first" near-misses are NOT
implicit-int (reproduced each through the byte-gate). Real classes:
(a) resident-callee LINK-miss: draft calls func_<ADDR> but the addr has a curated
name (0x8004CFEC = ratan2) -> "undefined reference to func_8004CFEC". NEW
tools/canon_resident_calls.py rewrites func_<ADDR> -> curated name (pure draft
text, body bytes unchanged) -> run FIRST in the recovery pipeline.
(b) shared-caller ARITY conflict: an engine_core.h caller macro declares the callee
file-scope `extern void func_X(void)` then calls func_X(); the real def takes an
arg -> "conflicting types". FIX = caller extern -> no-proto `extern void func_X();`
(byte-neutral for the caller; compatible with promotion-safe params int/long/ptr).
- 3 matches banked in ov_SC01_077: func_8017209C (ratan2 rewrite), func_80147514 (s32),
func_80168F40 (void*); 2 no-proto lines in src/shared/engine_core.h
- recovery pipeline: draft -> canon_resident_calls -> sig_unify -> harvest_verify --chunk 1
- make check-all 136/136 BYTE-IDENTICAL (full extract, R22); dedup 1450 validated/0 failed;
0 NON_MATCHING in any default build (G4); fleet 56.64% (the 3 are ov_SC01_077-local,
x134 propagation rides T3/Phase-20)
- cookbook §17a-3 corrected (R14/R16); SETUP tool inventory (R21); CURRENT_PHASE T2 done
- §17 rewritten: the call-crossing register-ORDER class is MATCHABLE with register
__asm__ pins + a scheduling barrier (byte-proven func_8012B8E4), not unsteerable.
The wall was a missing lever (the pins), not an impossibility. Recipe documented as
a TOOLKIT: triage with match_one -> reconstruct / array-decay / PINS / barrier.
- §16 cross-refs + hand-matching §8e corrected to match. The ONLY genuine stub-and-skip
left is the narrow-param loose-typing conflict (func_80146A6C).
- self-correction (P9/R14): I concluded 'unsteerable' before trying the pins; Drew's
hand-match-everything directive exposed the gap. Recorded honestly.
The 'unsteerable' regalloc-order class is NOT unsteerable. Drew's directive (hand-match
everything) exposed the lever I'd skipped: explicit register pinning.
- func_8012B8E4 (the flagship circular reach-134 'unsteerable' fn): 21 -> MATCH via
* register pins: register s32 d __asm__("$16"); register s32 s1ang __asm__("$17")
-> FORCES the $s0/$s1 global allocation gcc otherwise density-orders the other way
* branch-polarity invert (§3-T4): 24->21
* explicit clamp temps (t=u6+0x1000; iVar4=u5-t): fixes the reassociation, 7->3
* scheduling barrier __asm__ __volatile__("" : : "r"(u5)): anchors the last sll, 3->MATCH
- byte-gated (harvest_verify, G3/P9) + propagated x134 (per-overlay gate); 134 byte-identical
- recipe = §10-family hand technique (pins + barriers); labor-intensive (per-fn register map
from the asm) but it WORKS -> the circular tail IS hand-matchable, not asm-only
- supersedes the T1-T5 'unsteerable' verdict; cookbook §17 + PhaseEnd to be revised accordingly
- canonical-sig layer (session 4): tools/census_conflict_callees.py + derive_canonical_sigs.py
-> a 20-extern byte-neutral block atop ov_SC01_077.c (LOCAL, not engine_core.h); census
conflict callees 20->0, blocked targets 24->0; gate pipeline now draft -> sig_unify (MANDATORY)
-> harvest_verify --chunk 1; fleet 136/136 byte-identical (R22), 55.51% (no regression)
- FINDING (R14/P9): the conflict wall is 7%-reach not ~2x; the 4 reach-134 circular targets are
ALL gcc-quirk/regalloc/layout-bound (0 banked); the high-reach core IS the quirk tail; struct
types are byte-neutral for matching (the wall is gcc codegen, not knowledge)
- leverage analysis: fleet % is function-count-weighted (size adds no %); "unblock many" = the
layer (declaration, not matching); reach is the lever (already reach-sorted); 247 tractable
reach-134 stubs ~ +3-4% projected
- GO/NO-GO: NO-GO on brute waves at the current ceiling; GO on a compiler-quirk research phase
(read gcc-2.7.2 source + Xenogears + the §10/regalloc classes, R17) -> then resume the wave
- docs: cookbook §16 corrected + hand-matching-process.md §8 (the layer + the finding + handoff)
- worklog archived -> phase-ends/logs/Phase17.md (R19); bumps 1.15.0 -> 1.16.0
- CURRENT_PHASE.md: new session-4 START HERE (build the canonical-sig layer, then
scale the wave); old demo-protocol START HERE marked superseded
- hand-matching-process.md S7: the Ultracode harvest wave (pipeline, calibration
result 60% match_one / 33% whole-binary / +0.47%) + THE CANONICAL-SIG WALL (the
60->33 gap is 100% sig conflicts; surgical per-callee canonicalization = ~2x lever)
- matching-cookbook.md S16: guided-hand-matching breakthrough + new idioms (mask-local,
shared-ret0 goto, v0/v1 coalescing) + the wave/canonical-sig finding (R16 flywheel)
- SETUP.md: DecompileFunctions.java (headless batch Ghidra-C pre-pass) in the inventory (R21)
- recovered the engine actor/player struct cold via PCSX-Redux (R10/R11): base 0x80078E00,
~154 fields over 0x24C bytes, 32 pointer fields; HP/BP/gold/day/hour/position/BP-drain all
live-verified and anchored to the Phase-3 player state + the walk diff. -> docs/actor-struct.md
- tools/ram_probe.py: PCSX-Redux RAM capture/diff/read/ptr-scan workhorse for field-typing
- FINDING (the matching verdict): feeding the struct to m2c as --context is BYTE-NEUTRAL.
m2c adopts it (arg0->f_X) but structural-miss sample scores 0 better / 10 same / 2 worse vs
m2c-alone (full code+RAM pointer set). The misses are 50-98% mismatched = m2c decompiler
limits, not missing types. T4 fn-ptr context unblocks compilation but functions stay structural.
- conclusion: rich context (struct/type recovery) is a comprehension / decomp.me win, NOT a
byte-match lever -> Phase-16 wall re-confirmed from the type angle. struct-core-pivot.md updated.
- emulator field-naming pass deferred (comprehension, off the matching path); NEXT: T6 permuter
validation on the 146 genuine near-misses, then T7 go/no-go.
- banked 42 gate-proven SIG_FIXABLE_KR matches in ov_SC01_077 (of 48 predicted; 6 honestly
reverted by the whole-binary byte-gate, P9) + propagated 13 h_exact-shared x134 via
dedup_propagate (one body -> 134 overlays, each byte-gated, fail-closed)
- fleet byte-identical-from-source 54.48% -> 55.00% (+0.52%); REAL +1,778; dedup groups
1395 -> 1408; make check-all 136/136 byte-identical (R22); dedup-check 1408 validated/0 failed
- wall probe (R14): raw drafts 0/99 + sig_unify fails -> the K&R/sig_unify avenue is exhausted;
the residual conflict-class is ARITY_WALL (the §14e param-count dead-end) or structural (T4/T5)
- FIX a T1 classifier bug surfaced by verifying the wall vs the bytes: tu_compile_error captured
the baseline's pre-existing byte-NEUTRAL warnings ("makes integer from pointer", cc1 rc=0) and
mislabeled 99 fns LOOSE_TYPING_WALL; now keeps only non-warning errors -> there is NO loose-typing
wall, the real residual wall is ARITY_WALL=74. docs/wall-taxonomy.md amended with the correction.
- 134 overlay .c files carry the propagated DEFINE_func instantiations; a one-time full-fleet
make extract was required for the per-overlay propagation gate + check-all
- docs/struct-core-pivot.md: findings + decision + new research directions. Root cause = the
original engine is LOOSELY TYPED (K&R; same fn called with int/ptr, arg/no-arg across sites),
so no single canonical signature exists -> m2c guesses inconsistently, permuter can't fix
semantics, byte-gate (correctly) rejects. Yields ~3%, not the crack. New plan: emulator-recover
the actor struct/types -> Ghidra global type propagation -> Ghidra-C -> permuter+gate.
- harness bug-fixes (REAL, kept): p16_permute output-0-only match (killed the false '42%'),
base.c keeps callee externs, winner_to_draft line-strip; sig_unify canonicalizes m2c's
no-extern prototypes; gen_engine_decls.py (documents why a global canonical header breaks
loose-typed matches).
- a few byte-gated leaf matches banked in ov_SC01_077.c.
- S0 (cookbook §15): ran m2c on real ov_SC01_077 stubs (R14). Finding: m2c
--valid-syntax output COMPILES via m2c_macros.h (M2C_FIELD = byte-faithful cast);
30/30 sampled targets use only byte-faithful macros. Compiling != matching: the
residual is regalloc/schedule -> decomp-permuter is the byte-closer; struct types
are an enhancer, not the sole gate (S3/GATE-B measures the lift). ML parked (owner).
- common.h: add s64/u64/f64 + M2C_UNK* typedefs + byte-faithful M2C_FIELD/M2C_BITWISE
macros (non-faithful m2c macros left UNDEFINED = early 'defer' signal). Byte-neutral:
main 143dbb89 (clean rebuild, R22), resident 8e17e02f, ov_SC01_077 d19c9580 all OK.
- CURRENT_PHASE.md: Phase 16 plan + S0-S9 gated tasks + Sun-afternoon timeline +
known-answer oracle test method.
- tools/sig_unify.py (NEW): unifies a draft's FULL signature set to the banked-canonical
decls — both callee externs AND the draft's OWN definition signature (return + param
types, body param-names preserved). Recovery for the standalone-MATCH residual: a probe
found 30/30 sampled failures were type conflicts, 0 false-positives. Gated 191 -> 32 verified.
- tools/dedup_propagate.py find_site FIX: the inline-def detector required the opening brace
ON THE SAME LINE as the signature, silently dropping every next-line-brace def from
propagation (sig_unify/permuter outputs + ~stragglers all session). Now accepts brace on
the same OR next line -> unlocked a 61-function propagation backlog.
- propagated 61 fns fleet-wide (incl. 32 sig-unify + the permuter win func_801508F8 +
earlier-dropped next-line-brace matches), all 134 byte-identical; dedup-check 1394/0
- R22 CLEAN fleet rebuild: 136/136 byte-identical. REAL 177501 -> 185646. Zero agent tokens.
Drew T