- driver per-fn flow: m2c -> match_one prefilter -> if near-miss & --permute-secs>0, run
decomp-permuter (time-boxed, reuses p16_permute setup/run) -> use the score-0 winner as the
draft -> whole-binary byte-gate. The permuter is closing near-misses well in the overnight
batch, so it's the driver's core yield step for the unattended run.
- +--permute-j (RAM-bound on the 15GiB box).
- S0 (cookbook §15): ran m2c on real ov_SC01_077 stubs (R14). Finding: m2c
--valid-syntax output COMPILES via m2c_macros.h (M2C_FIELD = byte-faithful cast);
30/30 sampled targets use only byte-faithful macros. Compiling != matching: the
residual is regalloc/schedule -> decomp-permuter is the byte-closer; struct types
are an enhancer, not the sole gate (S3/GATE-B measures the lift). ML parked (owner).
- common.h: add s64/u64/f64 + M2C_UNK* typedefs + byte-faithful M2C_FIELD/M2C_BITWISE
macros (non-faithful m2c macros left UNDEFINED = early 'defer' signal). Byte-neutral:
main 143dbb89 (clean rebuild, R22), resident 8e17e02f, ov_SC01_077 d19c9580 all OK.
- CURRENT_PHASE.md: Phase 16 plan + S0-S9 gated tasks + Sun-afternoon timeline +
known-answer oracle test method.
- tools/sig_unify.py (NEW): unifies a draft's FULL signature set to the banked-canonical
decls — both callee externs AND the draft's OWN definition signature (return + param
types, body param-names preserved). Recovery for the standalone-MATCH residual: a probe
found 30/30 sampled failures were type conflicts, 0 false-positives. Gated 191 -> 32 verified.
- tools/dedup_propagate.py find_site FIX: the inline-def detector required the opening brace
ON THE SAME LINE as the signature, silently dropping every next-line-brace def from
propagation (sig_unify/permuter outputs + ~stragglers all session). Now accepts brace on
the same OR next line -> unlocked a 61-function propagation backlog.
- propagated 61 fns fleet-wide (incl. 32 sig-unify + the permuter win func_801508F8 +
earlier-dropped next-line-brace matches), all 134 byte-identical; dedup-check 1394/0
- R22 CLEAN fleet rebuild: 136/136 byte-identical. REAL 177501 -> 185646. Zero agent tokens.
- fix tools/build_engine_types.py to be ADDITIVE: merge new source types with the
types already in engine_types.h (a re-run after a prior --strip was destructive —
it dropped the ~34 already-migrated types and broke the build; caught + reverted)
- engine_types.h 34 -> 42 named types (the 8 from wave-1's struct-using matches);
inline defs stripped from ov_SC01_077.c (byte-neutral)
- dedup_propagate --auto-from: 8 of 15 struct-blocked matches now self-contained ->
propagated across all 134 overlays (byte-gated); 7 remain (anonymous/typedef'd types)
- R22 CLEAN fleet rebuild: 136/136 byte-identical; dedup-check 1333/0
- zero agent tokens (deterministic)
- canon_draft_decls recovery on the combined 505 quarantined drafts (285 small + 220 medium),
now canonicalizing against the much-larger banked symbol set: 47 verified (no API), 42
propagated fleet-wide (struct/local-type skipped), 134 overlays byte-identical; dedup
1141 -> 1183 groups, dedup_integrate --check 1183 validated / 0 failed.
- fleet REAL substantive 151,908 -> 157,541; byte-identical 44.64% -> 46.27%.
- R22 clean rebuild: 136/136 byte-identical, 0 extract failures.
- scoped the struct-using-shared-fn lever (read-only): 34 named struct/union types in 077,
ZERO same-name-different-layout collisions -> a shared-types-header is a clean path to
propagate the ~100 struct-using fns (next).
- gen_harvest_targets.py: now also resolves canonical DATA-symbol declarations per target
(scan the .s for D_XXXX refs -> `extern <type> D_XXXX;` from banked code) in addition to
callee function sigs — the §14c(c) fix at the source, so agents declare both functions and
data correctly and avoid the dominant remaining conflict class.
- v2 harvest (callee+data-sig-aware) on the 591 small still-stub call-heavy fns + a gap-fill
agent for 6 targets dropped by a connection-closed-mid-response failure (the retry-wave
misses truncated-non-null results). Gate: 306 verified (ov_SC01_077 1554 -> 1248 stubs).
- dedup_propagate.py: skip any struct/union-touching body — agents named structs with colliding
generic names (`struct S`/`struct vec`) inline, so two macros' types redefine/conflict when
instantiated in one overlay (ov_SC01_000 abort). 34 skip; struct-using shared fns stay banked
in 077 (shared-types-header follow-up).
- 281 functions propagated fleet-wide (134 overlays each, every overlay byte-gated); dedup
795 -> 1076 groups, dedup_integrate --check 1076 validated / 0 failed.
- fleet REAL substantive 105,764 -> 143,322; byte-identical 31.22% -> 42.14%.
- R22 clean rebuild: 136/136 byte-identical, 0 extract failures.
- cookbook §12: gap-fill is now a MANDATORY post-step for every multi-agent run (reconcile
produced-vs-expected: missing + truncated drafts) before gating.
- tools/canon_draft_decls.py: rewrites a quarantined harvest draft's extern/data
declarations to the banked-consistent set (functions from engine_core.h DEFINE
bodies / extern decls / inline defs; data from existing `extern <type> D_XXXX;`),
so a draft whose BODY is byte-correct (passes match_one) but whose DECLARATION
collided with already-banked code can re-gate cleanly. The whole-binary
harvest_verify byte-gate stays the sole arbiter — a semantics-changing
canonicalization just fails and reverts.
- recovered 19 of the 90 match_one-MATCH-but-conflicting quarantined failures
(no API — immune to the rate limiting), propagated fleet-wide, all 134 overlays
byte-identical; dedup 776 -> 795 groups, 0 failed.
- fleet byte-identical 30.48% -> 31.22%; ov_SC01_077 1574 -> 1535 stubs.
- R22 clean rebuild: 136/136 byte-identical, 0 extract failures.
- the remaining ~70 match_one-MATCH residual needs definition-widening (§14c(d),
narrow-return) or true data-type resolution — queued for the next pass, with the
un-started medium band (430) and the 54 genuine instruction-misses (permuter/§3a).
- progress.py: refactor to report(binary) + set_binary() + a per-binary .s index
(fleet run 6m38s -> 7s); add --fleet -> docs/progress.fleet.md (deterministic,
source-derived per-binary table + fleet totals). Single-binary output byte-unchanged.
- fleet now: 947 REAL / 959 LINKED / 13132 byte-identical / 344010 matchable = 3.82%
across 136 binaries (main + resident + 134 overlays)
- dup_report --cross: HONESTY FIX (R14) — an onboarded overlay is both a named BINARIES
entry AND a .run/sig.ov_* glob hit; it was counted twice, inflating collapsible bytes
to 58.7M once the fleet was onboarded. Dedupe by alias -> accurate 9366 h_exact groups
/ 28.6M collapsible (matches the Phase-11 figure)
- Makefile: wire progress.py --fleet into make report (main-only block)
- docs: progress.fleet.md (new), duplicates.cross.md regenerated, SETUP inventory
- make report green; dedup-check 8 validated / 0 failed
- ghidra/ churn NOT staged (R23)
- onboarded all 118 not-yet-onboarded overlays via tools/new_overlay.sh (0 failed)
-> fleet = 134 location overlays + main + resident = 136 binaries
- R22 CLEAN rebuild (make clean -> extract all 136 -> make check-all):
136 passed, 0 failed of 136 -- every overlay byte-identical at 100% INCLUDE_ASM
from a fully clean tree, 0 extract failures
- per-overlay config (splat.ov_*.yaml / check.ov_*.sha / symbols.ov_*.txt) +
config/overlays.mk var blocks + the 4 report/diff BINARIES dicts auto-registered
(sentinel-anchored, idempotent); src/ov_*/ stub records committed (harvest_verify
reverts via git checkout, so the records must be tracked before any harvest)
- ghidra/ churn NOT staged (R23); build/asm/assets/.run gitignored
- report-digest regen (fleet-wide duplicates.cross.md + progress.fleet.md) is T4
- tools/dedup_propagate.py: lift a matched body -> DEFINE_func_<ADDR>() macro in
src/shared/engine_core.h -> instantiate in place at every onboarded overlay sharing
that h_exact -> snapshot + per-overlay byte-gate (fail-closed revert) -> register in
config/dedup.us.yaml. Keyed by addr-int (sig lowercase vs splat func_%08X).
- proof: 4 medium funcs propagated across all 16 onboarded overlays; clean check-all
18/18 byte-identical; dedup_integrate 8 validated/0 failed; idempotent; negative test
(corrupt shared body) -> make check FAILS then restores
- Makefile: header-dependency tracking (cpp -MMD -MP + -include ) so editing a
shared header recompiles dependents — closes a stale-build/false-pass gap the negative
test exposed; output-neutral (R22 clean 18/18)
- structural self-check: a leftover INCLUDE_ASM stub is byte-identical, so the byte-gate
can't catch under-application -> assert DEFINE present + stub gone
- FINDING: 577 of ov_SC01_077's 785 matches are h_exact across all 134 overlays
(2.19 MB collapsible, already matched) -> T5 = bulk-propagate these + harvest the rest
- cookbook §14 (the propagation runbook + gotchas), SETUP tool inventory
- ghidra/ churn NOT staged (R23); build/asm/.run gitignored
- onboarded ov_{SC01_000,SC01_001,SC02_000,SC02_003,SC03_002,SC03_006,
SC04_000,SC04_018,SC04_019,SC05_000,SC06_000,SC07_000} via tools/new_overlay.sh
— spans all 7 SC areas + 2 full duplicate pairs (SC02_000≡003, SC04_018≡019)
- each byte-identical at 100% INCLUDE_ASM (non-4-aligned auto-handled); no
position-lock anomalies — the cross-shape risk is retired before the full onboard
- make check-all = 18/18 passed (main + resident + 4 prior + 12 new)
- config/overlays.mk + 4 report dicts auto-registered (sentinel, idempotent)
- ghidra/ DB churn NOT staged (R23); asm/build/assets/.run gitignored
Controlled: 80 fixed SC01/077 fns (nins 8-15), identical blind-draft Workflow,
only session effort varied. Arm A (Ultracode/xHigh) 13/80, Arm B (Max) 14/80;
A∩B=12. 13-vs-14 = noise -> per-agent depth does NOT change blind-draft harvest
yield; the 'pass 1 Max did better' observation was difficulty-confounded.
DECISION: prefer Ultracode for breadth harvests (same yield + orchestration),
Max for deep synthesis. Banked Arm B's 14 -> ov_SC01_077 REAL 704 (byte-identical).
(scriptPath re-invoke didn't forward args -> Arm B overwrote Arm A's drafts dir;
Arm A was already gated/recorded, so the comparison stands; 1 A-only fn unbanked.)
finding -> docs/effort-map.md (Evidence section)
- src/shared/ov_setters.h: generic SETTER/RETCONST macro bodies (cookbook §11
source-level share), instantiated IN PLACE (address order preserved) in both
ov_SC01_005.c and ov_SC01_006.c via ../shared/ relative include
- config/dedup.us.yaml: D1/D2/D3 cross-overlay groups (members in BOTH 005 & 006)
- both make check BINARY=ov_SC01_005 AND =ov_SC01_006 -> 56760dbe BYTE-IDENTICAL
from the one shared source; dedup_integrate --check: 4 validated, 0 failed;
REAL=3 credited to both members
match_one iterate pass throttled by a transient server rate-limit (20/36 batches
lost); banked 52 from the 16 completed. Build d19c9580. Resume with paced blind
draft + retry-wave (§12 resilience).
splat carved 2505 functions cleanly (func_* names; the top cross-report group
func_80144B9C carved correctly at 770 ins). The C record is durable (splat
preserves it across re-extract); matches land here in T1d/T1e.
Live save-to-slot-1 trace via the PCSX-Redux web API confirmed the memcard path
bu00:BASLUS-00726MUSASHI (libmcrd) + the directory/title-frame staging in low RAM. The
save-DATA serialization couldn't be isolated from a live before/after diff (running-game
churn; the one big changed region @0x8007CD28 is an unrelated UI list). Clean route = the
.mcd file / a paused capture -> deferred as Gen3-repack territory, NOT needed for the
SaveLoadRoutine byte-match (code-level). SaveLoadRoutine stays the documented hard-defer
(1139-ins multi-entry, Gen1 precedent, P9).
formats.md §6: SaveLoadRoutine @0x8002B154 = multi-entry save/load blob driving the linked
PsyQ libmcrd primitives; saveHeaderTemplate @0x80072DF0 decoded — memcard filename
"BASLUS-00726MUSASHI", SJIS title "BRAVE FENCER MUSASHI", and a @+0x54 handler DISPATCH
TABLE of 0x8002Bxxx pointers (the "multi-entry blob" IS this table of save/load sub-handlers).
Static layout VERIFIED. TBD (needs Ghidra-on-EXE + a Drew-operated PCSX-Redux save trace +
PsyQ-memcard CRC research): the save-DATA serialization (player/progress state 0x80078Exx +
checksum) + matching the tractable handlers. ram_savescreen.bin holds the live header.
- memory-map.md: RESOLVE the "Script VM instruction set" open question — BFM has NO bytecode
VM. "Scripting" is compiled-MIPS state machines dispatched via function-pointer tables
indexed by game-state vars, proven by the matched resident dispatchers (D_800D3430[
D_800B99F0](), D_800D3480/3488/3490[state](), the func_800D2Fxx cluster) mirroring the EXE's
GameModeDispatch[gameMode](). Overlays = compiled MIPS that call the resident, not bytecode.
Basis for the gen2-roadmap criterion #2 amendment (T6).
- formats.md §5: SQV driver = semi-custom — thin Square wrappers/glue over stock PsyQ libsnd;
DsMix @0x800D1BD8 is a custom 2-line wrapper (NOT stock libsnd; the 4.7 sig was coincidental,
T1). Bytes matched in the harvest; deep playback semantics deferred (Gen3, not needed for the
byte-match).
T3a recovery landed REAL 102->123 (85.62% byte-identical). T3b reassessed under Max:
intra-resident dedup collapse has no recovery value left (all 6 h_exact + 9 h_norm group
members are individually matched byte-identical) -> deferred as optional source-DRY elegance
(the §11 share mechanism is proven; real leverage is the Phase-13 overlays). The ~21
remaining stubs are documented compiler-internal residuals (cross-jump/regalloc) -> honest
P9 deferrals (decomp-permuter is deliberate per-function work, not a blind sweep). T3 done
at the "substantial not exhaustive" bar. Both binaries byte-identical.
The pass-4 functions that match_one matched standalone but the single-TU build rejected
on conflicting shared-symbol extern types are recovered by UNIFYING those types in
resident.c (widening definitions' return types where byte-identical — u16 f(){return
u16g} ≡ s32 f(){...} both lhu;jr; choosing asm-correct widths for globals; per-site
*(u16*)&g tricks where a neighbor pins the type). ~20 symbols reconciled
(func_800291B4/CFD68/D1714/D1078/801465C0/... + D_800AE6A8/80078E50/...).
20 of 21 recovered; func_800D00E4 stays an honest INCLUDE_ASM stub (its switch needs the
external jtbl_80113ED8 relocation, not a C-source fix — analysis preserved under #if 0).
REAL 102 -> 123 (85.62% byte-identical); resident 8e17e02f, main 143dbb89 (R22 clean).
22 stubs remain: ~20 documented compiler-internal residuals (permuter/§3a) + D00E4 + dedup.
- matching-cookbook.md §12: the parallel-draft + deterministic byte-gate + match_one
iteration loop that took the resident 1.4%->71.7% in one session. The pipeline (Workflow
draft -> harvest_verify.py gate -> redraft passes -> match_one.py iterate), retry-wave
resilience, the two TU gotchas (inline-typedef strip; standalone-MATCH-but-build-FAIL =
extern-type conflict -> unify types), and the idioms the swarm surfaced (void-dispatcher-
is-s32, struct-copy prologue, *10->2x+8x, masked-andi-range, callee-return-type-forces-cast).
- effort-map.md: standing instruction to PROACTIVELY prompt Drew to enable /effort ultracode
on breadth-shaped stretches during a Max session (with the Phase-12 evidence), keeping the
deep single-thread tasks at Max. Memories: ultracode-harvest-pattern, effort-prompt-
ultracode-on-breadth.
Drew T