4415fd9910
feat(phase-25): close — structural-family endgame + the giant endgame + the h_seq family reframe (v1.24.0)
...
- MECHANICAL FAMILY ENGINE: family_manifest/family_remap/family_sweep + canon_sig_reconcile v3.2 +
rtu_match; ~20k+ member-matches banked for ~0 agent tokens (h_norm families are TEMPLATES, T3).
Fleet 66.02% -> ~74.6% fn-count (58.2% instr-weighted / 30.3% distinct-code, progress.py --weighted).
- GIANT ENDGAME COMPLETE: every matchable region-a giant ×134 incl. the 399-ins flagship func_80133CD4
(§45 Fable5 gdb-on-cc1 crack: merged-accumulator-variables + 1-death local-alloc in-out-asm + /s +
goto-ret; the ~22-phase 'unsteerable' verdict retired). 3 flywheel giants + F-band permuter batch
WALLED -> Fable5 (recon-done pin-free seeds + gdb specs).
- THE REFRAME (R14, byte-verified): the '36k unique / 87% distinct-code tail' is a strict-h_norm
artifact; re-clustered by h_seq it collapses 90% into ~754 families (986 substantial / 1.88M ins,
top-20=52%), templatability confirmed. Endgame = crack ~986 exemplars -> template ×120.
- cookbook §40-§45; docs/{phase25-t6-curriculum,phase25-frontier-map,family-endgame-megaplan}.md.
136/136 byte-identical (R22), 0 NON_MATCHING (G4), dedup 1813/0. No new governance rules (techniques
-> cookbook; the reframe -> the megaplan). worklog -> logs/Phase25.md (R19). bumps 1.23.0 -> 1.24.0.
2026-07-11 19:30:11 -06:00
9194d27760
feat(phase-24): permuter overhaul + integration-recovery + the giant endgame (all giants ×134); fleet 64.86->66.02% (v1.23.0)
...
- PERMUTER OVERHAUL (tools/, no submodule edit): masked_diff.py floor-free relocation-masked
scorer (killed the ~1750-2100 stock floor -> true closeness); permuter_weights.py §31-class-
directed mutation; permuter_ils.py warm-restart ILS; p16_permute pin/typedef/-O0/comment fixes.
- GIANT ENDGAME COMPLETE: every matched region-a giant banked ×134 — the whale func_80144B9C
(770, -O0 struct-assign memcpy §38), the flagship func_80132784 (400, "irreducible 22 phases"),
the §G batch (6/8 cracked), + this close's func_801770E0 (152) + func_801372B0 (207). 2 permanent
walls remain (func_801412A8, func_80178004 — G4). NO matched giant remains ×1.
- INTEGRATION/PROPAGATION TOOLING: recover_integration.py, reconcile_decls.py, dedup_propagate
--recover + find_site fixes + overlay_files _o0b/_after (post-whale gap), lint_symbol_refs.py.
Fixed the Phase-21 latent clean-rebuild breakage (62 stale INCLUDE_ASM refs) -> true clean 136/136.
- TOOLKIT MATURED: cookbook §18-§39 + gcc-2.7.2-map (S11/S12/S13, RC-6..RC-13, giant idioms).
- STRATEGIC REFRAME (R14): the frontier grouped by STRUCTURE (h_norm) reveals 2764 families /
11.1 MB hidden by byte-grouping; 46% of large "solo monsters" are family members -> the idiom-
curriculum endgame (Phase 25). Survey scripts .run/analyze_frontier.py + probe_hnorm.py.
- check-all 136/136 BYTE-IDENTICAL (R22), dedup 1783->1813/0, 0 NON_MATCHING (G4). No new rules
(techniques -> cookbook; the reframe -> Phase-25 backlog). worklog -> logs/Phase24.md (R19).
2026-07-08 00:14:40 -06:00
80b89d8a2d
feat(phase-23): LLM matching tier + the Fable5 wall-breaker + the gcc-2.7.2 codegen map (v1.22.0)
...
- LLM TIER (T1-T10.9): tools/serve_local + api_draft + lora_grind + bulk_harvest + the LoRA
pipeline (v3 model). Local v3 saturated <=15 (~1300 banks, $0). Frontier GLM-5.2 (~$4)
confirmed the def-side wall INTRINSIC + the idiom well DRY at the leaf (both angles).
- THE PIVOT (07-02): Fable5Max reading the gcc-2.7.2 SOURCE matched func_8014EE14, a §20/§10
store-vs-load giant "CONFIRMED unsteerable" for 22 phases -> the "wall is intrinsic" verdict
is MODEL-RELATIVE. Generalized to Opus agents applying §30 (2 more giants, 3-5x cheaper).
- §31 THE CODEGEN MAP: 4 Fable5 agents read the whole gcc-2.7.2 source -> docs/gcc-2.7.2-map/
{sched,regalloc,loop,cse_expr}.md (935 lines, byte-proven residual->lever catalogs) +
cookbook §31 (index+triage). Broke hoist-vs-remat/delay-slot/coalescing walls. Correction:
gcc-papermario is 2.8.1 not 2.7.2 -> tools/reference/gcc-2.7.2/ (SETUP §5.6).
- THE WAVE (8 reach-134 near-misses, Opus + §31): 3 leaf-MATCH by lookup (47-137k tok vs
200-250k walls) + 5 tight permuter seeds (func_80132784 240->4-off!), zero dead-ends.
- 5 giants/hard-fns banked x134 (func_8014EE14/F2E0/150528/149374/144090); fleet 63.66->64.86%,
136/136 byte-identical, 0 NON_MATCHING. cookbook §30/§30a/§31. No new governance rules.
- FINDING: matching is SOLVED by the map; bottlenecks are now the permuter (S11 seeds) +
whole-binary integration (auto-bank leaf-MATCHes) -> Phase 24. worklog -> logs/Phase23.md.
2026-07-02 21:51:25 -06:00
64cc91ee83
feat(phase-22): close — tractable giants banked ×134 (fleet 63.18->63.66%), irreducible tail byte-deferred; pivot to the LLM tier (v1.21.0)
...
- BANKED ×134 (~17 fns): func_8015126C (canonical-extern recovery) + func_80156B74/func_80163C2C
(struct type-lift -> engine_types.h) + the 14-fn close=0 recovery batch (recover_giant.py). 136/136
byte-identical, dedup-check 1633/0, 0 NON_MATCHING throughout.
- IRREDUCIBLE GIANTS byte-deferred w/ evidence: func_80132784 (hoist-vs-remat + reg-lifetime),
func_801372B0 (scheduler last-mile, structural draft banked), func_8014F74C (masked). gcc-2.7.2
intrinsic walls -> backlog, not effort gaps. Grinder ROI audit: 7 all-time banks (all Phase 21), 0 since.
- TOOLKIT: recover_giant.py + the engine_types.h type-lift lever; cookbook §28/§28a/§28b/§28c.
gate_stage commit tag made phase-agnostic (GATE_PHASE).
- PIVOT: Phase 23 = the offline LLM matching tier (built this session; see CURRENT_PHASE.md).
- worklog -> phase-ends/logs/Phase22.md (R19); bumps project version 1.20.0 -> 1.21.0
2026-06-30 00:26:49 -06:00
ddf9266ab0
feat(phase-21): close — automation manager built + run + ceiling byte-proven; fleet 58.82%->63.18% (v1.20.0)
...
- THE AUTOMATION MANAGER (built + validated + run): worker_wave (parallel xHigh drafters) + grinder
(token-free permuter daemon) + orchestrator (ROI pool rotation) + gate_stage (whole-binary byte-gate,
sole arbiter) + backlog ledger + idiom_loop/distill flywheel. Drove fleet 58.82% -> 63.18% across ~10
sessions, 136/136 byte-identical, 0 NON_MATCHING.
- THE CEILING, BYTE-PROVEN: cheap close=0 recovery exhausted; giant auto-waves 0/6 (x2 — giants don't
auto-bank, they're hand-finish/permuter fuel); the cached reach-134 automated harvest COMPLETE at 63.17%.
- COLLABORATION SWEEP (5 scouts, clean verdicts): we're SOTA (no neural/GPU/diffusion shortcut); 4
gcc-2.7.2-psx twins (VS=CC0, the clean pool partner); no Square symbols; dedup probe CLEAN NEGATIVE
(BFM shares only PsyQ with sisters, AKAO refuted, 0 engine code) -> NO external shortcut to the gcc wall.
+62 PsyQ names banked (byte-safe).
- TOOLKIT MATURED: cookbook §18 (-O0 split) / §19 (wave recovery + propagation cap) / §20 (cast tool +
DEF-side wall) / §25 (coalescing + gate two-stage + h_exact over-count) / §26 (recovery exhausted) /
§27 (giant recipe). Novel byproduct: xsig cross-project dedup tool (open-source TODO logged).
- DELIVERABLE for Phase 22: the ranked hand-finish backlog (docs/backlog.md) + the giants (×134).
- worklog -> phase-ends/logs/Phase21.md (R19); bumps project version 1.19.0 -> 1.20.0
2026-06-26 01:34:09 -06:00
e803651f0c
feat(phase-20): close — cap tool + T3a %lo crack + 3 byte-proven findings; fleet 58.00%->58.82% (v1.19.0)
...
- THE CAP TOOL tools/cast_call_sites.py (§17a-1 callee-cast): per draft, callee whose canonical
TU sig != draft's intended sig -> decl->canonical + call-site cast ((ret(*)(args))func_X)(args)
(codegen-neutral; gcc folds the cast of a known symbol -> direct jal). Recovered 6 of T6 batch-1's
33 gate-fails byte-identical (5 x134 + 1 local). Pipeline canon_resident_calls -> cast_call_sites
-> sig_unify -> harvest_verify --chunk 1.
- T3a CRACKED (not cited): the %lo-folding indexed-global -O0 residual falls to the array-of-STRUCT
idiom (extern Struct base[], sizeof==stride, base[i].field; NOT *(T*)(&sym+i*stride) which
materializes &sym). func_8013B7AC banked. Reusable idiom fleet-wide. cookbook §18/§20.
- THREE byte-proven findings (R14): (1) the wave cap is the DEF-side loose-typing wall, not
callee-conflicts (the §20 "~33" projection was wrong); (2) the -O0 cluster is OVERLAY-LOCAL,
not reach-134 -> the ×134 rollout is INVALID; (3) data-cast is MOOT (0 real data conflicts).
- T1 typedef type-lift (+0.35%, 9 fns x134); T2 exemplar_miner router; T4 codegen map (cookbook §20);
T6 batch-1 (+0.28%, 8). dedup 1485->1513/0 failed. check-all 136/136 BYTE-IDENTICAL (R22);
0 NON_MATCHING (G4). fleet 58.00% -> 58.82%.
- rule R30 (capture context-dependent knowledge before a fresh session). worklog -> logs/Phase20.md.
- bumps 1.18.0 -> 1.19.0
2026-06-21 10:26:34 -06:00
de35505a63
feat(phase-19): scale the toolkit — 2 waves (88%/92%), fleet 56.64%->58.00%, propagation is the cap (v1.18.0)
...
- T1 (prior session, commit:0158/commit:0159): per-file -O0 split infra + 6 -O0 matches; deferred the
10 %lo-folding-quirk fns + the ×134 rollout (R14 corrected "free ×134")
- T2 (commit:0161): recovery tooling. R14: the 3 Phase-18 "implicit-int" near-misses are NOT
implicit-int -> resident-callee link-miss (NEW canon_resident_calls.py) + shared-caller arity
(no-proto caller-extern fix). 3 matches; cookbook §17a-3 corrected
- T3 (commit:0162/commit:0163/commit:0164): 2 Ultracode waves over reach-134 tractable residuals. batch-1
44/50 match_one (30 banked, 25 propagated ×134); batch-2 35/38 (25 banked, 10 propagated).
NEW fix_arity_callers.py (no-proto, ALL 18 batch-2 failures were the arity class); canon-first
2-stage gate (sig_unify REGRESSES canonical drafts); gen_harvest_targets garbled-sig fix
(0/749, 88%->92%); build_engine_types --strip lift (+4 propagated)
- FINDING: propagation (not matching) caps the fleet-% yield — typedef/local-type bodies can't
auto-propagate (-> Phase-20 type-lift lever). decomp.dev-comparable byte-weighted est ~30%
- check-all 136/136 BYTE-IDENTICAL (R22); dedup 1450->1485/0 failed; 0 NON_MATCHING (G4);
fleet 56.64% -> 58.00% (+1.36%). worklog -> phase-ends/logs/Phase19.md (R19)
- bumps 1.17.0 -> 1.18.0
2026-06-20 20:33:00 -06:00
a741d2b6b5
feat(phase-18): compiler-quirk research -> the matching TOOLKIT, taught to waves (v1.17.0)
...
- RESEARCH (T0-T7): read gcc-2.7.2 source (gcc-papermario) + mined Xenogears (our exact
compiler) -> per-class byte-gated verdicts. VERDICT REVERSAL (Drew "hand-match everything"):
the call-crossing register-ORDER class is STEERABLE via `register __asm__` pins + a scheduling
barrier (func_8012B8E4, the flagship "unsteerable" fn, byte-gated) -- not unsteerable.
- TOOLKIT (cookbook §17): pins+barrier (register swap), array-decay (hoist-vs-remat),
for-loop/statement-order (structural), sig_unify-mandatory; narrow-param = honest stub.
§16 reconciled (§10-vs-§16).
- TAUGHT TO WAVES (§17a): calibration 9/16=56% whole-binary (vs 33%); Step-1 (embedded
canonical callee sigs + call-site-cast/re-validate) 28/31=90% match_one, 22 verified.
~31 reach-134 matches banked, each propagated x134. fleet 55.51 -> 56.64% (+1.13%).
- FINDING: the match_one->gate gap is declaration plumbing (call-site casts not redeclaration;
stale M2C_UNK sibling-decl reconcile; implicit-int caller class) -- recovered deterministically.
- NEW residual classes: per-file -O0 (~18 fns, needs an -O0 split file like src/boot.c -> high
ROI) + gcc loop-guard operand-order (irreducible). W1: build-divergence ruled out by proxy.
- make check-all 136/136 byte-identical (R22); dedup 1450 validated/0 failed; 0 NON_MATCHING (G4).
- worklog -> phase-ends/logs/Phase18.md (R19); bumps 1.16.0 -> 1.17.0
2026-06-20 15:05:52 -06:00
c4b0cef86a
feat(phase-17): close — canonical-sig layer built; the wall is the compiler, not sigs; pivot to gcc research (v1.16.0)
...
- canonical-sig layer (session 4): tools/census_conflict_callees.py + derive_canonical_sigs.py
-> a 20-extern byte-neutral block atop ov_SC01_077.c (LOCAL, not engine_core.h); census
conflict callees 20->0, blocked targets 24->0; gate pipeline now draft -> sig_unify (MANDATORY)
-> harvest_verify --chunk 1; fleet 136/136 byte-identical (R22), 55.51% (no regression)
- FINDING (R14/P9): the conflict wall is 7%-reach not ~2x; the 4 reach-134 circular targets are
ALL gcc-quirk/regalloc/layout-bound (0 banked); the high-reach core IS the quirk tail; struct
types are byte-neutral for matching (the wall is gcc codegen, not knowledge)
- leverage analysis: fleet % is function-count-weighted (size adds no %); "unblock many" = the
layer (declaration, not matching); reach is the lever (already reach-sorted); 247 tractable
reach-134 stubs ~ +3-4% projected
- GO/NO-GO: NO-GO on brute waves at the current ceiling; GO on a compiler-quirk research phase
(read gcc-2.7.2 source + Xenogears + the §10/regalloc classes, R17) -> then resume the wave
- docs: cookbook §16 corrected + hand-matching-process.md §8 (the layer + the finding + handoff)
- worklog archived -> phase-ends/logs/Phase17.md (R19); bumps 1.15.0 -> 1.16.0
2026-06-20 09:35:18 -06:00
f3f15649f1
docs(phase-16): PhaseEnd — pivot from m2c+permuter to type-recovery (v1.15.0)
...
- PhaseEnd_Phase16.md: the loose-typing wall (byte-proven), the kept harness fixes, the unattended
driver (built/not-deployed), and the new research directions (emulator -> Ghidra type propagation).
- CURRENT_PHASE.md archived -> logs/Phase16.md (R19).
- Negative-but-decisive result (Phase-3.5 precedent); bumps 1.14.0 -> 1.15.0.
2026-06-19 10:12:22 -06:00
964cec8b61
docs(phase-15): close — PhaseEnd v1.14.0 (fleet 54.48%), continuation backlog -> Phase 16
...
- PhaseEnd_Phase15.md: synthesis of the open-ended overlay long-tail phase (136 binaries
byte-identical from clean; fleet 3.82% -> 54.48%); CURRENT_PHASE.md archived -> logs/Phase15.md (R19)
- the final-session finding: deterministic recovery (sig_unify + find_site fix) beat a 50-agent
wave ~7x/token (+2.67% vs +0.36%); cookbook §14d (the recovery) + §14e (() / m2c-struct dead-ends)
- Continuation Backlog -> Phase 16 'Crack the struct-heavy shared core': the actor-struct ->
m2c-context -> permuter compute pipeline (+ multi-day autonomous run), the 28 giants, decomp.me,
emulator capture, ML/CUDA wildcard, the arity-conflict dead-end
- SETUP tool inventory (sig_unify + recovery tools); gen2-roadmap Phase-15 CLOSED + Phase-16 added
- bumps project version 1.13.0 -> 1.14.0
2026-06-18 21:01:38 -06:00
a0865e78ca
docs(phase-13): PhaseEnd — representative overlays complete; 6 binaries byte-identical (v1.13.0)
...
Milestone confirmed (gate 2): make check-all 6/6 from clean (R22); ov_SC01_077
704 matched, dup-pair collapse, free-roam, new_overlay.sh + build-all/check-all.
Rules R27 (effort-transition discipline), R28 (task-list-after-plan), R29 (MCP
reconnect). CURRENT_PHASE.md archived -> phase-ends/logs/Phase13.md (R19).
2026-06-16 18:00:25 -06:00
b7b34f007e
docs(phase-12): PhaseEnd — key engine systems complete; resident 1.4%->85.6% (v1.12.0)
...
Phase 12 close (milestone confirmed by Drew, gate 2):
- resident engine REAL 1 -> 123/146 = 85.62% byte-identical, 0 NON_MATCHING; both binaries
byte-identical from a clean rebuild (resident 8e17e02f, main 143dbb89)
- pivots (byte-driven, Drew-approved): T1 the resident has NO PsyQ library footprint (all
custom engine code, REAL not LINKED); T4 NO bytecode VM (script/event = compiled-MIPS
fn-ptr-table dispatch, byte-backed); T5 save format documented, deep data + SaveLoadRoutine
code-match honestly deferred (Gen3/P9)
- method: Ultracode harvest = swarm parallel-draft + incorruptible byte-gate (5 passes) +
match_one iteration loop + extern-conflict recovery; tools harvest_verify.py + match_one.py
(-> Phase 13). cookbook §12, R26 (prompt for ultracode on breadth)
- gen2-roadmap milestone + EXIT criterion #2 AMENDED (script/event resolved; save documented)
- PhaseEnd_Phase12.md; CURRENT_PHASE.md archived -> logs/Phase12.md (R19); README status
- rule R26; bumps project version 1.11.0 -> 1.12.0
2026-06-16 14:42:52 -06:00
0cd9656f25
docs(phase-11): close-out — PhaseEnd_Phase11 + worklog archive (v1.11.0)
...
- PhaseEnd_Phase11.md: cross-binary dedup pipeline complete; milestone proven (clean
dual-binary rebuild main 143dbb89 / resident 8e17e02f; cross-report 9366 h_exact groups
across 136 binaries / 28.5 MB collapsible; one shared body byte-gated in both states)
- deviations D1 (source-level share, not object-swap) + D2 (self-consistent h_norm) recorded;
no new rules (captured as cookbook §11 techniques)
- CURRENT_PHASE.md archived -> phase-ends/logs/Phase11.md (R19)
- bumps project version 1.10.0 -> 1.11.0
- Next: Phase 12 — key engine systems, opening with the resident's PsyQ 4.7 linking (R24)
2026-06-16 09:45:49 -06:00
702b94fd7d
docs(phase-10): close-out — PhaseEnd_Phase10 + flat-blob recipe (v1.10.0)
...
- PhaseEnd_Phase10.md: resident engine blob byte-identical from source (8e17e02f) at 100%
INCLUDE_ASM; the binary-agnostic toolchain proven on a real 2nd binary; main still 143dbb89
- docs/SETUP.md §6.7: the resident first-instantiation + the reusable flat-blob <bin> recipe
(per-binary OBJS prune, build_path=build, flat config, leading-data-word-as-rodata); 2 new
Ghidra tools in the inventory (R21)
- rule R24 (per-binary compiler/SDK provenance — resident is PsyQ 4.7 vs EXE 4.0)
- worklog archived CURRENT_PHASE.md -> phase-ends/logs/Phase10.md (R19)
- bumps project version 1.9.0 -> 1.10.0
2026-06-15 23:35:11 -06:00
d380b13fc5
feat(phase-9): close-out — PhaseEnd_Phase9 + binary-agnostic toolchain (v1.9.0)
...
- PhaseEnd_Phase9.md written; CURRENT_PHASE.md archived -> phase-ends/logs/Phase9.md (R19)
- MILESTONE (gate-2 confirmed): the toolchain is binary-agnostic — EXE rebuilds
143dbb89 through the parameterized path WITH and WITHOUT SDK objects; make report
52/959/7/50.24%; every binary-specific value a required param (no EXE default);
wrong --vram-base -> cae22f7e (negative control). Landed as 11 per-tool checkpoints.
- R23: stop the Ghidra MCP + commit the DB at phase-end/RE-checkpoint (lock won't
release until MCP closes; SessionEnd is too late for a mid-session commit). This
commit reconciles the Ghidra program DB (db.12 -> db.15, 99% identical: Phase 9 did
zero RE writes) after a clean ghidra_mcp_stop.sh save.
- .gitignore: ignore /ghidra/**/*.lock~ (the lock-backup that churned every session)
- bumps project version 1.8.0 -> 1.9.0
2026-06-15 22:01:17 -06:00
1f5cc43981
feat(phase-8): close-out — PhaseEnd_Phase8 + worklog archived (v1.8.0)
...
- PhaseEnd_Phase8.md: 8 PsyQ libraries linked byte-identical, byte-identical coverage
20.31% -> 50.24% (LINKED 340 -> 959), 9 game-code matches (REAL 43 -> 52), milestone
confirmed by Drew (gate 2)
- rule R22 (verify byte-matches from a clean rebuild, never incremental)
- CURRENT_PHASE.md -> phase-ends/logs/Phase8.md (R19)
- bumps project version 1.7.0 -> 1.8.0
2026-06-15 19:22:36 -06:00
49db81d8ae
feat(phase-7): Gen1 close-out — matching loop industrialized, PsyQ libs linked, LZSS matched (v1.7.0)
...
- reports (Task 3): tools/{progress,difficulty,dup_report}.py + make report/sig-refresh
(deterministic digests); harvest to 43 REAL matches incl. LzssDecodeSector
- per-module -O0 split (Task 5.5): src/boot.c + Makefile per-file CC1FLAGS (cookbook §6)
- PsyQ SDK linking (sessions C-G): split/build/identify/link/integrate pipeline
(tools/psyq_*.py, make_libgs.sh, ld_interleave.py, split_src_region.py) — libcd (18 obj)
+ FULL libgs (31 obj / 6 blocks) linked BYTE-IDENTICAL from real PsyQ 4.0 libs; byte-
identical with or without the SDK objects (stub fallback); cookbook §7/§8/§9.1-9.5
- LZSS MATCHED byte-for-byte (session F): §5a cross-jump barrier + §10 regalloc/schedule
idioms, ground-truthed against pinned gcc-2.7.2 source (the Gen1 LZSS gate)
- file-loader cluster: 6 matched + 5 NON_MATCHING-drafted (CdReadStateMachine,
CdReadSectorReadyCB, StreamLoadStateMachine new this session, valid under -DNON_MATCHING);
SaveLoadRoutine DEFERRED to Q#5 (save/memcard, documented stub)
- README Gen1-complete; worklog preserved -> phase-ends/logs/Phase7.md
- rules R17 (web-research compiler quirks), R18 (plain-English session summaries),
R19 (preserve phase worklog out of load order)
- MILESTONE: Gen1 EXIT confirmed — byte-identical build, 43 matches incl LZSS, 7 zero-
regression sessions, loader cluster matched/drafted, overlay map emulator-proven
- bumps project version 1.6.0 -> 1.7.0
2026-06-15 13:44:32 -06:00
Drew T