mirror of
https://github.com/Druthulu/BFM-decomp
synced 2026-09-30 15:36:55 -04:00
feat(phase-23): LLM matching tier + the Fable5 wall-breaker + the gcc-2.7.2 codegen map (v1.22.0)
- LLM TIER (T1-T10.9): tools/serve_local + api_draft + lora_grind + bulk_harvest + the LoRA
pipeline (v3 model). Local v3 saturated <=15 (~1300 banks, $0). Frontier GLM-5.2 (~$4)
confirmed the def-side wall INTRINSIC + the idiom well DRY at the leaf (both angles).
- THE PIVOT (07-02): Fable5Max reading the gcc-2.7.2 SOURCE matched func_8014EE14, a §20/§10
store-vs-load giant "CONFIRMED unsteerable" for 22 phases -> the "wall is intrinsic" verdict
is MODEL-RELATIVE. Generalized to Opus agents applying §30 (2 more giants, 3-5x cheaper).
- §31 THE CODEGEN MAP: 4 Fable5 agents read the whole gcc-2.7.2 source -> docs/gcc-2.7.2-map/
{sched,regalloc,loop,cse_expr}.md (935 lines, byte-proven residual->lever catalogs) +
cookbook §31 (index+triage). Broke hoist-vs-remat/delay-slot/coalescing walls. Correction:
gcc-papermario is 2.8.1 not 2.7.2 -> tools/reference/gcc-2.7.2/ (SETUP §5.6).
- THE WAVE (8 reach-134 near-misses, Opus + §31): 3 leaf-MATCH by lookup (47-137k tok vs
200-250k walls) + 5 tight permuter seeds (func_80132784 240->4-off!), zero dead-ends.
- 5 giants/hard-fns banked x134 (func_8014EE14/F2E0/150528/149374/144090); fleet 63.66->64.86%,
136/136 byte-identical, 0 NON_MATCHING. cookbook §30/§30a/§31. No new governance rules.
- FINDING: matching is SOLVED by the map; bottlenecks are now the permuter (S11 seeds) +
whole-binary integration (auto-bank leaf-MATCHes) -> Phase 24. worklog -> logs/Phase23.md.
This commit is contained in:
Drew T
@@ -113,3 +113,4 @@ models/
|
||||
.venv-train/
|
||||
unsloth_compiled_cache/
|
||||
.venv-serve/
|
||||
.env
|
||||
|
||||
+2904
-2908
File diff suppressed because it is too large
Load Diff
+138
-138
@@ -3,148 +3,148 @@
|
||||
# source-derived (committed src/*.c + config/dedup.us.yaml). Live byte gate: `make check-all`;
|
||||
# cross-binary collapsible-byte leverage: docs/duplicates.cross.md.
|
||||
|
||||
FLEET REAL substantive : 221659 (of which dedup-shared 219118 via 1779 groups / 219169 instances)
|
||||
FLEET REAL substantive : 222196 (of which dedup-shared 219654 via 1783 groups / 219705 instances)
|
||||
FLEET LINKED PsyQ objs : 959
|
||||
FLEET byte-identical : 223188 / 344807 = 64.73% (REAL+LINKED+empties)
|
||||
FLEET byte-identical : 223725 / 344941 = 64.86% (REAL+LINKED+empties)
|
||||
FLEET NON_MATCHING : 7 (0 in any default build — G4)
|
||||
FLEET INCLUDE_ASM stubs : 121612
|
||||
FLEET matchable : 344807
|
||||
FLEET INCLUDE_ASM stubs : 121209
|
||||
FLEET matchable : 344941
|
||||
|
||||
| binary | REAL | shared | LINKED | byte-ident | matchable | byte-ident % |
|
||||
|---|---:|---:|---:|---:|---:|---:|
|
||||
| main | 54 | 2 | 959 | 1055 | 2096 | 50.3% |
|
||||
| resident | 123 | 0 | 0 | 125 | 146 | 85.6% |
|
||||
| ov_SC01_000 | 1652 | 1638 | 0 | 1652 | 2409 | 68.6% |
|
||||
| ov_SC01_001 | 1648 | 1638 | 0 | 1650 | 2473 | 66.7% |
|
||||
| ov_SC01_004 | 1641 | 1629 | 0 | 1642 | 2422 | 67.8% |
|
||||
| ov_SC01_005 | 1664 | 1651 | 0 | 1664 | 2510 | 66.3% |
|
||||
| ov_SC01_006 | 1663 | 1651 | 0 | 1663 | 2510 | 66.3% |
|
||||
| ov_SC01_008 | 1642 | 1629 | 0 | 1644 | 2434 | 67.5% |
|
||||
| ov_SC01_009 | 1663 | 1630 | 0 | 1664 | 2516 | 66.1% |
|
||||
| ov_SC01_074 | 1642 | 1630 | 0 | 1644 | 2433 | 67.6% |
|
||||
| ov_SC01_077 | 2157 | 1590 | 0 | 2159 | 2585 | 83.5% |
|
||||
| ov_SC01_080 | 1660 | 1633 | 0 | 1660 | 2521 | 65.8% |
|
||||
| ov_SC01_084 | 1665 | 1633 | 0 | 1670 | 2587 | 64.6% |
|
||||
| ov_SC02_000 | 1679 | 1668 | 0 | 1679 | 2690 | 62.4% |
|
||||
| ov_SC02_003 | 1679 | 1668 | 0 | 1679 | 2690 | 62.4% |
|
||||
| ov_SC02_004 | 1646 | 1633 | 0 | 1646 | 2409 | 68.3% |
|
||||
| ov_SC02_005 | 1705 | 1629 | 0 | 1715 | 2936 | 58.4% |
|
||||
| ov_SC02_011 | 1686 | 1636 | 0 | 1697 | 2902 | 58.5% |
|
||||
| ov_SC02_015 | 1646 | 1635 | 0 | 1646 | 2422 | 68.0% |
|
||||
| ov_SC02_016 | 1650 | 1635 | 0 | 1653 | 2554 | 64.7% |
|
||||
| ov_SC02_017 | 1670 | 1635 | 0 | 1678 | 2741 | 61.2% |
|
||||
| ov_SC02_021 | 1649 | 1635 | 0 | 1649 | 2445 | 67.4% |
|
||||
| ov_SC02_026 | 1654 | 1633 | 0 | 1660 | 2581 | 64.3% |
|
||||
| ov_SC02_027 | 1663 | 1633 | 0 | 1672 | 2701 | 61.9% |
|
||||
| ov_SC02_028 | 1671 | 1633 | 0 | 1681 | 2710 | 62.0% |
|
||||
| ov_SC02_031 | 1661 | 1634 | 0 | 1666 | 2571 | 64.8% |
|
||||
| ov_SC02_035 | 1652 | 1633 | 0 | 1655 | 2529 | 65.4% |
|
||||
| ov_SC02_039 | 1637 | 1633 | 0 | 1637 | 2424 | 67.5% |
|
||||
| ov_SC02_041 | 1644 | 1633 | 0 | 1647 | 2569 | 64.1% |
|
||||
| ov_SC03_001 | 1702 | 1634 | 0 | 1719 | 2876 | 59.8% |
|
||||
| ov_SC03_002 | 1663 | 1638 | 0 | 1678 | 2638 | 63.6% |
|
||||
| ov_SC03_003 | 1635 | 1633 | 0 | 1636 | 2431 | 67.3% |
|
||||
| ov_SC03_006 | 1676 | 1639 | 0 | 1685 | 2774 | 60.7% |
|
||||
| ov_SC03_007 | 1651 | 1633 | 0 | 1655 | 2632 | 62.9% |
|
||||
| ov_SC03_010 | 1645 | 1633 | 0 | 1645 | 2478 | 66.4% |
|
||||
| ov_SC03_011 | 1641 | 1633 | 0 | 1647 | 2536 | 64.9% |
|
||||
| ov_SC03_012 | 1636 | 1633 | 0 | 1637 | 2414 | 67.8% |
|
||||
| ov_SC03_013 | 1644 | 1633 | 0 | 1644 | 2501 | 65.7% |
|
||||
| ov_SC03_014 | 1667 | 1658 | 0 | 1667 | 2694 | 61.9% |
|
||||
| ov_SC03_015 | 1673 | 1658 | 0 | 1673 | 2694 | 62.1% |
|
||||
| ov_SC03_023 | 1638 | 1633 | 0 | 1639 | 2443 | 67.1% |
|
||||
| ov_SC03_024 | 1668 | 1638 | 0 | 1675 | 2650 | 63.2% |
|
||||
| ov_SC03_028 | 1659 | 1633 | 0 | 1663 | 2673 | 62.2% |
|
||||
| ov_SC03_029 | 1645 | 1635 | 0 | 1655 | 2653 | 62.4% |
|
||||
| ov_SC03_030 | 1644 | 1637 | 0 | 1646 | 2504 | 65.7% |
|
||||
| ov_SC03_031 | 1640 | 1633 | 0 | 1644 | 2523 | 65.2% |
|
||||
| ov_SC03_089 | 1650 | 1633 | 0 | 1657 | 2590 | 64.0% |
|
||||
| ov_SC03_090 | 1651 | 1633 | 0 | 1659 | 2634 | 63.0% |
|
||||
| ov_SC03_091 | 1650 | 1633 | 0 | 1658 | 2649 | 62.6% |
|
||||
| ov_SC03_092 | 1650 | 1633 | 0 | 1662 | 2596 | 64.0% |
|
||||
| ov_SC03_093 | 1647 | 1633 | 0 | 1651 | 2572 | 64.2% |
|
||||
| ov_SC03_094 | 1641 | 1633 | 0 | 1647 | 2585 | 63.7% |
|
||||
| ov_SC03_095 | 1639 | 1633 | 0 | 1642 | 2483 | 66.1% |
|
||||
| ov_SC03_096 | 1642 | 1633 | 0 | 1645 | 2474 | 66.5% |
|
||||
| ov_SC03_097 | 1646 | 1633 | 0 | 1653 | 2612 | 63.3% |
|
||||
| ov_SC03_098 | 1642 | 1633 | 0 | 1645 | 2550 | 64.5% |
|
||||
| ov_SC03_099 | 1639 | 1633 | 0 | 1642 | 2513 | 65.3% |
|
||||
| ov_SC03_100 | 1641 | 1633 | 0 | 1645 | 2548 | 64.6% |
|
||||
| ov_SC03_101 | 1643 | 1633 | 0 | 1647 | 2537 | 64.9% |
|
||||
| ov_SC03_102 | 1643 | 1633 | 0 | 1646 | 2501 | 65.8% |
|
||||
| ov_SC03_103 | 1641 | 1635 | 0 | 1644 | 2519 | 65.3% |
|
||||
| ov_SC03_104 | 1653 | 1633 | 0 | 1660 | 2625 | 63.2% |
|
||||
| ov_SC03_105 | 1647 | 1633 | 0 | 1654 | 2606 | 63.5% |
|
||||
| ov_SC03_108 | 1635 | 1633 | 0 | 1635 | 2452 | 66.7% |
|
||||
| ov_SC03_109 | 1636 | 1633 | 0 | 1638 | 2432 | 67.4% |
|
||||
| ov_SC03_110 | 1638 | 1633 | 0 | 1638 | 2477 | 66.1% |
|
||||
| ov_SC03_111 | 1643 | 1633 | 0 | 1646 | 2519 | 65.3% |
|
||||
| ov_SC03_112 | 1640 | 1635 | 0 | 1642 | 2535 | 64.8% |
|
||||
| ov_SC03_113 | 1637 | 1635 | 0 | 1640 | 2477 | 66.2% |
|
||||
| ov_SC03_114 | 1634 | 1633 | 0 | 1636 | 2423 | 67.5% |
|
||||
| ov_SC03_115 | 1642 | 1633 | 0 | 1644 | 2481 | 66.3% |
|
||||
| ov_SC03_116 | 1640 | 1633 | 0 | 1643 | 2447 | 67.1% |
|
||||
| ov_SC03_117 | 1641 | 1633 | 0 | 1647 | 2566 | 64.2% |
|
||||
| ov_SC03_118 | 1668 | 1657 | 0 | 1669 | 2694 | 62.0% |
|
||||
| ov_SC03_119 | 1664 | 1657 | 0 | 1665 | 2694 | 61.8% |
|
||||
| ov_SC03_121 | 1640 | 1633 | 0 | 1643 | 2468 | 66.6% |
|
||||
| ov_SC03_124 | 1665 | 1629 | 0 | 1685 | 2750 | 61.3% |
|
||||
| ov_SC03_125 | 1646 | 1633 | 0 | 1658 | 2597 | 63.8% |
|
||||
| ov_SC03_126 | 1637 | 1635 | 0 | 1637 | 2431 | 67.3% |
|
||||
| ov_SC04_000 | 1647 | 1637 | 0 | 1656 | 2553 | 64.9% |
|
||||
| ov_SC04_002 | 1650 | 1633 | 0 | 1654 | 2645 | 62.5% |
|
||||
| ov_SC04_003 | 1638 | 1633 | 0 | 1642 | 2511 | 65.4% |
|
||||
| ov_SC04_004 | 1645 | 1633 | 0 | 1647 | 2567 | 64.2% |
|
||||
| ov_SC04_005 | 1647 | 1633 | 0 | 1653 | 2621 | 63.1% |
|
||||
| ov_SC04_006 | 1636 | 1633 | 0 | 1638 | 2462 | 66.5% |
|
||||
| ov_SC04_007 | 1646 | 1633 | 0 | 1650 | 2578 | 64.0% |
|
||||
| ov_SC04_008 | 1636 | 1633 | 0 | 1636 | 2423 | 67.5% |
|
||||
| ov_SC04_009 | 1639 | 1633 | 0 | 1642 | 2449 | 67.0% |
|
||||
| ov_SC04_010 | 1639 | 1633 | 0 | 1640 | 2427 | 67.6% |
|
||||
| ov_SC04_011 | 1662 | 1633 | 0 | 1668 | 2812 | 59.3% |
|
||||
| ov_SC04_012 | 1636 | 1633 | 0 | 1637 | 2428 | 67.4% |
|
||||
| ov_SC04_015 | 1656 | 1634 | 0 | 1667 | 2620 | 63.6% |
|
||||
| ov_SC04_016 | 1634 | 1633 | 0 | 1636 | 2448 | 66.8% |
|
||||
| ov_SC04_018 | 1683 | 1668 | 0 | 1683 | 2864 | 58.8% |
|
||||
| ov_SC04_019 | 1687 | 1668 | 0 | 1687 | 2864 | 58.9% |
|
||||
| ov_SC04_020 | 1649 | 1633 | 0 | 1661 | 2576 | 64.5% |
|
||||
| ov_SC04_021 | 1635 | 1635 | 0 | 1635 | 2431 | 67.3% |
|
||||
| ov_SC05_000 | 1639 | 1637 | 0 | 1642 | 2428 | 67.6% |
|
||||
| ov_SC05_001 | 1639 | 1633 | 0 | 1644 | 2583 | 63.6% |
|
||||
| ov_SC05_002 | 1636 | 1633 | 0 | 1639 | 2451 | 66.9% |
|
||||
| ov_SC05_003 | 1637 | 1633 | 0 | 1638 | 2490 | 65.8% |
|
||||
| ov_SC05_004 | 1637 | 1633 | 0 | 1639 | 2473 | 66.3% |
|
||||
| ov_SC05_005 | 1638 | 1633 | 0 | 1639 | 2500 | 65.6% |
|
||||
| ov_SC05_006 | 1635 | 1633 | 0 | 1635 | 2438 | 67.1% |
|
||||
| ov_SC05_007 | 1640 | 1633 | 0 | 1645 | 2491 | 66.0% |
|
||||
| ov_SC05_008 | 1660 | 1633 | 0 | 1662 | 2552 | 65.1% |
|
||||
| ov_SC05_009 | 1640 | 1633 | 0 | 1644 | 2446 | 67.2% |
|
||||
| ov_SC05_010 | 1648 | 1633 | 0 | 1652 | 2597 | 63.6% |
|
||||
| ov_SC05_011 | 1635 | 1633 | 0 | 1636 | 2417 | 67.7% |
|
||||
| ov_SC05_017 | 1667 | 1630 | 0 | 1678 | 2851 | 58.9% |
|
||||
| ov_SC05_018 | 1658 | 1633 | 0 | 1671 | 2682 | 62.3% |
|
||||
| ov_SC05_019 | 1636 | 1635 | 0 | 1636 | 2431 | 67.3% |
|
||||
| ov_SC06_000 | 1670 | 1642 | 0 | 1673 | 2698 | 62.0% |
|
||||
| ov_SC06_006 | 1643 | 1633 | 0 | 1644 | 2520 | 65.2% |
|
||||
| ov_SC06_008 | 1647 | 1635 | 0 | 1653 | 2550 | 64.8% |
|
||||
| ov_SC06_010 | 1647 | 1633 | 0 | 1652 | 2526 | 65.4% |
|
||||
| ov_SC06_011 | 1639 | 1633 | 0 | 1643 | 2477 | 66.3% |
|
||||
| ov_SC06_013 | 1638 | 1633 | 0 | 1639 | 2433 | 67.4% |
|
||||
| ov_SC06_014 | 1640 | 1633 | 0 | 1642 | 2461 | 66.7% |
|
||||
| ov_SC06_015 | 1637 | 1633 | 0 | 1637 | 2429 | 67.4% |
|
||||
| ov_SC06_016 | 1648 | 1633 | 0 | 1650 | 2558 | 64.5% |
|
||||
| ov_SC06_018 | 1647 | 1636 | 0 | 1654 | 2674 | 61.9% |
|
||||
| ov_SC06_020 | 1641 | 1635 | 0 | 1642 | 2527 | 65.0% |
|
||||
| ov_SC06_022 | 1647 | 1633 | 0 | 1655 | 2651 | 62.4% |
|
||||
| ov_SC06_024 | 1652 | 1633 | 0 | 1658 | 2676 | 62.0% |
|
||||
| ov_SC06_025 | 1645 | 1633 | 0 | 1650 | 2581 | 63.9% |
|
||||
| ov_SC06_027 | 1633 | 1633 | 0 | 1634 | 2416 | 67.6% |
|
||||
| ov_SC06_029 | 1662 | 1633 | 0 | 1674 | 2671 | 62.7% |
|
||||
| ov_SC06_030 | 1640 | 1633 | 0 | 1640 | 2465 | 66.5% |
|
||||
| ov_SC06_032 | 1640 | 1634 | 0 | 1647 | 2667 | 61.8% |
|
||||
| ov_SC06_033 | 1648 | 1634 | 0 | 1655 | 2640 | 62.7% |
|
||||
| ov_SC07_000 | 1651 | 1637 | 0 | 1653 | 2528 | 65.4% |
|
||||
| ov_SC07_001 | 1640 | 1633 | 0 | 1642 | 2463 | 66.7% |
|
||||
| ov_SC07_002 | 1654 | 1633 | 0 | 1658 | 2588 | 64.1% |
|
||||
| ov_SC07_008 | 1634 | 1633 | 0 | 1634 | 2394 | 68.3% |
|
||||
| ov_SC07_009 | 1642 | 1633 | 0 | 1644 | 2438 | 67.4% |
|
||||
| ov_SC01_000 | 1656 | 1642 | 0 | 1656 | 2410 | 68.7% |
|
||||
| ov_SC01_001 | 1652 | 1642 | 0 | 1654 | 2474 | 66.9% |
|
||||
| ov_SC01_004 | 1645 | 1633 | 0 | 1646 | 2423 | 67.9% |
|
||||
| ov_SC01_005 | 1668 | 1655 | 0 | 1668 | 2511 | 66.4% |
|
||||
| ov_SC01_006 | 1667 | 1655 | 0 | 1667 | 2511 | 66.4% |
|
||||
| ov_SC01_008 | 1646 | 1633 | 0 | 1648 | 2435 | 67.7% |
|
||||
| ov_SC01_009 | 1667 | 1634 | 0 | 1668 | 2517 | 66.3% |
|
||||
| ov_SC01_074 | 1646 | 1634 | 0 | 1648 | 2434 | 67.7% |
|
||||
| ov_SC01_077 | 2162 | 1594 | 0 | 2164 | 2586 | 83.7% |
|
||||
| ov_SC01_080 | 1664 | 1637 | 0 | 1664 | 2522 | 66.0% |
|
||||
| ov_SC01_084 | 1669 | 1637 | 0 | 1674 | 2588 | 64.7% |
|
||||
| ov_SC02_000 | 1683 | 1672 | 0 | 1683 | 2691 | 62.5% |
|
||||
| ov_SC02_003 | 1683 | 1672 | 0 | 1683 | 2691 | 62.5% |
|
||||
| ov_SC02_004 | 1650 | 1637 | 0 | 1650 | 2410 | 68.5% |
|
||||
| ov_SC02_005 | 1709 | 1633 | 0 | 1719 | 2937 | 58.5% |
|
||||
| ov_SC02_011 | 1690 | 1640 | 0 | 1701 | 2903 | 58.6% |
|
||||
| ov_SC02_015 | 1650 | 1639 | 0 | 1650 | 2423 | 68.1% |
|
||||
| ov_SC02_016 | 1654 | 1639 | 0 | 1657 | 2555 | 64.9% |
|
||||
| ov_SC02_017 | 1674 | 1639 | 0 | 1682 | 2742 | 61.3% |
|
||||
| ov_SC02_021 | 1653 | 1639 | 0 | 1653 | 2446 | 67.6% |
|
||||
| ov_SC02_026 | 1658 | 1637 | 0 | 1664 | 2582 | 64.4% |
|
||||
| ov_SC02_027 | 1667 | 1637 | 0 | 1676 | 2702 | 62.0% |
|
||||
| ov_SC02_028 | 1675 | 1637 | 0 | 1685 | 2711 | 62.2% |
|
||||
| ov_SC02_031 | 1665 | 1638 | 0 | 1670 | 2572 | 64.9% |
|
||||
| ov_SC02_035 | 1656 | 1637 | 0 | 1659 | 2530 | 65.6% |
|
||||
| ov_SC02_039 | 1641 | 1637 | 0 | 1641 | 2425 | 67.7% |
|
||||
| ov_SC02_041 | 1648 | 1637 | 0 | 1651 | 2570 | 64.2% |
|
||||
| ov_SC03_001 | 1706 | 1638 | 0 | 1723 | 2877 | 59.9% |
|
||||
| ov_SC03_002 | 1667 | 1642 | 0 | 1682 | 2639 | 63.7% |
|
||||
| ov_SC03_003 | 1639 | 1637 | 0 | 1640 | 2432 | 67.4% |
|
||||
| ov_SC03_006 | 1680 | 1643 | 0 | 1689 | 2775 | 60.9% |
|
||||
| ov_SC03_007 | 1655 | 1637 | 0 | 1659 | 2633 | 63.0% |
|
||||
| ov_SC03_010 | 1649 | 1637 | 0 | 1649 | 2479 | 66.5% |
|
||||
| ov_SC03_011 | 1645 | 1637 | 0 | 1651 | 2537 | 65.1% |
|
||||
| ov_SC03_012 | 1640 | 1637 | 0 | 1641 | 2415 | 68.0% |
|
||||
| ov_SC03_013 | 1648 | 1637 | 0 | 1648 | 2502 | 65.9% |
|
||||
| ov_SC03_014 | 1671 | 1662 | 0 | 1671 | 2695 | 62.0% |
|
||||
| ov_SC03_015 | 1677 | 1662 | 0 | 1677 | 2695 | 62.2% |
|
||||
| ov_SC03_023 | 1642 | 1637 | 0 | 1643 | 2444 | 67.2% |
|
||||
| ov_SC03_024 | 1672 | 1642 | 0 | 1679 | 2651 | 63.3% |
|
||||
| ov_SC03_028 | 1663 | 1637 | 0 | 1667 | 2674 | 62.3% |
|
||||
| ov_SC03_029 | 1649 | 1639 | 0 | 1659 | 2654 | 62.5% |
|
||||
| ov_SC03_030 | 1648 | 1641 | 0 | 1650 | 2505 | 65.9% |
|
||||
| ov_SC03_031 | 1644 | 1637 | 0 | 1648 | 2524 | 65.3% |
|
||||
| ov_SC03_089 | 1654 | 1637 | 0 | 1661 | 2591 | 64.1% |
|
||||
| ov_SC03_090 | 1655 | 1637 | 0 | 1663 | 2635 | 63.1% |
|
||||
| ov_SC03_091 | 1654 | 1637 | 0 | 1662 | 2650 | 62.7% |
|
||||
| ov_SC03_092 | 1654 | 1637 | 0 | 1666 | 2597 | 64.2% |
|
||||
| ov_SC03_093 | 1651 | 1637 | 0 | 1655 | 2573 | 64.3% |
|
||||
| ov_SC03_094 | 1645 | 1637 | 0 | 1651 | 2586 | 63.8% |
|
||||
| ov_SC03_095 | 1643 | 1637 | 0 | 1646 | 2484 | 66.3% |
|
||||
| ov_SC03_096 | 1646 | 1637 | 0 | 1649 | 2475 | 66.6% |
|
||||
| ov_SC03_097 | 1650 | 1637 | 0 | 1657 | 2613 | 63.4% |
|
||||
| ov_SC03_098 | 1646 | 1637 | 0 | 1649 | 2551 | 64.6% |
|
||||
| ov_SC03_099 | 1643 | 1637 | 0 | 1646 | 2514 | 65.5% |
|
||||
| ov_SC03_100 | 1645 | 1637 | 0 | 1649 | 2549 | 64.7% |
|
||||
| ov_SC03_101 | 1647 | 1637 | 0 | 1651 | 2538 | 65.1% |
|
||||
| ov_SC03_102 | 1647 | 1637 | 0 | 1650 | 2502 | 65.9% |
|
||||
| ov_SC03_103 | 1645 | 1639 | 0 | 1648 | 2520 | 65.4% |
|
||||
| ov_SC03_104 | 1657 | 1637 | 0 | 1664 | 2626 | 63.4% |
|
||||
| ov_SC03_105 | 1651 | 1637 | 0 | 1658 | 2607 | 63.6% |
|
||||
| ov_SC03_108 | 1639 | 1637 | 0 | 1639 | 2453 | 66.8% |
|
||||
| ov_SC03_109 | 1640 | 1637 | 0 | 1642 | 2433 | 67.5% |
|
||||
| ov_SC03_110 | 1642 | 1637 | 0 | 1642 | 2478 | 66.3% |
|
||||
| ov_SC03_111 | 1647 | 1637 | 0 | 1650 | 2520 | 65.5% |
|
||||
| ov_SC03_112 | 1644 | 1639 | 0 | 1646 | 2536 | 64.9% |
|
||||
| ov_SC03_113 | 1641 | 1639 | 0 | 1644 | 2478 | 66.3% |
|
||||
| ov_SC03_114 | 1638 | 1637 | 0 | 1640 | 2424 | 67.7% |
|
||||
| ov_SC03_115 | 1646 | 1637 | 0 | 1648 | 2482 | 66.4% |
|
||||
| ov_SC03_116 | 1644 | 1637 | 0 | 1647 | 2448 | 67.3% |
|
||||
| ov_SC03_117 | 1645 | 1637 | 0 | 1651 | 2567 | 64.3% |
|
||||
| ov_SC03_118 | 1672 | 1661 | 0 | 1673 | 2695 | 62.1% |
|
||||
| ov_SC03_119 | 1668 | 1661 | 0 | 1669 | 2695 | 61.9% |
|
||||
| ov_SC03_121 | 1644 | 1637 | 0 | 1647 | 2469 | 66.7% |
|
||||
| ov_SC03_124 | 1669 | 1633 | 0 | 1689 | 2751 | 61.4% |
|
||||
| ov_SC03_125 | 1650 | 1637 | 0 | 1662 | 2598 | 64.0% |
|
||||
| ov_SC03_126 | 1641 | 1639 | 0 | 1641 | 2432 | 67.5% |
|
||||
| ov_SC04_000 | 1651 | 1641 | 0 | 1660 | 2554 | 65.0% |
|
||||
| ov_SC04_002 | 1654 | 1637 | 0 | 1658 | 2646 | 62.7% |
|
||||
| ov_SC04_003 | 1642 | 1637 | 0 | 1646 | 2512 | 65.5% |
|
||||
| ov_SC04_004 | 1649 | 1637 | 0 | 1651 | 2568 | 64.3% |
|
||||
| ov_SC04_005 | 1651 | 1637 | 0 | 1657 | 2622 | 63.2% |
|
||||
| ov_SC04_006 | 1640 | 1637 | 0 | 1642 | 2463 | 66.7% |
|
||||
| ov_SC04_007 | 1650 | 1637 | 0 | 1654 | 2579 | 64.1% |
|
||||
| ov_SC04_008 | 1640 | 1637 | 0 | 1640 | 2424 | 67.7% |
|
||||
| ov_SC04_009 | 1643 | 1637 | 0 | 1646 | 2450 | 67.2% |
|
||||
| ov_SC04_010 | 1643 | 1637 | 0 | 1644 | 2428 | 67.7% |
|
||||
| ov_SC04_011 | 1666 | 1637 | 0 | 1672 | 2813 | 59.4% |
|
||||
| ov_SC04_012 | 1640 | 1637 | 0 | 1641 | 2429 | 67.6% |
|
||||
| ov_SC04_015 | 1660 | 1638 | 0 | 1671 | 2621 | 63.8% |
|
||||
| ov_SC04_016 | 1638 | 1637 | 0 | 1640 | 2449 | 67.0% |
|
||||
| ov_SC04_018 | 1687 | 1672 | 0 | 1687 | 2865 | 58.9% |
|
||||
| ov_SC04_019 | 1691 | 1672 | 0 | 1691 | 2865 | 59.0% |
|
||||
| ov_SC04_020 | 1653 | 1637 | 0 | 1665 | 2577 | 64.6% |
|
||||
| ov_SC04_021 | 1639 | 1639 | 0 | 1639 | 2432 | 67.4% |
|
||||
| ov_SC05_000 | 1643 | 1641 | 0 | 1646 | 2429 | 67.8% |
|
||||
| ov_SC05_001 | 1643 | 1637 | 0 | 1648 | 2584 | 63.8% |
|
||||
| ov_SC05_002 | 1640 | 1637 | 0 | 1643 | 2452 | 67.0% |
|
||||
| ov_SC05_003 | 1641 | 1637 | 0 | 1642 | 2491 | 65.9% |
|
||||
| ov_SC05_004 | 1641 | 1637 | 0 | 1643 | 2474 | 66.4% |
|
||||
| ov_SC05_005 | 1642 | 1637 | 0 | 1643 | 2501 | 65.7% |
|
||||
| ov_SC05_006 | 1639 | 1637 | 0 | 1639 | 2439 | 67.2% |
|
||||
| ov_SC05_007 | 1644 | 1637 | 0 | 1649 | 2492 | 66.2% |
|
||||
| ov_SC05_008 | 1664 | 1637 | 0 | 1666 | 2553 | 65.3% |
|
||||
| ov_SC05_009 | 1644 | 1637 | 0 | 1648 | 2447 | 67.3% |
|
||||
| ov_SC05_010 | 1652 | 1637 | 0 | 1656 | 2598 | 63.7% |
|
||||
| ov_SC05_011 | 1639 | 1637 | 0 | 1640 | 2418 | 67.8% |
|
||||
| ov_SC05_017 | 1671 | 1634 | 0 | 1682 | 2852 | 59.0% |
|
||||
| ov_SC05_018 | 1662 | 1637 | 0 | 1675 | 2683 | 62.4% |
|
||||
| ov_SC05_019 | 1640 | 1639 | 0 | 1640 | 2432 | 67.4% |
|
||||
| ov_SC06_000 | 1674 | 1646 | 0 | 1677 | 2699 | 62.1% |
|
||||
| ov_SC06_006 | 1647 | 1637 | 0 | 1648 | 2521 | 65.4% |
|
||||
| ov_SC06_008 | 1651 | 1639 | 0 | 1657 | 2551 | 65.0% |
|
||||
| ov_SC06_010 | 1651 | 1637 | 0 | 1656 | 2527 | 65.5% |
|
||||
| ov_SC06_011 | 1643 | 1637 | 0 | 1647 | 2478 | 66.5% |
|
||||
| ov_SC06_013 | 1642 | 1637 | 0 | 1643 | 2434 | 67.5% |
|
||||
| ov_SC06_014 | 1644 | 1637 | 0 | 1646 | 2462 | 66.9% |
|
||||
| ov_SC06_015 | 1641 | 1637 | 0 | 1641 | 2430 | 67.5% |
|
||||
| ov_SC06_016 | 1652 | 1637 | 0 | 1654 | 2559 | 64.6% |
|
||||
| ov_SC06_018 | 1651 | 1640 | 0 | 1658 | 2675 | 62.0% |
|
||||
| ov_SC06_020 | 1645 | 1639 | 0 | 1646 | 2528 | 65.1% |
|
||||
| ov_SC06_022 | 1651 | 1637 | 0 | 1659 | 2652 | 62.6% |
|
||||
| ov_SC06_024 | 1656 | 1637 | 0 | 1662 | 2677 | 62.1% |
|
||||
| ov_SC06_025 | 1649 | 1637 | 0 | 1654 | 2582 | 64.1% |
|
||||
| ov_SC06_027 | 1637 | 1637 | 0 | 1638 | 2417 | 67.8% |
|
||||
| ov_SC06_029 | 1666 | 1637 | 0 | 1678 | 2672 | 62.8% |
|
||||
| ov_SC06_030 | 1644 | 1637 | 0 | 1644 | 2466 | 66.7% |
|
||||
| ov_SC06_032 | 1644 | 1638 | 0 | 1651 | 2668 | 61.9% |
|
||||
| ov_SC06_033 | 1652 | 1638 | 0 | 1659 | 2641 | 62.8% |
|
||||
| ov_SC07_000 | 1655 | 1641 | 0 | 1657 | 2529 | 65.5% |
|
||||
| ov_SC07_001 | 1644 | 1637 | 0 | 1646 | 2464 | 66.8% |
|
||||
| ov_SC07_002 | 1658 | 1637 | 0 | 1662 | 2589 | 64.2% |
|
||||
| ov_SC07_008 | 1638 | 1637 | 0 | 1638 | 2395 | 68.4% |
|
||||
| ov_SC07_009 | 1646 | 1637 | 0 | 1648 | 2439 | 67.6% |
|
||||
|
||||
@@ -0,0 +1,109 @@
|
||||
# PhaseEnd — Phase 23: The offline LLM matching tier → the Fable5 wall-breaker + the gcc-2.7.2 codegen map
|
||||
**Date:** 2026-07-02 · **Project Version:** 1.22.0 · **Phase Status:** Complete (milestone confirmed by Drew, gate 2) · **Generation:** Gen2 (15th phase of the arc; 14 deferred to Gen3+)
|
||||
|
||||
> Gen2 phase 15 (8→…→22→**23**). The **longest, highest-yield phase** — it began as "stand up a free local LLM matching tier" and, mid-run, **pivoted into the strategic breakthrough of the whole project**: a frontier model reading the gcc-2.7.2 source is a *wall-breaker*, and reading the source *once* into a **codegen map** (cookbook §31 + `docs/gcc-2.7.2-map/`) converts per-function reverse-engineering into cheap lookup. The granular per-task trail (T1–T10.9 LLM-tier, then the giant campaign + §31 study + wave) is in `phase-ends/logs/Phase23.md` (R19 — NOT auto-loaded). This file is the synthesis. Owner decisions (Drew): build the LLM tier → exhaust it → GLM frontier test → **pivot to hand-solving giants** → Fable5 experiment → generalize via Opus + §30 → **read the whole compiler source into §31** → prove it with a wave → **close here (option 3); permuter+integration next phase.**
|
||||
|
||||
## Build Log
|
||||
|
||||
**Files created/changed and complete — do not recreate:**
|
||||
|
||||
*The LLM matching tier (T1–T10, the phase's original substance):*
|
||||
- `tools/serve_local.py` (Unsloth GPU OpenAI endpoint — replaces the ejected LM Studio) · `tools/api_draft.py` (LEAN drafter + `MAXTOK`/`REASON`/OpenRouter env) · `tools/lora_grind.py` (mass-run) · `tools/bulk_harvest.py` (phase-separated parallel-gate harvester + `--cycles`) · `tools/{export_pairs,format_finetune,train_lora,eval_lora}.py` (the LoRA pipeline) · `tools/glm_reconcile.py` · `tools/idiom_hunt.py` · `tools/inject_capped_externs.py` (§28d). Models: `models/bfm-match-7b-v3` (Qwen2.5-Coder-7B QLoRA on corpus-v3; v2 fallback). Corpus `datasets/match_pairs/` (gitignored). Details in `docs/gen2-mips-matching-model.md`.
|
||||
|
||||
*The giant campaign + the §31 codegen map (this session — the pivot):*
|
||||
- **`tools/glm_parallel.sh`** — K concurrent OpenRouter/GLM cloud drafters (key from `.env` at runtime, never echoed/persisted).
|
||||
- **`docs/gcc-2.7.2-map/{sched,regalloc,loop,cse_expr}.md`** — **the codegen map (935 lines).** 4 Fable5 agents read the gcc-2.7.2 passes → source-cited, byte-proven `residual → C-lever | intrinsic→permuter` catalogs. **cookbook §31 = the index + triage table.**
|
||||
- **`tools/reference/gcc-2.7.2/`** (gitignored) — the **REAL vanilla gcc-2.7.2 source** (the loop agent proved `gcc-papermario` is gcc **2.8.1**, a behavioral diff). SETUP §5.6 + §31 flag it.
|
||||
- `docs/matching-cookbook.md` — **§30** (Fable5 store-vs-load `/s`-flag + def-side macro-widen + birthing-boost), **§30a** (§30 generalizes via Opus agents + IV-combine + inline-limit + the mechanical-integration finding), **§31** (the codegen-map index + triage + the gcc-version correction).
|
||||
- `src/shared/engine_core.h` + `src/shared/engine_types.h` + 134 overlay `.c` + `config/dedup.us.yaml` — the banked matches + ×134 propagations.
|
||||
- `docs/SETUP.md` §5.6 (gcc-version correction + the vanilla-2.7.2 reference).
|
||||
- `phase-ends/CURRENT_PHASE.md` → **`phase-ends/logs/Phase23.md`** (R19). `phase-ends/PhaseEnd_Phase23.md` — this file.
|
||||
|
||||
**Tools/packages installed:** Unsloth + torch (cu128) in a separate `.venv-train` (LLM tier, prior sessions). OpenRouter (cloud, key in `.env`). No new apt. `tools/reference/gcc-2.7.2/` staged (re-fetchable from GNU ftp).
|
||||
|
||||
**Verification results (literal):**
|
||||
- **`make check-all` → 136 passed, 0 failed of 136** (main `143dbb89…`, resident `8e17e02f…`, ov_SC01_077 `d19c9580…`, all 134 overlays) — reproduced clean after every bank (R22). *(Milestone proof run at phase close.)*
|
||||
- **Fleet: 63.66% (Phase-22 close) → 64.86% byte-identical-from-source** (function-count); ~30% byte-weighted (decomp.dev-comparable). **0 NON_MATCHING** in any default build (G4); `dedup-check` validated, 0 failed.
|
||||
- **LLM tier (T1–T10.9):** local v3 saturated the ≤15-ins band (~1,300 banks, **$0**, T10.6 → 64.6%); frontier GLM-5.2 (OpenRouter, ~$4/$25) confirmed the **def-side loose-typing wall is intrinsic** (triple-confirmed) and the **new-idiom well is DRY from both angles** (failed-residual T10.8 + fresh-hand-solve T10.9). Conclusion: the free tier harvests the easy bulk; the paid frontier tier is a modest hard-band drafter + idiom teacher, not a wall-breaker **at the leaf**.
|
||||
- **The giant campaign (this session) — 5 giants/hard-fns banked ×134** (each byte-gated, propagated, check-all-verified): `func_8014EE14` (248, Fable5), `func_8014F2E0` (66), `func_80150528` (53), `func_80149374` (23, §31 study), `func_80144090` (154, §31 wave). GLM parallel campaign: 2/102 on the walled reach-134 band (~$10; def-side wall confirmed at scale). Free v3 grind: +89 fns.
|
||||
- **The §31 wave (8 reach-134 near-misses, Opus agents applying the map):** **3 leaf-MATCH** (47–137k tokens each — vs 200–250k *walls* pre-map) + **5 substantially-advanced permuter seeds** with precise class verdicts (**`func_80132784` 240→4-off** — the 400-ins Phase-22 "HARD-DEFER" giant; `func_801412A8` 198/198 structure-exact; `func_8014E048` 129→35; `func_80176D94` 56→52; `func_80148094` front-half exact). **Zero dead-end walls.**
|
||||
- `git status`: only `config/`/`tools/`/`src/`/`docs/`/`phase-ends/` tracked; zero ROM-derived/generated bulk staged. **No Ghidra DB change** (R23 no-op — the `db.*.gbf` churn is SessionStart-restart noise, NOT staged).
|
||||
|
||||
**Milestone achieved (confirmed by Drew, gate 2):** the phase set out to build a free local LLM matching tier and did (v3, ≤15 saturated, $0). It then **pivoted to the project's breakthrough**: proving a frontier model (**Fable5Max reading the gcc-2.7.2 source**) is a *wall-breaker* — it matched `func_8014EE14`, a giant on the §20/§10 store-vs-load class **22 phases had "CONFIRMED unsteerable,"** refuting this phase's own "wall is intrinsic" verdict as *model-relative*. That generalized: **standard Opus agents *applying* the documented idioms** (§30/§30a) banked 2 more giants at 3–5× lower cost, and reading the **whole compiler source once** (4 Fable5 agents → **§31 codegen map**) broke *multiple* long-standing walls (**hoist-vs-remat**, delay-slot, coalescing) and was **proven by an 8-function wave** (3 leaf-MATCH by lookup + 5 tight permuter seeds, no walls). Fleet 63.66% → 64.86%, 136/136 byte-identical throughout. The map converts matching from expensive discovery into cheap lookup — the strategic unlock for the endgame.
|
||||
|
||||
**Next:** **Phase 24 — Fix + EVOLVE the permuter (§31-class-aware), build the integration-recovery tool, then scale the §31 wave.** The matching problem is *solved* by the map; the remaining bottlenecks are throughput plumbing (permuter for the register-permutation seeds; integration for auto-banking leaf-MATCHes). Full context in the **Phase-24 Backlog** below. Open a fresh session (plan mode, Max — Tier-1).
|
||||
|
||||
## Deviations
|
||||
| Item | Plan | Actual | Reason |
|
||||
|---|---|---|---|
|
||||
| Phase scope | build the offline LLM matching tier | tier built + exhausted, then **PIVOTED to the giant campaign + the gcc-2.7.2 codegen map** | the LLM tier hit the def-side wall (intrinsic); Drew's "hand-solve giants + Fable5" opened the wall-breaker; the map is the far bigger prize |
|
||||
| "The wall is intrinsic / idiom well dry" (T10.8/T10.9) | a firm conclusion | **REFUTED (model-relative)** — Fable5 reading gcc source cracked it | true for GLM/7B, false for a frontier model that reads the compiler internals (a clean R14 self-correction) |
|
||||
| Giant matching | Fable5 per giant (~375k tokens) | **generalized to Opus agents applying §30, then to §31 lookup** (47–137k tokens) | once documented, the idioms are cheap to *apply* — Fable5 is only needed to *discover* |
|
||||
| The compiler-source read | reactive, per-function | **systematic: 4 Fable5 agents mapped the whole source once (§31)** | Drew's insight — reading it per-function re-derives the same facts N times; one map serves all future matching + the cheap tier |
|
||||
| Source reference | `gcc-papermario` = "our compiler" (since Phase 18) | **it's gcc 2.8.1, not 2.7.2** — vanilla 2.7.2 staged | the loop agent caught a behavioral biv-elim difference; all byte-proven levers stand (match_one-validated vs the real cc1); §17/§30 citations were on 2.8.1 |
|
||||
| Giant banking | matches → auto-bank ×134 | **matching solved, but banking hit integration friction** (macro-widen + TU-context + propagation stragglers) — 1 of 3 wave leaf-MATCHes banked | the bottleneck moved from *matching* to *integration plumbing* → Phase-24 tool |
|
||||
| CURRENT_PHASE hygiene | kept current | the giant campaign initially ran under the stale "PhaseEnd next" stopping point → updated mid-session (Drew caught it earlier in the phase) | the session snowballed from "quick giant test" into the map breakthrough |
|
||||
| Effort | Max | Max throughout (deep synthesis + orchestration; Fable5/Opus agents via the Agent tool) | the campaign + map are deep single-thread orchestration, not breadth |
|
||||
|
||||
## Commit Message
|
||||
```
|
||||
(Phase 23 landed across many sessions T1–T10.9 + the 07-02 giant/map session; commits commit:0424..commit:0433 + this PhaseEnd.)
|
||||
|
||||
feat(phase-23): LLM matching tier + the Fable5 wall-breaker + the gcc-2.7.2 codegen map (v1.22.0)
|
||||
|
||||
- LLM TIER (T1-T10.9): tools/serve_local + api_draft + lora_grind + bulk_harvest + the LoRA
|
||||
pipeline (v3 model). Local v3 saturated <=15 (~1300 banks, $0). Frontier GLM-5.2 (~$4)
|
||||
confirmed the def-side wall INTRINSIC + the idiom well DRY at the leaf (both angles).
|
||||
- THE PIVOT (07-02): Fable5Max reading the gcc-2.7.2 SOURCE matched func_8014EE14, a §20/§10
|
||||
store-vs-load giant "CONFIRMED unsteerable" for 22 phases -> the "wall is intrinsic" verdict
|
||||
is MODEL-RELATIVE. Generalized to Opus agents applying §30 (2 more giants, 3-5x cheaper).
|
||||
- §31 THE CODEGEN MAP: 4 Fable5 agents read the whole gcc-2.7.2 source -> docs/gcc-2.7.2-map/
|
||||
{sched,regalloc,loop,cse_expr}.md (935 lines, byte-proven residual->lever catalogs) +
|
||||
cookbook §31 (index+triage). Broke hoist-vs-remat/delay-slot/coalescing walls. Correction:
|
||||
gcc-papermario is 2.8.1 not 2.7.2 -> tools/reference/gcc-2.7.2/ (SETUP §5.6).
|
||||
- THE WAVE (8 reach-134 near-misses, Opus + §31): 3 leaf-MATCH by lookup (47-137k tok vs
|
||||
200-250k walls) + 5 tight permuter seeds (func_80132784 240->4-off!), zero dead-ends.
|
||||
- 5 giants/hard-fns banked x134 (func_8014EE14/F2E0/150528/149374/144090); fleet 63.66->64.86%,
|
||||
136/136 byte-identical, 0 NON_MATCHING. cookbook §30/§30a/§31. No new governance rules.
|
||||
- FINDING: matching is SOLVED by the map; bottlenecks are now the permuter (S11 seeds) +
|
||||
whole-binary integration (auto-bank leaf-MATCHes) -> Phase 24. worklog -> logs/Phase23.md.
|
||||
```
|
||||
|
||||
## Rules Added This Phase
|
||||
| Rule | Reason |
|
||||
|---|---|
|
||||
| **None (governance).** Per the Phase-8+ precedent, the phase's lessons are *findings + techniques*, captured where they belong: the **matching toolkit + the codegen map** → cookbook §30/§30a/§31 + `docs/gcc-2.7.2-map/`; the **LLM-tier discipline** → `docs/gen2-mips-matching-model.md` + the `cheap-tier-ab-validated` memory; the **"a wall verdict can be model-relative — a frontier model reading the compiler SOURCE breaks classes GLM/7B can't"** meta-lesson is a direct extension of **R14** (verify vs bytes) + **R17** (web/read the compiler internals) + the `dont-conclude-unsteerable-try-register-pins` memory — now generalized to "read the source, build the map." The existing rules (G3/P9 byte-gate, R14, R16 flywheel, R17, R22, R26/R27 effort, R30 capture-during-session) covered all the new ground. | A build → pivot → breakthrough phase yields tools, a knowledge artifact (§31), and findings — not a new norm of conduct. Drew may later elevate the `dont-block-loop-with-askuserquestion` memory or a "map-before-grind" practice to a formal rule if he wants it binding. |
|
||||
|
||||
## PhaseEnd Changelog
|
||||
**v1.21.0 → v1.22.0 — Phase 23 complete (Gen2 phase 15; the breakthrough phase).** Built the **free local LLM matching tier** (v3 QLoRA + serving + the harvest drivers), which **saturated the ≤15-ins band for $0 (~1,300 banks)** and then **byte-proved its own ceiling** — the def-side loose-typing wall is intrinsic and the idiom well is dry *at the leaf, for a 7B/GLM*. Drew pivoted to **hand-solving giants**, and the phase's breakthrough followed: **Fable5Max, reading the actual gcc-2.7.2 source, matched a giant on a class 22 phases had "CONFIRMED unsteerable"** — proving the "wall is intrinsic" verdict was **model-relative**. This generalized twice: **standard Opus agents applying the now-documented idioms** (§30/§30a) banked more giants at 3–5× lower cost; and reading the **whole compiler source once** (4 Fable5 agents → the **§31 codegen map**, `docs/gcc-2.7.2-map/`, 935 byte-proven lines) **broke multiple long-standing walls** (hoist-vs-remat, delay-slot, coalescing) and was **proven by an 8-function wave** (3 leaf-MATCH by cheap lookup + 5 tight permuter seeds, incl. the 400-ins `func_80132784` driven to **4-off**). Along the way it corrected a 5-phase-old error (the source reference was gcc **2.8.1**, not 2.7.2). **5 giants/hard-fns banked ×134**, fleet **63.66% → 64.86%**, 136/136 byte-identical, 0 NON_MATCHING. **No new governance rules** (techniques → cookbook §30/§30a/§31 + the map). The decisive shift: **matching is now solved by the map** (cheap lookup, cheap-tier-usable); the remaining bottlenecks are throughput plumbing — the **permuter** (for the register-permutation seeds) and **whole-binary integration** (for auto-banking leaf-MATCHes) — which become Phase 24.
|
||||
|
||||
## Plain-English Recap
|
||||
This was the phase where the project cracked its hardest problem. We started by building a free, private AI (running on your own graphics card) that reconstructs the game's simple code for **no cost** — it finished off the small functions (~1,300 of them). But it hit a wall on the hard code, and we proved the wall was real *for a small model*. Then the breakthrough: we pointed **the best AI model in the world at the actual source code of the 1990s compiler** — and it *solved* functions we'd sworn were impossible, by understanding exactly *why* the old compiler arranged instructions the way it did. Even better, ordinary AI agents could then reuse those tricks cheaply. So we did the smart thing you suggested: instead of re-figuring-out the compiler for every function, we had four AI agents **read the entire compiler source once and write a permanent "cheat sheet"** — for every way the compiler scrambles our code, either the exact fix or an honest "this one needs the brute-forcer." We tested the cheat sheet on eight hard functions: three were solved instantly by *looking up the answer*, and the other five were driven to the brink (one giant went from "abandoned" to *four instructions* from perfect) — with **zero** of the expensive dead-ends we used to hit. The game is now **~65% rebuilt bit-for-bit**, and — more importantly — we've turned "reverse-engineer the compiler" from a per-function slog into a cheat-sheet lookup that even the cheap AI can use. What's left isn't *matching* anymore; it's two pieces of **plumbing** — a smarter brute-forcer and an auto-installer for the finished code — which is next phase.
|
||||
|
||||
## Phase-24 Backlog — "Fix + EVOLVE the permuter · build the integration tool · scale the wave" (the forward bridge — nothing here is lost on close; Drew emphasized storing full context)
|
||||
|
||||
> Phase 24 opens in **plan mode, Max** (Tier-1). The matching problem is solved by §31; **the two remaining bottlenecks are throughput plumbing.** Prioritised:
|
||||
|
||||
**1. [HIGHEST BYTE-WEIGHT] FIX + EVOLVE THE PERMUTER (Drew's "evolve it to help more" idea — a first-class design goal).**
|
||||
- **Why:** the §31 wave produced **register-permutation / schedule seeds** dominated by the **S11 LUID⊗alloc coupling** class — structure-exact drafts that no C lever moves (the map correctly routes them to the permuter). These are high byte-weight and *right there*: **`func_80132784` (4/400 — the 400-ins flagship, ×134 ≈ +1.6% byte-weight alone)**, `func_801412A8` (198/198 structure-exact), `func_8014E048` (35/143), `func_80176D94` (52/152), `func_80148094` (front-half exact), plus earlier `func_801770E0`/`func_8014EA4C`. **All seeds are in `.run/wave/` (+ `.run/toolkit/`, `.run/backlog_drafts/`).**
|
||||
- **The current permuter is BLOCKED** (tools/decomp-permuter + tools/permuter/ + tools/grinder.py; recipe cookbook §3/§3a): (a) **pycparser can't parse the `__asm__("$sN")` register pins** the toolkit drafts use → it strips/breaks them (both wave sched agents hit this); (b) it scores against a **leaf compile with relocation noise** (§5a floor), not the real bytes → it wanders (func_80176D94: base 225, best 210, never 0); (c) custom typedefs (GsOT/OTLINK) break its turnkey path. Phase-22 audit: grinder banked 7 all-time (all Phase 21), **0 since** — the *random* permuter's tail is exhausted.
|
||||
- **FIX (baseline):** (a) GNU-asm-declaration support — preprocess the pins to a permuter-safe placeholder + re-inject (or patch pycparser); (b) a **relocation-masked scorer** (mask jal/HI16/LO16 exactly like `tools/match_one.py`, which == the whole-binary byte-gate for a standalone reach-134 fn) so it scores true closeness; (c) custom-typedef dir setup.
|
||||
- **EVOLVE (the big idea — the §31 map GUIDES the search):** instead of random mutations, make the permuter **§31-class-aware** — the agent already diagnosed each seed's residual class, so DIRECT the mutation set: **S11 register-permutation** → mutate the levers §31 says drive `allocno_compare` density (**declaration order**, def/use placement, pin scope — regalloc.md RC-1/RC-2/RC-3); **S3 chain-priority** → schedule-affecting statement reorders (sched.md); **RC-6 pressure-lock** → a stabilizing pin + targeted perturbation. This turns a random walk into a **directed search over the §31 lever space** — the map's second payoff (it guides not just agents but the permuter). Implement as PERM_* macros / a class-seeded mutation set.
|
||||
- **EVOLVE further (the ML wildcard — carried from Phase 15/16, now well-motivated):** Drew has a 3080Ti/CUDA + LightGBM. Train a **candidate-ranker** on the banked corpus (asm/RTL features → likely-closeness) to prioritise the permuter's search, or an ML asm→C scorer. A real wildcard to evaluate once the directed permuter is running.
|
||||
|
||||
**2. [UNLOCKS WAVE THROUGHPUT] THE INTEGRATION-RECOVERY TOOL (§30a "integration is mechanical → build the tool", now urgent).**
|
||||
- **Why:** §31 leaf-MATCHes **do not auto-bank whole-binary** — the wave banked only 1 of 3. The friction classes seen this phase: (a) **def-side return-type macro-widen** — a `DEFINE_func_*` caller macro declares the fn `extern void` but the matching def needs `s32` (a void return DCEs a live local); widen the macro's extern `void→s32`, byte-neutral IFF the caller discards the return (§30#2; **all 3 banked giants** hit this — `func_8014EE14/F2E0/150528`, verified byte-neutral via check-all); (b) **TU-context perturbation** — a leaf-MATCH whose whole-binary bytes differ with **no caller macro involved** (`func_80155800`, `func_8014F4C0` — drafts in `.run/wave/`; a different declaration/context effect, needs diagnosis); (c) **propagation stragglers** — a banked fn's dedup group has a cross-overlay member that isn't byte-identical (loose-typing/overlay-local) → the ×134 lift drops to ×1 (`func_80144090` banked ×1; a pre-existing `0x80165CA0`/`0x8012A018` straggler); (d) **data-extern type conflicts** (`func_8014EE14`'s `D_80126720` `u8[]` vs `s16`).
|
||||
- **The tool:** a `gate_stage` recovery pass that, on a leaf-MATCH-but-whole-binary-near fn, tries in order: **macro-widen (a)** → re-gate; **TU-context diagnosis (b)**; **straggler recovery (c)** (bank ×134 where clean, ×1 where straggler-blocked, log the reason). Goal: **every §31 leaf-MATCH auto-banks** (with the reason logged when it can't reach ×134). Test cases: the unbanked wave leaf-MATCHes `.run/wave/func_80155800.c`, `func_8014F4C0.c`. Context: the recovery pipeline is `gate_stage.py` (canon_resident_calls → cast_call_sites → sig_unify → harvest_verify → dedup_propagate); the propagation cap/straggler is cookbook §19/§20; the macro-widen is §30#2 (manual now).
|
||||
|
||||
**3. [THE HARVEST] SCALE THE §31 WAVE (after 1+2).** With the permuter closing S11 seeds (1) and the integration tool auto-banking leaf-MATCHes (2), scale the §31-powered Opus-agent wave over the **full near-miss backlog** (~95 "schedule"/80 "regalloc"/31 "hoist" verdicts — most now steerable per §31). Each match ×134. Batch ~8; triage-first (bail to permuter on confirmed-intrinsic); gate winners together. The map + the two tools convert the backlog into a harvest.
|
||||
|
||||
**4. [FREE-TIER FLYWHEEL, carried] v4 retrain on the idiom corpus.** Once the wave banks many examples per idiom, retrain v4 on the banked-with-idioms corpus so the **free** local tier applies the *common* steerable idioms in straightforward cases (per the in-phase v4 discussion — a real but bounded gain; the 7B has a reasoning ceiling + does nothing for the intrinsic walls). Also worth a $0 experiment: §31 *in v3's prompt* on a few medium near-misses to gauge the ceiling. Post-wave.
|
||||
|
||||
**5. [BYTE-WEIGHT, carried] The giants + the compile-blocked 8.** The 20 giants are 35% of remaining byte-weight; the permuter (1) is the key to the S11 seeds. The **8 compile-blocked giants** (region `a`) are **mechanically** unblocked (ghidra-type aliases + data externs — NOT struct-walled; type-map finding this phase) — a cheap pass makes them draftable, then §31 + Opus.
|
||||
|
||||
**6. [STRATEGIC, carried] The public flip** (Fable5 §6.3, deferred to Gen3+) remains the community-labor lever; the LLM tier + the map make the repo an attractive contributor on-ramp when it's time.
|
||||
|
||||
**Carried LLM-tier state:** v3 = `models/bfm-match-7b-v3`; serve via `tools/serve_local.py` (Unsloth GPU, NOT LM Studio); `bulk_harvest.py` for the ≤15 tail (~92% saturated, ~383 low-value untried left); corpus `datasets/match_pairs/`; OpenRouter key in `.env` (`open_router_key`, ~$21/$25 left). serve_local is DOWN, GPU free.
|
||||
|
||||
## 🛑 Stop Here
|
||||
PhaseEnd written; `CURRENT_PHASE.md` archived → `phase-ends/logs/Phase23.md` (R19). The Phase-23 work is committed across the session (`commit:0424`..`commit:0433`); **Drew commits AND pushes** this PhaseEnd + the archived log + the regenerated digests (`docs/progress.fleet.md`) (R6/R8 — the milestone-close is the human-controlled gate). No Ghidra DB change (R23 no-op — the `db.*.gbf` churn is restart-noise; do NOT stage; the pre-session `.gitignore`/`docs/backlog.md` edits are unrelated). Keep this file forever. Gen2 continues — do **NOT** start Phase 24 here. Start a **fresh session** (effort **Max**, **plan mode**) for **Phase 24 — Fix + evolve the permuter (§31-class-aware) · build the integration-recovery tool · scale the §31 wave** (the Backlog above; permuter first — it's the highest byte-weight lever, and the flagship `func_80132784` is 4 instructions from ×134).
|
||||
Reference in New Issue
Block a user