diff --git a/phase-ends/PhaseEnd_Phase30.md b/phase-ends/PhaseEnd_Phase30.md new file mode 100644 index 0000000000..06b4dbff26 --- /dev/null +++ b/phase-ends/PhaseEnd_Phase30.md @@ -0,0 +1,134 @@ +# PhaseEnd — Phase 30 The Recovery & Concentration Campaign (overlays to their ceiling) +**Date:** 2026-08-14 · **Project Version:** 1.29.0 · **Phase Status:** Complete + +## Build Log + +**The phase in one line:** the overlay fleet was driven from 87.5% to **95.3% instruction-weighted** +(78.0→**90.0%** distinct, 92.00→**96.68%** fn-count) across 25 sessions (S26–S50) and 947 commits, +while the *denominator itself* grew from 140 to **213 byte-identical binaries** (the in-phase disc +audit + module-fleet onboarding), open stubs fell **28,296 → 12,059**, and the matching machinery +crossed from agent-driven to majority **zero-token mechanical** banking. + +**Files created/changed and complete — do not recreate** (category summaries; the per-session +detail is `phase-ends/logs/Phase30.md`, on-demand per R19): + +- `src/` — ~16,200 stub instances replaced with byte-verified C across 211 overlays + modules; + dedup registry 1,886 → **2,061 groups / 0 failures** (`config/dedup.us.yaml`). +- **Matching engine tools (new, all in `tools/`):** `o0_subsplit.py` (±O0 region carving; 277 -O0 + TUs), `prefetch_fleet.py` + Ghidra-C cache (`.run/ghidra_c/`), `family_cousins.py` (the §168 + similarity tier + adapt/A-prop card lanes), `gate_lane.py` (the §169 wave-gate contract), + `aprop_symfix.py` (§171 stale-seed-symbol oracle), `aprop_autodraft.py` + `draft_prechecks.py` + (§171a/b zero-token mechanical drafts), `cc1_dumps.sh` (§172 RTL-dump instrument), + `jtbl_family_bank`/`family_sweep --hseq` hardening, permuter ILS warm-restart in `grinder.py`. +- **Instrument repairs (T0):** `gate_stage.py` stage-0 + scaled/caught propagation timeout + + arity journal; `rtu_match.py` real-error surfacing; `family_hseq`/`progress` digest + reconciliation (scope-stamped); backlog scanner deleted (R33); 21-file absolute-include fix. +- **Fleet expansion (S41–S45):** the definitive disc audit (partition-asserted, R32-gated via + `make audit-binaries`/`audit-text-sources`); +73 code-bearing payloads onboarded emulator-free + (static-address derivation); the L3 emulator measurement tour; per-binary SHA gates for all 213. +- **Knowledge base:** cookbook **§122–§172b** (~50 sections) including the §31→§172 idiom arc, the + §168/§169/§170/§171 lane laws, and the §172/§172a/§172b compiler-internals models; + `docs/frontier-p30.md`; `docs/family-cousins.md`; `docs/decision-log.md` grew by ~12 entries + (R31); `docs/accelerators.md` seeded; `tools/cookbook_index.py` (self-asserting, in + tools-health). +- `docs/SETUP.md` — rows for every tool above (R21). + +**Tools/packages installed:** none external — all in-repo tooling (R3). + +**Verification results (literal, at close):** +- `make clean && make extract-all && make check-all` → **extract-all: 212 extracted, 0 failed of + 212 (+ main, serial)**; **check-all: 213 passed, 0 failed of 213** (R22 run 5× in S50 alone, + 213/213 every time; `.run/r22_s50f.log` EXIT=0). +- `make tools-health` → OK (sigs fresh; corpus+resident, cdecl, binaries, report lint+dedup, + cookbook-index all green; fail-closed). +- dedup-check: **2,061 validated, 0 failed**; C1 coverage 254,517/254,517 members signed. +- Fleet digest (`docs/progress.fleet.md`, audit-digest-asserted): instr 12,885,596/13,523,856 = + **95.3%** · distinct 5,268,903/5,851,963 = **90.0%** · fn-count 351,097/363,163 = **96.68%** · + INCLUDE_ASM stubs **12,059** · NON_MATCHING 0 in any default build (G4). + +**Milestone achieved:** the phase milestone — *"overlays at their measured ceiling: ≥95% instr +fleet, or every remaining overlay stub on a named wall/behemoth/queue ledger — byte-identical +throughout"* — is met on BOTH clauses: **95.3% ≥ 95%** with **213/213** byte-identical (the +original 140 plus the 73 onboarded in-phase), AND every remaining stub is on a named ledger +(`.run/family_hseq.json` / `family_cousins.json` frontier maps, `.run/backlog.jsonl` + +`docs/backlog.md` near-miss ledger, the §172 wall dossier, the P32 giant/behemoth list). + +**Next:** Phase 31 — plan fresh from `docs/roadmap-to-100.md` v2 + this file's Roadmap delta. +Start with the Phase Start Protocol in a fresh session (no plan is pre-committed here, per P8). + +## Roadmap delta (feeds the P31 plan) + +- **Overlays are at their measured ceiling for wave-scale work.** What remains in overlay-land: + the LEN+N draft pile (~400 stored drafts where the target is LONGER — §172b swaprepeat/extend + candidates), the A-prop mechanical reaches (PURE 437 members/37k ins for `aprop_autodraft`'s + data/jtbl extension; IMM 275/8.8k half-resolvable; STRUCT 238/4.3k agent-shaped), 13 + ambiguous-symbol stored drafts, and the md CARVE-REFUSED class (carve-tool extension). +- **main (SLUS EXE) is now the single largest coherent mass:** 79,510 weighted ins at 0.5%, + structurally barren of family leverage (§168 holds), Ghidra sig regenerated 2026-08-05 — + the genuine ~490-tok/ins agent tail. P31's core question is main's campaign design. +- **The wall track (P32) is fully provisioned:** func_8017C294 ×16 (§172 mechanism-complete, + NEAR-2 floor, siege kit banked), the 2 GIANT exemplar walls (close 91/110, 50,094 ins riding), + the behemoth list, func_8017EE78, and the roadmap-v2 W3-owner gap flagged at T5 remains OPEN — + assign W3's remainder an owner phase during P31 planning. +- **The zero-token machinery generalizes:** aprop_autodraft + symfix + prechecks + gate_lane is + a reusable pipeline (measured ~256 members banked for ~0 agent tokens vs ~20M as waves) — + P31 should route EVERY lane through the deterministic pipeline first, agents last (per the + offline-tooling-first standing goal). + +## Deviations +| Item | Plan | Actual | Reason | +|---|---|---|---| +| T2 route | "two-file atomic o0b substitution driver" (primary), Arm-A splat research (fallback) | BOTH refuted by probes; shipped `o0_subsplit.py` carve instead (§126) | o0b-append mis-places unless target abuts object END (×1 probe); the `%lo +0x20` premise measured byte-neutral | +| T1 prior | S16 "39%" stored-draft recovery expectation | 16% measured (18/~111) | the 39% was measured on FRESH same-wave drafts; stored backlog is the A10 integration-decayed class | +| T6/T7 execution | ordered wave lists (S1–S5 / S1–S7) | consumed as a measured continuation across S26–S50 with re-ranking by templatable weight; exclusions: 2 GIANT walls + behemoths → P32, main tail → P31, ×2-9 mass + singleton residue → ledgered | the pricing finding (h_exact under-prices family heads ×138) + per-lane ROI verdicts superseded static ordering, approved in-phase (P5d) | +| Fleet denominator | 140 binaries assumed for the phase | 213 by close (+73 onboarded in-phase) | the disc audit found the honest denominator; milestone restated against it a fortiori | +| Milestone wording | "140/140 byte-identical" | 213/213 | same binaries all green plus the new ones; no regression at any point | +| func_8017C294 (T6-adjacent) | crackable frontier head | mechanism-complete WALL (§172, ~240 refutations incl. the first whole-binary-gate run on its NEAR drafts) | the +32 frame bytes need structurally different source with identical bytes; P32 siege kit banked | +| R22 cadence | per banked batch | run 5× in the close session alone; 213/213 every time | fleet-shared writes multiplied late in phase | + +## Commit Message +*(for Drew's milestone-close commit)* +``` +chore(phase-30): CLOSE — overlays at their measured ceiling (95.3% instr / 90.0% distinct / 96.68% fn, 213/213) + +- PhaseEnd_Phase30.md written; CURRENT_PHASE.md archived to phase-ends/logs/Phase30.md (R19) +- 25 sessions (S26–S50), 947 commits, stubs 28,296 -> 12,059, dedup 2,061/0 +- milestone met on both clauses: >=95% instr AND every remaining stub on a named ledger +- v1.28.0 -> v1.29.0 +``` + +## Rules Added This Phase +| Rule | Reason | +|---|---| +| **R37 — Probe before costing** *(proposed at PhaseEnd_Phase29; operated as binding all phase — Drew to RATIFY formally)* | Exercised throughout P30 (the ×1 o0b probe that saved 2,131 builds; the pre-probe rule in T7; the S50 selector probes). | +| **R38 — Read the recorded failure verdicts before designing an experiment** *(proposed; Drew to accept/modify/reject)*: before scoping any measurement, experiment, or "test X to explain failure rate Y", first grep the classified failure ledgers the tools already wrote (`.run/harvest_failed.*.classified.txt`, gate results, autopsy rows) — the answer is frequently already recorded. Extends R35/R37 to the *diagnosis* step. | The S49→S50 A-prop arc: the checkpoint's top-priority "TU-spread test" was already answered three ways by artifacts on disk (`undefined reference to D_80181900` ×11, verbatim); §169 even had the law, unapplied cross-lane. A session's first hours went to a question the tools had answered the night before. | +| **R39 — Negative-control every new refusal-check against the already-succeeded population** *(proposed; Drew to accept/modify/reject)*: a pre-check/selector/filter that refuses work ships only after running it over everything that previously PASSED, requiring zero false positives — a checker that discards good work silently is worse than one that lets a few failures through. | `draft_prechecks.py`'s negative control (all 205 banked drafts) caught two bugs in the checks themselves (C89 `f()` = unspecified params, not zero; a member's own definition read as a self-call) — either would have silently discarded good drafts forever. Same pattern caught aprop_symfix's `local-only` class. | + +## PhaseEnd Changelog +v1.28.0 → v1.29.0 — Phase 30 complete: the overlay recovery & concentration campaign. Fleet +87.5→95.3% instr (78.0→90.0 distinct, 92.00→96.68 fn-count) over 25 sessions; denominator +140→213 byte-identical binaries (in-phase disc audit + module onboarding); stubs 28,296→12,059; +dedup 2,061/0. Matching industrialized to majority zero-token mechanical (cards → autodraft → +symfix/prechecks → gate_lane → per-fn propagation). Cookbook §122–§172b; the §172 +compiler-internals wall model; main + walls + LEN+N vein handed to P31/P32. + +## Plain-English Recap + +This phase took the game's hundreds of level files from "mostly translated" to "nearly finished" — +95.3% of all the game's code now rebuilds byte-for-byte from our C source, checked against the +original disc 213 files at a time. Along the way we discovered the disc held 73 more code files +than anyone had counted, added them to the checklist, and beat the target percentage anyway. The +biggest change was HOW the work gets done: at the start, an AI agent hand-translated most +functions; by the end, most conversions were produced by our own deterministic tools for free — +the AI is reserved for the genuinely novel cases. We also went deep into the original 1995 +compiler's source code and came back with a field guide to its quirks, which un-stuck 52 +functions in one sweep and fully explains the one function that still refuses (a haystack problem +saved for the endgame). What's left: the main executable (the last big untranslated mass), a +short list of named "wall" functions, and a pile of near-misses our new field guide is expected +to chip through. + +## 🛑 Stop Here +PhaseEnd written; `CURRENT_PHASE.md` archived to `phase-ends/logs/Phase30.md`. **Both are left +uncommitted for Drew** (R6): review, commit with the message above, and push — exclude the +`ghidra/db.*.gbf` rename churn (R23 noise; no RE writes this session; MCP stopped cleanly with +"Save succeeded"). Then start a **fresh session** for Phase 31 planning. Keep this file forever. diff --git a/phase-ends/CURRENT_PHASE.md b/phase-ends/logs/Phase30.md similarity index 100% rename from phase-ends/CURRENT_PHASE.md rename to phase-ends/logs/Phase30.md