BFM-decomp/phase-ends/PhaseEnd_Phase35.md

# PhaseEnd — Phase 35: Gen3 opens — the dedup phase, "one source per unique function"
**Date:** 2026-09-08 · **Project Version:** 2.1.0 · **Phase Status:** Complete (milestone: see §Milestone — every item with its literal output; gate 2 = Drew's close commit) · **Generation:** Gen3 (the first phase; Drew's Gen3 order: dedup → pins → structs → names, one phase each, planned one at a time)

> Chartered at gate 1 (S94, 2026-09-08, plan mode, Max) from `docs/gen3-handoff.md` + `docs/gen3-standards.md`, on evidence read from
> sotn-decomp's tree that reversed the 2026-09-02 "leave the dedup backlog" decision. Ran **three sessions (S94–S96, all 2026-09-08)** and
> **48 commits** (`48170fd7f..HEAD`, plus this close). The granular trail — every task's verify lines, the S94 death and the S95
> recovery, every tool defect with its cause — is at **`phase-ends/logs/Phase35.md`** (R19, on demand). Owner decisions (Drew): gate 1's
> four (twins → one source directory; cross-address classes censused and deferred; macro-era tools frozen with a loud refusal; R96–R99
> ratified); no "trivial" exception; the tooling must know there is one source per unique function (→ the S1 gate); the recovery session's
> scope ("capture only"); the effort levels per step (Max for two decisions delegated to Claude, high/xHigh otherwise); T9 run in the same
> session at xHigh.

## Build Log

**The phase in one line:** every shared function body became one plain-C header included at each member's site (sotn's shape), the
five identical-payload overlay twins build from one source directory, the 4,755 already-matched duplicate copies of 1,099 same-address
classes were shared down to 160 copies in 51 ledgered classes, 38 functions whose bytes vary per overlay share one text through a new
registry tier, the invariant "one source per unique function" is a `tools-health` gate with a second, disagreeing oracle, the macro-era
tools are frozen or retired behind a guard, the published numbers were corrected where the old counting was blind — and every step was
proven by the 218 hashes from a clean rebuild.

**Files created/changed and complete — do not recreate** (category summaries; the per-commit list is `git log 48170fd7f..HEAD` and the
archived log):

- **The shape (T3, T4, T5, T5b).** `src/shared/engine_prelude.h` (what every overlay/module TU includes at line 2), `src/shared/{ov,main,
  slot_800CAE08,slot_801E25E8,slot_801EF468}/func_<VRAM>[__<h8>|__t<h8>].h` — **3,175 headers**, one per shared body (2,215 converted from
  the macro header at T4, 914 + 1 + 38 written by the share tool); `src/shared/engine_core.h`, `ov_setters.h`, `clearTbl40.h` deleted;
  the whale `func_80144B9C.h` moved under `ov/`; **3,173 registry groups** in `config/dedup.us.yaml` (2,220 at the open; 38 of them
  `tier: h_text` in the verbose member form with per-member `h_exact`); `config/dedup_exceptions.tsv` (the S1 ledger: 51 rows, each with
  the compiler's diagnostic and a reason code); `config/overlays.mk` `<twin>_TWIN_OF/_SRC_DIR` for SC01_006, SC03_119, SC02_003, SC04_019,
  SC03_015 (their `src/` directories deleted, 166 files; the twins' yamls = the primaries' carves with `create_c_files: False`, their
  `JTBL_PADS` lines regenerated); the Makefile's twin block.
- **The tools (T1, T2, T4, T5, T5b, T6, T7).** `tools/share_census.py` (the S1 census + gate: every sig instance resolves to one source form;
  verdicts A/B/C/D/M, flags E/F/TWIN/PINS/ALIAS; `--selftest` fixture; `--check --strict-macros --strict-text`; `text_classes`; the second,
  sig-blind oracle keyed by (text, address)), `tools/macro_to_header.py` (the T4 converter, kept as `share_body`'s library), **`tools/
  share_body.py`** (the permanent share tool: `--plan / --apply --bucket extend|new|text / --reexemplar / --repair-registry`, the
  per-binary gate with object comparison and a snapshot-based bisect, the ledger, `--batches 1`), `tools/share_body_cycle.sh` (the
  unattended batch cycle: batch → exit code → phase-log entry → commit → periodic clean fleet run), `tools/frozen.py` (the one refusal
  14 FROZEN tools print from `main()`), `tool_census.py` (status FROZEN; the `ast` macro-form guard with `--guard-root` for the negative
  control), `dedup_integrate.py` (C2c/C2d; the `h_text` tier with C1 per member), `progress.py` (the include form as a definition, the
  source-dir oracle, the empty-shared fold, `unique_function_bodies`, `duplicate_source_copies/_classes`, `text_tier_*`,
  `deferred_same_address_*`, a dated `corrections` list, the README sentence), `corpus.py` (`src_dir/twin_of`, the one source-dir oracle),
  16 health-chain tools taught the include + twin forms (T2), 12 tools' dead macro branches dropped (T6); `tools/sunset/`: `dedup_propagate`,
  `dedup_extend` (their library surface moved verbatim into `share_body`), `macro_draft`, `test_reconcile_ledger`; the Makefile's
  `tools-health` gained `share_census --selftest` and the strict check before `report`.
- **The record (T8).** `docs/wiki/The-dedup-engine.md` rewritten for the shared-source model; `Repository-layout.md`,
  `Where-the-project-goes-next.md`, `Verification-and-progress.md`, how-to ch.10, README prose and the generated block (with the
  "One source per unique function" sentence), `docs/gen3-standards.md` (§4 row struck, the DoD's "0 macro bodies" met), `docs/gen3-
  handoff.md` (dated snapshots), cookbook **§453** + a retirement banner on §14, `docs/decision-log.md` P35 (R31), `docs/accelerators.md`
  P35 (five), DIGEST §0/§2/§3/§4, `docs/SETUP.md` P35 sections (T1, T3–T5, T6, T7/T5b, the context guard), `docs/tool-index.md` and the
  kit corpus regenerated, the memory `dedup-backlog-leave-it` rewritten as REVERSED.
- **Evidence (`.run/P35/`, allowlisted text):** `baseline/` (every R22 and tools-health log, the kit-corpus logs), `census/` (the census
  json/txt, `check_t5.txt`), `share/` (every batch record `batch_<label>.json`, every run log, the 146 per-binary gate logs, the registry
  repair record, the old header of the re-exemplar), `convert/` (T4's batch scripts, ledger and apply logs), `probe/` (T0's object A/B).
- **Outside the repository:** `~/.claude/statusline.sh` writes the context percentage; `~/.claude/ctx_guard.sh` (a user-settings
  `PostToolUse` hook) injects "90% context. decide on the best place to checkpoint for a fresh session" once per session — built after
  S94 died at 91 % without a checkpoint; it fired in S96 exactly at the right moment.

**Tools/packages installed:** none (no apt/pip). Agents: three at gate 1 (two Explore, one Plan; ~680k tokens); none during execution —
every task was the coordinator's own work and deterministic gates.

**Verification results (literal, at close — S96, HEAD `c1a901373` + this close):**
- T9 R22 from clean: `extract-all: 217 extracted, 0 failed of 217 (+ main, serial)` · **`check-all: 218 passed, 0 failed of 218`** · `wall=87.62 s` · exit 0` (`.run/P35/baseline/r22_t9.log`).
- T9 `make tools-health` (foreground): `tools-health: OK — sigs fresh; corpus(+resident) + cdecl + binaries + report(lint+dedup) + cookbook-index all green.` (run 2, 431 s, exit 0; run 1 was red only at the kit corpus, which had not yet copied this file and the DIGEST edit — regenerated, re-run) (`.run/P35/baseline/tools_health_t9.log`) carrying `S1: one source per unique function — 10,180 classes, 10,180 satisfied (3,507 twin-covered, 51 excepted, 3,801 deferred cross-address), 0 VIOLATION(S) — OK`, `dedup-check: 3173 validated, 0 failed | C1 coverage 262573/262573 members signed`,
  `macro-form guard: 0 LIVE tools reference the retired form (217 scanned, 14 frozen, 6 whitelisted detectors, 38 retired)`.
- `git grep -c '^#define DEFINE_func_' -- src` → empty; `tools/macro_to_header.py --verify` → `OK — 0 macro sites, no macro header, the
  prelude present`; the census `macro sites 0` (two block-comment lines in prose still mention the token — comments are the record);
  `src/shared/engine_core.h` absent.
- `tools/share_census.py --selftest` → `7/7 verdicts correct`; the strict check's second oracle: `0 same-address definition texts
  duplicated across TUs`, `381 … inside the deferred cross-address classes (1,668 sites)`.
- `tools/progress.py --json --readme --check` → fresh; `timeline --check: fresh (74 rows)`; `doc_links` strict `0 pending, 0 broken`;
  `wiki_render --selftest: 12 cases, 0 failed; 32 pages, 0 unlisted`; `cookbook-index OK (1170 sections)`; `kit_coverage: OK`;
  `tool_census --check: OK` (the guard: 217 scanned, 14 frozen, 6 whitelisted detectors, 38 retired).
- The negative controls on record (R39): the guard against a worktree of the pre-T4 tree flags exactly the 15 tools with macro-form code
  (`--guard-root`); one member's include reverted in place → `S1 … 1 VIOLATION(S) — FAIL` naming the class, C2d naming the member, restored → OK.

**Milestone achieved (gate 2 — Drew's close commit is the confirmation; each item with its literal output):**
1. `git grep -c '^#define DEFINE_func_' -- src` empty; no `DEFINE_func_X()` site (`macro_to_header --verify` 0; the census 0);
   `src/shared/engine_core.h` absent; every registered body a plain-C header under `src/shared/<space>/` (`dedup-check` C2c, 3,173 groups).
2. `tools/share_census.py --check --strict-macros --strict-text` exit 0: `S1: one source per unique function — 10,180 classes, 10,180
   satisfied (3,507 twin-covered, 51 excepted, 3,801 deferred cross-address), 0 VIOLATION(S) — OK` — same-vram duplicate copies 160, each
   in a ledgered class with its reason; the five twins built from one source directory each (`audit-binaries` CHECK 3b); the cross-vram
   classes published as a deferred count (3,801; plus the 381 tiny-body texts / 1,668 copies inside them).
3. `make tools-health` OK with S1 strict and the macro-form guard; every consumer updated (16 + 12), frozen (14) or retired (4) per its
   dictionary row.
4. `make clean && make extract-all && make check-all` → `check-all: 218 passed, 0 failed of 218` (R22, T9 and after every batch).
5. README/wiki/kit regenerated (R75), decision log (R31), SETUP rows (R21), this PhaseEnd + the DIGEST; v2.1.0.

**Next:** **Phase 36 — pins off** (Drew's Gen3 order). Open a fresh session in **plan mode, Max** (Tier 1) from `docs/gen3-handoff.md`
§5–§7 and `docs/gen3-standards.md`: the 44,243 `register … __asm__` pins (most come off byte-identical once the source shape is right —
Phase 32's finding), family-batched, byte-gated per binary, R22 per batch; the S1 gate stays in the chain (a pin removed inside a shared
header reaches every member — one edit, one gate). Inherited by name: the 51 ledgered declaration conflicts (the types phase), the 3,801
cross-address classes and the 381 tiny-body texts (the names phase's parameterized form), the kit split, xsig v2.

## Roadmap delta (what Phase 35 changed about the road ahead)

- **The Gen3 charter's first item is met and measured:** "shared engine functions live as C, not as macros" — 0 macro bodies; the
  charter's "5,147" counted define lines (3,516 distinct macros; R14/R41 correction published in the log and the wiki).
- **A new registry tier exists (`h_text`)** for the shape the byte tier cannot hold: identical C whose bytes vary per binary through
  the translation unit's declaration environment. The types phase must treat those 38 headers as compiling per unit by design.
- **The names phase inherits a precise list:** 3,801 cross-address classes (E flag), 381 same-address tiny-body texts inside them, and
  the parameterized `SHARED_FN` form as the intended shape (one worked example, `clearTbl40`).
- **The types phase inherits the ledger:** 50 declaration conflicts in the late-onboarded overlays (`config/dedup_exceptions.tsv`,
  each with its diagnostic) and one byte-variant body — exactly where a canonical declaration layer (R95) pays.
- **Scale, honestly (R41):** the plan priced the phase at 7–8 sessions; it took three (one of them a recovery), because the tooling was
  built to gate itself and the batches ran unattended. The cost that was real: one dead session's 35 minutes rebuilt from its transcript,
  four tool defects found from their artifacts, and nine red health-chain runs across T6–T7, every one an instrument finding.

## Deviations

| Item | Plan (gate-1 plan) | Actual | Reason |
|---|---|---|---|
| Sessions | 7–8 | 3 (S94–S96) + this close | the gates were the tool's; batches ran unattended (`share_body_cycle.sh`) |
| S94 | a checkpoint at every pause | died at 91 % context waiting on a background run, no checkpoint for T5; S95 rebuilt the state from the transcript with no runs (Drew's scope) | two bank commits carried no log line; rule candidate (b); the context guard built |
| `share_body` | designed at T5 | four defects found from S94's artifacts and fixed at S96: the cause extractor (no "error" token in gcc 2.7.2), `git checkout` as restore, a twin's duplicate edit, wholesale registry extension | each caught by a gate or a known-true control; cookbook §453 |
| Bucket 0's residue | ledgered with a message class | 48 late-overlay declaration conflicts ledgered TU-CONFLICT for the types phase; `E_func_80168B70` re-exemplared from the majority text (Claude's delegated decision at Max) | the header carried a 7-overlay minority spelling; the cast form compiles under both environments (141/141) |
| The second oracle | "a second, disagreeing oracle" | it disagreed: 38 same-address texts / 2,030 copies byte-variant per binary → **T5b added** (the `h_text` tier; Claude's delegated decision at Max); 381 tiny-body texts inside the deferred classes stay deferred | no later phase would remove the copies; one header per address for the empties would be undone by the names phase's form |
| T6's counts | freeze 7, retire 3 | freeze **14**, retire **4**; the guard's census measured 30 hits, not 22; `family_cousins` stays LIVE (`make atlas` runs it) with its dead branch dropped | the plan's counts came from a partial census |
| `progress.py` | "the +219 correction" | +459 (SC03_015's 219 + SC03_118's 3 + the never-extended members of five groups) and REAL −10,211 → EMPTY (the fold); both dated in `progress.json` `corrections` | the include form made invisible sites visible; the macro-era fold had counted empty shared bodies as REAL |
| T9 | Max, fresh session | run in S96 at xHigh (Drew) | owner's call; the checkpoint carried the brief |

## What we believed, what failed, and what we would do sooner (for the retrospective; detail in `docs/decision-log.md` P35)

- **Believed (2026-09-02):** sotn writes duplicate functions explicitly, so ~12,000 duplicate copies are the normal end state. **False:**
  read from its tree, sotn shares stage code once and instantiates it by an include. **Sooner:** read the target project's tree, never a
  memory of it, before setting a policy on its precedent.
- **Believed:** refreshing the checkpoint at every task close was enough. **S94 refreshed it five times and still died with 35 minutes
  of state only in its transcript** — two bank commits inside a task carried no log line, and a background wait consumed the rest.
  **Sooner:** every commit that advances a task carries its log line and the headline; past ~85 % context, checkpoint before any long wait.
- **Believed:** a green per-binary gate meant the tool's edits were what it reported. **Its restore was `git checkout`,** which wiped the
  previous batch's uncommitted shares and returned green; its cause lines read "Error 33" for 254 of 303 rejections. **Sooner:** a tool's
  restore path is part of the tool (R57); read a compiler's diagnostics by their form.
- **Believed:** the byte join sees every duplicate. **The text oracle found 38 functions in 2,030 copies it could never class.** **Sooner:**
  build the disagreeing oracle before the batch runs, and measure the disagreement before believing either side (R34).
- **Cost:** no model tokens beyond the coordinator and gate 1's three agents; the phase's expense was wall-clock in gates (≈2.5 h of
  `make check` and fleet runs) and the recovery (≈1 h reading a 6 MB transcript).

## Commit Message
``
(Phase 35 landed as 48 commits 48170fd7f..c1a901373, 2026-09-08 (S94–S96). This close = the archived log + this PhaseEnd + the DIGEST
update + the kit's record copies, Drew's milestone-close commit.)

chore(phase-35): CLOSE — Gen3's dedup phase: one source per unique function (3,175 shared headers, 3,173 registry groups incl. the
h_text tier, the macro header gone, the five twins from one directory, S1 a tools-health gate with a second oracle, 14 macro-era tools
frozen and 4 retired behind a guard, the published counts corrected) — v2.1.0

- PhaseEnd_Phase35.md written; CURRENT_PHASE.md archived to phase-ends/logs/Phase35.md (R19); DIGEST §0/§2 appended (step 3b)
- T0–T8 + T5b across S94–S96 (S94 died at 91 % without a checkpoint; S95 recovered it from the transcript; S96 finished)
- close: R22 clean fleet 218/218; tools-health OK (S1 10,180/10,180, 0 violations; dedup-check 3173/0; the guard 0)
- rules: R96–R99 ratified at gate 1; candidates (a)–(g) proposed for Phase 36 gate 1
- v2.0.0 -> v2.1.0
``

## Rules Added This Phase

**Ratified at gate 1 (2026-09-08):** **R96–R99** (the PhaseEnd_Phase34 candidates (a)–(d)); full text in `phase-ends/DIGEST.md` §3.

**Candidates recorded in the log, for Drew to accept, modify or reject at Phase 36 gate 1 (P10):**

| Rule | Reason |
|---|---|
| **(a) A shared body has exactly one source; a duplicate copy is a defect the health chain asserts, and its count is published with its rule.** | The phase's invariant (S1), a `tools-health` gate; 1,099 classes / 4,755 copies → 51 / 160, every remainder ledgered with its diagnostic. |
| **(b) Every commit that advances a task — an intra-task bank included — carries its log line and the 🛑 headline; a checkpoint older than HEAD is a dead session's checkpoint.** | S94 refreshed the block at every task close, made two T5 bank commits with no log line, then died at 91 % context; S95 rebuilt 35 minutes of state from the transcript. |
| **(c) A tool that restores files restores from its own snapshot, never `git checkout` on a tree it did not commit.** | `share_body`'s first bisect wiped the previous batch's uncommitted shares (~50 sites) and returned green. |
| **(d) A failure-cause extractor is negative-controlled against the compiler's real message forms, not against the word "error".** | gcc 2.7.2 prints errors without it and exits 33; 254 of 303 rejection lines read "Error 33". |
| **(e) Build the disagreeing oracle before the batch runs, and measure its disagreement before believing either side (R34 operated).** | The text oracle's 4,312 hits decomposed into twin false positives, the gate-1 deferral, and 38 genuine byte-variant functions — three different actions. |
| **(f) A registry lists a member only after the gate has spoken for it; a listed member whose site does not share is a defect the check names.** | The first pass extended 788 members wholesale; 317 were private after the run (C2d exists now). |
| **(g) A policy taken on a remembered precedent is a belief — read the target project's tree.** | The 2026-09-02 deferral rested on one parenthetical about a cross-jump idiom. |

## PhaseEnd Changelog
**v2.0.0 → v2.1.0 — Phase 35 complete (Gen3's first phase).** Every shared function body in the fleet is one plain-C header included at
each member's site, sotn's shape, verified from its tree; the 8.8 MB macro header and its 5,147 define lines are gone; the five overlay
twins build from one source directory each; the same-address duplicate backlog fell from 1,099 classes and 4,755 private copies to 51
ledgered classes and 160 copies, each with the compiler's own reason; 38 functions whose bytes vary per overlay share one text through a
new registry tier; the invariant "one source per unique function" is asserted on every health-chain run by a census with a self-test and
a second, sig-blind oracle, and by four registry checks; fourteen macro-era tools are frozen and four retired behind a guard that scans
every live tool; the published numbers were corrected where the macro-era counting had been blind, dated and explained. Every batch was
proven by the 218 hashes from a clean rebuild, which now takes 84–90 seconds instead of 157. R96–R99 ratified; seven candidates proposed.

## Plain-English Recap
This phase made the code say each thing once. Before it, when the same function appeared in many of the game's 141 level files, our source
held it as a macro in one enormous header, or as a private copy pasted into each level, thousands of times over. Now every shared function
is one small ordinary C file, included wherever it is used, exactly the way the best-known PlayStation decompilation does it, and a check that
runs with every health check proves that no function has two sources. Along the way the tool that does the sharing was caught making four
kinds of mistake by its own gates and fixed; one session ran out of memory in the middle and the next rebuilt its state from the transcript;
a second, independent check found 38 functions the first could never have seen, and they got their own way of being shared. Fifty-one
functions could not be shared yet because their surrounding declarations disagree between level files, and they are listed with the
compiler's own reason for the next phase to settle. The numbers on the public page were corrected where the old counting had been blind.
All 218 game files still rebuild byte for byte, and faster than before.

## 🛑 Stop Here
PhaseEnd written; `CURRENT_PHASE.md` archived → `phase-ends/logs/Phase35.md` (R19, via `git mv`); `phase-ends/DIGEST.md` §0/§2 appended
(step 3b); the kit's record corpus regenerated (`make kit-corpus`) — **all left uncommitted for Drew's milestone-close commit** (R6 gate-2
sign-off; the message is above). The Phase-35 work is committed across the sessions (`48170fd7f..c1a901373`; nothing pushed). The headless
Ghidra MCP was stopped via the sentinel (R23); nothing under `ghidra/` is tracked. **Drew, from the repository root:**

``
git add -A phase-ends decomp-architect docs .run/P35/baseline .run/P35/census
git commit -m "chore(phase-35): CLOSE — Gen3's dedup phase: one source per unique function (3,175 shared headers, 3,173 registry groups incl. the h_text tier, the macro header gone, the five twins from one directory, S1 a tools-health gate with a second oracle, 14 macro-era tools frozen and 4 retired behind a guard, the published counts corrected) — v2.1.0"
git tag -a v2.1.0 -m "v2.1.0 — Phase 35: one source per unique function; 218 binaries byte-identical"
git push origin main --tags
``

Gen3 continues — do **NOT** start Phase 36 here. Start a **fresh session** (effort **Max**, **plan mode**) for **Phase 36 — pins off**,
planning from `docs/gen3-handoff.md` and `docs/gen3-standards.md`. Keep this file forever.