From 98988e04ec2ef08faa4287088e17fb04381485ea Mon Sep 17 00:00:00 2001 From: Christopher Williams Date: Thu, 24 Sep 2026 16:41:11 -0400 Subject: [PATCH] =?UTF-8?q?phase12:=20PLAN=20approved=20=E2=80=94=20milest?= =?UTF-8?q?one=20750=20bodies=20(+148),=20stretch=20800?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Approved by the developer 2026-09-24: milestone 750, stretch 800; both harness repairs authorised; Goal B (the 92-unclassified negatives) in scope and bounded; 4 workers — two on the fresh band, one on the 101 named negatives, one on the third fresh slice plus the 92 unclassified as the Goal B worker; compaction-first. THE ONE STRUCTURAL FACT THIS PLAN IS BUILT ON, measured at the open: 0 of the 193 near_match_negatives rows appear in the 1001-row worklist. The negatives index is held OUT of the worklist (sf3_triage plan --negatives) and is not a queue — it is an index, and a worker dispatched only from the worklist can never reach it. So the negatives pool becomes a first-class partition with its own worker, and the phase's structural change is that it stops being an appendix. Two pools, measured: 1001 fresh rows (410 of them <=244 B) plus 193 attempted-but-unresolved negatives, of which 101 carry a NAMED mechanism and are overturnable by cookbook 182's rule, and 92 carry no class at all and cannot be attempted until someone re-reads them. That is what Goal B is for. The plan states plainly that +148 is LARGER THAN ANY PHASE SO FAR (Phase 11 closed +118) and that it cannot be reached from the fresh worklist alone — it depends on the negatives-overturn programme working at scale, which is a measured route but only a ONE-ROW demonstration so far (worker F's 0x8002622C). Two safeguards are built in rather than discovered late: the cycle-1 checkpoint must include at least 8 bodies from the negatives pool, so a low overturn rate is measured in cycle 1 and not at the close; and the leading-indicator rule stands unchanged — report the projection and ask, never redefine or self-certify the milestone. Also recorded: the fresh-band size distribution, the negatives class distribution, and the baseline revalidation table including the failed precondition that T0 repaired. --- phase-ends/Phase12_PLAN.md | 341 +++++++++++++++++++++++++++++++++++++ 1 file changed, 341 insertions(+) create mode 100644 phase-ends/Phase12_PLAN.md diff --git a/phase-ends/Phase12_PLAN.md b/phase-ends/Phase12_PLAN.md new file mode 100644 index 0000000..317f4b1 --- /dev/null +++ b/phase-ends/Phase12_PLAN.md @@ -0,0 +1,341 @@ +# Phase 12 Plan — Overturning the Named Negatives + +**Status:** **APPROVED** (2026-09-24). +**Prepared:** 2026-09-24, immediately after Phase 11 closure (602 bodies / 611 regions). +**Baseline:** commit `9f2543b` on `main`, working tree clean, 722 tracked files. +**Approved decisions (developer, 2026-09-24):** 1. milestone **750**, stretch **800**; +2. **both harness repairs authorised** (§10.1 the stale maspsx patch, §10.2 `sf3_free`'s ledger path); +3. **Goal B in scope, bounded** (the 92-unclassified classification only); +4. **4 workers** — two on the fresh band, one on the 101 named negatives, one on the third fresh slice +plus the 92 unclassified as the Goal B worker; orchestrator discretion to scale or retire; +5. **compaction-first** context policy. + +**Milestone:** **750 distinct matched bodies**, from 602 — i.e. **+148**. +**Stretch:** 800 bodies. + +> ### ⚠ That milestone is larger than any phase so far, and the plan says so plainly +> +> Phase 11's record was **+118** with 4–6 workers. **+148 is 25 % more in one phase**, and it cannot be +> reached from the fresh worklist alone (the ≤244 B band is only 410 rows and a third of it yields +> +98 on Phase 11's rate). **The milestone therefore depends on the negatives-overturn programme +> working at scale**, which is a measured route but only a *one-row* demonstration so far. +> +> Two honest safeguards are built in rather than discovered late: +> 1. **Only one of the 101 named-mechanism negatives has ever been overturned** (worker F's +> `0x8002622C`). The first checkpoint (P12-T2, +30 bodies) must include **at least 8 A2 bodies from +> the negatives pool**, so that a low overturn rate is measured in cycle 1 instead of at the close. +> 2. **The leading-indicator rule** (workflow §9) applies unchanged: a per-batch cluster of sub-8-byte +> residuals with no NEW mechanism means the band is exhausted — report the numbers and re-rank, do +> not grind. If cycle 2's measured rate projects below 750, the orchestrator reports the projection +> and asks; it does not redefine the milestone and does not self-certify it. + +--- + +## 0. The honest starting point + +Phase 11 closed at 602 bodies / 611 regions with two headline results that define this phase: + +1. **The 244-byte ceiling was a dispatch artefact, not a toolchain limit.** 28 regions now exist + above 244 B, the largest 1232 B. Nothing above 244 B had ever been *attempted*, and once rows were + attempted the ceiling evaporated. +2. **The dispatch criterion is tie-break density, not size** (cookbook 79/88/99, now implemented in + the tracked `tools/sf3_rank`): a 548 B body matched first attempt while a 176 B body took nine + spellings and never closed. +3. **A recorded negative with a NAMED mechanism is overturnable; one without is not** + (cookbook 182). Worker F took `0x8002622C` out of the negatives index — a row four earlier + spellings had written off — and matched it inside the hour by making an intermediate live past the + second operation, proving the mechanism with a `cc1 -da` dump. + +Result 3 is the measured route to 700, and it is larger than the Phase 11 coordinator first reported. + +### The two dispatch pools, measured at this baseline + +| pool | rows | note | +|---|---|---| +| `config/match_worklist.tsv` (fresh rows) | **1001** | regenerated 2026-09-24, byte-identical to the tracked file, `excluded_already_registered=611` | +| `config/near_match_negatives.tsv` (attempted, unresolved) | **193** | **held OUT of the worklist** — `sf3_triage plan --negatives` excludes them | +| **total unregistered candidates** | **1194** | | + +**The negatives index is a separate pool, and this is the single most important fact for Phase 12.** +Verified at this baseline: **0 of the 193 negatives rows appear in the 1001-row worklist.** A worker +dispatched only from the worklist can never reach them. They are not a queue; they are an index. + +### What is actually in the negatives index (measured) + +| class column | rows | +|---|---| +| named class or substantive note | **95** | +| `LENGTH-MISMATCH ...` prose note | 6 | +| **`-` (no class at all)** | **92** | +| **total** | **193** | + +So **101 rows carry a named mechanism** and are overturnable by cookbook 182's rule; 92 carry no +mechanism and must be re-read before they can be attempted at all. Mean stated size of the 116 rows +that have one is 119 B — this is a pool of *small* rows, and the milestone counts bodies. + +### Fresh-worklist size bands (measured) + +| band | rows | +|---|---| +| ≤120 B | 68 | +| 121–200 B | 243 | +| 201–244 B | 99 | +| **≤244 B subtotal** | **410** | +| 245–400 B | 225 | +| 401–800 B | 221 | +| >800 B | 145 | + +Grade: 999 `exact`, 2 `fallthrough`. Tier: 138 tier-1 (exact leaf), 861 tier-2, 2 tier-3, 0 tier-0 +(the duplicate-group representatives are all consumed). + +### The arithmetic of +148 + +| source | rows available | basis | +|---|---|---| +| fresh worklist ≤244 B | 410 | the proven band; Phase 11 consumed a comparable slice | +| fresh worklist 245–400 B | 225 | **partially proven** — Phase 11 put 28 regions above 244 B | +| negatives with a NAMED mechanism | 101 | overturnable by cookbook 182; **1 / 1 demonstrated** | +| negatives with NO mechanism | 92 | unattemptable until Goal B classifies them | +| fresh worklist 401–800 B / >800 B | 221 + 145 | proven tractable but expensive in context | + +**The milestone mixes three rates, and only one of them is proven at scale:** + +- the ≤244 B band is proven, and the fresh-band workers will consume it at Phase 11's per-worker rate; +- the 245–400 B band is 225 rows and Phase 11's 28 regions above 244 B (max 1232 B) show it is + reachable — this is where A1 must go once ≤244 B thins, and it is why `sf3_rank` (tie-break density) + rather than size drives dispatch; +- **the negatives are the swing factor.** 101 rows at a high overturn rate gives the milestone its + margin; at a low rate the phase lands nearer Phase 11's own +118 rate and the shortfall is reported + with its measured cause. + +**This phase's structural change is therefore: the negatives index stops being an appendix and becomes +a first-class partition with its own worker.** + +--- + +## 1. Goal + +**Goal A (the milestone): 750 distinct bodies.** Two partitions run in parallel: + +- **A1 — the fresh band**, dispatched by `sf3_rank` (tie-break density) with `--max-size`, not by size. +- **A2 — the named negatives**, 101 rows, dispatched from `config/near_match_negatives.tsv` directly. + Each row's named mechanism is the hypothesis; cookbook 182's rule is the licence to attempt it. The + rule is: **name the pass that does the damage and the dimension that was exhausted**, then try the + ONE lever that dimension implies. + +**Goal B (strategic, evidence deliverable): a counted answer to why 92 negatives rows carry no class.** +Deliverable is evidence, not bodies. Any of these outcomes is a success: (a) the 92 are classified into +named mechanisms and become attemptable; (b) they are shown to collapse into a small number of classes +already known; (c) they are characterised as out of reach with the class counted. **A Goal B result +that adds zero bodies is still a met Goal B.** + +**Goal C (carry-forward): the GTE class.** `include/gtemac.h` carries the `lwc2`/`swc2` macros, the +command-field caveat and the register-binding lever (cookbook 159/160/161); one row matched and four +more are fully derived. Bounded: harvest the derived rows, then report. + +--- + +## 2. Structure — orchestrator + 4 workers, spawned by the orchestrator + +Unchanged from Phase 11, and **the roster must be spawned from scratch** (`CURRENT_PHASE.md`, +workflow §4.0 — Phase 11 tore the roster down; there is nothing to reconnect to). + +- **Topology:** one tab, four agent panes, plus the orchestrator's own pane. 2×2 grid; no four-column + or four-row layouts. +- **Assignment by goal, not preference:** + | worker | partition | goal | + |---|---|---| + | A | fresh band, `sf3_rank` top slice | A1 | + | B | fresh band, `sf3_rank` second slice | A1 | + | C | **the 101 named negatives** | A2 | + | D | fresh band third slice + **the 92 unclassified negatives** | A1 + **B** | +- **D is the Goal B worker** and its artifact is a running investigation record, not a body count. +- Roster may scale to 5 or shrink, orchestrator's call, based on measured attempts-per-match. +- **Adjacency is the strongest dispatch heuristic found** (4-for-4) — but it crosses partition + boundaries, so the in-flight ledger is mandatory (§5 of the workflow). + +--- + +## 3. Context policy — compaction first, rotation second + +`pi-context-tools` is installed globally. Unchanged from Phase 11: + +- Every report quotes the exact `context_info` figure. **Never estimate.** +- Worker at 70–80% with work left in its band → `compact_context`, keep working. +- Band exhausted or staging stale → rotate (verify handoff first). +- Orchestrator compacts rather than stopping at the cap. +- **Keeping the ledger and staging current is a hard requirement**, because compaction summarises the + conversation away; only the files survive. + +--- + +## 4. The carried machinery (verify before relying, do not re-derive) + +- **Toolchain:** PsyQ 4.0 `CC1PSX` (GCC 2.7.2.SN32.3.7.0002); flags `-quiet -O2 -G0`; maspsx (ASPSX + 2.56 emulator) then GNU `as`. +- **Harness:** `sf3_match` (per-region `cc1=`, `as=`, `gp=-NAME`, + `maspsx=off|noreordernop|regread|moves|epilogue|nopmarker`), `sf3_diff`, `sf3_merge`, `sf3_triage`, + `sf3_cc`, `sf3_free`, `sf3_family`, `sf3_rank`. +- **The tracked maspsx patch** `tools/patches/maspsx-phase10-r1r2.patch` — **STALE, see §10.1. This is + a phase blocker, not a footnote.** +- **Evidence tables:** `function_inventory`, `function_extents`, `duplicate_bodies`, `match_worklist`, + `near_match_negatives`. +- **The hardened merge flow** (workflow §6): verify → merge to CANDIDATE → gate whole-binary → promote + only on `result=MATCH` → reconcile negatives → regenerate worklist + refilter partitions → commit → + push. Use the gate's **exit code**, never a `grep` for `result=`. + +--- + +## 5. Rules already in force (carried into every charter, with provenance in workflow §9) + +`git ls-files --error-unmatch` before any `rm` under `src/`; the dependent-claim rule; exact symbol-name +spelling (the gp marker is name-keyed); compute-don't-eyeball; the three-encoding-trap habit; the +amended register-field rule (verify the dataflow before calling it a tie-break); the two-step +two-arm-branch rule; the paired `&&`/named-boolean rule; **verify the file you are about to stage, at +the path you are about to claim**; **run the `src/` cleanup audit BEFORE the final report**; **read start +and end from the worklist row, never from a hand dump**; **one attempt on a named lever, then classify**; +**record the source md5 per claim**; **a rule every worker must follow belongs in a TRACKED tool** +(cookbook 179); **the `maspsx=epilogue` token is decided from the CANDIDATE's tail, never the +original's** (cookbook 165/180); **derive every absolute address from the `(page, signed immediate)` +pair** (cookbook 154 — the most repeated worker error of Phase 11, five occurrences, all one worker). + +--- + +## 6. Verification ladder + +| frequency | checks | +|---|---| +| every merge | candidate gate (whole binary, SHA-1 fixed) → promote → `make check` | +| every merge | worklist regen (`excluded_already_registered` == registry size) + partition refilter with the disjointness and union-equals-worklist proof | +| every 3rd merge | `make clean && make all`, `cmp`, SHA-1, registry audit, `git status --short src/`, firewall count | +| every worker retire | staging verified, `src/` cleanup audit, negatives extracted into the tracked index | +| phase close | full clean gates, negatives **reconciliation**, stale-taken audit, milestone confirmation requested, PhaseEnd, digest, ledger, roster teardown, **stop** | +| push | after every verified merge | + +--- + +## 7. Scope and safeguards + +- ROM/SDK firewall unchanged; worker staging ignored; `git clean -x`/`-fdx` forbidden. +- Workers may not weaken a gate, widen regions, or register a non-`exact` extent. +- **No scheduler-changing flag** for a match; diagnosis only, recorded as a negative. +- Inline asm limited to shapes plain C provably cannot express, per-file documented. +- The blocked classes stay excluded unless Goal B produces evidence that reopens them. +- Goal B is bounded: two distinct root-cause attempts per hypothesis, then record. +- **Do not copy a worker's staging file over a tracked tool** (cookbook 186; `tools/sf3_diff` was + destroyed and restored). Before promoting anything into `tools/`: `git log --oneline -1 -- `, + `wc -l `, `make test`, and ask *what does the old one already do that the new one does not?* +- **Do not ignore a shrinking test count.** 253 tests is the Phase 11 close figure. + +--- + +## 8. Tasks + +- **P12-T0 — REPRODUCIBILITY REPAIR (blocker, before any matching). — ✅ COMPLETE 2026-09-24.** + §10.1 the stale maspsx patch regenerated; §10.2 `sf3_free`'s per-phase ledger resolution. Both + verified; `make test` **281 tests OK** (253 at the Phase 11 close, +28 new). Evidence: + `docs/SETUP.md` ("Phase 12 repair"), cookbook findings **188** and **189**, and the two new guard + suites `tools/tests/test_maspsx_patch.py` (7 tests) and `tools/tests/test_sf3_free.py` (21 tests). +- **P12-T1 — phase open.** Control records, clean-baseline revalidation (done in part; see §9), + herdr roster spawn (4 workers), capability probes including `context_info`, partitions cut from both + pools, disjointness/union proofs, lever files with their measured basis, `CURRENT_PHASE.md`, ledger. +- **P12-T2 — Goal A cycle 1**, both pools live. Checkpoint: **+30 bodies, of which at least 8 must come + from the negatives pool (A2)** — so a low overturn rate is measured in cycle 1 rather than at the + close (see the milestone warning in the header). +- **P12-T3 — Goal B opening:** the 92 unclassified negatives. Re-read every row against the original + bytes, classify into named mechanisms, publish the classification as counted evidence. +- **P12-T4…T-n — the autonomy loop:** per-merge regen, full audit every 3rd merge, workers compacted + rather than rotated, Goal B probed in bounded slices, Goal C harvested early and reported. +- **P12-Tn — phase close:** cookbook/conventions/verification records, negatives reconciliation + (drop any row since registered, with the overturn recorded), stale-taken audit, milestone + confirmation requested from the developer, PhaseEnd, digest, ledger archive, **roster teardown + stated explicitly**, stop. + +--- + +## 9. Baseline revalidation — evidence gathered while preparing this plan + +| check | result | +|---|---| +| `git log --oneline -1` | `9f2543b` (Phase 11 close-record correction) | +| `git status --short` | clean at first check | +| tracked files | 722 | +| registry rows | **611**, distinct `src/*.c` = **602** | +| `sf3_triage plan` re-run to scratch | **listed=1001**, `excluded_already_registered=611`, `cmp` vs tracked worklist **exit 0** (byte-identical, deterministic) | +| negatives index | 193 rows, 92 unclassified, 101 with a class; **0 overlap with the worklist** | +| maspsx patch applied? | **NO — patch does not apply to pristine and does not reproduce the working tree (§10.1). Repaired in P12-T0.** | +| `make test` after P12-T0 | **281 tests, OK** (253 + 28 new guard tests) | +| `make clean && make all`, `cmp`, SHA-1, `make extents-verify`, `make gate` | run at the open; results recorded in the Phase 12 ledger before P12-T2 | + +--- + +## 10. Decisions — developer's answers (2026-09-24) + +| # | decision | answer | +|---|---|---| +| 10.3.1 | milestone | **750 bodies (+148)**, stretch **800** | +| 10.1 + 10.2 | harness repairs | **both authorised** | +| 10.3.2 | Goal B scope | **in scope, bounded** — the 92-unclassified classification only | +| 10.3.3 | roster | **4 workers**, 2 fresh + 1 negatives + 1 third-slice/Goal-B | +| 10.3.4 | context policy | **compaction-first**, unchanged from Phase 11 | + +### 10.1 ⚠ Harness repair: the tracked maspsx patch is STALE and the toolchain is not reproducible + +**This is a failed precondition, found during the open checklist, not a hypothesis.** Evidence: + +1. `patch -p1 < tools/patches/maspsx-phase10-r1r2.patch` against pristine `86ccd7d` (extracted with + `git archive HEAD`) → **exit 1, 4 of 10 hunks FAILED**, `.rej` files written for + `maspsx/__init__.py` and `maspsx.py`. +2. The patch carries the **shape-A-only** epilogue transform from commit `05be974` ("THE EPILOGUE + POST-PASS SHIPS"). The working tree carries the **shape-A+B** transform from commit `ea51ac9` + ("merge 40 + the epilogue transform now handles BOTH shapes"). +3. `ea51ac9` touched `tools/sf3_match`, `config/regions.tsv` and seven `src/*.c` files — **and not + `tools/patches/`**. `git log --oneline -- tools/patches/` still shows `05be974` as the newest commit + touching the patch. `tools/maspsx/` is git-ignored (`tools/maspsx/maspsx.py` mtime 10:46; patch + mtime 10:42). +4. **Functional demonstration** (`maspsx --fill-epilogue`, shape-B tail + `lw $31,24($sp) / lw $17,20($sp) / lw $16,16($sp) / addiu $sp,$sp,32 / jr $31 / nop`): + | build | output | + |---|---| + | pristine + tracked patch | `... / nop / jr $31 / addiu $sp,$sp,32` — **6 instructions** | + | working tree | `... / jr $31 / addiu $sp,$sp,32` — **5 instructions** | + The original at `0x800F44D0` is **5**. `ea51ac9`'s own message: *"My first implementation did A only + and left every B row 4 bytes long."* +5. **15 registered regions carry `maspsx=epilogue`.** A fresh clone would fail the whole-binary gate on + the shape-B members of that set. + +**Decision: AUTHORISED.** Regenerate the patch as a diff of pristine → working tree for exactly +`maspsx/__init__.py` and `maspsx.py`, then verify by reconstructing from pristine and diffing +byte-identically against the working tree, and record the verification in `docs/SETUP.md`. This is the +§11 remedy (*"do not edit a tool that is git-ignored without carrying the change as a tracked patch"*), +and it **strengthens** rather than weakens a gate. + +### 10.2 Harness repair: `sf3_free`'s ledger path is hard-coded to `.run/p11/` + +`tools/sf3_free` line 35 is `INFLIGHT = REPO / ".run/p11/inflight.tsv"`. The ledger is +session-scoped and git-ignored by design, so Phase 12's ledger is `.run/p12/inflight.tsv` and the tool +would read a 306-line stale Phase 11 ledger containing `wip` rows whose holders no longer exist — the +exact stale-taken failure of cookbook 187, reintroduced by a hard-coded path. + +**Decision: AUTHORISED.** Add a `--ledger` option (default: the current phase's path derived from the +partition directory), keeping last-row-wins semantics. Small, tracked, and covered by the existing +`tools/tests` suite if it has a case there. + +--- + +## 11. Preconditions + +- **P12-T0 — ✅ COMPLETE.** The regenerated maspsx patch applies cleanly to pristine (exit 0, no + rejects) and reproduces both modified files byte-identically (`cmp` exit 0), reverse-applies cleanly + (the tree *is* pristine+patch), and still carries all four opt-in modes. `tools/sf3_free` resolves the + current phase's ledger and no longer reads a dead one. Guard tests proven to catch both defects. Patch + sha1 `0d7f643ba1c31d1ae28de257737c16c623a9eb00`, replacing the stale + `4bd21823402d73659f76b3afbf60f8f93daaa3e8`. +- `make test` green at **281 tests** (253 at the Phase 11 close). +- `make check` green at 611 regions / 602 bodies. Registry audit clean; `git status --short src/` empty. +- herdr roster spawned and probed; `context_info` confirmed live in every worker. +- Partitions cut from **both** pools with a proven disjointness and union-equals-pool check. + +**Milestone:** 750 distinct matched bodies on the clean whole-binary gate, plus a counted Goal B +classification — or the evidence-backed blocker recorded. **The milestone is confirmed by the +developer, never self-certified.**