phase11: plan — Breaking the 244-Byte Ceiling (DRAFT, requires approval)

Prepared immediately after Phase 10 closure (484 bodies / 493 regions).

THE PLANNING FINDING THAT DEFINES THE PHASE: the matched corpus is n=493 with a
median of 52 bytes, p90 108 bytes and MAX 244 bytes -- NOT ONE BODY LARGER THAN 244
BYTES HAS EVER MATCHED. Against a remaining worklist of 1193 rows:

  <=120 B     115 rows   proven-matchable band
  121-200 B   284 rows   proven-matchable band
  201-400 B   367 rows   partially proven (up to 244 B)
  401-800 B   245 rows   UNPROVEN -- nothing has ever matched here
  >800 B      182 rows   UNPROVEN

So the ~400-450 rows at <=244 B are the finite proven band, and +116 bodies means
matching a quarter to a third of it. The 244-byte ceiling is therefore the real
subject, and the plan states TWO goals: A consume the proven band (the milestone
path), B break the 244-byte ceiling (a bounded investigation with a measured
deliverable, where a result that adds zero bodies is still a met goal).

Structural change: the orchestrator spawns and retires its own workers via herdr
(4 agent panes per tab, 2x2 verified at 115x31 each); the developer spawns only the
orchestrator. Context management changes too: pi-context-tools is installed
globally, so every session has context_info and compact_context -- measurement is
exact and self-service, and COMPACTION replaces rotation as the first response to a
full context, with rotation second. That is only safe because the state lives in
files, so keeping the ledger current becomes a hard requirement.

Four decisions requested: the milestone number; whether Goal B is in scope this
phase; the 4-worker default with orchestrator discretion to add a fifth; and
confirmation of the compaction-first policy.

phase-ends/Phase11_PLAN.md

@@ -0,0 +1,180 @@
+# Phase 11 Plan — Breaking the 244-Byte Ceiling
+
+**Status:** DRAFT — requires explicit developer approval before activation.
+**Prepared:** 2026-09-24, immediately after Phase 10 closure (484 bodies / 493 regions).
+**Milestone:** **600 distinct matched bodies** from 484 (i.e. **+116**), with **700 as a stretch**.
+Measured on the orchestrator's clean whole-binary gate, never as "keep matching".
+
+## 0. The honest starting point — and the thing this phase is actually about
+
+Phase 10 closed at 484 bodies with the size-band law as its headline result. Measuring the corpus
+against the remaining pool produces two facts that define this phase:
+
+| | n | median | p90 | **max** |
+|---|---|---|---|---|
+| **matched corpus** | 493 regions | 52 B | 108 B | **244 B** |
+| remaining worklist | 1193 rows | — | — | — |
+
+**Not one body larger than 244 bytes has ever been matched.** Every one of the 493 matches is ≤244 B.
+
+The remaining pool by size:
+
+| band | rows | status |
+|---|---|---|
+| ≤120 B | 115 | proven-matchable band |
+| 121–200 B | 284 | proven-matchable band |
+| 201–400 B | 367 | **partially proven** (up to 244 B) |
+| 401–800 B | 245 | **unproven — nothing has ever matched here** |
+| >800 B | 182 | **unproven** |
+
+So the ≈400–450 rows at ≤244 B are the finite, proven band, and **+116 bodies means matching roughly a
+quarter to a third of it.** That is achievable at Phase 10's measured rates with a fourth worker and
+better context management — but it is the *end* of that band, and the 244-byte ceiling is the real
+subject of this phase.
+
+**Therefore Phase 11 has two goals, not one:**
+
+- **A — consume the remaining proven band** (the milestone path: +116 from ≤244 B).
+- **B — break the 244-byte ceiling** (the strategic path: find out whether the 401–800 B and >800 B
+  bands are tractable at all, and if so what it takes).
+
+Goal B is what makes 700 reachable and what a Phase 12 would build on. It is a **bounded
+investigation**, not open-ended matching, and it is stated as such below.
+
+## 1. Goal
+
+**Goal A (the milestone): 600 distinct bodies.** Consume the remaining ≤244 B rows, size-band-first,
+using the accumulated 57-finding lever set.
+
+**Goal B (the strategic result): a measured answer to why nothing above 244 B has matched.** The
+deliverable is *evidence*, and any of these outcomes is a success: (a) the band is tractable and here is
+the lever; (b) the band needs a specific harness change, named and demonstrated on at least one row;
+(c) the band is dominated by a class that cannot be reproduced with this toolchain, with the class
+characterised and counted. A goal-B result that adds zero bodies is still a met goal B.
+
+**Goal C (carry-forward, small): close the three named loose ends** — document `0x80107C5C` (which
+already matches but has no documented source), settle the two unproven opt-in maspsx modes, and either
+demonstrate or retire the scheduler class. Each is bounded and each is worth bodies.
+
+## 2. Structure — orchestrator + 4 workers, spawned by the orchestrator
+
+**This phase changes who runs the show.** The orchestrator spawns, names, monitors, and retires its own
+workers; the developer spawns only the orchestrator.
+
+- **Topology: one tab, four agent panes** (developer's recommendation), plus the orchestrator's own pane.
+  Layout: split right for the first worker, then split down for the rest, so no column becomes unusably
+  narrow. All panes share the project cwd.
+- **Spawn:** `herdr pane split --current --direction <right|down> --cwd "$PWD" --no-focus` → read the new
+  pane id from `.result.pane.pane_id` → `herdr agent start <name> --kind pi --pane <id>`. The default
+  model is already configured, so no model flag is needed.
+- **Communicate:** `herdr agent prompt <name> "<charter>"` to charter, `herdr agent read <name>` to
+  inspect, `herdr agent wait <name> --until idle --timeout N` to wait. **Intercom remains the reporting
+  channel** — it is already proven and the workers' reports arrive as messages — but herdr is the
+  control channel for spawn/retire/read.
+- **Retire:** when a worker has compacted to exhaustion or a band is done, read its final state, verify
+  its staging, then close its pane. Never close a pane the orchestrator did not create.
+- **Scaling:** the roster is **4 by default**. A fifth worker is added only if the measured
+  attempts-per-match says the pool supports it; workers are retired rather than left idle.
+
+## 3. Context management — the phase's biggest mechanical change
+
+`pi-context-tools` is installed **globally**, so every session (orchestrator and workers) has
+`context_info` and `compact_context`. This changes the rotation calculus that governed Phases 8–10:
+
+- **Measurement is now exact and self-service.** Every worker reports its context by calling
+  `context_info` rather than estimating, and the orchestrator quotes that figure. Phase 10 had a worker
+  estimate 87% when it was at 50%; that class of error disappears.
+- **`compact_context` replaces rotation as the first response to a full context.** A worker at 70–80%
+  compacts and continues, keeping its partition, its staging, and its learned levers. Rotation becomes
+  the *second* response — for a worker whose band is done or whose staging has gone stale.
+- **This is only safe because the state lives in files.** The ledger, `CURRENT_PHASE.md`, the tracked
+  negatives index and each worker's staging are current at every merge, so a compaction that summarises
+  away the conversation loses nothing that matters. **The ledger discipline is what makes compaction
+  safe**, and keeping it current becomes a hard requirement rather than good practice.
+- **The orchestrator compacts too** rather than stopping at the cap. Its gate/merge role is the one that
+  cannot be delegated, so it must be able to run the whole phase.
+
+## 4. The carried machinery (do not re-derive; verify before relying)
+
+- Toolchain: PsyQ 4.0 `CC1PSX` (GCC 2.7.2.SN32.3.7.0002); flags `-quiet -O2 -G0`; maspsx (ASPSX 2.56
+  emulator) then GNU `as`.
+- Harness: `tools/sf3_match` (per-region `cc1=`, `as=`, `gp=-NAME`, `maspsx=off|noreordernop|regread`),
+  `tools/sf3_diff`, `tools/sf3_merge`, `tools/sf3_triage` (with `restores_unsaved`).
+- **The tracked maspsx patch** (`tools/patches/maspsx-phase10-r1r2.patch`) — `tools/maspsx/` is
+  git-ignored, so the patch is the only reproducible carrier. Verify it is applied before any build.
+- Evidence tables: `function_inventory`, `function_extents`, `duplicate_bodies`, `match_worklist`,
+  `near_match_negatives` (194 rows at close).
+- **The hardened merge flow**: verify → merge to CANDIDATE → gate whole-binary → promote only on MATCH →
+  reconcile negatives → regenerate worklist + refilter partitions → commit → push. The gate rejected
+  three batches in Phase 10 and the registry was never corrupted.
+- **The size-band law** (cookbook 41) as the dispatch rule.
+- **The md5-in-claim-row guard** and the four process rules in cookbook 57.
+
+## 5. Rules already in force (carry into every charter)
+
+`git ls-files --error-unmatch` before any `rm` under `src/`; the dependent-claim rule; exact symbol-name
+spelling (the gp marker is name-keyed); compute-don't-eyeball; the leading-indicator rule; the
+three-encoding-trap habit; the amended register-field rule (**verify the dataflow before calling it a
+tie-break**); the two-step two-arm-branch rule (invert-and-swap first, then `goto`); the paired
+`&&`/named-boolean rule; **verify the file you are about to stage, at the path you are about to claim**;
+**run the `src/` cleanup audit before the final report**; **read start and end from the worklist row,
+never from a hand dump**; **one attempt on a named lever, then classify**; **fold region options into the
+claim row**; **record the source md5 per claim**.
+
+## 6. Verification ladder
+
+| Frequency | Checks |
+|---|---|
+| Every merge | candidate gate (whole-binary, SHA-1 fixed) → promote → `make check` |
+| Every 3rd merge | `make clean && make all`, `cmp`, SHA-1, registry audit, `git status --short src/`, firewall |
+| Every merge | worklist regen (`excluded_already_registered` = registry size) + partition refilter with the disjointness/union proof |
+| Every worker retire | staging verified, `src/` cleanup audit, negatives extracted into the tracked index |
+| Phase close | full clean gates, negatives extraction, milestone confirmation requested, PhaseEnd, digest, ledger, stop |
+
+**Push after every verified merge** (the developer's standing preference).
+
+## 7. Scope and safeguards
+
+- ROM/SDK firewall unchanged; worker staging ignored; `git clean -x`/`-fdx` forbidden.
+- Workers may not weaken a gate, widen regions, or register a non-`exact` extent.
+- **No scheduler-changing flag** (`-fno-schedule-insns` etc.) may be granted for a match. It is a
+  diagnosis-only tool; taking it would mean the corpus is no longer all built by one compiler
+  configuration. Workers may run it for diagnosis and must record the row as a negative.
+- Inline asm remains limited to shapes plain C **provably cannot express**, per-file documented.
+- The blocked classes (trapping arithmetic, the `0x80012xxx` primitive-init family, the maspsx
+  rare-epilogue mutual exclusion) stay excluded unless Goal B produces evidence that reopens them.
+- **Goal B is bounded**: two distinct root-cause attempts per hypothesis, then record. It is an
+  investigation with a deliverable, not an open-ended grind.
+
+## 8. Tasks
+
+- **P11-T1** — control records, clean-baseline revalidation, herdr roster spawn (4 workers), capability
+  probes including a `context_info` check, size-band-first partitions.
+- **P11-T2** — Goal A cycle 1 on the ≤244 B band: first checkpoint (+30 bodies).
+- **P11-T3** — Goal B opening: the 244-byte ceiling investigation. Characterise what the 401–800 B band
+  actually contains (tie-break classes, scheduling share, harness-blocked share) with counted evidence
+  from the payload and the extents table, and name the candidate levers.
+- **P11-T4 … T-n** — the autonomy loop: per-cycle regeneration, full audits every 3rd merge, workers
+  compacted rather than rotated, Goal B probed in bounded slices alongside Goal A.
+- **P11-Tn** — phase close: cookbook/conventions/verification records, negatives extraction, milestone
+  confirmation requested, PhaseEnd, digest, ledger archive, stop.
+
+## 9. Decisions requested from the developer
+
+1. **Milestone:** 600 with 700 as a stretch, or a different number.
+2. **Goal B scope:** is the bounded 244-byte-ceiling investigation in scope for this phase (it is the
+   route to 700), or should Phase 11 be Goal A only?
+3. **Roster:** 4 workers as the default, with the orchestrator free to spawn a fifth if the measured rate
+   supports it.
+4. **Compaction:** confirm that workers compact and continue rather than rotating at the cap, and that
+   the orchestrator does the same.
+
+## 10. Preconditions
+
+- `make check` green at 493 regions / 484 bodies (Phase 10 close state), with the tracked maspsx patch
+  applied.
+- The Phase 10 closure commit on `main` and pushed.
+- herdr roster spawned and probed; `context_info` confirmed working in every worker.
+
+Milestone: **600 distinct matched bodies on the clean whole-binary gate, plus a measured Goal B answer** —
+or the evidence-backed blocker recorded.