From 4a731de2d5334da56773da896c0da349fd28b55c Mon Sep 17 00:00:00 2001 From: Christopher Williams Date: Thu, 24 Sep 2026 08:24:53 -0400 Subject: [PATCH] =?UTF-8?q?phase10:=20CLOSE=20=E2=80=94=20PhaseEnd,=20dige?= =?UTF-8?q?st,=20verification=20record,=20cookbook=2041-57?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit MILESTONE MET AND EXCEEDED: 484 distinct matched bodies / 493 registered regions (target 475, from the 400 baseline) — +84 bodies. Developer confirmation of the milestone was requested and given before any close record was written. Closing checklist all green from clean: make clean && make all exit 0 cmp exit 0 SHA-1 both files e173426c157384ebf1b6caf8c6fea18a85a14af9 make test 237 tests, OK make extents-verify regions=493 disagreements=0 AGREE make gate c_regions=493 differing_bytes=0 MATCH registry audit 493 rows, 0 overlaps, 0 unsorted, 0 bad extents, 0 missing sources, 484 distinct sources worklist listed=1193, excluded_already_registered=493 negatives index 194 rows, address-ordered, 0 registered git status --short src/ empty (0 untracked files) firewall 0 prohibited-root paths (591 tracked files) New records: phase-ends/PhaseEnd_Phase10.md the phase record docs/PHASE10_VERIFICATION.md the verification record docs/MATCHING_COOKBOOK.md findings 41-57 (57 total) phase-ends/CURRENT_PHASE.md CLOSED, with the checklist itemised phase-ends/DIGEST.md the Phase 10 digest entry The headline finding is methodological (finding 41, THE SIZE-BAND LAW): the matched corpus median is 48 bytes with 454/459 at <=200 B while the remaining levered rows had a median of 456 B, and two independent measurements — one controlled — put the small band at 1-2 attempts per row against 1-in-12 for 200-800 B. The phase's character: five of the coordinator's own generalisations were bounded by workers (rare-epilogue class, register-field diagnostic, polarity lever, goto trigger, load-delay consumer form). The rules that survived are the ones that were bounded. Five incidents recorded rather than smoothed over; the candidate gate rejected three batches and the tracked registry was never corrupted. Scope held: the blocked classes stay excluded, no scheduler-changing flag was granted, and inline asm was extended only to shapes C provably cannot express. STOPPING HERE. Phase 11 does not begin in this session. --- docs/MATCHING_COOKBOOK.md | 236 +++++++++++++++++++++++++++++++++ docs/PHASE10_VERIFICATION.md | 104 +++++++++++++++ phase-ends/CURRENT_PHASE.md | 65 ++++++++- phase-ends/DIGEST.md | 95 +++++++++++++ phase-ends/PhaseEnd_Phase10.md | 165 +++++++++++++++++++++++ 5 files changed, 661 insertions(+), 4 deletions(-) create mode 100644 docs/PHASE10_VERIFICATION.md create mode 100644 phase-ends/PhaseEnd_Phase10.md diff --git a/docs/MATCHING_COOKBOOK.md b/docs/MATCHING_COOKBOOK.md index 4900266..7e9a5a3 100644 --- a/docs/MATCHING_COOKBOOK.md +++ b/docs/MATCHING_COOKBOOK.md @@ -683,3 +683,239 @@ taken in Phase 10. The two opt-in maspsx modes shipped in Phase 10 (`maspsx=nore `maspsx=regread`) are implemented and default-off but **neither has been shown to close a region**: on `0x800FFF60` R1 makes the length *worse* (140 → 128) because it also removes `nop`s the original needs, and R2's predicate did not fire. + +## Phase 10 — findings 41+ (coordinated tail squeeze, 2026-09-24) + +Phase 10 took the corpus from 400 to **484 distinct bodies / 493 regions** with the full clean +audit green. Its headline result is methodological, and most of its levers are *two-sided* — each +acquired its limit by being tested against a row where it failed. + +### 41. THE SIZE-BAND LAW — the phase's headline finding + +**The matched corpus is 484 regions with a median size of 48 bytes and 454 of 459 at ≤200 B; the +remaining levered rows have a median of 456 B. Size is the strongest predictor of yield left in the +pool.** Two independent measurements, one of them controlled: + +| source | ≤200 B | 200 B–800 B | +|---|---|---| +| worker C — same worker, same levers, same day, band the only variable | **1, 1, 1, 2** attempts | **1-in-12** | +| worker B2 — within a single session | **1, 2, 2** attempts (claims 5-7) | the two >200 B rows it spent the *most* time on: **1 deferred + 1 harness, zero matches** | + +B2's version is the more persuasive because the confound of "different workers" is absent. +*Limit:* the band is where matching has been happening, not a claim that large bodies are +unmatchable — every measured residual had an identified mechanism. But when a worker-hour must be +placed, this is the number to use. + +### 42. The commutative-operand lever, and its coupled allocation side-effect + +`addu` takes the register of its **first dying source operand**, so `base + (index << 4)` and +`(index << 4) + base` allocate differently despite being mathematically identical. Diagnostic: count +right + a residual that is a small multiple of 4 in ONE basic block → compare **register fields** per +instruction before touching the expression shape. + +**COUPLING (Phase 10).** On `0x80027744` (4 bytes) and `0x80050674` (1 byte, only the register fields +of one `addu`) the operand order and the register allocation are **coupled**: every spelling giving the +original's `addu` operand order also flips which value takes `v0` vs `v1`, because operand order +changes the pseudo creation order. `base + a1*4` gives the right allocation and the wrong destination; +`a1*4 + base` the reverse. So re-spelling the addition is **necessary but not sufficient** — look for a +shape that keeps the index in `v1` while the base is still live at the addition. + +**AMENDMENT — A PERMUTED REGISTER SET CAN BE A SEMANTIC BUG (worker A).** The register-field +diagnostic was broadcast as "permuted registers = the commutative lever", and that generality is +**false**. On `0x800A5180` the source wrote `q[1]` where the original read `entry[1]` through the +table-entry pointer: right length, identical mnemonics and schedule, 11 differing bytes all in register +fields, and **not** an allocation tie-break. **Verify the DATAFLOW — which value each register actually +holds — before concluding tie-break.** Two different pointers swapping registers are indistinguishable +from register names alone. + +### 43. The two-arm branch family — three triggers, and the first two are ordered + +1. **POLARITY (try first).** For `a1 = (x < 2) ? a3 : saved`, both the natural order and the ternary + emit `beq v0,zero` with the arms swapped (right length, 5 differing bytes). Writing the + larger-than arm first — `if (x >= 2) a1 = saved; else a1 = a3;` — makes cc1 emit `bne v0,zero` with + `a1 = a3` in the **branch delay slot**, running on both paths and overwritten on the `>= 2` path: + the original exactly. +2. **A GUARD THAT MUST BRANCH *INTO* THE BODY NEEDS A `goto` (try second).** On `0x800A9C24` both + ordinary spellings produce the mirrored `beq v1,zero` at 96 differing bytes, and only + `if (a1 != 0) goto body; return -1; body:` gives the original's `bne v1,zero` with the `return -1` + block inline as the fall-through. +3. **SHARING ONE BLOCK NEEDS A NAMED `goto` LABEL.** On `0x800A6998` the two `func_800A6FAC(0)` calls + appear **exactly once** in the original, reached both by fall-through and by the `w != 0` branch. + `while`/`for` spellings either **invert the loop** (156 B) or make cc1 **cross-jump the two `== 0` + tests** so the x-load goes dead (124 B). + +*Limit (worker A, `0x800256F0`):* trigger 2 is **necessary but not sufficient** — the branch went into +the body and the ORDINARY spelling still reproduced the arm placement correctly. Keep the two-step +order and do not expect a `goto` every time the layout looks that way. + +### 44. The paired boolean rule — combine, or name, according to the original's shape + +- **Original has a BRANCH on a boolean expression → combine the conditions into ONE `&&` chain.** + Three separate `if (...) return 0;` statements let cc1 fold the third guard into a branchless + `xor`/`sltiu` (124 B); one `&&` chain with a single trailing `return 0` keeps the branch. + (`0x800B67B8`; also `0x8006AA10`, where nested `if`s give two `li a3,1` blocks and 124 B + LENGTH-MISMATCH versus one shared `li` for the combined form.) +- **Original has the BRANCHLESS form → NAME the boolean.** `rec[6] = ((a3 & 0xff) != 0) << 1` is + branchy at 92 B; only `int flag = (a3 & 0xff) != 0;` then `rec[6] = flag << 1` gives the original's + `andi`/`sltu`/`sll` at 88. Six spellings measured. + +*Limit:* on `0x800B67B8` a `goto fail; … fail: return 0;` spelling is **also** byte-identical, so the +distinguishing fact is **"one combined condition with one trailing `return`"**, not the operator. + +### 45. The named-locals family — three directions, plus scope + +The family has **five** distinct mechanisms. The three *directions* around one mechanism, all +byte-required, all measured: + +| direction | row | what the source must do | +|---|---|---| +| names it → cc1 keeps it | `0x8005E538` | the handle must be a named local loaded before the first call, or cc1 keeps only the object pointer in a callee-saved register and **reloads** the handle after the call (76 vs 88 B) | +| re-reads it → cc1 re-reads it | `0x800F66B8` | `d[0]`/`d[1]` are each loaded twice with neither held in a register — naming a local would be **wrong** | +| re-reads it → cc1 CSEs it anyway | `0x800256F0` | the original re-reads a field after a call; cc1 CSE'd the two reads | + +**Diagnostic: the load count, not a rule about locals.** + +Plus: **order-pinning** (a named local fixes a store order), **reload-prevention** (a local is not +memory, so a store through a possibly-aliasing pointer cannot invalidate it), the **inverted** case +where the repeated expression is correct and the local is the mistake, and the **scoping** lever — +a local shared by two guard blocks gets its live ranges **coalesced**; declaring each in its own +brace-scoped block prevents that (`0x8008BA80`). + +### 46. The per-site `gp` form, and its double-failure case + +The gp marker is a **per-symbol** property in the harness but a **per-access** property in the +original, so a registry-gp-marked symbol may need a per-region `gp=-NAME` override. The strongest +evidence is a single worker's batch: `D_80121BFC` is read **gp-relatively** in one row +(`lw v1,708(gp)`) and **absolutely** in another (`lui v1,0x8012` + `lw v1,7164(v1)`). + +**On `0x800A6998` BOTH spellings are wrong, for two independent reasons:** the **symbol** spelling +gives the gp-relative encoding the original does not use, while the **literal** spelling makes cc1 +**cache the address in `s0`** so the frame grows 24 → 32 bytes with an extra saved register. The +per-site override is the only correct route. + +### 47. The `-G` small-data threshold lever + +A symbol whose **address is taken** is invisible to `-G0`: cc1 emits a two-instruction large-data `la` +and CSE-hoists it into a callee-saved register (candidate LONGER by ~12 with an extra saved register +holding an `la`). A `-G` that admits the symbol makes cc1 emit one `la` per use with no hoist. Rule: +**the symbol is admitted as small data exactly when `declared size <= G`.** + +*Limit:* the harness's `-G0` + per-symbol gp marker model is exact for a direct `lw`/`sw`, which is why +400 bodies matched without any `-G` override; it is inexact only when the symbol's ADDRESS is taken. +Override-free routes were tested and closed (known-size declaration at `-G0` still mismatches; +`__attribute__((section(".sdata")))` still mismatches). **`-G8` is a working value, not a recovered +one** — the true `-G` is only bounded as `>= 4` by this row. The global `-G0` default is unchanged by +developer decision, so the existing corpus is untouched. + +### 48. Signedness shows up as an encoding — four forms + +- `addiu ` = signed destination, `ori ` = unsigned (`0x8010A940`: `0xC000` needs + `unsigned short`). +- A 1-byte residual on a `li`/`addiu` immediate whose **low byte is unchanged** is a signedness + difference, not a value difference (`0x80073F78`: `*(signed char *)` loads `-30`/`0xFFFFFFE2` where + `char *` loads `226`/`0x000000E2`). +- **Within one function**, two constants can need opposite signedness (`0x8008A758`: `0xC0FFFFU` needs + `ori`, `480` needs `addiu` — same immediate bits, one differing byte). +- The **loop-test** form: `sltiu` vs `slti` (opcode 0x0b vs 0x0a) means an `unsigned` counter + (`0x800504E4`). + +### 49. A self-inflicted typo presents as a 1-byte diff — and compute-don't-eyeball is the fix + +The harness's name-encoded symbol resolution is **faithful to what you typed**, so a mis-converted +displacement encodes the wrong address and presents as a mysterious 1-byte diff (`0x8008DB44`: +`addiu a2,a2,32140` vs `32172` — the source declared `D_80117DAC` for `D_80117D8C`). **When the +residual is one instruction's immediate, recompute the address from the instruction's OWN immediate +before theorising.** Four instances this phase; two of them were the same class read as decimals +(`0x80091370`'s -30200/-30172 are the `%lo` of `0x80138A08`/`0x80138A24`; `0x800ACA10`'s 28220 is +`0x6E3C`, not `0x6E7C`). + +### 50. The scheduler class — `-fno-schedule-insns` is the one-compile confirmation + +**Correct length + a residual that is a permutation of a few instructions + the unscheduled build +matching the original order ⇒ the residual is sched, not source shape** (worker C2). Measured on +`0x800A6C34` (16 differing) and `0x80016F80` (45 differing), both at correct length: the identical +source with `cc1 -quiet -O2 -G0 -fno-schedule-insns` produces the original's instruction ORDER +byte-for-byte. + +**The trap is that the two effects are coupled and opposed:** on `0x80016F80` sched-OFF gives the +original's order but the wrong register allocation (`i→s0/p→s1` vs the original's `i→s1/p→s0`), while +sched-ON gives the right allocation and the wrong order. On `0x800A6C34` sched-OFF gives the original's +load/store order but reorg then fills the `jal` slot the original leaves empty for maspsx's `nop`. +**Neither alone matches.** + +*Limit:* the override is for **diagnosis only**. Taking it would mean the corpus is no longer all built +by one compiler configuration. Record the row as a negative with the lever named. + +### 51. Dead-store elimination, and when `volatile` is the answer + +Two stores to the same address are DSE'd and cc1 deletes the first store **and its `andi`** (144 B vs +the correct 156) — and **a literal-address second store does NOT defeat the DSE**. Measured set +(`0x80016F80`): the repeated stores must be `volatile`; the ten zero stores must be `volatile` too or +the scheduler floats them above the mask block (68 → 45 differing); with both volatile the two `andi`s +stay adjacent only if the masked values are bound to **temporaries before the stores**; and the loop +bound must stay **non**-volatile or its load moves after the zero stores (160 B). + +### 52. maspsx drops an explicit `#nop` marker it should honour + +cc1 emits a bare `#nop` when it wants a load-delay nop. maspsx re-derived the need and could **overrule +cc1** for a **bare-symbol store** consumer: `uses_at('sw $2,D_801221C4')` is True (the store expands +via `$at`) while `nop_at_expansion` is False for ASPSX >= 2.30, so neither test fired and the nop was +dropped (`0x80107C5C` at 108 vs 112; `0x8003A9C8`). **Fixed in Phase 10** (tracked patch +`tools/patches/maspsx-phase10-r1r2.patch`): maspsx now honours the explicit marker. Verified +regression-free at 489 regions. + +*Limit and lesson:* the predicate `line_loads_from_reg` **already returned True** for a store source — +the worker's first diagnosis (extend the predicate) would have been a **no-op patch**. **A named +mechanism is a hypothesis until it is traced, even when the observation is solid and reproducible.** +The two other opt-in modes shipped in the same patch (`maspsx=noreordernop`, `maspsx=regread`) remain +unproven: neither has been shown to close a region. + +### 53. The record pointer must be the BASE, not an offset one + +`p = (unsigned char *)(node[0] + 6)` with `p[-2]/p[-1]/p[0]` makes cc1's combine pass split it into +**two** pointers (`addiu` to base+6 AND base+5) — 124 B, LENGTH-MISMATCH. `p = node[0]` with +`p[4]/p[5]/p[6]` reproduces the original's single `addiu a1,v1,6` base with -2/-1/0 offsets. +*Diagnostic:* candidate LONGER by 4 with two `addiu`s from one base where the original has one → +suspect the base choice, not the stride. + +### 54. A 2-D array's row stride is byte-load-bearing (a LENGTH-class lever) + +`D_8013C078[a3][a4]` emits `sll a3,4` + `sll a4,2` + `add` and the length is correct (120 B); +`D_8013C078[a3 * 4 + a4]` folds the outer `*4` into a second `sll` and the row comes out **4 bytes +SHORT**. *Diagnostic:* when a scaled index is one `sll` short, the source is a 2-D array whose row +stride the compiler can use directly, not a flattened index. + +### 55. The family lever's scope, measured on a large sample + +The argument-map template **transfers**; the body shape **does not**. Worker C matched `0x80027E1C` +(frame -48) and `0x80099078` (frame -40) first-attempt from the frame-size-as-shape-signature map, +then the map stopped transferring: the -48 siblings `0x800BD5E8` (380 B) and `0x800BE828` (416 B) are +`mult`-heavy vector maths, the -40 sibling `0x80052424` (488 B) is a 4-argument nested branch tree, and +`0x80015FC8` (288 B) carries a `div` with `break` guards. **The remaining cluster rows are structurally +distinct bodies that merely CALL the same target.** + +### 56. The `restores_unsaved` fragment class + +A body that **restores a callee-saved register it never saves** cannot be a whole function: the +register was established by an enclosing prologue the derived extent cut off. These are `jal` targets +*inside* a real function, so the extent walk began mid-body — distinct from `bad_extent_start`, which +flags starts that are not function entries at all. **Verified disjoint from the corpus: 0 of 462 +registered regions trip it; 11 worklist rows do.** Implemented in `sf3_triage` as a counted reason. +*Limit:* sufficient but not complete — it catches only rows that RESTORE an unsaved register, not rows +that merely read one, so the count is a lower bound. + +### 57. Process rules the phase earned + +- **Claim rows carry the source md5.** `src/func_XXXXXXXX.c` is a shared, unlocked namespace across + concurrent sessions, so a file can change **between a worker's verify and the coordinator's merge**. + A mismatch means re-verify rather than merge or reject. ("Verify the file at the path you are about + to claim" is necessary but **not sufficient** — the gap is between verify and merge.) +- **A variant is an experiment; `src/func_XXXXXXXX.c` is the claim.** Fold a winning spelling from a + scratch variant into the documented source before claiming. +- **Run the `src/` cleanup audit BEFORE the final report**, not after — a finding that is not also a + claim is invisible to the merge flow. +- **Read start and end straight out of the worklist row**, never reconstruct them from a hand dump: a + hand-derived range produces a LENGTH-MISMATCH indistinguishable from a real extent defect. +- **One attempt on a named lever, then classify.** A harness classification backed by a proof is worth + more than an assumption; the model case is `0x80023D40`, where cc1's duplicated `slti` in the `bgez` + slot is **correct code**, so no source shape can prevent it. diff --git a/docs/PHASE10_VERIFICATION.md b/docs/PHASE10_VERIFICATION.md new file mode 100644 index 0000000..bf55fc1 --- /dev/null +++ b/docs/PHASE10_VERIFICATION.md @@ -0,0 +1,104 @@ +# Phase 10 — Verification Record + +**Scope:** the checks that establish Phase 10's result. Every number below was produced by a command +run in this repository against the tracked state, not estimated. + +**Result: 484 distinct matched bodies / 493 registered regions** (from the Phase 9 close state of +400 / 409), all gates green from a clean tree. + +## The closing gate set (run from clean, in this order) + +| Check | Command | Result | +|---|---|---| +| Clean rebuild | `make clean && make all` | exit 0 | +| Byte comparison | `cmp build/scus_946_40.rebuilt 'extracted/SCUS_946.40;1'` | exit 0 | +| Full-file SHA-1 (both) | `sha1sum` | `e173426c157384ebf1b6caf8c6fea18a85a14af9` | +| Synthetic suite | `make test` | **237 tests, OK** | +| Extent agreement | `make extents-verify` | `regions=493 disagreements=0 result=AGREE` | +| Whole-binary gate | `make gate` | `c_regions=493 differing_bytes=0 result=MATCH` | +| Registry audit | scripted | 493 rows, 0 overlaps, 0 unsorted, 0 bad extents, 0 missing sources, 484 distinct sources | +| Worklist regeneration | `make worklist` | `listed=1193`, `excluded_already_registered=493` (= the registry size) | +| Negatives reconciled | scripted | index 194 rows, address-ordered, 0 duplicates, 0 registered; **0 unregistered negatives survive into the worklist** | +| `src/` cleanliness | `git status --short src/` | **empty** — 0 untracked files | +| Registry ↔ tracked sources | scripted | 484 distinct registry sources = 484 tracked `src/` files | +| Firewall | `git ls-files` under prohibited roots | **0** (591 tracked files) | + +## Per-merge verification (every merge, all 27) + +Every merge followed the hardened flow and the **candidate gate ran before promotion in every case**: + +1. `sf3_merge apply` → candidate regions + symbols (never the tracked registry) +2. `sf3_match gate` on the candidate, whole-binary, expecting the fixed SHA-1 +3. only on `result=MATCH`: promote, then `make check` +4. reconcile negatives, regenerate the worklist, refilter partitions, commit, push + +The gate rejected three batches during the phase, and **the tracked registry was never corrupted**: + +| Batch | Failure | Cause | Resolution | +|---|---|---|---| +| merge 8 | `differing_bytes=1117197` | a region-option request staged in a separate file rather than in the claim row, so the region merged without its `gp` override | folded the option into the claim row, re-gated to MATCH | +| merge 19/20 | `differing_bytes=96` | the claimed `src/` path held a different spelling from the one verified (a bare variant, not the documented source) | returned to the worker; the documented source was restored and re-verified | +| merge 20 | `differing_bytes=96` again | a **concurrent write** to the same `src/` path by another session (a coordinator double-assignment) | md5-in-claim-row guard adopted; the row was reassigned to one owner | + +## Audit cadence actually achieved + +- **Full clean audit** (`make clean && make all`, `cmp`, SHA-1, registry audit, `git status --short src/`, + firewall) at the milestone and at the close, plus at the 3rd-merge interval. +- **Every merge**: candidate gate + `make check`. +- **Every merge**: worklist regeneration and partition refiltering, with the disjointness and + union-equals-worklist proof re-run (`intersection = ∅`, `union == worklist`). +- **Negative reconciliation** run repeatedly during the phase rather than only at the close, so no + worker finding depended on ignored staging surviving. The final extraction imported 28 negatives and + dropped 28 rows that had since been registered. + +## Harness changes and their verification + +| Change | Tracked as | Verified by | +|---|---|---| +| `maspsx` opt-in modes `noreordernop` / `regread` | `tools/patches/maspsx-phase10-r1r2.patch` | patch reproduces both modified files byte-identically from the pristine pinned checkout; `make check` green at 493 with the modes off | +| **maspsx honours cc1's explicit `#nop` marker** | same patch (default-on) | **`make check` green at 489 regions / 237 tests with every one of the 489 regions byte-identical**, so the fix is regression-free | +| `restores_unsaved` exclusion class | `tools/sf3_triage` + 5 new tests | measured disjoint from the corpus: **0 of 462 registered regions** trip it, 11 worklist rows do | +| per-region `cc1=` / `gp=` override plumbing | `tools/sf3_match` + 5 new tests | suite 229 → 237, all green | + +`tools/maspsx/` is **git-ignored**, so an in-place edit would not survive a fresh clone. Every maspsx +change is therefore carried as a tracked patch and recorded in `docs/SETUP.md` with the apply command. + +## Registry-option decisions (each byte-verified with a failing control) + +| Row | Option | Without it | With it | +|---|---|---|---| +| `0x800A6BEC` | `cc1=-G8` | 84 B LENGTH-MISMATCH | 72 B, 0 differing | +| `0x80015D50` | `gp=-D_80121B88` | 104 B LENGTH-MISMATCH | 108 B, 0 differing | +| `0x8002D364` | `gp=-D_801226F4` | 140 B LENGTH-MISMATCH | 144 B, 0 differing | +| `0x800A6998` | `gp=-D_80121B88` | 120 B LENGTH-MISMATCH | 128 B, 0 differing | +| `0x80048128` | `gp=-D_80121BFC` | 88 B without the override is the MATCH; the symbol spelling is wrong | 88 B, 0 differing | + +No option was granted on a hunch: each was reproduced by the coordinator with a control that fails +without it. **No scheduler-changing flag was granted at any point** — a flag that changed cc1's +scheduler would mean the corpus is no longer all built by one compiler configuration. + +## Scoring honesty + +The milestone is **bodies on the clean whole-binary gate**, never "keep matching". A classified but +unmatched residual counts as a **recorded negative, not progress**. The phase therefore reports: + +- **484 matched bodies** (the milestone metric, target 475) +- **194 recorded negatives** in the tracked index, each with an address, size, status and class +- **2 harness rows with proven impossibility arguments** (`0x80023D40` — cc1's duplicated `slti` in the + `bgez` slot is *correct code*, so no source shape can prevent it; `0x800FBB20`) +- **the blocked classes unchanged**: trapping arithmetic, the `0x80012xxx` primitive-init family, and + the maspsx mutual exclusion all remain excluded with counted reasons, 0 of 493 registered regions + containing any of them + +## Known limits of this verification + +- The `restores_unsaved` and `classify_epilogue` scans are **lower bounds**: they catch rows that + restore an unsaved register / that have `jr ra` as the third tail word, not rows that merely read an + unsaved register or that restore two or more registers before the jump. +- `0x8010080C` remains a **false extent start** whose root cause (a `jal` target mid-body) is recorded + but not fixed in `sf3_extents`; a hand-edited extent would fail `make extents-verify` by design. +- `0x80107C5C` **matches** with the `#nop` fix (verified against worker B2's variant `X3.c`, 112 B, 0 + differing) but its source is a bare variant with no header, so it is recorded as unblocked-and-one- + documented-source-away rather than claimed. It is not in the 484. +- The two opt-in maspsx modes shipped alongside the `#nop` fix remain **unproven**: neither has been + shown to close a region. diff --git a/phase-ends/CURRENT_PHASE.md b/phase-ends/CURRENT_PHASE.md index 5dfa3c6..86e1973 100644 --- a/phase-ends/CURRENT_PHASE.md +++ b/phase-ends/CURRENT_PHASE.md @@ -1,10 +1,67 @@ # CURRENT_PHASE — Phase 10: Matching the Remaining Pool (Goal A — tail squeeze) -**Status:** ACTIVE (P10-T1 complete; P10-T2 ready to dispatch) +**Status:** **CLOSED** (2026-09-24) — milestone **MET at 484 distinct matched bodies / 493 regions** +(target 475, from the 400 baseline), with the developer's explicit confirmation of the milestone +given before any close record was written. PhaseEnd: `phase-ends/PhaseEnd_Phase10.md`. Verification: +`docs/PHASE10_VERIFICATION.md`. Ledger: `phase-ends/logs/Phase10.md`. **No further work is active in +this phase.** **Created:** 2026-09-24 (coordinating session `01a0d302-0201`, after developer approval) -**Plan:** `phase-ends/Phase10_PLAN.md` — approved 2026-09-24, **Goal A: 475 distinct -matched bodies** from the Phase 9 close state of **400** (i.e. **+75 new bodies**), -measured as bodies on the coordinator's clean whole-binary gate. +**Plan:** `phase-ends/Phase10_PLAN.md` — approved 2026-09-24, **Goal A: 475 distinct matched bodies**. + +## Final state + +| | Phase 9 close | Phase 10 close | +|---|---|---| +| Distinct matched bodies | 400 | **484** | +| Registered regions | 409 | **493** | +| Synthetic tests | 229 | **237** | +| Tracked files | 506 | 591 | + +All gates green from clean: `make clean && make all` exit 0, `cmp` exit 0, both SHA-1 +`e173426c157384ebf1b6caf8c6fea18a85a14af9`, `make test` 237 OK, `make extents-verify` +`regions=493 disagreements=0 AGREE`, `make gate` `c_regions=493 differing_bytes=0 MATCH`, registry audit +493 rows / 0 overlaps / 0 bad extents / 0 missing sources / 484 distinct, worklist `listed=1193` with +`excluded_already_registered=493`, negatives index 194 rows address-ordered with 0 registered, +`git status --short src/` empty, 0 firewall violations. + +## Sessions (all released) + +| Role | Short id | Claims | Outcome | +|---|---|---|---| +| **Coordinator** | `01a0d302-0201` | — | 27 merges, all gated; close records written | +| Worker A | `01a0d302-18e3` | 26 | final handoff accepted | +| Worker B | `01a0d302-8a6f` | 23 | stopped on a yield basis at 47% (recorded as **not** the measured criterion) | +| Worker B2 | `01a0d338-485d` | 9 | replacement for B; stopped clean | +| Worker C | `01a0d302-fad3` | 22 | stopped at the measured 68% cap | +| Worker C2 | `01a0d344-0c94` | 4 | replacement for C; stopped on budget | + +## Closing checklist — all items verified + +| # | Item | State | +|---|---|---| +| 1 | Every worker: final report, drafts removed from `src/`, complete negatives list | done (A, B, B2, C, C2 all released with handoffs) | +| 2 | `git status --short src/` = 0 | **empty** | +| 3 | Registry audit: ordered, non-overlapping, extents exact, sources present | 493 rows, 0 violations, 484 distinct | +| 4 | All gates from clean | all exit 0 (see above) | +| 5 | Worklist regenerates with `excluded_already_registered` = registry size | `493` = `493` | +| 6 | Negative metadata extracted before ignored staging is lost | index 194 rows; 0 unregistered negatives survive into the worklist | +| 7 | Firewall and Git review | 0 prohibited-root paths; history audited before the first push | +| 8 | Milestone confirmation requested | **requested and given** | +| 9 | PhaseEnd, digest update, ledger, commit, stop | written; **stopped without beginning Phase 11** | + +## Carried into Phase 11 + +- **The size-band law** (cookbook finding 41) as the dispatch rule: ≤200 B yields at 1–2 attempts per row + against 1-in-12 for 200 B–800 B. +- The **md5-in-claim-row guard** and the four process rules in cookbook finding 57. +- The **negatives index** (194 rows) as the queue-reconciliation input. +- The **tracked maspsx patch** (`tools/patches/maspsx-phase10-r1r2.patch`) with `SETUP.md` provenance — + `tools/maspsx/` is git-ignored, so the patch is the only reproducible carrier of those changes. +- Worker handoffs with ranked near-matches: worker A's 13 and worker C2's 4, each with an exact residual + and a named untried lever. +- Unresolved: the `0x8010080C` false extent start; the maspsx rare-epilogue mutual exclusion; the two + opt-in maspsx modes (unproven); the scheduler class; and `0x80107C5C` (matches, one documented source + away, not counted in the 484). ## Sessions diff --git a/phase-ends/DIGEST.md b/phase-ends/DIGEST.md index c235723..0ecbadd 100644 --- a/phase-ends/DIGEST.md +++ b/phase-ends/DIGEST.md @@ -113,3 +113,98 @@ the dependent-claim rule, the leading-indicator rule, and compute-don't- eyeball (extended to two-piece constants). Next: Phase 10 plan is written (`phase-ends/Phase10_PLAN.md`) and requires explicit developer approval. No changes to AGENTS.md. + +## Phase 10 — Matching the Remaining Pool, Goal A (2026-09-24) + +Phase 10 ran a tail squeeze from the Phase 9 close state and closed at **484 distinct matched bodies / +493 registered regions** (from 400 / 409 — **+84**, against a milestone of 475, so **+9 over target**), +verified on the coordinator's own clean whole-binary gate (`c_regions=493`, `differing_bytes=0`, SHA-1 +`e173426c157384ebf1b6caf8c6fea18a85a14af9`). All gates exit 0 from clean: `make clean && make all`, +`cmp`, `make test` (237 tests, up from 229), `make extents-verify` (AGREE), `make gate` (MATCH); the +registry audit reports 493 ordered non-overlapping regions, 0 bad extents, 0 missing sources and 484 +distinct sources; `git status --short src/` is **empty** (0 untracked files, so 484 registry sources = +484 tracked `src/` files); the worklist regenerates with `excluded_already_registered=493`; and 0 tracked +paths sit under any prohibited root (591 tracked files). + +**The phase's headline finding is methodological: THE SIZE-BAND LAW.** The matched corpus has a median +size of 48 bytes with 454 of 459 at ≤200 B, while the remaining levered rows had a median of 456 B, so +size is the strongest predictor of yield left in the pool. Two independent measurements, one controlled +(same worker, same levers, same day, band the only variable): **1, 1, 1, 2 attempts per row on the +≤200 B band against 1-in-12 on the 200 B–800 B band**; and within one session, **1, 2, 2** on the small +band against the two >200 B rows it spent the most time on producing **1 deferred + 1 harness row and +zero matches**. Acting on this mid-phase — every dispatch file re-ranked to size-band-first — caught a +worker heading back into the band it had just measured as exhausted. It is cookbook finding 41. + +**The phase's character is that every lever acquired a limit, and five of the coordinator's own +generalisations were bounded by workers** — more than in any prior phase. "Skip the rare-epilogue class" +was falsified (it is a three-way split and attemptable). "A permuted register set = the commutative +lever" was falsified (it can be a *semantic bug* — two pointers swapping registers look identical from +register names, so **verify the dataflow first**). "Residual at a two-arm branch → invert and swap" was +bounded (both ordinary spellings can mirror, so the guard may need a `goto`). "The `goto` trigger needs a +`goto`" was bounded (necessary but **not sufficient**). And a coordinator "negative result" on the +load-delay class was shown to have tested the wrong consumer form. **The rules that survived are the +ones that were bounded**, and that is the phase's most transferable process result. + +**The rare-epilogue class (finding 35) is mechanistically closed.** GNU `as` in reorder mode fills a jump +delay slot with the immediately-preceding instruction but **refuses when doing so would place `jr ra` in +the load-delay slot of `lw ra`**, so the class splits on whether any instruction intervenes before the +frame release. The obvious fix was measured and closed: maspsx forces each function into +`.set noreorder`, so suppressing only its jump-slot `nop` restores the length but leaves the slot +**empty**; also suppressing the function-level `.set noreorder` makes `as` fill the epilogue *exactly* +right and then over-fill other slots for +4 bytes. A tracked post-pass modelling ASPSX's fill for one +site is the remaining route and was not taken. + +**Harness work, all verified.** A new **`restores_unsaved` exclusion class** in `sf3_triage` (a body +restoring a callee-saved register it never saves cannot be a whole function; verified disjoint from the +corpus — 0 of 462 registered regions, 11 worklist rows — and it independently re-derived a defect another +worker had found by a different signal). A **maspsx fix honouring cc1's explicit `#nop` marker** +(developer-authorised): cc1 asked for a load-delay nop and maspsx overruled it for bare-symbol store +consumers, because `uses_at` is True for a macro store while `nop_at_expansion` is False for ASPSX +≥ 2.30; **verified regression-free at 489 regions, every one byte-identical**. Worker B2 found the gap +but its first diagnosis (extend the predicate) would have been a **no-op** — the predicate already +returned True — and the coordinator traced the real mechanism: **a named mechanism is a hypothesis until +it is traced, even when the observation is solid**. Because `tools/maspsx/` is **git-ignored**, every +maspsx change is carried as a **tracked patch** verified to reproduce the modified files byte-identically +from the pristine pinned checkout, with provenance in `docs/SETUP.md` — an in-place edit would not have +survived a fresh clone, and that was caught before it mattered. Per-region override plumbing was added +for the new modes (+5 tests). + +**Five incidents were recorded rather than smoothed over, and none corrupted the tracked registry.** +A **coordinator double-assignment** (one row redistributed to one worker and chartered to another, so two +sessions wrote one file) produced the **md5-in-claim-row guard**, now standing protocol, because +`src/func_XXXXXXXX.c` is a shared unlocked namespace and a file can change *between a worker's verify and +the coordinator's merge* — so "verify the file at the path you are about to claim" is necessary but **not +sufficient**. A worker verified a variant and claimed a different file. A worker dropped a matched row +from staging by reporting it as a finding without listing it as a claim (caught by its own cleanup +audit — hence "run the audit BEFORE the final report"). A coordinator commit message silently lost a fact +to unescaped backticks. A coordinator-recorded *reason* was wrong for a tested lever (the test was inert +by construction) and was corrected after the worker pointed it out. The candidate gate rejected three +batches and the tracked registry was never touched. + +**Scope held.** The blocked classes (trapping arithmetic, the `0x80012xxx` primitive-init family, the +maspsx mutual exclusion) stay excluded — 0 of 493 registered regions contains any of them — and the +phase's harness changes did not reopen them. **No scheduler-changing flag was granted** at any point: +`cc1=-G8` and `gp=-NAME` were granted because they are byte-required and each was verified with a failing +control, but a flag that changed cc1's scheduler would mean the corpus is no longer all built by one +compiler configuration; workers proposed it twice and were told no, and both recorded their rows as +negatives with the lever named. Inline asm was extended by developer decision from "instructions C cannot +*name*" to "shapes C provably cannot *express*", first instance the PSX scratchpad stack switch, proven +byte-exact twice. `git clean -x`/`-fdx` were never used, and the ROM firewall was audited across the +entire history before the first push to a new remote (largest blob in all of history: a 163 KB config +TSV). + +**Two workers rotated mid-phase on measured criteria and two fresh sessions replaced them; fresh context +beat exhausted context both times.** Worker A produced 26 claims (24 first-attempt with no override), +worker B 23, worker C 22, worker B2 9, worker C2 4. One worker stopped on a *yield* basis at 47% and the +stop was recorded as explicitly **not** the measured criterion rather than dressed as a rotation. + +Unresolved and recorded with evidence: the `0x8010080C` false extent start (`sf3_extents` deliberately +not changed — a hand-edited extent fails `make extents-verify` by design); the maspsx rare-epilogue +mutual exclusion; the two opt-in maspsx modes (`noreordernop`, `regread`) which are implemented, +default-off and regression-safe but **neither shown to close a region**; the scheduler class +(`-fno-schedule-insns` gives the original's order on at least two rows but the allocation or delay slot +then disagrees); `0x80107C5C`, which matches with the `#nop` fix but whose source is a bare variant with +no header, so it is recorded as unblocked-and-one-documented-source-away and **not counted in the 484**; +and 194 recorded negatives in the tracked index, each with an address, size, status and class. Cookbook +grew to 57 findings (41–57 new), `docs/PHASE10_VERIFICATION.md` records the verification, and +`phase-ends/logs/Phase10.md` is the ledger. No changes to AGENTS.md. diff --git a/phase-ends/PhaseEnd_Phase10.md b/phase-ends/PhaseEnd_Phase10.md new file mode 100644 index 0000000..a4737ed --- /dev/null +++ b/phase-ends/PhaseEnd_Phase10.md @@ -0,0 +1,165 @@ +# PhaseEnd — Phase 10: Matching the Remaining Pool (Goal A, tail squeeze) + +**Closed:** 2026-09-24, coordinator session `01a0d302-0201`. +**Milestone:** **475 distinct bodies** — **MET and exceeded at 484**, from the Phase 9 close state of +400. **484 distinct matched bodies / 493 registered regions.** +**Developer confirmation:** requested and given before any close record was written, per the plan. + +## Result + +| | Phase 9 close | Phase 10 close | +|---|---|---| +| Distinct matched bodies | 400 | **484** | +| Registered regions | 409 | **493** | +| Registered symbols | 380 | 386 | +| Synthetic tests | 229 | **237** | +| Tracked files | 506 | 591 | + +**+84 distinct bodies**, against a milestone of +75. All gates green from clean: + +``` +make clean && make all exit 0 +cmp exit 0 +SHA-1 (both files) e173426c157384ebf1b6caf8c6fea18a85a14af9 +make test 237 tests, OK +make extents-verify regions=493 disagreements=0 result=AGREE +make gate c_regions=493 differing_bytes=0 result=MATCH +registry audit 493 rows, 0 overlaps, 0 unsorted, 0 bad extents, + 0 missing sources, 484 distinct sources +worklist listed=1193, excluded_already_registered=493 +negatives index 194 rows, address-ordered, 0 duplicates, 0 registered +git status --short src/ empty (0 untracked files) +firewall 0 tracked paths under any prohibited root +``` + +## Roster and yield + +| Session | Role | Claims | Outcome | +|---|---|---|---| +| `01a0d302-0201` | coordinator | — | merged and gated every claim; 27 merges | +| `01a0d302-18e3` | worker A | **26** | 24 first-attempt with no override; stopped with a complete handoff | +| `01a0d302-8a6f` | worker B | 23 | stopped on a *yield* basis at 47% (recorded as explicitly **not** the measured criterion) | +| `01a0d338-485d` | worker B2 | 9 | replacement for B; produced the md5 guard and the `#nop` finding | +| `01a0d302-fad3` | worker C | 22 | stopped at the measured 68% cap; produced the array-locals lever and the size-band experiment | +| `01a0d344-0c94` | worker C2 | 4 | replacement for C; produced the scheduler-class finding | + +Two workers rotated mid-phase on measured criteria, and two fresh sessions replaced them. **Fresh +context beat exhausted context both times**, which is Phase 9's strongest lesson confirmed again. + +## The headline finding: THE SIZE-BAND LAW + +**The matched corpus has a median size of 48 bytes and 454 of 459 were at ≤200 B, while the remaining +levered rows had a median of 456 B. Size is the strongest predictor of yield left in the pool.** + +Two independent measurements, one of them controlled: + +| source | ≤200 B | 200 B–800 B | +|---|---|---| +| worker C — same worker, same levers, same day, band the only variable | **1, 1, 1, 2** attempts | **1-in-12** | +| worker B2 — within a single session | **1, 2, 2** attempts | the two >200 B rows it spent the *most* time on: **1 deferred + 1 harness, zero matches** | + +This was acted on mid-phase: every worker's dispatch file was **re-ranked to size-band-first**, which +caught worker C heading back into the band it had just measured as exhausted. It is cookbook finding 41 +and the number to use when placing a worker-hour in a future phase. + +## The phase's character: every lever acquired a limit + +Most of Phase 10's levers are **two-sided**, and each side was found by a worker testing the rule +against a row where it failed. Five of the coordinator's own generalisations were bounded this way — +more than in any prior phase: + +| coordinator's rule | bounded by | the limit | +|---|---|---| +| "skip the rare-epilogue class" | worker B | the class is a three-way split, and attemptable | +| "a permuted register set = the commutative lever" | worker A | a permuted set can be a **semantic bug** — verify dataflow first | +| "residual at a two-arm branch → invert and swap" | worker B2 | both ordinary spellings can mirror; the guard may need a `goto` | +| "the `goto` trigger needs a `goto`" | worker A | necessary but **not sufficient** | +| "R2 did not fire, so the load-delay class is closed" | worker B2 | R2 only extends to `jr`/`jalr`; the store consumer was untested | + +The rules that survived into the cookbook are the ones that were bounded. This is the checks-and-balances +structure working as designed, and it is the phase's most transferable process result. + +## Harness work + +- **`restores_unsaved` exclusion class** (`tools/sf3_triage` + 5 tests): a body restoring a callee-saved + register it never saves cannot be a whole function. Verified disjoint from the corpus (**0 of 462 + registered regions**; 11 worklist rows). It independently re-derived a defect another worker had found + by a different signal. +- **maspsx: honour cc1's explicit `#nop` marker** (developer-authorised). cc1 asked for a load-delay nop + and maspsx overruled it for bare-symbol store consumers. **Verified regression-free: all 489 regions + byte-identical.** Found by worker B2, whose first diagnosis (extend the predicate) would have been a + **no-op** — the predicate already returned True. The coordinator traced the real mechanism. +- **Per-region override plumbing** for the new maspsx modes, plus 5 tests. +- **`tools/maspsx/` is git-ignored**, so every maspsx change is carried as a **tracked patch** + (`tools/patches/maspsx-phase10-r1r2.patch`) verified to reproduce the modified files byte-identically + from the pristine pinned checkout, with provenance in `docs/SETUP.md`. **An in-place edit would not + have survived a fresh clone** — this was caught before it mattered. +- **md5-in-claim-row guard**, adopted after a real concurrent-write collision: `src/func_XXXXXXXX.c` is + a shared, unlocked namespace, so a file can change between a worker's verify and the coordinator's + merge. A mismatch means re-verify rather than merge or reject. + +## Incidents, recorded rather than smoothed over + +1. **A coordinator double-assignment.** `0x800A9C24` was redistributed to worker B2 and then chartered + to worker C2, so two sessions wrote one file and a correct claim came back DIFF. **The error was the + coordinator's**; neither worker did anything wrong. It produced the md5 guard. +2. **A worker verified a variant and claimed a different file** (merge 20). Returned to its owner; the + rule "verify the file you are about to stage, at the path you are about to claim" was added to every + charter. +3. **A worker dropped a matched row from staging** by reporting it as a *finding* without listing it as + a claim. Caught by that worker's own cleanup audit; the rule "run the audit BEFORE the final report" + was added. +4. **A coordinator commit message silently lost a fact** to unescaped backticks. Amended and + force-pushed; commit messages now go through a file. +5. **A coordinator-recorded reason was wrong** for a tested lever (the test was inert by construction). + Corrected in the tracked index after the worker pointed it out. + +Each was caught by the merge flow or by a worker, and none corrupted the tracked registry. + +## Scope, safeguards, and what did NOT change + +- **The blocked classes stay excluded**: trapping arithmetic, the `0x80012xxx` primitive-init family, and + the maspsx mutual exclusion. 0 of 493 registered regions contains any of them. The phase's harness + changes did **not** reopen them. +- **No scheduler-changing flag was granted.** `cc1=-G8` and `gp=-NAME` were granted because they are + byte-required and each was verified with a failing control; a flag that changed cc1's scheduler would + mean the corpus is no longer all built by one compiler configuration. Workers proposed it twice and + were told no. +- **Inline asm was extended** (developer decision) from "instructions C cannot *name*" to "shapes C + provably cannot *express*", with demonstrated triggers and per-file documentation. First instance: the + PSX scratchpad stack switch, proven byte-exact twice. +- **`git clean -x` / `-fdx` were never used.** No ROM-derived material entered a tracked path at any + point: the largest blob in the entire history is a 163 KB config TSV, and the firewall was audited + across all 200+ commits before the first push to the new remote. + +## Unresolved, recorded with evidence + +- **`0x8010080C` false extent start** — a `jal` target mid-body splits one real 252-byte function. Root + cause recorded; `sf3_extents` deliberately not changed (a hand-edited extent fails + `make extents-verify` by design, and a real fix alters the boundary rule all 493 regions derive from). +- **The maspsx rare-epilogue mutual exclusion** — finding 40 records the load-delay-hazard mechanism and + the measured failure of the obvious fix. The remaining route is a tracked post-pass modelling ASPSX's + fill for one site; developer-owned, not taken. +- **The two opt-in maspsx modes** (`noreordernop`, `regread`) are implemented, default-off and + regression-safe, but **neither has been shown to close a region**. +- **`0x80107C5C` matches** with the `#nop` fix but its source is a bare variant with no header, so it is + recorded as unblocked-and-one-documented-source-away, **not** counted in the 484. +- **The scheduler class** (`-fno-schedule-insns` gives the original's order on at least two rows, but the + allocation or delay slot then disagrees) is characterised and recorded, not solved. +- **194 recorded negatives** in the tracked index, each with an address, size, status and class, plus the + workers' full residual/spelling/untried-lever detail in the ledger and their handoffs. + +## Carried into Phase 11 + +- The **size-band law** (finding 41) as the dispatch rule. +- The **md5 guard** and the four process rules in finding 57. +- The **negatives index** (194 rows) as the queue-reconciliation input, already proven to keep closed + negatives off the worklist head. +- The **tracked maspsx patch** and its `SETUP.md` provenance. +- Two workers' handoffs with ranked near-matches: worker A's 13 and worker C2's 4, each with an exact + residual and a named untried lever. +- The unresolved items above. + +## Rules + +No changes to `AGENTS.md`.