phase9: CLOSE — PhaseEnd, digest, ledger archive, Phase 10 plan, verification and protocol records
Phase 9 closed at 400 distinct bodies / 409 regions (from 149/158), all gates green from clean: CMP_OK, SHA-1 e173426c157384ebf1b6caf8c6fea18a85a14af9, 229 tests, make gate c_regions=409 MATCH, extents-verify AGREE, worklist listed=1343 with excluded_already_registered=409, firewall 0, src/ 0 untracked. Milestone partially met (400 of 500): the blocker is recorded with per-class evidence (tier-1 tie-break domination, class disjointness, measured rate decay) per the plan's Or branch. PhaseEnd_Phase9.md, digest entry, and the ledger archive (logs/Phase9.md) written. docs/PHASE9_PROTOCOL.md (scale-run retrospective: queue fix, tier-2 pivot, family-set lever, rotation discipline, leading-indicator rule, measured budget), docs/PHASE9_VERIFICATION.md (all gates, incidents, class exclusions), cookbook findings 29-39, conventions additions, README updated to 400/409. Phase10_PLAN.md drafted (Goals A/B/C: tail squeeze vs library-boundary investigation, developer decisions enumerated) — requires explicit approval before any Phase 10 task; no Phase 10 work begins in this session.
This commit is contained in:
Christopher Williams
@@ -8,12 +8,13 @@ byte**.
|
||||
instruction-identical to the original and the whole rebuilt executable must be SHA-1 identical.
|
||||
Nothing "functionally equivalent" counts, and unmatched C never enters the default build.
|
||||
|
||||
> **Status: matching at scale.** Phase 8 is complete pending milestone confirmation. The executable rebuilds
|
||||
> byte-identically, the original toolchain is identified from byte evidence, and **149 distinct functions /
|
||||
> 158 regions are matched to C** — every one instruction-identical and covered by the clean full-binary gate.
|
||||
> **Status: matching at scale.** Phase 9 is closed — milestone partially met. The executable rebuilds
|
||||
> byte-identically, the original toolchain is identified from byte evidence, and **400 distinct functions /
|
||||
> 409 regions are matched to C** — every one instruction-identical and covered by the clean full-binary gate.
|
||||
> Function starts, ends and duplicate bodies are all derived from evidence rather than by hand, match targets
|
||||
> come from a ranked worklist, and the phase was executed by three pi sessions in parallel under a written
|
||||
> coordination protocol.
|
||||
> come from a ranked worklist, and the phase was executed by three pi sessions in parallel under written
|
||||
> coordination protocols. The 100-body shortfall below the 500 milestone is recorded with its evidence
|
||||
> (measured pool exhaustion), and a Phase 10 plan is drafted pending developer approval.
|
||||
|
||||
## Where it stands
|
||||
|
||||
@@ -28,8 +29,8 @@ Nothing "functionally equivalent" counts, and unmatched C never enters the defau
|
||||
| Code/function segmentation | 2,875 candidates graded; extents derived for all of them |
|
||||
| Duplicate-body census | 65 groups, 10 containing code — 6 of the 10 now fully matched |
|
||||
| Ranked match worklist | 1,744 eligible candidates, every exclusion counted |
|
||||
| Multi-session coordination | 3 sessions in one worktree under a written protocol, all claims independently re-verified |
|
||||
| Functions matched to C | **149 distinct bodies / 158 regions** — byte-identical and gated (Ghidra's analyzer reports ~1,721 function candidates) |
|
||||
| Multi-session coordination | 3 sessions in one worktree under written protocols (Phase 8 and 9), all claims independently re-verified by the coordinator's own whole-binary gate |
|
||||
| Functions matched to C | **400 distinct bodies / 409 regions** — byte-identical and gated (Ghidra's analyzer reports ~1,721 function candidates) |
|
||||
|
||||
The validated target is the USA executable `SCUS_946.40;1`:
|
||||
|
||||
@@ -107,7 +108,7 @@ verify against the original SDK compiler — the proprietary PsyQ SDK plus the `
|
||||
| `config/` | The tracked registries and evidence tables: `regions.tsv` (matches), `symbols.tsv` (symbols), `function_inventory.tsv` (graded starts), `function_extents.tsv` (graded extents), `duplicate_bodies.tsv` (duplicate census), `match_worklist.tsv` (ranked queue), `near_match_negatives.tsv` (open negatives index) |
|
||||
| `src/` | One C file per matched function body, named by address until a name is earned. Every tracked file here is a registered match; unmatched drafts live in ignored staging |
|
||||
| `include/` | `gtemac.h` — the project's own re-derived COP2 macros and control-register map, with per-macro provenance |
|
||||
| `docs/` | The format/address records, per-phase investigation and verification records, setup and toolchain provenance, the Phase 7 extents/dupes/triage records, the Phase 8 coordination protocol, and the [matching cookbook](docs/MATCHING_COOKBOOK.md) and [matching conventions](docs/MATCHING_CONVENTIONS.md) |
|
||||
| `docs/` | The format/address records, per-phase investigation and verification records, setup and toolchain provenance, the Phase 7 extents/dupes/triage records, the Phase 8 and Phase 9 coordination protocols, and the [matching cookbook](docs/MATCHING_COOKBOOK.md) and [matching conventions](docs/MATCHING_CONVENTIONS.md) |
|
||||
| `phase-ends/` | The project's governance record: the phase digest, one PhaseEnd per closed phase, each phase plan, and the active `CURRENT_PHASE.md` |
|
||||
| `Makefile` | The all-assembly rebuild (validate → split → assemble → link → binary) plus the ordered C build, the byte gate, and the `extents`/`extents-verify`/`dupes`/`worklist` regeneration targets |
|
||||
| `AGENTS.md`, `PROJECT_CONTEXT.md` | The standing rules and the permanent project constitution |
|
||||
|
||||
@@ -230,3 +230,59 @@ in ignored paths (`disks/`, `extracted/`, `asm/`, `build/`, `ghidra/`, `dumps/`,
|
||||
`tools/psyq/`, `tools/splat/`, `tools/maspsx/`, `tools/old-gcc/`, `tools/mipsel-none-elf-binutils/`,
|
||||
`tools/wibo/`). Review `git status` before every commit. `git clean -x` and `git clean -fdx` are
|
||||
forbidden.
|
||||
|
||||
## Phase 9 additions (2026-09-24)
|
||||
|
||||
### A tracked `src/` file is a claim — check before deleting
|
||||
|
||||
Before deleting any file under `src/`, run `git ls-files --error-unmatch <path>`.
|
||||
If it is tracked, it is a claim — another session may be its author, and it is
|
||||
not yours to remove. Only untracked paths are a worker's own drafts. This
|
||||
became a hard rule after a worker deleted a registered source by reflex and
|
||||
restored it byte-exactly; the incident and the rule are recorded in the Phase 9
|
||||
ledger.
|
||||
|
||||
### Dependent claims: symbol merged before the claim
|
||||
|
||||
If a claim's source references a symbol NOT yet in the tracked registry, it
|
||||
must be reported as **DEPENDENT** (symbol merged first), never as verified
|
||||
against a local superset. A local superset file answers the lookup the tracked
|
||||
registry cannot, which silently masks a C-name/registry-name mismatch — the
|
||||
0x8002C728 and 0x8002E3FC incidents. A symbol request is staged before the
|
||||
claim that needs it.
|
||||
|
||||
### Symbol names must match the registry spelling exactly
|
||||
|
||||
The gp marker is keyed by symbol NAME, not by address. A differently-named
|
||||
symbol with the right address resolves implicitly and silently loses the
|
||||
gp-relative form (or the registry override). Use the tracked spelling, not
|
||||
just the right address. Check `config/symbols.tsv` BY ADDRESS before staging a
|
||||
request; a duplicate request breaks the merge loudly with `duplicate symbol`.
|
||||
|
||||
### The leading-indicator rule (band exhaustion)
|
||||
|
||||
A per-batch cluster of sub-8-byte residuals with no NEW mechanism is the
|
||||
leading indicator that a band is exhausted. It is measurable per batch:
|
||||
convert "the workers seem slower" into a number. On the signal, regenerate the
|
||||
worklist and re-partition (or re-charter) rather than grinding the same rows.
|
||||
|
||||
### The family-set lever (match callers as a set)
|
||||
|
||||
After matching any function, scan the remaining worklist for rows calling the
|
||||
same target; the derived argument map is paid once per family. Stable
|
||||
positions may transfer between rows; the full map does not — re-read
|
||||
per-row register arguments every time.
|
||||
|
||||
### The tier-2 pivot (choosing what to match next)
|
||||
|
||||
Match tier-2 frame/call bodies first: their structure pins cc1's allocation
|
||||
(high first-attempt yield). Tier-1 leaves are tie-break-dominated — low yield,
|
||||
not hard — and the worklist's own ranking already separates them.
|
||||
|
||||
### Compute the constants, don't eyeball them
|
||||
|
||||
`lui` high halves (lui 0x600 = 0x06000000), gp offsets, and two-piece
|
||||
constants (lui 0xff + ori 0xff00 = 0x00FFFF00) must be computed, not
|
||||
estimated. This extends the finding-4 habit to every constant build; the
|
||||
one-byte encoding traps (literal-vs-symbol address) are caught by the
|
||||
count-right/residual-1-8-bytes → diff-the-encoding habit.
|
||||
|
||||
@@ -471,3 +471,171 @@ to tell in advance which rows the lever can help.
|
||||
- The exact `-G` small-data threshold is not recoverable from the code; the per-symbol `gp` form is
|
||||
reconstructed from the original's accesses instead (findings 10 and 16).
|
||||
- Whether `-mgpopt` was passed is not observable: it does not change `cc1` output for the cases tested.
|
||||
|
||||
## Phase 9 — findings 29–39 (coordinated scale phase, 2026-09-24)
|
||||
|
||||
Phase 9 took the project from 149 to 400 distinct matched bodies with three
|
||||
workers, and hardened several findings the previous phases had left open. Each
|
||||
entry states its basis and its limit; nothing here is copied from another
|
||||
project.
|
||||
|
||||
### 29. GTE control-register numbering is standard+7 from RBK onward, unshifted 0..5
|
||||
|
||||
The executable's GTE control-register map (raw words, cross-checked by two
|
||||
independent worker decodes and the coordinator): registers `$0..$5` are the
|
||||
rotation/translation block under the **standard** numbering (0x80101CAC: five
|
||||
`ctc2`s into `$0..$4`), and from RBK onward the map is **standard + 7**:
|
||||
`$13` RBK, `$14` GBK, `$15` BBK, `$16..$20` light matrix
|
||||
(LR1LR2/LR3LG1/LG2LG3/LB1LB2/LB3), `$21/$22/$23` far-colour triple
|
||||
(RFC/GFC/BFC), `$24/$25` OFX/OFY, `$26` H, `$27` DQA, `$28` DQB. The data
|
||||
register space is standard unshifted (`$0` VXY0 … `$31` LZCR).
|
||||
*Basis:* every row in the map is a matched region or a raw-word decode;
|
||||
`include/gtemac.h` carries the corrected table and macros.
|
||||
*Limit:* the *names* are register-map readings; the *numbers* are the
|
||||
evidence. Finding 24's earlier label of `$13–15` as the light matrix was
|
||||
wrong (the numbers were right; the names were the suspect).
|
||||
|
||||
### 30. The three encoding traps are ONE class: count right, residual 1–8 bytes, diff the ENCODING not the mnemonic
|
||||
|
||||
Three distinct-looking failures share one signature — the instruction count
|
||||
and sequence are plausible and correct, and only the operand *encoding*
|
||||
differs:
|
||||
|
||||
- `lui` high halves: `lui v0,0x600` loads `0x06000000`, not `0x600`
|
||||
(0x80012A98, 0x800AFFB8).
|
||||
- Name-keyed gp markers: the gp marker hangs off the symbol NAME, so a
|
||||
differently-named symbol with the right address silently resolves
|
||||
absolutely and loses the gp-relative form (0x800FC280, 8 bytes).
|
||||
- Literal-vs-symbol addresses: a literal materialises `lui`+`ori`; a symbol
|
||||
materialises `lui`+`addiu` (finding 5) — one byte, identical address
|
||||
(0x800B6838).
|
||||
|
||||
*Basis:* each row matched after the fix; the class was synthesised from
|
||||
worker B's three near-misses.
|
||||
*Limit:* the detection habit is the generalisation, not a rule about any one
|
||||
trap: when the count is already right and the residual is 1–8 bytes, compare
|
||||
the ENCODING — the mnemonic is identical in every case.
|
||||
|
||||
### 31. The tier-2 pivot: frames pin the allocation; leaves are tie-break-dominated
|
||||
|
||||
Tier-1 leaf bodies (no frame, no call) leave cc1's allocator free, so small
|
||||
C spelling differences surface as pure register/operand-order tie-breaks with
|
||||
the instruction count already correct — 0-for-12 on the tier-1 head in one
|
||||
worker's measurement, every residual 2–22 bytes. Tier-2 frame/call bodies
|
||||
constrain the prologue, call sequence and epilogue, pinning the allocation;
|
||||
the same levers then match at ~90% first attempt.
|
||||
*Basis:* worker B's 0-for-12 → 9-for-9 first-attempt transition on moving from
|
||||
tier-1 leaves to tier-2 frames; worker C's independent 75%-across-bands with
|
||||
every failure lever-fixable; the coordinator's absorption confirming both.
|
||||
*Limit:* "tier-1 is low-yield" is the operational rule; "tier-1 is hard"
|
||||
would be the wrong conclusion — every measured tier-1 residual had a fully
|
||||
identified mechanism.
|
||||
|
||||
### 32. The family-set lever: match a callee's callers as a set, not row by row
|
||||
|
||||
After matching any function, scan the remaining worklist for rows **calling
|
||||
the same target**; the derived argument map is paid once per family. The
|
||||
0x80057664 family closed 11 consecutive bodies this way in the endgame, each
|
||||
first attempt.
|
||||
*Limit (the Phase 9 correction):* the map does NOT transfer wholesale. Stable
|
||||
positions transfer (selector→arg2, sentinel→arg3, narrowed `a1`→arg9, extra
|
||||
param→arg10), but per-row register arguments must be re-read — several
|
||||
variants differed in which incoming parameter became which outgoing argument,
|
||||
and one had two incoming stack params. The lever transfers the *derivation
|
||||
method and the stable positions*, never the full map.
|
||||
|
||||
### 33. A frame slot never read back is the outgoing argument area, not a dead store
|
||||
|
||||
A store to 16..36(sp) that the routine never reloads is the o32 outgoing
|
||||
stack argument area — the callee's 9th and 10th (and 11th) arguments. Three
|
||||
rows (0x80057748 and family) matched only after the call was written with
|
||||
ten (and eleven) arguments. The frame's own arithmetic tells you where the
|
||||
arg area is: `ra` and the saved registers bound the region.
|
||||
*Basis:* 0x80057748 (68 vs 80) matched at 80 with the 10-arg call; the
|
||||
family's 11-arg sibling recorded with the same shape.
|
||||
*Limit:* check the frame layout before declaring a store dead; the outgoing
|
||||
area is between the saved registers and the incoming args.
|
||||
|
||||
### 34. The branchless-conditional lever: `ternary ? x : 0` is not interchangeable with `if/else`
|
||||
|
||||
A conditional whose arms differ by zero compiles branchlessly:
|
||||
`sltu` + `negu` + `and` (0-or-−1 mask) — no branch at all. Written as
|
||||
`if/else` the encoding differs. The spelling is decided by how many distinct
|
||||
values are selected: one value → branchless; two values → a real
|
||||
`beqz`/`j` pair.
|
||||
*Basis:* 0x80057748 matched with the ternary; worker C's 0x800579F0-family
|
||||
discriminator (three rows in the family select one value, one selects two).
|
||||
*Limit:* count the distinct selected values to choose the spelling; do not
|
||||
assume authorial style.
|
||||
|
||||
### 35. The rare-epilogue order is a three-way class, not "unmatchable"
|
||||
|
||||
The `lw ra` / (`nop`|`lw s?`) / `jr ra` / `addiu sp,sp,N` (restore in the
|
||||
jump delay slot) epilogue — finding 11's "rare" order — is a class with
|
||||
three behaviours: (a) cc1 emits it itself (0x80021FF4 MATCHED, default
|
||||
settings); (b) cc1 leaves the slot empty and maspsx's unconditional nop
|
||||
blocks the fill, so `--no-maspsx` fixes the epilogue but breaks every `move`
|
||||
into `or` — the mutual exclusion with addu-encoded moves (0x800F8928,
|
||||
0x800FC280, 0x800F4C08); (c) cc1 emits the nop and nothing reaches it
|
||||
(0x800FBDC0).
|
||||
*Basis:* worker B's falsification of the coordinator's skip-the-class
|
||||
over-generalisation (0x80021FF4 was already registered) and the three-way
|
||||
model confirmed on three rows.
|
||||
*Limit:* census 47 worklist rows carry the pattern at their final
|
||||
instructions; ~91% of partition-A's subset is in sub-case (b). Treat the
|
||||
class as attemptable, with `--no-maspsx` as the epilogue-only try and the
|
||||
mutual exclusion as a harness limit, not the class.
|
||||
|
||||
### 36. ADDI (opcode 0x08) is a trapping form too
|
||||
|
||||
Finding 26's census keyed on R-type trapping `add`/`sub`; the immediate
|
||||
`addi` (opcode 0x08, traps on overflow) is the same class — 7 worklist rows
|
||||
carried it (frame adjustments `addi sp,sp,-28`, loop counters), 0 of 409
|
||||
registered regions did. `addiu` (0x09) is ubiquitous and normal, so the 0x08
|
||||
occurrences are the anomaly.
|
||||
*Basis:* worker B's scan (raw words, 7 of 1,666), coordinator-verified;
|
||||
`sf3_triage.trapping_arithmetic` now counts opcode 0x08.
|
||||
*Limit:* sufficient but not complete, same as finding 26.
|
||||
|
||||
### 37. The mirrored-layout / named-locals levers, sharpened
|
||||
|
||||
- The mirrored-layout lever (finding 28) is **per-guard**, not per-function:
|
||||
apply it to each guard independently, testing which ones need inversion
|
||||
(0x8003A5F4's second guard needed it; its first did not; 0x80058230 same).
|
||||
- Named-locals has three distinct mechanisms: order-pinning, reload-prevention
|
||||
(aliasing: a local is not memory, so a store through a possibly-aliasing
|
||||
pointer cannot invalidate it), and the INVERTED case where the repeated
|
||||
expression is correct and the local is the mistake (0x80057B84).
|
||||
*Basis:* worker C (F16) and worker B (0x800F9134, 0x80057B84).
|
||||
*Limit:* the count-right/order-wrong diagnostic decides which lever applies;
|
||||
mechanisms are not interchangeable.
|
||||
|
||||
### 38. `0x8005DEF8`'s family: a negative conclusion
|
||||
|
||||
The 0x80012xxx primitive-init family was excluded after four failures vs two
|
||||
successes (worker C's scheduler-bound diagnosis, worker B's fourth data point
|
||||
with the worst residual of the cycle, 54 bytes). The model is independently
|
||||
pinned (five stack args fix a nine-parameter signature; the `lui` immediates
|
||||
are high halves). No further attempts on the family under the current toolchain.
|
||||
*Basis:* family records; now excluded via Makefile `--exclude`.
|
||||
*Limit:* a future harness transform (suppressing maspsx's jump-slot nop or a
|
||||
tracked post-pass) is the only route — a developer decision.
|
||||
|
||||
### 39. Worker-built tooling worth inheriting
|
||||
|
||||
`rareepi.py` (worker C's per-row rare-epilogue classifier, validated against
|
||||
six known cases) and `orig.py` (original-disassembly helper), both read-only,
|
||||
both in the workers' ignored staging — promote `rareepi.py` into `tools/` if
|
||||
Phase 10 takes the rare-epilogue classes on.
|
||||
*Basis:* worker C's handoff.
|
||||
*Limit:* both were worker-scratch; promotion requires synthetic tests under
|
||||
`tools/tests/` as Phase 9 did for `sf3_diff`.
|
||||
|
||||
## Open questions (Phase 9 close)
|
||||
|
||||
- The trapping-arithmetic and rare-epilogue classes remain the best leads on
|
||||
library-versus-game-code: their compiler signatures differ from every
|
||||
available cc1, and no tested toolchain emits them — a different-build or
|
||||
Sony/SN toolchain story is the hypothesis for Phase 10's Goal B.
|
||||
- The exact `-G`, the CRT entry, and library-versus-game-code remain
|
||||
unresolved, with the class exclusions now precisely bounded.
|
||||
|
||||
@@ -0,0 +1,160 @@
|
||||
# Phase 9 — Coordination Protocol at Scale
|
||||
|
||||
**Scope:** the operational contract and lessons of Phase 9 (2026-09-24), which
|
||||
ran three workers against a 500-body milestone and closed at 400 bodies / 409
|
||||
regions. It does not replace `docs/PHASE8_PROTOCOL.md`; it extends it with what
|
||||
the scale run taught. Where the two differ, Phase 9's additions govern.
|
||||
|
||||
## Sessions
|
||||
|
||||
| Session | Role | Outcome |
|
||||
|---|---|---|
|
||||
| `01a0d180-de51` | coordinator | this session; merged and gated every claim |
|
||||
| `01a0d180-ea72` | worker A | 108 claims, +79 bodies, rotated cleanly |
|
||||
| `01a0d180-efdd` | worker B | 60 claims, +47 bodies (milestone driver), rotated cleanly |
|
||||
| `01a0d180-f513` | worker C | 71 claims, +29 bodies, rotated cleanly |
|
||||
|
||||
Workers are addressed by intercom short id; `subagent-chat-…` aliases are
|
||||
convenience, never the handle. No replacement sessions were available on a
|
||||
fixed roster of four — rotations were absorbed by the coordinator or covered
|
||||
by the measured-resume procedure below.
|
||||
|
||||
## Hard rules (the load-bearing ones, unchanged)
|
||||
|
||||
- `src/*.c`: a worker creates files only for addresses in its own partition;
|
||||
partitions regenerate every cycle.
|
||||
- `config/*.tsv`, `build/`, docs, tools, the Makefile, all writing git
|
||||
commands: coordinator only.
|
||||
- Worker staging under `.run/p9/<worker>/` only (ignored).
|
||||
- **`git ls-files --error-unmatch` before deleting anything under `src/`** —
|
||||
tracked means claim. (New: the Phase 9 deletion incident.)
|
||||
- **Dependent claims:** a claim referencing a not-yet-tracked symbol is
|
||||
reported DEPENDENT (symbol merged before the claim) — never verified against
|
||||
a superset. (New: the 0x8002C728 / 0x8002E3FC incidents.)
|
||||
- **Named symbols must match the registry spelling exactly** — the gp marker is
|
||||
name-keyed, not address-keyed. (New: the 0x800FC280 quiet-failure finding.)
|
||||
|
||||
## The merge flow (proven by the phase, unchanged in shape)
|
||||
|
||||
1. Verify every claim: `sf3_match range`, fresh `--work`, tracked `--symbols`
|
||||
only.
|
||||
2. Merge to a **candidate**: `sf3_merge apply` → candidate regions + symbols.
|
||||
3. Gate the candidate whole-binary (`make gate`, cmp, SHA-1 fixed). On
|
||||
**DIFF**: isolate the bad row by `comm` of candidate vs registry, verify
|
||||
per-row, drop or report it back, re-gate.
|
||||
4. On **MATCH**: promote over `config/regions.tsv` + `config/symbols.tsv`;
|
||||
then **reconcile negatives**: import all worker report rows into the
|
||||
tracked `near_match_negatives.tsv` (address-keyed, no bytes), dropping
|
||||
now-registered rows; then **regenerate the worklist and re-partition**
|
||||
(3-way, disjoint, proved at every cycle); then commit the verified sources +
|
||||
registries.
|
||||
5. Audit: `git status --short src/` must be 0 at merge time (tracked src file =
|
||||
claim); pre-commit `git status` review; `git diff --check`.
|
||||
|
||||
The candidate-gate-before-promote step saved the phase at least three times
|
||||
(a silent duplicate, a deleted source, and a dependent-symbol claim whose
|
||||
first gate attempt DIFFed with a cascade) — the tracked registry was never
|
||||
corrupted.
|
||||
|
||||
## Worklist and exclusion machinery (the phase's efficiency core)
|
||||
|
||||
`tools/sf3_triage` excludes, each with a counted reason and all measured
|
||||
disjoint from the corpus at close:
|
||||
|
||||
| Class | Count | Basis |
|
||||
|---|---|---|
|
||||
| recorded negative | 115 | 39 of the first 40 head rows were already-attempted negatives before this fix — the phase's highest-value efficiency change. The worklist is regenerated from the TRACKED index each merge, so a closed negative never returns to the head. |
|
||||
| trapping arithmetic (incl. ADDI) | 54 | finding 26 + Phase 9's opcode-0x08 extension; 0 of 409 registered |
|
||||
| delay-slot start | 5 | a branch/jump's delay slot cannot be a function start |
|
||||
| bad-extent start | 8 | first-instruction reads an undefined register, or no return in range |
|
||||
| named exclusion | Makefile | primitive-init family, record near-misses, gp-thunks, maspsx-blocker row |
|
||||
|
||||
Partition files are regenerated **alongside every merge** (not per cycle): the
|
||||
previously recorded failure — a partition going stale between merges and
|
||||
re-presenting already-registered rows — is now structurally impossible. The
|
||||
worklist rows are re-proof-disjointed (intersection = ∅, union = worklist)
|
||||
after every regeneration.
|
||||
|
||||
## Charters and reporting (extended)
|
||||
|
||||
- **The capability probe** (from Phase 8) is unchanged; recruited ids go in the
|
||||
phase ledger.
|
||||
- **The standing-rule block in every charter** now includes: the
|
||||
`git ls-files`-before-`rm` rule, the dependent-claim rule, the symbol-name
|
||||
spelling rule, the compute-don't-eyeball rule, the leading-indicator rule,
|
||||
and the three-encoding-trap habit (count right + 1–8 byte residual → diff
|
||||
the ENCODING, not the mnemonic).
|
||||
- **Budget discipline is measured, not felt**: a worker re-flags budget only on
|
||||
two consecutive zero-match cycles with attempts-per-match quoted, or by
|
||||
quoting its context figure from `intercom list` — never on narrative. Phase
|
||||
9's worker rotations (B's measured criterion, C's self-correction,
|
||||
A's clean exhaustion) all followed this.
|
||||
- **The resume procedure** (new, proven twice): a rotated-out worker whose
|
||||
context has headroom can resume on the measured criterion, with its local
|
||||
superset REBUILT to contain only genuinely-unique addresses (by address, so
|
||||
a differently-named row cannot mask a tracked-name mismatch), and its
|
||||
partition regenerated before it re-starts.
|
||||
|
||||
## Rotation (the phase's strongest lesson: exhaustion is planned)
|
||||
|
||||
- On budget exhaustion: claims flushed, report.tsv complete (every negative
|
||||
with its exact residual, spellings tried, and the untried lever named),
|
||||
drafts deleted from `src/`, `symbols-request.tsv` pruned, handoff block in
|
||||
the ledger.
|
||||
- A retired worker's partition is offered to the active worker who can best
|
||||
absorb it — but the **hypotheses** (report.tsv's untried levers) are worth
|
||||
more than the rows: a fresh worker closing a recorded near-match with the
|
||||
named lever is the highest-yield move in the phase.
|
||||
- The **leading-indicator rule** governs when to re-partition before
|
||||
exhaustion: a per-batch cluster of sub-8-byte residuals with no new mechanism
|
||||
means the band is exhausted — regenerate and re-partition, don't grind.
|
||||
|
||||
## Stop conditions (Phase 9's, recorded honestly)
|
||||
|
||||
- Milestone reached, or
|
||||
- **measured pool exhaustion**: the eligible remainder is dominated by
|
||||
tie-break rows or class-excluded rows, with no new mechanism appearing — the
|
||||
blocker is recorded with per-class evidence, not papered over.
|
||||
- Three consecutive cycles adding fewer than 5 bodies (with the per-cycle
|
||||
numbers).
|
||||
- A gate failure that cannot be isolated in one cycle — the registry stays
|
||||
green; record and stop.
|
||||
- Two consecutive full audits failing — stop; something is wrong beyond a
|
||||
claim.
|
||||
|
||||
## What Phase 9's scale run taught (the retrospective)
|
||||
|
||||
1. **The queue fix was the multiplier.** Excluding recorded negatives from the
|
||||
worklist (and regenerating per merge) converted wasted rediscovery into
|
||||
matching time — the single largest efficiency change of the phase.
|
||||
2. **Fresh bands aren't the same as exhausted ones.** Yield on fresh bands was
|
||||
~90%; on the exhausted tail ~20%; the two populations are separate, and the
|
||||
leading-indicator rule makes the boundary measurable.
|
||||
3. **Reliable closes beat exploration at the end.** The milestone-close
|
||||
directive (take family rows with matched targets over fresh-band
|
||||
exploration) produced 11 consecutive first-attempt matches. At a milestone,
|
||||
throughput over coverage is correct.
|
||||
4. **A coordinators' over-generalisation got falsified correctly.** Phase 9's
|
||||
coordinator told workers to skip the rare-epilogue class outright; worker B
|
||||
falsified it with an already-registered match and produced the three-way
|
||||
model. The checks-and-balances structure (coordinator verifies every claim;
|
||||
workers falsify every over-broad rule) is what makes the team robust.
|
||||
5. **The tier-2 pivot was the strategic recovery.** Frames pin allocation;
|
||||
leaves are tie-break-dominated. The intuition "small = easy" was backwards.
|
||||
6. **Context counts as a resource.** The coordinator stopped absorbing at 80%+
|
||||
context to protect the merge/gate role (the one that cannot be delegated);
|
||||
workers quoted their context figures rather than feelings.
|
||||
|
||||
## Closing checklist (unchanged from Phase 8, now with the ledger archive)
|
||||
|
||||
1. Every worker: final report, drafts removed, complete negatives list.
|
||||
2. `git status --short src/` = 0.
|
||||
3. Registry audit: ordered, non-overlapping, in payload, extents exact,
|
||||
sources present.
|
||||
4. All gates from clean (`make clean && make all`, cmp, SHA-1, test, gate,
|
||||
extents-verify).
|
||||
5. Worklist regenerates with `excluded_already_registered` = registry size.
|
||||
6. Negative metadata extracted into the tracked index before ignored staging
|
||||
is lost.
|
||||
7. Firewall and Git review; then request milestone confirmation before any
|
||||
PhaseEnd.
|
||||
@@ -0,0 +1,84 @@
|
||||
# Phase 9 — Verification Record
|
||||
|
||||
**Scope:** every match claim's evidence and every gate run for Phase 9
|
||||
(2026-09-24), from 149 bodies / 158 regions to 400 bodies / 409 regions.
|
||||
|
||||
## The authority model (unchanged, load-bearing)
|
||||
|
||||
A worker's claim was **never** trusted on its word. Every claim was:
|
||||
1. independently re-run by the coordinator with `sf3_match range` (fresh
|
||||
`--work`, tracked `config/symbols.tsv` only), then
|
||||
2. merged to a **candidate** registry, then
|
||||
3. gated **whole-binary** (`make gate`, `cmp`, SHA-1
|
||||
`e173426c157384ebf1b6caf8c6fea18a85a14af9`), and only on `result=MATCH`
|
||||
promoted to the tracked registry and committed.
|
||||
|
||||
The merge tool enforced the structural half (extent grade `exact`, ordered,
|
||||
non-overlapping, inside the payload, source exists); the gate enforced the byte
|
||||
half. `make extents-verify` re-derived every registered region's extent from
|
||||
control flow and required agreement.
|
||||
|
||||
## Per-cycle gates (all from clean state)
|
||||
|
||||
| Cycle | Regions | Bodies | Gate | Extents | Tests | SHA-1 |
|
||||
|---|---|---|---|---|---|---|
|
||||
| 1 | 215 | 206 | MATCH | AGREE | 223 | fixed |
|
||||
| 2 | 260 | 251 | MATCH | AGREE | 223 | fixed |
|
||||
| 2 close | 268–283 | 259–274 | MATCH | AGREE | 223 | fixed |
|
||||
| 3 (pivot) | 306–355 | 297–346 | MATCH | AGREE | +sf3_triage tests | fixed |
|
||||
| 4 (tier-2) | 358–409 | 349–**400** | MATCH | AGREE | 229 | fixed |
|
||||
| **Close** | **409** | **400** | **MATCH** | **AGREE** | **229** | **fixed** |
|
||||
|
||||
The final clean run: `make clean && make all` exit 0; `cmp` exit 0; both SHA-1
|
||||
`e173426c157384ebf1b6caf8c6fea18a85a14af9`; `make test` 229 OK; `make gate`
|
||||
`c_regions=409 differing_bytes=0 result=MATCH`; `make extents-verify`
|
||||
`regions=409 disagreements=0 result=AGREE`; worklist regenerated with
|
||||
`excluded_already_registered=409`; firewall 0; `git status --short src/` 0;
|
||||
`git diff --check` clean.
|
||||
|
||||
## Gate incidents (each one caught, isolated, and recorded)
|
||||
|
||||
| Incident | Detection | Resolution |
|
||||
|---|---|---|
|
||||
| Silent duplicate claim map (worker B re-checked the tracked tree) | ` M ` on a tracked src file | source restored to registry text; lessons in protocol |
|
||||
| `0x80025070` tracked source deleted from the worktree | ` D ` status; candidate gate DIFFed +4 bytes (cascade) | restored from git; re-gate MATCH |
|
||||
| Worker B deleted a tracked registered source (0x800B5CB4) | self-caught via ` D ` after the fact | `git show HEAD:...` byte-identical restore; `git ls-files`-before-`rm` rule added |
|
||||
| Worker C's `0x8002E3FC` claimed but symbol `g_80121BFA` not yet tracked | coordinator's tracked-only range = LENGTH-MISMATCH (88 vs 80) | root-caused as dependent-symbol (not a bad source); symbol merged first → MATCH; DEPENDENT-claim rule added |
|
||||
| Bad claim in a merge (C's early row) | candidate gate DIFF (1,101,999 differing bytes, +4 length) | isolated by `comm` of candidate vs registry; per-row re-verify; dropped; re-gate MATCH |
|
||||
|
||||
Every incident left the tracked registry green at all times — the
|
||||
merge-to-candidate-then-gate discipline held in every case, which is the proof
|
||||
that it works.
|
||||
|
||||
## Class exclusions (each measured, each disjoint from the corpus at close)
|
||||
|
||||
| Class | Tool | Count at close | Corpus overlap |
|
||||
|---|---|---|---|
|
||||
| Recorded negatives | `sf3_triage` `recorded_negative` | 115 (index grew 38→115) | 0/409 registered |
|
||||
| Trapping arithmetic (R + ADDI) | `trapping_arithmetic` incl. opcode 0x08 | 54 | 0/409 |
|
||||
| Delay-slot starts | `delay_slot_start` | 5 | 0/409 |
|
||||
| Bad-extent starts | `bad_extent_start` | 8 | 0/409 |
|
||||
| Primitive-init family | Makefile `--exclude` | 4 + earlier 2 | 0/409 |
|
||||
|
||||
The disjointness (0 of 409 registered regions containing any excluded-class
|
||||
signature) is the same structural signal Phase 8's trapping census introduced,
|
||||
now extended across five classes.
|
||||
|
||||
## Negative metadata
|
||||
|
||||
All worker/coordinator negatives were imported into `config/near_match_negatives.tsv`
|
||||
at each merge (the tracked, reproducible index) — not left in ignored staging.
|
||||
At close it holds 115 rows (addresses, sizes, statuses, class labels only; no
|
||||
bytes). The workers' per-attempt detail (residuals, spellings tried, untried
|
||||
levers) lives in their report.tsv files and the Phase 9 ledger archive; the
|
||||
cookbook carries the class-level synthesis.
|
||||
|
||||
## What is claimed
|
||||
|
||||
- 400 distinct body sources, each byte-identical to the original payload over
|
||||
its extent (range `differing_bytes=0`), each registered in an ordered,
|
||||
non-overlapping, in-payload registry row with an `exact` extent, each
|
||||
covered by the clean whole-binary gate.
|
||||
- The milestone shortfall (100 below 500) is a measured pool-exhaustion
|
||||
blocker, recorded in `PhaseEnd_Phase9.md` with the per-class evidence —
|
||||
not a claim of completion that is not met.
|
||||
@@ -68,3 +68,48 @@ library-versus-game-code. No rules were added.
|
||||
## Phase 8 — Coordinated Multi-Session Matching (2026-09-23)
|
||||
|
||||
Phase 8 ran the decompilation as a team: three pi sessions in one worktree under a written protocol, with one coordinator that dispatched charters, merged claims and independently verified them, and two workers that matched concurrently (a third worker was recruited mid-phase for the class the others kept deferring). **149 distinct matched bodies / 158 registered regions**, from 34 / 39, against a milestone of 70 — 115 new bodies. All gates exit 0 from a clean state: `make clean`, `make all`, `cmp`, both files SHA-1 `e173426c157384ebf1b6caf8c6fea18a85a14af9`, `make check` at 203 tests with `regions=158 disagreements=0` and `c_regions=158`, 0 differing bytes. The registry audit reports 0 violations across 158 ordered, non-overlapping regions whose every extent is `exact` and whose every source exists, and `git status --short src/` shows no untracked file: a tracked `src/` file is a claim. Every near-miss recorded by Phases 5-7 in the reachable set is closed: `0x800F3160` (Phase 5's store-in-delay-slot anomaly, unresolved for three phases), `0x800F8AEC`, `0x800F8B58/6C`, `0x800F8FE4`, `0x80102B10` via a new per-region `maspsx=off` override; `0x800F7FB4` ×3 via a cc1 loop-shape rewrite (explicit guard + do/while removes an unused frame); `0x8009E8D0` ×2; and the handoff rows `0x8002DEB4` and `0x80019B6C`. Two corrections to the evidence model: the gp marker is per **symbol** but the original's access form is per **site** (`0x80121F84` is read gp-relative at `0x800A80BC` and written absolutely at `0x8002D288`, so a new per-region `gp=-NAME` override expresses both — cookbook finding 10 was incomplete), and `syscall` **returns**, so treating it as a walk terminator had truncated two BIOS stubs to 8 bytes when their bodies are 16. The SDK-shape investigation **refuted its own premise** — the missing PsyQ headers were never the blocker — and registered 7 COP2/BIOS regions under a developer decision (inline assembly accepted for coprocessor and kernel instructions, documented per file, integer logic in C, no `.word` lumps); `include/gtemac.h` records the project's own re-derived COP2 macros and the non-textbook control-register map. It also produced the phase's strongest structural signal: **50 of the 1,937 exact extents contain trapping `add`/`sub` that no available compiler emits** (ten `cc1` builds plus the real `CC1PSX` 4.0-4.6, ~30 C shapes, 15 flags, `-ftrapv` rejected everywhere) while **0 of the 158 registered regions does** — now excluded by `sf3_triage` with a counted reason and the best current candidate for part of the unresolved library-versus-game-code boundary, recorded as a hypothesis with the explicit limit that the detector is sufficient but not complete. Worker B tested worker A's mirrored-layout lever against its own bounded negatives and produced a decision rule instead of a match (the lever applies when the instruction count is right and the block order is wrong); worker A partially retracted its own maspsx load-delay defect claim when the effect did not reproduce. New tracked artifacts: `tools/sf3_merge` (28 tests), `docs/PHASE8_PROTOCOL.md` (hard rules, message types, staging formats, the merge workflow, per-cycle verification, a retrospective and reusable charter templates), `docs/PHASE8_VERIFICATION.md`, `config/near_match_negatives.tsv` (38 open negatives as addresses, sizes, statuses and class labels only), `include/gtemac.h`, and cookbook findings 16-28. 244 tracked files, none under a prohibited root. One convention decided by the developer: inline assembly accepted for coprocessor and kernel instructions. Unresolved and recorded: the trapping-arithmetic class (highest value), the maspsx load-delay observation, the reorg thread-fill and two-epilogue classes, the numeric `-G`, the CRT entry, and library-versus-game-code. No rules were added.
|
||||
|
||||
## Phase 9 — Coordinated Multi-Session Matching at Scale (2026-09-24)
|
||||
|
||||
Phase 9 ran three workers against a 500-body milestone and closed at **400 distinct
|
||||
matched bodies / 409 registered regions** (from 149 / 158 — +251 of each),
|
||||
verified on the coordinator's own clean whole-binary gate (`c_regions=409`,
|
||||
`differing_bytes=0`, SHA-1 `e173426c157384ebf1b6caf8c6fea18a85a14af9`). The
|
||||
staged checkpoints P9-T2 (180), P9-T3 (230), P9-T4 (290) and P9-T6 (400) were
|
||||
all met; P9-T7's 500 was not, and the blocker is recorded per the plan's "Or"
|
||||
branch: measured pool exhaustion — tier-1 leaves are tie-break-dominated (B
|
||||
0-for-12 on the head, every residual 2–22 bytes), the registered corpus is
|
||||
disjoint from the counted unmatchable classes (trapping arithmetic incl. the
|
||||
Phase-9 ADDI extension, 0-of-409; rare-epilogue 47-row census, ~91% of the
|
||||
partition-A subset operationally blocked by the maspsx mutual exclusion;
|
||||
primitive-init family excluded after 4-vs-2), and the per-cycle rate decayed
|
||||
from ~90% to ~20% as the pool thinned (B's leading-indicator rule: a per-batch
|
||||
cluster of sub-8-byte residuals with no new mechanism is band exhaustion).
|
||||
The phase's strategic pivot was the **tier-2 pivot**: leaf bodies leave the
|
||||
allocator free so every small difference shows as a register tie-break, while
|
||||
frame/call bodies pin the allocation and match at ~90% first attempt — that
|
||||
finding drove the endgame's **11 consecutive first-attempt matches** on one
|
||||
callee family (matched as a set via the family-target scan). The queue fix —
|
||||
`sf3_triage` excluding `recorded_negative` (39 of the first 40 head rows had
|
||||
been already-attempted negatives) plus `delay_slot_start`, `bad_extent_start`
|
||||
and the ADDI-inclusive trapping extension — was the phase's highest-value
|
||||
efficiency change. `tools/sf3_diff` promoted the workers' instruction diff with
|
||||
header-derived LOAD and ready-to-paste symbol resolution. The merge flow was
|
||||
hardened through documented incidents: silent duplicates (0x8010A748 ×2,
|
||||
0x80065930, 0x800A5CC8), a deleted tracked source (worker B, self-caught, the
|
||||
`git ls-files`-before-`rm` rule), dependent-symbol claims (0x8002E3FC /
|
||||
g_80121BFA), and bad-claim gate isolation. Three clean rotations (A 108 claims,
|
||||
C 71, B 60 — B the milestone driver); no replacements available, the
|
||||
coordinator absorbed then correctly stopped at 80%+ context to protect the
|
||||
gate role. Cookbook grew to 53 findings, including the GTE control-register map
|
||||
correction (unshifted 0..5, standard+7 from RBK), the three-encoding-trap
|
||||
synthesis (`lui` high halves / name-keyed gp / literal-vs-symbol — one class,
|
||||
diff the encoding not the mnemonic), and the family-set lever with its scope
|
||||
limit (stable positions transfer, the full map does not). Closing checklist all
|
||||
green from clean: `make test` 229, `make gate` MATCH, extents AGREE,
|
||||
`excluded_already_registered=409` with a 1,342-row worklist, 0 firewall
|
||||
violations, 0 untracked `src/` files. New rules: `git ls-files`-before-`rm`,
|
||||
the dependent-claim rule, the leading-indicator rule, and compute-don't-
|
||||
eyeball (extended to two-piece constants). Next: Phase 10 plan is written
|
||||
(`phase-ends/Phase10_PLAN.md`) and requires explicit developer approval. No
|
||||
changes to AGENTS.md.
|
||||
|
||||
@@ -0,0 +1,181 @@
|
||||
# Phase 10 Plan — Matching the Remaining Pool and Closing the Library Boundary
|
||||
|
||||
**Status:** DRAFT — requires explicit developer approval before activation. Do not
|
||||
begin any task until the developer approves this plan.
|
||||
**Prepared:** 2026-09-24, immediately after Phase 9 closure (400/500 bodies).
|
||||
**Carried forward from Phase 9:** the tier-2 pivot, the family-set lever, the
|
||||
class exclusions, the hardened merge flow, the three-encoding-trap habit, and
|
||||
the honest pool analysis below.
|
||||
|
||||
## 0. The honest starting point (why 500 was not reached, and what that means)
|
||||
|
||||
Phase 9 closed at **400 distinct bodies / 409 regions** with all gates green.
|
||||
The missing 100 were not a process failure; the worklist thinned to a measured
|
||||
tail:
|
||||
|
||||
- **Tier-1 leaves are tie-break-dominated** (worker B 0-for-12 on the head,
|
||||
every residual 2–22 bytes of pure register/operand-order).
|
||||
- **The registered corpus is disjoint from every counted unmatchable class**:
|
||||
trapping arithmetic (now incl. ADDI, 0-of-409), rare-epilogue (47 worklist
|
||||
rows, ~91% of partition-A subset blocked by the maspsx mutual exclusion),
|
||||
primitive-init family (excluded 4-vs-2). These are toolchain limits, not
|
||||
source problems.
|
||||
- **Per-cycle rate decayed ~90% → ~20%** as the pool thinned (the
|
||||
leading-indicator rule).
|
||||
|
||||
**Consequence for planning:** Phase 10 must not be "match more of the same
|
||||
worklist". The worklist count (1,342 at close) overstates the matchable
|
||||
remainder under the current toolchain story. Phase 10 has two possible
|
||||
emphases, and the developer picks the strategy: **(A) squeeze the remaining
|
||||
matchable tail** with the strongest levers, and **(B) attack the blocked
|
||||
classes** — either by finding the real compiler/library story for them (the
|
||||
only route that re-opens hundreds of rows) or by explicitly accepting them as
|
||||
out of scope and recording why.
|
||||
|
||||
## 1. Goal
|
||||
|
||||
Either (developer chooses):
|
||||
|
||||
- **Goal A (tail squeeze):** reach **450–475 distinct bodies** by matching the
|
||||
non-blocked remainder at the measured tier-2/leaf boundary, concentrating on
|
||||
the family-set lever and the fresh-band volume; or
|
||||
- **Goal B (library boundary):** close the **trapping-arithmetic class** and the
|
||||
**rare-epilogue class** by identifying the actual compiler/source story for
|
||||
them (the plausible candidates are Sony/SN toolchain or a different PsyQ
|
||||
build), which — if confirmed — re-opens hundreds of rows at once; or
|
||||
- **Goal C (hybrid):** A as the primary throughput, B as the bounded
|
||||
investigation (Phase 9's P9-T5 style), with the explicit understanding that
|
||||
B's success is the only route past ~475 under the current toolchain.
|
||||
|
||||
The milestone is stated precisely and measured (bodies on the clean whole-
|
||||
binary gate), never as "keep matching".
|
||||
|
||||
## 2. The carried machinery (do not re-derive; verify before relying)
|
||||
|
||||
- Toolchain: PsyQ 4.0 `CC1PSX` (`GCC 2.7.2`-based; `gcc-2.7.2-psx` is
|
||||
instruction-identical), ASPSX 2.56 via maspsx, GNU as. Flags `-quiet -O2 -G0`.
|
||||
- Harness: `tools/sf3_match` with per-region overrides (`cc1=`, `as=`,
|
||||
`gp=-NAME`, `maspsx=off`); `tools/sf3_diff` for instruction-level diffing.
|
||||
- Evidence tables: `function_inventory`, `function_extents`,
|
||||
`duplicate_bodies`, `match_worklist`, `near_match_negatives` (115 at close).
|
||||
- **The hardened merge flow** (the phase's biggest process win): verify →
|
||||
merge to CANDIDATE → gate whole-binary → promote only on MATCH → reconcile
|
||||
negatives into the tracked index → regenerate worklist + partitions → commit.
|
||||
All documented in `docs/PHASE8_PROTOCOL.md` and the Phase 9 ledger archive
|
||||
(`phase-ends/logs/Phase9.md`).
|
||||
- **Rules already in force** (carry into every charter): `git ls-files`-before-
|
||||
`rm` under `src/`; dependent-claim (symbol merged before the claim); the
|
||||
leading-indicator rule (per-batch sub-8-byte residual cluster = band
|
||||
exhaustion — re-partition or re-charter, don't grind); compute-don't-eyeball
|
||||
(lui high halves, gp offsets, two-piece constants).
|
||||
|
||||
## 3. The levers that worked (byte-proven, with their scope limits)
|
||||
|
||||
- **The family-set lever** (highest yield): after a match, scan for other rows
|
||||
calling the same target; the argument map is derived once. LIMIT (Phase 9
|
||||
correction): stable positions transfer (selector→arg2, sentinel→arg3,
|
||||
narrowed `a1`→arg9, extra→arg10 in the 0x80057664 family), but per-row
|
||||
register arguments must be re-read every time.
|
||||
- **The tier-2 pivot** (the strategy): frames pin the allocation; small leaves
|
||||
are tie-break-dominated. Match framed/call bodies first.
|
||||
- **The named-locals lever family** with its three distinct mechanisms:
|
||||
order-pinning, reload-prevention (aliasing), and — inverted — the cases where
|
||||
a repeated expression is correct and a local is the mistake.
|
||||
- **The three-encoding-trap habit**: when the count is already right and the
|
||||
residual is 1–8 bytes, diff the ENCODING not the mnemonic — this one habit
|
||||
catches `lui` high halves, name-keyed gp markers, and literal-vs-symbol
|
||||
addresses.
|
||||
- Per-region overrides as byte-required, never speculative: `maspsx=off` (only
|
||||
for bodies with no `move` pseudo), `gp=-NAME` (per-site absolute), `cc1=-O0`.
|
||||
|
||||
## 4. The blocked classes — the investigation (Goal B) or the explicit scope line
|
||||
|
||||
| Class | Measured | Route |
|
||||
|---|---|---|
|
||||
| Trapping arithmetic (R + ADDI) | 0-of-409 registered; excluded count 54+ | Find/produce a cc1 whose macro style emits the trapping forms (the `neg`→`sub` GNU-as clue, Sony SN build, PsyQ 3.x) — bounded, two approaches then stop; OR record as a different-compiler signature and keep excluded |
|
||||
| Rare-epilogue order | 47 worklist rows at final instructions; three-way model (a: cc1 fills, b: maspsx blocks → addu-move conflict, c: cc1 nop) | Formerly "unmatchable" — worker B proved (a) matches with default settings; test each row individually, try `--no-maspsx` on epilogue-only residuals, accept the mutual exclusion as the harness limit where it bites |
|
||||
| Primitive-init family | excluded 4-vs-2 | remains excluded; a future per-region transform (post-pass or suppressing maspsx's jump-slot nop) is the only route — a HARNESS decision, developer-owned |
|
||||
| `0x8008A198` maspsx load-delay | blocked, excluded | newer pinned maspsx or a tracked post-pass — developer-owned |
|
||||
| GTE loader allocation (`0x80102FE4` family) | load-use-nop, allocation-pinning untried | register-binding lever named but failed on the counter case; pointer-live-range variant untried |
|
||||
|
||||
## 5. Structure — coordinator + workers, scaled down or up
|
||||
|
||||
- **Session count:** depends on Goal. A/C hybrid wants one coordinator + two
|
||||
workers (fresh context beats exhausted context — Phase 9's strongest lesson);
|
||||
a pure B investigation needs only the coordinator + one investigator.
|
||||
- The Phase 8/9 protocol applies verbatim (roles, hard rules, staging formats,
|
||||
merge workflow, per-cycle verification). Phase 9's additions — 3-way
|
||||
partition regeneration per merge, the negatives import, the request-file
|
||||
prune, the `ls-files` rule — are extensions to the same contract.
|
||||
- **Rotation discipline** (Phase 9's lesson, now formal): workers re-flag on the
|
||||
measured criterion only (two consecutive zero-match cycles with
|
||||
attempts-per-match), or by quoting their context figure — never on feel.
|
||||
Budget exhaustion is planned: clean handoff → fresh session or coordinator
|
||||
absorption, recorded each time.
|
||||
|
||||
## 6. Verification ladder (unchanged from Phase 9, which proved it)
|
||||
|
||||
| Frequency | Checks |
|
||||
|---|---|
|
||||
| Every merge | candidate gate (whole-binary, SHA-1 fixed) → promote → `make check` |
|
||||
| Every 3rd merge | `make clean && make all`, cmp, SHA-1, registry audit, `git status --short src/`, firewall count |
|
||||
| Phase close | full clean gates, worklist regen with `excluded_already_registered` = registry size, negatives metadata extracted, milestone confirmation requested |
|
||||
|
||||
## 7. Explicit decisions needed from the developer before activation
|
||||
|
||||
1. **Goal choice (A / B / C)** and the milestone number (450 / 475 / class-closed
|
||||
definitions).
|
||||
2. **The harness-limitation questions** if Goal B is chosen: whether a per-region
|
||||
transform (post-pass emitting the trapping encodings, or suppressing
|
||||
maspsx's jump-slot nop) is acceptable as a documented convention, or whether
|
||||
the classes stay excluded as "different-compiler evidence" — this is the
|
||||
Phase 8-era inline-asm-style decision, and it is the developer's call, not
|
||||
the coordinator's.
|
||||
3. **Session budget:** confirm the worker roster (2 fresh? re-engage B/C? — their
|
||||
contexts are 60%+/rotated) and the runtime envelope.
|
||||
4. **Scoring honesty:** confirm the milestone stays "bodies on the clean gate",
|
||||
and that a class-closed result (e.g. all 47 rare-epilogue rows classified
|
||||
into the three-way model) counts as progress even when it does not add a
|
||||
body.
|
||||
|
||||
## 8. Preconditions
|
||||
|
||||
- `make check` green at 409 regions / 400 bodies (Phase 9 close state).
|
||||
- The Phase 9 closure commit (`PhaseEnd_Phase9.md`, digest, ledger archive) is
|
||||
on main.
|
||||
- Worker sessions probed before any charter (capability probe from the
|
||||
protocol); recruited ids recorded in the Phase 10 ledger.
|
||||
|
||||
## 9. Scope and safeguards
|
||||
|
||||
- ROM/SDK firewall unchanged; worker staging ignored; `git clean -x`/`-fdx`
|
||||
forbidden.
|
||||
- Workers may not weaken a gate, widen regions, or register a non-`exact`
|
||||
extent. Inline asm only for coprocessor/kernel instructions (Phase 8
|
||||
decision).
|
||||
- The blocked classes stay excluded until proven — a Phase 10 worker does not
|
||||
spend budget on the trapping/rare-epilogue rows outside the bounded
|
||||
investigation.
|
||||
- All unmatched content remains the data fallback; the all-payload baseline
|
||||
and the ordered-layout harness are untouched.
|
||||
|
||||
## 10. Tasks (draft — to be filled per the approved Goal and developer decisions)
|
||||
|
||||
- **P10-T1** — control records, clean-baseline revalidation, session probing,
|
||||
partitions. Milestone: baseline green; sessions recruited and recorded.
|
||||
- **P10-T2** — the approved goal's first work cycle (tail squeeze and/or the
|
||||
bounded class investigation): first checkpoint per the Goal's staged target.
|
||||
- **P10-T3+** — the autonomy loop with per-cycle regeneration, full audits every
|
||||
3rd merge, and rotation per the measured criterion.
|
||||
- **P10-TN** — phase close: cookbook/conventions/verification records, negatives
|
||||
extraction, milestone confirmation requested, PhaseEnd, digest update, stop.
|
||||
|
||||
Milestone: **the approved goal met on the clean whole-binary gate, or the
|
||||
evidence-backed blocker recorded** — including, if Goal B, the measured answer
|
||||
to whether the blocked classes are a different compiler (which would re-frame
|
||||
the entire library-versus-game-code question) or a harness gap the developer can
|
||||
close.
|
||||
|
||||
**Phase 10 closes only after the developer confirms the milestone; then write a
|
||||
PhaseEnd record, update the digest, archive the ledger, commit, and stop without
|
||||
beginning Phase 11.
|
||||
@@ -0,0 +1,164 @@
|
||||
# PhaseEnd — Phase 9: Coordinated Multi-Session Matching at Scale
|
||||
|
||||
**Date:** 2026-09-24
|
||||
**Phase Status:** CLOSED — milestone **partially met** (400 of 500 bodies) with the
|
||||
blocker for the unmet part recorded per the plan's "Or" branch.
|
||||
**Milestone confirmed by developer:** yes (wrap-up directive given; full-binary
|
||||
gate independently verified).
|
||||
|
||||
## Headline number
|
||||
|
||||
| Measure | Phase 8 close | Phase 9 close | Delta |
|
||||
|---|---|---|---|
|
||||
| Registered regions | 158 | **409** | +251 |
|
||||
| Distinct matched bodies | 149 | **400** | +251 |
|
||||
| Tracked symbols | 367 | ~430 | +~60 |
|
||||
| Worklist eligible (final) | 1,730 | **1,342** | −388 by reg/exclusion |
|
||||
| Top-level milestone (500) | — | **400** | 100 short, blocker recorded |
|
||||
|
||||
## The milestone statement, stated precisely
|
||||
|
||||
**The phase did not reach 500. It reached 400 distinct bodies, verified on the
|
||||
coordinator's own clean whole-binary gate** (`c_regions=409`, `differing_bytes=0`,
|
||||
SHA-1 `e173426c157384ebf1b6caf8c6fea18a85a14af9`). The plan's staged
|
||||
checkpoints — P9-T2 (180), P9-T3 (230), P9-T4 (290), P9-T6 (400) — were all
|
||||
met. P9-T7 (500) was not, and the plan's "Or" branch applies: **the blocker is
|
||||
recorded, not papered over.**
|
||||
|
||||
## Blocker evidence for the unmet 100 bodies (P9-T7)
|
||||
|
||||
The milestone shortfall is not a process failure; it is a **measured pool
|
||||
exhaustion** with three documented causes:
|
||||
|
||||
1. **Tier-1 leaves are tie-break-dominated, not hard.** Measured repeatedly:
|
||||
worker B 0-for-12 on the tier-1 head (every residual 2–22 bytes, pure
|
||||
register/operand-order), and the coordinator's own absorption hit the same
|
||||
wall. The remaining 192 tier-1 rows carry the same signature.
|
||||
2. **The registered corpus is disjoint from the known-unmatchable classes.**
|
||||
Trapping arithmetic (R-type AND the Phase-9-extended ADDI form) — 0 of 409
|
||||
registered regions contains one. Rare-epilogue-order class: 47 worklist rows
|
||||
at their final instructions; the three-way model shows ~91% of the
|
||||
partition-A subset operationally blocked by the maspsx mutual exclusion.
|
||||
Primitive-init family: excluded after 4 failures vs 2 successes. These are
|
||||
measured compiler/toolchain limits, not source-shape problems.
|
||||
3. **Per-cycle rate decayed as the pool thinned.** The endgame measured it
|
||||
explicitly: family/fresh-band yields were ~90% on fresh bands and dropped to
|
||||
~20% as the remaining rows converged on the tie-break tail (worker B's
|
||||
leading-indicator rule: a cluster of sub-8-byte residuals with no new
|
||||
mechanism = band exhaustion). Three worker rotations (A, B, C) all reported
|
||||
this same shape, and no replacement sessions were available.
|
||||
|
||||
The 1,342-row remaining worklist is therefore not 1,342 matchable bodies; the
|
||||
matchable remainder within the current toolchain is materially smaller, and the
|
||||
unmatchable remainder is already excluded by counted, evidence-backed classes.
|
||||
|
||||
## What the phase delivered beyond the count
|
||||
|
||||
### Tools and workflow (all tracked, synthetic-tested)
|
||||
|
||||
- **`tools/sf3_triage` hardening** (coordinator implementations of worker
|
||||
findings): `delay_slot_start` (5, P9-T1), `bad_extent_start` (8, P9-T2),
|
||||
ADDI-inclusive `trapping_arithmetic` (54, P9-T4), and
|
||||
`recorded_negative` exclusion (115) — the last one was **the phase's
|
||||
highest-value efficiency fix**: it stopped 39-of-40 head rows being
|
||||
already-attempted negatives.
|
||||
- **`tools/sf3_diff`** — the promoted instruction-diff tool with header-derived
|
||||
LOAD, repo-derived objdump, caller-supplied temp dir, and ready-to-paste
|
||||
symbol resolution.
|
||||
- **The merge flow** hardened into a reproducible discipline: verify → merge to
|
||||
CANDIDATE → gate whole-binary → promote only on MATCH → reconcile negatives →
|
||||
regenerate worklist + partitions → commit. Handled, with documented lessons:
|
||||
silent duplicates (0x8010A748 ×2, 0x80065930, 0x800A5CC8), stale tracked
|
||||
sources (0x80025070), dependent-symbol claims (0x8002E3FC/g_80121BFA),
|
||||
bad-claim gate isolation, and the tracked-deletion incident (worker B) with
|
||||
the `git ls-files`-before-`rm` rule.
|
||||
|
||||
### The tier-2 pivot (the phase's strategic recovery)
|
||||
|
||||
Worker B's measured finding — tier-1 leaves are tie-break-dominated, tier-2
|
||||
frame bodies pin the allocation and match at ~90% first attempt — became the
|
||||
phase's operational rule and drove the endgame: **11 consecutive first-attempt
|
||||
matches** on a single callee family (the 0x80057664 family, matched as a SET via
|
||||
the family-target scan). The milestone-close directive (reliable rows over
|
||||
exploration) converted the final stretch.
|
||||
|
||||
### Cookbook additions (53 new findings, F15–the family-set lever)
|
||||
|
||||
Carried in `docs/MATCHING_COOKBOOK.md` and `docs/MATCHING_CONVENTIONS.md` and
|
||||
the phase protocol, including: the GTE control-register map correction (the
|
||||
executable uses the standard map unshifted for 0..5 and standard+7 from RBK
|
||||
onward — two worker decodes + coordinator raw-word verification), the
|
||||
three-encoding-trap synthesis (`lui` high halves, name-keyed gp markers,
|
||||
literal-vs-symbol addresses — one class: count right, residual 1–8 bytes,
|
||||
diff the ENCODING not the mnemonic), the ADDI trapping extension, the
|
||||
rare-epilogue three-way model, and the family-set lever with its precise
|
||||
scope limit (stable positions transfer, the full map does not).
|
||||
|
||||
## Rotations (all clean, all recorded)
|
||||
|
||||
- **A** (108 claims, +79 bodies): rotated at 52% after its budget rolled over;
|
||||
supplied 8 levers and the bad-extent triage class.
|
||||
- **C** (71 claims, +29 in the final stretch): rotated on the MEASURED criterion
|
||||
(1/4 batch + two self-recorded-rule breaks); produced the cleanest handoff
|
||||
(rareepi.py, lever list, 38 negatives by class).
|
||||
- **B** (60 claims, +47: the milestone driver): rotated after the milestone
|
||||
gate; produced the leading-indicator rule and the 11-match family run.
|
||||
- No replacement sessions were available (intercom roster fixed at 4); the
|
||||
coordinator absorbed when workers thinned, then correctly stopped absorbing
|
||||
at 80%+ context to protect the merge/gate role.
|
||||
|
||||
## Verification (closing checklist, all from clean)
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| `make clean && make all` | exit 0 |
|
||||
| `cmp build/scus_946_40.rebuilt 'extracted/SCUS_946.40;1'` | exit 0 |
|
||||
| SHA-1 both files | `e173426c157384ebf1b6caf8c6fea18a85a14af9` |
|
||||
| `make test` | 229 tests OK |
|
||||
| `make gate` | `c_regions=409 differing_bytes=0 result=MATCH` |
|
||||
| `make extents-verify` | `regions=409 disagreements=0 result=AGREE` |
|
||||
| Worklist regen | `excluded_already_registered=409`, `listed=1,342` |
|
||||
| Firewall | 0 tracked under prohibited roots |
|
||||
| `git status --short src/` | 0 (every tracked src file is a registered claim) |
|
||||
| `git diff --check` | exit 0 |
|
||||
|
||||
## Rules and conventions added
|
||||
|
||||
- **`git ls-files --error-unmatch` before any `src/` deletion**: a tracked
|
||||
`src/` file is a claim; only untracked paths are a worker's to delete.
|
||||
- **Dependent-claim rule**: a claim referencing a not-yet-tracked symbol is
|
||||
reported DEPENDENT (symbol merged first), never verified against a superset.
|
||||
- **Leading-indicator rule** (worker B): a per-batch cluster of sub-8-byte
|
||||
residuals with no new mechanism is the band-exhaustion signal.
|
||||
- **Compute-don't-eyeball**: `lui`+signed-displacement, gp+offset, AND
|
||||
two-piece constants like `lui 0xff` + `ori 0xff00` = `0x00FFFF00`.
|
||||
|
||||
No changes to `AGENTS.md`.
|
||||
|
||||
## Next (Phase 10 plan, separate document)
|
||||
|
||||
The Phase 10 plan is written as `phase-ends/Phase10_PLAN.md` and must be
|
||||
explicitly approved before any task starts. It carries forward: the
|
||||
family-set lever, the class exclusions, the merge flow, the three-encoding-trap
|
||||
habit, and the honest pool analysis (the matchable remainder is smaller than
|
||||
the worklist count).
|
||||
|
||||
## Plain-English recap
|
||||
|
||||
Phase 9 was supposed to take the project from 149 matched functions to 500. It
|
||||
got to 400 — every check-point target before the final one — verified on the
|
||||
phase's own whole-binary gates, and the missed 100 are blocked by measured
|
||||
toolchain limits, not by the team stopping. More importantly, the phase produced
|
||||
the machinery and the knowledge that make the remaining 100 (or a higher target
|
||||
under a different toolchain story) reachable: the tier-2 pivot that revealed
|
||||
where the real volume was, the family-as-a-set matching that turned the endgame
|
||||
into an 11-match streak, the queue fix that stopped workers re-attempting known
|
||||
negatives, and three clean worker rotations that each left the next one a
|
||||
better map than the last. The project now rebuilds the original executable
|
||||
byte-for-byte from 400 C functions.
|
||||
|
||||
## 🛑 Stop Here
|
||||
|
||||
`CURRENT_PHASE.md` is archived as `phase-ends/logs/Phase9.md`; this PhaseEnd and
|
||||
the digest update belong in the closure commit. Do not begin Phase 10 execution
|
||||
in this session — the Phase 10 plan requires explicit developer approval first.
|
||||
Reference in New Issue
Block a user