+2 bodies (worker C claims 19-20) and 3 gp symbol rows (D_80121F90,
D_8012196E, D_80121964). Candidate gate MATCH before promotion.
NEW EXCLUSION CLASS `restores_unsaved` (worker A, coordinator-verified). A body
that restores a callee-saved register it never saves cannot be a whole function:
the register it restores was established by an enclosing prologue that the derived
extent cut off. These are jal targets INSIDE a real function, so the walk began
mid-body — distinct from bad_extent_start, which flags starts that are not
function entries at all.
Verified disjoint from the matched corpus before acting, as the project requires:
0 of 462 registered regions trip the rule; 11 worklist rows do.
excluded_restores_unsaved=11; worklist 1265 -> 1253 rows. 5 new tests; suite
229 -> 237, OK. make check green: regions=462 AGREE, differing_bytes=0 MATCH.
Independent corroboration worth recording: the new rule re-derives 0x8010080C,
the false extent start worker C reported earlier via a completely different signal
(the first instruction reads a register the range never defines). Two independent
detections of one defect class. bad_extent_start is False for that row, confirming
C's observation that the older rule missed it. Its Makefile --exclude entry is
retained only as the provenance record for that defect.
32 new bodies from 400, all verified on the candidate whole-binary gate before
promotion. SHA-1 e173426c157384ebf1b6caf8c6fea18a85a14af9 stable.
Registry requests granted (each byte-verified with a failing control):
cc1=-G8 on 0x800A6BEC; gp=-D_80121B88 on 0x80015D50
symbols D_80122700, D_80122704, D_80121AD4 (gp)
Harness: per-region maspsx modes wired through sf3_match (maspsx=noreordernop,
maspsx=regread) plus --no-jump-slot-nop/--nop-on-reg-read for range. Both are
opt-in and default-off; make check green at 441 with them off, suite 229 -> 232
tests. Carried as a TRACKED patch (tools/patches/maspsx-phase10-r1r2.patch)
because tools/maspsx/ is git-ignored, so an in-place edit would not survive a
fresh clone; patch verified to reproduce the working tree byte-identically.
R1/R2 are recorded as a MEASURED NEGATIVE: neither closes a region (cookbook
finding 40 has the mechanism and the remaining developer-owned route).
Docs: cookbook finding 40 (rare-epilogue mechanism + why the obvious maspsx fix
fails); SETUP.md maspsx patch provenance and apply step.
Negatives: 0x8010AA28 imported; index sorted by address (140 rows, 0 registered).
Full clean audit green: make clean && make all exit 0, cmp exit 0, both SHA-1
match, registry 441/0 overlaps/0 bad extents/0 missing sources, 0 firewall.
Worker B measured that recorded negatives cluster at the head of every
partition: 39 of the first 40 worklist rows and 57 of 271 tier-1 rows (21%)
were addresses an earlier session had already attempted and recorded. The
triage tool never read the negatives index, so every closed negative
returned to the queue head and workers rediscovered them.
Fix: sf3_triage reads the tracked config/near_match_negatives.tsv (a new
--negatives input) and excludes those rows as recorded_negative=69. The
tracked index was grown from 38 Phase-8 rows to 94 by importing 56 Phase-9
negatives from the workers' reports and the coordinator's absorption record
(filtered to unregistered; tracked rows win on conflict). Worklist
regenerated: the head is now genuinely fresh (0x800F9134, 0x80042D88,
0x800450C4, ...). 3 new synthetic tests; plan-level tests wired for the new
argument. The phase plan said extract negatives only at close; B's
measurement showed the harm is per-cycle, so the index is now a living
tracked input. Note for P9-T8: keep importing worker negatives each cycle.
Worker B found the exclusion detector's blind spot: opcode 0x08 (addi, traps
on overflow) appears in 7 of 1,666 worklist rows (frame adjustments
addi sp,sp,-28, loop counters) while 0 of 310 registered regions contains
one — the same disjointness signal as finding 26's R-type class. addiu (0x09)
is ubiquitous, so the 0x08 occurrences are the anomaly. Verified by the
coordinator from raw words: 7 rows, incl. 3 of partition A's 5 call-shape
rows (0x80011084, 0x800F3B10, 0x80010F30) and 0x800F3BB4 (previously
recorded as a separate coordinator negative — now explained by this class).
sf3_triage.trapping_arithmetic now counts opcode 0x08; worklist regenerated
(1,563 rows, excluded_trapping_arith=54). Synthetic test added.
P9-T5 route (a) widened: the immediate form is part of the class; the 7
ADDI rows are a ready-made minimal probe set for any cc1 candidate.
Also confirmed this cycle: the no-frame global-ra ISR family (ra saved to a
global, no stack frame — 0x801097A0 proved, 0x8010B420 pattern-transferred)
is a second library-asm class identical in kind to the CRT entry.
Worker A reported nine worklist rows graded exact that are not function
starts. The coordinator verified the tell independently: a candidate whose
first instruction reads a register the range never defines (non-parameter),
or whose range has no jr/jalr return, is a wrong extent. Measured disjoint
from the matched corpus (7 worklist rows flagged incl. the runaway
0x801800C4 and the fallthrough 4-byte rows 0x80100808/0x80180808; 0 of 288
registered flagged). Implemented as excluded_bad_extent_start in sf3_triage
with 4 synthetic tests; worklist regenerated to 1,570 eligible rows.
P9-T1: create CURRENT_PHASE.md as the phase ledger; baseline revalidated
clean (cmp + SHA-1 e173426c157384ebf1b6caf8c6fea18a85a14af9, make check
regions=158 disagreements=0 c_regions=158 MATCH); three workers probed and
recruited (01a0d180-ea72/-efdd/-f513); 3-way partitions generated and proved
disjoint (3x575 rows, pairwise intersections 0, union == worklist).
P9-T2 (tooling half): promote worker B's scratch instruction diff into
tools/sf3_diff, hardened per the plan: PS-X LOAD read from the header,
objdump path derived from the repo, temp files under the caller's work dir,
resolver printing ready-to-paste symbol names. 17 synthetic tests.
sf3_triage: new counted exclusion excluded_delay_slot_start=5, implemented
after independent verification of worker B's finding (a branch/jump always
executes its delay slot, so no function starts at X+4; 0 of 158 registered
regions does). Worklist regenerated: 1725 eligible rows, tiers 355/1365/5.
3 new synthetic tests; existing triage fixture extended.
Worker C found the strongest structural signal of the phase and asked me to act
on it rather than merely record it. I re-derived the census independently before
acting: 50 of the 1,937 exact extents contain trapping add/sub (funct 0x20/0x22,
410 instructions) and 0 of the 144 then-registered regions does. All four
unmatched duplicate groups are inside the class. Ten cc1 builds plus the real
CC1PSX 4.0-4.6 over ~30 C shapes and 15 flags never emit the trapping forms, and
-ftrapv is rejected by all of them.
tools/sf3_triage now detects the class and excludes it by default
(excluded_trapping_arith=50, --allow-trapping to list it), with three synthetic
tests, so no worker can spend budget on it by accident. Recorded as cookbook
finding 26 with the neg-macro clue that points at a macro-print-style difference
rather than a source difference. The class is now the best candidate for part of
the unresolved library-versus-game-code boundary - a hypothesis, not a claim.
Also recorded: worker A's maspsx load-delay defect (its predicate tests whether
the next instruction loads FROM the register, not whether it reads it at all, so
a load followed by a store of that register gets no delay nop), four mechanical
layout tells, and the struct-assignment lever for the base-in-register class.
Finding 22 is refined with worker C's third spelling: the written order of the +
operands decides the emitted addu order.
Cycle 5 merge: 65 claims, 8 accepted, 57 skipped, 0 rejected. Candidate gate
c_regions=152, 0 differing bytes; promoted; make check green at 203 tests.
Distinct matched bodies: 143.
Worker C found that config/function_extents.tsv mis-split the two BIOS stubs:
it recorded 0x80103FCC..0x80103FD4 (8 bytes) and 0x80103FEC..0x80103FF4, but the
real bodies are 16 bytes (li a0,N / syscall / jr ra / nop). Its evidence: the
first address has four jal callers and the following address has none, syscall
returns to EPC+4 so the body must continue to a jr ra, every neighbouring stub is
16 bytes with the same shape, and an 8-byte body is unreachable from C because
cc1 always appends an epilogue.
Root cause: the walk treated `syscall` as a terminal alongside `break`. On this
target syscall is the BIOS call instruction and it returns to the next word.
`break` still terminates. After the fix both stubs are 16 bytes ending at their
jr ra, and all 115 registered regions still agree with their derived extents.
The fix also exposed three walks in the 0x8017xxxx region that the accidental
syscall stop had been bounding: 0x8017D5C0 now runs out of the payload
(grade=outside), 0x801800C4 becomes a 1860-byte fallthrough, and 0x8018080C a
240708-byte indirect. All three are jal-graded starts whose bodies are almost
certainly data; none is registered and none is matchable. 200 tests pass.
Two harness gaps were closed as per-region overrides, both found by the workers
and implemented by the coordinator:
1. maspsx=off. Worker A isolated the $at-macro-store-in-the-jr-delay-slot shape
with a decisive experiment: cc1 emits an empty delay slot, maspsx fills it with
a nop, and GNU as in reorder mode fills it properly by moving the last half of
the expanded store. Six regions need this, including 0x800F3160 -- Phase 5's
first recorded bounded negative, unreachable for three phases -- and 0x800F8AEC,
0x800F8B58, 0x800F8B6C, 0x800F8FE4, 0x80102B10. Worker B reverse-engineered the
same mechanism independently from the cc1 output.
2. gp=-NAME. The gp marker is per symbol, but the original's access form is per
SITE: 0x80121F84 is read gp-relative at 0x800A80BC and written absolutely at
0x8002D288, in the same function. A byte search confirmed 6 gp-relative
accesses at d=0x64c, so worker B's census row was right and worker A's source
was right; only a per-region exclusion can express both. This makes cookbook
finding 10 incomplete.
Both overrides are validated (199 tests, 10 added), and sf3_merge gained
--skip-registered because a worker's claims file is naturally cumulative.
Cycle-2 merge: 66 claims, 58 accepted, 8 skipped, 0 rejected. Candidate gate:
c_regions=115, 0 differing bytes, SHA-1 e173426c. Promoted, then make check green:
199 tests, regions=115 disagreements=0, c_regions=115 MATCH.
Distinct matched bodies: 51 -> 109, past the phase milestone of 70.
P8-T2 plus the first verified cycle-1 merge.
tools/sf3_merge validates worker claims before anything tracked changes: a claim
is accepted only if its extent exists in the derived extents table and is graded
exact with exactly the claimed end, its source is a repo-relative
src/func_XXXXXXXX.c that exists, and it overlaps neither the registry nor another
worker's claim. Rejections are reported with reasons and nothing is written.
The workflow it enables is stronger than the plan's wording: merge to a CANDIDATE
registry, gate the candidate, and promote only on MATCH, so the tracked registry
never contains an unverified claim.
Worker A reported 10 claims (target 8) and correctly refused to edit the shared
symbol registry itself, instead requesting 9 gp-marked rows; each was checked
arithmetically as gp + d. Candidate gate: c_regions=49, 0 differing bytes, SHA-1
e173426c. Promoted, then make check green: 189 tests, regions=49
disagreements=0, c_regions=49 MATCH.
Worker A also reported a significant blocker: two of the remaining duplicate
groups are GTE (COP2) bodies -- 0x80018CB0 (3 addresses) and 0x8001084C (2
addresses, the 712-byte shared body) -- plus 0x80103A94 and 0x80103B60. That is
four GTE functions, which raises the value of the bounded SDK-shape
investigation in P8-T5.
Every cross-reference used to need a hand-written config/symbols.tsv row, which
does not scale to a batch. A symbol whose name is an address now resolves to that
address with no row: func_XXXXXXXX, D_XXXXXXXX, g_XXXXXXXX, lbl_XXXXXXXX. This is
the convention the registry already used, applied without the row.
The names come from the object's own undefined-symbol list (nm -u), not from a
guess about the source, so a name the source defines but never references is
never mistaken for one needing resolution. A registry row still wins, which is
how a gp marker or a real name is attached. Anything neither registered nor
address-shaped now fails before the link with a message naming the symbol and the
fix, instead of a bare ld diagnostic.
First match from the worklist: func_800F8F9C (36 bytes, worklist rank 3), a
duplicate-group representative with a frame and a call. It matched on the first
attempt and is registered twice (0x800F8F9C, 0x80109314) against one source, so
two functions were matched for one body. Its callee needed no registry row.
Ghidra's independent body agrees with the derived extent.
make gate: c_regions=14, 0 differing bytes, SHA-1 e173426c. 168 synthetic tests.
Phase 6 chose match targets by eye from the boundary inventory. tools/sf3_triage
now ranks every eligible candidate by (tier, size, address) from tracked inputs
alone and records why everything else was excluded.
Eligibility: an exact or fallthrough extent, a non-degenerate body, not already
registered, not the header entry, not named by --exclude. indirect, escape,
outside, runaway, contained and standalone are excluded and counted.
Tiers: 0 duplicate-group representative (one match, several addresses), 1 exact
leaf (no cross-references, so no symbol rows), 2 exact non-leaf, 3 fallthrough.
Result: 1916 listed (tier 0: 9, tier 1: 509, tier 2: 1394, tier 3: 4), with 252
degenerate bodies, 88 low-confidence grades, 12 registered, 1 header entry and
2 named near-misses excluded. The nine tier-0 entries are the real duplicate
groups: matching those nine bodies registers 22 function addresses.
The two deferred near-misses are excluded by name in the Makefile so the
exclusion stays visible rather than buried in the tool. 160 synthetic tests pass.
Matching conventions require a duplicate check before registering, because a
shared body is matched once and registered once per address. Phase 6 did that
check by hand and found one 12-byte pair. tools/sf3_dupes now hashes every
derived extent body and groups exact duplicates.
Results: 2284 extents, 65 multi-address groups, 2104 singletons. Only 10 groups
contain code (24 addresses, all exact-graded); 55 are all-zero bodies. The
hand-found pair 0x800262E0/0x800262EC is reproduced as g0002, which is the check
that the census measures what it claims. The largest real groups are 712 bytes
(0x8001084C/0x800189E8) and 436 bytes.
The zero groups are a real finding: 252 extents have all-zero bodies, 245 inside
the zero band 0x80147000..0x80170000. The cause is the inventory's jal grade,
which decodes every word as an instruction -- in a data region a word with
opcode 3 is graded as a call whose target lands in the zero band. The census
flags those groups rather than hiding them, and the worklist must exclude
degenerate bodies.
The census is tracked rather than ignored as the plan said, because it holds
addresses, sizes and grades only (the same class as the tracked inventory and
extents tables) and the worklist must be reproducible from tracked inputs. The
content hash is computed and never written.
Phase 6 graded function starts and left every end to be derived by hand. This
adds tools/sf3_extents, which explores all reachable control flow from each hard
start (jal/entry) and reports an extent plus how far it can be trusted.
Measured decisions, not stylistic ones:
- Soft starts are not walk boundaries. A body's second instruction can satisfy
the prologue grade exactly (0x800152AC is lw v1,8(gp) / addiu sp,sp,-176, so
0x800152B0 looks like a start). Enforcing soft boundaries stopped 155 of 416
walks inside a real body.
- The walk is a full reachability computation, not a first-terminal search: a
function whose paths return at different addresses must report the whole body.
Grades: exact 1940 (1666 packed, gap=0), fallthrough 256, indirect 73,
escape 15, contained 153, standalone 438; 63.8% of the payload covered.
Verification: all 12 registered regions reproduce exactly (make extents-verify,
now part of make check), 29 new synthetic tests (115 total), byte-identical
across two runs, and Ghidra's independent body for FUN_80017ad4 agrees. Two
defects were caught by writing the tests first and are recorded: reach had to be
an exclusive end, and a terminal j's delay slot must not continue linearly.
The table holds addresses, sizes, grades and site addresses only -- no bytes.
P5-T6. Adds docs/MATCHING_COOKBOOK.md (nine byte-proven compiler/assembler
findings, each with basis and limit), docs/MATCHING_CONVENTIONS.md (what counts
as a match, registry format, source naming, duplicate sharing, symbols, the
verification procedure, the firewall) and docs/PHASE5_VERIFICATION.md (outcome,
clean gates, bounded negatives, milestone request).
Tooling: make test and make check targets; region objects are now
symbol-localized so the documented duplicate-sharing mechanism (N registry rows
-> 1 source) links, covered by a synthetic test (53 tests).
Clean gates: 53/53 tests; make clean/all/cmp/SHA-1 green; make gate with one C
region -> c_regions=1, 0 differing bytes, SHA-1 e173426c...; 57 tracked files, 0
under any prohibited root. Phase 5 is not closed until the milestone is
confirmed.
make gate on the empty registry prints result=MATCH, which is the data baseline
being copied through the ordered layout, not a C match. The harness now prints
c_regions=<n> and, when zero, an explicit note that the build contains no C.
Adds two synthetic tests (52 total).
P5-T4: tools/sf3_match provides range/plan/build/gate. It compiles a C candidate
with the identified toolchain (egcs-2.91.66 psx, -O2 -G0 -mno-split-addresses),
compares the exact instruction range byte-for-byte against the original, and
builds the address-ordered executable from the tracked registry config/regions.tsv
(header 0x800 + data gaps + C regions, LMA 0x800, metadata discarded).
Promotes the Phase 4 ordered workflow from an ignored experiment to tracked
tooling. Verified: make gate with an empty registry and with one real C region
both reproduce SHA-1 e173426c...; the Phase 3 baseline is unchanged and green;
the synthetic suite is now 50 tests including deliberate-corruption failures and
safe refusal of an existing destination. No ROM-derived material is tracked.
Christopher Williams