Cycle 3 merged 22 claims (worker A's 8 new plus 9 gp rows, worker C's 6 pure-C
rows) at 137 regions / 128 distinct bodies, and closed every near-miss that
Phases 5-7 recorded in the reachable set: 0x800F3160, 0x800F8AEC, 0x800F8B58/6C,
0x800F8FE4, 0x80102B10 (maspsx=off), 0x800F7FB4 x3 (a cc1 loop-shape artifact),
0x8009E8D0 x2 (ordinary matching).
Worker C's SDK-shape investigation refuted its own premise: the COP2 shapes are
reachable, the blocker was never the missing SDK headers, and the stack accessor
needs no asm at all. The 8 COP2/syscall regions are parked pending the
developer's inline-asm convention decision.
Recorded as results, not footnotes: three rules that were tested and bounded. The
commutative-operand rule's direction is right but its trigger fails for a literal
base in a counted loop; the do/while loop rewrite does not transfer to
0x800FBF5C; and the reorg class is a cc1 reorder thread-fill decision that no
assembler can perform for a conditional branch, so it is a bounded negative
rather than a harness gap.
Control record updated with both verified cycles, the two per-region overrides
that came out of worker findings (maspsx=off and gp=-NAME, with the byte evidence
for each of the six anomaly-class regions), and the coordinator findings: the
stale worklist caught before dispatch, the candidate-first merge that contained a
bad cycle, Ghidra's COP2 pseudo-op collapsing, and the two $gp thunk halves.
Also excludes 0x80108034 and 0x8010804C from the worklist: they are the two halves
of a $gp-switch thunk and are not matchable as C regions.
Milestone met at 109 distinct bodies (target 70).
P8-T1 of the coordinated multi-session phase. Baseline re-run green: make clean,
make all, cmp, SHA-1 e173426c..., 168 synthetic tests, extents agreement 39/39,
and the ordered gate at c_regions=39 with 0 differing bytes. No tracked path
under a prohibited root.
Both peers were probed rather than assumed: 01a0d143 and 01a0d145 are live in
this repository at HEAD 42c9b1d with shell and a full tool set, but neither had
read the project docs, so both charters are self-contained. Hard rules were
restated with the probe: the tracked registries, build/ and all writing git
commands are coordinator-only, and only new src/*.c files inside a worker's own
partition may be created.
Before dispatching, the partitions were checked against config/regions.tsv and
the tracked worklist was found STALE -- it was generated in P7-T4 with only 12
regions registered, so it still listed 23 candidates Phase 7 later matched.
Regenerating it gives 1893 listed with 39 excluded as already registered, which
is exactly the 39 regions. Partitions are now 947/946 rows, disjoint
(intersection 0), union equal to the worklist. Worker B's charter was corrected
immediately because it quoted the old bounds.
The charters carry a standing safety net: a worker that finds a row already in
config/regions.tsv must skip it and report, because that means the file went
stale again.
Phase 7 made matching possible at scale; Phase 8 makes it parallel. Three pi
sessions in one worktree, coordinated over pi-intercom: one coordinator that
dispatches charters, merges claims and independently verifies them, and two
workers that match concurrently.
The plan's centre of gravity is concurrency safety, because three sessions share
one worktree: src/*.c is partitioned disjointly so no two sessions can write the
same file, the tracked registries and build/ and git are coordinator-only, and
worker deliverables stage in ignored paths. The safety property is that a
worker's claim is never trusted -- the coordinator's own clean make gate over the
whole 1,886,208-byte executable is the only authority, so a fabricated match
breaks the SHA-1, and make extents-verify stops a hand-chosen end address.
Target: at least 70 distinct bodies (from 34), staged across three cycles so a
stall shows up early, plus one bounded investigation into the SDK-header shapes
Phase 7 deferred.
Draft, awaiting approval. Decisions flagged for the developer: this session
coordinates, workers match while the coordinator verifies, the two already-live
peers (01a0d143, 01a0d145) are probed rather than assumed, and the target size.
Developer-confirmed milestone: 34 distinct byte-identical function bodies across
39 registered regions, up from 11 / 12 at Phase 6 close. Every region end is now
derived from control flow and enforced by make check, duplicate bodies are
censused, match targets come from a ranked worklist, and cross-references no
longer need hand-written symbol rows.
Adds PhaseEnd_Phase7.md and the digest entry, and archives CURRENT_PHASE.md as
logs/Phase7.md.
Final gates: 168 synthetic tests; make clean, make all, cmp and SHA-1
e173426c...; make check green at c_regions=39 with 0 differing bytes and
regions=39 disagreements=0. 121 tracked files, none under a prohibited root.
Phase 7 closes the milestone: 34 distinct byte-identical bodies across 39
registered regions, up from 11 bodies / 12 regions at Phase 6 close, with every
region end derived from control flow instead of by hand.
New cookbook findings, each byte-proven:
- 13: a store-only function leaves its constant in v0 as scratch. Writing
`return 1` costs an instruction in the delay slot (0x8003636C: 12 vs 16 bytes).
- 14: the gp-relative offsets proved so far, with the global each one names.
- 15: shapes plain C cannot reach without SDK headers (GTE/COP2 code, BIOS
syscall wrappers, stack-pointer accessors).
Conventions updated: a region end comes from config/function_extents.tsv and is
enforced by make check; the duplicate check is the tracked census rather than a
manual search; the worklist is the ranked queue; and address-named symbols
resolve implicitly while anything else fails loudly.
Clean-state gates all exit 0: make clean, make all, cmp, both files SHA-1
e173426c..., and make check (168 tests, regions=39 disagreements=0,
c_regions=39, 0 differing bytes). 121 tracked files, none under a prohibited
root. Milestone confirmation requested before any PhaseEnd.
Every cross-reference used to need a hand-written config/symbols.tsv row, which
does not scale to a batch. A symbol whose name is an address now resolves to that
address with no row: func_XXXXXXXX, D_XXXXXXXX, g_XXXXXXXX, lbl_XXXXXXXX. This is
the convention the registry already used, applied without the row.
The names come from the object's own undefined-symbol list (nm -u), not from a
guess about the source, so a name the source defines but never references is
never mistaken for one needing resolution. A registry row still wins, which is
how a gp marker or a real name is attached. Anything neither registered nor
address-shaped now fails before the link with a message naming the symbol and the
fix, instead of a bare ld diagnostic.
First match from the worklist: func_800F8F9C (36 bytes, worklist rank 3), a
duplicate-group representative with a frame and a call. It matched on the first
attempt and is registered twice (0x800F8F9C, 0x80109314) against one source, so
two functions were matched for one body. Its callee needed no registry row.
Ghidra's independent body agrees with the derived extent.
make gate: c_regions=14, 0 differing bytes, SHA-1 e173426c. 168 synthetic tests.
Phase 6 chose match targets by eye from the boundary inventory. tools/sf3_triage
now ranks every eligible candidate by (tier, size, address) from tracked inputs
alone and records why everything else was excluded.
Eligibility: an exact or fallthrough extent, a non-degenerate body, not already
registered, not the header entry, not named by --exclude. indirect, escape,
outside, runaway, contained and standalone are excluded and counted.
Tiers: 0 duplicate-group representative (one match, several addresses), 1 exact
leaf (no cross-references, so no symbol rows), 2 exact non-leaf, 3 fallthrough.
Result: 1916 listed (tier 0: 9, tier 1: 509, tier 2: 1394, tier 3: 4), with 252
degenerate bodies, 88 low-confidence grades, 12 registered, 1 header entry and
2 named near-misses excluded. The nine tier-0 entries are the real duplicate
groups: matching those nine bodies registers 22 function addresses.
The two deferred near-misses are excluded by name in the Makefile so the
exclusion stays visible rather than buried in the tool. 160 synthetic tests pass.
Matching conventions require a duplicate check before registering, because a
shared body is matched once and registered once per address. Phase 6 did that
check by hand and found one 12-byte pair. tools/sf3_dupes now hashes every
derived extent body and groups exact duplicates.
Results: 2284 extents, 65 multi-address groups, 2104 singletons. Only 10 groups
contain code (24 addresses, all exact-graded); 55 are all-zero bodies. The
hand-found pair 0x800262E0/0x800262EC is reproduced as g0002, which is the check
that the census measures what it claims. The largest real groups are 712 bytes
(0x8001084C/0x800189E8) and 436 bytes.
The zero groups are a real finding: 252 extents have all-zero bodies, 245 inside
the zero band 0x80147000..0x80170000. The cause is the inventory's jal grade,
which decodes every word as an instruction -- in a data region a word with
opcode 3 is graded as a call whose target lands in the zero band. The census
flags those groups rather than hiding them, and the worklist must exclude
degenerate bodies.
The census is tracked rather than ignored as the plan said, because it holds
addresses, sizes and grades only (the same class as the tracked inventory and
extents tables) and the worklist must be reproducible from tracked inputs. The
content hash is computed and never written.
Phase 6 graded function starts and left every end to be derived by hand. This
adds tools/sf3_extents, which explores all reachable control flow from each hard
start (jal/entry) and reports an extent plus how far it can be trusted.
Measured decisions, not stylistic ones:
- Soft starts are not walk boundaries. A body's second instruction can satisfy
the prologue grade exactly (0x800152AC is lw v1,8(gp) / addiu sp,sp,-176, so
0x800152B0 looks like a start). Enforcing soft boundaries stopped 155 of 416
walks inside a real body.
- The walk is a full reachability computation, not a first-terminal search: a
function whose paths return at different addresses must report the whole body.
Grades: exact 1940 (1666 packed, gap=0), fallthrough 256, indirect 73,
escape 15, contained 153, standalone 438; 63.8% of the payload covered.
Verification: all 12 registered regions reproduce exactly (make extents-verify,
now part of make check), 29 new synthetic tests (115 total), byte-identical
across two runs, and Ghidra's independent body for FUN_80017ad4 agrees. Two
defects were caught by writing the tests first and are recorded: reach had to be
an exclusive end, and a terminal j's delay slot must not continue linearly.
The table holds addresses, sizes, grades and site addresses only -- no bytes.
Approved plan: phase-ends/Phase7_PLAN.md. Goal is scale -- evidence-graded
function extents and a duplicate-body census first, then a batch that takes the
project past thirty distinct byte-identical functions.
P7-T1 re-ran every entry gate green: 86 synthetic tests, clean make all with
cmp exit 0 and both files SHA-1 e173426c..., make gate at c_regions=12 with 0
differing bytes. No tracked path under a prohibited root.
Triage splits the Phase 6 open items into four class blockers (missing function
ends, unknown duplicate bodies, no candidate ranking, per-function symbol-row
cost) and the deferred or unresolved single items (0x8005DEF8, 0x800F3160, the
numeric -G, the CRT entry, library versus game code).
Also ignore .pi/, the agent harness's local task-log state, which was untracked
but not ignored.
Closes Phase 5: writes phase-ends/PhaseEnd_Phase5.md, adds the Phase 5 entry to
the digest, and archives CURRENT_PHASE.md as phase-ends/logs/Phase5.md.
Phase 5 outcome: the original compiler is identified with byte evidence as
egcs-2.91.66 (egcs-1.1.2) target mips-sony-psx -- PsyQ 4.5 CC1PSX -- with the
open gcc-2.91.66-psx build verified instruction-identical to the real SDK binary;
the entry is determined to be CRT startup rather than C; and func_80017AD4 is
matched instruction-identical with a green clean full-binary gate. Gates: 53
tests, make clean/all/cmp/SHA-1 green, 62 tracked files with none under a
prohibited root.
Also adds phase-ends/Phase6_PLAN.md as a DRAFT requiring developer approval, and
updates the README status. No Phase 6 task is started.
P5-T6. Adds docs/MATCHING_COOKBOOK.md (nine byte-proven compiler/assembler
findings, each with basis and limit), docs/MATCHING_CONVENTIONS.md (what counts
as a match, registry format, source naming, duplicate sharing, symbols, the
verification procedure, the firewall) and docs/PHASE5_VERIFICATION.md (outcome,
clean gates, bounded negatives, milestone request).
Tooling: make test and make check targets; region objects are now
symbol-localized so the documented duplicate-sharing mechanism (N registry rows
-> 1 source) links, covered by a synthetic test (53 tests).
Clean gates: 53/53 tests; make clean/all/cmp/SHA-1 green; make gate with one C
region -> c_regions=1, 0 differing bytes, SHA-1 e173426c...; 57 tracked files, 0
under any prohibited root. Phase 5 is not closed until the milestone is
confirmed.
P5-T5. Part A: the entry [0x800FB368,0x800FB410) is not compiler output. The
return address is round-tripped through an absolute global around the first
call, the range ends in break, the stack pointer is built from linker globals,
and the clear loop falls through with no jr ra. Two bounded compile experiments
failed and were stopped; the entry stays fallback and no C is claimed.
Part B: first byte-identical C match -- func_80017AD4 at 0x80017AD4..0x80017AE8
(20 bytes), src/func_80017AD4.c, registered in config/regions.tsv. make gate
reports c_regions=1, 0 differing bytes, SHA-1 e173426c157384ebf1b6caf8c6fea18a85a14af9.
Body is unique with one caller at 0x80014C14; a shared-tail near-miss was
checked and rejected as a duplicate. The Phase 3 baseline is unaffected.
Records six codegen findings, notably that GNU as expands the la macro with ori
while the original assembler (ASPSX 2.81) uses addiu, so la-using functions will
need maspsx. No ROM-derived material is tracked.
P5-T4: tools/sf3_match provides range/plan/build/gate. It compiles a C candidate
with the identified toolchain (egcs-2.91.66 psx, -O2 -G0 -mno-split-addresses),
compares the exact instruction range byte-for-byte against the original, and
builds the address-ordered executable from the tracked registry config/regions.tsv
(header 0x800 + data gaps + C regions, LMA 0x800, metadata discarded).
Promotes the Phase 4 ordered workflow from an ignored experiment to tracked
tooling. Verified: make gate with an empty registry and with one real C region
both reproduce SHA-1 e173426c...; the Phase 3 baseline is unchanged and green;
the synthetic suite is now 50 tests including deliberate-corruption failures and
safe refusal of an existing destination. No ROM-derived material is tracked.
P5-T3 revised: obtained the real PsyQ 4.4/4.5/4.6 SDK compilers (proprietary,
kept ignored) and ran CC1PSX 4.5 under the ignored wibo Win32 loader. The SDK's
own README and the binary agree: PsyQ 4.5 CC1PSX = egcs-2.91.66 (egcs-1.1.2),
target mips-sony-psx, assembler ASPSX 2.81.
The open decompals/old-gcc gcc-2.91.66-psx produces instruction-identical
output to the real CC1PSX across ~990 instructions in twelve probe files, so no
proprietary compiler is needed for the matching build. Five reconstructed
original functions are byte-identical with -O2 -G0 -mno-split-addresses.
Corrects the earlier gcc-2.7.2-psx selection, which was an artifact of the
invalid -mcpu=3000 spelling. Records the unresolved 0x8005DEF8 reconstruction.
P5-T3: fingerprinted all eight decompals/old-gcc 0.17 cc1 candidates with
self-authored probes. gcc-2.7.2-psx is the only candidate matching the
ASPSX-style $at symbolic store, the same-register symbolic load, the
unsigned-char default, and the mfhi <scratch> magic-division allocation.
Five reconstructed original functions are byte-identical; 2.6.3-psx, vanilla
2.7.2, 2.7.2-cdk and 2.8.0/2.8.1/2.91.66/2.95.2 are eliminated. One bounded
constant-multiply synthesis divergence is recorded. No ROM-derived material is
tracked.
P5-T1: revalidated the clean payload-data baseline (28 tests, make clean/all,
cmp, SHA-1 e173426c...), reviewed the firewall and Git state, and inventoried the
USA toolchain-relevant codegen signatures and SDK version-marker provenance.
No ROM-derived bytes, strings, or listings are tracked.
Christopher Williams