1be3026446f24de6bada899252e4cb7f00a448bf
57 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Christopher Williams
|
00f951d502 |
phase12: SESSION STOP at 684 bodies / 693 regions (+82) -- clean tree, handover in the ledger
Gate exit 0 (693 regions, MATCH), 344 tests OK, make check exit 0, tracked patch regenerated so
pristine + patch reproduces the working tree.
+4 this commit (worker D): 0x80100334, 0x800FBDC0, 0x800F88F0 (all maspsx=epilogue), 0x80042DD4.
Phase: 602 -> 684 = +82. Milestone 750 was not reached; +66 remains.
TWO TOOL INCIDENTS OF MINE, both reported by workers before I saw them, both repaired:
1. I edited tools/sf3_match in place while four workers verified against it. The new add_argument
block landed between `--fill-epilogue` and its help=, raising IndentationError at line 1129 --
EVERY worker's verification failed for the duration. A verified with a probe copy and re-verified
with the official tool once it parsed; D reported it independently.
2. Reverting an untested transform, my scripted edit DELETED `_LOAD_RA`, the load-delay regex
fill_epilogue needs, breaking maspsx=epilogue for all 21 regions carrying the token. The
whole-binary gate caught it immediately (exit 2, no result line) and it is repaired at the point
of damage.
Guard adopted for every future tool edit: ast.parse immediately after saving, and one atomic write
rather than a scripted multi-step edit. A worker cannot tell a transient tool break from a broken row,
and the cost lands in their context.
ONE DEFERRED ITEM COSTS A BODY: `gp=-` is SINGLE-VALUED per region. D's 0x80102F58 (60 B) needs
D_801221A8/AA/AC registered with gp markers (proven: 60/0 with, 72 without), but A's merged 0x800FFF60
needs those same three ABSENT from the registry because it uses them as SYMBOLS with per-site macro
expansion. Measured: adding the rows breaks A's 140 -> 124; adding them AND giving A three gp=- overrides
restores A to 140/0 -- but the parser refuses with `duplicate override key 'gp'`. The resolution is
proven; only the option key's multiplicity blocks it. Cheapest remaining body in the phase.
The la SYM+N harness transform the developer directed was implemented and REVERTED, because it did not
close its target and an untested transform in the shipped tool is worse than a documented design. Its
mechanism, its guard, where it belongs in the pipeline, and the open question about WHICH STAGE's text
it must rewrite are all in the ledger so it can be resumed without re-deriving.
Also recorded: the gp-rewrite bug worker A found and I fixed (a stale %hi from an unrecorded register
redefinition); the compiler-boundary class (3 rows no vendored cc1 can build); the per-region codegen
flag policy; D's measured tier finding (4/4 with a call or a frame, 0/4 on leaves); and the reading
rules discovered this session, each with at least two instances.
|
||
|
Christopher Williams
|
354e90d5e5 |
phase12: merge 24-25 (670 bodies) + worker A's sf3_match gp-rewrite bug, found and fixed
**670 bodies / 680 regions. Phase: 602 -> 670 = +68.**
**+3 bodies, all verified by me from fresh --work dirs at their REAL extents:**
0x80107CCC (76 B, cc1bin=gcc-2.8.1-psx) -- A's row, unlocked by the tool fix below
0x8002D060 (72 B, D) 0x8006FAEC (88 B, D)
And +2 earlier this round: 0x8004D7C8 (220 B, B), 0x8003A9C8 (240 B, B).
**WORKER A FOUND A REAL BUG IN tools/sf3_match AND I FIXED IT.** `rewrite_gp_accesses`'s `_GP_LO`
branch rewrote `lw $2,%lo(SYM)($R)` to `lw $2,%gp_rel(SYM)($gp)` and then `continue`d WITHOUT recording
that the rewrite redefines its destination register. `carrying` kept the stale `%hi`, the register's
next use was scored as an unrewritten use, and the `%hi` was deliberately kept -- so the region came
out exactly 4 bytes long, one instruction.
I reproduced the evidence independently before touching the code, in 0x80107CCC's pre-fix output:
lui $2,%hi(D_801221CC) # high
lw $2,%gp_rel(D_801221CC)($gp)
The same failure the docstring says was fixed for the adjacent-lines case, still live for the
load-whose-result-is-used-later case. One-line fix, A's reasoning and name in the comment, plus why it
stayed invisible: nothing breaks until a gp-marked LOAD feeds something.
Measured: 80/LENGTH-MISMATCH -> 76/0/MATCH. **The tool change was GATED, not trusted**: the pass
touches every gp region, so the full gate was re-run -- c_regions=678, differing_bytes=0, MATCH. That
also answers A, which asked whether the four merged cc1bin rows needed re-verification; the gate is
what proves it and it was green.
**AND WORKER D FOUND THE SAME PASS FROM THE OPPOSITE DIRECTION** on 0x800FFBBC: cc1 2.8.1 puts a safe
`lui $2,%hi(...)` in a branch delay slot, the rewrite folds the hi/lo pair to gp-relative, the lui
DISAPPEARS and the store slides into the delay slot (48 -> 44). So A's case is the rewrite failing to
fire and D's is the rewrite firing and changing a schedule -- two faces of one pass, on two rows, found
by two workers who could not see each other's row. D's framing is the open item: a maspsx mode that
re-derives the delay slot after the gp-relative rewrite would likely close a class.
**cc1bin's measured boundary, and a correction to my own guidance.** B swept the compilers: 2.5.7 /
2.6.0 / 2.6.3 / 2.7.2 / 2.7.2-cdk / 2.7.2-psx give a MERGED epilogue; 2.8.0-psx / 2.8.1-psx give
SEPARATE jr $31 per return. So return merging is a 2.7.2-family behaviour. BUT 2.8.x also changes the
GLOBAL ACCESS FORM (explicit lui %hi / lw %lo pair instead of the symbolic load maspsx turns into
lw v0,2080(gp)). **The lever moves two axes at once and is not free** -- which corrects what I told A,
B and D when routing the multi-exit rows.
**0x80107D7C sits BETWEEN the compilers and is a genuine open question.** B: default 112, 2.8.1 104,
target 108. I swept all TEN vendored builds because B's table listed eight: only 2.7.2-cdk produces 108
bytes, and it is 30 bytes off in content. 2.91.66 and 2.95.2 give 100. So the length is reachable and
the content is not, on any vendored compiler.
**Two corrections of record, both mine:** I guessed extents instead of reading
config/function_extents.tsv (testing A's 76-byte row against an 80-byte extent and reporting
LENGTH-MISMATCH from a correct candidate); and I told three workers cc1bin was a clean lever for the
multi-exit rows when B's sweep shows it moves two axes.
**Ruling on A's per-region codegen flag request (0x800AB504): (b) class-bound, with the diagnosis
kept** -- because `-fno-strength-reduce` gives 152 against a 148 target. A flag that closes a row is a
lever; one that moves a row 16 bytes closer to a different number is a diagnosis. But the PRINCIPLE is
recorded for the next such row: a codegen flag is admissible on the same terms as cc1bin, i.e. a named
mechanism in the bytes, and A has one (address normalisation creating a second loop-carried pointer).
**D: the indexed and pointer loop forms COEXIST and neither carries across rows.** 0x8002D060 matches
only as an INDEXED loop; 0x80107B40 wants the pointer form. A direct refutation of pattern-carrying
from the worker with the most reason to trust a pattern. Also: the both-sides-volatile rule TRANSFERS
(0x8006FAEC), with the narrowing measured (only interleaved accesses need it); and READ the displacement
and compute the symbol rather than inferring it from an adjacent symbol's name (gp+0xBA4 -> 0x801224DC).
**Charter note: `--cc1-flag=X` REPLACES the default `-quiet -O2 -G0` set**, so a diagnosis run must
pass all four explicitly. A's first -fthread-jumps probe read 200 B purely because -O2 was dropped.
|
||
|
Christopher Williams
|
bf7b6ceac2 |
phase12: merge 22 (665 bodies) + the pool band filter, and I had the first call backwards
MERGE 22: +2 bodies, both from worker D's un-attempted band. 0x801092C0 (52 B) and 0x80085B44 (60 B), both re-verified by me from fresh --work dirs first. Gate: c_regions=674, differing_bytes=0, MATCH. POOL FILTER, SECOND LEAK, MY ERROR CORRECTED. D found 0x80012D54 -- the charter-blocked 0x80012xxx primitive-init family, with class , so the address-parsing rule I installed after the FIRST leak (0x80012A98) had no text to match on. D read its bytes and confirmed the family. I had explicitly DECLINED band-blocking the first time, arguing it 'would silently discard unclassified rows on an inference the charter does not make'. That was backwards: the charter names the family as 'the 0x80012xxx primitive-init family', i.e. BY ADDRESS BAND, and earlier phases already acted on it -- 0x80012A10/AE0/B20/CFC, all class '-', are excluded BY NAME in the Makefile. Two rows leaked by label phrasing and the second had no label at all. The band is the predicate. Blocked 8 -> 15, pools 70 -> 54. Also applying my own corrected commit idiom: this commit names the two claim sources rather than using 'git add -A src/', which swept three of D's in-flight rows into an earlier commit. New levers recorded: LOAD/STORE INTERLEAVING NEEDS BOTH SIDES VOLATILE (a volatile load can still hoist above a plain store and vice versa; cc1 orders volatile only against volatile) -- and from 0x801092C0, unsigned-parameter signedness read from the instruction AFTER the test, volatile on a table pointer for an alias reason, cookbook 19 generalising to a forward loop, and `p += i + start; p->p0 = 0;` over the subscript form. |
||
|
Christopher Williams
|
50270a95f6 |
phase12: merge 20-21 (663 bodies) -- worker B's maspsx predicate patch closes a released row
**+4 bodies: 0x80102A80 (132, B, maspsx=epilogue), 0x800FFF60 (140, A, maspsx=epilogue),
0x80102A00 (128, B's source + the NEW maspsx=regread predicate), and the positioning is now
663 bodies / 672 regions. Phase: 602 -> 663 = +61.** Gate byte-exact at every merge, SHA-1 unchanged,
make check exit 0, 344 tests green.
**THE HARNESS PATCH IS THE HEADLINE, because it turns a class into a predicate.** Worker B reported
`0x80102A00` as an UNCLOSABLE row with the mechanism pinned to the harness's own `--work`
intermediates, and refused to touch `tools/` -- correct, since it could not know the default path's
invariants. Its diagnosis: finding 27's gap was fixed by `line_jumps_via_reg` (a load feeding a
REGISTER JUMP), and **this row is the complementary case in the opposite direction -- the jump
ignores the register, but the instruction in its DELAY SLOT reads it.**
lw $2, D_8011FD2C <- loads $2
jal func_80103FCC
sb $0, 0($2) <- the SLOT FILLER reads $2
cc1 emits that with no `#nop` and no marker, correctly by the documented load-to-use rule (gap of 1).
**ASPSX was conservative ACROSS a jump** -- the slot filler is part of the jump -- and maspsx's own
trace says so: `#nop # DEBUG: 'jal func_80103FCC' does not load from $2`. It declined the row for
exactly the reason the original accepted it.
Implementation: `_jump_slot_filler_reads_reg`, on the **existing opt-in** `--nop-on-reg-read` /
`maspsx=regread` token, so the default path is byte-identical and the green gate cannot move. Confirmed
safe rather than assumed: **no registered region named that token before this commit.** Restricted to
`j`/`jal` -- `jr`/`jalr` slot fillers are the same shape but UNMEASURED, and an opt-in predicate whose
whole purpose is default-identity has no place for an unmeasured widening.
Measured, on the source unchanged: **default 124 LENGTH-MISMATCH -> `maspsx=regread` 128 / 0 differing
/ MATCH.** So B's "unclosable" row is a body, and B's candidate source is now `src/func_80102A00.c`
with a header recording that the token is REQUIRED and must not be "cleaned up".
**THE PATCH WAS REGENERATED IN THE SAME COMMIT, which is the whole point of the discipline.**
`tools/patches/maspsx-phase10-r1r2.patch` went 229 -> 289 lines, and all 7 `test_maspsx_patch` tests
pass -- so pristine + patch reconstructs the working tree byte-for-byte and a fresh clone can still
rebuild the gate. Phase 11's `ea51ac9` did exactly this correction WITHOUT regenerating the patch and
left a fresh clone broken; that is why the test exists and it is why this was done in one commit.
**Cookbook, two new entries:**
* **61c -- THE REGISTRY DECIDES THE SPELLING.** A closed two rows today in OPPOSITE directions and the
deciding fact is the registry entry, not the code's shape: `0x800FB758`'s address IS `gp`-marked and
the original writes it absolutely, so the source must use the LITERAL (a symbol lets the harness
rewrite to `%gp_rel`); `0x800FFF60`'s three addresses are NOT registered, so the source must use
SYMBOLS (as literals the address becomes a value and cc1 CSEs it -- the index's `cc1=-G4` attempt was
chasing the wrong thing). Rule: check the registry first. This is the source-side companion to
finding 46.
* **61d -- the `nop_on_reg_read` second gap**, above, with the trace line as the receipt.
Also recorded: worker A's `0x80011484` is no longer the index's "UNRESOLVED" row -- A derived it from
the bytes as an octagonal distance approximation (`s=|dx|+|dz|`, `d=||dx|-|dz||`, then `d<h -> s-q`,
`h+q<d -> s`, else `s-(s>>3)`) with both absolutes as SWAPPED SUBTRACTIONS, and the build is 120 with
a scheduler residual. And A's `0x8002FB54` gave the phase a new lever type: **a DUPLICATED MASK is
evidence of the variable's WIDTH** (`unsigned char r` vs `int r &= 0xFF`), because a byte-typed object
masks at every read and no placement can fake that.
|
||
|
Christopher Williams
|
5af450c491 |
phase12: the three-worker triage key was implemented, TESTED, and FAILED -- disarmed, kept as a label
Three workers converged independently on the same split within one hour, which is normally the
signal to act on. This time acting on it would have been wrong, and the reason is measurable.
D, from its scoreboard: what matters is whether the row has been attempted WITH THE CURRENT LEVER
SET, not which pool it is in.
C, from 11 classifications: prio-1 does not distinguish "a name that implies a SOURCE SHAPE" from
"a name that implies a COMPILER BEHAVIOUR".
A, from 14 rows touched: what predicts closability is the WORDING of the row's own note, not its
prio -- and a grep would separate the two groups in one pass.
**The insight is right and is kept. The proposed discriminator is not predictive, and I measured it
rather than arguing about it.** I classified every row A and C named, by outcome, against the pool's
own record:
MATCHED (8): 0x8010036C 0x800690E4 0x80018284 0x80094370 0x80065494 0x800C3514 0x80022E44 0x8002DF1C
-> source-shape 2 | unknown 2 | **residual-class 4**
RESISTED (7): 0x8010400C 0x800A6C34 0x80107CCC 0x80050674 0x80023D40 0x80024630 0x80101E50
-> source-shape 1 | **unknown 3** | residual-class 3
Precision of "residual-class => will resist": **3/7, worse than a coin flip**, and the rule would have
**deprioritised four of the eight actual successes.** The decisive examples are the proposers' own:
C's `0x800C3514` is classed `alloc+layout (priority selector; ...)` and MATCHED -- the lever was
ARITY; B's `0x8002DF1C` is classed `candidate_bytes=104 = CORRECT length, 58 differing bytes` and
MATCHED -- the lever was a `goto` into the body. Both would have gone to the back of the queue.
So `kind` is DISARMED as an ordering rule and retained only as a LABEL written into every pool row,
with the failure and its numbers in the header so no worker trusts it. A test asserts it is not in
the sort key and cannot drop a row, so it cannot be quietly re-enabled.
**This is defect 5's shape exactly** -- a text heuristic over free prose, on this same field, which
inverted 62 rows. The difference is only that this one was tested before being believed. The real
discriminator is whether a residual has been LOCALISED to an allocation or scheduling decision, and
that requires reading the bytes rather than the note. **Free prose is not evidence about closability.**
Also in this commit: `negatives-d-unattempted.tsv`, the 74-row UN-PUSHED band that worker D asked for
and that D's own numbers justify (2-for-7 on rows a previous session pushed to 1-6 bytes, against
one-spelling closes on rows nobody had pushed). Rows with no mechanism named, not registered, not
excluded, not charter-blocked, not held in the ledger. This is D's criterion applied as a band rather
than as a preference.
Pools after the re-cut: 70 rows (a 16 / b 19 / c 21 / d 14).
|
||
|
Christopher Williams
|
ca5eb6e613 |
phase12: merge 12-14 (652 bodies) + the phase's first inline-asm row + charter s7's not-counted row
MERGES. +13 bodies across rounds 12-14, each re-verified by me from a fresh --work dir first:
0x80042CE0 (80, D) 0x80018284 (80, A) 0x80094370 (84, A) 0x8002DF1C (104, B)
0x80065494 (80, C) 0x800C3514 (88, C) 0x801097A0 (128, D)
Gate every time: differing_bytes=0 result=MATCH, SHA-1 unchanged. make check exit 0.
**652 bodies / 662 regions. Phase: 602 -> 652 = +50.**
**A DEPENDENT ROW WAS RE-VERIFIED WHEN ITS DEPENDENCY LANDED, and this is a new rule.** Worker C's
`0x80091490` was merged as a DEPENDENT claim whose callee `0x80018284` was unreconstructed, with a
3-argument prototype INFERRED from the call site. Worker A then reconstructed `0x80018284`. Merging it
changes `config/symbols.tsv`, which is the environment that row was verified IN -- so I re-ran
`0x80091490` after the merge: still `differing_bytes=0 result=MATCH`. The arity check I promised:
A's `int func_80018284(int *a0, int a1, struct V8 *a2)` vs C's inferred `(int, int, int *)` --
**same arity (3)**, differing only in pointee types on args 1 and 3, which are **byte-invisible** at a
pass-through and a literal-0 site. So the inference held. Rule recorded: **a dependent claim must be
re-verified whenever its dependency is merged**, because the symbol set is part of its input.
**`0x801097A0` -- THE PHASE'S FIRST GENUINE INLINE-ASM ROW IS IN.** Worker D needed seven spellings and
each of the four asm properties is a *measurement with its counter-shape*, which is what makes this an
authorised use rather than a shortcut:
1. `$31` cannot be WRITTEN from C. `register int ra __asm__("$31"); ra = ...;` makes cc1 keep a frame
(132 B) and DELETES the assignment -- reading `$31` is a documented binding, writing it is not
expressible. That is the "provably cannot express" clause demonstrated rather than asserted.
2. The call must NOT be a C call: as a C call cc1 preserves `$31` itself and adds a prologue and an
epilogue (140 B), while **the original has NO FRAME AT ALL**. Written as an asm `jal`, cc1 never
sees a call. This single fact is what makes the row expressible.
3. Zero operands must be `$0` LITERALLY: `gte_ldOFX(0)` passes `"r"(0)`, cc1 materialises a register,
and the original's `ctc2 zero,$24` becomes `move` + `ctc2`.
4. The constant register must be pinned to `$8` (cookbook 160); otherwise the seven constants land in
`a0` (10 differing bytes = five `li` plus five `ctc2` register fields).
Also measured: maspsx supplies the jump-slot `nop` after the asm `jal` (writing one explicitly gives
132 B -- two nops), and the CP0 constant is `0x40000000`, not `0x4000`: bit 30 is the COP2-enable bit,
and `0x4000` compiles to `ori` and does not match.
**I wrote `include/gtemac.h` myself, because worker D refused to and was RIGHT.** D's charter section 4
forbids it from writing under `include/`, and my authorisation conflicted with the charter. D held its
write scope and handed me the macro text with the note that "I followed the charter because it is a
file and it is unambiguous". **That is the correct resolution of a coordinator error and it is the
behaviour I want**: the charter is a file, it was unambiguous, and D did not quietly exceed it. The two
macros (`gte_ldZSF3`/`gte_ldZSF4`, $29/$30) are written in exactly the form of the existing entries,
with the evidence in the comment -- the NUMBERS are the evidence, not the register names. The open
question D raises is recorded for the next charter: whether `include/` should be a worker write scope
or whether worker-authored macro text passed to the coordinator is the right division of labour.
**CHARTER SECTION 7'S NOT-COUNTED ROW IS NOW EXCLUDED BY NAME, and this one is my miss.** Worker C found
`0x80107C5C` (112 B) in its pool and skipped it because section 7 says it "matches but has no
documented source -- it is NOT counted; do not claim it". **I had READ that sentence earlier in this
session and did not act on it.** A worker who did not know the sentence would have produced a body the
phase does not count. It is now in `NAMED_EXCLUSIONS` -- the one place an exclusion lives -- with a
comment explaining that its reason differs from the rest of the list.
**`sf3_free` CRASHED, worker B caught it, and the fix is in.** My `--claim` addition used `free` and
`failed` without initialising them, so **every FREE address raised NameError and exited 1**. Worker B
reported the reproduction and, critically, the asymmetry that makes it matter: **the crash happened
AFTER the `FREE` line was printed, so a worker reading stdout saw the right answer while `$?` said 1**
-- a false BLOCKED for anyone checking the exit code or running under `set -e`. The harmless direction
of the two, but it costs a row per occurrence and it would have looked like a ledger problem rather
than a tool problem. Fixed (`free: list[int] = []`, `failed = False`), B's exact reproduction now
exits 0, and the 25-test suite passes. **The test suite caught this on the very change that introduced
it** -- recorded because that is the tests earning their keep, and because the tool whose whole purpose
is to be trusted unread is the one that must never be wrong about its exit code.
**`--claim` added so check-and-claim is ONE step.** The race is real and worker B hit it: B appended its
`wip` row in the same shell line as the check, *before* reading the output, so the ledger's last row for
`0x80042CE0` said B while D held it. B caught it, did not touch the row, and restored D's hold by
appending the row back (last-row-wins). **In the safe direction** -- B's own row masked the true holder
as TAKEN -- but the inverse ordering, appending on a stale read, is how two workers end up on one
address. `--claim WORKER` now appends only if every address is free, and a refusal writes nothing.
**Worker C found the FOURTH row where the class name pointed at the right place and the mechanism was
not what the note said.** `0x800C3514`'s note described "all stack loads hoisted, beqz+nop+li groups";
the lever was **ARITY** -- the function takes SIX `unsigned char` parameters, the fifth and sixth
arriving on the stack (`lbu 16(sp)`/`lbu 20(sp)`), which the note read as hoisted loads. Measured: the
four-argument spelling gives 56 bytes, twelve short, with neither stack load present. The tally is now
`alloc-tiebreak` -> a pass-through argument (finding 164) | `constant-materialisation-order` -> source
statement order | `return-merge` -> a literal twin and a compiler revision | `alloc+layout` -> arity.
|
||
|
Christopher Williams
|
9053a4dc51 |
phase12: merge 11 (646 bodies / 655 regions) + sf3_free: claimed is HELD, not FREE
MERGE 11: +7 bodies. All seven re-verified by me from fresh --work dirs BEFORE merging:
0x800A6B38 (104 B, D) 0x8010036C (56 B, A, maspsx=epilogue)
0x800690E4 (68 B, A) 0x800FE970 (84 B, C, maspsx=epilogue)
0x80022E44 (52 B, B) 0x80047468 (72 B, B, gp=-D_80121BFC)
0x80042D88 (76 B, B)
Gate: c_regions=655 differing_bytes=0 result=MATCH, SHA-1 unchanged. make check exit 0.
Registry: 646 distinct bodies, 0 missing sources, 4 carry cc1bin. Phase: 602 -> 646 = +44.
**sf3_free: a `claimed` ledger row is HELD, not FREE.** Worker D found this and reported it rather
than changing a tracked tool on its own judgement, which is the right instinct. The old rule was
`wip` -> TAKEN, else FREE, so a row whose last ledger row was `claimed` -- a worker holding a
STAGED, UNMERGED claim -- read as free. That is a false FREE in the dangerous direction: the row is
neither registered nor abandoned, so the next worker re-derives it and the merge can then receive two
claims for one address.
**This was not hypothetical. It is exactly what the credit outage produced**: all four workers
stopped with claims in flight, and I re-dispatched the pools underneath them. I recovered those rows
by auditing the staging files, but that was luck of ordering, not a mechanism. D reported the two
live instances it could see (`0x800460AC` held by B, `0x800FE970` held by C). `0x800460AC` I had
already merged, which masked the first one; `0x800FE970` was live when D wrote.
A registered row never reaches the changed branch -- `region_owner()` returns TAKEN first -- so the
new rule only ever fires for a claim whose merge has NOT landed, which is precisely the window at
issue. Measured after the fix:
0x800FE970 TAKEN by inflight (claim staged, merge pending)
0x80012A98 FREE (last ledger row 'released') <- a genuine release still reads FREE
0x800FECF8 FREE (last ledger row 'released')
0x800ACA10 FREE (last ledger row 'released')
The residual risk is the opposite direction and is now an orchestrator DUTY, recorded here: if a
claim is ever REJECTED rather than merged, the merge path must append a `released` row, or that
address stays held forever. Nothing has been rejected so far this phase.
Also recorded from this batch:
* **B's `0x80047468` is the second worked case of cookbook 46's per-site override**: the registry
marks `D_80121BFC` gp, but this row reads it absolutely (`lui v1,0x8012 / lw v1,7164(v1)`), so
without `gp=-D_80121BFC` the candidate is one instruction SHORT (68 vs 72). C independently hit the
same class on `0x800A6658` with `D_801226E0`. **The registry's gp marker is per-SYMBOL and the
access form is per-SITE**, and two workers in one hour each lost time to that.
* **B and C independently converged on the same conclusion about what these negatives are worth:**
B -- "the recorded class named the SYMPTOM (a merged store, a folded shift, a strength-reduce)
while the fix was an ORDER or a claim-row option"; C -- "in every one the recorded class NAME was
right and the recorded MECHANISM was half wrong". Two workers, different partitions, same finding.
That is the strongest argument yet for dispatching onto named negatives, and simultaneously a
warning: **the class is a signpost, not an instruction.**
|
||
|
Christopher Williams
|
7090663275 |
phase12: sf3_negpool -- kill the five defects that reached the workers, and merge 10 (639 bodies)
MERGE 10: +4 bodies -> 639 bodies / 648 regions. All four re-verified by me from fresh --work dirs
with --symbols first: 0x80099E34 (40 B), 0x800AC9D8 (56 B), 0x800460AC (40 B), 0x80045540 (56 B).
Gate: c_regions=648 differing_bytes=0 result=MATCH, SHA-1 unchanged. make check exit 0.
The tool, and why it exists. The Phase 12 re-dispatch put all four workers on the ~193-row
classified-negatives index. I generated those pools with a one-off script, and that script was
wrong in FIVE ways. Every one of them shipped, and **every one was found by a worker, not by me**:
1. The index documents its own schema on line 2 -- `# Columns: address<TAB>size<TAB>status<TAB>class`
-- and my script **never read `status`**. 5 rows `blocked` + 1 `blocked,deferred` are rows
charter section 7 forbids attempting (trapping arithmetic, the 0x80012xxx primitive-init family,
the maspsx rare-epilogue mutual exclusion). **Six forbidden rows went out at prio 1, the top of
the list.** Worker C found three in its own pool and refused them.
2. `status` is not enough either: `0x80012A48` is status `near-match` with class `primitive-init
scheduler-bound family`, so it reached prio 1 as well. Blocked families are now matched on the
CHARTER's own wording, in both the status and the class column -- 7 rows dropped.
Deliberately NOT blocked: `rare-epilogue-ORDER`. Worker A drew that line precisely (the order is
work; only the maspsx mutual EXCLUSION is blocked), and cookbook 140/147/165 shipped
`maspsx=epilogue` for it.
3. The Makefile's `NAMED_EXCLUSIONS` (12 rows) was invisible to my script: **8 of the 12 leaked back**
as fresh work -- $gp-switch thunk halves, a fragment, false extent starts. Worker A found two and
asked whether they should be filtered. Eight were leaking, not two. The list now lives in ONE
place (`NAMED_EXCLUSIONS`), and both `worklist` and the new `negpool` target are given it.
4. The workers' own `negatives.tsv` staging was invisible: **94 of the 115 rows they had already
classified were served as fresh work, at the TOP of the prio-1 order.** Worker C: "they are the
first four rows in the file's own prio-1 order, so a worker starting at the top spends its first
hours re-deriving my floor." 58 rows dropped as classified-with-no-mechanism (a second worker
re-deriving a row that already yielded nothing is pure duplication), and a worker's OWN
classification is dropped from that worker's own pool (12 rows) while still being dispatched to
the others, who get the recorded mechanism as a lead.
5. `named = class not in ('-', '')` is a test for a NON-EMPTY STRING, not for a mechanism. Worker D's
staging labels 56 of its 92 rows `no-mechanism-yet` and 6 more `no-extent`, so **62 rows were
ranked prio 1, "cheapest, mechanism already named", when their label says the exact opposite.**
That is why worker A's pool led with rows it could not close. `names_a_mechanism()` now rejects
absences, numbers and bare status words, and accepts prose (workers A and C write their class as
a sentence, so a token-only rule would have missed their mechanisms).
This is a SCHEMA error, not a logic error, and it is worth naming as such: five separate bugs all
trace to generating a dispatch file from a table whose header I had not read.
One coordinator aside that belongs in the record: I had already "fixed" defect 3 by hand, adding
`--exclude 0x8001DC20` to the `worklist` target -- and **the counter caught me**:
`excluded_named_exclusion` did not move, because that row was already excluded from the worklist
under a different rule and had reached the worker through the negatives pool instead. Fixing one row
by hand while the generator leaked was treating a symptom, and the tool said so.
Pools: 176 unregistered -> **93 rows** (a 24 / b 23 / c 28 / d 18), every exclusion and blocked row
gone. C's pool now leads with `index-only` rows, i.e. genuinely fresh work.
Tests: 17 new, `tools/tests/test_sf3_negpool.py`. Fail-first is demonstrated rather than asserted --
the two INTEGRATION tests (which read `NAMED_EXCLUSIONS` out of the Makefile rather than hard-coding
addresses, because the whole defect was two lists that must agree living in different files) were run
against the pools the workers were holding at that moment and **failed, naming all 8 leaked rows**.
Two further bugs found in the process: my own `names_a_mechanism` accepted the numeric dashboard
columns (`'1'` is not a mechanism), which made the rule vacuous and dropped 0 rows instead of ~62; and
one of my tests asserted output that its own `--quiet` flag suppressed.
|
||
|
Christopher Williams
|
567b069495 |
phase12: cc1bin= — the alternative-cc1 lever, RESTRICTED BY THE GATE (developer-approved)
Worker C found that the project's cc1 cannot build a whole class of the original's functions,
and the census is triply verified (C, worker D, coordinator):
623/623 REGISTERED regions contain EXACTLY ONE `jr $31`. ZERO contain two or more.
12 of the 193 NEGATIVES rows contain two or more. That is the entire blocked class.
NOT a flag: 14 flag sets on the default cc1 all yield one exit.
And it produces bodies: 0x800FF43C / 0x800FF47C / 0x80108578 are byte-exact with
gcc-2.8.1-psx/cc1 where the default gives 56/60/52 B LENGTH-MISMATCH.
THE RESTRICTION IS ENFORCED HERE, NOT LEFT TO A CONVENTION. Allowing a second compiler
BINARY widens the corpus, and the project's rule against per-function compiler choice exists
because the gate CANNOT catch a wrong choice -- byte-exact is byte-exact. So a region naming
`cc1bin` is REFUSED unless its original body has >= 2 function-exit jumps, i.e. unless it has
a NAMED MECHANISM in the bytes:
region 0x8006B5F0..0x8006B66C names cc1bin=gcc-2.8.1-psx but its original body has only
1 function-exit jump(s) (`jr $31`); 2 are required. -> exit 2
`cc1bin` takes a BARE vendored directory name under tools/old-gcc/ (no slashes, no paths), so
the nameable set is exactly the pinned set already in the repo. Validated by `sf3_merge` at
merge time and enforced by `sf3_match` before anything is compiled.
THE MECHANISM SENTENCE IN THIS FILE WAS WRONG AND IS CORRECTED, NOT DELETED. My first version
said "2.7.2 emits one shared return epilogue while 2.8.x emits one per return". C's 4-return
probe reproduces that and I reproduced C's probe exactly -- but worker D could NOT reproduce
it: on both a 3-return framed probe and a 3-return frameless probe, ALL TEN builds emitted one
shared exit. Both probes are real, so the compiler-side result is SHAPE-DEPENDENT and the
general claim is false. The census and the three bodies justify the lever; the mechanism was a
hypothesis I recorded as a finding. Left in place, marked as corrected, because the next reader
will otherwise re-derive it and believe it.
ALSO IN THIS COMMIT:
* **The 2.8.x gp access pair.** 2.8.x addresses a global as `lui $R,%hi(SYM)` +
`<op> $r,%lo(SYM)($R)` where 2.7.2 emits a bare `sw $2,SYM`. Both lines were invisible to
the existing regexes, so a gp-touching region built with an alt cc1 came out 4 BYTES LONG.
Now rewritten as a pair: the access becomes `%gp_rel(SYM)($gp)` and the `%hi` is dropped.
My first two attempts at this were both wrong and both are recorded in the code:
it required the two lines to be ADJACENT and maspsx emits a BLANK LINE between them (so it
silently did nothing), and it deleted the `%hi` unconditionally, which leaves a later access
to the same global addressing a register nothing wrote -- right length, plausible, wrong.
The deletion is now gated on every use of the register before it is redefined being rewritten,
with redefinition tracked explicitly and `jalr` deliberately treated as non-defining so the
rule can only ever keep a `%hi` alive, never delete a live one.
* **`check-claims` reported a healthy CUMULATIVE file as a failure.** Worker B hit this: an
already-registered row is exactly what `apply --skip-registered` exists to skip, and a
worker's claims file is cumulative, so `result=PROBLEMS`/exit 1 was the NORMAL case for a
good file. A checker whose healthy output is a failure teaches workers to ignore its exit
code, and then it catches nothing. Already-registered is now information
(`already_registered=N`, exit 0); format errors stay exit 2, missing sources and duplicate
starts stay exit 1.
Tests: 306 -> 318, with the restriction pinned both ways (a 1-exit region is refused, a 2-exit
region is allowed), the census checked on a synthetic payload, and the gp pair rewrite's three
shapes covered. Full gate on the tracked registry stays byte-exact at 623 regions, so none of
this changes the existing corpus. `make check` exit 0.
|
||
|
Christopher Williams
|
d2cf1b8f10 |
phase12: sf3_cc's maspsx stage NEVER RAN, then doubled the output — fixed (worker B's finding)
Worker B measured this and was right, and the diagnosis is exact. The call was
maspsx.py --aspsx-version=2.56 "$OUT/$BASE.s" "$OUT/$BASE.ms.s"
but maspsx takes ONE positional (an INPUT) and writes to STDOUT -- it has no output-file
argument, which is precisely why sf3_match runs it as a stdin->stdout filter (run_filter).
So maspsx tried to open the OUTPUT path as its input, failed, and the failure was hidden
TWICE: by `2>/dev/null`, and by a `|| cp` fallback that quietly copied the raw cc1 output
over `.ms.s`.
Measured on src/func_8006B5F0.c before the fix:
run 1, fresh scratch : .ms.s IDENTICAL to .s (71 lines) -- maspsx NEVER APPLIED
run 2, scratch exists: 133 lines = TWO body copies with 6 `addu`, because maspsx now read
the STALE .ms.s as input, succeeded, printed its real output STRAIGHT
TO STDOUT (unredirected), and `cat` printed the stale raw copy after it
After the fix: 62 lines both runs, IDENTICAL to each other, stdout == .ms.s, and .ms.s differs
from .s for the right reason (the stage actually runs).
**So the tool was wrong on the first run and wrong-and-doubled on every later one, from its
Phase 11 promotion (
|
||
|
Christopher Williams
|
1e620cde50 |
phase12: a CHARTER is a file every worker copies — my claims format was wrong (cookbook 190)
The phase's FIRST merge was rejected: "expected three or four fields", while all four workers
were already staging files in the format my charter had given them.
charter said: range<TAB>source<TAB>md5<TAB>differing_bytes<TAB>result<TAB>options
with range written 0xSTART..0xEND
tool reads: start<TAB>end<TAB>source[<TAB>overrides]
Finding 179 says a rule every worker must follow belongs in a TRACKED tool, not in a file each
worker copies, because a copied artefact cannot be fixed for the people who already copied it.
Phase 11 earned that from a worker's free.sh. **A charter IS a file each worker copies**, so
writing a FORMAT into prose recreates the defect one level up — and this time the unfixable
copied artefact was the coordinator's.
AND THE TOOL HAD ALREADY LEARNED IT. validate_overrides exists because a Phase 11 worker put
md5= in the 4th column, and its error message says in terms: "Per-claim metadata such as a
source md5 belongs in report.tsv, not in the registry row." The convention was documented
INSIDE THE TOOL, I did not read it, and I wrote prose contradicting it — including renaming
the evidence file to evidence.tsv when report.tsv is the established name in the tool's own
error string and in every Phase 8/9/11 worker's staging directory.
Standing rule: before writing a staging format, an interface, or an exit-code contract into a
charter, READ THE TOOL THAT ENFORCES IT. "The tool is the contract" is not advice for workers
only. Workers compute addresses rather than eyeballing them; the coordinator must derive
formats rather than inventing them.
FIXED AS A COMMAND, NOT AS CORRECTED PROSE. New `sf3_merge check-claims --claims F [--regions R]`
validates the format, flags a duplicate start, a missing source file and an already-registered
row, so a worker answers "is my staging mergeable?" itself before reporting. 11 tests, one of
which asserts THE CHARTER'S OWN WRONG FORMAT IS REJECTED, so the message stays honest for the
next coordinator — who will also write prose.
Charter §10 rewritten to specify the tool's format by READING THE TOOL, to separate claims.tsv
(the merge input) from report.tsv (the evidence, where the md5 lives), and to name
check-claims as a required pre-report step; §5 gained rule 17.
What worked: fail-fast validation caught it in under a second at the merge, not as a confusing
failure at the gate. Phase 11's override-key guard paid off again. A staging format a tool can
CHECK is worth more than one that is documented well.
Cost: one rejected merge, four correction messages, ~10 minutes. Cheap because the tool refuses
to guess.
make test 301 tests, OK (from 290)
|
||
|
Christopher Williams
|
afce5aa108 |
phase12: sf3_cc --help was broken — three of four workers tripped on it
Worker B, worker C and worker D all probed `tools/sf3_cc` with `--help` during the Phase 12 capability probe, and all three got a raw coreutils `basename` error instead of help: ./tools/sf3_cc: line 29: .run/sf3_cc/Usage: basename NAME [SUFFIX] ... : No such file ANY leading-`-` argument was taken as a source path. `basename --help .c` prints coreutils' own help text, and that entire text then became the output filename, so the failure surfaced as a confusing redirection/cpp error. Every worker lost time on it, on the first command they tried, and worker D spent probe effort diagnosing it as a "genuinely broken path". This is cookbook 179 from the other side. That finding says a rule every worker must follow belongs in a TRACKED tool rather than a file each worker copies. Here the tool WAS tracked -- and it was still unusable the way every single worker reaches for it first. Tracking a tool is necessary but not sufficient; it also has to survive first contact. Fixed: `--help`/`-h` print real help and exit 0, an unknown option is rejected BY NAME, and a missing source file says so rather than failing inside cpp. The compile path is unchanged. Also documented its LIMIT, which was previously unwritten and is easy to misread as a match: the maspsx stage always runs with DEFAULT options and no region option is passed, so this shows a SPELLING's shape and NOT a region's final bytes. A region needing `maspsx=epilogue`, `nopmarker`, `moves`, `regread`, `off` or `gp=`/`cc1=`/`as=` looks different here than under `sf3_match range`. Sweep spellings with this; decide matches with `sf3_match range` -- and decide an `epilogue` token from the CANDIDATE's tail, never the original's (finding 165/180). 9 new tests (tools/tests/test_sf3_cc.py): the argument-handling cases run without a toolchain, including one that pins the exact symptom (no coreutils help text in the output, and no file created named after it); the compile-path test skips cleanly when the ignored toolchain or any src/func_*.c is absent, and one test pins the documented LIMIT so it cannot be dropped. make test 290 tests, OK (from 281) |
||
|
Christopher Williams
|
e53086a59e |
phase12: T0 — regenerate the STALE maspsx patch and guard it with a test + sf3_free reads the CURRENT phase's ledger
Both defects were found by the Phase 12 open checklist, and both were then PROVEN by direct test rather than by inspection. *** 1. THE TRACKED PATCH WAS STALE, AND THE TOOLCHAIN WAS NOT REPRODUCIBLE *** |
||
|
Christopher Williams
|
4824b7af9a |
phase11: restore tools/sf3_diff (overwritten blind) + cookbook 186
|
||
|
Christopher Williams
|
b29963eafb |
phase11: promote sf3_cc + sf3_diff to tools/ (worker F) + cookbook 184-185
Worker F's judgement, which I agree with: these two together are the highest-value tooling built
this phase, because the first removes the COST of a spelling and the second removes the GUESSWORK
about what is wrong.
tools/sf3_cc <file.c> one-file cpp -> cc1 -> maspsx, printing the assembly.
A spelling costs ~0.1s, which turns 'try a few variants'
into 'grid the whole space'.
tools/sf3_diff 0xS 0xE <workdir> opcode-level diff of the candidate object against the
original, difflib-aligned, so you see the SHAPE of the
residual rather than a byte count.
A differing-byte count tells you HOW WRONG a candidate is; it does not tell you WHICH DIMENSION
the error lives in. sf3_diff marks differing instructions with '<<', so a run of identical
mnemonics with differing register names is the allocator class -- a CLASSIFY signal, not a
spelling signal.
184: a load's BASIC BLOCK is not source-movable -- sched2 cannot cross blocks, so a load in a
different block than the candidate's is a SOURCE-ORDER fact.
|
||
|
Christopher Williams
|
3519afe441 |
phase11: tools/sf3_free + cookbook 179-180 — the EIGHTH rule defect, and the token table
179: a rule every worker must follow belongs in a TRACKED tool, not a copied script. The registry free-check lived in a worker's staging dir and the orchestrator told everyone to copy it. It was WRONG -- it reported TAKEN if ANY ledger row for the address was not 'released', so a row that was wip and later released stayed blocked FOREVER. Worker E found it and measured 13 released rows reading as taken, several of them the cheapest rows left, and the same stale pattern existed for workers A and D too. Fixed and promoted to tools/sf3_free (tracked, docstring explains the bug). A copied script cannot be fixed for the people who already copied it. 180: worker E's maspsx=epilogue token table, MEASURED not inferred -- 5 rows REQUIRED, 3 HARMFUL or NO-OP, 2 HARMFUL-but-unmatched. Three of E's fourteen claims sit on that list and TWO would have failed outright if the token had been applied by shape. The cheap read: '4 bytes SHORT with the token on' means the token was unnecessary. |
||
|
Christopher Williams
|
696b7dfbfb |
phase11: cookbook 169 + sf3_family bug fix — 0.96-0.99 is idiom noise, CONFIRMED by raw-word diff
Worker A calibrated sf3_family by checking two 0.97 entries and finding neither shared its sibling's body. I have now confirmed that on eight candidates by raw-word diff, which is the decisive test: 0x8006EBA0 vs 0x80028CE0 differs in 60 of 61 words; 0x800FFFEC vs 0x8007E8B8 in 18 of 19; 0x8005E17C vs 0x800FB54C in 25 of 26. Contrast the genuine sibling 0x800F3DC0 vs 0x800F3E18: 1 of 22 words. So a high cosine with ratio 1.00 is NOT evidence of a shared body -- at 0.96-0.99 the histogram matches common IDIOMS. The useful band is ratio 1.000 AND a near-zero raw-word diff. Tool bug fixed: sf3_family did not exclude already-claimed rows, so its top hit was a row matching ITSELF. The registry is now the authority and claimed rows are skipped. |
||
|
Christopher Williams
|
ea51ac9629 |
phase11: merge 40 + the epilogue transform now handles BOTH shapes — 571 bodies / 580 regions
Worker A's two epilogue-class rows (0x800F42AC 96 B, 0x80100038 104 B), both carrying the
maspsx=epilogue token -- the first rows closed through the new mode.
AND THE TRANSFORM IS NOW CORRECT FOR BOTH SHAPES, which worker A's report 24 showed was
necessary. The 120 rows split:
A) lw $31 IMMEDIATELY before the release -> the release moves into the slot AND a nop must be
inserted after lw $31, or j $31 lands in its load-delay slot. 0x800FFBEC.
B) other loads between lw $31 and the release -> the release moves into the slot and the
trailing nop is DROPPED; no load-delay nop is needed. Worker A's 0x800F44D0.
My first implementation did A only and left every B row 4 bytes long. Both are handled now, and
the discriminator is whether the jump's own register was loaded immediately before the release.
A BUG WORTH RECORDING: reading out[-1] to find that preceding instruction saw maspsx's own
'#nop # DEBUG: ...' comment instead of the lw, silently producing the shape-B answer for a
shape-A row and turning a MATCH back into a LENGTH-MISMATCH. The scan now skips comments.
|
||
|
Christopher Williams
|
05be974ce2 |
phase11: THE EPILOGUE POST-PASS SHIPS (maspsx=epilogue) — 565 bodies / 574 regions
Finding 84 named the transform; it is now implemented and 0x800FFBEC matches (80 B, 0 differing) where it was 6 differing bytes without it. IT IS A SWAP, NOT A MOVE, and getting that wrong cost one implementation: the candidate is lw $31,16(sp) / addiu sp,sp,24 / jr $31 / nop and the original is lw $31 / nop / jr $31 / addiu sp,sp,24 -- SAME instruction count, two words swapped. My first version moved the release after the jump and dropped the nop, producing 3 instructions instead of 4 and turning an 80-byte row into a 76-byte LENGTH-MISMATCH. A 'small mechanical transform' still has to be checked against the bytes. SCALE: 120 unclaimed rows have the filled epilogue in the ORIGINAL (scanned every worklist row's tail for jr $31 followed by a positive addiu sp,sp,N). They are mostly SMALL -- 76, 76, 80, 92, 92, 96, 104 B -- so this is a large class of cheap rows that were blocked on a HARNESS GAP rather than on source shape. 770 other rows have the unfilled shape and need nothing. The tracked patch is regenerated and verified to reproduce both modified maspsx files from the pristine checkout. |
||
|
Christopher Williams
|
686e906b97 |
phase11: merge 37 + calibrate sf3_family + cookbook 136-137 — 563 bodies / 572 regions
Worker A's 0x80036F70 (460 B, first attempt, family score 1.000 AND adjacent to its own 0x80036DA4). Its family run finished 7 for 7 with five first-spelling matches. 136: worker A CALIBRATED the family tool. It checked the two 0.97-scoring entries and NEITHER shares its sibling's body at all -- one is a table-allocation routine, the other a summing loop. '1.000 is the useful band; below ~0.99 the histogram is matching common idioms, not bodies.' That is the same false-positive mode as the redundancy ranker (finding 110). The default threshold is now 0.99. 137: a family's signature can be a CONSTANT TRIPLE -- worker A's 0x80036F70 differs from its sibling only in six constants, whose signature is (A, A+12, A-58). Searchable in a way no similarity metric can be, because the shapes are identical and only the immediates differ. |
||
|
Christopher Williams
|
9ff344834c |
phase11: cookbook 130-131 + document the ranker's size bias
130: worker D found the ranker's top is SIZE-BIASED -- redundancy and size are correlated because a bigger body has more chances to repeat a 3-gram. Its top was 3288 B at 0.91 while the rows it was matching sat at 0.64-0.74 in the small tail. The metric is not wrong about rows of equal size; the raw score just cannot be compared across bands. Since the milestone counts BODIES, a 3288 B row and a 248 B row are worth one body each and the large one costs many times the context. Worker D's effective filter was redundancy PER UNIT SIZE, now approximated by --max-size, which is documented in the tool. 131: an unreferenced ARRAY local homes but an unreferenced SCALAR does not -- second independent instance, identical mechanism, so it is a rule: when the frame is a clean multiple of 8 bytes short and everything else is identical, add an unreferenced array local of that size. |
||
|
Christopher Williams
|
6589668e85 |
phase11: tools/sf3_family — search for a matched row's siblings (cookbook 125)
Worker D's finding 125 said three members of one family were found by three different means and 'the finder varies, the price does not', concluding that families should be SEARCHED FOR explicitly rather than waited for. This implements that: for every unmatched worklist row, find the already-matched row with the highest similarity, where similarity is an opcode-histogram cosine (registers erased, nops dropped, per finding 120) multiplied by the size ratio so a shared multiset at a different scale does not count. A BUG WORTH RECORDING: the first version read config/match_worklist.tsv with the regions column layout, so column 0 (the RANK) was read as the address. It returned ZERO candidates at every threshold, which is what exposed it -- a silently wrong address yields no matches rather than an error. Both layouts are now parsed by named functions with the offset documented. |
||
|
Christopher Williams
|
46c260c252 |
phase11: worker B's adjudication makes the fragment check DISJOINT — 0 of 555
The first version of the fragment check fired on any nonzero sp offset and flagged 2 of the 555 registered regions. Worker B read its own hit (0x800B704C) before committing and showed it is a LEGAL FRAMELESS LEAF WITH EIGHT ARGUMENTS: in o32 the callee's sp is unchanged at entry, so sp+16..sp+28 IS the caller's outgoing area -- arguments 4-7 -- and reading it before any addiu sp,sp,-N is exactly what a frameless >4-argument leaf looks like. Its evidence: exactly one jr ra, zero jal, zero addiu sp,sp,-N, zero sw ra/lw ra across all 324 bytes, and no callee-saved register touched. Excluding the incoming argument area (sp+0..sp+31) and flagging only a negative offset or an offset beyond the 8-argument area makes the check DISJOINT: registered regions flagged: 2 of 555 -> 0 of 555 suspects across 4 partitions: 5 -> 1 The one remaining suspect is worker A's 0x800C3490. This is the cleanest example in the phase of a worker ADJUDICATING a tool's output rather than obeying it -- B was explicitly told 'advisory, do not skip', read the row anyway, and its adjudication turned a noisy heuristic into a precise one. |
||
|
Christopher Williams
|
74199d1ab3 |
phase11: tools/sf3_rank --fragments + cookbook 114 — worker A's fragment anomaly
Worker A found 0x800C3490 is not a matchable body: it starts mid-expression with sw v0,32(sp) before any frame setup, and its identical tail also appears at 0x800C3470, so it is a shared/jump-target block Ghidra promoted to a function -- inside no region, with the code before it in no worklist, so nobody can match it standalone. Its generalised rule is narrower than 'first instruction is not prologue-like', because a function may legally start with beq/sh/move: a row is a FRAGMENT if its first instruction touches the stack before any addiu sp,sp,-N, reads a stack slot, or uses a callee-saved register that is never saved. Implemented as a --fragments scan. Measured: 5 suspects across all four partitions, but 2 false positives across the 555 REGISTERED regions, so it is ADVISORY not an exclusion -- sufficient-but-not-complete like the trapping check. A hit means read before spending a spelling, never skip. |
||
|
Christopher Williams
|
17e3b4496f |
phase11: pin the ranker's statistic + cookbook 107-108
Worker B's comparability caution: the redundancy score is the MAXIMUM of the 2/3/4-gram repeat ratios, so a high score means 'some length scale is very repetitive', not 'repetitive at every scale'. Any implementation using a mean or a different normalisation gives non-comparable numbers, which matters because workers were sharing rankings across partitions. Pinned in the tool's docstring with this tool named as the reference implementation. 107: finding 43's goto lever is NECESSARY BUT NOT SUFFICIENT -- worker B found it overshoots (152 vs 140) on 0x800FCA90, and this is the second independent instance after worker A's 0x800256F0. Treat it as one attempt, not as the fix. |
||
|
Christopher Williams
|
bb1e3e5685 |
phase11: tools/sf3_rank — the redundancy ranker as shared tooling
Promoted from worker B's staging implementation (which independently reproduced worker A's metric). Ranks worklist rows by tie-break density: for n = 2/3/4, the fraction of n-instruction opcode subsequences already seen earlier in the body, best of three, with NOPS EXCLUDED (worker A's refinement -- lw/nop pairs otherwise inflate arithmetic rows). This is the phase's central dispatch finding made reproducible. Evidence: worker A matched 548 B / 460 B / 356 B / 204 B all on the FIRST spelling while its one nine-attempt failure was the SMALLEST row it attacked (176 B); worker D matched 1232 B on the 2nd spelling and 700 B on the 3rd while its 248 B row took 4. Cost is set by tie-break density, not size. Companion rule (cookbook 99): once a row is matched, the row ADJACENT to it beats even the top of this list -- the binary is laid out by translation unit, so neighbours share the author's habits. 4-for-4 across two workers. |
||
|
Christopher Williams
|
c4f4bc0cfa |
phase11: CORRECTION — the Phase 10 #nop-honouring fix was TOO BROAD; now opt-in (maspsx=nopmarker)
Worker C found the defect while characterising an above-ceiling row: the Phase 10 fix that made maspsx honour cc1's explicit `#nop` marker unconditionally is wrong for a bare-symbol store consumer, because the store's own `lui $at` expansion fills the delay slot and the marker is SPURIOUS. On 0x800AFDBC the original is `lhu` / `lui at` / `sh` with NO nop, and honouring the marker costs 2 instructions -- turning a row that was otherwise byte-identical into a LENGTH-MISMATCH. But the fix is genuinely needed for 0x80107C5C (112 vs 108). So the two rows want opposite behaviour from the same instruction shape, and the honest resolution is to make it a per-region mode rather than a global default. `maspsx=nopmarker` -> --honour-nop-marker (default OFF) VERIFIED BOTH WAYS: make check with the mode OFF -> regions=510 AGREE, differing_bytes=0 MATCH, 253 tests OK 0x80107C5C WITH the mode -> 112 B, differing_bytes=0 MATCH 0x80107C5C WITHOUT it -> 108 B LENGTH-MISMATCH So the default is the long-standing behaviour, and rows that need the marker opt in. The tracked patch is regenerated and verified to reproduce both modified files exactly from the pristine pinned checkout (tools/maspsx is git-ignored, so the patch is the only reproducible carrier). docs/SETUP.md records the correction. This is the third time this phase a worker found a defect in work the coordinator had already shipped as verified -- the pattern is worth noting: a fix proven regression-free against the CORPUS can still be wrong for an UNMATCHED row, because the corpus only exercises the paths that already work. |
||
|
Christopher Williams
|
55e59b3a4a |
phase11: merges 6-7 + the maspsx=moves mode — 501 bodies / 510 regions, MAX 700 B
+5 bodies: worker A claims 5-8 (0x800507A0, 0x80017B50, 0x800BBAC8, 0x800AFACC) and
worker D's 0x8006BC74 (700 B). Every candidate gate MATCH before promotion; all md5s
verified on disk.
*** 700 B IS THE LARGEST BODY EVER MATCHED IN THIS PROJECT *** — 175 instructions,
2.9x the old 244 B ceiling, and the FIRST match in the 401-800 B band. It cost THREE
spellings, fewer than worker D's own 248 B row (four). Both residuals were mechanical:
a missing `li 4096 / sw` pair hidden inside a run of 46 zero stores ("a run of repeated
stores is not a run of identical stores -- read every immediate"), and four extra
pointer reloads fixed by naming the sub-object pointer ONCE for the three byte stores
of 255 while leaving the fourth store its own re-read (cookbook 45's named-locals
family at its cheapest). Nothing about 700 bytes was hard: the body is large but highly
REDUNDANT, and redundancy is what a matcher keys off.
NEW HARNESS MODE `maspsx=moves` (worker B's oracle result, developer-authorized).
Worker B ran all five SDK assemblers (ASPSX 2.56/2.67/2.79/2.81/2.86) and every
supported option as a read-only oracle and found that **ASPSX does NOT fill delay slots
at all** -- it produces maspsx's exact shape. So maspsx is FAITHFUL to ASPSX, and the
fills in the original did not come from ASPSX. That overturns the "model ASPSX's fill"
framing: what fills the slots is GNU `as` in REORDER mode, i.e. maspsx OFF, and the only
real gap is ONE MNEMONIC -- `as` expands cc1's `move` to `or` where ASPSX emits `addu`.
So the mode is `maspsx=off` plus a single `move`->`addu` rewrite, letting `as` fill
exactly the slots cc1 left empty while cc1's own `.set noreorder` windows are preserved.
DEMONSTRATED: 0x800FA5D8 now reports 132 bytes / differing_bytes=0 MATCH where default
maspsx gives 148 LENGTH-MISMATCH. 7 new tests; suite 246 -> 253.
REGRESSION-VERIFIED: make check green at 510 regions / 253 tests with the mode OFF, so
every one of the 510 regions is byte-identical. The mode stays opt-in per region --
worker B measured the counterexample 0x8002D2BC, which has the SAME cc1 shape but whose
original keeps the store before the jr with a nop, so the original's assembler behaves
differently in different files.
|
||
|
Christopher Williams
|
7d7f41fbaa |
phase11: merge 3 — 493 bodies / 502 regions, TWO bodies above the old 244 B ceiling
+8 bodies: worker A claims 1-4 (0x8005E820, 0x80012DE8, 0x8001644C, 0x800A6880)
and worker C claims 1-4 (0x80048180, 0x800B107C, 0x80082868, 0x80036134).
Candidate gate MATCH before promotion; all 8 md5s matched on disk.
FOUR gp symbol rows added for 0x80017C6C (D_80121A2C/34/3C/44), coordinator-verified
against the payload: the original materialises them with addiu $2,gp,244/252/260/268.
C's claim 4 (0x80036134, 248 B) is the SECOND body above the old ceiling, matched on
lever-c-large row 1 — so two independent workers have now matched above 244 B, and the
dispatch-artefact verdict is confirmed by result rather than by inference.
sf3_merge format fixes, both triggered by real worker files:
- a bare header row is now rejected with "looks like a column HEADER" instead of a
confusing "not a hex address: 'start'"
- a lone `-` in the override column means "no overrides", matching the absent-value
convention the other tracked tables use
Suite 246 tests OK; make check green: regions=502 AGREE, differing_bytes=0 MATCH.
|
||
|
Christopher Williams
|
5d41f97421 |
phase11: *** CEILING BROKEN *** — 485 bodies / 494 regions, new max 248 B
Worker D's claim 0x8009F6A0..0x8009F798 (248 B) MATCHES. Verified independently by the coordinator on a fresh work dir (candidate_bytes=248 differing_bytes=0 MATCH) and gated on the whole binary before promotion. This is the FIRST body above 244 B ever matched, and it sets a new corpus maximum (previous max 244 B at 0x80099078). THE LEVER (worker D, 4 spellings): the local working buffer must be a 3x4 word array (`int t[3][4]`, only columns 0..2 used), NOT `int t[9]`. The 4-WORD ROW STRIDE IS BYTE-LOAD-BEARING: it moves the 2nd and 3rd triples to 0x10 and 0x20, makes the frame 48 B instead of 40 B, and leaves the unused 0x0C/0x1C slots the original shows. New instance of cookbook 54 (a 2-D array's row stride is byte-load-bearing). The element type is the other half: `short` locals let cc1 drop the sign extension (lhu/subu, no frame); `int` locals keep it (lh/negu). Diagnostic broadcast: correct length + right instruction multiset and order + residual concentrated on the FRAME ADJUSTMENT and every sp-relative offset => suspect a local aggregate's row stride / element size, not the control flow. D's variant (c) was a textbook case: 19 differing bytes, all of them the frame size and the address shift that follows from it, closed by one array-shape change. Also in this commit — a fail-fast fix to sf3_merge. Worker D placed the source md5 in the claim row's 4th column, which sf3_merge passed through as a region override, so the row MERGED and only `sf3_match gate` failed later with "unknown override key 'md5'". sf3_merge now validates override keys at merge time and rejects the row with a message naming the valid keys and pointing at report.tsv for per-claim metadata. 4 new tests, suite 242 -> 246, OK. The candidate gate caught it; the tracked registry was untouched. |
||
|
Christopher Williams
|
ced0951b6c |
phase11: new exclusion class division_check (worker D's census) — worklist 1193 -> 1118
Worker D's Goal B census found a second trapping-class-style blocker: `break` NEVER appears without `div` and `div` NEVER appears without `break` -- 75 worklist rows, 0 exceptions. The shape is GCC's divmodsi4 with MASK_CHECK_ZERO_DIV|MASK_CHECK_RANGE_DIV (div / bnez+break 7 zero check / li at,-1 + bne + lui at,0x8000 + bne quotient,at + break 6 range check / mflo). `break` cannot be produced from C. COORDINATOR-VERIFIED before implementing, as the project requires: 0 of 493 registered regions contains a div, a rem or a break (disjoint from corpus) 75 worklist rows have div+break; 0 div-only; 0 break-only (perfect correlation) 69 of the 75 are >244 B Compiler matrix (worker D): every available cc1 either emits a BARE div with no check (open 2.5.7-2.91.66-psx, and Sony CC1PSX 4.0-4.5, which reject -mcheck-zero-division), or emits a check with a DIFFERENT shape (open 2.95.2-psx and CC1PSX 4.6: mflo before the check, scratch $3/$4, comparing the DIVISOR against 0x80000000; the original checks before mflo, uses $at, and compares the QUOTIENT). Not reproducible with this toolchain -- the route is a cc1 build with those masks in TARGET_DEFAULT, which is a developer-owned toolchain decision, not a source lever. Implemented as `division_check` with a counted reason; excluded_division_check=81; worklist 1193 -> 1118. 5 new tests, suite 237 -> 242, OK. make check green: regions=493 AGREE, differing_bytes=0 MATCH. |
||
|
Christopher Williams
|
a7b842faf5 |
phase10: maspsx fix — honour cc1's explicit #nop marker (developer-authorised)
cc1 emits an explicit `#nop` marker when it wants a load-delay nop. maspsx used to
RE-DERIVE the need and could overrule cc1 for a BARE-SYMBOL STORE consumer:
uses_at('sw\t$2,D_801221C4') -> True (macro store, expands via $at)
uses_at('sw\t$2,0($4)') -> False (register+offset, no macro)
nop_at_expansion is False for ASPSX >= 2.30
so neither test in _handle_nop_before_next_instruction fired, nop_required stayed
False, and an instruction cc1 had explicitly asked for was dropped (0x80107C5C at
108 vs 112; worker B's 0x8003A9C8). The fix honours the marker instead of
overruling it; only that path changes.
Worker B2 found the gap but mis-diagnosed it: its proposed fix was to extend the
`line_loads_from_reg` predicate, which ALREADY returns True for a store source, so
that patch would have been a no-op. The coordinator traced the actual call and found
the real mechanism in the uses_at / nop_at_expansion interaction. B2 then appended a
CORRECTION row to its own staged report superseding its paragraph — the right
response, and it records the general lesson: a named mechanism is a hypothesis until
it is traced, even when the observation is solid and reproducible.
REGRESSION VERIFICATION (the whole point of gating this):
make check exit 0
regions=489 disagreements=0 AGREE
c_regions=489 differing_bytes=0 MATCH
237 tests OK
All 489 previously-matched regions are byte-identical with the fix in place.
Carried as a tracked patch (tools/maspsx/ is git-ignored, so an in-place edit would
not survive a fresh clone); patch verified to reproduce both modified files exactly
from the pristine pinned checkout. docs/SETUP.md records the fix and its provenance.
0x80107C5C NOW MATCHES (112 B, 0 differing, verified against worker B2's variant
X3.c) — but that is a BARE variant with no header, and the project convention requires
a documented source stating the observed instructions and limits. So the row is
UNBLOCKED and one documented source away, not claimed. Recorded as a carry-forward.
|
||
|
Christopher Williams
|
091a9020f7 |
phase10: merge 16 + new worklist exclusion class — 453 bodies / 462 regions
+2 bodies (worker C claims 19-20) and 3 gp symbol rows (D_80121F90, D_8012196E, D_80121964). Candidate gate MATCH before promotion. NEW EXCLUSION CLASS `restores_unsaved` (worker A, coordinator-verified). A body that restores a callee-saved register it never saves cannot be a whole function: the register it restores was established by an enclosing prologue that the derived extent cut off. These are jal targets INSIDE a real function, so the walk began mid-body — distinct from bad_extent_start, which flags starts that are not function entries at all. Verified disjoint from the matched corpus before acting, as the project requires: 0 of 462 registered regions trip the rule; 11 worklist rows do. excluded_restores_unsaved=11; worklist 1265 -> 1253 rows. 5 new tests; suite 229 -> 237, OK. make check green: regions=462 AGREE, differing_bytes=0 MATCH. Independent corroboration worth recording: the new rule re-derives 0x8010080C, the false extent start worker C reported earlier via a completely different signal (the first instruction reads a register the range never defines). Two independent detections of one defect class. bad_extent_start is False for that row, confirming C's observation that the older rule missed it. Its Makefile --exclude entry is retained only as the provenance record for that defect. |
||
|
Christopher Williams
|
3dd4b4537e |
phase10: cycle-1 merges 1-7 — 432 distinct bodies / 441 regions
32 new bodies from 400, all verified on the candidate whole-binary gate before promotion. SHA-1 e173426c157384ebf1b6caf8c6fea18a85a14af9 stable. Registry requests granted (each byte-verified with a failing control): cc1=-G8 on 0x800A6BEC; gp=-D_80121B88 on 0x80015D50 symbols D_80122700, D_80122704, D_80121AD4 (gp) Harness: per-region maspsx modes wired through sf3_match (maspsx=noreordernop, maspsx=regread) plus --no-jump-slot-nop/--nop-on-reg-read for range. Both are opt-in and default-off; make check green at 441 with them off, suite 229 -> 232 tests. Carried as a TRACKED patch (tools/patches/maspsx-phase10-r1r2.patch) because tools/maspsx/ is git-ignored, so an in-place edit would not survive a fresh clone; patch verified to reproduce the working tree byte-identically. R1/R2 are recorded as a MEASURED NEGATIVE: neither closes a region (cookbook finding 40 has the mechanism and the remaining developer-owned route). Docs: cookbook finding 40 (rare-epilogue mechanism + why the obvious maspsx fix fails); SETUP.md maspsx patch provenance and apply step. Negatives: 0x8010AA28 imported; index sorted by address (140 rows, 0 registered). Full clean audit green: make clean && make all exit 0, cmp exit 0, both SHA-1 match, registry 441/0 overlaps/0 bad extents/0 missing sources, 0 firewall. |
||
|
Christopher Williams
|
a38aa2603a |
phase9: recorded-negatives exclusion in sf3_triage (worker B, verified) + phase negatives index import
Worker B measured that recorded negatives cluster at the head of every partition: 39 of the first 40 worklist rows and 57 of 271 tier-1 rows (21%) were addresses an earlier session had already attempted and recorded. The triage tool never read the negatives index, so every closed negative returned to the queue head and workers rediscovered them. Fix: sf3_triage reads the tracked config/near_match_negatives.tsv (a new --negatives input) and excludes those rows as recorded_negative=69. The tracked index was grown from 38 Phase-8 rows to 94 by importing 56 Phase-9 negatives from the workers' reports and the coordinator's absorption record (filtered to unregistered; tracked rows win on conflict). Worklist regenerated: the head is now genuinely fresh (0x800F9134, 0x80042D88, 0x800450C4, ...). 3 new synthetic tests; plan-level tests wired for the new argument. The phase plan said extract negatives only at close; B's measurement showed the harm is per-cycle, so the index is now a living tracked input. Note for P9-T8: keep importing worker negatives each cycle. |
||
|
Christopher Williams
|
d1b9cf7ad1 |
phase9: extend trapping-arithmetic exclusion to the immediate ADDI form (worker B, verified)
Worker B found the exclusion detector's blind spot: opcode 0x08 (addi, traps on overflow) appears in 7 of 1,666 worklist rows (frame adjustments addi sp,sp,-28, loop counters) while 0 of 310 registered regions contains one — the same disjointness signal as finding 26's R-type class. addiu (0x09) is ubiquitous, so the 0x08 occurrences are the anomaly. Verified by the coordinator from raw words: 7 rows, incl. 3 of partition A's 5 call-shape rows (0x80011084, 0x800F3B10, 0x80010F30) and 0x800F3BB4 (previously recorded as a separate coordinator negative — now explained by this class). sf3_triage.trapping_arithmetic now counts opcode 0x08; worklist regenerated (1,563 rows, excluded_trapping_arith=54). Synthetic test added. P9-T5 route (a) widened: the immediate form is part of the class; the 7 ADDI rows are a ready-made minimal probe set for any cc1 candidate. Also confirmed this cycle: the no-frame global-ra ISR family (ra saved to a global, no stack frame — 0x801097A0 proved, 0x8010B420 pattern-transferred) is a second library-asm class identical in kind to the CRT entry. |
||
|
Christopher Williams
|
6d3a6ba434 |
phase9: bad-extent classification in sf3_triage (worker A's class, coordinator-verified)
Worker A reported nine worklist rows graded exact that are not function starts. The coordinator verified the tell independently: a candidate whose first instruction reads a register the range never defines (non-parameter), or whose range has no jr/jalr return, is a wrong extent. Measured disjoint from the matched corpus (7 worklist rows flagged incl. the runaway 0x801800C4 and the fallthrough 4-byte rows 0x80100808/0x80180808; 0 of 288 registered flagged). Implemented as excluded_bad_extent_start in sf3_triage with 4 synthetic tests; worklist regenerated to 1,570 eligible rows. |
||
|
Christopher Williams
|
54af82aa95 |
phase9: ledger, delay-slot exclusion, and the promoted instruction-diff tool
P9-T1: create CURRENT_PHASE.md as the phase ledger; baseline revalidated clean (cmp + SHA-1 e173426c157384ebf1b6caf8c6fea18a85a14af9, make check regions=158 disagreements=0 c_regions=158 MATCH); three workers probed and recruited (01a0d180-ea72/-efdd/-f513); 3-way partitions generated and proved disjoint (3x575 rows, pairwise intersections 0, union == worklist). P9-T2 (tooling half): promote worker B's scratch instruction diff into tools/sf3_diff, hardened per the plan: PS-X LOAD read from the header, objdump path derived from the repo, temp files under the caller's work dir, resolver printing ready-to-paste symbol names. 17 synthetic tests. sf3_triage: new counted exclusion excluded_delay_slot_start=5, implemented after independent verification of worker B's finding (a branch/jump always executes its delay slot, so no function starts at X+4; 0 of 158 registered regions does). Worklist regenerated: 1725 eligible rows, tiers 355/1365/5. 3 new synthetic tests; existing triage fixture extended. |
||
|
Christopher Williams
|
591923a32f |
phase8: exclude the trapping-arithmetic class and record the shape tells
Worker C found the strongest structural signal of the phase and asked me to act on it rather than merely record it. I re-derived the census independently before acting: 50 of the 1,937 exact extents contain trapping add/sub (funct 0x20/0x22, 410 instructions) and 0 of the 144 then-registered regions does. All four unmatched duplicate groups are inside the class. Ten cc1 builds plus the real CC1PSX 4.0-4.6 over ~30 C shapes and 15 flags never emit the trapping forms, and -ftrapv is rejected by all of them. tools/sf3_triage now detects the class and excludes it by default (excluded_trapping_arith=50, --allow-trapping to list it), with three synthetic tests, so no worker can spend budget on it by accident. Recorded as cookbook finding 26 with the neg-macro clue that points at a macro-print-style difference rather than a source difference. The class is now the best candidate for part of the unresolved library-versus-game-code boundary - a hypothesis, not a claim. Also recorded: worker A's maspsx load-delay defect (its predicate tests whether the next instruction loads FROM the register, not whether it reads it at all, so a load followed by a store of that register gets no delay nop), four mechanical layout tells, and the struct-assignment lever for the base-in-register class. Finding 22 is refined with worker C's third spelling: the written order of the + operands decides the emitted addu order. Cycle 5 merge: 65 claims, 8 accepted, 57 skipped, 0 rejected. Candidate gate c_regions=152, 0 differing bytes; promoted; make check green at 203 tests. Distinct matched bodies: 143. |
||
|
Christopher Williams
|
62983c9a8a |
phase8: syscall returns, so it is not a walk terminator
Worker C found that config/function_extents.tsv mis-split the two BIOS stubs: it recorded 0x80103FCC..0x80103FD4 (8 bytes) and 0x80103FEC..0x80103FF4, but the real bodies are 16 bytes (li a0,N / syscall / jr ra / nop). Its evidence: the first address has four jal callers and the following address has none, syscall returns to EPC+4 so the body must continue to a jr ra, every neighbouring stub is 16 bytes with the same shape, and an 8-byte body is unreachable from C because cc1 always appends an epilogue. Root cause: the walk treated `syscall` as a terminal alongside `break`. On this target syscall is the BIOS call instruction and it returns to the next word. `break` still terminates. After the fix both stubs are 16 bytes ending at their jr ra, and all 115 registered regions still agree with their derived extents. The fix also exposed three walks in the 0x8017xxxx region that the accidental syscall stop had been bounding: 0x8017D5C0 now runs out of the payload (grade=outside), 0x801800C4 becomes a 1860-byte fallthrough, and 0x8018080C a 240708-byte indirect. All three are jal-graded starts whose bodies are almost certainly data; none is registered and none is matchable. 200 tests pass. |
||
|
Christopher Williams
|
9a49ee62f7 |
phase8: close the assembler-anomaly and gp-site classes, reaching 109 bodies
Two harness gaps were closed as per-region overrides, both found by the workers and implemented by the coordinator: 1. maspsx=off. Worker A isolated the $at-macro-store-in-the-jr-delay-slot shape with a decisive experiment: cc1 emits an empty delay slot, maspsx fills it with a nop, and GNU as in reorder mode fills it properly by moving the last half of the expanded store. Six regions need this, including 0x800F3160 -- Phase 5's first recorded bounded negative, unreachable for three phases -- and 0x800F8AEC, 0x800F8B58, 0x800F8B6C, 0x800F8FE4, 0x80102B10. Worker B reverse-engineered the same mechanism independently from the cc1 output. 2. gp=-NAME. The gp marker is per symbol, but the original's access form is per SITE: 0x80121F84 is read gp-relative at 0x800A80BC and written absolutely at 0x8002D288, in the same function. A byte search confirmed 6 gp-relative accesses at d=0x64c, so worker B's census row was right and worker A's source was right; only a per-region exclusion can express both. This makes cookbook finding 10 incomplete. Both overrides are validated (199 tests, 10 added), and sf3_merge gained --skip-registered because a worker's claims file is naturally cumulative. Cycle-2 merge: 66 claims, 58 accepted, 8 skipped, 0 rejected. Candidate gate: c_regions=115, 0 differing bytes, SHA-1 e173426c. Promoted, then make check green: 199 tests, regions=115 disagreements=0, c_regions=115 MATCH. Distinct matched bodies: 51 -> 109, past the phase milestone of 70. |
||
|
Christopher Williams
|
d7047e08c7 |
phase8: add the claim-merge tooling and merge worker A's first ten claims
P8-T2 plus the first verified cycle-1 merge. tools/sf3_merge validates worker claims before anything tracked changes: a claim is accepted only if its extent exists in the derived extents table and is graded exact with exactly the claimed end, its source is a repo-relative src/func_XXXXXXXX.c that exists, and it overlaps neither the registry nor another worker's claim. Rejections are reported with reasons and nothing is written. The workflow it enables is stronger than the plan's wording: merge to a CANDIDATE registry, gate the candidate, and promote only on MATCH, so the tracked registry never contains an unverified claim. Worker A reported 10 claims (target 8) and correctly refused to edit the shared symbol registry itself, instead requesting 9 gp-marked rows; each was checked arithmetically as gp + d. Candidate gate: c_regions=49, 0 differing bytes, SHA-1 e173426c. Promoted, then make check green: 189 tests, regions=49 disagreements=0, c_regions=49 MATCH. Worker A also reported a significant blocker: two of the remaining duplicate groups are GTE (COP2) bodies -- 0x80018CB0 (3 addresses) and 0x8001084C (2 addresses, the 712-byte shared body) -- plus 0x80103A94 and 0x80103B60. That is four GTE functions, which raises the value of the bounded SDK-shape investigation in P8-T5. |
||
|
Christopher Williams
|
399da98a83 |
phase7: resolve address-named symbols implicitly and fail loudly otherwise
Every cross-reference used to need a hand-written config/symbols.tsv row, which does not scale to a batch. A symbol whose name is an address now resolves to that address with no row: func_XXXXXXXX, D_XXXXXXXX, g_XXXXXXXX, lbl_XXXXXXXX. This is the convention the registry already used, applied without the row. The names come from the object's own undefined-symbol list (nm -u), not from a guess about the source, so a name the source defines but never references is never mistaken for one needing resolution. A registry row still wins, which is how a gp marker or a real name is attached. Anything neither registered nor address-shaped now fails before the link with a message naming the symbol and the fix, instead of a bare ld diagnostic. First match from the worklist: func_800F8F9C (36 bytes, worklist rank 3), a duplicate-group representative with a frame and a call. It matched on the first attempt and is registered twice (0x800F8F9C, 0x80109314) against one source, so two functions were matched for one body. Its callee needed no registry row. Ghidra's independent body agrees with the derived extent. make gate: c_regions=14, 0 differing bytes, SHA-1 e173426c. 168 synthetic tests. |
||
|
Christopher Williams
|
a6af8b3cd2 |
phase7: replace hand-picked targets with a ranked, reproducible worklist
Phase 6 chose match targets by eye from the boundary inventory. tools/sf3_triage now ranks every eligible candidate by (tier, size, address) from tracked inputs alone and records why everything else was excluded. Eligibility: an exact or fallthrough extent, a non-degenerate body, not already registered, not the header entry, not named by --exclude. indirect, escape, outside, runaway, contained and standalone are excluded and counted. Tiers: 0 duplicate-group representative (one match, several addresses), 1 exact leaf (no cross-references, so no symbol rows), 2 exact non-leaf, 3 fallthrough. Result: 1916 listed (tier 0: 9, tier 1: 509, tier 2: 1394, tier 3: 4), with 252 degenerate bodies, 88 low-confidence grades, 12 registered, 1 header entry and 2 named near-misses excluded. The nine tier-0 entries are the real duplicate groups: matching those nine bodies registers 22 function addresses. The two deferred near-misses are excluded by name in the Makefile so the exclusion stays visible rather than buried in the tool. 160 synthetic tests pass. |
||
|
Christopher Williams
|
2000cc4101 |
phase7: census duplicate bodies and expose a zero band of false positives
Matching conventions require a duplicate check before registering, because a shared body is matched once and registered once per address. Phase 6 did that check by hand and found one 12-byte pair. tools/sf3_dupes now hashes every derived extent body and groups exact duplicates. Results: 2284 extents, 65 multi-address groups, 2104 singletons. Only 10 groups contain code (24 addresses, all exact-graded); 55 are all-zero bodies. The hand-found pair 0x800262E0/0x800262EC is reproduced as g0002, which is the check that the census measures what it claims. The largest real groups are 712 bytes (0x8001084C/0x800189E8) and 436 bytes. The zero groups are a real finding: 252 extents have all-zero bodies, 245 inside the zero band 0x80147000..0x80170000. The cause is the inventory's jal grade, which decodes every word as an instruction -- in a data region a word with opcode 3 is graded as a call whose target lands in the zero band. The census flags those groups rather than hiding them, and the worklist must exclude degenerate bodies. The census is tracked rather than ignored as the plan said, because it holds addresses, sizes and grades only (the same class as the tracked inventory and extents tables) and the worklist must be reproducible from tracked inputs. The content hash is computed and never written. |
||
|
Christopher Williams
|
6988ca96b0 |
phase7: derive evidence-graded function extents from control flow
Phase 6 graded function starts and left every end to be derived by hand. This adds tools/sf3_extents, which explores all reachable control flow from each hard start (jal/entry) and reports an extent plus how far it can be trusted. Measured decisions, not stylistic ones: - Soft starts are not walk boundaries. A body's second instruction can satisfy the prologue grade exactly (0x800152AC is lw v1,8(gp) / addiu sp,sp,-176, so 0x800152B0 looks like a start). Enforcing soft boundaries stopped 155 of 416 walks inside a real body. - The walk is a full reachability computation, not a first-terminal search: a function whose paths return at different addresses must report the whole body. Grades: exact 1940 (1666 packed, gap=0), fallthrough 256, indirect 73, escape 15, contained 153, standalone 438; 63.8% of the payload covered. Verification: all 12 registered regions reproduce exactly (make extents-verify, now part of make check), 29 new synthetic tests (115 total), byte-identical across two runs, and Ghidra's independent body for FUN_80017ad4 agrees. Two defects were caught by writing the tests first and are recorded: reach had to be an exclusive end, and a terminal j's delay slot must not continue linearly. The table holds addresses, sizes, grades and site addresses only -- no bytes. |
||
|
Christopher Williams
|
2507994ac3 | phase6: correct the compiler to PsyQ 4.0 (gcc-2.7.2-psx) and register the framed batch | ||
|
Christopher Williams
|
d951e49b1d | phase6: add the evidence-graded function-boundary inventory | ||
|
Christopher Williams
|
d379b84ec3 | phase6: reproduce a gp-relative function and record the small-data evidence | ||
|
Christopher Williams
|
4688662cf4 | phase6: wire maspsx and link-time symbols, resolving the ASPSX la form |