Christopher Williams
677d982398
Worker A's third claim, verified from a fresh --work dir against the exact md5 in the claim,
merged to a candidate, gated whole-binary, promoted only on result=MATCH. It is the largest body
any worker has taken this phase (572 B).
sf3_match gate c_regions=630 differing_bytes=0 result=MATCH
sha1 e173426c157384ebf1b6caf8c6fea18a85a14af9 (unchanged)
make check exit 0 make extents-verify regions=630 disagreements=0 AGREE
make worklist listed=989, excluded_already_registered=630
registry audit 630 rows, 621 distinct sources, 0 missing, 3 carry cc1bin
**ADJACENCY IS NOW 3-FOR-3 FOR WORKER A, ACROSS THREE BANDS.** 0x800297F4 (204 B, band 0) ->
0x800298C0 (392 B, band 1) -> 0x80029A48 (572 B, band 2): three CONSECUTIVE bodies, 1168 bytes,
four spellings total, with the density ranker not involved in any of the three picks. The chain
ends where no row starts at the next boundary. That is the strongest single piece of dispatch
evidence in the phase, and it is why the fresh-band files were re-cut to put adjacency above
density inside a band.
Worker A's lever, recorded because it is an operator RECOVERY rule rather than a spelling tip: the
original is `if ((x1 < 0 && x2 > 0) || (x1 > 0 && x2 < 0))`, and the truth-table-equivalent
if/else ladder is EXACTLY ONE INSTRUCTION SHORT. The emitted stream is `bgez x1` / `bgtz x2` /
`blez x1` / `bgez x2` -- each term's first test branches over its own second test -- and the ladder
has no `bgez x1` to emit. So the four BRANCH SENSES let you write the operator down without
guessing; cookbook 83's "branch direction distinguishes && from ||" turned into a recovery rule.
And `(x1 ^ x2) < 0` is the same predicate with the wrong codegen: the original compares.
**0x800C3490 IS EXCLUDED FROM THE WORKLIST.** Cookbook 114 (worker A, Phase 10) records it as a
FRAGMENT: it starts mid-expression, its body is a SHARED TAIL (`addiu sp,sp,48; jr ra`) that also
appears at 0x800C3470-0x800C348C, and it cannot be matched standalone. The extents table still
grades it `exact` with `term=jr_ra` because the boundary walk sees a well-formed terminal, so the
tool cannot catch it -- which is exactly why it needs a recorded exclusion rather than a tool rule.
Worker A recognised it for the SECOND time in Phase 12 and skipped it instead of spending reading
budget, which is the signal that the exclusion belongs in the Makefile: a row that has to be
re-identified by hand every phase is a row the dispatch should not be offering.
Added with its provenance in the Makefile comment, the same mechanism Phase 10 used for
0x8010080C's false extent start. excluded_named_exclusion 8 -> 9.
Ledger updated with merges 2-5, the cc1bin lever and its gate, the mechanism correction, the gp
pair rewrite and both of my failures in it, the dispatch measurement (369 of 993 rows abut SOME
region = noise; only 5 abut a PHASE-12 match = the signal), and the partition-membership fix.
161 lines
6.3 KiB
Makefile
161 lines
6.3 KiB
Makefile
# Phase 3 all-assembly baseline orchestration.
|
|
# All original and generated ROM-derived inputs/outputs remain ignored.
|
|
|
|
SHELL := /usr/bin/bash
|
|
|
|
EXE := extracted/SCUS_946.40;1
|
|
MANIFEST := extracted/MANIFEST.tsv
|
|
SPLAT := tools/splat/.venv/bin/splat
|
|
AS := tools/mipsel-none-elf-binutils/prefix/usr/bin/mipsel-none-elf-as
|
|
LD := tools/mipsel-none-elf-binutils/prefix/usr/bin/mipsel-none-elf-ld
|
|
OBJCOPY := tools/mipsel-none-elf-binutils/prefix/usr/bin/mipsel-none-elf-objcopy
|
|
|
|
ASM_ROOT := asm
|
|
SPLAT_CONFIG := $(ASM_ROOT)/scus_946_40.yaml
|
|
SPLAT_OUTPUT := $(ASM_ROOT)/generated
|
|
LINKER_SCRIPT := $(ASM_ROOT)/scus_946_40.ld
|
|
BUILD := build
|
|
ELF := $(BUILD)/scus_946_40.elf
|
|
REBUILT := $(BUILD)/scus_946_40.rebuilt
|
|
INPUT_REPORT := .run/p3-exe-info.tsv
|
|
|
|
# Phase 5 ordered code build: C regions listed in the tracked registry are
|
|
# compiled with the identified toolchain and linked in address order between
|
|
# data fallbacks taken from the original executable. An empty registry
|
|
# reproduces the all-payload data baseline exactly.
|
|
REGIONS := config/regions.tsv
|
|
SYMBOLS := config/symbols.tsv
|
|
INVENTORY := config/function_inventory.tsv
|
|
EXTENTS := config/function_extents.tsv
|
|
DUPES := config/duplicate_bodies.tsv
|
|
NEGATIVES := config/near_match_negatives.tsv
|
|
WORKLIST := config/match_worklist.tsv
|
|
MATCH := tools/sf3_match
|
|
EXTENTS_TOOL := tools/sf3_extents
|
|
DUPES_TOOL := tools/sf3_dupes
|
|
TRIAGE_TOOL := tools/sf3_triage
|
|
CODE_OUT := build/code
|
|
EXPECTED_SHA1 := e173426c157384ebf1b6caf8c6fea18a85a14af9
|
|
|
|
.PHONY: all validate split assemble link binary code gate extents extents-verify dupes worklist test check clean
|
|
|
|
all: binary
|
|
|
|
validate:
|
|
@test -f "$(EXE)"
|
|
@test -f "$(MANIFEST)"
|
|
@mkdir -p .run
|
|
@./tools/sf3_extract psx-exe-info "$(EXE)" "$(MANIFEST)" > "$(INPUT_REPORT)"
|
|
|
|
split: validate
|
|
@test -f "$(SPLAT)"
|
|
@test -f "$(SPLAT_CONFIG)"
|
|
@cd "$(ASM_ROOT)" && ../"$(SPLAT)" split "$$(basename "$(SPLAT_CONFIG)")" --disassemble-all
|
|
|
|
assemble: split
|
|
@test -d "$(SPLAT_OUTPUT)" && test ! -L "$(SPLAT_OUTPUT)"
|
|
@test -f "$(AS)"
|
|
@test ! -L "$(BUILD)"
|
|
@mkdir -p "$(BUILD)"
|
|
@while IFS= read -r source; do \
|
|
rel="$${source#$(ASM_ROOT)/}"; \
|
|
object="$(BUILD)/$${rel}.o"; \
|
|
mkdir -p "$$(dirname "$$object")"; \
|
|
"$(AS)" -march=r3000 -G0 -I "$(ASM_ROOT)/include" -o "$$object" "$$source"; \
|
|
done < <(find "$(SPLAT_OUTPUT)" -type f -name '*.s' -print | sort)
|
|
|
|
link: assemble
|
|
@test -f "$(LINKER_SCRIPT)"
|
|
@test -f "$(LD)"
|
|
@rm -f "$(ELF)"
|
|
@"$(LD)" -T "$(LINKER_SCRIPT)" -o "$(ELF)"
|
|
|
|
binary: link
|
|
@test -f "$(OBJCOPY)"
|
|
@rm -f "$(REBUILT)"
|
|
@"$(OBJCOPY)" -O binary "$(ELF)" "$(REBUILT)"
|
|
|
|
# Ordered code build and full-binary gate (Phase 5). `code` builds only;
|
|
# `gate` builds and then requires the whole executable to be byte-identical.
|
|
code: validate
|
|
@test -f "$(MATCH)"
|
|
@test -f "$(REGIONS)"
|
|
@test -f "$(SYMBOLS)"
|
|
@rm -rf "$(CODE_OUT)"
|
|
@"$(MATCH)" build --exe "$(EXE)" --regions "$(REGIONS)" --symbols "$(SYMBOLS)" --out "$(CODE_OUT)"
|
|
|
|
gate: validate
|
|
@test -f "$(MATCH)"
|
|
@test -f "$(REGIONS)"
|
|
@test -f "$(SYMBOLS)"
|
|
@rm -rf "$(CODE_OUT)"
|
|
@"$(MATCH)" gate --exe "$(EXE)" --regions "$(REGIONS)" --symbols "$(SYMBOLS)" --out "$(CODE_OUT)" \
|
|
--expect-sha1 "$(EXPECTED_SHA1)"
|
|
|
|
# Function extents (Phase 7). `extents` regenerates the tracked table from the
|
|
# inventory; `extents-verify` re-derives it and requires every registered region
|
|
# to agree, so a region can never silently disagree with the derived extent.
|
|
extents: validate
|
|
@test -f "$(EXTENTS_TOOL)"
|
|
@test -f "$(INVENTORY)"
|
|
@"$(EXTENTS_TOOL)" scan --exe "$(EXE)" --inventory "$(INVENTORY)" --out "$(EXTENTS)" --force
|
|
|
|
extents-verify: validate
|
|
@test -f "$(EXTENTS_TOOL)"
|
|
@test -f "$(INVENTORY)"
|
|
@test -f "$(EXTENTS)"
|
|
@"$(EXTENTS_TOOL)" verify --exe "$(EXE)" --inventory "$(INVENTORY)" \
|
|
--extents "$(EXTENTS)" --regions "$(REGIONS)"
|
|
|
|
# Duplicate-body census (Phase 7): groups byte-identical bodies so a shared body
|
|
# is matched once and registered once per address.
|
|
dupes: validate
|
|
@test -f "$(DUPES_TOOL)"
|
|
@test -f "$(EXTENTS)"
|
|
@"$(DUPES_TOOL)" census --exe "$(EXE)" --extents "$(EXTENTS)" --out "$(DUPES)" --force
|
|
|
|
# Ranked match worklist (Phase 7). The --exclude addresses are recorded
|
|
# near-misses or known-bad rows deferred by developer direction; naming them here
|
|
# keeps the exclusion visible instead of burying it in the tool.
|
|
# 0x8010080C was added in Phase 10 as a FALSE EXTENT START (worker C): its first
|
|
# instruction is `beq s0,zero` with s0 never written in range, and the walk split
|
|
# one real 252-byte function (0x801007E0..0x801008DC) at a spurious `jal`-target
|
|
# boundary. It is now caught earlier by sf3_triage's general `restores_unsaved`
|
|
# rule (worker A's independent scan re-derived the same row from a different
|
|
# signal), so this row is retained only as the provenance record for that defect.
|
|
#
|
|
# 0x800C3490 was added in Phase 12 as a FRAGMENT, not a function (cookbook 114,
|
|
# found by worker A and re-confirmed by the coordinator against the registry). It
|
|
# starts mid-expression, its body is a SHARED TAIL (`addiu sp,sp,48; jr ra`) that
|
|
# also appears at 0x800C3470-0x800C348C, and it cannot be matched standalone. The
|
|
# extents table still grades it `exact` with `term=jr_ra`, because the boundary walk
|
|
# sees a well-formed terminal -- so the tool cannot catch this one. Excluded rather
|
|
# than left in the dispatch files, where it costs every fresh-band worker a reading
|
|
# budget before it is recognised again. Worker A identified it for the second time in
|
|
# Phase 12 and skipped it rather than spending that budget.
|
|
worklist: validate
|
|
@test -f "$(TRIAGE_TOOL)"
|
|
@test -f "$(EXTENTS)"
|
|
@test -f "$(DUPES)"
|
|
@"$(TRIAGE_TOOL)" plan --exe "$(EXE)" --extents "$(EXTENTS)" --inventory "$(INVENTORY)" \
|
|
--census "$(DUPES)" --regions "$(REGIONS)" --negatives "$(NEGATIVES)" \
|
|
--exclude 0x8005DEF8 --exclude 0x800F3160 \
|
|
--exclude 0x80108034 --exclude 0x8010804C \
|
|
--exclude 0x80012A10 --exclude 0x80012AE0 \
|
|
--exclude 0x80012CFC --exclude 0x80012B20 \
|
|
--exclude 0x8008A198 --exclude 0x8010080C \
|
|
--exclude 0x800C3490 --out "$(WORKLIST)" --force
|
|
|
|
# Verification gates. `test` is synthetic-only and needs no game input;
|
|
# `check` adds the extents check and the full-binary byte gate (which do).
|
|
test:
|
|
@PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s tools/tests
|
|
|
|
check: test extents-verify gate
|
|
|
|
clean:
|
|
@if test -e "$(BUILD)"; then \
|
|
test -d "$(BUILD)" && test ! -L "$(BUILD)"; \
|
|
rm -rf "$(BUILD)"; \
|
|
fi
|