phase7: replace hand-picked targets with a ranked, reproducible worklist
Phase 6 chose match targets by eye from the boundary inventory. tools/sf3_triage now ranks every eligible candidate by (tier, size, address) from tracked inputs alone and records why everything else was excluded. Eligibility: an exact or fallthrough extent, a non-degenerate body, not already registered, not the header entry, not named by --exclude. indirect, escape, outside, runaway, contained and standalone are excluded and counted. Tiers: 0 duplicate-group representative (one match, several addresses), 1 exact leaf (no cross-references, so no symbol rows), 2 exact non-leaf, 3 fallthrough. Result: 1916 listed (tier 0: 9, tier 1: 509, tier 2: 1394, tier 3: 4), with 252 degenerate bodies, 88 low-confidence grades, 12 registered, 1 header entry and 2 named near-misses excluded. The nine tier-0 entries are the real duplicate groups: matching those nine bodies registers 22 function addresses. The two deferred near-misses are excluded by name in the Makefile so the exclusion stays visible rather than buried in the tool. 160 synthetic tests pass.
This commit is contained in:
Christopher Williams
@@ -28,13 +28,15 @@ SYMBOLS := config/symbols.tsv
|
||||
INVENTORY := config/function_inventory.tsv
|
||||
EXTENTS := config/function_extents.tsv
|
||||
DUPES := config/duplicate_bodies.tsv
|
||||
WORKLIST := config/match_worklist.tsv
|
||||
MATCH := tools/sf3_match
|
||||
EXTENTS_TOOL := tools/sf3_extents
|
||||
DUPES_TOOL := tools/sf3_dupes
|
||||
TRIAGE_TOOL := tools/sf3_triage
|
||||
CODE_OUT := build/code
|
||||
EXPECTED_SHA1 := e173426c157384ebf1b6caf8c6fea18a85a14af9
|
||||
|
||||
.PHONY: all validate split assemble link binary code gate extents extents-verify dupes test check clean
|
||||
.PHONY: all validate split assemble link binary code gate extents extents-verify dupes worklist test check clean
|
||||
|
||||
all: binary
|
||||
|
||||
@@ -111,6 +113,17 @@ dupes: validate
|
||||
@test -f "$(EXTENTS)"
|
||||
@"$(DUPES_TOOL)" census --exe "$(EXE)" --extents "$(EXTENTS)" --out "$(DUPES)" --force
|
||||
|
||||
# Ranked match worklist (Phase 7). The two --exclude addresses are the recorded
|
||||
# near-misses deferred by developer direction in Phase 7; naming them here keeps
|
||||
# the exclusion visible instead of burying it in the tool.
|
||||
worklist: validate
|
||||
@test -f "$(TRIAGE_TOOL)"
|
||||
@test -f "$(EXTENTS)"
|
||||
@test -f "$(DUPES)"
|
||||
@"$(TRIAGE_TOOL)" plan --exe "$(EXE)" --extents "$(EXTENTS)" --inventory "$(INVENTORY)" \
|
||||
--census "$(DUPES)" --regions "$(REGIONS)" \
|
||||
--exclude 0x8005DEF8 --exclude 0x800F3160 --out "$(WORKLIST)" --force
|
||||
|
||||
# Verification gates. `test` is synthetic-only and needs no game input;
|
||||
# `check` adds the extents check and the full-binary byte gate (which do).
|
||||
test:
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,103 @@
|
||||
# Phase 7 — Ranked Match Worklist
|
||||
|
||||
**Scope:** P7-T4 — replace hand-picked match targets with a single deterministic, reproducible queue.
|
||||
**Status:** complete. Tracked output: `config/match_worklist.tsv`. Tool: `tools/sf3_triage`
|
||||
(25 synthetic tests).
|
||||
|
||||
## Why
|
||||
|
||||
Phase 6 chose match targets by reading the boundary inventory and judging by eye. The inventory itself
|
||||
says the heuristic out loud — "prefer `jal`-graded, unique, small bodies" — but nothing enforced it,
|
||||
nothing excluded known-bad candidates, and two people would not have produced the same list. Phase 7
|
||||
now has the evidence to do better: extents (`config/function_extents.tsv`), duplicate groups
|
||||
(`config/duplicate_bodies.tsv`), and a demonstrated class of degenerate candidates.
|
||||
|
||||
## Eligibility
|
||||
|
||||
A candidate is listed only if all of these hold:
|
||||
|
||||
| Condition | Why |
|
||||
|---|---|
|
||||
| Carries an extent, graded `exact` or `fallthrough` | `indirect` (jump-table switch), `escape` (control flow leaves the region), `outside`, `runaway`, `contained` and `standalone` are all excluded — and counted |
|
||||
| Body is not entirely zero bytes | An all-zero body is a zero-filled region the walk ran through, not code — 252 candidates. See `docs/PHASE7_DUPES.md` |
|
||||
| Not already in `config/regions.tsv` | Already matched |
|
||||
| Not graded `entry` in the inventory | CRT startup is not compiler output |
|
||||
| Not named by `--exclude` | The two deferred near-misses |
|
||||
|
||||
## Ranking
|
||||
|
||||
Rows are ordered by `(tier, size, address)`. The order is reproducible from the tracked inputs alone —
|
||||
no judgement, no timestamps, no hashes.
|
||||
|
||||
| Tier | Rule | Rationale |
|
||||
|---|---|---|
|
||||
| 0 | Duplicate-group representative with a code body | One match registers several addresses at once; the `registers` column says how many |
|
||||
| 1 | `exact` and `leaf` (no call instruction in the body) | No cross-references, so no symbol rows are needed |
|
||||
| 2 | `exact` and not a leaf | Needs symbol rows for its callees |
|
||||
| 3 | `fallthrough` | No reachable terminal, so the extent may include alignment padding; the body may still be matchable |
|
||||
|
||||
Only the **lowest-address member** of a duplicate group is listed, so a group cannot be attempted
|
||||
twice. `shape` is `frame` when the body sets up a stack frame (`addiu sp,sp,-N`, N > 0), `leaf` when it
|
||||
makes no call and has no frame, and `call` otherwise.
|
||||
|
||||
## Results
|
||||
|
||||
| Measure | Count |
|
||||
|---|---|
|
||||
| Listed | **1,916** |
|
||||
| — tier 0 (duplicate representative) | 9 |
|
||||
| — tier 1 (`exact` leaf) | 509 |
|
||||
| — tier 2 (`exact` non-leaf) | 1,394 |
|
||||
| — tier 3 (`fallthrough`) | 4 |
|
||||
| Excluded: already registered | 12 |
|
||||
| Excluded: degenerate (all-zero) body | 252 |
|
||||
| Excluded: low-confidence grade | 88 |
|
||||
| Excluded: header entry | 1 |
|
||||
| Excluded: named (`--exclude`) | 2 |
|
||||
|
||||
The head of the queue is exactly what the phase wants to attack first:
|
||||
|
||||
| Rank | Address | Size | Tier | Shape | Registers |
|
||||
|---|---|---|---|---|---|
|
||||
| 1 | `0x80042088` | 8 | 0 | leaf | 4 |
|
||||
| 2 | `0x800F7FB4` | 36 | 0 | leaf | 3 |
|
||||
| 3 | `0x800F8F9C` | 36 | 0 | frame | 2 |
|
||||
| 4 | `0x80010810` | 60 | 0 | leaf | 2 |
|
||||
| 5 | `0x80018CB0` | 120 | 0 | leaf | 3 |
|
||||
| 6 | `0x800FB13C` | 128 | 0 | leaf | 2 |
|
||||
| 7 | `0x8009E8D0` | 140 | 0 | leaf | 2 |
|
||||
| 8 | `0x8001D98C` | 436 | 0 | frame | 2 |
|
||||
| 9 | `0x8001084C` | 712 | 0 | frame | 2 |
|
||||
| 10 | `0x80100808` | 4 | 3 | leaf | 1 |
|
||||
|
||||
The nine tier-0 entries are the real duplicate groups from the census; matching those nine bodies
|
||||
registers **22** function addresses. Behind them sit 509 leaf functions whose smallest members are
|
||||
8 bytes (`0x80013C88`, `0x800179CC`, `0x8002F2F8`, `0x80036378`, `0x80038788`, `0x80057DFC`,
|
||||
`0x800FB5D4`, `0x80103FCC`, `0x80103FEC`).
|
||||
|
||||
## Limits
|
||||
|
||||
- A worklist entry is **not** a match. It says where to look, not what is correct; the byte gate
|
||||
remains the only authority.
|
||||
- `exact` is a statement about control flow, not about library-versus-game code, which is still
|
||||
unresolved.
|
||||
- `shape` is read from the body's own instructions. A `leaf` with an indirect `jalr` to an unmapped
|
||||
address still counts as a call; a body whose calls are all in unreachable bytes would be misread.
|
||||
- The exclusion counts do not sum to the total: a duplicate group's non-representative members are
|
||||
dropped without a reason counter, because they are represented by the group's listed entry.
|
||||
|
||||
## Regenerate
|
||||
|
||||
```bash
|
||||
make worklist
|
||||
# or, explicitly:
|
||||
./tools/sf3_triage plan --exe 'extracted/SCUS_946.40;1' \
|
||||
--extents config/function_extents.tsv --inventory config/function_inventory.tsv \
|
||||
--census config/duplicate_bodies.tsv --regions config/regions.tsv \
|
||||
--exclude 0x8005DEF8 --exclude 0x800F3160 \
|
||||
--out config/match_worklist.tsv --force
|
||||
```
|
||||
|
||||
The two excluded addresses are the recorded near-misses deferred by developer direction in Phase 7
|
||||
(`0x8005DEF8`, 5-byte register tie-break; `0x800F3160`, store-in-delay-slot scheduling). Naming them
|
||||
explicitly in the `Makefile` keeps the exclusion visible rather than burying it in the tool.
|
||||
@@ -12,7 +12,7 @@ function bodies, bringing the project past **thirty** distinct byte-identical fu
|
||||
- [x] **P7-T1 — Phase control records, baseline revalidation, and open-item triage** (complete)
|
||||
- [x] **P7-T2 — Evidence-graded function extents** (complete)
|
||||
- [x] **P7-T3 — Duplicate-body census** (complete)
|
||||
- [ ] **P7-T4 — Candidate triage worklist**
|
||||
- [x] **P7-T4 — Candidate triage worklist** (complete)
|
||||
- [ ] **Rules check**
|
||||
- [ ] **P7-T5 — Symbol rows at scale**
|
||||
- [ ] **P7-T6 — Scaled batch A: at least fifteen new bodies**
|
||||
@@ -156,6 +156,55 @@ reproducible from tracked inputs alone. The plan's intent — no ROM content com
|
||||
**Limits:** a group is evidence of a shared body, not that every member is a function start; a shared
|
||||
*tail* is not a duplicate function; and the census inherits the extents' bounds.
|
||||
|
||||
## P7-T4 — Candidate triage worklist (2026-09-23)
|
||||
|
||||
**Delivered:**
|
||||
|
||||
- `tools/sf3_triage plan` — ranks eligible candidates by `(tier, size, address)` and records every
|
||||
exclusion reason; 25 synthetic tests.
|
||||
- `config/match_worklist.tsv` — tracked queue,
|
||||
`rank<TAB>address<TAB>end<TAB>size<TAB>grade<TAB>tier<TAB>shape<TAB>calls<TAB>duplicate<TAB>registers`,
|
||||
plus a comment block with the exclusion counts.
|
||||
- `make worklist`; `docs/PHASE7_TRIAGE.md`.
|
||||
|
||||
**Results:** **1,916 listed** — tier 0 (duplicate representative) 9, tier 1 (`exact` leaf) 509,
|
||||
tier 2 (`exact` non-leaf) 1,394, tier 3 (`fallthrough`) 4. Excluded: 252 degenerate (all-zero) bodies,
|
||||
88 low-confidence grades, 12 already registered, 1 header entry, 2 named (`--exclude`).
|
||||
|
||||
The nine tier-0 entries are the real duplicate groups from P7-T3: matching those nine bodies registers
|
||||
**22** function addresses. Behind them are 509 leaf functions, the smallest 8 bytes.
|
||||
|
||||
**Eligibility rule:** an extent graded `exact`/`fallthrough`; a non-degenerate body; not already
|
||||
registered; not the header entry; not named by `--exclude`. `indirect`, `escape`, `outside`,
|
||||
`runaway`, `contained` and `standalone` are excluded and counted.
|
||||
|
||||
**Verification:**
|
||||
|
||||
| Check | Command | Result |
|
||||
|---|---|---|
|
||||
| Ranked order is deterministic | two `plan` runs | identical; order is `(tier, size, address)` from tracked inputs only |
|
||||
| Every exclusion is counted | `plan` stdout and the file's comment block | counts printed and recorded |
|
||||
| Duplicate group listed once | `awk '$9 != "-"'` | 9 rows, one per code group |
|
||||
| Synthetic suite | `python3 -m unittest discover -s tools/tests` | 25 new tests pass |
|
||||
| Firewall | output columns | addresses, sizes, grades, counts, group labels only |
|
||||
|
||||
**Limits:** a worklist entry is not a match; `exact` says nothing about library-versus-game code;
|
||||
`shape` is read from the body's instructions and can misread unreachable bytes; the exclusion counts do
|
||||
not sum to the total because a duplicate group's non-representative members are dropped without a
|
||||
reason counter.
|
||||
|
||||
## Rules check
|
||||
|
||||
Re-read `AGENTS.md` mandatory behavior after P7-T4. One task at a time; explain before changing code or
|
||||
architecture; never overwrite blind; evidence before assumption with Ghidra and PCSX-Redux as the
|
||||
oracles; a match is byte-for-byte with the full-binary hash green; unmatched code stays fallback;
|
||||
verify from clean state and read exit codes; check duplicates before matching and share a verified body
|
||||
through the documented mechanism; document uncertainty; stop after two distinct unexplained failures.
|
||||
The ROM/repository firewall and the `git clean -x` prohibition stand, as does the phase discipline of
|
||||
one task at a time with a checkpoint before ending a session.
|
||||
|
||||
`Rules check — re-read complete. Continuing with P7-T5.`
|
||||
|
||||
## Notes and limits
|
||||
|
||||
- The four class items are the plan's P7-T2..T5 work; the single-function items are explicitly out of
|
||||
|
||||
Executable
+391
@@ -0,0 +1,391 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Ranked match worklist for the USA executable.
|
||||
|
||||
Phase 6 selected match targets by hand from the boundary inventory. This tool
|
||||
turns the Phase 7 evidence into a single deterministic queue: which candidates
|
||||
to attempt, in what order, and what each one costs.
|
||||
|
||||
## Eligibility
|
||||
|
||||
A candidate is listed only if all of these hold:
|
||||
|
||||
* it carries an extent (`tools/sf3_extents` grade `exact` or `fallthrough`);
|
||||
* its body is not degenerate -- an all-zero body is a zero-filled region the
|
||||
walk ran through, not code (see `docs/PHASE7_DUPES.md`);
|
||||
* it is not already registered in the region registry;
|
||||
* it is not the header entry (CRT startup is not compiler output);
|
||||
* it is not named by `--exclude`.
|
||||
|
||||
Candidates graded `indirect` (jump-table switch), `escape` (control flow leaves
|
||||
the region), `outside`, `runaway`, `contained` or `standalone` are excluded and
|
||||
counted, not listed.
|
||||
|
||||
## Ranking
|
||||
|
||||
Rows are ordered by `(tier, size, address)`, so the order is reproducible from
|
||||
the tracked inputs alone:
|
||||
|
||||
* **tier 0** -- a duplicate group representative with a code body: one match
|
||||
registers several addresses at once (the `registers` column).
|
||||
* **tier 1** -- `exact` and `leaf` (no call instruction in the body): no
|
||||
cross-references, so no symbol rows are needed.
|
||||
* **tier 2** -- `exact` and not a leaf: needs symbol rows for its callees.
|
||||
* **tier 3** -- `fallthrough`: no reachable terminal, so the extent may include
|
||||
alignment padding; the body may still be matchable.
|
||||
|
||||
`shape` is `frame` when the body sets up a stack frame (`addiu sp,sp,-N`, N > 0),
|
||||
`leaf` when it makes no call and has no frame, and `call` otherwise.
|
||||
|
||||
## Output
|
||||
|
||||
A tracked TSV of `rank<TAB>address<TAB>end<TAB>size<TAB>grade<TAB>tier<TAB>shape<TAB>calls<TAB>duplicate<TAB>registers`,
|
||||
followed by a comment block recording the exclusion counts by reason. Addresses,
|
||||
sizes, grades, counts and group labels only -- no instruction bytes.
|
||||
|
||||
Nothing here promotes a candidate to a match: a match still requires an
|
||||
instruction-identical `sf3_match range` comparison and the clean full-binary
|
||||
gate. The worklist says where to look, not what is correct.
|
||||
|
||||
Exit codes: 0 success, 2 usage or environment error.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
from pathlib import Path
|
||||
import struct
|
||||
import sys
|
||||
from typing import Sequence
|
||||
|
||||
|
||||
EXE_MAGIC = b"PS-X EXE"
|
||||
HEADER_SIZE = 0x800
|
||||
PAYLOAD_LMA = 0x800
|
||||
|
||||
ELIGIBLE_GRADES = ("exact", "fallthrough")
|
||||
|
||||
HEADER_LINES = (
|
||||
"# Syphon Filter 3 (USA) match worklist.",
|
||||
"# Columns: rank<TAB>address<TAB>end<TAB>size<TAB>grade<TAB>tier<TAB>shape<TAB>calls"
|
||||
"<TAB>duplicate<TAB>registers.",
|
||||
"# Ordered by (tier, size, address): tier 0 = duplicate group with code (one",
|
||||
"# match, several addresses), 1 = exact leaf, 2 = exact non-leaf, 3 = fallthrough.",
|
||||
"# shape: frame = sets up a stack frame, leaf = no call and no frame, else call.",
|
||||
"# Addresses, sizes, grades, counts and group labels only; no bytes.",
|
||||
"# A worklist entry is not a match: verify it with sf3_match range and make gate.",
|
||||
)
|
||||
|
||||
|
||||
class ToolError(Exception):
|
||||
"""A usage or environment problem; maps to exit code 2."""
|
||||
|
||||
|
||||
def parse_hex(text: str, label: str) -> int:
|
||||
try:
|
||||
return int(text, 16)
|
||||
except ValueError as exc:
|
||||
raise ToolError(f"{label}: not a hex address: {text!r}") from exc
|
||||
|
||||
|
||||
def require_file(path: Path, label: str) -> Path:
|
||||
if not path.is_file():
|
||||
raise ToolError(f"{label} is not a regular file: {path}")
|
||||
return path
|
||||
|
||||
|
||||
def resolve_output(path: Path, force: bool) -> Path:
|
||||
if path.exists() or path.is_symlink():
|
||||
if not force:
|
||||
raise ToolError(f"output already exists (use --force to overwrite): {path}")
|
||||
if not path.is_file() or path.is_symlink():
|
||||
raise ToolError(f"output is not a regular file: {path}")
|
||||
return path
|
||||
|
||||
|
||||
def parse_psx_exe(header: bytes) -> tuple[int, int, int]:
|
||||
if len(header) < HEADER_SIZE:
|
||||
raise ToolError("executable is smaller than a PS-X EXE header")
|
||||
if header[:8] != EXE_MAGIC:
|
||||
raise ToolError("executable does not carry the PS-X EXE magic")
|
||||
entry, _gp, text_address, text_size = struct.unpack_from("<IIII", header, 0x10)
|
||||
if text_size == 0:
|
||||
raise ToolError("PS-X EXE header declares an empty payload")
|
||||
return entry, text_address, text_size
|
||||
|
||||
|
||||
def load_extents(path: Path) -> list[tuple[int, int, str]]:
|
||||
"""Read a generated extents table; keep rows that carry an extent."""
|
||||
rows: list[tuple[int, int, str]] = []
|
||||
for number, raw in enumerate(path.read_text(encoding="utf-8").splitlines(), 1):
|
||||
line = raw.split("#", 1)[0].strip()
|
||||
if not line:
|
||||
continue
|
||||
fields = line.split("\t")
|
||||
if len(fields) != 7:
|
||||
raise ToolError(f"extents line {number}: expected seven fields")
|
||||
if fields[1] == "-":
|
||||
continue
|
||||
address = parse_hex(fields[0], f"extents line {number}")
|
||||
end = parse_hex(fields[1], f"extents line {number}")
|
||||
if end <= address:
|
||||
raise ToolError(f"extents line {number}: end is not after address")
|
||||
rows.append((address, end, fields[5]))
|
||||
if not rows:
|
||||
raise ToolError("extents table contains no rows with an extent")
|
||||
return rows
|
||||
|
||||
|
||||
def load_entry_addresses(path: Path) -> set[int]:
|
||||
"""Addresses graded `entry` in the boundary inventory."""
|
||||
entries: set[int] = set()
|
||||
for number, raw in enumerate(path.read_text(encoding="utf-8").splitlines(), 1):
|
||||
line = raw.split("#", 1)[0].strip()
|
||||
if not line:
|
||||
continue
|
||||
fields = line.split("\t")
|
||||
if len(fields) != 2:
|
||||
raise ToolError(f"inventory line {number}: expected 'address<TAB>grades'")
|
||||
names = {name.strip() for name in fields[1].split(",")}
|
||||
if "entry" in names:
|
||||
entries.add(parse_hex(fields[0].strip(), f"inventory line {number}"))
|
||||
return entries
|
||||
|
||||
|
||||
def load_registered_starts(path: Path) -> set[int]:
|
||||
"""Starts already registered in the region registry."""
|
||||
starts: set[int] = set()
|
||||
for number, raw in enumerate(path.read_text(encoding="utf-8").splitlines(), 1):
|
||||
line = raw.split("#", 1)[0].strip()
|
||||
if not line:
|
||||
continue
|
||||
fields = line.split("\t")
|
||||
if len(fields) < 3:
|
||||
raise ToolError(f"region line {number}: expected at least three fields")
|
||||
starts.add(parse_hex(fields[0].strip(), f"region line {number}"))
|
||||
return starts
|
||||
|
||||
|
||||
def load_duplicate_groups(path: Path) -> dict[int, tuple[str, int]]:
|
||||
"""Map every member address to (group label, member count)."""
|
||||
members: dict[int, tuple[str, int]] = {}
|
||||
for number, raw in enumerate(path.read_text(encoding="utf-8").splitlines(), 1):
|
||||
line = raw.split("#", 1)[0].strip()
|
||||
if not line:
|
||||
continue
|
||||
fields = line.split("\t")
|
||||
if len(fields) != 6:
|
||||
raise ToolError(f"census line {number}: expected six fields")
|
||||
label, _size, count, addresses, _grades, flags = fields
|
||||
if flags == "zero":
|
||||
continue
|
||||
for text in addresses.split(","):
|
||||
members[parse_hex(text.strip(), f"census line {number}")] = (label, int(count))
|
||||
return members
|
||||
|
||||
|
||||
def body_shape(body: bytes) -> tuple[str, int]:
|
||||
"""Classify a body as `frame`, `leaf` or `call`, and count its calls."""
|
||||
calls = 0
|
||||
framed = False
|
||||
for offset in range(0, len(body) - 3, 4):
|
||||
word = struct.unpack_from("<I", body, offset)[0]
|
||||
opcode = word >> 26
|
||||
if opcode == 0x03:
|
||||
calls += 1
|
||||
elif opcode == 0 and (word & 0x3F) == 0x09:
|
||||
calls += 1
|
||||
elif opcode == 0x09:
|
||||
rs = (word >> 21) & 0x1F
|
||||
rt = (word >> 16) & 0x1F
|
||||
immediate = word & 0xFFFF
|
||||
if rs == 29 and rt == 29 and immediate != 0:
|
||||
framed = True
|
||||
if framed:
|
||||
return "frame", calls
|
||||
if calls:
|
||||
return "call", calls
|
||||
return "leaf", calls
|
||||
|
||||
|
||||
class Candidate:
|
||||
"""One ranked worklist entry."""
|
||||
|
||||
__slots__ = ("address", "end", "grade", "tier", "shape", "calls", "duplicate", "registers")
|
||||
|
||||
def __init__(self, address: int, end: int, grade: str, tier: int, shape: str,
|
||||
calls: int, duplicate: str, registers: int) -> None:
|
||||
self.address = address
|
||||
self.end = end
|
||||
self.grade = grade
|
||||
self.tier = tier
|
||||
self.shape = shape
|
||||
self.calls = calls
|
||||
self.duplicate = duplicate
|
||||
self.registers = registers
|
||||
|
||||
@property
|
||||
def size(self) -> int:
|
||||
return self.end - self.address
|
||||
|
||||
@property
|
||||
def sort_key(self) -> tuple[int, int, int]:
|
||||
return (self.tier, self.size, self.address)
|
||||
|
||||
|
||||
def build_worklist(payload: bytes, text_address: int, rows: Sequence[tuple[int, int, str]],
|
||||
duplicates: dict[int, tuple[str, int]], excluded: set[int],
|
||||
registered: set[int], entries: set[int]) -> tuple[list[Candidate], dict[str, int]]:
|
||||
"""Rank eligible candidates and count every exclusion reason."""
|
||||
reasons = {
|
||||
"no_extent": 0,
|
||||
"low_confidence_grade": 0,
|
||||
"degenerate_body": 0,
|
||||
"already_registered": 0,
|
||||
"header_entry": 0,
|
||||
"named_exclusion": 0,
|
||||
}
|
||||
candidates: list[Candidate] = []
|
||||
seen_duplicates: set[str] = set()
|
||||
|
||||
for address, end, grade in rows:
|
||||
size = end - address
|
||||
if grade not in ELIGIBLE_GRADES:
|
||||
reasons["low_confidence_grade"] += 1
|
||||
continue
|
||||
start = address - text_address
|
||||
body = payload[start:start + size]
|
||||
if len(body) != size:
|
||||
raise ToolError(f"extent 0x{address:08X}..0x{end:08X} is outside the payload")
|
||||
if not any(body):
|
||||
reasons["degenerate_body"] += 1
|
||||
continue
|
||||
if address in registered:
|
||||
reasons["already_registered"] += 1
|
||||
continue
|
||||
if address in entries:
|
||||
reasons["header_entry"] += 1
|
||||
continue
|
||||
if address in excluded:
|
||||
reasons["named_exclusion"] += 1
|
||||
continue
|
||||
|
||||
shape, calls = body_shape(body)
|
||||
group = duplicates.get(address)
|
||||
if group is not None:
|
||||
label, count = group
|
||||
if label in seen_duplicates:
|
||||
# Only the lowest-address member of a group is listed.
|
||||
continue
|
||||
seen_duplicates.add(label)
|
||||
tier = 0
|
||||
registers = count
|
||||
duplicate = label
|
||||
else:
|
||||
if grade == "fallthrough":
|
||||
tier = 3
|
||||
elif shape == "leaf":
|
||||
tier = 1
|
||||
else:
|
||||
tier = 2
|
||||
registers = 1
|
||||
duplicate = "-"
|
||||
candidates.append(Candidate(address, end, grade, tier, shape, calls, duplicate, registers))
|
||||
|
||||
candidates.sort(key=lambda candidate: candidate.sort_key)
|
||||
return candidates, reasons
|
||||
|
||||
|
||||
def format_worklist(candidates: Sequence[Candidate], reasons: dict[str, int]) -> str:
|
||||
lines = list(HEADER_LINES)
|
||||
for rank, candidate in enumerate(candidates, 1):
|
||||
lines.append("\t".join((
|
||||
str(rank), f"0x{candidate.address:08X}", f"0x{candidate.end:08X}",
|
||||
str(candidate.size), candidate.grade, str(candidate.tier), candidate.shape,
|
||||
str(candidate.calls), candidate.duplicate, str(candidate.registers),
|
||||
)))
|
||||
lines.append("#")
|
||||
lines.append(f"# listed={len(candidates)}")
|
||||
for reason in sorted(reasons):
|
||||
lines.append(f"# excluded_{reason}={reasons[reason]}")
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
def command_plan(args: argparse.Namespace) -> int:
|
||||
exe_path = require_file(args.exe, "executable")
|
||||
extents_path = require_file(args.extents, "function extents")
|
||||
inventory_path = require_file(args.inventory, "function inventory")
|
||||
census_path = require_file(args.census, "duplicate-body census")
|
||||
regions_path = require_file(args.regions, "region registry")
|
||||
out = resolve_output(args.out, args.force)
|
||||
|
||||
data = exe_path.read_bytes()
|
||||
_entry, text_address, text_size = parse_psx_exe(data)
|
||||
payload = data[PAYLOAD_LMA:PAYLOAD_LMA + text_size]
|
||||
|
||||
rows = load_extents(extents_path)
|
||||
duplicates = load_duplicate_groups(census_path)
|
||||
entries = load_entry_addresses(inventory_path)
|
||||
registered = load_registered_starts(regions_path)
|
||||
excluded = {parse_hex(text, "--exclude") for text in args.exclude}
|
||||
|
||||
candidates, reasons = build_worklist(payload, text_address, rows, duplicates,
|
||||
excluded, registered, entries)
|
||||
if args.limit is not None:
|
||||
if args.limit < 1:
|
||||
raise ToolError("--limit must be at least 1")
|
||||
candidates = candidates[:args.limit]
|
||||
|
||||
out.parent.mkdir(parents=True, exist_ok=True)
|
||||
out.write_text(format_worklist(candidates, reasons), encoding="ascii")
|
||||
|
||||
tiers = {tier: 0 for tier in range(4)}
|
||||
for candidate in candidates:
|
||||
tiers[candidate.tier] = tiers.get(candidate.tier, 0) + 1
|
||||
print(f"listed={len(candidates)}")
|
||||
for tier in sorted(tiers):
|
||||
print(f"tier_{tier}={tiers[tier]}")
|
||||
for reason in sorted(reasons):
|
||||
print(f"excluded_{reason}={reasons[reason]}")
|
||||
print(f"output={out}")
|
||||
return 0
|
||||
|
||||
|
||||
def build_parser() -> argparse.ArgumentParser:
|
||||
parser = argparse.ArgumentParser(
|
||||
description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
|
||||
)
|
||||
subparsers = parser.add_subparsers(dest="command", required=True)
|
||||
|
||||
plan_parser = subparsers.add_parser("plan", help="build the ranked worklist")
|
||||
plan_parser.add_argument("--exe", required=True, type=Path)
|
||||
plan_parser.add_argument("--extents", required=True, type=Path)
|
||||
plan_parser.add_argument("--inventory", required=True, type=Path)
|
||||
plan_parser.add_argument("--census", required=True, type=Path)
|
||||
plan_parser.add_argument("--regions", required=True, type=Path)
|
||||
plan_parser.add_argument("--out", required=True, type=Path)
|
||||
plan_parser.add_argument("--force", action="store_true",
|
||||
help="overwrite an existing output file")
|
||||
plan_parser.add_argument("--exclude", action="append", default=[],
|
||||
help="address to exclude (repeatable)")
|
||||
plan_parser.add_argument("--limit", type=int, default=None,
|
||||
help="keep only the first N entries")
|
||||
plan_parser.set_defaults(handler=command_plan)
|
||||
|
||||
return parser
|
||||
|
||||
|
||||
def main(argv: Sequence[str] | None = None) -> int:
|
||||
parser = build_parser()
|
||||
args = parser.parse_args(argv)
|
||||
try:
|
||||
return args.handler(args)
|
||||
except ToolError as exc:
|
||||
print(f"error: {exc}", file=sys.stderr)
|
||||
return 2
|
||||
except OSError as exc:
|
||||
print(f"error: {exc}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,362 @@
|
||||
"""Synthetic-only tests for the match-worklist triage tool.
|
||||
|
||||
Fixtures are self-authored bytes in temporary directories. They never read the
|
||||
local disc or the extracted game executable.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import importlib.machinery
|
||||
import importlib.util
|
||||
import io
|
||||
from pathlib import Path
|
||||
import struct
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
|
||||
TOOL_PATH = Path(__file__).resolve().parents[1] / "sf3_triage"
|
||||
|
||||
|
||||
def _load_tool() -> object:
|
||||
loader = importlib.machinery.SourceFileLoader("sf3_triage_under_test", str(TOOL_PATH))
|
||||
spec = importlib.util.spec_from_loader(loader.name, loader)
|
||||
if spec is None:
|
||||
raise RuntimeError("could not create an import specification")
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
sys.modules[spec.name] = module
|
||||
loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
sf3_triage = _load_tool()
|
||||
|
||||
PAYLOAD = 0x80010000
|
||||
|
||||
# Payload-relative layout of the fixture.
|
||||
LEAF = 0x00 # duplicate of LEAF_TWIN, exact
|
||||
LEAF_TWIN = 0x08
|
||||
CALL = 0x10 # has a jal, exact
|
||||
FRAME = 0x20 # sets up a frame, exact
|
||||
ZERO = 0x30 # all-zero body, exact
|
||||
FALL = 0x38 # fallthrough grade
|
||||
REGISTERED = 0x44 # already in the region registry
|
||||
ENTRY_FN = 0x4C # graded `entry` in the inventory
|
||||
EXCLUDED = 0x54 # named by --exclude
|
||||
LOW_CONF = 0x5C # escape grade
|
||||
LEAF2 = 0x64 # exact leaf, no twin
|
||||
|
||||
LEAF_BODY = (0x03E00008, 0x00000000) # jr ra; nop
|
||||
|
||||
|
||||
def _words() -> list[int]:
|
||||
words = [0] * (0x70 // 4)
|
||||
|
||||
def put(offset: int, *values: int) -> None:
|
||||
for index, value in enumerate(values):
|
||||
words[offset // 4 + index] = value
|
||||
|
||||
put(LEAF, *LEAF_BODY)
|
||||
put(LEAF_TWIN, *LEAF_BODY)
|
||||
put(CALL, 0x0C000018, 0x00000000, 0x03E00008, 0x00000000) # jal 0x60; nop; jr ra; nop
|
||||
put(FRAME, 0x27BDFFF0, 0xAFBF000C, 0x03E00008, 0x27BD0010) # frame setup and restore
|
||||
put(FALL, 0x00000000, 0x00000000, 0x03E00008) # nop; nop; jr ra
|
||||
put(REGISTERED, 0x03E00008, 0x24020001)
|
||||
put(ENTRY_FN, 0x03E00008, 0x24020002)
|
||||
put(EXCLUDED, 0x03E00008, 0x24020003)
|
||||
put(LOW_CONF, 0x03E00008, 0x24020004)
|
||||
put(LEAF2, 0x03E00008, 0x24020005)
|
||||
return words
|
||||
|
||||
|
||||
def _payload() -> bytes:
|
||||
return b"".join(struct.pack("<I", word) for word in _words())
|
||||
|
||||
|
||||
def _synthetic_exe(payload: bytes, entry: int) -> bytes:
|
||||
header = bytearray(0x800)
|
||||
header[:8] = b"PS-X EXE"
|
||||
struct.pack_into("<IIII", header, 0x10, entry, 0, PAYLOAD, len(payload))
|
||||
return bytes(header) + payload
|
||||
|
||||
|
||||
def _extents_text() -> str:
|
||||
lines = ["# synthetic extents",
|
||||
"# Columns: address<TAB>end<TAB>size<TAB>next<TAB>gap<TAB>grade<TAB>evidence."]
|
||||
|
||||
def row(offset: int, size: int, grade: str) -> str:
|
||||
start = PAYLOAD + offset
|
||||
return f"0x{start:08X}\t0x{start + size:08X}\t{size}\t-\t-\t{grade}\tevidence=1"
|
||||
|
||||
lines.append(row(LEAF, 8, "exact"))
|
||||
lines.append(row(LEAF_TWIN, 8, "exact"))
|
||||
lines.append(row(CALL, 16, "exact"))
|
||||
lines.append(row(FRAME, 16, "exact"))
|
||||
lines.append(row(ZERO, 8, "exact"))
|
||||
lines.append(row(FALL, 12, "fallthrough"))
|
||||
lines.append(row(REGISTERED, 8, "exact"))
|
||||
lines.append(row(ENTRY_FN, 8, "exact"))
|
||||
lines.append(row(EXCLUDED, 8, "exact"))
|
||||
lines.append(row(LOW_CONF, 8, "escape"))
|
||||
lines.append(row(LEAF2, 8, "exact"))
|
||||
lines.append(f"0x{PAYLOAD + 0x6C:08X}\t-\t-\t-\t-\tcontained\tinside=0x80010000")
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
def _inventory_text() -> str:
|
||||
return (f"# synthetic inventory\n"
|
||||
f"0x{PAYLOAD + LEAF:08X}\tjal\n"
|
||||
f"0x{PAYLOAD + ENTRY_FN:08X}\tentry,jal\n")
|
||||
|
||||
|
||||
def _census_text() -> str:
|
||||
return (
|
||||
"# synthetic census\n"
|
||||
f"g0001\t8\t2\t0x{PAYLOAD + LEAF:08X},0x{PAYLOAD + LEAF_TWIN:08X}\texact,exact\t-\n"
|
||||
f"g0002\t8\t2\t0x{PAYLOAD + ZERO:08X},0x{PAYLOAD + 0x34:08X}\texact,exact\tzero\n"
|
||||
)
|
||||
|
||||
|
||||
def _regions_text() -> str:
|
||||
return f"0x{PAYLOAD + REGISTERED:08X}\t0x{PAYLOAD + REGISTERED + 8:08X}\tsrc/f.c\n"
|
||||
|
||||
|
||||
def _rows() -> list[tuple[int, int, str]]:
|
||||
return [
|
||||
(PAYLOAD + LEAF, PAYLOAD + LEAF + 8, "exact"),
|
||||
(PAYLOAD + LEAF_TWIN, PAYLOAD + LEAF_TWIN + 8, "exact"),
|
||||
(PAYLOAD + CALL, PAYLOAD + CALL + 16, "exact"),
|
||||
(PAYLOAD + FRAME, PAYLOAD + FRAME + 16, "exact"),
|
||||
(PAYLOAD + ZERO, PAYLOAD + ZERO + 8, "exact"),
|
||||
(PAYLOAD + FALL, PAYLOAD + FALL + 12, "fallthrough"),
|
||||
(PAYLOAD + REGISTERED, PAYLOAD + REGISTERED + 8, "exact"),
|
||||
(PAYLOAD + ENTRY_FN, PAYLOAD + ENTRY_FN + 8, "exact"),
|
||||
(PAYLOAD + EXCLUDED, PAYLOAD + EXCLUDED + 8, "exact"),
|
||||
(PAYLOAD + LOW_CONF, PAYLOAD + LOW_CONF + 8, "escape"),
|
||||
(PAYLOAD + LEAF2, PAYLOAD + LEAF2 + 8, "exact"),
|
||||
]
|
||||
|
||||
|
||||
def _build(**overrides: object) -> tuple[list[object], dict[str, int]]:
|
||||
options = {
|
||||
"duplicates": {PAYLOAD + LEAF: ("g0001", 2), PAYLOAD + LEAF_TWIN: ("g0001", 2)},
|
||||
"excluded": {PAYLOAD + EXCLUDED},
|
||||
"registered": {PAYLOAD + REGISTERED},
|
||||
"entries": {PAYLOAD + ENTRY_FN},
|
||||
}
|
||||
options.update(overrides)
|
||||
return sf3_triage.build_worklist(
|
||||
_payload(), PAYLOAD, _rows(), options["duplicates"], options["excluded"],
|
||||
options["registered"], options["entries"])
|
||||
|
||||
|
||||
class ParseTests(unittest.TestCase):
|
||||
def test_rejects_missing_magic(self) -> None:
|
||||
with self.assertRaises(sf3_triage.ToolError):
|
||||
sf3_triage.parse_psx_exe(bytes(0x800))
|
||||
|
||||
|
||||
class LoaderTests(unittest.TestCase):
|
||||
def test_extents_skip_rows_without_an_extent(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
path = Path(tmp) / "extents.tsv"
|
||||
path.write_text(_extents_text(), encoding="ascii")
|
||||
self.assertEqual(len(sf3_triage.load_extents(path)), 11)
|
||||
|
||||
def test_inventory_finds_entry_grades(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
path = Path(tmp) / "inventory.tsv"
|
||||
path.write_text(_inventory_text(), encoding="ascii")
|
||||
self.assertEqual(sf3_triage.load_entry_addresses(path), {PAYLOAD + ENTRY_FN})
|
||||
|
||||
def test_regions_yield_starts(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
path = Path(tmp) / "regions.tsv"
|
||||
path.write_text(_regions_text(), encoding="ascii")
|
||||
self.assertEqual(sf3_triage.load_registered_starts(path), {PAYLOAD + REGISTERED})
|
||||
|
||||
def test_zero_flagged_groups_are_ignored(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
path = Path(tmp) / "census.tsv"
|
||||
path.write_text(_census_text(), encoding="ascii")
|
||||
members = sf3_triage.load_duplicate_groups(path)
|
||||
self.assertEqual(members[PAYLOAD + LEAF], ("g0001", 2))
|
||||
self.assertNotIn(PAYLOAD + ZERO, members)
|
||||
|
||||
def test_rejects_a_malformed_census_row(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
path = Path(tmp) / "census.tsv"
|
||||
path.write_text("g0001\t8\t2\t0x80010000\texact\n", encoding="ascii")
|
||||
with self.assertRaises(sf3_triage.ToolError):
|
||||
sf3_triage.load_duplicate_groups(path)
|
||||
|
||||
|
||||
class ShapeTests(unittest.TestCase):
|
||||
def _body(self, offset: int, size: int) -> bytes:
|
||||
return _payload()[offset:offset + size]
|
||||
|
||||
def test_leaf_has_no_calls(self) -> None:
|
||||
self.assertEqual(sf3_triage.body_shape(self._body(LEAF, 8)), ("leaf", 0))
|
||||
|
||||
def test_jal_counts_as_a_call(self) -> None:
|
||||
self.assertEqual(sf3_triage.body_shape(self._body(CALL, 16)), ("call", 1))
|
||||
|
||||
def test_frame_setup_is_detected(self) -> None:
|
||||
self.assertEqual(sf3_triage.body_shape(self._body(FRAME, 16)), ("frame", 0))
|
||||
|
||||
def test_jalr_counts_as_a_call(self) -> None:
|
||||
body = struct.pack("<II", 0x0100F809, 0x00000000)
|
||||
self.assertEqual(sf3_triage.body_shape(body), ("call", 1))
|
||||
|
||||
|
||||
class WorklistTests(unittest.TestCase):
|
||||
def test_excludes_every_documented_reason(self) -> None:
|
||||
candidates, reasons = _build()
|
||||
self.assertEqual(reasons["already_registered"], 1)
|
||||
self.assertEqual(reasons["header_entry"], 1)
|
||||
self.assertEqual(reasons["named_exclusion"], 1)
|
||||
self.assertEqual(reasons["degenerate_body"], 1)
|
||||
self.assertEqual(reasons["low_confidence_grade"], 1)
|
||||
self.assertEqual(reasons["no_extent"], 0)
|
||||
|
||||
def test_ranked_order_is_tier_then_size_then_address(self) -> None:
|
||||
candidates, _ = _build()
|
||||
addresses = [candidate.address for candidate in candidates]
|
||||
self.assertEqual(addresses, [
|
||||
PAYLOAD + LEAF, # tier 0: duplicate representative
|
||||
PAYLOAD + LEAF2, # tier 1: exact leaf
|
||||
PAYLOAD + CALL, # tier 2: exact, has a call
|
||||
PAYLOAD + FRAME, # tier 2: exact, frame
|
||||
PAYLOAD + FALL, # tier 3: fallthrough
|
||||
])
|
||||
|
||||
def test_duplicate_group_is_listed_once_with_its_member_count(self) -> None:
|
||||
candidates, _ = _build()
|
||||
listed = [candidate for candidate in candidates if candidate.duplicate != "-"]
|
||||
self.assertEqual(len(listed), 1)
|
||||
self.assertEqual(listed[0].address, PAYLOAD + LEAF)
|
||||
self.assertEqual(listed[0].registers, 2)
|
||||
self.assertEqual(listed[0].tier, 0)
|
||||
|
||||
def test_registers_is_one_for_a_single_body(self) -> None:
|
||||
candidates, _ = _build()
|
||||
single = [candidate for candidate in candidates if candidate.address == PAYLOAD + LEAF2]
|
||||
self.assertEqual(single[0].registers, 1)
|
||||
|
||||
def test_shapes_and_tiers_are_recorded(self) -> None:
|
||||
candidates, _ = _build()
|
||||
by_address = {candidate.address: candidate for candidate in candidates}
|
||||
self.assertEqual(by_address[PAYLOAD + CALL].shape, "call")
|
||||
self.assertEqual(by_address[PAYLOAD + FRAME].shape, "frame")
|
||||
self.assertEqual(by_address[PAYLOAD + FALL].tier, 3)
|
||||
self.assertEqual(by_address[PAYLOAD + LEAF2].tier, 1)
|
||||
|
||||
def test_an_entry_graded_candidate_is_never_listed(self) -> None:
|
||||
candidates, _ = _build()
|
||||
self.assertNotIn(PAYLOAD + ENTRY_FN, [candidate.address for candidate in candidates])
|
||||
|
||||
def test_deterministic(self) -> None:
|
||||
first, _ = _build()
|
||||
second, _ = _build()
|
||||
self.assertEqual([candidate.address for candidate in first],
|
||||
[candidate.address for candidate in second])
|
||||
|
||||
|
||||
class FormatTests(unittest.TestCase):
|
||||
def test_header_columns_and_exclusion_counts(self) -> None:
|
||||
candidates, reasons = _build()
|
||||
text = sf3_triage.format_worklist(candidates, reasons)
|
||||
body = [line for line in text.splitlines() if not line.startswith("#")]
|
||||
self.assertTrue(body)
|
||||
for line in body:
|
||||
self.assertEqual(len(line.split("\t")), 10)
|
||||
self.assertIn("# listed=5", text)
|
||||
self.assertIn("# excluded_degenerate_body=1", text)
|
||||
|
||||
def test_ranks_are_sequential_from_one(self) -> None:
|
||||
candidates, reasons = _build()
|
||||
text = sf3_triage.format_worklist(candidates, reasons)
|
||||
ranks = [line.split("\t")[0] for line in text.splitlines() if not line.startswith("#")]
|
||||
self.assertEqual(ranks, ["1", "2", "3", "4", "5"])
|
||||
|
||||
def test_no_body_bytes_are_written(self) -> None:
|
||||
candidates, reasons = _build()
|
||||
text = sf3_triage.format_worklist(candidates, reasons)
|
||||
self.assertNotIn("03e00008", text.lower())
|
||||
|
||||
|
||||
class MainTests(unittest.TestCase):
|
||||
def _write_inputs(self, root: Path) -> dict[str, Path]:
|
||||
exe = root / "synthetic.exe"
|
||||
exe.write_bytes(_synthetic_exe(_payload(), PAYLOAD))
|
||||
paths = {"exe": exe}
|
||||
for name, text in (("extents", _extents_text()), ("inventory", _inventory_text()),
|
||||
("census", _census_text()), ("regions", _regions_text())):
|
||||
path = root / f"{name}.tsv"
|
||||
path.write_text(text, encoding="ascii")
|
||||
paths[name] = path
|
||||
return paths
|
||||
|
||||
def _argv(self, paths: dict[str, Path], out: Path, *extra: str) -> list[str]:
|
||||
return ["plan", "--exe", str(paths["exe"]), "--extents", str(paths["extents"]),
|
||||
"--inventory", str(paths["inventory"]), "--census", str(paths["census"]),
|
||||
"--regions", str(paths["regions"]), "--out", str(out),
|
||||
"--exclude", f"0x{PAYLOAD + EXCLUDED:08X}", *extra]
|
||||
|
||||
def test_plan_writes_the_worklist(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
root = Path(tmp)
|
||||
paths = self._write_inputs(root)
|
||||
out = root / "worklist.tsv"
|
||||
stdout = io.StringIO()
|
||||
with contextlib.redirect_stdout(stdout):
|
||||
rc = sf3_triage.main(self._argv(paths, out))
|
||||
self.assertEqual(rc, 0)
|
||||
text = out.read_text(encoding="ascii")
|
||||
self.assertIn(f"1\t0x{PAYLOAD + LEAF:08X}", text)
|
||||
self.assertIn("listed=5", stdout.getvalue())
|
||||
self.assertIn("excluded_named_exclusion=1", stdout.getvalue())
|
||||
|
||||
def test_limit_keeps_the_first_entries(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
root = Path(tmp)
|
||||
paths = self._write_inputs(root)
|
||||
out = root / "worklist.tsv"
|
||||
with contextlib.redirect_stdout(io.StringIO()):
|
||||
rc = sf3_triage.main(self._argv(paths, out, "--limit", "2"))
|
||||
self.assertEqual(rc, 0)
|
||||
body = [line for line in out.read_text(encoding="ascii").splitlines()
|
||||
if not line.startswith("#")]
|
||||
self.assertEqual(len(body), 2)
|
||||
|
||||
def test_rejects_a_bad_limit(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
root = Path(tmp)
|
||||
paths = self._write_inputs(root)
|
||||
rc = sf3_triage.main(self._argv(paths, root / "w.tsv", "--limit", "0"))
|
||||
self.assertEqual(rc, 2)
|
||||
|
||||
def test_refuses_an_existing_output(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
root = Path(tmp)
|
||||
paths = self._write_inputs(root)
|
||||
out = root / "worklist.tsv"
|
||||
out.write_text("", encoding="ascii")
|
||||
self.assertEqual(sf3_triage.main(self._argv(paths, out)), 2)
|
||||
|
||||
def test_force_overwrites(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
root = Path(tmp)
|
||||
paths = self._write_inputs(root)
|
||||
out = root / "worklist.tsv"
|
||||
out.write_text("stale\n", encoding="ascii")
|
||||
with contextlib.redirect_stdout(io.StringIO()):
|
||||
rc = sf3_triage.main(self._argv(paths, out, "--force"))
|
||||
self.assertEqual(rc, 0)
|
||||
self.assertNotIn("stale", out.read_text(encoding="ascii"))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in New Issue
Block a user