diff --git a/Makefile b/Makefile index f2c31c3..0be941c 100644 --- a/Makefile +++ b/Makefile @@ -28,7 +28,7 @@ MATCH := tools/sf3_match CODE_OUT := build/code EXPECTED_SHA1 := e173426c157384ebf1b6caf8c6fea18a85a14af9 -.PHONY: all validate split assemble link binary code gate clean +.PHONY: all validate split assemble link binary code gate test check clean all: binary @@ -81,6 +81,13 @@ gate: validate @"$(MATCH)" gate --exe "$(EXE)" --regions "$(REGIONS)" --out "$(CODE_OUT)" \ --expect-sha1 "$(EXPECTED_SHA1)" +# Verification gates. `test` is synthetic-only and needs no game input; +# `check` adds the full-binary byte gate (which does). +test: + @PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s tools/tests + +check: test gate + clean: @if test -e "$(BUILD)"; then \ test -d "$(BUILD)" && test ! -L "$(BUILD)"; \ diff --git a/README.md b/README.md index 7c1d7ac..20dd504 100644 --- a/README.md +++ b/README.md @@ -99,7 +99,7 @@ verify against the original SDK compiler — the proprietary PsyQ SDK plus the ` | Path | What | |---|---| | `tools/` | The project's own tooling: `sf3_extract` (disc/extraction), `sf3_probe` (structural probe), `sf3_fingerprint_probe`, and the synthetic test suite in `tools/tests/` | -| `docs/` | The format/address records, per-phase investigation and verification records, setup and toolchain provenance, and the Phase 5 toolchain fingerprint | +| `docs/` | The format/address records, per-phase investigation and verification records, setup and toolchain provenance, the Phase 5 toolchain fingerprint, and the [matching cookbook](docs/MATCHING_COOKBOOK.md) and [matching conventions](docs/MATCHING_CONVENTIONS.md) | | `phase-ends/` | The project's governance record: the phase digest, one PhaseEnd per closed phase, each phase plan, and the active `CURRENT_PHASE.md` | | `Makefile` | The all-assembly rebuild: validate → split → assemble → link → binary | | `AGENTS.md`, `PROJECT_CONTEXT.md` | The standing rules and the permanent project constitution | @@ -121,6 +121,11 @@ match oracle is a clean rebuild plus `cmp` and SHA-1 over the whole binary. Comp reached by differential fingerprinting against the real executable, never from a version label — the Phase 5 record documents one such conclusion that was found wrong and corrected. +What has been learned about the compiler is written down where it can be reused: +[`docs/MATCHING_COOKBOOK.md`](docs/MATCHING_COOKBOOK.md) (byte-proven findings, each with its basis +and limit) and [`docs/MATCHING_CONVENTIONS.md`](docs/MATCHING_CONVENTIONS.md) (what counts as a match +and how one is registered). + ## Standards, and the no-ROM policy **Accuracy.** A function is matched only when its compiled instructions are identical to the diff --git a/docs/MATCHING_CONVENTIONS.md b/docs/MATCHING_CONVENTIONS.md new file mode 100644 index 0000000..7991872 --- /dev/null +++ b/docs/MATCHING_CONVENTIONS.md @@ -0,0 +1,94 @@ +# Matching Conventions + +How a function becomes a match in this project, and what evidence it must carry. These are the +rules the tooling enforces; they exist so a match claim is never a matter of opinion. + +## What counts as a match + +A function is matched only when **both** hold: + +1. `tools/sf3_match range` reports `differing_bytes=0` for the function's exact instruction range; and +2. `make gate` reports `differing_bytes=0` and the rebuilt executable's SHA-1 equals + `e173426c157384ebf1b6caf8c6fea18a85a14af9`. + +Functional equivalence, a plausible-looking decompilation, or a diff that is "only register +allocation" is **not** a match. If the bytes differ, the function stays fallback. + +## Registering a match + +Add one row to the tracked registry `config/regions.tsv`: + +``` +# startendsource +0x80017AD4 0x80017AE8 src/func_80017AD4.c +``` + +Rules the harness enforces: + +- Regions must be ordered, non-overlapping and inside the declared payload. +- A region's compiled `.text` must be **exactly** the declared length; anything else is + `LENGTH-MISMATCH` and fails. A size disagreement is a failure, never something to widen the region + for. +- The source must exist and be a regular file. + +## Source layout and naming + +- One function per file under `src/`, named `func_XXXXXXXX.c` using the function's address in + uppercase hex, until evidence supports a real name. Names are earned, not guessed. +- Every reconstructed type, field and function name is a **hypothesis**. State that in the file + header, and state the limits, exactly as `src/func_80017AD4.c` does. +- Do not invent semantics that the bytes do not support. Where a struct's other fields are unknown, + pad or leave them unnamed rather than inventing a layout. + +## Unmatched code stays fallback + +Unmatched code remains raw data taken from the original. It does not appear as `NON_MATCHING` C, and +it never enters the default build. A failed candidate is recorded as a bounded negative, not hidden +behind a guard. + +## Duplicate bodies + +A body shared by several addresses is matched **once** and registered several times against the same +source: + +``` +0x80017AD4 0x80017AE8 src/func_80017AD4.c +0x80012DAC 0x80012DC0 src/func_80017AD4.c +``` + +The harness compiles each region object and then **localizes its symbols** +(`objcopy --keep-global-symbol=`), so the same source can be instantiated repeatedly without +a duplicate-symbol clash. This is safe because region objects are placed by the linker script rather +than by symbol name, and cross-references are supplied as absolute `--defsym` values — so symbol +visibility cannot affect the emitted bytes. The behaviour is covered by a synthetic test. + +Before registering any match, check for duplicates: search the payload for the function's exact byte +body. A shared *tail* is not a duplicate function — verify the candidate address is a real function +start before sharing. + +## Symbols + +Cross-references to functions and globals that are not themselves matched are supplied as absolute +assembler definitions (`--defsym NAME=0xADDR`). The harness currently accepts these **globally** for a +whole build; per-region symbols are not yet supported, which is why a symbol-referencing match such as +`0x8002D2A0` is verified but not yet registered. + +## Verification procedure + +```bash +make test # synthetic-only suite; needs no game input +make check # make test + the full-binary byte gate (needs the extracted executable) +``` + +`make gate` prints `c_regions=` and, when the registry is empty, an explicit note that the build +contains no C. A `MATCH` with `c_regions=0` means the data baseline was reproduced, not that anything +was matched. + +## Firewall + +Never commit: the disc image, extracted files, generated disassembly or assembly, build outputs, +expected binaries, the Ghidra database, memory dumps, or proprietary SDK material. All of those live +in ignored paths (`disks/`, `extracted/`, `asm/`, `build/`, `ghidra/`, `dumps/`, `assets/`, +`tools/psyq/`, `tools/splat/`, `tools/maspsx/`, `tools/old-gcc/`, `tools/mipsel-none-elf-binutils/`, +`tools/wibo/`). Review `git status` before every commit. `git clean -x` and `git clean -fdx` are +forbidden. diff --git a/docs/MATCHING_COOKBOOK.md b/docs/MATCHING_COOKBOOK.md new file mode 100644 index 0000000..3962943 --- /dev/null +++ b/docs/MATCHING_COOKBOOK.md @@ -0,0 +1,111 @@ +# Matching Cookbook + +Compiler and assembler behaviour that this project has **proved on this executable**, recorded so it +is not rediscovered. Every entry states its basis and its limit. Nothing here is copied from another +project; each claim was observed here first. + +## The toolchain + +| Role | Identity | Basis | +|---|---|---| +| Compiler | `egcs-2.91.66 19990314` (egcs-1.1.2), target `mips-sony-psx` — PsyQ 4.5 `CC1PSX` | SDK README banner; the real binary executed and compared | +| Open substitute | `decompals/old-gcc` 0.17 `gcc-2.91.66-psx` | Instruction-identical to the real `CC1PSX` across ~990 probe instructions | +| Assembler | `ASPSX` 2.81 (Sony) | SDK banner | +| Working invocation | `cc1 -quiet -O2 -G0 -mno-split-addresses`, then GNU `as -march=r3000 -G0` | Byte-identical ranges; `make gate` | + +## Findings + +### 1. `-mno-split-addresses` is mandatory + +`cc1` has two address-materialisation modes, and only one of them matches. + +- **`-msplit-addresses`** (the real `CC1PSX` default) emits explicit sequences such as + `lui $3,%hi(sym)` / `lw $2,%lo(sym)($3)`. **Does not match.** +- **`-mno-split-addresses`** emits assembler **macros** (`lw $2,sym`, `sw $4,sym`, `la $2,sym`) that + the assembler expands. **Matches.** + +*Basis:* oracle ranges `0x80085B80`, `0x800F3160` — byte-identical only under `-mno-split-addresses`. +*Limit:* proven for symbol loads, symbol stores and `la`; not yet exhaustively for every construct. + +### 2. Symbol load — the macro expands into the destination register + +`lw rt,sym` cannot need a separate scratch, so the assembler expands it as +`lui rt,%hi(sym)` + `lw rt,%lo(sym)(rt)` — the **same** register twice. + +*Basis:* `0x80085B80` (`lui v0,0x8014` / `lw v0,-31652(v0)`) byte-identical. +*Limit:* single observation class; `lhu`/`lbu` assumed to behave the same, unverified. + +### 3. Symbol store — the macro expands through `$at` + +`sw rt,sym` must not clobber its source register, so the assembler expands it through `$at`: +`lui $at,%hi(sym)` + `sw rt,%lo(sym)($at)`. + +*Basis:* `0x800F3160` (`lui at,0x8014` / `jr ra` / `sw a0,11996(at)`) byte-identical. +*Limit:* as above. + +### 4. `la` differs between assemblers — GNU `as` uses `ori`, ASPSX uses `addiu` + +This is the project's current sharpest edge. + +- GNU `as` expands `la rt,sym` with **`ori`** (zero-extended low half). +- The original — assembled by **ASPSX 2.81** — uses **`addiu`** (sign-extended `%lo`). + +So any function that materialises a symbol **address** (as opposed to loading/storing through one) +will mismatch until `maspsx` is wired into the harness. + +*Basis:* the original's clear loop (`lui v0,0x8012` / `addiu v0,v0,8664`) versus GNU `as` output for +the same `la` under the same flags. +*Limit:* `maspsx` is not yet exercised here; the exact `--aspsx-version` is not chosen. + +### 5. A literal address is not a symbol address + +A **literal** address in the C is materialised as `lui`+`ori`; a **symbol** address uses the +macro/`%hi`/`%lo` split, which the assembler turns into `addiu`. A probe written with a literal +address therefore cannot reproduce the original's `addiu` form. + +*Basis:* the two entry clear-loop experiments (literal vs `extern`). +*Limit:* observed for constant-vs-symbol; interaction with `-G` is untested. + +### 6. `cc1` consumes preprocessed input only + +`cc1` is the compiler proper, not the driver. It rejects `/* … */` comments and `#` directives with a +parse error, so a `cpp` stage must run first. The harness always preprocesses. + +*Basis:* direct parse failures on commented sources. +*Limit:* none; this is a hard requirement. + +### 7. `char` is unsigned by default + +The PlayStation target defaults plain `char` to **unsigned**: a `char` load emits `lbu`, and an +`unsigned char` parameter is masked with `andi …,0xff`. + +*Basis:* `v_strlen`-style probe (`lbu`) and `func_80017AD4` (`andi a2,a2,0xff` before `sltu`). +*Limit:* explicit `signed char` is assumed to emit `lb`; the original contains both `lb` and `lbu`, +so per-variable signedness still has to be read from the bytes. + +### 8. Delay-slot filling + +- `jr ra` takes the last independent instruction; a trailing store commonly lands in the slot. +- Branch delay slots take `nop` when no independent instruction is available. + +*Basis:* `0x80017AD4` (store in the `jr ra` slot), `0x80017AE8` (three stores, last in the slot), +the entry's clear loop (`nop` after `bnez`). +*Limit:* scheduling is optimisation-level dependent; only `-O2` is characterised. + +### 9. Magic-division scratch register + +Signed and unsigned division by a non-power-of-two constant is expanded into a multiply by a magic +constant plus `mfhi`. The `mfhi` destination is a scratch register chosen by register pressure — in +this executable overwhelmingly `t0`. + +*Basis:* 74 magic-constant sites censused; `mfhi` histogram `t0` ×25, never `v0` in complex code. +*Limit:* the register choice is context-dependent, so it cannot be used to identify the compiler; it +only matters when reproducing a specific function. + +## Open questions + +- The exact `-G` small-data threshold (the executable clearly uses `gp`-relative data in places). +- Whether `-mgpopt` was passed. +- `maspsx`'s `--aspsx-version` for this SDK (ASPSX 2.81 is reported by the SDK banner). +- One reconstructed function (`0x8005DEF8`) whose constant multiply the original emits as a real + `mult` while every compiler tested synthesises it — the reconstruction is the likely cause. diff --git a/docs/PHASE5_VERIFICATION.md b/docs/PHASE5_VERIFICATION.md new file mode 100644 index 0000000..de0fded --- /dev/null +++ b/docs/PHASE5_VERIFICATION.md @@ -0,0 +1,74 @@ +# Phase 5 Verification Record + +**Scope:** the consolidated Phase 5 outcome and the clean gates for the phase milestone. +**Task:** P5-T6 +**Status:** complete pending developer milestone confirmation. The phase is not closed. + +## Outcome summary + +| Task | Result | +|---|---| +| P5-T1 | Baseline revalidated and toolchain evidence inventoried (`docs/PHASE5_TOOLCHAIN_EVIDENCE.md`). | +| P5-T2 | The `decompals/old-gcc` 0.17 candidate ladder acquired with provenance (`docs/SETUP.md`). | +| P5-T3 | Compiler identified as **`egcs-2.91.66` (egcs-1.1.2), target `mips-sony-psx` — PsyQ 4.5 `CC1PSX`**; the open `gcc-2.91.66-psx` is instruction-identical to the real SDK binary (`docs/PHASE5_TOOLCHAIN_FINGERPRINT.md`). | +| P5-T4 | Tracked matching harness and ordered-layout gate (`docs/PHASE5_MATCH_HARNESS.md`). | +| P5-T5 | The entry determined to be CRT startup, not C; **first byte-identical C match** `func_80017AD4` (`docs/PHASE5_FIRST_MATCH.md`). | +| P5-T6 | Cookbook, conventions, verification record, milestone request (`docs/MATCHING_COOKBOOK.md`, `docs/MATCHING_CONVENTIONS.md`). | + +**The Phase 5 milestone is met:** the original toolchain is identified with byte evidence, and one C +function is matched instruction-for-instruction with a green clean full-binary rebuild. + +## Clean verification gates (last run) + +1. **Synthetic suite** — `PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s tools/tests` — + exit 0; **53 tests pass** (25 new this phase). +2. **Phase 3 baseline** — `make clean` exit 0; `make all` exit 0; + `cmp -s build/scus_946_40.rebuilt 'extracted/SCUS_946.40;1'` exit 0; both files SHA-1 + `e173426c157384ebf1b6caf8c6fea18a85a14af9`. +3. **Ordered gate with one C region** — `make gate`: `c_regions=1`, `differing_bytes=0`, + `result=MATCH`, rebuilt SHA-1 `e173426c157384ebf1b6caf8c6fea18a85a14af9`. +4. **Firewall / Git review** — 57 tracked files; 0 tracked paths under any prohibited root + (`disks/`, `extracted/`, `asm/`, `assets/`, `expected/`, `build/`, `ghidra/`, `dumps/`, + `tools/psyq/`, `tools/splat/`, `tools/maspsx/`, `tools/mipsel-none-elf-binutils/`, + `tools/old-gcc/`, `tools/wibo/`); `git diff --check` exit 0. + +## Match evidence + +`func_80017AD4`, `0x80017AD4..0x80017AE8` (20 bytes), `src/func_80017AD4.c`: + +- `sf3_match range` — 0 differing bytes. +- `make gate` — 0 differing bytes and SHA-1 identical. +- Duplicate review — the body occurs exactly once; one caller at `0x80014C14`. + +Everything else in the executable remains the raw data fallback. The registry contains exactly one +region, and `make gate` reports `c_regions=1` so the single match cannot be confused with the +baseline. + +## Bounded negatives and open items + +Recorded, not hidden: + +- The entry `[0x800FB368, 0x800FB410)` is **not** compiler output and no C is claimed for it. +- `0x8005DEF8` — the original emits a real `mult` for a constant multiply that every compiler tested + (including the real `CC1PSX`) synthesises into shifts. The reconstruction is the likely cause. +- `0x8002D2A0` and `0x80017AE8` are verified byte-identical by `sf3_match range` but not registered + (the first needs symbol support; the second was deliberately left for the next batch). +- GNU `as` expands the `la` macro with `ori` while ASPSX uses `addiu`, so `la`-using functions need + `maspsx` (cookbook finding 4). +- The `-G` small-data threshold and `-mgpopt` remain undetermined. + +## Rules added this phase + +None. No rule in `AGENTS.md` was added, changed or removed. + +## Requested action + +The developer is asked to confirm the Phase 5 milestone: + +- the original compiler is identified with byte evidence as `egcs-2.91.66` (PSX) / PsyQ 4.5 + `CC1PSX`, with the open `gcc-2.91.66-psx` verified instruction-identical; and +- one C function (`func_80017AD4`) is matched instruction-for-instruction, with the clean full-binary + rebuild passing exact `cmp` and SHA-1. + +On confirmation, a PhaseEnd record follows, `phase-ends/DIGEST.md` is updated, `CURRENT_PHASE.md` is +archived to `phase-ends/logs/Phase5.md`, and the phase closes. Do not begin Phase 6 in this session. diff --git a/phase-ends/CURRENT_PHASE.md b/phase-ends/CURRENT_PHASE.md index b1f19b4..696b074 100644 --- a/phase-ends/CURRENT_PHASE.md +++ b/phase-ends/CURRENT_PHASE.md @@ -2,8 +2,8 @@ **Phase:** Phase 5 — Original Toolchain Identification and First C Match **Plan:** `phase-ends/Phase5_PLAN.md` -**Status:** active; P5-T5 complete — the entry is CRT startup (not C) and the first C match is byte-identical -**Active task:** P5-T5 complete; next is P5-T6 (cookbook, registry conventions, phase gate) +**Status:** active; all six tasks complete — awaiting developer milestone confirmation +**Active task:** P5-T6 complete; phase is not closed until the milestone is confirmed > Approval note: `Phase5_PLAN.md` was authored as a DRAFT requiring explicit developer > approval. The developer's instruction to begin Phase 5 is being treated as that approval @@ -69,10 +69,31 @@ The Phase 4 ordered workflow is now tracked tooling rather than an ignored exper - Six codegen findings were recorded, including the key one: GNU `as` expands the `la` macro with `ori` while the original assembler (ASPSX 2.81) uses `addiu`, so `la`-using functions need `maspsx`. -## Next task +## P5-T6 completed evidence -P5-T6 — matching cookbook, registry/verification conventions, and the phase gate. Known gaps to -close: symbol support in the region registry, `maspsx` integration, and a per-region flag override. +- `docs/MATCHING_COOKBOOK.md` — nine byte-proven compiler/assembler findings, each with its basis and + limit, plus the open questions. +- `docs/MATCHING_CONVENTIONS.md` — what counts as a match, registry format, source naming, the + duplicate-sharing mechanism, symbol handling, the verification procedure and the firewall. +- `docs/PHASE5_VERIFICATION.md` — the consolidated outcome, the clean gates, the bounded negatives, + and the milestone request. +- Tooling: `make test` and `make check` targets added; region objects are now symbol-localized, which + makes the documented duplicate-sharing mechanism (N registry rows -> 1 source) work and is covered + by a synthetic test (53 tests total). + +## Clean gates (last run) + +Synthetic suite 53/53 pass; `make clean`, `make all`, `cmp` and SHA-1 green; `make gate` with one C +region gives `c_regions=1`, 0 differing bytes, SHA-1 `e173426c157384ebf1b6caf8c6fea18a85a14af9`; 57 +tracked files and 0 under any prohibited root. + +## Awaiting developer milestone confirmation + +The Phase 5 milestone is ready: the original compiler is identified with byte evidence as +`egcs-2.91.66` (PSX) / PsyQ 4.5 `CC1PSX` (open substitute verified instruction-identical), and one C +function is matched with a green clean full-binary rebuild. On confirmation: write the PhaseEnd, +update `phase-ends/DIGEST.md`, archive this file to `phase-ends/logs/Phase5.md`, commit, and stop +without beginning Phase 6. ## P5-T2 completed evidence diff --git a/phase-ends/Phase5_PLAN.md b/phase-ends/Phase5_PLAN.md index c3667c1..e18afb1 100644 --- a/phase-ends/Phase5_PLAN.md +++ b/phase-ends/Phase5_PLAN.md @@ -64,7 +64,7 @@ Identify and verify the original Syphon Filter 3 (USA) build toolchain — compi - Iterate only on comparison-supported root causes; stop after two distinct unexplained failures. - **Verify:** the direct instruction-identical comparison AND the clean full-binary `cmp` + SHA-1 check both pass before C is enabled by default. On failure, keep fallback, record the bounded result, and do not claim a match. -- [ ] **P5-T6 — Cookbook, registry, verification record, and phase gate** *(xHigh)* +- [x] **P5-T6 — Cookbook, registry, verification record, and phase gate** *(xHigh)* — complete; `docs/MATCHING_COOKBOOK.md`, `docs/MATCHING_CONVENTIONS.md`, `docs/PHASE5_VERIFICATION.md`; awaiting developer milestone confirmation. - If P5-T5 succeeds: record reusable compiler/codegen findings in a matching cookbook, add ROM-safe registration and verification conventions for matched functions, and share verified duplicate bodies through the documented mechanism. - Otherwise: document the evidence-backed blocker. - Complete the clean test, baseline/full-binary comparison, firewall, and Git-review gates, then request milestone confirmation. @@ -72,4 +72,4 @@ Identify and verify the original Syphon Filter 3 (USA) build toolchain — compi ## Milestone -The original build toolchain is identified with byte evidence and used to produce at least one instruction-identical C function match whose clean full-binary rebuild passes exact `cmp` and SHA-1, **or** Phase 5 records the evidence-backed blocker that prevents a reproducible toolchain and leaves all content as assembly/data fallback. Phase 5 closes only after the developer confirms the milestone; write a PhaseEnd record, update `phase-ends/DIGEST.md`, archive `CURRENT_PHASE.md` to `phase-ends/logs/Phase5.md`, commit, and stop without beginning Phase 6. +The original build toolchain is identified with byte evidence and used to produce at least one instruction-identical C function match whose clean full-binary rebuild passes exact `cmp` and SHA-1, **or** Phase 5 records the evidence-backed blocker that prevents a reproducible toolchain and leaves all content as assembly/data fallback. **Met:** compiler identified as `egcs-2.91.66` (PSX) / PsyQ 4.5 `CC1PSX`; first C match `func_80017AD4` gated byte-identical. Phase 5 closes only after the developer confirms the milestone; write a PhaseEnd record, update `phase-ends/DIGEST.md`, archive `CURRENT_PHASE.md` to `phase-ends/logs/Phase5.md`, commit, and stop without beginning Phase 6. diff --git a/tools/sf3_match b/tools/sf3_match index 832dd00..fcf547c 100755 --- a/tools/sf3_match +++ b/tools/sf3_match @@ -317,6 +317,22 @@ def extract_text_bytes(object_path: Path, destination: Path, tools: Toolchain) - return destination.read_bytes() +# A symbol that never exists: `--keep-global-symbol` then localizes everything else. +LOCALIZE_SENTINEL = "__sf3_keep_no_global_symbol" + + +def localize_symbols(object_path: Path, tools: Toolchain) -> None: + """Make a region object's symbols local. + + Region objects are placed by the linker script, not by symbol name, and any + cross-reference is supplied as an absolute `--defsym`. Localizing therefore + changes nothing about the emitted bytes, and it lets one shared source file + be instantiated for several regions without a duplicate-symbol clash. + """ + run([str(tools.objcopy), f"--keep-global-symbol={LOCALIZE_SENTINEL}", + str(object_path)]) + + # -------------------------------------------------------------------------- # subcommands # -------------------------------------------------------------------------- @@ -411,6 +427,7 @@ def _build(args: argparse.Namespace) -> tuple[Path, Path]: assemble_asm(item.source, object_path, tools) else: compile_c(item.source, object_path, work, tools) + localize_symbols(object_path, tools) objects.append(item.object_name) script = out / "link.ld" diff --git a/tools/tests/test_sf3_match.py b/tools/tests/test_sf3_match.py index de7afaf..fa8ebf3 100644 --- a/tools/tests/test_sf3_match.py +++ b/tools/tests/test_sf3_match.py @@ -253,6 +253,28 @@ class EndToEndTests(unittest.TestCase): "--out", str(root / "out")]) self.assertEqual(rc, 1) + def test_two_regions_can_share_one_source(self) -> None: + """The registry's documented duplicate-sharing mechanism: N rows -> 1 source.""" + with tempfile.TemporaryDirectory() as tmp: + root = Path(tmp) + exe, code_length = self._build_synthetic(root, corrupt=False) + # Place a second, identical copy of the body later in the payload. + payload = bytearray(exe.read_bytes()) + body = payload[0x800:0x800 + code_length] + payload[0x800 + 0x20:0x800 + 0x20 + code_length] = body + exe.write_bytes(bytes(payload)) + regions = root / "regions.tsv" + regions.write_text( + f"0x{PAYLOAD:X} 0x{PAYLOAD + code_length:X} {root / 'synthetic.c'}\n" + f"0x{PAYLOAD + 0x20:X} 0x{PAYLOAD + 0x20 + code_length:X} {root / 'synthetic.c'}\n", + encoding="ascii", + ) + out = root / "out" + rc = sf3_match.main(["gate", "--exe", str(exe), "--regions", str(regions), + "--out", str(out)]) + self.assertEqual(rc, 0) + self.assertEqual((out / "scus_946_40.rebuilt").read_bytes(), exe.read_bytes()) + def test_refuses_an_existing_destination(self) -> None: with tempfile.TemporaryDirectory() as tmp: root = Path(tmp)