# Syphon Filter 3 (USA) — matching decompilation

A **matching decompilation** of *Syphon Filter 3* (PlayStation, USA release, `SCUS-94640`): C source
that, compiled with the game's own late-1990s toolchain, rebuilds the shipped executable **byte for
byte**.

"Matching" here has a narrow, machine-checkable meaning: the compiled output must be
instruction-identical to the original and the whole rebuilt executable must be SHA-1 identical.
Nothing "functionally equivalent" counts, and unmatched C never enters the default build.

> **Status: early but scaled.** Phase 7 is complete pending milestone confirmation. The executable
> rebuilds byte-identically, the original toolchain is identified from byte evidence, and **34 distinct
> functions / 39 regions are matched to C** — every one instruction-identical and covered by the clean
> full-binary gate. Function *ends* are now derived from control flow rather than by hand, duplicate
> bodies are censused, and match targets come from a ranked worklist.

## Where it stands

| Milestone | State |
|---|---|
| Deterministic disc extraction and manifest | done (Phase 1) — two fresh runs, byte-identical manifests |
| First Ghidra import and static oracle | done (Phase 1) |
| All-assembly byte-identical rebuild of the executable | done (Phase 3–4) |
| Address-ordered code-recovery path (entry as real MIPS) | done (Phase 4) |
| Original compiler identified from byte evidence | done (Phase 5) |
| Matching harness and full-binary gate | done (Phase 5) |
| Code/function segmentation | 2,875 candidates graded; extents derived for all of them |
| Duplicate-body census | 65 groups, 10 containing code (all `exact`-graded) |
| Ranked match worklist | 1,916 eligible candidates, every exclusion counted |
| Functions matched to C | **34 distinct bodies / 39 regions** — byte-identical and gated (Ghidra's analyzer reports ~1,721 function candidates) |

The validated target is the USA executable `SCUS_946.40;1`:

| Property | Value |
|---|---|
| Size | 1,886,208 bytes |
| SHA-1 | `e173426c157384ebf1b6caf8c6fea18a85a14af9` |
| Declared payload | `[0x80010000, 0x801DC000)` |
| Entry PC | `0x800FB368` |

The current tracked build is an all-payload **assembly** representation: it reproduces the executable
byte for byte but asserts no code, function, section, or object model. It is a build/comparison
baseline, not a recovery of the original program structure.

## Toolchain (pinned by byte evidence)

The original compiler was re-identified in Phase 6 by comparing candidate compilers against the
SDK's own binaries and the executable:

| Role | Component | Evidence |
|---|---|---|
| Compiler | **`GNU C 2.7.2.SN32.3.7.0002`** — the `CC1PSX.EXE` of PsyQ SDK 4.0 | the real binary executed and compared; the open `gcc-2.7.2-psx` build is instruction-identical to it across all 21 probe files |
| Assembler | **`ASPSX` 2.56** (Sony) | SDK 4.0 banner; `maspsx` is the open emulator |
| Linker / binary tools | GNU `mipsel-none-elf` binutils | Phase 3 local build, recorded in `docs/SETUP.md` |
| Splitter | `splat` (+ `spimdisasm`, `rabbitizer`) | Phase 3 |

Working compiler invocation (input must be **preprocessed**; `cc1` rejects comments and directives):

```
gcc-2.7.2-psx/cc1 -quiet -O2 -G0          # then maspsx, then GNU as
```

The macro address form is this compiler's default. Phase 5 had selected `egcs-2.91.66` (PsyQ 4.5),
which matches simple functions but emits a different framed epilogue; the executable's framed code
identifies PsyQ 4.0. The compiler is **open**: `decompals/old-gcc` release 0.17 publishes
`gcc-2.7.2-psx`, and no proprietary SDK is needed for the matching build. The full derivation is in
[`docs/PHASE6_TOOLCHAIN_CORRECTION.md`](docs/PHASE6_TOOLCHAIN_CORRECTION.md).

## Build it from your own disc

The repository contains **no game data**: no disc image, no executable, no disassembly, no assets, no
memory dumps — only source, configuration, tooling, hashes and documentation. You need your own
MODE2/2352 dump of the USA disc (`SCUS94640`; the local input is 691,530,336 bytes over 294,018
sectors, SHA-1 `4abe30077c2b449ea68239083df7932d47ae0b69`; a CUE is not required for filesystem
extraction).

```bash
# 1. extract the disc deterministically and record a manifest
./tools/sf3_extract extract 'disks/Syphon Filter 3 (USA).bin' extracted

# 2. rebuild the executable from assembly and compare
make clean && make all
cmp -s build/scus_946_40.rebuilt 'extracted/SCUS_946.40;1' && echo byte-identical
sha1sum build/scus_946_40.rebuilt 'extracted/SCUS_946.40;1'
```

The rebuild must print `byte-identical` and both files must hash to
`e173426c157384ebf1b6caf8c6fea18a85a14af9`.

Run the project's synthetic test suite with:

```bash
PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s tools/tests -v
```

The build needs the ignored local tooling recorded in [`docs/SETUP.md`](docs/SETUP.md): pinned
`splat`, a GNU MIPS binutils cross toolchain, the vintage `cc1` candidates, and — only if you want to
verify against the original SDK compiler — the proprietary PsyQ SDK plus the `wibo` Win32 loader.

## What is in the repository

| Path | What |
|---|---|
| `tools/` | The project's own tooling: `sf3_extract` (disc/extraction), `sf3_probe` (structural probe), `sf3_fingerprint_probe`, `sf3_boundaries` (function starts), `sf3_extents` (function ends), `sf3_dupes` (duplicate bodies), `sf3_triage` (match worklist), `sf3_match` (the matching harness), and the synthetic test suite in `tools/tests/` |
| `config/` | The tracked registries and evidence tables: `regions.tsv` (matches), `symbols.tsv` (symbols), `function_inventory.tsv` (graded starts), `function_extents.tsv` (graded extents), `duplicate_bodies.tsv` (duplicate census), `match_worklist.tsv` (ranked queue) |
| `src/` | One C file per matched function body, named by address until a name is earned |
| `docs/` | The format/address records, per-phase investigation and verification records, setup and toolchain provenance, the Phase 5 toolchain fingerprint, the Phase 7 extents/dupes/triage records, and the [matching cookbook](docs/MATCHING_COOKBOOK.md) and [matching conventions](docs/MATCHING_CONVENTIONS.md) |
| `phase-ends/` | The project's governance record: the phase digest, one PhaseEnd per closed phase, each phase plan, and the active `CURRENT_PHASE.md` |
| `Makefile` | The all-assembly rebuild (validate → split → assemble → link → binary) plus the ordered C build, the byte gate, and the `extents`/`extents-verify`/`dupes`/`worklist` regeneration targets |
| `AGENTS.md`, `PROJECT_CONTEXT.md` | The standing rules and the permanent project constitution |

Everything under `disks/`, `extracted/`, `asm/`, `build/`, `ghidra/`, `dumps/`, `assets/`, and the
local tool directories (`tools/splat/`, `tools/maspsx/`, `tools/old-gcc/`, `tools/mipsel-none-elf-binutils/`,
`tools/psyq/`, `tools/wibo/`) is ignored and never committed.

## How it was made

The project runs under a written constitution ([`PROJECT_CONTEXT.md`](PROJECT_CONTEXT.md)) and standing
agent rules ([`AGENTS.md`](AGENTS.md)): one task at a time, evidence before assumptions, a byte-level
match or nothing, an explicit ROM firewall, and a hard stop at every phase boundary. Each phase has an
approved plan, a verification gate, a PhaseEnd record, and a digest entry under
[`phase-ends/`](phase-ends/).

The static oracle is Ghidra with the locally built PSX loader; the runtime oracle is PCSX-Redux; the
match oracle is a clean rebuild plus `cmp` and SHA-1 over the whole binary. Compiler conclusions are
reached by differential fingerprinting against the real executable, never from a version label — the
Phase 5 record documents one such conclusion that was found wrong and corrected.

What has been learned about the compiler is written down where it can be reused:
[`docs/MATCHING_COOKBOOK.md`](docs/MATCHING_COOKBOOK.md) (byte-proven findings, each with its basis
and limit) and [`docs/MATCHING_CONVENTIONS.md`](docs/MATCHING_CONVENTIONS.md) (what counts as a match
and how one is registered).

## Standards, and the no-ROM policy

**Accuracy.** A function is matched only when its compiled instructions are identical to the
original's and the full executable passes `cmp` and the SHA-1 check. Every match claim must be
reproducible from a clean state, and unmatched content stays behind an explicit fallback.

**No game data.** Nothing derived from the game may be committed: no disc image, no extracted files,
no disassembly, no assets, no memory dumps, no Ghidra database, and no proprietary SDK files. Review
`git status` before every commit; `git clean -x`/`-fdx` are forbidden because they can destroy ignored
reverse-engineering data.

## Third-party components

This project uses, but does not redistribute, other people's work. Third-party components keep their
own licenses:

- [`splat`](https://github.com/ethteck/splat), [`spimdisasm`](https://github.com/Decompollaborate/spimdisasm),
  [`rabbitizer`](https://github.com/Decompollaborate/rabbitizer) — the split and the disassembly
- [`maspsx`](https://github.com/mkst/maspsx) (Mark Street) — Sony ASPSX's assembler quirks, reproduced
- [`old-gcc`](https://github.com/decompals/old-gcc) (decompals) — the vintage `cc1` builds (GPL-licensed builds of GCC)
- [`wibo`](https://github.com/decompals/wibo) (decompals) — the minimal Win32 loader used to run the SDK compiler
- [Ghidra](https://github.com/NationalSecurityAgency/ghidra) and
  [`ghidra_psx_ldr`](https://github.com/lab313ru/ghidra_psx_ldr) (lab313ru) — the static oracle and its PSX loader
- [PCSX-Redux](https://github.com/grumpycoders/pcsx-redux) — the runtime oracle
- GNU binutils — the MIPS assembler, linker and `objcopy`
- The Sony PsyQ SDK — proprietary; **never distributed**, only referenced by checksum. The PsyQ 4.0/4.1/4.4/4.5/4.6
  compiler binaries were obtained from [`mkst/esa`](https://github.com/mkst/esa)'s `psyq-binaries` release
  and are used only as a verification reference

The reimplementation of game code that this project will produce carries no license. No license has
yet been chosen for the project's own tooling and documentation.

*Syphon Filter 3 is © its respective rights holders (Sony Computer Entertainment / 989 Studios). This
project is not affiliated with or endorsed by them.*