BFM-decomp/docs/wiki/The-ROM-firewall.md

# The ROM firewall

The repository ships no byte of the game. Symbol names, addresses, hashes, configuration and the decompiled C are what a
decompilation publishes (the sotn-decomp precedent); everything derived from the disc stays on the maintainer's disk
and is regenerated by anyone who owns the game. This page is the policy in full: the rule and the one exemption never
to grant, the nine classes of content the rule covers with the tracked form each one takes instead, a copyable
`.gitignore`, the three layers that enforce it, what to do if bytes got in anyway, and the one residue this project
records.

## The rule, and the exemption never to grant

**Nothing derived from the game enters git — from the first commit, private or not.** This project's constitution
carried the rule (H1) from day one, then relaxed it while the repository was private ("ROM-derived content MAY be
committed; accidental inclusion is not a violation; compliance is a scrub before going public", rule R1, 2026-06-10).
The relaxation bought a few weeks of convenience and cost, three months later, a full-history rewrite of 4,031 commits:
two sessions of tooling, two rehearsals, an archive repository, a force-push, a GitHub Support ticket and a daily probe
of the host's object store. Rule R74 now says it plainly — no ROM-derived bytes in *any* published artifact, test
fixtures, JSON, badges, reports, wiki pages and tracked scratch included — and the day-one lesson for any other
project is the retrospective's first item: keep the bytes out from the first commit, even while private, and cite
commits by date and subject in long-lived documents, never by hash alone, because a rewrite changes every hash.

## What counts as ROM-derived — nine classes, and the form each takes instead

"ROM-derived" is wider than the executable. Every class below ended up in this repository's history while the rule was
relaxed and had to be purged; the right-hand column is what a public tree carries instead.

| # | Class | In this project | What is tracked instead |
|---|---|---|---|
| 1 | The executable(s) | `SLUS_007.26`, under two historical paths — a rename is why the audit checks content hashes, not names | its SHA1 in the per-binary contract; `make disc-extract` regenerates it from your disc |
| 2 | Extracted payloads | `extracted/**`, 1,801 files | a manifest of hashes (`extracted/retail/manifest.jsonl`), which doubles as the oracle your own extraction is verified against |
| 3 | Generated disassembly, assets, build output — **including any listing of the target's instructions in a text file** | `asm/`, `assets/`, `build/`, `expected/`; a notes file that pastes a function's disassembly | regenerated by `make extract` / `make build`; the durable record is the symbol file |
| 4 | Memory images | `dumps/*.bin`, 28 × 2 MB | `dumps/INDEX.md` (what each state is) + `dumps/CHECKSUMS.sha1` (its identity); a re-capture is a new snapshot |
| 5 | The reverse-engineering database | `ghidra/` — a Ghidra project embeds the program's bytes under a page XOR mask | a text export of the hand-authored annotations (`config/ghidra/*.jsonl`) and a rebuild script that *proves* the round trip — [Ghidra rebuild from text](Ghidra-rebuild-from-text.md) |
| 6 | The vendor SDK | `tools/psyq/` (Sony's PsyQ libraries and objects) | checksums of the archives a user must supply + a fetch script; `make sdk-dual` proves the build is byte-identical without it |
| 7 | Session transcripts that quote disassembly | the agent-session archive (~260,000 lines of the target's `.s` inside) | nothing public; a private archive repository for the owner's recovery only |
| 8 | Training pairs and weights derived from the target | the drafter model's dataset | the pipeline write-up is published; the data and weights are not |
| 9 | Re-downloadable third-party binaries | extension zips, a compiled GPL tool | a sha256 in the ops reference and a download step |

Class 3 is the subtle one: the disassembly of a function whose C is already in the tree adds nothing a build does not
reproduce, and it is still class 3 — the rule is about what the repository *distributes*, not about what it *reveals*.

## The `.gitignore` template

The block below is the firewall as a template: generic paths, the contents-exclude idiom for scratch, one comment per
class. It is also, byte for byte, the file the day-one kit stamps into a new repository (`tools-health` asserts the two
copies are identical). Adjust the path names to your layout; do not delete a class.

```gitignore
# ============================================================
# ROM firewall — in force from the FIRST commit; no private-repository exemption, ever.
# The repository ships NO game-derived bytes and NO vendor SDK. Symbol names, addresses,
# hashes, configuration and the decompiled C are what a decompilation publishes.
# ============================================================

# 1. The disc / ROM dump — never committed; your own copy goes here
/disks/

# 2. Extracted payloads — regenerated from YOUR dump by the extractor and verified against the
#    committed manifest of hashes, the only thing tracked under extracted/
/extracted/*
!/extracted/retail/
/extracted/retail/*
!/extracted/retail/manifest.jsonl
!/extracted/retail/manifest.sha1

# 3. Generated disassembly, assets and build output — `make extract` / `make build` recreate them;
#    the durable record is the symbol file. (A pasted listing of target instructions is class 3 too.)
/asm/
/assets/
/build/
/expected/
/undefined_syms_auto.txt
/undefined_funcs_auto.txt

# 4. Memory images — local only; INDEX.md + CHECKSUMS.sha1 stay tracked and describe them
/dumps/*.bin

# 5. The reverse-engineering database — embeds the program's bytes; tracked as a TEXT export
#    (config/ghidra/) with a rebuild script that proves the round trip. NEVER `git clean -x` here.
/ghidra/

# 6. The vendor SDK — user-supplied, optional, never distributed; its checksums are tracked
/tools/psyq/

# 7. Session transcripts — they quote the target's disassembly; keep them out of any public tree
#    (a governance system that keeps the agent's transcripts inside the repository writes them under
#    .claude-state/transcripts/ — ignored here; the memory files beside them stay tracked)
/session-archive/
/.claude-state/transcripts/

# 8. Training data and weights derived from the target
/datasets/
/models/
*.gguf
*.safetensors

# 9. Re-downloadable binaries — a sha256 in the ops reference and a fetch step, not a commit.
#    (A vendored compiler tarball may be tracked only if its license allows; keep its checksum beside it.)
/tools/bin/*
!/tools/bin/*.sha256
/tools/ghidra-ext/*.zip
*.tar.gz

# ---- Scratch: ignored by CONTENTS (/.run/* not /.run/) so that dated `!` exceptions can re-include ----
# ---- irreplaceable work. Rule of thumb: commit what a rerun CANNOT reproduce (hand analysis,      ----
# ---- harnesses, ledgers, the recorded contract run); leave what a script regenerates ignored.    ----
/.run/*
!/.run/README.md
# One dated block per exception — re-include the directory, re-exclude its contents, re-include
# the wanted files — under a comment naming the phase, the session and the rule. For example:
#   # P<N> <task> (<session>, <date>): the recorded contract run's per-step logs. Evidence, tracked.
#   !/.run/P<N>/
#   /.run/P<N>/*
#   !/.run/P<N>/verify/
#   /.run/P<N>/verify/*
#   !/.run/P<N>/verify/*.log

# ---- Toolchain caches, research clones and editor noise ----
/.venv/
__pycache__/
/tools/reference/
.vscode/
Thumbs.db
```

## Enforcement — three layers, one list

1. **The ignore firewall** above. It stops the ordinary case: a `git add` of a working directory.
2. **The audit** — `tools/audit_public.py`, the first-push gate reused as CI. It refuses any tracked file under a purged
   path; any tracked file whose SHA1 appears in the ROM-hash set, which it *derives* from the extraction manifest, every
   per-binary contract and the disc's own track hash rather than from a typed list (rule R33); anything over GitHub's
   100 MiB threshold (GitHub's hard limit; raised from its 50 MiB warning on 2026-09-12); and, since Phase 33.5, any tracked text file with a long contiguous run of disassembly-shaped
   lines (an assembler listing, an objdump, a splat `/* ADDR HEX */` block) — the class-3 case the path and hash checks
   cannot see. It asserts its own coverage (a missing manifest, zero contracts or a short manifest is a failure, rule
   R32), refuses rules it cannot interpret rather than mishandling them (R43), prints every count with its denominator
   (R41), and has a **negative control**: an expected-offenders fixture the scan must fail on before it is trusted to
   pass (R39). One subtlety it had to learn: two zero-length disc payloads share the empty file's SHA1, so empty files
   are exempt by content, not by name.
3. **CI** — `.github/workflows/no-rom.yml` runs the audit on every push and pull request and weekly, then compiles every
   eligible translation unit with the pinned toolchain. CI proves everything that does not need the disc; the
   byte-identity of the 218 binaries is verified locally with the disc and recorded — [Verification and
   progress](Verification-and-progress.md).

**One list, in one file.** The purge set (`tools/public_rewrite/purge_set.txt`) is read by the audit *and* by the
history rewrite, so the two can never disagree about what is forbidden. Anything untracked *after* the rewrite without
a second rewrite is listed in a sibling file the audit also reads, `tools/public_rewrite/untracked_after_rewrite.txt`,
so that the purge set stays exactly the set the rewrite removed and the history gate that checks it stays green.

## If bytes got in anyway

The rewrite procedure is `docs/public-flip-runbook.md` and the transferable version is
[How to AI-decomp, chapter 11](../how-to-ai-decomp/11-publishing.md). The lessons it cost, now rules:

- Rehearse every irreversible repository operation on a scratch copy and prove it pair by pair with **positive**
  assertions (R76) — the first rehearsal caught a defect a single run would have shipped.
- A linked worktree's HEAD is a ref; audit `git worktree list` before any gc or purge (R77) — twelve stale worktrees
  pinned 3,729 old commits through a gc.
- A probe or guard must never write into the repository it guards (R81) — a "read-only" `git fetch <old-sha>` pulled
  the purged history back into the working repository, twice.
- A rewritten history is not private until the objects are gone from the **host** (R82): GitHub's Activity view lists
  every pre-force-push tip, and each still served the purged files. Gate the flip on a probe of the old hashes, or
  recreate the repository; never on a clean tree.
- Afterwards the purged paths are ignored-but-present on disk: never `git clean -x` (R78).

## The one recorded residue

Two notes files from a Phase-25 crack — the giant-crack reconnaissance directory's `mine_full.txt` and
`target_full.txt`, both named in `tools/public_rewrite/untracked_after_rewrite.txt` — held the disassembly of one
matched function (`func_80133CD4`, 398 and 399 lines) — class 3, re-included by a `*.txt` allow-list line and invisible
to a path-and-hash audit. They were found at the Phase-33.5 audit, after the rewrite had been
force-pushed. The owner's decision (2026-09-07): untrack them and list them in `untracked_after_rewrite.txt`; add the
content check so the class is caught from now on; do **not** rewrite history a second time for eight kilobytes that the
tracked C and the pinned compiler already reproduce, at the cost of another set of old tips for the host to purge. The
residue therefore exists in the published history and is named here.

## For a new project

Everything on this page is stamped on day one by the decomp kit this project produced: the template above, an audit
with its sources declared in one config file and a planted-fixture negative control, the CI workflow, extraction that
verifies against a committed manifest, and the text-export policy for the reverse-engineering database. The one
decision the kit cannot make for you is the one this page opens with.