BFM-decomp/docs/wiki/Docs-and-scratch-conventions.md

Docs and scratch conventions

This wiki is the source of truth for the project's documentation. docs/ holds the records and references the wiki summarises and links; .run/ holds scratch. This page states where each kind of knowledge goes, which files are generated rather than written, how links are checked, what the archive is for, and what may be tracked under .run/. It was written at the Phase-33.5 consolidation (September 2026), when the tree was reorganised around these rules.

docs/ — what goes where

Knowledge has a home by kind, not by the session that produced it. A note written for one session is scratch; the durable result it produced goes into one of these files, and the note is archived.

Kind of knowledge	Where it goes	The rule behind it
A compiler idiom proven on the bytes (the residual, the mechanism, the lever, the byte proof)	a numbered section of docs/matching-cookbook.md, and its symptom index docs/cookbook-index.md (regenerated, never edited)	R16 — consult the knowledge base before a match, feed the lesson back after
A strategic pivot: what was believed, what failed, the measurement, the hindsight	an entry in docs/decision-log.md, written while fresh	R31
The post-100 % story (Gen3): the narrative of the readability phases	docs/story.md §10 and docs/retrospective.md §7, advanced at the END OF EVERY SESSION from that session's decision-log entry; tools/timeline.py draws the lever and readability series as the chart's lower panel	the owner's rule, 2026-09-12
A late discovery that would have sped up an earlier phase — what it is, when it was found, when it could have been found, what it would have saved	docs/accelerators.md	the ledger a future project starts from
A procedure people run	a runbook — docs/wave-playbook.md for the matching campaign, docs/public-flip-runbook.md for the publication; a superseded runbook carries a banner at the top and is then archived	R21 for the ops reference
An environment or tool fact (a version, a flag, a hook, a row per tool)	docs/SETUP.md, in the same change as the tool	R21
An address, with its source and its verification status	docs/memory-map.md	G5
A file or container format	docs/formats.md	—
The state of the open phase; then the phase's synthesis; then the one-page digest every session starts from	phase-ends/CURRENT_PHASE.md → phase-ends/PhaseEnd_Phase<N>.md → phase-ends/DIGEST.md	R64
The transferable method (any console, any compiler)	the How to AI-decomp chapters	—
How this project is used: building, verifying, contributing, its layout and conventions	this wiki	—
A design note, a frontier analysis, a triage ladder, a worklist for one session	.run/<session>/ while live (see below); once its result is in the record above, the note moves to the archive — it is never a reference	see The archive

Two consequences. A reader looking for "how does X work" starts at the wiki page for X; the wiki links the record when the record is the answer (the cookbook section, the decision-log entry), never a session note. And a document that describes a plan or a state carries the date and the phase it was true for, so a later reader can tell a record from a current instruction.

Authored versus generated

Numbers are generated, never typed (rule R75). Every progress figure, badge, timeline row and census in a published document is produced by a tool from the tree, and the tool asserts the published copy is fresh. A generated file says so on its first line; edits go to the generator, not the file.

Generated file	Generator	Regenerated by
docs/progress.md, docs/progress.fleet.md, docs/progress.json, the README's progress block, docs/badges/*.json	tools/progress.py	make report BINARY=main; make audit-digest asserts freshness
docs/difficulty.md, docs/duplicates.md, docs/duplicates.cross.md, docs/backlog.md	tools/difficulty.py, tools/dup_report.py, tools/backlog.py	make report BINARY=main
docs/cookbook-index.md	tools/cookbook_index.py	make tools-health (--check)
docs/disc-ledger.md	tools/disc_audit.py	make audit-disc
docs/story-timeline.md, docs/story-timeline.svg	tools/timeline.py	make report BINARY=main; --check in make audit-digest
docs/commit-map.tsv	tools/public_rewrite/build_commit_map.py	one-shot, at the history rewrite

A number that has to appear in prose — a census figure in a standards document, say — is written as a dated snapshot with the command that produced it, so the next reader re-derives it rather than trusting it. The register-pin count in the Gen3 documents is the worked example: two documents once carried two figures (44,243 and 43,857) from two different greps; the reconciled figure carries its command and its date.

Snapshots of a state that no longer exists are frozen, not regenerated: the frontier atlas and family surveys were generated while functions remained unmatched; run at the closed frontier their generators write empty documents, so the last populated copies are kept in the archive and the generators are not re-run.

Links, and how they are checked

• A document links the wiki page for a topic, as a relative link to docs/wiki/<Page>.md, not the docs/ file behind it. The README is the one exception (it links a few records directly). The wiki renderer maps those relative links to wiki page names at publication; on GitHub's file view they work as ordinary links.
• A wiki page links into docs/ only through the Reference index, which lists every live reference and generated file with what it is and how to read it. A page that needs a record links the index's row's target; the link checker derives its allow-list from the index page itself.
• Nothing links into an archived document. The Archive index names each archived file as a backticked path with the version it was archived at, so that removing the file breaks no link — which is what happened at Phase 34.
• tools/doc_links.py checks every relative link in the governing documents, every wiki page and every how-to chapter; it runs in make tools-health. A link to a page a later task of the same phase will create is registered in docs/doc_links_pending.txt (rule R80: a missing promised page is PENDING, not BROKEN); the strict mode used at a phase close refuses any pending entry. The renderer (tools/wiki_render.py) refuses to publish a dead link at all.
• A backticked path such as `.run/P32/t4e/NOTES.md` is a citation, not a link. The checker classifies each such citation of a docs/ or .run/ path as TRACKED or UNTRACKED by asking git, never the disk (so a fresh clone gets the same verdict), and refuses an UNTRACKED citation only in a wiki page or a how-to chapter — a reader of the wiki must be able to follow every path named there. In the cookbook, the decision log and the phase records, a citation of an untracked .run/ path is a breadcrumb into the maintainer's private tree, and is allowed as such.

The archive and the Archive index

A document leaves docs/ when its purpose is fulfilled and its information lives elsewhere: a closed-phase plan whose outcome is in the PhaseEnd, a design note whose tool shipped and whose findings are cookbook sections, a snapshot of a frontier that is now empty. At the Phase-33.5 consolidation such files were moved with git mv into an archive folder (history intact) for the owner's review; at Phase 34 (2026-09-08) the owner removed that folder from the tree — the last tracked versions are in the history at v1.32.1. From here on a retired document gets its row in the Archive index — what it was, what came of it, and where its information lives now (a wiki page, a PhaseEnd, a cookbook section) — and is deleted from the tree in the same commit; history keeps it. Before any retirement, every referrer is re-pointed — a command over the tree (git grep -F <basename> over Markdown, Python, shell and the Makefile, excluding the phase records) must return only the index page.

The historical folder docs/history/ (the original project brief, the methodology version the constitution was generated from, an early experiment) is the same idea for the project's beginnings; its README says nothing there is current instruction.

.run/ — scratch, with dated exceptions

.run/ is the project-local replacement for /tmp (rule R12: no project data outside the repository, ever). Build and extract logs, signature dumps, permuter and compile scratch, agent work directories, per-session probes: everything a rerun can reproduce lives here and is never committed. By the Phase-33 close it held about 29 GB in some 455,000 top-level entries — none tracked, all regenerable. make clean never touches it; pruning it is a hand decision.

It is ignored by contents, not as a directory. The .gitignore rule is /.run/* rather than /.run/, because git will not look inside an excluded directory and no ! re-include could then work. That form lets the irreplaceable part be tracked by exception. Each exception is a three-line idiom under a dated comment that names the phase, the session and the rule that justified it:

# P33 A5 (S87, 2026-09-06): THE recorded contract run — tools/verify_contract.sh's per-step logs, the two link
# maps of the with/without-SDK dual and SUMMARY.md (quoted by docs/verification.md). Evidence, tracked.
!/.run/P33/
/.run/P33/*
!/.run/P33/verify/
/.run/P33/verify/*
!/.run/P33/verify/*.log

The test for an exception is rule R20's: commit what a rerun cannot reproduce — hand or frontier-model analysis, the harness that produced a verdict, a ledger, the recorded contract run — and leave what a script regenerates (RTL dumps, build logs, drafts, compile directories) ignored. What that leaves tracked, as of Phase 33.5: .run/P33/verify/ (the 218-of-218 contract run that Verification and progress quotes), the Phase-32 drafts, notes, banks and reproducers under .run/P32/, the giant-crack reconnaissance under .run/giants/, the crack ledgers the cookbook cites (s42, s43, s45, wave22, near6, fable_80178004, probe_jtbl, S79w), and two fleet ledgers (backlog.jsonl, fuel_manifest.json). Finished session artifacts — wave slates, capture scripts, bank logs — were untracked at the same consolidation; they remain on the maintainer's disk and in the private archive's history.

Per-session layout, from the matching campaign's own hard lessons (the wave playbook's S80 addendum): a session or wave gets one directory, .run/<session>/; each agent works in its own work/<function>/ under it and may clean only that; deliverables (drafts, verdict lines, reports) go to a drafts/, verdicts/ or reports/ directory no agent owns; an agent never runs find, rm or mv outside its own work directory; and an agent's final message is one JSON line, with the prose in reports/<function>.md. Work that turns out to be irreplaceable gets its dated ! block the day it is recognised as such, not at the phase close.

Two fail-safes. Never git clean -x or git clean -fdx in this tree: the purged paths (the disc, the dumps, the Ghidra project, the SDK) are ignored-but-present on a maintainer's disk, and a -x clean deletes the reverse-engineering database. And a tracked .run/ file is published: it is subject to The ROM firewall like anything else — a listing of the target's instructions is ROM-derived even inside a notes file, and the audit's content check looks for exactly that.

For a new project

The day-one kit built from this project stamps these conventions into a fresh repository as a README in its docs folder and one in its scratch folder, together with the .gitignore firewall, so that the first session already has a place for each kind of knowledge and never has to consolidate the way this one did.