mirror/BFM-decomp
1
0
Fork 0
mirror of https://github.com/Druthulu/BFM-decomp synced 2026-09-27 05:56:00 -04:00
Code Issues Packages Projects Releases Wiki Activity

docs(phase-33.5): task 4 — wiki: Reference-index (48 rows + 4 directory rows; the only sanctioned wiki→docs link site), Where-the-project-goes-next (Phase 34, then Gen3: the floor, the bar, the census with one pin count, the invariant, the order, the levers, conduct, shiftability, parked ideas), Contributing 'AI use — conduct', Home rewritten with the source-of-truth statement, sidebar sections Reference / Where next / History; gen3-handoff §8 R1–R83 + 43,925; Reference-index pending entry cleared; coverage check: uncovered == the task-6 leavers; log + checkpoint; P6 rules check

Browse Source
This commit is contained in:
Drew T
2026-09-07 17:32:09 -06:00
parent 5d10a0d126
commit 9970f1e628
8 changed files with 308 additions and 59 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/doc_links_pending.txt
-1
View File
@@ -1,5 +1,4 @@
# docs/doc_links_pending.txt — link targets promised by a document and created by a later task of the SAME phase.
# Format: <repo-relative path><TAB><the task that creates it>. tools/doc_links.py reports these as PENDING instead of
# BROKEN; `--strict` (gate 2) refuses any entry, so this file must be EMPTY before a PhaseEnd.
docs/wiki/Reference-index.md Phase 33.5 task 4 (the Reference index — the only sanctioned place for wiki→docs links)
docs/wiki/Archive-index.md Phase 33.5 task 6 (the Archive index — one row per sunset document)
docs/gen3-handoff.md
+2 -2
View File
@@ -204,11 +204,11 @@ PCSX-Redux, then the disc rebuild with recompressed payloads.
## 8. Governance for Gen3
The framework carries over unchanged: the constitution, the session protocol, the digest, the two gates, one task at
a time, the rules R1–R73 plus the Phase-33 candidates (a)–(h) once ratified. A new generation starts with a fresh
a time, the rules R1–R83 (the Phase-33 candidates were ratified as R74–R83 at the Phase-33.5 gate). A new generation starts with a fresh
plan in plan mode at Max. Three things the record says to do first:
1. **Measure the shape before choosing** (`docs/hindsight-study.md` §2.1): which structures own most of the 61,898
data symbols, and which families own most of the 44,243 pins — a census with a self-asserting scanner, checked
data symbols, and which families own most of the 43,925 pins — a census with a self-asserting scanner, checked
against a case whose answer is known.
2. **Build the differential harness for the new question** before the campaign: "is this rename/type change
byte-neutral?" has two paths — the per-binary gate and the fleet run — and a scanner that asserts every reference
docs/wiki/Contributing-and-the-no-ROM-policy.md
+20
View File
@@ -36,6 +36,26 @@ The matching frontier is empty, so new matches are not the need. The work that h
- **Tooling and documentation** — every tool has a row in [`docs/SETUP.md`](../SETUP.md)'s inventory; a tool change
ships with its siblings wired to use it and its docs in the same change.
## AI use — conduct
The project was carried out by an AI coding agent and says so once, at the project level; that is the disclosure. What
keeps it from being a liability is a short set of conduct rules, which apply to any contribution made with an agent's
help (the charter is [`docs/gen3-standards.md`](../gen3-standards.md) §3; the direction they serve is
[Where the project goes next](Where-the-project-goes-next.md)):
1. **The byte gate is the only claim of success.** No match is reported that the gate has not proven; "the compiler ran"
and "looks equivalent" are not results, and every "done" is written from a tool's printed success line.
2. **Names and types are evidence-based, never guessed.** A function or variable is renamed only on a recorded basis — a
string it prints, a cross-reference chain, the debug menu, a live-RAM datapoint, a community label with provenance. No
evidence, no name: `func_80xxxxxx` is honest and greppable. A model may propose; it may never assert.
3. **Outward text is written by a person.** Issues, pull requests, posts and notes to other projects are written by the
maintainer the way a developer writes — short and plain — never a model draft with the tells removed; and the target
project's own AI policy is read first (sotn-decomp, for one, requires pull requests made without autonomous tooling).
4. **No automated traffic against community infrastructure.** decomp.me asks that nobody scrape it or hook an LLM to it;
the project's use of it is a person in a browser, and its local replica of that toolchain exists so proofs run at home.
5. **Agents assist; a person owns.** Every change is justifiable by a person from the record — the phase logs, the
decision log, the cookbook's byte proofs.
## The one invariant
Every change to `src/`, `config/` or `include/` is held to the byte gate: `make check BINARY=<alias>` stays green for
docs/wiki/Home.md
+31 -13
View File
@@ -8,14 +8,18 @@ on every build. The repository is [github.com/Druthulu/BFM-decomp](../../README.
The claim is narrow and machine-checkable: *matching* means byte-identical output, nothing "functionally equivalent"
counts, and what the repository claims is exactly what `make check-all` proves. The live numbers are generated into the
README's progress block and [`docs/progress.json`](../progress.json), never typed by hand; at the Phase-33 close
(September 2026) all three metrics — functions, instructions, distinct code — read 100%, with two things deliberately
not our C and stated as such: Sony's PsyQ library objects linked into the main executable (1,256 functions) and five
hand-written assembly routines kept verbatim.
README's progress block and `docs/progress.json`, never typed by hand; at the Phase-33 close (September 2026) all three
metrics — functions, instructions, distinct code — read 100%, with two things deliberately not our C and stated as such:
Sony's PsyQ library objects linked into the main executable (1,256 functions) and five hand-written assembly routines
kept verbatim.
The project was carried out end to end by an AI coding agent (Claude Code) under a written constitution and a two-gate
phase system, in twelve weeks. How that was done — and what it cost — is the second half of this wiki.
**This wiki is the source of truth for the project's documentation.** The records behind it (the cookbook, the decision
log, the memory map, the environment reference) live under `docs/` and are listed, with how to read each one, in the
[Reference index](Reference-index.md); when a page and a record disagree, the record wins and the page gets fixed.
## Using the project
| Page | What it answers |
@@ -28,9 +32,22 @@ phase system, in twelve weeks. How that was done — and what it cost — is the
| [Overlays and modules](Overlays-and-modules.md) | The disc, the 218 binaries, where each loads and how every load address was proven |
| [Ghidra rebuild from text](Ghidra-rebuild-from-text.md) | The reverse-engineering database is not in git; here is how it regenerates from text + the disc, and the proof that it does |
| [Verification and progress](Verification-and-progress.md) | The contract run, the three metrics and how they are computed, what CI proves without a disc, the published numbers |
| [Contributing and the no-ROM policy](Contributing-and-the-no-ROM-policy.md) | What may never enter the repository, what a useful contribution looks like now that the frontier is empty, the license split |
| [Contributing and the no-ROM policy](Contributing-and-the-no-ROM-policy.md) | What may never enter the repository, what a useful contribution looks like now that the frontier is empty, the AI-use conduct rules, the license split |
| [Tools from this project](Tools-from-this-project.md) | xsig, the permuter driver, the codegen map, the decomp.me replica, the drafter write-up — what stands on its own for other projects |
## Working conventions
| Page | What it answers |
|---|---|
| [Docs and scratch conventions](Docs-and-scratch-conventions.md) | Where each kind of knowledge goes, which files are generated, how links are checked, what the archive is, what may be tracked under `.run/` |
| [The ROM firewall](The-ROM-firewall.md) | The rule with no private exemption, the nine classes of ROM-derived content, a copyable `.gitignore`, the audit and CI, what a history rewrite costs |
## Reference
The [Reference index](Reference-index.md): every live reference and generated file under `docs/` — the environment
reference, the formats and the memory map, the cookbook and the codegen map, the progress files, the releases, the record,
the Gen3 inputs — with what each is, how to read it and who writes it.
## How to AI-decomp
[How to AI-decomp](How-to-AI-decomp.md) — the transferable part: thirteen chapters on running a byte-exact decompilation
@@ -39,16 +56,17 @@ gate, the bootstrap order, oracles and instruments, cards/lanes/waves, the knowl
models and budgets, the economics as measured, integration and propagation, publishing, and a museum of the failures
that looked right at the time.
## The records behind the wiki
## Where the project goes next
Everything here is a summary of files kept *as it happened*; when a page and a record disagree, the record wins.
[Where the project goes next](Where-the-project-goes-next.md): the flip and the Gen2 exit (Phase 34), then Gen3 —
readability on a byte-exact floor: the style bar, the measured gap, the one invariant, the order of work, the levers.
- [`docs/story.md`](../story.md) — the narrative, first commit to public flip; [`docs/retrospective.md`](../retrospective.md) — what we believed, what failed, what it cost
- [`docs/matching-cookbook.md`](../matching-cookbook.md) (some 500 compiler idioms, each byte-proven; grep it by section — it is 3.5 MB) and its symptom index [`docs/cookbook-index.md`](../cookbook-index.md); [`docs/gcc-2.7.2-map/`](../gcc-2.7.2-map/) — the codegen map of the pinned compiler, by pass
- [`docs/decision-log.md`](../decision-log.md) — every strategic pivot with its measurement and its hindsight; [`docs/accelerators.md`](../accelerators.md) — what would have made earlier phases faster
- [`phase-ends/`](../../phase-ends/) — one PhaseEnd per phase, the digest of every phase and every rule ([`DIGEST.md`](../../phase-ends/DIGEST.md)), the per-phase logs
- [`docs/SETUP.md`](../SETUP.md) — the environment reference, every tool's row; [`docs/verification.md`](../verification.md) — the contract and the last recorded run
- [`docs/memory-map.md`](../memory-map.md) and [`docs/formats.md`](../formats.md) — addresses with provenance; the disc, container, archive and LZSS formats
## History
The narrative and the retrospective are in the Reference index (`docs/story.md`, `docs/story-timeline.md`,
`docs/retrospective.md`); the phase-by-phase record is `phase-ends/` ([Repository layout](Repository-layout.md)); the
documents that served one phase and were then retired are in the [Archive index](Archive-index.md), each with what came
of it and where its information lives now.
*Brave Fencer Musashi is © 1998 Square. This project is not affiliated with or endorsed by Square Enix. `src/` asserts no
license ([`src/NOTICE.md`](../../src/NOTICE.md)); the project's own tools and documents are AGPL-3.0 ([`LICENSE`](../../LICENSE)).*
docs/wiki/Reference-index.md
+92
View File
@@ -0,0 +1,92 @@
# Reference index
Every live reference and generated file under `docs/`, in one place: what it is, how to read it, and who writes it.
This page is the **only** place a wiki page links into `docs/` — every other page links a wiki page for its topic, and
the link checker (`tools/doc_links.py`) derives its allow-list of `docs/` targets from the rows below (a directory row
covers everything under it). The two index pages, this one and the [Archive index](Archive-index.md), are exempt from
that rule; the README keeps its own handful of direct links. Files that left `docs/` at the Phase-33.5 consolidation are
in the Archive index, not here. The conventions behind this page: [Docs and scratch conventions](Docs-and-scratch-conventions.md).
## Environment and verification
| File | What it is | How to read it | Written by |
|---|---|---|---|
| [`SETUP.md`](../SETUP.md) | The environment reference: every tool's install row, the pinned compiler triple (§5.4), the daily command crib, the MCP lifecycle, the scratch conventions (§1a). 270 KB. | Grep by `§` or by tool name; §5.4 for the triple; the `## Tooling inventory` table for a tool's one-line row. | authored; kept current in the same change as any tool (R21) |
| [`verification.md`](../verification.md) | The byte-identity contract and the recorded run that proved it (218 of 218 from a clean rebuild, both SDK legs). | Read whole; §2 quotes the run's summary from `.run/P33/verify/`. | authored; quotes `tools/verify_contract.sh`'s output |
| [`second-oracle.md`](../second-oracle.md) | The independent boundary oracle (R34): what it covers, where it disagrees with the primary one by design. | Read whole. | authored; read by the Makefile and four tools |
| [`effort-map.md`](../effort-map.md) | The reasoning-effort doctrine for the AI agent: depth vs breadth, the per-phase map, the decision rule. Governs where `CLAUDE.md`'s tier language differs. | Read whole. | authored |
| [`doc_links_pending.txt`](../doc_links_pending.txt) | Link targets promised by a document and created by a later task of the same phase (R80). Empty at every phase close. | One `path<TAB>task` per line. | authored per task |
| [`public-flip-runbook.md`](../public-flip-runbook.md) | The history-rewrite and publication procedure, its controls, its risk register, and what the host kept serving afterwards (§11). | By section; §11 for the purge and the probe. | authored |
## Formats and addresses
| File | What it is | How to read it | Written by |
|---|---|---|---|
| [`formats.md`](../formats.md) | The disc, the `.CD` containers, the PAC archives, the LZSS codec (game semantics), the SQV split. Anything not byte-verified is marked. | By section. | authored |
| [`memory-map.md`](../memory-map.md) | Every address with its source and verification status (G5): the loader, the overlay slots, player state, the RAM-dump datapoints, the community labels. | By section (`§S…` = the session that established it); Ghidra is authoritative once an address is imported. | authored |
| [`idxtab-map.md`](../idxtab-map.md) | The master load map: every payload's index-table and destination-pointer entries. | Read whole. | authored from `tools/idxtab_map.py` |
| [`debug-menu-list.txt`](../debug-menu-list.txt) | The retail debug menu's AREA/SCENE list, transcribed from the live menu. | Read whole. | authored (primary data) |
| [`disc-completeness.md`](../disc-completeness.md) | The disc-completeness audit: every payload claimed by a binary or explicitly excluded, with evidence. | Read whole. | authored; reproduce with `tools/disc_code_sweep.py` |
| [`disc-ledger.md`](../disc-ledger.md) | The completeness partition, residue 0. | Read the summary line; the table is per payload. | **generated** — `tools/disc_audit.py`, `make audit-disc` |
## The knowledge base
| File | What it is | How to read it | Written by |
|---|---|---|---|
| [`matching-cookbook.md`](../matching-cookbook.md) | Some 500 compiler idioms, each with its residual, mechanism, lever and byte proof. **3.5 MB — never read whole.** | Grep by `§` number or by symptom; the index below maps symptoms to sections. | authored, one section per proven idiom (R16) |
| [`cookbook-index.md`](../cookbook-index.md) | The symptom → section index of the cookbook. 560 KB. | Grep the symptom. | **generated** — `tools/cookbook_index.py`, checked in `make tools-health` |
| [`gcc-2.7.2-map/`](../gcc-2.7.2-map/) | The codegen map of the pinned compiler by pass (scheduling, register allocation, loop, CSE/expression), with a citation audit; its README is the catalogue. | Start at the README's pass → residual → lever table. | authored; `tools/gccmap_cites.py --check` in tools-health |
| [`wave-playbook.md`](../wave-playbook.md) | The matching-campaign procedure: draw, cards, packs, draft, gate, harvest, checkpoint — each guard paired with the measurement that produced it. | By numbered step; the S80 addenda for the one-agent-per-function shape. | authored |
| [`accelerators.md`](../accelerators.md) | What was learned late that a future project should know on day one: each entry with when it was found, when it could have been, what it would have saved. | Read whole; the entries are short. | authored |
| [`generic-decomp-package.md`](../generic-decomp-package.md) | The thesis for the next project: what it inherits and does before cracking — the knowledge base seeded from the compiler's source and sibling projects, the structural tooling, the similarity join as a band not a point, the differential-oracle harness, the widening review. | Read whole. | authored |
| [`decision-log.md`](../decision-log.md) | Every strategic pivot with its context, what failed, the measurement and the hindsight (R31). 280 KB, append-only. | By date heading. | authored |
| [`psyq-worklist.md`](../psyq-worklist.md) | The per-object record of the 1,256 Sony library functions linked into the executable, and the named residue. | By library. | authored from `tools/psyq_identify.py`; complete |
| [`psyq-sample-idioms.md`](../psyq-sample-idioms.md) | Style notes mined from the SDK's sample sources. | Read whole. | authored (a seed) |
| [`concurrency-design.md`](../concurrency-design.md) | The concurrency contract for overlapping draft, gate, propagate and fleet-verify runs; implemented in the gate snapshot layer. | Read whole. | authored |
| [`tooling-audit.md`](../tooling-audit.md) | The Phase-26 tooling-integrity audit, finding by finding — the evidence that several "walls" were the instruments. 400 KB. | By finding number. | authored |
## Verification and progress (generated)
| File | What it is | Written by |
|---|---|---|
| [`progress.md`](../progress.md), [`progress.fleet.md`](../progress.fleet.md), [`progress.json`](../progress.json) | The executable's digest, the fleet roll-up, and the published data the README block, the badges and CI read. | `tools/progress.py`, `make report BINARY=main`; freshness asserted by `make audit-digest` |
| [`badges/`](../badges/) | The four shields endpoints. | `tools/progress.py` |
| [`difficulty.md`](../difficulty.md), [`duplicates.md`](../duplicates.md), [`duplicates.cross.md`](../duplicates.cross.md) | The executable's harvest queue and self-duplicates; the cross-binary duplicate groups. | `tools/difficulty.py`, `tools/dup_report.py`, `make report BINARY=main` |
| [`backlog.md`](../backlog.md) | The near-miss ledger (0 open). | `tools/backlog.py`, `make report BINARY=main` |
| [`story-timeline.md`](../story-timeline.md), [`story-timeline.svg`](../story-timeline.svg) | The dated digest timeline and its chart. | `tools/timeline.py`; `--check` in `make audit-digest` |
| [`commit-map.tsv`](../commit-map.tsv) | The ordinal → hash map for commits cited in historical documents after the history rewrite. | `tools/public_rewrite/build_commit_map.py`, one-shot |
## Releases and outreach
| File | What it is | How to read it | Written by |
|---|---|---|---|
| [`permuter-ils.md`](../permuter-ils.md) | The permuter recipe, the masked scorer and the local-search driver, and the upstream offer's outcome. | Read whole. | authored |
| [`matching-drafter-pipeline.md`](../matching-drafter-pipeline.md) | The local drafter model's pipeline, its measured arc, and what is not published. | Read whole. | authored |
| [`gen2-mips-matching-model.md`](../gen2-mips-matching-model.md) | The design and every measurement behind that write-up. | By section. | authored |
| [`community-matching-model-plan.md`](../community-matching-model-plan.md) | The parked plan to publish the drafter's weights and dataset (waits on a licensing decision). | Read whole. | authored |
| [`decompme-preset.md`](../decompme-preset.md) | The decomp.me preset for the pinned compiler, its local proof, and the post-flip request procedure. | §5 for the procedure. | authored |
| [`outreach/`](../outreach/) | The notes to neighbouring projects, written for the maintainer to send after the flip. | Read whole. | authored |
## The record
| File | What it is | How to read it | Written by |
|---|---|---|---|
| [`story.md`](../story.md) | The narrative, first commit to public flip. | Read whole. | authored |
| [`retrospective.md`](../retrospective.md) | What was believed, what failed, what it cost, what to do sooner. | Read whole; §4 is the list for the next project. | authored; assembled with `tools/mine_hindsight.py` |
| [`phase34-seed.md`](../phase34-seed.md) | The plan seed for the flip phase (Phase 34): every task, actor, effort and carrying document. | Read whole. | authored |
| [`history/`](../history/) | The provenance archive: the original brief (superseded, with its corrected claims), the methodology version the constitution was generated from, an early experiment. Nothing current. | Its README. | authored |
| `phase-ends/` | Not under `docs/`, but the record proper: one PhaseEnd per phase, the digest of every phase and every rule, the working logs. | [Repository layout](Repository-layout.md). | authored per phase |
## Gen3 inputs
| File | What it is | How to read it | Written by |
|---|---|---|---|
| [`gen3-standards.md`](../gen3-standards.md) | The charter for the readability generation: accuracy met, the style bar (sotn's guide plus four rules), the AI-use conduct, the measured gap, a definition of done. | Read whole. | authored |
| [`gen3-handoff.md`](../gen3-handoff.md) | Where Gen2 ends, the starter census with its commands, the one invariant, the levers, shiftability scoped, the parked ideas. | §3 for the census, §5 for the levers. | authored |
| [`gen3-parking-lot.md`](../gen3-parking-lot.md) | The asset-export / native-rebuild survey, with the decisive probes. | Read whole. | authored |
| [`actor-struct.md`](../actor-struct.md) | The recovered player/actor structure (~154 fields, live-verified) and the byte-neutrality finding. | Read whole. | authored |
| [`struct-core-pivot.md`](../struct-core-pivot.md) | What struct recovery can and cannot do for matching — the Phase-16 wall and the pivot. | Read whole. | authored |
| [`wall-taxonomy.md`](../wall-taxonomy.md) | The census of residual stubs by blocker class, validated against the bytes. | By class. | authored from `tools/wall_taxonomy.py` |
Not listed: `docs/wiki/` and `docs/how-to-ai-decomp/` are this wiki's sources; `docs/sunset/` is the archive, indexed by
the [Archive index](Archive-index.md).
docs/wiki/Where-the-project-goes-next.md
+97
View File
@@ -0,0 +1,97 @@
# Where the project goes next
The matching is finished: 218 binaries rebuild byte for byte and the frontier is empty. What remains is in two parts.
The first is short and mostly administrative — the public flip and the close of the second generation. The second is the
third generation: making code that *matches* into code a maintainer would call *finished*.
## Phase 34 — the flip and the Gen2 exit
The repository's history was rewritten before publication to remove every game-derived byte and force-pushed in
place. The flip itself waits on one thing outside the project's control: GitHub Support purging the old objects, which
the host's Activity view still serves for every pre-force-push tip ([The ROM firewall](The-ROM-firewall.md)). A probe
of the old hashes runs daily; the flip is gated on it printing PASS (or, if the purge stalls, on recreating the
repository under the same name).
After that, in order: the visibility change and the checks that follow it (the probe after the flip, the Actions tab
green, the wiki enabled — GitHub offers wikis only on public repositories on the free plan); the outward actions the
maintainer does in his own words — the decomp.me preset request, a note to the neighbouring Archipelago project whose
RAM map cited this one, the progress boards (decomp.dev, frogress), the wiki push, the tool announcement and the
topics on the released tool repositories; the aftercare (other clones reset to the rewritten history, the rewrite
scratch deleted); and the Phase-34 close at **v2.0.0**, which is the Gen2 exit. The seed for that phase, with every
task, actor and effort level, is [`docs/phase34-seed.md`](../phase34-seed.md).
## Gen3 — readability, on a byte-exact floor
**The floor.** Accuracy is not a Gen3 concern; it is the floor Gen3 stands on. "Matched" means instruction-identical
including register allocation *and* a whole-binary hash equal to the original, checked inside every build of every
binary, with a clean rebuild after every batch. Nothing functionally equivalent has ever been counted, and what is not
the project's own C is stated: 1,256 Sony library objects linked byte-identical into the executable, five hand-written
assembly routines kept verbatim.
**The bar.** The community's expectations are written down, and the project adopts sotn-decomp's style guide as its
baseline: `camelCase` locals, `g_PascalCase` globals, `PascalCase` types and functions, the project's own `u8…u32`
types, clang-format at 4 spaces and 80 columns, decimal for counts and hexadecimal for addresses and masks,
`// !FAKE:` on anything that exists only to force a match, functions in assembly order in the file of the system they
implement — and, ranked first, *"if you are not sure what something does, it is better to leave it unnamed than name it
wrongly."* To that the project adds four rules of its own, because its code was produced by a machine at scale:
1. **Every name carries evidence** — a string, a cross-reference chain, the debug menu, a live-RAM datapoint, a community
label with provenance — recorded the way addresses are. A model may propose a name; it may never assert one.
2. **Every register pin goes**, or stays with a `// !FAKE:` line naming the residual. The count is a published metric until
it is zero.
3. **Shared engine functions live as C, not as macros** — the dedup engine's macro bodies become `.c` files per system,
byte-neutrally, proven per family.
4. **Types come from the code, names from observation** — structure from the access patterns the compiler locked into the
bytes; the 1,232 struct definitions unified before any field is named.
**The gap, as measured.** The starter census below was derived on 2026-09-07 with the commands in
[`docs/gen3-handoff.md`](../gen3-handoff.md) §3 — re-derive it, do not trust it:
| Quantity | Value | Note |
|---|---|---|
| Raw address casts `*(T *)0x80…` | 143 | the last of a class the type-lift tools already handle |
| Distinct `D_80xxxxxx` data symbols | 61,898 | most are fields of a handful of structures the engine indexes |
| Distinct `func_80xxxxxx` names | 16,335 | across 4,287 C files |
| Register-pin declarations | 43,925 | `grep -rhoE 'register [^;/]*__asm__\("\$?[a-z0-9]+"\)' src --include=*.c --include=*.h \| wc -l`; 43,857 in the `"$reg"` form, 68 in the bare form |
| Shared bodies living as macros | 5,147 | in one 8.4 MB header |
| Struct definitions | 1,232 | many drafter-invented variants of one type |
| Curated names in the symbol files | 1,083 | the names the build already knows |
**The one invariant.** Every Gen3 edit is gated exactly like a match was: the per-binary check for every binary a change
touches, and the clean fleet run after anything that touches a shared body, a shared header or the executable. A rename
is a symbol-file change mirrored into Ghidra by the headless script, never a hand edit of assembly. A shared body is one
source and up to 138 binaries, so the gate runs on all of them. Types are a comprehension lever, not a byte lever — the
compiler does not care what a field is called — which is exactly why struct work is safe and exactly why it still goes
through the gate.
**The order of work:** pins off → the macro bodies into shared C files → struct unification → names with evidence →
formatting and comments. Three things the record says to do first: measure the shape (which structures own most of the
data symbols, which families own most of the pins — with a self-asserting scanner checked against a known case), build
the differential harness for the new question ("is this edit byte-neutral?" has two paths, and a scanner must assert
every reference to a renamed symbol was rewritten), and batch by leverage — a pin in a shared body comes off for 138
binaries at once.
**The levers Gen3 inherits** are tools already in the tree: the fleet-wide type lift, the call-site cast and canonical-
signature tools, the declaration layer of the reconcile ladder, the allocation-order and RTL-dump readers to consult
before touching a pin, the propagation and registry machinery that keeps 2,220 dedup groups consistent through a rename,
the headless symbol mirror into Ghidra, and the atlas for grouping candidates. The table with each lever's purpose is
[`docs/gen3-handoff.md`](../gen3-handoff.md) §5.
**The use of AI — conduct.** The project is openly machine-carried and says so once, at the project level. The rules that
keep that from being a liability: the byte gate is the only claim of success; names and types are evidence-based, never
guessed; outward text is written by a person, the way a developer writes, after reading the target project's own AI
policy; no automated traffic against community infrastructure; agents assist, a person owns. They are stated in full on
the [Contributing](Contributing-and-the-no-ROM-policy.md) page and in [`docs/gen3-standards.md`](../gen3-standards.md) §3.
**Shiftability, honestly scoped.** Overlays are position-locked at one slot by the loader; LZSS recompression is not
byte-stable, so a rebuilt disc verifies by booting, not by hash; the executable's Sony regions are linked objects.
The order that follows: symbolic references first (byte-neutral, gated), then a shifted overlay that boots in the
emulator, then the disc rebuild. Parked beyond that, each with its state recorded in the handoff: asset export and a
native rebuild, a PC port, randomizer-grade tooling, the community matching model (pipeline published; data and weights
wait on a licensing decision), the Japanese release and the prototypes as extra versions, the Sony libraries from source.
## What a new project takes from this
The order this project would use if it started again — and the firewall, the conventions and the conduct rules above —
are packaged as a day-one kit for the next decompilation, described on the wiki page *Start a new decomp project* once
Phase 33.5 ships it.
docs/wiki/_Sidebar.md
+9
View File
@@ -16,6 +16,9 @@
- [Docs and scratch conventions](Docs-and-scratch-conventions.md)
- [The ROM firewall](The-ROM-firewall.md)
**Reference**
- [Reference index](Reference-index.md)
**How to AI-decomp**
- [Index](How-to-AI-decomp.md)
- [00 Read me first](../how-to-ai-decomp/00-README.md)
@@ -31,3 +34,9 @@
- [10 Integration and propagation](../how-to-ai-decomp/10-integration-and-propagation.md)
- [11 Publishing](../how-to-ai-decomp/11-publishing.md)
- [12 The failure museum](../how-to-ai-decomp/12-failure-museum.md)
**Where the project goes next**
- [Where the project goes next](Where-the-project-goes-next.md)
**History**
- [Archive index](Archive-index.md)
phase-ends/CURRENT_PHASE.md
+57 -43
View File
@@ -42,7 +42,7 @@ in-tree links to `docs/wiki/<Page>.md`. 8. `.run/`: only what git tracks; no sca
- [x] **1** Unblock, code side: tool docstrings re-pointed (`decl_from_use.py`, `integration_resolver.py`, `jtbl_carve.py`, `autopsy.py`, `residual_class.py`); CLAUDE.md lines 11 / 61 / the effort-default sentence — xHigh — see Log 2026-09-07 Task 1
- [x] **2** Unblock, doc side: banners (`concurrency-design`, `gen2-mips-matching-model`, `psyq-worklist`, `distill/*`, `hindsight-study` HISTORICAL + supersession table); SETUP §1a rewritten; the pin count re-derived (one dated number + its command, both gen3 docs) — xHigh — see Log 2026-09-07 Task 2
- [x] **3** Wiki — conventions + firewall: `Docs-and-scratch-conventions.md`, `The-ROM-firewall.md` (the gitignore template block); sidebar rows; Repository-layout/Contributing/ch.11 links; forward links in `doc_links_pending.txt` — Max — see Log 2026-09-07 Task 3
- [ ] **4** Wiki — reference + direction: `Reference-index.md` (list-driven), `Where-the-project-goes-next.md`, Contributing "AI use — conduct", Home rewritten, sidebar sections — xHigh — **then P6 rules check**
- [x] **4** Wiki — reference + direction: `Reference-index.md` (list-driven), `Where-the-project-goes-next.md`, Contributing "AI use — conduct", Home rewritten, sidebar sections — xHigh — see Log 2026-09-07 Task 4 — **P6 rules check done after it**
- [ ] **5** Integrate the six (hand-matching-process; wave-metrics + the s37w block → wave-playbook; portable-decomp-workflow vs the 13 chapters; generic-decomp-package attributed; hindsight §2.1/§7 → ch.03 + permuter-ils; the gen3 docs → the direction page); re-point every referrer incl. `retrospective.md:10–11` — xHigh
- [ ] **6** Sunset moves (~61 files incl. the 6 frozen snapshots) with the referrer census by command; `docs/sunset/README.md`; `Archive-index.md` rows (backticked paths + `v1.32.1`); DIGEST §4 + `docs/history/README.md` refreshed; ledger coverage: every `git ls-files docs` path outside wiki/how-to/sunset is KEEP in the ledger — xHigh
- [ ] **7** Checkers: `doc_links.py` (sunset refusal, index-derived allowlist, wiki-first warnings, TRACKED/UNTRACKED citation classifier + `--disk`, coverage ⊆); reachability in `wiki_render --selftest`; `timeline.py` regen in `report BINARY=main` + `--check` in `audit-digest`; the 14 dangling cookbook cites; the gitignore-template diff; tools-health wiring; SETUP rows — xHigh
@@ -259,57 +259,71 @@ one-list rule with the `untracked_after_rewrite.txt` sibling, the rewrite lesson
layout (`docs/` row, `.run/` row, the not-in-repo section, R1…R83), Contributing (the firewall paragraph; "83 numbered rules"), how-to
ch.11 (the day-one section + the proposed-rules heading now says "ratified as R74–R80"). Forward links registered as PENDING:
`Reference-index.md` (task 4), `Archive-index.md` (task 6). Verify: `wiki_render --selftest` OK; `doc_links` 0 broken / 2 pending.
Commit `5d10a0d12`.
## 🛑 SESSION CHECKPOINT — Tasks 0–3 ✓; NEXT = task 4 (wiki: Reference-index + Where-the-project-goes-next + Contributing conduct + Home/Sidebar; xHigh — prompt Drew to drop back, R27; then the P6 rules check)
### 2026-09-07 — Task 4 — Wiki: the Reference index, the direction page, Contributing's conduct section, Home + sidebar (xHigh)
`Reference-index.md`: 47 rows in seven groups (environment and verification; formats and addresses; the knowledge base; verification
and progress (generated); releases and outreach; the record; Gen3 inputs) + four directory rows (`gcc-2.7.2-map/`, `badges/`,
`outreach/`, `history/`) — each with what it is, how to read it (the 3.5 MB / 560 KB warnings), who writes it; the page states it is the
only sanctioned wiki→`docs/` link site and that the two index pages are exempt. `Where-the-project-goes-next.md`: Phase 34 (the
probe-gated flip, the outward actions in the maintainer's words, aftercare, v2.0.0), Gen3 (the floor, the bar — sotn's guide + the four
rules —, the census table with the ONE pin count and its command, the invariant, the order of work, the levers, the conduct rules,
shiftability scoped, the parked ideas), and the day-one-kit pointer in prose. Contributing gained "AI use — conduct" (the five rules of
gen3-standards §3, linking the direction page). `Home.md` rewritten: the source-of-truth statement + sections Using / Working conventions
/ Reference / How to AI-decomp / Where next / History; `_Sidebar.md` gained Reference, Where-next and History (Archive index, pending).
`gen3-handoff.md` §8: "R1–R73 plus candidates" → R1–R83; the last 44,243 → 43,925. `doc_links_pending.txt`: the Reference-index line
removed (1 entry left: Archive-index, task 6). **Coverage check** (a Python one-liner over `doc_links`' LINK_RE — the task-7 checker's
prototype): 132 `docs/` files outside wiki/how-to; index ∪ README covers 66; the 66 uncovered are EXACTLY the task-6 leavers (33 root
sunset + 4 integrate + 6 frozen + 12 per-binary snapshots + 11 tool-designs + 2 distill + 4 history files with spaces in their names)
plus one KEEP file the first draft omitted — `generic-decomp-package.md`, added as a row (how-to 00/06 link it). Verify:
`wiki_render --selftest` 12/12; `doc_links` 0 broken / 1 pending.
**P6 rules check — re-read complete** (CLAUDE.md's Mandatory Behavior + fail-safes, DIGEST §3 R1–R83): one task at a time, one commit
per task after this log; never overwrite blind (H5 — every banner was added, none replaced); never commit ROM content (the two
`fable_cd4` listings leave at task 8, before any push Drew makes of this phase would matter — they are already in the pushed history);
never `git clean -x`; a match is byte-for-byte (no build input has changed). Continuing with task 5.
## 🛑 SESSION CHECKPOINT — Tasks 0–4 ✓ (P6 check done); NEXT = task 5 (integrate the six documents into the wiki; xHigh)
### 0. How to use this block
You are a FRESH SESSION that has read `PROJECT_CONTEXT.md`, `phase-ends/DIGEST.md`, `PhaseEnd_Phase31/32/33.md` and this file, and
nothing else (R64). Replay this block verbatim, state phase / done / NEXT / effort, list the rules from the digest (R1–R83), then
WAIT for Drew. Rebuild the harness task list (16 rows, R28) marking tasks 0–3 completed and task 4 in progress.
WAIT for Drew. Rebuild the harness task list (16 rows, R28) marking tasks 0–4 completed and task 5 in progress.
### 1. Where we are
**Phase 33.5** (sub-phase; v1.32.0 → v1.32.1), gate 1 approved 2026-09-07 by Drew in plan mode at Max; effort follows the plan's
column (Max for tasks 3, 10, 12, 15 — Drew toggled Max for task 3; xHigh otherwise; prompt at each transition, R27). Baseline HEAD
`80d45b29b`; task 0 = `39d524991`; task 1 = `a0cf302e5`; task 2 = `d06923a06`; task 3 = the commit after it (the two wiki pages).
Nothing under `src/`, `config/` or the `Makefile` has changed; the fleet is 218/218 at the Phase-33 close. `docs/doc_links_pending.txt`
carries TWO entries (`docs/wiki/Reference-index.md` → task 4; `docs/wiki/Archive-index.md` → task 6) — remove each when its page lands.
Wiki pages now: the 12 original + `Docs-and-scratch-conventions.md` + `The-ROM-firewall.md`; `_Sidebar.md` has a "Working conventions"
section between "Using the project" and "How to AI-decomp".
column (Max for tasks 10, 12, 15 — prompt at each transition, R27; xHigh now). Baseline HEAD `80d45b29b`; task 0 = `39d524991`;
task 1 = `a0cf302e5`; task 2 = `d06923a06`; task 3 = `5d10a0d12`; task 4 = the commit after it. Nothing under `src/`, `config/` or the
`Makefile` has changed; the fleet is 218/218 at the Phase-33 close. `docs/doc_links_pending.txt` carries ONE entry
(`docs/wiki/Archive-index.md` → task 6). Wiki pages now: 12 original + Docs-and-scratch-conventions, The-ROM-firewall, Reference-index,
Where-the-project-goes-next (16 + the 13 how-to chapters); the sidebar has sections Using / Working conventions / Reference / How to
AI-decomp / Where next / History (Archive index, pending). The coverage one-liner in the task-4 log entry is the prototype of task 7's
check: the 66 files it lists as uncovered are the task-6 leavers.
### 2. What NEXT does (task 4, xHigh) — exact steps
1. `docs/wiki/Reference-index.md` — list-driven from ledger L1 (this file): one row per KEEP-LIVE / KEEP-GEN file, grouped: environment
(`SETUP.md`, `effort-map.md`, `verification.md`, `second-oracle.md`, `doc_links_pending.txt`) · formats and addresses (`formats.md`,
`memory-map.md`, `idxtab-map.md`, `debug-menu-list.txt`, `disc-completeness.md`, `disc-ledger.md`) · the knowledge base
(`matching-cookbook.md`, `cookbook-index.md`, `gcc-2.7.2-map/` as a directory row, `wave-playbook.md`, `accelerators.md`,
`decision-log.md`, `psyq-sample-idioms.md`, `psyq-worklist.md`, `concurrency-design.md`, `tooling-audit.md`) · verification and
progress (`progress.md/.json/.fleet.md`, `difficulty.md`, `duplicates.md/.cross.md`, `backlog.md`, `badges/` directory row,
`story-timeline.md/.svg`, `commit-map.tsv`, `public-flip-runbook.md`) · releases and outreach (`permuter-ils.md`,
`matching-drafter-pipeline.md`, `gen2-mips-matching-model.md`, `community-matching-model-plan.md`, `decompme-preset.md`,
`outreach/` directory row) · the record (`story.md`, `retrospective.md`, `phase34-seed.md`, `history/` directory row) · Gen3 inputs
(`gen3-standards.md`, `gen3-handoff.md`, `gen3-parking-lot.md`, `actor-struct.md`, `struct-core-pivot.md`, `wall-taxonomy.md`).
Columns: file (a relative link `../<path>`) · what it is · how to read it (grep by §, the size warning for the 3.5 MB cookbook and
the 566 KB index) · generated by (tool + target) or "authored". The INTEGRATE→SUNSET four (`hand-matching-process`, `wave-metrics`,
`portable-decomp-workflow`, `hindsight-study`) are NOT rows — they leave in task 6; the 6 frozen snapshots are NOT rows. The
directory rows (`gcc-2.7.2-map/`, `badges/`, `outreach/`, `history/`) are prefixes for the task-7 allow-list. State at the top that
this page is the only sanctioned place for wiki→`docs/` links and that the index pages are exempt from that rule.
2. `docs/wiki/Where-the-project-goes-next.md` — Phase 34 (from `docs/phase34-seed.md` §1: the daily probe, the flip, the outward
actions, aftercare, v2.0.0 = Gen2 exit; the Support ticket as the gate) and Gen3 (from `gen3-standards.md` §1–§3 in summary —
accuracy met, the readability bar, the AI-use conduct — and `gen3-handoff.md`: the census with the ONE pin count 43,925 + its
command + date, the invariant "every Gen3 edit is gated like a match", the levers table condensed, the order of work
pins → macro bodies → struct unification → names → formatting, the parked ideas). Link both docs via the Reference index row form.
3. `Contributing-and-the-no-ROM-policy.md` — a new section "AI use — conduct" after "What a useful contribution looks like": the five
rules of `gen3-standards.md` §3 (the byte gate is the only claim of success; names/types evidence-based never guessed — unnamed
beats wrong; outward text is written by a person, the target project's AI policy read first; no automated traffic against
community infrastructure; agents assist, a person owns), linking the direction page.
4. `Home.md` rewritten: the lead as is; sections "Using the project" (table as is + the two conventions rows), "Working conventions",
"Reference" (one paragraph → the index), "How to AI-decomp" (as is), "Where the project goes next" (one paragraph), "History"
(story / timeline / retrospective / the Archive index — pending) ; the "records behind the wiki" list becomes the index pointer.
`_Sidebar.md`: add "Reference" (Reference index), "Where the project goes next", "History" (Story and timeline → via index? — no:
sidebar rows must be wiki pages; History = Archive index (pending) + the direction page); keep the how-to list.
5. Remove the `Reference-index.md` line from `docs/doc_links_pending.txt`. Verify: `wiki_render --selftest`; `doc_links` 0 broken /
1 pending; the list-driven check — every L1 KEEP path appears as a link target or under a directory row (a small Python one-liner
over the page with `doc_links.LINK_RE`). Log + checkpoint; commit by explicit path. **Then the P6 rules check** (re-read CLAUDE.md's
Mandatory Behavior + the fail-safes; state "Rules check — re-read complete. Continuing with task 5"). Task 5 is xHigh.
### 2. What NEXT does (task 5, xHigh) — exact steps
Fold the six INTEGRATE documents into the wiki/how-to and re-point every referrer, so task 6 can move the four that leave:
1. `docs/hand-matching-process.md` (32 KB, P17 Ghidra-driven hand-matching guide) → read it; fold what the wiki lacks into
`The-matching-workflow.md` (a "Matching by hand" section: the Ghidra-driven loop, the differ discipline, the register-pin toolkit
pointer §17, when to hand the near-miss to the permuter) and, for the transferable parts, how-to `07-compiler-source.md`; record
the "unique claims folded" list in this log.
2. `docs/wave-metrics.md` (20 KB, one row per crack wave) → the measurement source of how-to `09-economics.md` and `05-cards-lanes-
waves.md`: fold the per-wave table's headline numbers (agents, tokens, wall, banked per wave; the 76%→77%→100% bank-rate arc) where
the chapters cite economics without a table; move its two `.run/` prose citations FIRST: line 62's "Use `.run/s37w.js`'s execution
block for every future wave" → copy the execution block itself into `docs/wave-playbook.md` (a short fenced block in §5 Draft) and
line 153's `.run/s6f_gate.py` assertion → name it as archived. Then the file can leave.
3. `docs/portable-decomp-workflow.md` (14 KB, §0–§11) → diff each § against the 13 chapters (this is the breadth-shaped part — three
read-only agents may take §0–§4 / §5–§8 / §9–§11); fold any unique claim into ch.03 (bootstrap order), ch.05 (lanes/cards), ch.09
(economics), ch.04 (refuse work the pipeline cannot bank — R43/R45 framing); re-point how-to `00-README.md:59`'s link to the chapters.
4. `docs/generic-decomp-package.md` (KEEP) → attribute it in ch.03 and ch.06 where its claims are used without citation (the twin BAND
3.4× widening; "rank by work, not distance"; the widening review); it stays and is in the Reference index.
5. `docs/hindsight-study.md` → re-point how-to `00-README.md:60`, `02-byte-gate.md:16`, `docs/retrospective.md:10–11` and
`docs/gen3-handoff.md` §8 item 1 (which cites its §2.1: "measure the shape before choosing") to the chapters/the retrospective;
add to ch.03 Phase 2b the "mine the permuter's failures" paragraph (its §7 → `tools/autopsy.py` + `residual_class.py`; the
docstrings of both tools already cite ch.03 Phase 2b) and to `docs/permuter-ils.md` a one-line pointer to the autopsy.
6. `docs/gen3-standards.md` / `gen3-handoff.md` (KEEP) → already summarised in the direction page (task 4); nothing more.
7. ch.04 gains a citation of `docs/tooling-audit.md` as its evidence base (one sentence + a Reference-index-form link).
8. Referrer census for the four leavers (the D3 command) → must be 0 outside PhaseEnds/logs before task 6. Verify `doc_links` 0 broken;
log the folded-claims checklist; commit by explicit path. Task 6 is xHigh.
### 3. Standing facts for every task of this phase
- One commit per task, after this file's log line (R8/R42 form); commit by explicit path; no trailers (R5); Drew pushes (R6).