From db68150f5eddef6320850f781b1abc66db7ec75c Mon Sep 17 00:00:00 2001 From: Drew T <50529377+Druthulu@users.noreply.github.com> Date: Tue, 29 Sep 2026 19:42:44 -0600 Subject: [PATCH] T1.c1: tools-health off docs/retired: census reads ops inventory, links fixed, cookbook_index off --- Makefile | 7 ++++- README.md | 2 +- docs/gcc-2.7.2-map/README.md | 2 +- docs/how-to-ai-decomp/00-README.md | 2 +- docs/how-to-ai-decomp/01-governance.md | 2 +- docs/how-to-ai-decomp/06-knowledge-base.md | 4 +-- .../how-to-ai-decomp/08-models-and-budgets.md | 2 +- docs/ops/tooling-inventory.md | 2 +- docs/wiki/Archive-index.md | 10 +++---- .../Contributing-and-the-no-ROM-policy.md | 2 +- docs/wiki/Docs-and-scratch-conventions.md | 6 ++-- docs/wiki/Reference-index.md | 10 ++++--- docs/wiki/Repository-layout.md | 2 +- docs/wiki/The-matching-workflow.md | 4 +-- docs/wiki/Toolchain-setup.md | 2 +- phase-ends/current/logs/T1.c1.md | 30 +++++++++++++++++++ tools/doc_links.py | 4 +-- tools/tool_census.py | 11 +++---- tools/wiki_render.py | 2 +- 19 files changed, 72 insertions(+), 34 deletions(-) create mode 100644 phase-ends/current/logs/T1.c1.md diff --git a/Makefile b/Makefile index 5df9634aa8..2d796a9a24 100644 --- a/Makefile +++ b/Makefile @@ -314,7 +314,12 @@ tools-health: $(MAKE) --no-print-directory audit-digest # The cookbook index is DERIVED (R33) and self-asserts its coverage (R32). Stale = agents can't # find documented idioms and re-derive them at full token cost (measured, P30 wave 1). - $(VENV_PY) tools/cookbook_index.py --check + # DISABLED: cookbook_index.py --check — 2026-09-29 / Phase 38 T1 + # Original intent: assert docs/cookbook-index.md is fresh against docs/matching-cookbook.md. + # Why disabled: both files moved to docs/retired/ at the PA3 migration (b8a6cbc54); the check crashed on the missing + # source. Successor: cookbook/INDEX.md, kept by tools/cookbook_add.sh. + # Re-enable if: docs/matching-cookbook.md becomes a live source again. + # $(VENV_PY) tools/cookbook_index.py --check $(VENV_PY) tools/gccmap_cites.py --check # P33 B5: the Ghidra roster is DERIVED from config/ghidra/*.jsonl (R33); a stale roster misreports # which programs' RE work is tracked as text. Pure text check, no Ghidra needed. diff --git a/README.md b/README.md index 1917e32a25..9774e97542 100644 --- a/README.md +++ b/README.md @@ -105,7 +105,7 @@ The reverse-engineering side ran on Ghidra driven over MCP, with a PCSX-Redux ru byte gate over the whole binary, a permuter, a compiler-idiom cookbook, and — decisively — reading the gcc 2.7.2 source to explain the bytes the compiler emits. Everything that was learned is written down where it can be reused: -- [`docs/matching-cookbook.md`](docs/matching-cookbook.md) — the idioms, each byte-proven, and the compiler-internals laws +- [`cookbook/INDEX.md`](cookbook/INDEX.md) — the idioms, each byte-proven, and the compiler-internals laws - [`docs/gcc-2.7.2-map/`](docs/gcc-2.7.2-map/) — the codegen map of the pinned compiler, by pass - [`docs/decision-log.md`](docs/decision-log.md) — every strategic pivot with its measurement and its hindsight - [`docs/accelerators.md`](docs/accelerators.md) — what would have made earlier phases faster diff --git a/docs/gcc-2.7.2-map/README.md b/docs/gcc-2.7.2-map/README.md index 81d8887954..e62eaf2d0e 100644 --- a/docs/gcc-2.7.2-map/README.md +++ b/docs/gcc-2.7.2-map/README.md @@ -6,7 +6,7 @@ that moves it — each lever **byte-proven** on a named function with the diff b Phase 23 by four frontier-class agents reading the gcc 2.7.2 passes directly, after six phases in which the same residuals had been called "unsteerable"; it is the artifact that lets a mid-tier agent, or a local model, apply compiler-internal levers without reading 80,000 lines of source. The index and triage table live in the cookbook -([`docs/matching-cookbook.md`](../matching-cookbook.md) §31); the method is chapter 07 of the how-to +([`cookbook/INDEX.md`](../../cookbook/INDEX.md) §31); the method is chapter 07 of the how-to ([`docs/how-to-ai-decomp/07-compiler-source.md`](../how-to-ai-decomp/07-compiler-source.md)). ## The files diff --git a/docs/how-to-ai-decomp/00-README.md b/docs/how-to-ai-decomp/00-README.md index ca93b258df..480903109c 100644 --- a/docs/how-to-ai-decomp/00-README.md +++ b/docs/how-to-ai-decomp/00-README.md @@ -67,7 +67,7 @@ the machine that builds it is universal.** That is the whole point of these chap - [`docs/accelerators.md`](../accelerators.md) — what was learned late that day one should know, each entry dated with when it *could* have been found - [`docs/retrospective.md`](../retrospective.md) and [`docs/story.md`](../story.md) — what we believed, what failed, what it cost; and the narrative - [`docs/wave-playbook.md`](../wave-playbook.md) — the running procedure, each guard paired with the measurement that earned it -- [`docs/decision-log.md`](../decision-log.md) — every pivot with its measurement and hindsight; [`docs/effort-map.md`](../effort-map.md) — the reasoning-effort doctrine +- [`docs/decision-log.md`](../decision-log.md) — every pivot with its measurement and hindsight; [`docs/retired/effort-map.md`](../retired/effort-map.md) — the reasoning-effort doctrine - [`phase-ends/DIGEST.md`](../../phase-ends/DIGEST.md) — every phase's synopsis and the full text of every rule cited here as `R` The framework the governance chapter describes is published separately as diff --git a/docs/how-to-ai-decomp/01-governance.md b/docs/how-to-ai-decomp/01-governance.md index 36b6830902..64b839ee54 100644 --- a/docs/how-to-ai-decomp/01-governance.md +++ b/docs/how-to-ai-decomp/01-governance.md @@ -86,7 +86,7 @@ The last thirty sessions of the project resumed from the digest and the checkpoi ## Effort, and who toggles it The orchestrating agent's reasoning depth is a setting the human controls, and the project mapped it per task -([`docs/effort-map.md`](../effort-map.md)): the deepest setting for phase plans, PhaseEnds, architectural forks and +([`docs/retired/effort-map.md`](../retired/effort-map.md)): the deepest setting for phase plans, PhaseEnds, architectural forks and non-obvious debugging; the highest *persistent* setting for execution whose design is already settled; multi-agent breadth (many independent items) for surveys, audits and bulk matching. The rule that made this workable is R27: **every transition is prompted and waited for** — the agent never launches a fan-out on a verbal yes, and never rolls diff --git a/docs/how-to-ai-decomp/06-knowledge-base.md b/docs/how-to-ai-decomp/06-knowledge-base.md index 781d7ba288..4f1c8bd34a 100644 --- a/docs/how-to-ai-decomp/06-knowledge-base.md +++ b/docs/how-to-ai-decomp/06-knowledge-base.md @@ -8,8 +8,8 @@ behaviour; yours will be about your compiler. The machine that builds it is what | Layer | BFM's instance | What it holds | |---|---|---| -| **The cookbook** | [`docs/matching-cookbook.md`](../matching-cookbook.md) (3.5 MB; grep it by section, never read it whole) | one section per idiom or law, each **byte-proven** on a named function with the diff before and after; the compiler-internals laws; the integration classes; the wall proofs and their later refutations | -| **The symptom index** | [`docs/cookbook-index.md`](../cookbook-index.md), derived by a self-asserting tool | *symptom → section*: what a residual looks like in the diff, and where the lever is | +| **The cookbook** | [`cookbook/INDEX.md`](../../cookbook/INDEX.md) (3.5 MB; grep it by section, never read it whole) | one section per idiom or law, each **byte-proven** on a named function with the diff before and after; the compiler-internals laws; the integration classes; the wall proofs and their later refutations | +| **The symptom index** | [`cookbook/INDEX.md`](../../cookbook/INDEX.md), derived by a self-asserting tool | *symptom → section*: what a residual looks like in the diff, and where the lever is | | **The codegen map** | [`docs/gcc-2.7.2-map/`](../gcc-2.7.2-map/) by compiler pass | pass → residual pattern → C-level lever, from reading the compiler's source ([chapter 07](07-compiler-source.md)) | | **The pinned triple and the permuter recipes** | the build, `tools/permuter/` | the executable form of the same knowledge: what the tools try before an agent is asked | | **The journal** | per-function past-attempt notes | what every earlier agent tried on *this* function — pack fuel | diff --git a/docs/how-to-ai-decomp/08-models-and-budgets.md b/docs/how-to-ai-decomp/08-models-and-budgets.md index 5ebe489dbf..988f5e5d54 100644 --- a/docs/how-to-ai-decomp/08-models-and-budgets.md +++ b/docs/how-to-ai-decomp/08-models-and-budgets.md @@ -46,7 +46,7 @@ differently. ## The orchestrator's effort — depth vs. breadth The orchestrating agent's reasoning depth is a human-controlled setting, and it maps onto two axes -([`docs/effort-map.md`](../effort-map.md)): +([`docs/retired/effort-map.md`](../retired/effort-map.md)): | | Depth — how hard one agent thinks | Breadth — how many angles at once | |---|---|---| diff --git a/docs/ops/tooling-inventory.md b/docs/ops/tooling-inventory.md index 3a15da025b..50dbcda794 100644 --- a/docs/ops/tooling-inventory.md +++ b/docs/ops/tooling-inventory.md @@ -18,7 +18,7 @@ Every script under `tools/` (plus the two report make-targets), grouped by purpo | | `DecompileAt.java` | Decompile the function at a given address (scripted scaffold). | | | `DefineFunctions.java` | Disassemble + create functions at splat's validated entry points (`.run/_funcs.txt`) — completes a raw-blob program's function set (Phase 10). | | | `ApplySymbols.java` + `tools/ghidra_apply_symbols.sh` | **(P31 S78) The Ghidra MIRROR of the curated symbol file (R15/G6), headless with a real save.** `tools/ghidra_apply_symbols.sh [PROG] [symbols files…]` (defaults `SLUS_007.26 config/symbols.us.txt`; MCP must be STOPPED first) reads `name = 0xADDR;` rows and sets every function/label to its curated name; a name held by another address is moved to that address's own curated name first (`firstfile`/`firstfile2`), else to `__at_`. Idempotent; prints `BFMAPPLY renamed_funcs=… unchanged=…`; R9-verify with `ghidra_mcp_verify.sh`. **Use this, not MCP `rename_symbol`/`batch_rename`, for renames:** S78 observed 47 MCP renames NOT persisting through the sentinel stop ("Save succeeded", DB grew, names gone — R9 caught it; cause not yet isolated), while the postScript path persisted 73/73 on the first run. | -| **Public flip / CI** | `.github/workflows/no-rom.yml` | **(P33 B7) The ROM-free CI**: job `audits` (audit_public, audit_text_sources, verbatim_check --strict, cookbook_index --check, ghidra_roster --check, work_evidence --selftest, test_lzss, lint_symbol_refs — ≈45 s of checks) + job `compile-only` (binutils-mipsel + `cpp-mipsel-linux-gnu` from apt, cc1 from the tracked tarball sha256-checked, maspsx submodule; PR scope `main resident ov_SC01_077 md_MAIN_013`; `--all` weekly Mon 06:17 UTC + `workflow_dispatch`). Byte-identity is NOT proven in CI (needs the disc) — `docs/verification.md`. | +| **Public flip / CI** | `.github/workflows/no-rom.yml` | **(P33 B7) The ROM-free CI**: job `audits` (audit_public, audit_text_sources, verbatim_check --strict, cookbook_index --check (retired from `make tools-health` at P38 T1: its sources moved to `docs/retired/`; successor `cookbook/INDEX.md` via `tools/cookbook_add.sh`), ghidra_roster --check, work_evidence --selftest, test_lzss, lint_symbol_refs — ≈45 s of checks) + job `compile-only` (binutils-mipsel + `cpp-mipsel-linux-gnu` from apt, cc1 from the tracked tarball sha256-checked, maspsx submodule; PR scope `main resident ov_SC01_077 md_MAIN_013`; `--all` weekly Mon 06:17 UTC + `workflow_dispatch`). Byte-identity is NOT proven in CI (needs the disc) — `docs/verification.md`. | | | `tools/timeline.py [--check]` | **(P33 F1)** The progress timeline from the repository's own committed digests: every commit touching `docs/progress.fleet.md` / `docs/progress.md` (both historical formats parsed), one row per DATE (never keyed by hash — the rewrite changed them), PhaseEnd ticks from the headers, commits per day → `docs/story-timeline.md` + `docs/story-timeline.svg` (three polylines, phase ticks, the 07-22 denominator step annotated from the rows); self-check: the last row == `docs/progress.json`; `--check` = stale detection. 72 rows, 1.5 s. **Wired P33.5 task 7:** regenerated by `make report BINARY=main` (after `progress.py --json`), asserted fresh by `make audit-digest` (it had been wired to nothing and sat stale at the Phase-33 close). | | | `tools/mine_hindsight.py [--out …]` | **(P33 F2)** Gathers every recorded hindsight with `file:line` anchors — the decision-log's `Hindsight` bullets and `### Hindsight` sections (19 over 79 entries), the PhaseEnds' "What we believed…" sections (2), every Deviations table (237 rows / 32 PhaseEnds) → `.run/P33/hindsight.md` (scratch; `docs/retrospective.md` cites the sources, never the working set). Prints the census. | | **Docs** | `tools/doc_links.py [--strict] [--disk] [FILE…]` | **(P33 D5; extended P33.5 task 7)** Six checks, each printed with its denominator: (1) every relative Markdown link in the public-facing docs (the DEFAULT set + every wiki page and how-to chapter) resolves; targets listed in `docs/doc_links_pending.txt` (`pathcreating task`) count as PENDING, not broken — `--strict` (gate 2) refuses any pending entry; (2) nothing links into `docs/sunset/`; (3) a wiki page/chapter links into `docs/` only at a target the Reference index or the README links (the allow-list is DERIVED from those two pages; the two index pages are exempt); (4) every tracked `docs/` file outside wiki/how-to/sunset is covered the same way; (5) backticked `docs/…` / `.run/…` citations are TRACKED or UNTRACKED by `git ls-files` (never the disk) — a wiki page citing an UNTRACKED path fails, elsewhere it is counted; `--disk` adds the PRIVATE/DANGLING split for the maintainer; (6) wiki-first WARNINGS (exit 0) for a non-wiki document linking a `docs/` file whose topic has a wiki page. In `tools-health`. Controls: a broken link → rc 1; the P33.5 cookbook control (`--disk docs/matching-cookbook.md` listed the stale `.run/` citations before they were fixed). | diff --git a/docs/wiki/Archive-index.md b/docs/wiki/Archive-index.md index 8bedf1674e..7b81e37f7b 100644 --- a/docs/wiki/Archive-index.md +++ b/docs/wiki/Archive-index.md @@ -21,7 +21,7 @@ Records that stay — the PhaseEnds, the decision log, the cookbook — keep the | `frontier-p30.md`, `frontier-s39.md` | Point-in-time frontier reports (2026-07-30, 2026-08-05), never re-baselined by design | The frontier they described emptied | `PhaseEnd_Phase30/31.md`; `docs/progress.fleet.md` (218 of 218) | | `frontier-p32.md` | The last 21 functions and the five parked payloads, by blocker (2026-09-05) — the Phase-32 jumping-off document | All 21 banked and all 5 payloads onboarded; PhaseEnd_Phase31 asked that it be kept forever, so it is archived, never deleted | `PhaseEnd_Phase32.md`; the decision log P32 entries | | `recovery-queue-s65.md` | 69 unbanked wave targets classified into four lanes, "build it, run it next session" | Absorbed by the P31–P32 finish; 0 open near-misses at the P32 close | `docs/backlog.md` (0 open); `PhaseEnd_Phase31.md` | -| `next-session-triage-ladder.md` | The triage-ladder specification and its three corrections | Built and shipped (`tools/triage_ladder.py`); the corrections are cookbook §376/§378 | `docs/wave-playbook.md` §4b; `docs/SETUP.md`'s tool row | +| `next-session-triage-ladder.md` | The triage-ladder specification and its three corrections | Built and shipped (`tools/triage_ladder.py`); the corrections are cookbook §376/§378 | `docs/wave-playbook.md` §4b; `docs/ops/INDEX.md`'s tool row | ## Studies and syntheses superseded by the wiki @@ -45,8 +45,8 @@ Records that stay — the PhaseEnds, the decision log, the cookbook — keep the | File (`docs/sunset/…`) | What it was | What came of it | Where it lives now | |---|---|---|---| -| `distill/atbhbkbl.md` | 165 candidate idiom notes from waves at/bh/bk/bl, each disposed of with its A/B evidence | Landed as cookbook §265 (the verbatim-asm bank lane), §266 (the inert-rider law) and §267 (the addenda) — §267 cites this ledger as its evidence | `docs/matching-cookbook.md` §265–§267 | -| `distill/axbm.md` | 82 candidate notes from waves ax/bm | Landed as cookbook §268 (the register-pin law on call-clobbered registers) and §269 | `docs/matching-cookbook.md` §268–§269 | +| `distill/atbhbkbl.md` | 165 candidate idiom notes from waves at/bh/bk/bl, each disposed of with its A/B evidence | Landed as cookbook §265 (the verbatim-asm bank lane), §266 (the inert-rider law) and §267 (the addenda) — §267 cites this ledger as its evidence | `cookbook/INDEX.md` §265–§267 | +| `distill/axbm.md` | 82 candidate notes from waves ax/bm | Landed as cookbook §268 (the register-pin law on call-clobbered registers) and §269 | `cookbook/INDEX.md` §268–§269 | ## Session tool designs and analyses (Phase 31, sessions 58–61) @@ -81,8 +81,8 @@ documents (0 open stubs), so the last populated snapshot is kept instead of bein | File (`docs/sunset/history/…`) | What it was | Where it lives now | |---|---|---| -| `Fable5-ultracode-Full research findings-w42n9cp9t.output` | 216 KB of raw multi-agent research JSON from the first day (the sotn workflow, the Ghidra/PSX stack, WSL) | Everything actionable went into `docs/SETUP.md` and `docs/formats.md` in June 2026 | -| `debug menu link.txt`, `ghidra psx plugin link.txt`, `sotn-decomp wiki link.txt` | The original three reference URLs | `docs/SETUP.md` §2 and the memory map's source index | +| `Fable5-ultracode-Full research findings-w42n9cp9t.output` | 216 KB of raw multi-agent research JSON from the first day (the sotn workflow, the Ghidra/PSX stack, WSL) | Everything actionable went into `docs/ops/INDEX.md` and `docs/formats.md` in June 2026 | +| `debug menu link.txt`, `ghidra psx plugin link.txt`, `sotn-decomp wiki link.txt` | The original three reference URLs | `docs/ops/INDEX.md` §2 and the memory map's source index | ## Tools retired at Phase 33.5 (`tools/sunset/`, v1.32.1) diff --git a/docs/wiki/Contributing-and-the-no-ROM-policy.md b/docs/wiki/Contributing-and-the-no-ROM-policy.md index f2769db707..e263eea1d9 100644 --- a/docs/wiki/Contributing-and-the-no-ROM-policy.md +++ b/docs/wiki/Contributing-and-the-no-ROM-policy.md @@ -33,7 +33,7 @@ The matching frontier is empty, so new matches are not the need. The work that h are the type-lift, call-site cast and canonical-signature tools under `tools/`; the constraint is below. - **Shiftability** — a build whose addresses can move; the LZSS repack (recompression is not byte-stable, so this is a Gen3 problem) and asset tooling. -- **Tooling and documentation** — every tool has a row in [`docs/SETUP.md`](../SETUP.md)'s inventory; a tool change +- **Tooling and documentation** — every tool has a row in [`docs/ops/INDEX.md`](../ops/INDEX.md)'s inventory; a tool change ships with its siblings wired to use it and its docs in the same change. ## AI use — conduct diff --git a/docs/wiki/Docs-and-scratch-conventions.md b/docs/wiki/Docs-and-scratch-conventions.md index a072a66cfd..29e629e0a1 100644 --- a/docs/wiki/Docs-and-scratch-conventions.md +++ b/docs/wiki/Docs-and-scratch-conventions.md @@ -12,12 +12,12 @@ durable result it produced goes into one of these files, and the note is archive | Kind of knowledge | Where it goes | The rule behind it | |---|---|---| -| A compiler idiom proven on the bytes (the residual, the mechanism, the lever, the byte proof) | a numbered section of `docs/matching-cookbook.md`, and its symptom index `docs/cookbook-index.md` (regenerated, never edited) | R16 — consult the knowledge base before a match, feed the lesson back after | +| A compiler idiom proven on the bytes (the residual, the mechanism, the lever, the byte proof) | an entry in `cookbook/INDEX.md`, added with `tools/cookbook_add.sh` (never hand-edited) | R16 — consult the knowledge base before a match, feed the lesson back after | | A strategic pivot: what was believed, what failed, the measurement, the hindsight | an entry in `docs/decision-log.md`, written while fresh | R31 | | The post-100 % story (Gen3): the narrative of the readability phases | `docs/story.md` §10 and `docs/retrospective.md` §7, advanced at the END OF EVERY SESSION from that session's decision-log entry; `tools/timeline.py` draws the lever and readability series as the chart's lower panel | the owner's rule, 2026-09-12 | | A late discovery that would have sped up an earlier phase — what it is, when it was found, when it *could* have been found, what it would have saved | `docs/accelerators.md` | the ledger a future project starts from | | A procedure people run | a runbook — `docs/wave-playbook.md` for the matching campaign, `docs/public-flip-runbook.md` for the publication; a superseded runbook carries a banner at the top and is then archived | R21 for the ops reference | -| An environment or tool fact (a version, a flag, a hook, a row per tool) | `docs/SETUP.md`, in the same change as the tool | R21 | +| An environment or tool fact (a version, a flag, a hook, a row per tool) | `docs/ops/INDEX.md`, in the same change as the tool | R21 | | An address, with its source and its verification status | `docs/memory-map.md` | G5 | | A file or container format | `docs/formats.md` | — | | The state of the open phase; then the phase's synthesis; then the one-page digest every session starts from | `phase-ends/CURRENT_PHASE.md` → `phase-ends/PhaseEnd_Phase.md` → `phase-ends/DIGEST.md` | R64 | @@ -40,7 +40,7 @@ on its first line; edits go to the generator, not the file. |---|---|---| | `docs/progress.md`, `docs/progress.fleet.md`, `docs/progress.json`, the README's progress block, `docs/badges/*.json` | `tools/progress.py` | `make report BINARY=main`; `make audit-digest` asserts freshness | | `docs/difficulty.md`, `docs/duplicates.md`, `docs/duplicates.cross.md`, `docs/backlog.md` | `tools/difficulty.py`, `tools/dup_report.py`, `tools/backlog.py` | `make report BINARY=main` | -| `docs/cookbook-index.md` | `tools/cookbook_index.py` | `make tools-health` (`--check`) | +| `cookbook/INDEX.md` | `tools/cookbook_add.sh` | none (`cookbook_index.py --check` retired from `make tools-health`, P38 T1) | | `docs/disc-ledger.md` | `tools/disc_audit.py` | `make audit-disc` | | `docs/story-timeline.md`, `docs/story-timeline.svg` | `tools/timeline.py` | `make report BINARY=main`; `--check` in `make audit-digest` | | `docs/commit-map.tsv` | `tools/public_rewrite/build_commit_map.py` | one-shot, at the history rewrite | diff --git a/docs/wiki/Reference-index.md b/docs/wiki/Reference-index.md index 3c07e31f0b..212d956c0a 100644 --- a/docs/wiki/Reference-index.md +++ b/docs/wiki/Reference-index.md @@ -11,10 +11,13 @@ in the Archive index, not here. The conventions behind this page: [Docs and scra | File | What it is | How to read it | Written by | |---|---|---|---| -| [`SETUP.md`](../SETUP.md) | The environment reference: every tool's install row, the pinned compiler triple (§5.4), the daily command crib, the MCP lifecycle, the scratch conventions (§1a). 270 KB. | Grep by `§` or by tool name; §5.4 for the triple; the `## Tooling inventory` table for a tool's one-line row. | authored; kept current in the same change as any tool (R21) | +| [`ops/INDEX.md`](../ops/INDEX.md) | The environment reference: every tool's install row, the pinned compiler triple (§5.4), the daily command crib, the MCP lifecycle, the scratch conventions (§1a). 270 KB. | Grep by `§` or by tool name; §5.4 for the triple; the `## Tooling inventory` table for a tool's one-line row. | authored; kept current in the same change as any tool (R21) | +| [`ops/`](../ops/) | The environment reference split one page per section at the PA3 migration (2026-09); `ops/INDEX.md` lists every page, `ops/tooling-inventory.md` is the per-tool table. | Open `ops/INDEX.md`, then one page. | authored; kept current in the same change as any tool (R21) | +| [`retired/`](../retired/) | Files retired at the PA3 migration: the monolithic `SETUP.md`, `matching-cookbook.md` and its index, `effort-map.md`, the PA2-era files. Kept for history; never in an agent's load order. | Grep only. | frozen | +| [`project-architect.md`](../project-architect.md) | The Project Architect 3.0 methodology this repository runs under. | Read whole. | authored | | [`verification.md`](../verification.md) | The byte-identity contract and the recorded run that proved it (218 of 218 from a clean rebuild, both SDK legs). | Read whole; §2 quotes the run's summary from `.run/P33/verify/`. | authored; quotes `tools/verify_contract.sh`'s output | | [`second-oracle.md`](../second-oracle.md) | The independent boundary oracle (R34): what it covers, where it disagrees with the primary one by design. | Read whole. | authored; read by the Makefile and four tools | -| [`effort-map.md`](../effort-map.md) | The reasoning-effort doctrine for the AI agent: depth vs breadth, the per-phase map, the decision rule. Governs where `CLAUDE.md`'s tier language differs. | Read whole. | authored | +| [`retired/effort-map.md`](../retired/effort-map.md) | The reasoning-effort doctrine for the AI agent: depth vs breadth, the per-phase map, the decision rule. Governs where `CLAUDE.md`'s tier language differs. | Read whole. | authored | | [`doc_links_pending.txt`](../doc_links_pending.txt) | Link targets promised by a document and created by a later task of the same phase (R80). Empty at every phase close. | One `pathtask` per line. | authored per task | | [`public-flip-runbook.md`](../public-flip-runbook.md) | The history-rewrite and publication procedure, its controls, its risk register, and what the host kept serving afterwards (§11). | By section; §11 for the purge and the probe. | authored | @@ -33,8 +36,7 @@ in the Archive index, not here. The conventions behind this page: [Docs and scra | File | What it is | How to read it | Written by | |---|---|---|---| -| [`matching-cookbook.md`](../matching-cookbook.md) | Some 500 compiler idioms, each with its residual, mechanism, lever and byte proof. **3.5 MB — never read whole.** | Grep by `§` number or by symptom; the index below maps symptoms to sections. | authored, one section per proven idiom (R16) | -| [`cookbook-index.md`](../cookbook-index.md) | The symptom → section index of the cookbook. 560 KB. | Grep the symptom. | **generated** — `tools/cookbook_index.py`, checked in `make tools-health` | +| [`cookbook/INDEX.md`](../../cookbook/INDEX.md) | The cookbook since the PA3 migration: one line per technique, each a `cookbook/C.md` file (the monolith is in `docs/retired/`). Some 500 compiler idioms, each with its residual, mechanism, lever and byte proof. **3.5 MB — never read whole.** | Grep by `§` number or by symptom; the index below maps symptoms to sections. | authored, one section per proven idiom (R16) | | [`docs/tool-index.md`](../tool-index.md) | every tool, keyed by the NEED it answers, with what proved it (its consumers) and what it hard-codes; the retired ones last | generated by `tools/tool_census.py` from the tree and `config/tool_dictionary.tsv` (the authored phase/need/verdict); `--check` in tools-health; the same data generates the day-one kit's manifest and its verbatim tool corpus | read it by need: find the phrase that matches what you are trying to do | | [`gcc-2.7.2-map/`](../gcc-2.7.2-map/) | The codegen map of the pinned compiler by pass (scheduling, register allocation, loop, CSE/expression), with a citation audit; its README is the catalogue. | Start at the README's pass → residual → lever table. | authored; `tools/gccmap_cites.py --check` in tools-health | | [`wave-playbook.md`](../wave-playbook.md) | The matching-campaign procedure: draw, cards, packs, draft, gate, harvest, checkpoint — each guard paired with the measurement that produced it. | By numbered step; the S80 addenda for the one-agent-per-function shape. | authored | diff --git a/docs/wiki/Repository-layout.md b/docs/wiki/Repository-layout.md index 2abe436309..a1ed1ba45c 100644 --- a/docs/wiki/Repository-layout.md +++ b/docs/wiki/Repository-layout.md @@ -6,7 +6,7 @@ |---|---| | [`src/`](../../src/) | **The C.** One directory per binary: `src/` itself is the main executable (`SLUS_007.26`), `src/resident/` the always-resident engine, `src/ov_*/` the 141 overlays, `src/md_*/` the 75 code modules. [`src/shared/`](../../src/shared/) holds the bodies shared across binaries by [the dedup engine](The-dedup-engine.md) (`engine_core.h`, ~8.8 MB of `DEFINE_func_…()` macros — every translation unit that shares code includes it). Sony's library regions in main are `INCLUDE_ASM` tiles (`src/lib*.c`, `src/apicard*.c`); the five original hand-written routines are verbatim `__asm__` bodies. [`src/NOTICE.md`](../../src/NOTICE.md) states the licensing position. `src/shared/` holds every shared function body ONCE as a plain-C header (`ov/`, `slot_*/`, `main/`, Phase 35), included at each member's site — see [The dedup engine](The-dedup-engine.md). | | [`config/`](../../config/) | The build definition: one splat yaml per binary (`splat..yaml`), the symbol files (`symbols.us*.txt`, curated and build-valid — rule R15), the per-binary SHA1 contracts (`check..sha`), the overlay and module registries ([`overlays.mk`](../../config/overlays.mk), [`modules.mk`](../../config/modules.mk) — generated by the onboarding scripts, never edited by hand), the dedup registry ([`dedup.us.yaml`](../../config/dedup.us.yaml)), the verbatim-assembly manifest ([`verbatim_manifest.json`](../../config/verbatim_manifest.json)), the Ghidra annotations as text ([`config/ghidra/`](../../config/ghidra/)). | -| [`tools/`](../../tools/) | Everything the project built (about 236 Python tools, 25 shell tools, 12 Ghidra scripts): the disc extractor, the byte gates, the matching harness, the dedup and family engines, the campaign machinery, the audits, the public-flip rewrite package. The pinned submodules live here too (`maspsx`, `asm-differ`, `decomp-permuter`, `m2c`) and the tracked `cc1` tarballs (`tools/bin/`). [`tools/README.md`](../../tools/README.md) carries the one AGPL statement; [`docs/SETUP.md`](../SETUP.md)'s tooling inventory has a one-line row for every tool. | +| [`tools/`](../../tools/) | Everything the project built (about 236 Python tools, 25 shell tools, 12 Ghidra scripts): the disc extractor, the byte gates, the matching harness, the dedup and family engines, the campaign machinery, the audits, the public-flip rewrite package. The pinned submodules live here too (`maspsx`, `asm-differ`, `decomp-permuter`, `m2c`) and the tracked `cc1` tarballs (`tools/bin/`). [`tools/README.md`](../../tools/README.md) carries the one AGPL statement; [`docs/ops/INDEX.md`](../ops/INDEX.md)'s tooling inventory has a one-line row for every tool. | | [`docs/`](../) | The knowledge base and the record: the matching cookbook and its index, the gcc-2.7.2 codegen map, the memory map (addresses with provenance), the formats, the decision log, the accelerators, the wave playbook, the runbooks, the story, the retrospective, this wiki's sources (`docs/wiki/`, `docs/how-to-ai-decomp/`), the generated progress digests, badges and timeline, and the [Archive index](Archive-index.md) of closed-phase documents (their files left the tree at Phase 34; the history keeps them). Where each kind of knowledge goes, which files are generated, and how links are checked: [Docs and scratch conventions](Docs-and-scratch-conventions.md). | | [`phase-ends/`](../../phase-ends/) | The governance record: `PhaseEnd_Phase.md` for every phase, [`DIGEST.md`](../../phase-ends/DIGEST.md) (every phase's synopsis and every rule's full text), `CURRENT_PHASE.md` while a phase is open, `logs/` (each closed phase's working log, on demand only). | | [`include/`](../../include/) | Headers, including the four generic splat preset headers (`include_asm.h`, `macro.inc`, `labels.inc`, `gte_macros.inc`). | diff --git a/docs/wiki/The-matching-workflow.md b/docs/wiki/The-matching-workflow.md index 122960c4d3..aa61999726 100644 --- a/docs/wiki/The-matching-workflow.md +++ b/docs/wiki/The-matching-workflow.md @@ -129,8 +129,8 @@ earlier inline match never propagated — and a propagated body ships its signat ## The knowledge base, and feeding it -Before each match, consult [`docs/matching-cookbook.md`](../matching-cookbook.md) (grep it by symptom through -[`docs/cookbook-index.md`](../cookbook-index.md)); after each match, write the generalizable lesson back into the +Before each match, consult [`cookbook/INDEX.md`](../../cookbook/INDEX.md) (grep it by symptom through +[`cookbook/INDEX.md`](../../cookbook/INDEX.md)); after each match, write the generalizable lesson back into the cookbook *and* into the tooling, so the next similar function is a one-shot (rule R16). Two laws learned at cost: only harvest lessons from byte-proven results, and strip a credited lever from the accepted body and recompile before it enters the base — one in three credited levers was byte-inert. diff --git a/docs/wiki/Toolchain-setup.md b/docs/wiki/Toolchain-setup.md index 40c84ec13f..ac40030669 100644 --- a/docs/wiki/Toolchain-setup.md +++ b/docs/wiki/Toolchain-setup.md @@ -4,7 +4,7 @@ Everything runs inside one Linux environment (the project uses WSL2 Ubuntu 24.04 on a single ext4 clone; builds never run on a Windows-mounted path — the 9P bridge is slow and breaks file watching). The authoritative, evolvable reference -for every version, path and command is [`docs/SETUP.md`](../SETUP.md); this page is the map of it. +for every version, path and command is [`docs/ops/INDEX.md`](../ops/INDEX.md); this page is the map of it. ## The pinned triple — and why it is pinned diff --git a/phase-ends/current/logs/T1.c1.md b/phase-ends/current/logs/T1.c1.md new file mode 100644 index 0000000000..a0d9c59e4e --- /dev/null +++ b/phase-ends/current/logs/T1.c1.md @@ -0,0 +1,30 @@ +# T1.c1 — tools-health: repoint off docs/retired/ (PA3 migration b8a6cbc54) + +## Changed +- `tools/tool_census.py:14,42-43,51,143,331` — setup_rows() reads `docs/ops/tooling-inventory.md` (same `| group | tool | purpose |` shape, cells[2]); docstring updated. Beyond brief (needed for --check to get past the crash): COOKBOOK_SOURCES -> `docs/retired/{matching-cookbook,cookbook-index}.md`, RECORD_SOURCES effort-map -> `docs/retired/effort-map.md`; corpus_plan keeps the old corpus dest names (`replace("docs/retired/","docs/",1)`). Moves were pure renames (0 lines changed per `git diff --stat -M b8a6cbc54^ b8a6cbc54`). +- `Makefile:317-322` — `cookbook_index.py --check` commented out under a house DISABLED block (tool kept; CI `.github/workflows/no-rom.yml` still runs it, untouched). +- `tools/doc_links.py:38,54` — DEFAULT + TOPIC_PAGE entry `docs/SETUP.md` -> `docs/ops/INDEX.md` (kept, not dropped: Toolchain-setup.md still covers the topic). +- `tools/wiki_render.py:112` — selftest case g `/docs/SETUP.md` -> `/docs/ops/INDEX.md`. +- 36 BROKEN links retargeted: README.md:108, docs/gcc-2.7.2-map/README.md, docs/wiki/{Archive-index,Contributing-and-the-no-ROM-policy,Docs-and-scratch-conventions,Reference-index,Repository-layout,The-matching-workflow,Toolchain-setup}.md, docs/how-to-ai-decomp/{00-README,01-governance,06-knowledge-base,08-models-and-budgets}.md. Mapping per brief. Reference-index: the matching-cookbook row now describes cookbook/INDEX.md; the cookbook-index row dropped (would duplicate it). Docs-and-scratch-conventions rows reworded to cookbook/INDEX.md + cookbook_add.sh. +- `docs/wiki/Reference-index.md` — 3 new rows (`ops/`, `retired/`, `project-architect.md`): after the links were fixed doc_links still rc 1 on 60 UNCOVERED docs files (53 docs/ops/*, 6 docs/retired/*, docs/project-architect.md); a directory link covers its files (covered(), doc_links.py:128). +- `docs/ops/tooling-inventory.md:21` — note cookbook_index retired from tools-health (no own row exists; the CI row is the only one naming it). +- docs/wiki/Home.md: no `SETUP` link present at HEAD (grep); nothing to change. The dead link was the wiki_render selftest case. + +## Verified +- `.venv/bin/python tools/doc_links.py` -> rc 0, `0 broken`, coverage 126 of 126, `doc_links: OK` +- `.venv/bin/python tools/wiki_render.py --selftest` -> rc 0, 12 cases 0 failed +- `.venv/bin/python tools/kit_lint.py` -> rc 0 `kit_lint: OK` +- `.venv/bin/python tools/kit_coverage.py` -> rc 0 `kit_coverage: OK` +- `.venv/bin/python tools/tool_census.py --check` -> rc 1, 65 gaps (NOT green; see Blocker) +- `make -n tools-health >/dev/null` -> rc 2 at `Makefile:516: sdk-dual`; control with HEAD Makefile (`make -n -f .run/t1c1/Makefile.head tools-health`) fails identically -> pre-existing, not this change. + +## Blocker: tool_census --check +Past the SETUP crash, 65 gaps, all PA3-migration drift outside the brief: +- 22 `no dictionary row` — new PA3 tools (card.py, commit_task.sh, cookbook_add.sh, plan_edit.py, run.sh, outline.py, launch.py, ...) lack rows in `config/tool_dictionary.tsv` (authored data, 8 cols each). +- 43 corpus gaps in `decomp-architect/corpus/` (generated): missing copies for those tools, stale copies of changed tools/docs, missing `record/phase-ends/PhaseEnd_Phase37{,.5}.md`. Fix = `tools/tool_census.py --corpus` after the rows exist; that copies the untracked `phase-ends/PhaseEnd_Phase37.md` into the kit (brief forbids touching it) -> expert decision. + +## Incident (expert must know) +`make -n tools-health` is NOT a dry run here: `.ONESHELL` + a recipe containing `$(MAKE)` makes make execute the whole recipe; sub-makes stay dry (-n in MAKEFLAGS) but plain shell lines run. Within sdk-dual that ran `rm -rf build/psyq` and two `cp $(main_MAPFILE) .run/P33/verify/main_{with,no}_sdk.map` (19:40, after the in-flight tools-health1 had printed `sdk-dual: OK`). Ran 3x (twice via tools-health, once `make -n sdk-dual` while diagnosing). +- `.run/P33/verify/main_no_sdk.map` (tracked) was clobbered with the WITH map -> restored with `git checkout --` (clean now). +- `build/psyq/` is gone. Restore = leg 3 of sdk-dual: `make extract BINARY=main && make check BINARY=main` (not run: brief forbids make builds and a run is in flight). The in-flight tools-health1 was past sdk-dual (at `report`). +- generalizable: `make -n` on a `.ONESHELL` target whose recipe calls `$(MAKE)` executes the recipe's non-make lines; the brief's `make -n tools-health` check is destructive on an SDK machine. diff --git a/tools/doc_links.py b/tools/doc_links.py index 6af940b5c1..5329995caf 100644 --- a/tools/doc_links.py +++ b/tools/doc_links.py @@ -35,7 +35,7 @@ import subprocess import sys REPO = pathlib.Path(__file__).resolve().parent.parent -DEFAULT = ["README.md", "THIRD_PARTY.md", "CLAUDE.md", "src/NOTICE.md", "tools/README.md", "docs/SETUP.md", +DEFAULT = ["README.md", "THIRD_PARTY.md", "CLAUDE.md", "src/NOTICE.md", "tools/README.md", "docs/ops/INDEX.md", "docs/verification.md", "docs/public-flip-runbook.md", "docs/decision-log.md", "docs/accelerators.md", "docs/story.md", "docs/story-timeline.md", "docs/retrospective.md", "phase-ends/README.md", "phase-ends/DIGEST.md", "docs/gcc-2.7.2-map/README.md", "tools/xsig/README.md", @@ -51,7 +51,7 @@ INDEX_PAGES = ("docs/wiki/Reference-index.md", "docs/wiki/Archive-index.md") ARCHIVE_DIR = "docs/sunset/" # the wiki page that covers a docs/ file's topic — for the wiki-first WARNING only (a hand-kept map; small on purpose) TOPIC_PAGE = { - "docs/SETUP.md": "docs/wiki/Toolchain-setup.md", + "docs/ops/INDEX.md": "docs/wiki/Toolchain-setup.md", "docs/verification.md": "docs/wiki/Verification-and-progress.md", "docs/public-flip-runbook.md": "docs/wiki/The-ROM-firewall.md", "docs/gen3-standards.md": "docs/wiki/Where-the-project-goes-next.md", diff --git a/tools/tool_census.py b/tools/tool_census.py index 8a7185b1c4..c489d1800b 100644 --- a/tools/tool_census.py +++ b/tools/tool_census.py @@ -11,7 +11,7 @@ two verbatim corpora, all from one hand-kept dictionary (P33.5 task 13.5; `--che Two kinds of fact, deliberately separated (R33): DERIVED on every run from the tree — the file set (two independent enumerations that must agree: `find` and `git ls-files`, - R34), each tool's docstring line, its SETUP row (`docs/SETUP.md`, matched by file name), its CONSUMERS (Makefile + config/*.mk + R34), each tool's docstring line, its SETUP row (`docs/ops/tooling-inventory.md`, matched by file name), its CONSUMERS (Makefile + config/*.mk targets, the CI workflows, the wave playbook, and every other tool that imports it or names it), and hence its class: LIVE (a runtime consumer), REFERENCED (only a SETUP/doc row), ORPHAN (neither). AUTHORED once, in `config/tool_dictionary.tsv` (tab-separated, LF line endings, the eight columns below) — the ladder phase, the portability (PORTABLE / ADAPT / PROJECT-ONLY), the NEED @@ -39,7 +39,8 @@ KIT = REPO / "decomp-architect" MANIFEST = KIT / "tools" / "MANIFEST.md" CORPUS_TOOLS = KIT / "corpus" / "tools" CORPUS_COOK = KIT / "corpus" / "cookbook" -COOKBOOK_SOURCES = ["docs/matching-cookbook.md", "docs/cookbook-index.md"] + [p.relative_to(REPO).as_posix() for p in sorted((REPO / "docs" / "gcc-2.7.2-map").glob("*")) if p.is_file()] +# the two cookbook files moved to docs/retired/ at the PA3 migration (b8a6cbc54, bytes unchanged); the corpus keeps their old names +COOKBOOK_SOURCES = ["docs/retired/matching-cookbook.md", "docs/retired/cookbook-index.md"] +[p.relative_to(REPO).as_posix() for p in sorted((REPO / "docs" / "gcc-2.7.2-map").glob("*")) if p.is_file()] # P33.5 task 14.5 (Drew: "the whole of our experience"): the RECORD as the third dictionary — the distilled records and the # phase-by-phase record, verbatim. The phase worklogs (phase-ends/logs/) are deliberately NOT here (R19: on-demand archives, 30k # lines); their distillation is the task-14.5 mining pass, banked into the kernels/accelerators. Layout under corpus/record/: @@ -47,7 +48,7 @@ COOKBOOK_SOURCES = ["docs/matching-cookbook.md", "docs/cookbook-index.md"] + [p. CORPUS_RECORD = KIT / "corpus" / "record" RECORD_SOURCES = ([p.relative_to(REPO).as_posix() for p in sorted((REPO / "docs" / "how-to-ai-decomp").glob("*.md"))] + ["docs/decision-log.md", "docs/accelerators.md", "docs/retrospective.md", "docs/story.md", "docs/wave-playbook.md", - "docs/effort-map.md", "docs/gen3-standards.md", "docs/gen3-handoff.md", "phase-ends/DIGEST.md"] + "docs/retired/effort-map.md", "docs/gen3-standards.md", "docs/gen3-handoff.md", "phase-ends/DIGEST.md"] + [p.relative_to(REPO).as_posix() for p in sorted((REPO / "phase-ends").glob("PhaseEnd_Phase*.md"), key=lambda q: [float(x) if x.replace(".", "").isdigit() else x for x in re.split(r"(\d+(?:\.\d+)?)", q.stem)])]) @@ -139,7 +140,7 @@ def docstring_line(path): def setup_rows(): """file name -> the SETUP row's purpose cell (first 160 chars).""" out = {} - for ln in (REPO / "docs" / "SETUP.md").read_text(encoding="utf-8", errors="replace").splitlines(): + for ln in (REPO / "docs" / "ops" / "tooling-inventory.md").read_text(encoding="utf-8", errors="replace").splitlines(): if not ln.startswith("|"): continue cells = [c.strip() for c in ln.strip().strip("|").split("|")] @@ -327,7 +328,7 @@ def corpus_plan(recs): dest = CORPUS_TOOLS / r["phase"] / r["path"].removeprefix("tools/") copies.append((REPO / r["path"], dest)) for s in COOKBOOK_SOURCES: - copies.append((REPO / s, CORPUS_COOK / pathlib.Path(s).relative_to("docs"))) + copies.append((REPO / s, CORPUS_COOK / pathlib.Path(s.replace("docs/retired/", "docs/", 1)).relative_to("docs"))) for s in RECORD_SOURCES: copies.append((REPO / s, record_dest(s))) return copies, pointers diff --git a/tools/wiki_render.py b/tools/wiki_render.py index a8859c6754..f9958026b2 100644 --- a/tools/wiki_render.py +++ b/tools/wiki_render.py @@ -109,7 +109,7 @@ def selftest(): ("[d](../verification.md)", f"[d]({GH}/blob/main/docs/verification.md)"), ("[e](../../README.md#license)", f"[e]({GH}/blob/main/README.md#license)"), ("[f](../../tools/)", f"[f]({GH}/tree/main/tools)"), - ("[g](/docs/SETUP.md)", f"[g]({GH}/blob/main/docs/SETUP.md)"), + ("[g](/docs/ops/INDEX.md)", f"[g]({GH}/blob/main/docs/ops/INDEX.md)"), ("![h](../story-timeline.svg)", f"![h]({RAW}/docs/story-timeline.svg)"), ("[i](https://example.org/x.md)", "[i](https://example.org/x.md)"), ("[j](#anchor)", "[j](#anchor)"),