T1.c1: tools-health off docs/retired: census reads ops inventory, links fixed, cookbook_index off

This commit is contained in:
Drew T
2026-09-29 19:42:44 -06:00
parent 7e6e5dda0d
commit db68150f5e
19 changed files with 72 additions and 34 deletions
+6 -1
View File
@@ -314,7 +314,12 @@ tools-health:
$(MAKE) --no-print-directory audit-digest $(MAKE) --no-print-directory audit-digest
# The cookbook index is DERIVED (R33) and self-asserts its coverage (R32). Stale = agents can't # The cookbook index is DERIVED (R33) and self-asserts its coverage (R32). Stale = agents can't
# find documented idioms and re-derive them at full token cost (measured, P30 wave 1). # find documented idioms and re-derive them at full token cost (measured, P30 wave 1).
$(VENV_PY) tools/cookbook_index.py --check # DISABLED: cookbook_index.py --check — 2026-09-29 / Phase 38 T1
# Original intent: assert docs/cookbook-index.md is fresh against docs/matching-cookbook.md.
# Why disabled: both files moved to docs/retired/ at the PA3 migration (b8a6cbc54); the check crashed on the missing
# source. Successor: cookbook/INDEX.md, kept by tools/cookbook_add.sh.
# Re-enable if: docs/matching-cookbook.md becomes a live source again.
# $(VENV_PY) tools/cookbook_index.py --check
$(VENV_PY) tools/gccmap_cites.py --check $(VENV_PY) tools/gccmap_cites.py --check
# P33 B5: the Ghidra roster is DERIVED from config/ghidra/*.jsonl (R33); a stale roster misreports # P33 B5: the Ghidra roster is DERIVED from config/ghidra/*.jsonl (R33); a stale roster misreports
# which programs' RE work is tracked as text. Pure text check, no Ghidra needed. # which programs' RE work is tracked as text. Pure text check, no Ghidra needed.
+1 -1
View File
@@ -105,7 +105,7 @@ The reverse-engineering side ran on Ghidra driven over MCP, with a PCSX-Redux ru
byte gate over the whole binary, a permuter, a compiler-idiom cookbook, and — decisively — reading the gcc 2.7.2 source byte gate over the whole binary, a permuter, a compiler-idiom cookbook, and — decisively — reading the gcc 2.7.2 source
to explain the bytes the compiler emits. Everything that was learned is written down where it can be reused: to explain the bytes the compiler emits. Everything that was learned is written down where it can be reused:
- [`docs/matching-cookbook.md`](docs/matching-cookbook.md) — the idioms, each byte-proven, and the compiler-internals laws - [`cookbook/INDEX.md`](cookbook/INDEX.md) — the idioms, each byte-proven, and the compiler-internals laws
- [`docs/gcc-2.7.2-map/`](docs/gcc-2.7.2-map/) — the codegen map of the pinned compiler, by pass - [`docs/gcc-2.7.2-map/`](docs/gcc-2.7.2-map/) — the codegen map of the pinned compiler, by pass
- [`docs/decision-log.md`](docs/decision-log.md) — every strategic pivot with its measurement and its hindsight - [`docs/decision-log.md`](docs/decision-log.md) — every strategic pivot with its measurement and its hindsight
- [`docs/accelerators.md`](docs/accelerators.md) — what would have made earlier phases faster - [`docs/accelerators.md`](docs/accelerators.md) — what would have made earlier phases faster
+1 -1
View File
@@ -6,7 +6,7 @@ that moves it — each lever **byte-proven** on a named function with the diff b
Phase 23 by four frontier-class agents reading the gcc 2.7.2 passes directly, after six phases in which the same Phase 23 by four frontier-class agents reading the gcc 2.7.2 passes directly, after six phases in which the same
residuals had been called "unsteerable"; it is the artifact that lets a mid-tier agent, or a local model, apply residuals had been called "unsteerable"; it is the artifact that lets a mid-tier agent, or a local model, apply
compiler-internal levers without reading 80,000 lines of source. The index and triage table live in the cookbook compiler-internal levers without reading 80,000 lines of source. The index and triage table live in the cookbook
([`docs/matching-cookbook.md`](../matching-cookbook.md) §31); the method is chapter 07 of the how-to ([`cookbook/INDEX.md`](../../cookbook/INDEX.md) §31); the method is chapter 07 of the how-to
([`docs/how-to-ai-decomp/07-compiler-source.md`](../how-to-ai-decomp/07-compiler-source.md)). ([`docs/how-to-ai-decomp/07-compiler-source.md`](../how-to-ai-decomp/07-compiler-source.md)).
## The files ## The files
+1 -1
View File
@@ -67,7 +67,7 @@ the machine that builds it is universal.** That is the whole point of these chap
- [`docs/accelerators.md`](../accelerators.md) — what was learned late that day one should know, each entry dated with when it *could* have been found - [`docs/accelerators.md`](../accelerators.md) — what was learned late that day one should know, each entry dated with when it *could* have been found
- [`docs/retrospective.md`](../retrospective.md) and [`docs/story.md`](../story.md) — what we believed, what failed, what it cost; and the narrative - [`docs/retrospective.md`](../retrospective.md) and [`docs/story.md`](../story.md) — what we believed, what failed, what it cost; and the narrative
- [`docs/wave-playbook.md`](../wave-playbook.md) — the running procedure, each guard paired with the measurement that earned it - [`docs/wave-playbook.md`](../wave-playbook.md) — the running procedure, each guard paired with the measurement that earned it
- [`docs/decision-log.md`](../decision-log.md) — every pivot with its measurement and hindsight; [`docs/effort-map.md`](../effort-map.md) — the reasoning-effort doctrine - [`docs/decision-log.md`](../decision-log.md) — every pivot with its measurement and hindsight; [`docs/retired/effort-map.md`](../retired/effort-map.md) — the reasoning-effort doctrine
- [`phase-ends/DIGEST.md`](../../phase-ends/DIGEST.md) — every phase's synopsis and the full text of every rule cited here as `R<n>` - [`phase-ends/DIGEST.md`](../../phase-ends/DIGEST.md) — every phase's synopsis and the full text of every rule cited here as `R<n>`
The framework the governance chapter describes is published separately as The framework the governance chapter describes is published separately as
+1 -1
View File
@@ -86,7 +86,7 @@ The last thirty sessions of the project resumed from the digest and the checkpoi
## Effort, and who toggles it ## Effort, and who toggles it
The orchestrating agent's reasoning depth is a setting the human controls, and the project mapped it per task The orchestrating agent's reasoning depth is a setting the human controls, and the project mapped it per task
([`docs/effort-map.md`](../effort-map.md)): the deepest setting for phase plans, PhaseEnds, architectural forks and ([`docs/retired/effort-map.md`](../retired/effort-map.md)): the deepest setting for phase plans, PhaseEnds, architectural forks and
non-obvious debugging; the highest *persistent* setting for execution whose design is already settled; multi-agent non-obvious debugging; the highest *persistent* setting for execution whose design is already settled; multi-agent
breadth (many independent items) for surveys, audits and bulk matching. The rule that made this workable is R27: breadth (many independent items) for surveys, audits and bulk matching. The rule that made this workable is R27:
**every transition is prompted and waited for** — the agent never launches a fan-out on a verbal yes, and never rolls **every transition is prompted and waited for** — the agent never launches a fan-out on a verbal yes, and never rolls
+2 -2
View File
@@ -8,8 +8,8 @@ behaviour; yours will be about your compiler. The machine that builds it is what
| Layer | BFM's instance | What it holds | | Layer | BFM's instance | What it holds |
|---|---|---| |---|---|---|
| **The cookbook** | [`docs/matching-cookbook.md`](../matching-cookbook.md) (3.5 MB; grep it by section, never read it whole) | one section per idiom or law, each **byte-proven** on a named function with the diff before and after; the compiler-internals laws; the integration classes; the wall proofs and their later refutations | | **The cookbook** | [`cookbook/INDEX.md`](../../cookbook/INDEX.md) (3.5 MB; grep it by section, never read it whole) | one section per idiom or law, each **byte-proven** on a named function with the diff before and after; the compiler-internals laws; the integration classes; the wall proofs and their later refutations |
| **The symptom index** | [`docs/cookbook-index.md`](../cookbook-index.md), derived by a self-asserting tool | *symptom → section*: what a residual looks like in the diff, and where the lever is | | **The symptom index** | [`cookbook/INDEX.md`](../../cookbook/INDEX.md), derived by a self-asserting tool | *symptom → section*: what a residual looks like in the diff, and where the lever is |
| **The codegen map** | [`docs/gcc-2.7.2-map/`](../gcc-2.7.2-map/) by compiler pass | pass → residual pattern → C-level lever, from reading the compiler's source ([chapter 07](07-compiler-source.md)) | | **The codegen map** | [`docs/gcc-2.7.2-map/`](../gcc-2.7.2-map/) by compiler pass | pass → residual pattern → C-level lever, from reading the compiler's source ([chapter 07](07-compiler-source.md)) |
| **The pinned triple and the permuter recipes** | the build, `tools/permuter/` | the executable form of the same knowledge: what the tools try before an agent is asked | | **The pinned triple and the permuter recipes** | the build, `tools/permuter/` | the executable form of the same knowledge: what the tools try before an agent is asked |
| **The journal** | per-function past-attempt notes | what every earlier agent tried on *this* function — pack fuel | | **The journal** | per-function past-attempt notes | what every earlier agent tried on *this* function — pack fuel |
@@ -46,7 +46,7 @@ differently.
## The orchestrator's effort — depth vs. breadth ## The orchestrator's effort — depth vs. breadth
The orchestrating agent's reasoning depth is a human-controlled setting, and it maps onto two axes The orchestrating agent's reasoning depth is a human-controlled setting, and it maps onto two axes
([`docs/effort-map.md`](../effort-map.md)): ([`docs/retired/effort-map.md`](../retired/effort-map.md)):
| | Depth — how hard one agent thinks | Breadth — how many angles at once | | | Depth — how hard one agent thinks | Breadth — how many angles at once |
|---|---|---| |---|---|---|
+1 -1
View File
@@ -18,7 +18,7 @@ Every script under `tools/` (plus the two report make-targets), grouped by purpo
| | `DecompileAt.java` | Decompile the function at a given address (scripted scaffold). | | | `DecompileAt.java` | Decompile the function at a given address (scripted scaffold). |
| | `DefineFunctions.java` | Disassemble + create functions at splat's validated entry points (`.run/<prog>_funcs.txt`) — completes a raw-blob program's function set (Phase 10). | | | `DefineFunctions.java` | Disassemble + create functions at splat's validated entry points (`.run/<prog>_funcs.txt`) — completes a raw-blob program's function set (Phase 10). |
| | `ApplySymbols.java` + `tools/ghidra_apply_symbols.sh` | **(P31 S78) The Ghidra MIRROR of the curated symbol file (R15/G6), headless with a real save.** `tools/ghidra_apply_symbols.sh [PROG] [symbols files…]` (defaults `SLUS_007.26 config/symbols.us.txt`; MCP must be STOPPED first) reads `name = 0xADDR;` rows and sets every function/label to its curated name; a name held by another address is moved to that address's own curated name first (`firstfile`/`firstfile2`), else to `<name>__at_<addr>`. Idempotent; prints `BFMAPPLY renamed_funcs=… unchanged=…`; R9-verify with `ghidra_mcp_verify.sh`. **Use this, not MCP `rename_symbol`/`batch_rename`, for renames:** S78 observed 47 MCP renames NOT persisting through the sentinel stop ("Save succeeded", DB grew, names gone — R9 caught it; cause not yet isolated), while the postScript path persisted 73/73 on the first run. | | | `ApplySymbols.java` + `tools/ghidra_apply_symbols.sh` | **(P31 S78) The Ghidra MIRROR of the curated symbol file (R15/G6), headless with a real save.** `tools/ghidra_apply_symbols.sh [PROG] [symbols files…]` (defaults `SLUS_007.26 config/symbols.us.txt`; MCP must be STOPPED first) reads `name = 0xADDR;` rows and sets every function/label to its curated name; a name held by another address is moved to that address's own curated name first (`firstfile`/`firstfile2`), else to `<name>__at_<addr>`. Idempotent; prints `BFMAPPLY renamed_funcs=… unchanged=…`; R9-verify with `ghidra_mcp_verify.sh`. **Use this, not MCP `rename_symbol`/`batch_rename`, for renames:** S78 observed 47 MCP renames NOT persisting through the sentinel stop ("Save succeeded", DB grew, names gone — R9 caught it; cause not yet isolated), while the postScript path persisted 73/73 on the first run. |
| **Public flip / CI** | `.github/workflows/no-rom.yml` | **(P33 B7) The ROM-free CI**: job `audits` (audit_public, audit_text_sources, verbatim_check --strict, cookbook_index --check, ghidra_roster --check, work_evidence --selftest, test_lzss, lint_symbol_refs — ≈45 s of checks) + job `compile-only` (binutils-mipsel + `cpp-mipsel-linux-gnu` from apt, cc1 from the tracked tarball sha256-checked, maspsx submodule; PR scope `main resident ov_SC01_077 md_MAIN_013`; `--all` weekly Mon 06:17 UTC + `workflow_dispatch`). Byte-identity is NOT proven in CI (needs the disc) — `docs/verification.md`. | | **Public flip / CI** | `.github/workflows/no-rom.yml` | **(P33 B7) The ROM-free CI**: job `audits` (audit_public, audit_text_sources, verbatim_check --strict, cookbook_index --check (retired from `make tools-health` at P38 T1: its sources moved to `docs/retired/`; successor `cookbook/INDEX.md` via `tools/cookbook_add.sh`), ghidra_roster --check, work_evidence --selftest, test_lzss, lint_symbol_refs — ≈45 s of checks) + job `compile-only` (binutils-mipsel + `cpp-mipsel-linux-gnu` from apt, cc1 from the tracked tarball sha256-checked, maspsx submodule; PR scope `main resident ov_SC01_077 md_MAIN_013`; `--all` weekly Mon 06:17 UTC + `workflow_dispatch`). Byte-identity is NOT proven in CI (needs the disc) — `docs/verification.md`. |
| | `tools/timeline.py [--check]` | **(P33 F1)** The progress timeline from the repository's own committed digests: every commit touching `docs/progress.fleet.md` / `docs/progress.md` (both historical formats parsed), one row per DATE (never keyed by hash — the rewrite changed them), PhaseEnd ticks from the headers, commits per day → `docs/story-timeline.md` + `docs/story-timeline.svg` (three polylines, phase ticks, the 07-22 denominator step annotated from the rows); self-check: the last row == `docs/progress.json`; `--check` = stale detection. 72 rows, 1.5 s. **Wired P33.5 task 7:** regenerated by `make report BINARY=main` (after `progress.py --json`), asserted fresh by `make audit-digest` (it had been wired to nothing and sat stale at the Phase-33 close). | | | `tools/timeline.py [--check]` | **(P33 F1)** The progress timeline from the repository's own committed digests: every commit touching `docs/progress.fleet.md` / `docs/progress.md` (both historical formats parsed), one row per DATE (never keyed by hash — the rewrite changed them), PhaseEnd ticks from the headers, commits per day → `docs/story-timeline.md` + `docs/story-timeline.svg` (three polylines, phase ticks, the 07-22 denominator step annotated from the rows); self-check: the last row == `docs/progress.json`; `--check` = stale detection. 72 rows, 1.5 s. **Wired P33.5 task 7:** regenerated by `make report BINARY=main` (after `progress.py --json`), asserted fresh by `make audit-digest` (it had been wired to nothing and sat stale at the Phase-33 close). |
| | `tools/mine_hindsight.py [--out …]` | **(P33 F2)** Gathers every recorded hindsight with `file:line` anchors — the decision-log's `Hindsight` bullets and `### Hindsight` sections (19 over 79 entries), the PhaseEnds' "What we believed…" sections (2), every Deviations table (237 rows / 32 PhaseEnds) → `.run/P33/hindsight.md` (scratch; `docs/retrospective.md` cites the sources, never the working set). Prints the census. | | | `tools/mine_hindsight.py [--out …]` | **(P33 F2)** Gathers every recorded hindsight with `file:line` anchors — the decision-log's `Hindsight` bullets and `### Hindsight` sections (19 over 79 entries), the PhaseEnds' "What we believed…" sections (2), every Deviations table (237 rows / 32 PhaseEnds) → `.run/P33/hindsight.md` (scratch; `docs/retrospective.md` cites the sources, never the working set). Prints the census. |
| **Docs** | `tools/doc_links.py [--strict] [--disk] [FILE…]` | **(P33 D5; extended P33.5 task 7)** Six checks, each printed with its denominator: (1) every relative Markdown link in the public-facing docs (the DEFAULT set + every wiki page and how-to chapter) resolves; targets listed in `docs/doc_links_pending.txt` (`path<TAB>creating task`) count as PENDING, not broken — `--strict` (gate 2) refuses any pending entry; (2) nothing links into `docs/sunset/`; (3) a wiki page/chapter links into `docs/` only at a target the Reference index or the README links (the allow-list is DERIVED from those two pages; the two index pages are exempt); (4) every tracked `docs/` file outside wiki/how-to/sunset is covered the same way; (5) backticked `docs/…` / `.run/…` citations are TRACKED or UNTRACKED by `git ls-files` (never the disk) — a wiki page citing an UNTRACKED path fails, elsewhere it is counted; `--disk` adds the PRIVATE/DANGLING split for the maintainer; (6) wiki-first WARNINGS (exit 0) for a non-wiki document linking a `docs/` file whose topic has a wiki page. In `tools-health`. Controls: a broken link → rc 1; the P33.5 cookbook control (`--disk docs/matching-cookbook.md` listed the stale `.run/` citations before they were fixed). | | **Docs** | `tools/doc_links.py [--strict] [--disk] [FILE…]` | **(P33 D5; extended P33.5 task 7)** Six checks, each printed with its denominator: (1) every relative Markdown link in the public-facing docs (the DEFAULT set + every wiki page and how-to chapter) resolves; targets listed in `docs/doc_links_pending.txt` (`path<TAB>creating task`) count as PENDING, not broken — `--strict` (gate 2) refuses any pending entry; (2) nothing links into `docs/sunset/`; (3) a wiki page/chapter links into `docs/` only at a target the Reference index or the README links (the allow-list is DERIVED from those two pages; the two index pages are exempt); (4) every tracked `docs/` file outside wiki/how-to/sunset is covered the same way; (5) backticked `docs/…` / `.run/…` citations are TRACKED or UNTRACKED by `git ls-files` (never the disk) — a wiki page citing an UNTRACKED path fails, elsewhere it is counted; `--disk` adds the PRIVATE/DANGLING split for the maintainer; (6) wiki-first WARNINGS (exit 0) for a non-wiki document linking a `docs/` file whose topic has a wiki page. In `tools-health`. Controls: a broken link → rc 1; the P33.5 cookbook control (`--disk docs/matching-cookbook.md` listed the stale `.run/` citations before they were fixed). |
+5 -5
View File
@@ -21,7 +21,7 @@ Records that stay — the PhaseEnds, the decision log, the cookbook — keep the
| `frontier-p30.md`, `frontier-s39.md` | Point-in-time frontier reports (2026-07-30, 2026-08-05), never re-baselined by design | The frontier they described emptied | `PhaseEnd_Phase30/31.md`; `docs/progress.fleet.md` (218 of 218) | | `frontier-p30.md`, `frontier-s39.md` | Point-in-time frontier reports (2026-07-30, 2026-08-05), never re-baselined by design | The frontier they described emptied | `PhaseEnd_Phase30/31.md`; `docs/progress.fleet.md` (218 of 218) |
| `frontier-p32.md` | The last 21 functions and the five parked payloads, by blocker (2026-09-05) — the Phase-32 jumping-off document | All 21 banked and all 5 payloads onboarded; PhaseEnd_Phase31 asked that it be kept forever, so it is archived, never deleted | `PhaseEnd_Phase32.md`; the decision log P32 entries | | `frontier-p32.md` | The last 21 functions and the five parked payloads, by blocker (2026-09-05) — the Phase-32 jumping-off document | All 21 banked and all 5 payloads onboarded; PhaseEnd_Phase31 asked that it be kept forever, so it is archived, never deleted | `PhaseEnd_Phase32.md`; the decision log P32 entries |
| `recovery-queue-s65.md` | 69 unbanked wave targets classified into four lanes, "build it, run it next session" | Absorbed by the P31–P32 finish; 0 open near-misses at the P32 close | `docs/backlog.md` (0 open); `PhaseEnd_Phase31.md` | | `recovery-queue-s65.md` | 69 unbanked wave targets classified into four lanes, "build it, run it next session" | Absorbed by the P31–P32 finish; 0 open near-misses at the P32 close | `docs/backlog.md` (0 open); `PhaseEnd_Phase31.md` |
| `next-session-triage-ladder.md` | The triage-ladder specification and its three corrections | Built and shipped (`tools/triage_ladder.py`); the corrections are cookbook §376/§378 | `docs/wave-playbook.md` §4b; `docs/SETUP.md`'s tool row | | `next-session-triage-ladder.md` | The triage-ladder specification and its three corrections | Built and shipped (`tools/triage_ladder.py`); the corrections are cookbook §376/§378 | `docs/wave-playbook.md` §4b; `docs/ops/INDEX.md`'s tool row |
## Studies and syntheses superseded by the wiki ## Studies and syntheses superseded by the wiki
@@ -45,8 +45,8 @@ Records that stay — the PhaseEnds, the decision log, the cookbook — keep the
| File (`docs/sunset/…`) | What it was | What came of it | Where it lives now | | File (`docs/sunset/…`) | What it was | What came of it | Where it lives now |
|---|---|---|---| |---|---|---|---|
| `distill/atbhbkbl.md` | 165 candidate idiom notes from waves at/bh/bk/bl, each disposed of with its A/B evidence | Landed as cookbook §265 (the verbatim-asm bank lane), §266 (the inert-rider law) and §267 (the addenda) — §267 cites this ledger as its evidence | `docs/matching-cookbook.md` §265–§267 | | `distill/atbhbkbl.md` | 165 candidate idiom notes from waves at/bh/bk/bl, each disposed of with its A/B evidence | Landed as cookbook §265 (the verbatim-asm bank lane), §266 (the inert-rider law) and §267 (the addenda) — §267 cites this ledger as its evidence | `cookbook/INDEX.md` §265–§267 |
| `distill/axbm.md` | 82 candidate notes from waves ax/bm | Landed as cookbook §268 (the register-pin law on call-clobbered registers) and §269 | `docs/matching-cookbook.md` §268–§269 | | `distill/axbm.md` | 82 candidate notes from waves ax/bm | Landed as cookbook §268 (the register-pin law on call-clobbered registers) and §269 | `cookbook/INDEX.md` §268–§269 |
## Session tool designs and analyses (Phase 31, sessions 58–61) ## Session tool designs and analyses (Phase 31, sessions 58–61)
@@ -81,8 +81,8 @@ documents (0 open stubs), so the last populated snapshot is kept instead of bein
| File (`docs/sunset/history/…`) | What it was | Where it lives now | | File (`docs/sunset/history/…`) | What it was | Where it lives now |
|---|---|---| |---|---|---|
| `Fable5-ultracode-Full research findings-w42n9cp9t.output` | 216 KB of raw multi-agent research JSON from the first day (the sotn workflow, the Ghidra/PSX stack, WSL) | Everything actionable went into `docs/SETUP.md` and `docs/formats.md` in June 2026 | | `Fable5-ultracode-Full research findings-w42n9cp9t.output` | 216 KB of raw multi-agent research JSON from the first day (the sotn workflow, the Ghidra/PSX stack, WSL) | Everything actionable went into `docs/ops/INDEX.md` and `docs/formats.md` in June 2026 |
| `debug menu link.txt`, `ghidra psx plugin link.txt`, `sotn-decomp wiki link.txt` | The original three reference URLs | `docs/SETUP.md` §2 and the memory map's source index | | `debug menu link.txt`, `ghidra psx plugin link.txt`, `sotn-decomp wiki link.txt` | The original three reference URLs | `docs/ops/INDEX.md` §2 and the memory map's source index |
## Tools retired at Phase 33.5 (`tools/sunset/`, v1.32.1) ## Tools retired at Phase 33.5 (`tools/sunset/`, v1.32.1)
@@ -33,7 +33,7 @@ The matching frontier is empty, so new matches are not the need. The work that h
are the type-lift, call-site cast and canonical-signature tools under `tools/`; the constraint is below. are the type-lift, call-site cast and canonical-signature tools under `tools/`; the constraint is below.
- **Shiftability** — a build whose addresses can move; the LZSS repack (recompression is not byte-stable, so this is a - **Shiftability** — a build whose addresses can move; the LZSS repack (recompression is not byte-stable, so this is a
Gen3 problem) and asset tooling. Gen3 problem) and asset tooling.
- **Tooling and documentation** — every tool has a row in [`docs/SETUP.md`](../SETUP.md)'s inventory; a tool change - **Tooling and documentation** — every tool has a row in [`docs/ops/INDEX.md`](../ops/INDEX.md)'s inventory; a tool change
ships with its siblings wired to use it and its docs in the same change. ships with its siblings wired to use it and its docs in the same change.
## AI use — conduct ## AI use — conduct
+3 -3
View File
@@ -12,12 +12,12 @@ durable result it produced goes into one of these files, and the note is archive
| Kind of knowledge | Where it goes | The rule behind it | | Kind of knowledge | Where it goes | The rule behind it |
|---|---|---| |---|---|---|
| A compiler idiom proven on the bytes (the residual, the mechanism, the lever, the byte proof) | a numbered section of `docs/matching-cookbook.md`, and its symptom index `docs/cookbook-index.md` (regenerated, never edited) | R16 — consult the knowledge base before a match, feed the lesson back after | | A compiler idiom proven on the bytes (the residual, the mechanism, the lever, the byte proof) | an entry in `cookbook/INDEX.md`, added with `tools/cookbook_add.sh` (never hand-edited) | R16 — consult the knowledge base before a match, feed the lesson back after |
| A strategic pivot: what was believed, what failed, the measurement, the hindsight | an entry in `docs/decision-log.md`, written while fresh | R31 | | A strategic pivot: what was believed, what failed, the measurement, the hindsight | an entry in `docs/decision-log.md`, written while fresh | R31 |
| The post-100 % story (Gen3): the narrative of the readability phases | `docs/story.md` §10 and `docs/retrospective.md` §7, advanced at the END OF EVERY SESSION from that session's decision-log entry; `tools/timeline.py` draws the lever and readability series as the chart's lower panel | the owner's rule, 2026-09-12 | | The post-100 % story (Gen3): the narrative of the readability phases | `docs/story.md` §10 and `docs/retrospective.md` §7, advanced at the END OF EVERY SESSION from that session's decision-log entry; `tools/timeline.py` draws the lever and readability series as the chart's lower panel | the owner's rule, 2026-09-12 |
| A late discovery that would have sped up an earlier phase — what it is, when it was found, when it *could* have been found, what it would have saved | `docs/accelerators.md` | the ledger a future project starts from | | A late discovery that would have sped up an earlier phase — what it is, when it was found, when it *could* have been found, what it would have saved | `docs/accelerators.md` | the ledger a future project starts from |
| A procedure people run | a runbook — `docs/wave-playbook.md` for the matching campaign, `docs/public-flip-runbook.md` for the publication; a superseded runbook carries a banner at the top and is then archived | R21 for the ops reference | | A procedure people run | a runbook — `docs/wave-playbook.md` for the matching campaign, `docs/public-flip-runbook.md` for the publication; a superseded runbook carries a banner at the top and is then archived | R21 for the ops reference |
| An environment or tool fact (a version, a flag, a hook, a row per tool) | `docs/SETUP.md`, in the same change as the tool | R21 | | An environment or tool fact (a version, a flag, a hook, a row per tool) | `docs/ops/INDEX.md`, in the same change as the tool | R21 |
| An address, with its source and its verification status | `docs/memory-map.md` | G5 | | An address, with its source and its verification status | `docs/memory-map.md` | G5 |
| A file or container format | `docs/formats.md` | — | | A file or container format | `docs/formats.md` | — |
| The state of the open phase; then the phase's synthesis; then the one-page digest every session starts from | `phase-ends/CURRENT_PHASE.md` → `phase-ends/PhaseEnd_Phase<N>.md` → `phase-ends/DIGEST.md` | R64 | | The state of the open phase; then the phase's synthesis; then the one-page digest every session starts from | `phase-ends/CURRENT_PHASE.md` → `phase-ends/PhaseEnd_Phase<N>.md` → `phase-ends/DIGEST.md` | R64 |
@@ -40,7 +40,7 @@ on its first line; edits go to the generator, not the file.
|---|---|---| |---|---|---|
| `docs/progress.md`, `docs/progress.fleet.md`, `docs/progress.json`, the README's progress block, `docs/badges/*.json` | `tools/progress.py` | `make report BINARY=main`; `make audit-digest` asserts freshness | | `docs/progress.md`, `docs/progress.fleet.md`, `docs/progress.json`, the README's progress block, `docs/badges/*.json` | `tools/progress.py` | `make report BINARY=main`; `make audit-digest` asserts freshness |
| `docs/difficulty.md`, `docs/duplicates.md`, `docs/duplicates.cross.md`, `docs/backlog.md` | `tools/difficulty.py`, `tools/dup_report.py`, `tools/backlog.py` | `make report BINARY=main` | | `docs/difficulty.md`, `docs/duplicates.md`, `docs/duplicates.cross.md`, `docs/backlog.md` | `tools/difficulty.py`, `tools/dup_report.py`, `tools/backlog.py` | `make report BINARY=main` |
| `docs/cookbook-index.md` | `tools/cookbook_index.py` | `make tools-health` (`--check`) | | `cookbook/INDEX.md` | `tools/cookbook_add.sh` | none (`cookbook_index.py --check` retired from `make tools-health`, P38 T1) |
| `docs/disc-ledger.md` | `tools/disc_audit.py` | `make audit-disc` | | `docs/disc-ledger.md` | `tools/disc_audit.py` | `make audit-disc` |
| `docs/story-timeline.md`, `docs/story-timeline.svg` | `tools/timeline.py` | `make report BINARY=main`; `--check` in `make audit-digest` | | `docs/story-timeline.md`, `docs/story-timeline.svg` | `tools/timeline.py` | `make report BINARY=main`; `--check` in `make audit-digest` |
| `docs/commit-map.tsv` | `tools/public_rewrite/build_commit_map.py` | one-shot, at the history rewrite | | `docs/commit-map.tsv` | `tools/public_rewrite/build_commit_map.py` | one-shot, at the history rewrite |
+6 -4
View File
@@ -11,10 +11,13 @@ in the Archive index, not here. The conventions behind this page: [Docs and scra
| File | What it is | How to read it | Written by | | File | What it is | How to read it | Written by |
|---|---|---|---| |---|---|---|---|
| [`SETUP.md`](../SETUP.md) | The environment reference: every tool's install row, the pinned compiler triple (§5.4), the daily command crib, the MCP lifecycle, the scratch conventions (§1a). 270 KB. | Grep by `§` or by tool name; §5.4 for the triple; the `## Tooling inventory` table for a tool's one-line row. | authored; kept current in the same change as any tool (R21) | | [`ops/INDEX.md`](../ops/INDEX.md) | The environment reference: every tool's install row, the pinned compiler triple (§5.4), the daily command crib, the MCP lifecycle, the scratch conventions (§1a). 270 KB. | Grep by `§` or by tool name; §5.4 for the triple; the `## Tooling inventory` table for a tool's one-line row. | authored; kept current in the same change as any tool (R21) |
| [`ops/`](../ops/) | The environment reference split one page per section at the PA3 migration (2026-09); `ops/INDEX.md` lists every page, `ops/tooling-inventory.md` is the per-tool table. | Open `ops/INDEX.md`, then one page. | authored; kept current in the same change as any tool (R21) |
| [`retired/`](../retired/) | Files retired at the PA3 migration: the monolithic `SETUP.md`, `matching-cookbook.md` and its index, `effort-map.md`, the PA2-era files. Kept for history; never in an agent's load order. | Grep only. | frozen |
| [`project-architect.md`](../project-architect.md) | The Project Architect 3.0 methodology this repository runs under. | Read whole. | authored |
| [`verification.md`](../verification.md) | The byte-identity contract and the recorded run that proved it (218 of 218 from a clean rebuild, both SDK legs). | Read whole; §2 quotes the run's summary from `.run/P33/verify/`. | authored; quotes `tools/verify_contract.sh`'s output | | [`verification.md`](../verification.md) | The byte-identity contract and the recorded run that proved it (218 of 218 from a clean rebuild, both SDK legs). | Read whole; §2 quotes the run's summary from `.run/P33/verify/`. | authored; quotes `tools/verify_contract.sh`'s output |
| [`second-oracle.md`](../second-oracle.md) | The independent boundary oracle (R34): what it covers, where it disagrees with the primary one by design. | Read whole. | authored; read by the Makefile and four tools | | [`second-oracle.md`](../second-oracle.md) | The independent boundary oracle (R34): what it covers, where it disagrees with the primary one by design. | Read whole. | authored; read by the Makefile and four tools |
| [`effort-map.md`](../effort-map.md) | The reasoning-effort doctrine for the AI agent: depth vs breadth, the per-phase map, the decision rule. Governs where `CLAUDE.md`'s tier language differs. | Read whole. | authored | | [`retired/effort-map.md`](../retired/effort-map.md) | The reasoning-effort doctrine for the AI agent: depth vs breadth, the per-phase map, the decision rule. Governs where `CLAUDE.md`'s tier language differs. | Read whole. | authored |
| [`doc_links_pending.txt`](../doc_links_pending.txt) | Link targets promised by a document and created by a later task of the same phase (R80). Empty at every phase close. | One `path<TAB>task` per line. | authored per task | | [`doc_links_pending.txt`](../doc_links_pending.txt) | Link targets promised by a document and created by a later task of the same phase (R80). Empty at every phase close. | One `path<TAB>task` per line. | authored per task |
| [`public-flip-runbook.md`](../public-flip-runbook.md) | The history-rewrite and publication procedure, its controls, its risk register, and what the host kept serving afterwards (§11). | By section; §11 for the purge and the probe. | authored | | [`public-flip-runbook.md`](../public-flip-runbook.md) | The history-rewrite and publication procedure, its controls, its risk register, and what the host kept serving afterwards (§11). | By section; §11 for the purge and the probe. | authored |
@@ -33,8 +36,7 @@ in the Archive index, not here. The conventions behind this page: [Docs and scra
| File | What it is | How to read it | Written by | | File | What it is | How to read it | Written by |
|---|---|---|---| |---|---|---|---|
| [`matching-cookbook.md`](../matching-cookbook.md) | Some 500 compiler idioms, each with its residual, mechanism, lever and byte proof. **3.5 MB — never read whole.** | Grep by `§` number or by symptom; the index below maps symptoms to sections. | authored, one section per proven idiom (R16) | | [`cookbook/INDEX.md`](../../cookbook/INDEX.md) | The cookbook since the PA3 migration: one line per technique, each a `cookbook/C<nnnn>.md` file (the monolith is in `docs/retired/`). Some 500 compiler idioms, each with its residual, mechanism, lever and byte proof. **3.5 MB — never read whole.** | Grep by `§` number or by symptom; the index below maps symptoms to sections. | authored, one section per proven idiom (R16) |
| [`cookbook-index.md`](../cookbook-index.md) | The symptom → section index of the cookbook. 560 KB. | Grep the symptom. | **generated** — `tools/cookbook_index.py`, checked in `make tools-health` |
| [`docs/tool-index.md`](../tool-index.md) | every tool, keyed by the NEED it answers, with what proved it (its consumers) and what it hard-codes; the retired ones last | generated by `tools/tool_census.py` from the tree and `config/tool_dictionary.tsv` (the authored phase/need/verdict); `--check` in tools-health; the same data generates the day-one kit's manifest and its verbatim tool corpus | read it by need: find the phrase that matches what you are trying to do | | [`docs/tool-index.md`](../tool-index.md) | every tool, keyed by the NEED it answers, with what proved it (its consumers) and what it hard-codes; the retired ones last | generated by `tools/tool_census.py` from the tree and `config/tool_dictionary.tsv` (the authored phase/need/verdict); `--check` in tools-health; the same data generates the day-one kit's manifest and its verbatim tool corpus | read it by need: find the phrase that matches what you are trying to do |
| [`gcc-2.7.2-map/`](../gcc-2.7.2-map/) | The codegen map of the pinned compiler by pass (scheduling, register allocation, loop, CSE/expression), with a citation audit; its README is the catalogue. | Start at the README's pass → residual → lever table. | authored; `tools/gccmap_cites.py --check` in tools-health | | [`gcc-2.7.2-map/`](../gcc-2.7.2-map/) | The codegen map of the pinned compiler by pass (scheduling, register allocation, loop, CSE/expression), with a citation audit; its README is the catalogue. | Start at the README's pass → residual → lever table. | authored; `tools/gccmap_cites.py --check` in tools-health |
| [`wave-playbook.md`](../wave-playbook.md) | The matching-campaign procedure: draw, cards, packs, draft, gate, harvest, checkpoint — each guard paired with the measurement that produced it. | By numbered step; the S80 addenda for the one-agent-per-function shape. | authored | | [`wave-playbook.md`](../wave-playbook.md) | The matching-campaign procedure: draw, cards, packs, draft, gate, harvest, checkpoint — each guard paired with the measurement that produced it. | By numbered step; the S80 addenda for the one-agent-per-function shape. | authored |
+1 -1
View File
@@ -6,7 +6,7 @@
|---|---| |---|---|
| [`src/`](../../src/) | **The C.** One directory per binary: `src/` itself is the main executable (`SLUS_007.26`), `src/resident/` the always-resident engine, `src/ov_*/` the 141 overlays, `src/md_*/` the 75 code modules. [`src/shared/`](../../src/shared/) holds the bodies shared across binaries by [the dedup engine](The-dedup-engine.md) (`engine_core.h`, ~8.8 MB of `DEFINE_func_…()` macros — every translation unit that shares code includes it). Sony's library regions in main are `INCLUDE_ASM` tiles (`src/lib*.c`, `src/apicard*.c`); the five original hand-written routines are verbatim `__asm__` bodies. [`src/NOTICE.md`](../../src/NOTICE.md) states the licensing position. `src/shared/` holds every shared function body ONCE as a plain-C header (`ov/`, `slot_*/`, `main/`, Phase 35), included at each member's site — see [The dedup engine](The-dedup-engine.md). | | [`src/`](../../src/) | **The C.** One directory per binary: `src/` itself is the main executable (`SLUS_007.26`), `src/resident/` the always-resident engine, `src/ov_*/` the 141 overlays, `src/md_*/` the 75 code modules. [`src/shared/`](../../src/shared/) holds the bodies shared across binaries by [the dedup engine](The-dedup-engine.md) (`engine_core.h`, ~8.8 MB of `DEFINE_func_…()` macros — every translation unit that shares code includes it). Sony's library regions in main are `INCLUDE_ASM` tiles (`src/lib*.c`, `src/apicard*.c`); the five original hand-written routines are verbatim `__asm__` bodies. [`src/NOTICE.md`](../../src/NOTICE.md) states the licensing position. `src/shared/` holds every shared function body ONCE as a plain-C header (`ov/`, `slot_*/`, `main/`, Phase 35), included at each member's site — see [The dedup engine](The-dedup-engine.md). |
| [`config/`](../../config/) | The build definition: one splat yaml per binary (`splat.<alias>.yaml`), the symbol files (`symbols.us*.txt`, curated and build-valid — rule R15), the per-binary SHA1 contracts (`check.<alias>.sha`), the overlay and module registries ([`overlays.mk`](../../config/overlays.mk), [`modules.mk`](../../config/modules.mk) — generated by the onboarding scripts, never edited by hand), the dedup registry ([`dedup.us.yaml`](../../config/dedup.us.yaml)), the verbatim-assembly manifest ([`verbatim_manifest.json`](../../config/verbatim_manifest.json)), the Ghidra annotations as text ([`config/ghidra/`](../../config/ghidra/)). | | [`config/`](../../config/) | The build definition: one splat yaml per binary (`splat.<alias>.yaml`), the symbol files (`symbols.us*.txt`, curated and build-valid — rule R15), the per-binary SHA1 contracts (`check.<alias>.sha`), the overlay and module registries ([`overlays.mk`](../../config/overlays.mk), [`modules.mk`](../../config/modules.mk) — generated by the onboarding scripts, never edited by hand), the dedup registry ([`dedup.us.yaml`](../../config/dedup.us.yaml)), the verbatim-assembly manifest ([`verbatim_manifest.json`](../../config/verbatim_manifest.json)), the Ghidra annotations as text ([`config/ghidra/`](../../config/ghidra/)). |
| [`tools/`](../../tools/) | Everything the project built (about 236 Python tools, 25 shell tools, 12 Ghidra scripts): the disc extractor, the byte gates, the matching harness, the dedup and family engines, the campaign machinery, the audits, the public-flip rewrite package. The pinned submodules live here too (`maspsx`, `asm-differ`, `decomp-permuter`, `m2c`) and the tracked `cc1` tarballs (`tools/bin/`). [`tools/README.md`](../../tools/README.md) carries the one AGPL statement; [`docs/SETUP.md`](../SETUP.md)'s tooling inventory has a one-line row for every tool. | | [`tools/`](../../tools/) | Everything the project built (about 236 Python tools, 25 shell tools, 12 Ghidra scripts): the disc extractor, the byte gates, the matching harness, the dedup and family engines, the campaign machinery, the audits, the public-flip rewrite package. The pinned submodules live here too (`maspsx`, `asm-differ`, `decomp-permuter`, `m2c`) and the tracked `cc1` tarballs (`tools/bin/`). [`tools/README.md`](../../tools/README.md) carries the one AGPL statement; [`docs/ops/INDEX.md`](../ops/INDEX.md)'s tooling inventory has a one-line row for every tool. |
| [`docs/`](../) | The knowledge base and the record: the matching cookbook and its index, the gcc-2.7.2 codegen map, the memory map (addresses with provenance), the formats, the decision log, the accelerators, the wave playbook, the runbooks, the story, the retrospective, this wiki's sources (`docs/wiki/`, `docs/how-to-ai-decomp/`), the generated progress digests, badges and timeline, and the [Archive index](Archive-index.md) of closed-phase documents (their files left the tree at Phase 34; the history keeps them). Where each kind of knowledge goes, which files are generated, and how links are checked: [Docs and scratch conventions](Docs-and-scratch-conventions.md). | | [`docs/`](../) | The knowledge base and the record: the matching cookbook and its index, the gcc-2.7.2 codegen map, the memory map (addresses with provenance), the formats, the decision log, the accelerators, the wave playbook, the runbooks, the story, the retrospective, this wiki's sources (`docs/wiki/`, `docs/how-to-ai-decomp/`), the generated progress digests, badges and timeline, and the [Archive index](Archive-index.md) of closed-phase documents (their files left the tree at Phase 34; the history keeps them). Where each kind of knowledge goes, which files are generated, and how links are checked: [Docs and scratch conventions](Docs-and-scratch-conventions.md). |
| [`phase-ends/`](../../phase-ends/) | The governance record: `PhaseEnd_Phase<N>.md` for every phase, [`DIGEST.md`](../../phase-ends/DIGEST.md) (every phase's synopsis and every rule's full text), `CURRENT_PHASE.md` while a phase is open, `logs/` (each closed phase's working log, on demand only). | | [`phase-ends/`](../../phase-ends/) | The governance record: `PhaseEnd_Phase<N>.md` for every phase, [`DIGEST.md`](../../phase-ends/DIGEST.md) (every phase's synopsis and every rule's full text), `CURRENT_PHASE.md` while a phase is open, `logs/` (each closed phase's working log, on demand only). |
| [`include/`](../../include/) | Headers, including the four generic splat preset headers (`include_asm.h`, `macro.inc`, `labels.inc`, `gte_macros.inc`). | | [`include/`](../../include/) | Headers, including the four generic splat preset headers (`include_asm.h`, `macro.inc`, `labels.inc`, `gte_macros.inc`). |
+2 -2
View File
@@ -129,8 +129,8 @@ earlier inline match never propagated — and a propagated body ships its signat
## The knowledge base, and feeding it ## The knowledge base, and feeding it
Before each match, consult [`docs/matching-cookbook.md`](../matching-cookbook.md) (grep it by symptom through Before each match, consult [`cookbook/INDEX.md`](../../cookbook/INDEX.md) (grep it by symptom through
[`docs/cookbook-index.md`](../cookbook-index.md)); after each match, write the generalizable lesson back into the [`cookbook/INDEX.md`](../../cookbook/INDEX.md)); after each match, write the generalizable lesson back into the
cookbook *and* into the tooling, so the next similar function is a one-shot (rule R16). Two laws learned at cost: only cookbook *and* into the tooling, so the next similar function is a one-shot (rule R16). Two laws learned at cost: only
harvest lessons from byte-proven results, and strip a credited lever from the accepted body and recompile before it harvest lessons from byte-proven results, and strip a credited lever from the accepted body and recompile before it
enters the base — one in three credited levers was byte-inert. enters the base — one in three credited levers was byte-inert.
+1 -1
View File
@@ -4,7 +4,7 @@
Everything runs inside one Linux environment (the project uses WSL2 Ubuntu 24.04 on a single ext4 clone; builds never Everything runs inside one Linux environment (the project uses WSL2 Ubuntu 24.04 on a single ext4 clone; builds never
run on a Windows-mounted path — the 9P bridge is slow and breaks file watching). The authoritative, evolvable reference run on a Windows-mounted path — the 9P bridge is slow and breaks file watching). The authoritative, evolvable reference
for every version, path and command is [`docs/SETUP.md`](../SETUP.md); this page is the map of it. for every version, path and command is [`docs/ops/INDEX.md`](../ops/INDEX.md); this page is the map of it.
## The pinned triple — and why it is pinned ## The pinned triple — and why it is pinned
+30
View File
@@ -0,0 +1,30 @@
# T1.c1 — tools-health: repoint off docs/retired/ (PA3 migration b8a6cbc54)
## Changed
- `tools/tool_census.py:14,42-43,51,143,331` — setup_rows() reads `docs/ops/tooling-inventory.md` (same `| group | tool | purpose |` shape, cells[2]); docstring updated. Beyond brief (needed for --check to get past the crash): COOKBOOK_SOURCES -> `docs/retired/{matching-cookbook,cookbook-index}.md`, RECORD_SOURCES effort-map -> `docs/retired/effort-map.md`; corpus_plan keeps the old corpus dest names (`replace("docs/retired/","docs/",1)`). Moves were pure renames (0 lines changed per `git diff --stat -M b8a6cbc54^ b8a6cbc54`).
- `Makefile:317-322` — `cookbook_index.py --check` commented out under a house DISABLED block (tool kept; CI `.github/workflows/no-rom.yml` still runs it, untouched).
- `tools/doc_links.py:38,54` — DEFAULT + TOPIC_PAGE entry `docs/SETUP.md` -> `docs/ops/INDEX.md` (kept, not dropped: Toolchain-setup.md still covers the topic).
- `tools/wiki_render.py:112` — selftest case g `/docs/SETUP.md` -> `/docs/ops/INDEX.md`.
- 36 BROKEN links retargeted: README.md:108, docs/gcc-2.7.2-map/README.md, docs/wiki/{Archive-index,Contributing-and-the-no-ROM-policy,Docs-and-scratch-conventions,Reference-index,Repository-layout,The-matching-workflow,Toolchain-setup}.md, docs/how-to-ai-decomp/{00-README,01-governance,06-knowledge-base,08-models-and-budgets}.md. Mapping per brief. Reference-index: the matching-cookbook row now describes cookbook/INDEX.md; the cookbook-index row dropped (would duplicate it). Docs-and-scratch-conventions rows reworded to cookbook/INDEX.md + cookbook_add.sh.
- `docs/wiki/Reference-index.md` — 3 new rows (`ops/`, `retired/`, `project-architect.md`): after the links were fixed doc_links still rc 1 on 60 UNCOVERED docs files (53 docs/ops/*, 6 docs/retired/*, docs/project-architect.md); a directory link covers its files (covered(), doc_links.py:128).
- `docs/ops/tooling-inventory.md:21` — note cookbook_index retired from tools-health (no own row exists; the CI row is the only one naming it).
- docs/wiki/Home.md: no `SETUP` link present at HEAD (grep); nothing to change. The dead link was the wiki_render selftest case.
## Verified
- `.venv/bin/python tools/doc_links.py` -> rc 0, `0 broken`, coverage 126 of 126, `doc_links: OK`
- `.venv/bin/python tools/wiki_render.py --selftest` -> rc 0, 12 cases 0 failed
- `.venv/bin/python tools/kit_lint.py` -> rc 0 `kit_lint: OK`
- `.venv/bin/python tools/kit_coverage.py` -> rc 0 `kit_coverage: OK`
- `.venv/bin/python tools/tool_census.py --check` -> rc 1, 65 gaps (NOT green; see Blocker)
- `make -n tools-health >/dev/null` -> rc 2 at `Makefile:516: sdk-dual`; control with HEAD Makefile (`make -n -f .run/t1c1/Makefile.head tools-health`) fails identically -> pre-existing, not this change.
## Blocker: tool_census --check
Past the SETUP crash, 65 gaps, all PA3-migration drift outside the brief:
- 22 `no dictionary row` — new PA3 tools (card.py, commit_task.sh, cookbook_add.sh, plan_edit.py, run.sh, outline.py, launch.py, ...) lack rows in `config/tool_dictionary.tsv` (authored data, 8 cols each).
- 43 corpus gaps in `decomp-architect/corpus/` (generated): missing copies for those tools, stale copies of changed tools/docs, missing `record/phase-ends/PhaseEnd_Phase37{,.5}.md`. Fix = `tools/tool_census.py --corpus` after the rows exist; that copies the untracked `phase-ends/PhaseEnd_Phase37.md` into the kit (brief forbids touching it) -> expert decision.
## Incident (expert must know)
`make -n tools-health` is NOT a dry run here: `.ONESHELL` + a recipe containing `$(MAKE)` makes make execute the whole recipe; sub-makes stay dry (-n in MAKEFLAGS) but plain shell lines run. Within sdk-dual that ran `rm -rf build/psyq` and two `cp $(main_MAPFILE) .run/P33/verify/main_{with,no}_sdk.map` (19:40, after the in-flight tools-health1 had printed `sdk-dual: OK`). Ran 3x (twice via tools-health, once `make -n sdk-dual` while diagnosing).
- `.run/P33/verify/main_no_sdk.map` (tracked) was clobbered with the WITH map -> restored with `git checkout --` (clean now).
- `build/psyq/` is gone. Restore = leg 3 of sdk-dual: `make extract BINARY=main && make check BINARY=main` (not run: brief forbids make builds and a run is in flight). The in-flight tools-health1 was past sdk-dual (at `report`).
- generalizable: `make -n` on a `.ONESHELL` target whose recipe calls `$(MAKE)` executes the recipe's non-make lines; the brief's `make -n tools-health` check is destructive on an SDK machine.
+2 -2
View File
@@ -35,7 +35,7 @@ import subprocess
import sys import sys
REPO = pathlib.Path(__file__).resolve().parent.parent REPO = pathlib.Path(__file__).resolve().parent.parent
DEFAULT = ["README.md", "THIRD_PARTY.md", "CLAUDE.md", "src/NOTICE.md", "tools/README.md", "docs/SETUP.md", DEFAULT = ["README.md", "THIRD_PARTY.md", "CLAUDE.md", "src/NOTICE.md", "tools/README.md", "docs/ops/INDEX.md",
"docs/verification.md", "docs/public-flip-runbook.md", "docs/decision-log.md", "docs/accelerators.md", "docs/verification.md", "docs/public-flip-runbook.md", "docs/decision-log.md", "docs/accelerators.md",
"docs/story.md", "docs/story-timeline.md", "docs/retrospective.md", "phase-ends/README.md", "phase-ends/DIGEST.md", "docs/story.md", "docs/story-timeline.md", "docs/retrospective.md", "phase-ends/README.md", "phase-ends/DIGEST.md",
"docs/gcc-2.7.2-map/README.md", "tools/xsig/README.md", "docs/gcc-2.7.2-map/README.md", "tools/xsig/README.md",
@@ -51,7 +51,7 @@ INDEX_PAGES = ("docs/wiki/Reference-index.md", "docs/wiki/Archive-index.md")
ARCHIVE_DIR = "docs/sunset/" ARCHIVE_DIR = "docs/sunset/"
# the wiki page that covers a docs/ file's topic — for the wiki-first WARNING only (a hand-kept map; small on purpose) # the wiki page that covers a docs/ file's topic — for the wiki-first WARNING only (a hand-kept map; small on purpose)
TOPIC_PAGE = { TOPIC_PAGE = {
"docs/SETUP.md": "docs/wiki/Toolchain-setup.md", "docs/ops/INDEX.md": "docs/wiki/Toolchain-setup.md",
"docs/verification.md": "docs/wiki/Verification-and-progress.md", "docs/verification.md": "docs/wiki/Verification-and-progress.md",
"docs/public-flip-runbook.md": "docs/wiki/The-ROM-firewall.md", "docs/public-flip-runbook.md": "docs/wiki/The-ROM-firewall.md",
"docs/gen3-standards.md": "docs/wiki/Where-the-project-goes-next.md", "docs/gen3-standards.md": "docs/wiki/Where-the-project-goes-next.md",
+6 -5
View File
@@ -11,7 +11,7 @@ two verbatim corpora, all from one hand-kept dictionary (P33.5 task 13.5; `--che
Two kinds of fact, deliberately separated (R33): Two kinds of fact, deliberately separated (R33):
DERIVED on every run from the tree — the file set (two independent enumerations that must agree: `find` and `git ls-files`, DERIVED on every run from the tree — the file set (two independent enumerations that must agree: `find` and `git ls-files`,
R34), each tool's docstring line, its SETUP row (`docs/SETUP.md`, matched by file name), its CONSUMERS (Makefile + config/*.mk R34), each tool's docstring line, its SETUP row (`docs/ops/tooling-inventory.md`, matched by file name), its CONSUMERS (Makefile + config/*.mk
targets, the CI workflows, the wave playbook, and every other tool that imports it or names it), and hence its class: targets, the CI workflows, the wave playbook, and every other tool that imports it or names it), and hence its class:
LIVE (a runtime consumer), REFERENCED (only a SETUP/doc row), ORPHAN (neither). LIVE (a runtime consumer), REFERENCED (only a SETUP/doc row), ORPHAN (neither).
AUTHORED once, in `config/tool_dictionary.tsv` (tab-separated, LF line endings, the eight columns below) — the ladder phase, the portability (PORTABLE / ADAPT / PROJECT-ONLY), the NEED AUTHORED once, in `config/tool_dictionary.tsv` (tab-separated, LF line endings, the eight columns below) — the ladder phase, the portability (PORTABLE / ADAPT / PROJECT-ONLY), the NEED
@@ -39,7 +39,8 @@ KIT = REPO / "decomp-architect"
MANIFEST = KIT / "tools" / "MANIFEST.md" MANIFEST = KIT / "tools" / "MANIFEST.md"
CORPUS_TOOLS = KIT / "corpus" / "tools" CORPUS_TOOLS = KIT / "corpus" / "tools"
CORPUS_COOK = KIT / "corpus" / "cookbook" CORPUS_COOK = KIT / "corpus" / "cookbook"
COOKBOOK_SOURCES = ["docs/matching-cookbook.md", "docs/cookbook-index.md"] + [p.relative_to(REPO).as_posix() for p in sorted((REPO / "docs" / "gcc-2.7.2-map").glob("*")) if p.is_file()] # the two cookbook files moved to docs/retired/ at the PA3 migration (b8a6cbc54, bytes unchanged); the corpus keeps their old names
COOKBOOK_SOURCES = ["docs/retired/matching-cookbook.md", "docs/retired/cookbook-index.md"] +[p.relative_to(REPO).as_posix() for p in sorted((REPO / "docs" / "gcc-2.7.2-map").glob("*")) if p.is_file()]
# P33.5 task 14.5 (Drew: "the whole of our experience"): the RECORD as the third dictionary — the distilled records and the # P33.5 task 14.5 (Drew: "the whole of our experience"): the RECORD as the third dictionary — the distilled records and the
# phase-by-phase record, verbatim. The phase worklogs (phase-ends/logs/) are deliberately NOT here (R19: on-demand archives, 30k # phase-by-phase record, verbatim. The phase worklogs (phase-ends/logs/) are deliberately NOT here (R19: on-demand archives, 30k
# lines); their distillation is the task-14.5 mining pass, banked into the kernels/accelerators. Layout under corpus/record/: # lines); their distillation is the task-14.5 mining pass, banked into the kernels/accelerators. Layout under corpus/record/:
@@ -47,7 +48,7 @@ COOKBOOK_SOURCES = ["docs/matching-cookbook.md", "docs/cookbook-index.md"] + [p.
CORPUS_RECORD = KIT / "corpus" / "record" CORPUS_RECORD = KIT / "corpus" / "record"
RECORD_SOURCES = ([p.relative_to(REPO).as_posix() for p in sorted((REPO / "docs" / "how-to-ai-decomp").glob("*.md"))] RECORD_SOURCES = ([p.relative_to(REPO).as_posix() for p in sorted((REPO / "docs" / "how-to-ai-decomp").glob("*.md"))]
+ ["docs/decision-log.md", "docs/accelerators.md", "docs/retrospective.md", "docs/story.md", "docs/wave-playbook.md", + ["docs/decision-log.md", "docs/accelerators.md", "docs/retrospective.md", "docs/story.md", "docs/wave-playbook.md",
"docs/effort-map.md", "docs/gen3-standards.md", "docs/gen3-handoff.md", "phase-ends/DIGEST.md"] "docs/retired/effort-map.md", "docs/gen3-standards.md", "docs/gen3-handoff.md", "phase-ends/DIGEST.md"]
+ [p.relative_to(REPO).as_posix() for p in sorted((REPO / "phase-ends").glob("PhaseEnd_Phase*.md"), + [p.relative_to(REPO).as_posix() for p in sorted((REPO / "phase-ends").glob("PhaseEnd_Phase*.md"),
key=lambda q: [float(x) if x.replace(".", "").isdigit() else x for x in re.split(r"(\d+(?:\.\d+)?)", q.stem)])]) key=lambda q: [float(x) if x.replace(".", "").isdigit() else x for x in re.split(r"(\d+(?:\.\d+)?)", q.stem)])])
@@ -139,7 +140,7 @@ def docstring_line(path):
def setup_rows(): def setup_rows():
"""file name -> the SETUP row's purpose cell (first 160 chars).""" """file name -> the SETUP row's purpose cell (first 160 chars)."""
out = {} out = {}
for ln in (REPO / "docs" / "SETUP.md").read_text(encoding="utf-8", errors="replace").splitlines(): for ln in (REPO / "docs" / "ops" / "tooling-inventory.md").read_text(encoding="utf-8", errors="replace").splitlines():
if not ln.startswith("|"): if not ln.startswith("|"):
continue continue
cells = [c.strip() for c in ln.strip().strip("|").split("|")] cells = [c.strip() for c in ln.strip().strip("|").split("|")]
@@ -327,7 +328,7 @@ def corpus_plan(recs):
dest = CORPUS_TOOLS / r["phase"] / r["path"].removeprefix("tools/") dest = CORPUS_TOOLS / r["phase"] / r["path"].removeprefix("tools/")
copies.append((REPO / r["path"], dest)) copies.append((REPO / r["path"], dest))
for s in COOKBOOK_SOURCES: for s in COOKBOOK_SOURCES:
copies.append((REPO / s, CORPUS_COOK / pathlib.Path(s).relative_to("docs"))) copies.append((REPO / s, CORPUS_COOK / pathlib.Path(s.replace("docs/retired/", "docs/", 1)).relative_to("docs")))
for s in RECORD_SOURCES: for s in RECORD_SOURCES:
copies.append((REPO / s, record_dest(s))) copies.append((REPO / s, record_dest(s)))
return copies, pointers return copies, pointers
+1 -1
View File
@@ -109,7 +109,7 @@ def selftest():
("[d](../verification.md)", f"[d]({GH}/blob/main/docs/verification.md)"), ("[d](../verification.md)", f"[d]({GH}/blob/main/docs/verification.md)"),
("[e](../../README.md#license)", f"[e]({GH}/blob/main/README.md#license)"), ("[e](../../README.md#license)", f"[e]({GH}/blob/main/README.md#license)"),
("[f](../../tools/)", f"[f]({GH}/tree/main/tools)"), ("[f](../../tools/)", f"[f]({GH}/tree/main/tools)"),
("[g](/docs/SETUP.md)", f"[g]({GH}/blob/main/docs/SETUP.md)"), ("[g](/docs/ops/INDEX.md)", f"[g]({GH}/blob/main/docs/ops/INDEX.md)"),
("![h](../story-timeline.svg)", f"![h]({RAW}/docs/story-timeline.svg)"), ("![h](../story-timeline.svg)", f"![h]({RAW}/docs/story-timeline.svg)"),
("[i](https://example.org/x.md)", "[i](https://example.org/x.md)"), ("[i](https://example.org/x.md)", "[i](https://example.org/x.md)"),
("[j](#anchor)", "[j](#anchor)"), ("[j](#anchor)", "[j](#anchor)"),