From 0e38b51a37b78ea62ab07dbfbd777afa59a96201 Mon Sep 17 00:00:00 2001 From: Drew T <50529377+Druthulu@users.noreply.github.com> Date: Tue, 8 Sep 2026 17:33:42 -0600 Subject: [PATCH] =?UTF-8?q?build(phase-35):=20T3=20close=20=E2=80=94=20the?= =?UTF-8?q?=20five=20twin=20pairs=20collapsed=20onto=20one=20source=20dire?= =?UTF-8?q?ctory=20each=20(10=20aliases,=20166=20files=20deleted);=20R22?= =?UTF-8?q?=20clean=20fleet=20run=20check-all:=20218=20passed,=200=20faile?= =?UTF-8?q?d=20of=20218=20(309=20s);=20the=20census=20after=20the=20twins:?= =?UTF-8?q?=20same-vram=20backlog=204,667=20->=201,099=20classes,=203,580?= =?UTF-8?q?=20classes=20twin-covered,=20S1=209,081/10,180=20satisfied;=20r?= =?UTF-8?q?ules=20check=20(P6)=20done;=20kit=20corpus=20for=20pads=5Faudit?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .run/P35/baseline/kit_corpus_t3.log | 5 + .run/P35/baseline/kit_corpus_t3a.log | 5 + .run/P35/baseline/r22_t3.log | 476 ++++++++++++++++++ .run/P35/census/share_census.json | 57 ++- .run/P35/census/share_census.txt | 16 +- .../corpus/tools/P6/pads_audit.py | 9 +- docs/tool-index.md | 2 +- phase-ends/CURRENT_PHASE.md | 32 +- 8 files changed, 560 insertions(+), 42 deletions(-) create mode 100644 .run/P35/baseline/kit_corpus_t3.log create mode 100644 .run/P35/baseline/kit_corpus_t3a.log create mode 100644 .run/P35/baseline/r22_t3.log diff --git a/.run/P35/baseline/kit_corpus_t3.log b/.run/P35/baseline/kit_corpus_t3.log new file mode 100644 index 000000000..e5869de42 --- /dev/null +++ b/.run/P35/baseline/kit_corpus_t3.log @@ -0,0 +1,5 @@ +.venv/bin/python tools/tool_census.py --all +tool_census: 295 tool files (find == git ls-files) + 34 retired; classes {'LIVE': 235, 'ORPHAN': 31, 'REFERENCED': 29, 'RETIRED': 34}; dictionary rows 329 +tool_census: wrote docs/tool-index.md +tool_census: wrote decomp-architect/tools/MANIFEST.md +tool_census: corpus materialised — 361 verbatim copies, 28 pointers, INDEX.md diff --git a/.run/P35/baseline/kit_corpus_t3a.log b/.run/P35/baseline/kit_corpus_t3a.log new file mode 100644 index 000000000..e5869de42 --- /dev/null +++ b/.run/P35/baseline/kit_corpus_t3a.log @@ -0,0 +1,5 @@ +.venv/bin/python tools/tool_census.py --all +tool_census: 295 tool files (find == git ls-files) + 34 retired; classes {'LIVE': 235, 'ORPHAN': 31, 'REFERENCED': 29, 'RETIRED': 34}; dictionary rows 329 +tool_census: wrote docs/tool-index.md +tool_census: wrote decomp-architect/tools/MANIFEST.md +tool_census: corpus materialised — 361 verbatim copies, 28 pointers, INDEX.md diff --git a/.run/P35/baseline/r22_t3.log b/.run/P35/baseline/r22_t3.log new file mode 100644 index 000000000..af2accb12 --- /dev/null +++ b/.run/P35/baseline/r22_t3.log @@ -0,0 +1,476 @@ +clean: removed build/, expected/, and the regenerated splat tree (asm/, assets/, undefined_*_auto.txt). +disc-extract: up to date — extracted/retail/ matches the committed manifest (sha1 e5c6019249dec3c71bb2cf40658a78439dddb037) +splat 0.41.0 (powered by spimdisasm 1.41.0) + 0%| | 0/1096 [00:00=2 instances - verdicts (classes): A 1,712 · B 282 · C 6,135 · D 1,833 · M 218 - instances by verdict: A 214,478 · B 45,226 · C 14,895 · D 5,740 · M 462 - flags: ALIAS 44 · E 3,826 · F 54 · PINS 686 · TWIN-COVERED 575 · TWIN-PENDING 3,173 · TYPEDEF 96 - SAME-VRAM UNREGISTERED (the phase's backlog): 4,105 classes · 34,852 instances · 10,632 private copies · 6,529 collapsible · 1,632,227 ins; of which twin-pending 3,006 classes / 6,012 instances - TWIN-COVERED (one source directory per payload, T3): 565 classes · 1,136 instances - CROSS-ADDRESS / CROSS-SPACE (deferred to the names phase): 3,801 classes · 30,347 instances · 11,600 private copies - S1 violations (scope as run): 4,105 · multi-form instances 0 · unresolved names 0 + verdicts (classes): A 1,712 · B 282 · C 6,640 · D 1,545 · M 1 + instances by verdict: A 214,478 · B 45,226 · C 15,917 · D 5,178 · M 2 + flags: ALIAS 38 · E 3,826 · F 54 · PINS 684 · TWIN-COVERED 3,748 · TYPEDEF 93 + SAME-VRAM UNREGISTERED (the phase's backlog): 1,099 classes · 28,840 instances · 4,755 private copies · 3,658 collapsible · 1,257,027 ins; of which twin-pending 0 classes / 0 instances + TWIN-COVERED (one source directory per payload, T3): 3,580 classes · 7,186 instances + CROSS-ADDRESS / CROSS-SPACE (deferred to the names phase): 3,801 classes · 30,347 instances · 11,340 private copies + S1 violations (scope as run): 1,099 · multi-form instances 0 · unresolved names 0 controls (R39): func_80144B9C got B/141/{'include': 141}/missing 7 expected B/141/{'include': 141}/missing 7 (the registry lists 134; T5 bucket 0 extends it) clearTbl40 got A+E/2 expected A+E/2 @@ -37,4 +37,4 @@ share_census: 218 binaries · 362,389 sig instances · 362,389 classified · 0 U B E,F 11852 x 2 0x80014554,0x8001455C,0x8001513C func_80014554,func_8001455c A - 141 x 167 0x80163C2C func_80163c2c A - 141 x 165 0x80178004 func_80178004 - macro sites 255,947 · duplicate-text classes 6,874 (23,325 sites) · elapsed 27.6 s + macro sites 255,728 · duplicate-text classes 7,385 (24,359 sites) · elapsed 46.9 s diff --git a/decomp-architect/corpus/tools/P6/pads_audit.py b/decomp-architect/corpus/tools/P6/pads_audit.py index c890109df..245525f0e 100644 --- a/decomp-architect/corpus/tools/P6/pads_audit.py +++ b/decomp-architect/corpus/tools/P6/pads_audit.py @@ -22,7 +22,14 @@ def carves(tu): out.append((a, end)) return out def compiled_tables(tu): - cmd = subprocess.run(['make', '-n', '-W', 'src/%s/%s.c' % (b, tu), 'build/src/%s/%s.o' % (b, tu), 'BINARY=%s' % b], capture_output=True, text=True).stdout + # Phase 35 T3: the TU's source comes from the binary's source dir through the oracle — a twin's objects + # (build/src//_x.o) are compiled from src//_x.c (R33; a hand-built src// path + # found no compile line for a twin and raised IndexError here). + sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) + import corpus + prim = corpus.twin_of(b) or b + src = os.path.join(corpus.src_dir(b), tu.replace(b, prim, 1) + '.c') + cmd = subprocess.run(['make', '-n', '-W', src, 'build/src/%s/%s.o' % (b, tu), 'BINARY=%s' % b], capture_output=True, text=True).stdout line = [l for l in cmd.splitlines() if 'cc1' in l][0] line = re.sub(r'\|\s*\.venv/bin/python\s+tools/jtbl_rodata_pads\.py[^|]*', '', line) line = re.sub(r'\|\s*[^|]*mipsel-linux-gnu-as.*$', '', line) diff --git a/docs/tool-index.md b/docs/tool-index.md index 1812b6104..bda1f6fa4 100644 --- a/docs/tool-index.md +++ b/docs/tool-index.md @@ -85,7 +85,7 @@ then what proved it. The same data generates the day-one kit's manifest and its | When you need to… | Tool | What it does | Proven by | Adapt | Class | |---|---|---|---|---|---| -| answer from one derived model whether each function is open, matched or shared | `corpus.py` | The single derived model of the source tree: open, matched or shared, per function | Makefile, api_agent.py, aprop_autodraft.py, aprop_symfix.py (+66) | repo src/config layout | LIVE | +| answer from one derived model whether each function is open, matched or shared | `corpus.py` | The single derived model of the source tree: open, matched or shared, per function | Makefile, api_agent.py, aprop_autodraft.py, aprop_symfix.py (+67) | repo src/config layout | LIVE | | assert every consumer knows about each newly onboarded binary | `audit_binaries.py` | Gate asserting every consumer knows about each newly onboarded binary | Makefile | repo config/registry paths | LIVE | | assert every tracked source is plain text so text searches never skip it | `audit_text_sources.py` | Every tracked C source must be plain text, or text searches silently skip it | .github/workflows/no-rom.yml, Makefile | repo src paths | LIVE | | check that the independent what-remains views agree with the corpus oracle | `audit_frontier.py` | Checks that the independent "what remains" views agree with the corpus oracle | Makefile | repo paths | LIVE | diff --git a/phase-ends/CURRENT_PHASE.md b/phase-ends/CURRENT_PHASE.md index d5db47e53..fa00eca07 100644 --- a/phase-ends/CURRENT_PHASE.md +++ b/phase-ends/CURRENT_PHASE.md @@ -41,8 +41,10 @@ green (audit-binaries OK · dedup-check 2220/0 · cdecl · report + `progress.py --check` fresh · doc_links · wiki_render · kit_lint · cookbook-index · gccmap_cites · ghidra_roster), one red rung — the kit's verbatim copies of the edited tools — fixed by `make kit-corpus` (17 copies) with the rungs after it re-run individually by exit code. The +7 count correction published. -- ☐ **T3** — twin binaries → one source directory: SC01_005/006 (probe), SC03_118/119, SC02_000/003, SC04_018/019, SC03_014/015 (carve - alignment, R60); one commit per pair; the clean fleet run after the last. +- ☑ **T3** (S94) — twin binaries → one source directory: the five pairs collapsed (probe SC01_005/006 with the race test; SC03_118/119 + + SC02_000/003 equal carves; SC04_018/019 + SC03_014/015 with the primary's carve, interleave + pads regenerated, R60 checks), each + twin and primary BYTE-IDENTICAL, 166 source files deleted; the census: same-vram backlog 4,667 → 1,099 classes; the clean fleet + run after the last pair (see the log). - ☐ **T4** — `tools/macro_to_header.py`: the fleet converted in L0-gated batches; `engine_core.h`/`ov_setters.h`/`clearTbl40.h` deleted (clearTbl40 → the parameterized control); `func_80144B9C.h` moved; the registry's `source:`/`func:` rewritten surgically; clean fleet 218/218. - ☐ **T5** — `tools/share_body.py`: bucket 0 `--extend` (206 classes), then the same-vram buckets largest reach first (no trivial exception); @@ -157,6 +159,30 @@ `TWIN_SRC_DIR` matched the `_SRC_DIR` oracle regex as a binary named `TWIN` → renamed `TWIN_SRCDIR`; after it the census shows **TWIN-COVERED 575 classes** for the pair (twin-pending 3,748 → 3,173), twin symmetry 653/653 unchanged. The census's `copies` now counts DISTINCT private sites (a twin's instance resolves to its primary's TU — one source), `collapsible` = copies − 1. + Commit `9b0816e74`. +- **S94 — T3 pairs 2–3 (equal carves, mechanical):** `ov_SC03_119` TWIN_OF `ov_SC03_118` (31 files removed; both `77ee78dd…` + BYTE-IDENTICAL), `ov_SC02_003` TWIN_OF `ov_SC02_000` (37 files; both `5ece4bca…`); `audit-binaries: OK`; dedup-check per twin + 0 failed. Commit `e79cfcc06`. +- **S94 — T3 pairs 4–5 (carve alignment, R60):** the twin's yaml := the primary's with the alias and `FILE_nnn` substituted (+ the + twin note + `create_c_files: False`), its `_JTBL_INTERLEAVE` := the primary's substituted — and, found by the first build's + failure (`missing .end at end of assembly` on `ov_SC04_019_jr_8017AE2C.o`): the per-object **`JTBL_PADS` lines are keyed by + OBJECT PATH** (`build/src//…`), which my alias-normalized block diff (lines starting with the alias) never compared, so the + twin still carried its retired carve's pads (4 differing lines for SC04, 2 for SC03; the three committed pairs re-checked: 0, 0, 0). + Regenerated from the primaries by substitution → 0 differing. `pads_audit.py` built `src//.c` by hand and raised + IndexError on a twin — now reads the source through the oracle (`corpus.src_dir` + `twin_of`). **Verify:** `ov_SC04_019` + + `ov_SC04_018` BYTE-IDENTICAL (`fe9b413f…`), `ov_SC03_015` + `ov_SC03_014` (`d84b01a2…`); `interleave_check` ALIGNED (n=51, n=46); + `pads_audit` ok on every carve object; dedup-check 0 failed; `audit-binaries: OK` (CHECK 3b incl. equal carves). Commits + `2648aa5a9`, `6ebb3dac3`. +- **S94 — T3 census after the five collapses:** TWIN-COVERED 3,748 classes (3,580 non-A, 7,186 instances); **same-vram unregistered + 4,667 → 1,099 classes · 4,755 private sites · 3,658 collapsible · 28,840 instances** (twin-pending 0); `S1: … 10,180 classes, 9,081 + satisfied (3,580 twin-covered, 0 excepted, 3,801 deferred cross-address), 1,099 VIOLATION(S)` — the backlog T5 shares. The controls + unchanged (twin symmetry 653/653). `make kit-corpus` for `pads_audit` + `share_census`; `tool_census --check: OK`. +- **S94 — Rules check (P6, after T3 = four tasks): re-read complete (CLAUDE.md's eight mandatory behaviours; PROJECT_CONTEXT's 23 + P/G/H/X rules). Continuing with T4.** +- **S94 — T3 close, R22:** `make clean && make extract-all JOBS=16 && make check-all JOBS=16` → `extract-all: 217 extracted, 0 failed of + 217 (+ main, serial)` · **`check-all: 218 passed, 0 failed of 218`** · `wall=308.78 s` (the census and the kit corpus ran beside it) · + `exit=0` (`.run/P35/baseline/r22_t3.log`). Twins: 5 pairs / 10 aliases / 166 source files deleted; every twin builds from + its primary's directory. **T3 ☑.** ## Approved plan (verbatim, gate 1 — 2026-09-08) @@ -450,7 +476,7 @@ same commit — history keeps it. Candidate for the PhaseEnd: "a shared body has exactly one source; a duplicate copy is a defect the health chain asserts, and a count of them is published with its rule" (this phase's invariant). -## 🛑 SESSION CHECKPOINT — S94 (2026-09-08): Phase 35 OPEN at gate 1; T0 ☑, T1 ☑ (the census), T2 ☑ (the health chain knows the include + twin forms; the +7 correction published); NEXT = T3 twin binaries → one source directory (probe pair SC01_005/006) +## 🛑 SESSION CHECKPOINT — S94 (2026-09-08): Phase 35 OPEN at gate 1; T0–T3 ☑ (the census; the health chain knows the include + twin forms; the five twin pairs collapsed, fleet 218/218 from clean); NEXT = T4 `tools/macro_to_header.py` (written, its --plan being shaken out) ### 0. How to use this block A fresh session reads CLAUDE.md's load order, replays THIS block verbatim, and resumes at the first unchecked task in §Tasks above.