mirror of
https://github.com/Druthulu/BFM-decomp
synced 2026-09-30 07:31:59 -04:00
tools(phase-34): cdecl --audit — create its scratch dir and refuse (never misgrade) when the gcc adjudication cannot run; the prune had removed .run/audit and every unparsed draft line read as a DEFECT
This commit is contained in:
Drew T
@@ -0,0 +1,26 @@
|
||||
.venv/bin/python tools/cdecl.py --audit --gcc --limit 60
|
||||
[coverage] 60 TUs (cpp-derived) + 600 drafts (raw)
|
||||
[coverage] depth-0 statements: 175222 declarators parsed: 166814
|
||||
[coverage] unparsed: 4 -> PARSER DEFECTS (gcc accepts, cdecl does not): 0 NOT-C (gcc rejects it too — corrupt input): 4
|
||||
NOT-C in 1 dir(s) (all scratch, none in src/): ['.run/drafts-T6-recanon']
|
||||
e.g. 'extern if ((func_80029178(0x119) & 0xFF) != 0);' -> expected ')', got '!=' in 'extern if ((func_800
|
||||
|
||||
[differential] expanded 5147 DEFINE_ macros through cpp -> 7430 names
|
||||
[differential] engine_core.h — D_ symbols (raw-text scan vs cpp-expanded file scope)
|
||||
incumbent theirs cdecl cdecl-only MY DEFECT their block-scope error
|
||||
gen_harvest_targets.DATA_DECL_RE 2238 2319 601 0 520
|
||||
they count BLOCK-scope externs as file-scope canonicals: ['D_800636C8', 'D_80078EAF', 'D_800A5E8C', 'D_800A5E90']
|
||||
sig_unify.DATA_DECL_RE 2238 2319 601 0 520
|
||||
they count BLOCK-scope externs as file-scope canonicals: ['D_800636C8', 'D_80078EAF', 'D_800A5E8C', 'D_800A5E90']
|
||||
the audit's named blind symbols (engine_core.h), now resolved WITH THEIR KIND:
|
||||
D_80127530 kind=array (expected array) [ok] s32 [4]
|
||||
D_80127088 kind=fnptr (expected fnptr) [ok] void (*)(void)
|
||||
D_8011DB28 kind=fnptr (expected fnptr) [ok] void (*)(s32)
|
||||
D_801274D0 kind=fnptr (expected fnptr) [ok] s32 (*)(s32)
|
||||
D_8018E858 kind=fnptr_array (expected fnptr_array) [ok] void (*[])(void)
|
||||
|
||||
[gcc] 4777 distinct declarations -> the real cross-compiler
|
||||
[gcc] adjudicated: 4777 env-skipped (gcc cannot compile the original out of its TU): 0
|
||||
[gcc] accepted: 4777 REJECTED (my reconstruction is a DIFFERENT TYPE): 0
|
||||
|
||||
cdecl: ALL ORACLES GREEN
|
||||
@@ -892,3 +892,17 @@ lessons that were fixed but never generalised; on this project one in five candi
|
||||
~4.3M tokens, ~13 min wall each in two batches; the coordinator's cost was the brief, the harvest script, the clustering and the
|
||||
provenance generation.
|
||||
|
||||
## P34 S93 (2026-09-08) — the flip day: a third-party step proven through the third party's toolchain, and the public-tree decisions
|
||||
|
||||
The one failure of the flip day was a browser step nobody could have exercised while the repository was private: `docs/decompme-preset.md`
|
||||
told the owner to paste the regenerated target listing into decomp.me, and decomp.me's assembler refused it (`invalid operands 'li a2,2'`).
|
||||
The listing is the project's word oracle — nothing on our side assembles it; its bare register names resolve only through `macro.inc` and
|
||||
its branch targets are absolute addresses — and the replica that had "proven the preset" compared words from the listing's comments,
|
||||
never the listing itself. **Accelerator: a step that hands a file to a third party is proven through that party's own toolchain, on the
|
||||
file itself, before the owner's browser session — and the tool that proves it writes the paste** (`verbatim_target_s.py --gas`;
|
||||
`decompme_replica.sh` steps D and E; the retry matched 100% first time). Two smaller ones from the same day. **A calendar gate is read
|
||||
the morning after the ticket, not "daily for a week":** the Support purge took one day; the probe passed during the previous sub-phase and
|
||||
the flip could have opened a day earlier. **Decide what a public tree carries before the flip:** the sent outreach drafts and the
|
||||
document archive were always going to leave (the owner removed both within hours of going public), and deciding at the consolidation
|
||||
sub-phase would have spared two untrack commits on a public history; the Archive-index rule that made the removal harmless — name
|
||||
archived files as paths, never links — held.
|
||||
|
||||
@@ -3645,3 +3645,41 @@ register lever (accelerators (16)), and treat "PROVED" as "proved against this l
|
||||
above any mid-campaign harvest's. Both are now the kit's own rules: the coverage check is in the kit's health target, and the record's
|
||||
front page tells the next project that its logs deserve one final read.
|
||||
|
||||
## P34 S93 (2026-09-08) — the flip: a calendar gate that opened early, and what a public tree carries
|
||||
|
||||
- **Context and belief.** Phase 33 closed re-chartered because the flip waited on GitHub Support purging the old objects (the Activity
|
||||
view served every pre-force-push tip). The seed planned a daily probe "until PASS, ~a week", with a delete-and-recreate fallback. The
|
||||
probe passed on 2026-09-07, during Phase 33.5 — the purge took a day, not a week — so Phase 34 opened with its gate already open and the
|
||||
fallback never needed. Gate 1 (plan mode, Max) ratified the twelve Phase-33.5 candidates as R84–R95 and added three things the seed did
|
||||
not have: repository metadata and a `protect-main` ruleset (no force-push, no deletion, no bypass actors) set by Claude on the owner's
|
||||
word, a GitHub Release for v2.0.0, and the finding that a Phase-1 relic clone of the OLD history still existed on the Windows
|
||||
filesystem — deleted by Drew during planning, so C11's clone-reset step became "not applicable, with evidence".
|
||||
- **What failed, once.** The decomp.me browser step failed on its first try: the doc told Drew to paste the regenerated target listing,
|
||||
which is the project's WORD ORACLE (nothing on our side assembles it — bare register names that resolve only through `macro.inc`,
|
||||
branch targets as absolute addresses). decomp.me's assembler refused it (`invalid operands 'li a2,2'`). The step had been written for a
|
||||
private repo and never exercised (R14). The fix went into the tools, not the doc alone: `verbatim_target_s.py --gas` (the assemblable
|
||||
form), the replica's step D (the paste itself through decomp.me's own `as` wrapper, 26/26 words) and step E (the bundle written from
|
||||
the proven run); the retry matched 100% first time. Lesson, the same as P33's: a step that talks to a third party is proven through
|
||||
that party's own toolchain before the owner sits down in the browser.
|
||||
- **The outward day.** Public at 11:2x MDT; the after-flip probe 33/33 gone; both workflows green on the public repo; decomp.dev
|
||||
registered (100.00% / 100.00%, hero image); the wiki pushed (32 pages; Claude ran the push on Drew's explicit one-off R6 waiver, the
|
||||
P33 xsig precedent); the preset request (decompme/decomp.me#2106); the Archipelago note (AegeusEvander/…#1); xsig topics + homepage;
|
||||
the tools post and Drew's own `#ai` reply in the decomp.me Discord; two Decompedia rows. frogress skipped on measurement: alive, 38
|
||||
projects, admin-created, and at 100% with generated badges and decomp.dev live it would add a list row only — pending by name.
|
||||
- **Two owner decisions about the public tree (the pivot in this entry).** After the notes were sent, Drew took `docs/outreach/` out of
|
||||
the tree ("just gitignore"), and then the document archive `docs/sunset/` ("remove sunset from the pushed repo … keep the old tools in
|
||||
the sunset folder for the index, but not the other stuff"). The 33.5 design had anticipated exactly this: nothing ever linked into the
|
||||
archive, the Archive index names each file as a path, so the removal broke no link — but the folder was still cited as PRESENT in four
|
||||
wiki pages, and a trial "untrack both" pass measured that the tool census `--check` needs the retired files (a fresh clone would have
|
||||
gone red), which is why the tools half stayed. Convention going forward (on the conventions page): a retired document gets its
|
||||
Archive-index row and is deleted in the same commit; history keeps it. No rewrite for either — the drafts and the archive are in the
|
||||
published history and contain nothing ROM-derived.
|
||||
- **Scratch.** The untracked `.run/` campaign scratch that the frontier's emptying made worthless — per-function draft trees, permuter
|
||||
work dirs, wave shards, resolver and rtu shadow trees, the P29/P32 crack scratch — was pruned after sampling each tree (≈9 GB; disk
|
||||
46 → 37 GB used); `.run/public_rewrite/` kept only the probe's two inputs (1.1 GB → 352 KB) so the probe stays runnable; every
|
||||
tracked `.run` file (952) and the keep-set (`obj40`, the sigs, `decompme`, `wiki`, `verbatim_targets`, `ghidra_export`) untouched.
|
||||
- **Hindsight — the better path.** (1) Exercise every third-party step through that party's own toolchain before the owner's browser
|
||||
session, and let the tool write the paste (done now for decomp.me; the same rule would have caught it in P33). (2) Decide what a
|
||||
public tree carries BEFORE the flip, not after: the outreach drafts and the archive were always going to leave; deciding at 33.5 would
|
||||
have spared two untrack commits on a public history. (3) A "daily probe for a week" should start the day the ticket is filed and be
|
||||
read the next morning — it passed in one day and the phase could have opened a day earlier.
|
||||
|
||||
@@ -19,8 +19,9 @@ database regenerable from `config/ghidra/`). The whole history was rewritten wit
|
||||
no license asserted over `src/`, NOTICE, THIRD_PARTY, badges, ROM-free CI), the releases (gcc-2.7.2 map, xsig, the permuter driver —
|
||||
offered upstream and declined, the drafter write-up, the decomp.me preset proven through decomp.me's own toolchain), the outreach notes,
|
||||
story, retrospective, wiki (13 pages) + how-to (13 chapters), and the **Gen3 charter** (`docs/gen3-handoff.md`, `docs/gen3-standards.md`)
|
||||
are written. **The repository is still PRIVATE:** GitHub Support must purge the old objects first (ticket #4736982; the Activity view
|
||||
publishes every pre-force-push tip, so a clean tree is not enough). **Phase 33.5 CLOSED 2026-09-08 (v1.32.1; `PhaseEnd_Phase33.5.md`):** the docs sub-phase before the flip — the wiki is the single
|
||||
are written. **The repository went PUBLIC on 2026-09-08 (Phase 34 task 1, Drew)** after the purge probe passed twice — 2026-09-07 and
|
||||
again immediately before the flip (33 of 33 sampled old hashes gone; Support ticket #4736982 had done its work) — with the `protect-main`
|
||||
ruleset (no force-push, no deletion) guarding the published history. **Phase 33.5 CLOSED 2026-09-08 (v1.32.1; `PhaseEnd_Phase33.5.md`):** the docs sub-phase before the flip — the wiki is the single
|
||||
source of truth (six new pages incl. the Reference index and the Archive index; 59 documents and 34 tools retired into indexed sunset
|
||||
folders; `doc_links` with six checks), the tracked `.run/` is flip-ready (1,086 → 868 + the phase's own evidence; the two ROM listings
|
||||
untracked under an audit-only rule + a content check), the memory store reconciled (80 == 80; an 18-file seed), and **the day-one decomp
|
||||
@@ -30,8 +31,9 @@ kit-corpus`, asserted equal in tools-health), `tools/kit_coverage.py` (every rul
|
||||
its first run found 26 + 21 uncited, three genuine gaps), and the worklog mining pass (777 candidates / 634 banked / 143 new → DK-69–80).
|
||||
R74–R83 ratified at its gate 1; candidates (i)–(xii) proposed for Phase 34 gate 1. **Phase 34's gate is OPEN: the purge probe PASSED
|
||||
2026-09-07.** **Phase 34 = the flip + the outward actions + C11 + the v2.0.0 PhaseEnd = Gen2 EXIT** (`docs/phase34-seed.md`).
|
||||
**Phase 34 OPENED 2026-09-08 (S93; gate 1 in plan mode at Max; R84–R95 ratified; the plan and the live state are in
|
||||
`phase-ends/CURRENT_PHASE.md`).** **Gen3 opens at Phase 35** — its seed now includes the S91-b types doctrine (R95, the canonical
|
||||
**Phase 34 IN PROGRESS (opened 2026-09-08, S93; gate 1 in plan mode at Max; R84–R95 ratified; the plan and the live state are in
|
||||
`phase-ends/CURRENT_PHASE.md`): the flip done, decomp.dev live, the wiki live (32 pages), the decomp.me preset requested, the
|
||||
Archipelago note sent, the tools announced; `docs/outreach/` and `docs/sunset/` left the tree on the owner's decision.** **Gen3 opens at Phase 35** — its seed now includes the S91-b types doctrine (R95, the canonical
|
||||
type layer).
|
||||
|
||||
## 1. Corrections and supersessions of PROJECT_CONTEXT.md recorded in PhaseEnds (P1: the constitution is never edited)
|
||||
@@ -42,7 +44,7 @@ type layer).
|
||||
backups → today: **one commit per completed task after CURRENT_PHASE.md is updated; a bank commits the moment it
|
||||
exists (R42)**. Claude commits; **Drew pushes** (R6); the PhaseEnd + archived log are left uncommitted for Drew's
|
||||
milestone-close commit; no `Co-Authored-By` (R5).
|
||||
- **H1** was relaxed while private (R1, 2026-06-10 → 2026-09-06) and is **IN FORCE again since Phase 33 C3 (2026-09-06)**: the ROM-derived and proprietary paths left the index and were purged from the whole history before the public flip (`docs/public-flip-runbook.md`); R1 is historical. **R20's backup home** is now the text export `config/ghidra/` (+ `tools/ghidra_rebuild.sh --proof`), `dumps/CHECKSUMS.sha1`, `tools/psyq_CHECKSUMS.sha256`, and the private archive repo `Druthulu/BFM-decomp-archive` (the pre-rewrite history); **never `git clean -x`** (CLAUDE.md fail-safe).
|
||||
- **H1** was relaxed while private (R1, 2026-06-10 → 2026-09-06) and is **IN FORCE again since Phase 33 C3 (2026-09-06)**: the ROM-derived and proprietary paths left the index and were purged from the whole history before the public flip (`docs/public-flip-runbook.md`); R1 is historical. **The flip itself happened on 2026-09-08 (Phase 34 task 1).** **R20's backup home** is now the text export `config/ghidra/` (+ `tools/ghidra_rebuild.sh --proof`), `dumps/CHECKSUMS.sha1`, `tools/psyq_CHECKSUMS.sha256`, and the private archive repo `Druthulu/BFM-decomp-archive` (the pre-rewrite history); **never `git clean -x`** (CLAUDE.md fail-safe).
|
||||
- **Environment:** all-in-WSL (R2); Ghidra runs **headless** (MCP server via `tools/ghidra_mcp_start.sh` / the
|
||||
SessionStart hook; symbols persist only via `tools/ghidra_apply_symbols.sh`, MCP renames do not); PCSX-Redux is
|
||||
Windows-native bridged over the web API (R11); no `/tmp`, scratch under `.run/` (R12).
|
||||
@@ -355,8 +357,8 @@ the binding text. R44–R63 one-liners are the P31 table, ratified at P32 gate 1
|
||||
## 4. Where things live (the doc map a session needs)
|
||||
**The wiki is the source of truth for documentation (Phase 33.5):** `docs/wiki/Reference-index.md` lists every live reference and
|
||||
generated file with how to read it — start there; `docs/wiki/Docs-and-scratch-conventions.md` says where each kind of knowledge goes;
|
||||
`docs/wiki/The-ROM-firewall.md` is the no-ROM policy; `docs/wiki/Archive-index.md` records every document retired to `docs/sunset/`
|
||||
(kept for the owner's review; nothing links into it). The files a session touches most: `docs/SETUP.md` (environment, every tool's
|
||||
`docs/wiki/The-ROM-firewall.md` is the no-ROM policy; `docs/wiki/Archive-index.md` records every document retired at Phase 33.5
|
||||
(the archive folder itself left the tree at Phase 34, 2026-09-08 — the rows are the record; nothing ever linked into it). The files a session touches most: `docs/SETUP.md` (environment, every tool's
|
||||
row, commands) · `docs/matching-cookbook.md` §1–§500 (idioms; 3.5 MB — grep by §, never read whole) + `docs/cookbook-index.md`
|
||||
(symptom-keyed, derived, 566 KB — grep) · `docs/gcc-2.7.2-map/` (the compiler map) · `docs/wave-playbook.md` (THE wave procedure;
|
||||
the OpenRouter-era `automation-runbook.md` is archived) · `docs/effort-map.md` · `docs/phase34-seed.md` (the next phase's seed;
|
||||
|
||||
@@ -1004,15 +1004,19 @@ def audit_coverage(limit=None, verbose=False):
|
||||
defects, notc = [], []
|
||||
for p, e in bad:
|
||||
stmt = e.split(' -> ')[0].strip("'\"")
|
||||
# The scratch dir is created here (P34 S93): it lived under .run/audit/, which a scratch prune removed, and the
|
||||
# write raised — the bare `except` below then counted every unparsed line as a parser DEFECT (the last close had
|
||||
# read the same four lines as NOT-C). An adjudication that cannot run is a REFUSAL, never a verdict (R43/R57).
|
||||
src = os.path.join(REPO, '.run/audit/cdecl/adj.c')
|
||||
os.makedirs(os.path.dirname(src), exist_ok=True)
|
||||
try:
|
||||
src = os.path.join(REPO, '.run/audit/cdecl/adj.c')
|
||||
open(src, 'w').write(_PROBE_HDR + stmt.encode().decode('unicode_escape') + '\n')
|
||||
r = subprocess.run([CC, '-fsyntax-only', '-w', '-I' + os.path.join(REPO, 'include'),
|
||||
'-I' + os.path.join(REPO, 'src'), src],
|
||||
capture_output=True, text=True, cwd=REPO)
|
||||
(notc if r.returncode else defects).append((p, e))
|
||||
except Exception:
|
||||
defects.append((p, e))
|
||||
except Exception as ex:
|
||||
sys.exit(f'cdecl --audit: cannot adjudicate an unparsed statement with gcc ({ex}); refusing to grade it either way (R43)')
|
||||
(notc if r.returncode else defects).append((p, e))
|
||||
|
||||
print(f'[coverage] {len(tus)} TUs (cpp-derived) + {len(drafts)} drafts (raw)')
|
||||
print(f'[coverage] depth-0 statements: {stmts} declarators parsed: {decls}')
|
||||
|
||||
@@ -242,7 +242,22 @@ output) · `.run/P34/c11/` (the prune listing with sizes, `git worktree list`, t
|
||||
`master` = `be8a6cc`, the live Archive-index page (200) carries "removed from the tree at Phase 34" (`.run/P34/outward/wiki_push_c11.log`).
|
||||
`gh auth logout` stays Drew's last command after the close push. **Task 7 CLOSED.**
|
||||
|
||||
## 🛑 SESSION CHECKPOINT — PHASE 34 OPEN (S93, 2026-09-08): tasks 0a, 0–7 ✓; task 8 IN PROGRESS (Tier 1, Max confirmed by Drew "ready for task 8"): the close tools-health ran and FAILED at audit-cdecl — under diagnosis (R40: the instrument first); then the P7 walk, gate 2, the PhaseEnd v2.0.0
|
||||
### 2026-09-08 — Task 8 (part 1) — the close health chain: one instrument defect, exposed by the prune and fixed (Max; S93)
|
||||
- The first `make tools-health` of the close (`.run/P34/tools_health_close.log`, 3,416 lines) went **red at `audit-cdecl`**: "PARSER
|
||||
DEFECTS (gcc accepts, cdecl does not): 4", all four the same non-C line `extern if (...)` in `.run/drafts-T6-recanon/` drafts. At the
|
||||
33.5 close the SAME four lines were "NOT-C (gcc rejects it too): 4 → ALL ORACLES GREEN" on an unchanged sample (the sorted glob of
|
||||
`.run/drafts*/*.c`; no draft dir was pruned). **Root cause (R40 → the instrument, R57 → its write path):** the gcc adjudication writes
|
||||
its probe to `.run/audit/cdecl/adj.c`; task 7's prune removed `.run/audit/`; the write raised; the bare `except Exception` counted every
|
||||
unparsed line as a DEFECT. A refusal reported as a verdict.
|
||||
- **Fix (`tools/cdecl.py`):** `os.makedirs` for the scratch dir (the other two `.run/audit/cdecl` users already had it); an adjudication
|
||||
that cannot run now `sys.exit`s with a refusal (R43) instead of grading either way. `make audit-cdecl` alone → **unparsed 4 → DEFECTS 0,
|
||||
NOT-C 4 (`.run/drafts-T6-recanon`), gcc accepted 4,777 / REJECTED 0, "cdecl: ALL ORACLES GREEN"** (`.run/P34/c11/audit_cdecl_after_fix.log`).
|
||||
`make kit-corpus` regenerated (cdecl.py is in the corpus); `tool_census --check` OK. The full chain relaunched detached (its log
|
||||
overwrites the red one; the red run's verdict is quoted here).
|
||||
- Lesson for the ledger: a scratch prune is an instrument change — every tool that writes under a pruned tree must be re-run before the
|
||||
prune is called done (a candidate rule for the PhaseEnd).
|
||||
|
||||
## 🛑 SESSION CHECKPOINT — PHASE 34 OPEN (S93, 2026-09-08): tasks 0a, 0–7 ✓; task 8 IN PROGRESS (Tier 1, Max): the cdecl instrument fixed, the full tools-health re-running detached (`.run/P34/tools_health_close.log`); then the P7 walk, gate 2, the PhaseEnd v2.0.0
|
||||
|
||||
### 0. How to use this block
|
||||
A fresh session reads CLAUDE.md's load order, replays THIS block verbatim, and resumes at the first unchecked task above. Everything
|
||||
|
||||
+8
-4
@@ -1004,15 +1004,19 @@ def audit_coverage(limit=None, verbose=False):
|
||||
defects, notc = [], []
|
||||
for p, e in bad:
|
||||
stmt = e.split(' -> ')[0].strip("'\"")
|
||||
# The scratch dir is created here (P34 S93): it lived under .run/audit/, which a scratch prune removed, and the
|
||||
# write raised — the bare `except` below then counted every unparsed line as a parser DEFECT (the last close had
|
||||
# read the same four lines as NOT-C). An adjudication that cannot run is a REFUSAL, never a verdict (R43/R57).
|
||||
src = os.path.join(REPO, '.run/audit/cdecl/adj.c')
|
||||
os.makedirs(os.path.dirname(src), exist_ok=True)
|
||||
try:
|
||||
src = os.path.join(REPO, '.run/audit/cdecl/adj.c')
|
||||
open(src, 'w').write(_PROBE_HDR + stmt.encode().decode('unicode_escape') + '\n')
|
||||
r = subprocess.run([CC, '-fsyntax-only', '-w', '-I' + os.path.join(REPO, 'include'),
|
||||
'-I' + os.path.join(REPO, 'src'), src],
|
||||
capture_output=True, text=True, cwd=REPO)
|
||||
(notc if r.returncode else defects).append((p, e))
|
||||
except Exception:
|
||||
defects.append((p, e))
|
||||
except Exception as ex:
|
||||
sys.exit(f'cdecl --audit: cannot adjudicate an unparsed statement with gcc ({ex}); refusing to grade it either way (R43)')
|
||||
(notc if r.returncode else defects).append((p, e))
|
||||
|
||||
print(f'[coverage] {len(tus)} TUs (cpp-derived) + {len(drafts)} drafts (raw)')
|
||||
print(f'[coverage] depth-0 statements: {stmts} declarators parsed: {decls}')
|
||||
|
||||
Reference in New Issue
Block a user