From e9c66a67209cf6aa7f8736c9ef6457e3592c8671 Mon Sep 17 00:00:00 2001 From: Drew T <50529377+Druthulu@users.noreply.github.com> Date: Wed, 5 Aug 2026 00:37:59 -0600 Subject: [PATCH] =?UTF-8?q?fix(phase-30=20S39):=20close=20the=20=C2=A7134?= =?UTF-8?q?=20class=20=E2=80=94=20the=20last=20two=20line-shape=20scanners?= =?UTF-8?q?=20route=20through=20cdecl.=5Fmask?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit §134 had been patched individually in six tools; the standing note said the fix is ONE masking oracle, not a seventh regex (R33). The last two holdouts are migrated. progress.py.strip_comments — a private 2-line regex, NOT string-aware, feeding three line-shape decisions in classify(): the {-vs-; definition/declaration scan, the count('{')-count('}') body walk, and the empty-vs-real body test. A brace inside a string literal therefore mis-buckets a function in the FN-COUNT metric. Negative control: void f(void) { puts("}"); x = 1; } old regex -> body-depth -1 (the string's brace was counted) cdecl._mask -> body-depth 0 (correct) Metrics IDENTICAL before/after on today's corpus (341365/353717; REAL 339510, empty 896, stubs 12345) -- a latent defect, harmless until someone banks a function containing "{". lint_symbol_refs.strip_comments_strings — correct, but a SECOND implementation of the same masking. Deleted in favour of cdecl._mask. The one behavioural difference (_mask blanks the quote DELIMITERS, the private scanner kept them) was CHECKED not assumed: irrelevant because every token the linter hunts lives outside the quotes. Gated on the linter's OUTPUT being byte-identical across the change (it is), not on the two masks being byte-identical -- the right gate is the tool's answer, not its internals. cookbook §141 + index regenerated. No src/ or config/ change; no bytes touched. --- docs/cookbook-index.md | 6 ++++-- docs/matching-cookbook.md | 38 +++++++++++++++++++++++++++++++++++++ tools/lint_symbol_refs.py | 40 +++++++++++++++------------------------ tools/progress.py | 19 +++++++++++++++++-- 4 files changed, 74 insertions(+), 29 deletions(-) diff --git a/docs/cookbook-index.md b/docs/cookbook-index.md index 4207c558c..8ca383caa 100644 --- a/docs/cookbook-index.md +++ b/docs/cookbook-index.md @@ -2,7 +2,7 @@ > **Generated by `tools/cookbook_index.py` — do not hand-edit** (R33). Regenerate after adding a cookbook section. > -> `docs/matching-cookbook.md` is ~716 KB / 398 sections. Grepping it blind is how three P30 wave-1 agents each "discovered" an idiom that was already written down. **Start here, then read the section.** A section appears under every symptom it addresses. +> `docs/matching-cookbook.md` is ~716 KB / 399 sections. Grepping it blind is how three P30 wave-1 agents each "discovered" an idiom that was already written down. **Start here, then read the section.** A section appears under every symptom it addresses. **How to use:** name what you SEE in the diff (a stolen delay slot, an extra `la`, a swapped register pair, a `conflicting types` error), find that symptom below, read those sections first. If nothing fits, THEN grind — and add a section when you win. @@ -173,7 +173,7 @@ - **§3-Do** — NOT "strip the duplicate typedef" — it breaks the extern that uses it L8030 - **§3-The** — type-form rules L8872 -### declarations, prototypes & K&R (52) +### declarations, prototypes & K&R (53) - **§3-T4** — Branch polarity: invert the source condition to flip gcc's chosen branch L90 - **§8c** — Splitting a TU means rebuilding its DECLARATION ENVIRONMENT, not moving text (Phase 26 session 6) L437 @@ -227,6 +227,7 @@ - **§136f** — Two declaration sub-cases the reconcile lane surfaced (lane now 15/15 lifetime) L9176 - **§138** — The propagation lanes: a gate refusal is a DECLARATION, and which lever you owe depends on blast radius L9405 - **Reconciling** — a gate-refused draft: which way you edit depends on WHERE the TU's decl is L9555 +- **§141** — The §134 class is CLOSED: every line-shape decision now routes through `cdecl._mask` (P30 S39) L9730 ### jump tables & switches (26) @@ -1058,3 +1059,4 @@ - **§3-The** — two instrument defects it exposed L9679 - **§3-The** — same swallow, twice more, in the integration spine L9699 - **§3-Two** — wrong mechanisms I chased first, and why they were wrong L9710 +- **§141** — The §134 class is CLOSED: every line-shape decision now routes through `cdecl._mask` (P30 S39) L9730 diff --git a/docs/matching-cookbook.md b/docs/matching-cookbook.md index 144c93eb3..9f6d6b367 100644 --- a/docs/matching-cookbook.md +++ b/docs/matching-cookbook.md @@ -9724,3 +9724,41 @@ metrics moved in opposite directions"** · **"a digest disagrees with the tree i **The law:** *a committed number is a claim about a tree; if it cannot be recomputed from that tree, it is not evidence — and it must never gate a lever.* (R32/R34/R35; and R14 — I asserted two mechanisms before deriving either.) + +--- + +## §141 — The §134 class is CLOSED: every line-shape decision now routes through `cdecl._mask` (P30 S39) + +§134 ("multi-line / comment / string blindness in a hand-rolled line scanner") had been fixed +*individually* in six tools, each time by patching that tool. The standing note said the real fix is +routing every line-shape decision through the ONE masking oracle rather than writing a seventh +regex. Done — the last two holdouts are migrated: + +**`progress.py.strip_comments`** — was a private two-line regex stripping `/*…*/` and `//…`, **not +string-aware**. Every caller is a line-shape decision on its output: the `{`-vs-`;` scan that +separates a definition from a declaration, the `count('{') - count('}')` body-depth walk, and the +empty-vs-real body test. So a brace inside a string literal mis-buckets a function in the **fn-count +metric**. Demonstrated: + + void f(void) { puts("}"); x = 1; } + old regex -> body-depth walk = -1 (unbalanced: the string's brace was counted) + cdecl._mask-> body-depth walk = 0 (correct) + +Metrics were **identical before and after** on today's corpus (341,365 / 353,717; REAL 339,510, +empty 896, stubs 12,345) — i.e. no live source currently trips it. That is what a latent defect looks +like: harmless until the day someone banks a function containing `"{"`, and then silently wrong. + +**`lint_symbol_refs.strip_comments_strings`** — was *correct* (char-by-char, escape-aware) but was a +SECOND implementation of the same masking. Deleted in favour of `cdecl._mask`. One behavioural +difference existed and was checked rather than assumed: `_mask` blanks the quote DELIMITERS too, +where the private scanner kept them — irrelevant, because every token the linter hunts +(`func_`, `D_`, and the bare 2nd arg of `INCLUDE_ASM("...", func_X)`) lives outside the +quotes either way. **Gated on the linter's own output being byte-identical across the change, not on +the two masks being byte-identical** — the right gate is the tool's answer, not its internals. + +**The general law (R33):** when the same defect class has been patched N times in N tools, the fix is +not the N+1th patch — it is deleting N−1 implementations. A private copy of a shared decision is a +divergence waiting to happen, and it diverges silently. + +**Symptom line for the index:** **"a scanner miscounts braces/semicolons"** · **"two tools disagree +about what a line is"**. diff --git a/tools/lint_symbol_refs.py b/tools/lint_symbol_refs.py index 389615ffe..dfe12c366 100644 --- a/tools/lint_symbol_refs.py +++ b/tools/lint_symbol_refs.py @@ -17,6 +17,8 @@ Exit 0 = clean; exit 1 = stale refs found (printed as file:line func_ -> c """ import re, glob, os, sys +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +import cdecl # §134/R33: the ONE comment/string masking oracle REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) # The files a real binary actually stacks (main/resident/overlays). The proto symbol files @@ -64,31 +66,19 @@ def asm_labeled_addrs(files): def strip_comments_strings(src): - """blank out /* */ + // comments and string/char-literal CONTENTS (keeping newlines so line - numbers stay correct), so a `func_` mentioned only in a comment/string isn't flagged. - A bare INCLUDE_ASM(func_) token lives OUTSIDE the quotes, so it survives.""" - out = [] - i, n = 0, len(src) - while i < n: - c = src[i] - two = src[i:i+2] - if two == "/*": - j = src.find("*/", i + 2) - j = n if j < 0 else j + 2 - out.append("".join(ch if ch == "\n" else " " for ch in src[i:j])); i = j - elif two == "//": - j = src.find("\n", i) - j = n if j < 0 else j - out.append(" " * (j - i)); i = j - elif c in '"\'': - q = c; j = i + 1 - while j < n and src[j] != q: - j += 2 if src[j] == "\\" else 1 - j = min(j + 1, n) - out.append(q + " " * (j - i - 2) + q if j - i >= 2 else src[i:j]); i = j - else: - out.append(c); i += 1 - return "".join(out) + """Blank comments + string/char-literal contents, via the ONE masking oracle (`cdecl._mask`). + + §134 / R33 (P30 S39). This used to be a private char-by-char scanner — correct, but a SECOND + implementation of the masking `cdecl._mask` already owns, and the §134 class (a line-shape + decision made against unmasked text) had by then appeared in six tools precisely because each + one kept its own copy. One oracle cannot diverge from itself. + + Behavioural note, verified before the swap: `_mask` blanks the quote DELIMITERS as well as the + content, where this scanner kept the quotes. That is irrelevant here — both blank the contents, + and every token this linter hunts (`func_`, `D_`, and the bare 2nd argument of + `INCLUDE_ASM("...", func_X)`) lives OUTSIDE the quotes either way. Gated on the linter's own + output being byte-identical across the change, not on the masks being byte-identical.""" + return cdecl._mask(src) def main(): diff --git a/tools/progress.py b/tools/progress.py index 5aa2a4b4e..bbfdcb458 100644 --- a/tools/progress.py +++ b/tools/progress.py @@ -12,6 +12,8 @@ Usage: tools/progress.py --binary # report a non-default binary (default: main = the EXE) """ import os, re, sys, hashlib, pathlib +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +import cdecl # §134/R33: the ONE comment/string masking oracle ROOT = pathlib.Path(__file__).resolve().parent.parent @@ -397,8 +399,21 @@ def find_s(name): INSTR = re.compile(r'^\s*/\*\s*[0-9A-Fa-f]+\s+[0-9A-Fa-f]+\s+[0-9A-Fa-f]+\s*\*/\s+[a-z]') def strip_comments(s): - s = re.sub(r'/\*.*?\*/', '', s, flags=re.S) - return re.sub(r'//[^\n]*', '', s) + """Blank comments AND string/char-literal contents, via the ONE masking oracle (`cdecl._mask`). + + §134 / R33 (P30 S39). This was a private two-line regex that removed `/*…*/` and `//…` and was + NOT string-aware — so a brace or a semicolon inside a C string literal (`printf("}")`, a path + like `"a;b"`) was seen as real syntax by the callers below, every one of which is a LINE-SHAPE + decision on this function's output: the `{`-vs-`;` scan that separates a definition from a + declaration, the `count('{') - count('}')` body-depth walk, and the empty-vs-real body test. + A miscount there mis-buckets a function in the fn-count metric. + + That is the §134 class, which had by this point appeared in SIX tools; the fix is routing every + line-shape decision through `cdecl._mask` rather than writing a seventh regex. `_mask` is + length-preserving (it blanks rather than deletes), which is strictly better here: the callers + compare offsets (`br < sm`) and count characters, and blanking keeps those offsets valid + against the ORIGINAL line while removing the false tokens.""" + return cdecl._mask(s) def is_data_blob(name): """A .s with a code label (glabel/jlabel) is a function; data-only (dlabel, no code) is a blob."""