fix(phase-30 S39): close the §134 class — the last two line-shape scanners route through cdecl._mask

§134 had been patched individually in six tools; the standing note said the fix is ONE masking
oracle, not a seventh regex (R33). The last two holdouts are migrated.

progress.py.strip_comments — a private 2-line regex, NOT string-aware, feeding three line-shape
decisions in classify(): the {-vs-; definition/declaration scan, the count('{')-count('}') body
walk, and the empty-vs-real body test. A brace inside a string literal therefore mis-buckets a
function in the FN-COUNT metric. Negative control:

    void f(void) { puts("}"); x = 1; }
      old regex   -> body-depth -1  (the string's brace was counted)
      cdecl._mask -> body-depth  0  (correct)

Metrics IDENTICAL before/after on today's corpus (341365/353717; REAL 339510, empty 896, stubs
12345) -- a latent defect, harmless until someone banks a function containing "{".

lint_symbol_refs.strip_comments_strings — correct, but a SECOND implementation of the same
masking. Deleted in favour of cdecl._mask. The one behavioural difference (_mask blanks the quote
DELIMITERS, the private scanner kept them) was CHECKED not assumed: irrelevant because every token
the linter hunts lives outside the quotes. Gated on the linter's OUTPUT being byte-identical across
the change (it is), not on the two masks being byte-identical -- the right gate is the tool's
answer, not its internals.

cookbook §141 + index regenerated. No src/ or config/ change; no bytes touched.
This commit is contained in:
Drew T
2026-08-05 00:37:59 -06:00
parent 85519f1e32
commit e9c66a6720
4 changed files with 74 additions and 29 deletions
+4 -2
View File
@@ -2,7 +2,7 @@
> **Generated by `tools/cookbook_index.py` — do not hand-edit** (R33). Regenerate after adding a cookbook section.
>
> `docs/matching-cookbook.md` is ~716 KB / 398 sections. Grepping it blind is how three P30 wave-1 agents each "discovered" an idiom that was already written down. **Start here, then read the section.** A section appears under every symptom it addresses.
> `docs/matching-cookbook.md` is ~716 KB / 399 sections. Grepping it blind is how three P30 wave-1 agents each "discovered" an idiom that was already written down. **Start here, then read the section.** A section appears under every symptom it addresses.
**How to use:** name what you SEE in the diff (a stolen delay slot, an extra `la`, a swapped register pair, a `conflicting types` error), find that symptom below, read those sections first. If nothing fits, THEN grind — and add a section when you win.
@@ -173,7 +173,7 @@
- **§3-Do** — NOT "strip the duplicate typedef" — it breaks the extern that uses it <sub>L8030</sub>
- **§3-The** — type-form rules <sub>L8872</sub>
### declarations, prototypes & K&R (52)
### declarations, prototypes & K&R (53)
- **§3-T4** — Branch polarity: invert the source condition to flip gcc's chosen branch <sub>L90</sub>
- **§8c** — Splitting a TU means rebuilding its DECLARATION ENVIRONMENT, not moving text (Phase 26 session 6) <sub>L437</sub>
@@ -227,6 +227,7 @@
- **§136f** — Two declaration sub-cases the reconcile lane surfaced (lane now 15/15 lifetime) <sub>L9176</sub>
- **§138** — The propagation lanes: a gate refusal is a DECLARATION, and which lever you owe depends on blast radius <sub>L9405</sub>
- **Reconciling** — a gate-refused draft: which way you edit depends on WHERE the TU's decl is <sub>L9555</sub>
- **§141** — The §134 class is CLOSED: every line-shape decision now routes through `cdecl._mask` (P30 S39) <sub>L9730</sub>
### jump tables & switches (26)
@@ -1058,3 +1059,4 @@
- **§3-The** — two instrument defects it exposed <sub>L9679</sub>
- **§3-The** — same swallow, twice more, in the integration spine <sub>L9699</sub>
- **§3-Two** — wrong mechanisms I chased first, and why they were wrong <sub>L9710</sub>
- **§141** — The §134 class is CLOSED: every line-shape decision now routes through `cdecl._mask` (P30 S39) <sub>L9730</sub>
+38
View File
@@ -9724,3 +9724,41 @@ metrics moved in opposite directions"** · **"a digest disagrees with the tree i
**The law:** *a committed number is a claim about a tree; if it cannot be recomputed from that tree,
it is not evidence — and it must never gate a lever.* (R32/R34/R35; and R14 — I asserted two
mechanisms before deriving either.)
---
## §141 — The §134 class is CLOSED: every line-shape decision now routes through `cdecl._mask` (P30 S39)
§134 ("multi-line / comment / string blindness in a hand-rolled line scanner") had been fixed
*individually* in six tools, each time by patching that tool. The standing note said the real fix is
routing every line-shape decision through the ONE masking oracle rather than writing a seventh
regex. Done — the last two holdouts are migrated:
**`progress.py.strip_comments`** — was a private two-line regex stripping `/*…*/` and `//…`, **not
string-aware**. Every caller is a line-shape decision on its output: the `{`-vs-`;` scan that
separates a definition from a declaration, the `count('{') - count('}')` body-depth walk, and the
empty-vs-real body test. So a brace inside a string literal mis-buckets a function in the **fn-count
metric**. Demonstrated:
void f(void) { puts("}"); x = 1; }
old regex -> body-depth walk = -1 (unbalanced: the string's brace was counted)
cdecl._mask-> body-depth walk = 0 (correct)
Metrics were **identical before and after** on today's corpus (341,365 / 353,717; REAL 339,510,
empty 896, stubs 12,345) — i.e. no live source currently trips it. That is what a latent defect looks
like: harmless until the day someone banks a function containing `"{"`, and then silently wrong.
**`lint_symbol_refs.strip_comments_strings`** — was *correct* (char-by-char, escape-aware) but was a
SECOND implementation of the same masking. Deleted in favour of `cdecl._mask`. One behavioural
difference existed and was checked rather than assumed: `_mask` blanks the quote DELIMITERS too,
where the private scanner kept them — irrelevant, because every token the linter hunts
(`func_<ADDR>`, `D_<ADDR>`, and the bare 2nd arg of `INCLUDE_ASM("...", func_X)`) lives outside the
quotes either way. **Gated on the linter's own output being byte-identical across the change, not on
the two masks being byte-identical** — the right gate is the tool's answer, not its internals.
**The general law (R33):** when the same defect class has been patched N times in N tools, the fix is
not the N+1th patch — it is deleting N−1 implementations. A private copy of a shared decision is a
divergence waiting to happen, and it diverges silently.
**Symptom line for the index:** **"a scanner miscounts braces/semicolons"** · **"two tools disagree
about what a line is"**.
+15 -25
View File
@@ -17,6 +17,8 @@ Exit 0 = clean; exit 1 = stale refs found (printed as file:line func_<ADDR> -> c
"""
import re, glob, os, sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import cdecl # §134/R33: the ONE comment/string masking oracle
REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
# The files a real binary actually stacks (main/resident/overlays). The proto symbol files
@@ -64,31 +66,19 @@ def asm_labeled_addrs(files):
def strip_comments_strings(src):
"""blank out /* */ + // comments and string/char-literal CONTENTS (keeping newlines so line
numbers stay correct), so a `func_<ADDR>` mentioned only in a comment/string isn't flagged.
A bare INCLUDE_ASM(func_<ADDR>) token lives OUTSIDE the quotes, so it survives."""
out = []
i, n = 0, len(src)
while i < n:
c = src[i]
two = src[i:i+2]
if two == "/*":
j = src.find("*/", i + 2)
j = n if j < 0 else j + 2
out.append("".join(ch if ch == "\n" else " " for ch in src[i:j])); i = j
elif two == "//":
j = src.find("\n", i)
j = n if j < 0 else j
out.append(" " * (j - i)); i = j
elif c in '"\'':
q = c; j = i + 1
while j < n and src[j] != q:
j += 2 if src[j] == "\\" else 1
j = min(j + 1, n)
out.append(q + " " * (j - i - 2) + q if j - i >= 2 else src[i:j]); i = j
else:
out.append(c); i += 1
return "".join(out)
"""Blank comments + string/char-literal contents, via the ONE masking oracle (`cdecl._mask`).
§134 / R33 (P30 S39). This used to be a private char-by-char scanner — correct, but a SECOND
implementation of the masking `cdecl._mask` already owns, and the §134 class (a line-shape
decision made against unmasked text) had by then appeared in six tools precisely because each
one kept its own copy. One oracle cannot diverge from itself.
Behavioural note, verified before the swap: `_mask` blanks the quote DELIMITERS as well as the
content, where this scanner kept the quotes. That is irrelevant here — both blank the contents,
and every token this linter hunts (`func_<ADDR>`, `D_<ADDR>`, and the bare 2nd argument of
`INCLUDE_ASM("...", func_X)`) lives OUTSIDE the quotes either way. Gated on the linter's own
output being byte-identical across the change, not on the masks being byte-identical."""
return cdecl._mask(src)
def main():
+17 -2
View File
@@ -12,6 +12,8 @@ Usage:
tools/progress.py --binary <alias> # report a non-default binary (default: main = the EXE)
"""
import os, re, sys, hashlib, pathlib
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import cdecl # §134/R33: the ONE comment/string masking oracle
ROOT = pathlib.Path(__file__).resolve().parent.parent
@@ -397,8 +399,21 @@ def find_s(name):
INSTR = re.compile(r'^\s*/\*\s*[0-9A-Fa-f]+\s+[0-9A-Fa-f]+\s+[0-9A-Fa-f]+\s*\*/\s+[a-z]')
def strip_comments(s):
s = re.sub(r'/\*.*?\*/', '', s, flags=re.S)
return re.sub(r'//[^\n]*', '', s)
"""Blank comments AND string/char-literal contents, via the ONE masking oracle (`cdecl._mask`).
§134 / R33 (P30 S39). This was a private two-line regex that removed `/*…*/` and `//…` and was
NOT string-aware — so a brace or a semicolon inside a C string literal (`printf("}")`, a path
like `"a;b"`) was seen as real syntax by the callers below, every one of which is a LINE-SHAPE
decision on this function's output: the `{`-vs-`;` scan that separates a definition from a
declaration, the `count('{') - count('}')` body-depth walk, and the empty-vs-real body test.
A miscount there mis-buckets a function in the fn-count metric.
That is the §134 class, which had by this point appeared in SIX tools; the fix is routing every
line-shape decision through `cdecl._mask` rather than writing a seventh regex. `_mask` is
length-preserving (it blanks rather than deletes), which is strictly better here: the callers
compare offsets (`br < sm`) and count characters, and blanking keeps those offsets valid
against the ORIGINAL line while removing the false tokens."""
return cdecl._mask(s)
def is_data_blob(name):
"""A .s with a code label (glabel/jlabel) is a function; data-only (dlabel, no code) is a blob."""