mirror of
https://github.com/Druthulu/BFM-decomp
synced 2026-09-29 07:10:32 -04:00
fix(phase-30 S39): close the §134 class — the last two line-shape scanners route through cdecl._mask
§134 had been patched individually in six tools; the standing note said the fix is ONE masking
oracle, not a seventh regex (R33). The last two holdouts are migrated.
progress.py.strip_comments — a private 2-line regex, NOT string-aware, feeding three line-shape
decisions in classify(): the {-vs-; definition/declaration scan, the count('{')-count('}') body
walk, and the empty-vs-real body test. A brace inside a string literal therefore mis-buckets a
function in the FN-COUNT metric. Negative control:
void f(void) { puts("}"); x = 1; }
old regex -> body-depth -1 (the string's brace was counted)
cdecl._mask -> body-depth 0 (correct)
Metrics IDENTICAL before/after on today's corpus (341365/353717; REAL 339510, empty 896, stubs
12345) -- a latent defect, harmless until someone banks a function containing "{".
lint_symbol_refs.strip_comments_strings — correct, but a SECOND implementation of the same
masking. Deleted in favour of cdecl._mask. The one behavioural difference (_mask blanks the quote
DELIMITERS, the private scanner kept them) was CHECKED not assumed: irrelevant because every token
the linter hunts lives outside the quotes. Gated on the linter's OUTPUT being byte-identical across
the change (it is), not on the two masks being byte-identical -- the right gate is the tool's
answer, not its internals.
cookbook §141 + index regenerated. No src/ or config/ change; no bytes touched.
This commit is contained in:
@@ -2,7 +2,7 @@
|
||||
|
||||
> **Generated by `tools/cookbook_index.py` — do not hand-edit** (R33). Regenerate after adding a cookbook section.
|
||||
>
|
||||
> `docs/matching-cookbook.md` is ~716 KB / 398 sections. Grepping it blind is how three P30 wave-1 agents each "discovered" an idiom that was already written down. **Start here, then read the section.** A section appears under every symptom it addresses.
|
||||
> `docs/matching-cookbook.md` is ~716 KB / 399 sections. Grepping it blind is how three P30 wave-1 agents each "discovered" an idiom that was already written down. **Start here, then read the section.** A section appears under every symptom it addresses.
|
||||
|
||||
**How to use:** name what you SEE in the diff (a stolen delay slot, an extra `la`, a swapped register pair, a `conflicting types` error), find that symptom below, read those sections first. If nothing fits, THEN grind — and add a section when you win.
|
||||
|
||||
@@ -173,7 +173,7 @@
|
||||
- **§3-Do** — NOT "strip the duplicate typedef" — it breaks the extern that uses it <sub>L8030</sub>
|
||||
- **§3-The** — type-form rules <sub>L8872</sub>
|
||||
|
||||
### declarations, prototypes & K&R (52)
|
||||
### declarations, prototypes & K&R (53)
|
||||
|
||||
- **§3-T4** — Branch polarity: invert the source condition to flip gcc's chosen branch <sub>L90</sub>
|
||||
- **§8c** — Splitting a TU means rebuilding its DECLARATION ENVIRONMENT, not moving text (Phase 26 session 6) <sub>L437</sub>
|
||||
@@ -227,6 +227,7 @@
|
||||
- **§136f** — Two declaration sub-cases the reconcile lane surfaced (lane now 15/15 lifetime) <sub>L9176</sub>
|
||||
- **§138** — The propagation lanes: a gate refusal is a DECLARATION, and which lever you owe depends on blast radius <sub>L9405</sub>
|
||||
- **Reconciling** — a gate-refused draft: which way you edit depends on WHERE the TU's decl is <sub>L9555</sub>
|
||||
- **§141** — The §134 class is CLOSED: every line-shape decision now routes through `cdecl._mask` (P30 S39) <sub>L9730</sub>
|
||||
|
||||
### jump tables & switches (26)
|
||||
|
||||
@@ -1058,3 +1059,4 @@
|
||||
- **§3-The** — two instrument defects it exposed <sub>L9679</sub>
|
||||
- **§3-The** — same swallow, twice more, in the integration spine <sub>L9699</sub>
|
||||
- **§3-Two** — wrong mechanisms I chased first, and why they were wrong <sub>L9710</sub>
|
||||
- **§141** — The §134 class is CLOSED: every line-shape decision now routes through `cdecl._mask` (P30 S39) <sub>L9730</sub>
|
||||
|
||||
@@ -9724,3 +9724,41 @@ metrics moved in opposite directions"** · **"a digest disagrees with the tree i
|
||||
**The law:** *a committed number is a claim about a tree; if it cannot be recomputed from that tree,
|
||||
it is not evidence — and it must never gate a lever.* (R32/R34/R35; and R14 — I asserted two
|
||||
mechanisms before deriving either.)
|
||||
|
||||
---
|
||||
|
||||
## §141 — The §134 class is CLOSED: every line-shape decision now routes through `cdecl._mask` (P30 S39)
|
||||
|
||||
§134 ("multi-line / comment / string blindness in a hand-rolled line scanner") had been fixed
|
||||
*individually* in six tools, each time by patching that tool. The standing note said the real fix is
|
||||
routing every line-shape decision through the ONE masking oracle rather than writing a seventh
|
||||
regex. Done — the last two holdouts are migrated:
|
||||
|
||||
**`progress.py.strip_comments`** — was a private two-line regex stripping `/*…*/` and `//…`, **not
|
||||
string-aware**. Every caller is a line-shape decision on its output: the `{`-vs-`;` scan that
|
||||
separates a definition from a declaration, the `count('{') - count('}')` body-depth walk, and the
|
||||
empty-vs-real body test. So a brace inside a string literal mis-buckets a function in the **fn-count
|
||||
metric**. Demonstrated:
|
||||
|
||||
void f(void) { puts("}"); x = 1; }
|
||||
old regex -> body-depth walk = -1 (unbalanced: the string's brace was counted)
|
||||
cdecl._mask-> body-depth walk = 0 (correct)
|
||||
|
||||
Metrics were **identical before and after** on today's corpus (341,365 / 353,717; REAL 339,510,
|
||||
empty 896, stubs 12,345) — i.e. no live source currently trips it. That is what a latent defect looks
|
||||
like: harmless until the day someone banks a function containing `"{"`, and then silently wrong.
|
||||
|
||||
**`lint_symbol_refs.strip_comments_strings`** — was *correct* (char-by-char, escape-aware) but was a
|
||||
SECOND implementation of the same masking. Deleted in favour of `cdecl._mask`. One behavioural
|
||||
difference existed and was checked rather than assumed: `_mask` blanks the quote DELIMITERS too,
|
||||
where the private scanner kept them — irrelevant, because every token the linter hunts
|
||||
(`func_<ADDR>`, `D_<ADDR>`, and the bare 2nd arg of `INCLUDE_ASM("...", func_X)`) lives outside the
|
||||
quotes either way. **Gated on the linter's own output being byte-identical across the change, not on
|
||||
the two masks being byte-identical** — the right gate is the tool's answer, not its internals.
|
||||
|
||||
**The general law (R33):** when the same defect class has been patched N times in N tools, the fix is
|
||||
not the N+1th patch — it is deleting N−1 implementations. A private copy of a shared decision is a
|
||||
divergence waiting to happen, and it diverges silently.
|
||||
|
||||
**Symptom line for the index:** **"a scanner miscounts braces/semicolons"** · **"two tools disagree
|
||||
about what a line is"**.
|
||||
|
||||
+15
-25
@@ -17,6 +17,8 @@ Exit 0 = clean; exit 1 = stale refs found (printed as file:line func_<ADDR> -> c
|
||||
"""
|
||||
import re, glob, os, sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
import cdecl # §134/R33: the ONE comment/string masking oracle
|
||||
REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
# The files a real binary actually stacks (main/resident/overlays). The proto symbol files
|
||||
@@ -64,31 +66,19 @@ def asm_labeled_addrs(files):
|
||||
|
||||
|
||||
def strip_comments_strings(src):
|
||||
"""blank out /* */ + // comments and string/char-literal CONTENTS (keeping newlines so line
|
||||
numbers stay correct), so a `func_<ADDR>` mentioned only in a comment/string isn't flagged.
|
||||
A bare INCLUDE_ASM(func_<ADDR>) token lives OUTSIDE the quotes, so it survives."""
|
||||
out = []
|
||||
i, n = 0, len(src)
|
||||
while i < n:
|
||||
c = src[i]
|
||||
two = src[i:i+2]
|
||||
if two == "/*":
|
||||
j = src.find("*/", i + 2)
|
||||
j = n if j < 0 else j + 2
|
||||
out.append("".join(ch if ch == "\n" else " " for ch in src[i:j])); i = j
|
||||
elif two == "//":
|
||||
j = src.find("\n", i)
|
||||
j = n if j < 0 else j
|
||||
out.append(" " * (j - i)); i = j
|
||||
elif c in '"\'':
|
||||
q = c; j = i + 1
|
||||
while j < n and src[j] != q:
|
||||
j += 2 if src[j] == "\\" else 1
|
||||
j = min(j + 1, n)
|
||||
out.append(q + " " * (j - i - 2) + q if j - i >= 2 else src[i:j]); i = j
|
||||
else:
|
||||
out.append(c); i += 1
|
||||
return "".join(out)
|
||||
"""Blank comments + string/char-literal contents, via the ONE masking oracle (`cdecl._mask`).
|
||||
|
||||
§134 / R33 (P30 S39). This used to be a private char-by-char scanner — correct, but a SECOND
|
||||
implementation of the masking `cdecl._mask` already owns, and the §134 class (a line-shape
|
||||
decision made against unmasked text) had by then appeared in six tools precisely because each
|
||||
one kept its own copy. One oracle cannot diverge from itself.
|
||||
|
||||
Behavioural note, verified before the swap: `_mask` blanks the quote DELIMITERS as well as the
|
||||
content, where this scanner kept the quotes. That is irrelevant here — both blank the contents,
|
||||
and every token this linter hunts (`func_<ADDR>`, `D_<ADDR>`, and the bare 2nd argument of
|
||||
`INCLUDE_ASM("...", func_X)`) lives OUTSIDE the quotes either way. Gated on the linter's own
|
||||
output being byte-identical across the change, not on the masks being byte-identical."""
|
||||
return cdecl._mask(src)
|
||||
|
||||
|
||||
def main():
|
||||
|
||||
+17
-2
@@ -12,6 +12,8 @@ Usage:
|
||||
tools/progress.py --binary <alias> # report a non-default binary (default: main = the EXE)
|
||||
"""
|
||||
import os, re, sys, hashlib, pathlib
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
import cdecl # §134/R33: the ONE comment/string masking oracle
|
||||
|
||||
ROOT = pathlib.Path(__file__).resolve().parent.parent
|
||||
|
||||
@@ -397,8 +399,21 @@ def find_s(name):
|
||||
INSTR = re.compile(r'^\s*/\*\s*[0-9A-Fa-f]+\s+[0-9A-Fa-f]+\s+[0-9A-Fa-f]+\s*\*/\s+[a-z]')
|
||||
|
||||
def strip_comments(s):
|
||||
s = re.sub(r'/\*.*?\*/', '', s, flags=re.S)
|
||||
return re.sub(r'//[^\n]*', '', s)
|
||||
"""Blank comments AND string/char-literal contents, via the ONE masking oracle (`cdecl._mask`).
|
||||
|
||||
§134 / R33 (P30 S39). This was a private two-line regex that removed `/*…*/` and `//…` and was
|
||||
NOT string-aware — so a brace or a semicolon inside a C string literal (`printf("}")`, a path
|
||||
like `"a;b"`) was seen as real syntax by the callers below, every one of which is a LINE-SHAPE
|
||||
decision on this function's output: the `{`-vs-`;` scan that separates a definition from a
|
||||
declaration, the `count('{') - count('}')` body-depth walk, and the empty-vs-real body test.
|
||||
A miscount there mis-buckets a function in the fn-count metric.
|
||||
|
||||
That is the §134 class, which had by this point appeared in SIX tools; the fix is routing every
|
||||
line-shape decision through `cdecl._mask` rather than writing a seventh regex. `_mask` is
|
||||
length-preserving (it blanks rather than deletes), which is strictly better here: the callers
|
||||
compare offsets (`br < sm`) and count characters, and blanking keeps those offsets valid
|
||||
against the ORIGINAL line while removing the false tokens."""
|
||||
return cdecl._mask(s)
|
||||
|
||||
def is_data_blob(name):
|
||||
"""A .s with a code label (glabel/jlabel) is a function; data-only (dlabel, no code) is a blob."""
|
||||
|
||||
Reference in New Issue
Block a user