mirror/BFM-decomp
1
0
Fork 0
mirror of https://github.com/Druthulu/BFM-decomp synced 2026-10-01 23:52:03 -04:00
Code Issues Packages Projects Releases Wiki Activity

docs(phase-29): T66 — item 4: the distinct-code anomaly modelled and closed (it was never a bug)

Browse Source 
Seven sweeps moved distinct-code by +125/+125/+129 and +0 four times; I had logged it four times as
"unexplained, still not guessed at". Modelled in one pass:

    delta_distinct = (distinct h_exact classes in the family) - (classes already matched)

weighted_metrics counts distinct h_exact classes with >=1 matched instance. EXACT on all 7, no
residual: func_80135260 131-6=125; func_80133AB0 131-6=125; func_80156044 130-1=129; the four +0
families have EXACTLY 1 class across all 138 overlays (every member byte-identical), already matched
via the exemplar.

IT IS A REAL SIGNAL, NOT NOISE. A byte-IDENTICAL family is ONE piece of distinct code — the
exemplar's crack already reconstructed it, so the other 137 banks pay fleet/instr in full (each
binary now builds from source instead of pasted asm) but add NO new reverse-engineering. A
byte-VARIANT family is ~130 genuinely different functions and pays both. The two headline metrics
rank the same work differently, and both are now predictable BEFORE spending a sweep.

THE REMAINING FRONTIER, PRICED BOTH WAYS (49 eligible non-jr families):
  byte-identical  13 families   80,085 ins       0 distinct
  byte-variant    36 families  114,331 ins   2,962 distinct
  total           49          194,416 ins (~1.48 pp instr)

MY OWN BUG, CAUGHT BY VERIFYING (R14): my first ranking reported ALL 49 families as byte-identical /
0 distinct yield. Defect in my probe — I wrote int(x,16) on the member address in one comprehension
and forgot it in the next, so every sig lookup missed and every family collapsed to one class. Caught
only by spot-checking two entries against a direct count (func_80143D28 is 130 classes, not 1). Had I
reported it, the conclusion "the entire remaining harvest is worthless for distinct-code" would have
been exactly backwards for 36 of 49 families.

cookbook §111, with §106 applied: the ranking is two lines over the sigs, so it is derivable on
demand and deliberately NOT committed as a table that rots.

No src/ or config/ change: no bank, no metric move.
This commit is contained in:
Drew T
2026-07-28 22:41:46 -06:00
parent 350cc1c34c
commit a6f6ccf545
2 changed files with 95 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/matching-cookbook.md
+49
View File
@@ -7668,3 +7668,52 @@ all five here extracted byte-identically before and after.
> nothing, because its output looks plausible and the failure surfaces N members later wearing the
> compiler's clothes. Assert the shape you require; and when the assertion is about "is this a
> definition", remember C lets a line be two declarations and a definition at once.
---
## §111 — The distinct-code metric is not noisy: a family pays it only if its members are byte-VARIANT (Phase 29 T66)
Seven family sweeps moved `distinct-code` by +125, +125, +129, and **+0 four times**. It read as a
metric bug for four tasks. It is not — it is the metric reporting something the instruction-weighted
number cannot.
`weighted_metrics` counts `dedup_fns = len(matched_cls)`, where a class is **one distinct `h_exact`**
and is marked matched if **ANY** binary has it matched. So:
> **Δdistinct = (distinct `h_exact` classes in the family) − (classes already matched)**
Exact on all seven, with no residual:
| family | classes | already matched | predicted | observed |
|---|---|---|---|---|
| `func_80135260` | 131 | 6 | +125 | **+125** |
| `func_80133AB0` | 131 | 6 | +125 | **+125** |
| `func_80156044` | 130 | 1 | +129 | **+129** |
| four others | **1** | 1 | +0 | **+0** |
**The meaning.** A family whose 138 members are byte-IDENTICAL is ONE piece of distinct code. The
exemplar's crack already reconstructed it; the other 137 banks are real (each binary now builds that
function from source instead of pasted asm, so `fleet`/instr-weighted pays in full) but they add
**no new reverse-engineering**. A byte-VARIANT family is ~130 genuinely different functions and pays
both.
**So the two headline metrics rank the same work differently**, and you can predict both before
spending a sweep — count the family's distinct `h_exact` across overlays:
```python
cls = collections.Counter(sig[ov][addr] for ov in sigs if addr in sig[ov])
instr_yield = live_members * nins
distinct_yield = len(cls) - len(classes_already_matched)
```
Measured over the 49 currently-eligible non-jr families: **194,416 instructions (~1.48 pp)** total,
of which **13 families / 80,085 ins are byte-identical and pay ZERO distinct-code**, and 36 families
/ 114,331 ins pay **2,962** distinct classes. Pick by which number you are trying to move.
> **The law:** before calling a metric noisy, model it. Two behaviours with no identified variable is
> not noise — it is a variable you have not found. This one took a `Counter` and ten minutes, after
> four tasks of writing "still unexplained, still not guessed at" in the log. Logging the anomaly
> honestly was right; leaving it unmodelled that long was not.
*(And the derived-not-persisted rule from §106 applies to the table above: it is two lines of code
over the sigs, so regenerate it — do not commit a ranking that rots the moment a family banks.)*
phase-ends/CURRENT_PHASE.md
+46
View File
@@ -8337,3 +8337,49 @@ the answer every time).
- **`.run/autopsy/residuals.jsonl` is dated Jul 21** — `route_for` protects the ROUTE from staleness
(T54) but the `klass` measurements are old; a re-collect is owed before trusting the grinder.
**DO NOT close P29 on ROI** — +0.8pp instr today and item 5 is an un-mined systematic lever.
## ✅ T66 — item 4: the distinct-code anomaly is **modelled and closed** (it was never a bug)
Seven sweeps had moved `distinct-code` by +125/+125/+129 and **+0 four times**, and I had logged it
four times as "unexplained, still not guessed at". Modelled in one pass:
> **Δdistinct = (distinct `h_exact` classes in the family) − (classes already matched)**
`weighted_metrics` counts distinct `h_exact` classes with ≥1 matched instance. **Exact on all 7, no
residual:** `func_80135260` 131−6=125 ✓ · `func_80133AB0` 131−6=125 ✓ · `func_80156044` 130−1=129 ✓ ·
the four `+0` families have **exactly 1 class** across all 138 overlays (every member byte-identical),
already matched via the exemplar ✓.
**It is a real signal, not noise.** A byte-IDENTICAL family is ONE piece of distinct code — the
exemplar's crack already reconstructed it, so the other 137 banks pay `fleet`/instr in full (each
binary now builds from source) but add **no new RE**. A byte-VARIANT family is ~130 genuinely
different functions and pays both. **The two headline metrics therefore rank the same work
differently**, and both are now predictable *before* spending a sweep.
### THE REMAINING FRONTIER, PRICED BOTH WAYS (49 eligible non-jr families)
| | families | instructions | distinct classes |
|---|---|---|---|
| byte-**identical** | 13 | 80,085 | **0** |
| byte-**variant** | 36 | 114,331 | **2,962** |
| **total** | **49** | **194,416** (~**1.48 pp** instr) | **2,962** |
### ⚠️ MY OWN BUG, CAUGHT BY VERIFYING (R14)
My first ranking table reported **all 49 families as byte-identical / 0 distinct yield**. That was a
defect in my probe: I wrote `int(x,16)` on the member address in one comprehension and **forgot it in
the next**, so every sig lookup missed and every family collapsed to one class. I only caught it by
spot-checking two entries against a direct count — `func_80143D28` is **130** classes, not 1. Had I
reported it, the conclusion "the entire remaining harvest is worthless for distinct-code" would have
been exactly backwards for 36 of 49 families.
Recorded as cookbook **§111**, with the §106 rule applied: the ranking is two lines over the sigs, so
it is derivable on demand and deliberately NOT committed as a table that rots.
## ▶ NEXT (ranked, all measured — unchanged except the new pricing)
1. **`0x80143d28`** (136 members · 10,880 ins · **128 distinct**) — also the last un-diagnosed of the
original five (`ApplyMatrixSV` conflict). Highest combined value on the board.
2. **`0x801457a4`** (137 · 10,823 ins · **129 distinct**) — re-measure with `match_one` before routing
as codegen (T62: a "clean DIFF" can be tool-manufactured).
3. **Sweep the 36 byte-VARIANT families** (114,331 ins · 2,962 distinct) ahead of the 13
byte-identical ones (80,085 ins · 0 distinct) — same tooling, strictly better on the honest metric.
4. **Audit `engine_core.h` for decls contradicting byte truth** — three found this session, each
unblocking 137 members. Still the likeliest systematic lever.