mirror of
https://github.com/Druthulu/BFM-decomp
synced 2026-09-26 13:33:34 -04:00
phase-37: CHECKPOINT for a fresh session — T0–T2 ☑; the record banked: decision log P37 S106 (struct spelling moves bytes; the /s per-access dial), accelerators P37 S106 (→ DK-65 corrected in the kit), cookbook §458 (the func_801814AC byte-read), the handoff and wiki 'byte-neutral' claims date-stamped; the 🛑 block rewritten with T3's full design brief, the counters, the probe's findings and every command | 🛑 T3 NEXT (the tools, Max design)
This commit is contained in:
@@ -65,3 +65,4 @@ rule R103 DK-69 a failure-cause extractor is negative-controlled against the com
|
||||
rule R104 DK-9 build the disagreeing oracle before the batch runs and measure the disagreement first — DK-9 (the differential-oracle harness) with DK-61's known-true case
|
||||
rule R105 DK-70 a registry lists a member only after the gate spoke for it; a listed member whose site does not share is a named defect — DK-70 (the health suite asserts the work was done)
|
||||
rule R106 DK-71 a policy taken on a remembered precedent is a belief — read the target project's tree — DK-71 (what earns belief)
|
||||
accelerator P37 S106 DK-65 a stress-test before the fleet run and an audit's rejections read before believed: the struct-spelling byte finding corrects DK-65 (per-access spelling under the gate); the probe-first order and the known-true discipline are DK-61; DK-65
|
||||
|
||||
|
Can't render this file because it contains an unexpected character in line 52 and column 78.
|
@@ -2,7 +2,7 @@
|
||||
|
||||
> **Generated by `tools/cookbook_index.py` — do not hand-edit** (R33). Regenerate after adding a cookbook section.
|
||||
>
|
||||
> `docs/matching-cookbook.md` is ~716 KB / 1175 sections. Grepping it blind is how three P30 wave-1 agents each "discovered" an idiom that was already written down. **Start here, then read the section.** A section appears under every symptom it addresses.
|
||||
> `docs/matching-cookbook.md` is ~716 KB / 1176 sections. Grepping it blind is how three P30 wave-1 agents each "discovered" an idiom that was already written down. **Start here, then read the section.** A section appears under every symptom it addresses.
|
||||
|
||||
**How to use:** name what you SEE in the diff (a stolen delay slot, an extra `la`, a swapped register pair, a `conflicting types` error), find that symptom below, read those sections first. If nothing fits, THEN grind — and add a section when you win.
|
||||
|
||||
@@ -455,7 +455,7 @@
|
||||
- **§472** — 🔴 §148-A's HOIST THRESHOLD IS 29, NOT 58, WHEN THE LOOP CONTAINS A CALL <sub>L35499</sub>
|
||||
- **§501** — ★★★ — A LEVER THAT MEASURES WORSE MAY BE A CASCADE: READ THE `.loop` DUMP FOR THE DESIRABILITY FLIP BEFORE DISCARDING IT (P32 T4b, `main:func_800391D4`, a pinned wall banked by a Fable agent) <sub>L37157</sub>
|
||||
|
||||
### structs, block moves & memcpy (92)
|
||||
### structs, block moves & memcpy (93)
|
||||
|
||||
- **§3-T2** — Source statement order drives instruction scheduling <sub>L78</sub>
|
||||
- **§5** — Known hard-residual classes (instruction-identical, one byte-exact blocker) <sub>L199</sub>
|
||||
@@ -549,6 +549,7 @@
|
||||
- **§468** — THE `%lo`-FOLD EXTENDS TO STORES ONLY VIA `extern Struct SYM[]`, AND MASKING HID THE OPERAND ORDER <sub>L35390</sub>
|
||||
- **§469** — THE `MEM_IN_STRUCT_P` ALIAS UNLOCK (and §463's spill law, independently confirmed) <sub>L35424</sub>
|
||||
- **§497** — ★ — A BODILESS `typedef struct Tag Alias;` DEFINES THE ALIAS, NOT THE TAG: THE CARRIER'S FALSE "CONFLICTING BODIES" REFUSAL (P32 T1b; ov_SC02_017 `func_80186C64` isolation) <sub>L36705</sub>
|
||||
- **§458** — Struct spelling is a per-access dial: the T2 probe's byte evidence (Phase 37 S106) <sub>L37886</sub>
|
||||
|
||||
### types, signedness & load/store width (98)
|
||||
|
||||
@@ -1320,7 +1321,7 @@
|
||||
- **§453** — ★★★ — ONE SOURCE PER UNIQUE FUNCTION: THE INCLUDE-AT-SITE SHARE, ITS GATE, AND THE FOUR DEFECTS THAT SHAPED IT (P35 S94–S96) <sub>L37561</sub>
|
||||
- **§455** — Rung G, the guided search: score the object, classify the residual, compose the moves (Phase 36 S101) <sub>L37692</sub>
|
||||
|
||||
### process, measurement & doctrine (143)
|
||||
### process, measurement & doctrine (144)
|
||||
|
||||
- **§8e** — The jtbl ALIGNMENT LAW + the pad-spec filter — multi-table .rodata spans (Phase 29, byte-proven; `.run/probe_jtbl/verdict.md`) <sub>L549</sub>
|
||||
- **§3-The** — mechanism: game-code dedup is SOURCE-LEVEL, not an object swap (R-D1, the key lesson) <sub>L945</sub>
|
||||
@@ -1465,6 +1466,7 @@
|
||||
- **§479** — ★★★ — WHERE THE PERMUTER ACTUALLY PAYS: A MEASURED YIELD CURVE (P31 S77, 8 candidates) <sub>L35817</sub>
|
||||
- **§501** — ★★★ — A LEVER THAT MEASURES WORSE MAY BE A CASCADE: READ THE `.loop` DUMP FOR THE DESIRABILITY FLIP BEFORE DISCARDING IT (P32 T4b, `main:func_800391D4`, a pinned wall banked by a Fable agent) <sub>L37157</sub>
|
||||
- **§454a** — the recipe rung's real shape: replication, not discovery (S99, measured) <sub>L37677</sub>
|
||||
- **§458** — Struct spelling is a per-access dial: the T2 probe's byte evidence (Phase 37 S106) <sub>L37886</sub>
|
||||
|
||||
### (unbucketed — title matched no symptom vocabulary) (338)
|
||||
|
||||
@@ -2985,6 +2987,7 @@
|
||||
- **§455** — Rung G, the guided search: score the object, classify the residual, compose the moves (Phase 36 S101) <sub>L37692</sub>
|
||||
- **§456** — The lever-removal move catalog: what the S104 agents found under ~205 pins, barriers and launders (Phase 36 T7) <sub>L37765</sub>
|
||||
- **§457** — The lever-removal move catalog, S105: eight TU batches, 42 of 42 at 0 (Phase 36 T7) <sub>L37814</sub>
|
||||
- **§458** — Struct spelling is a per-access dial: the T2 probe's byte evidence (Phase 37 S106) <sub>L37886</sub>
|
||||
|
||||
|
||||
---
|
||||
@@ -4172,3 +4175,4 @@ Notes routinely quote that as a section id. This table resolves it. Grep bait: `
|
||||
| L37692 | §455 | Rung G, the guided search: score the object, classify the residual, compose the moves (Pha |
|
||||
| L37765 | §456 | The lever-removal move catalog: what the S104 agents found under ~205 pins, barriers and l |
|
||||
| L37814 | §457 | The lever-removal move catalog, S105: eight TU batches, 42 of 42 at 0 (Phase 36 T7) |
|
||||
| L37886 | §458 | Struct spelling is a per-access dial: the T2 probe's byte evidence (Phase 37 S106) |
|
||||
|
||||
@@ -37882,3 +37882,22 @@ closed; the ledger (`.run/P36/delever/ledger.jsonl`) is the record of which rung
|
||||
**The do-while tell.** Both marked do-while closes this session were reference-weight levers with the same arithmetic: the
|
||||
loser needs EXACTLY +1 flow-time ref (`flow.c:2067`) on a statement that does not mention the winner; a real callee argument
|
||||
does it when one exists — never a fake one.
|
||||
|
||||
## §458 — Struct spelling is a per-access dial: the T2 probe's byte evidence (Phase 37 S106)
|
||||
|
||||
**The fact.** In gcc 2.7.2 `*(T *)(p + k)` is a MEM without `MEM_IN_STRUCT_P`; `((S *)p)->f`, `p->f`, `q[k]` (typed `q`) and `*(Blk *)p`
|
||||
carry it (`expr.c:4568-4577`, `:4888`); `sched.c:837-865` lets a `/s` varying-address non-QImode access pass a non-`/s` FIXED-address one,
|
||||
and cse's kill table differs by `/s` (`docs/gcc-2.7.2-map/cse_expr.md` §4). So converting a cast to a member is a byte edit.
|
||||
**The measurement (`tools/restruct.py --probe`, 165 bodies, seed 37).** Rung 1 — every raw cast on ONE base of a body → `((P37S *)base)->unkK`
|
||||
with the type built from the struct map — is IDENTICAL on 126/139 judged bodies (716 sites moved); the 13 DIFFERS are COUNT/ORDER
|
||||
residuals, never a register change. Rung S2 — leave one site as a cast, else revert cumulatively in source order and minimise — closes
|
||||
13/13 with 27 casts kept.
|
||||
**The byte-read exemplar: `func_801814AC` (ov_SC05_010, `src/ov_SC05_010/ov_SC05_010_jr_80180F84.c:3030`).** Six `D_801C7E30`-based sites
|
||||
(`*(u16 *)(D_801C7E30 + 0x34) = 0; … + 2) = 9; *(u8 *)(… + 0xC2) = 0xE; var = D_801C7E30; …`). Every site a member EXCEPT the `+0x34` store →
|
||||
IDENTICAL. That store a member → the next statement's reload of the pointer global (`lui/lw D_801C7E30`, fixed, non-`/s`) hoists above the
|
||||
store (`/s`, varying): `nop; sh zero,52(v0); lui …` becomes `lui …; sh zero,52(v0)`. The pointer global itself read as a struct member
|
||||
(`((struct { S *p; } *)&D_801C7E30)->p` — both sides `/s`, they conflict, the order is kept) fixes that store — and moves a `li v0,10`
|
||||
belonging to the `param_1 + 0x1C` store elsewhere. The original therefore set `/s` per access; there is no uniform spelling of this body.
|
||||
**The rule.** Convert uniformly first (rung 1), then find the minimal kept-cast set (S2), then hand the kept casts to the readings (which
|
||||
side of each pair the original spelled scalar — a walked pointer, a reinterpretation, a cached global). A kept cast is not a fake: it is the
|
||||
original's spelling until a reading proves otherwise, counted and ledgered with its pass (SCHED-ALIAS here). Related: §30, §351, §379, §469.
|
||||
|
||||
@@ -800,8 +800,15 @@ data symbols; 3,558 of 4,287 C files carve splits; no formatter configuration; a
|
||||
```
|
||||
provenance: BFM gen3-standards (2026-09-07) + gen3-handoff §2.2; the sotn-decomp style guide read as data
|
||||
|
||||
### DK-65 — Types are a banking lever and a width lever, not a byte lever
|
||||
- **Kernel:** a type name never moves a byte — the compiler does not care what a field is called — and the source
|
||||
### DK-65 — Types are a banking lever and a width lever — and, in gcc 2.7.2, a struct MEMBER is a scheduling lever too
|
||||
- **Correction (source project, Phase 37 T2, 2026-09-12):** the kernel below was written from Phase 17's measurement, which tested a
|
||||
decompiler's drafting with a struct in context — not the spelling of a matched body. On the bytes, `*(T *)(p + k)` and `p->f` are NOT
|
||||
the same code in gcc 2.7.2: a member/array access carries `MEM_IN_STRUCT_P` (`expr.c:4568-4577`, `:4888`), a cast on a sum does not, and
|
||||
the scheduler's alias escape (`sched.c:837-865`) and cse's kill table read that flag. A struct-spelling probe on 165 matched bodies was
|
||||
byte-neutral on 90.6 % and moved instructions on the rest; a per-site minimal kept-cast set closed every one. So: a canonical type
|
||||
layer still pays as a banking and width lever from the first bank — and it must be introduced under the byte gate, per access, never by
|
||||
a blanket rewrite that assumes neutrality. The name still moves nothing; the SPELLING can.
|
||||
- **Kernel (as first written):** a type name never moves a byte — the compiler does not care what a field is called — and the source
|
||||
project measured exactly that early, correctly, and then drew the wrong second conclusion: that types could wait until
|
||||
after 100%. Read the other way, its record says types are the largest *banking* lever it underweighted. Banking, not
|
||||
cracking, was the bottleneck, and a large share of gate failures were declaration conflicts — a signed halfword against
|
||||
|
||||
@@ -962,3 +962,19 @@ regen pass reports its refusal REASONS as a histogram (`&p`, `no base`, `re-seat
|
||||
defect, not a property of the population (R32's "assert your coverage" for generators); (3) when an agent closes with a move
|
||||
a generator claims to make, run the generator on the agent's START text before the bank — the cheapest instrument check in
|
||||
the loop, and this session it paid three times.
|
||||
|
||||
## P37 S106 (2026-09-12) — the Plan agent's stress-test before the fleet run, and an audit's rejections that were the probe's
|
||||
|
||||
Four premises of the approved plan were corrected by a Plan agent reading the compiler source and the P36 packs BEFORE `type_census.py`
|
||||
ran on the fleet (the relocation-spelling oracle blind spot, function identity by body hash, the sixth site form `((T *)p)->f`, duplicates
|
||||
by layout + evidence) — each would otherwise have surfaced as a wrong number in a committed census. The struct map's clustering then went
|
||||
through four rounds against the classic over-merge (one cluster of 297,668 sites): typed-use parameters only, a width-conflict veto,
|
||||
single-source locals, positive shared evidence (≥1 common offset, the callee's map ≥2 offsets) — each rule measured by the biggest cluster's
|
||||
size. The layout audit against cc1 started at 4,498 rejections of 5,295 and ended at 0 of 5,283: five were the probe's (the environment
|
||||
lacked the canonical header; typedefs out of order; block-scope typedefs hoisted; a shared scratch name raced across workers; dead
|
||||
`#ifndef BFM_ENGINE_TYPES_H` blocks), two were the engine's (nested-paren attributes, per-field `aligned(N)`).
|
||||
**What would have been sooner:** (1) a struct-spelling probe on 150 bodies (6 min) as the FIRST thing in a types phase — it priced the
|
||||
campaign and rewrote the rewrite table five times; (2) rung S2 (leave-one-out, then cumulative revert) as the second rung of any
|
||||
per-site spelling campaign — 13/13 DIFFERS closed at ≤2n+1 compiles, no agent; (3) read every rejection message of a new audit before
|
||||
believing any (R40 for audits): 4,498 → 9 → 0 with two real findings along the way; (4) unique scratch names per probe (R111) — the
|
||||
cross-address copies produce IDENTICAL probe bodies, and a hash-keyed scratch file raced.
|
||||
|
||||
@@ -3813,3 +3813,42 @@ packs (21 classes: comment-boundary strips, macro-carried launders, one-off inst
|
||||
pack-builder gap an agent works around from `body_tree.c` — fix the six comment-boundary strips in the tool, draw the rest
|
||||
with the note. (4) The three parked signature changes of this session join the structs phase's list with ready patches;
|
||||
none of them needed a new mechanism to read.
|
||||
|
||||
## P37 S106 (2026-09-11/12) — the structs phase opens: struct spelling moves bytes, and the plan is built on that
|
||||
|
||||
**Context/belief.** The Gen3 charter and `docs/gen3-handoff.md` §4 carried Phase 17's verdict forward — "types are a comprehension lever,
|
||||
not a byte lever: struct-ification is byte-neutral by construction as long as the layout and the access widths are right" — and
|
||||
`include/common.h:28-30` says `M2C_FIELD(p,t,o)` has "identical codegen to `p->field`". Phase 36 had found, one agent at a time, that the
|
||||
last third of its lever residue was signatures, structs, carves and one GTE spelling; its evidence (d10's register union, a18's
|
||||
`extern u16 X[]`, f10's body-local structs, f3's `s16[3]` arrays) said the struct spelling was the MISSING piece.
|
||||
**What the gate-1 reading found.** The claim is false in gcc 2.7.2 and the tree already knew it (cookbook §30/§351/§379/§469,
|
||||
`docs/gcc-2.7.2-map/cse_expr.md` §4): `*(T*)(p+k)` gets no `MEM_IN_STRUCT_P`, `p->f`/`q[k]`/`*(Blk*)p` do, and `sched.c:830-865`'s
|
||||
alias escape and cse's kill table read that flag. Phase 17 had measured m2c DRAFTING with a struct in context (0/10/2 on a
|
||||
structural-miss sample), not member-vs-cast spelling in a matched body — and the project generalised it into "types can wait". A Plan
|
||||
agent's stress-test then corrected four more premises before any tool ran: caller bytes DO depend on a narrow prototype's widths in both
|
||||
directions (`c-typeck.c:1740` converts to the parameter type, then re-promotes); a whole-object oracle says DIFFERS on a correct
|
||||
global-block edit because the relocation SPELLING changes while the linked bytes are identical (proven: f3's `D_801F8870[3]`, object
|
||||
DIFFERS, `make build BINARY=md_SC07_004` byte-identical); a function's identity is never its bare name (6,415 names carry more than one
|
||||
body text); and "one definition per layout" is too strong (`{s16 ×4}` carries 177 names, some genuinely different types) — a duplicate is
|
||||
same layout AND identical meaningful member names, an opaque copy block, or flow evidence.
|
||||
**The decisions (Drew, gate 1).** Grind to zero on raw casts, lying declarations and levers, with each "zero" defined so it is honest and
|
||||
reachable (reinterpret macros/unions for the genuine reinterpretations; byte-proven K&R sites counted apart; legitimate GTE sequences as
|
||||
project-local macros in the one header); placeholder names + cited evidence only; the P36 agent lane at his cap, started on his word; the
|
||||
full canonical declaration layer. After T0 he asked whether the phase should first analyse every function and build a map of every needed
|
||||
struct — my answer: yes, as T1's deliverable (the struct map), because it makes "zero casts" a checkable statement and lets T4's prototypes
|
||||
carry the parameter types in one pass; he approved. Also: no tags or releases at phase closes (the `v2.2.0` tag deleted; the P36 close
|
||||
pushed as a normal commit on his word).
|
||||
**What the census and the probe then measured (T1/T2).** 503,016 raw dereferences in four forms (the readability series' regex saw one,
|
||||
411,850 over raw text) + 18,912 address-of + 36,681 typed cast-member accesses that are already COMPONENT_REFs; 7,255 struct definitions
|
||||
(5,995 inside `.c` files) over 527 layouts; a `--strict` lever finish line of 4,010 + 6,717 direct GTE statements + 314 per-TU macros.
|
||||
Rung 1 (body-local `((T *)base)->unkK`) is byte-neutral on 126/139 judged bodies; the 13 DIFFERS are the `/s` asymmetry between a
|
||||
converted pointer base and the body's bare fixed globals, and a per-site greedy (rung S2) closes all 13 with 27 of 716 sites kept as
|
||||
casts — **the `/s` flag is a dial the original set per ACCESS, so a uniform conversion is the first rung and a per-site minimal kept-cast
|
||||
set is the second, before any agent.** The definition's ANSI signature is a free declaration for 92.6 % of (TU, callee) pairs with zero
|
||||
DIFFERS; the refusals are arity contradictions (K&R sites) and multi-spelled callees. The layout engine agrees with cc1 on 5,283
|
||||
definitions after the audit found two real gaps in it.
|
||||
**Hindsight (the better path).** (1) Re-read the compiler map before restating a Phase-17 verdict in a charter: the `/s` model was in
|
||||
the tree's own documents for two months while the handoff said "byte-neutral". (2) Price a struct campaign by a byte probe on the tree
|
||||
(one afternoon) before designing the engine — the probe fixed the rewrite table five times. (3) Build the disagreeing oracle for the new
|
||||
question BEFORE the campaign (R104): the relocation-spelling control took one binary build to settle and would otherwise have read every
|
||||
global-block conversion as a failure.
|
||||
|
||||
@@ -962,3 +962,19 @@ regen pass reports its refusal REASONS as a histogram (`&p`, `no base`, `re-seat
|
||||
defect, not a property of the population (R32's "assert your coverage" for generators); (3) when an agent closes with a move
|
||||
a generator claims to make, run the generator on the agent's START text before the bank — the cheapest instrument check in
|
||||
the loop, and this session it paid three times.
|
||||
|
||||
## P37 S106 (2026-09-12) — the Plan agent's stress-test before the fleet run, and an audit's rejections that were the probe's
|
||||
|
||||
Four premises of the approved plan were corrected by a Plan agent reading the compiler source and the P36 packs BEFORE `type_census.py`
|
||||
ran on the fleet (the relocation-spelling oracle blind spot, function identity by body hash, the sixth site form `((T *)p)->f`, duplicates
|
||||
by layout + evidence) — each would otherwise have surfaced as a wrong number in a committed census. The struct map's clustering then went
|
||||
through four rounds against the classic over-merge (one cluster of 297,668 sites): typed-use parameters only, a width-conflict veto,
|
||||
single-source locals, positive shared evidence (≥1 common offset, the callee's map ≥2 offsets) — each rule measured by the biggest cluster's
|
||||
size. The layout audit against cc1 started at 4,498 rejections of 5,295 and ended at 0 of 5,283: five were the probe's (the environment
|
||||
lacked the canonical header; typedefs out of order; block-scope typedefs hoisted; a shared scratch name raced across workers; dead
|
||||
`#ifndef BFM_ENGINE_TYPES_H` blocks), two were the engine's (nested-paren attributes, per-field `aligned(N)`).
|
||||
**What would have been sooner:** (1) a struct-spelling probe on 150 bodies (6 min) as the FIRST thing in a types phase — it priced the
|
||||
campaign and rewrote the rewrite table five times; (2) rung S2 (leave-one-out, then cumulative revert) as the second rung of any
|
||||
per-site spelling campaign — 13/13 DIFFERS closed at ≤2n+1 compiles, no agent; (3) read every rejection message of a new audit before
|
||||
believing any (R40 for audits): 4,498 → 9 → 0 with two real findings along the way; (4) unique scratch names per probe (R111) — the
|
||||
cross-address copies produce IDENTICAL probe bodies, and a hash-keyed scratch file raced.
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
> **Generated by `tools/cookbook_index.py` — do not hand-edit** (R33). Regenerate after adding a cookbook section.
|
||||
>
|
||||
> `docs/matching-cookbook.md` is ~716 KB / 1175 sections. Grepping it blind is how three P30 wave-1 agents each "discovered" an idiom that was already written down. **Start here, then read the section.** A section appears under every symptom it addresses.
|
||||
> `docs/matching-cookbook.md` is ~716 KB / 1176 sections. Grepping it blind is how three P30 wave-1 agents each "discovered" an idiom that was already written down. **Start here, then read the section.** A section appears under every symptom it addresses.
|
||||
|
||||
**How to use:** name what you SEE in the diff (a stolen delay slot, an extra `la`, a swapped register pair, a `conflicting types` error), find that symptom below, read those sections first. If nothing fits, THEN grind — and add a section when you win.
|
||||
|
||||
@@ -455,7 +455,7 @@
|
||||
- **§472** — 🔴 §148-A's HOIST THRESHOLD IS 29, NOT 58, WHEN THE LOOP CONTAINS A CALL <sub>L35499</sub>
|
||||
- **§501** — ★★★ — A LEVER THAT MEASURES WORSE MAY BE A CASCADE: READ THE `.loop` DUMP FOR THE DESIRABILITY FLIP BEFORE DISCARDING IT (P32 T4b, `main:func_800391D4`, a pinned wall banked by a Fable agent) <sub>L37157</sub>
|
||||
|
||||
### structs, block moves & memcpy (92)
|
||||
### structs, block moves & memcpy (93)
|
||||
|
||||
- **§3-T2** — Source statement order drives instruction scheduling <sub>L78</sub>
|
||||
- **§5** — Known hard-residual classes (instruction-identical, one byte-exact blocker) <sub>L199</sub>
|
||||
@@ -549,6 +549,7 @@
|
||||
- **§468** — THE `%lo`-FOLD EXTENDS TO STORES ONLY VIA `extern Struct SYM[]`, AND MASKING HID THE OPERAND ORDER <sub>L35390</sub>
|
||||
- **§469** — THE `MEM_IN_STRUCT_P` ALIAS UNLOCK (and §463's spill law, independently confirmed) <sub>L35424</sub>
|
||||
- **§497** — ★ — A BODILESS `typedef struct Tag Alias;` DEFINES THE ALIAS, NOT THE TAG: THE CARRIER'S FALSE "CONFLICTING BODIES" REFUSAL (P32 T1b; ov_SC02_017 `func_80186C64` isolation) <sub>L36705</sub>
|
||||
- **§458** — Struct spelling is a per-access dial: the T2 probe's byte evidence (Phase 37 S106) <sub>L37886</sub>
|
||||
|
||||
### types, signedness & load/store width (98)
|
||||
|
||||
@@ -1320,7 +1321,7 @@
|
||||
- **§453** — ★★★ — ONE SOURCE PER UNIQUE FUNCTION: THE INCLUDE-AT-SITE SHARE, ITS GATE, AND THE FOUR DEFECTS THAT SHAPED IT (P35 S94–S96) <sub>L37561</sub>
|
||||
- **§455** — Rung G, the guided search: score the object, classify the residual, compose the moves (Phase 36 S101) <sub>L37692</sub>
|
||||
|
||||
### process, measurement & doctrine (143)
|
||||
### process, measurement & doctrine (144)
|
||||
|
||||
- **§8e** — The jtbl ALIGNMENT LAW + the pad-spec filter — multi-table .rodata spans (Phase 29, byte-proven; `.run/probe_jtbl/verdict.md`) <sub>L549</sub>
|
||||
- **§3-The** — mechanism: game-code dedup is SOURCE-LEVEL, not an object swap (R-D1, the key lesson) <sub>L945</sub>
|
||||
@@ -1465,6 +1466,7 @@
|
||||
- **§479** — ★★★ — WHERE THE PERMUTER ACTUALLY PAYS: A MEASURED YIELD CURVE (P31 S77, 8 candidates) <sub>L35817</sub>
|
||||
- **§501** — ★★★ — A LEVER THAT MEASURES WORSE MAY BE A CASCADE: READ THE `.loop` DUMP FOR THE DESIRABILITY FLIP BEFORE DISCARDING IT (P32 T4b, `main:func_800391D4`, a pinned wall banked by a Fable agent) <sub>L37157</sub>
|
||||
- **§454a** — the recipe rung's real shape: replication, not discovery (S99, measured) <sub>L37677</sub>
|
||||
- **§458** — Struct spelling is a per-access dial: the T2 probe's byte evidence (Phase 37 S106) <sub>L37886</sub>
|
||||
|
||||
### (unbucketed — title matched no symptom vocabulary) (338)
|
||||
|
||||
@@ -2985,6 +2987,7 @@
|
||||
- **§455** — Rung G, the guided search: score the object, classify the residual, compose the moves (Phase 36 S101) <sub>L37692</sub>
|
||||
- **§456** — The lever-removal move catalog: what the S104 agents found under ~205 pins, barriers and launders (Phase 36 T7) <sub>L37765</sub>
|
||||
- **§457** — The lever-removal move catalog, S105: eight TU batches, 42 of 42 at 0 (Phase 36 T7) <sub>L37814</sub>
|
||||
- **§458** — Struct spelling is a per-access dial: the T2 probe's byte evidence (Phase 37 S106) <sub>L37886</sub>
|
||||
|
||||
|
||||
---
|
||||
@@ -4172,3 +4175,4 @@ Notes routinely quote that as a section id. This table resolves it. Grep bait: `
|
||||
| L37692 | §455 | Rung G, the guided search: score the object, classify the residual, compose the moves (Pha |
|
||||
| L37765 | §456 | The lever-removal move catalog: what the S104 agents found under ~205 pins, barriers and l |
|
||||
| L37814 | §457 | The lever-removal move catalog, S105: eight TU batches, 42 of 42 at 0 (Phase 36 T7) |
|
||||
| L37886 | §458 | Struct spelling is a per-access dial: the T2 probe's byte evidence (Phase 37 S106) |
|
||||
|
||||
@@ -3813,3 +3813,42 @@ packs (21 classes: comment-boundary strips, macro-carried launders, one-off inst
|
||||
pack-builder gap an agent works around from `body_tree.c` — fix the six comment-boundary strips in the tool, draw the rest
|
||||
with the note. (4) The three parked signature changes of this session join the structs phase's list with ready patches;
|
||||
none of them needed a new mechanism to read.
|
||||
|
||||
## P37 S106 (2026-09-11/12) — the structs phase opens: struct spelling moves bytes, and the plan is built on that
|
||||
|
||||
**Context/belief.** The Gen3 charter and `docs/gen3-handoff.md` §4 carried Phase 17's verdict forward — "types are a comprehension lever,
|
||||
not a byte lever: struct-ification is byte-neutral by construction as long as the layout and the access widths are right" — and
|
||||
`include/common.h:28-30` says `M2C_FIELD(p,t,o)` has "identical codegen to `p->field`". Phase 36 had found, one agent at a time, that the
|
||||
last third of its lever residue was signatures, structs, carves and one GTE spelling; its evidence (d10's register union, a18's
|
||||
`extern u16 X[]`, f10's body-local structs, f3's `s16[3]` arrays) said the struct spelling was the MISSING piece.
|
||||
**What the gate-1 reading found.** The claim is false in gcc 2.7.2 and the tree already knew it (cookbook §30/§351/§379/§469,
|
||||
`docs/gcc-2.7.2-map/cse_expr.md` §4): `*(T*)(p+k)` gets no `MEM_IN_STRUCT_P`, `p->f`/`q[k]`/`*(Blk*)p` do, and `sched.c:830-865`'s
|
||||
alias escape and cse's kill table read that flag. Phase 17 had measured m2c DRAFTING with a struct in context (0/10/2 on a
|
||||
structural-miss sample), not member-vs-cast spelling in a matched body — and the project generalised it into "types can wait". A Plan
|
||||
agent's stress-test then corrected four more premises before any tool ran: caller bytes DO depend on a narrow prototype's widths in both
|
||||
directions (`c-typeck.c:1740` converts to the parameter type, then re-promotes); a whole-object oracle says DIFFERS on a correct
|
||||
global-block edit because the relocation SPELLING changes while the linked bytes are identical (proven: f3's `D_801F8870[3]`, object
|
||||
DIFFERS, `make build BINARY=md_SC07_004` byte-identical); a function's identity is never its bare name (6,415 names carry more than one
|
||||
body text); and "one definition per layout" is too strong (`{s16 ×4}` carries 177 names, some genuinely different types) — a duplicate is
|
||||
same layout AND identical meaningful member names, an opaque copy block, or flow evidence.
|
||||
**The decisions (Drew, gate 1).** Grind to zero on raw casts, lying declarations and levers, with each "zero" defined so it is honest and
|
||||
reachable (reinterpret macros/unions for the genuine reinterpretations; byte-proven K&R sites counted apart; legitimate GTE sequences as
|
||||
project-local macros in the one header); placeholder names + cited evidence only; the P36 agent lane at his cap, started on his word; the
|
||||
full canonical declaration layer. After T0 he asked whether the phase should first analyse every function and build a map of every needed
|
||||
struct — my answer: yes, as T1's deliverable (the struct map), because it makes "zero casts" a checkable statement and lets T4's prototypes
|
||||
carry the parameter types in one pass; he approved. Also: no tags or releases at phase closes (the `v2.2.0` tag deleted; the P36 close
|
||||
pushed as a normal commit on his word).
|
||||
**What the census and the probe then measured (T1/T2).** 503,016 raw dereferences in four forms (the readability series' regex saw one,
|
||||
411,850 over raw text) + 18,912 address-of + 36,681 typed cast-member accesses that are already COMPONENT_REFs; 7,255 struct definitions
|
||||
(5,995 inside `.c` files) over 527 layouts; a `--strict` lever finish line of 4,010 + 6,717 direct GTE statements + 314 per-TU macros.
|
||||
Rung 1 (body-local `((T *)base)->unkK`) is byte-neutral on 126/139 judged bodies; the 13 DIFFERS are the `/s` asymmetry between a
|
||||
converted pointer base and the body's bare fixed globals, and a per-site greedy (rung S2) closes all 13 with 27 of 716 sites kept as
|
||||
casts — **the `/s` flag is a dial the original set per ACCESS, so a uniform conversion is the first rung and a per-site minimal kept-cast
|
||||
set is the second, before any agent.** The definition's ANSI signature is a free declaration for 92.6 % of (TU, callee) pairs with zero
|
||||
DIFFERS; the refusals are arity contradictions (K&R sites) and multi-spelled callees. The layout engine agrees with cc1 on 5,283
|
||||
definitions after the audit found two real gaps in it.
|
||||
**Hindsight (the better path).** (1) Re-read the compiler map before restating a Phase-17 verdict in a charter: the `/s` model was in
|
||||
the tree's own documents for two months while the handoff said "byte-neutral". (2) Price a struct campaign by a byte probe on the tree
|
||||
(one afternoon) before designing the engine — the probe fixed the rewrite table five times. (3) Build the disagreeing oracle for the new
|
||||
question BEFORE the campaign (R104): the relocation-spelling control took one binary build to settle and would otherwise have read every
|
||||
global-block conversion as a failure.
|
||||
|
||||
@@ -162,6 +162,9 @@ Two subtleties the census hides:
|
||||
it to the decompiler produced identical bytes (`docs/struct-core-pivot.md`): the compiler does not care what you call
|
||||
a field. That is good news for Gen3 — struct-ification is byte-neutral by construction *as long as the layout and
|
||||
the access widths are right* — and it is why every such edit still goes through the gate.
|
||||
*Corrected 2026-09-12 (Phase 37 T2): in gcc 2.7.2 the SPELLING moves bytes — a member/array access carries `MEM_IN_STRUCT_P`, a cast on a
|
||||
sum does not, and the scheduler's alias escape and cse read that flag; a 165-body probe was byte-neutral on 90.6 % and a per-site
|
||||
minimal kept-cast set closed the rest (cookbook §458, `docs/decision-log.md` P37 S106). The gate is the arbiter, per access.*
|
||||
|
||||
## 5. Levers Gen3 inherits
|
||||
|
||||
|
||||
@@ -37882,3 +37882,22 @@ closed; the ledger (`.run/P36/delever/ledger.jsonl`) is the record of which rung
|
||||
**The do-while tell.** Both marked do-while closes this session were reference-weight levers with the same arithmetic: the
|
||||
loser needs EXACTLY +1 flow-time ref (`flow.c:2067`) on a statement that does not mention the winner; a real callee argument
|
||||
does it when one exists — never a fake one.
|
||||
|
||||
## §458 — Struct spelling is a per-access dial: the T2 probe's byte evidence (Phase 37 S106)
|
||||
|
||||
**The fact.** In gcc 2.7.2 `*(T *)(p + k)` is a MEM without `MEM_IN_STRUCT_P`; `((S *)p)->f`, `p->f`, `q[k]` (typed `q`) and `*(Blk *)p`
|
||||
carry it (`expr.c:4568-4577`, `:4888`); `sched.c:837-865` lets a `/s` varying-address non-QImode access pass a non-`/s` FIXED-address one,
|
||||
and cse's kill table differs by `/s` (`docs/gcc-2.7.2-map/cse_expr.md` §4). So converting a cast to a member is a byte edit.
|
||||
**The measurement (`tools/restruct.py --probe`, 165 bodies, seed 37).** Rung 1 — every raw cast on ONE base of a body → `((P37S *)base)->unkK`
|
||||
with the type built from the struct map — is IDENTICAL on 126/139 judged bodies (716 sites moved); the 13 DIFFERS are COUNT/ORDER
|
||||
residuals, never a register change. Rung S2 — leave one site as a cast, else revert cumulatively in source order and minimise — closes
|
||||
13/13 with 27 casts kept.
|
||||
**The byte-read exemplar: `func_801814AC` (ov_SC05_010, `src/ov_SC05_010/ov_SC05_010_jr_80180F84.c:3030`).** Six `D_801C7E30`-based sites
|
||||
(`*(u16 *)(D_801C7E30 + 0x34) = 0; … + 2) = 9; *(u8 *)(… + 0xC2) = 0xE; var = D_801C7E30; …`). Every site a member EXCEPT the `+0x34` store →
|
||||
IDENTICAL. That store a member → the next statement's reload of the pointer global (`lui/lw D_801C7E30`, fixed, non-`/s`) hoists above the
|
||||
store (`/s`, varying): `nop; sh zero,52(v0); lui …` becomes `lui …; sh zero,52(v0)`. The pointer global itself read as a struct member
|
||||
(`((struct { S *p; } *)&D_801C7E30)->p` — both sides `/s`, they conflict, the order is kept) fixes that store — and moves a `li v0,10`
|
||||
belonging to the `param_1 + 0x1C` store elsewhere. The original therefore set `/s` per access; there is no uniform spelling of this body.
|
||||
**The rule.** Convert uniformly first (rung 1), then find the minimal kept-cast set (S2), then hand the kept casts to the readings (which
|
||||
side of each pair the original spelled scalar — a walked pointer, a reinterpretation, a cached global). A kept cast is not a fake: it is the
|
||||
original's spelling until a reading proves otherwise, counted and ledgered with its pass (SCHED-ALIAS here). Related: §30, §351, §379, §469.
|
||||
|
||||
@@ -64,7 +64,7 @@ wrongly."* To that the project adds four rules of its own, because its code was
|
||||
**The one invariant.** Every Gen3 edit is gated exactly like a match was: the per-binary check for every binary a change
|
||||
touches, and the clean fleet run after anything that touches a shared body, a shared header or the executable. A rename
|
||||
is a symbol-file change mirrored into Ghidra by the headless script, never a hand edit of assembly. A shared body is one
|
||||
source and up to 138 binaries, so the gate runs on all of them. Types are a comprehension lever, not a byte lever — the
|
||||
source and up to 138 binaries, so the gate runs on all of them. Types are a comprehension lever, not a byte lever — the *(Corrected 2026-09-12, Phase 37 T2: the struct SPELLING does move bytes in gcc 2.7.2 through `MEM_IN_STRUCT_P` — byte-neutral on 90.6 % of a 165-body probe, the rest closed per site; the gate decides every access. Cookbook §458.)*
|
||||
compiler does not care what a field is called — which is exactly why struct work is safe and exactly why it still goes
|
||||
through the gate.
|
||||
|
||||
|
||||
+138
-58
@@ -271,66 +271,146 @@ bucketed by what each bucket needs before any plan is proposed. **Candidates for
|
||||
probe tables above · the census re-run `type_census: 7255 definitions (527 layouts, 206 duplicate classes, 40 variant camps) · 503016
|
||||
cast sites … coverage OK … controls 4/4`; `readability_progress --snapshot` (a T2 row) + `progress.py --check` fresh.
|
||||
|
||||
## 🛑 SESSION CHECKPOINT — S106 (2026-09-11/12, LIVE — refreshed at every task close): gate 1 APPROVED, **T0 ☑ T1 ☑ T2 ☑** (census, map, the probe — every number in the T2 log entry) — 🛑 **T3 NEXT (the tools: `restruct.py`'s full form, Max design)** | R22 `check-all: 218 passed, 0 failed of 218` at `db212f167` | the P36 close `79b2f6f15` is committed AND PUSHED (`origin/main`); no tag (Drew: no tags/releases at closes); T0 `0a55cb0fd` local
|
||||
## 🛑 SESSION CHECKPOINT — S106 (2026-09-11/12, FINAL — written for a FRESH session; the session's last commit follows this): gate 1 APPROVED, **T0 ☑ T1 ☑ T2 ☑** (baseline · the type census + the struct map · the probe) — 🛑 **T3 NEXT (the tools: `restruct.py`'s full form + the linked-relocation oracle + `struct_layout.py` + the canonical type writer; Max for the design, xHigh to finish)** | R22 `check-all: 218 passed, 0 failed of 218` at `db212f167` (no `src/` byte changed since) | HEAD after this commit is the checkpoint's commit; tree clean; nothing pushed after `79b2f6f15`
|
||||
|
||||
**Replay this block into the chat at the next session start (R64); it is the ONLY in-phase context the next session inherits.**
|
||||
**Replay this block into the chat at the next session start (R64); it is the ONLY in-phase context the next session inherits. Everything below
|
||||
is what S106 knew and the next session must not re-derive.**
|
||||
|
||||
1. **Where we are.** Phase 37 (the structs phase) opened 2026-09-11 in S106 at Max. Gate 1 approved with Drew's four decisions: **grind to
|
||||
zero** (raw casts, lying declarations, levers — the plan's §"What zero means" defines each honestly); **placeholders + cited evidence**
|
||||
for every type/field name; **T7 = the P36 agent lane at Drew's cap, started only on his word**; **the full canonical declaration layer**.
|
||||
R107–R117 ratified. T0, T1 (census + map) and T2 (the probe; its five measurements and the pricing are the T2 log entry) are
|
||||
committed. **NEXT = T3 — the tools (Max for the design, xHigh to finish):** grow `tools/restruct.py` from its T2 form into the engine —
|
||||
rungs L (definition fold WITH member renames), D (the declaration solver: per callee, every declaration in the TU replaced; admissible =
|
||||
the definition's TU and every caller TU IDENTICAL; the named causes: definition width/return/arity, K&R sites marked `// K&R:`), S (rung 1
|
||||
per base, all bases of a body), S2 (the greedy kept-cast set), S+A (levers stripped), X (the residual classified by pass), R (recipes),
|
||||
the ledger `.run/P37/restruct/ledger.jsonl` keyed alias+tu+fn+body-hash with per-site verdicts, `inflight.json` the only restore,
|
||||
`--plan/--apply/--only/--batch/--label`, workers own whole TUs, header edits serialized and judged on every includer with ONE retry of a
|
||||
COMPILE-ERROR, selftest + a known-true no-op run + negative controls; `tools/struct_layout.py` factored out of the census (the o32
|
||||
engine, now cc1-validated on 5,283 definitions) with the field assertion; the canonical type files' writer (final style: hex offset
|
||||
comments, `unk<HEX>`, `// size = 0x..`; `Unkstruct_<addr>`; PsyQ names for SDK-identical layouts; one file per symbol space); the
|
||||
reinterpret macros in `include/common.h`; **the linked-relocation oracle mode** (the f3 control: object DIFFERS, binary identical) with
|
||||
its known-true and negative controls, and the committed linker fragment for any new base symbol; `tools/restruct_cycle.sh` (detached).
|
||||
Verify: selftest; the calibration line; one batch on one overlay end-to-end with R22 green.
|
||||
2. **What the next session must know (T1's findings, for T2–T5).** (a) The phase's counters, re-derived by `tools/type_census.py` (coverage OK, controls 4/4): **503,016 raw dereferences** (P 409,007 +
|
||||
I 60,666 + X 13,800 + M 19,543) in 69,497 bodies + 18,912 address-of + 36,681 typed cast-member (rung 1's form, counted apart);
|
||||
7,261 struct definitions (1,179 canonical, 5,995 inside `.c`), 525 layouts, 206 duplicate classes (2,789 names; 46 layout-twin classes /
|
||||
1,844 names are NOT duplicates without flow evidence), 39 VARIANT names, 141 dead canonical names; 98,648 lying declarations / 1,609
|
||||
callees; 6,415 names with >1 body text (identity = body hash, never the name); **the lever finish line is 4,010 marked sites + 6,717
|
||||
direct GTE statements in bodies + 314 per-TU asm macro definitions** (`lever_census --check --strict`, now literal). (b) The struct map
|
||||
(`.run/P37/census/struct_map_top.json`, `docs/struct-map.md`): 18,760 types explain 99.2 % of the sites; the two largest — 107,598 sites /
|
||||
0x10C over 9,753 pointer globals (partly fused at low offsets) and 65,397 sites / 0x24C with the instance global `D_80126B58` (the
|
||||
actor record) — carry width conflicts the map reports; the map's merge rules are in SETUP §P37 S106; it is a MODEL — T2 tests it on
|
||||
bytes (does the member spelling hold?) before T5 authors anything from it. (c) The Plan agent's four corrections (the log entry "T1
|
||||
design"): the relocation-spelling oracle mode + linker fragment for base symbols (T3), body-hash identity (done), form C (done),
|
||||
duplicates = layout + evidence (done; milestone item 1 reads "per type"). (d) Two doc corrections owed at T9: `actor-struct.md`'s
|
||||
`s16` at +0x108/+0x10C (the code declares 4-byte fields; the bytes decide), and `include/common.h:28-30`'s "identical codegen to
|
||||
`p->field`" for `M2C_FIELD` (false — the `/s` flag differs; `gen3-handoff.md:161-164` says the same). (e) The 24 parked
|
||||
P36 classes are now ONE file, `.run/P37/census/parked.tsv` (fn, alias, tu, copies, sites, kinds, regs, needs {signature 13, struct 7, ? 4},
|
||||
pack, patches, headline; 453 bodies / 691 needed sites; 22 more PARK names are minimum-lever banks) — generated by the census from the
|
||||
P36 ledger + `pick.py`'s PARK set + each pack's `mechanism.md`; patch files exist for 6 (`main__func_800385C0/scratch/signature.patch`,
|
||||
`main__func_80029D3C/scratch/patch_signature.c`, `ov_SC07_007__func_80180FA4/scratch/signature_3args.patch`,
|
||||
`ov_SC02_027__func_80180A54/scratch/signature.patch`, `md_SC07_004__func_801A1E94/signature.diff`, `…__func_801A5C44/signature.diff`),
|
||||
whole-TU scratch copies for 8 more; `func_80180200` has no pack (it lives in `ov_SC03_115__func_8017FC5C/`). (f) `func_8013D9B0`'s 268
|
||||
markers (kind `gte-unsigned`) are now inside `--strict`'s direct-GTE count (resolved: they are asm statements outside the header, to
|
||||
become macro calls). `lever_census`'s controls are re-keyed and 4/4 OK. (g) The 4,010 reconcile exactly against the ledger: 1,637 drawable RESIDUE + 1,080 minimum-lever banks in PARK + 593 parked RESIDUE +
|
||||
450 GTE clobber variants (`gte-lever`; 64 per-TU `_m` macro definitions, `gte_rt_m` 55) + 94 arg-only pins (83 classes) + 83 whose latest
|
||||
ledger row is a failed attempt (invisible to `pick.py`; `delever.py --redraw`) + 24 + 18 + 13 asm-body + 10 drawn + 8 file-scope.
|
||||
(h) The §396(a) refuting body IS recorded: `.run/P36/agents/ov_SC03_110__func_801800F0/mechanism.md:64-70` (d11) — T9 writes the correction.
|
||||
(i) The type tools' reach: `lift_types.py` scans only `src/ov_*/*.c` (line 49); `engine_types.h` is included by 3,818 TUs via
|
||||
`engine_prelude.h` but by 0 of the 85 root `.c` and 12 of 87 `md_*`; `decl_repair.py` compares `.text/.rodata/.data` sections (the scratch
|
||||
path lands in the object) while `delever_oracle` compares whole objects compiled in place.
|
||||
3. **Environment.** `~/bfm-decomp` on ext4; `.venv/bin/python`; every build with `JOBS=16`/`-j`; the headless Ghidra MCP was started by
|
||||
the SessionStart hook (pid in `.run/ghidra-mcp.log`) and is NOT needed until T8 — stop it via the sentinel before any commit that touches
|
||||
`ghidra/` (nothing does until T8; R23). The oracle's snapshot baseline: `.run/P36/delever/baseline/` (7,428 objects at `db212f167`, taken
|
||||
after the green R22 — R112). The Plan agent (tool design stress-test) may still be running or have reported: its report goes into T3's
|
||||
design notes; if it is lost, T3's design at Max re-derives it from the plan's §B questions.
|
||||
4. **Commands that work.** Probe/engine: `.venv/bin/python tools/restruct.py --try <TU> <fn> [--base bclass:name]`; `--probe --seed 37 -j 12`
|
||||
(6.5 min); `--rerun-differs`; `--probe-decls -j 12` (16 min); `--probe-defs`; `--audit-layouts -j 8` (80 s); `--fanout-cost -j 16` (35 s;
|
||||
NOTHING else may compile meanwhile — it writes `engine_types.h` in place). Census: `.venv/bin/python tools/type_census.py -j 16 [--sites]` (≈3.5 min; `--selftest` 21/21; outputs under
|
||||
`.run/P37/census/`; `docs/struct-map.md` regenerated). R22: `make clean && make extract-all JOBS=16 && make check-all JOBS=16` (≈86 s; read the exit code with
|
||||
`pipefail`). Oracle: `.venv/bin/python tools/delever_oracle.py --calibrate <aliases> -j 16` (147/147 in 2.3 s). Census (P36's, for the
|
||||
lever side): `.venv/bin/python tools/lever_census.py --check -j 16`. Series: `tools/readability_progress.py --snapshot <label>`,
|
||||
`tools/lever_progress.py --check`. Tools-health (foreground, ~8–15 min): `make tools-health`.
|
||||
1. **Where we are.** Phase 37 (the structs phase) opened 2026-09-11 in S106 (plan mode, Max; three Explore agents + one Plan agent at gate 1).
|
||||
Drew's four decisions: **grind to zero** on raw casts, lying declarations and levers (each "zero" defined in the plan's §"What zero
|
||||
means" — reinterpret macros/unions for genuine reinterpretations; byte-proven K&R sites kept as `()` with `// K&R: n of m args (P37 …)`,
|
||||
counted apart; legitimate GTE sequences as project-local macros in `include/gte_inline.h`, clobber-only steers closed by the shape);
|
||||
**placeholders + cited evidence** (`Unkstruct_<addr>`, `unk<HEX>`, hex offset comments, `// size = 0x..`; a field named only where
|
||||
memory-map §3.4 / actor-struct.md cite evidence); **T7 = the P36 agent lane** at his cap, started only on his word (the S98 rule), never an
|
||||
Ultracode wave without direct approval; **the full canonical declaration layer** (per definition, per TU, byte-arbitrated — never the
|
||||
Phase-16 header forced on every TU). After T0: the struct map became T1's deliverable (my recommendation on his question — he asked, he
|
||||
did not propose); **no tags/releases at phase closes** (the `v2.2.0` tag deleted; the P36 close `79b2f6f15` pushed as a normal commit on
|
||||
his word — two one-off R6 waivers, R6 unchanged otherwise: Drew pushes). R107–R117 ratified (the P36 candidates (a)–(k)). Phase 36's
|
||||
§🛑 template's `git tag` lines are retired from T10 on. Commits this session: `79b2f6f15` (P36 close), `0a55cb0fd` T0, `89c1efc54`
|
||||
+ `23fc5d215` (gate-1 adjustment + attribution), `da26c05d6` T1, `cd39c79de` + `48e68c5b7` T2, then this checkpoint's commit.
|
||||
|
||||
2. **The counters (T1, `tools/type_census.py`, coverage OK, controls 4/4; every verify line in the T1/T2 log entries).**
|
||||
Raw dereferences **503,016** = P `*(T *)(…)` 409,007 + I `*(T *)ident|&D_|0x80…` 60,666 + X `((T *)e)[i]` 13,800 + M `M2C_FIELD` 19,543,
|
||||
in 69,497 bodies (+18,912 A address-of casts and +36,681 C `((T *)p)->f` typed cast-member accesses, counted apart — C is rung 1's own
|
||||
form, already a COMPONENT_REF); by base: param 276,860 · local 149,915 · `&D_` 40,122 · nested 24,076 · `D_` by value 8,860 · other 1,678;
|
||||
the first parameter (`a0`/`param_1`/`arg0`) 241,554 = 48 %. Definitions **7,255** (canonical header 1,179 · `.c` file-scope 4,085 ·
|
||||
`.c` block-scope 1,904 · per-function shared headers 63 · other headers 24) over **527 layouts**; 206 duplicate classes / 2,789 names
|
||||
(tier 1 same-named 122, tier 2 opaque 84; **46 layout-twin classes / 1,844 names are NOT duplicates without flow evidence** — milestone
|
||||
item 1 reads "one definition per TYPE"); 40 VARIANT names; 141 dead canonical names; 75 canonical names redefined in `.c`.
|
||||
Declarations: 99,130 definitions (1,889 K&R); 4,279,365 extern function declarations over 12,139 names (3,870 spelled >1 way);
|
||||
**6,415 names carry >1 body text** (identity = body hash `F:<nhash>` / shared header `H:<path>` / external `X:<space>:<name>`, never the
|
||||
name); 1,078,837 data externs over 55,569 symbols (18,442 typed >1 way); asm-label aliases 5,749 (225 names); builtins memcpy 301 / abs
|
||||
144; attributes packed 64 / aligned 12; **lying declarations 98,648 over 1,609 callees** (K&R-empty 91,357, narrow 7,291).
|
||||
**Levers (`lever_census --check --strict`, made literal at T1): 4,010 marked (2,141 pins + 1,869 asm) + 6,717 direct GTE asm statements
|
||||
in bodies (kinds `gte`/`gte-unsigned`, incl. func_8013D9B0's 268) + 314 per-TU asm macro definitions** — the `--check` headline (4,010)
|
||||
keeps the P36 definition for the series. Global blocks: 2,123 runs of ≥4 adjacent scalar `D_` symbols (14,251 symbols). The 24 parked
|
||||
P36 classes with reasons: `.run/P37/census/parked.tsv` (453 bodies / 691 sites; needs: signature 13, struct 7, ? 4).
|
||||
**The struct map** (`.run/P37/census/struct_map_top.json` tracked; `struct_map.json` 6.4 MB and `body_base_type.json` 11 MB ignored,
|
||||
regenerable): 18,760 types explain 99.2 % of the deref sites; the two largest are `Unkstruct_800B5CB8` (107,598 sites / 0x10C over 9,753
|
||||
pointer globals — partly fused at low offsets) and `Unkstruct_80126B58` (65,397 sites / 0x24C, one instance global — the actor record of
|
||||
`docs/actor-struct.md`); merge rules in SETUP §P37 S106 (typed-use parameters; positive shared evidence ≥1 offset with the callee's map
|
||||
≥2 offsets; single-source locals; a width-conflict veto; `AT:`/`PTR:` global nodes seeded by the declared type; FLD nodes for pointer
|
||||
fields). It is a MODEL: T2 tested it on bytes (below); T5 authors from it after T3's engine can judge every type per body.
|
||||
|
||||
3. **What T2 proved on the bytes (the design inputs for T3; the full tables are the T2 log entry and `.run/P37/probe/*.md`).**
|
||||
(a) **Rung 1** (body-local `((P37S_<hash> *)base)->unk<HEX>` with the type from the cluster layout; misaligned layout entries skipped;
|
||||
pointer fields only where the site's pointee equals the field's; sign-mismatch loads stay casts; the typedef inserted at file scope before
|
||||
the function) is byte-neutral on **126/139 judged bodies (90.6 %), 716 sites moved**; 226 of 942 candidate sites (24 %) were skipped
|
||||
because the cluster layout cannot hold them (no-field 133, sign 44, width 33, index 13, negative 3) — T5's type-authoring residue.
|
||||
(b) **The 13 DIFFERS are the `/s` asymmetry** (COUNT/ORDER residuals, never REG): a converted store carries `MEM_IN_STRUCT_P`, the body's
|
||||
bare fixed globals do not, `sched.c:837`'s escape hoists a fixed load over it. Byte-read on `func_801814AC` (ov_SC05_010 :3030): five
|
||||
`D_801C7E30` sites as members + the `+0x34` store as a cast → IDENTICAL; the pointer global itself as a member fixes that store and moves a
|
||||
`li v0,10` elsewhere. **The `/s` flag is a per-ACCESS dial** (cookbook §458, §351). (c) **Rung S2** (leave-one-out, then cumulative
|
||||
revert in source order, then minimise) **closes 13/13 keeping 27 of 716 sites as casts** — ≤2n+1 compiles, no agent. (d) S+A (levers
|
||||
stripped too) closed 0/5 lever bodies — unmeasured at this size; T6 measures it on all 1,980 lever bodies. (e) **Declarations:** the
|
||||
definition's ANSI signature written into a callee's lying declarations is IDENTICAL for **4,473/4,828 (TU, callee) pairs (92.6 %), 0
|
||||
DIFFERS**; 355 compile refusals = 136 arity contradictions (`too few/many arguments` → the K&R sites), 121 conflicting declarations
|
||||
elsewhere in the TU (a repair must replace EVERY declaration of the callee in the TU), 98 unread causes (main's `At top level:` 22; a
|
||||
`#define NULL` context line 14) — T3's solver reads full messages (R103). (f) **Definition fold** (`typedef <Canon> <Local>;` for a
|
||||
same-layout tier-1/2 duplicate): 12/20 TUs IDENTICAL, 8 COMPILE-ERROR `structure has no member named a/f0/v` — an opaque-tier fold
|
||||
renames members, so the fold must rewrite the TU's member accesses too. (g) **The layout engine agrees with cc1 on 5,283 file-scope
|
||||
definitions / 29,248 named fields (0 rejected, 0 unresolved)** — `restruct --audit-layouts`; on the way two engine gaps were fixed
|
||||
(nested-paren `__attribute__((aligned(4)))`, per-field `aligned(N)`) and 27 dead `#ifndef BFM_ENGINE_TYPES_H` standalone blocks (21 TUs)
|
||||
are now blanked by the census. (h) **The oracle:** the f3 control (`func_801A4258` in md_SC07_004 with `s16 *p = &D_801F8870; p[1]…`)
|
||||
is **whole-object DIFFERS and `make build BINARY=md_SC07_004` BYTE-IDENTICAL** (`87ac0de3…`) — relocation SPELLING differs, linked bytes
|
||||
do not → T3's linked-relocation mode is required for global blocks/arrays; `engine_types.h` fan-out = 3,818 TUs / 3,975 objects in **35 s
|
||||
wall at -j16** (0.141 s/object), with 26 TRANSIENT compile errors at 16 threads (0 on re-run: retry a fan-out failure once). Oracle
|
||||
tags must contain no spaces (the scratch object path lands in the pipeline). `judge_text` writes the file IN PLACE and restores it —
|
||||
two workers must never own one file; header judgements compile other workers' TUs, so headers run serially after the `.c` bodies.
|
||||
(i) **Pricing (R41):** T4 ≈ 1 unattended session (94k pairs, ~93 % mechanical, ~6.6k to the solver); T5 ≈ 1 session (5,995 in-`.c`
|
||||
definitions, ~60 % by alias, the rest with member renames; header batches of dozens per 35-s fan-out); T6 ≈ 1–2 sessions (69,497 bodies ×
|
||||
2 s + S2 on ~10 %; expected floor after S2 ≈ 4 % of converted sites kept + the 24 % the types cannot hold yet); T7 open-ended by
|
||||
decision 1.
|
||||
|
||||
4. **T3 — the design brief (Max).** Grow `tools/restruct.py` (T2 form: `--try/--probe/--rerun-differs/--probe-decls/--probe-defs/
|
||||
--audit-layouts/--fanout-cost`; the Plan agent's stress-test — the S106 "T1 design" log entry — is its design review) into the engine:
|
||||
- **Rungs.** L (definition fold WITH member renames — the fold's field map rewrites `.a`→`.unk0` in the TU; tier 1/2 only; layout twins
|
||||
never); D (the declaration solver: per callee, EVERY declaration in the TU replaced by the candidate prototype; candidates = the
|
||||
definition's ANSI signature > its K&R-promoted form > the return axis flipped > `()`; admissible iff the definition's TU AND every
|
||||
caller TU are IDENTICAL; the named causes when none is — the definition's width (`s16`→`s32`, `main__func_800385C0/scratch/
|
||||
signature.patch`), a `(void)` definition that reads `$a0`, a `void` one that returns `$v0`, a caller passing fewer args (→ `// K&R:`
|
||||
kept `()`, counted apart); `argcheck.definitions()` re-keyed by body identity, not first-definition-per-name); S (rung 1 on EVERY typed
|
||||
base of a body, not one — the probe did one; bases from `body_base_type.json`); S2 (the greedy kept-cast set, then the kept casts
|
||||
ledgered with the pass); S+A (the marked levers stripped, `delever_search.lever_free_body`); X (the residual classified: reuse
|
||||
`delever_search.classify` + name the pass: SCHED-ALIAS / CSE-KILL / WIDTH / ADDRESS-FOLD / ALIGNMENT / PROMOTION / OTHER); R (recipes:
|
||||
`u16 X[]` vs scalar, the `s16[3]` array walked through a pointer, the register union, `p = &tbl[i]` to drop `/s`, the cast-at-the-shift,
|
||||
§357's one struct pointer).
|
||||
- **The engine pattern (from `tools/delever.py`, read its docstring + `work_file`/`includers`/`site_edits`/`apply_edits`/`restore_file`):**
|
||||
positional token-exact edits with REFUSAL on a token mismatch (R43); the FILE as the write unit and its final compile through every
|
||||
recipe as the proof; the ledger `.run/P37/restruct/ledger.jsonl` keyed alias+tu+function+body-hash (R48/R114) with per-site verdicts
|
||||
MEMBER / KEPT(pass) / REFUSED(reason) / DECL-CANON / DECL-KEPT(cause); `inflight.json` the ONLY restore (R102); `--plan/--apply/--class/
|
||||
--only <base|tu|type>/--batch/--batches 1/--label`; workers own whole TUs; header edits serialized and judged on every includer with
|
||||
one retry of a COMPILE-ERROR; progress logs (R55); a selftest with fixtures for every form + a known-true NO-OP run (a body already
|
||||
spelled with members → zero edits; the tree's own text → every object IDENTICAL) + negative controls (a wrong offset, width, sign, a
|
||||
scaled offset through an `s32 *` base, a merge across a width conflict — each must DIFFER or be REFUSED); `tools/restruct_cycle.sh`
|
||||
cloned from `delever_cycle.sh` (batch → exit code → R22 → census → commit with the log line and the 🛑 headline), DETACHED with
|
||||
`setsid nohup` (R115), never a harness background task.
|
||||
- **The oracle's linked mode (`delever_oracle.judge_linked`):** compile as today, then compare `.text/.rodata/.data` with relocations
|
||||
RESOLVED (symbol address + addend; a `D_/func_` name IS its address, else `corpus.symbols(alias)` / `build/<alias>/<alias>.map`) and the
|
||||
symbol table modulo renamed/new symbols; `tools/masked_diff.py:270-290` compares reloc operands as strings — add a resolver hook.
|
||||
Known-true: the f3 spelling IDENTICAL-linked and DIFFERS-whole-object; negative: a wrong addend DIFFERS in both. Default stays
|
||||
whole-object; linked only for edits the ledger marks relocation-spelling (global blocks, base symbols). A NEW base symbol (e.g.
|
||||
`D_80078E00`, which no map carries) goes into one committed linker fragment (`config/symbols.structs.txt`, linked as a `-T` like
|
||||
`LIBCD_SYMS` at `Makefile:1182-1186`) in its own R22-gated commit BEFORE any TU references it; unsized in splat.
|
||||
- **`tools/struct_layout.py`:** factor `Resolver`/`layout_of_fields`/`layout_hash`/`field_offsets` out of `type_census.py` (import it
|
||||
there and in `restruct.py`); the field assertion "every converted site's (base type, offset, width, sign) is a field of its type"
|
||||
(the `SYNC_FIELD` equivalent) as a `--check` the census runs. The engine is cc1-validated (probe (g)); keep `--audit-layouts` as its
|
||||
regression test.
|
||||
- **The canonical type files' writer:** final style (sotn's: `/* 0xNN */` hex offset comments, `unk<HEX>` fields, anonymous or `u8
|
||||
pad<HEX>[n]` padding — decide one, `// size = 0x..`), `Unkstruct_<addr>` (the lowest instance global's address; else the exemplar
|
||||
function's), PsyQ's names where a layout equals an SDK type, the PsyQ ground-truth block of `engine_types.h:10-35` preserved,
|
||||
`MATRIX_L48` kept as documented; one file per symbol space (`src/shared/engine_types.h` stays the `ov`/resident file; `src/shared/
|
||||
main/types.h`, `src/shared/<slot>/types.h` added; main's 85 root `.c` and 75 of 87 `md_*` TUs gain the include); the reinterpret
|
||||
macros (`LOH/LOW/LOHU/…`, sotn `common.h:80-97`) in `include/common.h`, each proven byte-identical to the cast it replaces.
|
||||
- **Verify (the plan's):** selftest; the calibration line (`delever_oracle --calibrate`); one batch on one overlay end-to-end with R22
|
||||
green. Then **the rules check (P6) after T3** — four tasks will be complete.
|
||||
|
||||
5. **Files and artifacts.** Tools: `tools/type_census.py` (T1; selftest 21/21; `-j 16 --sites` 3.5 min; writes `.run/P37/census/
|
||||
{type_census.json,type_census.txt,struct_map_top.json,parked.tsv}` tracked + `struct_map.json,sites.jsonl (213 MB),body_base_type.json,
|
||||
cache/` ignored + `docs/struct-map.md`), `tools/restruct.py` (T2 form), `tools/lever_census.py` (controls re-keyed: func_800226C0 14,
|
||||
func_80178004.h 8, func_801896EC 2 bare; `--strict` literal), `tools/progress.py` (`counts.types` + README sentence + 2 dated
|
||||
corrections), `tools/readability_progress.py` (16 census columns; the TSV read by column NAME; a snapshot REFUSES a stale census — its
|
||||
`head` must be HEAD). Docs: SETUP §P37 S106 (two sections), dictionary rows for both tools, `docs/struct-map.md` + its Reference-index
|
||||
row, cookbook **§458**, decision log **P37 S106**, accelerators **P37 S106** (→ DK-65, corrected in the kit: the struct SPELLING is a
|
||||
scheduling lever), `docs/gen3-handoff.md` + `docs/wiki/Where-the-project-goes-next.md` date-stamped corrections of "byte-neutral". Probe
|
||||
evidence `.run/P37/probe/{probe_table.md,probe_rows.jsonl,decl_table.md,decl_rows.jsonl,defs_rows.jsonl,layout_audit.json}` tracked;
|
||||
`probe/cc1/`, `probe/obj/` scratch ignored. Baseline logs `.run/P37/baseline/{r22_t0.log,oracle_t0.log}`. **Owed at T9 (three doc
|
||||
corrections, one a build input):** `include/common.h:28-30` ("identical codegen to `p->field`" is false — comment-only edit, R22 after),
|
||||
`docs/actor-struct.md` (+0x108/+0x10C are 4-byte in the code, not `s16`), `include/common.h`'s `M2C_FIELD` note; plus the §396(a)
|
||||
correction (`.run/P36/agents/ov_SC03_110__func_801800F0/mechanism.md:64-70`).
|
||||
|
||||
6. **Environment and commands.** `~/bfm-decomp` on ext4; `.venv/bin/python`; every build `JOBS=16`/`-j`. The headless Ghidra MCP is
|
||||
started by the SessionStart hook and stopped by the SessionEnd hook (`tools/ghidra_mcp_stop.sh`, the sentinel) — not needed until T8;
|
||||
nothing under `ghidra/` is tracked. Oracle baseline snapshot `.run/P36/delever/baseline/` (7,428 objects at `db212f167`; the tree's
|
||||
`src/` bytes are unchanged since — a comment-only or doc commit does not invalidate it; refresh with `delever_oracle --snapshot-baseline`
|
||||
after any green R22 that follows a `src/` change). R22: `make clean && make extract-all JOBS=16 && make check-all JOBS=16` (≈86 s; read the
|
||||
exit code, `pipefail`). Oracle: `.venv/bin/python tools/delever_oracle.py --calibrate <aliases> -j 16`. Census: `tools/type_census.py -j 16
|
||||
--sites`. Engine probes: `tools/restruct.py --try <TU> <fn> [--base bclass:name]` · `--probe --seed 37 -j 12` (6.5 min) · `--probe-decls
|
||||
-j 12` (16 min) · `--probe-defs` · `--audit-layouts -j 8` (80 s) · `--fanout-cost -j 16` (35 s; NOTHING else may compile meanwhile — it
|
||||
writes `engine_types.h` in place). Series: `tools/readability_progress.py --snapshot <label>` (needs a fresh census), `tools/lever_progress.py
|
||||
--snapshot <label>` / `--check`, `tools/progress.py --json --readme [--check]`. Health: `make tools-health` (foreground, 8–15 min);
|
||||
`tools/doc_links.py --strict`, `tools/tool_census.py --check`, `tools/kit_lint.py`, `tools/kit_coverage.py`, `make kit-corpus`,
|
||||
`tools/cookbook_index.py [--check]` — all OK at this checkpoint. No harness task-list tool exists in this build (R28 = this file's ☐/☑).
|
||||
|
||||
## Approved plan (verbatim, gate 1 — 2026-09-11)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user