- Project: BFM (PS1) matching decomp, 100 % byte-identical since P32. Gen 3 = readability. P35 dedup, P36 levers off, P37 (legacy) = the structs phase part one: census, struct map, `restruct` engine, the declaration layer to its floor. This phase = legacy P38 / P37-carry T5 ("rung L"): one canonical definition per TYPE.
- Seed sources (not to be re-read by experts): `phase-ends/PhaseEnd_Phase37.md:75-99` (carry list), `phase-ends/logs/Phase37.md` S108 checkpoint (~L820-965), cookbook C0507/C0508. Digest below is what they say.
- Oracle = the byte gate. Every edit batch is judged by `tools/restruct.py` (whole-object byte compare; LINKED mode for global-block edits whose relocation spelling moves but linked bytes do not) and committed only after a green R22 (`make clean && make extract-all JOBS=16 && make check-all JOBS=16`, ~95 s, read the exit code; green = `extract-all: 217 extracted, 0 failed of 217` + `check-all: 218 passed, 0 failed of 218`).
- Interpreter: BFM tools run under `.venv/bin/python` (Makefile `VENV_PY`), `-j 16` / `JOBS=16`, repo on ext4. PA tools (`plan_edit`, `task_log`, `research_add`) run under `/usr/bin/python3`. `tools/type_census.py`, `tools/restruct.py`, `tools/restruct_cycle.sh`, `tools/lift_types.py`, `tools/delever_oracle.py`, `tools/lever_progress.py` are BFM's own product tooling (not Project Architect's) and are in scope to edit.
- Census state: `.run/P37/census/type_census.txt` is STALE (2026-09-13 at bd7d32ec2, before T4). Its DEFINITIONS line: 7,255 bodies = canonical header 1,179 · `.c` file-scope 4,085 · `.c` block-scope 1,904 · per-function shared headers 63 · other headers 24 · include/ 0; 3,143 names, 527 layouts, 206 duplicate classes over 2,823 names (tier 1 122 same-named, tier 2 84 opaque; 46 layout-twin classes / 1,867 names kept apart); 40 variant camps. T4-close figures: 1,796 lying declarations / 76 callees (`K&R-empty` 1,373, `narrow` 423), 5,262 K&R sites counted apart (the original's calling convention, not debt), 418 cross-binary apart. T1 re-baselines.
- Canonical type files: `CANON_HEADERS = ("src/shared/engine_types.h",)` at `tools/type_census.py:55`; comment there says T3/T5 add `src/shared/main/types.h` and the slots'. Existing trees: `src/shared/{main,ov,slot_800CAE08,slot_801E25E8,slot_801E7B28,slot_801ED988,slot_801EF468}/`, `src/shared/engine_prelude.h`, `src/800_shared.h` (main). Only `include/{common.h,gte_inline.h,include_asm.h}` in `include/`.
- Head types: the three names `Unkstruct_80126B58` (entity/actor, 0x24C, 146 fields, preview skipped {overlap 28, misaligned 1}), `Unkstruct_800B5CB8` (0x10C, 90 fields, skipped {overlap 44}), `Unkstruct_80078E00` (the player block, instance at 0x80078E00; `docs/actor-struct.md:3` "~154 fields over 0x24C") exist ONLY in the generated preview `.run/P37/restruct/types_preview/ov.h` (from `restruct.py --write-types --top 6 --space ov`); 0 occurrences in `src/`, `include/`, `docs/`. The overlaps are the struct map's width conflicts: the union-vs-split call is per offset. Source map: `.run/P37/census/struct_map.json` (18,760 types, explains 498,896/503,016 sites, 99.2 %), rendered `docs/struct-map.md` (generated, never hand-edited). Parked P36 work: `.run/P37/census/parked.tsv` (24 classes, 453 bodies; struct 7).
- Naming law (G62): `Unkstruct_<addr>` placeholders, `unkHEX` fields with `/* 0xNN */` comments and `// size = 0x..`; a field gets a name only with cited evidence (`docs/memory-map.md` §3.4, `docs/actor-struct.md`); PsyQ layouts take Sony's SDK names. Layout twins without evidence stay apart (S106 "per type", not "per layout").
- Byte facts (C0507 §458): struct spelling is a per-access dial — `*(T*)(p+k)` carries no `MEM_IN_STRUCT_P`, a member access does, so a cast→member rewrite can move scheduling. An opaque-tier fold renames members (T2 probe: 8 of 20 TUs), so rung L rewrites accesses too, and every fold is judged, never assumed neutral. (C0508 §459): a K&R site is per-TU, but a definition in a shared header makes it per-definition; a return type is observable in a delay slot; what a prototype cannot hide: types the TU cannot see (1,120 `parse error` → `DECL-KEPT` "type not visible", `tools/restruct.py:1741-1745`), `conflicting types` (417, `:1737-1739`), jr-merged callers (10, → 3.44), void defs whose callers use `$v0` (5, → 3.40), zero-arg callers passing their own `$a0` (→ 3.40).
- Owed from P37: `make tools-health` (last green at P36 close; >15 min, `audit-cdecl` ~787 s) and `make kit-corpus` (the kit's `tool_census --check` may read red until then); `docs/actor-struct.md`: +0x108/+0x10C are 4-byte and `D_80078F08/0C` are code-width 4, not 2 (the census control prints them as `doc-disputed`); the 45 TU-CONFLICT rows; the 13 `syntax error` units in `src/800.c` (600 KB; the marker lives in the restruct ledger / census logs, not the source); the cookbook §396(a) correction.
- Tree at plan time: `config/kit_coverage_map.tsv` modified (uncommitted), `phase-ends/PhaseEnd_Phase37.md` untracked (the router's migration ceremony owns it; tasks never stage it). Disk: ~22 GB free; `.run/P32/` (6.6 GB) and parts of `.run/P36/` are prunable scratch, but `.run/P36/delever/baseline/` and `.run/P36/delever/ledger.jsonl` are live inputs of `delever_oracle` / `lever_progress`.
- Gotchas carried (S108, binding for every task):
- Stop a judge with `kill -INT <pid>`, WAIT for exit (~2 min, finishes in-flight units), only then `--restore`; an early restore consumes `inflight.json` (S108: 6 files hand-reverted, 197 orphan rows).
- Never put a literal of your own command line in `pgrep -f`/`pkill -f`; bracket a char (`restruct_cycl[e]`).
- Never edit tracked files the cycle rewrites between a cycle's R22 and its commit; check `ps` for `make`.
-`delever_oracle --snapshot-baseline` only after a GREEN clean fleet run; `--apply` refuses a stale snapshot; `restruct_cycle.sh` refreshes snapshot + calibration itself when HEAD moved.
-`DECL-NONE` is a DONE verdict: redraw with `--redraw VERDICT:cause`, never bare `DECL-NONE`.
- A flat yield is not "nothing drawable": bucket the remainder by ledger verdict before stopping (`md_*` hid 3,700 never-judged pairs).
- A pack's "one caller" is a claim: `grep -rnE '\bfunc_X\s*\(' src` before a signature unit.
- Census blind spots that moved counts by thousands: include graph not followed, name-keyed maps (6,415 names carry >1 body), column-0 definition regex, one-sided redraw key, snapshot stamped fresh over empty `build/`. Key every map by (binary, address, body).
- Shared headers are judged serially (~35 s per judgement over 141 includers): budget header batches accordingly.
- Cycle launch form (detached, outlives the agent): `setsid nohup bash -c 'LABEL_PREFIX=<p> TASK=<T> CENSUS=0 tools/restruct_cycle.sh START END 400 <RUNG>; echo "cycle exit=$?"' > .run/P38/restruct/cycle_<x>.log 2>&1 &`; `HEADERS=1` for header batches; `REDRAW="<VERDICT>:<cause>"` to redraw. Scratch for this phase: `.run/P38/`.
- Long commands: one tool call stays under 285 s. Anything longer: `bash tools/run.sh --bg <name> -- <cmd>` then `bash tools/run.sh --wait <name> --max 280` (repeat). R22 (~95 s) fits one foreground call.
## Rationale
- Instrument first (T2). `type_census --check` (`tools/type_census.py:1621-1634`) also fails on raw cast sites (~503k, phase 3.39) and lying declarations (K&R-empty/narrow residue, phase 3.40), so it cannot go green this phase; it does not gate dead names at all, and no mode counts parse-error declarations. The generation milestone's content is kept exactly; T2 adds the phase-scoped `--check-structs` that asserts it, and the closing expert runs that. `--check` itself stays unchanged for 3.39/3.41. Rejected: reading numbers off the census text by hand (R85: a check compares derived figures; G58: generated, never typed).
- The parse-error count comes from the restruct ledger's latest verdict per unit, read through restruct's own latest-verdict function (imported), never a second filter (P37 proposed rule (c): one "done" filter used by every side).
- Head types before the long tail (gen-plan order): the two big types own most of the 206 classes' members and the 1,120 type-not-visible declarations; folding them first makes the long tail mostly mechanical `lift_types.py` / rung L work, and T6's redraw depends on the types being visible.
-`effort: high` on T3 only (1 of 7): the union-vs-split decision per overlapping offset (28+1 on the entity, 44 on `Unkstruct_800B5CB8`) and the canonical-file layout are judgments the byte gate cannot arbitrate — both spellings can be byte-identical; the choice rests on reading the access evidence in `struct_map.json` (which functions read which width at which offset) and on what a reader of the C should see. T4 then applies that design mechanically at medium.
- T3 designs + proves on exemplars; T4 applies fleet-wide. Splitting keeps the Fable context on the judgment and the rollout on cycles at medium.
- T5 (long tail) separate from T4: different instrument (`lift_types.py` + rung L cycles vs a designed layout), different evidence, and a natural checkpoint where `--check-structs` shows the head types gone from the dup classes.
- T6 (declarations) after T5: the gen plan's "most of the 417 `conflicting types` fall to the canonical files" and the 1,120 `parse error` redraw only pay once every type is in a canonical file every TU includes. The 13 `src/800.c` syntax-error units and 45 TU-CONFLICT rows are the same class (declaration environment) and ride along.
-`docs/actor-struct.md` fix goes in T2: the census control `player_block_widths` reads it and currently prints two `doc-disputed` rows; the instrument task owns making its controls clean.
- Coder tier opus55 throughout (the only tier). Test design: no new suites; the gates are R22, the restruct judge, `type_census --selftest` + `--check-structs`, and T2's known-true case (the current tree must FAIL `--check-structs` with its current counts before any fold lands).
- No `## Developer decides` items: naming follows G62, file layout and union/split are engineering (T3), agent caps do not apply (no wave lanes this phase).
- Makefile: `kit-corpus` :255-256 (`$(VENV_PY) tools/tool_census.py --all`); `tools-health` :269-360 (last line `tools-health: OK …`); `check-all` :1236 (`check-all: N passed, M failed of 218`, exit 1 on shortfall); R22 recipe :166/:1209.
- New in T2: `type_census.py --check-structs` → prints `type_census --check-structs: OK|FAIL — <violations>` and one counts line `dup_classes=… outside_canon=… dead=… variants=… pad_names=… parse_error_decls=… controls=a/b` plus non-gating report `conflicting_types=… tu_conflict=… types_floor_lying=…`; exit 1 on any gating violation.
## Cookbook
C0507 (§458 struct spelling per-access dial), C0508 (§459 declaration-layer byte facts), C0067 (§64 lift_types' three laws: fold the tagged typedef, check visibility, strip only textually identical), C0224 (§203 a deduped typedef must precede every splice point), C0012 (§8c a TU's declaration environment), C0013 (§8d templating must not change the declaration environment), C0103 (§92 conforming a declaration: pointer changes caller-neutral, width changes not), C0114 (§103 a file-scope extern constrains every later function), C0257 (§236 the declaration layer's nine gate failures).
## Research
- (no new reports) — planning digest of `PhaseEnd_Phase37.md` §Carried, `logs/Phase37.md` S108 checkpoint, C0507/C0508, and a code retrieval of the tool interfaces above; no report ids were allocated.
## Developer decides
- (none)
## Triage
- (none)
## Tasks
- T1 | next | expert-opus55 | title: phase-start health and baselines | coder: opus55 | effort: medium | files: config/kit_coverage_map.tsv, .run/P38/baseline/ | done-when: `make tools-health` last line `tools-health: OK`; `make kit-corpus` run and its outputs committed (tree clean of kit drift, `config/kit_coverage_map.tsv` either regenerated-and-committed or explained); R22 green; `delever_oracle --snapshot-baseline` then `--calibrate ov_SC04_011 ov_SC03_015 md_SC07_004 main -j 16` OK; fresh `type_census.py --out-dir .run/P38/census` + `restruct.py --audit-types` baseline numbers recorded in the summary (defs, dup classes, variant camps, dead names, pad names, lying by kind, controls) | verify: `make clean && make extract-all JOBS=16 && make check-all JOBS=16 && .venv/bin/python tools/delever_oracle.py --calibrate ov_SC04_011 ov_SC03_015 md_SC07_004 main -j 16` | reads: — | deps: — | est-ctx: 70k | review: no | wait-for: —
tools-health: `bash tools/run.sh --bg tools-health -- make tools-health` then `--wait tools-health --max 280` repeated (>15 min); delete leftover `src/shared/.cdecl_allmacros.c`; `ps` for orphaned workers after. A red item is a product-tool defect → coder fixes, commit, re-run; three distinct red items → `blocked` with the list.
Order: kit-corpus → tools-health → commit → R22 → snapshot-baseline → calibrate (snapshot only after the green clean run). Record `df -h .` in the summary; do not prune scratch.
- T2 | queued | expert-opus55 | title: --check-structs instrument and actor-struct fix | coder: opus55 | effort: medium | files: tools/type_census.py, docs/actor-struct.md | done-when: `type_census.py --check-structs` exists per ## Interfaces; gates dup classes, outside-canon defs, dead names, variant names, legacy `pad<SIZE>` names (same rule as `restruct --audit-types`, imported not re-implemented), parse-error declarations (restruct ledger latest verdict `DECL-KEPT` cause `type not visible`, via restruct's own latest-verdict function), controls; known-true case: on the T1 tree it FAILS with exactly T1's baseline counts; `--selftest` gains ≥2 tests for the new mode and passes; `docs/actor-struct.md` +0x108/+0x10C and `D_80078F08/0C` corrected to 4-byte and the control line shows no `doc-disputed` rows | verify: `.venv/bin/python tools/type_census.py --selftest && .venv/bin/python tools/type_census.py --out-dir .run/P38/census --check-structs; test $? -eq 1` | reads: tasks/T1.md | deps: T1 | est-ctx: 80k | review: no | wait-for: —
`CANON_HEADERS` (:55) becomes the list T3 decides; T2 makes it a single constant both census and `--check-structs` read. Do not change `--check` semantics (3.39/3.41 use it).
- T3 | queued | expert-fable | title: head-type layouts and canonical files | coder: opus55 | effort: high | files: src/shared/engine_types.h, src/shared/main/, src/800_shared.h, tools/type_census.py (CANON_HEADERS only), docs/struct-map-decisions.md | done-when: `Unkstruct_80126B58`, `Unkstruct_800B5CB8`, `Unkstruct_80078E00` defined once each in the canonical file(s) with every overlapping offset resolved (union or split) and a per-offset decision record citing the struct-map access evidence (`docs/struct-map-decisions.md`); canonical file set decided (ov / main / md_* / slots) and written into `CANON_HEADERS`; each layout proven on ≥2 known exemplar TUs per space via `restruct --try`/`--try-file` IDENTICAL (incl. one main and one `md_*` TU); `func_801814AC` known-true control unchanged; R22 green | verify: `.venv/bin/python tools/restruct.py --selftest --real && make clean && make extract-all JOBS=16 && make check-all JOBS=16` | reads: tasks/T1.md, tasks/T2.md | deps: T2 | est-ctx: 120k | review: no | wait-for: —
Inputs: `.run/P37/restruct/types_preview/ov.h` (regenerate with `restruct.py --write-types --type <name> --space <s> --pointees`), `.run/P37/census/struct_map.json`, `docs/actor-struct.md`, `docs/memory-map.md` §3.4. Field names only with cited evidence (G62); else `unkHEX`. Decide whether main (`src/800_shared.h`) and `md_*` share the ov definition or carry a per-space canonical file, by the layouts' identity in the map, never by name.
- T4 | queued | expert-opus55 | title: fold the head types fleet-wide (rung L) | coder: opus55 | effort: medium | files: src/**/*.c, src/**/*.h (via restruct cycles) | done-when: every local definition and variant of the three head types folded onto T3's canonical definitions through judged rung-L batches (accesses rewritten where an opaque-tier fold renames members); `--check-structs` shows none of the three in any dup class or variant camp; remainder bucketed by ledger verdict (kept units listed with cause); each batch committed after its own R22 | verify: `.venv/bin/python tools/type_census.py --out-dir .run/P38/census --check-structs; .venv/bin/python tools/restruct.py --check-ledger && make clean && make extract-all JOBS=16 && make check-all JOBS=16` | reads: tasks/T2.md, tasks/T3.md | deps: T3 | est-ctx: 110k | review: no | wait-for: —
Launch: cycle form in ## Context with `TASK=T4 LABEL_PREFIX=t38_4 … L`, `--only <type names>`; shared headers with `HEADERS=1` (serial). Onboard main and `md_*` in this task. `delever_oracle --calibrate` refresh when HEAD moved (the cycle does it).
- T5 | queued | expert-opus55 | title: long tail — one definition per type | coder: opus55 | effort: medium | files: src/shared/engine_types.h, src/shared/**, src/**/*.c, tools/lift_types.py | done-when: remaining duplicate classes folded via `lift_types.py --apply` and rung-L cycles; 40 variant camps uniquified; dead canonical names deleted; the 26 legacy `pad<SIZE>` names renamed to `unkHEX`/`pad_<off>` form; PsyQ layouts carry Sony's SDK names; layout twins without evidence left apart and listed; `--check-structs` gating counts dup_classes/outside_canon/dead/variants/pad_names all 0 | verify: `.venv/bin/python tools/type_census.py --out-dir .run/P38/census --check-structs; make clean && make extract-all JOBS=16 && make check-all JOBS=16` | reads: tasks/T2.md, tasks/T3.md, tasks/T4.md | deps: T4 | est-ctx: 130k | review: no | wait-for: —
`lift_types.py --candidates` → dry-run → `--apply` per type list; C0067's three laws and C0224 (typedef before every splice point). A tool defect in `lift_types.py` is fixed in this task (product tooling). Commit per batch (R42) before the next gate.
- T6 | queued | expert-opus55 | title: declaration layer on the canonical types | coder: opus55 | effort: medium | files: src/**/*.c, src/**/*.h, src/800.c | done-when: `restruct.py --redraw DECL-KEPT:type` (type-not-visible) and `DECL-KEPT:conflicting` redrawn to exhaustion and the remainder bucketed; `--check-structs` parse_error_decls = 0; the 13 `src/800.c` syntax-error units and the 45 TU-CONFLICT rows each resolved or kept with a named cause; summary reports before/after for types-floor lying (1,537), conflicting types (417), TU-CONFLICT (45) | verify: `.venv/bin/python tools/type_census.py --out-dir .run/P38/census --check-structs; .venv/bin/python tools/restruct.py --check-ledger && make clean && make extract-all JOBS=16 && make check-all JOBS=16` | reads: tasks/T1.md, tasks/T5.md | deps: T5 | est-ctx: 110k | review: no | wait-for: —
Out of scope (other phases): jr-merged callers (3.44), void-def/`$v0` and zero-arg `$a0` callers, return-type lies (3.40), cast sites (3.39). Grep callers before any signature unit.
- T7 | queued | expert-opus55 | title: milestone gate and record | coder: none | effort: medium | files: docs/struct-map.md, docs/levers.md, cookbook/, phase-ends/current/ | done-when: `--check-structs` exit 0 on a fresh census; R22 218/218 at the closing commit; `type_census --selftest`, `restruct --selftest --real`, `lever_progress --check` OK; `docs/struct-map.md` regenerated; cookbook §396(a) correction and the phase's generalizable gotchas added via `tools/cookbook_add.sh` | verify: `.venv/bin/python tools/type_census.py --out-dir .run/P38/census --no-cache --check-structs && make clean && make extract-all JOBS=16 && make check-all JOBS=16` | reads: tasks/T1.md, tasks/T4.md, tasks/T5.md, tasks/T6.md | deps: T6 | est-ctx: 60k | review: no | wait-for: —
## Risks
-`tools-health` red on a stale tool after P36/P37 edits → T1 stalls. Detection: its log's failing stage; fix as product-tool work in T1, `blocked` if three distinct causes.
-`--check-structs` measures the wrong scope (the P37 blind-spot class). Detection: T2's known-true requirement (current tree FAILS with T1's exact counts) and selftests; T7 re-runs with `--no-cache`.
- An overlapping offset admits no single layout that keeps every TU IDENTICAL (union spelling changes codegen per C0507). Detection: T3's exemplar probes DIFFER; resolution in T3 is a split or a kept per-TU view, recorded; if the kept set is large, T3 returns `blocked` with counts (a scope question for REPLAN).
- Header batches are serial (~35 s × 141 includers): T4/T5 wall-clock may be long. Detection: cycle logs; mitigate by batching per space, never by skipping the judge.
- Disk (~22 GB free) under cycle scratch + snapshots. Detection: `df` in T1 and each cycle log; ENOSPC → stop, report; prune only `.run/P32/` scratch, never `.run/P36/delever/`.
-`DECL-KEPT` remainder after T6 may be non-zero for reasons outside types (arity). Only the type-not-visible cause gates; others are reported and fall to 3.40.
- Stale snapshot / early restore corrupts the tree (S108). Detection: `--apply` refusal; the SIGINT-wait-restore order in ## Context.
A PhaseEnd closed without a '### Carry audit — phase <N>' section is audited as "no tables"; the auditor returns blocked, not a hand-measured substitute.
A function is keyed by (binary, address, body), never by its bare name, in every index, map and census. Scope: tooling keys only; source symbol names stay fully readable.
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Drew T