mirror of
https://github.com/Druthulu/BFM-decomp
synced 2026-10-01 23:52:03 -04:00
phase-36: T8 — lever_census --selftest + --check wired into make tools-health (0 UNMARKED, 0 orphans); --strict stays the structs phase's finish line
This commit is contained in:
@@ -303,6 +303,11 @@ tools-health:
|
||||
# (cb2fb5e6d, 2026-09-09) turned CI red for a day while every local chain stayed green (R54: a guard that is not
|
||||
# running locally is not a guard). Fix a GONE row with `tools/verbatim_check.py --update` (a one-row diff since S103).
|
||||
$(VENV_PY) tools/verbatim_check.py --strict
|
||||
# P36 T8: the lever census's self-test and its gate — every surviving register pin / asm statement outside the GTE header
|
||||
# carries a `// !FAKE:` marker (0 UNMARKED) and no marker is an orphan; a marker on a kept ordinary-C fake (do-while,
|
||||
# dead-init) is counted apart. `--strict` (0 pins, 0 asm) is the STRUCTS phase's finish line (Drew, S104), not this rung's.
|
||||
$(VENV_PY) tools/lever_census.py --selftest
|
||||
$(VENV_PY) tools/lever_census.py --check -j 16 --quiet
|
||||
$(MAKE) --no-print-directory report BINARY=main
|
||||
# AFTER report (which regenerates the digest), so this asserts the freshly-written digest agrees
|
||||
# with the tree — and, on a tree whose digest was committed stale, says so instead of staying green.
|
||||
|
||||
+1
-1
@@ -782,7 +782,7 @@ Every script under `tools/` (plus the two report make-targets), grouped by purpo
|
||||
| | `tools/kit_lint.py [--selftest] [--paths …]` | **(P33.5 task 11)** The day-one decomp kit (`decomp-architect/`) stays de-specialised: (1) LEAK — no line outside a ```` ```calibration ```` fence and off a `provenance:` line matches `SLUS|Musashi|BFM|Druthulu|func_80|ov_SC|/home/musashi|/mnt/z|172\.17\.|\bR[0-9]{1,2}\b|§[0-9]+` (fence-aware: `grep -v calibration` would drop only lines containing the word); (2) PLACEHOLDERS — the `{{NAME}}` set used under the package equals the backticked set in `templates/PLACEHOLDERS.md` (the contract); (3) SYNTAX — `bash -n` / `py_compile` (no bytecode written) / JSON+YAML parse; (4) the gitignore template diff (delegated); (5) `TODO(platform)` / `TODO(phase-N)` counts; (6) coverage — zero files is a failure. rc 1 findings, rc 2 package absent (R43). `--selftest` = the R39 control (a planted leak line + a planted unlisted placeholder must be caught, fenced and provenance lines must not). In `tools-health` (selftest, then the real run). |
|
||||
| | `tools/tool_census.py [--check | --manifest | --corpus | --all | --consumers FILE]` | **(P33.5 task 13.5)** The tools audit as a derived instrument: two independent enumerations of every tool file under `tools/` (`find` vs `git ls-files`, submodules/vendored/downloaded excluded — they must agree, R34); per tool the docstring line, its SETUP row, its CONSUMERS (Makefile/`.mk` targets, CI, the wave playbook, other tools by import or by name) and hence its class (LIVE · REFERENCED · ORPHAN); the AUTHORED facts live in `config/tool_dictionary.tsv` (phase · portability · the NEED the tool answers · what · what it hard-codes · the retirement verdict with its successor or product) with coverage asserted BOTH ways (R32 — a new tool without a row fails `--check`). Generates `docs/tool-index.md` (the need-keyed dictionary; KEEP-GEN), the kit's `tools/MANIFEST.md` (`--manifest`) and the three verbatim corpora `decomp-architect/corpus/tools/<phase>/` + `corpus/cookbook/` + `corpus/record/` (the how-to, the decision log, the accelerators, the retrospective, the story, the playbook, the effort map, the gen3 docs, the digest, every PhaseEnd — task 14.5) (`--corpus`; superseded tools as pointer files; sha1-equal to their sources). `--check` in `tools-health`; `make kit-corpus` = `--all`. `--consumers FILE` is the referrer census before any `git mv` of a tool. |
|
||||
| | `tools/share_census.py` | **(P35 T1)** The fleet-wide census of byte-identical function classes and the S1 "one source per unique function" checker (`--check`, `--selftest`, `--scope`, `--strict-macros`, `--strict-text`); its ledger is `config/dedup_exceptions.tsv`; details in the P35 T1 section below. |
|
||||
| | `tools/lever_census.py` | **(P36 T1)** The census of every compiler-forcing construct ("lever") in the fleet's C — register pins, asm statements by kind (barrier / launder / keepalive / instruction / GTE / verbatim-body), volatile levers, bare `register`, plus the deferred asm-label aliases, builtins and attributes — derived from `share_census`'s scanner with a per-token coverage assertion against the raw text, four known-true controls, the `// !FAKE:` marker split, `--check` (0 unmarked pins/asm AND 0 orphan markers — a `// !FAKE:` line with no pin/asm site on it nor on the line below) and `--check --strict` (0 pins, 0 asm outside the GTE header) gates, cross-file macro names (a use of a macro defined in another file — the prelude's `ENGINE_SHB`, a unit's `SHB` inside an included header, the GTE header's names — is classed by the majority definition's kind; it was invisible before), the JSON's `head` + stat-based `src_stamp` (delever refuses a census that does not describe the tree) and a per-file walk cache keyed on the walker's own hash (a tool change invalidates it, R35), `--sites` (every site to `.run/P36/census/lever_sites.jsonl`, the delever ledger's input), `--selftest`. Evidence: `.run/P36/census/lever_census.{json,txt}`; `progress.py` publishes the `levers` block and a README sentence from it. |
|
||||
| | `tools/lever_census.py` | **(P36 T1)** The census of every compiler-forcing construct ("lever") in the fleet's C — register pins, asm statements by kind (barrier / launder / keepalive / instruction / GTE / verbatim-body), volatile levers, bare `register`, plus the deferred asm-label aliases, builtins and attributes — derived from `share_census`'s scanner with a per-token coverage assertion against the raw text, four known-true controls, the `// !FAKE:` marker split, `--check` (0 unmarked pins/asm AND 0 orphan markers — a `// !FAKE:` line with no pin/asm site on it nor on the line below) and `--check --strict` (0 pins, 0 asm outside the GTE header) gates, cross-file macro names (a use of a macro defined in another file — the prelude's `ENGINE_SHB`, a unit's `SHB` inside an included header, the GTE header's names — is classed by the majority definition's kind; it was invisible before), the JSON's `head` + stat-based `src_stamp` (delever refuses a census that does not describe the tree) and a per-file walk cache keyed on the walker's own hash (a tool change invalidates it, R35), `--sites` (every site to `.run/P36/census/lever_sites.jsonl`, the delever ledger's input), `--selftest`. Evidence: `.run/P36/census/lever_census.{json,txt}`; `progress.py` publishes the `levers` block and a README sentence from it. **(P36 T8, S105)** `--selftest` + `--check -j 16 --quiet` are a `make tools-health` rung (after `verbatim_check --strict`, before `report`): 0 UNMARKED pins/asm and 0 orphan markers on every health run; `--strict` (0 pins, 0 asm) is the STRUCTS phase's finish line by Drew's S104 amendment, not a rung. |
|
||||
| | `tools/delever.py --recipes` | **(P36 T6, rung R)** `tools/delever.py --recipes --label L [--only …] [--limit N] [--control 8]`: the cookbook's byte-neutral SHAPE recipes tried mechanically on every RESIDUE body, seeded with that body's lever-free text — R2 the formerly-pinned declarations permuted among their own lines, R4 one of them moved to every other slot of the body's declaration run (§76/§501-R: the allocation order is the bank), R3 an initializer split into a declaration and a first assignment placed after the WHOLE declaration run (C89), deduplicated and capped at 40 candidates; the first candidate whose object is IDENTICAL replaces the body and its `// !FAKE:` markers are scrubbed within its own line span only. The control runs first (R39) on the drawn bodies: the identity permutation through the same splice code must reproduce its input text exactly (a text assertion, no compile) and the untouched file must still judge IDENTICAL. Scope, stated: only a body with a formerly-PINNED declaration has candidates. |
|
||||
| | `tools/delever_oracle.py` | **(P36 T2)** The fast byte oracle for a single-translation-unit edit: `--recipes` captures every object's exact build command through `make -n -W <src> <obj> BINARY=<alias>` (the Makefile's own pipeline — the jump-table pad stage, the per-object `-O0` overrides, the twin rule — into `.run/P36/delever/recipes.json`, regenerable in ~15 s); a candidate is compiled IN PLACE with `-o`/`-MF` redirected to scratch and its bytes compared with `build/`'s object from the fleet run (build/ is never written); `--calibrate <aliases>` proves 100 % equality untouched + twin == primary + a positive control (a nop injected at the end of a real body → DIFFERS) and writes `calibration.json` (HEAD, config stamp, per-object seconds); `--status`. Measured at T2: main 0.08–0.77 s per object, overlays ~0.14 s. |
|
||||
| | `tools/delever.py` | **(P36 T2/T3)** The de-lever engine AND the campaign tool. Rewrites per lever class on the raw text at the census's positions (a token mismatch REFUSES, never guesses): pin → plain declaration (type/qualifiers/initializer kept; the `$0` zero-register variable's uses → 0, refused if it is ever assigned), barrier / keep-alive → deleted, launder → deleted when it launders a value into itself, **an ASSIGNMENT `out = in;` when its output and input differ** (deleting those made cc1 2.7.2 abort — the T2 probe counted them NEEDED), a hand-placed instruction → its C (addu-$zero/move/la/lh/lw/addiu/sll/srl/and/andi/lui/li and the lui+addiu / lui+ori pairs), a macro-carried site → deleted for a pure launder statement macro (`SHB`), its value `((T)(p))` for a launder statement-expression, REFUSED for a compound macro (XFER/DRAW/RTP_SND: the lever is in the `#define`, T5), volatile / register → dropped. Per body: replay (a ledger exemplar with the same normalized text: 1 compile) → rung A strip-all → rung B greedy, through `delever_oracle` (verdicts IDENTICAL / DIFFERS / COMPILE-ERROR / COMPILE-CRASH). **The file is the write unit and its final compile the proof**: every body's accepted edits + the file-scope volatile edits + the `// !FAKE:` markers spliced once, compiled through every recipe of the file (a twin's object; every includer of a header, in parallel) — IDENTICAL or the file-scope edits are dropped, or the file is restored and REFUSED (COMBINATION-FAILED). `--plan` / `--apply --batch N --label L [--headers] [--only …] [-j 12]` (TUs in parallel, exemplar files before their copies; headers serial; the tree must be clean, the calibration current, the census's src stamp the tree's — it reruns the census itself), `--redraw REFUSED NOTHING-USABLE` (draw again the bodies whose latest ledger verdict is one of these — after a tool fix), `--restore` (from `inflight.json`, the only restore — R102; it also drops the in-flight batch's ledger rows into an ignored `ledger.jsonl.killed_<label>`, so a killed batch leaves no trace and its bodies are drawn again), `--status`, `--scrub [--only …]` (remove the census's ORPHAN markers — a `// !FAKE:` whose site is gone — each file judged through every recipe; a removed or rewritten site consumes its own trailing marker, so orphans arise only from older tool versions; **S104:** every `lever_census.walk_file` call outside the census now defaults to the tree's cross-file asm-macro table, `lever_census.tree_asm_macros()` (~24 s once per process) — S103's scrub walked a shared header alone, read its prelude-defined `ENGINE_SHB` launder as no site, and deleted a live marker), `--apply-body TU FN FILE --label L [--rung E]` (T6/T7: a reshaped body judged on the bytes, refused if a class A/B lever remains), `--selftest` (the fixture through the census's own parser: every rewrite, the ladder against a stub oracle with a crashing site, the markers, a replay and a disagreeing replay, the ledger index, the oracle's crash forms), `--probe` (T2). The ledger `.run/P36/delever/ledger.jsonl`: one row per (tu, fn, addr, aliases) per judgement with every site's verdict REMOVED / REWRITTEN / NEEDED(oracle word) / REFUSED(why) / DEFERRED, the rung, the calibration id, the body's nhash BEFORE and AFTER — "done" = the current nhash is some row's after-hash; a body with a row's before-hash replays it. Class C/D survivors are never marked (decision 3); class A/B survivors carry `// !FAKE: <kind> <detail> — <verdict> (P36 <rung> <label>)` at the end of their line. Evidence: `apply_<label>.log`, `batch_<label>.json`. Per-batch setup: the per-file include lists are cached in the ignored `includers_cache.json` on (mtime, size) — 29 s → 1.3 s, proven equal to the uncached map. |
|
||||
|
||||
Reference in New Issue
Block a user