From b1d6c8f78ed71c179a6fb0fbf5a1c548ef45276d Mon Sep 17 00:00:00 2001 From: Drew T <50529377+Druthulu@users.noreply.github.com> Date: Wed, 9 Sep 2026 07:18:13 -0600 Subject: [PATCH] =?UTF-8?q?phase-36:=20T6=20opened=20=E2=80=94=20the=20res?= =?UTF-8?q?idue=20measured=20from=20the=20ledger=20(12,970=20bodies=20/=20?= =?UTF-8?q?1,804=20classes;=20median=201=20needed=20site),=20the=20permute?= =?UTF-8?q?r=20harness=20read,=20the=20two=20feasibility=20probes=20passed?= =?UTF-8?q?=20(a=20target=20listing=20regenerated=20from=20the=20ROM=20ima?= =?UTF-8?q?ge;=20a=20one-function=20lever-free=20base=20through=20make=5Fb?= =?UTF-8?q?ase=5Fc=20and=20compile.sh=20=E2=86=92=20an=20object=20holding?= =?UTF-8?q?=20only=20the=20exemplar);=20the=20rung-D=20driver's=20design?= =?UTF-8?q?=20settled=20in=20the=20=F0=9F=9B=91=20block=20=C2=A72=20for=20?= =?UTF-8?q?the=20next=20session;=20S98=20closes=20at=2090=20%=20context?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- phase-ends/CURRENT_PHASE.md | 79 +++++++++++++++++++++++++++++-------- 1 file changed, 62 insertions(+), 17 deletions(-) diff --git a/phase-ends/CURRENT_PHASE.md b/phase-ends/CURRENT_PHASE.md index d5c961bc3..52832d8f2 100644 --- a/phase-ends/CURRENT_PHASE.md +++ b/phase-ends/CURRENT_PHASE.md @@ -514,7 +514,32 @@ accumulate here as the phase produces them.** cross-file names; the sweep's per-definition uses. Commits: `0bd784c64` (header), `145ab7719`/`666d4b5ae`/`02810debd` (batches), `5b6e1a2fe` (repair), `3b2c5178b`, `4bc987426` (sweep), this close. **T5 β˜‘.** -## πŸ›‘ SESSION CHECKPOINT β€” S98 (2026-09-09): T0 β˜‘ T1 β˜‘ T1b β˜‘ T2 β˜‘ T3 β˜‘ T4 β˜‘ T5 β˜‘ β€” all committed (this close on top of `4bc987426`); NEXT = T6 rung R (mechanical shape recipes) + rung D (the permuter on the residue exemplars) | the number: 34,091 sites in 12,712 bodies (1,759 distinct) Β· marked 34,091 Β· UNMARKED 0 Β· orphans 0 Β· GTE levers 462 Β· per-TU asm macro definitions 314 (0 canonical duplicates) β€” `lever_census --check` OK +- **S98 β€” T6 opened: reconnaissance and the two feasibility probes (the design is in the πŸ›‘ block Β§2; the build is the next session's).** + The residue from the ledger (latest row per body): **12,970 bodies in 1,804 distinct after-hash classes** β€” 1,450 singletons, 80 classes + of 134 copies (the overlay-wide functions), 75 pairs, 60 triples; the median exemplar needs ONE site (p90 4, max 25); 904 exemplars need + only pins, 48 only `$0` pins; needed sites Γ— copies: pin 20,643 Β· launder 4,635 Β· barrier 4,576 Β· keep-alive 2,000 Β· instruction 1,667 Β· + cast 1,509. The cookbook on the classes: the `$0` pin is the "$0-ADD OPAQUE COPY" (Β§34/Β§17 RC-12: `y = x + zr` emits a `move` CSE + cannot see through β€” no plain-C spelling; T7's); Β§137 says a clean register swap is a TWO-COMPILE ARITHMETIC PROBLEM (read R and L from + `cc1 -dl -dg`, `tools/alloc_table.py`), not a permuter job; Β§76 the allocno class through declaration scope. The permuter harness as + built: `tools/permuter/run_masked.py` (the masked scorer rebound over decomp-permuter's Scorer: target.o and cand.o each hold ONE + function, whole-.text compared), `tools/p16_permute.py` (`make_base_c` = comments β†’ cpp WITHOUT includes β†’ M2C_FIELD β†’ `hide_asm` (asm + statements and pins carried as decomp-permuter's own `#pragma _permuter b64literal`, decoded by its serializer, so cc1 sees the real + asm) β†’ typedefs; `setup(fn, draft, asm_subdir, klass)` builds base.c + target.o from `/.s` + settings.toml; + `run_permuter(pd, secs, j)`; `winner_to_draft`), `tools/permuter_ils.py` (warm restarts from the best waypoint), `tools/permuter_weights.py` + (profiles regalloc / schedule / cse / length via `render_settings_toml(fn, klass=…)`), `tools/permuter/compile.sh` (the faithful + cppβ†’cc1β†’maspsxβ†’as tail, no pad stage β€” .text only, fine). Per-function `.s` targets do NOT exist on disk for matched functions; + `tools/verbatim_target_s.py --binary --fn ` regenerates one from the extracted ROM image (`.run/verbatim_targets//.s`, + splat format β€” the independent oracle). **Probe 1:** `verbatim_target_s.py --binary ov_SC04_011 --fn func_80135D20` β†’ `100 ins @ + 0x80135D20 -> .run/verbatim_targets/ov_SC04_011/func_80135D20.s`. **Probe 2 (the base):** the TU text with the exemplar's body lever-free + (delever's rung-A edits for that body: 1 site, the `$17` pin), every OTHER definition replaced by its prototype (`sc.scan_text` defs β†’ + the signature up to `{` + `;`), every `#include "../shared/…/func_*.h"` replaced by the prototypes of the definitions that header holds, + `INCLUDE_ASM`/`INCLUDE_RODATA` lines dropped, then `mipsel-linux-gnu-cpp -P -lang-c -Iinclude -I -undef -Dmips -D__GNUC__=2 + -D__OPTIMIZE__ -Dpsx -D_PSYQ -D_MIPSEL -D_LANGUAGE_C` (the real include paths β€” common.h, the prelude, engine_types.h, the GTE header + expand; make_base_c's own cpp runs without includes and would lose them) β†’ `p16_permute.make_base_c(expanded)` β†’ `base.c` 5,218 lines, + `defines_fn` True, 0 other bodies β†’ `tools/permuter/compile.sh base.c -o base.o` rc 0 β†’ `nm`: `T func_80135D20` alone (the callees `U`). + The pipeline is proven; scratch under `.run/P36/permuter/probe/` (ignored). + +## πŸ›‘ SESSION CHECKPOINT β€” S98 (2026-09-09): T0 β˜‘ T1 β˜‘ T1b β˜‘ T2 β˜‘ T3 β˜‘ T4 β˜‘ T5 β˜‘ β€” all committed (this close on top of `4bc987426`); NEXT = T6 BUILD: `tools/delever_permute.py` per Β§2 (the design is settled and its pipeline proven by two probes β€” the T6 log entry); S98 ended here at 90 % context | the number: 34,091 sites in 12,712 bodies (1,759 distinct) Β· marked 34,091 Β· UNMARKED 0 Β· orphans 0 Β· GTE levers 462 Β· per-TU asm macro definitions 314 (0 canonical duplicates) β€” `lever_census --check` OK ### 0. How to use this block A fresh session reads CLAUDE.md's load order, replays this block verbatim, confirms the effort (T6/T8/T9 xHigh per the plan; S98 ran at Max β€” @@ -544,22 +569,42 @@ batch: `tools/delever.py --restore` (drops its ledger rows). The calibration is - **Environment:** WSL2, `~/bfm-decomp`, `.venv`; R22 clean fleet run β‰ˆ 115 s; the census β‰ˆ 35 s (a full rewalk when a tool or a table changes); `make report BINARY=main` β‰ˆ 285 s; `make kit-corpus` β‰ˆ 25 s; 16 cores; 31 GB RAM; ~30 GB disk free. -### 2. T6 β€” rung R (mechanical shape recipes) + rung D (the permuter on the residue exemplars) (xHigh; design first, X1) -The plan's two yield lines: `recipes: of NEEDED sites removed` and `permuter: of exemplars matched lever-free in h`. -(a) **Rung R** = byte-neutral shape recipes applied mechanically per NEEDED site, each negative-controlled on a body it must not change: -the cookbook's Β§501-P (a fresh single-set temp at the use), Β§501-R (the inline-expression-thrice / `u16` accumulator), Β§76 (declaration-scope -moves), T4 (branch-polarity flip), Β§17a (the explicit temp before a call), the `zr` β†’ `0` copy-arm rewrite for the `$0` class (1,318 needed -`$0` pins), and for the 462 GTE levers "drop the clobber" (already tried per definition in T5 β€” the 70 kept ARE needed there) β€” read -`docs/matching-cookbook.md` by Β§ (grep `^## Β§501`) and `docs/cookbook-index.md` by symptom; measure each recipe on a stratified sample of -its class before the sweep (R37). Implement as `tools/delever.py --recipes` (or a `tools/delever_recipes.py` that delever imports): -per body, per NEEDED site, the recipe's rewrite β†’ the same oracle β†’ REMOVED/NEEDED; the ledger row's site verdict gains `rule`. -(b) **Rung D** = `tools/permuter/run_masked.py` + `permuter_ils.py` on the residue EXEMPLARS (one per distinct after-hash class, β‰ˆ1,759; -largest copy count first), seeded with the lever-free body (every A/B site stripped β€” the target bytes from `build/`'s object of the fleet -run), `permuter_weights.py`'s class profile, a time budget per body, 16 workers, DETACHED; a score-0 candidate β†’ `tools/delever.py ---apply-body TU FN FILE --label d --rung D` (refused if a class A/B lever remains) β†’ the copies replay through the ledger's exemplar -rows β†’ the cycle's R22 + commit. Read `docs/SETUP.md` Β§permuter and cookbook Β§3/Β§5a/Β§12 for the harness (`.run/permuter//`, `base.c` -self-contained, `target.o` from the object, `settings.toml`); `tools/verbatim_target_s.py` for the target listing. Report both yield lines -with denominators (R41), then the T6 log entry + checkpoint. +### 2. T6 β€” rung R (mechanical shape recipes) + rung D (the permuter on the residue exemplars) (xHigh) β€” DESIGN SETTLED, BUILD NEXT +The two yield lines to report (R41): `recipes: of NEEDED sites removed` Β· `permuter: of exemplars matched lever-free in h`. +**Rung D first (the bigger lever; its pipeline is proven by the two probes in the T6 log entry).** Build `tools/delever_permute.py`: +1. `--plan`: the exemplars from the ledger (`dl.load_ledger()`, latest row per (tu, fn), verdict RESIDUE, grouped by `nhash_after`, one + exemplar per class = the first (tu, fn); order: copies desc, then needed sites asc; skip classes in `.run/P36/permuter/outcomes.jsonl`); + `--only`, `--limit`. +2. Per exemplar `prepare(tu, fn)` EXACTLY as probe 2: (a) the lever-free TU text = `dl.site_edits` for every REMOVABLE site of the body + (a site that REFUSES β†’ the exemplar is `UNSTRIPPABLE`, skipped and recorded); a `gte-lever` site (kind not in `dl.REMOVABLE` yet) should + get a rewrite too: `gte_consolidate.set_clobbers(statement, canonical clobbers)` β€” add ("B", "gte-lever") to delever's rewrite table + (the canonical from `.run/P36/gte/canonical.json` by the statement's signature) so rung A/B in delever can judge direct GTE levers as + well (T5 tried only the definitions); (b) every other definition β†’ prototype; (c) every `#include "…/shared/…/func_*.h"` β†’ the + prototypes of that header's definitions (a static function's prototype keeps `static`); (d) `INCLUDE_ASM`/`INCLUDE_RODATA` lines and + file-scope `__asm__` blocks dropped; (e) cpp with `-Iinclude -I` and the build's -D flags, `-P`; (f) write the expanded text + as the DRAFT `.run/P36/permuter/__/draft.c`; (g) the target: `tools/verbatim_target_s.py --binary --fn ` β†’ + copy `.run/verbatim_targets//.s` into the scratch dir (p16's `setup(fn, draft_c, asm_subdir=)` reads + `/.s`); the alias = the body's first alias (`aliases[0]`; a shared header's body β†’ its first includer's alias); + (h) klass from the needed-site mix for `permuter_weights`: pins only β†’ "REGALLOC-PERM"; barriers/launders/keep-alives β†’ "SCHEDULE-REORDER"; + mixed β†’ "REGALLOC-PERM"; (i) `permuter_ils.py --draft draft.c --asm-subdir --klass --cycles 3 --secs 240 --j ` + (its `setup` runs `make_base_c` on the draft β€” already expanded, so its include-less cpp changes nothing β€” and asserts `defines_fn`; + the winner lands in `.run/permuter-winners/.c` β€” KEY BY alias+fn (R48): copy it to the scratch dir at once). +3. A winner β†’ `p16_permute.winner_to_draft` β†’ extract the function's definition span β†’ write `/body.c` β†’ `tools/delever.py + --apply-body /body.c --label d --rung D --dirty-ok` (refused if a class A/B lever remains; the copies replay + through the ledger when their class is drawn again: `delever --apply --redraw RESIDUE --only …` or the exemplar's copies by nhash) β†’ + then `tools/gte_consolidate.py --apply --only --rejudge --label dg` re-folds the cpp-expanded GTE asm in that function into + canonical calls (the winner is macro-expanded: readability regained by the re-fold; `#define`d constants stay expanded β€” the names + phase) β†’ the batch's R22 β†’ commit (the cycle's FINISH mode or by hand with the log line + headline, R101). +4. The campaign: DETACHED (`setsid nohup … &` + a `Monitor`), K exemplars concurrently with `-j 16/K` each, `--secs 240 --cycles 3` per + exemplar as the first measurement on ~16 exemplars (the 134-copy classes first: one match banks 134 bodies), then price the rest + from the measured yield (the P29 sweep's honest prior: ~3 in 8 for small residuals). `outcomes.jsonl` = exemplar, class copies, needed + sites, klass, secs, cycles, best score, winner path, banked commit. +**Rung R second (cheap, measured on samples, R37):** in `tools/delever.py --recipes --only …`: per RESIDUE body, candidates from +(R2) the declaration-ORDER permutations of the pinned callee-saved variables (Β§76/Β§501-R: the allocation order is the bank; ≀ 4 pins β†’ +all 24 orders, else 24 random) applied to the lever-free text; (R3) a pin with an initializer split into declaration + assignment, or the +declaration moved next to its first use (Β§17a); each candidate judged by the oracle; the first IDENTICAL replaces the body (its ledger row +updated, its markers consumed); negative-control each recipe on a LEVER-FREE body first (must stay identical, R39). Report `recipes: k of r`. +Then the T6 log entry (both lines with denominators), `make report BINARY=main`, the checkpoint refresh, commit. ### 3. Numbers to re-derive, never trust The T2 probe's rates β†’ T4's measured rates β†’ T5 changed the population (the number went UP by the 462 GTE levers and 4 surfaced