Drew: "make it more multi-threaded... I still see my cpu idle for far too long."
Measured, fixed, and regression-tested against the S46-3 bank as a KNOWN ANSWER.
- THE MEASUREMENT: 31s saturated (33 makes/48 cc1/load 27) then ~25s with ONE build alive
while 31 cores idled, repeating. Causes: ex.map starts in list order so the giants land
last, and apply/restore is single-threaded.
- gate_all -> gate_failures: return EVERY failure the sweep already computed (~138 rounds -> 1).
- Longest-first gate scheduling; results re-sorted into `changed` order so the verdict stays
bit-identical to the serial loop's.
- PER-OVERLAY INDEPENDENT SEARCH, IN PROCESSES. My first cut used threads and the box refuted
it: 0-4 builds alive at load 3, because the work is regex over 15k-line files and 138
"parallel" searches all queued on the GIL. Same logic in a ProcessPoolExecutor: 14-29 builds,
load 34.75, search phase ~100s. Safe because the shared header is written ONCE by the parent
and each overlay owns its own .c files + build/<bin>/. Seeded with one in-process search
first — a pool submitted at once gives every worker an empty suspect list and makes all 138
pay a full bisection. place_in_overlay extracted to module level so the worker and the
in-process apply cannot drift (R33); compiles_standalone's fixed t.c is per-call now.
- THE REGRESSION (the point, not the stopwatch): revert src/+config to pre-bank, re-run the
identical command -> 29 functions (same), 141 overlays byte-identical, 682s vs ~1440s, and
285 exclusions vs ~350 => +62 MORE member instances (249,161). The old prefix-based
necessity probe was OVER-EXCLUDING (charging 4 fns to 9 overlays that did not all need
them); the per-overlay shrink minimises per overlay. The faster path is also more correct —
a timing comparison would never have shown it. R22 213/213 + tools-health green.
- STILL SERIAL, now the actual wall-clock (neither is a build): ~3min setup before the first
gate (registered_addrs() yaml-parsing a 1949-group/249k-instance registry + 213 sig loads)
and ~2.5min of sequential reconcile_caller_extern after the search.
- Captured as defaults: docs/accelerators.md A8 + memory fleet-tool-parallelism-defaults.
cookbook index regenerated (my §155c append left it stale — the gate caught it, exit 1).
The S45p9 blocker is closed, and the recovery loop that kept it from finishing is rewritten.
- BANKED: dedup_propagate --auto-from ov_SC02_037 --recover -> 29 functions propagated,
141 overlays byte-identical, dedup 1920 -> 1949 groups, member instances 246,284 ->
249,099 (+2,815). make clean && extract-all && check-all -> 213 passed / 0 failed (R22).
- WHY IT FINISHED THIS TIME: gate_all -> gate_failures returns EVERY failure from the sweep
that already computed them, and the recovery loop resolves them all per round. Converged in
3 rounds; the old one-overlay-per-sweep design needed ~138. That reframes the S45 run — it
was not nearly done when it died, it had barely started.
- Batching did NOT cost capability: per-overlay necessity probes excluded four of the nine
culprits from only the 9 overlays that needed it (not all 138), and ov_SC07_006 was
RECOVERED by the Part-B caller-extern reconcile instead of excluded.
- Plan phase parallelised: 5 min -> 26 s, plan + skip classification byte-identical. Its
compiles_standalone temp file is per-call now — the fixed `t.c` was the same fake-isolation
class as match_one's shared --work dir (P28 T5), latent until something ran it in parallel.
- docs/accelerators.md (NEW, Drew 2026-08-07): the reusable-workflow ledger — what we learned
late that a future decomp should know on day one, each entry with when we found it, when it
WAS findable, what it cost, and the honest prerequisite where one exists.
Drew T