From 675b4702b32a4d2234b32e9e882b332fc7a7feea Mon Sep 17 00:00:00 2001
From: Drew T <50529377+Druthulu@users.noreply.github.com>
Date: Mon, 31 Aug 2026 14:18:05 -0600
Subject: [PATCH] =?UTF-8?q?perf(gate):=20pass=20-j=20to=20the=20per-binary?=
 =?UTF-8?q?=20build=20=E2=80=94=206.1x=20on=20the=20inner=20loop=20of=20ev?=
 =?UTF-8?q?ery=20gate?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

MEASURED on ov_SC03_010 (35 objects), clean each time, byte-verified against the locked SHA:
    make build            7.18 s real / 6.84 s user   <- SERIAL, one core, on a 32-thread box
    make -j16 build       1.18 s real / 11.3 s user   <- 6.1x, IDENTICAL bytes
Negative control at -j32 over ov_SC03_010 + ov_SC01_004 + md_MAIN_031: all rc=0, all byte-identical
to config/check.<bin>.sha.

WHY IT WAS MISSED: the Makefile's `JOBS ?= 16` is parallelism ACROSS binaries (`xargs -P`), which
parallel_gate already uses for extract-all/check-all. Parallelism WITHIN one binary's ~35 objects was
never passed by any tool, though docs/SETUP.md:416 documents `make -j$(nproc) build` as the form.

PATCHED the two hot sites:
* harvest_verify.py — the gate's build, run ONCE PER DRAFT (--chunk 1). Every gate in the project.
* dedup_propagate.py byte_gate — run once per propagation candidate, which is why a wide
  propagation dominated a 33-minute gate this session.
Both honour BFM_BUILD_JOBS, else os.cpu_count().

SAFE BY CONSTRUCTION: these builds feed a locked-SHA comparison, so a bad parallel build FAILS the
gate rather than banking wrong bytes. The error direction is a false NEGATIVE, never a false bank;
G3/P9 remains the sole arbiter.

Correction recorded: I earlier extrapolated "9 serial binaries x 30 min" from ONE 33-minute
measurement. That was unfounded — propagation time scales with how many sites a body reaches, and
other gates today propagated x19/x8/x7 quickly. One slow binary is not a rate (R41).
---
 tools/dedup_propagate.py |  7 ++++++-
 tools/harvest_verify.py  | 14 +++++++++++++-
 2 files changed, 19 insertions(+), 2 deletions(-)

diff --git a/tools/dedup_propagate.py b/tools/dedup_propagate.py
index 01656446a7..06a3fbdbda 100644
--- a/tools/dedup_propagate.py
+++ b/tools/dedup_propagate.py
@@ -341,9 +341,14 @@ def append_groups(groups):
     return len(blocks)
 
 
+_BJOBS = int(os.environ.get('BFM_BUILD_JOBS') or (os.cpu_count() or 8))
+
+
 # ---------------------------------------------------------------- byte gate
 def byte_gate(ov):
-    r = subprocess.run(["make", "build", f"BINARY={ov}"], cwd=ROOT,
+    # -j — see harvest_verify's note: ~6x on a per-binary build, and this gate runs once per
+    # propagation candidate, which is why a wide propagation dominated a 33-minute gate in P31 S67.
+    r = subprocess.run(["make", "-j%d" % _BJOBS, "build", f"BINARY={ov}"], cwd=ROOT,
                        capture_output=True, text=True)
     return r.returncode == 0, r.stdout + r.stderr
 
diff --git a/tools/harvest_verify.py b/tools/harvest_verify.py
index 996a982eac..e805438f00 100644
--- a/tools/harvest_verify.py
+++ b/tools/harvest_verify.py
@@ -72,6 +72,9 @@ if not a.good_sha:
 # mismatch. Strip exactly the names the TARGET TU provides, per-TU (cdecl.typedef_names + the T4
 # primitive). The old `_TD` here was scalar-only (no M2C_UNK, no struct typedefs) and dropped 39
 # still-open functions this way; the strip-set is now derived from the tree, not hand-listed.
+_JOBS = int(os.environ.get('BFM_BUILD_JOBS') or (os.cpu_count() or 8))
+
+
 def sha1(path):
     return hashlib.sha1(open(path, 'rb').read()).hexdigest() if os.path.exists(path) else None
 
@@ -101,7 +104,16 @@ def build():
     global _last_err, _last_sha
     if os.path.exists(a.out):
         os.remove(a.out)
-    p = subprocess.run(['make', 'build', 'BINARY=' + a.binary], capture_output=True, text=True)
+    # -j: THE INNER LOOP OF EVERY GATE. A binary is ~35 objects and the build was SERIAL — measured
+    # 7.18 s serial vs 1.18 s at -j16 on ov_SC03_010, byte-identical and equal to the locked SHA
+    # (6.1x). `JOBS` in the Makefile is parallelism ACROSS binaries (xargs -P); this is parallelism
+    # WITHIN one binary and was simply never passed. docs/SETUP.md:416 already documents
+    # `make -j$(nproc) build` as the sanctioned form.
+    # SAFE BY CONSTRUCTION: this build feeds a locked-SHA comparison, so a bad parallel build FAILS
+    # the gate rather than banking wrong bytes — the error direction is a false NEGATIVE, never a
+    # false bank (G3/P9 remains the sole arbiter).
+    p = subprocess.run(['make', '-j%d' % _JOBS, 'build', 'BINARY=' + a.binary],
+                       capture_output=True, text=True)
     _last_err = (p.stderr or '') + (p.stdout or '')
     _last_sha = sha1(a.out)
     return _last_sha