mirror of
https://github.com/Druthulu/BFM-decomp
synced 2026-10-07 09:27:55 -04:00
perf(gate): pass -j to the per-binary build — 6.1x on the inner loop of every gate
MEASURED on ov_SC03_010 (35 objects), clean each time, byte-verified against the locked SHA:
make build 7.18 s real / 6.84 s user <- SERIAL, one core, on a 32-thread box
make -j16 build 1.18 s real / 11.3 s user <- 6.1x, IDENTICAL bytes
Negative control at -j32 over ov_SC03_010 + ov_SC01_004 + md_MAIN_031: all rc=0, all byte-identical
to config/check.<bin>.sha.
WHY IT WAS MISSED: the Makefile's `JOBS ?= 16` is parallelism ACROSS binaries (`xargs -P`), which
parallel_gate already uses for extract-all/check-all. Parallelism WITHIN one binary's ~35 objects was
never passed by any tool, though docs/SETUP.md:416 documents `make -j$(nproc) build` as the form.
PATCHED the two hot sites:
* harvest_verify.py — the gate's build, run ONCE PER DRAFT (--chunk 1). Every gate in the project.
* dedup_propagate.py byte_gate — run once per propagation candidate, which is why a wide
propagation dominated a 33-minute gate this session.
Both honour BFM_BUILD_JOBS, else os.cpu_count().
SAFE BY CONSTRUCTION: these builds feed a locked-SHA comparison, so a bad parallel build FAILS the
gate rather than banking wrong bytes. The error direction is a false NEGATIVE, never a false bank;
G3/P9 remains the sole arbiter.
Correction recorded: I earlier extrapolated "9 serial binaries x 30 min" from ONE 33-minute
measurement. That was unfounded — propagation time scales with how many sites a body reaches, and
other gates today propagated x19/x8/x7 quickly. One slow binary is not a rate (R41).
This commit is contained in:
@@ -341,9 +341,14 @@ def append_groups(groups):
|
||||
return len(blocks)
|
||||
|
||||
|
||||
_BJOBS = int(os.environ.get('BFM_BUILD_JOBS') or (os.cpu_count() or 8))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- byte gate
|
||||
def byte_gate(ov):
|
||||
r = subprocess.run(["make", "build", f"BINARY={ov}"], cwd=ROOT,
|
||||
# -j — see harvest_verify's note: ~6x on a per-binary build, and this gate runs once per
|
||||
# propagation candidate, which is why a wide propagation dominated a 33-minute gate in P31 S67.
|
||||
r = subprocess.run(["make", "-j%d" % _BJOBS, "build", f"BINARY={ov}"], cwd=ROOT,
|
||||
capture_output=True, text=True)
|
||||
return r.returncode == 0, r.stdout + r.stderr
|
||||
|
||||
|
||||
+13
-1
@@ -72,6 +72,9 @@ if not a.good_sha:
|
||||
# mismatch. Strip exactly the names the TARGET TU provides, per-TU (cdecl.typedef_names + the T4
|
||||
# primitive). The old `_TD` here was scalar-only (no M2C_UNK, no struct typedefs) and dropped 39
|
||||
# still-open functions this way; the strip-set is now derived from the tree, not hand-listed.
|
||||
_JOBS = int(os.environ.get('BFM_BUILD_JOBS') or (os.cpu_count() or 8))
|
||||
|
||||
|
||||
def sha1(path):
|
||||
return hashlib.sha1(open(path, 'rb').read()).hexdigest() if os.path.exists(path) else None
|
||||
|
||||
@@ -101,7 +104,16 @@ def build():
|
||||
global _last_err, _last_sha
|
||||
if os.path.exists(a.out):
|
||||
os.remove(a.out)
|
||||
p = subprocess.run(['make', 'build', 'BINARY=' + a.binary], capture_output=True, text=True)
|
||||
# -j: THE INNER LOOP OF EVERY GATE. A binary is ~35 objects and the build was SERIAL — measured
|
||||
# 7.18 s serial vs 1.18 s at -j16 on ov_SC03_010, byte-identical and equal to the locked SHA
|
||||
# (6.1x). `JOBS` in the Makefile is parallelism ACROSS binaries (xargs -P); this is parallelism
|
||||
# WITHIN one binary and was simply never passed. docs/SETUP.md:416 already documents
|
||||
# `make -j$(nproc) build` as the sanctioned form.
|
||||
# SAFE BY CONSTRUCTION: this build feeds a locked-SHA comparison, so a bad parallel build FAILS
|
||||
# the gate rather than banking wrong bytes — the error direction is a false NEGATIVE, never a
|
||||
# false bank (G3/P9 remains the sole arbiter).
|
||||
p = subprocess.run(['make', '-j%d' % _JOBS, 'build', 'BINARY=' + a.binary],
|
||||
capture_output=True, text=True)
|
||||
_last_err = (p.stderr or '') + (p.stdout or '')
|
||||
_last_sha = sha1(a.out)
|
||||
return _last_sha
|
||||
|
||||
Reference in New Issue
Block a user