Files
BFM-decomp/tools
Drew T 26ba449684 perf(tools-health): parallelise the sig targets; fix a latent probe-file race; MEASURE the real cost
Drew asked why `make tools-health` runs 15+ min. Measured per step rather than
guessed (I guessed wrong twice first, and both are recorded in the comments):

  sig-overlays  ~52s serial  -> 3.9s wall / 51.8s user  (xargs -P$(JOBS), 32 cores)
  sig-modules   0s   sig-resident 0s   audit-corpus 17s
  audit-cdecl   >9 MINUTES  <-- the actual bottleneck, and NOT the gcc probes:
                the `[gcc] N distinct declarations` line never printed inside a
                10-minute run, so not one cc1 call had happened. `tu_statements`
                over 4,168 TUs is ~787s single-core, all of it before the probes.

SHIPPED
  * sig-overlays / sig-modules: xargs -P$(JOBS), same pattern extract-all and
    check-all already use in this file. sig_image has exactly one write path
    (its own per-alias .jsonl), verified before fanning out. NEGATIVE CONTROL:
    141/141 sig files BYTE-IDENTICAL to the serial output. Also adds the failure
    detection the serial loops never had -- a sig_image crash used to vanish (R32).
  * cdecl._gcc_probe: `probe_{tag}.c` was ONE FIXED FILENAME PER TAG, correct only
    while _sift is serial. Now unique per call, so concurrent probes cannot
    overwrite each other's source between write and compile and return a verdict
    about another chunk's declarations.
  * cdecl._sift: threads over chunks + over the bisection probes (gcc is a
    subprocess, so the GIL is released), results written back BY INDEX so the
    output stays deterministic. A/B on --limit 6: IDENTICAL verdicts (829/829).

NOT SHIPPED, and the measurement is left in the code
  A ProcessPoolExecutor over the collection phase was tried and REVERTED: 12 TUs
  yield 32,352 statements, so the full pass ships ~11M strings through IPC and the
  pickling costs more than the parse it saves. The fix is to dedupe/filter INSIDE
  the worker or memoise per-TU by content hash -- left measured, not guessed.
2026-09-01 20:10:51 -06:00
..