mirror/BFM-decomp
1
0
Fork 0
mirror of https://github.com/Druthulu/BFM-decomp synced 2026-09-26 13:33:34 -04:00
Code Issues Packages Projects Releases Wiki Activity
Files
f6e48b60c54bdb3038f36a8b2b6d2afdece17f13
BFM-decomp/docs/progress.fleet.md
T
Drew T f6e48b60c5 perf(phase-30 S46-4): parallelise the propagation — 24min -> 11.4min, and +62 MORE instances 
Drew: "make it more multi-threaded... I still see my cpu idle for far too long."
Measured, fixed, and regression-tested against the S46-3 bank as a KNOWN ANSWER.

- THE MEASUREMENT: 31s saturated (33 makes/48 cc1/load 27) then ~25s with ONE build alive
  while 31 cores idled, repeating. Causes: ex.map starts in list order so the giants land
  last, and apply/restore is single-threaded.
- gate_all -> gate_failures: return EVERY failure the sweep already computed (~138 rounds -> 1).
- Longest-first gate scheduling; results re-sorted into `changed` order so the verdict stays
  bit-identical to the serial loop's.
- PER-OVERLAY INDEPENDENT SEARCH, IN PROCESSES. My first cut used threads and the box refuted
  it: 0-4 builds alive at load 3, because the work is regex over 15k-line files and 138
  "parallel" searches all queued on the GIL. Same logic in a ProcessPoolExecutor: 14-29 builds,
  load 34.75, search phase ~100s. Safe because the shared header is written ONCE by the parent
  and each overlay owns its own .c files + build/<bin>/. Seeded with one in-process search
  first — a pool submitted at once gives every worker an empty suspect list and makes all 138
  pay a full bisection. place_in_overlay extracted to module level so the worker and the
  in-process apply cannot drift (R33); compiles_standalone's fixed t.c is per-call now.
- THE REGRESSION (the point, not the stopwatch): revert src/+config to pre-bank, re-run the
  identical command -> 29 functions (same), 141 overlays byte-identical, 682s vs ~1440s, and
  285 exclusions vs ~350 => +62 MORE member instances (249,161). The old prefix-based
  necessity probe was OVER-EXCLUDING (charging 4 fns to 9 overlays that did not all need
  them); the per-overlay shrink minimises per overlay. The faster path is also more correct —
  a timing comparison would never have shown it. R22 213/213 + tools-health green.
- STILL SERIAL, now the actual wall-clock (neither is a build): ~3min setup before the first
  gate (registered_addrs() yaml-parsing a 1949-group/249k-instance registry + 213 sig loads)
  and ~2.5min of sequential reconcile_caller_extern after the search.
- Captured as defaults: docs/accelerators.md A8 + memory fleet-tool-parallelism-defaults.
  cookbook index regenerated (my §155c append left it stale — the gate caught it, exit 1).
2026-08-07 23:14:15 -06:00

12 KiB
Raw Blame History

BFM FLEET matching progress (generated by tools/progress.py --fleet — authoritative)

213 binaries: main + resident + 211 location overlays. DETERMINISTIC,

source-derived (committed src/*.c + config/dedup.us.yaml). Live byte gate: make check-all;

cross-binary collapsible-byte leverage: docs/duplicates.cross.md.

THREE progress metrics (all matter — see the labels):

FLEET fn-count byte-ident: 347723 / 363378 = 95.69% (REAL+LINKED+empties; FUNCTION-count, ×134-inflated — one crack counts per overlay) FLEET instr-weighted : 12685370 / 13523856 = 93.8% (shipped .text across main + resident + 211 overlays; the decomp.dev-DISPLAY number) FLEET distinct-code(uniq): 5100166 / 5851963 = 87.2% (78596/90928 unique fns; the DISTINCT-RE number) MAIN game-code weighted : 436 / 79510 = 0.5% (INCLUDED in the fleet numbers above since 2026-07-22 — roadmap §1 metrics contract; LINKED-excluding Ghidra sig dated 2026-08-05; caveat is R34: no independent second oracle for a PS-X EXE, NOT drift) (fleet EXCLUDING main, for continuity with pre-2026-07-22 readings: 12684934 / 13444346 = 94.4%)

FLEET REAL substantive : 345541 (of which dedup-shared 249159 via 1949 groups / 249233 instances) FLEET LINKED PsyQ objs : 959 FLEET NON_MATCHING : 7 (0 in any default build — G4) FLEET INCLUDE_ASM stubs : 15648 FLEET matchable : 363378

binary REAL shared LINKED byte-ident matchable byte-ident %
main 54 2 959 1055 2096 50.3%
resident 129 0 0 131 145 90.3%
md_MAIN_001 1 0 0 1 11 9.1%
md_MAIN_003 0 0 0 0 60 0.0%
md_MAIN_008 0 0 0 0 6 0.0%
md_MAIN_011 0 0 0 0 21 0.0%
md_MAIN_013 3 0 0 3 16 18.8%
md_MAIN_014 3 0 0 4 13 30.8%
md_MAIN_015 3 0 0 3 20 15.0%
md_MAIN_016 2 0 0 3 13 23.1%
md_MAIN_017 3 0 0 3 10 30.0%
md_MAIN_018 3 0 0 4 11 36.4%
md_MAIN_019 3 0 0 3 22 13.6%
md_MAIN_020 5 0 0 6 14 42.9%
md_MAIN_021 3 0 0 4 11 36.4%
md_MAIN_022 10 0 0 12 33 36.4%
md_MAIN_023 2 0 0 3 6 50.0%
md_MAIN_024 2 0 0 4 6 66.7%
md_MAIN_025 3 0 0 3 30 10.0%
md_MAIN_026 3 0 0 3 24 12.5%
md_MAIN_027 6 0 0 8 29 27.6%
md_MAIN_028 3 0 0 3 22 13.6%
md_MAIN_029 3 0 0 4 11 36.4%
md_MAIN_030 3 0 0 3 10 30.0%
md_MAIN_031 2 0 0 2 18 11.1%
md_MAIN_032 2 0 0 2 10 20.0%
md_MAIN_033 2 0 0 2 19 10.5%
md_MAIN_034 2 0 0 3 28 10.7%
md_MAIN_035 7 0 0 8 16 50.0%
md_MAIN_036 2 0 0 3 28 10.7%
md_MAIN_037 2 0 0 2 17 11.8%
md_MAIN_038 3 0 0 3 20 15.0%
md_MAIN_039 2 0 0 2 26 7.7%
md_MAIN_040 2 0 0 2 19 10.5%
md_MAIN_041 3 0 0 3 15 20.0%
md_MAIN_042 0 0 0 0 4 0.0%
md_MAIN_043 2 0 0 3 23 13.0%
md_MAIN_044 7 0 0 8 44 18.2%
md_MAIN_045 0 0 0 1 21 4.8%
md_MAIN_046 4 0 0 5 39 12.8%
md_MAIN_047 3 0 0 4 12 33.3%
md_SC02_009 0 0 0 0 40 0.0%
md_SC03_073 0 0 0 2 28 7.1%
md_SC03_074 0 0 0 2 29 6.9%
md_SC03_075 0 0 0 2 41 4.9%
md_SC03_076 0 0 0 2 66 3.0%
md_SC03_077 0 0 0 2 35 5.7%
md_SC03_078 0 0 0 2 27 7.4%
md_SC03_079 0 0 0 2 29 6.9%
md_SC03_132 0 0 0 2 28 7.1%
md_SC03_133 0 0 0 2 29 6.9%
md_SC03_134 0 0 0 2 41 4.9%
md_SC03_135 0 0 0 2 66 3.0%
md_SC03_136 0 0 0 2 35 5.7%
md_SC03_137 0 0 0 2 27 7.4%
md_SC03_138 0 0 0 4 37 10.8%
md_SC04_024 0 0 0 2 28 7.1%
md_SC04_025 0 0 0 2 29 6.9%
md_SC04_026 0 0 0 2 41 4.9%
md_SC04_027 0 0 0 2 66 3.0%
md_SC04_028 0 0 0 2 35 5.7%
md_SC04_029 0 0 0 2 27 7.4%
md_SC04_030 0 0 0 4 31 12.9%
md_SC05_023 0 0 0 2 28 7.1%
md_SC05_024 0 0 0 2 29 6.9%
md_SC05_025 0 0 0 2 41 4.9%
md_SC05_026 0 0 0 2 66 3.0%
md_SC05_027 0 0 0 2 35 5.7%
md_SC05_028 0 0 0 2 27 7.4%
md_SC05_029 0 0 0 4 31 12.9%
md_SC07_003 12 0 0 12 100 12.0%
md_SC07_004 40 0 0 40 315 12.7%
ov_MAIN_012 1788 1635 0 1867 2403 77.7%
ov_SC01_000 2389 1775 0 2389 2403 99.4%
ov_SC01_001 2396 1776 0 2398 2466 97.2%
ov_SC01_004 2393 1767 0 2394 2414 99.2%
ov_SC01_005 2425 1790 0 2425 2503 96.9%
ov_SC01_006 2425 1790 0 2425 2503 96.9%
ov_SC01_008 2393 1767 0 2395 2426 98.7%
ov_SC01_009 2420 1768 0 2421 2507 96.6%
ov_SC01_074 2395 1768 0 2397 2425 98.8%
ov_SC01_077 2561 1733 0 2563 2586 99.1%
ov_SC01_080 2424 1771 0 2424 2512 96.5%
ov_SC01_084 2448 1776 0 2453 2579 95.1%
ov_SC02_000 2520 1815 0 2520 2683 93.9%
ov_SC02_003 2520 1815 0 2520 2683 93.9%
ov_SC02_004 2389 1771 0 2389 2401 99.5%
ov_SC02_005 2539 1767 0 2549 2927 87.1%
ov_SC02_011 2626 1774 0 2637 2893 91.2%
ov_SC02_015 2399 1773 0 2399 2414 99.4%
ov_SC02_016 2482 1774 0 2485 2545 97.6%
ov_SC02_017 2550 1774 0 2558 2732 93.6%
ov_SC02_021 2404 1773 0 2404 2437 98.6%
ov_SC02_026 2502 1771 0 2508 2572 97.5%
ov_SC02_027 2558 1776 0 2567 2692 95.4%
ov_SC02_028 2563 1776 0 2573 2701 95.3%
ov_SC02_031 2493 1772 0 2498 2562 97.5%
ov_SC02_035 2487 1771 0 2490 2520 98.8%
ov_SC02_037 1852 1642 0 1934 2516 76.9%
ov_SC02_039 2394 1771 0 2394 2416 99.1%
ov_SC02_041 2475 1771 0 2478 2561 96.8%
ov_SC03_001 2669 1772 0 2686 2869 93.6%
ov_SC03_002 2541 1776 0 2556 2631 97.1%
ov_SC03_003 2398 1771 0 2399 2423 99.0%
ov_SC03_006 2553 1778 0 2562 2767 92.6%
ov_SC03_007 2462 1771 0 2466 2623 94.0%
ov_SC03_010 2414 1771 0 2414 2471 97.7%
ov_SC03_011 2415 1771 0 2421 2527 95.8%
ov_SC03_012 2388 1771 0 2389 2406 99.3%
ov_SC03_013 2420 1771 0 2420 2493 97.1%
ov_SC03_014 2573 1802 0 2573 2685 95.8%
ov_SC03_015 2572 1802 0 2572 2685 95.8%
ov_SC03_023 2400 1771 0 2401 2435 98.6%
ov_SC03_024 2516 1776 0 2523 2641 95.5%
ov_SC03_028 2462 1772 0 2466 2664 92.6%
ov_SC03_029 2499 1774 0 2509 2644 94.9%
ov_SC03_030 2418 1776 0 2420 2495 97.0%
ov_SC03_031 2446 1771 0 2449 2514 97.4%
ov_SC03_089 2512 1776 0 2519 2581 97.6%
ov_SC03_090 2527 1776 0 2535 2625 96.6%
ov_SC03_091 2541 1776 0 2549 2640 96.6%
ov_SC03_092 2505 1776 0 2517 2587 97.3%
ov_SC03_093 2515 1776 0 2519 2563 98.3%
ov_SC03_094 2504 1771 0 2510 2577 97.4%
ov_SC03_095 2431 1771 0 2434 2474 98.4%
ov_SC03_096 2428 1771 0 2431 2465 98.6%
ov_SC03_097 2525 1776 0 2532 2603 97.3%
ov_SC03_098 2509 1776 0 2512 2541 98.9%
ov_SC03_099 2485 1771 0 2488 2504 99.4%
ov_SC03_100 2520 1776 0 2524 2539 99.4%
ov_SC03_101 2483 1776 0 2487 2528 98.4%
ov_SC03_102 2469 1771 0 2472 2492 99.2%
ov_SC03_103 2493 1773 0 2496 2510 99.4%
ov_SC03_104 2541 1776 0 2548 2616 97.4%
ov_SC03_105 2448 1772 0 2455 2597 94.5%
ov_SC03_107 1797 1635 0 1880 2497 75.3%
ov_SC03_108 2399 1771 0 2399 2443 98.2%
ov_SC03_109 2407 1771 0 2409 2424 99.4%
ov_SC03_110 2440 1778 0 2440 2468 98.9%
ov_SC03_111 2471 1771 0 2474 2510 98.6%
ov_SC03_112 2436 1773 0 2438 2526 96.5%
ov_SC03_113 2434 1773 0 2437 2468 98.7%
ov_SC03_114 2389 1771 0 2391 2415 99.0%
ov_SC03_115 2430 1771 0 2432 2472 98.4%
ov_SC03_116 2399 1771 0 2402 2438 98.5%
ov_SC03_117 2467 1771 0 2473 2557 96.7%
ov_SC03_118 2542 1796 0 2543 2685 94.7%
ov_SC03_119 2539 1796 0 2540 2685 94.6%
ov_SC03_121 2402 1771 0 2405 2459 97.8%
ov_SC03_124 2576 1767 0 2596 2741 94.7%
ov_SC03_125 2527 1771 0 2539 2588 98.1%
ov_SC03_126 2399 1773 0 2399 2423 99.0%
ov_SC04_000 2473 1775 0 2482 2546 97.5%
ov_SC04_002 2503 1771 0 2507 2636 95.1%
ov_SC04_003 2438 1771 0 2442 2502 97.6%
ov_SC04_004 2469 1771 0 2471 2558 96.6%
ov_SC04_005 2499 1771 0 2505 2612 95.9%
ov_SC04_006 2429 1771 0 2431 2454 99.1%
ov_SC04_007 2493 1771 0 2497 2569 97.2%
ov_SC04_008 2399 1771 0 2399 2415 99.3%
ov_SC04_009 2426 1771 0 2429 2441 99.5%
ov_SC04_010 2401 1771 0 2402 2419 99.3%
ov_SC04_011 2515 1776 0 2521 2803 89.9%
ov_SC04_012 2396 1771 0 2397 2420 99.0%
ov_SC04_015 2532 1772 0 2543 2611 97.4%
ov_SC04_016 2400 1771 0 2402 2440 98.4%
ov_SC04_018 2672 1808 0 2672 2857 93.5%
ov_SC04_019 2672 1808 0 2672 2857 93.5%
ov_SC04_020 2513 1771 0 2525 2567 98.4%
ov_SC04_021 2399 1773 0 2399 2423 99.0%
ov_SC05_000 2405 1775 0 2408 2422 99.4%
ov_SC05_001 2477 1772 0 2482 2574 96.4%
ov_SC05_002 2429 1771 0 2432 2442 99.6%
ov_SC05_003 2412 1771 0 2413 2481 97.3%
ov_SC05_004 2422 1771 0 2424 2464 98.4%
ov_SC05_005 2437 1777 0 2438 2491 97.9%
ov_SC05_006 2421 1772 0 2421 2430 99.6%
ov_SC05_007 2430 1771 0 2435 2482 98.1%
ov_SC05_008 2441 1771 0 2443 2543 96.1%
ov_SC05_009 2408 1771 0 2412 2438 98.9%
ov_SC05_010 2448 1772 0 2452 2588 94.7%
ov_SC05_011 2395 1771 0 2396 2409 99.5%
ov_SC05_017 2648 1768 0 2659 2842 93.6%
ov_SC05_018 2545 1771 0 2558 2673 95.7%
ov_SC05_019 2399 1773 0 2399 2423 99.0%
ov_SC06_000 2528 1780 0 2531 2691 94.1%
ov_SC06_006 2422 1771 0 2423 2511 96.5%
ov_SC06_008 2487 1773 0 2493 2542 98.1%
ov_SC06_010 2459 1771 0 2464 2517 97.9%
ov_SC06_011 2414 1771 0 2418 2468 98.0%
ov_SC06_013 2404 1771 0 2405 2425 99.2%
ov_SC06_014 2415 1771 0 2417 2453 98.5%
ov_SC06_015 2409 1771 0 2409 2421 99.5%
ov_SC06_016 2451 1772 0 2453 2549 96.2%
ov_SC06_018 2499 1773 0 2506 2665 94.0%
ov_SC06_020 2445 1773 0 2446 2518 97.1%
ov_SC06_022 2480 1771 0 2488 2642 94.2%
ov_SC06_024 2500 1771 0 2506 2667 94.0%
ov_SC06_025 2451 1771 0 2456 2572 95.5%
ov_SC06_027 2390 1771 0 2391 2408 99.3%
ov_SC06_029 2476 1772 0 2488 2662 93.5%
ov_SC06_030 2412 1772 0 2412 2456 98.2%
ov_SC06_032 2486 1772 0 2493 2658 93.8%
ov_SC06_033 2487 1772 0 2494 2631 94.8%
ov_SC07_000 2431 1775 0 2433 2521 96.5%
ov_SC07_001 2403 1771 0 2405 2454 98.0%
ov_SC07_002 2459 1772 0 2463 2579 95.5%
ov_SC07_006 2323 1595 0 2403 2455 97.9%
ov_SC07_007 2336 1628 0 2420 2612 92.6%
ov_SC07_008 2384 1771 0 2384 2386 99.9%
ov_SC07_009 2396 1771 0 2398 2430 98.7%
ov_SC07_010 2320 1649 0 2403 2523 95.2%
ov_SC07_011 2338 1627 0 2418 2448 98.8%
Reference in New Issue View Git Blame Copy Permalink