mirror of
https://github.com/Druthulu/BFM-decomp
synced 2026-09-30 07:31:59 -04:00
afd1aeea4f10ff25100b68a2dd16694fc447cee5
340 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Drew T
|
afd1aeea4f |
feat(phase-29 T6): broad --fix-def-sig harvest +19 (def-sig lever tapped beyond the mega-pools)
- fleet-wide --band substantial (17) + tiny (2) with --fix-def-sig (all families) = 19 more members; the def-sig conflict was concentrated in the 2 tiny-IMM mega-pools (already banked, now not-stub) - R22 clean-fleet 140/140 byte-identical; pure-reduction; dedup 1840/0; 0 NON_MATCHING (G4) - fleet instr 71.0% (steady) / distinct 53.2->53.3% / fn-count 86.30%. Task-6 mega-pool track complete; permuter backlog (grinder/permuter_ils close-1..4) remains as the other Task-6 half |
||
|
Drew T
|
941cd37b19 |
feat(phase-29 T6): tiny-IMM mega-pools CRACKED +4,801 via family_sweep --fix-def-sig (fleet 70.4->71.0% instr)
- THE FIX (new): family_sweep --fix-def-sig (header_sig_map + reconcile_def_sig, 1005 mapped fns) — rewrites each member draft's DEF signature to the shared-header (engine_core.h) canonical decl. Root cause (byte-proven, R14/R35 after 3 masked-metric mis-reads): engine_core.h forward-declares the member (extern void func_8015FAAC(s32 *a0), a shared fn calls it) while family_remap copies the EXEMPLAR sig (void *a0) -> 'conflicting types' -> member TU never compiles. Invisible to standalone diff_regions/match_one (no header conflict) AND to --reconcile. Byte-neutral (ptr-type param, gate arbitrates G3/P9); one member hand-verified byte-identical first. - 0x80131eec 2331/2470 (94%) + 0x80130d0c 2470/2496 (99%) = 4,801 members banked across 405 overlay files. R22 clean-fleet 140/140 byte-identical; pure-reduction (0 new/dup stubs); dedup 1840/0; 0 NON_MATCHING (G4). Fleet instr 70.4->71.0% / distinct 52.3->53.2% / fn-count 84.94->86.30%. - CORRECTS the commit:0665 'symbol-definition gap' scout (WRONG). The 4th 'reproduce the build step' instance (§53-carve, -O0-flag, now the member's canonical DECLARATION). cookbook §54, decision-log R31. |
||
|
Drew T
|
53ee974947 |
docs(phase-29 T6-scout): tiny-IMM mega-pools = a symbol-definition gap, not a wall (~4,966 members)
- 0x80131eec (2887) + 0x80130d0c (2679): family_sweep --hseq staged 4966, banked 1/4966 (0.0%) - diff_regions: TEMPLATES / O2:MATCH(0) — remapped C is byte-correct at -O2, jump-table symbol IS remapped (D_8018708C[idx]() -> D_801815EC[idx]()), but D_801815EC is splat-LOCAL and NOT a defined linker symbol (absent from config/symbols*) -> named C ref can't resolve -> whole-binary DIFF; masked_diff hid it (masks %hi/%lo) — R35 (a masked-MATCH contradicting the gate = the tool hides the divergence) - FIX (deferred, focused sub-project): define/export per-overlay jump-table data symbols so remapped C links -> banks ~4,966 members mechanically (the biggest cheap lever left). Same 'reproduce the build step' class as §53-carve / -O0-flag; here the missing step is symbol DEFINITION - no banks kept (tree reverted clean); characterization only. CURRENT_PHASE per-task log updated |
||
|
Drew T
|
f6f89781ff |
feat(phase-29 T2 Arm A): swing verdict = BANKED FACT (9/9 -O0 members on ov_SC07_010); fleet -O0 rollout deferred at the splat wall
- tools/rollout_o0_cluster.py (new) + Makefile O0_CLUSTER_OBJS -O0 wildcard: the -O0-cluster carve (0x13410..0x14834), adapting rollout_whale_o0.py to a 3-way <ov>/<ov>_o0/<ov>_o2b split - ov_SC07_010: carve byte-neutral -> family_sweep --hseq banked 9/9 -O0 exemplar-family members whole-binary (R22 clean-fleet 140/140). The Task-1 masked-MATCH swing verdict is now a BANKED FACT: -O0 cluster members DO bank at -O0 (§52b). Phase-20 'func_8013B7AC overlay-local' refuted. - THE WALL (byte-proven, TOOLING not compiler): the same carve on 006/007/011 byte-shifts the whole image (+0x20 %lo data-symbol shift, 34% diff) from a CLEAN build; boundaries verified as real fn-starts. Root cause = splat re-disassembly of a 3-way-split subseg that still holds INCLUDE_ASM stubs (the whale's stub-free _o0b shape avoids it). The Phase-20 '-O0 split infra' wall, root-caused. - DEFERRED (ROI): full -O0 fleet rollout (~1,233 / ~0.6pp) — 3/4 sampled walled + 134 jr-embedded + bigger levers (Task 3 core-cracks, Task 6 tiny-IMM ~5,566). decision-log R31 + cookbook §18-P29. - 140/140 byte-identical; dedup 1840/0; 0 NON_MATCHING (G4); main 143dbb89. Task 2 substantively done. |
||
|
Drew T
|
908dded511 |
feat(phase-29 T2a): Arm B tail — 748 members banked in the 4 SC07 tail overlays (fleet 70.2->70.4% instr)
- resumed the SIGTERM-interrupted comprehensive --band all sweep as 3 band-bounded family_sweep --hseq --allow-pins passes (substantial 53 + mid 372 + tiny 323 = 748), each exit 0 (the --band all SIGTERM lesson: band-bounded + committed-per-batch) - all 748 in the 4 files=1 SC07 tail overlays (ov_SC07_010 897->516, ov_SC07_011 797->436, 006 457->454, 007 595->592); verified pure-reduction (0 new/dup stubs, R14/H5) - 006/007 residual = jr-families (§53 carve, Task 3) + plumbing; plain-sweep tail drained - R22 clean-fleet 140/140 byte-identical; audit-binaries OK; dedup 1840/0; main 143dbb89; 0 NON_MATCHING (G4). fleet instr 70.2->70.4% / distinct 51.9->52.3% / fn-count 84.73->84.94% |
||
|
Drew T
|
5b6a8ae6cb |
feat(phase-29 T2a): Arm B type-lift sweep — 3,407 member-matches banked (fleet 68.9->70.2% instr)
Acting on Task-1's verdict (the legacy-PURE "~3%" is tooling, not a wall): the -O2 type-lift arm. family_sweep --hseq --no-preclassify --band all --allow-pins templates each matched exemplar's C onto its unbanked same-family members and whole-binary byte-gates every one (the sole arbiter, G3/P9). The pin-crash wall being dissolved (Phase-27/28 _carry_macros) let --allow-pins retry the pinned exemplars. - BANKED 3,407 member-matches across 136 overlays (INCLUDE_ASM stubs -> matched C). - R22 clean-fleet verify: 136/137 modified overlays byte-identical from a clean rebuild; the 1 FAIL (ov_SC07_010) was the SIGTERM mid-gate partial -> reverted, byte-identical. - make check-all: 140 passed / 140 BYTE-IDENTICAL; dedup-check 1840/0; audit-binaries OK; audit-cdecl green; 0 NON_MATCHING in any default build (G4). - Fleet: instr-weighted 68.9 -> 70.2% (+1.3pp) . distinct-code 49.5 -> 51.9% (+2.4pp) . fn-count 83.94 -> 84.73% (+0.79pp). - Process lesson (CURRENT_PHASE.md): --band all sweeps are too long for one background pass (got SIGTERM'd); future Arm B runs go band-bounded + committed-per-batch (resumable). Residual FAILED members (pin/drift/plumbing) + the -O0 Arm A carve remain. |
||
|
Drew T
|
2e84b53b2d |
feat(phase-29 T1): the swing number RESOLVED — (a) tooling, an -O0 compile-flag artifact (not a wall)
Phase-29 opens by running the Phase-28 disambiguating probe before scaling any "(cores)x(reach)" arithmetic on the legacy-PURE-non-jr "~3% as-tooled" swing number. - NEW tools/diff_regions.py (the deferred roadmap tool): remap the exemplar exactly as family_sweep --hseq stages it, compile at the EXEMPLAR's real opt level (auto-detected from the Makefile -O0 rules), masked-diff vs target, classify each member O0-FLAG / TEMPLATES / PLUMBING-ISO / REGALLOC / NO-TARGET. Composes family_remap + match_one + masked_diff (R33). - VERDICT (byte-proven): the 274 DIFF is dominated (~272) by two -O0 families (0x8013c964, 0x8013c938) that family_sweep compiled -O2 (member stub files are -O2); an -O2 compile of an -O0 target can never match. Compiled -O0 they masked-MATCH (func_8013C964->MATCH(10), func_8013C938->MATCH(11)). 106-member sample across nins 2..133: O0-FLAG 45 / already-banked 29 / TEMPLATES 17 / type-lift-plumbing 15 / REGALLOC 0. ZERO codegen walls. - The "~3% ceiling" is RETIRED (a tooling artifact — the 3rd structural wall to resolve to tooling after B2 and SC07). The member track is a mechanical -O0 split rollout + type-lift sweep (Task 2a), NOT a per-member grind; member_adapt.py not needed here. - Honest scope (§52b, R14/R35): masked-MATCH is a CANDIDATE; whole-binary banking is Task 2a's gate. Verdict + byte evidence -> docs/calibration.md + docs/decision-log.md (R31). |
||
|
Drew T
|
3827d01bfb |
feat(phase-28): the endgame engine -> the instrument-repair phase; B2 lives, SC07 wired, R36 (v1.27.0)
- RE-SCOPED at plan time (R35): the roadmap's swing number rested on a broken-tool probe. B2's "structural families bank ~0%" (0/8, which reshaped 2 phases of strategy) was a MISSING CARVE -> 8/8, then 102/115 (88.7%). The ~0% doctrine has NO surviving post-fix evidence. - T0 img_path derive-not-guess -> un-hid a 230,612-ins SC07 pool doubly hidden (P27 onboarded 4 overlays, never regenerated the map; the hardcode would have called every member "LEN"). - T3-A: the SC07 pool is h_exact + UNWIRED, not h_seq. T4 dedup_extend (NEW) wired 6174/6457 (95.6%) -> groups 134->138 binaries, C1 coverage +6174. NOT member_adapt (the number said build nothing). - T5: resident 21->14 (90.34%, 7 banked via an Ultracode wave) + honest dossier for the 14; fixed progress.py (#if 0 + len()-sum) and match_one's FAKE isolation (found by an agent mid-wave). - T2 purged the poisoned grinder blacklist (8/22 matched anyway). T6 killed the --chunk 1 double-build. - T7: make audit-binaries (the R36 citizenship gate, negative-control-proven) + fixed disc_code_sweep's BLINDNESS to compressed code (the type-4 row was vacuous for 138 known binaries) + the worklist key-bug. - T3b: the legacy h_seq swing number = ~3% AS-TOOLED, CLASSIFIED (274 DIFF / 37 PLUMBING), ceiling UNKNOWN -- the 274 DIFF is byte-PURE members whose remapped bodies don't reproduce (the same tooling-vs-wall ambiguity that resolved to TOOLING twice this phase). P29 disambiguates before scaling. - THREE SELF-INFLICTED DEFECTS fixed forward: the registry yaml.safe_dump (H5, destroyed 47 comments + 1832 hex fields, invisible to every byte-gate), two DIFF mis-reports (R14), the match_one shared scratch. - rule R36 (a newly-discovered binary is not real until every consumer knows it; enforced by audit-binaries). cookbook §53. R22 140/140 throughout; tools-health OK; dedup 1840/0. |
||
|
Drew T
|
6996d25379 |
feat(phase-28 T3b): the legacy h_seq swing number — ~3% as-tooled, CLASSIFIED, ceiling unknown
The roadmap's actual swing number (the LEGACY h_seq templatability rate; T3-A's SC07 pool was a different question — h_exact + unwired, banked 95.6%). family_sweep --hseq --chunk 1 over 6 legacy PURE non-jr families (has_mid_jr excluded per §53): - 9 BANKED / 37 PLUMBING / 274 DIFF (173 skipped not-stub/pinned) = ~3% (9/320). R22 140/140. UNLIKE Phase 26, the failures are CLASSIFIED — 274 genuine gate-DIFF, not an unclassified 0%. - THE LOAD-BEARING NUANCE (R14/R35 on my own probe): the 274 DIFF is NOT structural variance. The members are byte-level PURE (classify_member = reloc-only, 20/20 sampled), genuine h_seq (all DIFF_BYTES vs the exemplar, so family_sweep is the RIGHT tool not dedup_extend), at the SAME vram. A PURE family should reproduce once its relocs are remapped -> 274 non-reproducing members means the remapped body FAILS TO RECOMPILE to the member's bytes: either (a) an incomplete symbol_map (the recurring jtbl/prefix bug -- B2's 0/8 and T4's 12 DIFFs BOTH resolved to tooling THIS phase) or (b) genuine TU-context regalloc divergence (a real wall). - HONEST VERDICT: ~3% as-tooled, CEILING UNKNOWN. Provisionally consistent with "legacy h_seq doesn't mechanically template" but on a probe whose dominant failure mode is the exact tooling-vs-wall ambiguity that keeps resolving to TOOLING. Did NOT rush-resolve it at ~40% context (that is how Phase 26 manufactured a wrong 0%). P29 MUST run the disambiguating probe (diff one PURE DIFF member's staged bytes region-by-region: reloc-position mismatch = fixable remap; regalloc-away- from-relocs = TU wall) BEFORE scaling "(cores)x(reach)" on 3%. - calibration.md + decision-log R31 record the measurement + the named next probe. 9 real banks. |
||
|
Drew T
|
064e762a00 |
docs(phase-28): checkpoint — 9/11 tasks done; resume point for T7 + T3b recorded
Fleet 67.0 -> 68.9% instr / 47.8 -> 49.5% distinct, 140/140 byte-identical, 0 NON_MATCHING. Gate items met (swing number measured; resident resolved 90.34% + dossier). T7 (expanded to the audit-binaries R32/R36 gate + the blind disc_code_sweep fix) and T3b (the legacy h_seq rate) are recorded in CURRENT_PHASE with full state; both benefit from a fresh session's context. R36 drafted for PhaseEnd ratification. |
||
|
Drew T
|
af05f6e412 |
docs(phase-28 T5b): the resident wall dossier — 14 remaining, classed and byte-grounded
The flag-plant did NOT reach 100% (21 -> 14). This records what is left and WHY, per function, so the next attempt starts from evidence instead of re-deriving it. - docs/resident-dossier.md: all 14 remaining stubs in 3 honest classes + the 5 deferred jtbl. Each entry is the agent's own byte-grounded analysis — the levers tried, the exact gcc pass that blocked it, why it stuck. That analysis cost ~2.4M tokens and IS the durable asset (R30); the 7 banks were the cheap part. Raw verdicts preserved at .run/resident_wave_verdicts.json (R20, force-added past the .run/ ignore since they are not regenerable). - 5 PLUMBING: reached match_one MATCH standalone, failed IN-TU on `conflicting types` (D_8010EDEC / D_80115110 / func_800D1984 / CdReadRequest / cdFileLocTable). The Phase-16 loose-typing wall: the C is byte-correct, the TU cannot hold both spellings. gate_stage's recovery banked 0/5 — this needs a resident-scoped §41 def-side lever, not more drafting. - 4 DIFF: genuine gcc-2.7.2 residuals, each with a NAMED mechanism (func_800D2650 close=4 and func_800CFAD0 close=5 are permuter-class seeds; func_800D0E30 and func_800D27DC carry intrinsic allocno-priority verdicts). - 5 jtbl DEFERRED: need the rodata-island carve (§53 + the Phase-7 workflow), which the resident has no split infra for. Deliberately NOT forced — sweeping a jr function without its carve is EXACTLY how the "≈0% structural families don't template" doctrine was manufactured (T1), and that mistake cost two phases of strategy. - The dossier records the two caveats a future reader needs: the wave ran on a broken match_one (shared scratch — fixed in commit:0652, verdicts may carry that noise, the GATE results do not), and the func_800CEDFC / func_800D33E0 sig_image boundary question (defined in resident.c, absent from the 2nd oracle, while audit-corpus reports 0 PHANTOM/TRUNCATED) -> T7 audit-binaries. |
||
|
Drew T
|
fda9eebb42 |
fix(phase-28 T4): wire all 4 SC07 overlays (6174/6457, 95.6%) + REPAIR the registry I destroyed
Completes T4 and corrects two defects I introduced, both landed in commit:0649. - WIRED: 006 1543/1614 · 007 1544/1615 · 010 1544/1614 · 011 1543/1614 = 6174/6457 = 95.6%, ~0 agent tokens. Stubs/overlay ~2400 -> 831/984/898/825. Fleet instr 67.0 -> 68.9%, fn-count 82.16 -> 83.94%. dedup-check 1840 validated / 0 failed; groups now read "138 members [138 binaries]" (was 134); C1 coverage 227211 -> 233385 = exactly +6174. R22 make clean && extract-all && check-all -> 140 passed, 0 failed of 140 at every stage. - FIX #1 — I DESTROYED THE REGISTRY'S DOCUMENTATION, AND EVERY GATE CALLED IT GREEN (H5). The first cut wrote config/dedup.us.yaml with yaml.safe_dump, round-tripping the whole file: 47 comment lines -> 0 (including the curated Phase-11 header explaining WHY the share is source-level) and 1832 `vram: 0x80162FF4` -> `vram: 2148937716` (PyYAML parses YAML-1.1 hex to int; dumps int as decimal). 25,948 lines rewritten. It passed dedup-check 1840/0 AND check-all 140/140 because _addr() accepts both forms: THE DATA WAS CORRECT AND THE DOCUMENT WAS RUINED. Fixed forward (R6, no history rewrite): restored from commit:0649~1 and re-applied the 6174 memberships via a surgical text edit (add_members_surgical). Verified: 1545 insertions / 1545 deletions, 0 non-`binaries:` lines changed, 47 comments + 1908 hex fields intact, and the rebuilt fleet is byte-identical to the destructive version (140/140). THE LESSON: every oracle this project owns measures BYTES, so a formatting-destructive write is invisible to all of them by construction. R34 says the byte-gate is a null COVERAGE oracle; this is the same hole one layer out — it is a null DOCUMENT oracle too. - FIX #2 — I MIS-REPORTED THE DIFFs, TWICE (R14). (a) commit:0649 claims ov_SC07_006's 71 non-banks were "ALL PLUMBING, ZERO DIFF". FALSE — I read head -6 of the classified file and generalized. It has the same 4 DIFFs as the others. (b) I then built the jr guard assuming those 4 were the §53 jr class BECAUSE ov_SC01_077 hosts them in _jr_8017A4AC.c / _jr_80182268.c. has_mid_jr is FALSE for all four (33-52 ins, no jump table): they merely live in a carved jr-REGION split, which sweeps in every function in its address range. HOSTING FILE != FUNCTION CLASS. The guard is KEPT (preventive, §53-correct, currently skips 0 — no jr fn is in the extendable set) with its docstring corrected to record what it is NOT. The 12 DIFFs (0.19%) are UNDIAGNOSED and logged, correctly left as stubs by the gate — not dressed in a story. - The 283 non-banks: 271 PLUMBING (the loose-typing conflict class + the whale, whose body lives in src/shared/func_80144B9C.h so no DEFINE macro exists to expand) + 12 DIFF. Existing tools cover the plumbing (cast_call_sites / canon_sig_reconcile / reconcile_tu). |
||
|
Drew T
|
515d003dbe |
feat(phase-28 T3-A): ZERO DIFF — the SC07 pool is the EASY (h_exact) class and is simply UNWIRED
Stratum A of the swing-number probe, on the cleanest test available: the 4 highest-byte-weight
SC07-only families are the GIANTS (func_80144B9C 770 "the whale", func_80141CA4 476,
func_80132784 400 "irreducible for 22 phases", func_80133CD4 399 the §45 Fable5 crack).
Exemplars already byte-proven, members PURE, non-jr -> every confound removed.
- RESULT 4 banked / 8 failed / 4 skipped of 16, and the CLASSIFICATION is the finding:
* BANKED 4 = func_80133CD4, a 399-ins Fable5 giant, into 4/4 new overlays, free
* PLUMBING 8 = "parse error before ')'" (func_80144B9C, func_80132784) — never compiled
* skipped 4 = pinned-exemplar (§42e guard; P27 T5 dissolved the wall behind it)
* DIFF 0 <- NOT ONE failure is a byte mismatch
Of the members that reached the gate as valid C: 4/4 = 100%. This is exactly the
DIFF-vs-CC1-FAIL distinction Phase 26 never recorded, and why its 0% couldn't be trusted.
- ROOT CAUSE (byte-verified, far bigger than the parse error): the 4 new overlays were
onboarded byte-clean but NEVER WIRED INTO THE SHARED-BODY ECOSYSTEM.
established ov_SC01_001 : common.h + ../shared/engine_core.h ; DEFINE_func_*() ; ~2150 matched
the 4 new SC07 : common.h ONLY ; ~2400 raw stubs ; ~80 matched
refs in config/dedup.us.yaml: 0. 1689 registry groups say "134 binaries", never 138.
The parse error is a symptom: the drafts need types (P10/P14/P18/P1C/HDR/ENT) that live in
src/shared/func_80144B9C.h — a header the SC07 TU never includes.
- SCALE (measured vs the registry): 6,513 live stubs across the 4 new overlays are byte-identical
to an ALREADY-REGISTERED h_exact group (1625/1628/1627/1633). That is the h_exact class —
which calibration.md itself rates ~xN near-100% — NOT h_seq, and NOT a member_adapt problem.
- MECHANISM PROVEN BY HAND (probe-before-investing): +#include "../shared/engine_core.h" and ONE
stub -> DEFINE_func_80128158() in ov_SC07_006.c -> make build -> 7ca772be... BYTE-IDENTICAL.
Probe reverted; the tool should do it uniformly.
- THE TOOLING GAP -> T4: dedup_propagate --auto-from plans only 11 fns (it authors macros from
ov_SC01_077 INLINE DEFS; the ~1600 shared bodies are ALREADY DEFINE_func_* macros in
engine_core.h), and --addr errors "no source overlay has it matched" because no overlay holds
an inline def. There is NO mode for "extend an existing macro-backed group to a
newly-onboarded binary". T4 builds it (NOT member_adapt — the number says build nothing else).
- R22: make clean && extract-all && check-all -> 140 passed, 0 failed of 140.
- CARRIED -> T3b: strata B (legacy PURE non-jr, 95 fam / 7993 members) and C (legacy IMM, 36 fam
/ 6644) — the LEGACY h_seq rate is still genuinely unmeasured. The SC07 pool answered a
different, cheaper question than T3 set out to ask.
|
||
|
Drew T
|
65e96e5d5a |
feat(phase-28 T2): purge the poisoned grinder blacklist — 8 of its 22 had MATCHED anyway
The blacklist recorded "permuter won, byte-gate rejected => plumbing-bound, never
re-permute" for 22 fns. It was manufactured by a gate that no longer exists.
- R14 ON THE PREMISE, TWICE (before touching anything):
* The roadmap's two named grinder bugs are ALREADY FIXED (commit:0327, commit:0200) — stale line.
* docs/tooling-audit.md:933-937 DOWNGRADED ITS OWN FINDING with three corrections: the
prescribed fix is a NO-OP (deleting the scanner banks zero fns — harvest_verify was
single-TU BY CONSTRUCTION, the defect was mislocalized to it); nothing is being discarded
now (grinder STOPPED since 2026-07-02 — confirmed via .run/auto/STOP); queue magnitude
inflated (1252, not 1298). T2's only real content was the persisted artifact.
- AND THE AUDIT'S SNAPSHOT IS ITSELF STALE (verified in code): harvest_verify is now fully
multi-TU — _stubs derives every stub across every TU from the corpus oracle (:122), render()
splices "each draft into the TU that actually holds its stub", _write() writes multiple
paths, un-stubbed drafts are REPORTED not silently dropped (R32). The gate that manufactured
the blacklist is gone. No harvest_verify change was needed or made.
- THE PROOF IT WAS MANUFACTURED, NOT OBSERVED: of the 22 entries, 8 have since MATCHED anyway
(func_80131D68/80149374/8014FE60/80150528/8016BBE0/80171C64/80174684/8017F290); the other 14
are still stubs and would have been skipped FOREVER on a dead gate's verdict. (16 of 22 were
split-hosted, so harvest_verify never compiled them — the permuter's byte-matches were
discarded UNBUILT. The audit predicted 5 matched-anyway; it is 8.)
- DONE: blacklist -> [] (poisoned copy preserved at grinder_blacklist.json.poisoned-pre-T2);
grinder.py now carries the RULE (R35): a blacklist entry is a verdict from a SPECIFIC GATE
and EXPIRES when that gate changes — purge and re-derive from the fixed gate, never inherit.
A persisted negative verdict is only as good as the instrument that produced it.
- No byte claim (analysis tooling + a scratch artifact only) -> no R22 cycle owed.
|
||
|
Drew T
|
4db79a2060 |
feat(phase-28 T1b): the B2 family swept — 102/115 banked (88.7%), fleet 67.0 -> 67.7% instr
The family the roadmap recorded as 0/8 ("~0%, structural families do not template" — the
number that rewrote P29's arithmetic to "(cores cracked) x (reach)") banks at 88.7% when
swept with the carve its own exemplar required. ~0 agent tokens.
- SWEEP: jtbl_family_bank.py over the remaining 107 members ->
{'BANKED': 94, 'gate-fail': 7, 'remap-refuse': 6}. Family total 8 (T1) + 94 = 102/115.
R22: make clean && extract-all && check-all -> 140 passed, 0 failed of 140, 0 FAIL lines.
- FLEET (measured, make report): instr-weighted 67.0 -> 67.7% (+0.7pp, +97,104 ins);
distinct-code 47.8 -> 49.4% (+1.6pp); fn-count 82.16 -> 82.19%. 102 x 952 = 97,104 =
the exact measured instruction delta — the arithmetic reconciles to the byte.
- THE 13-MEMBER TAIL is the predicted shape, and both halves are data for T3:
* 6 remap-refuse = EXACTLY the family's 6 IMM members (cls_counts PURE 109 / IMM 6).
imm_map_tier1 REFUSED rather than guessed: "unresolved immediates: [(512,
'asm-ambiguous')]" — 512 also occurs at a non-differing position, so a blind swap could
corrupt it. This is the concrete shape of T3's IMM stratum.
* 7 gate-fail = genuine byte-DIFFs, correctly rejected. Verified to leave NO residue
(all 7: split_file=none, cfg_refs=0) — no false-bank risk.
- HYGIENE: the 7 "git checkout ... did not match any file" errors are benign (revert of a
never-tracked path). Verified 0 untracked splits belong to a non-banked member; 91 new
splits + 3 banked into existing splits = 94.
- SCOPE (P9, unchanged): still n=1 family, and jr is the rarest class (3/163 matched-exemplar
families). This demonstrates the mechanism at family scale; it does NOT give a rate for the
PURE/IMM mass (98% of the population). T3 measures the swing number.
|
||
|
Drew T
|
a4640e3a51 |
feat(phase-28 T1): B2 LIVES — 8/8 banked; the "families don't template" doctrine was a missing carve
The roadmap's decisive P28/P29 input (h_seq families bank at ~0%) is byte-refuted. Same
family, same era, through the carve path its own exemplar required: 8 of 8 BANKED.
- THE PROBE: jtbl_family_bank.py func_8017BEBC ov_SC01_000 0x8017bebc --raw
.run/phase26-cracks/func_8017BEBC.c over 8 of 115 members (4 same-address + 4
CROSS-address, exercising to_addr) -> {'BANKED': 8}.
R22: make clean && extract-all && check-all -> 140 passed, 0 failed of 140.
- ROOT CAUSE of the P27 0/8, byte-verified: 0x8017BEBC is a jr/switch core. §47 banked its
exemplar as "lazy isolation -> carve (9-piece interleave) -> splice -> BYTE-IDENTICAL" and
called the fix "×N template-safe". family_sweep.hseq_sweep stages C and gates -- it has NO
CARVE STEP -- so gcc's generated jump table is never placed at the sibling's address. The
entire residual is TWO WORDS: classify_member -> PURE, ndiff=2 @ idx 343/345 =
lui/lw %hi/%lo(jtbl_801EC44C). overlays.mk:112 carves ov_SC01_000_jr_8017BEBC.o for the
exemplar; :134 has no such entry for the member. tools/jtbl_family_bank.py exists to do
exactly this per sibling and had NEVER been run on this family.
- THREE COMPOUNDING FAILURES made the doctrine: (1) wrong tool for the class; (2) n=1 on the
LEAST representative family -- has_mid_jr is 3 of 163 matched-exemplar families (120 of
13,232 members) -- generalized to the whole frontier; (3) its corroborating Phase-26 probes
(tiny-IMM 0/241, PURE 0/134, pinned 0/133) ALL predate _carry_macros (P27 T5, commit:0637).
P27's decision-log calls its own re-probe "a FOURTH phantom exhaustion proof" -- naming the
mechanism that would have faked the first three, and never re-running them. The ~0% doctrine
has NO surviving post-fix evidence.
- SCOPE HONESTY (P9): this refutes the EVIDENCE for ~0%; it does NOT establish a general rate.
n=1, and jr is the rarest class by construction. T3 measures the rate over the population
that actually exists: 1418 matched-exemplar families / 21,889 members (PURE 78% / IMM 20% /
STRUCT 1.8% -- note the roadmap sizes its swing number on STRUCT = 1.8% of the input).
- TWO SELF-CORRECTIONS (R14), both mine: (a) the approved plan's "add jtbl_ to symbol_map" was
a WRONG FIX FROM A TRUE DIAGNOSIS -- a compiler-generated switch table is never named in C,
so there is no token to substitute; the fix is PLACEMENT. No symbol_map change was made and
T1 became a run, not a code change. (b) func_8017BEBC.md's header still says "close=2 of 952"
(pre-§47-slider); the .c was updated, the .md was not -- templating from the header's premise
would have produced zeros indistinguishable from a wall.
- DISTILLED IN-SESSION (R30/R16): cookbook §53 (sweep a family with the tool its exemplar
needed: the carve law, the --raw rule, the symbol_map-jtbl trap, and the "before a 0%
retires a lever" three-question test); calibration.md's decisive table REWRITTEN (the ~0%
row marked an artifact, not a rate; the addressable pool tabulated); decision-log R31.
- Carried: the family's remaining 107 members (~101,864 ins, ~0 agent tokens) -> T1b.
|
||
|
Drew T
|
76db32455c |
feat(phase-28 T0): fix img_path (derive, don't guess) — the SC07 pool was doubly hidden
R35 sequencing: fix the instrument before the probe that scopes the phase.
- family_remap.img_path: DERIVE the payload from config/splat.<bin>.yaml's target_path
(R33 — the file the BUILD reads, so it cannot drift from the bytes) instead of
reconstructing `.../FILE_{nnn}.dir/0.4.dec` from the alias. RAISES on a missing
payload (R32) — the silent None WAS the defect.
Negative control (the fix must change an answer the old tool gave):
ov_SC01_001 -> 0.4.dec UNCHANGED (no regression)
ov_SC07_006 -> None -> .../1.4.dec
resident -> (n/a) -> MAIN.CD.dir/FILE_010.dir/1.1 (free; feeds T5)
ov_SC99_999 -> None -> raises
Downstream: all 233 shared substantial fns between ov_SC07_006 and ov_SC01_001
classify PURE (reloc-only). Under the old tool every one returned LEN = "not
templatable" AND poisoned its family's diff_class to MIXED (family_hseq.py:141-143).
Same bug class as new_overlay.sh's hardcoded 0.4.dec glob (which hid these four
overlays for a month) — left uncorrected in a second tool. Fourth instance of the
project's dominant defect class, sitting directly under the number P28 must measure.
- .run/family_hseq.json regenerated: 134 -> 138 overlays (the 4 P27 SC07 overlays newly
visible); metrics re-baselined 68.9 -> 67.0% instr (now agreeing with the committed
progress.fleet.md); LEN across the whole frontier = 0 (a phantom-LEN from a missing
image is now structurally impossible). Proven consistent against the post-tools-health
sigs by a second run (byte-identical) rather than assuming sig_image is deterministic.
- FINDING — a large, doubly-hidden target pool: 1255 families / 6268 members / 230,612 ins
whose ONLY unmatched members are in the 4 new SC07 overlays (0 elsewhere — a clean
partition), each behind an ALREADY-MATCHED, byte-proven ov_SC01_077 exemplar. Classes
PURE 5575 (89%) / IMM 633 (10%) / STRUCT 60 (1%). Hidden twice: P27's disc audit created
it by onboarding the overlays but never regenerated the map — and had it, img_path would
have classified every member LEN. Corroborated independently by tools-health: the 4 new
overlays are ~97% unmatched (stubs ~2,400, matched ~80) vs ~85% matched for their
siblings. PREDICTION, not a bank — h_seq predicts, the whole-binary gate decides (G3/P9).
-> T3's headline stratum, and a better probe than planned: the exemplar is already
byte-proven, so a failure isolates the templating mechanism with no drafting variable.
- SELF-CORRECTION (R14): the approved plan's own population figures (163 families /
13,232 members) came from the STALE map — my numbers were an instance of the defect this
phase is about. Honest: 1418 matched-exemplar families / 21,889 unmatched members
(PURE 17,024 = 78% / IMM 4,473 = 20% / STRUCT 392 = 1.8% — the roadmap's "register-drift"
swing class stays ~2% of the input, so that framing is unchanged). Legacy pool unchanged
at 163 families (the fix + the 4 overlays are purely additive).
- T3 strata (honest): SC07-only 1255 fam / 6268 mem / 230,612 ins · legacy PURE non-jr
95 / 7993 / 478,379 · legacy IMM 36 / 6644 / 212,707 · legacy MIXED 30 / 968 / 10,462 ·
legacy PURE w/ jr 2 / 16 / 5,088. Total addressable 937,248 ins = 21.7% of all remaining
weight = 7.16pp of fleet instr if it all banked — the prize the roadmap declared dead.
- tools-health GREEN: sigs fresh; corpus(+resident) 0 PHANTOM + 0 TRUNCATED; cdecl;
report(lint + dedup 1840 validated / 0 failed, C1 coverage 227211/227211). No source or
build input touched (analysis tooling + regenerated digests only) -> no byte claim, no
R22 cycle owed. docs/duplicates.cross.md regenerated: overlays 134x -> 138x, h_exact
cross-binary 9366 -> 9484 groups; resident sig now the sig_image one (P27 T10 intent).
|
||
|
Drew T
|
3b31508a24 |
feat(phase-27): the honest frontier — fix the instruments, audit the disc, dissolve a wall (v1.26.0)
- INSTRUMENTS FIXED: Makefile fail-closed (report's gates were swallowed); one cdecl typedef-strip primitive (was 6 regexes); scanners derived not hand-listed (difficulty/exemplar_miner); the second boundary oracle extended to resident. Each fix CHANGED an answer the old tool hid. - DISC AUDITED HONEST: 4 hidden SC07 overlays onboarded (136->140, code at PAC entry 1) + 39 un-onboarded type-1 code modules found (resident-class, load-address RE pending). The byte-gate is blind to un-onboarded code (R34); game-code TRUE 100% now spans 140 + ~39. Instr 68.9->67.0% (denominator correction, not regression). - PIN-CRASH WALL DISSOLVED: the §42e "cc1 SIGABRTs the sibling TU" wall is the extract_unit macro- drop (sched.c:2725), fixed (T5 _carry_macros); pinned families stage 133/133 clean -> P31 open. - HONEST FRONTIER: worklist --assert-partition (R32, caught 5 stale rows); ledger corruption fixed; calibration.md (the templatability swing: h_exact cores ~xN, h_seq families ~0% -> B2 refuted). - FABLE5 SPRINT: 4 cracks + the SIGABRT, 0 banks, but 3 wall reclassifications + the wall dissolved + ~9 pin-free levers distilled (cookbook §42e/§44 + regalloc/cse_expr §H + decision-log R31). - 140/140 byte-identical (R22), 0 NON_MATCHING (G4), audit gates green + fail-closed. No tools installed. rules R35 (fix the instrument before trusting its measurement). bumps 1.25.0 -> 1.26.0. |
||
|
Drew T
|
d1ef983af0 |
docs(phase-27 T1 close): func_80176734 crack + distill — the CSE address-fold antidote
The last Fable5 pass of the sprint (Drew capped further waves at 86% context). func_80176734 (371 ins, fresh un-drafted core): NO bank (mine=370 vs 371, 5 permuter-shaped clusters — entry-schedule tie, caller-saved shuffles, a combine-merge missing insn, qty ties), pin-free, honestly handed off (P9; match_one confirms the DIFF). Draft -> decomp-permuter warm-start (P29). Idiom harvest (cookbook cse_expr §H): - THE CSE ADDRESS-FOLD ANTIDOTE (zero asm): find_best_addr's cost-ungated qty-const fold + from_plus re-association eat reg-based global accesses on every cse walk; a balanced if/else DIAMOND makes the merge label barrier-preceded -> fresh cse table -> both folds die with no #APP. Replaced two asm dials. - update_equiv_regs doubles live_length for single-set REG_EQUIV pseudos (local-alloc.c:1064) — a 2nd set forfeits the doubling, ~4x the allocno priority; explains a "my dial broke the $s-order" class. - record_jump_equiv fall-through delete (cse.c:7511) — a recognition tell for genuine dead source logic. T1 sprint COMPLETE: 4 cracks + the SIGABRT characterization, 0 direct banks, but 3 wall reclassifications + 2 cracked roots + the pin-crash wall dissolved + ~9 new pin-free levers. Fable5 DISCOVERS, cheap-Opus APPLIES — the ROI is idioms, not banks (docs/calibration.md). |
||
|
Drew T
|
ce88baf365 |
feat(phase-27 T9): calibration — the templatability swing measured (structural families are NOT cheap)
docs/calibration.md — the byte-gate-grounded rates that size P28/P29 (roadmap §6 held yield projections until this). - VELOCITY: instr 68.9->67.0% (a T7 denominator re-baselining DOWN, not a regression) + ~0 matches banked (an infrastructure/findings phase). The honest flip-checkpoint read: denominator correction + unblocking findings, NOT 0 progress/session — velocity resumes at P29, re-measure there. - THE TEMPLATABILITY SWING (decisive for P28/P29): h_exact reach-N cores propagate ~xN near-100% (§52: 5 cores -> 670 instances) vs h_seq/h_norm structural families ~0% (0x8017BEBC: 106/112 stage but 0/8 bank, all genuine DIFF). So remaining yield = per-member cracking + mechanical xN for the h_exact cores, NOT "template x120 the 986 families" — B1/B2's cheap-harvest hope is byte-refuted. The 223-stub frontier: 101 reach-134 (xN-able if cracked) + 119 reach-1. - COST/TIER: Fable5 ~230k tok/fn, 0 banks / 5 — ROI is idioms + the pin-crash wall dissolved, not banks (the doctrine held). cheap-Opus is the banking tier; permuter tail exhausted; local-v3 $0/<=15. - HONEST GAP: the headline member-adapt close-rate on register-drift members needs P28's member_adapt tool (chicken-and-egg) -> P28 opens by measuring it on a byte-gated sample, per the risk register. - NEW un-projected fuel: ~20 PINS-class stubs now harvestable (pin-crash dissolved, T5). |
||
|
Drew T
|
0f68a83cfa |
feat(phase-27 T8): worklist --assert-partition (R32) + honest re-scan + ledger corruption fixed
- worklist --assert-partition: the audit's literal R32 prescription (tooling-audit.md:1173) — enumerate live stubs from corpus.stubs (the invariant, R33), assert the fuel manifest partitions its source overlay's stubs, exactly one row each. Scoped honestly (worklist's universe is ONE overlay ~223 stubs, not the fleet's 53k — a fleet partition is a scope change, not a flag). PROVEN: it caught 5 stale rows (pin-free cores Phase-26 banked, manifest never re-derived) -> FAIL exit 1. - honest re-scan: build_fuel_manifest on the fixed tools + 140 binaries. Giants re-verified reach-138 (was 134 — the SC07 overlays now counted). Partition PASSES 223==223 after refresh. - ledger corruption fixed: func_80178004's 2 false `close=0 "MATCH"` records (a Phase-26-retracted myth — the seed's best was 5 pinned, and a real close=0 whole-binary match BANKS; it is still a stub) -> corrected to the honest close=91 regalloc wall. func_8012E364 already honest (close=23 — the "stale closeness" flag was itself stale). No real duplicate rows (load_best dedups by addr; the uniq hits were func names in where_stuck prose). docs/worklist.md + docs/backlog.md regenerated. - the 1,670-untriaged near-miss triage SCOPED TO P29 (P5d): Phase-21 automation leftovers whose class labels re-derive at harvest, and the pin-crash finding re-buckets the PINS class — an Ultracode fan-out buys low-durable labels; the gate's residue map is the partition + the class summary, done. |
||
|
Drew T
|
54bdf96218 |
docs(phase-27 T1): distill the Fable5 wave — the pin-crash wall refuted + 3 RC-6 downgrades
The flywheel step (R16/R30): turn the wave-1 + SIGABRT byte-proven findings into cookbook/codegen-map knowledge, in the producing session. The distillation REWRITES wall verdicts, so accuracy is load-bearing. - cookbook §42e-CORRECTION: the "pin-crash wall" (register-pin-heavy families "SIGABRT the sibling TU, ov077-TU-context-specific, NOT ×134-recoverable") is REFUTED. The SIGABRT is real (sched.c:2725 create_reg_dead_note, a sched1 REG_DEAD-note conservation bug) but was TRIGGERED by extract_unit dropping file-scope #define macros (the T5 bug) -> implicit-call GTE ops -> caller-saved pins in the fatal shape. Only 1 of 4 families genuinely crashed; 3 were exit-33 plumbing folded into one crash bucket. Fixed, all 4 stage 133/133 clean. Per-pin predicate recorded. P31's pin route is OPEN. - cookbook §44-Lever-5: the 3 functions it cited as intrinsic (func_8014D820/8016CBC0/801670E4) are each oracle-refuted (2 cracked roots + 1 RC-6-not-S3). Corrected the "NEVER ship pinned, it SIGABRTs" claim per §42e-CORRECTION. - gcc-2.7.2-map/regalloc.md §H: THE reg_renumber-swap oracle (discriminate RC-6 allocation from S3 scheduling in one gdb run — patch reg_renumber at reload entry, swap the contested regs; byte-exact = pure allocation), RC-14 reused-load-temp serialization (the MERGE pole; pin-free, cheap-Opus), RC-15 the density dial across a floor_log2 boundary (subsumes "coalescing knife-edge"), and the local-vs- global allocation tie as a precisely-named honest sub-class. Continues the §F/§G RC-6-downgrade series. - decision-log.md R31: the 3 Phase-27 strategic findings (disc is bigger: 140 + 39 modules; a wall was our tool again; a cheap win is dead) + the through-line — the roadmap's numbers were red-teamed, the tools under them were not, until this phase. func_80176734 (fresh-core wave-2 agent) still running; its findings fold in before the PhaseEnd. |
||
|
Drew T
|
ee4b3a02e8 |
feat(phase-27 T5): extract_unit carries file-scope #define macros — honest 0x8017BEBC probe + the pin-crash wall dissolved
family_remap.extract_unit dropped the file-scope function-like #define macros a body references (the gte_* C inline-asm GTE-op macros live ABOVE the function; the backward preamble walk stopped at the first #define/continuation line). A staged sibling saw every GTE op as an implicit-declaration CALL. Two consequences in one bug: - staging failed: 0x8017BEBC's 112 members all CC1-FAIL'd -> a FAKE 0% probe that reads "mechanical harvest dead" when the tool was broken (a 4th phantom exhaustion proof, exactly the 26-A audit class). - the SIGABRT: with a caller-saved register PIN present, the phantom call pushes cc1's sched1 into create_reg_dead_note's abort (sched.c:2725) — the §42e "pin-crash wall". The wave-2 SIGABRT agent proved this IS the cause (.run/giants/pin_crash_sigabrt.md): pinned families stage 133/133 clean once their macros ride along. A propagation wall recorded as a compiler limit for phases = a staging-tool artefact. - _carry_macros: prepend the function-like #define macros the unit body references (file order), not already inside the unit. Safe by construction: feeds only the templating path (remap_hseq), never make_macro's engine_core.h lift (no #define embedded in a DEFINE_func_*() macro); gather_externs only scans func_/D_ so no bogus extern; regression-verified non-GTE exemplars carry 0 macros (a no-op where it should be). THE HONEST PROBE (R14): staging 0 -> 106/112 (6 skip = IMM tier-2). Bounded 8-member gate sample = 0 banked / 8, ALL genuine DIFF (T4 classifier: compiled, wrong bytes — NOT plumbing). 0x8017BEBC is BYTE-PROVEN NOT TEMPLATABLE: the roadmap's "largest cheap win left" (B2) is REFUTED. The h_seq match is necessary, not sufficient. This 0% MEANS something because the tool is fixed first. Carried to T8: harvest the now-unblocked pin families (verify the 133/133 claim + bank). |
||
|
Drew T
|
427baba3bf |
feat(phase-27 T10): completion dashboard (main in the weighted metric) + the resident second oracle
The metrics contract (roadmap §1) wants all three metrics WITH main in the denominators, and the
second, independent boundary oracle (R34) extended beyond the overlays. Both had landmines.
10a — main into the weighted metric, safely:
- weighted_metrics off the func_-only src_stubs regex onto corpus.stubs (R33). THE LANDMINE IS
REAL: src_stubs("SLUS_007.26") globs src/SLUS_007.26/*.c -> 0 files -> every row "matched" ->
main 100% + fleet % silently inflates. Routing through corpus.stubs is a PROVEN 0.000pp no-op on
the existing fleet (overlays are all func_) and closes the curated-name leak.
- a SEPARATE "MAIN game-code weighted" line (0.7%): main's Ghidra sig excludes the LINKED PsyQ
objects (Ghidra never analysed them), which is exactly right for a game-code metric (LINKED is
complete, counted in fn-count). Reported un-folded and caveated (month-stale sig, PROVISIONAL) —
folding a stale/incomplete value into the decomp.dev headline would mislead the flip checkpoint.
10b — the resident second oracle:
- make sig-resident: sig_image on the resident flat blob (byte-derived, not Ghidra). corpus.
sig_is_independent now covers resident -> audit-corpus checks its boundaries too. Probed clean
BEFORE wiring (144 fns, all 21 stubs present, 0 phantom), verified 0 phantom + 0 truncated.
- sig-overlays now derives its payload list from config/overlays.mk, not a 0.4.dec glob that
silently dropped the 4 SC07 index-1 overlays (the audit's own silent-skip class). tools-health
regenerates sig-overlays + sig-resident first so the audit never crashes on an absent sig.
10c — main's second oracle: docs/second-oracle.md. sig_image can't sign the PS-X EXE yet (0x800
header offset, interleaved data/linked islands, one text range); seeding from splat would destroy
independence for the PHANTOM class specifically. Honest deferral + scoped design, not a fake oracle.
- docs/progress.fleet.md regenerated: 140 binaries · fn-count 82.16% · instr-weighted 67.0%
(the honest post-T7 drop from 68.9%) · distinct 47.8% · MAIN game-code 0.7% (separate).
- SETUP §6.3 updated (R21).
|
||
|
Drew T
|
264fe6c115 |
feat(phase-27 T7): disc-completeness audit — onboard 4 hidden SC07 overlays (136->140) + the type sweep
The whole-binary byte-gate is structurally blind to code nobody onboarded (R34): check-all is
green over the onboarded set no matter what code sits unbuilt on the disc. This reconciles the
onboarded set against every code-bearing PAC payload.
- new_overlay.sh: optional [ENTRY] arg (default 0.4) reaches a non-0.4.dec payload. Onboarded
ov_SC07_{006,007,010,011} from 1.4.dec (they put graphics at PAC entry 0, the code overlay at
entry 1 — invisible to the 0.4 hardcode for a month). Each byte-identical (7ca772be / b3b95547 /
d7b5875d / 9885af74). FLEET 136 -> 140; check-all 140/140 (T2's pass==N re-baselined cleanly).
difficulty.py NOT in the insertion set anymore (it derives, T6) -> only 3 tool dicts touched.
- tools/disc_code_sweep.py: decode every payload (reusing sig_image.make_insn) and gate code on
BOTH valid>=0.90 AND jr_$ra density>=0.01. The jr_$ra gate is decisive: isValid() alone flags
389 false hits (type-0/2 structured data decodes ~100% valid but has ZERO returns); jr_$ra
separates code (~2.9-3.4%) from data (0.000%), validated on positive+negative controls.
- FINDING (docs/disc-completeness.md): type-4 location overlays are COMPLETE (138/138). All other
types are data EXCEPT type-1 = 40 code payloads, 1 onboarded (the resident), 39 HIDDEN
resident-class modules (mostly MAIN.CD/FILE_XXX/1.1). They load at UNKNOWN addresses (not the
shared overlay slot), so they are NOT mechanically onboardable — byte-verifying a build binary
needs its load address (P9), knowable only by runtime RE (the Phase-3 method). Deferred with
evidence, NOT force-onboarded at a guess.
- CONSEQUENCE: game-code TRUE 100% now spans 140 onboarded binaries PLUS ~39 type-1 modules
pending load-address RE. The roadmap assumed 136 — this is a real re-baselining (the +4 overlays
also add ~2.45 MB to the denominator; every family propagation is now x138). Flows to T10/T11.
- SETUP §6.3 tool inventory updated (R21).
|
||
|
Drew T
|
8a1a1a0d79 |
feat(phase-27 T6): migrate difficulty + exemplar_miner off hand-lists/proxies (R33) — before T7
The 26-A audit named difficulty's 136-entry BINARIES dict as the exact root cause corpus.py:9 describes (a hand-maintained allowlist over a filesystem that already answers the question), and exemplar_miner's registered_addrs() as a ~60%-wrong proxy for "is this still work?". T7 onboards new overlays, so these are migrated FIRST or the new binaries silently miss make report. - exemplar_miner.py: "still a residual?" now = corpus.stubs(source) membership (the INCLUDE_ASM invariant, R33), not dp.registered_addrs() (config/dedup.us.yaml — a matched-but-unregistered fn, e.g. banked inline or matched-but-local, stayed wrongly in the residual pool). - difficulty.py: the 136-entry hand-dict -> cfg_for(alias), deriving the mechanical layout (src/<a> + asm/<a>/nonmatchings; main/resident the two specials). Validated against the tree (src/<a> must exist -> a typo is a clean error, R32), not a hand-list. A newly-onboarded overlay now needs zero difficulty registration. - new_overlay.sh: DROPPED difficulty from the sentinel-insertion set (T6 made it obsolete; leaving it would insert a dead dict entry into a file that no longer has a dict). The other 3 tools (diff_settings/progress/dup_report) keep their hand-lists — migrated one-at-a-time, byte-gated, per the audit's cadence; NOT dup_report.BINARIES, which corpus depends on as the binary list. VERIFIED: - difficulty derivation is BYTE-EXACT vs the old dict for all 136 aliases (0 mismatches), and old-tool vs new-tool output is byte-identical (.md AND .csv) on the same tree — the diff vs the committed docs was pure staleness (committed 2026-06-20, tree at 2026-07-15), NOT my change (R14). - unknown alias -> clean error, not silent-empty output. - exemplar_miner runs -> 223 residual stubs (corrected; it's a manual tool, not in make report). - new_overlay.sh: bash + embedded-python both parse; difficulty absent from the insertion set. |
||
|
Drew T
|
e0a0becfaa |
feat(phase-27 T4): one cdecl typedef-strip primitive (was six regexes) + surface cc1 stderr
The plan named two defective regexes; the tree had SIX with complementary holes, each silently recording the resulting compile failure as "not a match" — a plumbing error wearing a compiler wall's clothes, the exact class the 26-A audit exists to end (R32). - cdecl.py: the canonical primitive — typedef_names(tu_path) + strip_provided_typedefs (draft, provided). Built on tu_statements (robust) NOT tu_scope (which coverage-asserts -> would crash the byte-gate on any unrelated unparseable file-scope statement). Splits multi-typedef lines (split_statements, depth-aware); covers scalar AND struct typedefs; keeps draft-local types. lru_cached. - harvest_verify.py: strips PER-TU (cdecl.typedef_names of the draft's real target TU) -> unblocks the 39 struct-typedef drafts the scalar-only _TD dropped. And SURFACES cc1 stderr: build() stashes it; a single-draft failure is classified DIFF / PLUMBING:… / CC1-FAIL / SKIP -> .run/harvest_failed.classified.txt. A `redefinition` is no longer recorded byte-identically to a codegen miss. - masked_diff.py: strip_scalar_typedefs() (common.h set derived from the header once, R33, cached) replaces SCALAR_TYPEDEF_RE.sub for the ISOLATED compile; wired into match_one + p16_permute. Fixes the multi-typedef-LINE skip that discarded 42 masked-MATCH drafts over whitespace. Unblocks B4's func_8015C32C (redefinition of 's16'). - canon_sig_reconcile / eval_lora / format_finetune keep their own copies — migrate per-bank, byte-gated (the audit-prescribed cadence, not a big-bang swap). VERIFIED: - HEADLINE known-answer: func_8015C030 -> MATCH (23 ins) UNEDITED via match_one (was CC1-FAIL; the multi-line typedef split alone fixes it — a live x134-family draft that was being discarded over whitespace). - unit: 7/7 scalars stripped; a local struct KEPT; a TU-provided Blk16 stripped. - classifier unit: DIFF / PLUMBING:… / CC1-FAIL / SKIP all label correctly. - all 5 edited tools import + AST-parse clean. - R22 clean-fleet: check-all 136/136; main clean-rebuild 143dbb89. (A mid-test c4546248 "mismatch" was a stale-incremental artifact from concurrent compiles, cleared by a clean rebuild — the R22 lesson; edits touch only tools/, src/ stayed git-clean.) - SAFETY: a strip bug can only fail-to-bank, never falsely bank (INCLUDE_ASM pastes the original asm; a wrong draft always changes bytes -> always fails SHA1). |
||
|
Drew T
|
ebdef9012b |
feat(phase-27 T2): make the Makefile fail-closed — the enabling fix for every downstream gate
The roadmap §5 asserted `make report` is fail-closed. It was NOT: .ONESHELL sends each whole recipe to one `bash -c`, so with no -e only the LAST command's exit survives and every earlier failure is swallowed. `dedup-check` "gated" purely by being last; lint_symbol_refs / progress --audit / difficulty / dup_report were non-gates. That is the 26-A audit's own thesis (a loud failure nobody counts is as invisible as a silent one) biting the audit's infrastructure — and until it's fixed, any R32 assertion added to a report-invoked tool is swallowed on arrival. - .SHELLFLAGS := -ec (global fail-closed). ONE documented opt-out: check-env (set +e — its contract is accumulate-every-failure-and-report, which -e would truncate at the first missing tool). - check-all:610 grep -c landmine fixed (|| true): grep -c exits 1 on zero matches, which -e treats as fatal in a command substitution -> check-all would FAIL exactly when nothing did. - check-all / extract-all: assert COVERAGE (pass == N), not the absence of a failure marker. The old `fail == 0` / `! grep -q` form was a VACUOUS PASS on an empty pipeline (R32). - new `make tools-health` = audit-corpus + audit-cdecl + report, fail-closed — the deliberate pre-matching ritual the roadmap's standing invariant names, and the dependent the two derived oracles never had (nothing invoked them). NOT a report/build prereq — audit-cdecl cross-compiles every C decl through gcc (~minutes). SETUP §6.3 documents it (R21). VERIFIED: - NEGATIVE CONTROL (the proof): a broken lint_symbol_refs makes `make report` exit 0 under the old .SHELLFLAGS=-c and exit 2 under -ec. The swallow was real, not theoretical. - the grep -c landmine + the vacuous-pass both reproduced and fixed in isolation. - check-env still exits 0 (the opt-out works); recipe sweep found the Makefile already -e-aware (set -o pipefail, explicit || true) — line 610 was the only real hazard. - R22 clean-fleet: make check-all -> 136/136 byte-identical; a forced main re-extract+rebuild drove the full splat->cpp->cc1->maspsx->as->ld->objcopy->check pipeline under -e -> 143dbb89. - audit-corpus 7s / audit-cdecl green / tools-health wired. |
||
|
Drew T
|
6e99157bcf |
chore(phase-27 T3 addendum): widen the .run/giants allowlist — cookbook §45 cited untracked files
Found while reading the seeds for T1: cookbook §45 names
.run/giants/func_80133CD4.fable.c as its worked example and .run/giants/fable_cd4/
as the flagship's gdb oracle — BOTH were untracked. The docs cite artifacts that
were not in the repo.
- .gitignore: widen by FILE TYPE, not directory — .run/giants/*.{c,md,sh} +
fable_cd4/*.{c,md,sh,gdb,txt}. +49 files / 460K.
- Now preserved: the flagship func_80133CD4 crack + its gdb oracle (§45's cited
worked example); the byte-verified pf*.c regression ladder (the seeds' own
Method/reproducibility section cites it: pf2 78, pf_c2 30, pf_d1 35, pf_h1 280);
the dump.sh/mon*.sh RTL harnesses; the banked giants' drafts (80135480, 80163EC8,
80166994).
- Still ignored (regenerable via dump.sh, R33): d_pf*.i.*, *.s, dumps_m*/, and the
ILS/permuter .log files. Negative control re-verified: all 5 probes IGNORED, no
db.*.gbf staged (R23).
Lesson (R31 candidate): a doc that cites a path is an untested claim about the repo.
The §45 citation and its file were 4 days out of sync; only reading the seed for an
unrelated reason caught it. Candidate lint: cookbook path citations must resolve to
tracked files.
|
||
|
Drew T
|
2351c43e66 |
chore(phase-27 T3): preserve the irreplaceable .run/ recon (R20) — 2.2M, not 12.3M
Pulled ahead of T1: the Fable5 sprint's agents work inside .run/, and its Phase-25
seed recons were untracked — an agent overwriting .run/giants/*.opus.c would have
destroyed irreplaceable input. 5 minutes to remove that risk.
- .gitignore: /.run/ -> contents-exclude form (/.run/* + ! exceptions), following the
/tools/bin/*.sha256 precedent. Resolves R20 (commit irreplaceable RE work) vs R12
(.run/ is scratch) by splitting the directory on the real axis: what a rerun CANNOT
reproduce.
- PRESERVED (~2.2M / 31 files): the 6 Phase-25 *.opus.{c,md} giant seed recons (49K);
the func_80178004 gdb-on-cc1 harness + ORACLE_PROOF.md + the v00-v07 draft ladder +
the sched/combine .lst evidence (~110K — the distilled output of a 477k-token Fable5
pass, and the method §52 lever 6 depends on); backlog.jsonl (1.9M) + fuel_manifest.json.
- STILL IGNORED (regenerable, R33): dumps_v00..v07/ and d_pf*.i.* gcc RTL scratch —
12.3M reproducible via runorc.sh + the .gdb scripts; the MCP log; draft scratch.
The plan said "track the dirs"; the bytes said the dirs are 96% regenerable.
- VERIFIED both directions: git add --dry-run stages exactly the 30 intended files and
0 bulk; negative control — ghidra-mcp.log / dumps_v00 / d_pf.i.sched / d_pf.s all
still IGNORED. No db.*.gbf staged (R23 restart-noise).
|
||
|
Drew T
|
002f6d7c7b |
chore(phase-27 T0): Phase Start — plan approved (gate 1) + the P27 verification corrections
- CURRENT_PHASE.md: the approved plan (11 tasks, effort-annotated per R7), the
Planning-verification section, blockers, and the Task-0 log entry
- harness task list built before work (R28); deps enforced T6->T7, T2->T8,
T2+T7->T10, T5->T9, all->T11
VERIFICATION (R14 at planning scale — the roadmap's own §0 mandate): 3 read-only
agents checked every P27 premise against the repo. ~28 specifics corrected. The
three that reshaped the plan:
- the 0x8017BEBC probe ("possibly the largest cheap win left") would fail 112/112
today on a TOOLING defect — extract_unit drops the 8 file-scope gte_* macros the
banked exemplar needs; 0 of 112 member TUs define them. It would have been logged
as a 4th 0% exhaustion probe: the 26-A audit's thesis, about to recur.
- `make report` is NOT fail-closed (roadmap §5 asserts it is): .ONESHELL + no -e in
.SHELLFLAGS => only the last command's exit survives; lint_symbol_refs/progress
--audit/difficulty/dup_report are swallowed. check-all/extract-all assert fail==0,
not pass==N => an empty pipeline is a vacuous pass.
- 4 code-bearing SC07 payloads (~2.45 MB, 98.0% plausible-opcode) are invisible to
every tool: they sit at PAC entry index 1 while new_overlay.sh hardcodes 0.4.dec.
Zero mentions in docs/ or config/.
DROPPED with reasons: 0x8013C414 (x134 contested by an explicit verified_reach:1;
already drafted) · func_801549F8 (from the SUPERSEDED megaplan; Phase 26 walled it
17/31 and wrote "do NOT hand-grind it"; inside the 0/958 re-gate) · func_8012E364
(the "stale closeness" label is itself stale — close=23) · jtbl_carve "fix first,
load-bearing for B5" (the audit measured it twice: 0 of 5043 jtbl ends differ).
B4 dissolves into T4 — its remedy already ran (A9b, 1/7) and the 1 is banked.
Owner decisions (Drew, 2026-07-15): curated .run/ preservation (R20 vs .gitignore —
every Fable5-sprint input is currently untracked) · full disc audit incl. the
type-sweep, accepting the denominator expansion · Fable5 in 2 waves, distill between.
|
||
|
Drew T
|
1bbab78c65 |
feat(phase-26): close — family engine + the tooling-integrity audit + the §52 discovery-flywheel; mechanical harvest byte-proven exhausted; fleet 58.2->68.9% instr (v1.25.0)
- FAMILY ENGINE (Tasks 1-6): family_remap (extended reloc tracker + single-pass subst) + family_hseq (the h_seq reframe) + family_sweep (crack-one -> template-x134 -> byte-gate) + canon_sig_reconcile v3.2 + rtu_match. Sessions 6-8 cracked 13 cores (58.2->66.5% instr). - 26-A TOOLING-INTEGRITY AUDIT (inserted half-phase, A0-A11): the tools WERE several of the walls. Fixed ~15; DELETED decaying scanners (R33); built corpus.py + cdecl.py (derived, coverage-asserted oracles) + make audit-corpus (a SECOND, disagreeing oracle, R34); the listCdBuffer 193-slice corpus defect -> 0; masked_diff 150 closeness-lies -> 4; the stale- object false-pass closed. Payoff 66.5->68.6% instr. docs/tooling-audit.md AUDIT-CLOSE LEDGER. - 52 DISCOVERY-FLYWHEEL (Task 7): single Fable5 on func_80178004 = intrinsic 3-integer regalloc wall, BUT distilled the walker-family idiom (52); two cheap-Opus waves applied it -> 5 pin-free cores banked x134 = 670 instances (68.6->68.9%). 4 named wall classes; 52/52a/52b. pin-guard comment false-positive fixed; family_sweep --allow-pins. - FINDING (R14/P9, 3 probes 0%): the matched-sib mechanical harvest is EXHAUSTED; the manifest's ~13k "templatable" members are an h_seq prediction the byte-gate refuses. Phase-26's templating thesis is spent -> close, open Phase 27 with byte-gate-honest re-scans. - R22 clean-fleet 136/136 BYTE-IDENTICAL throughout; dedup 1840/0; 0 NON_MATCHING (G4). - rules R32 (coverage assertion) / R33 (derive, don't re-derive) / R34 (a second, disagreeing oracle). worklog -> phase-ends/logs/Phase26.md (R19). bumps 1.24.0 -> 1.25.0. |
||
|
Drew T
|
a33c6f85f5 |
docs(phase-26a): A11 — distill + close the tooling-integrity audit (26-A COMPLETE)
Closed the inserted half-phase. tooling-audit.md: DIAGNOSIS -> AUDIT-CLOSE LEDGER (A1-A10 outcomes + the payoff 66.5->68.6% instr + remaining/handoff); the "two rules" -> R32/R33/R34 crisp for P10 ratification at the Phase-26 PhaseEnd. decision-log: the A10 wall-re-test verdict (R31 -- the broken tools WERE the walls; the payoff was banked by the fixes; the closeness-0 residual is genuine; the real deliverable is the 3 rules + the derived-oracle pattern). SETUP: the A9d-A10 tool changes (R21). Cookbook §51 verified complete; LAW 3 tagged R34. Observables green: final R22 clean-fleet 136/136 BYTE-IDENTICAL; make report EXIT 0 (dedup 1840/0, C1 227211/227211 signed, lint_symbol_refs wired + passing); audit-corpus 0 slices; audit-cdecl green. Zero src/config changes this session. Phase 26 resumes at Task 7 (fresh session). |
||
|
Drew T
|
e9038a04ea |
docs(phase-26a): A10 COMPLETE — wall re-test verdict on all 5 walls
The audit thesis is CONFIRMED: the tooling-walls were dissolved by the FIXES and the payoff banked there (A3f/g/h + A9a/b, 66.5->68.6% instr), while the re-tests confirm the residual walls are real. - fuel/closeness-0: CONFIRMED REAL (0/958 bank at fleet scale). - arity (Phase-15 dead-end): was tooling (A3c order-dependent rule); 13/18 banked. - def-side loose-typing: was partly tooling; A9b banked func_8017A4AC x134. - type-heavy: blocking tool build_engine_types was broken (A7 fixed it, now runs); the ~1,200-member family harvest is Task-8 integration, not a pure re-test. - 780 h_seq callee-oracle rejections: was tooling; A3h banked +2,675. A re-confirmed wall is as valuable as a dissolved one (P9). Next: A11 close. |
||
|
Drew T
|
b1c58d7668 |
docs(phase-26a): A10 wave 1 — closeness-0 wall re-test CONFIRMED REAL (0/958 bank)
Re-gated all 958 closeness-0 open-stub backlog drafts through the FIXED gate across 135 binaries in parallel: banked=0, near=957, failed=71. The closeness-0 backlog is genuine whole-binary near-misses, NOT tooling misses -- match_one's isolated closeness==0 systematically overstates whole-binary bankability, and the repaired gate recovers none. P9: a re-confirmed wall is as valuable as a dissolved one. (The audit's tooling-walls were already banked by A3f/g/h + A9b, +2.1% instr.) backlog.py: env-gated BACKLOG_NO_RENDER so parallel workers skip the render race (append is atomic) -- backward-compatible parallel-safety. backlog.md refreshed with the re-test's whole-binary-informed scores. |
||
|
Drew T
|
f2b2a9d778 |
docs(phase-26a): A10 Max-phase — wall re-test scoping + arity measurement (fan-out pending R27)
Scoped the 5 walls. The audit fixes already dissolved the easy wall (A3f/g/h + A9b, 66.5->68.6% instr). Remaining re-test pool = 2,218 open-stub backlog drafts (958 at closeness 0). #2 arity: 13/18 Phase-15 arity-conflict fns already banked (wall largely fell). Confirmed the remaining re-test is breadth (serial gate_stage timed out). R27 boundary reached: prompting Drew for /effort ultracode before the parallel re-gate fan-out. Tree clean (HEAD commit:0618 before this). |
||
|
Drew T
|
ea20bdf9f3 |
fix(phase-26a): A9g — jr_inventory: retire the ephemeral roster, derive banked from the image (R33)
jr_inventory's `banked` set was filtered by an EPHEMERAL, gitignored .run/banked_func_*.json roster: a `rm -rf .run` / fresh clone would blind ALL banked jr at once, cross-address siblings (roster named after the exemplar) were structurally invisible, and non-leader banked jr were missed. "The purest R33 case in the group" (audit). FIX (the audit's exact prescription): delete the roster glob + `cand` filter; `banked` is DERIVED FROM THE IMAGE — a real-C def/define fn is a banked jr iff family_remap.reloc_targets shows it references a committed .rodata carve offset (config + image, both durable; cross-address- and non-leader-immune). R32 assertion: every committed carve must resolve to EXACTLY ONE owner or abort (a stranded/duplicated carve is the §8b func_801734BC incident, never silent). Also fixed the adjacent finding: the asm_jr scan's func_-fullmatch dropped the curated-name listCdBuffer jr; now resolved via oss.addr_of(). (The --only path's own fullmatch is left — it parses user input, not the corpus.) Perf: read the overlay image ONCE and pass it to reloc_targets(..., data=) — a new backward-compatible param on family_remap (regression: 0/80 mismatch vs the re-read path). Verified: data-param behavior-identical; the R33 win — ov_SC02_000 now finds the cross-address sibling func_8017FCB0 the roster missed; full-fleet parallel run = 134/134 OK, 0 false aborts, 1336 banked jr == 1336 carves -> 1:1 ownership holds fleet-wide. Byte-safe: jr_isolate_all is not in the make build/extract path (R22-neutral); the change makes future isolations strictly more correct. |
||
|
Drew T
|
96e025a324 |
fix(phase-26a): A9f — overlay_src_split swallowed 2 real definitions; the selftest was blind
scan_construct's force_decl latched from the FIRST token and returned at the
first depth-0 `;`, so a definition sharing a physical line with leading externs
(`extern A; extern B; void f(){...}`) was never anchored — absorbed into the
next anchor's preamble. The parser jr_isolate_all rewrites source from was short
two functions in the exemplar overlay. The round-trip selftest is a SERIALISATION
check (a miss lands in a preamble -> round-trip still exact BY CONSTRUCTION), so
it was structurally incapable of seeing this.
FIX (byte-safe): force_decl no longer survives a same-line `;` with trailing
code — re-classify from the remainder and keep scanning so the def anchors (its
leading externs stay in its whole-line item text -> round-trip byte-identical).
Rejected the audit's "split into 3 constructs": round-trip joins whole-line
chunks with `\n`, so sub-line splitting would insert a newline where a space was.
def_name now names the LAST top-level header before `{` (the definition, not the
first same-line extern; byte-identical on every single-def construct).
R32: hidden_definitions() coverage oracle wired into selftest — an independent
detector of `func_XXXX(...){` bodies not anchored. The selftest is now a coverage
check, not just serialisation.
Verified: 2 swallowed -> 0; regression over 1738 overlay .c = 0 round-trip fails,
0 non-monotonic, 0 non-additive changes, +2 anchored defs. Byte-safe: tool not in
the build path (R22-neutral); ov_SC01_077 rebuilds d19c9580; neither def straddles
a committed subseg boundary. Audit ledger line refs were stale (src rewritten);
real cases are ov_SC01_077_after.c:2020 + ov_SC01_077_jr_8015444C.c:1495.
|
||
|
Drew T
|
68d29ba8af |
fix(phase-26a): A9e — reconcile_tu already wired into bank_exemplar (A3d); document the ladder
NULL RESULT (P9/R14): the session-12 "wire reconcile_tu into bank_exemplar"
handoff item was stale — A3d (commit:0601) already wired reconcile_tu into
jtbl_family_bank.recover() ("on BOTH banking paths"), and bank_exemplar's
`recovered` stage delegates to fb.recover = cast_call_sites + reconcile_tu.
Proven working by A9b (func_8017A4AC banked at the recovered stage, reconcile_tu
resolving its struct + fn-ptr conflicts). No live tool references the RETIRED
reconcile_decls (only docstrings + the audit-cdecl differential harness).
No code change warranted. Byte-neutral hardening only: document the
raw/scoped/recovered/reconciled fallback-ladder composition inline in
bank_exemplar so a future session does not re-run this "is it wired?" trace.
|
||
|
Drew T
|
40477281ce |
fix(phase-26a): A9d — retire the dead Phase-17 canonical-sig chain (R33)
DELETE tools/census_conflict_callees.py + tools/derive_canonical_sigs.py. - census_conflict_callees: audit-CONFIRMED marked-for-deletion (commit:0593; decision-log 836). It re-derives from C text the per-TU "defined/declared/ stubbed/external?" question that reconcile_tu (Phase 26) answers FROM THE BUILD — and does it WRONG in the unsafe direction (unknown -> conflict-free). - derive_canonical_sigs (census's ONLY consumer): genuinely dead — last touched Phase-17 (commit:0140), output .run/canonical_sigs.json read by nothing (no Makefile/workflow/import), no-ops on the 2-byte [] input, asm-arity heuristic 36% wrong vs byte-exact banked C. Its purpose was retired in A3d (fleet-majority oracle -> reconcile_tu's per-TU oracle). Deleting census orphans it, so the whole dead chain ceases to exist (R33: the best outcome is a DELETED SCANNER, not a fixed regex). Byte-neutral by construction (neither tool is in any build/report path): module-import smoke over the 13 importable harvest/bank/report/reconcile tools = all clean; bank_exemplar is a run-only script (indexes sys.argv at module scope), imports neither deleted module. No src/config change -> no byte moves. Doc-pointer hygiene: hand-matching-process.md 8a, matching-cookbook.md (canonical-sig-layer entry), tooling-audit.md (ledger row + derive entry) all annotated DELETED/historical so nothing points at a nonexistent tool. |
||
|
Drew T
|
181191b6af | docs(phase-26a): log A9c (lint_symbol_refs green + wired into make report) | ||
|
Drew T
|
49004f5b06 | docs(phase-26a): log A9a (canon_sig_reconcile fn-ptr fix) + A9b (func_8017A4AC ×134 wall re-test) | ||
|
Drew T
|
959cc9b7c1 | docs(phase-26a): log A3h — standing-lead harvest (measured, banked +1.3% instr, Bucket X honest-stop) | ||
|
Drew T
|
7a3921a3f3 | docs(phase-26a): log A3f+A3g (harvest + propagation) + disk-hygiene note | ||
|
Drew T
|
b89fcc2edc |
fix(phase-26a): A3e — gate_stage pinned the byte-gate back to 4.9%, OF A3'S OWN FIX
THE WORST DEFECT IN THE AUDIT IS NOT IN A SCANNER. It is one default argument in the CALLER of a
scanner we had already fixed.
# tools/gate_stage.py:315
summary = run_gate(a.drafts, binary=b, src=a.src or f"src/{b}/{b}.c", ...) # ALWAYS the main .c
`src` RESTRICTS the byte-gate to ONE translation unit, and _gate1 does `if src: cmd += ["--src", src]`
-- always truthy. A3 had just taught harvest_verify to DERIVE each draft's home TU *when --src is
omitted*, lifting the byte-gate's reach from 4.9% to 100%. gate_stage NEVER OMITS IT. The fix was
neutralised by its own caller's default, and the PRIMARY BANKING PATH -- every wave, the grinder, the
orchestrator, bulk_harvest -- remained structurally unable to bank 250 of ov_SC01_077's 263 stubs.
WHY IT SURVIVED 26 PHASES: harvest_verify cannot splice a draft whose stub is not in the TU it was
pointed at, so the draft never verifies -- and is then logged as near/failed, i.e. AS A MATCHING
PROBLEM. The wave reports a poor close-rate; the function goes to the backlog as a compiler residual.
A tool that CANNOT bank a function is indistinguishable, in every log this project keeps,
from a function that CANNOT BE banked.
PROOF, same draft / same gate / same second: gate_stage rejected func_80129C40; harvest_verify run
directly (no --src) VERIFIED it byte-identical and banked it.
AND A COUNTING BUG THAT HID THE HIDING (gate_stage:261): when match_one says MATCH but the whole-binary
gate rejects, the record is logged status="near" and THE COUNTER IS NEVER INCREMENTED. A 63-draft run
printed `banked 0, near 0, failed 0` -- three zeros that do not sum to 63 -- for phases. Nobody ever
added them up. (The number was not wrong. It was ABSENT.)
ALSO FIXED, sig_unify (the same disease, one level down): it SILENTLY DROPPED 190 of 196 drafts (97%).
`cur_stubs` was read from the main .c (13 of 263 stubs), so any draft whose stub lives in a _jr_ carve
hit `if fn not in cur_stubs: continue` -- dropped BEFORE THE WRITE: never copied to --out, never gated,
never logged, while the summary printed "drafts unified: 6" and read like success. THIS IS GATE_STAGE'S
STAGE-2 RECOVERY -- the pass whose whole job is to rescue the stage-1 failures -- and it has been a
no-op for nearly every draft it was meant to save. Now: TU derived per draft (corpus.stubs), canon
derived from cdecl.tu_scope (cpp -- macro-injected decls finally visible), and _keep() so an
already-acceptable decl is left alone (the §19 "sig_unify regresses canonical drafts" failure mode).
Reach: 6 -> 196 drafts; callee-externs rewritten 2 -> 90; own def-sig 2 -> 86.
MEASURED, all three consumers migrated (196 never-banked drafts):
near 5 -> 116 failed 190 -> 17
=> 173 of 190 "failures" were PLUMBING, not codegen: now compiling and SCORED instead of invisible.
THE PRIZE (measured, not claimed): the backlog holds 1,588 entries at closeness==0 -- body byte-exact
per match_one, whole-binary gate rejected. 1,215 have been banked since by other paths. 373 ARE STILL
OPEN STUBS WHOSE BODIES ARE ALREADY BYTE-EXACT, sitting in a ledger that calls them unrecoverable.
⚠ THE HARVEST ITSELF IS NOT IN THIS COMMIT, AND IS NOT CLAIMED (P9). Gating the 63 ov_SC01_077 ones
dragged `dedup_propagate --auto-from --recover` behind it; it ran >1h and hit its timeout -- its
first-ever run over the FULL corpus (A6/A7 unblocked the 407 files it could never see). It MUTATES THE
TREE BEFORE IT GATES, so the kill left 859 files + engine_core.h (+544 lines) written and UN-GATED with
the registry never updated. R22 on that tree: 44 passed / 92 FAILED -> `git checkout -- src/ config/`,
fleet restored to 136/136. Nothing lost (H4: the tree was clean, so the revert was one command).
Two real lessons, recorded: dedup_propagate is NOT crash-safe and must never run under a timeout it can
hit; and a 63-draft experiment must not drag an unbounded fleet-wide propagation behind it.
R22 clean-fleet after revert: 136 passed, 0 failed of 136. src/ and config/ clean.
cookbook §51g LAW 11: A FIX IS NOT LANDED UNTIL ITS CALLER STOPS OVERRIDING IT. After fixing a
scanner, grep every call site and ask whether a caller's default re-disables it. An audit that stops
at the callee is half an audit.
|
||
|
Drew T
|
4aae5e7589 |
fix(phase-26a): A3d — retire the fleet-majority oracle: it was WRONG for the TU 16% of the time, on both banking paths
R33 applied to the worst finding in the audit: this oracle was not fixed, it was RETIRED.
reconcile_decls asks "what does the FLEET call this symbol?"
C asks "what does THIS TRANSLATION UNIT declare?"
The engine is loosely typed -- the same address is legitimately declared with incompatible types in
different overlays -- so a single fleet-wide answer is WRONG FOR SOME TU BY CONSTRUCTION. And it is
worse than a silent skip: it writes an ACTIVELY WRONG declaration into the draft, which then
collides with the very TU it was meant to conform to.
MEASURED across ov_SC01_077's 12 TUs, against what cpp says each TU really declares:
the fleet oracle AGREES with the TU ................ 2883
the fleet oracle CONFLICTS with it (cc1 REJECTS) .. 548 <- 16%
the TU declares it, the oracle has NO answer ...... 357
and it was LIVE ON BOTH BANKING PATHS:
* gate_stage -- rewrote 60 of 196 drafts in the current batch
* jtbl_family_bank -- EVERY SIBLING of the ×134 family sweep, the project's economic engine.
A poisoned decl means that sibling silently does not bank, and the loss is invisible: the sweep
simply reports a smaller number. The irony is exact -- that function's own docstring already
knew the conflicting symbols are PER-OVERLAY, which is precisely why a FLEET oracle could never
have been right.
reconcile_tu.py (written in Phase 26 but NEVER WIRED) now supersedes it, rebuilt on cdecl:
* ask cpp what the TU declares (macro-injected DEFINE_func_* externs included -- a raw scan
cannot see them, §8c / §51g LAW 7);
* ask cc1 whether the draft's decl can coexist (cdecl.compatible, validated against the real
gcc-2.7.2 front end on 1,485 live pairs -- NOT the C standard, NOT modern gcc; §51g LAW 9);
* NOT declared -> leave the draft alone (its extern types are load-bearing: %lo-folding, access
width, alignment); compatible -> nothing; CONFLICTING -> the TU wins + cast at every USE so the
draft's intended access survives byte-for-byte;
* derives WHICH TU from corpus.stubs() rather than a hand-passed --src-file (§51g LAW 10).
* handles the fn-ptr kind NATIVELY -- which is why it supersedes rather than patches: teaching
reconcile_decls' parser to see `extern void (*D_x[])(void);` would have ARMED its fn-ptr-blind
data_access_subs to rewrite a call-through `D_x[i]()` into `((u8 *)D_x)[i]()`. Fixing the regex
would have detonated a dormant bug.
AND THE NULL RESULT, AGAIN, REPORTED AS SUCH (P9/R14): on the 196 never-banked historical drafts the
new oracle banks EXACTLY AS MANY AS THE OLD ONE -- zero. That tail fails on CODEGEN, not on decl
plumbing. The two disagree on 45 of 196 drafts and the outcome does not move. This is a CORRECTNESS
fix (548 wrong declarations removed from two live pipelines, protecting all FUTURE drafts and every
future family sweep), not a banking win, and it is not being sold as one. Three nulls in one session.
reconcile_decls.py is kept as EVIDENCE, marked RETIRED, with no live caller.
R22 clean-fleet: make clean + extract-all + check-all -> 136 passed, 0 failed of 136
src/ untouched (0 changes) reconcile_tu: 0 coverage defects over 196 drafts
NOTE: the family-sweep path gets its real exercise at Task 8 -- watch the per-sibling bank rate.
|
||
|
Drew T
|
f4502f11bb |
fix(phase-26a): A3c — the recovery passes were reconciling 95% of drafts against the WRONG TU
FIRST CONSUMER MIGRATION onto the cdecl oracle — and the compiler taught me two things I had
wrong, one of which reopens a wall that has been closed since Phase 15.
1. cdecl.compatible() — "will cc1 accept these two declarations of one name?"
The predicate four tools each half-implement and get wrong: norm_sig / _norm_type collapse the
int family to ONE token, so a SIGNEDNESS change reads as "already compatible" and gets no
rewrite -- while cc1 REJECTS that redeclaration. Right about codegen, wrong about the front end,
which never reaches codegen.
2. THE ADJUDICATOR MUST BE THE COMPILER THAT COMPILES YOUR CODE (cookbook §51g LAW 9).
I wrote the rules from the C standard, then let a compiler judge. It contradicted me -- and then
the RIGHT compiler contradicted the first one. Three different answers:
declarations in one TU | standard | modern gcc | gcc-2.7.2 cc1
typedef int X; twice | error | ACCEPTS | ERROR
extern u16 X; + volatile u16 X| error | error | ACCEPTS
void X(s16); then void X(); | error | error | ACCEPTS
void X(); then void X(s16)| error | error | ERROR
--compat now adjudicates with tools/bin/gcc-2.7.2-psx/cc1, the front end that actually
arbitrates the build: 1,485/1,485 live corpus pairs agree, 0 disagree, 0 skipped.
3. THE PRIZE: the Phase-15 narrow-param wall rests on a false premise.
The no-prototype rule is ORDER-DEPENDENT. `void X(s16); void X();` COMPILES; only the reverse
fails. Phase 15 closed "the 159 arity/narrow-param conflicts" as "no clean deterministic fix --
it is simply C's default-promotion rule". cc1 does not enforce that rule in the direction the
wall assumed. Four three-line probes, 90 seconds, zero tokens. -> A10 RE-TEST TARGET.
Probe the compiler for FACTS; read its source only for LEVERS; byte-validate both. (We read
gcc-papermario for five phases believing it was 2.7.2. It was 2.8.1.)
4. THE MIGRATION: cast_call_sites canonicalized 95.1% of drafts against a TU that would never
compile them. `--src-file` is an OPTIONAL HAND-PASSED flag defaulting to src/<ov>/<ov>.c, and no
caller knows about the Phase-26 _jr_<ADDR> carves: ov_SC01_077 has 263 open stubs across 12 TUs
and only 13 are in the main .c -- while harvest_verify (A3) correctly splices into the real one.
Now DERIVED from corpus.stubs() (the INCLUDE_ASM line is self-describing), with the canonical map
derived from cdecl.tu_scope() (cpp -- so macro-injected DEFINE_func_* decls are finally visible).
Callee-conflict repair reach: 8 -> 58 of 196 drafts (7x).
5. AND THE NULL RESULT, REPORTED AS SUCH (P9/R14). Those 58 banked ZERO functions. The historical
draft tail fails on CODEGEN, not plumbing -- func_801387B8, which the audit blames on a single
unparsed `[4]`, is really 67/100 instructions off with a $s0/$s1 swap (that claim does not
reproduce on today's tree). The real gain is narrower and still worth having: 52 drafts moved
from "won't compile" to "compiles, N instructions off" -- from an INVISIBLE failure that reads as
a compiler wall into a SCORED near-miss the permuter and the §47/§48 dials can act on. That is
the audit's thesis, not a bank. THREE times in one session a confirmed mechanism produced a null
consequence.
Also: my own new audit printed "ALL ORACLES GREEN" while silently skipping 100% of its corpus (a
missing -Isrc). The exact bug class, in the tool written to hunt it. An unadjudicable check is not
a passed check.
R22 clean-fleet: make clean + extract-all + check-all -> 136 passed, 0 failed of 136
src/ untouched (0 changes) make audit-cdecl: green --compat: 1485/1485
NEXT: sig_unify + reconcile_decls carry the SAME wrong-TU bug (same --src-file flag).
|
||
|
Drew T
|
f9742cf9c0 |
feat(phase-26a): A3b — cdecl.py, THE C-declaration oracle: one grammar, fifteen deleted models
Fifteen tools each carried their own regex model of "what is a C declaration", and they
disagreed — two tools in ONE pipeline disagree today about whether `extern s32 D_a, D_b;`
is a declaration at all. All fifteen shared one character class,
extern\s+([A-Za-z_][\w\s\*]*?\bD_[0-9A-Fa-f]+\s*(?:\[\s*\])?)\s*;
which cannot hold '(', ',', or a non-empty [N] — so three whole shapes were invisible to
every one of them: fn-ptr/jump-table arrays, sized arrays (one unparsed `[4]` has blocked
func_801387B8 in 134 TUs), and multi-declarators (the WHOLE line dropped, not just #2..N).
REJECTED the audit's own prescription (a shape-aware alternation per tool, ~15 coordinated
regex edits) on R33 grounds: fifteen hand-maintained models are exactly what diverged, and
an alternation only ever covers the shapes somebody remembered. The thing being scanned HAS
A GRAMMAR. C's declarator grammar is small, closed and TOTAL — it describes fn-ptr arrays,
sized/2-D arrays, multi-declarators, fn-ptr params and K&R identifier-lists without being
told they exist. ~250 lines of recursive descent: LESS code than the regexes it deletes, and
exhaustive by construction rather than by memory. (decision-log 2026-07-14.)
Two statement paths, because the inputs genuinely differ:
* tu_statements() - a TU's file scope, derived from cpp. A decl inside a DEFINE_func_*
macro body declares NOTHING until the macro is invoked (the §8c law);
a raw scan is wrong in both directions. cpp answers it exactly, in
54 ms/TU (~20 s for the fleet, cacheable).
* split_statements() - span-preserving raw split, for drafts (which get rewritten).
THREE ORACLES, whole corpus — a measurement, not a belief:
* coverage 2,952,246 depth-0 statements -> 2,731,521 declarators, 0 PARSER DEFECTS
* the real gcc 50,405 distinct declarations compiled beside this parser's reconstruction
of each one -> 0 REJECTED
* differential 0 file-scope symbols the incumbents see that cdecl misses; 26 in
engine_core.h they cannot see; 6 they wrongly promote from BLOCK scope
Two ideas worth keeping (cookbook §51g, LAWS 4-8):
* THE CANDIDATE SET IS DERIVED TOO (R33 applied to R32). At file scope C admits nothing but
declarations, so R32's over-approximating detector is *every depth-0 statement* — supplied
by the grammar, with no hand-maintained candidate regex to rot.
* GCC ADJUDICATES MY OWN COVERAGE GAP. Deciding for myself which failures "don't count" is
grading my own homework — the habit that wrote the fifteen bugs. A statement gcc ALSO
rejects is not C (my rejection is correct, the INPUT is corrupt); one gcc ACCEPTS and I do
not is MY defect. All 33 residual: NOT-C, all dead .run/drafts* scratch, none in src/.
NEW findings (docs/tooling-audit.md):
* reconcile_decls.DATA_DECL_LINE_RE finds ZERO decls in engine_core.h — it is line-anchored
and every decl there ends in a '\'. Its "authoritative tier" has ALWAYS been empty.
* gen_harvest_targets + sig_unify count BLOCK-SCOPE externs (6, byte-proven inside a macro's
function body) as file-scope canonicals — the §8d `conflicting types` confusion.
* tu_ambient's func regex ([^()]* params) drops ANY callee with a fn-ptr parameter.
* R14 near-miss: 33 drafts contain `extern if ((func_80029178(0x119) & 0xFF) != 0);`, written
by a RECOVERY TOOL — but the source bug was already fixed in Phase 19 (0 garbage / 300 sigs
today). Mechanism confirmed, consequence nil. Note what it cost while live: a draft that
cannot compile fails the byte-gate and reads downstream as an INTRINSIC COMPILER WALL.
Bugs the oracles caught in ME (and would otherwise have shipped): `extern s32 (*D_801274D0)(s32);`
parsed the BASE TYPE as the name; a K&R declaration-list flushes as SEVERAL spans, so the body
attached to the wrong one and leaked the K&R parameter names into file scope as fake globals.
SCOPE, deliberate: NO consumer is migrated here, so this cannot move a byte. The audit warns
that making the parser see more ARMS dormant transforms (reconcile_decls.data_access_subs would
mangle `D_1[i]()` -> `((u8 *)D_1)[i]()` the moment fn-ptr decls become visible to it). Migration
is one tool at a time, each byte-gated.
R22 clean-fleet: make clean + extract-all + check-all -> 136 passed, 0 failed of 136
make audit-corpus: 0 PHANTOM + 0 TRUNCATED make audit-cdecl: ALL ORACLES GREEN (new gate)
|
||
|
Drew T
|
c7772bc452 |
docs(phase-26a): SESSION-9 CLOSE — cookbook §51 (the tooling-integrity laws) + handoff
R30/R16: the context-dependent artifacts, written while the context is live.
cookbook §51 — the SILENT SKIP: the bug class, why the byte-gate cannot see it, the
over-approximating-detector method, and FOUR LAWS:
1. Derive, don't re-derive — the best outcome is a DELETED SCANNER (28 findings -> one
derived oracle + ~10 deleted scanners). A derived fact cannot rot; a hand-maintained
copy of it is a liability that grows with every structural change.
2. Assert your COVERAGE, not merely your correctness. *** A LOUD FAILURE THAT NOBODY
COUNTS IS EXACTLY AS INVISIBLE AS A SILENT ONE *** — build_engine_types printed
'[overlap] handle manually' every single time for four phases while dead on 81% of its
own corpus. This CORRECTS the first draft of R32 ('fail loud'), which was not enough.
3. When an oracle is structurally blind to a class of error, add a SECOND ORACLE THAT CAN
DISAGREE WITH IT — not a better assertion inside it. We had two all along and never made
them argue. (And scope the comparison to where the second oracle is genuinely independent:
the same check run outside its domain reports 914 slices when the truth is 193.)
4. A rule that needs a human to remember it is not a gate. Make it structural.
+ the FALSE-WALL PIPELINE (a silent skip -> a wasted draft -> a backlog 'matching failure'
-> reserved_walls() PERMANENTLY blacklists a function that was never attempted), and a
checklist for any new corpus-scanning tool.
CURRENT_PHASE: session-9 handoff — what is done, what remains (each with its spec on disk),
and the R32-corrected / R33 / R34-new rule candidates for P10 ratification.
|
||
|
Drew T
|
a2a507d079 | docs(phase-26a): A4/A5 + the stale-object false-pass hole recorded |