mirror of
https://github.com/Druthulu/BFM-decomp
synced 2026-09-27 05:56:00 -04:00
8d8afe6b44e1771c19f8af48dcc8d7ebdd923576
254 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Drew T
|
ba0cfb751e | fix(workflow): wave_binary drafters MUST copy the winner back to draftDir/<fn>.c (a wave's 24 winners were written under scratch names and recovered only from agent transcripts) | ||
|
Drew T
|
e393c320e6 |
feat(phase-29): VARIANT camps -> UNIQUIFY (not reconcile); validated on Buf, R22 140/140 (§64a)
MEASUREMENT CORRECTED THE PLAN. The checkpoint called for a "per-camp field-access reconcile";
measuring the camps refutes that: Vec8 = {s32 w[8]} (32B) in 180 files AND {s16 unk0..} (8B) in
139 files; MATRIX 48B/32B/32B; Buf 16B / 0x20+ / DrawEnv. These are DIFFERENT types sharing an
identifier across TUs of the same overlay — reconciling to a canonical layout MERGES them, the
same failure that broke 103 binaries on Prim. The right op is UNIQUIFY: rename the non-majority
camp (byte-neutral — a type name emits no code; TU-local by construction), which makes every camp
single-def and liftable by the existing lift_types rules.
- NEW tools/uniquify_type.py: deterministic camp ordering (file-count desc, then normalized text,
so re-runs assign the same suffixes); majority keeps the name, camp n -> <T>_c<n>; rewrites ONLY
files that DEFINE that camp (a file that merely USES the name gets it elsewhere and is untouched);
\bT\b word boundaries so `Buf` never matches `Buf80153978`.
- VALIDATED on Buf (578/6/1 files): 11 identifiers across 7 files -> 3 camps LIFTABLE -> lifted
(585 local copies stripped) -> R22 140/140 -> blocked core queue 13 -> 11 (0x8012ea90, 0x801749c8
freed). Propagated 0x8012EA90 ×138; 0x801749C8 dropped (straggler in ov_SC07_006).
- YIELD, HONESTLY (P9): ZERO new matched functions. fn-count 88.61% / instr 79.3% / stubs 40281 all
UNCHANGED; dedup 1867->1868, C1 +138. 0x8012EA90's members were ALREADY matched in all 138
overlays — the propagation consolidated duplication into one shared macro (DRY), not coverage.
The value is the PROVEN RECIPE + the queue moving 13->11, not the numbers.
- dedup_propagate (R32): the skip line printed a COUNT and no names, and aggregated three unrelated
causes into n_local — a body skipped merely for a `//` comment (macro-unsafe, 1-line fix) read
identically to one genuinely using an overlay-local type. Now named and split by cause.
- cookbook §64a (uniquify-vs-reconcile + the validated recipe + remaining camps by cost).
|
||
|
Drew T
|
5ab80517de |
feat(phase-29): broad §20 type-lift lands — 154 types fleet-wide, R22 140/140 (§64)
The 3-session-carried blocker ("needs collision-vetting + -O0 strip precision") was
misdiagnosed on all three counts; fixing the instruments first (R35) changed every answer.
- (1) the "case-variant collision" is a TAGGED TYPEDEF counted twice with OVERLAPPING spans:
the inner span starts at `struct` so it emits a VARIABLE definition, the alias is redeclared,
and the highest-first strip leaves the outer end offset STALE -> over-deletes past the span.
MEASURED 13 pairs / 6142 occurrences / 0 standalone tags. build_engine_types.resolve_type_defs()
is now the ONE shared model (R33) + assert_disjoint() enforces span disjointness (R32).
The case-insensitive exclude was a heuristic over a structural fact — it would also have
wrongly dropped the legitimate Obj/obj + Vec/vec pairs. Key by (kind, name) — the C namespace.
- (2) the "-O0 strip precision" bug is a VISIBILITY bug: ov_SC01_077_o0.c is the 1 TU of 3226
that deliberately omits engine_core.h, so the strip DELETED its types; `multiple definition of
D_801DAA08` was 3 steps downstream (undeclared -> parse error -> implicit int -> tentative def
-> link collision) and named a symbol no diff touched. bet.type_visible() derives the visible-
header set from the include graph and keeps such defs local, named.
- (3) a third blocker, introduced this session and caught by R22: --candidates classifies per
ENTITY but emits per NAME, so passing `Prim` dragged in the deferred VARIANT `typedef Prim` and
repointed 103 overlays at the header's different layout. Compiled clean, per-binary pre-filter
green, R22 37/140 — the 103 failures were EXACTLY the 103 Prim-stripped overlays (set equality).
Fixed by the strip invariant "remove a local def only if what becomes visible is TEXTUALLY
IDENTICAL", placed at the MUTATION so a selector bug cannot reach the source.
- lift_types.py: (kind,name) entity keying, --candidates derived selector (retires the ad-hoc
102-type pipeline), divergence + visibility reports, whole-line strip (411 whitespace-churn
lines -> 0), complement-based single-pass rewrite (no mid-loop offset mutation).
- RESULT: 154 types lifted, 2958 files stripped, engine_types.h +510 lines. R22 clean-fleet
140/140 BYTE-IDENTICAL; tools-health OK (corpus 0/0, cdecl ALL GREEN, audit-binaries 140,
dedup-check 1854/0, C1 235170/235170); 0 NON_MATCHING (G4). Metrics unchanged 79.0/67.6/88.38
— honest: a type-lift banks no functions, it unblocks the NEXT propagate.
- DEFERRED + NAMED: 8 VARIANT entities (MATRIX 3-def, Buf 3-def, Vec8, Prim, Handler, Blk8, V8,
Prim_8016E7C8) = the per-camp field-access reconcile, the remaining hard part of roadmap B4;
14 carried tags; 5 types kept local in the -O0 TU.
- cookbook §64 (the three laws + the pre-filter lesson: a pre-filter is evidence ONLY about what
it filtered — ov_SC01_077 passed the Prim-broken run too; pre-filter on a binary that FAILED),
decision-log (R31), CURRENT_PHASE.md log.
|
||
|
Drew T
|
f62d3bcf37 | feat(phase-29): lift_types.py topo-sort (dep ordering) + broad-lift finding (needs collision-vet + -O0 strip precision; clean 2-type lift stands) | ||
|
Drew T
|
b921c00912 |
feat(phase-29): lift_types.py — fleet-wide §20 type-lift; Mat32+Cam8012E138 lifted (R22 140/140)
Builds tools/lift_types.py: targeted fleet-wide lift of a SPECIFIC type list into src/shared/engine_types.h, using build_engine_types' brace-aware parser (NOT the [^;]* regex that under-counted multi-field structs — R35, caught twice). It picks each type's CANONICAL (majority) def across the fleet, writes it once to the header, strips every local copy, and reports variant overlays. The whole-binary gate (R22) is the byte-arbiter. Applied to the 2 CLEAN types the §20-blocked fresh-138 cores need: - Mat32 (138 canonical + 1 copy-only draft variant in ov_SC07_006) + Cam8012E138 (unique) -> lifted, 139 local copies stripped. R22 clean-fleet 140/140. - Unblocks func_8012B4B8 + func_8012E138 for x138 propagation (next). DEFERRED (variant-heavy, high-risk — fleet genuinely split): MATRIX (3 defs), Vec8 (2 near-even 180/139), Buf (3), M8 (2) — lifting them would force ~929 files to one camp's field layout and break variant field-access. Needs a per-camp reconcile pass. func_80175308/func_8012A1BC still blocked (need those or other types). |
||
|
Drew T
|
f8d8f13b98 |
feat(phase-29): fix_header_decl v2 (--reconcile-externs) + SESSION-13 cont checkpoint
v2 generalizes v1: also reconciles CALLEE decls in shared headers that a byte-perfect draft declares differently than engine_core.h (sources the byte-true sig from the draft's `extern` line; reuses v1's compatible/build_decl machinery). --check validated: func_80174CB0 -> func_80012C6C s32->s16 (all SAFE); v1 unbroken (func_8014CD80 ALREADY-OK). The byte-gate + R22 arbitrate scalar-width changes. Known limitation (byte-gate caught it, fail-closed, nothing banked): v2 fixes only src/shared/*.h, not per-overlay-local decls — func_80174CB0 stays blocked by a local decl of func_80012C6C in ov_SC07_006_jr_8015C32C.c:8523. v3 = extend to the overlay split .c (carried in the checkpoint). Also in flight (checkpoint): crack wave w9lidyi5b (24 fresh LIVE=138 families, running) + permuter winner func_8014D12C (score-0, staged) — both gate-on-completion. |
||
|
Drew T
|
cbb52ebce1 |
feat(phase-29): fix_header_decl.py — automate the fresh-138 def-side header reconcile (§63)
Builds + self-tests the tool that reconciles a shared-header forward-decl to a function's byte-true signature — the integration half of the proven func_8014CD80 x138 pipeline. - tools/fix_header_decl.py: --fn --draft [--check|--apply]. Parses the byte-true def sig, canon-compares (typedef-aware int==s32, unsigned short*==u16* -> skip ALREADY-OK), REFUSES ABI-changing rewrites (param-count / ptr<->scalar / scalar class), preserves the macro `\` continuation, snapshots + prints the git-restore (§61: undo = restore, validate fleet-wide by R22). - 4 self-tests pass: (1) idempotent on the already-fixed func_8014CD80; (2) correct rewrite + `\` preserved; (3) REFUSE on a 2-vs-3-param ABI mismatch; (4) end-to-end --apply on func_8014D12C turned the `conflicting types` PLUMBING into a clean codegen DIFF (plumbing dissolved; its body is a near-miss so no bank), reverted clean. The self-test CAUGHT two fleet-corrupting bugs before any apply (a dropped macro `\`, and non-idempotent int/s32). - cookbook §63 + CURRENT_PHASE. NEXT: fresh-family wave over the 38-family market -> fix_header_decl --apply -> harvest_verify -> dedup_propagate -> R22. |
||
|
Drew T
|
57ef4ebcb5 |
feat(phase-29): ov_SC07_006 reach-138 batch-1 propagation + live-count re-scope (SESSION-13)
Propagation of the 6 batch-1 x1 banks (§55b: banks committed first in commit:0848, then targeted propagate as a standalone step): - dedup_propagate --addr: func_801325B8 -> +3 onboarded-tail siblings (ov_SC07_007/010/011). func_8014A048/func_801678F0 byte-diverge in the SC07 cluster (kept x1); func_8014FE60/func_80167540 local-type-blocked §20 (x1). - func_80165CA0: consolidated its h_exact subgroup (dedup group registered, +0 new), then family_sweep --hseq 0/135 — a PER-MEMBER WALL (cf func_80133AB0 0/136). The x135 "fresh family" prize does not exist here. - Net batch-1 yield ~9 newly-matched functions; fleet 78.6->78.7% instr, distinct flat; ov_SC07_006 84.6->84.8%. R22 clean-fleet 140/140; tools-health 1850/0. The finding (R14/R35, decision-log 2026-07-23): nins*reach leverage over-counts — rank by LIVE-siblings. build_wave_args.py --rank live now ranks by the true lever and reports the fresh(76)/onboarded-tail(44) split. The reach-138 family well is largely SPENT via wave+gate; the fresh families are the hard tail (def-side plumbing/DIFF/per-member walls), not free x138 fuel. |
||
|
Drew T
|
b76ad85157 |
feat(phase-29): ov_SC07_006 reach-138 crack-wave batch-1 — 6 x1 banks
Option (b) from the SESSION-12 checkpoint: fresh-exemplar crack-wave on the P27-onboarded ov_SC07_006 (84.6% -> 84.8%). New reusable builder tools/build_wave_args.py emits wave_binary.js args from a fuel manifest. - Drafting wave (wave_binary.js, 24 xHigh drafters over the top-24 reach-138 WAVE families by leverage): 16 self-assessed MATCH, 8 hit the session usage limit (redraft later). - Byte-gate: plain harvest_verify banked 2 (func_801325B8, func_80165CA0); gate_stage reconcile ladder (arity pre-pass + cast/sig_unify) banked +4 (func_8014FE60, func_8014A048, func_801678F0, func_80167540). 7 near, 6 reconcile-fail, 2 CC1-FAIL, 2 DIFF (the incomplete session-limit drafts). - R14/R35 leverage reality-check: nins*reach over-counts. 5 of 6 are already matched in ~135 overlays (only the onboarded SC07 tail ov_SC07_007/010/011 is live, +2-3 each). func_80165CA0 is a TRUE fresh family (135 live) — the x135 sweep prize, propagated next. - R22 clean-fleet 140/140 (engine_core.h arity edit fleet-safe); tools-health green (dedup 1849/0). Propagation is the next commit (§55b: banks first). |
||
|
Drew T
|
57af7ea889 |
feat(phase-29): ov_SC06_018 non-jtbl wave batch-1 (8 x1 + 30 swept = 38 instances) + fix gate_stage arity-undo revert bug
- batch-1 wave: 25 non-jtbl ov_SC06_018 targets (reach 3-14 modal), 13/25 match_one MATCH; whole-binary banked 8 (4 plain + 4 via gate_stage --src-file), swept 4 families -> 30 members across 13 overlays. 5 matches deferred (missing-sym/s58/deeper plumbing); 12 nears are permuter fuel (several close=2/3/4). - KEY: non-jtbl fns in a jr-split file need gate_stage --src-file <the jr TU> (the default- TU reconcile misses them -- same class as the jtbl --src-file fix). - BUG FIX (R33): gate_stage's arity-undo snapshotted src/<bin>/*.c BEFORE _gate1 splices the banks there, so an unbanked draft in the batch triggered a snapshot-restore that SILENTLY REVERTED the banks (measured: a 9-draft run banked 4, the 5 unbanked reverted all 4 to INCLUDE_ASM). Fix: restore ONLY src/shared/ (the fleet hazard the snapshot exists for); the binary's own TU arity edits are local + byte-neutral. GATE_NO_ARITY=1 was the interim workaround. s61's law from below (undo scope must not EXCEED write scope). - incremental check-all 140/140 (concurrent with batch-2 drafting; full R22 after batch 2). fleet distinct 67.5->67.6% (+38 unique fns), instr 78.6% steady. |
||
|
Drew T
|
3ebdd91fc6 |
feat(phase-29): jtbl post-carve reconcile (harvest_verify._jtbl_reconcile) + REFUTE the reach-138 jtbl families as plumbing wins
- ROOT CAUSE: the jtbl carve's s8b carried-decl layer conflicts with each draft's externs; the reconcile chain (cast_call_sites+reconcile_tu, --src-file-aware) exists but gate_stage runs it PRE-carve against the wrong TU (a jtbl fn's real TU is the split file, which doesn't exist until harvest_verify carves). - FIX (cookbook s62): harvest_verify._jtbl_reconcile runs the chain POST-carve against the CARVED TU, draft-only rewrite, guarded by _jsnap is not None. Validated: func_80135260 (callee) + func_80191C50 (data) both conflicting-types -> genuine DIFF. - FINDING (R14/R31 -> decision-log): dissolving the plumbing REVEALED all 4 jtbl drafts have a deeper issue -- func_80135260/80191C50 real %hi-share regalloc residual (agents' reloc-masked match_one MATCH over-claimed it); func_8012AAAC def-side-arity + FLEET- SHARED + still DIFFs after arity fix (def-side wall); func_80135EB0 isolate FAILED. The '+0.58pp from 3 reach-138 jtbl families' is REFUTED -- near-misses/walls, not plumbing. The fix banks any plumbing-ONLY jtbl fn + makes the jtbl gate honest. - s61 traps re-confirmed (s62): gate jtbl ONE-AT-A-TIME (mid-batch isolate-FAIL corrupts the batch); fix_arity on an engine_core.h fn leaks fleet-wide (restore src/shared/ too). - R22 clean-fleet 140/140; tool change only (no committed-byte change). |
||
|
Drew T
|
f142a6858f |
feat(phase-29): ov_SC06_018 crack-wave — func_801365B8 x138 (fresh-exemplar sweep CONFIRMED), thesis is family-specific
- binary-aware crack wave (new tools/workflows/wave_binary.js): 8-target calibration over ov_SC06_018 substantial stubs, 7/8 match_one MATCH - func_801365B8 (155, reach 133): cracked FRESH in ov_SC06_018, swept 132/132 siblings via family_sweep --hseq --source ov_SC06_018 --allow-pins -- SESSION-10 refused this family 0/133 from an ov077 exemplar. THESIS CONFIRMED (fresh exemplar unlocks it). - func_80133AB0 (137, reach 137): cracked fresh + banked x1 (+ a byte-neutral s17a-1 cast reconcile of banked caller func_801343C4), but the family sweep FAILED 0/136 even from the fresh exemplar (reverted clean) -- THESIS REFUTED for this family. - FINDING (R14/R31 -> decision-log): the fresh-exemplar sweep is FAMILY-SPECIFIC, not a blanket mechanical x137. A fresh crack is necessary but not sufficient; the byte-gate arbitrates each family (~50% on this 2-family sample -> discount the ~1.5pp estimate). - tooling (R33): family_sweep --source override now searches matched_members (a fresh member leaves 'members' after a sig-regen); cdecl._depth0_spans consumes backslash line-continuations so a raw-draft #define macro no longer trips audit-cdecl. - R22 clean-fleet 140/140 byte-identical; tools-health green (dedup 1849/0, C1 234615); 0 NON_MATCHING. fleet 78.4->78.5% instr / 67.1->67.5% distinct / 88.14->88.18% fn-count. |
||
|
Drew T
|
80d570195c |
feat(phase-29): add main to the weighted denominators (roadmap §1 metrics contract)
The contract requires all three headline metrics to include the main EXE. Since Phase-27 T10 main was reported as a SEPARATE provisional line, so the headline silently measured 139 of 140 binaries — and flattered itself by omitting the LEAST-decompiled one. RESTATED CAVEAT (the old "stale, PROVISIONAL" wording was misleading): main's sig is Ghidra's (2026-06-14), but function BOUNDARIES derive from the original bytes and do not change, and matched-vs-stub comes from the LIVE corpus.stubs — so the numbers do NOT drift. The real limitation is R34: sig_image cannot independently validate a PS-X EXE's boundaries, so main has no SECOND, DISAGREEING oracle for the PHANTOM/TRUNCATED class. The sig also excludes the LINKED PsyQ objects, which is exactly right for a GAME-CODE contract. EFFECT — the headline DROPS, which is the point: instr-weighted 78.7% -> 78.4% (10,299,493 / 13,141,652) distinct-code 67.9% -> 67.1% (3,782,990 / 5,634,875) A continuity line reports the ex-main figure so pre-2026-07-22 readings stay comparable, and the binary-count label is corrected to "main + resident + 138 overlays" (it read "resident+139 overlays"). Metrics-only change; no build impact (ov_SC06_018 re-verified BYTE-IDENTICAL). |
||
|
Drew T
|
c6515a6cdc |
fix(phase-29): the FLOOR VERDICT was an artifact of snapshot frequency, not yield decay
burndown.py averaged the last 3 INTER-COMMIT deltas. The phase's ROI criterion is "per-SESSION yield floors out", and historically one snapshot was taken per session — but I seeded THREE inside this session (9, 9b, 9c). Averaging them drove the mean to +0.23 and printed "AT THE FLOOR — consider closing P29" for a session that actually yielded +0.7pp instr (78.0 -> 78.7), comparable to recent sessions. I nearly closed the phase on it. Same error class as everything else this session: an instrument answering a DIFFERENT QUESTION than the one asked, where the wrong answer is indistinguishable from the right one. - --session-close marks a snapshot as a session boundary; the floor verdict is now computed ONLY from those (older records predate the flag, so a label containing "close" counts too). Mid-session snapshots still record for tracking. - honest output now: "0 SESSION-to-SESSION delta(s) logged — need >=3 for a floor verdict (1 session-close snapshot on record)". => P29 must NOT close on ROI grounds yet: the floor is UNDETERMINED and needs 3 session closes to become computable. The eyeballed "+2.5 -> +1.1 -> +0.6 -> +0.2" trend is not the instrument's verdict either. |
||
|
Drew T
|
372bff1153 |
fix(phase-29): jtbl_family_bank's revert left the tree GIT-CLEAN BUT UNBUILDABLE
Restoring config/ from git does NOT rewind asm/, so a reverted overlay was left with stale extraction — and `git status` is structurally blind to it (the R22 corollary, here inside a tool's own undo path). Within a sweep each sibling is a DIFFERENT overlay, so the next sibling's own extract never repairs the previous one: the damage persists to whatever runs next. Byte-witnessed on the 0x8013C414 probe: after 3 gate-fails `git status` was EMPTY and ov_SC01_004 failed to link (`undefined reference to jtbl_8018DAC8`). - revert() now RE-EXTRACTS (extract=True default); the clean-slate call at the top of bank_one passes extract=False since it extracts immediately afterwards anyway - revert() only `git checkout`s a TRACKED path: an isolation creates region .c files that were never added, and checkout on those emitted `error: pathspec ... did not match any file(s) known to git` while doing nothing. The keep_regions cleanup removes them instead. NEGATIVE CONTROL: the same probe now gate-fails 3/3 as before, and all three overlays build BYTE-IDENTICAL afterwards with a clean git status. |
||
|
Drew T
|
64bfe59bc0 |
fix(phase-29): gate_stage gated EVERY non-077 binary against ov_SC01_077's SHA — one-line default, one month
THE BUG (tools/gate_stage.py main(), introduced commit:0181, 2026-06-21, Phase 21 T3):
good_sha=a.good_sha or DEF_SHA, # DEF_SHA = ov_SC01_077's locked hash
DEF_SHA is TRUTHY, so it beat run_gate's per-binary lookup
(`good_sha or _check_sha(binary) or DEF_SHA`) and made that lookup DEAD CODE on
every CLI invocation. gate_stage therefore BUILT one binary and compared it to a
DIFFERENT binary's hash: it can never match, every draft reports as "near", and
NOTHING COULD EVER BANK outside ov_SC01_077 from the CLI. A "near" is
indistinguishable from a genuine codegen residual, so the failure looked like a
compiler wall for a month.
WHY IT HID: the programmatic callers take a different path and were all correct —
grinder/idiom_hunt pass good_sha=None (per-binary lookup), lora_grind/bulk_harvest
pass an explicit per-binary sha, orchestrator is 077-only where DEF_SHA is right.
That is exactly why the grinder banked func_80181F78 in ov_SC03_014 (Task 13B)
while my CLI ladder banked 0/10 on the same tree. Two paths disagreed for a month
and nothing compared them (R34's lesson, from the inside).
BLAST RADIUS, MEASURED (not assumed): 0 of 6,708 backlog records come from the
affected path — by source: worker 2724 / bulk-harvest 2275 / lora-grind 766 /
grinder 522, all correct. THE BACKLOG NEEDS NO RE-RUN. The void verdicts are the
manual CLI gates on non-077 binaries, i.e. exactly the Task-5 wave's "the gate
banked ZERO" on 12 preserved cracks — never a codegen finding at all.
FIXED: pass a.good_sha through; run_gate reads config/check.<bin>.sha (R33).
VALIDATED end-to-end: the raw draft that produced {banked:0, near:1} now gives
{banked:1}.
RE-RUN RESULT — 7 of the 12 preserved t5wave cracks are now banked:
func_8018F694 (478) · func_8019059C (673) · func_80135A4C (181) ·
func_80135888 (113) · func_80135D20 (100) · func_801749C8 (105) ·
func_801299C8 (158, was filed "PLUMBING: prototype declaration")
STILL BLOCKED (5, believed genuine): func_8012AAAC, func_80135260, func_801365B8,
func_80165CA0, func_80191C50.
⚠️ SUPERSEDES the earlier retraction: the "0/10 ladder" was not vague interference
— it was the gate comparing against the wrong binary's hash. Task 14 stages 2-3
were priced against a number that could only ever have been zero.
- R22 clean-fleet 140/140 BYTE-IDENTICAL; all 6 verified stub-free
|
||
|
Drew T
|
52e772628b |
feat(phase-29): isolate-with-body FIX + 3 more preserved cracks banked (incl. a 673-ins giant)
THE FIX (harvest_verify._jtbl_prep_one): isolate WITH THE BODY STILL SPLICED.
jr_isolate_all accumulates each object's file-scope decls as the new region's
`ambient` set, so partitioning around an INCLUDE_ASM stub hands the region a
DIFFERENT decl context than the draft's body needs — and the gate then produced a
byte-DIFF rather than a compile error, which is why a batch of these read as
"9 compile / 0 bank" and looked like a codegen wall. §61b's law extends one step:
THE CARVE MUST FOLLOW THE SPLICE — AND SO MUST THE ISOLATION.
The un-splice existed only because the tool is stub-centric (a spliced fn is no
longer in corpus.stubs). New _unsplice_body() handles that properly: it finds the
file that NOW holds the body (isolation may have MOVED it into a fresh region file)
and writes back the stub line for THAT subseg, so the gate re-splices identical text.
BANKED via the corrected path (each whole-binary byte-gated, one per invocation):
func_80135D20 (100 ins, reach 138)
func_801749C8 (105 ins, reach 136)
func_8019059C (673 ins, reach 3 — a giant)
Together with func_80135888 and func_8018F694 earlier: 5 of the 11 preserved
t5wave cracks are now banked.
THE REMAINING 6, honestly classed: 2 genuine DIFF (func_80135260, func_80191C50),
2 CC1-FAIL (func_801365B8, func_80165CA0), 2 §57 self-decl plumbing
(func_801299C8 `prototype declaration`, func_8012AAAC own-name conflict).
⚠️ RETRACTION: the earlier "the ladder converts 0/10, which prices Task 14 stages
2-3" is WRONG and is withdrawn. Three of those ten bank through harvest_verify
alone with the SAME transformed drafts that gate_stage rejected — so that 0/10 was
measuring gate_stage's own interference, not the residuals. Prime suspect is the
arity pre-pass (fix_arity_callers --apply --any-proto edits engine_core.h before
the gate; the §19 regression mode). GATE_NO_ARITY=1 is the ready A/B. Task 14
stages 2-3 remain UNPRICED until that runs.
- R22 clean-fleet 140/140 BYTE-IDENTICAL; tree clean before and after
|
||
|
Drew T
|
02b16fa7a7 |
refactor(phase-29): DELETE gate_stage's jtbl pre-pass (R33) + the honest ladder measurement
gate_stage._jtbl_prepare carried the SAME config-only undo as harvest_verify's did,
and ate the tree again on the first ladder run: 5 orphan region files, truncated
TUs, `undefined reference to func_80192F64`. That INVALIDATED the run's 0/10, so it
was re-measured rather than reported (R35 — a probe from a broken tool is not
evidence). Tree restored from HEAD and re-verified byte-identical first.
DELETED, not patched (R33 — the best outcome is a deleted stage). It was wrong on
two independent axes:
1. §61b already byte-proved THE CARVE MUST FOLLOW THE SPLICE. A batch pre-pass
carving unspliced functions reports "prepared" and yields a spec that fails
once the body lands — which is why it banked nothing.
2. Its undo snapshotted only config/, while jr_isolate_all rewrites region 0 back
over the ORIGINAL src/<ov>/<nm>.c truncated.
harvest_verify's per-draft prep is the correct mechanism, snapshots the full source
set, and undoes per function. Two implementations of one capability, the outer one
ineffective AND destructive.
THE HONEST RE-MEASUREMENT (clean tree; tree verified clean after):
- 0/10 bank, but 9/10 now COMPILE and land as whole-binary byte-DIFF; 1/10 plumbing.
- match_one close=0 on several (the function's own bytes exact) and rtu_match says
MATCH-in-real-TU for func_80135888 — while func_801299C8's transformed draft does
not compile in its real TU at all. The residual is MIXED, not uniform; at least one
is an IMAGE-level effect rather than the draft or its TU decl context (prime
suspect: jtbl/rodata carve placement). NOT generalized from one data point.
- This PRICES Task 14 stages 2-3 by measurement: the existing ladder converts 0 of
10, so they are not "wire in normalize_self_decls + the type-lift and collect ten
banks" — the projection error §57a already caught once this phase.
- R22 clean-fleet 140/140 BYTE-IDENTICAL with the giant func_8018F694 banked and the
func_80135A4C family swept 138/138
- cookbook §61d (the tree-eating undo in two tools; the constant-label defect; the
re-probe + ladder measurements; the general rule: an undo whose scope is narrower
than its write scope destroys work no byte-gate can see)
- decision-log + CURRENT_PHASE updated (R30/R31)
|
||
|
Drew T
|
fcd83f705e |
fix(phase-29): the undo was EATING the tree — snapshot src too; kill the §58 label; giant func_8018F694 banked
THE DEFECT (byte-witnessed, and it is the §61c mechanism). `jr_isolate_all`
repartitions a code object by writing region 0 back over the ORIGINAL
src/<ov>/<nm>.c — TRUNCATED to just that region — and emitting the rest as new
_jr_<lo>.c files. `_jtbl_restore` undid only config/ + the new region files, so
every gate-REJECTED draft left the original TU permanently truncated and its
stubs gone. Nothing regenerates them (splat does not rewrite a committed overlay
.c). Measured across an 11-draft re-probe: live stubs 419 -> 414 -> 406 -> 395,
ending in `undefined reference to func_80191C50`.
It is INVISIBLE to the gate that causes it: the incremental build keeps linking
stale objects (§42b) so `make build` stays green while a CLEAN rebuild fails.
That is exactly the "139/140, twice" signature that became the §61c blocker —
the tree was being eaten by the undo meant to protect it. The §61c attribution
(batch _jtbl_prep residue) is now a demonstrated defect, not an inference.
FIXES (both negative-control-validated on a draft that fails):
- _jtbl_snapshot captures every src/<binary>/*.c; _jtbl_restore restores them and
removes exactly the files the attempt created (derived from the snapshot's file
set, not re-guessed from the _jr_* name shape, R33). Undo by SNAPSHOT-RESTORE,
never an inverse transform — §61's law one level deeper. Same failing draft that
previously broke the tree now leaves it byte-identical, git status clean.
- classify_fail ignores `warning:` lines. The benign `conflicting types for
built-in function 'memcpy'` warning was winning the match on 8 of 8 failures
across four different real causes — a label identical for every input, which the
cookbook had to work around by hand ("the gate label is useless here, splice
individually and read real cc1 stderr"). Now reports the real error, and falls
through to CC1-FAIL:<last error line> rather than guessing.
THE RE-PROBE THIS ENABLED (11 preserved t5wave cracks, one invocation each):
- BANKED: func_8018F694 (478 ins) — one of the wave's three giants, previously
recorded as part of "the gate banked ZERO".
- The other 10 now carry TEN DISTINCT diagnoses: 4 data-decl conflicts (D_80193B64
x2, D_8011D030, D_80126B5C), 3 callee-decl (func_80135480 x2, func_8012F14C),
3 self-decl/own-sig (§57). ZERO jtbl-drift, ZERO local-type redefinition, ZERO
codegen DIFF.
- So §61a's "§8e-2 jtbl table-count drift blocks 10 of 12" does NOT survive the
carve-follows-splice prep: the carve now succeeds and what is left is ordinary
decl plumbing the existing ladder already handles (cast_call_sites / reconcile_tu
/ normalize_self_decls / fix_arity_callers) = Task 14 stages 2-3, no new tooling.
- ov_SC06_018 BYTE-IDENTICAL cbbc4f44 with the giant banked; nothing committed broken
at any point (tree was restored from HEAD and re-verified before these fixes)
- cookbook: the `git add -u` complementary hole (an isolation's NEW region file is
untracked, so a carve/isolation bank needs `git add -A src/ config/`)
|
||
|
Drew T
|
41d65af73f |
fix(phase-29): per-function jtbl prep + stranded-carve undo; the clean-rebuild blocker named
THE BLOCKING FINDING (§61c): the jtbl carve+isolation path yields a state that is
INCREMENTALLY valid and CLEAN-INVALID. func_80135A4C banks every time through the automated
path (BYTE-IDENTICAL at harvest_verify's gate) and fails `make clean && extract-all &&
check-all` twice, identically (139/140, [FAIL] ov_SC06_018). The bank is therefore NOT
reproducible from committed config + source, and the gate that authorises it cannot see the
defect because the gate IS the incremental build (§42b, most expensive form).
=> NO jtbl core can be banked until that divergence is diagnosed. Next step is to diff the
incremental vs clean build/ov_SC06_018/** object set + generated .ld/asm for the carved
subseg — NOT to bank more. All 12 wave cracks stay preserved at .run/giants/t5wave_*.
TWO REAL DESIGN FAULTS FOUND AND FIXED IN harvest_verify:
1. A stranded carve poisons the overlay. _jtbl_prep carved a draft the gate then REJECTED;
the carve remained with NO owner (the fn is still INCLUDE_ASM), and jr_inventory's 1:1
ownership assertion then refused every later isolation in that overlay
([('UNOWNED','0x801d288c')] = func_801299C8's table). THE ASSERTION WAS RIGHT AND CAUGHT IT
— R32/R33 working as designed; the defect was mine for leaving the carve behind. Fixed:
per-function carve + exact snapshot-restore (config text + only this attempt's region files)
on gate rejection.
2. Per-function undo is UNSOUND IN A BATCH: isolation REPARTITIONS shared source, so restoring
one draft's snapshot deletes region files now hosting OTHER pending drafts, and their stubs
vanish (KeyError in render). jtbl drafts must run one per harvest_verify invocation, or the
undo must be region-aware.
MEASURED so it is not re-derived: of 11 preserved cracks exactly ONE (func_80135A4C) reaches
byte-identical through the carve path; the other 4 table-bearing ones fail one-at-a-time too,
on PLUMBING (§57 self-decl et al), not the carve.
Tree reverted; clean-fleet 140/140; nothing banked. cookbook §61c.
|
||
|
Drew T
|
aa1b8b3c1a |
feat(phase-29 T14 stage 4): jtbl prep moved into harvest_verify — it BANKS; new blocker named
THE LAW IMPLEMENTED: the carve must follow the splice. harvest_verify._jtbl_prep() splices each
table-bearing draft TEMPORARILY, asks jtbl_carve, isolates on the §8b walls, un-splices, re-extracts,
and RE-DERIVES the stub map + baseline (isolation MOVES a stub's TU, so both are keyed on stale
paths otherwise). gate_stage's batch pre-pass could not work: while a fn is still INCLUDE_ASM the
non-contiguity is undetectable, so jtbl_carve reports success and yields a spec that fails when the
body lands.
BYTE-PROVEN AUTOMATED: func_80135A4C (181 ins, 138 members) ->
[jtbl] carved 1/1 table-bearing draft(s): func_80135A4C
+ chunk(1): func_80135A4C verified 1 / failed 0 BYTE-IDENTICAL
NEW BLOCKER, PRECISELY NAMED — banking a jtbl core makes its own carve UNOWNED to jr_inventory,
which then refuses every subsequent isolation in that overlay:
"committed .rodata carve ownership is not 1:1 (R32/R33) — a stranded/duplicated carve
(§8b func_801734BC class): [('UNOWNED', '0x801d288c')]"
Byte-proven both ways: on the COMMITTED tree `jr_isolate_all --only func_80135260 --dry-run`
succeeds; with func_80135A4C banked it fails the assertion. The assertion is RIGHT (a banked fn's
stub .s is pruned, so the owner lookup finds nobody) but its conclusion is wrong — the carve IS
owned, by C rather than a stub. SO TODAY JTBL CORES BANK ONE PER OVERLAY. 10-draft batch: 6
table-bearing -> 1 carved, 4 isolate-FAILED on this assertion, 1 stale-asm carve failure.
NEXT INCREMENT (precise): resolve jr_inventory's carve owners from corpus.matched U stubs, not
stubs alone (R33 — the same derive-don't-reparse move that fixed the corpus oracle).
BANK NOT KEPT: R22 clean-fleet showed 139/140 (ov_SC06_018 FAILS from a clean tree) even though the
INCREMENTAL build read BYTE-IDENTICAL — the stale-incremental false pass R22 exists to catch (§42b).
Reverted; clean-fleet re-verified 140/140, tools-health OK (dedup 1848/0). All 12 wave cracks remain
preserved at .run/giants/t5wave_*. cookbook §61b updated.
|
||
|
Drew T
|
96d1324acd |
feat(phase-29 T14 stage 4): jtbl isolate+carve stage built; the CARVE-MUST-FOLLOW-SPLICE law
DIAGNOSIS CORRECTED: the wave's 10/12 blocker is NOT "§8e-2 table-count drift" (the symptom the filter reports) but a NON-CONTIGUOUS .rodata carve — the new function's table is separated from the TU's existing carve by an UNMATCHED function's table, and one object cannot straddle that gap. jtbl_carve names its own remedy in the refusal message. RECIPE BYTE-PROVEN (func_80135A4C, 181 ins / 138 members): jr_isolate_all --only <fn> ; make extract ; <splice> ; jtbl_carve --func <fn> ; make extract ; make build -> BYTE-IDENTICAL (isolation verified byte-neutral on its own first; a config change needs extract, not just build.) BUILT: gate_stage._jtbl_prepare — per-draft carve + auto-isolate on the §8b walls, logic LIFTED from jtbl_family_bank (R33: one implementation, two callers — their divergence IS this bug), snapshot-restore undo, GATE_NO_ARITY A/B guard. Ladder: canon -> cast -> reconcile_tu -> jtbl -> arity -> gate -> sig_unify -> gate. IT DOES NOT YET BANK, and that is the finding: THE CARVE MUST FOLLOW THE SPLICE. The non-contiguity is only DETECTABLE once the body is in the object; while the fn is still INCLUDE_ASM, jtbl_carve reports SUCCESS and produces a spec that fails when the body lands. Byte-witnessed both ways (spliced -> NON-CONTIGUOUS 0xaa810/0xaa920; unspliced -> "prepared 1/1" then byte-DIFF). Innocent suspects A/B'd out: the draft is IDENTICAL through canon/cast/ reconcile_tu, and GATE_NO_ARITY=1 changes nothing. FIX = per-draft prep inside harvest_verify's splice loop (it owns the splice), not a batch pre-pass in gate_stage. SUB-FINDINGS: (a) a wholesale `git checkout -- config/` undo is WRONG in a batch gate — it discarded a previously-banked-but-UNCOMMITTED carve, leaving that bank's source with no subseg (undefined reference to func_80136C90 at link). Now snapshot-restore + drop only this run's region files (§61's constraint, which I had written and then not applied here). (b) being in a _jr_* TU != having a table: only 4 of 8 wave drafts reference a jtbl_. Tree restored byte-identical; nothing banked. cookbook §61a corrected + §61b. |
||
|
Drew T
|
1074f2f202 |
feat(phase-29 Task-14 stage 1): the ARITY pre-pass — and the shared-state constraint it cost
DIAGNOSED, not assumed. The 12-draft integration probe banked 1/12 and reported the SAME label for 10 of the 11 failures: `conflicting types for built-in function 'memcpy'` — the §58 red-herring (a WARNING, from an unrelated TU position). Splicing three top-reach failures individually and reading real cc1 stderr gave the actual causes: conflicting types for `func_XXXX' 3/3 <- loose-typing ARITY conflict redefinition of `struct V8' <- a SECOND class (type-lift), stage 2 A banked shared caller macro in engine_core.h declares the function with FEWER params than its byte-true definition takes (the original calls K&R-style with fewer args than the callee reads); a C89 prototype makes that a hard error. tools/fix_arity_callers.py --any-proto already fixes it and was simply NEVER WIRED into gate_stage's ladder (only family_sweep carried §57). Now wired as a TU-side pre-pass. MEASURED: 2 of 7 top integration candidates banked (func_8016EFC8, func_80164418, both reach-138) vs the 1/12 old-ladder baseline. R22 140/140; tools-health OK (dedup 1848/0). INCIDENT — this stage BROKE 138/140 AND R22 CAUGHT IT (nothing was ever committed): pairing `--apply --any-proto` with `--revert` for the unbanked drafts corrupted declarations fleet-wide. `--revert` rewrites ()->(void), which inverts a PLAIN apply but NOT --any-proto, so an unbanked fn whose real decl was `extern void func_801708B0(void *a0)` came back as `(void)` — in engine_core.h (included by all 138 overlays) and 6 sites in ov_SC01_077's own sources. harvest_verify --binary ov_SC01_077 reported BYTE-IDENTICAL and was RIGHT about that binary; the other 137 were structurally invisible to it. Repaired to the exact lines. ROOT CAUSE FIXED: the ladder now snapshots every file the pre-pass touches and undoes by RESTORE + re-apply-for-the-banked-set-only — exact by construction, cannot invent a signature. NEW HARD CONSTRAINT (cookbook §61): any ladder stage mutating SHARED state must be undone by snapshot restore, never an inverse transform, and validated FLEET-WIDE (R22) rather than by the per-binary gate that authorised it. §55b's propagation law, one level down. The planned type-lift stage edits engine_types.h and inherits it by default. ALSO FIXED: the first wiring passed only --drafts (the narrow-param FILTER) without the required --funcs, so the stage exited `no funcs given` as a SILENT NO-OP and the gate reported 0/6 as though diagnosed. sh() does not raise on non-zero exit -> explicit rc check added. |
||
|
Drew T
|
835c45905d |
feat(phase-29 Task-13B close): plateau autopsy = ZERO missing-transforms; narrow the admission instead
The hindsight-study §7 taxonomy predicts plateaus decompose into missing-transform (the "highest-value bucket and the whole point"), seed-structural, and genuine-wall. Run against real plateaus this class produced NO missing-transforms, and the answer needed no LLM. MEASURED: `length` probe, 20 targets, 1 win. tail 1/6; partial 0/12. AUTOPSY (read directly from the bytes, 3 partial plateaus): - func_8017F0C0 / func_801806C8: target has `sltiu $v0,$v0,1` = gcc's codegen for `!x`/`x==0`; the drafts wrote `(u32)(D_x ^ 1)` which emits `xori`. No local mutation crosses that. - func_8017FF90: draft stores to arg0+8, target stores to a GLOBAL. Different function. => these are WRONG DRAFTS wearing a small closeness, i.e. seed-structural, not a mutation gap. THE FIX IS THE OPPOSITE OF "ADD TRANSFORMS" — a tighter ADMISSION rule: - _drift_route: permuter only when |d|<=2 AND explains=="tail" (the shape that measurably converts). length pool 339 -> 34; permuter bucket 389 -> 84. - SIZE-MISMATCH: added a PROPORTIONAL test (|d| >= 0.5*nt). max(2,0.15*nt) is far too permissive on a tiny target — a 2-ins draft vs a 4-ins target read as a near-miss. permuter_weights needs NO extension for this class. Transferable (cookbook §60b): raising a search-closer's yield is at least as often about refusing it unreachable work as widening its mutation set. Same knife as Task-13A's targeting fix, one cut finer. Drafter idiom recorded: `sltiu rd,rs,1` => `!x`, never `x^1`. 17 unit tests green; corpus re-collected (1654 rows, closeness cross-check clean). |
||
|
Drew T
|
5c894c1e62 |
feat(phase-29 Task-13B): func_80141B90 x138 + the reach repricing + the length profile
PROPAGATION (§55b, its own targeted batch): dedup_propagate --addr 0x80141B90 --recover -> "138 overlays byte-identical after propagation"; 117 remaining stubs -> 0; 1 new dedup group. This was the ONLY one of the 21 directed-run banks worth propagating. THE REPRICING (R14 — measure a bucket's VALUE, not just its conversion rate): the directed run converted 27% (21/77) but moved the fleet ~0.03pp, because h_exact reach of the 21 is: func_80141B90=138, TEN at reach-1 (nothing to propagate), rest 2-10. Instruction-weighted, the ENTIRE permuter bucket is worth ~0.36pp at 100% conversion. The mechanism is validated; the fuel was small. Priced frontier (ins-weighted / 13.08M): LENGTH-DRIFT |d|<=2 472,178 ~3.6pp (339 fns) <- the real permuter-adjacent lever integration 419,162 ~3.2pp (305 fns) <- Task 14's ladder WIDTH 71,593 ~0.55pp (45) permuter (current) 46,571 ~0.36pp (74) BRANCH-POLARITY 9,462 ~0.07pp (22) So WIDTH/BRANCH-POLARITY are NOT worth prioritizing; my earlier "~200 candidates" framing undersold LENGTH-DRIFT 10x and oversold WIDTH. NEW: permuter_weights._LENGTH profile (perm_temp_for_expr/perm_expand_expr are the only passes that change instruction COUNT; the reorder/decl-order levers that dominate the regalloc+schedule profiles cannot, so they are down-weighted here) + residual_class ._drift_route (|d|<=2 -> permuter/`length`, larger stays structural — same class, opposite tool) + classify() accepts a PROFILE NAME directly (the measured profile beats re-parsing a free-text label). 17 unit tests green. grinder: --profile filter (probe ONE residual class's conversion) + a PERSISTENT attempt ledger. `tried` was in-process only, so every fresh --once run re-permuted the previous run's losers — the permuter is deterministic given (base.c, target.o), so that CPU can never produce a new win. Measured: a 20-target probe drew 19 already-tried targets. Keyed by draft_sig so an improved draft legitimately re-opens the function. |
||
|
Drew T
|
b8ee9fa9d2 |
feat(phase-29 Task-13B): directed permuter wins first try; 3 latent gate defects fixed at root
First run of the Task-13A-targeted grinder (--once --batch 8 --permute-secs 90):
targeting: ON — 1665 classified; only bucket=permuter is admitted
targeting skipped 1601 non-permuter candidates
{redraft 707, structural 583, integration 305, unclassified 4, unknown 2}
permuter WON func_80181F78 @ ov_SC03_014 (close was 1) [~6 min]
BANKED 2 (both whole-binary byte-gated, R22 clean-fleet 140/140):
- func_80181F78 (8 ins) — classified DELAY-SLOT / schedule
- func_80141B90 (29 ins) — classified IMM-VALUE / cse
Both were classified bucket=permuter by residual_class BEFORE the run: the classifier
predicted a search-closer could reach them, and one did. First end-to-end validation of
the targeting thesis.
THREE latent defects, all pre-existing, all unreachable while the grinder banked nothing
since Phase 21 — the fix made it win and every one fired at once (cookbook §60a):
1. gate_stage commit path crashed on src=None. `src` is DELIBERATELY never defaulted
(Phase 26-A: a default silently PINS the gate to the main .c), but the commit did
`git add src …` unconditionally -> every caller that omits it (grinder, orchestrator,
idiom_hunt) crashes THE MOMENT IT BANKS. Now `git add -u src/`, which also retires the
`src/ov_*/*.c` filename glob that once omitted 4 R22-verified banks from a commit.
2. _xform ladder dirs (-cn/-cast/-rc/-uni) ACCUMULATE across runs: they held 34 stale
drafts, so the gate processed 34 when the grinder submitted 1, and banked one function
it was never asked to try — which would have been committed under a message naming a
different one. Nothing wrong entered the tree (G3/P9 held; the gate banks only
byte-identical output) but report and work had diverged. Now cleared per run.
3. grinder called gate_stage with the default propagate=True = `dedup_propagate
--auto-from`, the §55b fleet-wide path that timed out at 3600s and left 90/140 overlays
broken — and being INSIDE the gate it takes the banks down with it. Now propagate=False;
banks commit cheap, propagation is its own targeted --addr batch (§55b law).
R22: make clean && extract-all && check-all -> 140 passed, 0 failed of 140.
tools-health OK (dedup 1847/0, C1 234343/234343); 0 NON_MATCHING (G4).
|
||
|
Drew T
|
b3ab5c2253 |
feat(phase-29 Task-13A): deterministic residual classifier — the permuter's problem is TARGETING
The autopsy (hindsight-study §7) assumed the permuter loses for want of a mutation. Measured over the whole open backlog, it loses because it is aimed at work a search-closer provably cannot close. - NEW tools/residual_class.py: decide a near-miss's class FROM THE BYTES. Decodes each mismatching MIPS word -> (op-skeleton, register-fields, immediate); drift FIRST (one inserted insn inflates `closeness` by the tail length), then consistent-injective register map -> REGALLOC-PERM (§31 S11/RC-3), same-multiset-reorder -> SCHEDULE-REORDER, DELAY-SLOT, WIDTH/BRANCH-POLARITY/STRENGTH/ADDRESSING/IMM-OFFSET/IMM-VALUE. Every class routes to a BUCKET = which tool the failure wants. Uncovered opcode -> UNKNOWN, COUNTED (R32). 16 synthetic unit tests (test_residual_class.py). - NEW tools/autopsy.py: `collect` materialises the corpus Task-12's telemetry never filled (1 of 6,169 records had a residual) by recompiling every open draft through the EXISTING match_one path (R33) — 1,752 drafts in 21s at -j12. `report` -> docs/autopsy.md. - NEW corpus.o0_sources()/is_o0(): the opt-level oracle DERIVED from the Makefile's own -O0 rules, coverage-asserted. Scoring an -O0 target at -O2 makes the residual 100% artefact (the trap this phase hit four times). - R34 cross-check baked in: residual_class's closeness vs masked_diff.structured_diff's, asserted per row; 1,673/1,673 agree, 0 classifier errors. FINDING: of the 972 records the grinder's own filter admits, only 75 (7.7%) are permuter-shaped; 547 are structural and 348 are drafts that are not the function at all. ~92% of the daemon's CPU went where it could not win — the byte-grounded explanation of "7 banks all-time, all Phase 21, 0 since" (Phase-22 audit). grinder.candidates() now filters on the measured bucket (1,303 -> 78) and takes its directed profile from the measured class, not the logged label (91% carry none -> it ran on gcc defaults). Degrades to undirected if uncollected and says so; --no-targeting A/Bs it. Two measured corollaries (R14, not projections): - 699 records rank as near-misses at closeness up to 278 purely from a length artefact: un-attempted work misfiled as a backlog of hard functions -> new `redraft` bucket. - a 12-draft gate probe of the `integration` bucket banked 1/12 (11 PLUMBING), so the 306 prices Task 14's reconcile ladder rather than promising free banks. func_80167714 (104 ins, reach-134) banked x1, un-propagated by design (§55b). Two defects fixed forward: - masked_diff._common_typedefs() used ONE shared probe path, so parallel match_one processes clobbered each other: 14 of 1,752 drafts lost in a single 12-way run (0.8%), silently, in every parallel wave ever run. Now per-PID. - gate_stage.match_one_closeness never passed --o0 -> phantom residuals for every -O0 function, written straight into the backlog this autopsy reads. R22 clean-fleet: check-all 140 passed, 0 failed of 140; tools-health OK (dedup 1847/0, C1 234343/234343); 0 NON_MATCHING (G4). Flywheel captured in-session (R30/R31): cookbook §60, decision-log entry, SETUP.md inventory. |
||
|
Drew T
|
eec96f76a4 |
feat(phase-29 Task-12): structured failure telemetry + durable permuter-winner save
The permuter-autopsy prerequisite (Drew-directed). Captures the WHAT/WHERE of every
non-match so the classifier/autopsy (Task-13) can mine it, not just a scalar closeness:
- masked_diff.structured_diff(): the per-instruction masked residual [(idx,mine,tgt),...]
- match_one --json: emits {status,closeness,nins,residual} (refactored to share the primitive)
- backlog FIELDS += residual, passes_tried
- gate_stage.match_one_closeness returns + logs the residual on every near/fail (verified
end-to-end: a near-miss's structured residual now lands in backlog.jsonl)
- grinder: durably save the winning C to .run/permuter-winners/ BEFORE gate_stage banks
(this session's lesson — 3 permuter wins were lost to a mid-flight revert)
|
||
|
Drew T
|
a5bad7f48d |
feat(phase-29 Task-11): burn-down + velocity tracker (the ROI-close instrument)
tools/burndown.py snapshots the 3 fleet metrics + remaining substantial frontier each run (docs/burndown.jsonl, git-HEAD-keyed), computes velocity vs the previous commit-snapshot, and flags the yield floor (mean instr-pp over last 3 snapshots < threshold => 'consider closing P29'). Seeded with the session-4-close baseline. Reads progress.fleet.md + family-hseq.md (regen those first for a true reading). |
||
|
Drew T
|
09a561e3e7 |
fix(phase-29): jtbl_family_bank auto-isolates on the span-fit wall too
Previously only auto-isolated on NON-CONTIGUOUS carve collisions; the span-fit
wall ('table starts do not fit the span', when --like's merged span doesn't match
a sibling's jtbl layout) fell through to carve-fail. jr_isolate unblocks it too
(byte-proven: func_8017AE2C ov_SC01_000 member BANKED). The distinct table-count-
drift error ('more rodata .align than pad specs') stays excluded (not isolate-fixable).
|
||
|
Drew T
|
9591d61f78 |
fix(phase-29): family_sweep --hseq --source now overrides the manifest exemplar
The --hseq path templated from pick_exemplar's choice (hard-prefers ov_SC01_077), silently ignoring --source (wave-2 finding). Now when --source names a different overlay carrying a MATCHED member of a family, template from IT (self-correcting: only a non-stub source member is used; whole-binary gate stays the arbiter). The ov077-default path is unchanged (override skipped). |
||
|
Drew T
|
53433df1b3 |
feat(phase-29): func_8013D53C 14/137 + NSD hardened (rewrite-to-canonical, (void) fix) + Cmd lift
- func_8013D53C: NSD (self-decl caller-thread) + a byte-neutral Cmd_8013D53C type-lift clear the plumbing -> 14/137 h_seq members bank; 123 have genuine per-member codegen DIFFs (hard cse.c-wall crack -> permuter fuel). Reverted the 123 bankless byte-neutral NSD edits (churn on matched code, §57a NB). - NSD hardened, 2 bug-fixes validated on D53C: REWRITE the divergent decl to canonical instead of DROPPING it (drop broke def-after-caller forward visibility -> 'func undeclared'); and '(void)' is a genuine 0-param conflict, not skippable like a true no-proto '()' (the bug that filed D53C's arity class as 'not this pass'). - earlier this session: broad --fix-def-sig sweep = HONEST +137 only (staging != banking, R14/R35); NSD is SURGICAL-ONLY (broad application poisons whole groups; --fix-def-sig is broad-safe). cookbook §57a. - R22 clean-fleet 140/140 byte-identical; tools-health OK (dedup 1846/0, C1 234205/234205); 0 NON_MATCHING (G4). fleet 75.2% instr / 60.6% distinct / 87.04% fn-count. session-total from 74.6: +0.6pp instr, ~+421 members. |
||
|
Drew T
|
ad11c491a5 |
feat(phase-29 decl-normalize): func_801670E4 family 4->137/137 (+133) via the 3rd §17a-1 direction
- new tools/normalize_self_decls.py — the SAME-FUNCTION decl-normalize: drop each decl of the templated fn F that the sibling TU's OWN already-banked callers declare divergently (block-scope, a different C form than the exemplar's fn-ptr cast) + cast its in-scope calls (byte-neutral §17a-1). cast_call_sites does the callee direction, reconcile_tu the data direction; this is the third: F itself. - family_sweep --normalize-self-decls: new per-sibling stage after reconcile_def_sig (edits the sibling TU file = harvest_verify's baseline, like edit_remap_sweep) + snapshot/final-SHA-MISMATCH revert backstop. - byte-proven the sole blocker was the caller-decl conflict, NOT --fix-def-sig (which renamed F's def params a0..a3 while the body used arg0..arg3 -> arg0 undeclared; R14 confound removed). 133/133 banked, 0 failed, 0 backstop fires; 0 func_801670E4 stubs remain fleet-wide. - R22 clean-fleet 140/140 byte-identical; tools-health OK (dedup 1846/0, C1 234205/234205); 0 NON_MATCHING linked (G4). fleet 74.6->74.9% instr / 59.4->60.1% distinct / 86.92->86.96% fn-count. cookbook §57. - R14: func_8016CBC0 (also stuck 137-family) has NO divergent self-decl -> type-lift-blocked, a SEPARATE lever (6-typedef cluster, like 8012956c), not this pass. Route by the real cc1 error. |
||
|
Drew T
|
041c9c7553 | feat(phase-29 §8e): jtbl_family_bank passes --like <exemplar> to sibling carves (span-structure transfer) | ||
|
Drew T
|
1cb018bc00 |
feat(phase-29 §8e): bank giant func_8013F350 (490) x1 — the 4-table [0,0,4,0] span + tables= persistence
- jtbl_carve §8e hardening (the F350 lesson): a pre-§8e Phase-26 merged-double span had NO
recoverable structure — spec derivation now uses the payload ZERO-WORD rule over persisted
table starts (tables= comment on the JTBL_PADS line), with source priority
{untouched+line=reuse verbatim | untouched+no-line=skip | touched=union of .s refs,
line tables=, --span-tables override, --like exemplar role-transfer}; spec_from_starts
replaces interval-carry; None-tolerant legacy comments; --like/--span-tables CLI.
- F350 carve: tables 8860(8e)+8880(5e,trimmed) fused BEFORE the existing 8898/88B8 double via
the zero-checked 4-gap -> ov_SC01_077.o JTBL_PADS := 0,0,4,0 (tables= persisted).
- splice reconciles (§56, byte-neutral): §30#2 def-side widen void->s32 (TU extern +
engine_core discarding-caller macro extern); D_80115158/D_8011515C macro-canonical redecls
+ §18 width-preserving store casts (sh under u8[]/u8); func_801416D4 canonical (s16) redecl
+ §17a-1 fn-ptr (s32) call cast; D_80187BD0 block-scalar decl dropped (file array covers).
- whole-binary gate [ OK ] sha1 d19c9580 == check. 3 of 4 giants now banked x1.
|
||
|
Drew T
|
db2daf47e4 |
feat(phase-29 §8e): bank giant func_80159C84 (337) x1 — isolate + the FIRST pad=4 spec
- NON-CONTIGUOUS -> jr_isolate_all --only (new jr_80159C84 subseg, 2 region files); jtbl_carve func_jtbls: stale-location .s CONTENT fallback (ownership stays config-derived) — needed once a spliced fn is re-extracted (no fresh .s anywhere). - carve: jtbl_801D8AFC (7e, trimmed) + pad word + jtbl_801D8B1C (5e, trimmed) -> JTBL_PADS := 0,4 — the first reproduced ORIGINAL interior pad (the §8a 'handle then' address). - splice reconciles (§8d carried-layer-wins, all byte-neutral): draft scalar typedefs stripped; 3 draft decls dropped for carried (D_801891B8 void*, D_8018911C u8 addr-only, func_80149FB0 s32); 3 call-site casts + the §17a-1 fn-ptr cast for canonical-(void) func_80161208. - whole-binary gate [ OK ] sha1 d19c9580 == check. |
||
|
Drew T
|
5e78dccf07 |
feat(phase-29 §8e): jtbl 8-align pad-spec filter — the 4-giant unblock tooling (fleet-neutral)
- ROOT CAUSE PINNED (the session-2 half-pin was INVERTED; both probes were vacuous, R35): cc1 emits .align 3 before EVERY jump table; maspsx passes it VERBATIM (the :435 'drop' is an inventory-only pass); as bakes the pad SECTION-RELATIVE; link placement was never guilty (SUBALIGN(2) + ALIGN(.,4) place 4-mod-8 carve starts tight). Merging originally-separate TUs fires an intra-TU align where the original packed tight -> +4 at rodata 0xCC -> image-wide %lo shift. Honest probes persisted: .run/probe_jtbl/ (verdict.md + objdumps). - NEW tools/jtbl_rodata_pads.py: post-maspsx filter replaces each rodata .align 3 with the ORIGINAL's exact pad bytes per a JTBL_PADS spec; fail-loud on table-count drift / non-align-3 / non-jtbl rodata content. Byte-proven: verbatim 0xE4 pad-at-0xCC -> filtered 0xE0 tight (= the merged carve span). - jtbl_carve.py: spec-aware same-subseg merge (gap 0 or 4-with-zero-payload-word; else NON-CONTIGUOUS -> isolate), interval-arithmetic pad specs (committed values CARRIED, never re-derived), JTBL_PADS target-var emission into overlays.mk + revert() restore + stale-.o invalidation; the false 'maspsx drops .align' docstring corrected (H5). - Makefile: $(if $(JTBL_PADS),| jtbl_rodata_pads.py ...) stage in build/src/%.o + file-scope empty default (env-shield). jtbl_family_bank.stub_file: duplicate-stub fail-loud (the earlier 'ladder failure' was a wrong-TU splice into a stale _a.c stub, byte-witnessed). - R22 clean-fleet WITH the fix wired: 140/140 byte-identical, tools-health green (dedup 1846/0, C1 234205/234205), ZERO new banks -- fleet-neutral by construction. - cookbook §8e (the jtbl alignment law) + §8a/§8a-pad corrections; decision-log R31 entry; SETUP.md tool row; .gitignore allowlist for the probe verdict artifacts. |
||
|
Drew T
|
c916115291 |
feat(phase-29 T3): the §53 has_mid_jr INTERLOCK + worker_wave -O0 build-step fix
- family_sweep §53 INTERLOCK (the Task-3 deliverable): a has_mid_jr family CANNOT bank through the carve-less --hseq path — its exemplar's own bank needed a jump-table carve. The sweep now SKIPS them LOUDLY, names the right tool (jtbl_family_bank), and states that a 0% from this path is a TOOL ARTIFACT, not a wall. Uses the manifest's has_mid_jr (family_hseq already derived it — R33, one oracle, shared with dedup_extend; no re-derivation). --allow-jr is the escape hatch. WHY: a silent 0% from the wrong tool is exactly what manufactured the Phase-26 "structural families ≈0% / don't template" doctrine and steered two phases of strategy (§53, Phase-28 T1). The interlock makes the omission impossible to misread. VERIFIED (live-fire + negative control, R32/R35): manifest carries 11 has_mid_jr families / 163 member-slots of 1418 sweepable; --band substantial -> skips 9 families / 155 slots (159 -> 150 families); --allow-jr -> 159 (does NOT skip). It correctly flags 0x8017bebc (n=115) = B2, the very family whose missing carve produced the ≈0% doctrine (0/8 -> 102/115 once carved). - worker_wave.js: pass --o0 to the match_one self-check for -O0 targets (new `o0` target flag) + an -O0 prompt block (§18/§18-P29: the %lo-fold residual and the array-of-struct crack). Without it an agent iterates a -O0 target against an -O2 compile and can NEVER match — the same wrong-build-step trap that produced the Phase-28 "~3%" swing number (Task 1). Exercised live by func_8013C414. - byte-neutral: tooling only, no src/config touched. |
||
|
Drew T
|
941cd37b19 |
feat(phase-29 T6): tiny-IMM mega-pools CRACKED +4,801 via family_sweep --fix-def-sig (fleet 70.4->71.0% instr)
- THE FIX (new): family_sweep --fix-def-sig (header_sig_map + reconcile_def_sig, 1005 mapped fns) — rewrites each member draft's DEF signature to the shared-header (engine_core.h) canonical decl. Root cause (byte-proven, R14/R35 after 3 masked-metric mis-reads): engine_core.h forward-declares the member (extern void func_8015FAAC(s32 *a0), a shared fn calls it) while family_remap copies the EXEMPLAR sig (void *a0) -> 'conflicting types' -> member TU never compiles. Invisible to standalone diff_regions/match_one (no header conflict) AND to --reconcile. Byte-neutral (ptr-type param, gate arbitrates G3/P9); one member hand-verified byte-identical first. - 0x80131eec 2331/2470 (94%) + 0x80130d0c 2470/2496 (99%) = 4,801 members banked across 405 overlay files. R22 clean-fleet 140/140 byte-identical; pure-reduction (0 new/dup stubs); dedup 1840/0; 0 NON_MATCHING (G4). Fleet instr 70.4->71.0% / distinct 52.3->53.2% / fn-count 84.94->86.30%. - CORRECTS the commit:0665 'symbol-definition gap' scout (WRONG). The 4th 'reproduce the build step' instance (§53-carve, -O0-flag, now the member's canonical DECLARATION). cookbook §54, decision-log R31. |
||
|
Drew T
|
f6f89781ff |
feat(phase-29 T2 Arm A): swing verdict = BANKED FACT (9/9 -O0 members on ov_SC07_010); fleet -O0 rollout deferred at the splat wall
- tools/rollout_o0_cluster.py (new) + Makefile O0_CLUSTER_OBJS -O0 wildcard: the -O0-cluster carve (0x13410..0x14834), adapting rollout_whale_o0.py to a 3-way <ov>/<ov>_o0/<ov>_o2b split - ov_SC07_010: carve byte-neutral -> family_sweep --hseq banked 9/9 -O0 exemplar-family members whole-binary (R22 clean-fleet 140/140). The Task-1 masked-MATCH swing verdict is now a BANKED FACT: -O0 cluster members DO bank at -O0 (§52b). Phase-20 'func_8013B7AC overlay-local' refuted. - THE WALL (byte-proven, TOOLING not compiler): the same carve on 006/007/011 byte-shifts the whole image (+0x20 %lo data-symbol shift, 34% diff) from a CLEAN build; boundaries verified as real fn-starts. Root cause = splat re-disassembly of a 3-way-split subseg that still holds INCLUDE_ASM stubs (the whale's stub-free _o0b shape avoids it). The Phase-20 '-O0 split infra' wall, root-caused. - DEFERRED (ROI): full -O0 fleet rollout (~1,233 / ~0.6pp) — 3/4 sampled walled + 134 jr-embedded + bigger levers (Task 3 core-cracks, Task 6 tiny-IMM ~5,566). decision-log R31 + cookbook §18-P29. - 140/140 byte-identical; dedup 1840/0; 0 NON_MATCHING (G4); main 143dbb89. Task 2 substantively done. |
||
|
Drew T
|
2e84b53b2d |
feat(phase-29 T1): the swing number RESOLVED — (a) tooling, an -O0 compile-flag artifact (not a wall)
Phase-29 opens by running the Phase-28 disambiguating probe before scaling any "(cores)x(reach)" arithmetic on the legacy-PURE-non-jr "~3% as-tooled" swing number. - NEW tools/diff_regions.py (the deferred roadmap tool): remap the exemplar exactly as family_sweep --hseq stages it, compile at the EXEMPLAR's real opt level (auto-detected from the Makefile -O0 rules), masked-diff vs target, classify each member O0-FLAG / TEMPLATES / PLUMBING-ISO / REGALLOC / NO-TARGET. Composes family_remap + match_one + masked_diff (R33). - VERDICT (byte-proven): the 274 DIFF is dominated (~272) by two -O0 families (0x8013c964, 0x8013c938) that family_sweep compiled -O2 (member stub files are -O2); an -O2 compile of an -O0 target can never match. Compiled -O0 they masked-MATCH (func_8013C964->MATCH(10), func_8013C938->MATCH(11)). 106-member sample across nins 2..133: O0-FLAG 45 / already-banked 29 / TEMPLATES 17 / type-lift-plumbing 15 / REGALLOC 0. ZERO codegen walls. - The "~3% ceiling" is RETIRED (a tooling artifact — the 3rd structural wall to resolve to tooling after B2 and SC07). The member track is a mechanical -O0 split rollout + type-lift sweep (Task 2a), NOT a per-member grind; member_adapt.py not needed here. - Honest scope (§52b, R14/R35): masked-MATCH is a CANDIDATE; whole-binary banking is Task 2a's gate. Verdict + byte evidence -> docs/calibration.md + docs/decision-log.md (R31). |
||
|
Drew T
|
45fb6affbb |
fix(phase-28 T7): worklist --assert-partition read the wrong manifest key (silent default)
- worklist.py:194 read manifest.get("source", "ov_SC01_077") but the manifest key is
`source_overlay` -> the ", ov_SC01_077" DEFAULT always fired, so the R32 partition assertion
checked a hardcoded binary regardless of the manifest, and would print "PARTITION OK" while
validating the wrong one. Now reads source_overlay and FAILS LOUD if the key is truly absent
(refuses to guess a default — R32/R35). Still passes: 223 live stubs, 223 rows -> PARTITION OK.
- worklist.py:100 n_overlays default 134 -> 138 (defensive; the manifest always provides 138 since
the P27 disc audit, so this default is never actually hit).
NOTE on the other "134" mentions (backlog.py:22, second-oracle.md:60, build_fuel_manifest docstrings,
worklist.py:13): those are descriptive ×134-LEVERAGE prose, not functional — the actual reach is
recomputed live from the 138 sigs everywhere (reach_histogram globs .run/sig.ov_*.jsonl). Left as-is
rather than burn a phase on cosmetic docstring edits that aren't wrong (the leverage concept holds).
|
||
|
Drew T
|
6d9af19482 |
fix(phase-28 T7): disc_code_sweep was blind to COMPRESSED code — the type-4 row was vacuous
The disc-completeness oracle (R34) decoded only the RAW payload bytes, so LZSS-compressed type-4 overlay code read as noise: every type-4 row said "code 0" — a VACUOUS row for 138 known-code binaries. This is not cosmetic. The tool exists to answer "what code did nobody onboard", and it could NOT have found the 4 hidden SC07 overlays (their code is compressed like every type-4) — they were caught by hand-reconciling 138-vs-134. It found the 39 type-1 modules ONLY because those happen to be uncompressed. - FIX: code_signals now decodes BOTH layers — the raw bytes AND the lzss.decompress() output — and takes the stronger code signal, recording which layer (raw|dec) in the report. Uncompressed code (type-1 resident-class) lives in raw; compressed code (type-4 overlays) lives in the decompressed layer. Reuses the extractor's own game-semantics lzss decoder (R33), not a second one. - onboarded_payloads() now also keys by the .dec-stripped raw path: a type-4 _EXE is the .dec, but the sweep iterates raw payloads, so without this all 138 type-4 overlays — now correctly seen as code — would false-flag as HIDDEN. - RESULT: type-4 row 138 payloads / 138 code / 138 onboarded / 0 HIDDEN (was "code 0"). The 138 detections are all via the `dec` layer (verified: 100% valid, 3.39% jr). A 139th un-onboarded type-4 overlay would NOW flag HIDDEN — structurally impossible before. The 39 type-1 modules are unchanged and reconcile with the committed disc-completeness.md. - Coverage still asserted (R32): 1189/1189 classified. Tonight's separate exhaustive ad-hoc sweep independently confirmed no further hidden overlays; this makes that a REPRODUCIBLE tool, not a one-off script. Ranked list -> .run/disc_code_sweep.txt (the file disc-completeness.md references). |
||
|
Drew T
|
758c4365b6 |
feat(phase-28 T7): make audit-binaries — the R36 citizenship gate (R32 enforcement)
R36: a newly-discovered binary is not real until every consumer knows it. Onboarding produces a
byte-CLEAN binary (check-all green) that is not yet a CITIZEN — the tools that enumerate binaries
can each be silently unaware of it, and the byte-gate is structurally blind to that (R34).
Not hypothetical: P27 onboarded 4 SC07 overlays byte-clean; P28 found FOUR consumers silently
ignoring them (family_remap.img_path, .run/family_hseq.json, config/dedup.us.yaml, and the overlays'
own .c), hiding ~6,400 already-matched bodies. Every failure was silent.
- tools/audit_binaries.py asserts, coverage-checked BOTH directions (R32), against the config the
BUILD reads (R33 — onboarded = main + resident + every config/splat.ov_*.yaml):
1. dup_report.BINARIES (what corpus/family_hseq/progress all derive from) EXACTLY equals the
onboarded set — a missing binary is invisible to every derived tool; a phantom is invented.
2. every onboarded binary has a byte-derived sig.
3. THE LOAD-BEARING SC07 CHECK: every onboarded OVERLAY's .c includes ../shared/engine_core.h,
or no shared body can ever reach it (main/resident have their own bodies, exempt).
4. every onboarded overlay is represented in the family map (warn — regenerable/may post-date).
INFO: dedup-group membership (0 = onboarded-but-un-harvested, a dedup_extend candidate).
- NEGATIVE CONTROL: stripping the shared include from ov_SC07_006.c makes the gate FAIL loudly and
exit 1 — it catches the exact bug that hid 6,400 bodies for a month. Restored clean.
- Wired into `make tools-health` (the pre-matching ritual) — cheap (config + text scans, no build),
so it sits in the fast lane. Passes today: 140 onboarded, all full citizens.
- Reads config/dedup.us.yaml as TEXT (never a YAML round-trip — the H5 lesson from T4).
|
||
|
Drew T
|
1dbf5477d0 |
perf(phase-28 T6): harvest_verify --chunk 1 no longer double-builds every failing draft
The gate's hot path is --chunk 1 (the prescribed default — chunked failures mis-attribute innocent neighbours, cookbook:1568). With an atomic chunk the old code fell into the bisect loop and re-ran attempt([fn]) on the SAME single element against the SAME baseline: a byte-identical DUPLICATE build. classify_fail reads _last_sha/_last_err, which the failed attempt(chunk) ALREADY set, so the re-attempt bought nothing but a second cc1+maspsx+as+ld. - FIX: an `elif len(chunk) == 1:` branch classifies + reports directly, skipping the bisect. Per-draft failure build cost 2 -> 1 in the loop; ~26% fewer builds overall at the measured 65% bank rate (successes were always 1 build; only failures doubled). Every wave, one branch. - CORRECTNESS unchanged: verified a guaranteed-failing draft still classifies DIFF and is reported, not silently dropped. Measured make-build calls for one failing draft = 2 (attempt + the single final confirming build), the redundant re-attempt gone. - Gate tooling only — no src/config/build change, cannot alter any binary's bytes (a bug here can only fail-to-bank, never falsely bank; G3/P9). No R22 owed. Found by the explore agent's gate-substrate survey (Phase-28 planning); the shard-farm half of T6 is deferred (conditional on P29 volume — the distinct-binary farm already exists in bulk_harvest, and the same-binary axis is a directory-shard build, documented in CURRENT_PHASE). |
||
|
Drew T
|
876dc7f053 |
feat(phase-28 T5): resident 21 -> 14 stubs (7 banked, 90.34%) + fix match_one's fake isolation
Ultracode wave: 16 isolated drafters over the resident's non-jtbl stubs (the 5 jtbl deferred —
they need the rodata-island carve, §53). Drafts only; the whole-binary byte-gate arbitrated after.
- BANKED 7/16, byte-gated: func_800CEFD0(77) func_800D0D7C(45) func_800D1B80(22) func_800D1E28(37)
func_800D1FC8(62) func_800D29F8(172) func_800D2D10(39).
Resident REAL 122 -> 129, stubs 21 -> 14, byte-ident 124/145 (85.52%) -> 131/145 (90.34%).
FLEET instr 9017152 -> 9017606 (+454 ins). R22 make clean && extract-all && check-all ->
140 passed, 0 failed of 140 (the first R22 was killed by a terminal crash and RE-RUN, not assumed).
Ground truth on 14 agrees 3 ways: source grep, splat-emitted stub .s count, progress.py.
- §52b's LAW, MEASURED AGAIN INDEPENDENTLY: the agents self-reported 11 match_one MATCH; the
whole-binary gate banked 7 (64%). All 4 blocked MATCHes died on `conflicting types`
(D_8010EDEC / D_80115110 / func_800D1984 / cdFileLocTable) — the loose-typing def-side wall, NOT
codegen. gate_stage's recovery banked 0/5 on them. A match_one MATCH is a CANDIDATE (G3/P9).
- FIX — match_one's isolation was FAKE, and its own docstring was the false spec. It promises
"Fully isolated (own temp dir) so many run in PARALLEL with no shared build -- a real asm-differ
loop for an agent to iterate against", while `--work` defaulted to the SHARED '.run/match': every
concurrent caller compiled into the same t.c/t.o. FOUND BY AN AGENT MID-WAVE, the only way it can
be found — it read another agent's function out of its own scratch ("found another agent's
func_800D2650 in my t.c") and reported it. Every other agent steered by a loop that could hand it
someone else's compile: a CONFIDENT WRONG verdict, worse than a crash. Default is now a private
.run/match/<fn>.<pid>; the default IS the promise. (Some agents had already worked around it by
passing --work themselves.) The byte-gate was never at risk — it is the sole arbiter — but the
iteration loop the agents steer by absolutely was.
- The 14 remaining: 5 PLUMBING (loose-typing) + 4 DIFF (genuine codegen: func_800D2650 close=4,
func_800CFAD0 close=5, func_800D0E30 close=12, func_800D27DC close=48) + 5 jtbl deferred.
Dossier next (T5b) — the agents' per-function residual analyses are the durable asset (R30).
|
||
|
Drew T
|
2c4e2344da |
fix(phase-28 T5): progress.py — #if 0 blindness + the len()-sum; resident 123/146 -> 122/145
R35: fix the instrument before planting a flag on its denominator.
- #if 0 BLINDNESS: classify() knew `#ifdef NON_MATCHING` (:425) but not `#if 0`, so a dead
analysis body was read as a live definition AND its real INCLUDE_ASM stub counted separately —
the SAME function in BOTH `real` and `stubs`. Live case: resident.c:868-925 wraps a full
void func_800D00E4(s32){...} in #if 0 (its jtbl dossier) and re-declares the stub at :926.
Now the block is skipped entirely: dead code is neither matched nor stubbed.
- THE len()-SUM (the dual defect): `placed` was a set union, so it caught a function in NO bucket
— but `matchable` SUMMED len()s, so a function in TWO buckets counted twice and nothing
complained. matchable/byteident are now SET unions, plus a new OVER-coverage assertion that
fails loudly if any fn lands in multiple buckets. R32 means both directions: nothing missing,
nothing double-counted.
- NEGATIVE CONTROL (the fix must change an answer the old tool gave):
resident REAL 123 -> 122 | matchable 146 -> 145 | 85.62% -> 85.52% | func_800D00E4 no longer
double-counted. FLEET instr 68.9% UNCHANGED (no #if 0 in the overlays) — the fix is scoped.
- FINDING (logged for T7's audit-binaries, does NOT block the flag-plant): the two INDEPENDENT
oracles now agree exactly at 144 — corpus (21 stubs + 123 matched, derived from the tree) and
sig_image (the 2nd oracle) — with EMPTY set difference both ways. progress.py still reports 145
because it counts func_800CEDFC and func_800D33E0, which are DEFINED in resident.c but absent
from sig_image. 0x800CEDFC is the resident's vram base +4 (the first function, code starts at
file offset 0x4 after the leading data word), yet make audit-corpus reports 0 PHANTOM +
0 TRUNCATED. Either sig_image has a boundary blind spot or those defs are not image functions.
progress.py's text-scanning classify() is exactly the re-parsing R33 says should be DERIVED from
corpus instead — a real refactor, logged not rushed.
The flag-plant claim is unaffected: it rests on corpus.stubs('resident') == 21 (tree-derived,
verified 5 ways), not on the contested denominator.
|
||
|
Drew T
|
fda9eebb42 |
fix(phase-28 T4): wire all 4 SC07 overlays (6174/6457, 95.6%) + REPAIR the registry I destroyed
Completes T4 and corrects two defects I introduced, both landed in commit:0649. - WIRED: 006 1543/1614 · 007 1544/1615 · 010 1544/1614 · 011 1543/1614 = 6174/6457 = 95.6%, ~0 agent tokens. Stubs/overlay ~2400 -> 831/984/898/825. Fleet instr 67.0 -> 68.9%, fn-count 82.16 -> 83.94%. dedup-check 1840 validated / 0 failed; groups now read "138 members [138 binaries]" (was 134); C1 coverage 227211 -> 233385 = exactly +6174. R22 make clean && extract-all && check-all -> 140 passed, 0 failed of 140 at every stage. - FIX #1 — I DESTROYED THE REGISTRY'S DOCUMENTATION, AND EVERY GATE CALLED IT GREEN (H5). The first cut wrote config/dedup.us.yaml with yaml.safe_dump, round-tripping the whole file: 47 comment lines -> 0 (including the curated Phase-11 header explaining WHY the share is source-level) and 1832 `vram: 0x80162FF4` -> `vram: 2148937716` (PyYAML parses YAML-1.1 hex to int; dumps int as decimal). 25,948 lines rewritten. It passed dedup-check 1840/0 AND check-all 140/140 because _addr() accepts both forms: THE DATA WAS CORRECT AND THE DOCUMENT WAS RUINED. Fixed forward (R6, no history rewrite): restored from commit:0649~1 and re-applied the 6174 memberships via a surgical text edit (add_members_surgical). Verified: 1545 insertions / 1545 deletions, 0 non-`binaries:` lines changed, 47 comments + 1908 hex fields intact, and the rebuilt fleet is byte-identical to the destructive version (140/140). THE LESSON: every oracle this project owns measures BYTES, so a formatting-destructive write is invisible to all of them by construction. R34 says the byte-gate is a null COVERAGE oracle; this is the same hole one layer out — it is a null DOCUMENT oracle too. - FIX #2 — I MIS-REPORTED THE DIFFs, TWICE (R14). (a) commit:0649 claims ov_SC07_006's 71 non-banks were "ALL PLUMBING, ZERO DIFF". FALSE — I read head -6 of the classified file and generalized. It has the same 4 DIFFs as the others. (b) I then built the jr guard assuming those 4 were the §53 jr class BECAUSE ov_SC01_077 hosts them in _jr_8017A4AC.c / _jr_80182268.c. has_mid_jr is FALSE for all four (33-52 ins, no jump table): they merely live in a carved jr-REGION split, which sweeps in every function in its address range. HOSTING FILE != FUNCTION CLASS. The guard is KEPT (preventive, §53-correct, currently skips 0 — no jr fn is in the extendable set) with its docstring corrected to record what it is NOT. The 12 DIFFs (0.19%) are UNDIAGNOSED and logged, correctly left as stubs by the gate — not dressed in a story. - The 283 non-banks: 271 PLUMBING (the loose-typing conflict class + the whale, whose body lives in src/shared/func_80144B9C.h so no DEFINE macro exists to expand) + 12 DIFF. Existing tools cover the plumbing (cast_call_sites / canon_sig_reconcile / reconcile_tu). |
||
|
Drew T
|
c0486fe5f8 |
feat(phase-28 T4): dedup_extend — wire newly-onboarded binaries in; ov_SC07_006 1543/1614 (95.6%)
The 4 SC07 overlays P27 onboarded were byte-clean but NOT citizens: their .c included only
common.h (never ../shared/engine_core.h), so no shared body could reach them, and they
appeared in ZERO dedup groups (1689 groups read "134 binaries", never 138). Each sat at ~80
matched / ~2400 stubs while its siblings were ~2150 matched.
- NEW tools/dedup_extend.py — the missing mode. dedup_propagate is built for CRACK -> AUTHOR
MACRO -> INSTANTIATE: --auto-from scans INLINE DEFS (planned only 11 here; the ~1600 shared
bodies are ALREADY DEFINE_func_* macros in engine_core.h) and --addr dies "no source overlay
has it matched" because no overlay holds an inline def. Extending an existing MACRO-BACKED
group to a newly-onboarded binary is a different operation and nothing implemented it.
- SAFETY (explicit — this feeds the byte-gate): h_exact is the SHA1 of RAW INSTRUCTION BYTES, so
two instances sharing one are identical INCLUDING their jal/lui/%lo reloc immediates — same
callees, same data addresses, same symbols. The body that compiles byte-identically at one
member does so at the other with NO remap. (Exactly why dup_report calls h_exact "guaranteed
byte-match" and h_norm "candidate-only".) A bug here can only FAIL TO BANK, never falsely bank.
- REUSE, DON'T REBUILD (R33): owns only the set computation + the registry edit. The splice and
the gate are harvest_verify verbatim (it already derives each stub's home TU from the corpus
oracle, chunks + bisects, reverts on failure). h_exact members are byte-identical by
construction -> the happy path is ~1 build per binary, not one per function.
- RESULT ov_SC07_006: 1543 / 1614 banked = 95.6%, ~0 agent tokens. Stubs 2374 -> 831.
The 71 non-banks are ALL PLUMBING, ZERO DIFF, in two named classes with existing tools:
* func_80144B9C "undefined reference" — the whale's body lives in src/shared/func_80144B9C.h
(the -O0 shared header), not engine_core.h, so no DEFINE macro exists to expand.
* "conflicting types for D_800A5E60 / func_8012C750 / func_8012C0EC" — the loose-typing
conflict class (cast_call_sites / canon_sig_reconcile / reconcile_tu already exist for it).
- GATES: R22 make clean && extract-all && check-all -> 140 passed, 0 failed of 140, 0 FAIL lines.
dedup-check 1840 validated / 0 failed; groups now read "135 members [135 binaries]" (was 134);
C1 coverage 227211 -> 228754 = exactly +1543. The second oracle accepts the extension.
- Mechanism had been proven by hand first (probe-before-investing): +include + ONE stub ->
DEFINE_func_80128158() -> ov_SC07_006 built 7ca772be BYTE-IDENTICAL, then reverted.
|
||
|
Drew T
|
65e96e5d5a |
feat(phase-28 T2): purge the poisoned grinder blacklist — 8 of its 22 had MATCHED anyway
The blacklist recorded "permuter won, byte-gate rejected => plumbing-bound, never
re-permute" for 22 fns. It was manufactured by a gate that no longer exists.
- R14 ON THE PREMISE, TWICE (before touching anything):
* The roadmap's two named grinder bugs are ALREADY FIXED (commit:0327, commit:0200) — stale line.
* docs/tooling-audit.md:933-937 DOWNGRADED ITS OWN FINDING with three corrections: the
prescribed fix is a NO-OP (deleting the scanner banks zero fns — harvest_verify was
single-TU BY CONSTRUCTION, the defect was mislocalized to it); nothing is being discarded
now (grinder STOPPED since 2026-07-02 — confirmed via .run/auto/STOP); queue magnitude
inflated (1252, not 1298). T2's only real content was the persisted artifact.
- AND THE AUDIT'S SNAPSHOT IS ITSELF STALE (verified in code): harvest_verify is now fully
multi-TU — _stubs derives every stub across every TU from the corpus oracle (:122), render()
splices "each draft into the TU that actually holds its stub", _write() writes multiple
paths, un-stubbed drafts are REPORTED not silently dropped (R32). The gate that manufactured
the blacklist is gone. No harvest_verify change was needed or made.
- THE PROOF IT WAS MANUFACTURED, NOT OBSERVED: of the 22 entries, 8 have since MATCHED anyway
(func_80131D68/80149374/8014FE60/80150528/8016BBE0/80171C64/80174684/8017F290); the other 14
are still stubs and would have been skipped FOREVER on a dead gate's verdict. (16 of 22 were
split-hosted, so harvest_verify never compiled them — the permuter's byte-matches were
discarded UNBUILT. The audit predicted 5 matched-anyway; it is 8.)
- DONE: blacklist -> [] (poisoned copy preserved at grinder_blacklist.json.poisoned-pre-T2);
grinder.py now carries the RULE (R35): a blacklist entry is a verdict from a SPECIFIC GATE
and EXPIRES when that gate changes — purge and re-derive from the fixed gate, never inherit.
A persisted negative verdict is only as good as the instrument that produced it.
- No byte claim (analysis tooling + a scratch artifact only) -> no R22 cycle owed.
|