mirror of
https://github.com/Druthulu/BFM-decomp
synced 2026-09-27 05:56:00 -04:00
daec0e9f54a540aefac34bfafe57dcdc8eee5311
290 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Drew T
|
55cd894024 |
perf(phase-29): family_sweep gates groups in PARALLEL by default (the SESSION-20 adapter, finally wired)
SESSION-20 measured serial family-sweep gating as "roughly an 8-16x throughput loss on a 32-thread box" and BUILT tools/sweep_parallel.py for it — but only reachable via a manual `--stage-only` two-step, so this path stayed serial and three sweeps in SESSION-22 (133 + 273 + 137 members) ran serially for no reason. §101, the stale-default class. SHAPE OF THE CHANGE — deliberately minimal after two failed attempts earlier today. A parallel PRE-PASS (phase 2a) runs only the per-group `harvest_verify` subprocess; phase 2b then consumes the results IN THE ORIGINAL SERIAL ORDER, so every line of post-processing (the MISMATCH backstop, the zero-bank restore, the counters, the prints) is untouched and output stays deterministic. No closure restructuring — that is exactly what broke it twice before. SAFETY, not a new claim: the Makefile already builds binaries concurrently (check-all/extract-all use `xargs -P$(JOBS)`, JOBS=16) and bulk_harvest's farm does the same with a per-binary lock. The §28 hazard is two makes racing on the SAME artifacts, prevented by the per-overlay lock (two splits of one overlay build the same binary and therefore serialise). NEGATIVE-CONTROLLED BOTH WAYS: `--stage-only` stages identically under -j1 and -j12 (4 groups each); a full gate returns IDENTICAL tallies (0 banked / 4 failed) parallel vs serial; tree clean after both. HONEST MEASUREMENT: on the only sample available (4 groups, and they fail FAST on a compile error rather than running full builds) parallel was 4s vs serial 6s — ~1.5x, NOT the 8-16x. That figure needs a large family (137 groups of full builds) to show, and every such family was already banked today. The wiring is proven correct here; the throughput claim remains SESSION-20's measurement, not mine. `-j 1` restores the old behaviour. |
||
|
Drew T
|
ab065b1646 |
fix(phase-29): sweep pinned exemplars BY DEFAULT — the §42e guard's cause was removed in Phase 27
The guard skipped any exemplar carrying a `register __asm__("$N")` pin because templating it
cc1-CRASHED the sibling TUs (§42e). Phase 27 BYTE-PROVED that SIGABRT was `extract_unit` dropping the
body's file-scope macros — OUR bug — and fixed it (_carry_macros); its own roadmap delta then put the
PINS class "back on the mechanical-harvest table". The cause was removed and the default never
changed, so the guard kept skipping real work.
MEASURED THIS SESSION on one family: func_80175AB8 reported `skipped {'pinned-exemplar': 137}` and
then banked 133/137 the moment it was bypassed (R22 140/140). A protection whose cause is gone is not
free — it is a silent skip (R32) wearing a safety label, and the whole-binary byte-gate was always the
real arbiter here.
--allow-pins kept as an accepted no-op so existing recipes/docs keep working; --no-pins restores the
old behaviour. Negative control: --no-pins still reports `pinned-exemplar: 4`; the default stages them.
This is the THIRD stale default found today, after sweep_parallel being opt-in (8-16x throughput left
on the table) and conform_decls refusing a remedy it could perform. Same shape each time: correct when
written, cause since removed, still the default, opt-out only if you remember the flag.
|
||
|
Drew T
|
5f1fc5b5eb |
feat(phase-29): caller pair banked (57,822 templ ins) via K&R defs — §92's remedy corrected (§99)
func_80175AB8 + func_80175DA8 both banked. R22 clean-fleet 140 passed / 0 failed of 140.
§92 SAID these need "the §17a-1 caller pair, NOT a bare conform" — the diagnosis was right (conforming
a narrow param changes argument promotion at every call site, measured PLUMBING -> DIFF) but the
remedy was the expensive one. The actual fix touches NO declaration: convert the DEFINITION to K&R,
where a narrow param PROMOTES to int (C89 6.3.2.2) and is therefore already compatible with the
fleet's existing `s32` prototype, while still emitting narrow-param codegen. §43 applied to the def
side. T0 draft-only, ZERO blast radius, versus a 524-site fleet conform.
THREE reconcile_tu BUGS SURFACED, ONE OF THEM MINE:
(a) BLIND TO BLOCK SCOPE. split_statements is depth-0 BY DESIGN, and §8d deliberately demotes data
externs into the function body — so the tool saw one statement and no declarations, printing
"reconciled: 0 draft(s), 0 data symbol(s); coverage defects: 0" for a draft cc1 rejected with
`conflicting types for D_8011F7BC`. A silent skip (R32). Fixed: descend one level.
(b) MY BUG, introduced by (a): descending into ANY `{` also enters struct/union/enum definitions, so
MEMBERS parse as declarations and get conformed — `u32 code;` became the TU's
`typedef void (*code)(unsigned short*);` INSIDE the struct, and `p->code` became
`p->(*(u32 *)&code)`. Caught by DIFFING THE TOOL'S OUTPUT AGAINST ITS INPUT before trusting it;
the byte-gate would have said PLUMBING and explained nothing. Guard: function bodies only.
(c) LATENT since the tool was written: _cast_sub matched bare identifiers and rewrote MEMBER ACCESSES
as globals. Unreachable until (a) existed. Guard: (?<![.\w])(?<!->).
cookbook §99.
|
||
|
Drew T
|
445b149279 |
perf(phase-29): -j16 on jtbl_family_bank's make calls — measured 16s -> 14s per sibling (12%)
MEASURED, not assumed. Baseline ~18s/sibling (92 siblings in 27:37). Profiling a realistic cold cycle: `make extract` 3s + `make build` 2s = ~5s of the 16s, so MAKE IS NOT THE BOTTLENECK and -j cannot be the 8-16x lever. Confirmed end-to-end on one sibling: 16s -> 14s. Where the rest goes: the per-sibling loop tries up to FOUR stages (raw -> scoped -> recovered -> reconciled) and EACH runs its own `make build`, plus jtbl_carve and remap/canon_sig_reconcile. WHY THE REAL LEVER IS NOT DONE HERE. Cross-sibling parallelism is worth ~8-16x on this 32-core box (each sibling is an independent binary, and the Makefile already proves per-binary parallel builds safe: check-all/extract-all run `xargs -P$(JOBS)` at JOBS=16). It is blocked on a specific hazard, not on effort: `revert()` restores config/ from git, and `config/overlays.mk` is SHARED, so a concurrent revert would clobber peers' carve entries — the same "revert-from-HEAD eats another worker's state" failure this tool's own precondition check warns about. Safe parallelisation needs line-scoped + locked + atomic edits to overlays.mk and a revert that never wholesale-restores shared paths. Designed, not built. ALSO REVERTED THIS SESSION: an attempt to make parallel gating the default in family_sweep. The adapter for it already exists (tools/sweep_parallel.py, built SESSION-20 after measuring the same 8-16x loss) but is only reachable via the manual `--stage-only` two-step, so the default path stayed serial — and three sweeps today (133 + 273 + 137 members) ran serially for no reason. Wiring it is right, but my patch broke the tool twice (missed import, then a closure-scope error) and family_sweep banked 543 members today. Restructuring a proven tool with blind string replaces at the end of a long session is how a working thing gets broken; reverted and left as a specified next-session task. |
||
|
Drew T
|
2f1aa8659d |
feat(phase-29): func_801789AC banked — conform_decls can now FIX the arity precondition it diagnoses
STUCK SINCE SESSION-21, and conform_decls was RIGHT to refuse it: the byte-true signature takes a parameter while 138 zero-arg CALL SITES exist across 138 files, so conforming the declarations alone would turn every one into `too few arguments` — a fleet-wide compile break the per-binary gate cannot see. The tool printed the exact remedy in its refusal message and could not perform it, so the function sat blocked for two sessions. NEW --cast-zero-arg-calls: cast every 0-arg call site to ((s32 (*)(void))func_801789AC)() — gcc folds the cast of a known symbol to a direct jal, so caller bytes are unchanged — then re-run the conform normally. PLAN -> VALIDATE -> WRITE like the decl axis, because a partial cast set is itself a fleet-wide compile break. Not a macro this time (unlike func_8015B950's single shared site): 138 genuine per-overlay call sites, one each. VERIFIED IN STAGES, not all at once: the 138 casts ALONE are byte-neutral (d19c9580); then the 660-site declaration conform (axis complete, 0 remaining); then the gate -> verified 1 / failed 0; then R22 clean-fleet 140 passed / 0 failed of 140. Reach 138 x 91 ins = 12,558 templated instructions unlocked for the sweep. |
||
|
Drew T
|
ee55bd5cae |
feat(phase-29): func_801330E0 banked — conform_decls taught to read K&R definitions
THE GAP: conform_decls could not parse a K&R definition at all — it exited "no DEFINITION found, refusing to guess". Honest, but §43 (a narrow param declared K&R-style, producing the in-place `sll $a2,$a2,16` tell) is a documented, load-bearing idiom here for exactly the narrow-param class. So the tool was silently refusing the drafts that most need it: a whole idiom family read as "nothing to conform" (R32 coverage). THE SUBTLE PART IS PROMOTION (C89 6.3.2.2). A K&R definition promotes each narrow parameter, so a prototype in scope must declare the PROMOTED type or gcc rejects the pair with `argument 'x' doesn't match prototype`. That is why the fleet prototype reads `s32 a2` for a parameter the definition declares `s16` — and why emitting the declared (unpromoted) type would RE-CREATE the narrow-param conflict this tool exists to remove. The parser now promotes s8/u8/char/s16/u16/short -> s32 and float -> f64, pointers untouched, and reports (R32) any K&R param with no declaration. RESULT: byte-true signature read as `void func_801330E0(void *, s16 *, s32)`; the only real change vs the fleet's 973 declarations was param_1 `s16 *` -> `void *` (a pointer shape, caller-neutral). 973 sites / 973 files rewritten, axis complete. Gate: verified 1 / failed 0, d19c9580 BYTE-IDENTICAL. R22 clean-fleet 140 passed / 0 failed of 140. Reach 138 x 110 ins = 15,180 templated instructions unlocked for the family sweep. |
||
|
Drew T
|
ad57c16e61 |
feat(phase-29): func_8014CF04 + func_8015D1B8 banked; conform_decls had 3 defects R22 caught (§98)
THE BANK: the T14 PLUMBING census showed func_8014CF04 blocking THREE drafts at once. Conforming its
decl axis banked func_8014CF04 + func_8015D1B8 (func_80135260 is a genuine DIFF, agreeing with its
independent SESSION-21 diagnosis). R22 clean-fleet 140/140; report fail-closed green (dedup 1886/0,
0 NON_MATCHING). fn-count 317,896 -> 317,898; distinct 66,110 -> 66,111.
BUT THE AXIS WAS A 1,748-FILE T2 WRITE SET (the --check per-form counts read "1"), and R22 came back
139/140 -- TWICE -- on a change the per-binary gate called BYTE-IDENTICAL. Three defects (§98):
1. THE REGEX CROSSED NEWLINES. `[^;]*` matches '\n', so a match starting at a DEFINITION line ran
past the `{` to the first `;`, swallowing `s32 func_8014CF04(...) {` PLUS the register pin on the
next line and replacing both with a prototype -> undefined reference. Fixed to `[^;{\n]*`: a
definition is now unmatchable by construction.
2. IT REWROTE INSIDE COMMENTS (H5, 3 lines). Now scans cdecl._mask() and rewrites by SPAN (R33 --
that length-preserving primitive already existed for exactly this).
3. THE REAL CAUSE -- IT ASSUMED ONE SIGNATURE FITS THE FLEET. ov_SC07_006 carries its own banked
definition with a DIFFERENT byte-true signature ((s32,s32,void*) vs (s32,void*,void*)), under a
decl marked "per-overlay-local decl (byte-true sig); do NOT re-macroize". That is the Phase-16
loose-typing wall inside a tool that structurally assumes it away. NEW RULE: a TU that DEFINES the
function owns its own declarations; a fleet axis is meaningful only for CONSUMING TUs. This grows
more common as banking proceeds -- every overlay that banks a function becomes an exception.
Then the R32 completion assertion cried wolf on its own by-design skip ("HALF-AXIS -- DO NOT BUILD"
for a complete rewrite): an assertion must be exact about its DOMAIN, not just its condition. Scoped
to consuming TUs -> 1,747 sites, 1 excluded by design. Also hardened to PLAN -> VALIDATE -> WRITE;
the refusal path had aborted mid-write while claiming nothing was modified, creating the very
half-axis §85 calls a guaranteed break.
META (R22's premise, re-earned): after fixing defect 1 I EXPECTED R22 to pass; it failed again for an
unrelated reason, and an individual `make build` of the failing binary SUCCEEDED by reusing objects
the clean run rebuilds. An incremental pass does not refute a clean-tree failure.
|
||
|
Drew T
|
97cd2739b5 |
fix(phase-29): the gate manufactured 3 false CC1-FAIL verdicts — carve-refusal, tree hygiene, R32 (§97)
A 15-draft harvest_verify batch reported CC1-FAIL=4 and `final SHA None`. Three of the four were the HARNESS, not the compiler. Checked the tree FIRST (the MISMATCH is a tree alarm, not a result), reverted to the committed baseline rather than reasoning about a half-applied state, rebuilt -> d19c9580 BYTE-IDENTICAL. No banked result was ever at risk: the byte-gate cannot manufacture a match, but it CAN manufacture a verdict — and verdicts are what the backlog and roadmap are built from. ORDERING PROVED THE CASCADE (R14): items 1-11 are real (9 PLUMBING, 2 DIFF), all before item 12 — jtbl_carve REFUSING func_8013B83C (§59(3) non-contiguous same-subseg table). Items 13-16 are four CC1-FAILs on the SAME ov_SC01_077_o0.o = one refused carve counted four times. THREE DEFECTS FIXED: 1. `_ok` was computed and IGNORED — a refused carve was spliced and built anyway into a guaranteed Error 33, filed as CC1-FAIL. Now a named CARVE-REFUSED class, skipped (one build cheaper). 2. attempt() never restored on failure, so the tree was dirty BETWEEN drafts — and _jtbl_snapshot() snapshots the tree AS IT FINDS IT, so a later carve captured an earlier FAILED draft's splice and its undo faithfully RE-APPLIED it, after the final _write(baseline). That is the entire `final SHA None` mechanism. Invariant restored: the tree is at baseline except while a draft is under test (atomic AND bisect branches). 3. The recovery's own `make extract` rc was unchecked (_sh does not raise — §93's sibling). Now loud. Plus an R32 assertion on the cleanup: at 0 verified a non-empty git status is residue, not a result; it names the files and the recovery command. It fired correctly on its first real run. MEASURED RECOVERY (same drafts, clean tree): func_8013B83C -> CARVE-REFUSED; func_801789AC -> PLUMBING (actionable); func_8017C974 -> DIFF (corroborates its agent's global_alloc spill diagnosis); func_80140958 -> CC1-FAIL (genuinely its own). final SHA None -> d19c9580; tracked diff empty. BLAST RADIUS OF §96, HONESTLY: the reconcile_tu span fix unblocked func_80176218 (banked, swept 133/137) and no other draft in the batch. 7 of the 9 PLUMBING are `conflicting types for <the function itself>` = the DEF-side self-decl axis conform_decls owns — the next lever, now a measured target list rather than a guess. cookbook §97. |
||
|
Drew T
|
8c36ede849 |
feat(phase-29): func_80176218 banked (45,126 templ ins) + reconcile_tu span/R32 fix
THE DRAFT was failing in a CHAIN, one "next conflict" per gate cycle. Applied §95's own diagnostic law instead — splice once, dump EVERY cc1 error — and the whole set named the cause immediately: three errors on TWO axes (one data decl, two callee decls), not three problems. THE DATA ERROR WAS reconcile_tu AGAIN, ONE SHAPE DOWN (§96). split_statements returns comment- STRIPPED text WITH SPANS; the rewrite re-found each planned statement by comparing that text to a raw LINE, so `extern u8 D_80078E78; /* cur base ($s5) */` never matched. The decl was left unconformed WHILE THE USE-CAST PASS STILL FIRED -> a draft whose uses are cast for the TU's storage against the draft's own declaration -> cc1 reports `conflicting types` AT THE VERY DECL THE TOOL JUST CLAIMED TO FIX, exit 0, "reconciled: 3 symbols". FIX: rewrite by SPAN (the primitive existed — its docstring says spans are preserved *because drafts get rewritten*). Plus the R32 assertion the old code was missing: it had a dropped_check counter incremented in two places and NEVER COMPARED — "a loud failure nobody counts is exactly as invisible as a silent one" in miniature. Now declarators-in vs -out AND a per-symbol check that each planned tu.declaration() actually landed, both as `!!` notes so --strict exits non-zero. MEASURED: 3 -> 4 data symbols reconciled on the same draft; trailing comments preserved (H5). THE TWO CALLEE CONFLICTS were the other axis (reconcile_tu skips kind=='func' by construction): cast_call_sites (§20) conformed func_80177AD4 (TU `void (int, unsigned int)`) and func_80178298 (TU `(u32*, u8*, short, short)`) and cast each call site to the draft's intended widths. GATE: verified 1 / failed 0, d19c9580 BYTE-IDENTICAL. Write set is one overlay-local TU = T1 per the §63/§85 blast-radius taxonomy, so the per-binary gate is sufficient; the ×137 sweep is the T2 case and takes a full R22. |
||
|
Drew T
|
db620d4b8d |
fix(phase-29): reconcile_tu dropped the sibling declarators of a multi-symbol extern line
THE DEFECT (on the banking path — gate_stage runs reconcile_tu): its rewrite replaced the draft's declaration LINE with the TU's declaration of the ONE conflicting symbol. A statement can declare several: 'extern u16 D_80078EB2, D_8011F82A, D_8011F82C, D_80078EB4, D_8011F8C4;' where only EB4 conflicts became 'extern s16 D_80078EB4;' — four symbols silently gone. WHY IT HID: the draft does not fail at the declaration. It fails later with 'D_8011F82A undeclared' at a USE, several conflicts down a peeling chain, nowhere near the cause. I peeled four separate 'next conflicts' out of func_80176218 before dumping ALL cc1 errors in ONE build and seeing three undeclared symbols that the tool itself had removed. FIX: group the plan by STATEMENT rather than by symbol; re-emit EVERY declarator (TU's version for the conflicting ones, the draft's own for the rest); note multi-declarator statements; and when a statement cannot be re-parsed, say so loudly instead of emitting only the planned symbols. VERIFIED: all 5 declarators survive, and the same draft now reconciles 3 symbols instead of 2 — the dropped ones had been hiding a further conflict. cookbook §95. The law (R32 again): a transform that REPLACES a syntactic unit must account for everything that unit contained — the STATEMENT, not the line, is the unit of a C declaration. Diagnostic: when a draft fails in a chain, stop peeling one error per gate cycle; splice once and dump every cc1 error, because the shape of the whole set names the cause. |
||
|
Drew T
|
7d9dd6deca |
feat(phase-29): func_8014D820 banked (41,952 ins) + conform_decls scalar-narrowing guard
WAVE 2 (9 never-drafted exemplars, ultracode): 9/9 returned, 5 MATCH / 4 near, 2.25M tokens. BANKED: func_8014D820 (304 ins ×138) — and its agent ROOT-CAUSED the failure I left undiagnosed. It was never an assembler problem: cc1 exit 33, `conflicting types for 'Ent'` vs engine_types.h:434, surfaced by the recipe's `set -o pipefail` and MISATTRIBUTED to `as` because `as` is the last stage in the pipe (Makefile:560). Fixed by moving V4/Desc/Ent to BLOCK scope — byte-neutral and collision-proof across all 138 member TUs. Vindicates flagging it to the agent as UNVERIFIED rather than passing my own guess forward as fact (§88e). R22 clean-fleet 140/140. NEW GUARD — SCALAR NARROWING IS NOT CALLER-NEUTRAL (byte-proven, and it cost 3 gate cycles): conform_decls treated all decl type changes alike. A POINTER change is caller-neutral (func_80179B74 conformed 1,600 sites s16*/short* -> u16* and stayed byte-identical fleet-wide). A SCALAR WIDTH change is NOT: narrowing `s32 a0` -> `u16 param_1` changes argument promotion at every call site. MEASURED on func_80175DA8: decls reverted -> gate says PLUMBING; conform applied -> gate says DIFF. The conform did not fix the draft, it changed the CALLERS. Now warned explicitly (not refused — the draft's sig is still byte-truth for the callee and the gate arbitrates), with the instruction that a DIFF after this conform means examine the callers (§17a-1 pair), not the body. Verified the guard discriminates: fires on func_80175DA8 (s32->u16), silent on func_80179B74. STILL OPEN from wave 2: func_80176218 + func_80175AB8 (DATA-symbol conflicts, D_80078EB4 / D_8011F7BC -> reconcile_decls) · func_80175DA8 + func_80135EB0 (need the §17a-1 caller pair, not a bare conform) · 4 near-misses with precise residuals recorded (func_80176734 129 length-drift, func_80140958 49 inverted-hoist, func_80177B5C 19 sched tie, func_8017C974 83 -> permuter). |
||
|
Drew T
|
e721452526 |
feat(phase-29): 3 more exemplars banked + 3 family sweeps; conform_decls extern-optional fix
BANKED: func_8015B950 (271 ins) · func_8016AE5C (85) · func_80179B74 (111).
SWEPT: func_8015B950 137/137 · func_8016AE5C 136/137 (ov_SC03_108 refused, left a stub rather than
forced). Three full family sweeps this session, all unblocked by the --like role guard.
FLEET 82.4% instr · 70.4% distinct-code (crossed 70%) · 89.72% fn-count. R22 140/140 throughout.
conform_decls has now been right in BOTH directions: it REFUSED func_8015B950 (which by hand broke
138 binaries) and CLEARED func_80179B74 (1,600 sites / 523 files, three decl forms, pointer-type
only). Then it found its OWN coverage gap: it required a leading `extern`, so it reported "no
declaration found" for a TU declaring the function on line 23 without one — a silent miss that reads
exactly like "nothing to do" (R32). `extern` is now optional and PRESERVED where present.
⚠️ INSTRUMENT FAILURE, recorded: mid-session `grep <pat> <file> | head` began printing NOTHING while
exiting rc=0 (i.e. matching). Read showed the line plainly; re-done in Python the file has 3
occurrences including a CALL at line 68. It cost one wrong intermediate claim ("no extern anywhere"),
which conform_decls immediately contradicted. NO banked result is affected — every bank passed the
whole-binary byte-gate and a clean-tree R22, neither of which reads shell output. A broken diagnostic
wastes time; it cannot manufacture a match. Diagnostics moved to Python. §90a, aimed at the shell.
func_8013BD74 is NOT a wall: its byte-true def takes a draft-LOCAL struct `A`, conforming the
prototype fails `parse error before '*'` (A undeclared that early), and the prototype cannot be
deleted because a call at line 68 precedes the definition at 71. Needs the §20/§64 type-lift.
|
||
|
Drew T
|
7c1640888f |
feat(phase-29): func_8016AE5C banked + tools/conform_decls.py; a 138-binary break R22 caught
BANKED: func_8016AE5C (85 ins ×138). R22 clean-fleet 140/140.
⚠️ I BROKE 138 OF 140 BINARIES AND R22 CAUGHT IT — the per-binary gate could not.
Conforming func_8015B950's decl from `(void)` to its byte-true `(s32 arg0)` across 926 sites gated
BYTE-IDENTICAL on ov_SC01_077 and broke 138 other binaries with Error 33. §63/§85 exactly: a T2
write set is provable only by R22, and the binary the gate authorises is not the binary that breaks.
MECHANISM: conforming a decl to a signature that TAKES parameters makes every existing 0-ARG CALL
SITE a hard `too few arguments` error once a prototype is in scope. Not a declaration-only change.
PROCESS NOTE (mine): a first R22 reported 138 failures, an individual rebuild of a "failing" binary
said BYTE-IDENTICAL, and I nearly filed it as a flake. The second clean R22 reproduced it exactly —
the individual build passed only by reusing objects the clean run rebuilds. An incremental pass does
not refute a clean-tree failure; that is R22's whole premise, pointed at me. Reverted to a known-good
baseline (a stray jr_isolate region file was also in the tree) and redid the one good bank cleanly.
NEW tools/conform_decls.py — because applying this axis by hand three times in one session is how a
half-axis happens. Derives the byte-true signature from the DRAFT's definition (§58b), rewrites EVERY
site, asserts completion (R32). Encodes both preconditions: the §85 return axis (refuse if any caller
consumes the return) and a NEW arity precondition (refuse if 0-arg call sites exist, naming the cost).
The guard immediately gave a better diagnosis than my hand-fix had: func_8015B950's 0-arg call is in
ONE place — src/shared/engine_core.h, a DEFINE macro body — expanded into all 926 TUs. That fix is a
SINGLE cast, not 926 edits. Named as the next step rather than run on tired context.
Also lands cookbook §91 (the --like role trap) from the previous step.
|
||
|
Drew T
|
a5e5ea45ef |
fix(phase-29): jtbl_carve — guard the --like role-transfer; the ×137 sweep goes 0/3 -> 3/3
ROOT CAUSE of the 0/3 (found by reading the tool's ACTUAL invocation, not by guessing): jtbl_family_bank calls `jtbl_carve <sibling> --func <fn> --like <exemplar_ov>`, and the role-transfer keys on the SUBSEG ROLE (`ov_SC01_077_a` -> `_a`). Its premise — "same family => same span structure" — silently breaks when the exemplar and the sibling host the function in subsegs with DIFFERENT roles, which happens whenever the exemplar has a split the sibling does not. MEASURED: func_8012AAAC lives in `ov_SC01_077_a` (role `_a`) in the exemplar but in the MAIN subseg (role ``) in every sibling. The transfer therefore looked up `ov_SC01_077` — an unrelated SEVEN-table span belonging to entirely different functions — and stamped those starts onto a sibling span holding one table. jtbl_rodata_pads then refused with `consumed 1 rodata .align(s) but 2 pad spec(s) given — table-count drift`, and jtbl_family_bank deliberately does NOT treat that error as isolate-fixable, so all 137 siblings returned a bare `gate-fail` with the cause discarded. THE FIX: transfer only when the exemplar's subseg for THIS FUNCTION has the sibling's role; otherwise derive the span from the sibling's own carve (which was already computing it correctly). Fail-open is not acceptable here — a wrong table set corrupts the image, so the guard defaults to local derivation. MEASURED RESULT: the 3-member probe goes 0/3 -> 3/3 BANKED. R22 clean-fleet: extract-all 139/139, check-all 140 passed / 0 failed. Two earlier hypotheses were tested and are recorded honestly in CURRENT_PHASE.md: the sibling call-site casts (real conflict, fixed, byte-neutral — but NOT the blocker) and my own carve-alone test (which fails by construction for this shape, because a stub object does not emit the table its 2-entry spec describes — the tool splices the body BEFORE building, so its path is the valid one). |
||
|
Drew T
|
2cc49d0310 |
feat(phase-29): SESSION-21 — func_8012AAAC banked via the §81 carve chain (R22 140/140)
The first jtbl-routed bank of the session, and it validates the whole chain end-to-end:
1. jtbl_carve SPLIT-TABLE repair (this session): jtbl_801D7FB0 28 -> 50 words (112 -> 200 B),
authorized by func_8012AAAC's own `sltiu 0x32`.
2. NEW FIX — SINGLE-TABLE PREDECESSOR: adding a second table to a subseg whose existing carve was
single-table lost the FIRST table's start entirely (new_offs has only the new one;
overlay_jtbl_addrs cannot see the old one because its owner is banked and extract PRUNED the
stub .s; and single-table carves persist no tables= to rebase). The span then failed its own
validator with "first must equal the span start" — the invariant naming the missing entry.
A single-table carve spans exactly its one table, so ITS SPAN START *IS* THAT TABLE'S START:
inference, not persistence, so it also works for spans carved before tables= existed. This is
the RECOVERABLE half of the documented func_8013F350 lesson (that one was a pre-§8e merged
DOUBLE — two tables, no record, genuinely unrecoverable).
Result: ov_SC01_077_a JTBL_PADS := 0,0 tables=+0x0,+0x14. Carve alone byte-gated BYTE-IDENTICAL
BEFORE the bank was attempted (§81 step 2).
3. ARITY axis, all-or-nothing: 1,244 decl sites / 1,240 files `(void)` -> `()` + an R32 completion
assertion (old-form remaining: 0).
4. ONE call-site cast: the definition lands at line 811 and a 0-arg call sits at 822, so gcc sees
the prototype and rejects it — `((void (*)(void))func_8012AAAC)()` (§17a-1; gcc folds the cast
of a known symbol to a direct jal). Only 1 of the 1,386 fleet-wide 0-arg call sites needed it:
the others see only the `extern ()` decl, which permits a 0-arg call.
DIAGNOSIS NOTE: the failure read CC1-FAIL with only a warning visible under make. Running the
pipeline stage-by-stage (cpp | cc1 | maspsx | jtbl_rodata_pads | as) put it on cc1 rc=33, and cc1's
own stderr named it exactly: "too few arguments to function func_8012AAAC" at line 994. Isolating
the stage was what turned an opaque Error 33 into a one-line fix.
R22 clean-fleet: extract-all 139/139, check-all 140 passed / 0 failed.
family_sweep correctly REFUSED this exemplar (§53: a jr-family must route through
jtbl_family_bank.py; "a 0% from this path would be a TOOL artifact, not a wall") — the ×137 member
sweep is the next step and needs a clean tree, which this commit provides.
|
||
|
Drew T
|
5be4c21480 |
feat(phase-29): SESSION-21 — the ×138 member sweep: 274 members from 3 exemplar cracks (R22 140/140)
- family_sweep --hseq over the 3 newly-banked exemplars: BANKED 274 member-matches / 137 failed across 137 overlays, for ~0 agent tokens. Session total: 3 exemplars + 274 members = 277 fns. - THE STALE-MAP STEP, hit and handled: the first sweep returned "0 matched-exemplar families" because .run/family_hseq.json still listed the fresh cracks as draft-ov077. Regenerated (matched-sib families 60 -> 63) and the sweep found them — the documented bank-x1 -> regen -> sweep path (memory crack-wave-sweep-map-regen). - §86 REPRODUCED CLEANLY: 2 of 3 families templated ~137/137; the third failed ~137/137. Not a rate — a BIMODALITY. One probe per family, then sweep or skip; never a blended pool average. - R22 clean-fleet: extract-all 139/139, check-all 140 passed / 0 failed (second clean-tree verification this session). dedup 1886/0, C1 coverage 239,604/239,604, 0 NON_MATCHING (G4). - FLEET 81.7 -> 81.9% instr · 89.52 -> 89.60% fn-count · distinct-code 69.3% UNCHANGED — correct and expected: these are h_seq PURE propagation-class families, and SESSION-20's routing rule says propagation moves only the DISPLAY metric (members were already counted once via their exemplar). To move RE-completeness, target byte-VARIANT families. Stated plainly so the next session picks targets by the metric it means to move. - drive-by: family_sweep --help crashed (argparse %-expands help text; a literal "0%" needed "0%%"). |
||
|
Drew T
|
87b02b044f |
fix(phase-29): jtbl_carve — repair the SPLIT-TABLE undercount, gated on the function's own sltiu
THE BUG (real, found by a wave agent): jtbl_range() ends a carve at the next data dlabel, assuming every dlabel is an object boundary. spimdisasm can CUT ONE JUMP TABLE IN HALF and emit the tail under an invented D_ label — func_8012AAAC's 50-word table is jtbl_801D7FB0 (28) + D_801D8020 (22). The carve then reserves 112 B for an object supplying 200 B of .rodata, shifting every later symbol. §84-class: match_one is structurally blind; it surfaces only as a whole-binary DIFF. THE AGENT'S EVIDENCE WAS WRONG (R14): it reported D_801D8020 as having "ZERO xrefs anywhere in the tree" and proposed deleting the label. It has TWO (.word D_801D8020 and +0x2 in tail.data.s) — almost certainly spimdisasm mis-symbolizing packed halfword data, but "almost certainly" is not a gate, and the proposed remedy would have deleted a symbol two emitted words reference. I built the xref census first, watched it refuse, and only then found the references. THE GATE USED INSTEAD — the function's own `sltiu N` range check, which gcc emits right before the indexed load, so the PROGRAM declares its own table length (func_8012AAAC: sltiu 0x32 = 50). Absorb only when the next label is immediately adjacent, its words are all code addresses in the overlay's text, and absorbing lands on an EXACT sltiu bound (the SET, not max() — a multi-switch function has several and no way to say which owns this table). Three further corrections, each caught by testing rather than assumed: - the absorption fired and the trailing-pad trim immediately UNDID it (re-trimming against the first dlabel's 28 words); the trim now sees the whole absorbed table; - a continuation ends at ITS OWN last .word, not the next dlabel (D_801D8020 ends 0x801D8078; the next dlabel is 0x801D8158, 224 B on) — using the next dlabel is the assumption being repaired; - the shortfall warning now fires only on an unambiguous single-bound pairing (it fired ~90 times across 38 tables before the guard — a warning that fires on ambiguity is noise, not a signal). VERIFIED: the split table 28 -> 50 words (112 -> 200 B), matching the agent's 3 independent confirmations; and across 38 jtbls x 6 functions = 228 combinations, EXACTLY ONE range changes — that table, for its owning function only. |
||
|
Drew T
|
d0322d473c |
feat(phase-29): promote tools/reloc_verify.py — resolve every relocation before paying a gate cycle
The SESSION-20 carry item ("promote it — it closes 3 of the 4 blindness classes"), generalized:
base vram DERIVED from the target .s (was hard-coded to one function, R33) and the parse
coverage-asserted (R32 — a target that parses to zero instructions refuses to report a verdict
rather than reading "ALL RESOLVED"). Resolves jal callees, %hi/%lo data addresses (recovering the
implicit REL addend objdump -r never prints — the §84 trap) and internal j destinations.
IT TOOK TWO OF ITS OWN BUGS TO TRUST IT — both found by cross-checking masked_diff (R34):
1. `objdump -dr` instead of `-drz`: without -z objdump ELIDES identical-instruction runs, so
func_801330E0 read 104 ins vs masked_diff's 110 (6 elided nops) and every later index compared
against the wrong instruction — 2 phantom mismatches on a clean draft. A comparison tool MUST
share its reference oracle's index space exactly.
2. the .s word field is little-endian HEX TEXT, not the instruction integer; masked_diff byte-swaps
it and this did not — reporting "word differs" on three byte-IDENTICAL sites.
Now classifies instead of alarming: JTBL (gcc emits its own switch table via a local label => nothing
to relocate; the §81 routing signal — bank via jtbl_family_bank, never plain harvest_verify) ·
BAKED-LITERAL (same constant materialized inline: byte-correct here, but if the symbol is
per-overlay the exemplar matches and every SIBLING breaks — the §84 shape) · real mismatch.
Recorded: measuring a live wave's drafts is itself the §87 staleness error — a draft rewritten 12s
before the check gave a different verdict. Draft QA happens after the wave returns.
(The wave's gate driver lives at .run/s21_gate.py — gitignored scratch, §55b orchestration law
built in: --no-propagate per TU group, commit before the fleet propagate, and BANKED derived from
the stub set rather than read from gate_stage's accumulating verified-file.)
|
||
|
Drew T
|
ccbc65e9c3 |
fix(phase-29): progress.linked_subsegs fails closed (R32) + the main-EXE bucket re-measured
- progress.linked_subsegs() was FAIL-OPEN: gated on the module global BINARY that set_binary() assigns, it returned an EMPTY SET when imported as a library without that call — i.e. "no linked library subsegs", which for main is confidently wrong (there are 49) and silently reclassifies ~960 already-byte-identical PsyQ-linked stubs as outstanding game-code work. Now raises when unconfigured; the CLI path is untouched (set_binary assigns before calling). Caught by hitting it myself while measuring bucket #2. - ENDGAME-MAP CORRECTION (measured, zero-token): the map's "main EXE game code ~59,765 ins / ~1,048 stubs" conflates two populations. Correctly split: game code 1,042 stubs / 31,888 measurable ins; LINKED PsyQ library 960 stubs / 27,877 ins (already byte-identical). Bucket #2 is ~47% smaller than quoted. Caveat kept: 467 game-code stubs have NO sig row (the documented main second-oracle gap), so the true weight is above 31,888 and not currently measurable — re-price when the main second oracle lands, do not quote either number alone. - .run/s21_zerocrack.json: the 60-family zero-crack pool enumerated (45 plain / 15 jr) and honestly discounted — its top entries (0x8013c414 -O0 wall, 0x80144090 LENGTH-DRIFT, 0x80133ab0 pinned) are already-diagnosed refusals, so ~95k of the 208,499 is not available. |
||
|
Drew T
|
2b38c68333 |
feat(phase-29): SESSION-21 T1-T3 — the frontier measured, the family-exemplar wave, 3 tool fixes
- T1 FRONTIER MEASURED (zero-token, R35: family map regenerated on fresh sigs first — it was stale by ~657 banked members): 36,020 stubs / 2,345,599 weighted ins remain, and only 9.0% are h_exact-FREE. PROPAGATION IS TAPPED (238 distinct classes / 3,245 instances); 22,498 distinct classes / 1,680,097 distinct ins is what is actually left. The mass is FLAT across all 139 binaries (~300-550 sub-500 stubs each) -> "pick the best overlay" is not a strategy. .run/s21_frontier.py + .run/s21_frontier.json - T2 THE AXIS IS THE FAMILY, NOT THE LOCATION: 1,342 substantial h_seq families / 1,298,135 templatable ins = 55% of ALL remaining weighted instructions. Routed by blocker: jr/§81 181 fams (33.6%) · DRAFT-with-cached-Ghidra-C 91 (28.6%) · DRAFT-modal 1,023 (27.5%) · zero-crack 45 (6.5%) · permanent walls 2 (3.9%). Live+cached+non-wall in ov_SC01_077 = 54 families / 589,502 ins, value steeply concentrated (top 24 = 96%). .run/s21_targets.py + .run/s21_targets.json + .run/s21_draft_pool.json - T3 WAVE 1 LAUNCHED: tools/workflows/family_core_wave.js (NEW) — 24 xHigh drafters, one per family exemplar, stake 575,488 templatable ins (24% of remaining). Supersedes worker_wave.js for family work: carries each target's family STAKE, encodes the four §58/§87 integration rules at source (splat D_<UPPERHEX> not Ghidra DAT_; never invent a symbol; canonical callee sigs; leave decl plumbing to the ladder), and requires symcheck.py on any claimed MATCH. - T3b LADDER HYGIENE, both SESSION-20 carry items fixed — one defect, two masks: a byte-NEUTRAL transform was left in the tree when it banked nothing. family_sweep's --normalize-self-decls backstop only fired on MISMATCH (left 123 files of dead diff on a 0/123 run); gate_stage's ARITY undo narrowed to src/shared/ and left ~40 TUs. Both now restore the full snapshot when NOTHING banked (no banks to preserve => the splice hazard cannot apply). §61 on the success path. - T3c BACKLOG addr DEFECT fixed (R32/R33): new addr_of() derives the address from `name`, assert_addr_coverage() fails loud on an unkeyable row, append_record fills both directions. Found a latent bug doing it: load_best() keyed on `addr or name`, splitting one function into two "best" records. Keyable rows 128/1,701 (7.5%) -> 1,701/1,701 (100%). |
||
|
Drew T
|
5c422e8426 |
feat(phase-29): tools/sweep_parallel.py + §89 — the parallel gate farm, reachable from the family path (step 2)
bulk_harvest's Phase B has been a ProcessPoolExecutor over DISTINCT binaries (per-binary flock, per-worker result files, compute_fleet=False) since Phase 23 — but welded to Phase A's LLM drafting. Family sweeps stage drafts differently (family_sweep --stage-only), so the farm was UNREACHABLE from that path, and SESSION-20 gated 389 + 268 + 104 members SERIALLY for no architectural reason (~8-16x throughput loss on a 32-thread box). This is a thin adapter: same gate_stage.run_gate, same per-binary lock, NO new gate logic. Also fixes the phantom-dir bug at source: a bare .run/sweep/*/ glob matches gate_stage's own intermediate ladder dirs (-cn/-cast/-rc/-s2in/-uni) and calls them as binaries — 24 phantom PARTIAL 0/1 lines that inflated one run's notbanked from 0 to 56. Requires config/splat.<bin>.yaml to exist (R33/R36: derive the binary set, never glob it). Smoke-tested: the phantom is skipped and named, real binaries kept. §89 records both throughput rules the project already had and was not following. |
||
|
Drew T
|
1748eabe62 |
feat(phase-29): tools/blast_radius.py — MEASURE the write set, enforce the §63 tier (step 1)
§63 has defined T0/T1/T2 since Phase 26 and never enforced it. Two measured consequences in SESSION-20: ~13 full clean-fleet verifies (~15 min each) for batches that were provably T1 (over-verification), AND two cases where the write set was LARGER than the belief about it — the §85 widen believed contained to one overlay broke ov_SC01_077, and gate_stage's ARITY pre-pass silently rewrote 40 TUs (under-verification, the dangerous half). A tier is a CLAIM about the write set; this turns it into a MEASUREMENT. --expect t1 fails loud when the tree disagrees. Binary list DERIVED from config/splat.*.yaml (R33, never hardcoded — R36's incident was a hardcoded set missing 4 real binaries). Coverage ASSERTED (R32): an unclassified path exits 2 and names itself rather than being silently skipped. overlays.mk is attributed per-binary by parsing its diff, so a change confined to one binary's var-block stays T1. |
||
|
Drew T
|
9d9cd58028 |
feat(phase-29): T0.4 harvest — 272 return-axis members banked (+26,928 ins) + the §84 recompute in family_remap
THE §85 WIDEN PAID OFF AS PREDICTED. It is a ONE-TIME fleet edit, so once committed the
`conflicting types` blocker was gone for EVERY member of both families at once:
- sample 8 first (probe-before-scale): 8/8 banked with PLAIN harvest_verify, no ladder needed
- full sweep: 264 banked / 0 failed across 132 overlays; 10 skipped as not-stub
- total 272 members ~= 27k ins, ZERO agent tokens
R22 clean-fleet 140/140 BYTE-IDENTICAL; dedup 1886/0; 0 NON_MATCHING (G4).
Fleet: instr 80.6 -> 80.8% (10,589,503 -> 10,616,431, +26,928) · fn-count 89.19 -> 89.26%.
distinct-code UNCHANGED at 68.3% — propagation moves the DISPLAY metric, not the RE-completeness
one (the SESSION-19 split, reconfirmed).
§84 RECOMPUTE now implemented in tools/family_remap.py (fix_derived_offsets), wired into all three
apply_remap call sites as a PRE-pass on the exemplar body (the literal is ambiguous as a substitution
token, so it cannot be a table entry):
correct_literal = mapped(aliased_sym) - mapped(base_sym)
Verified by negative control: the hand-solved case recomputes 0x20 -> 0x18 exactly, and a site whose
target endpoint is NOT a mapped symbol is left byte-for-byte alone AND REPORTED in info
["derived_offsets"] (R32 — a silent skip is a defect, and a silent skip is how this bug survived).
|
||
|
Drew T
|
57ee715f3c |
feat(phase-29): T0.1 frontier survey re-run (138 ovs) — the family lever is ALIVE; a 224,410-ins zero-crack FREE pool
- verified the tool BEFORE trusting its scan (R35): load() correctly globs all 138 overlays, but the generated header hardcoded '134' -> fixed to derive from the same glob (a doc misreporting its own scope is the P28 img_path shape, one severity down) - stale(07-23,134ov) -> fresh(07-26,138ov): fleet 88.5/79.0/68.4 -> 89.4/80.9/69.0%; families 2721 -> 2688; substantial 558 -> 544; with-matched-sibling 74 -> 76. Structure STABLE => the P25 family reframe is NOT an artifact and P26's ~0% stays unsupported post-fix - FINDING: 3,419 instances banked but only 85 distinct CLASSES fell -> recent yield was propagation, not new classes (SESSION-19's split, now fleet-wide) - THE POOL: 76 zero-crack families (exemplar already matched) = 347,892 ins = 19.4% of remaining distinct code, decomposed by real blocker: FREE(PURE/non-jr/non-O0) 61 fams/224,410 ins = 12.5% of remaining; jr 13/57,311 (§81 chain); -O0 2/66,171 (known deferred build-infra, Arm A proved 9/9 bank) - STILL A PREDICTION (R14/G3): T0.2 re-targeted from this data to measure the GATE conversion rate on 8 members sampled across the FREE subset before any arithmetic scales |
||
|
Drew T
|
faf4547345 |
feat(phase-29): func_8017C954 BANKED — jr carve chain cleared; a shared type was PRESENT but INVISIBLE
- BANKED (1,194 ins, ×1 distinct-code). Chain cleared, each step byte-gated before the next was
built on it: one-line fix to jr_isolate_all._engine_types() -> jr_isolate_all --only
func_8017C954 (2 fns / 1 object, NOT the bare 47-fn / 21-object resegment) -> BYTE-IDENTICAL
b7b0d4ae -> jtbl_carve --func func_8017C954 (44-piece carve set + interleave order) ->
BYTE-IDENTICAL -> harvest_verify VERIFIED BYTE-IDENTICAL -> R22 clean-fleet 140/140,
tools-health OK. instr 80.5 -> 80.6%; distinct-code 3,842,906 -> 3,844,100.
- THE DEFECT (tools/jr_isolate_all.py): _engine_types() harvested shared type names with four
patterns -- `typedef ... X;`, `} X;`, forward-decl `struct X;`, fn-ptr typedef -- and a TAGGED
DEFINITION WITH A BODY matches NONE of them. So `struct PW8017E6D8 { int w; }
__attribute__((packed));` at engine_types.h:658 was present in the shared header yet invisible
to the carried-type check, and `extern struct PW8017E6D8 D_801E1EC4;` could not be placed.
MEASURED BLAST RADIUS: 77 such tags in engine_types.h were invisible. One added pattern fixes
all 77.
- WHY THIS COST 20 MINUTES INSTEAD OF A MYSTERY BYTE-DIFF THREE PHASES LATER: the Phase-26 audit
had already turned this predicate's SILENT DROP into a LOUD REFUSAL. The original bug dropped
4,040 col-0 decls, 683 of them function PROTOTYPES -- and a dropped prototype is a SILENT
BYTE-CHANGER (C89 implicit `int f()`; return type drives delay-slot fill in this codebase). The
refusal named the exact symbols and the exact remedy. A loud "I cannot place this" is worth far
more than a green build -- the audit paying for itself, live.
- §81: the 3-step jr-carve chain + why match_one CANNOT see the problem (it masks jal/HI16/LO16,
so a jump-table function reports MATCH while the whole-binary gate reports DIFF, correctly).
Detect with `grep -cE 'jr \$(v0|v1|a0|t[0-9])'` on the target .s + a jtbl_ in asm/<ov>/data/.
ALWAYS use --only: bare would have resegmented 47 jr-functions across 21 objects.
|
||
|
Drew T
|
9924b26968 |
fix(phase-29): dedup_extend stripped a load-bearing include on a 0-banked run (§61 class)
- THE DEFECT: `if not banked: ensure_include_revert(b)` fired UNCONDITIONALLY.
`ensure_include()` returns True only when IT inserted the line, but the revert ignored that
return value — so on a binary that ALREADY had `#include "../shared/engine_core.h"` from
earlier work, a zero-bank run REMOVED it, leaving every `DEFINE_func_*()` in that overlay
unresolvable.
- BLAST RADIUS AS IT HAPPENED: the §75a class-B probe banked 0 across 135 already-wired
binaries, so the include was stripped from ALL 135 in one run. Caught by reading `git status`
before moving on; `git checkout -- src/` restored (nothing was committed, nothing lost).
- WHY IT SURVIVED THIS LONG: the tool's designed case is NEWLY-onboarded binaries (which do not
have the include, so the revert is correct there), and prior runs banked >=1 per binary so the
branch never fired.
- WHY NO BYTE-GATE SAW IT (R34): the damage lands AFTER the last gate runs. harvest_verify had
already finished and reverted its drafts; the byte-gate is a null oracle for state mutated
after it. This is the §61/§63 class — an undo written as an INVERSE TRANSFORM instead of a
snapshot restore, applied without checking whether the forward action was ever taken. Same
shape as the SESSION-14 `fix_arity_callers --revert` incident.
- FIX: capture `added_include = ensure_include(b)` and revert ONLY if this run added it.
- NEGATIVE CONTROL: stripping the include from ov_SC01_004 makes `make audit-binaries` fail loud
("[FAIL] ... does NOT include ../shared/engine_core.h", make Error 1) — the R36 citizenship
gate is exactly the detector for this class, confirmed by experiment, then restored.
|
||
|
Drew T
|
e112eff601 |
fix(phase-29): find_site — a comment-only line halted the extern scan (§68); func_80174CB0 x1 -> x3
TWO mislabels in one tool, both found by making it print what the compiler actually said.
1) compiles_standalone() returned a bare False and the caller filed EVERY failure under
"overlay-local TYPE (the real cap)". The dominant real cause is undeclared FILE-SCOPE EXTERNS.
Now returns (ok, stderr) and the skip is classified by actual cc1 output.
2) find_site()'s backward walk over "preceding contiguous externs" skipped BLANK lines but not
COMMENT-ONLY lines, so a full-line /* ---- */ between two extern groups dropped every extern
above it. Comment lines are now skipped like blanks and filtered out of the emitted body so
make_macro never meets a `//`.
RESULT, measured honestly: func_80174CB0 went from "not self-contained" to a 138-member PLAN, but
--recover banked only x3 (ov_SC07_006/007/011); 135 overlays excluded. Those exclusions are NOT
byte divergence (all 138 share h_exact) -- they are the CARRIED EXTERNS colliding with each target
overlay's own decls. The carry is necessary but not sufficient: it must reconcile per-target-TU
(cdecl.compatible(), the shape reconcile_tu already uses). Spec updated in CURRENT_PHASE.md.
- R22 clean-fleet 140/140, 0 failed. dedup-check 1883 validated / 0 failed, C1 coverage complete.
- fleet instr 79.9% (10,501,384 / 13,141,652); +246 ins from the x3.
- WHY THIS MATTERS beyond the numbers: the Phase-21 backlog already prescribed "macro-extern-
injection frees them x134 (~+0.3%)" and it was never built, because the mislabel told every later
session these were the known-hard type wall. A wrong diagnostic label cost ~4 phases.
- cookbook §68. NOTE the exclusion message is ALSO mislabelled ("byte-diverge / irreconcilable"
conflates differing bytes with a non-compiling instantiation) -- logged to fix.
|
||
|
Drew T
|
00b6448f4e |
fix(phase-29): dedup_propagate — the "overlay-local TYPE (the real cap)" skip was a MISLABEL
compiles_standalone() returned a bare False and the caller attributed EVERY failure to the overlay-local type cap. The dominant real cause is undeclared FILE-SCOPE EXTERNS: the body references extern decls living outside the extracted def block (func_80174CB0: 22 of them; carrying them makes it compile cc1 rc=0). - compiles_standalone now returns (ok, stderr); the skip is classified by actual cause: "missing file-scope extern (CARRY-FIXABLE): <names>" vs "overlay-local TYPE (the real cap)". - FLEET SIZING (--auto-from ov_SC01_077 --check-only): 7 skipped, ALL 7 carry-fixable, 0 genuine type-cap. Three of them (0x80142B2C/0x801535F4/0x80155800) are on the Phase-21 backlog list whose note ALREADY said "macro-extern-injection frees them x134 (~+0.3%)" -- never built, because the mislabel told every later session they were the type wall. A wrong label cost ~4 phases. - also: func_80174CB0's local Mtx_/Svec_ typedefs swapped for the canonical shared MATRIX/SVECTOR (byte-identical layouts); ov_SC07_006 still BYTE-IDENTICAL 7ca772be. - value behind the real fix: the 7 (~+0.3pp) + func_80174CB0 x138 (16,974 ins, ~+0.13pp), ~0 tokens. - _carry_externs itself is SPEC'd but NOT built here: it writes 138 overlay files (§63 class) and wants a fresh session with an R22 budget. func_80174CB0 (banked x1) is the test case. |
||
|
Drew T
|
b8e7250c78 |
feat(phase-29): tools/symcheck.py — the pre-gate symbol-set guard (§67a)
Builds the guard SESSION-17 left as a TODO after the func_801463A0 `_s`-alias trap, where a draft invented extern aliases no symbol table defines, read MATCH under rtu_match, and could never bank. - diffs the symbols a draft's object references (reloc records) against the target .s's %hi/%lo/jal set; reports MISSING (invented-alias signature) and INVENTED separately. - fills a real hole: match_one/masked_diff compare relocation-MASKED words (object-vs-.s is symbol-agnostic BY CONSTRUCTION) and rtu_match COMPILES WITHOUT LINKING -- so both are structurally blind to this class. R34: a second oracle that can disagree with the first. - NEGATIVE-CONTROL PROVEN: with one data extern renamed to an invented alias, match_one reports the SAME 14 mismatched as the correct draft; symcheck exits 1 naming both symbols. - --c compiles via match_one so the pinned triple/flags can never drift (R33); or --obj. - applied to the live func_8014D820 close=14 draft: 12/12 symbols agree, so a match there will link cleanly -- the §65c class is ruled out for it in advance. - cookbook §67a + SETUP tooling-inventory row (R21). Necessary condition, NOT a match oracle: still finish on the whole-binary byte-gate (G3/P9). |
||
|
Drew T
|
a795680727 |
feat(phase-29): T17 probe — the permuter DOES ingest pinned giant drafts; func_80177940 5->1
- REFUTES the .run/giants README's "pycparser/permuter CANNOT ingest it as-is": p16_permute.setup's b64-pragma pin carrier handles it (6 pins -> 6 carriers, 0 raw __asm__, target.o built, §31 profile). Checked against the tool, not the note (R35) — the 3rd recorded wall this session to dissolve. - Drift-check first (R14): all preserved drafts reproduce their recorded closeness exactly (5/33/76/116). func_801670E4 (close=16) is ALREADY BANKED fleet-wide — the README is stale; it is not work. - 900s @ -j12: 5 -> 1. The permuter closed the 4-ins INSN_LUID scheduler tie the drafting agent had recorded as un-steerable after sweeping all 6 assign orders + pin combos by hand. - Last instruction (andi vs addu) diagnosed from the byte-verified sibling func_801778A8, whose "nib = uVar1;" plain-copy idiom (both vars hard-pinned) after the identical (x << 16) >> 28 shift pair is what materializes the addu. Dropping my redundant & 0xf alone COLLAPSES the copy (100 vs 101 ins, 52 mismatched), so the target needs a distinct pinned register. Pinning n to $a2 -> 101/101 with 6 left, class ADDRESSING [permuter] -> handed back to the permuter from the structurally-correct seed rather than hand-designed. - FIX: run_permuter's cleanup pkill matched EVERY concurrent run (two permuters silently killed each other); scoped to the run's own scratch dir -> concurrent giant grinding is now safe. |
||
|
Drew T
|
ffca2e4803 |
feat(phase-29): T16.10 — the driver's SUCCESS path verified (free re-bank test); 2 defects fixed
- THE FREE TEST (cookbook §66): reverted func_801778A8's bank to its INCLUDE_ASM stub (stub state rebuilds BYTE-IDENTICAL 7ca772be — a faithful revert proves itself; needs `make extract` first, the R22 corollary) and re-banked it THROUGH recover_integration.py --commit --r22. pass1 1/1 -> exact restore -> pass2 1/1 -> commit commit:0928 -> R22 140/140 -> report.json. Bank confirmed from SOURCE (stub gone), never the report (§55b trap 4). EQUIVALENCE: git diff vs the pre-revert commit = ONE blank line (mine) -> the driver reproduced SESSION-16's state exactly. - DEFECT 1 (SAFETY, found by reading before firing): PROPAGATION is a fleet-tier write (dedup_propagate --auto-from -> src/shared/engine_core.h + up to 138 overlay .c) that was both UNDECLARED and the DEFAULT, so --max-tier binary still permitted the widest write in the toolchain. assert_write_set cannot catch it (it runs before the gate; under --commit git status is clean). FIXED up front: propagate now requires --max-tier fleet AND --r22, and is REFUSED after a demacroize stage (those banks are x1 by construction; --auto-from would re-macroize and undo them). Both refusals negative-control-tested, exit 1. The "standing hazard" is now a refusal. - DEFECT 2 (METRIC): gate_stage scraped the fleet % via a progress.py label that no longer exists -> fp=None -> 50 gate commits recorded "fleet None%". Now reads FLEET instr-weighted (legacy fallback + loud stderr warning if neither matches); parses 79.6. - STALE DIGEST (R14): docs/progress.fleet.md at HEAD disagreed with HEAD's own source by 45 in the dedup-shared column — generated during the §65g local_type trial whose edits were then reverted. Regenerated (reproduced identically in-gate + standalone); headline %s unaffected. - cookbook §66/§66a/§66b distilled in-session (R30); SETUP.md gains the missing recover_integration row (R21 debt). tools-health OK: corpus 0/0, cdecl green, audit-binaries 140 citizens, lint OK, dedup-check 1879/0. Fleet unchanged 79.6% instr / 67.7% distinct / 88.86% fn-count. |
||
|
Drew T
|
9137d3b784 |
feat(phase-29): T16.4 — encode the recovery recipe in recover_integration.py (tiers ENFORCED, §55b trap 4 closed)
Extended, not replaced: it already owned exact snapshot/restore, split-aware grouping and the two-pass
gate-all -> restore -> re-stage-winners protocol. A new driver would be a 7th snapshot impl (R33).
- --draft-dir (repeatable): consume a WAVE dir instead of the backlog (unreliable closeness,
overlay-specific drafts). Strict ^func_[0-9A-Fa-f]{8}\.c$ filter -- the wave dirs carry scratch
(_b.c, try2.c, scratch/) and run_gate globs *.c blindly.
- --run-id: all scratch under .run/recover/<id>/, and run-local verified_out/failed_out passed into
run_gate -- closes §55b trap 4 (the accumulating .run/harvest_verified.txt phantom bank), which the
cookbook still lists as "still armed".
- --stages with the new demacroize stage; --max-tier; --r22; --probe-only; --report.
- TIERS ENFORCED not documented: stages declare T0/T1/T2, the driver MEASURES the write set
(git status before/after) and ABORTS if a stage writes outside its blast radius (§61d). A fleet-tier
stage is refused without --max-tier fleet AND --r22. Both refusals negative-control-tested.
- stub_map now derives from corpus.stubs (R33, coverage-asserting) instead of a private regex that
could silently return a short map; banked_from_source() is the sole bank oracle for reporting.
- End-to-end on the remaining 22: excluded 26 already-banked by name, demacroize ran on 12, banked 0,
restored exactly (src/ clean), 14/14 prior banks intact. A clean negative -- it does not manufacture
banks. Those 22 need normalize_self_decls / draft type-uniquify wired next.
|
||
|
Drew T
|
f1f6238251 |
feat(phase-29): T16.6/T16.7 — 14th bank via the generalized de-macroize; §65 distilled; R21 inventory debt cleared
- func_8012F40C banked (the callee-conflict variant): relaxing demacroize from "the draft's own function" to "any decl the DRAFT declares incompatibly" reaches macros that declare a CALLEE differently than the draft does (RotTransPers/RotTransSV). 14 banks total, R22 140/140. - THE ONE FAILURE, kept honest: func_8012F49C was rtu-MATCH but the whole-binary gate REJECTED it. rtu_match is relocation-masked, so a wrong call TARGET is invisible to it -- and this was a callee case, exactly where the mask hides the error. Trust rtu MATCH for self-decl corrections, distrust it for callee ones (§65c). Reverted its edits and re-banked only the winner rather than leave byte-neutral churn on matched code (§57a-4). - DISTILLED IN-SESSION (R30/R16/R31/R21): cookbook §65 + §65a-§65e (blast-radius tiers; the de-macroize escape and the §20 refutation; the rtu-vs-gate divergence; the existing-ladder baseline; two-oracle practice); decision-log entry with the HONEST multiple (~2.3x, not the projected 3.7x, and it lands on distinct-code not the display number); calibration.md measured table; SETUP.md rows for blocker_probe + demacroize PLUS the three the inventory was missing (lift_types, uniquify_type, fix_header_decl-as-retired). - Carried and NAMED, not dropped: 10 match_one-MATCH drafts still blocked by stacked classes, and the 11 `near` drafts which are unfinished drafts, not recovery fuel. |
||
|
Drew T
|
09fb069cf2 |
feat(phase-29): T16.2 — GATE A passed on measured numbers; the §20 DEF-conflict doctrine refuted per-overlay; func_8012CC88 banked
S0 measured all 36 stranded drafts in 9.2s, two oracles agreeing 36/36. - POPULATION CORRECTED (R14): the audit's "~92% byte-correct" is a whole-wave figure; among the STRANDED residue match_one says 24/36 MATCH / 11 near / 1 ERR = 67%. The 11 near are unfinished drafts, not integration problems -- and they are exactly the ones that compile and DIFF. - BLOCKERS (they STACK; cc1 reveals only the first): self_decl_hdr=21, callee_decl=19, data_decl=16, self_decl_tu=5, local_type=5. Per function by MAX tier: T0=6, T1=26, not-integration=4. - T3 BASELINE, measured free: the existing draft-side ladder clears callee_decl 19->3 and data_decl 16->0 yet converts 1 of 36 to compiling, which then DIFFs -- §61d verbatim. The wave's gate orchestration was NOT broken; the ladder simply cannot reach this population. - NEW tools/demacroize.py: the conflicting extern lives INSIDE a DEFINE_func_* macro BODY, so it exists only where instantiated. Expanding those instantiations in the overlay's OWN TU, correcting only the conflicting decl to the draft's byte-true sig (never dropping it, §57a-1), dissolves the conflict with nothing written outside src/ov_SC07_006/. This is §63's own unexplored "per-overlay-local decl" -- and it refutes §20's "the DEF-conflict class is byte-proven unrecoverable by text transform" for the per-overlay case. - MEASURED on the 14 clean candidates: 13 MATCH / 1 DIFF in the real TU. - END-TO-END: func_8012CC88 banked whole-binary BYTE-IDENTICAL (stub gone per grep, not per report); R22 clean-fleet 140 passed / 0 failed of 140 -> the T1 blast-radius claim validated empirically (the difference from fix_header_decl, which broke 139/140 from the same per-binary green light). - GATE A: 32/36 real decl errors (>=12) and 13/36 rtu-MATCH simulated (>=12). PASS. Price stated honestly: a de-macroized bank is x1 -- full distinct-code credit, ~1/138 of instr. |
||
|
Drew T
|
1ea54fa594 |
feat(phase-29): T16.1 — the two-oracle blocker probe (read-only); rtu_match full-stderr + multi-line //@EDIT
Task 16 (the integration-recovery pass) T1. The SESSION-15 audit measured the wave bottleneck as INTEGRATION (~92% of drafts byte-correct, ~27% bank); 36 stranded byte-correct reach-138 drafts are the fuel. Before building any recovery, measure the REAL blocker per draft. - tools/rtu_match.py: --stderr-out (atexit flush, covers every sys.exit path) + \n in //@EDIT replacements (T6 needs a multi-line macro expansion). The inline tail is truncated and these TUs emit hundreds of benign warnings -- on the first real run it was 100% warnings while the actual errors sat ~180 lines earlier (the §58 red-herring, one level down). - tools/blocker_probe.py (NEW, read-only, two oracles R34): static (cdecl.compatible -- never text equality, which is what made the deleted scanner report u8-vs-unsigned-char as a conflict) and real cc1 via rtu_match (ONE compile implementation, R33). Leads with the DISAGREEMENT table. - DELETED .run/diag_plumbing.py (R3 tooling under tools/; R33 net -1 scanner). - Two build-forced corrections: (1) cdecl.tu_scope runs real cpp, so it already expands instantiated DEFINE_func_* macros -- the macro scan's job is ATTRIBUTION (tu-text vs shared-header macro body: different transforms, same T1 tier), byte-checked against engine_core.h:7533 for func_80161374; (2) blockers STACK and cc1 reveals only the first, so a function's tier is the MAX over blockers. - Smoke test 3 fns: oracles agree 3/3. src/ untouched (write-set asserted). Not yet population evidence -- that is T2 / KILL GATE A. |
||
|
Drew T
|
ef85803b1f |
feat(phase-29): backlog prune — compact the append-only near-miss log + wire into make report
The near-miss ledger (.run/backlog.jsonl) is append-only, so it filled with already-banked noise: 6,867 rows, ~98% banked. load_best()/render() already filtered on READ (docs/backlog.md was correct), but the raw log drifted stale and every render re-scanned all 6,867 rows against the stub oracle. - backlog.py: new `prune` subcommand — atomic rewrite (temp + os.replace) to load_best()'s output (drop-now-matched P9 + best-per-addr collapse). Idempotent. 6,867 -> 1,704 open near-misses. - Makefile: `backlog.py prune` wired into `make report` (BINARY=main block) so the ledger tracks reality every cycle instead of drifting. - Finding (Drew's question): crack waves DO log every non-byte-match to the backlog durably (gate_stage copies best_draft -> .run/backlog_drafts/). BUT the `closeness` field is UNRELIABLE — byte-correct drafts (match_one MATCH) are logged with closeness>0 (e.g. func_8012F49C logged 29, actually MATCH). And a reach-N function's draft is overlay-SPECIFIC (per-location symbols), so the backlog is a messy recovery source vs the fresh per-wave stranded drafts. Integration-recovery should consume the fresh wave-dir strandeds, not re-derive from the backlog. |
||
|
Drew T
|
ba0cfb751e | fix(workflow): wave_binary drafters MUST copy the winner back to draftDir/<fn>.c (a wave's 24 winners were written under scratch names and recovered only from agent transcripts) | ||
|
Drew T
|
e393c320e6 |
feat(phase-29): VARIANT camps -> UNIQUIFY (not reconcile); validated on Buf, R22 140/140 (§64a)
MEASUREMENT CORRECTED THE PLAN. The checkpoint called for a "per-camp field-access reconcile";
measuring the camps refutes that: Vec8 = {s32 w[8]} (32B) in 180 files AND {s16 unk0..} (8B) in
139 files; MATRIX 48B/32B/32B; Buf 16B / 0x20+ / DrawEnv. These are DIFFERENT types sharing an
identifier across TUs of the same overlay — reconciling to a canonical layout MERGES them, the
same failure that broke 103 binaries on Prim. The right op is UNIQUIFY: rename the non-majority
camp (byte-neutral — a type name emits no code; TU-local by construction), which makes every camp
single-def and liftable by the existing lift_types rules.
- NEW tools/uniquify_type.py: deterministic camp ordering (file-count desc, then normalized text,
so re-runs assign the same suffixes); majority keeps the name, camp n -> <T>_c<n>; rewrites ONLY
files that DEFINE that camp (a file that merely USES the name gets it elsewhere and is untouched);
\bT\b word boundaries so `Buf` never matches `Buf80153978`.
- VALIDATED on Buf (578/6/1 files): 11 identifiers across 7 files -> 3 camps LIFTABLE -> lifted
(585 local copies stripped) -> R22 140/140 -> blocked core queue 13 -> 11 (0x8012ea90, 0x801749c8
freed). Propagated 0x8012EA90 ×138; 0x801749C8 dropped (straggler in ov_SC07_006).
- YIELD, HONESTLY (P9): ZERO new matched functions. fn-count 88.61% / instr 79.3% / stubs 40281 all
UNCHANGED; dedup 1867->1868, C1 +138. 0x8012EA90's members were ALREADY matched in all 138
overlays — the propagation consolidated duplication into one shared macro (DRY), not coverage.
The value is the PROVEN RECIPE + the queue moving 13->11, not the numbers.
- dedup_propagate (R32): the skip line printed a COUNT and no names, and aggregated three unrelated
causes into n_local — a body skipped merely for a `//` comment (macro-unsafe, 1-line fix) read
identically to one genuinely using an overlay-local type. Now named and split by cause.
- cookbook §64a (uniquify-vs-reconcile + the validated recipe + remaining camps by cost).
|
||
|
Drew T
|
5ab80517de |
feat(phase-29): broad §20 type-lift lands — 154 types fleet-wide, R22 140/140 (§64)
The 3-session-carried blocker ("needs collision-vetting + -O0 strip precision") was
misdiagnosed on all three counts; fixing the instruments first (R35) changed every answer.
- (1) the "case-variant collision" is a TAGGED TYPEDEF counted twice with OVERLAPPING spans:
the inner span starts at `struct` so it emits a VARIABLE definition, the alias is redeclared,
and the highest-first strip leaves the outer end offset STALE -> over-deletes past the span.
MEASURED 13 pairs / 6142 occurrences / 0 standalone tags. build_engine_types.resolve_type_defs()
is now the ONE shared model (R33) + assert_disjoint() enforces span disjointness (R32).
The case-insensitive exclude was a heuristic over a structural fact — it would also have
wrongly dropped the legitimate Obj/obj + Vec/vec pairs. Key by (kind, name) — the C namespace.
- (2) the "-O0 strip precision" bug is a VISIBILITY bug: ov_SC01_077_o0.c is the 1 TU of 3226
that deliberately omits engine_core.h, so the strip DELETED its types; `multiple definition of
D_801DAA08` was 3 steps downstream (undeclared -> parse error -> implicit int -> tentative def
-> link collision) and named a symbol no diff touched. bet.type_visible() derives the visible-
header set from the include graph and keeps such defs local, named.
- (3) a third blocker, introduced this session and caught by R22: --candidates classifies per
ENTITY but emits per NAME, so passing `Prim` dragged in the deferred VARIANT `typedef Prim` and
repointed 103 overlays at the header's different layout. Compiled clean, per-binary pre-filter
green, R22 37/140 — the 103 failures were EXACTLY the 103 Prim-stripped overlays (set equality).
Fixed by the strip invariant "remove a local def only if what becomes visible is TEXTUALLY
IDENTICAL", placed at the MUTATION so a selector bug cannot reach the source.
- lift_types.py: (kind,name) entity keying, --candidates derived selector (retires the ad-hoc
102-type pipeline), divergence + visibility reports, whole-line strip (411 whitespace-churn
lines -> 0), complement-based single-pass rewrite (no mid-loop offset mutation).
- RESULT: 154 types lifted, 2958 files stripped, engine_types.h +510 lines. R22 clean-fleet
140/140 BYTE-IDENTICAL; tools-health OK (corpus 0/0, cdecl ALL GREEN, audit-binaries 140,
dedup-check 1854/0, C1 235170/235170); 0 NON_MATCHING (G4). Metrics unchanged 79.0/67.6/88.38
— honest: a type-lift banks no functions, it unblocks the NEXT propagate.
- DEFERRED + NAMED: 8 VARIANT entities (MATRIX 3-def, Buf 3-def, Vec8, Prim, Handler, Blk8, V8,
Prim_8016E7C8) = the per-camp field-access reconcile, the remaining hard part of roadmap B4;
14 carried tags; 5 types kept local in the -O0 TU.
- cookbook §64 (the three laws + the pre-filter lesson: a pre-filter is evidence ONLY about what
it filtered — ov_SC01_077 passed the Prim-broken run too; pre-filter on a binary that FAILED),
decision-log (R31), CURRENT_PHASE.md log.
|
||
|
Drew T
|
f62d3bcf37 | feat(phase-29): lift_types.py topo-sort (dep ordering) + broad-lift finding (needs collision-vet + -O0 strip precision; clean 2-type lift stands) | ||
|
Drew T
|
b921c00912 |
feat(phase-29): lift_types.py — fleet-wide §20 type-lift; Mat32+Cam8012E138 lifted (R22 140/140)
Builds tools/lift_types.py: targeted fleet-wide lift of a SPECIFIC type list into src/shared/engine_types.h, using build_engine_types' brace-aware parser (NOT the [^;]* regex that under-counted multi-field structs — R35, caught twice). It picks each type's CANONICAL (majority) def across the fleet, writes it once to the header, strips every local copy, and reports variant overlays. The whole-binary gate (R22) is the byte-arbiter. Applied to the 2 CLEAN types the §20-blocked fresh-138 cores need: - Mat32 (138 canonical + 1 copy-only draft variant in ov_SC07_006) + Cam8012E138 (unique) -> lifted, 139 local copies stripped. R22 clean-fleet 140/140. - Unblocks func_8012B4B8 + func_8012E138 for x138 propagation (next). DEFERRED (variant-heavy, high-risk — fleet genuinely split): MATRIX (3 defs), Vec8 (2 near-even 180/139), Buf (3), M8 (2) — lifting them would force ~929 files to one camp's field layout and break variant field-access. Needs a per-camp reconcile pass. func_80175308/func_8012A1BC still blocked (need those or other types). |
||
|
Drew T
|
f8d8f13b98 |
feat(phase-29): fix_header_decl v2 (--reconcile-externs) + SESSION-13 cont checkpoint
v2 generalizes v1: also reconciles CALLEE decls in shared headers that a byte-perfect draft declares differently than engine_core.h (sources the byte-true sig from the draft's `extern` line; reuses v1's compatible/build_decl machinery). --check validated: func_80174CB0 -> func_80012C6C s32->s16 (all SAFE); v1 unbroken (func_8014CD80 ALREADY-OK). The byte-gate + R22 arbitrate scalar-width changes. Known limitation (byte-gate caught it, fail-closed, nothing banked): v2 fixes only src/shared/*.h, not per-overlay-local decls — func_80174CB0 stays blocked by a local decl of func_80012C6C in ov_SC07_006_jr_8015C32C.c:8523. v3 = extend to the overlay split .c (carried in the checkpoint). Also in flight (checkpoint): crack wave w9lidyi5b (24 fresh LIVE=138 families, running) + permuter winner func_8014D12C (score-0, staged) — both gate-on-completion. |
||
|
Drew T
|
cbb52ebce1 |
feat(phase-29): fix_header_decl.py — automate the fresh-138 def-side header reconcile (§63)
Builds + self-tests the tool that reconciles a shared-header forward-decl to a function's byte-true signature — the integration half of the proven func_8014CD80 x138 pipeline. - tools/fix_header_decl.py: --fn --draft [--check|--apply]. Parses the byte-true def sig, canon-compares (typedef-aware int==s32, unsigned short*==u16* -> skip ALREADY-OK), REFUSES ABI-changing rewrites (param-count / ptr<->scalar / scalar class), preserves the macro `\` continuation, snapshots + prints the git-restore (§61: undo = restore, validate fleet-wide by R22). - 4 self-tests pass: (1) idempotent on the already-fixed func_8014CD80; (2) correct rewrite + `\` preserved; (3) REFUSE on a 2-vs-3-param ABI mismatch; (4) end-to-end --apply on func_8014D12C turned the `conflicting types` PLUMBING into a clean codegen DIFF (plumbing dissolved; its body is a near-miss so no bank), reverted clean. The self-test CAUGHT two fleet-corrupting bugs before any apply (a dropped macro `\`, and non-idempotent int/s32). - cookbook §63 + CURRENT_PHASE. NEXT: fresh-family wave over the 38-family market -> fix_header_decl --apply -> harvest_verify -> dedup_propagate -> R22. |
||
|
Drew T
|
57ef4ebcb5 |
feat(phase-29): ov_SC07_006 reach-138 batch-1 propagation + live-count re-scope (SESSION-13)
Propagation of the 6 batch-1 x1 banks (§55b: banks committed first in commit:0848, then targeted propagate as a standalone step): - dedup_propagate --addr: func_801325B8 -> +3 onboarded-tail siblings (ov_SC07_007/010/011). func_8014A048/func_801678F0 byte-diverge in the SC07 cluster (kept x1); func_8014FE60/func_80167540 local-type-blocked §20 (x1). - func_80165CA0: consolidated its h_exact subgroup (dedup group registered, +0 new), then family_sweep --hseq 0/135 — a PER-MEMBER WALL (cf func_80133AB0 0/136). The x135 "fresh family" prize does not exist here. - Net batch-1 yield ~9 newly-matched functions; fleet 78.6->78.7% instr, distinct flat; ov_SC07_006 84.6->84.8%. R22 clean-fleet 140/140; tools-health 1850/0. The finding (R14/R35, decision-log 2026-07-23): nins*reach leverage over-counts — rank by LIVE-siblings. build_wave_args.py --rank live now ranks by the true lever and reports the fresh(76)/onboarded-tail(44) split. The reach-138 family well is largely SPENT via wave+gate; the fresh families are the hard tail (def-side plumbing/DIFF/per-member walls), not free x138 fuel. |
||
|
Drew T
|
b76ad85157 |
feat(phase-29): ov_SC07_006 reach-138 crack-wave batch-1 — 6 x1 banks
Option (b) from the SESSION-12 checkpoint: fresh-exemplar crack-wave on the P27-onboarded ov_SC07_006 (84.6% -> 84.8%). New reusable builder tools/build_wave_args.py emits wave_binary.js args from a fuel manifest. - Drafting wave (wave_binary.js, 24 xHigh drafters over the top-24 reach-138 WAVE families by leverage): 16 self-assessed MATCH, 8 hit the session usage limit (redraft later). - Byte-gate: plain harvest_verify banked 2 (func_801325B8, func_80165CA0); gate_stage reconcile ladder (arity pre-pass + cast/sig_unify) banked +4 (func_8014FE60, func_8014A048, func_801678F0, func_80167540). 7 near, 6 reconcile-fail, 2 CC1-FAIL, 2 DIFF (the incomplete session-limit drafts). - R14/R35 leverage reality-check: nins*reach over-counts. 5 of 6 are already matched in ~135 overlays (only the onboarded SC07 tail ov_SC07_007/010/011 is live, +2-3 each). func_80165CA0 is a TRUE fresh family (135 live) — the x135 sweep prize, propagated next. - R22 clean-fleet 140/140 (engine_core.h arity edit fleet-safe); tools-health green (dedup 1849/0). Propagation is the next commit (§55b: banks first). |
||
|
Drew T
|
57af7ea889 |
feat(phase-29): ov_SC06_018 non-jtbl wave batch-1 (8 x1 + 30 swept = 38 instances) + fix gate_stage arity-undo revert bug
- batch-1 wave: 25 non-jtbl ov_SC06_018 targets (reach 3-14 modal), 13/25 match_one MATCH; whole-binary banked 8 (4 plain + 4 via gate_stage --src-file), swept 4 families -> 30 members across 13 overlays. 5 matches deferred (missing-sym/s58/deeper plumbing); 12 nears are permuter fuel (several close=2/3/4). - KEY: non-jtbl fns in a jr-split file need gate_stage --src-file <the jr TU> (the default- TU reconcile misses them -- same class as the jtbl --src-file fix). - BUG FIX (R33): gate_stage's arity-undo snapshotted src/<bin>/*.c BEFORE _gate1 splices the banks there, so an unbanked draft in the batch triggered a snapshot-restore that SILENTLY REVERTED the banks (measured: a 9-draft run banked 4, the 5 unbanked reverted all 4 to INCLUDE_ASM). Fix: restore ONLY src/shared/ (the fleet hazard the snapshot exists for); the binary's own TU arity edits are local + byte-neutral. GATE_NO_ARITY=1 was the interim workaround. s61's law from below (undo scope must not EXCEED write scope). - incremental check-all 140/140 (concurrent with batch-2 drafting; full R22 after batch 2). fleet distinct 67.5->67.6% (+38 unique fns), instr 78.6% steady. |
||
|
Drew T
|
3ebdd91fc6 |
feat(phase-29): jtbl post-carve reconcile (harvest_verify._jtbl_reconcile) + REFUTE the reach-138 jtbl families as plumbing wins
- ROOT CAUSE: the jtbl carve's s8b carried-decl layer conflicts with each draft's externs; the reconcile chain (cast_call_sites+reconcile_tu, --src-file-aware) exists but gate_stage runs it PRE-carve against the wrong TU (a jtbl fn's real TU is the split file, which doesn't exist until harvest_verify carves). - FIX (cookbook s62): harvest_verify._jtbl_reconcile runs the chain POST-carve against the CARVED TU, draft-only rewrite, guarded by _jsnap is not None. Validated: func_80135260 (callee) + func_80191C50 (data) both conflicting-types -> genuine DIFF. - FINDING (R14/R31 -> decision-log): dissolving the plumbing REVEALED all 4 jtbl drafts have a deeper issue -- func_80135260/80191C50 real %hi-share regalloc residual (agents' reloc-masked match_one MATCH over-claimed it); func_8012AAAC def-side-arity + FLEET- SHARED + still DIFFs after arity fix (def-side wall); func_80135EB0 isolate FAILED. The '+0.58pp from 3 reach-138 jtbl families' is REFUTED -- near-misses/walls, not plumbing. The fix banks any plumbing-ONLY jtbl fn + makes the jtbl gate honest. - s61 traps re-confirmed (s62): gate jtbl ONE-AT-A-TIME (mid-batch isolate-FAIL corrupts the batch); fix_arity on an engine_core.h fn leaks fleet-wide (restore src/shared/ too). - R22 clean-fleet 140/140; tool change only (no committed-byte change). |
||
|
Drew T
|
f142a6858f |
feat(phase-29): ov_SC06_018 crack-wave — func_801365B8 x138 (fresh-exemplar sweep CONFIRMED), thesis is family-specific
- binary-aware crack wave (new tools/workflows/wave_binary.js): 8-target calibration over ov_SC06_018 substantial stubs, 7/8 match_one MATCH - func_801365B8 (155, reach 133): cracked FRESH in ov_SC06_018, swept 132/132 siblings via family_sweep --hseq --source ov_SC06_018 --allow-pins -- SESSION-10 refused this family 0/133 from an ov077 exemplar. THESIS CONFIRMED (fresh exemplar unlocks it). - func_80133AB0 (137, reach 137): cracked fresh + banked x1 (+ a byte-neutral s17a-1 cast reconcile of banked caller func_801343C4), but the family sweep FAILED 0/136 even from the fresh exemplar (reverted clean) -- THESIS REFUTED for this family. - FINDING (R14/R31 -> decision-log): the fresh-exemplar sweep is FAMILY-SPECIFIC, not a blanket mechanical x137. A fresh crack is necessary but not sufficient; the byte-gate arbitrates each family (~50% on this 2-family sample -> discount the ~1.5pp estimate). - tooling (R33): family_sweep --source override now searches matched_members (a fresh member leaves 'members' after a sig-regen); cdecl._depth0_spans consumes backslash line-continuations so a raw-draft #define macro no longer trips audit-cdecl. - R22 clean-fleet 140/140 byte-identical; tools-health green (dedup 1849/0, C1 234615); 0 NON_MATCHING. fleet 78.4->78.5% instr / 67.1->67.5% distinct / 88.14->88.18% fn-count. |
||
|
Drew T
|
80d570195c |
feat(phase-29): add main to the weighted denominators (roadmap §1 metrics contract)
The contract requires all three headline metrics to include the main EXE. Since Phase-27 T10 main was reported as a SEPARATE provisional line, so the headline silently measured 139 of 140 binaries — and flattered itself by omitting the LEAST-decompiled one. RESTATED CAVEAT (the old "stale, PROVISIONAL" wording was misleading): main's sig is Ghidra's (2026-06-14), but function BOUNDARIES derive from the original bytes and do not change, and matched-vs-stub comes from the LIVE corpus.stubs — so the numbers do NOT drift. The real limitation is R34: sig_image cannot independently validate a PS-X EXE's boundaries, so main has no SECOND, DISAGREEING oracle for the PHANTOM/TRUNCATED class. The sig also excludes the LINKED PsyQ objects, which is exactly right for a GAME-CODE contract. EFFECT — the headline DROPS, which is the point: instr-weighted 78.7% -> 78.4% (10,299,493 / 13,141,652) distinct-code 67.9% -> 67.1% (3,782,990 / 5,634,875) A continuity line reports the ex-main figure so pre-2026-07-22 readings stay comparable, and the binary-count label is corrected to "main + resident + 138 overlays" (it read "resident+139 overlays"). Metrics-only change; no build impact (ov_SC06_018 re-verified BYTE-IDENTICAL). |
||
|
Drew T
|
c6515a6cdc |
fix(phase-29): the FLOOR VERDICT was an artifact of snapshot frequency, not yield decay
burndown.py averaged the last 3 INTER-COMMIT deltas. The phase's ROI criterion is "per-SESSION yield floors out", and historically one snapshot was taken per session — but I seeded THREE inside this session (9, 9b, 9c). Averaging them drove the mean to +0.23 and printed "AT THE FLOOR — consider closing P29" for a session that actually yielded +0.7pp instr (78.0 -> 78.7), comparable to recent sessions. I nearly closed the phase on it. Same error class as everything else this session: an instrument answering a DIFFERENT QUESTION than the one asked, where the wrong answer is indistinguishable from the right one. - --session-close marks a snapshot as a session boundary; the floor verdict is now computed ONLY from those (older records predate the flag, so a label containing "close" counts too). Mid-session snapshots still record for tracking. - honest output now: "0 SESSION-to-SESSION delta(s) logged — need >=3 for a floor verdict (1 session-close snapshot on record)". => P29 must NOT close on ROI grounds yet: the floor is UNDETERMINED and needs 3 session closes to become computable. The eyeballed "+2.5 -> +1.1 -> +0.6 -> +0.2" trend is not the instrument's verdict either. |