mirror of
https://github.com/Druthulu/BFM-decomp
synced 2026-09-26 21:36:06 -04:00
b8ee9fa9d27fc06082bc8f02ca59f0d656dfa9a8
138 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Drew T
|
b8ee9fa9d2 |
feat(phase-29 Task-13B): directed permuter wins first try; 3 latent gate defects fixed at root
First run of the Task-13A-targeted grinder (--once --batch 8 --permute-secs 90):
targeting: ON — 1665 classified; only bucket=permuter is admitted
targeting skipped 1601 non-permuter candidates
{redraft 707, structural 583, integration 305, unclassified 4, unknown 2}
permuter WON func_80181F78 @ ov_SC03_014 (close was 1) [~6 min]
BANKED 2 (both whole-binary byte-gated, R22 clean-fleet 140/140):
- func_80181F78 (8 ins) — classified DELAY-SLOT / schedule
- func_80141B90 (29 ins) — classified IMM-VALUE / cse
Both were classified bucket=permuter by residual_class BEFORE the run: the classifier
predicted a search-closer could reach them, and one did. First end-to-end validation of
the targeting thesis.
THREE latent defects, all pre-existing, all unreachable while the grinder banked nothing
since Phase 21 — the fix made it win and every one fired at once (cookbook §60a):
1. gate_stage commit path crashed on src=None. `src` is DELIBERATELY never defaulted
(Phase 26-A: a default silently PINS the gate to the main .c), but the commit did
`git add src …` unconditionally -> every caller that omits it (grinder, orchestrator,
idiom_hunt) crashes THE MOMENT IT BANKS. Now `git add -u src/`, which also retires the
`src/ov_*/*.c` filename glob that once omitted 4 R22-verified banks from a commit.
2. _xform ladder dirs (-cn/-cast/-rc/-uni) ACCUMULATE across runs: they held 34 stale
drafts, so the gate processed 34 when the grinder submitted 1, and banked one function
it was never asked to try — which would have been committed under a message naming a
different one. Nothing wrong entered the tree (G3/P9 held; the gate banks only
byte-identical output) but report and work had diverged. Now cleared per run.
3. grinder called gate_stage with the default propagate=True = `dedup_propagate
--auto-from`, the §55b fleet-wide path that timed out at 3600s and left 90/140 overlays
broken — and being INSIDE the gate it takes the banks down with it. Now propagate=False;
banks commit cheap, propagation is its own targeted --addr batch (§55b law).
R22: make clean && extract-all && check-all -> 140 passed, 0 failed of 140.
tools-health OK (dedup 1847/0, C1 234343/234343); 0 NON_MATCHING (G4).
|
||
|
Drew T
|
b3ab5c2253 |
feat(phase-29 Task-13A): deterministic residual classifier — the permuter's problem is TARGETING
The autopsy (hindsight-study §7) assumed the permuter loses for want of a mutation. Measured over the whole open backlog, it loses because it is aimed at work a search-closer provably cannot close. - NEW tools/residual_class.py: decide a near-miss's class FROM THE BYTES. Decodes each mismatching MIPS word -> (op-skeleton, register-fields, immediate); drift FIRST (one inserted insn inflates `closeness` by the tail length), then consistent-injective register map -> REGALLOC-PERM (§31 S11/RC-3), same-multiset-reorder -> SCHEDULE-REORDER, DELAY-SLOT, WIDTH/BRANCH-POLARITY/STRENGTH/ADDRESSING/IMM-OFFSET/IMM-VALUE. Every class routes to a BUCKET = which tool the failure wants. Uncovered opcode -> UNKNOWN, COUNTED (R32). 16 synthetic unit tests (test_residual_class.py). - NEW tools/autopsy.py: `collect` materialises the corpus Task-12's telemetry never filled (1 of 6,169 records had a residual) by recompiling every open draft through the EXISTING match_one path (R33) — 1,752 drafts in 21s at -j12. `report` -> docs/autopsy.md. - NEW corpus.o0_sources()/is_o0(): the opt-level oracle DERIVED from the Makefile's own -O0 rules, coverage-asserted. Scoring an -O0 target at -O2 makes the residual 100% artefact (the trap this phase hit four times). - R34 cross-check baked in: residual_class's closeness vs masked_diff.structured_diff's, asserted per row; 1,673/1,673 agree, 0 classifier errors. FINDING: of the 972 records the grinder's own filter admits, only 75 (7.7%) are permuter-shaped; 547 are structural and 348 are drafts that are not the function at all. ~92% of the daemon's CPU went where it could not win — the byte-grounded explanation of "7 banks all-time, all Phase 21, 0 since" (Phase-22 audit). grinder.candidates() now filters on the measured bucket (1,303 -> 78) and takes its directed profile from the measured class, not the logged label (91% carry none -> it ran on gcc defaults). Degrades to undirected if uncollected and says so; --no-targeting A/Bs it. Two measured corollaries (R14, not projections): - 699 records rank as near-misses at closeness up to 278 purely from a length artefact: un-attempted work misfiled as a backlog of hard functions -> new `redraft` bucket. - a 12-draft gate probe of the `integration` bucket banked 1/12 (11 PLUMBING), so the 306 prices Task 14's reconcile ladder rather than promising free banks. func_80167714 (104 ins, reach-134) banked x1, un-propagated by design (§55b). Two defects fixed forward: - masked_diff._common_typedefs() used ONE shared probe path, so parallel match_one processes clobbered each other: 14 of 1,752 drafts lost in a single 12-way run (0.8%), silently, in every parallel wave ever run. Now per-PID. - gate_stage.match_one_closeness never passed --o0 -> phantom residuals for every -O0 function, written straight into the backlog this autopsy reads. R22 clean-fleet: check-all 140 passed, 0 failed of 140; tools-health OK (dedup 1847/0, C1 234343/234343); 0 NON_MATCHING (G4). Flywheel captured in-session (R30/R31): cookbook §60, decision-log entry, SETUP.md inventory. |
||
|
Drew T
|
e4e8ec5ca9 |
docs(phase-29): SESSION-5 CHECKPOINT — full session-checkpoint block for a fresh session
Session-5 (crack-waves 1-4 + 2 tooling fixes + burndown + hindsight-study + Task-12 telemetry). 140/140 byte-identical, fleet 78.0% instr. NEXT: Task 13 (classifier + autopsy) then Task 14 (gate_stage ladder + auto-R22). Written BEFORE the pause per the new checkpoint-before-pause rule. |
||
|
Drew T
|
a5bad7f48d |
feat(phase-29 Task-11): burn-down + velocity tracker (the ROI-close instrument)
tools/burndown.py snapshots the 3 fleet metrics + remaining substantial frontier each run (docs/burndown.jsonl, git-HEAD-keyed), computes velocity vs the previous commit-snapshot, and flags the yield floor (mean instr-pp over last 3 snapshots < threshold => 'consider closing P29'). Seeded with the session-4-close baseline. Reads progress.fleet.md + family-hseq.md (regen those first for a true reading). |
||
|
Drew T
|
1ad5cc359a |
docs(phase-29 crack-wave2): session-4 checkpoint — +819 members (76.3% instr), frontier honest
Crack-wave 2 (absent-from-ov077 pool) + propagation: 6 families banked ~815 members fleet-wide (R22 140/140). Session-4 total: 75.2->76.3% instr / 60.6->62.8% distinct / 87.04->87.47% fn-count across 2 crack-waves (~1500 banks). Tooling finding: family_sweep --source is ignored (uses manifest exemplar). 2 preserved permuter seeds. Non-MCP substantial frontier worked out; next fresh cracks need /mcp prefetch. |
||
|
Drew T
|
d7092de63a |
docs(phase-29 crack-wave): session-4 close — +412 residuals verified, cookbook §59
func_80165240 137/137 + func_8016D1D8/D688 274/274 + func_8013C0F8 x1 banked by the residual agent; 2 jtbl cores blocked on §8c/§8e-2 tooling walls. R22 clean-fleet 140/140, tools-health OK, 0 NON_MATCHING. Session-4 fleet 75.2->75.7% instr / 60.6->61.3% distinct / 87.04->87.24% fn-count (~687 member-banks). Cookbook §58/§59. |
||
|
Drew T
|
c9e12a079b |
feat(phase-29 crack-wave): +6 cores banked ×1 in ov_SC01_077 (R22 140/140)
Ultracode 11-core crack-wave over the freshly-regenerated draftable structural frontier (R35: the Jul-14 manifest still listed already-banked families). 9 match_one MATCH / 2 near; 6 of 9 banked whole-binary byte-identical. Banked (ov_SC01_077, ×1 — ×138 sweep deferred, Drew paused after the bank): - func_80150170 (95, hexR=138 dedup core) — engine_core void->s32 narrow - func_8016D1D8 (148) — data-label + typedef-scope - func_8016D688 (60) — data-label (D_801D9C20) - func_80165240 (63) — normalize_self_decls (caller-decl -> void(void*x3)) - func_80164E40 (25) — engine_core void->s32 narrow - func_801457A4 (79) — -O0 (relocated into _o0b object) - engine_core.h: 3 decls narrowed void->s32 (byte-neutral fleet-wide; callers ignore the return) — R22 clean-fleet 140/140 confirms neutrality. - config/symbols.ov_SC01_077.txt: D_801D9C20/60 u8 data-label mirror so a re-extract re-emits the labels the D1D8/D688 banks reference (R22 corollary; fixed a type:data->type:u8 splat-format bug that broke ov_SC01_077 extract). - 3 NOT banked (801549F8, 8013BD74, 8013C0F8): all match standalone, blocked ONLY by the §8 jtbl-rodata carve (NOT codegen; C0F8 was NOT a real DIFF). - .run/giants: 2 near (8014D820 close-11 intrinsic-sched, 8012E364 close-22) + the 3 jtbl-blocked drafts preserved (R20). R22 clean-fleet: 140/140 byte-identical; tools-health OK (dedup 1846/0, C1 234205/234205); 0 NON_MATCHING linked (G4). Fleet 75.2/60.6/87.04 (flat — ×1 banks; the ×138 sweep is the deferred fleet-mover). cookbook §58 (R30): match_one MATCH != bank — it compiles standalone so it is blind to (a) Ghidra symbol names, (b) def-sig conflicts vs the fleet, (c) callee-decl conflicts, (d) -O0-vs-O2; crack-wave drafts need a reconcile pass. |
||
|
Drew T
|
53433df1b3 |
feat(phase-29): func_8013D53C 14/137 + NSD hardened (rewrite-to-canonical, (void) fix) + Cmd lift
- func_8013D53C: NSD (self-decl caller-thread) + a byte-neutral Cmd_8013D53C type-lift clear the plumbing -> 14/137 h_seq members bank; 123 have genuine per-member codegen DIFFs (hard cse.c-wall crack -> permuter fuel). Reverted the 123 bankless byte-neutral NSD edits (churn on matched code, §57a NB). - NSD hardened, 2 bug-fixes validated on D53C: REWRITE the divergent decl to canonical instead of DROPPING it (drop broke def-after-caller forward visibility -> 'func undeclared'); and '(void)' is a genuine 0-param conflict, not skippable like a true no-proto '()' (the bug that filed D53C's arity class as 'not this pass'). - earlier this session: broad --fix-def-sig sweep = HONEST +137 only (staging != banking, R14/R35); NSD is SURGICAL-ONLY (broad application poisons whole groups; --fix-def-sig is broad-safe). cookbook §57a. - R22 clean-fleet 140/140 byte-identical; tools-health OK (dedup 1846/0, C1 234205/234205); 0 NON_MATCHING (G4). fleet 75.2% instr / 60.6% distinct / 87.04% fn-count. session-total from 74.6: +0.6pp instr, ~+421 members. |
||
|
Drew T
|
65c93ea035 |
feat(phase-29 type-lift): func_8016CBC0 family 1->138/138 (+137) via the 6-typedef engine_types.h lift
- func_8016CBC0's 137 members were NOT self-decl-blocked (§57 survey: 0 divergent self-decls) but blocked on the exemplar's 6 LOCAL typedefs (Rec_/Blk_/SVEC_/CLR_/Poly_/Mtx8_8016CBC0). Lifted all 6 verbatim into src/shared/engine_types.h (fleet-visible via engine_core.h) + removed the locals from the exemplar TU. No fleet name-collision; exemplar rebuilds byte-identical d19c9580 (the type-lift is byte-neutral). - family_sweep --hseq --only 0x8016CBC0 --no-preclassify --normalize-self-decls -> 137/137 banked, 0 failed (the decl-normalize stage a correct no-op here — 0 self-decls). 0 func_8016CBC0 stubs remain fleet-wide. - R22 clean-fleet 140/140 byte-identical; tools-health OK (dedup 1846/0, C1 234205/234205); 0 NON_MATCHING linked (G4). fleet 74.9->75.1% instr / 60.1->60.6% distinct / 86.96->87.00% fn-count. - R14 lesson: two families that both looked like a 'stuck 137-family' needed DIFFERENT levers (decl-normalize §57 vs type-lift) — route by the real cc1 error, not the shape. session-total +270 members. |
||
|
Drew T
|
514383de6a |
chore(phase-29 crack-wave): log wave2+fable+type-lift outcome (fleet 74.6% instr) + preserve drafts
Session-total 72.1->74.6% instr (+2.5pp). 8 exemplars + ~815 members banked; 3 Fable source-read cracks (cse.c address-CSE, post-reload sched, reorg j-slot). Follow-ups: 801670e4/8016cbc0 per-sibling decl-norm, 8013d53c caller-thread, 801412a8 §29 wall, 4 permuter near-misses. All drafts preserved (R20). |
||
|
Drew T
|
8bd1b2b175 |
chore(phase-29 crack-wave): log wave outcome (3 MATCH/4 near, 8013CB84 x138) + preserve drafts + fleet 73.7%
Ultracode 7-agent fan-out. func_8013CB84 banked x138 (+34k ins). 2 jr MATCHes deferred on def-side/-O0 walls (characterized+preserved R20). 4 near-misses + 8014D820 perm27 = permuter/Fable5 fuel. Crack-wave sweep recipe learned (map regen -> matched-ov077 -> standard sweep). R22 140/140. |
||
|
Drew T
|
4310269bb8 |
feat(phase-29 §8e): campaign close — 3 families swept (410 banks), fleet 73.4% instr / 56.9% distinct (v-batch R22 140/140)
- func_80131340 138/138 · func_80159C84 134/138 (4 SC07 named) · func_8013F350 138/138 · func_8013C414 x1 (family deferred on the -O0-cluster rollout wall, uniform + fail-loud). - cookbook §8e-2: clean-draft law, zero-word spec rule + tables= persistence + --like transfer, named deferral classes. progress.fleet.md digest regenerated. Clean drafts preserved (R20). |
||
|
Drew T
|
1dbf8310ef | chore(phase-29 T4): regen progress.fleet.md digest (fleet 72.1% instr after session-2 banks) | ||
|
Drew T
|
fd564a2cf7 |
feat(phase-29 T3): propagate the 3 self-contained cores ×137 (+410 instances; fleet 71.0->71.4% instr)
- targeted dedup_propagate --addr per core (NOT --auto-from), --recover for stragglers:
0x8014ADE0 -> 138 overlays byte-identical
0x801325B8 -> 134 (ov_SC07_011 byte-diverges -> auto-excluded, kept x1 — what --recover is for)
0x801387B8 -> 138 overlays byte-identical
= ~410 member-instances; 3 new dedup groups (1840 -> 1843), C1 coverage 233795/233795.
- 2 of the 5 banked cores (func_8014E284, func_80137DD4) stay ×1: "not self-contained (local types)"
-> blocked on the build_engine_types type-lift (the §19/§20 propagation cap). Carried.
- R22 clean-fleet 140/140 BYTE-IDENTICAL; audit-binaries OK; dedup 1843/0; 0 NON_MATCHING (G4).
Fleet instr 71.0 -> 71.4% / fn-count 86.30 -> 86.42% / distinct-code 53.3%.
- SELF-CORRECTION (R14/R35), now fixed in cookbook §55c + CURRENT_PHASE: my earlier claim that this
propagate "needs ~2h+" was WRONG. That timing was taken while the tree still carried the partial
damage of a killed --auto-from (90/140 overlays broken), so every member-gate was failing/retrying.
On a HEALTHY tree a targeted --addr propagate is ~233s/core (all 3 = ~27 min) — ~20x faster. Only
--auto-from is genuinely fleet-slow. A timing taken on a broken tree measures the breakage, not the
tool — recover the tree FIRST, then measure.
- cookbook §55: the wave's new byte-proven levers (§49-variant birthing-boost suppression via
reg_n_sets 1->2; sched1 birthing/LUID + "cc1 -dL" movable introspection; switch-tree vs jtbl
CASE_VALUES_THRESHOLD=5; block-scope-extern beats *(T*)&sym) + the GATE-ORCHESTRATION law
(--no-propagate per group then ONE targeted --addr; commit banks BEFORE propagating; a reverted src
needs a re-extract; gate_stage's default harvest_verified.txt accumulates -> phantom banks).
|
||
|
Drew T
|
b5340f920b |
feat(phase-29 T3): core-crack wave — 5 cores banked x1 (13 agents: 7 match/6 near); propagate deferred + 3 tooling traps logged
- ULTRACODE worker_wave: 13 xHigh drafters over 6 cores (260-371 ins) + 7 B3 near-misses (100-141).
Usage-limit hit at 2/13 -> RESUMED (cached replay) -> 13/13 done, 7 match / 6 near.
- BANKED x1 (whole-binary byte-gate; R22 clean-fleet 140/140): func_8014ADE0 (139), func_8014E284 (108),
func_80137DD4 (129), func_801325B8 (113), func_801387B8 (100). NEAR: func_8013FAF8 (312 giant, def-sig
s16/s16 vs canonical s32/s32), func_8014F4C0 (in-TU byte-verified; unresolved decl conflict).
- NEW levers for the flywheel: §49-variant birthing-boost suppression via reg_n_sets 1->2 (func_801325B8);
sched1 birthing/LUID + "cc1 -dL" movable introspection (func_80177940); switch-tree vs jtbl
CASE_VALUES_THRESHOLD (func_801387B8). 6 near-misses carry byte-proven residual analyses.
- THREE TOOLING TRAPS (mine; ~3.5h lost, 0 data lost — all recovered):
(1) gate_stage's propagate is FLEET-WIDE (--auto-from): running gate_stage per-group ran it 4x
redundantly; 3 timed out at 3600s -> partial damage (90/140 broken, 887 files). FIX: --no-propagate
per group, then ONE targeted --addr propagate.
(2) a reverted src needs a RE-EXTRACT (R22 corollary): asm/ kept the banked state -> corpus.CorpusError
'5 stubs have NO .s on disk' (R34's second oracle caught it, working as designed).
(3) gate_stage's default .run/harvest_verified.txt ACCUMULATES + no --verified-out CLI -> PHANTOM
'banked:1' for a still-stubbed fn (R32/R35 class, still armed). Trust the SOURCE, not the report.
LAW: commit cheap verified banks BEFORE the expensive propagate.
- PROPAGATE DEFERRED, properly sized: 3/5 self-contained x138 = ~414 instances; 2 blocked on local types
(type-lift); straggler ov_SC03_093 needs --recover; measured ~2h+ (my 3000s guards killed it twice).
- fleet instr 71.0% / distinct 53.3% / fn-count 86.30%; dedup 1840/0; 0 NON_MATCHING (G4); main 143dbb89.
|
||
|
Drew T
|
afd1aeea4f |
feat(phase-29 T6): broad --fix-def-sig harvest +19 (def-sig lever tapped beyond the mega-pools)
- fleet-wide --band substantial (17) + tiny (2) with --fix-def-sig (all families) = 19 more members; the def-sig conflict was concentrated in the 2 tiny-IMM mega-pools (already banked, now not-stub) - R22 clean-fleet 140/140 byte-identical; pure-reduction; dedup 1840/0; 0 NON_MATCHING (G4) - fleet instr 71.0% (steady) / distinct 53.2->53.3% / fn-count 86.30%. Task-6 mega-pool track complete; permuter backlog (grinder/permuter_ils close-1..4) remains as the other Task-6 half |
||
|
Drew T
|
941cd37b19 |
feat(phase-29 T6): tiny-IMM mega-pools CRACKED +4,801 via family_sweep --fix-def-sig (fleet 70.4->71.0% instr)
- THE FIX (new): family_sweep --fix-def-sig (header_sig_map + reconcile_def_sig, 1005 mapped fns) — rewrites each member draft's DEF signature to the shared-header (engine_core.h) canonical decl. Root cause (byte-proven, R14/R35 after 3 masked-metric mis-reads): engine_core.h forward-declares the member (extern void func_8015FAAC(s32 *a0), a shared fn calls it) while family_remap copies the EXEMPLAR sig (void *a0) -> 'conflicting types' -> member TU never compiles. Invisible to standalone diff_regions/match_one (no header conflict) AND to --reconcile. Byte-neutral (ptr-type param, gate arbitrates G3/P9); one member hand-verified byte-identical first. - 0x80131eec 2331/2470 (94%) + 0x80130d0c 2470/2496 (99%) = 4,801 members banked across 405 overlay files. R22 clean-fleet 140/140 byte-identical; pure-reduction (0 new/dup stubs); dedup 1840/0; 0 NON_MATCHING (G4). Fleet instr 70.4->71.0% / distinct 52.3->53.2% / fn-count 84.94->86.30%. - CORRECTS the commit:0665 'symbol-definition gap' scout (WRONG). The 4th 'reproduce the build step' instance (§53-carve, -O0-flag, now the member's canonical DECLARATION). cookbook §54, decision-log R31. |
||
|
Drew T
|
f6f89781ff |
feat(phase-29 T2 Arm A): swing verdict = BANKED FACT (9/9 -O0 members on ov_SC07_010); fleet -O0 rollout deferred at the splat wall
- tools/rollout_o0_cluster.py (new) + Makefile O0_CLUSTER_OBJS -O0 wildcard: the -O0-cluster carve (0x13410..0x14834), adapting rollout_whale_o0.py to a 3-way <ov>/<ov>_o0/<ov>_o2b split - ov_SC07_010: carve byte-neutral -> family_sweep --hseq banked 9/9 -O0 exemplar-family members whole-binary (R22 clean-fleet 140/140). The Task-1 masked-MATCH swing verdict is now a BANKED FACT: -O0 cluster members DO bank at -O0 (§52b). Phase-20 'func_8013B7AC overlay-local' refuted. - THE WALL (byte-proven, TOOLING not compiler): the same carve on 006/007/011 byte-shifts the whole image (+0x20 %lo data-symbol shift, 34% diff) from a CLEAN build; boundaries verified as real fn-starts. Root cause = splat re-disassembly of a 3-way-split subseg that still holds INCLUDE_ASM stubs (the whale's stub-free _o0b shape avoids it). The Phase-20 '-O0 split infra' wall, root-caused. - DEFERRED (ROI): full -O0 fleet rollout (~1,233 / ~0.6pp) — 3/4 sampled walled + 134 jr-embedded + bigger levers (Task 3 core-cracks, Task 6 tiny-IMM ~5,566). decision-log R31 + cookbook §18-P29. - 140/140 byte-identical; dedup 1840/0; 0 NON_MATCHING (G4); main 143dbb89. Task 2 substantively done. |
||
|
Drew T
|
908dded511 |
feat(phase-29 T2a): Arm B tail — 748 members banked in the 4 SC07 tail overlays (fleet 70.2->70.4% instr)
- resumed the SIGTERM-interrupted comprehensive --band all sweep as 3 band-bounded family_sweep --hseq --allow-pins passes (substantial 53 + mid 372 + tiny 323 = 748), each exit 0 (the --band all SIGTERM lesson: band-bounded + committed-per-batch) - all 748 in the 4 files=1 SC07 tail overlays (ov_SC07_010 897->516, ov_SC07_011 797->436, 006 457->454, 007 595->592); verified pure-reduction (0 new/dup stubs, R14/H5) - 006/007 residual = jr-families (§53 carve, Task 3) + plumbing; plain-sweep tail drained - R22 clean-fleet 140/140 byte-identical; audit-binaries OK; dedup 1840/0; main 143dbb89; 0 NON_MATCHING (G4). fleet instr 70.2->70.4% / distinct 51.9->52.3% / fn-count 84.73->84.94% |
||
|
Drew T
|
5b6a8ae6cb |
feat(phase-29 T2a): Arm B type-lift sweep — 3,407 member-matches banked (fleet 68.9->70.2% instr)
Acting on Task-1's verdict (the legacy-PURE "~3%" is tooling, not a wall): the -O2 type-lift arm. family_sweep --hseq --no-preclassify --band all --allow-pins templates each matched exemplar's C onto its unbanked same-family members and whole-binary byte-gates every one (the sole arbiter, G3/P9). The pin-crash wall being dissolved (Phase-27/28 _carry_macros) let --allow-pins retry the pinned exemplars. - BANKED 3,407 member-matches across 136 overlays (INCLUDE_ASM stubs -> matched C). - R22 clean-fleet verify: 136/137 modified overlays byte-identical from a clean rebuild; the 1 FAIL (ov_SC07_010) was the SIGTERM mid-gate partial -> reverted, byte-identical. - make check-all: 140 passed / 140 BYTE-IDENTICAL; dedup-check 1840/0; audit-binaries OK; audit-cdecl green; 0 NON_MATCHING in any default build (G4). - Fleet: instr-weighted 68.9 -> 70.2% (+1.3pp) . distinct-code 49.5 -> 51.9% (+2.4pp) . fn-count 83.94 -> 84.73% (+0.79pp). - Process lesson (CURRENT_PHASE.md): --band all sweeps are too long for one background pass (got SIGTERM'd); future Arm B runs go band-bounded + committed-per-batch (resumable). Residual FAILED members (pin/drift/plumbing) + the -O0 Arm A carve remain. |
||
|
Drew T
|
3827d01bfb |
feat(phase-28): the endgame engine -> the instrument-repair phase; B2 lives, SC07 wired, R36 (v1.27.0)
- RE-SCOPED at plan time (R35): the roadmap's swing number rested on a broken-tool probe. B2's "structural families bank ~0%" (0/8, which reshaped 2 phases of strategy) was a MISSING CARVE -> 8/8, then 102/115 (88.7%). The ~0% doctrine has NO surviving post-fix evidence. - T0 img_path derive-not-guess -> un-hid a 230,612-ins SC07 pool doubly hidden (P27 onboarded 4 overlays, never regenerated the map; the hardcode would have called every member "LEN"). - T3-A: the SC07 pool is h_exact + UNWIRED, not h_seq. T4 dedup_extend (NEW) wired 6174/6457 (95.6%) -> groups 134->138 binaries, C1 coverage +6174. NOT member_adapt (the number said build nothing). - T5: resident 21->14 (90.34%, 7 banked via an Ultracode wave) + honest dossier for the 14; fixed progress.py (#if 0 + len()-sum) and match_one's FAKE isolation (found by an agent mid-wave). - T2 purged the poisoned grinder blacklist (8/22 matched anyway). T6 killed the --chunk 1 double-build. - T7: make audit-binaries (the R36 citizenship gate, negative-control-proven) + fixed disc_code_sweep's BLINDNESS to compressed code (the type-4 row was vacuous for 138 known binaries) + the worklist key-bug. - T3b: the legacy h_seq swing number = ~3% AS-TOOLED, CLASSIFIED (274 DIFF / 37 PLUMBING), ceiling UNKNOWN -- the 274 DIFF is byte-PURE members whose remapped bodies don't reproduce (the same tooling-vs-wall ambiguity that resolved to TOOLING twice this phase). P29 disambiguates before scaling. - THREE SELF-INFLICTED DEFECTS fixed forward: the registry yaml.safe_dump (H5, destroyed 47 comments + 1832 hex fields, invisible to every byte-gate), two DIFF mis-reports (R14), the match_one shared scratch. - rule R36 (a newly-discovered binary is not real until every consumer knows it; enforced by audit-binaries). cookbook §53. R22 140/140 throughout; tools-health OK; dedup 1840/0. |
||
|
Drew T
|
876dc7f053 |
feat(phase-28 T5): resident 21 -> 14 stubs (7 banked, 90.34%) + fix match_one's fake isolation
Ultracode wave: 16 isolated drafters over the resident's non-jtbl stubs (the 5 jtbl deferred —
they need the rodata-island carve, §53). Drafts only; the whole-binary byte-gate arbitrated after.
- BANKED 7/16, byte-gated: func_800CEFD0(77) func_800D0D7C(45) func_800D1B80(22) func_800D1E28(37)
func_800D1FC8(62) func_800D29F8(172) func_800D2D10(39).
Resident REAL 122 -> 129, stubs 21 -> 14, byte-ident 124/145 (85.52%) -> 131/145 (90.34%).
FLEET instr 9017152 -> 9017606 (+454 ins). R22 make clean && extract-all && check-all ->
140 passed, 0 failed of 140 (the first R22 was killed by a terminal crash and RE-RUN, not assumed).
Ground truth on 14 agrees 3 ways: source grep, splat-emitted stub .s count, progress.py.
- §52b's LAW, MEASURED AGAIN INDEPENDENTLY: the agents self-reported 11 match_one MATCH; the
whole-binary gate banked 7 (64%). All 4 blocked MATCHes died on `conflicting types`
(D_8010EDEC / D_80115110 / func_800D1984 / cdFileLocTable) — the loose-typing def-side wall, NOT
codegen. gate_stage's recovery banked 0/5 on them. A match_one MATCH is a CANDIDATE (G3/P9).
- FIX — match_one's isolation was FAKE, and its own docstring was the false spec. It promises
"Fully isolated (own temp dir) so many run in PARALLEL with no shared build -- a real asm-differ
loop for an agent to iterate against", while `--work` defaulted to the SHARED '.run/match': every
concurrent caller compiled into the same t.c/t.o. FOUND BY AN AGENT MID-WAVE, the only way it can
be found — it read another agent's function out of its own scratch ("found another agent's
func_800D2650 in my t.c") and reported it. Every other agent steered by a loop that could hand it
someone else's compile: a CONFIDENT WRONG verdict, worse than a crash. Default is now a private
.run/match/<fn>.<pid>; the default IS the promise. (Some agents had already worked around it by
passing --work themselves.) The byte-gate was never at risk — it is the sole arbiter — but the
iteration loop the agents steer by absolutely was.
- The 14 remaining: 5 PLUMBING (loose-typing) + 4 DIFF (genuine codegen: func_800D2650 close=4,
func_800CFAD0 close=5, func_800D0E30 close=12, func_800D27DC close=48) + 5 jtbl deferred.
Dossier next (T5b) — the agents' per-function residual analyses are the durable asset (R30).
|
||
|
Drew T
|
2c4e2344da |
fix(phase-28 T5): progress.py — #if 0 blindness + the len()-sum; resident 123/146 -> 122/145
R35: fix the instrument before planting a flag on its denominator.
- #if 0 BLINDNESS: classify() knew `#ifdef NON_MATCHING` (:425) but not `#if 0`, so a dead
analysis body was read as a live definition AND its real INCLUDE_ASM stub counted separately —
the SAME function in BOTH `real` and `stubs`. Live case: resident.c:868-925 wraps a full
void func_800D00E4(s32){...} in #if 0 (its jtbl dossier) and re-declares the stub at :926.
Now the block is skipped entirely: dead code is neither matched nor stubbed.
- THE len()-SUM (the dual defect): `placed` was a set union, so it caught a function in NO bucket
— but `matchable` SUMMED len()s, so a function in TWO buckets counted twice and nothing
complained. matchable/byteident are now SET unions, plus a new OVER-coverage assertion that
fails loudly if any fn lands in multiple buckets. R32 means both directions: nothing missing,
nothing double-counted.
- NEGATIVE CONTROL (the fix must change an answer the old tool gave):
resident REAL 123 -> 122 | matchable 146 -> 145 | 85.62% -> 85.52% | func_800D00E4 no longer
double-counted. FLEET instr 68.9% UNCHANGED (no #if 0 in the overlays) — the fix is scoped.
- FINDING (logged for T7's audit-binaries, does NOT block the flag-plant): the two INDEPENDENT
oracles now agree exactly at 144 — corpus (21 stubs + 123 matched, derived from the tree) and
sig_image (the 2nd oracle) — with EMPTY set difference both ways. progress.py still reports 145
because it counts func_800CEDFC and func_800D33E0, which are DEFINED in resident.c but absent
from sig_image. 0x800CEDFC is the resident's vram base +4 (the first function, code starts at
file offset 0x4 after the leading data word), yet make audit-corpus reports 0 PHANTOM +
0 TRUNCATED. Either sig_image has a boundary blind spot or those defs are not image functions.
progress.py's text-scanning classify() is exactly the re-parsing R33 says should be DERIVED from
corpus instead — a real refactor, logged not rushed.
The flag-plant claim is unaffected: it rests on corpus.stubs('resident') == 21 (tree-derived,
verified 5 ways), not on the contested denominator.
|
||
|
Drew T
|
fda9eebb42 |
fix(phase-28 T4): wire all 4 SC07 overlays (6174/6457, 95.6%) + REPAIR the registry I destroyed
Completes T4 and corrects two defects I introduced, both landed in commit:0649. - WIRED: 006 1543/1614 · 007 1544/1615 · 010 1544/1614 · 011 1543/1614 = 6174/6457 = 95.6%, ~0 agent tokens. Stubs/overlay ~2400 -> 831/984/898/825. Fleet instr 67.0 -> 68.9%, fn-count 82.16 -> 83.94%. dedup-check 1840 validated / 0 failed; groups now read "138 members [138 binaries]" (was 134); C1 coverage 227211 -> 233385 = exactly +6174. R22 make clean && extract-all && check-all -> 140 passed, 0 failed of 140 at every stage. - FIX #1 — I DESTROYED THE REGISTRY'S DOCUMENTATION, AND EVERY GATE CALLED IT GREEN (H5). The first cut wrote config/dedup.us.yaml with yaml.safe_dump, round-tripping the whole file: 47 comment lines -> 0 (including the curated Phase-11 header explaining WHY the share is source-level) and 1832 `vram: 0x80162FF4` -> `vram: 2148937716` (PyYAML parses YAML-1.1 hex to int; dumps int as decimal). 25,948 lines rewritten. It passed dedup-check 1840/0 AND check-all 140/140 because _addr() accepts both forms: THE DATA WAS CORRECT AND THE DOCUMENT WAS RUINED. Fixed forward (R6, no history rewrite): restored from commit:0649~1 and re-applied the 6174 memberships via a surgical text edit (add_members_surgical). Verified: 1545 insertions / 1545 deletions, 0 non-`binaries:` lines changed, 47 comments + 1908 hex fields intact, and the rebuilt fleet is byte-identical to the destructive version (140/140). THE LESSON: every oracle this project owns measures BYTES, so a formatting-destructive write is invisible to all of them by construction. R34 says the byte-gate is a null COVERAGE oracle; this is the same hole one layer out — it is a null DOCUMENT oracle too. - FIX #2 — I MIS-REPORTED THE DIFFs, TWICE (R14). (a) commit:0649 claims ov_SC07_006's 71 non-banks were "ALL PLUMBING, ZERO DIFF". FALSE — I read head -6 of the classified file and generalized. It has the same 4 DIFFs as the others. (b) I then built the jr guard assuming those 4 were the §53 jr class BECAUSE ov_SC01_077 hosts them in _jr_8017A4AC.c / _jr_80182268.c. has_mid_jr is FALSE for all four (33-52 ins, no jump table): they merely live in a carved jr-REGION split, which sweeps in every function in its address range. HOSTING FILE != FUNCTION CLASS. The guard is KEPT (preventive, §53-correct, currently skips 0 — no jr fn is in the extendable set) with its docstring corrected to record what it is NOT. The 12 DIFFs (0.19%) are UNDIAGNOSED and logged, correctly left as stubs by the gate — not dressed in a story. - The 283 non-banks: 271 PLUMBING (the loose-typing conflict class + the whale, whose body lives in src/shared/func_80144B9C.h so no DEFINE macro exists to expand) + 12 DIFF. Existing tools cover the plumbing (cast_call_sites / canon_sig_reconcile / reconcile_tu). |
||
|
Drew T
|
c0486fe5f8 |
feat(phase-28 T4): dedup_extend — wire newly-onboarded binaries in; ov_SC07_006 1543/1614 (95.6%)
The 4 SC07 overlays P27 onboarded were byte-clean but NOT citizens: their .c included only
common.h (never ../shared/engine_core.h), so no shared body could reach them, and they
appeared in ZERO dedup groups (1689 groups read "134 binaries", never 138). Each sat at ~80
matched / ~2400 stubs while its siblings were ~2150 matched.
- NEW tools/dedup_extend.py — the missing mode. dedup_propagate is built for CRACK -> AUTHOR
MACRO -> INSTANTIATE: --auto-from scans INLINE DEFS (planned only 11 here; the ~1600 shared
bodies are ALREADY DEFINE_func_* macros in engine_core.h) and --addr dies "no source overlay
has it matched" because no overlay holds an inline def. Extending an existing MACRO-BACKED
group to a newly-onboarded binary is a different operation and nothing implemented it.
- SAFETY (explicit — this feeds the byte-gate): h_exact is the SHA1 of RAW INSTRUCTION BYTES, so
two instances sharing one are identical INCLUDING their jal/lui/%lo reloc immediates — same
callees, same data addresses, same symbols. The body that compiles byte-identically at one
member does so at the other with NO remap. (Exactly why dup_report calls h_exact "guaranteed
byte-match" and h_norm "candidate-only".) A bug here can only FAIL TO BANK, never falsely bank.
- REUSE, DON'T REBUILD (R33): owns only the set computation + the registry edit. The splice and
the gate are harvest_verify verbatim (it already derives each stub's home TU from the corpus
oracle, chunks + bisects, reverts on failure). h_exact members are byte-identical by
construction -> the happy path is ~1 build per binary, not one per function.
- RESULT ov_SC07_006: 1543 / 1614 banked = 95.6%, ~0 agent tokens. Stubs 2374 -> 831.
The 71 non-banks are ALL PLUMBING, ZERO DIFF, in two named classes with existing tools:
* func_80144B9C "undefined reference" — the whale's body lives in src/shared/func_80144B9C.h
(the -O0 shared header), not engine_core.h, so no DEFINE macro exists to expand.
* "conflicting types for D_800A5E60 / func_8012C750 / func_8012C0EC" — the loose-typing
conflict class (cast_call_sites / canon_sig_reconcile / reconcile_tu already exist for it).
- GATES: R22 make clean && extract-all && check-all -> 140 passed, 0 failed of 140, 0 FAIL lines.
dedup-check 1840 validated / 0 failed; groups now read "135 members [135 binaries]" (was 134);
C1 coverage 227211 -> 228754 = exactly +1543. The second oracle accepts the extension.
- Mechanism had been proven by hand first (probe-before-investing): +include + ONE stub ->
DEFINE_func_80128158() -> ov_SC07_006 built 7ca772be BYTE-IDENTICAL, then reverted.
|
||
|
Drew T
|
4db79a2060 |
feat(phase-28 T1b): the B2 family swept — 102/115 banked (88.7%), fleet 67.0 -> 67.7% instr
The family the roadmap recorded as 0/8 ("~0%, structural families do not template" — the
number that rewrote P29's arithmetic to "(cores cracked) x (reach)") banks at 88.7% when
swept with the carve its own exemplar required. ~0 agent tokens.
- SWEEP: jtbl_family_bank.py over the remaining 107 members ->
{'BANKED': 94, 'gate-fail': 7, 'remap-refuse': 6}. Family total 8 (T1) + 94 = 102/115.
R22: make clean && extract-all && check-all -> 140 passed, 0 failed of 140, 0 FAIL lines.
- FLEET (measured, make report): instr-weighted 67.0 -> 67.7% (+0.7pp, +97,104 ins);
distinct-code 47.8 -> 49.4% (+1.6pp); fn-count 82.16 -> 82.19%. 102 x 952 = 97,104 =
the exact measured instruction delta — the arithmetic reconciles to the byte.
- THE 13-MEMBER TAIL is the predicted shape, and both halves are data for T3:
* 6 remap-refuse = EXACTLY the family's 6 IMM members (cls_counts PURE 109 / IMM 6).
imm_map_tier1 REFUSED rather than guessed: "unresolved immediates: [(512,
'asm-ambiguous')]" — 512 also occurs at a non-differing position, so a blind swap could
corrupt it. This is the concrete shape of T3's IMM stratum.
* 7 gate-fail = genuine byte-DIFFs, correctly rejected. Verified to leave NO residue
(all 7: split_file=none, cfg_refs=0) — no false-bank risk.
- HYGIENE: the 7 "git checkout ... did not match any file" errors are benign (revert of a
never-tracked path). Verified 0 untracked splits belong to a non-banked member; 91 new
splits + 3 banked into existing splits = 94.
- SCOPE (P9, unchanged): still n=1 family, and jr is the rarest class (3/163 matched-exemplar
families). This demonstrates the mechanism at family scale; it does NOT give a rate for the
PURE/IMM mass (98% of the population). T3 measures the swing number.
|
||
|
Drew T
|
427baba3bf |
feat(phase-27 T10): completion dashboard (main in the weighted metric) + the resident second oracle
The metrics contract (roadmap §1) wants all three metrics WITH main in the denominators, and the
second, independent boundary oracle (R34) extended beyond the overlays. Both had landmines.
10a — main into the weighted metric, safely:
- weighted_metrics off the func_-only src_stubs regex onto corpus.stubs (R33). THE LANDMINE IS
REAL: src_stubs("SLUS_007.26") globs src/SLUS_007.26/*.c -> 0 files -> every row "matched" ->
main 100% + fleet % silently inflates. Routing through corpus.stubs is a PROVEN 0.000pp no-op on
the existing fleet (overlays are all func_) and closes the curated-name leak.
- a SEPARATE "MAIN game-code weighted" line (0.7%): main's Ghidra sig excludes the LINKED PsyQ
objects (Ghidra never analysed them), which is exactly right for a game-code metric (LINKED is
complete, counted in fn-count). Reported un-folded and caveated (month-stale sig, PROVISIONAL) —
folding a stale/incomplete value into the decomp.dev headline would mislead the flip checkpoint.
10b — the resident second oracle:
- make sig-resident: sig_image on the resident flat blob (byte-derived, not Ghidra). corpus.
sig_is_independent now covers resident -> audit-corpus checks its boundaries too. Probed clean
BEFORE wiring (144 fns, all 21 stubs present, 0 phantom), verified 0 phantom + 0 truncated.
- sig-overlays now derives its payload list from config/overlays.mk, not a 0.4.dec glob that
silently dropped the 4 SC07 index-1 overlays (the audit's own silent-skip class). tools-health
regenerates sig-overlays + sig-resident first so the audit never crashes on an absent sig.
10c — main's second oracle: docs/second-oracle.md. sig_image can't sign the PS-X EXE yet (0x800
header offset, interleaved data/linked islands, one text range); seeding from splat would destroy
independence for the PHANTOM class specifically. Honest deferral + scoped design, not a fake oracle.
- docs/progress.fleet.md regenerated: 140 binaries · fn-count 82.16% · instr-weighted 67.0%
(the honest post-T7 drop from 68.9%) · distinct 47.8% · MAIN game-code 0.7% (separate).
- SETUP §6.3 updated (R21).
|
||
|
Drew T
|
ed09ee749f |
feat(phase-26 T7): §52 sibling wave 2 — 3 more cores banked ×134 (402 instances)
Second cheap-Opus §52 wave over the close=0 regalloc cluster (armed with §52a):
- BANKED ×134: func_801379FC (97), func_801497A8 (47), func_801495C4 (34) —
3 exemplars + 399 members = 402 function-instances, 0 gate failures.
- 2 whole-binary-near (func_8012E138, func_8012F40C — match_one MATCH, A10 gap),
1 new wall (func_8012B4B8 — symbol-address-base wins-low-needs-high, a 3rd class).
- §52b: new verified de-pin levers (per-loop pseudos for register role-swap; the
RC-7 second-set dial to defeat rematerialization; value-barriers dissolve the
CSE-stack-address-common wall) + the new wall class + the match_one→whole-binary
gap-at-scale finding.
- TOOL FIX: family_sweep §42e pin-guard was a FALSE POSITIVE — it matched
'__asm__("$N")' inside COMMENTS that document a REMOVED pin (recovered
func_801495C4's 133 members). Now strips comments before the pin check.
- R22 clean-fleet 136/136 BYTE-IDENTICAL; dedup 1840/0.
- Wave 1+2 combined: 5 pin-free cracks -> 670 instances, from the walled flagship's idiom.
|
||
|
Drew T
|
06e43873e9 |
feat(phase-26 T7): §52 regalloc sibling wave — 2 cores banked ×134 (268 instances)
The Fable5 walker-family idiom (§52, from the func_80178004 wall) applied by a 6-agent cheap-Opus wave over the regalloc-order reach-134 cluster: - BANKED ×134: func_80171FFC (40 ins), func_801775E0 (67 ins) — 2 exemplars + 266 members = 268 function-instances, 0 gate failures (family_sweep byte-gate). - 4 precisely-characterized walls (P9), each yielding a byte-verified lever: func_80167714 (whole-binary near), func_80177AD4 (non-coalescing delay-slot copy), func_80169228 (NEW caller-saved priority-first-fit wall), func_80131A34 (save-order/ load-hoist tension; new const-unchanging-load lever). - §52a: the wave's new banking levers (pass-real-args/RC-10, store-base-both-arms, copy-chain-direction, pp-decl-schedule, const-unchanging-load/RC-3) + the two new intrinsic-wall classes. Fable5 DISCOVERS, cheap-Opus APPLIES. - fn-count 84.27→84.35% (+268), instr-weighted 68.6→68.7% (+14,338 ins), distinct +2. - R22 clean-fleet 136/136 BYTE-IDENTICAL; dedup 1840/0. |
||
|
Drew T
|
3509acf4b7 |
feat(phase-26a): A9b — func_8017A4AC banked ×134 (536-ins giant, wall re-test payoff)
The A10 re-test payoff. func_8017A4AC (536 ins, reach-134) — "blocked on plumbing" since session 8 — banks now that the audit repaired the recover path (A3d reconcile_tu / A3e gate). jtbl_family_bank --raw swept all 133 siblings (per-sibling isolate → jtbl carve → remap_hseq + canon_sig_reconcile → whole-binary gate): 133/133 BANKED, 0 failed. 0 still-stub overlays. R22 CLEAN-FLEET (make clean + extract-all + check-all): 136 passed, 0 failed of 136. dedup-check 1840/0 (jtbl sweep banks are per-overlay src, not registry). DELTA: instr-weighted 68.1% -> 68.6% (+0.5%, ~71,824 shipped .text instructions) distinct-code 48.0% -> 49.2% (+1.2% — the siblings are per-location byte-variants) The audit thesis, demonstrated: a giant "wall" that stood for many phases was our TOOLING (the recover path could not resolve its struct/fn-ptr conflicts), not an intrinsic compiler residual. Once the oracle was fixed, the wall dissolved and banked ×134. |
||
|
Drew T
|
6139800e6d |
feat(phase-26a): A3h — family_sweep --hseq re-harvest, +2,675 member banks (post-audit-fix)
Re-ran the mechanical h_seq family sweep after the audit tool-fixes (A3–A8) + the 14 new
Bucket-P exemplars regenerated the manifest. family_sweep --hseq --band all templated each
matched exemplar's still-stubbed members per sibling (reloc/immediate remap) and byte-gated:
BANKED 2,675 member-matches / 9,698 failed (22% — the hard residual; the easy bands were
harvested in earlier sessions, so what's left fails the whole-binary gate on jtbl / type /
plumbing, correctly rejected by G3/P9). Skipped: 3,319 pinned-exemplar (×1-only, cc1-crash
siblings, §42e), 212 unresolved-immediates, 137 STRUCT.
223 families with a matched ov077 exemplar contributed. R22 CLEAN-FLEET (make clean +
extract-all + check-all): 136 passed, 0 failed of 136. dedup-check 1840/0 (unchanged — hseq
banks are per-overlay src, not registry).
DELTA:
instr-weighted 67.4% -> 68.1% (+0.7%)
distinct-code 46.8% -> 48.0% (+1.2%, +1,435 unique fns — templated members are distinct
byte-variants, so this moves BOTH metrics unlike propagation)
876 overlay .c files + progress.fleet.md + family-hseq.md
Only 33 of the 567 substantial frontier families have a matched exemplar — the rest need
CRACKING (Phase-26 Task 7, Fable5). Remaining standing lead: Bucket X (~905 absent-from-ov077
byte-exact drafts) via per-overlay gate — next.
|
||
|
Drew T
|
2f38e31e76 |
feat(phase-26a): A3h — propagate 14 fleet-wide byte-exact stubs ×134 (Bucket P)
The standing-lead harvest (A3f/A3g continuation), measured precisely first (R14). Of the
~1,060 still-open byte-exact functions in the backlog:
- Bucket G (67 open in ov_SC01_077): re-gated through the A3e-fixed gate_stage
--no-propagate -> 0 banked. HONEST: A3f already took the bankable 33; the residual is
the known hard classes (jtbl-rodata / register-pins / struct-collision) + stale backlog
rows whose LATEST state is a WAVE mismatch. Correct G3/P9 rejection.
- Bucket P (88 matched in ov077, open in siblings): the clean lead. dedup_propagate --addr
(A3g primitive) skipped 70 as h_exact reach<2 (per-location byte VARIANTS -> family_sweep
territory, not plain propagation) and propagated the 14 genuine PURE fleet families:
5 top (func_80129C40/8012A6D0/80130A18/80131D68/80136DFC) + 9 more; 2 stragglers
dropped all-or-nothing (0x80173A60, 0x8014C568 -> --recover candidates).
Each propagated x~133 (dedup_propagate internal gate: 134 overlays byte-identical).
R22 CLEAN-FLEET (make clean + extract-all + check-all): 136 passed, 0 failed of 136.
dedup-check: 1826 -> 1840 validated, 0 failed | C1 227211/227211.
DELTA:
instr-weighted 66.8% -> 67.4% (+~1,862 member instantiations shipped from C)
distinct-code 46.8% -> 46.8% (flat: propagation adds MEMBERS, not new distinct code)
673 files (671 overlay .c instantiations + engine_core.h) + dedup.us.yaml + progress.fleet.md
Remaining standing lead: the ~72 variant Bucket-P + ~905 Bucket-X (absent from ov077) fns,
all latest-row closeness==0 -> route through family_sweep --hseq (per-sibling remap), next.
|
||
|
Drew T
|
60e26e07f8 |
feat(phase-26a): A3g — propagate the 3 fleet-wide banks ×134 (bounded, gated, R22-clean)
The 3 of A3f's 33 banks that are shared fleet-wide, stamped across all 134 overlays. Done the way
the earlier run should have been: TARGETED (--addr, not --auto-from), dry-run-sized first
(3 functions × 134 members = ~400 gates, not an unbounded fleet sweep), on a clean tree at HEAD.
func_80130650 (31 ins) · func_80149450 (13 ins) · func_80174684 (9 ins) — each ×134.
dedup_propagate internal gate : 134 overlays byte-identical, 3 groups registered
R22 CLEAN-FLEET (the real proof, not the tool's incremental check that lied during the crash):
make clean + extract-all + check-all -> 136 passed, 0 failed of 136
dedup-check: 1823 -> 1826 validated, 0 failed | C1 coverage 225335/225335
DELTA (reconciles exactly):
functions byte-identical 284,559 -> 284,958 (+399 = 3 fns × 133 other overlays)
instr-weighted 66.7% -> 66.8% (+13,167 shipped .text instructions)
distinct-code 46.8% -> 46.8% (flat: propagation adds MEMBERS, not new distinct
code — the 3 bodies were counted at A3f)
403 src files (3 ×134 instantiations + engine_core.h) + config/dedup.us.yaml
The other 30 of A3f's 33 are overlay-unique (×1) and need no propagation. The larger prize remains
the ~310 byte-exact stubs in the OTHER overlays (A3e), not yet attempted.
|
||
|
Drew T
|
8ffcf9646e |
feat(phase-26a): A3f — 33 functions banked that the project had written off as compiler walls
The payoff of A3e, byte-verified. These 33 sat in the backlog at closeness==0 -- match_one said
their bodies were BYTE-EXACT -- and the whole-binary gate rejected them, so they were logged as
`near`/`failed`, i.e. AS MATCHING PROBLEMS, and filed as intrinsic compiler residuals.
They were not hard. They were UNREACHABLE. gate_stage passed `--src src/<ov>/<ov>.c`
unconditionally, which restricts the byte-gate to ONE translation unit -- and every one of these
functions has its stub in a SPLIT TU. Look at where they landed:
src/ov_SC01_077/ov_SC01_077_a.c
src/ov_SC01_077/ov_SC01_077_after.c
src/ov_SC01_077/ov_SC01_077_jr_8012ACE0.c
src/ov_SC01_077/ov_SC01_077_jr_8015AE2C.c
src/ov_SC01_077/ov_SC01_077_jr_8016AB6C.c
src/ov_SC01_077/ov_SC01_077_jr_801734BC.c
src/ov_SC01_077/ov_SC01_077_jr_80178D40.c
src/ov_SC01_077/ov_SC01_077_jr_80182268.c
8 files. SEVEN of them are _jr_/_a/_after carves. NOT ONE is the main .c -- the only file the gate
was ever allowed to look at.
gate: 63 drafts -> banked 33, near 30, FAILED 0 (--no-propagate; the gate and the propagation
are different jobs, and letting an experiment tow an unbounded fleet-wide propagation is
what broke the tree an hour ago)
R22 CLEAN-FLEET: make clean + extract-all + check-all -> 136 passed, 0 failed of 136
dedup-check: 1823 validated, 0 failed | C1 coverage 224933/224933
METRICS, ×1, HONESTLY (no propagation yet -- the multiplier is still ahead):
functions byte-identical 284,526 -> 284,559 (+33)
instructions 8,470,381 -> 8,471,912 (+1,531)
fn-count % 82.79% -> 82.80%
instr-weighted % 66.7% -> 66.7% (flat: ×1 banks do not move the headline)
WHAT THIS MEASURES, beyond the 33: the backlog holds 1,588 entries at closeness==0. 1,215 have been
banked since by other paths. 373 ARE STILL OPEN STUBS WITH BYTE-EXACT BODIES. 63 of them were in
ov_SC01_077 and 33 banked -- a 52% rate on functions the ledger calls unrecoverable. The other 310
are spread across the remaining overlays: same class, same fix, not yet attempted.
Cookbook §51g LAW 11 -- a fix is not landed until its caller stops overriding it. And the reason
this hid for 26 phases, which belongs in the posterity doc: A TOOL THAT CANNOT BANK A FUNCTION IS
INDISTINGUISHABLE, IN EVERY LOG THIS PROJECT KEEPS, FROM A FUNCTION THAT CANNOT BE BANKED.
|
||
|
Drew T
|
82d79e7a32 |
fix(phase-26a): A6/A7 — the family engine could not see half its corpus; 17 fns banked x134 free
R22: check-all 136 PASSED / 0 FAILED. dedup-check 1823 validated / 0 failed (C1 coverage 224,933/224,933).
Fleet instr-weighted 66.5% -> 66.7%.
=== dedup_propagate: it was blind to HALF the corpus ===
overlay_files() used a hardcoded suffix allowlist ("_a","_o0","_o0b","_after") that predated the
Phase-26 jr carves -> 404 of the fleet's 811 overlay .c. The 407-file gap held 36,135 INCLUDE_ASM stubs
and ~32,000 inline defs, and overlay_files gates ALL of dedup_propagate (source_text / find_site /
apply_plan / struct_check / reconcile_caller_extern). Now a GLOB — never an allowlist, because the NEXT
split family would re-open it. The asm_subdir is always the file stem, an invariant the old four entries
already satisfied.
find_site's def-detector required the signature line to END in ')' and the next non-blank line to START
with '{'. It therefore silently dropped THREE shapes: K&R definitions (`s32 f(arg0)` / `s32 arg0;` / `{`),
multi-line signatures, and single-line bodies. K&R is the project's house style for exactly the biggest,
highest-reach functions — func_8015AE2C (562 ins), func_80166994, func_80133CD4, func_8015A3C8 — and they
live in the _jr_* files overlay_files could not even open. Fixing either alone would have been useless:
the glob exposes the files, and find_site would still drop their biggest prizes. Both fixed together.
* The signature's closing paren is now found by a real paren-walk, not line.count() or split(')')[-1]:
a single-line body containing a call (`void f(int a){ g(a); }`) has balanced parens of its own, so
both shortcuts land on the WRONG paren and then misread the body's ';' as a prototype terminator.
* AGREEMENT ASSERTION (the audit's): find_site vs family_remap.extract_unit -> 701 agree / 0 disagree.
Negative controls hold (a prototype+call is rejected; a 1-line body with a call is a def).
=== THE HARVEST (free work, byte-gated) ===
--auto-from ov_SC01_077 now nominates what it could never see: 20 planned, 17 propagated x134, 3 dropped
as cross-overlay stragglers. 134 overlays rebuilt BYTE-IDENTICAL; 17 new dedup groups.
Includes ALL FOUR functions A1 caught the registry lying about (func_80128ED8 / 8012C098 / 8012C0EC /
8012C750): 0 stubs remaining, real shared macros. THE LOOP CLOSES — A1 found the lie, and THIS is the
bug that had made it true (3 of the 4 are defined in ov_SC01_077_jr_8012ACE0.c, which the allowlist could
not open, so the propagation never ran and dedup_integrate greenlit the result).
=== family_remap: 96 PHANTOM exemplars -> 0 ===
extract_unit globbed only src/<ov>/<ov>*.c, so a function matched via a SHARED body had no source form
and read as NOT MATCHED. 93-96 of 218 h_seq "matched" exemplars were phantom, carrying 2,157 candidate
members of which 1,834 are still-stubbed, PURE/IMM-clean, symbol_map-clean and unpinned — staged and
gated today, dropped before the first build then. It is now TOTAL over BOTH shared-body mechanisms:
(1) the DEFINE_func_<ADDR>() macro — reconstructed as the exact INVERSE of dedup_propagate.make_macro
(derived from the generator, not re-guessed from the text);
(2) a DIRECT definition in a shared header, #included per overlay — the whale (func_80144B9C, 770 ins,
-O0), which the registry explicitly records as "NOT a DEFINE_ macro".
CENSUS: 216 matched exemplars, 216 real, 0 PHANTOM.
symbol_map named the symbol by HOW IT WAS LOADED, not by WHAT IT IS: reloc_targets labels every lui/%lo
pair "data", and a FUNCTION's address taken via lui/%lo (an address-taken callback) is exactly that shape
(splat's own .s: %lo(func_8017E1D4), 7 occurrences). The map got a D_<ADDR> key while the C writes
func_<ADDR>, so the word-bounded substitution matched NOTHING and silently no-op'd — the sibling kept the
EXEMPLAR's function pointer and the loss was booked as a BYTE failure, indistinguishable from a compiler
wall. Now emits both keys (addresses are unique; the pass is simultaneous, so the extra key is free).
gather_externs was line-oriented, so a WRAPPED comma extern was invisible in both directions (the first
line has no ';', the continuation has no `extern`). ov_SC01_077.c:271-272 declares NINE symbols that way,
and the exemplar referencing them (func_8013D178) is a 133-member family — every sibling was staged with
NO declaration, failed to compile, and bisect-stormed its whole gate group. Now statement-oriented, and
an unresolved symbol is REPORTED, never silently dropped.
=== family_sweep.stub_map / build_engine_types ===
stub_map: func_-only -> a curated-name stub read as "already matched" -> phantom exemplar. Now corpus-derived.
build_engine_types hard-exited on 1,070 of 1,470 type-bearing overlay .c (73%; the audit measured 573/709
= 81% on its narrower set) because 1,929 TAGGED-struct typedefs tripped a guard whose own comment asserts
"our source has only ANONYMOUS-struct typedefs" — true in Phase 20, false since the harvest agents started
writing tagged structs. inject_capped_externs routes every type-bearing body HERE as the type-heavy tail's
ONLY sanctioned unblocker, so the tail's unblocker could not run on the corpus the tail lives in.
A contained def (the typedef's span encloses the body) is liftable — it just must not be counted twice;
only a PARTIAL overlap is malformed. Verified on a file that used to hard-exit: 5 tagged typedefs folded +
forward-declared, 46 types written, exit 0.
** AND THE SHARPEST LESSON IN THE AUDIT: this one was never silent. It printed "[overlap] ... handle
manually" every single time. But the message reads like a rare edge case rather than a four-fifths
coverage failure, so nobody ever COUNTED it. A loud failure that nobody counts is exactly as
invisible as a silent one. R32 must be "assert your coverage", not merely "fail loud". **
R14 self-catches, recorded because I hit both while fixing them: my first shared-header scan read a macro
body's `extern void f(void); \` as a DEFINITION (the trailing continuation means the line does not end in
';', so the decl guard never fired) — the exact bug fixed at commit:0552, reintroduced by me and caught only
because the whale resolved from the WRONG file. Column-0 anchoring fixes it by construction. And my
phantom census returned 0/0 twice because I guessed the manifest schema instead of reading it.
|
||
|
Drew T
|
bb65d36341 |
fix(phase-26a): A1 — dedup_integrate was a gate that could print a FALSE GREEN
The audit's priority #1: a fail-closed byte-honesty validator whose silent skips nothing downstream can catch. Three false-green paths, all measured, all now fail-closed with negative controls. R33 FIRST (derive, don't re-derive). The registry makes two claims; the tool only ever checked one, and mis-described that one: C1 EQUIVALENCE ("these vrams hold the same code in the ORIGINAL") — checked against the sigs, which sign the ORIGINAL bytes. KEPT. But the docstring claimed it also caught SOURCE drift: it cannot. A sig is a property of the ROM, immutable w.r.t. src/. Source drift is caught by the BUILD. Docstring corrected (P9). C2 BANK ("matched once in the source header, instantiated at every member") — NEVER CHECKED. Now DERIVED from the build invariant: INCLUDE_ASM pastes the ORIGINAL asm, so a member NOT wrapped in it is byte-exact, and one that IS wrapped is not banked — whatever the registry says. C2a: the group's macro token must occur in its source file. C2b: no member may still be an INCLUDE_ASM stub. THE THREE FALSE GREENS 1. 1808 groups claimed a DEFINE_func_* macro; only 1801 exist. The 7 ghosts printed [ OK ] — hiding 532 member-instances / 22,344 instructions of REAL, UNBANKED work (4 fns matched in ov_SC01_077, still INCLUDE_ASM in the other 133 overlays). 2. An absent .run/sig.<bin>.jsonl degraded to "0 validated, 0 failed" and EXIT 0. On a fresh clone the gate validated NOTHING and passed. Now fails; --allow-unsigned is the escape. 3. The bank claim was never checked at all. THE CAUSAL CHAIN (the audit's thesis in one example). 3 of the 4 hidden fns are defined in ov_SC01_077_jr_8012ACE0.c — a _jr_* split file. dedup_propagate.overlay_files allowlists only ("_a","_o0","_o0b","_after"), so the propagator could not SEE them; the group was registered anyway; dedup_integrate greenlit the lie. TWO silent-skip bugs compounding: one created the hole, the other hid it. Harvest fuel -> .run/audit/a1_harvest_fuel.json, banked in A5. BLAST RADIUS, MEASURED NOT PREDICTED (R14). Headline metrics UNCHANGED to the decimal (instr-weighted 66.5%, distinct-code 46.8%) — weighted_metrics() derives from the invariant and was structurally immune to the lying registry. FLEET REAL substantive unchanged (282,466): progress.py had already been taught to distrust it (commit:0574). Only dedup_integrate still believed it. A null result that CONFIRMS R33: the tool that refused to re-derive was the one that was right. - registry repaired: 1813 -> 1806 groups (7 ghosts removed; instances 223,725 -> 222,787) - make report GREEN end-to-end: 1806 validated, 0 failed | C1 coverage 222,787/222,787 signed - negative controls: stubbed member -> exit 1; missing sig -> exit 1; --allow-unsigned -> exit 0 - report-only tool: no compiled artifact depends on it, so no R22 clean-fleet is owed here |
||
|
Drew T
|
6e8c459ade |
docs(phase-26): SESSION-8 CLOSE — checkpoint for a fresh session; the tooling-integrity audit gates what comes next
RESULTS. Fleet instr-weighted 63.0 -> 66.5%, distinct-code 39.1 -> 46.8%, fn-count 82.61%.
FINAL R22: make clean + extract-all + check-all -> 136/136 BYTE-IDENTICAL, 0 coverage defects.
dedup 1813/0. 0 NON_MATCHING (G4). 31 commits.
13 CORES CRACKED incl. the four heaviest functions in the game (952/890/562/536 ins). The 12-agent
Ultracode wave returned 11/12 first-pass MATCH, each adversarially verified (a skeptic re-ran match_one
+ the §8a jump-table check). Banked x134 this session: func_8015AE2C, func_80178D40, func_8015A3C8,
func_8013FFD8, func_8016AB6C, func_8015444C, func_801380E0 (+ func_8017BEBC x1).
THE TOOLKIT CROSSED A LINE — three ZERO-BYTE DIALS now cover the three passes that produce essentially
every "irreducible" residual, each with a diagnostic signature a cheap agent recognises on sight:
registers rotated -> global.c allocno priority -> §47 slider / §48-A pricing dials
two insns swapped, SAME regs -> sched.c rank_for_schedule LUID tiebreak -> §49 LUID dial
structure right, count wrong -> loop peel / cross-jump -> §46 / §48-D
That is why 9/12 fell first-pass to ORDINARY agents. Fable5 DISCOVERS a class; everyone else APPLIES it.
New: §46 §47 §48(+A4) §49 §50. Read §50-B before using §48-A1/A4 — it BOUNDS them (the "cross_jump
refunds the bytes" claim is FALSE for a 1-insn tail reached by two jumps; jump.c:1993 minimum=2).
DREW'S DIRECTIVE (binding): the TOOLING-INTEGRITY AUDIT comes BEFORE any further matching work, and is
NOT part of this phase. First act of the fresh session is a Tier-1 phase-boundary call (close Phase 26
early, or run the audit as an inserted phase — Drew decides).
WHY: seven silent-skip tool bugs in one session, and they are a STRUCTURAL blind spot — a scanner
extracts N items, the truth is M > N, and nobody ever compared N to M. The byte-gate is a perfect
CORRECTNESS oracle and a NULL COVERAGE oracle: it has been green since Phase 5 at 0% decompiled (
INCLUDE_ASM pastes the ORIGINAL asm), so a green gate is compatible with ANY decomp %. One 10% hole in
the callee oracle made NINE byte-exact functions look like an intrinsic compiler wall. The real question
the audit answers: how many walls we have already "byte-proven" across 26 phases were lookup misses
wearing a wall's clothes? (The def-side loose-typing wall, the 159 arity conflicts, the type-heavy tail
were ALL diagnosed on top of that hole.) Audit scope so far is 19 of 82 tools (23%), by risk — NOT
comprehensive; dedup_integrate.py is unaudited and can print a FALSE GREEN.
RULE CANDIDATES (P10, Drew ratifies at PhaseEnd):
R32 Coverage assertion — a corpus scanner must assert its own coverage and fail loud on unparsed input.
R33 Derive, don't re-derive — where a proven invariant answers the question, derive from it. The best
audit outcome is not a fixed regex; it is a DELETED scanner.
SELF-CORRECTION ON THE RECORD (P9/R14): I told Drew the headline numbers under-reported by ~190k
instructions. WRONG. weighted_metrics() never calls classify(), so it was structurally immune; the
published numbers were correct all along. I verified the DEFECT but not its BLAST RADIUS. A null result
against a strong prediction is a refutation — chase it.
|
||
|
Drew T
|
cc08601ae7 |
feat(phase-26): func_80178D40 swept ×134 — the heaviest core in the game, fleet-wide
- 132/132 siblings banked (0 failures) via jtbl_family_bank --raw + the lazy-isolation chain. Each sibling: isolate -> jtbl carve -> remap from the raw crack -> stage ladder (raw -> scoped §8d -> recovered -> reconciled) -> WHOLE-BINARY byte-gate. - R22 clean-fleet 136/136 BYTE-IDENTICAL from `make clean`; 0 NON_MATCHING (G4). - METRICS: instr-weighted 63.8 -> 64.7%; distinct-code 40.7 -> 42.8% (+2.1 points from ONE core — 890 ins x 133 overlays = ~118K instructions of unique engine code); fn-count 82.43%. - tools/bank_exemplar.py promoted from scratch: bank a cracked EXEMPLAR ×1 through the same stage ladder jtbl_family_bank uses for siblings (carve/lazy-isolate -> raw/scoped/recovered/reconciled -> whole-binary gate). The exemplar path was previously hand-run each time. |
||
|
Drew T
|
3a67dd609f |
feat(phase-26): func_8017BEBC (952 ins, ×113) CLOSED + banked ×1 — the §47 live-length slider
The largest unmatched core in the game, walled at close=2 for the permuter (25 min, no close) and
queued for a gdb-on-cc1 read. Closed WITHOUT gdb — the RTL dumps were the oracle:
- THE TIE, byte-measured (.lreg): &g.sz1 pseudo 228 refs 13 / live_length 783; &g.sz2 pseudo 230
refs 13 / 782 -> pri = int(390000/L) = 498 == 498, an exact int-truncation tie in global.c:594
allocno_compare. Tie-break = creation order -> allocation follows emission; the target needs them
to DIFFER (allocation sz2-first, emission sz1-first). The shipped operand-permutation workaround
could only pick one (close=2 vs close=10).
- THE FIX (§47): restore NATURAL operand order (emission correct) + ONE zero-byte
`__asm__ volatile ("")` placed BETWEEN two existing GTE volatile asms (no new cse/sched barrier —
one is already there) -> +1 static insn at global-alloc time -> L 784/783 -> pri 497 vs 498 ->
the tie SPLITS toward the shorter-lived (later-created) pseudo, which is ALWAYS the direction
"allocation != creation" requires. All 10 grants cascade; MATCH 952/952 first try; the slider
emits only #APP/#NO_APP (zero bytes). PIN-FREE, ×113 template-safe.
- BANKED ×1 in ov_SC01_000 through the WHOLE-BINARY gate (jr fn — match_one is not the arbiter,
§8a): lazy isolation -> new region ov_SC01_000_jr_8017BEBC + 9-piece jtbl interleave -> splice ->
BYTE-IDENTICAL. One TU-visible decl reconcile en route (D_800B9A02: declare the TU's `short`,
force the unsigned halfword at use `(*(u16*)&D_800B9A02)` — §8d sub-class (b)).
- R22 clean-fleet 136/136 BYTE-IDENTICAL; 0 NON_MATCHING (G4). The ×113 sibling sweep is IMM-class
(scattered addresses) -> Task-8 mechanical work via the imm engine.
- cookbook §47 (the slider method + the placement rule + the direction law); decision-log (R31).
|
||
|
Drew T
|
b8a525bb98 |
feat(phase-26): h_seq substantial-band re-sweep — 266 free member-matches (~0 agent tokens)
The extract_unit fix (commit:0552) revealed 82 families with a genuinely-matched exemplar and UNSWEPT siblings (~2.03M templatable bytes) — mostly exemplars cracked AFTER the session-2/3 mechanical band sweeps ran (the giant campaign + recent cores), so the sweep had simply never seen them. - re-ran `family_sweep --hseq --band substantial` on a regenerated manifest: 29 matched-exemplar families, 1046 member drafts staged, 1643 correctly skipped as pinned-exemplar. - BANKED 266 member-matches / 780 gate-rejected. The whole-binary byte-gate (G3/P9) arbitrated every one; R22 clean-fleet 136/136 BYTE-IDENTICAL from `make clean`. - metrics: instr-weighted 63.6 -> 63.8%; distinct-code 40.5 -> 40.7%; fn-count 82.39%. The 780 gate-rejections are the next lever: family_sweep's h_seq path does NOT yet carry the §8d `scoped` stage (it prepends carried data externs at FILE scope, the exact class that blocked the jr sweeps), so a large share are expected to be the same decl-environment conflict. Investigated next. |
||
|
Drew T
|
1ab9905368 |
feat(phase-26): §8d scope_data_externs — the ×133 sweep blocker fixed; func_8015AE2C banked ×134
- ROOT CAUSE (R14 — the session-7 diagnosis was half right): the isolated region builds [ OK ] WITHOUT the body, so §8b isolation was never implicated. `family_remap.gather_externs` prepends carried decls at FILE scope; D_801812A4 is a fn-ptr dispatch table the sibling declares FOUR incompatible ways at BLOCK scope inside its own later functions, so the carried file-scope decl ESTABLISHES A GLOBAL THE TU NEVER HAD and every later block-scope extern must now agree with it. Byte-proven asymmetry: BLOCK(int)->BLOCK(struct*)->FILE(void*) builds; FILE(void*)->BLOCK(int) errors. It was the ONLY hard error in the build — all 27 carried function externs were fine raw. - THE FIX (demote, don't reconcile): tools/scope_data_externs.py emits a carried D_ extern at BLOCK scope inside the function body when the TU has no file-scope decl of it above the insertion point. Byte-neutral (an extern emits no code; type + access opcodes unchanged) and never worse than raw, so it needs no oracle, no type comparator, no fn-ptr parser. Restores fidelity — the original declares these symbols at block scope in exactly this way. Wired into jtbl_family_bank as the `scoped` stage: raw -> scoped -> recovered -> reconciled (scoped is the base for the later stages). - reconcile_decls is the WRONG instrument for this class, twice: its oracle answers "what does the FLEET call this symbol" when the question is "what can THIS TU see", and its DATA_DECL_LINE_RE cannot parse `extern void (*D_x[])(void *);` — silently skipping the very symbols that were failing (the phase's third silent-skip bug, after find_site braces + overlay_files splits). - R17 TRIAGE RULE, first real test, held: `conflicting types` = the compiler REFUSED TO COMPILE = a C front-end diagnostic = our Python. Reading cse.c/global.c would have taught nothing. - RESULT: func_8015AE2C (562 ins, reach 134) swept 133/133 siblings, 0 failures. R22 clean-fleet 136/136 BYTE-IDENTICAL (534 changed src files); dedup-check 1813 validated / 0 failed; 0 NON_MATCHING (G4). instr-weighted 63.0 -> 63.6%; distinct-code 39.1 -> 40.5% (+256 unique fns / +79,957 ins) — one core, ~0 agent tokens. - knowledge captured during the producing session (R30/R31/R21): cookbook §8d, decision-log 2026-07-13 session 8, SETUP tool-inventory row; CURRENT_PHASE session-8 checkpoint. |
||
|
Drew T
|
5a08180617 |
feat(phase-26): §8 ×134 automation — func_8012ACE0 banked fleet-wide (133/133, R22 136/136)
- the jr-function ×134 harvest pipeline, proven end-to-end: per family sibling,
jtbl_carve (per-sibling jtbl-rodata carve, computed from THAT sibling's own jtbl
address — the fn is at the same vram across overlays but its jtbl floats) -> make
extract (auto ld_interleave) -> remap_hseq + canon_sig_reconcile -> whole-binary gate
- tools/jtbl_carve.py: per-overlay §8 carve generator (config data-tail split +
<ov>_JTBL_INTERLEAVE var)
- tools/jtbl_family_bank.py: the sibling sweep driver (idempotent, revert-on-fail, byte-gated)
- tools/family_remap.py: extract_unit now carries single-line typedefs (jr-function bodies
define local `typedef struct{} Foo_<addr>;` that must template with the body — the
propagation cap for these; additive, byte-gate-protected)
- func_8012ACE0 family: 133/133 siblings BANKED, 0 failures; R22 clean-fleet 136/136
byte-identical; 0 NON_MATCHING (G4)
- metrics: distinct-code 39.1% (50,698 unique fns), instr-weighted 63.0%
- opportunity (has_mid_jr families): 237 total (5,805 members) = 46 small mid/tiny
(771 members, same mechanical pipeline) + 191 substantial (the Fable5 cores, Task 7 paused)
- NEXT: R22 profiling/parallelization; then the other 45 small jr families
|
||
|
Drew T
|
095a611e75 |
feat(phase-26): §8 jtbl-rodata tooling — overlay PoC proven (func_8012ACE0, R22 136/136)
- overlay jr-functions can now bank as C: gcc switch jump tables form a .rodata island at the overlay TAIL; carve a matched fn's jtbl into a dotted [.rodata, <code-subseg>] subseg + ld_interleave (data->rodata->data sandwich) places it byte-exact. cookbook §8a + SETUP. - tools/ld_interleave.py: --section .<binary> param (derives the <binary>_TEXT/DATA/RODATA/ DATA2/BSS symbol prefix); default .main = the EXE, byte-identical (backward-compat proven) - Makefile + config/overlays.mk: <bin>_JTBL_INTERLEAVE hook + a $(strip)-guarded extract branch (gotcha caught: a trailing #comment on the := left whitespace -> non-empty -> the branch misfired on resident with the EXE defaults) - PoC: func_8012ACE0 (25-ins jr-fn in ov_SC01_077) reconciled (canon_sig_reconcile) + banked BYTE-IDENTICAL d19c9580 -- the first overlay jr-function matched through the C pipeline - R22 FULL-FLEET clean rebuild: 136 passed, 0 failed (main 143dbb89 unaffected by the ld_interleave change); 0 NON_MATCHING in any default build (G4) - P9 findings: func_80159C84/func_8015444C (the 2 carried Fable5 jr bodies) are rtu_match FALSE-matches (incomplete jtbls: 52B vs 56B -> never bank); the maspsx "hang" scare was a truncated experimental-file artifact (real pipeline builds in ~1s) - metrics: distinct-code 39.1% (50,572 unique fns), instr-weighted 62.9% - NEXT: the ×134 automation (generate the per-overlay carve + template the reconciled body) |
||
|
Drew T
|
025cc03f69 |
feat(phase-26): tiny-band mechanical harvest — 17,975 member-matches, R22 136/136
- family_sweep --hseq --band tiny: 180 tiny matched-exemplar families -> 17,975 member-matches BANKED / 5,617 gate-rejected (h_seq-collision false-templates — the whole-binary byte-gate refused every one; G3/P9), 266 overlay .c files touched (~76% bank rate) - R22 clean-fleet (make clean + extract-all-136 + check-all) -> 136/136, 0 failed; dedup-check 1813 validated / 0 failed; 0 NON_MATCHING in any default build (G4) - metrics: distinct-code 35.2 -> 39.1% (50,571/84,996 unique fns), instr-weighted 60.9 -> 62.9%. mechanical size-bands (substantial/mid/tiny) now harvested; remaining levers = the two harvest gaps (§8 jtbl-rodata, reconcile fn-ptr-extern) - regen docs/family-hseq.md + docs/progress.fleet.md; CURRENT_PHASE.md log |
||
|
Drew T
|
89162fd6b8 |
feat(phase-25): task A giant batch 1 — 2 more giants ×134 (func_80135480, func_80163EC8) + §44 levers; 4 Fable5 seeds
- cheap-Opus batch over the 6 frontier giants: 3 banked ×134 this session (func_80166994 §43 committed earlier; + func_80135480 258-ins block-scoped-pointer-split; func_80163EC8 234-ins cross-jump-duplicated-tail, 1 benign $v0 pin). 266 siblings byte-identical, 0 failed - R22 clean-fleet 136/136; instr-weighted 57.2->57.7%, distinct-code 28.2->29.3%, dedup 1813/0 - cookbook §44 — 5 reusable levers: §43-extension (K&R s16 also covers callee-stash sign-extend), pointer-var-decl (avoid &sym CSE-hoist), block-scoped-pointer-split, cross-jump-duplicated-tail (cracks a §31-D1/D2-"permuter-only" class), + the intrinsic-wall taxonomy (what cheap-Opus can't) - R14: §43 does NOT universally transfer (only 1 of 6 giants was K&R-s16; each is its own class); giant #1's prior "MATCH" note was stale/false (verify vs bytes) - 4 giants -> pin-free/structural Fable5+permuter SEEDS (func_80133CD4 flagship §37 allocno-tie, func_8014D820 RC-6 pin-free, func_801670E4 scheduling, func_8016CBC0 coalescing); escalation = permuter-first then Fable5 §34, queued in the CURRENT_PHASE resume block - SESSION CHECKPOINT: phase 25 OPEN; fresh session resumes the giant-seed escalation round |
||
|
Drew T
|
c62fe7f7ea |
feat(phase-25): task A giant #1 — func_80166994 (369 ins) cracked ×134 via Fable5 + the K&R s16-param idiom (§43)
- Fable5 subagent cracked func_80166994 (trail/afterimage ring recorder, 369 ins) — FULLY STRUCTURAL, zero register pins -> swept ×134 CLEAN (exemplar + 133 siblings byte-identical). R22 clean-fleet 136/136; instr-weighted 56.8% -> 57.2%; distinct-code 27.3% -> 28.2% - NEW IDIOM cookbook §43: a K&R s16-param DEFINITION dissolves the §17/§29 "narrow-param wall". On MIPS K&R promotes s16->int (ABI-identical to the canon-sig s32), body keeps the in-place sll aN,16 narrow/extend the (s16)cast form can't reproduce. void->s32 return-flip pair: split //@EDIT (self-fn, ov077-specific) + engine_core.h ec_edit ×5 (byte-neutral, callers discard) - family_sweep --edit-remap: split-edits now OPTIONAL (apply where present, never skip; the whole-binary byte-gate is the sole arbiter, G3/P9) — a sibling lacking the ov077 canon-sig decl still banks via ec_edit + body. edit-absent tracked, not skipped - R14: the prior wave's "@stuck: none — MATCH" note on func_80166994 was STALE/FALSE (re-ran DIFF 366/369). Verify a MATCH claim vs the bytes, never a stale note - structural cracks are the ×134-SAFE ones (contrast §42e pin-heavy families that cc1-SIGABRT in sibling TUs). Other 6 giants -> cheap-Opus applying §43+§31, Fable5 only on new-class evidence |
||
|
Drew T
|
c0379f0738 |
feat(phase-25): progress.py --weighted — byte/instruction-weighted metrics (the honest headline numbers)
- weighted_metrics() from .run/sig.*.jsonl + src stubs (executable code only, resident + 134 overlays; main EXE excluded). Two framings: fleet instr-weighted (per-overlay, the decomp.dev -display number) + dedup distinct-code (each unique h_exact once, the distinct-RE number) - --fleet now emits THREE labeled metrics into docs/progress.fleet.md: fn-count 74.48% (×134- inflated), instr-weighted 56.8% (shipped .text), distinct-code 27.3% (of 84,996 unique fns) - --weighted prints the two weighted numbers standalone; degrades gracefully if sigs absent - corrects the stale "~30-35% byte-weighted" estimate: the giant campaign since Phase 19 raised the fleet instr-weighted number to 56.8%; the distinct-code 27.3% is the unique-monster-tail truth - SETUP §tooling row updated (R21) |
||
|
Drew T
|
5fcb040dc3 |
feat(phase-25): task B — family_sweep --edit-remap; 2 array-decay families ×134 (+266 fns), 4 cc1-crash-walled
- family_sweep.py: new --edit-remap MANIFEST mode (§42e) — per family, symbol-remap the split-scope //@EDIT old||new per sibling + apply once-global engine_core.h ec_edits (byte-neutral), stage the family_remap body, gate via harvest_verify (the sole arbiter) - BANKED 266/266 (0 failed): func_80136824 + func_80136334 (array-decay ptr-flip) ×133 siblings each — full ×134. R22 clean-fleet 136/136, fleet 74.40% -> 74.48%, dedup 1813/0 - R14 FINDING (cookbook §42e addendum + decision-log): the other 4 byte-drift families (func_80133AB0 zero-reg pin, func_8016DF5C/8013D9B0 GTE-pin, func_80156044 trampoline) cc1-SIGABRT (Error 134) in the SIBLING TU — hand pins are ov077-TU-context-specific, NOT mechanically ×134-recoverable; backlogged as ×1/permuter fuel. rtu_match/match_one are blind here (neutralized/isolation compiles crash too); only make build is truth - 0 NON_MATCHING in any default build (G4) |
||
|
Drew T
|
0241772225 |
fix(phase-25): canon_sig_reconcile def-finder (\n -> (?:^|\n)) unblocks crack propagation; recover func_8014FE60 x134; fleet 74.36->74.40%, R22 136/136
- R14 CORRECTION of the prior "family_remap limitation" call: it was a MISDIAGNOSIS. family_remap
succeeds on all droppers; the "remap-fail" family_sweep reports was a mislabeled canon_sig_reconcile
throw ("no definition of func_X found in draft") — the def-finder regex required a leading \n, so a
//@EDIT-stripped raw draft with the fn definition on line 1 was not found.
- FIX: def-finder regex \n -> (?:^|\n) (also match a def at draft start; strictly additive, low-risk).
- Recovered func_8014FE60 fully: 133/133 siblings banked (fix + engine_core.h DEFINE_func_8014FDF4
extern void->s32 global flip, byte-neutral fleet-wide; caller discards return).
- Residual (the genuine, small --edit-remap): func_8016DF5C/80136334/8013D9B0/80156044 reconcile but
byte-drift per sibling (out-of-body fixes: pointer //@EDIT, no-proto, return-flip not carried per sibling).
- cookbook §42e (the two-layer diagnosis + the forward ×134-leverage-realism rule); decision-log corrected.
- R22 clean-fleet 136/136 BYTE-IDENTICAL from a fully clean tree; NON_MATCHING 7 (0 in default build, G4).
|
||
|
Drew T
|
6c0887b097 |
feat(phase-25): crack fan-out over the frontier map — wave 4, 24/26 MATCH + 1330 swept; fleet 73.97->74.36% (+1350 fns), R22 136/136
- Frontier map (docs/phase25-frontier-map.md, committed commit:0506): rtu_match-measured all 42 draftable families -> 37 crack targets; endgame = 42 draftable + 235 matched-free + 2481 absent. - Crack fan-out wf_b54b5d98-380 (26 tractable-band exemplars): 24/26 MATCH -> 20 banked byte-identical, incl. func_8014FBC0 (22 ins x1996 inline-asm trampoline), func_80132144 (27 x539), func_8016CF04 (engine_core.h void->short flip). Swept x134: 1330 siblings / 931 failed (//@EDIT/trampoline/engine_core families -> --edit-remap backlog). Batch = 20 exemplars + 1330 = 1350 fns. - 5 durable levers -> cookbook §42d: return-type flip BOTH ways (void<->s32/short); address-recompute-vs-cache (the unifying read-global rule); the full-inline-asm trampoline idiom + family_remap-inside-asm; memcpy-> struct-assign at scale; phantom-frame induction. - Deferred (backlog, map tiers): func_8016D1D8/80165240 (M-linkwall), func_80164E40 (sig-reconcile), func_801457A4 (-O0), func_8012E364/801549F8 (permuter). - R22 clean-fleet 136/136 BYTE-IDENTICAL from a fully clean tree; NON_MATCHING 7 (0 in default build, G4). |