diff --git a/phase-ends/CURRENT_PHASE.md b/phase-ends/CURRENT_PHASE.md index 8c71ce9801..acf72232e6 100644 --- a/phase-ends/CURRENT_PHASE.md +++ b/phase-ends/CURRENT_PHASE.md @@ -181,7 +181,79 @@ stub on a named wall/behemoth/queue ledger** β€” 140/140 byte-identical througho --- -# πŸ›‘ SESSION S45 CHECKPOINT part 8 (2026-08-07 late) β€” the SC03 trio SOLVED Β· Stage 1+2 landed Β· propagation in flight +# πŸ›‘ SESSION S45 CHECKPOINT part 9 (2026-08-07, session close) β€” FRESH SESSION SAFE HERE +> **Tree CLEAN** (`src/`+`config/` = 0 modified) but for R23 `db.*.gbf` churn β€” never stage. +> **Nothing running.** Effort ultracode. **NO phase close** β€” T5 unopened, needs Drew's gate-2. + +## β–Ά RESUME HERE β€” 3 items, in this order + +### 1. FIX `dedup_propagate`'s apply_plan/struct_check, THEN resume the banking +The propagation is the one unfinished item. **~30 fns β†’ ~3,163 member-instances** (~0 tokens). +It FAILED on its second run β€” a genuine tool bug, NOT the parallel-gate change (0 gate batches +ran; it never reached that code): +``` +[FAIL] ov_MAIN_012: 0x80156600 not instantiated β€” REVERTED +``` +- **Inputs verified SOUND at HEAD:** func_80156600 IS in ov_MAIN_012's sig (18 ins), `find_site` + returns `stub`, and the stub line matches `stub_line()` exactly. So the bug is in the + MULTI-FUNCTION edit path of `apply_plan`, not in the data. +- **WHY run #1 didn't hit it:** run #1 launched BEFORE the 15 wave-3-re-gate banks were committed; + those landed mid-flight, so run #2's plan is LARGER and reaches a path run #1 never did. +- **SECOND DEFECT, same failure:** it printed `REVERTED` but left **38 files dirty** incl. + `src/shared/engine_core.h`. Same incomplete-restore CLASS as the reconcile-ledger bug fixed + earlier (cookbook Β§156), on a DIFFERENT path. `struct_check`'s failure exit needs the same + ledger treatment. +- **Pragmatic option if the fix is slow:** propagate with an explicit `--addr` list EXCLUDING + 0x80156600 to bank the other ~29 now, and diagnose separately. +- **Recovery if a run dies dirty:** `git checkout -- src/ config/` (deterministic; done 3Γ— tonight). + +### 2. THE MASTER IDXTAB MAP (Drew's idea β€” feasibility PROVEN, high value) +Repeat the ov_SC03_001 decode for EVERY binary β†’ a ledger of **payload β†’ owning binary β†’ load +address**. Resolves MAIN/7+9 ownership (or proves absence across every reference table β€” the +strongest dead-code evidence obtainable), gives the disc-completeness claim its evidence base +(R34), validates Β§S44 exhaustively, and turns "where does X load?" into a lookup. +- **DESTPTR half ALREADY WORKS: 14/14 sampled overlays, first try**, reproducing Β§S44's one + documented case exactly (`ov_SC01_000 *0x801A3234 = 0x801A58E8`). Method: register-track + `func_80128CFC` (same vram in every overlay) for its `lui`+`lw`, then read that word. +- **IDXTAB half β€” the one idea still needed:** find `-1`-terminated s16 index runs, then REQUIRE a + register-verified code reference to the run's address (`tools/find_addr_refs.py`). That + reference is the discriminator the fleet-wide shape scan lacked (Β§155a). Validate against the + two known-good tables FIRST: `ov_SC01_000` @0x8017EEC8 (37 entries), `ov_SC03_001` @0x8018D7BC (5). + +### 3. Then waves β€” 1,168 seeded targets in the big-3; Haiku ≀30 ins (86%), Sonnet β‰₯50 (Β§157). +Use `tools/wave_snapshot.py`; reach-sort; carve `main` out (1,061 sub-25 stubs, barren, Γ—1). +**Read args FROM THE FILE** β€” never hand-type them (that cost a 50-agent wave tonight). + +## βœ… WHAT THIS SESSION LANDED (all committed, tree clean) +- **44 functions banked**, R22 213/213 Β· ov_SC02_037 626β†’597 stubs +- **SC03/53/54/56 SOLVED** β€” live script modules owned by `ov_SC03_001` (IDXTAB @0x8018D7BC = + 224/231/232/234/233) β†’ `func_80128CFC` β†’ `*DESTPTR 0x801EBC68` = **0x801EF468**. Load BASE not + yet proved; the byte-gate arbitrates on onboarding. (`docs/memory-map.md` Β§S45 p6) +- **MAIN/7 + MAIN/9** β€” absent from a full 304s attract cycle + 7 static lines. Dead-code case + strong, not proved. Their owning binary is unknown β€” item 2 would settle it. +- **Concurrency work:** Stage 1 (reconcile ledger + shared-state RW lock, 3 NCs) and Stage 2 + (`verify_worktree`, GREEN 87s, NC fires RED) DONE. **Stage 5 CANCELLED** (87s can't lag). + Stage 3 deferred for drafting gates β€” but **propagation IS the cross-binary workload it fits**, + so the parallel gate (`gate_all`, 32-way, verdict-identical to serial) landed here instead. +- **6 new tools:** `cdtrace` (runtime CD-load oracle; 7 routing addresses confirmed live), + `find_addr_refs` (register-tracked, Β§155), `wave_snapshot`, `shared_lock`, `verify_worktree`, + `test_reconcile_ledger` +- **Cookbook Β§155a/Β§155b/Β§156/Β§157**; model ladder recalibrated (Haiku ≀30 = 86%, β‰₯50 = 20%, 4Γ— cost) +- **Measured, worth keeping:** propagation setup is 8.6s of a 5,700s run (0.15%) β€” ALL the time is + byte-gating. Don't optimise setup; if the parallel gate underdelivers, attack the O(fnsΓ—overlays) + trial loop, not the JSON parsing. + +## 🧰 MY ERROR LEDGER THIS SESSION (8, one root cause) β€” R37-adjacent rule candidate for T5 +Fabricated workflow args Β· string-vs-integer membership test Β· two gate verdicts read off a broken +tree Β· the F1 misattribution Β· a `head -8`-truncated grep Β· an over-broad "wave_snapshot decouples +waves" claim Β· an invalid timing comparison (ran parallel first, so serial found everything cached). +**Root cause is ONE: asserting a mechanism or number I had not personally derived.** The byte-gate +caught every one before it reached anything load-bearing β€” but the pattern is a governance gap, not +eight separate slips. Propose at T5 alongside R37. + +--- + +# πŸ›‘ (superseded by part 9) SESSION S45 CHECKPOINT part 8 (2026-08-07 late) β€” the SC03 trio SOLVED Β· Stage 1+2 landed Β· propagation in flight > **Effort ultracode.** HEAD `commit:1528`. ⚠️ **A PROPAGATION WAS RUNNING AT CHECKPOINT TIME** β€” > see "IF THE TREE IS DIRTY" below before anything else. diff --git a/tools/dedup_propagate.py b/tools/dedup_propagate.py index ed771e5d87..3bb8ff6991 100644 --- a/tools/dedup_propagate.py +++ b/tools/dedup_propagate.py @@ -347,6 +347,59 @@ def byte_gate(ov): return r.returncode == 0, r.stdout + r.stderr +def _jobs(): + """Parallel overlay gates. Env JOBS wins; otherwise use EVERY core. + + Deliberately NOT capped at the Makefile's conservative `JOBS ?= 16` default: each worker is a + `make build` that spends nearly all its wall-clock in cc1/as/ld subprocesses, so the box is the + limit, not Python. Measured 2026-08-07: the serial version ran a propagation for 95 minutes at + load 1.6 on a 32-core machine β€” ~5% utilisation. Saturate it (Drew, same session). + """ + try: + j = int(_os.environ.get("JOBS", "0")) + except ValueError: + j = 0 + return max(1, j or (_os.cpu_count() or 4)) + + +def gate_all(changed, label=""): + """Byte-gate EVERY touched overlay in parallel; return the first failure in `changed` ORDER. + + WHY (measured 2026-08-07): this loop used to be serial β€” one `make build BINARY=` at a + time, over up to 141 members, for each of ~30 functions. A propagation ran 95 minutes at load + 1.6 on a 32-core box: ~5% utilisation. The Makefile has parallelised `extract-all`/`check-all` + since Phase 26 (`xargs -P$(JOBS)`), but dedup_propagate predates that and drives the + SINGLE-binary `build` target from Python, so it never saw any of it. + + SAFE for the same reason `check-all` is: byte_gate only runs `make build BINARY=`, which + writes solely to the per-binary-disjoint `build//**`; it mutates NO source. The splice has + already happened before this is called, and the restore happens after β€” only the *verification* + is parallel. Threads, not processes: subprocess.run releases the GIL while the build runs. + + DETERMINISM: ThreadPoolExecutor.map preserves input order, so the reported first failure is the + first in `changed` order β€” identical to the serial loop's verdict, not whichever build finished + first. The serial version short-circuited on the first failure and so did fewer builds; this + does them all, but in parallel, and the all-pass case (the common one) is a straight win. + """ + order = list(changed) + if not order: + return None + j = min(_jobs(), len(order)) + if j <= 1: + for ov in order: + if not byte_gate(ov)[0]: + return ov + return None + from concurrent.futures import ThreadPoolExecutor + print(f"[gate] byte-gating {len(order)} overlay(s) with {j} parallel builds{label}", flush=True) + with ThreadPoolExecutor(max_workers=j) as ex: + results = list(ex.map(lambda o: (o, byte_gate(o)[0]), order)) + for ov, ok in results: + if not ok: + return ov + return None + + # ---------------------------------------------------------------- straggler caller-extern reconcile (--recover) def reconcile_caller_extern(ov, addr): """no-proto every conflicting `extern func_();` caller decl in ov's src files @@ -662,10 +715,7 @@ def main(): struct_check(plan, changed, touched) if a.no_gate: break - fail_ov = None - for ov in changed: - if not byte_gate(ov)[0]: - fail_ov = ov; break + fail_ov = gate_all(changed, f" ({len(plan)} fn(s) in plan)") if fail_ov is None: print(f"[ OK ] {len(changed)} overlays byte-identical after propagation") break