tools(phase-35): T6 part 1 — 14 macro-era tools FROZEN (tools/frozen.py: one refusal from main(), imports never exit; the plan's 7 + 7 the guard census found, family_remap's CLI among them), 4 retired to tools/sunset/ (dedup_propagate + dedup_extend -> share_body.py with add_members_surgical and the load_sig library moved verbatim; macro_draft; test_reconcile_ledger), six lane callers repointed to share_body --apply --bucket new, validate_targets imports share_body, tool_census accepts FROZEN; dictionary + sunset README rows; kit corpus

This commit is contained in:
Drew T
2026-09-08 20:32:21 -06:00
parent cd43347791
commit 5a9f455e60
67 changed files with 356 additions and 1645 deletions
+5
View File
@@ -0,0 +1,5 @@
.venv/bin/python tools/tool_census.py --all
tool_census: 295 tool files (find == git ls-files) + 38 retired; classes {'LIVE': 236, 'ORPHAN': 30, 'REFERENCED': 29, 'RETIRED': 38}; dictionary rows 333
tool_census: wrote docs/tool-index.md
tool_census: wrote decomp-architect/tools/MANIFEST.md
tool_census: corpus materialised — 361 verbatim copies, 30 pointers, INDEX.md
+19 -18
View File
@@ -7,7 +7,7 @@ tools/alloc_table.py P7 PORTABLE list each pseudo-register's refs, live length a
tools/api_agent.py P8 ADAPT give an external chat model the same multi-turn tool harness an agent gets Gives an external chat model the same multi-turn tool harness an agent gets provider endpoint, repo doc paths LIVE
tools/api_draft.py P8 ADAPT draft one function against any chat-completions endpoint, provider-agnostic Provider-agnostic single-shot drafting worker against any chat completions endpoint provider endpoint, repo doc paths LIVE
tools/api_rate.py P8 ADAPT measure a drafting provider's real request rate and rate-limit attribution Reports measured request rate and rate-limit attribution from append-only telemetry repo telemetry path STILL-NEEDED measure a drafting provider's true request rate and rate-limit attribution
tools/aprop_autodraft.py P6 ADAPT draft a family member mechanically from a seed body and symbol rebase Mechanically drafts a family member from the seed body plus a positional symbol rebase repo scratch paths LIVE
tools/aprop_autodraft.py P6 ADAPT draft a family member mechanically from a seed body and symbol rebase Mechanically drafts a family member from the seed body plus a positional symbol rebase repo scratch paths FROZEN tools/share_body.py
tools/aprop_symfix.py P6 ADAPT rebase a remapped draft's stale seed symbols onto the target's own Rebases stale seed symbols in a mechanically adapted draft onto the target's own symbols repo symbol/config sources LIVE
tools/asm_verbatim.py P10 ADAPT emit a function's target assembly as an inline-assembly body Emits the file-scope inline-assembly body form from a disassembly file repo asm layout LIVE
tools/atlas.py P5 ADAPT partition every open function into exactly one lever-labelled work group Partitions all open functions into exactly one lever-labelled work group repo scratch paths LIVE
@@ -38,7 +38,7 @@ tools/bfm_extract/pac.py PROJECT-ONLY PROJECT-ONLY parse a bespoke archive-chain
tools/bfm_extract/test_lzss.py PROJECT-ONLY PROJECT-ONLY unit-test a bespoke compression decoder against known cases Unit tests for that game-specific compression variant project-only LIVE
tools/bisect_slate.py P8 ADAPT isolate the byte-wrong drafts in a batch, with a null control first Isolates the byte-wrong drafts in a batch, with a null control first repo build paths SUPERSEDED tools/gate_main_parallel.py
tools/blast_radius.py P8 ADAPT measure a change's write set and name the verification it actually requires Measures a change's write set and names the verification it actually requires repo tree layout LIVE
tools/blocker_probe.py P5 ADAPT explain with two oracles why a byte-correct draft fails the whole-binary gate Read-only two-oracle explanation of why a byte-correct draft fails the whole-binary gate repo build paths LIVE
tools/blocker_probe.py P5 ADAPT explain with two oracles why a byte-correct draft fails the whole-binary gate Read-only two-oracle explanation of why a byte-correct draft fails the whole-binary gate repo build paths FROZEN tools/share_body.py
tools/bootstrap.sh P9 ADAPT set up a fresh clone idempotently: packages, environments, pinned tools Idempotent fresh-clone setup: check packages, create environments, fetch pinned tools repo layout, package names LIVE
tools/build_engine_types.py P10 ADAPT extract inline-defined types and typedefs from a source file into a shared header Extracts inline-defined named types and typedefs from a source file into a shared header repo shared-header path LIVE
tools/build_fuel_manifest.py P8 ADAPT unify live stubs into one ranked, class-tagged, cache-annotated target pool Unifies live stubs into one ranked, class-tagged, cache-annotated target pool canonical source binary, repo scratch paths LIVE
@@ -50,7 +50,7 @@ tools/burndown.py P5 ADAPT track per-session yield and velocity so a diminishing
tools/campaign_status.py P8 ADAPT report every concurrent lane with its own metrics, not only the loud one One status view covering every lane, not only the loud one lane names, repo scratch paths STILL-NEEDED report every concurrent lane with its own metrics, not only the loud one
tools/canon_draft_decls.py P6 ADAPT canonicalize a draft's extern and data declarations to the banked-consistent set Canonicalizes a draft's extern and data declarations to the banked-consistent set repo shared-header paths SUPERSEDED tools/sig_unify.py
tools/canon_resident_calls.py P6 ADAPT rewrite address-named calls in a draft to their curated symbol names Rewrites address-named calls in a draft to the curated symbol name when one exists repo symbol file path LIVE
tools/canon_sig_reconcile.py P6 ADAPT reconcile a definition's typed signature with what its destination unit declares — part of banking against a canonical type layer from Phase 6; again at readability (Phase 10) Reconciles a definition's typed signature with what its destination unit already declares repo src layout LIVE
tools/canon_sig_reconcile.py P6 ADAPT reconcile a definition's typed signature with what its destination unit declares — part of banking against a canonical type layer from Phase 6; again at readability (Phase 10) Reconciles a definition's typed signature with what its destination unit already declares repo src layout FROZEN the types phase (Phase 37)
tools/cast_call_sites.py P10 ADAPT add per-site function-pointer casts so a draft can call a differently typed callee Adds per-site function-pointer casts so a draft can call a differently typed callee repo src layout LIVE
tools/cast_self_callers.py P6 ADAPT let a unit keep calling, through per-site casts, the function it now defines Lets a unit keep calling, through a per-site cast, the function it is about to define repo src layout LIVE
tools/cc1_dumps.sh P4 ADAPT dump every compiler pass file for a self-contained draft Dumps every compiler pass file for a self-contained draft into a private directory compiler triple, repo scratch paths LIVE
@@ -59,17 +59,17 @@ tools/cdecl.py P6 PORTABLE parse C declarators once, for every declaration tool
tools/cdtrace.py PROJECT-ONLY PROJECT-ONLY observe a live loader's requests to learn which payload loads where Reads one game's loader request structures at fixed RAM addresses project-only STILL-NEEDED observe a live loader's requests to learn which payload is read to which address
tools/claude_wave_packs.py P8 ADAPT build the per-function prompt packs a drafting agent wave works from Builds the per-function prompt packs an agent wave drafts from repo scratch paths, agent harness LIVE
tools/compile_only.py P9 ADAPT compile every eligible translation unit with the pinned toolchain, using no derived bytes Compiles every eligible translation unit with the pinned toolchain, without any game bytes compiler triple, repo makefile parsing LIVE
tools/conform_decls.py P6 ADAPT conform every declaration of a function fleet-wide to its byte-true definition — width and signedness proven by the bytes at bank time (Phase 6) Conforms every declaration of a function fleet-wide to its byte-true definition repo src/shared-header layout LIVE
tools/conform_decls.py P6 ADAPT conform every declaration of a function fleet-wide to its byte-true definition — width and signedness proven by the bytes at bank time (Phase 6) Conforms every declaration of a function fleet-wide to its byte-true definition repo src/shared-header layout FROZEN the types phase (Phase 37)
tools/cookbook_index.py P9 ADAPT generate a symptom-keyed index of a large technique document Generates a symptom-keyed index of the large technique document repo doc paths LIVE
tools/corpus.py P5 ADAPT answer from one derived model whether each function is open, matched or shared The single derived model of the source tree: open, matched or shared, per function repo src/config layout LIVE
tools/decl_from_use.py P6 ADAPT infer a minimal extern for a data symbol the destination unit does not declare Infers a minimal extern for a data symbol a draft uses but its destination does not declare repo src layout LIVE
tools/decl_prior.py P6 ADAPT compute the fleet's consensus declaration for every symbol, as card fuel — the seed of a canonical type layer (Phase 6) Computes the fleet's consensus declaration for every symbol, as card fuel repo src layout LIVE
tools/decompile.py P4 ADAPT turn one function's disassembly into a C scaffold to start a match Wrapper locating a function's disassembly and running the C-scaffold generator on it repo asm paths, decompiler target name STILL-NEEDED turn one function's disassembly into a C scaffold to start a match
tools/decompme_replica.sh P4 ADAPT reproduce an external reference toolchain locally and compare its words to target Runs a function through an external reference toolchain build and compares words against the target pinned toolchain versions, network fetch STILL-NEEDED kept: a public preset request needs the replica of the community toolchain to prove equivalence
tools/dedup_extend.py P6 ADAPT extend an existing code-share registry to newly onboarded binaries Extends the existing share registry to newly onboarded binaries repo config/src layout LIVE
tools/dedup_extend.py P6 ADAPT extend an existing code-share registry to newly onboarded binaries Extends the existing share registry to newly onboarded binaries repo config/src layout SUPERSEDED tools/share_body.py
tools/dedup_integrate.py P6 ADAPT validate a code-share registry byte-honestly, failing closed on signature drift Byte-honesty validator for the code-share registry; fails closed on signature drift repo config registry path LIVE
tools/dedup_propagate.py P6 ADAPT lift one matched body into a shared macro and instantiate it everywhere Lifts one matched body into a shared macro and instantiates it at every duplicate site repo shared-header/src layout LIVE
tools/demacroize.py P6 ADAPT escape a shared-header declaration conflict locally by unmacroizing one body Per-binary local escape from a shared-header declaration conflict by unmacroizing one body repo shared-header layout LIVE
tools/dedup_propagate.py P6 ADAPT lift one matched body into a shared macro and instantiate it everywhere Lifts one matched body into a shared macro and instantiates it at every duplicate site repo shared-header/src layout SUPERSEDED tools/share_body.py
tools/demacroize.py P6 ADAPT escape a shared-header declaration conflict locally by unmacroizing one body Per-binary local escape from a shared-header declaration conflict by unmacroizing one body repo shared-header layout FROZEN tools/share_body.py
tools/diff_autopsy.sh P7 ADAPT find which words diverge when the whole-binary gate says different but not where Reproduces exactly what the gate saw, then decodes the diverging words and restores the tree repo src/build paths STILL-NEEDED find WHICH words diverge when the whole-binary gate says DIFF but not where
tools/diff_regions.py P7 ADAPT classify where a remapped sibling's compiled bytes diverge from its target Classifies where a remapped member's compiled bytes diverge from its target repo build paths STILL-NEEDED classify where a mechanically remapped sibling's compiled bytes diverge from its target
tools/difficulty.py P5 ADAPT rank unmatched functions easiest-first by size, control flow and call count Ranks unmatched functions easiest-first by size, control flow, table presence and call count repo asm layout LIVE
@@ -87,8 +87,8 @@ tools/family_align.py P6 ADAPT classify drifted structural siblings by aligned w
tools/family_cousins.py P6 ADAPT cluster the open frontier one tier looser than exact skeleton hashing Similarity clustering one tier looser than exact skeleton hashing over the open frontier repo signature files LIVE
tools/family_hseq.py P6 ADAPT rank clusters of the unmatched frontier by mnemonic-skeleton hash Ranked clustering of the unmatched frontier by mnemonic-skeleton hash repo signature files LIVE
tools/family_manifest.py P6 ADAPT name the one exemplar to draft per structural family, ranked Ranked structural-family target manifest naming the one exemplar to draft per family repo signature files LIVE
tools/family_remap.py P6 ADAPT remap a matched exemplar's source onto a structural sibling by positional symbol pairing Mechanically remaps a matched exemplar's C onto a structural sibling by positional symbol pairing repo src layout LIVE
tools/family_sweep.py P6 ADAPT sweep a matched exemplar across every same-structure sibling, gate-arbitrated Sweeps a matched exemplar across every same-structure sibling, gate-arbitrated repo src/config layout LIVE
tools/family_remap.py P6 ADAPT remap a matched exemplar's source onto a structural sibling by positional symbol pairing Mechanically remaps a matched exemplar's C onto a structural sibling by positional symbol pairing repo src layout FROZEN tools/share_body.py
tools/family_sweep.py P6 ADAPT sweep a matched exemplar across every same-structure sibling, gate-arbitrated Sweeps a matched exemplar across every same-structure sibling, gate-arbitrated repo src/config layout FROZEN tools/share_body.py
tools/fetch_psyq.sh P9 ADAPT acquire and checksum-verify vendor SDK pieces for a linked build Optional acquisition and checksum verification of vendor SDK pieces for the linked build PS1 SDK sources, repo paths LIVE
tools/find_addr_refs.py P2 PORTABLE find code that materializes an absolute address, tracking registers rather than pairing windows Register-tracked scan for code that materializes an absolute address; never window-paired MIPS encodings, repo binary registry LIVE
tools/fix_arity_callers.py P6 ADAPT repair a shared caller's argument-count conflict for the no-prototype failure class Repairs the shared-caller argument-count conflict for the no-prototype failure class repo shared-header layout LIVE
@@ -109,7 +109,7 @@ tools/gate_wave.py P8 ADAPT gate a whole wave by splitting table-bearing drafts
tools/gater_lane.py P8 ADAPT drain finished drafts into the gate continuously while drafting still streams Continuous gater draining a wave's finished drafts into the parallel gate, grouped by binary repo scratch paths STILL-NEEDED drain finished drafts into the gate continuously while drafting still streams
tools/gccmap_cites.py P7 ADAPT tag every compiler-source citation with the source tree it refers to Tags every compiler-source citation in the codegen-map docs with the tree it refers to repo doc/reference paths LIVE
tools/gen_engine_decls.py P6 ADAPT declare every shared function and data symbol once, canonically, in a generated header Declares every shared function and data symbol once, canonically, in a generated header repo shared-header path SUPERSEDED tools/conform_decls.py
tools/gen_harvest_targets.py P8 ADAPT generate a callee-signature-aware target manifest with per-function fleet reach Generates a callee-signature-aware target manifest with per-function fleet reach canonical source binary, repo paths LIVE
tools/gen_harvest_targets.py P8 ADAPT generate a callee-signature-aware target manifest with per-function fleet reach Generates a callee-signature-aware target manifest with per-function fleet reach canonical source binary, repo paths FROZEN tools/share_census.py
tools/gen_lib_subsegs.py P3 ADAPT generate segment lines and a stub list for a multi-block vendor library region Generates segment lines and a stub list for a multi-block vendor library region repo config layout, PS1 SDK libs LIVE
tools/ghidra_annotations_delta.py P2 ADAPT derive the hand-authored annotations by subtracting a fresh analysis baseline Derives the hand-authored annotation rows by subtracting a fresh rebuild baseline repo paths LIVE
tools/ghidra_apply_symbols.sh P10 ADAPT mirror the curated symbol file into the analysis database with a real save Mirrors the curated symbol file into the analysis program headlessly, with a real save repo symbol path, project name STILL-NEEDED mirror the curated symbol file into the analysis database headlessly with a real save
@@ -144,7 +144,7 @@ tools/idiom_hunt.py P8 ADAPT discover new compiler idioms from grouped near-miss
tools/idiom_loop.py P8 ADAPT drain the easy fuel, learn the next idiom, hand off, repeat Never-ending meta-loop: drain the easy fuel, learn the next idiom, hand off repo scratch paths SUPERSEDED tools/idiom_serial.py
tools/idiom_serial.py P8 ADAPT learn idioms serially so each target inherits what the previous one taught The serial idiom-learning lane where each step inherits what the previous learned repo scratch paths LIVE
tools/idxtab_map.py PROJECT-ONLY PROJECT-ONLY decode an on-disc index table into a payload-to-binary naming map Decodes one game's on-disc index table and its payload-to-binary naming convention — the load map itself is a Phase-2 task project-only LIVE
tools/inject_capped_externs.py P6 ADAPT free duplicate matches a share propagation skips as not self-contained Frees high-reach inline matches skipped as not self-contained by injecting their file-scope externs repo shared-header layout STILL-NEEDED free duplicate matches a share propagation skips as not self-contained
tools/inject_capped_externs.py P6 ADAPT free duplicate matches a share propagation skips as not self-contained Frees high-reach inline matches skipped as not self-contained by injecting their file-scope externs repo shared-header layout FROZEN tools/share_body.py
tools/integration_resolver.py P8 ADAPT re-judge already byte-correct drafts and bank them for zero model tokens Zero-token lane re-judging drafts that are already byte-correct and banking them repo scratch paths LIVE
tools/interleave_check.py P6 ADAPT assert a read-only-data carve's interleave order equals the segment sequence Asserts the carve interleave order equals the segment sequence, position by position repo config layout STILL-NEEDED assert a read-only-data carve's interleave order equals the segment sequence
tools/journal_notes.py P8 ADAPT mine each function's own past-attempt history out of agent journals Mines per-function past-attempt notes out of the agent journals into card fuel agent transcript paths LIVE
@@ -184,7 +184,7 @@ tools/lenmiss_route.py P7 ADAPT route a length-drift near-miss pile through the
tools/lift_types.py P6 ADAPT lift a named list of types fleet-wide into the shared type header — the canonical type layer from the first bank (Phase 6, the multiplier); again at readability (Phase 10) Lifts a named list of types fleet-wide into the shared type header repo shared-header path LIVE
tools/lint_symbol_refs.py P10 ADAPT flag address-named references whose address now has a curated name Flags address-named references in committed sources whose address now has a curated name repo symbol/src paths LIVE
tools/lora_grind.py P8 ADAPT rotate binaries drafting small stubs with a served model, gating and propagating Mass-run driver rotating binaries, drafting small stubs with a served model, gating, propagating repo scratch paths, model endpoint LIVE
tools/macro_draft.py P6 ADAPT materialize a shared macro body back into a compilable standalone draft Materializes a shared macro body back into a compilable standalone draft repo shared-header layout STILL-NEEDED materialize a shared macro body back into a compilable standalone draft
tools/macro_draft.py P6 ADAPT materialize a shared macro body back into a compilable standalone draft Materializes a shared macro body back into a compilable standalone draft repo shared-header layout ONE-OFF the per-function headers under src/shared/ (Phase 35 T4; the macro form is gone)
tools/main_diff_locate.py P7 ADAPT turn a red whole-binary result into a named list of divergent symbols Turns a red whole-binary result into a named list of divergent symbols repo build/config layout LIVE
tools/main_lane.py P8 ADAPT run the executable's own draft, gate and commit cadence beside other lanes The executable's own draft, gate and commit cadence beside the other lanes repo build/make targets LIVE
tools/main_queue_rebuild.py P8 ADAPT rebuild the executable lane's queue from the tree, never from a stale list Rebuilds the executable lane's queue from the tree, never from a stale list repo src layout LIVE
@@ -201,17 +201,17 @@ tools/mk_write.py P3 ADAPT write the generated binary-registry makefile safely,
tools/neighbor_ref.py P8 ADAPT rank already-matched functions worth reading as worked examples for an open stub For an open stub, ranks already-matched functions worth reading as worked examples repo signature files LIVE
tools/new_binary.sh P3 ADAPT onboard any flat blob in one command: config, registry entry, first build One-command onboarding of any flat blob: config from template, registry entry, first build repo config/template paths, the source game's payload classes LIVE
tools/new_overlay.sh PROJECT-ONLY PROJECT-ONLY onboard one payload by its container and file-index naming Takes one game's container and file-index naming as its command-line contract project-only LIVE
tools/normalize_self_decls.py P6 ADAPT normalize a function's own declaration when a templated definition lands in a new unit Normalizes a function's own declaration when a templated definition lands in a new unit repo src layout LIVE
tools/normalize_self_decls.py P6 ADAPT normalize a function's own declaration when a templated definition lands in a new unit Normalizes a function's own declaration when a templated definition lands in a new unit repo src layout FROZEN tools/share_body.py
tools/o0_boundary.py P6 ADAPT find and bank unoptimized functions stranded at an optimized region's boundary Finds and banks unoptimized functions stranded at the end of an unoptimized region repo config layout STILL-NEEDED find and bank unoptimized functions stranded at an optimized region's boundary
tools/o0_detect.py P6 ADAPT answer from a prologue whether a function was built unoptimized The single place that answers whether a function was built unoptimized, from its prologue compiler prologue shape LIVE
tools/o0_subsplit.py P6 ADAPT route an unoptimized address range sitting inside an otherwise optimized object Routes an unoptimized address range that sits inside an otherwise optimized object repo config layout LIVE
tools/o0_subsplit.py P6 ADAPT route an unoptimized address range sitting inside an otherwise optimized object Routes an unoptimized address range that sits inside an otherwise optimized object repo config layout FROZEN the Makefile -O0 globs (tools/rollout_o0.py)
tools/objdiff_report.py P9 PORTABLE convert progress metrics into an external progress-report schema Converts the progress JSON into the external progress-report schema external report schema LIVE
tools/oracle/reg_renumber_swap.sh P7 ADAPT decide whether a residual reorder is scheduling or a register-grant consequence Mechanized oracle separating a register-allocation swap from a scheduling reorder compiler triple, repo scratch paths STILL-NEEDED decide whether a residual reorder is a scheduling choice or a register-grant consequence
tools/oracle_reorder.py P5 ADAPT decide whether a near-miss residual is a source defect or an assembler artifact Decides whether a near-miss residual is a source defect or an assembler artifact pinned assembler flags LIVE
tools/orchestrator.py P8 ADAPT pick the pool, emit the batch and record the cycle, deterministically The deterministic half of the unattended loop: pick the pool, emit the batch, record the cycle repo scratch paths LIVE
tools/overlay_src_split.py P6 ADAPT partition a source file at object boundaries, aware of comment state Comment-state-aware partition of a source file, the oracle for the split chain repo src layout LIVE
tools/ox_campaign.py P8 ADAPT loop draw, draft, pre-filter, gate, commit and ledger unattended The unattended wave loop: draw, draft, pre-filter, gate, commit, ledger, repeat repo scratch paths, provider config LIVE
tools/p16_improve.py P5 ADAPT revert known answers to stubs and categorize every harness failure it exposes Known-answer loop: reverts matched functions to stubs and measures the real pipeline repo src/build paths LIVE
tools/p16_improve.py P5 ADAPT revert known answers to stubs and categorize every harness failure it exposes Known-answer loop: reverts matched functions to stubs and measures the real pipeline repo src/build paths FROZEN tools/share_body.py
tools/p16_known_answer.py P5 ADAPT validate a scaffold pipeline against known answers across difficulty bands Graduated known-answer validation of the scaffold pipeline across difficulty bands repo paths SUPERSEDED tools/p16_improve.py
tools/p16_permute.py P7 ADAPT drive the permuter per function, building base, target and settings Per-function permuter driver building base, target and settings, then reporting closeness repo scratch paths, permuter layout LIVE
tools/pads_audit.py P6 ADAPT derive an object's inter-table padding spec from the bytes instead of searching Derives each object's padding spec from the bytes instead of searching for it repo build/config layout STILL-NEEDED derive an object's inter-table padding spec from the bytes instead of searching
@@ -252,7 +252,7 @@ tools/ram_probe.py P2 ADAPT capture and diff live console memory across scenario
tools/reconcile_slate.py P6 ADAPT drive a whole batch to declaration-compatible before any rebuild is spent Drives a whole batch to declaration-compatible before any rebuild is spent repo src layout SUPERSEDED tools/pregate_check.py
tools/reconcile_tu.py P6 ADAPT conform a draft's data declarations to what its destination unit can see Conforms a draft's data declarations to what the destination translation unit can actually see repo src layout LIVE
tools/recover_drafts.py P8 ADAPT recover deleted agent draft files from a run's transcripts Recovers agent-written draft files from a run's transcripts after they were deleted agent transcript paths SUPERSEDED tools/agent_drafts_restore.py
tools/recover_giant.py P8 ADAPT recover a large draft rejected only on declaration plumbing, with canonical externs Canonical-extern recovery for a large draft rejected only on declaration plumbing repo shared-header layout LIVE
tools/recover_giant.py P8 ADAPT recover a large draft rejected only on declaration plumbing, with canonical externs Canonical-extern recovery for a large draft rejected only on declaration plumbing repo shared-header layout FROZEN tools/share_body.py
tools/recover_integration.py P8 ADAPT drive batch integration recovery over a directory of correct-but-rejected drafts Batch integration recovery driver over a wave directory of correct-but-rejected drafts repo scratch/src layout LIVE
tools/recover_rejects.py P8 ADAPT turn pre-gate rejects back into bankable drafts for zero model tokens Turns pre-gate rejects back into bankable drafts for zero model tokens repo scratch paths LIVE
tools/recover_route.py P8 ADAPT name the one recovery tool that actually applies to a drop verdict Given a drop verdict, names the one recovery tool that actually applies repo scratch paths LIVE
@@ -263,7 +263,7 @@ tools/residual_class.py P7 PORTABLE map a residual to a named codegen class by d
tools/residual_rules.py P7 ADAPT turn a measured residual into the specific documented lever that addresses it Turns a measured residual into the specific documented lever that addresses it repo doc paths SUPERSEDED tools/residual_rules_b.py
tools/residual_rules_b.py P7 ADAPT classify a residual's shape to its documented lever, as an independent implementation Independent second implementation of residual-shape to rule classification, for head-to-head repo doc paths LIVE
tools/restage_matching.py P8 ADAPT re-stage only drafts that match in their real unit so one failure is isolated Re-stages only the drafts that compile and match in their real unit, so one failure is isolated repo src layout STILL-NEEDED re-stage only drafts that match in their real unit so one failure is isolated
tools/restore_dropped_decls.py P6 ADAPT restore file-scope declarations a share propagation deleted with the body Puts back file-scope declarations a share propagation deleted with the body repo src layout STILL-NEEDED restore file-scope declarations a code-share propagation deleted with the body
tools/restore_dropped_decls.py P6 ADAPT restore file-scope declarations a share propagation deleted with the body Puts back file-scope declarations a share propagation deleted with the body repo src layout FROZEN tools/share_body.py
tools/rollout_o0.py P6 ADAPT roll an unoptimized definition atomically across every binary carrying it Generalized two-file atomic rollout of an unoptimized definition across every binary carrying it repo src/config layout LIVE
tools/rtu_match.py P4 ADAPT splice a candidate into a copy of its real translation unit and compare bytes Splices a candidate into a copy of the real translation unit and checks the same bytes repo src layout, compiler triple LIVE
tools/rtu_second_chance.py P4 ADAPT re-judge standalone compile failures against their real translation unit before dropping Re-judges standalone compile-failures against their real translation unit before dropping them repo scratch paths LIVE
@@ -292,7 +292,7 @@ tools/t5_targets.py P8 ADAPT draw one routed slate from the classified frontier
tools/t7_bank.py P8 ADAPT bank a batch, reconciling declarations at bank time before the whole-binary gate Batch banking driver applying reconcile-at-bank-time plus the whole-binary gate repo src/config layout SUPERSEDED tools/gate_stage.py
tools/test_jtbl_parse_config.py P5 ADAPT regression-test the table-carve configuration parser with a negative control Regression plus negative control for the table-carve configuration parser repo config fixtures LIVE
tools/test_o0_detect.py P5 ADAPT negative-control the detector that decides a function's optimization level Negative control for the matcher's optimization-level auto-detection repo paths STILL-NEEDED negative-control the detector that decides a function's optimization level
tools/test_reconcile_ledger.py P5 ADAPT exercise a guarded repair path the full negative control never reaches Targeted proof of the propagation ledger guard the full control never exercised repo paths STILL-NEEDED exercise a guarded repair path the full negative control never reaches
tools/test_reconcile_ledger.py P5 ADAPT exercise a guarded repair path the full negative control never reaches Targeted proof of the propagation ledger guard the full control never exercised repo paths ONE-OFF the S45p7 reconcile-ledger proof, retired with its subject dedup_propagate
tools/test_residual_class.py P5 PORTABLE unit-test a residual classifier on hand-encoded words, not its own decoder Synthetic hand-encoded unit tests for the deterministic residual classifier MIPS encodings STILL-NEEDED unit-test a residual classifier on hand-encoded words, not on its own decoder
tools/timeline.py P9 ADAPT build a progress timeline from the repository's own committed digests Builds a progress timeline from the repository's own committed digests, with a self-check repo doc paths LIVE
tools/tool_census.py P5 ADAPT derive the tool census, the need-keyed index, the kit manifest and the verbatim corpora from one dictionary Derives the tool census, the need-keyed tool index, the kit manifest and the verbatim corpora from one dictionary, coverage-asserted both ways repo layout (tools/, docs/SETUP.md rows, the kit paths) LIVE
@@ -331,3 +331,4 @@ tools/share_census.py P10 ADAPT measure duplicate function bodies across the fle
tools/macro_to_header.py P10 PROJECT-ONLY convert a shared-body macro header into per-function plain-C headers included at each site The Phase-35 converter: every DEFINE_ macro body becomes a plain-C header under src/shared/<space>/ included at its site, the legacy name-parameterized headers converted, the registry text-edited, the macro header deleted; --plan / --apply / --finalize / --verify the macro form is this project's; a kit-born project shares headers from its first bank LIVE
tools/share_body.py P10 ADAPT share one byte-identical function class across its binaries through an include-at-site header, gated per binary The Phase-35 successor of dedup_propagate + dedup_extend for the include-at-site form: exemplar by majority text, the header written once, every private copy replaced by the include at its position, per-binary byte gate with object comparison, bisect on red, the registry appended or extended by text, the exception ledger on refusal; --plan / --apply --bucket extend|new repo paths, the registry and signature schemas LIVE
tools/share_body_cycle.sh P10 ADAPT run the share-body batch cycle unattended: batch, verify by exit code, log, commit, periodic clean fleet check The Phase-35 T5 driver for bucket new: per batch share_body --apply --bucket new --batches 1 --label new<k>, the gated N/N line read, the phase log entry appended, the bank committed (R42), the clean fleet run every N batches (R22) — stops on the first red repo paths, the phase-log format, the fleet count from config/check.*.sha LIVE
tools/frozen.py P10 PORTABLE refuse, from one place, the command line of a tool the project has frozen The one refusal a FROZEN tool prints from its main(): the tool, the successor and why; imports never exit (libraries stay usable) nothing LIVE
1 path phase portability need what adapts status successor_or_product
7 tools/api_agent.py P8 ADAPT give an external chat model the same multi-turn tool harness an agent gets Gives an external chat model the same multi-turn tool harness an agent gets provider endpoint, repo doc paths LIVE
8 tools/api_draft.py P8 ADAPT draft one function against any chat-completions endpoint, provider-agnostic Provider-agnostic single-shot drafting worker against any chat completions endpoint provider endpoint, repo doc paths LIVE
9 tools/api_rate.py P8 ADAPT measure a drafting provider's real request rate and rate-limit attribution Reports measured request rate and rate-limit attribution from append-only telemetry repo telemetry path STILL-NEEDED measure a drafting provider's true request rate and rate-limit attribution
10 tools/aprop_autodraft.py P6 ADAPT draft a family member mechanically from a seed body and symbol rebase Mechanically drafts a family member from the seed body plus a positional symbol rebase repo scratch paths LIVE FROZEN tools/share_body.py
11 tools/aprop_symfix.py P6 ADAPT rebase a remapped draft's stale seed symbols onto the target's own Rebases stale seed symbols in a mechanically adapted draft onto the target's own symbols repo symbol/config sources LIVE
12 tools/asm_verbatim.py P10 ADAPT emit a function's target assembly as an inline-assembly body Emits the file-scope inline-assembly body form from a disassembly file repo asm layout LIVE
13 tools/atlas.py P5 ADAPT partition every open function into exactly one lever-labelled work group Partitions all open functions into exactly one lever-labelled work group repo scratch paths LIVE
38 tools/bfm_extract/test_lzss.py PROJECT-ONLY PROJECT-ONLY unit-test a bespoke compression decoder against known cases Unit tests for that game-specific compression variant project-only LIVE
39 tools/bisect_slate.py P8 ADAPT isolate the byte-wrong drafts in a batch, with a null control first Isolates the byte-wrong drafts in a batch, with a null control first repo build paths SUPERSEDED tools/gate_main_parallel.py
40 tools/blast_radius.py P8 ADAPT measure a change's write set and name the verification it actually requires Measures a change's write set and names the verification it actually requires repo tree layout LIVE
41 tools/blocker_probe.py P5 ADAPT explain with two oracles why a byte-correct draft fails the whole-binary gate Read-only two-oracle explanation of why a byte-correct draft fails the whole-binary gate repo build paths LIVE FROZEN tools/share_body.py
42 tools/bootstrap.sh P9 ADAPT set up a fresh clone idempotently: packages, environments, pinned tools Idempotent fresh-clone setup: check packages, create environments, fetch pinned tools repo layout, package names LIVE
43 tools/build_engine_types.py P10 ADAPT extract inline-defined types and typedefs from a source file into a shared header Extracts inline-defined named types and typedefs from a source file into a shared header repo shared-header path LIVE
44 tools/build_fuel_manifest.py P8 ADAPT unify live stubs into one ranked, class-tagged, cache-annotated target pool Unifies live stubs into one ranked, class-tagged, cache-annotated target pool canonical source binary, repo scratch paths LIVE
50 tools/campaign_status.py P8 ADAPT report every concurrent lane with its own metrics, not only the loud one One status view covering every lane, not only the loud one lane names, repo scratch paths STILL-NEEDED report every concurrent lane with its own metrics, not only the loud one
51 tools/canon_draft_decls.py P6 ADAPT canonicalize a draft's extern and data declarations to the banked-consistent set Canonicalizes a draft's extern and data declarations to the banked-consistent set repo shared-header paths SUPERSEDED tools/sig_unify.py
52 tools/canon_resident_calls.py P6 ADAPT rewrite address-named calls in a draft to their curated symbol names Rewrites address-named calls in a draft to the curated symbol name when one exists repo symbol file path LIVE
53 tools/canon_sig_reconcile.py P6 ADAPT reconcile a definition's typed signature with what its destination unit declares — part of banking against a canonical type layer from Phase 6; again at readability (Phase 10) Reconciles a definition's typed signature with what its destination unit already declares repo src layout LIVE FROZEN the types phase (Phase 37)
54 tools/cast_call_sites.py P10 ADAPT add per-site function-pointer casts so a draft can call a differently typed callee Adds per-site function-pointer casts so a draft can call a differently typed callee repo src layout LIVE
55 tools/cast_self_callers.py P6 ADAPT let a unit keep calling, through per-site casts, the function it now defines Lets a unit keep calling, through a per-site cast, the function it is about to define repo src layout LIVE
56 tools/cc1_dumps.sh P4 ADAPT dump every compiler pass file for a self-contained draft Dumps every compiler pass file for a self-contained draft into a private directory compiler triple, repo scratch paths LIVE
59 tools/cdtrace.py PROJECT-ONLY PROJECT-ONLY observe a live loader's requests to learn which payload loads where Reads one game's loader request structures at fixed RAM addresses project-only STILL-NEEDED observe a live loader's requests to learn which payload is read to which address
60 tools/claude_wave_packs.py P8 ADAPT build the per-function prompt packs a drafting agent wave works from Builds the per-function prompt packs an agent wave drafts from repo scratch paths, agent harness LIVE
61 tools/compile_only.py P9 ADAPT compile every eligible translation unit with the pinned toolchain, using no derived bytes Compiles every eligible translation unit with the pinned toolchain, without any game bytes compiler triple, repo makefile parsing LIVE
62 tools/conform_decls.py P6 ADAPT conform every declaration of a function fleet-wide to its byte-true definition — width and signedness proven by the bytes at bank time (Phase 6) Conforms every declaration of a function fleet-wide to its byte-true definition repo src/shared-header layout LIVE FROZEN the types phase (Phase 37)
63 tools/cookbook_index.py P9 ADAPT generate a symptom-keyed index of a large technique document Generates a symptom-keyed index of the large technique document repo doc paths LIVE
64 tools/corpus.py P5 ADAPT answer from one derived model whether each function is open, matched or shared The single derived model of the source tree: open, matched or shared, per function repo src/config layout LIVE
65 tools/decl_from_use.py P6 ADAPT infer a minimal extern for a data symbol the destination unit does not declare Infers a minimal extern for a data symbol a draft uses but its destination does not declare repo src layout LIVE
66 tools/decl_prior.py P6 ADAPT compute the fleet's consensus declaration for every symbol, as card fuel — the seed of a canonical type layer (Phase 6) Computes the fleet's consensus declaration for every symbol, as card fuel repo src layout LIVE
67 tools/decompile.py P4 ADAPT turn one function's disassembly into a C scaffold to start a match Wrapper locating a function's disassembly and running the C-scaffold generator on it repo asm paths, decompiler target name STILL-NEEDED turn one function's disassembly into a C scaffold to start a match
68 tools/decompme_replica.sh P4 ADAPT reproduce an external reference toolchain locally and compare its words to target Runs a function through an external reference toolchain build and compares words against the target pinned toolchain versions, network fetch STILL-NEEDED kept: a public preset request needs the replica of the community toolchain to prove equivalence
69 tools/dedup_extend.py P6 ADAPT extend an existing code-share registry to newly onboarded binaries Extends the existing share registry to newly onboarded binaries repo config/src layout LIVE SUPERSEDED tools/share_body.py
70 tools/dedup_integrate.py P6 ADAPT validate a code-share registry byte-honestly, failing closed on signature drift Byte-honesty validator for the code-share registry; fails closed on signature drift repo config registry path LIVE
71 tools/dedup_propagate.py P6 ADAPT lift one matched body into a shared macro and instantiate it everywhere Lifts one matched body into a shared macro and instantiates it at every duplicate site repo shared-header/src layout LIVE SUPERSEDED tools/share_body.py
72 tools/demacroize.py P6 ADAPT escape a shared-header declaration conflict locally by unmacroizing one body Per-binary local escape from a shared-header declaration conflict by unmacroizing one body repo shared-header layout LIVE FROZEN tools/share_body.py
73 tools/diff_autopsy.sh P7 ADAPT find which words diverge when the whole-binary gate says different but not where Reproduces exactly what the gate saw, then decodes the diverging words and restores the tree repo src/build paths STILL-NEEDED find WHICH words diverge when the whole-binary gate says DIFF but not where
74 tools/diff_regions.py P7 ADAPT classify where a remapped sibling's compiled bytes diverge from its target Classifies where a remapped member's compiled bytes diverge from its target repo build paths STILL-NEEDED classify where a mechanically remapped sibling's compiled bytes diverge from its target
75 tools/difficulty.py P5 ADAPT rank unmatched functions easiest-first by size, control flow and call count Ranks unmatched functions easiest-first by size, control flow, table presence and call count repo asm layout LIVE
87 tools/family_cousins.py P6 ADAPT cluster the open frontier one tier looser than exact skeleton hashing Similarity clustering one tier looser than exact skeleton hashing over the open frontier repo signature files LIVE
88 tools/family_hseq.py P6 ADAPT rank clusters of the unmatched frontier by mnemonic-skeleton hash Ranked clustering of the unmatched frontier by mnemonic-skeleton hash repo signature files LIVE
89 tools/family_manifest.py P6 ADAPT name the one exemplar to draft per structural family, ranked Ranked structural-family target manifest naming the one exemplar to draft per family repo signature files LIVE
90 tools/family_remap.py P6 ADAPT remap a matched exemplar's source onto a structural sibling by positional symbol pairing Mechanically remaps a matched exemplar's C onto a structural sibling by positional symbol pairing repo src layout LIVE FROZEN tools/share_body.py
91 tools/family_sweep.py P6 ADAPT sweep a matched exemplar across every same-structure sibling, gate-arbitrated Sweeps a matched exemplar across every same-structure sibling, gate-arbitrated repo src/config layout LIVE FROZEN tools/share_body.py
92 tools/fetch_psyq.sh P9 ADAPT acquire and checksum-verify vendor SDK pieces for a linked build Optional acquisition and checksum verification of vendor SDK pieces for the linked build PS1 SDK sources, repo paths LIVE
93 tools/find_addr_refs.py P2 PORTABLE find code that materializes an absolute address, tracking registers rather than pairing windows Register-tracked scan for code that materializes an absolute address; never window-paired MIPS encodings, repo binary registry LIVE
94 tools/fix_arity_callers.py P6 ADAPT repair a shared caller's argument-count conflict for the no-prototype failure class Repairs the shared-caller argument-count conflict for the no-prototype failure class repo shared-header layout LIVE
109 tools/gater_lane.py P8 ADAPT drain finished drafts into the gate continuously while drafting still streams Continuous gater draining a wave's finished drafts into the parallel gate, grouped by binary repo scratch paths STILL-NEEDED drain finished drafts into the gate continuously while drafting still streams
110 tools/gccmap_cites.py P7 ADAPT tag every compiler-source citation with the source tree it refers to Tags every compiler-source citation in the codegen-map docs with the tree it refers to repo doc/reference paths LIVE
111 tools/gen_engine_decls.py P6 ADAPT declare every shared function and data symbol once, canonically, in a generated header Declares every shared function and data symbol once, canonically, in a generated header repo shared-header path SUPERSEDED tools/conform_decls.py
112 tools/gen_harvest_targets.py P8 ADAPT generate a callee-signature-aware target manifest with per-function fleet reach Generates a callee-signature-aware target manifest with per-function fleet reach canonical source binary, repo paths LIVE FROZEN tools/share_census.py
113 tools/gen_lib_subsegs.py P3 ADAPT generate segment lines and a stub list for a multi-block vendor library region Generates segment lines and a stub list for a multi-block vendor library region repo config layout, PS1 SDK libs LIVE
114 tools/ghidra_annotations_delta.py P2 ADAPT derive the hand-authored annotations by subtracting a fresh analysis baseline Derives the hand-authored annotation rows by subtracting a fresh rebuild baseline repo paths LIVE
115 tools/ghidra_apply_symbols.sh P10 ADAPT mirror the curated symbol file into the analysis database with a real save Mirrors the curated symbol file into the analysis program headlessly, with a real save repo symbol path, project name STILL-NEEDED mirror the curated symbol file into the analysis database headlessly with a real save
144 tools/idiom_loop.py P8 ADAPT drain the easy fuel, learn the next idiom, hand off, repeat Never-ending meta-loop: drain the easy fuel, learn the next idiom, hand off repo scratch paths SUPERSEDED tools/idiom_serial.py
145 tools/idiom_serial.py P8 ADAPT learn idioms serially so each target inherits what the previous one taught The serial idiom-learning lane where each step inherits what the previous learned repo scratch paths LIVE
146 tools/idxtab_map.py PROJECT-ONLY PROJECT-ONLY decode an on-disc index table into a payload-to-binary naming map Decodes one game's on-disc index table and its payload-to-binary naming convention — the load map itself is a Phase-2 task project-only LIVE
147 tools/inject_capped_externs.py P6 ADAPT free duplicate matches a share propagation skips as not self-contained Frees high-reach inline matches skipped as not self-contained by injecting their file-scope externs repo shared-header layout STILL-NEEDED FROZEN free duplicate matches a share propagation skips as not self-contained tools/share_body.py
148 tools/integration_resolver.py P8 ADAPT re-judge already byte-correct drafts and bank them for zero model tokens Zero-token lane re-judging drafts that are already byte-correct and banking them repo scratch paths LIVE
149 tools/interleave_check.py P6 ADAPT assert a read-only-data carve's interleave order equals the segment sequence Asserts the carve interleave order equals the segment sequence, position by position repo config layout STILL-NEEDED assert a read-only-data carve's interleave order equals the segment sequence
150 tools/journal_notes.py P8 ADAPT mine each function's own past-attempt history out of agent journals Mines per-function past-attempt notes out of the agent journals into card fuel agent transcript paths LIVE
184 tools/lift_types.py P6 ADAPT lift a named list of types fleet-wide into the shared type header — the canonical type layer from the first bank (Phase 6, the multiplier); again at readability (Phase 10) Lifts a named list of types fleet-wide into the shared type header repo shared-header path LIVE
185 tools/lint_symbol_refs.py P10 ADAPT flag address-named references whose address now has a curated name Flags address-named references in committed sources whose address now has a curated name repo symbol/src paths LIVE
186 tools/lora_grind.py P8 ADAPT rotate binaries drafting small stubs with a served model, gating and propagating Mass-run driver rotating binaries, drafting small stubs with a served model, gating, propagating repo scratch paths, model endpoint LIVE
187 tools/macro_draft.py P6 ADAPT materialize a shared macro body back into a compilable standalone draft Materializes a shared macro body back into a compilable standalone draft repo shared-header layout STILL-NEEDED ONE-OFF materialize a shared macro body back into a compilable standalone draft the per-function headers under src/shared/ (Phase 35 T4; the macro form is gone)
188 tools/main_diff_locate.py P7 ADAPT turn a red whole-binary result into a named list of divergent symbols Turns a red whole-binary result into a named list of divergent symbols repo build/config layout LIVE
189 tools/main_lane.py P8 ADAPT run the executable's own draft, gate and commit cadence beside other lanes The executable's own draft, gate and commit cadence beside the other lanes repo build/make targets LIVE
190 tools/main_queue_rebuild.py P8 ADAPT rebuild the executable lane's queue from the tree, never from a stale list Rebuilds the executable lane's queue from the tree, never from a stale list repo src layout LIVE
201 tools/neighbor_ref.py P8 ADAPT rank already-matched functions worth reading as worked examples for an open stub For an open stub, ranks already-matched functions worth reading as worked examples repo signature files LIVE
202 tools/new_binary.sh P3 ADAPT onboard any flat blob in one command: config, registry entry, first build One-command onboarding of any flat blob: config from template, registry entry, first build repo config/template paths, the source game's payload classes LIVE
203 tools/new_overlay.sh PROJECT-ONLY PROJECT-ONLY onboard one payload by its container and file-index naming Takes one game's container and file-index naming as its command-line contract project-only LIVE
204 tools/normalize_self_decls.py P6 ADAPT normalize a function's own declaration when a templated definition lands in a new unit Normalizes a function's own declaration when a templated definition lands in a new unit repo src layout LIVE FROZEN tools/share_body.py
205 tools/o0_boundary.py P6 ADAPT find and bank unoptimized functions stranded at an optimized region's boundary Finds and banks unoptimized functions stranded at the end of an unoptimized region repo config layout STILL-NEEDED find and bank unoptimized functions stranded at an optimized region's boundary
206 tools/o0_detect.py P6 ADAPT answer from a prologue whether a function was built unoptimized The single place that answers whether a function was built unoptimized, from its prologue compiler prologue shape LIVE
207 tools/o0_subsplit.py P6 ADAPT route an unoptimized address range sitting inside an otherwise optimized object Routes an unoptimized address range that sits inside an otherwise optimized object repo config layout LIVE FROZEN the Makefile -O0 globs (tools/rollout_o0.py)
208 tools/objdiff_report.py P9 PORTABLE convert progress metrics into an external progress-report schema Converts the progress JSON into the external progress-report schema external report schema LIVE
209 tools/oracle/reg_renumber_swap.sh P7 ADAPT decide whether a residual reorder is scheduling or a register-grant consequence Mechanized oracle separating a register-allocation swap from a scheduling reorder compiler triple, repo scratch paths STILL-NEEDED decide whether a residual reorder is a scheduling choice or a register-grant consequence
210 tools/oracle_reorder.py P5 ADAPT decide whether a near-miss residual is a source defect or an assembler artifact Decides whether a near-miss residual is a source defect or an assembler artifact pinned assembler flags LIVE
211 tools/orchestrator.py P8 ADAPT pick the pool, emit the batch and record the cycle, deterministically The deterministic half of the unattended loop: pick the pool, emit the batch, record the cycle repo scratch paths LIVE
212 tools/overlay_src_split.py P6 ADAPT partition a source file at object boundaries, aware of comment state Comment-state-aware partition of a source file, the oracle for the split chain repo src layout LIVE
213 tools/ox_campaign.py P8 ADAPT loop draw, draft, pre-filter, gate, commit and ledger unattended The unattended wave loop: draw, draft, pre-filter, gate, commit, ledger, repeat repo scratch paths, provider config LIVE
214 tools/p16_improve.py P5 ADAPT revert known answers to stubs and categorize every harness failure it exposes Known-answer loop: reverts matched functions to stubs and measures the real pipeline repo src/build paths LIVE FROZEN tools/share_body.py
215 tools/p16_known_answer.py P5 ADAPT validate a scaffold pipeline against known answers across difficulty bands Graduated known-answer validation of the scaffold pipeline across difficulty bands repo paths SUPERSEDED tools/p16_improve.py
216 tools/p16_permute.py P7 ADAPT drive the permuter per function, building base, target and settings Per-function permuter driver building base, target and settings, then reporting closeness repo scratch paths, permuter layout LIVE
217 tools/pads_audit.py P6 ADAPT derive an object's inter-table padding spec from the bytes instead of searching Derives each object's padding spec from the bytes instead of searching for it repo build/config layout STILL-NEEDED derive an object's inter-table padding spec from the bytes instead of searching
252 tools/reconcile_slate.py P6 ADAPT drive a whole batch to declaration-compatible before any rebuild is spent Drives a whole batch to declaration-compatible before any rebuild is spent repo src layout SUPERSEDED tools/pregate_check.py
253 tools/reconcile_tu.py P6 ADAPT conform a draft's data declarations to what its destination unit can see Conforms a draft's data declarations to what the destination translation unit can actually see repo src layout LIVE
254 tools/recover_drafts.py P8 ADAPT recover deleted agent draft files from a run's transcripts Recovers agent-written draft files from a run's transcripts after they were deleted agent transcript paths SUPERSEDED tools/agent_drafts_restore.py
255 tools/recover_giant.py P8 ADAPT recover a large draft rejected only on declaration plumbing, with canonical externs Canonical-extern recovery for a large draft rejected only on declaration plumbing repo shared-header layout LIVE FROZEN tools/share_body.py
256 tools/recover_integration.py P8 ADAPT drive batch integration recovery over a directory of correct-but-rejected drafts Batch integration recovery driver over a wave directory of correct-but-rejected drafts repo scratch/src layout LIVE
257 tools/recover_rejects.py P8 ADAPT turn pre-gate rejects back into bankable drafts for zero model tokens Turns pre-gate rejects back into bankable drafts for zero model tokens repo scratch paths LIVE
258 tools/recover_route.py P8 ADAPT name the one recovery tool that actually applies to a drop verdict Given a drop verdict, names the one recovery tool that actually applies repo scratch paths LIVE
263 tools/residual_rules.py P7 ADAPT turn a measured residual into the specific documented lever that addresses it Turns a measured residual into the specific documented lever that addresses it repo doc paths SUPERSEDED tools/residual_rules_b.py
264 tools/residual_rules_b.py P7 ADAPT classify a residual's shape to its documented lever, as an independent implementation Independent second implementation of residual-shape to rule classification, for head-to-head repo doc paths LIVE
265 tools/restage_matching.py P8 ADAPT re-stage only drafts that match in their real unit so one failure is isolated Re-stages only the drafts that compile and match in their real unit, so one failure is isolated repo src layout STILL-NEEDED re-stage only drafts that match in their real unit so one failure is isolated
266 tools/restore_dropped_decls.py P6 ADAPT restore file-scope declarations a share propagation deleted with the body Puts back file-scope declarations a share propagation deleted with the body repo src layout STILL-NEEDED FROZEN restore file-scope declarations a code-share propagation deleted with the body tools/share_body.py
267 tools/rollout_o0.py P6 ADAPT roll an unoptimized definition atomically across every binary carrying it Generalized two-file atomic rollout of an unoptimized definition across every binary carrying it repo src/config layout LIVE
268 tools/rtu_match.py P4 ADAPT splice a candidate into a copy of its real translation unit and compare bytes Splices a candidate into a copy of the real translation unit and checks the same bytes repo src layout, compiler triple LIVE
269 tools/rtu_second_chance.py P4 ADAPT re-judge standalone compile failures against their real translation unit before dropping Re-judges standalone compile-failures against their real translation unit before dropping them repo scratch paths LIVE
292 tools/t7_bank.py P8 ADAPT bank a batch, reconciling declarations at bank time before the whole-binary gate Batch banking driver applying reconcile-at-bank-time plus the whole-binary gate repo src/config layout SUPERSEDED tools/gate_stage.py
293 tools/test_jtbl_parse_config.py P5 ADAPT regression-test the table-carve configuration parser with a negative control Regression plus negative control for the table-carve configuration parser repo config fixtures LIVE
294 tools/test_o0_detect.py P5 ADAPT negative-control the detector that decides a function's optimization level Negative control for the matcher's optimization-level auto-detection repo paths STILL-NEEDED negative-control the detector that decides a function's optimization level
295 tools/test_reconcile_ledger.py P5 ADAPT exercise a guarded repair path the full negative control never reaches Targeted proof of the propagation ledger guard the full control never exercised repo paths STILL-NEEDED ONE-OFF exercise a guarded repair path the full negative control never reaches the S45p7 reconcile-ledger proof, retired with its subject dedup_propagate
296 tools/test_residual_class.py P5 PORTABLE unit-test a residual classifier on hand-encoded words, not its own decoder Synthetic hand-encoded unit tests for the deterministic residual classifier MIPS encodings STILL-NEEDED unit-test a residual classifier on hand-encoded words, not on its own decoder
297 tools/timeline.py P9 ADAPT build a progress timeline from the repository's own committed digests Builds a progress timeline from the repository's own committed digests, with a self-check repo doc paths LIVE
298 tools/tool_census.py P5 ADAPT derive the tool census, the need-keyed index, the kit manifest and the verbatim corpora from one dictionary Derives the tool census, the need-keyed tool index, the kit manifest and the verbatim corpora from one dictionary, coverage-asserted both ways repo layout (tools/, docs/SETUP.md rows, the kit paths) LIVE
331 tools/macro_to_header.py P10 PROJECT-ONLY convert a shared-body macro header into per-function plain-C headers included at each site The Phase-35 converter: every DEFINE_ macro body becomes a plain-C header under src/shared/<space>/ included at its site, the legacy name-parameterized headers converted, the registry text-edited, the macro header deleted; --plan / --apply / --finalize / --verify the macro form is this project's; a kit-born project shares headers from its first bank LIVE
332 tools/share_body.py P10 ADAPT share one byte-identical function class across its binaries through an include-at-site header, gated per binary The Phase-35 successor of dedup_propagate + dedup_extend for the include-at-site form: exemplar by majority text, the header written once, every private copy replaced by the include at its position, per-binary byte gate with object comparison, bisect on red, the registry appended or extended by text, the exception ledger on refusal; --plan / --apply --bucket extend|new repo paths, the registry and signature schemas LIVE
333 tools/share_body_cycle.sh P10 ADAPT run the share-body batch cycle unattended: batch, verify by exit code, log, commit, periodic clean fleet check The Phase-35 T5 driver for bucket new: per batch share_body --apply --bucket new --batches 1 --label new<k>, the gated N/N line read, the phase log entry appended, the bank committed (R42), the clean fleet run every N batches (R22) — stops on the first red repo paths, the phase-log format, the fleet count from config/check.*.sha LIVE
334 tools/frozen.py P10 PORTABLE refuse, from one place, the command line of a tool the project has frozen The one refusal a FROZEN tool prints from its main(): the tool, the successor and why; imports never exit (libraries stay usable) nothing LIVE
+5 -6
View File
@@ -8,8 +8,8 @@
> its platform SDK need the marked adaptation. The last table lists the tools that are project-only in code (their *shape* is a task;
> their code does not transfer). *TODO(platform): the MIPS and PlayStation SDK hard-codes are the ones another platform replaces first.*
>
> **Coverage:** 298 tool files in scope (submodules, vendored and downloaded code excluded), of which 298 live rows
> below; per phase: P1 2 · P2 26 · P3 17 · P4 9 · P5 27 · P6 54 · P7 20 · P8 86 · P9 27 · P10 14 · PROJECT-ONLY 16. Superseded tools appear only as pointers to their successor (28 pointer rows); one-offs are omitted. Table rows in all: 326 (the installer checks its copy against this figure).
> **Coverage:** 295 tool files in scope (submodules, vendored and downloaded code excluded), of which 295 live rows
> below; per phase: P1 2 · P2 26 · P3 17 · P4 9 · P5 26 · P6 51 · P7 20 · P8 86 · P9 27 · P10 15 · PROJECT-ONLY 16. Superseded tools appear only as pointers to their successor (30 pointer rows); one-offs are omitted. Table rows in all: 325 (the installer checks its copy against this figure).
## P1 — extraction + manifest
@@ -98,7 +98,6 @@
| `oracle_reorder.py` | decide whether a near-miss residual is a source defect or an assembler artifact | Decides whether a near-miss residual is a source defect or an assembler artifact | pinned assembler flags |
| `tool_census.py` | derive the tool census, the need-keyed index, the kit manifest and the verbatim corpora from one dictionary | Derives the tool census, the need-keyed tool index, the kit manifest and the verbatim corpora from one dictionary, coverage-asserted both ways | repo layout (tools/, docs/SETUP.md rows, the kit paths) |
| `wall_sweep.py` | enumerate fleet-wide every instance of one known assembler-level blocker class | Enumerates fleet-wide instances of one known assembler-level blocker class | repo asm layout, toolchain quirk |
| `test_reconcile_ledger.py` | exercise a guarded repair path the full negative control never reaches | Targeted proof of the propagation ledger guard the full control never exercised | repo paths |
| `blocker_probe.py` | explain with two oracles why a byte-correct draft fails the whole-binary gate | Read-only two-oracle explanation of why a byte-correct draft fails the whole-binary gate | repo build paths |
| `atlas_features.py` | extract one deterministic feature record per function across every registered binary | Extracts one deterministic feature record per function across all registered binaries | repo signature files |
| `worklist.py` | join the target pool and near-miss ledger into one byte-weighted queue | Joins the target pool and the near-miss ledger into one byte-weighted ranked queue | repo scratch paths |
@@ -138,7 +137,6 @@
| `pads_audit.py` | derive an object's inter-table padding spec from the bytes instead of searching | Derives each object's padding spec from the bytes instead of searching for it | repo build/config layout |
| `aprop_autodraft.py` | draft a family member mechanically from a seed body and symbol rebase | Mechanically drafts a family member from the seed body plus a positional symbol rebase | repo scratch paths |
| `demacroize.py` | escape a shared-header declaration conflict locally by unmacroizing one body | Per-binary local escape from a shared-header declaration conflict by unmacroizing one body | repo shared-header layout |
| `dedup_extend.py` | extend an existing code-share registry to newly onboarded binaries | Extends the existing share registry to newly onboarded binaries | repo config/src layout |
| `o0_boundary.py` | find and bank unoptimized functions stranded at an optimized region's boundary | Finds and banks unoptimized functions stranded at the end of an unoptimized region | repo config layout |
| `inject_capped_externs.py` | free duplicate matches a share propagation skips as not self-contained | Frees high-reach inline matches skipped as not self-contained by injecting their file-scope externs | repo shared-header layout |
| `decl_from_use.py` | infer a minimal extern for a data symbol the destination unit does not declare | Infers a minimal extern for a data symbol a draft uses but its destination does not declare | repo src layout |
@@ -146,8 +144,6 @@
| `match_protos.py` | join two related builds' signature dumps into a function correspondence | Joins per-function signature dumps of two related builds into a function correspondence | repo signature files |
| `cast_self_callers.py` | let a unit keep calling, through per-site casts, the function it now defines | Lets a unit keep calling, through a per-site cast, the function it is about to define | repo src layout |
| `lift_types.py` | lift a named list of types fleet-wide into the shared type header — the canonical type layer from the first bank (Phase 6, the multiplier); again at readability (Phase 10) | Lifts a named list of types fleet-wide into the shared type header | repo shared-header path |
| `dedup_propagate.py` | lift one matched body into a shared macro and instantiate it everywhere | Lifts one matched body into a shared macro and instantiates it at every duplicate site | repo shared-header/src layout |
| `macro_draft.py` | materialize a shared macro body back into a compilable standalone draft | Materializes a shared macro body back into a compilable standalone draft | repo shared-header layout |
| `scope_tu_externs.py` | move a unit's file-scope data externs down into their consumers | Moves a unit's own file-scope data externs down into their consumers to legalize a block-scope type | repo src layout |
| `family_manifest.py` | name the one exemplar to draft per structural family, ranked | Ranked structural-family target manifest naming the one exemplar to draft per family | repo signature files |
| `normalize_self_decls.py` | normalize a function's own declaration when a templated definition lands in a new unit | Normalizes a function's own declaration when a templated definition lands in a new unit | repo src layout |
@@ -338,6 +334,7 @@
| `verbatim_check.py` | guard that every inline-assembly body still reproduces its target bytes | Regression guard that every inline-assembly body still reproduces its target bytes | repo src layout |
| `share_census.py` | measure duplicate function bodies across the fleet and assert one source per unique function | The census of byte-identical function classes across every binary with their source forms and verdicts, plus the S1 invariant check with its exception ledger and a fixture self-test | repo paths, the registry and signature schemas |
| `ghidra_apply_symbols.sh` | mirror the curated symbol file into the analysis database with a real save | Mirrors the curated symbol file into the analysis program headlessly, with a real save | repo symbol path, project name |
| `frozen.py` | refuse, from one place, the command line of a tool the project has frozen | The one refusal a FROZEN tool prints from its main(): the tool, the successor and why; imports never exit (libraries stay usable) | nothing |
| `verbatim_target_s.py` | regenerate a splitter-format target disassembly for a function no longer stubbed | Regenerates a splitter-format target disassembly for a function that is no longer a stub; --gas emits the assemblable gas-syntax form ($-registers, .L labels, noreorder) that a web diff service accepts as a pasted target | repo build/asm layout |
| `share_body_cycle.sh` | run the share-body batch cycle unattended: batch, verify by exit code, log, commit, periodic clean fleet check | The Phase-35 T5 driver for bucket new: per batch share_body --apply --bucket new --batches 1 --label new<k>, the gated N/N line read, the phase log entry appended, the bank committed (R42), the clean fleet run every N batches (R22) — stops on the first red | repo paths, the phase-log format, the fleet count from config/check.*.sha |
| `share_body.py` | share one byte-identical function class across its binaries through an include-at-site header, gated per binary | The Phase-35 successor of dedup_propagate + dedup_extend for the include-at-site form: exemplar by majority text, the header written once, every private copy replaced by the include at its position, per-binary byte gate with object comparison, bisect on red, the registry appended or extended by text, the exception ledger on refusal; --plan / --apply --bucket extend|new | repo paths, the registry and signature schemas |
@@ -374,6 +371,8 @@
| `build_wave.py` | `draw_waves.py` | Builds the next wave from a card pool, ranked by fleet leverage |
| `build_wave_args.py` | `wave_args.py` | Emits workflow arguments for a wave from the target manifest, resolving each path |
| `canon_draft_decls.py` | `sig_unify.py` | Canonicalizes a draft's extern and data declarations to the banked-consistent set |
| `dedup_extend.py` | `share_body.py` | Extends the existing share registry to newly onboarded binaries |
| `dedup_propagate.py` | `share_body.py` | Lifts one matched body into a shared macro and instantiates it at every duplicate site |
| `exemplar_miner.py` | `atlas.py` | Routes every residual stub to its lever and ranks the highest-reach work |
| `fix_header_decl.py` | `conform_decls.py` | Rewrites a shared-header caller declaration to a draft's byte-true signature |
| `gen_engine_decls.py` | `conform_decls.py` | Declares every shared function and data symbol once, canonically, in a generated header |
@@ -0,0 +1,16 @@
#!/usr/bin/env python3
"""frozen.py — the ONE refusal every FROZEN tool prints from its main() (Phase 35 T6, 2026-09-08).
A frozen tool is a matching-era tool that reads or writes the `DEFINE_func_` macro form Phase 35 retired (the shared bodies are
plain-C headers under src/shared/ now, one source per unique function — the S1 invariant). Its code stays in tools/ for the record
and for the libraries other tools import from it; only its command line refuses, so an import never exits (cdecl imports
gen_harvest_targets; eighteen tools import family_remap). The dictionary row carries status FROZEN and the successor.
import frozen; frozen.refuse("family_sweep.py", "tools/share_body.py", "it rewrites the macro form Phase 35 retired")
"""
import sys
def refuse(tool, successor, why):
sys.exit(f"{tool}: FROZEN since Phase 35 (2026-09-08) — {why}. Successor: {successor}. The code is kept for the record and its "
f"library functions still import; the command line does nothing (R43). See docs/SETUP.md and config/tool_dictionary.tsv.")
@@ -11,7 +11,7 @@ WHAT ONE SHARE IS. A class = one h_exact (the raw instruction bytes) at one vram
are still PRIVATE COPIES (a definition in that binary's own TU). Sharing it: the class's ONE body goes to (or already is) a plain-C
header under src/shared/<space>/ (macro_to_header's naming: func_<VRAM>[__h8].h, keyed by the class — R48), every private copy becomes
`#include "../shared/<space>/<header>"` at the SAME position (the copy's definition lines, nothing else — the TU keeps its own
declarations), and the registry gains the group (shorthand form, appended by text) or the members (dedup_extend.add_members_surgical).
declarations), and the registry gains the group (shorthand form, appended by text) or the members (add_members_surgical, inherited from the retired dedup_extend).
THE EXEMPLAR (for an unregistered class) — printed with every share: the copy whose normalized text the MOST copies share; a tie goes
to a pin-free copy; a further tie to the shortest. A body defined under an asm-label alias (aF<ADDR>) gets its own binding
@@ -47,12 +47,102 @@ REPO = pathlib.Path(__file__).resolve().parent.parent
sys.path.insert(0, str(REPO / "tools"))
import share_census as sc # noqa: E402
import macro_to_header as m2h # noqa: E402 — Oracles (naming, spaces, sigs), bind_alias_header, include_line, banner
import dedup_extend as de # noqa: E402 — add_members_surgical (the registry's surgical text edit)
LEDGER = REPO / "config/dedup_exceptions.tsv"
RUN = REPO / ".run/P35/share"
# ----------------------------------------------------------------------------------------------------------------------------
# The library surface inherited from the retired propagate/extend tools in tools/sunset (Phase 35 T6; verbatim, so their importers
# change one line: `import share_body as dp`). load_sig/sig_path read the per-binary sig files; registered_addrs the registry.
def sig_path(ov):
return REPO / f".run/sig.{ov}.jsonl"
def onboarded_overlays():
"""Overlay aliases + (S44) module aliases — every non-main/resident binary a shared body can
propagate into. Modules carry engine functions too; excluding them re-creates the SC07
invisible-work bug one class over."""
out = []
for f, var in (("config/overlays.mk", "OVERLAY_BINARIES"), ("config/modules.mk", "MODULE_BINARIES")):
fp = REPO / f
if fp.exists():
m = re.search(rf"^{var}\s*:=\s*(.*)$", fp.read_text(), re.M)
if m:
out += m.group(1).split()
return out
def load_sig(ov):
"""addr(int) -> {h_exact, h_norm, nins, ...}; {} if absent."""
p = sig_path(ov)
if not p.exists():
return {}
out = {}
for ln in p.read_text().splitlines():
ln = ln.strip()
if ln:
r = json.loads(ln)
out[int(r["addr"], 16)] = r
return out
def registered_addrs():
"""Vaddrs already in config/dedup.us.yaml (shared via ANY mechanism — engine_core, ov_setters,
clearTbl40). --auto-from skips these so the bulk is purely additive and never collides with an
existing share (e.g. a SETTER-matched function) — which would trip the structural check."""
p = REPO / "config/dedup.us.yaml"
if not p.exists():
return set()
try:
import yaml
sys.path.insert(0, str(REPO / "tools"))
from dedup_integrate import group_members
data = yaml.safe_load(p.read_text()) or {}
return {v for g in (data.get("groups") or []) for (_b, v, _n) in group_members(g)}
except Exception:
return set()
def sym(addr):
return f"func_{addr:08X}" # splat convention: uppercase 8-hex
def add_members_surgical(additions):
"""Append binaries to each group's `binaries: [...]` list by TEXT EDIT, in place.
NEVER `yaml.safe_dump` this file. The first cut of this tool round-tripped it through
safe_dump and silently destroyed BOTH of the things a human needs from it (H5 — "never
silently drop comments on a rewrite"):
* all 47 comment lines — including the curated Phase-11 header explaining WHY the share is
source-level (the linker cannot excise bytes interior to an object) — dumped to nothing;
* every `vram: 0x80162FF4` re-serialized as `vram: 2148937716` (PyYAML parses YAML-1.1 hex
to int, and dumps int as decimal), making 1,832 entries unreadable.
It was invisible to every gate: dedup-check passed 1840/0 and check-all stayed 140/140,
because `_addr()` accepts both forms — the data was fine and the DOCUMENT was ruined. A
formatting-destructive write that all your oracles call green is exactly the class this
project keeps re-learning: the gate measures bytes, not intent.
`additions` = {group_id: [binary, ...]}. Idempotent: a binary already listed is skipped.
"""
p = str(REPO / "config/dedup.us.yaml")
lines = open(p).read().splitlines(keepends=True)
cur, n = None, 0
seen = set()
for i, ln in enumerate(lines):
m = re.match(r"^\s*-?\s*id:\s*(\S+)\s*$", ln)
if m:
cur = m.group(1)
continue
if cur and cur in additions and re.match(r"^\s*members:\s*$", ln):
# a VERBOSE group has no `binaries:` line — silently skipping it left five 141-member groups listed at 16 (P35 S94/S96)
raise SystemExit(f"add_members_surgical: {cur} is in the verbose `members:` form — convert it to shorthand first "
f"(share_body.py --repair-registry does), never skip it (R43)")
if cur and cur in additions and re.match(r"^\s*binaries:\s*\[", ln):
seen.add(cur)
add = [b for b in additions[cur] if re.search(rf"\b{re.escape(b)}\b", ln) is None]
if add:
lines[i] = ln.rstrip("\n").rstrip()[:-1].rstrip() + ", " + ", ".join(add) + "]\n"
n += len(add)
cur = None
open(p, "w").write("".join(lines))
return n
def log(msg):
print(msg, flush=True)
@@ -397,7 +487,7 @@ def run_batch(orc, cen, classes, label):
bins = [x for x in b.extend.get(gid, []) if x not in rejected_bins.get(c["h"], set())]
if bins:
ext[gid] = bins
n_ext = de.add_members_surgical(ext) if ext else 0
n_ext = add_members_surgical(ext) if ext else 0
ok_n = sum(1 for a in gated if results[a][0])
log(f" [{label}] gated {ok_n}/{len(gated)} binaries green · registered {len(reg_entries)} groups · extended {n_ext} members · "
f"rejected classes {len(failed_classes)}")
@@ -176,7 +176,7 @@ def main():
zero = sorted(b for b in onb if b.startswith("ov_") and memb.get(b, 0) == 0)
if zero:
warns.append(f"{len(zero)} overlay(s) in 0 dedup groups (onboarded but un-harvested — "
f"candidate for tools/dedup_extend.py): {zero if verbose else zero[:6]}")
f"candidate for tools/share_body.py --plan): {zero if verbose else zero[:6]}")
# --- report
n_ov = len([b for b in onb if b.startswith("ov_")])
@@ -223,6 +223,7 @@ def cc1_verdict(fn, stub, draft_path, work):
# ---------------------------------------------------------------------------------------------
def main():
import frozen; frozen.refuse("blocker_probe.py", "tools/share_body.py (its per-binary gate names the blocker)", "it probes macro-scope blockers in engine_core.h (gone)") # Phase 35 T6: FROZEN
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument('--binary', required=True)
@@ -66,6 +66,7 @@ def classify_fail(fn, cpath):
def main():
import frozen; frozen.refuse("p16_improve.py", "tools/share_body.py", "it rewrites macro sites in ov_SC01_077 against engine_core.h (gone)") # Phase 35 T6: FROZEN
ap = argparse.ArgumentParser()
ap.add_argument("--band", choices=list(BANDS), default="easy")
ap.add_argument("--n", type=int, default=40)
@@ -92,7 +92,7 @@ PHASE_TITLES = {"P1": "extraction + manifest", "P2": "the oracles + the load map
"P7": "the codegen map, the dumps, the permuter", "P8": "the campaign: cards, lanes, gates, recovery, harvest",
"P9": "publish", "P10": "readability", "PROJECT-ONLY": "project-only in code (the shape is a task; the code does not transfer)"}
COLS = ["path", "phase", "portability", "need", "what", "adapts", "status", "successor_or_product"]
STATUSES = {"LIVE", "STILL-NEEDED", "SUPERSEDED", "ONE-OFF"}
STATUSES = {"LIVE", "STILL-NEEDED", "FROZEN", "SUPERSEDED", "ONE-OFF"} # FROZEN (P35 T6): in the tree, command line refuses, libraries import
def sh(cmd):
@@ -191,7 +191,7 @@ def census():
files, retired = enumerate_tools()
rows = read_dict()
by_path = {r["path"]: r for r in rows}
live_rows = {p for p, r in by_path.items() if r["status"] in ("LIVE", "STILL-NEEDED")}
live_rows = {p for p, r in by_path.items() if r["status"] in ("LIVE", "STILL-NEEDED", "FROZEN")}
retired_rows = {p for p, r in by_path.items() if r["status"] in ("SUPERSEDED", "ONE-OFF")}
gaps = []
for f in files:
@@ -574,6 +574,7 @@ def _local_verdict(fn, text, sub, o0):
def main():
import frozen; frozen.refuse("aprop_autodraft.py", "tools/share_body.py", "it seeds drafts from the macro bodies in engine_core.h (gone)") # Phase 35 T6: FROZEN
ap = argparse.ArgumentParser()
ap.add_argument("--cards", default=CARDS)
ap.add_argument("--outdir", default=".run/aprop_autodraft")
@@ -518,6 +518,7 @@ def reconcile(fn, draft, canon_sig=None, tu_path=None):
def main():
import frozen; frozen.refuse("canon_sig_reconcile.py", "the types phase (Phase 37): one canonical declaration per symbol", "it reconciles declarations against engine_core.h (gone)") # Phase 35 T6: FROZEN
ap = argparse.ArgumentParser()
ap.add_argument('--fn', required=True)
ap.add_argument('--draft', required=True)
@@ -207,6 +207,7 @@ def consumers(fn):
def main():
import frozen; frozen.refuse("conform_decls.py", "the types phase (Phase 37)", "it conforms the fleet's declarations to engine_core.h (gone)") # Phase 35 T6: FROZEN
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--fn", required=True)
@@ -0,0 +1,3 @@
# dedup_extend.py — superseded
Superseded by `share_body.py` (see its row in the index). What it did: Extends the existing share registry to newly onboarded binaries
@@ -0,0 +1,3 @@
# dedup_propagate.py — superseded
Superseded by `share_body.py` (see its row in the index). What it did: Lifts one matched body into a shared macro and instantiates it at every duplicate site
@@ -122,6 +122,7 @@ def plan(binary, fn, draft_text):
def main():
import frozen; frozen.refuse("demacroize.py", "tools/share_body.py", "it turns a macro site back into a private copy — the opposite of the S1 invariant") # Phase 35 T6: FROZEN
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument('--binary', required=True)
@@ -1322,6 +1322,7 @@ def remap(addr, from_ov, to_ov, to_addr=None, imm_map=None):
def main():
import frozen; frozen.refuse("family_remap.py", "tools/share_body.py --reexemplar / --apply", "its command line writes a DEFINE_func_ macro head; the library (stream_words, nins_of, …) stays") # Phase 35 T6: FROZEN
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--addr", required=True)
ap.add_argument("--from", dest="frm", required=True)
@@ -876,6 +876,7 @@ def hseq_sweep(a):
def main():
import frozen; frozen.refuse("family_sweep.py", "tools/share_body.py", "it remaps and rewrites the DEFINE_func_ macro bodies Phase 35 retired") # Phase 35 T6: FROZEN
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--source", default="ov_SC01_077")
ap.add_argument("--limit", type=int, default=0, help="cap #exemplars (0 = all)")
@@ -30,7 +30,7 @@ import argparse, re, sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
import dedup_propagate as dp
dp = None # the propagate library is retired to tools/sunset (P35 T6); this tool is FROZEN — main() refuses
SYM_RE = re.compile(r'\b(func_[0-9A-Fa-f]{6,8}|D_[0-9A-Fa-f]{6,8})\b')
@@ -81,6 +81,7 @@ def inject(defblock, name, ext_map):
def main():
import frozen; frozen.refuse("inject_capped_externs.py", "tools/share_body.py", "it fixed drafts a macro-era propagation skipped, on dedup_propagate (retired)") # Phase 35 T6: FROZEN
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--overlay", default="ov_SC01_077")
@@ -191,6 +191,7 @@ def fix(tu_text, fn, ref_decl=None):
def main():
import frozen; frozen.refuse("normalize_self_decls.py", "tools/share_body.py", "it derives canonical declarations from engine_core.h (gone)") # Phase 35 T6: FROZEN
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument('--tu', required=True, help='the TU file to normalize (edited in place unless --out)')
ap.add_argument('--fn', required=True, help='func_XXXXXXXX to normalize decls of')
@@ -62,6 +62,7 @@ def free_letters(ov, n):
def main():
import frozen; frozen.refuse("o0_subsplit.py", "the Makefile -O0 globs (rollout_o0 for the record)", "it splits -O0 macro sites out of a TU against the macro form") # Phase 35 T6: FROZEN
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("ov")
@@ -71,6 +71,7 @@ def insert_at_file_scope(path, decls):
def main():
import frozen; frozen.refuse("restore_dropped_decls.py", "tools/share_body.py", "it repairs declarations a macro-era propagation deleted; share_body edits definition lines only") # Phase 35 T6: FROZEN
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--binary", required=True)
ap.add_argument("--ref", required=True, help="git ref holding the pre-deletion sources")
+2 -2
View File
@@ -348,8 +348,8 @@ def main():
f"total {banked}; fleet {fp}%")
if s.get("verified"):
log(" -> propagate as its own batch: "
+ "; ".join("tools/dedup_propagate.py --addr 0x%s --recover" % f.split('_')[-1]
for f in s["verified"][:4]))
+ "tools/share_body.py --apply --bucket new --batches 1 (P35 T6: shares every new same-address class; "
+ "verified: " + " ".join(s["verified"][:4]) + ")")
# A permuter win the whole-binary gate STILL rejects is plumbing-bound (not regalloc/sched) —
# re-permuting can never bank it. Blacklist so the grinder stops churning it (the §20 trap).
rejected = [f for f, _b in won if f not in verified]
@@ -137,7 +137,7 @@ def byte_gate(draft_dir):
def propagate():
sh([PY, "tools/dedup_propagate.py", "--auto-from", OV, "--min-reach", "2"], timeout=3600)
sh([PY, "tools/share_body.py", "--apply", "--bucket", "new", "--batches", "1"], timeout=3600) # P35 T6: the include-at-site share
def main():
@@ -264,7 +264,7 @@ def run_cycle(a, api_base, model, tried):
for b, fns in banked_by_bin.items():
if any((lg.reach_of(b, fn) or 1) >= 2 for fn in fns):
before = gate_stage._dedup_group_count()
sh([PY, "tools/dedup_propagate.py", "--auto-from", b, "--min-reach", "2"], timeout=3600)
sh([PY, "tools/share_body.py", "--apply", "--bucket", "new", "--batches", "1"], timeout=3600) # P35 T6
propagated += max(0, gate_stage._dedup_group_count() - before)
merge_backlogs()
fp = fleet_pct()
@@ -83,7 +83,7 @@ json.dump(banked,open(outp.replace('.json','_banked.json'),'w'))
# propagate per function
for fn in banked:
addr="0x"+fn.split("_")[1].lower()
r=subprocess.run([".venv/bin/python","tools/dedup_propagate.py","--addr",addr],
r=subprocess.run([".venv/bin/python","tools/share_body.py","--apply","--bucket","new","--batches","1"], # P35 T6
capture_output=True,text=True)
if dirty():
tag=os.environ.get("GATE_PHASE","decomp") # P31 T1: was hardcoded "phase-30 S49"
@@ -519,10 +519,9 @@ def _run_gate_locked(drafts, binary, src, asm, out, good_sha, propagate, source_
# cover write scope, and a killed process performs no undo at all).
_budget = min(6 * 3600, 1800 + 1800 * len(verified))
try:
pr = sh([PY, "tools/dedup_propagate.py", "--auto-from", binary, "--min-reach", "2",
"--recover"], timeout=_budget)
pr = sh([PY, "tools/share_body.py", "--apply", "--bucket", "new", "--batches", "1"], timeout=_budget) # P35 T6: the include-at-site share (dedup_propagate retired)
except subprocess.TimeoutExpired:
print(f"[gate] FATAL: dedup_propagate exceeded {_budget}s with {len(verified)} banks — "
print(f"[gate] FATAL: share_body exceeded {_budget}s with {len(verified)} banks — "
f"the fleet is HALF-PROPAGATED and the tree is DIRTY. Revert (`git checkout -- src/`), "
f"then re-gate with --no-propagate and propagate separately.", file=sys.stderr)
return {"drafts": len(draft_fns), "banked": len(verified), "propagated": 0,
@@ -152,6 +152,7 @@ def collect_stubs(c_path):
def main():
import frozen; frozen.refuse("gen_harvest_targets.py", "tools/share_census.py", "it collects harvest targets from the macro header engine_core.h (gone)") # Phase 35 T6: FROZEN
ap = argparse.ArgumentParser()
ap.add_argument('--source', default='ov_SC01_077')
ap.add_argument('--min-reach', type=int, default=2)
@@ -227,8 +227,8 @@ def main():
# periodic fleet propagate sweep (the multiplier) — every N batches
if batch_i % a.propagate_every == 0 and propagated_since:
for pb in sorted(propagated_since):
subprocess.run([".venv/bin/python", "tools/dedup_propagate.py", "--auto-from", pb,
"--min-reach", "2"], cwd=REPO, capture_output=True, timeout=3600)
subprocess.run([".venv/bin/python", "tools/share_body.py", "--apply", "--bucket", "new",
"--batches", "1"], cwd=REPO, capture_output=True, timeout=3600) # P35 T6
propagated_since.clear()
log("propagate sweep done")
# heartbeat + flywheel stats. progress.py --fleet is ~14s (a full 136-binary scan), so
@@ -69,6 +69,7 @@ def recover(fn, src, ec):
def main():
import frozen; frozen.refuse("recover_giant.py", "tools/share_body.py", "it recovers a giant into the macro header engine_core.h (gone)") # Phase 35 T6: FROZEN
if len(sys.argv) != 4:
raise SystemExit(__doc__)
fn, inp, outp = sys.argv[1], sys.argv[2], sys.argv[3]
@@ -41,7 +41,7 @@ import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import corpus
import dedup_propagate as DP
import share_body as DP # load_sig lives in share_body since dedup_propagate retired (P35 T6)
REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+5 -6
View File
@@ -8,8 +8,8 @@
> its platform SDK need the marked adaptation. The last table lists the tools that are project-only in code (their *shape* is a task;
> their code does not transfer). *TODO(platform): the MIPS and PlayStation SDK hard-codes are the ones another platform replaces first.*
>
> **Coverage:** 298 tool files in scope (submodules, vendored and downloaded code excluded), of which 298 live rows
> below; per phase: P1 2 · P2 26 · P3 17 · P4 9 · P5 27 · P6 54 · P7 20 · P8 86 · P9 27 · P10 14 · PROJECT-ONLY 16. Superseded tools appear only as pointers to their successor (28 pointer rows); one-offs are omitted. Table rows in all: 326 (the installer checks its copy against this figure).
> **Coverage:** 295 tool files in scope (submodules, vendored and downloaded code excluded), of which 295 live rows
> below; per phase: P1 2 · P2 26 · P3 17 · P4 9 · P5 26 · P6 51 · P7 20 · P8 86 · P9 27 · P10 15 · PROJECT-ONLY 16. Superseded tools appear only as pointers to their successor (30 pointer rows); one-offs are omitted. Table rows in all: 325 (the installer checks its copy against this figure).
## P1 — extraction + manifest
@@ -98,7 +98,6 @@
| `oracle_reorder.py` | decide whether a near-miss residual is a source defect or an assembler artifact | Decides whether a near-miss residual is a source defect or an assembler artifact | pinned assembler flags |
| `tool_census.py` | derive the tool census, the need-keyed index, the kit manifest and the verbatim corpora from one dictionary | Derives the tool census, the need-keyed tool index, the kit manifest and the verbatim corpora from one dictionary, coverage-asserted both ways | repo layout (tools/, docs/SETUP.md rows, the kit paths) |
| `wall_sweep.py` | enumerate fleet-wide every instance of one known assembler-level blocker class | Enumerates fleet-wide instances of one known assembler-level blocker class | repo asm layout, toolchain quirk |
| `test_reconcile_ledger.py` | exercise a guarded repair path the full negative control never reaches | Targeted proof of the propagation ledger guard the full control never exercised | repo paths |
| `blocker_probe.py` | explain with two oracles why a byte-correct draft fails the whole-binary gate | Read-only two-oracle explanation of why a byte-correct draft fails the whole-binary gate | repo build paths |
| `atlas_features.py` | extract one deterministic feature record per function across every registered binary | Extracts one deterministic feature record per function across all registered binaries | repo signature files |
| `worklist.py` | join the target pool and near-miss ledger into one byte-weighted queue | Joins the target pool and the near-miss ledger into one byte-weighted ranked queue | repo scratch paths |
@@ -138,7 +137,6 @@
| `pads_audit.py` | derive an object's inter-table padding spec from the bytes instead of searching | Derives each object's padding spec from the bytes instead of searching for it | repo build/config layout |
| `aprop_autodraft.py` | draft a family member mechanically from a seed body and symbol rebase | Mechanically drafts a family member from the seed body plus a positional symbol rebase | repo scratch paths |
| `demacroize.py` | escape a shared-header declaration conflict locally by unmacroizing one body | Per-binary local escape from a shared-header declaration conflict by unmacroizing one body | repo shared-header layout |
| `dedup_extend.py` | extend an existing code-share registry to newly onboarded binaries | Extends the existing share registry to newly onboarded binaries | repo config/src layout |
| `o0_boundary.py` | find and bank unoptimized functions stranded at an optimized region's boundary | Finds and banks unoptimized functions stranded at the end of an unoptimized region | repo config layout |
| `inject_capped_externs.py` | free duplicate matches a share propagation skips as not self-contained | Frees high-reach inline matches skipped as not self-contained by injecting their file-scope externs | repo shared-header layout |
| `decl_from_use.py` | infer a minimal extern for a data symbol the destination unit does not declare | Infers a minimal extern for a data symbol a draft uses but its destination does not declare | repo src layout |
@@ -146,8 +144,6 @@
| `match_protos.py` | join two related builds' signature dumps into a function correspondence | Joins per-function signature dumps of two related builds into a function correspondence | repo signature files |
| `cast_self_callers.py` | let a unit keep calling, through per-site casts, the function it now defines | Lets a unit keep calling, through a per-site cast, the function it is about to define | repo src layout |
| `lift_types.py` | lift a named list of types fleet-wide into the shared type header — the canonical type layer from the first bank (Phase 6, the multiplier); again at readability (Phase 10) | Lifts a named list of types fleet-wide into the shared type header | repo shared-header path |
| `dedup_propagate.py` | lift one matched body into a shared macro and instantiate it everywhere | Lifts one matched body into a shared macro and instantiates it at every duplicate site | repo shared-header/src layout |
| `macro_draft.py` | materialize a shared macro body back into a compilable standalone draft | Materializes a shared macro body back into a compilable standalone draft | repo shared-header layout |
| `scope_tu_externs.py` | move a unit's file-scope data externs down into their consumers | Moves a unit's own file-scope data externs down into their consumers to legalize a block-scope type | repo src layout |
| `family_manifest.py` | name the one exemplar to draft per structural family, ranked | Ranked structural-family target manifest naming the one exemplar to draft per family | repo signature files |
| `normalize_self_decls.py` | normalize a function's own declaration when a templated definition lands in a new unit | Normalizes a function's own declaration when a templated definition lands in a new unit | repo src layout |
@@ -338,6 +334,7 @@
| `verbatim_check.py` | guard that every inline-assembly body still reproduces its target bytes | Regression guard that every inline-assembly body still reproduces its target bytes | repo src layout |
| `share_census.py` | measure duplicate function bodies across the fleet and assert one source per unique function | The census of byte-identical function classes across every binary with their source forms and verdicts, plus the S1 invariant check with its exception ledger and a fixture self-test | repo paths, the registry and signature schemas |
| `ghidra_apply_symbols.sh` | mirror the curated symbol file into the analysis database with a real save | Mirrors the curated symbol file into the analysis program headlessly, with a real save | repo symbol path, project name |
| `frozen.py` | refuse, from one place, the command line of a tool the project has frozen | The one refusal a FROZEN tool prints from its main(): the tool, the successor and why; imports never exit (libraries stay usable) | nothing |
| `verbatim_target_s.py` | regenerate a splitter-format target disassembly for a function no longer stubbed | Regenerates a splitter-format target disassembly for a function that is no longer a stub; --gas emits the assemblable gas-syntax form ($-registers, .L labels, noreorder) that a web diff service accepts as a pasted target | repo build/asm layout |
| `share_body_cycle.sh` | run the share-body batch cycle unattended: batch, verify by exit code, log, commit, periodic clean fleet check | The Phase-35 T5 driver for bucket new: per batch share_body --apply --bucket new --batches 1 --label new<k>, the gated N/N line read, the phase log entry appended, the bank committed (R42), the clean fleet run every N batches (R22) — stops on the first red | repo paths, the phase-log format, the fleet count from config/check.*.sha |
| `share_body.py` | share one byte-identical function class across its binaries through an include-at-site header, gated per binary | The Phase-35 successor of dedup_propagate + dedup_extend for the include-at-site form: exemplar by majority text, the header written once, every private copy replaced by the include at its position, per-binary byte gate with object comparison, bisect on red, the registry appended or extended by text, the exception ledger on refusal; --plan / --apply --bucket extend|new | repo paths, the registry and signature schemas |
@@ -374,6 +371,8 @@
| `build_wave.py` | `draw_waves.py` | Builds the next wave from a card pool, ranked by fleet leverage |
| `build_wave_args.py` | `wave_args.py` | Emits workflow arguments for a wave from the target manifest, resolving each path |
| `canon_draft_decls.py` | `sig_unify.py` | Canonicalizes a draft's extern and data declarations to the banked-consistent set |
| `dedup_extend.py` | `share_body.py` | Extends the existing share registry to newly onboarded binaries |
| `dedup_propagate.py` | `share_body.py` | Lifts one matched body into a shared macro and instantiates it at every duplicate site |
| `exemplar_miner.py` | `atlas.py` | Routes every residual stub to its lever and ranks the highest-reach work |
| `fix_header_decl.py` | `conform_decls.py` | Rewrites a shared-header caller declaration to a draft's byte-true signature |
| `gen_engine_decls.py` | `conform_decls.py` | Declares every shared function and data symbol once, canonically, in a generated header |
+22 -21
View File
@@ -5,7 +5,7 @@ freshness). The derived columns come from the tree on every run; the authored on
dictionary, whose coverage is asserted both ways. Read it by NEED: find the phrase that matches what you are trying to do, then the tool,
then what proved it. The same data generates the day-one kit's manifest and its verbatim tool corpus.*
**Coverage:** 298 tool files in scope (submodules, vendored and downloaded code excluded; `find` and `git ls-files` agree) + 34 retired under `tools/sunset/`. Classes: LIVE 237 (a runtime consumer), REFERENCED 29 (a SETUP row only), ORPHAN 32 (neither) — of 298. Portability: PORTABLE 22, ADAPT 259, PROJECT-ONLY 17.
**Coverage:** 295 tool files in scope (submodules, vendored and downloaded code excluded; `find` and `git ls-files` agree) + 38 retired under `tools/sunset/`. Classes: LIVE 236 (a runtime consumer), REFERENCED 29 (a SETUP row only), ORPHAN 30 (neither) — of 295. Portability: PORTABLE 23, ADAPT 255, PROJECT-ONLY 17.
## P1 — extraction + manifest
@@ -85,7 +85,7 @@ then what proved it. The same data generates the day-one kit's manifest and its
| When you need to… | Tool | What it does | Proven by | Adapt | Class |
|---|---|---|---|---|---|
| answer from one derived model whether each function is open, matched or shared | `corpus.py` | The single derived model of the source tree: open, matched or shared, per function | Makefile, api_agent.py, aprop_autodraft.py, aprop_symfix.py (+67) | repo src/config layout | LIVE |
| answer from one derived model whether each function is open, matched or shared | `corpus.py` | The single derived model of the source tree: open, matched or shared, per function | Makefile, api_agent.py, aprop_autodraft.py, aprop_symfix.py (+66) | repo src/config layout | LIVE |
| assert every consumer knows about each newly onboarded binary | `audit_binaries.py` | Gate asserting every consumer knows about each newly onboarded binary | Makefile | repo config/registry paths | LIVE |
| assert every tracked source is plain text so text searches never skip it | `audit_text_sources.py` | Every tracked C source must be plain text, or text searches silently skip it | .github/workflows/no-rom.yml, Makefile | repo src paths | LIVE |
| check that the independent what-remains views agree with the corpus oracle | `audit_frontier.py` | Checks that the independent "what remains" views agree with the corpus oracle | Makefile | repo paths | LIVE |
@@ -94,7 +94,6 @@ then what proved it. The same data generates the day-one kit's manifest and its
| decide whether a near-miss residual is a source defect or an assembler artifact | `oracle_reorder.py` | Decides whether a near-miss residual is a source defect or an assembler artifact | Makefile, docs/wave-playbook.md, match_one.py, reorder_passthrough.py (+1) | pinned assembler flags | LIVE |
| derive the tool census, the need-keyed index, the kit manifest and the verbatim corpora from one dictionary | `tool_census.py` | Derives the tool census, the need-keyed tool index, the kit manifest and the verbatim corpora from one dictionary, coverage-asserted both ways | Makefile | repo layout (tools/, docs/SETUP.md rows, the kit paths) | LIVE |
| enumerate fleet-wide every instance of one known assembler-level blocker class | `wall_sweep.py` | Enumerates fleet-wide instances of one known assembler-level blocker class | docs/wave-playbook.md, residual_rules_b.py, triage_ladder.py | repo asm layout, toolchain quirk | LIVE |
| exercise a guarded repair path the full negative control never reaches | `test_reconcile_ledger.py` | Targeted proof of the propagation ledger guard the full control never exercised | — | repo paths | ORPHAN |
| explain with two oracles why a byte-correct draft fails the whole-binary gate | `blocker_probe.py` | Read-only two-oracle explanation of why a byte-correct draft fails the whole-binary gate | recover_integration.py | repo build paths | LIVE |
| extract one deterministic feature record per function across every registered binary | `atlas_features.py` | Extracts one deterministic feature record per function across all registered binaries | Makefile, atlas.py, len_tells.py | repo signature files | LIVE |
| join the target pool and near-miss ledger into one byte-weighted queue | `worklist.py` | Joins the target pool and the near-miss ledger into one byte-weighted ranked queue | build_fuel_manifest.py | repo scratch paths | LIVE |
@@ -120,7 +119,7 @@ then what proved it. The same data generates the day-one kit's manifest and its
| answer from a prologue whether a function was built unoptimized | `o0_detect.py` | The single place that answers whether a function was built unoptimized, from its prologue | match_one.py, neighbor_ref.py, o0_boundary.py, residual_rules_b.py (+2) | compiler prologue shape | LIVE |
| apply one proven codegen lever everywhere the bytes say it belongs | `weave_sweep.py` | Applies one known prologue-scheduling lever everywhere the bytes say it belongs | — | repo src layout, compiler behaviour | REFERENCED |
| assert a read-only-data carve's interleave order equals the segment sequence | `interleave_check.py` | Asserts the carve interleave order equals the segment sequence, position by position | — | repo config layout | REFERENCED |
| bank a matched jump-table exemplar across its siblings, reverting on failure | `jtbl_family_bank.py` | Banks a matched jump-table exemplar across its structural siblings, revert-on-fail | dedup_extend.py, family_sweep.py, reloc_verify.py | repo config/src layout | LIVE |
| bank a matched jump-table exemplar across its siblings, reverting on failure | `jtbl_family_bank.py` | Banks a matched jump-table exemplar across its structural siblings, revert-on-fail | family_sweep.py, reloc_verify.py | repo config/src layout | LIVE |
| bank a refused draft by copying the destination unit's own declarations into it — the declaration-conflict class a canonical type layer prevents (Phase 6) | `sync_tu_decls.py` | Banks a refused draft by copying the destination unit's own declarations into it | docs/wave-playbook.md, recover_route.py | repo src layout | LIVE |
| bank every open stub that already has a banked structural twin | `twin_sweep.py` | Banks every open stub that has an already-banked structural twin, for near-zero tokens | docs/wave-playbook.md | repo signature files | LIVE |
| bind a void definition to a symbol the destination declares as returning a value | `fix_decl_mirror.py` | Binds a void definition to its symbol when the destination declares a value return | — | repo src layout | ORPHAN |
@@ -129,12 +128,11 @@ then what proved it. The same data generates the day-one kit's manifest and its
| cluster the open frontier one tier looser than exact skeleton hashing | `family_cousins.py` | Similarity clustering one tier looser than exact skeleton hashing over the open frontier | Makefile, aprop_autodraft.py, atlas.py, atlas_features.py (+3) | repo signature files | LIVE |
| compute the fleet's consensus declaration for every symbol, as card fuel — the seed of a canonical type layer (Phase 6) | `decl_prior.py` | Computes the fleet's consensus declaration for every symbol, as card fuel | build_wave_atlas.py, family_remap.py, t5_cards.py | repo src layout | LIVE |
| conform a draft's data declarations to what its destination unit can see | `reconcile_tu.py` | Conforms a draft's data declarations to what the destination translation unit can actually see | gate_stage.py, harvest_verify.py, integration_resolver.py, jtbl_family_bank.py | repo src layout | LIVE |
| conform every declaration of a function fleet-wide to its byte-true definition — width and signedness proven by the bytes at bank time (Phase 6) | `conform_decls.py` | Conforms every declaration of a function fleet-wide to its byte-true definition | family_sweep.py, macro_draft.py | repo src/shared-header layout | LIVE |
| conform every declaration of a function fleet-wide to its byte-true definition — width and signedness proven by the bytes at bank time (Phase 6) | `conform_decls.py` | Conforms every declaration of a function fleet-wide to its byte-true definition | family_sweep.py | repo src/shared-header layout | LIVE |
| demote declaration scope as one rung of the recovery ladder | `scope_demote_drafts.py` | The scope-demote step, wired as a rung of the recovery ladder | gate_stage.py, recover_route.py | repo src layout | LIVE |
| derive an object's inter-table padding spec from the bytes instead of searching | `pads_audit.py` | Derives each object's padding spec from the bytes instead of searching for it | — | repo build/config layout | REFERENCED |
| draft a family member mechanically from a seed body and symbol rebase | `aprop_autodraft.py` | Mechanically drafts a family member from the seed body plus a positional symbol rebase | decl_from_use.py, maintenance.sh, sibling_lane.sh | repo scratch paths | LIVE |
| escape a shared-header declaration conflict locally by unmacroizing one body | `demacroize.py` | Per-binary local escape from a shared-header declaration conflict by unmacroizing one body | recover_integration.py | repo shared-header layout | LIVE |
| extend an existing code-share registry to newly onboarded binaries | `dedup_extend.py` | Extends the existing share registry to newly onboarded binaries | audit_binaries.py, share_body.py | repo config/src layout | LIVE |
| find and bank unoptimized functions stranded at an optimized region's boundary | `o0_boundary.py` | Finds and banks unoptimized functions stranded at the end of an unoptimized region | — | repo config layout | REFERENCED |
| free duplicate matches a share propagation skips as not self-contained | `inject_capped_externs.py` | Frees high-reach inline matches skipped as not self-contained by injecting their file-scope externs | — | repo shared-header layout | REFERENCED |
| infer a minimal extern for a data symbol the destination unit does not declare | `decl_from_use.py` | Infers a minimal extern for a data symbol a draft uses but its destination does not declare | aprop_autodraft.py, integration_resolver.py, residual_rules_b.py | repo src layout | LIVE |
@@ -142,21 +140,19 @@ then what proved it. The same data generates the day-one kit's manifest and its
| join two related builds' signature dumps into a function correspondence | `match_protos.py` | Joins per-function signature dumps of two related builds into a function correspondence | dup_report.py | repo signature files | LIVE |
| let a unit keep calling, through per-site casts, the function it now defines | `cast_self_callers.py` | Lets a unit keep calling, through a per-site cast, the function it is about to define | docs/wave-playbook.md, recover_integration.py, recover_route.py | repo src layout | LIVE |
| lift a named list of types fleet-wide into the shared type header — the canonical type layer from the first bank (Phase 6, the multiplier); again at readability (Phase 10) | `lift_types.py` | Lifts a named list of types fleet-wide into the shared type header | build_engine_types.py, uniquify_type.py | repo shared-header path | LIVE |
| lift one matched body into a shared macro and instantiate it everywhere | `dedup_propagate.py` | Lifts one matched body into a shared macro and instantiates it at every duplicate site | auto_driver.py, bulk_harvest.py, gate_lane.py, gate_stage.py (+5) | repo shared-header/src layout | LIVE |
| materialize a shared macro body back into a compilable standalone draft | `macro_draft.py` | Materializes a shared macro body back into a compilable standalone draft | — | repo shared-header layout | ORPHAN |
| move a unit's file-scope data externs down into their consumers | `scope_tu_externs.py` | Moves a unit's own file-scope data externs down into their consumers to legalize a block-scope type | family_sweep.py, jtbl_family_bank.py, recover_integration.py | repo src layout | LIVE |
| name the one exemplar to draft per structural family, ranked | `family_manifest.py` | Ranked structural-family target manifest naming the one exemplar to draft per family | family_hseq.py | repo signature files | LIVE |
| normalize a function's own declaration when a templated definition lands in a new unit | `normalize_self_decls.py` | Normalizes a function's own declaration when a templated definition lands in a new unit | family_sweep.py | repo src layout | LIVE |
| parse C declarators once, for every declaration tool to share | `cdecl.py` | The single recursive-descent C declarator parser every declaration tool shares | Makefile, blocker_probe.py, canon_sig_reconcile.py, cast_call_sites.py (+21) | C89 grammar only | LIVE |
| parse C declarators once, for every declaration tool to share | `cdecl.py` | The single recursive-descent C declarator parser every declaration tool shares | Makefile, blocker_probe.py, canon_sig_reconcile.py, cast_call_sites.py (+20) | C89 grammar only | LIVE |
| partition a source file at object boundaries, aware of comment state | `overlay_src_split.py` | Comment-state-aware partition of a source file, the oracle for the split chain | jr_isolate_all.py, neighbor_ref.py, o0_subsplit.py, share_census.py (+1) | repo src layout | LIVE |
| place a templated body's data externs at a scope the destination accepts | `scope_data_externs.py` | Places a templated body's data externs at the scope the destination unit can accept | family_sweep.py, jtbl_family_bank.py, scope_demote_drafts.py, scope_tu_externs.py | repo src layout | LIVE |
| property-test the signature tool on committed fixtures, needing no compiler | `xsig/tests/test_xsig.py` | Property tests for the signature tool on committed fixtures, needing no compiler | .github/workflows/no-rom.yml, Makefile | fixture paths | LIVE |
| rank clusters of the unmatched frontier by mnemonic-skeleton hash | `family_hseq.py` | Ranked clustering of the unmatched frontier by mnemonic-skeleton hash | Makefile, audit_binaries.py, audit_frontier.py, build_wave_atlas.py (+6) | repo signature files | LIVE |
| rank clusters of the unmatched frontier by mnemonic-skeleton hash | `family_hseq.py` | Ranked clustering of the unmatched frontier by mnemonic-skeleton hash | Makefile, audit_binaries.py, audit_frontier.py, build_wave_atlas.py (+5) | repo signature files | LIVE |
| rebase a remapped draft's stale seed symbols onto the target's own | `aprop_symfix.py` | Rebases stale seed symbols in a mechanically adapted draft onto the target's own symbols | aprop_autodraft.py, family_cousins.py, integration_resolver.py, recover_rejects.py | repo symbol/config sources | LIVE |
| reconcile a definition's typed signature with what its destination unit declares — part of banking against a canonical type layer from Phase 6; again at readability (Phase 10) | `canon_sig_reconcile.py` | Reconciles a definition's typed signature with what its destination unit already declares | family_sweep.py, jtbl_family_bank.py | repo src layout | LIVE |
| regenerate the signature tool's fixtures with the pinned toolchain | `xsig/tests/make_fixtures.sh` | Regenerates the signature-tool fixtures with the pinned toolchain | test_xsig.py | compiler triple, repo tool paths | LIVE |
| remap a matched exemplar's source onto a structural sibling by positional symbol pairing | `family_remap.py` | Mechanically remaps a matched exemplar's C onto a structural sibling by positional symbol pairing | aprop_autodraft.py, atlas.py, atlas_features.py, dedup_extend.py (+17) | repo src layout | LIVE |
| repair a shared caller's argument-count conflict for the no-prototype failure class | `fix_arity_callers.py` | Repairs the shared-caller argument-count conflict for the no-prototype failure class | docs/wave-playbook.md, cdecl.py, dedup_propagate.py, gate_stage.py (+2) | repo shared-header layout | LIVE |
| remap a matched exemplar's source onto a structural sibling by positional symbol pairing | `family_remap.py` | Mechanically remaps a matched exemplar's C onto a structural sibling by positional symbol pairing | aprop_autodraft.py, atlas.py, atlas_features.py, diff_regions.py (+16) | repo src layout | LIVE |
| repair a shared caller's argument-count conflict for the no-prototype failure class | `fix_arity_callers.py` | Repairs the shared-caller argument-count conflict for the no-prototype failure class | docs/wave-playbook.md, cdecl.py, gate_stage.py, recover_integration.py (+1) | repo shared-header layout | LIVE |
| repair a stale padding spec by search plus byte proof, never by guessing | `jtbl_pads_fix.py` | Repairs a stale padding spec by search plus byte proof, never by guessing | maintenance.sh | repo config layout | LIVE |
| report which open stubs became mechanical remaps the moment one was banked | `twin_rescan.py` | After a bank, reports which open stubs just became mechanical remaps instead of drafts | — | repo signature files | REFERENCED |
| reproduce the original inter-table padding in a multi-table carve | `jtbl_rodata_pads.py` | Post-assembler filter reproducing the original inter-table padding in a multi-table carve | Makefile, compile_only.py, gate_main.py, jtbl_carve.py | compiler/assembler behaviour, repo build paths | LIVE |
@@ -166,11 +162,11 @@ then what proved it. The same data generates the day-one kit's manifest and its
| roll an unoptimized definition atomically across every binary carrying it | `rollout_o0.py` | Generalized two-file atomic rollout of an unoptimized definition across every binary carrying it | Makefile, match_one.py | repo src/config layout | LIVE |
| route an unoptimized address range sitting inside an otherwise optimized object | `o0_subsplit.py` | Routes an unoptimized address range that sits inside an otherwise optimized object | match_one.py | repo config layout | LIVE |
| set up the read-only-data carve for a binary's matched jump-table functions | `jtbl_carve.py` | Sets up the read-only-data carve for a binary's matched jump-table functions | Makefile, config/overlays.mk, build_wave_atlas.py, harvest_verify.py (+7) | repo config/build layout | LIVE |
| sign every function of a flat image at a known base, without a disassembler | `sig_image.py` | Disassembler-free per-function signer for a flat image at a known base, field-identical to the dumper | Makefile, atlas_features.py, corpus.py, dedup_extend.py (+6) | MIPS encodings, repo scratch paths | LIVE |
| sign every function of a flat image at a known base, without a disassembler | `sig_image.py` | Disassembler-free per-function signer for a flat image at a known base, field-identical to the dumper | Makefile, atlas_features.py, corpus.py, disc_audit.py (+5) | MIPS encodings, repo scratch paths | LIVE |
| sign functions with relocations masked for cross-project code identification | `xsig/xsig.py` | Relocation-masked per-function signatures for cross-project code identification | .github/workflows/no-rom.yml, Makefile, test_xsig.py | MIPS/relocation model only | LIVE |
| sweep a matched exemplar across every same-structure sibling, gate-arbitrated | `family_sweep.py` | Sweeps a matched exemplar across every same-structure sibling, gate-arbitrated | recover_integration.py, twin_sweep.py | repo src/config layout | LIVE |
| sweep a matched exemplar across every same-structure sibling, gate-arbitrated | `family_sweep.py` | Sweeps a matched exemplar across every same-structure sibling, gate-arbitrated | frozen.py, recover_integration.py, twin_sweep.py | repo src/config layout | LIVE |
| unify a draft's callee externs and its own signature to the banked-canonical set | `sig_unify.py` | Unifies a draft's callee externs and its own definition signature to the banked-canonical set | auto_driver.py, cast_call_sites.py, cdecl.py, gate_stage.py (+1) | repo shared-header paths | LIVE |
| validate a code-share registry byte-honestly, failing closed on signature drift | `dedup_integrate.py` | Byte-honesty validator for the code-share registry; fails closed on signature drift | Makefile, dedup_propagate.py, macro_to_header.py, progress.py (+1) | repo config registry path | LIVE |
| validate a code-share registry byte-honestly, failing closed on signature drift | `dedup_integrate.py` | Byte-honesty validator for the code-share registry; fails closed on signature drift | Makefile, macro_to_header.py, progress.py, share_body.py (+1) | repo config registry path | LIVE |
## P7 — the codegen map, the dumps, the permuter
@@ -234,7 +230,7 @@ then what proved it. The same data generates the day-one kit's manifest and its
| generate a callee-signature-aware target manifest with per-function fleet reach | `gen_harvest_targets.py` | Generates a callee-signature-aware target manifest with per-function fleet reach | cast_call_sites.py, cdecl.py, sig_unify.py | canonical source binary, repo paths | LIVE |
| give a wave its own immutable copy of the disassembly its agents read | `wave_snapshot.py` | Gives a wave its own immutable copy of the disassembly files its agents read | build_wave_atlas.py | repo asm paths | LIVE |
| give an external chat model the same multi-turn tool harness an agent gets | `api_agent.py` | Gives an external chat model the same multi-turn tool harness an agent gets | claude_wave_packs.py, idiom_serial.py, jtbl_lane.py, ox_campaign.py | provider endpoint, repo doc paths | LIVE |
| hold one reader/writer lock over the shared source state for concurrent lanes | `shared_lock.py` | One reader/writer lock over the shared source state for concurrent lanes | bulk_harvest.py, dedup_propagate.py, fix_arity_callers.py, gate_stage.py | repo lock paths | LIVE |
| hold one reader/writer lock over the shared source state for concurrent lanes | `shared_lock.py` | One reader/writer lock over the shared source state for concurrent lanes | bulk_harvest.py, fix_arity_callers.py, gate_stage.py | repo lock paths | LIVE |
| keep an unattended campaign loop alive across run caps and crashes | `lanes/forever.sh` | Keeps the wave campaign running indefinitely across caps and crashes | — | repo scratch paths | ORPHAN |
| keep an unattended driver alive across crashes until a clean exit | `auto_supervisor.sh` | Keeps an unattended driver alive across crashes until a clean or requested exit | grinder.py, lora_grind.py | repo scratch paths | LIVE |
| keep drafting running continuously, touching only scratch state | `lanes/drafter.sh` | The drafting lane that must never stop; touches only scratch state | relaunch_drafter_shell.sh | repo scratch paths, provider config | LIVE |
@@ -274,14 +270,14 @@ then what proved it. The same data generates the day-one kit's manifest and its
| run the executable's own draft, gate and commit cadence beside other lanes | `main_lane.py` | The executable's own draft, gate and commit cadence beside the other lanes | main.sh | repo build/make targets | LIVE |
| run the executable's own lane loop on its clean-rebuild cadence | `lanes/main.sh` | The executable's own lane loop, on its clean-rebuild cadence | gate_stage.py, restart_main_lane_when_idle.sh | repo make targets | LIVE |
| run the free recovery and re-judgement passes whenever the gate is idle | `lanes/maintenance.sh` | The maintenance lane running the free recovery and re-judgement passes | — | repo scratch paths | ORPHAN |
| run the one shared recovery, byte-gate and log ladder every producer calls | `gate_stage.py` | The shared deterministic recovery, gate and log ladder every producer calls | docs/wave-playbook.md, api_agent.py, bulk_harvest.py, dedup_extend.py (+12) | repo src/config layout | LIVE |
| run the one shared recovery, byte-gate and log ladder every producer calls | `gate_stage.py` | The shared deterministic recovery, gate and log ladder every producer calls | docs/wave-playbook.md, api_agent.py, bulk_harvest.py, fix_tu_ret_decls.py (+11) | repo src/config layout | LIVE |
| run the token-free permuter grinder beside a saturated gate | `lanes/grinder_lane.sh` | Runs the model-free permuter grinder beside a saturated gater | — | repo scratch paths | ORPHAN |
| run two drafting pools against one serial gate | `lanes/campaign.sh` | Dual-pool campaign lane: two drafting pools, one serial gate | forever.sh | provider config, repo paths | LIVE |
| save each subagent's full final report to one file per target | `agent_reports.py` | Saves each subagent's full final prose report to one file per target | docs/wave-playbook.md, transcript_dump.py | agent transcript paths | LIVE |
| select a worker wave's target batch from the manifest with cache filters | `wave_targets.py` | Selects a worker-wave target batch from the target manifest with cache filters | build_fuel_manifest.py, gate_stage.py, orchestrator.py | repo scratch paths | LIVE |
| serve a fine-tuned model locally behind a chat-completions endpoint | `serve_local.py` | Serves the fine-tuned model locally behind a chat-completions endpoint | bulk_harvest.py | training venv, GPU assumptions | LIVE |
| spend spare provider capacity on other work when the gate is the constraint | `lanes/elastic.sh` | Spends spare provider capacity on extra work when the main lane cannot | — | provider config, repo paths | ORPHAN |
| substitute, build, keep only if byte-identical, else revert | `harvest_verify.py` | The whole-binary byte gate: substitute, build, keep only if identical, else revert | auto_driver.py, dedup_extend.py, family_sweep.py, gate_stage.py (+5) | repo src/build layout | LIVE |
| substitute, build, keep only if byte-identical, else revert | `harvest_verify.py` | The whole-binary byte gate: substitute, build, keep only if identical, else revert | auto_driver.py, family_sweep.py, gate_stage.py, gater_lane.py (+4) | repo src/build layout | LIVE |
| turn mined pairs into a chat-template instruction dataset | `format_finetune.py` | Turns the mined pairs into a chat-template instruction dataset | api_draft.py, train_lora.py | dataset paths, model chat template | LIVE |
| turn pre-gate rejects back into bankable drafts for zero model tokens | `recover_rejects.py` | Turns pre-gate rejects back into bankable drafts for zero model tokens | maintenance.sh | repo scratch paths | LIVE |
| unify live stubs into one ranked, class-tagged, cache-annotated target pool | `build_fuel_manifest.py` | Unifies live stubs into one ranked, class-tagged, cache-annotated target pool | orchestrator.py, worklist.py | canonical source binary, repo scratch paths | LIVE |
@@ -332,11 +328,12 @@ then what proved it. The same data generates the day-one kit's manifest and its
| flag address-named references whose address now has a curated name | `lint_symbol_refs.py` | Flags address-named references in committed sources whose address now has a curated name | .github/workflows/no-rom.yml, Makefile | repo symbol/src paths | LIVE |
| give each conflicting camp of a same-named type its own name | `uniquify_type.py` | Gives each conflicting camp of a same-named type its own name so every camp becomes liftable | — | repo src layout | REFERENCED |
| guard that every inline-assembly body still reproduces its target bytes | `verbatim_check.py` | Regression guard that every inline-assembly body still reproduces its target bytes | .github/workflows/no-rom.yml, verbatim_target_s.py, verbatim_to_stub.py | repo src layout | LIVE |
| measure duplicate function bodies across the fleet and assert one source per unique function | `share_census.py` | The census of byte-identical function classes across every binary with their source forms and verdicts, plus the S1 invariant check with its exception ledger and a fixture self-test | audit_binaries.py, dedup_integrate.py, macro_to_header.py, overlay_src_split.py (+2) | repo paths, the registry and signature schemas | LIVE |
| measure duplicate function bodies across the fleet and assert one source per unique function | `share_census.py` | The census of byte-identical function classes across every binary with their source forms and verdicts, plus the S1 invariant check with its exception ledger and a fixture self-test | audit_binaries.py, dedup_integrate.py, gen_harvest_targets.py, macro_to_header.py (+3) | repo paths, the registry and signature schemas | LIVE |
| mirror the curated symbol file into the analysis database with a real save | `ghidra_apply_symbols.sh` | Mirrors the curated symbol file into the analysis program headlessly, with a real save | — | repo symbol path, project name | REFERENCED |
| refuse, from one place, the command line of a tool the project has frozen | `frozen.py` | The one refusal a FROZEN tool prints from its main(): the tool, the successor and why; imports never exit (libraries stay usable) | aprop_autodraft.py, blocker_probe.py, canon_sig_reconcile.py, conform_decls.py (+10) | nothing | LIVE |
| regenerate a splitter-format target disassembly for a function no longer stubbed | `verbatim_target_s.py` | Regenerates a splitter-format target disassembly for a function that is no longer a stub; --gas emits the assemblable gas-syntax form ($-registers, .L labels, noreorder) that a web diff service accepts as a pasted target | decompme_replica.sh | repo build/asm layout | LIVE |
| run the share-body batch cycle unattended: batch, verify by exit code, log, commit, periodic clean fleet check | `share_body_cycle.sh` | The Phase-35 T5 driver for bucket new: per batch share_body --apply --bucket new --batches 1 --label new<k>, the gated N/N line read, the phase log entry appended, the bank committed (R42), the clean fleet run every N batches (R22) — stops on the first red | — | repo paths, the phase-log format, the fleet count from config/check.*.sha | ORPHAN |
| share one byte-identical function class across its binaries through an include-at-site header, gated per binary | `share_body.py` | The Phase-35 successor of dedup_propagate + dedup_extend for the include-at-site form: exemplar by majority text, the header written once, every private copy replaced by the include at its position, per-binary byte gate with object comparison, bisect on red, the registry appended or extended by text, the exception ledger on refusal; --plan / --apply --bucket extend|new | dedup_extend.py, share_body_cycle.sh, share_census.py | repo paths, the registry and signature schemas | LIVE |
| share one byte-identical function class across its binaries through an include-at-site header, gated per binary | `share_body.py` | The Phase-35 successor of dedup_propagate + dedup_extend for the include-at-site form: exemplar by majority text, the header written once, every private copy replaced by the include at its position, per-binary byte gate with object comparison, bisect on red, the registry appended or extended by text, the exception ledger on refusal; --plan / --apply --bucket extend|new | aprop_autodraft.py, audit_binaries.py, auto_driver.py, blocker_probe.py (+17) | repo paths, the registry and signature schemas | LIVE |
| turn an inline-assembly body back into a stub the toolchain can reach | `verbatim_to_stub.py` | Turns an inline-assembly body back into an include-assembly stub | — | repo src/asm layout | REFERENCED |
## PROJECT-ONLY — project-only in code (the shape is a task; the code does not transfer)
@@ -354,7 +351,7 @@ then what proved it. The same data generates the day-one kit's manifest and its
| list a sound library's address window and per-address exclusions | `make_snd_used.py` | Hard-codes one executable's sound-library address window and per-address exclusions | Makefile, fetch_psyq.sh | project-only | LIVE |
| list the interface library's objects and address window the executable actually links | `make_apicard_used.py` | Hard-codes one executable's interface-library address window and object roster | Makefile, fetch_psyq.sh | project-only | LIVE |
| observe a live loader's requests to learn which payload loads where | `cdtrace.py` | Reads one game's loader request structures at fixed RAM addresses | — | project-only | ORPHAN |
| onboard one payload by its container and file-index naming | `new_overlay.sh` | Takes one game's container and file-index naming as its command-line contract | Makefile, config/overlays.mk, dedup_extend.py, dup_report.py (+3) | project-only | LIVE |
| onboard one payload by its container and file-index naming | `new_overlay.sh` | Takes one game's container and file-index naming as its command-line contract | Makefile, config/overlays.mk, dup_report.py, family_remap.py (+2) | project-only | LIVE |
| parse a bespoke archive-chain header format | `bfm_extract/pac.py` | Parser for one game's bespoke archive-chain header format | crosscheck.py, extract.py | project-only | LIVE |
| read a bespoke container's table of contents to locate its files | `bfm_extract/cd_archive.py` | Reader for one game's bespoke container table-of-contents format | crosscheck.py, extract.py | project-only | LIVE |
| unit-test a bespoke compression decoder against known cases | `bfm_extract/test_lzss.py` | Unit tests for that game-specific compression variant | .github/workflows/no-rom.yml | project-only | LIVE |
@@ -370,6 +367,8 @@ then what proved it. The same data generates the day-one kit's manifest and its
| `build_wave.py` | SUPERSEDED | tools/draw_waves.py | Builds the next wave from a card pool, ranked by fleet leverage |
| `build_wave_args.py` | SUPERSEDED | tools/wave_args.py | Emits workflow arguments for a wave from the target manifest, resolving each path |
| `canon_draft_decls.py` | SUPERSEDED | tools/sig_unify.py | Canonicalizes a draft's extern and data declarations to the banked-consistent set |
| `dedup_extend.py` | SUPERSEDED | tools/share_body.py | Extends the existing share registry to newly onboarded binaries |
| `dedup_propagate.py` | SUPERSEDED | tools/share_body.py | Lifts one matched body into a shared macro and instantiates it at every duplicate site |
| `exemplar_miner.py` | SUPERSEDED | tools/atlas.py | Routes every residual stub to its lever and ranks the highest-reach work |
| `fix_header_decl.py` | SUPERSEDED | tools/conform_decls.py | Rewrites a shared-header caller declaration to a draft's byte-true signature |
| `gen_engine_decls.py` | SUPERSEDED | tools/conform_decls.py | Declares every shared function and data symbol once, canonically, in a generated header |
@@ -381,6 +380,7 @@ then what proved it. The same data generates the day-one kit's manifest and its
| `lanes/launch_ox.sh` | ONE-OFF | the sharded overnight campaign run it launched (its key file, endpoint and model are that campaign's) | Launcher for a sharded campaign run with a tag and shard count |
| `lanes/toolwork.sh` | ONE-OFF | the tooling-design briefs it ran against an external model (its key, endpoint and model are that era's) | Runs tooling-design briefs against an external model with build config as context |
| `lenmiss_route.py` | ONE-OFF | the routed outcome of one length-drift near-miss pile (it consumes that audit's classified ledger) | Routes the length-drift near-miss pile through the documented lenses, re-verified against the tree |
| `macro_draft.py` | ONE-OFF | the per-function headers under src/shared/ (Phase 35 T4; the macro form is gone) | Materializes a shared macro body back into a compilable standalone draft |
| `p16_known_answer.py` | SUPERSEDED | tools/p16_improve.py | Graduated known-answer validation of the scaffold pipeline across difficulty bands |
| `plumbing_groups.py` | SUPERSEDED | tools/gate_triage.py | Groups recorded declaration-conflict failures into sweepable work groups from the ledgers |
| `reconcile_slate.py` | SUPERSEDED | tools/pregate_check.py | Drives a whole batch to declaration-compatible before any rebuild is spent |
@@ -394,6 +394,7 @@ then what proved it. The same data generates the day-one kit's manifest and its
| `t5_distill_args.py` | SUPERSEDED | tools/distill_scan.py | Builds the post-wave distillation slate from a wave's directories |
| `t5_targets.py` | SUPERSEDED | tools/draw_waves.py | Draws one routed slate from the frontier classes |
| `t7_bank.py` | SUPERSEDED | tools/gate_stage.py | Batch banking driver applying reconcile-at-bank-time plus the whole-binary gate |
| `test_reconcile_ledger.py` | ONE-OFF | the S45p7 reconcile-ledger proof, retired with its subject dedup_propagate | Targeted proof of the propagation ledger guard the full control never exercised |
| `treelock.sh` | SUPERSEDED | tools/shared_lock.py | The mutex wrapper for tree-writing campaigns, with status and stale detection |
| `verify_map_findings.py` | ONE-OFF | the machine-checked verdicts of one codegen-map audit (docs/gcc-2.7.2-map/regalloc.md) | Machine-checks a codegen-map audit's findings against the actual compiler sources |
| `wall_taxonomy.py` | SUPERSEDED | tools/frontier_classify.py | Census-classifies every unmatched residual by blocker class to size each avenue |
+16
View File
@@ -458,6 +458,22 @@
3,135; shared headers 2,215 → 3,137; same-vram backlog 1,099 classes / 4,755 private copies / 3,658 collapsible → 51 / 160 / 109
(all ledgered); verdict A 1,712 → 2,760; fleet clean-run wall 145 s → 84 s (CPU 2,230 s → 1,335 s); 23 commits in T5. **T5 ☑.**
- **S96 — T6 part 1: freeze + retire.** `tools/frozen.py` (the one refusal, printed from main() — never at import: cdecl imports
gen_harvest_targets, 18 tools import family_remap). **FROZEN (14, status FROZEN + successor in the dictionary):** the plan's 7
(family_sweep, gen_harvest_targets, p16_improve, recover_giant, restore_dropped_decls, normalize_self_decls, o0_subsplit) + 7 the
guard census found reading or writing the macro form (inject_capped_externs, aprop_autodraft, canon_sig_reconcile, conform_decls,
blocker_probe, demacroize, and family_remap's command line — it writes a `#define DEFINE_func_` head; its library stays). Deviation
from the plan's "7": the S95 ast census measured 30 hits, not 22; twin_sweep/recover_integration (which exec frozen CLIs) stay LIVE
and would print the refusal; family_cousins stays LIVE (`make atlas` runs it) and loses its dead branch in part 2. **RETIRED (4, git mv
to tools/sunset/, README rows):** dedup_propagate + dedup_extend → share_body.py (add_members_surgical and the library surface
onboarded_overlays / load_sig / registered_addrs / sym moved verbatim; validate_targets imports share_body; inject_capped_externs
frozen), macro_draft (product: the headers), test_reconcile_ledger (retired with its subject). Six lane callers repointed to
`share_body.py --apply --bucket new --batches 1` (gate_stage, gate_lane, grinder's message, lora_grind, auto_driver, bulk_harvest);
audit_binaries' hint reworded. `tool_census` accepts FROZEN. Verify: every touched file compiles; `family_sweep.py --source x` →
`FROZEN since Phase 35 … Successor: tools/share_body.py` rc 1; `import share_body, family_remap, gen_harvest_targets,
validate_targets` OK; no live tool names a retired file; `tool_census --check: OK` after `make kit-corpus` (the first chain checked
BEFORE regenerating and a `;` let a partial commit through — R97 again; amended into one commit).
## Approved plan (verbatim, gate 1 — 2026-09-08)
# Phase 35 — Gen3 opens: the dedup phase, "one source per unique function" (v2.0.0 → v2.1.0)
+1
View File
@@ -574,6 +574,7 @@ def _local_verdict(fn, text, sub, o0):
def main():
import frozen; frozen.refuse("aprop_autodraft.py", "tools/share_body.py", "it seeds drafts from the macro bodies in engine_core.h (gone)") # Phase 35 T6: FROZEN
ap = argparse.ArgumentParser()
ap.add_argument("--cards", default=CARDS)
ap.add_argument("--outdir", default=".run/aprop_autodraft")
+1 -1
View File
@@ -176,7 +176,7 @@ def main():
zero = sorted(b for b in onb if b.startswith("ov_") and memb.get(b, 0) == 0)
if zero:
warns.append(f"{len(zero)} overlay(s) in 0 dedup groups (onboarded but un-harvested — "
f"candidate for tools/dedup_extend.py): {zero if verbose else zero[:6]}")
f"candidate for tools/share_body.py --plan): {zero if verbose else zero[:6]}")
# --- report
n_ov = len([b for b in onb if b.startswith("ov_")])
+1 -1
View File
@@ -137,7 +137,7 @@ def byte_gate(draft_dir):
def propagate():
sh([PY, "tools/dedup_propagate.py", "--auto-from", OV, "--min-reach", "2"], timeout=3600)
sh([PY, "tools/share_body.py", "--apply", "--bucket", "new", "--batches", "1"], timeout=3600) # P35 T6: the include-at-site share
def main():
+1
View File
@@ -223,6 +223,7 @@ def cc1_verdict(fn, stub, draft_path, work):
# ---------------------------------------------------------------------------------------------
def main():
import frozen; frozen.refuse("blocker_probe.py", "tools/share_body.py (its per-binary gate names the blocker)", "it probes macro-scope blockers in engine_core.h (gone)") # Phase 35 T6: FROZEN
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument('--binary', required=True)
+1 -1
View File
@@ -264,7 +264,7 @@ def run_cycle(a, api_base, model, tried):
for b, fns in banked_by_bin.items():
if any((lg.reach_of(b, fn) or 1) >= 2 for fn in fns):
before = gate_stage._dedup_group_count()
sh([PY, "tools/dedup_propagate.py", "--auto-from", b, "--min-reach", "2"], timeout=3600)
sh([PY, "tools/share_body.py", "--apply", "--bucket", "new", "--batches", "1"], timeout=3600) # P35 T6
propagated += max(0, gate_stage._dedup_group_count() - before)
merge_backlogs()
fp = fleet_pct()
+1
View File
@@ -518,6 +518,7 @@ def reconcile(fn, draft, canon_sig=None, tu_path=None):
def main():
import frozen; frozen.refuse("canon_sig_reconcile.py", "the types phase (Phase 37): one canonical declaration per symbol", "it reconciles declarations against engine_core.h (gone)") # Phase 35 T6: FROZEN
ap = argparse.ArgumentParser()
ap.add_argument('--fn', required=True)
ap.add_argument('--draft', required=True)
+1
View File
@@ -207,6 +207,7 @@ def consumers(fn):
def main():
import frozen; frozen.refuse("conform_decls.py", "the types phase (Phase 37)", "it conforms the fleet's declarations to engine_core.h (gone)") # Phase 35 T6: FROZEN
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--fn", required=True)
-298
View File
@@ -1,298 +0,0 @@
#!/usr/bin/env python3
"""Phase-28 T4: EXTEND the existing dedup registry to newly-onboarded binaries.
THE GAP THIS FILLS (byte-measured, P28 T3-A). `tools/new_overlay.sh` produces a byte-clean binary
that is NOT a citizen of the shared-body ecosystem: its `.c` includes only `common.h` (never
`../shared/engine_core.h`), so not one shared engine body can reach it, and it appears in ZERO of
`config/dedup.us.yaml`'s groups. The 4 SC07 overlays the Phase-27 disc audit onboarded therefore sat
at ~80 matched / ~2,400 stubs while their siblings were ~2,150 matched — and 1,689 registry groups
still read "134 binaries", never 138.
WHY THE EXISTING TOOL CANNOT DO IT. `dedup_propagate` is built for CRACK -> AUTHOR MACRO ->
INSTANTIATE: `--auto-from` scans a source overlay's INLINE DEFS (it planned only 11 fns here,
skipping the ~1,600 whose bodies are ALREADY `DEFINE_func_*` macros in engine_core.h), and `--addr`
dies with "no source overlay has it matched" because no overlay holds an inline def — every one of
the 134 uses the macro. Extending an existing MACRO-BACKED group to a new binary is a different
operation, and nothing implemented it.
WHY IT IS SAFE (the correctness argument — this is a byte-gate feeder, so it must be explicit).
`h_exact` is the SHA1 of RAW INSTRUCTION BYTES (tools/sig_image.py). Two instances sharing an
h_exact are therefore identical *including* their `jal`/`lui`/`%lo` reloc immediates — same callee
addresses, same data addresses, same symbols. So the body that compiles byte-identically at one
member compiles byte-identically at the other, with no remap at all. (This is exactly why
dup_report calls h_exact "guaranteed byte-match" and h_norm "candidate-only".) The whole-binary
byte-gate remains the sole arbiter (G3/P9): a wrong instantiation changes the bytes and fails SHA1.
A bug here can make this tool FAIL TO BANK; it cannot make it falsely bank.
REUSE, DON'T REBUILD (R33). This owns only the SET COMPUTATION and the REGISTRY EDIT. The splice and
the gate are `tools/harvest_verify.py` verbatim — which already derives each stub's home TU from the
corpus oracle, chunks + bisects, and reverts on failure. h_exact members are byte-identical by
construction, so the happy path is ONE build per binary (not one per function).
tools/dedup_extend.py --binaries ov_SC07_006,ov_SC07_007,ov_SC07_010,ov_SC07_011 [--check-only]
[--chunk N] [--limit N]
"""
import argparse
import glob
import json
import os
import re
import subprocess
import sys
import yaml
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import corpus
import family_hseq # has_mid_jr — the ONE jr oracle (R33)
import family_remap as FR # stream_words/nins_of
REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
PY = sys.executable
REGISTRY = "config/dedup.us.yaml"
SHARED_INCLUDE = '#include "../shared/engine_core.h"'
EXTEND_DIR = ".run/extend"
def sig_hashes(binary):
"""{vram_int: h_exact} from the binary's sig — the ORIGINAL bytes, independent of splat."""
out = {}
p = os.path.join(REPO, f".run/sig.{binary}.jsonl")
for line in open(p):
d = json.loads(line)
out[int(d["addr"], 16)] = d["h_exact"]
return out
def load_groups():
doc = yaml.safe_load(open(os.path.join(REPO, REGISTRY)))
groups = doc if isinstance(doc, list) else doc.get("groups", doc)
if not isinstance(groups, list):
sys.exit(f"dedup_extend: unexpected {REGISTRY} shape: {type(groups).__name__}")
return doc, groups
def add_members_surgical(additions):
"""Append binaries to each group's `binaries: [...]` list by TEXT EDIT, in place.
NEVER `yaml.safe_dump` this file. The first cut of this tool round-tripped it through
safe_dump and silently destroyed BOTH of the things a human needs from it (H5 — "never
silently drop comments on a rewrite"):
* all 47 comment lines — including the curated Phase-11 header explaining WHY the share is
source-level (the linker cannot excise bytes interior to an object) — dumped to nothing;
* every `vram: 0x80162FF4` re-serialized as `vram: 2148937716` (PyYAML parses YAML-1.1 hex
to int, and dumps int as decimal), making 1,832 entries unreadable.
It was invisible to every gate: dedup-check passed 1840/0 and check-all stayed 140/140,
because `_addr()` accepts both forms — the data was fine and the DOCUMENT was ruined. A
formatting-destructive write that all your oracles call green is exactly the class this
project keeps re-learning: the gate measures bytes, not intent.
`additions` = {group_id: [binary, ...]}. Idempotent: a binary already listed is skipped.
"""
p = os.path.join(REPO, REGISTRY)
lines = open(p).read().splitlines(keepends=True)
cur, n = None, 0
seen = set()
for i, ln in enumerate(lines):
m = re.match(r"^\s*-?\s*id:\s*(\S+)\s*$", ln)
if m:
cur = m.group(1)
continue
if cur and cur in additions and re.match(r"^\s*members:\s*$", ln):
# a VERBOSE group has no `binaries:` line — silently skipping it left five 141-member groups listed at 16 (P35 S94/S96)
raise SystemExit(f"add_members_surgical: {cur} is in the verbose `members:` form — convert it to shorthand first "
f"(share_body.py --repair-registry does), never skip it (R43)")
if cur and cur in additions and re.match(r"^\s*binaries:\s*\[", ln):
seen.add(cur)
add = [b for b in additions[cur] if re.search(rf"\b{re.escape(b)}\b", ln) is None]
if add:
lines[i] = ln.rstrip("\n").rstrip()[:-1].rstrip() + ", " + ", ".join(add) + "]\n"
n += len(add)
cur = None
open(p, "w").write("".join(lines))
return n
def _addr(v):
return int(v, 16) if isinstance(v, str) else int(v)
def plan_for(binary, groups):
"""[(group, vram, macro)] for every h_exact group this binary could join, plus the skip tally.
A group is extendable into `binary` iff: it is h_exact tier, the binary is not already a member,
the binary has a LIVE STUB at the group's vram, that stub's original bytes hash to the group's
recorded h_exact (the C1 equivalence the registry itself asserts), and it is NOT a jr/switch
function (see below).
THE jr GUARD (cookbook §53). A function with a mid-body `jr` on a non-$ra register dispatches
through a compiler-generated jump table the LINKER must place at the sibling's exact address — a
per-sibling `jtbl_carve` + interleave (`jtbl_family_bank.py`). This tool has no carve step and a
newly-onboarded overlay has no `_jr_*` split at all, so a jr body instantiated here would leave
its table unplaced and the gate would (correctly) reject it. The guard is PREVENTIVE: it skips
and REPORTS rather than feeding the gate drafts that cannot pass (R32 — a skip is only honest if
it is counted and named). On the SC07 set it currently skips ZERO: no jr function is in the
extendable set.
⚠️ R14 — WHAT THIS GUARD IS *NOT*. It does NOT explain the 12 DIFFs the first run produced
(`func_80162FF4/801630C4/80163194/8016325C`, the same 4 in every overlay, 12 of 6457 = 0.19%).
I first assumed those were the jr class because ov_SC01_077 hosts them in `_jr_8017A4AC.c` /
`_jr_80182268.c` — a natural read, and WRONG: `has_mid_jr` is **False** for all four (33-52 ins,
no jump table). They merely LIVE in a carved jr-REGION split (the carve region for
`func_8017A4AC` sweeps in every function in its address range), which says nothing about them.
Their DIFF cause is UNDIAGNOSED and logged for follow-up; they are correctly left as stubs by the
gate. Hosting file != function class — do not infer one from the other.
Detector reused verbatim from `family_hseq.has_mid_jr` (R33 — one oracle, not two).
"""
sig = sig_hashes(binary)
stubs = corpus.stubs(binary)
out, skips = [], {}
for g in groups:
if g.get("tier") != "h_exact":
continue
if "vram" not in g or "binaries" not in g:
skips["verbose-form"] = skips.get("verbose-form", 0) + 1
continue # not position-locked
if binary in g["binaries"]:
continue
vram = _addr(g["vram"])
if vram not in stubs:
continue # not a live stub here (matched already, or absent)
if sig.get(vram) != g["hash"]:
continue # DIFFERENT CODE at the same vram — the point of the check
words = FR.stream_words(binary, vram, FR.nins_of(binary, vram))
if words and family_hseq.has_mid_jr(words):
skips["jr-needs-carve"] = skips.get("jr-needs-carve", 0) + 1
continue # §53: route via jtbl_family_bank, not here
out.append((g, vram, g["func"]))
return out, skips
def ensure_include(binary, apply=True):
"""Add the shared-header include if absent. Byte-neutral: engine_core.h is only `#define`s, so
including it without instantiating emits no code (proven: ov_SC07_006 built 7ca772be with the
include + one macro). Returns True if the file was changed."""
p = os.path.join(REPO, f"src/{binary}/{binary}.c")
t = open(p).read()
if SHARED_INCLUDE in t:
return False
if apply:
t = t.replace('#include "common.h"', '#include "common.h"\n' + SHARED_INCLUDE, 1)
open(p, "w").write(t)
return True
def write_drafts(binary, plan):
d = os.path.join(REPO, EXTEND_DIR, binary)
subprocess.run(["rm", "-rf", d], check=False)
os.makedirs(d, exist_ok=True)
for g, vram, macro in plan:
fn = f"func_{vram:08X}"
body = f"{macro}() /* dedup: shared engine-core @0x{vram:08x} (src/shared) */\n"
open(os.path.join(d, fn + ".c"), "w").write(body)
return d
def gate(binary, drafts_dir, chunk, ladder=True):
"""The whole-binary byte-gate is the sole arbiter (G3/P9). Returns the banked fn set.
LADDER (S46): this called `harvest_verify` VERBATIM, which is the bare gate with no recovery.
Measured cost of that: 0 of 142 extendable groups banked, and the classified reasons were 118
PLUMBING / 21 CC1-FAIL / 3 DIFF — i.e. ~1 in 50 was a real byte divergence and everything else
was a declaration conflict in the TARGET TU (`conflicting types for memcpy / ApplyMatrixSV /
D_800AE620 / func_8014C568`). `gate_stage` wraps the same gate in exactly the reconcile ladder
those need (canon_resident_calls -> cast_call_sites -> sig_unify -> harvest_verify), and it
carries the §156 reconcile LEDGER so a failed candidate cannot leave a fleet-shared edit behind.
Measured on the same failure class in the S46 wave residue: 6 of 19 PLUMBING recovered (~32%).
GATE_NO_ARITY=1 is set for the child: gate_stage's arity pre-pass writes the fleet-shared
engine_core.h BEFORE the gate, and on a failing draft that edit can survive — the F1 defect that
broke 141 of 213 binaries in S45 and made every later gate report `near`. The ladder's other
rungs are draft-local and safe.
"""
good = open(os.path.join(REPO, f"config/check.{binary}.sha")).read().split()[0]
vout = f".run/extend_verified.{binary}.txt"
fout = f".run/extend_failed.{binary}.txt"
if ladder:
cmd = [PY, "tools/gate_stage.py", "--binary", binary,
"--drafts", os.path.relpath(drafts_dir, REPO), "--no-propagate",
"--verified-out", vout, "--failed-out", fout]
else:
cmd = [PY, "tools/harvest_verify.py", "--binary", binary,
"--out", f"build/{binary}/{binary}", "--good-sha", good,
"--drafts", os.path.relpath(drafts_dir, REPO), "--chunk", str(chunk),
"--verified-out", vout, "--failed-out", fout]
env = dict(os.environ, GATE_NO_ARITY="1")
subprocess.run(cmd, cwd=REPO, timeout=7200, env=env)
p = os.path.join(REPO, vout)
return set(open(p).read().split()) if os.path.exists(p) else set()
def main():
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--binaries", required=True, help="comma list of newly-onboarded binaries to wire in")
ap.add_argument("--chunk", type=int, default=512,
help="drafts per build (h_exact is byte-identical by construction, so the happy "
"path is ONE build; harvest_verify bisects on failure). default 512")
ap.add_argument("--limit", type=int, default=0, help="cap #groups per binary (0 = all)")
ap.add_argument("--check-only", action="store_true", help="print the plan, touch nothing")
a = ap.parse_args()
os.chdir(REPO)
doc, groups = load_groups()
targets = [b.strip() for b in a.binaries.split(",") if b.strip()]
dirty = subprocess.run("git status --porcelain -- config/ src/", shell=True,
capture_output=True, text=True).stdout.strip()
if dirty and not a.check_only:
sys.exit("dedup_extend: config/ or src/ is dirty — commit first so a failed gate reverts "
"cleanly.\n" + dirty[:400])
total_banked, total_planned = 0, 0
for b in targets:
plan, skips = plan_for(b, groups)
if a.limit:
plan = plan[:a.limit]
total_planned += len(plan)
print(f"[{b}] {len(plan)} extendable h_exact group(s); "
f"include {'ABSENT -> add' if ensure_include(b, apply=False) else 'present'}"
+ (f"; skipped {skips}" if skips else ""))
if a.check_only or not plan:
continue
added_include = ensure_include(b) # True IFF this run inserted the line
d = write_drafts(b, plan)
banked = gate(b, d, a.chunk)
print(f"[{b}] BANKED {len(banked)} / {len(plan)}")
total_banked += len(banked)
if not banked:
# ONLY undo what THIS run did (§61/§63). The revert is an INVERSE TRANSFORM, not a
# snapshot restore, so firing it on a binary that ALREADY had the include strips a
# load-bearing line: every `DEFINE_func_*()` in that overlay stops resolving. It was
# unconditional until Phase 29 SESSION-19, where a 0-banked run over 135 already-wired
# binaries removed the include from all 135 at once. Invisible to every byte-gate (R34)
# because the damage lands AFTER the last gate has run.
if added_include:
ensure_include_revert(b)
continue
adds = {g["id"]: [b] for g, vram, _ in plan # registry: only what the GATE accepted (P9)
if f"func_{vram:08X}" in banked}
n = add_members_surgical(adds) # TEXT edit — never safe_dump (see the docstring)
print(f"[{b}] registry: +{n} membership(s)")
print(f"\n=== dedup_extend: banked {total_banked} / {total_planned} planned "
f"across {len(targets)} binaries ===")
if a.check_only:
print("(--check-only: no files touched)")
def ensure_include_revert(binary):
p = os.path.join(REPO, f"src/{binary}/{binary}.c")
t = open(p).read().replace('#include "common.h"\n' + SHARED_INCLUDE, '#include "common.h"', 1)
open(p, "w").write(t)
if __name__ == "__main__":
main()
File diff suppressed because it is too large Load Diff
+1
View File
@@ -122,6 +122,7 @@ def plan(binary, fn, draft_text):
def main():
import frozen; frozen.refuse("demacroize.py", "tools/share_body.py", "it turns a macro site back into a private copy — the opposite of the S1 invariant") # Phase 35 T6: FROZEN
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument('--binary', required=True)
+1
View File
@@ -1322,6 +1322,7 @@ def remap(addr, from_ov, to_ov, to_addr=None, imm_map=None):
def main():
import frozen; frozen.refuse("family_remap.py", "tools/share_body.py --reexemplar / --apply", "its command line writes a DEFINE_func_ macro head; the library (stream_words, nins_of, …) stays") # Phase 35 T6: FROZEN
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--addr", required=True)
ap.add_argument("--from", dest="frm", required=True)
+1
View File
@@ -876,6 +876,7 @@ def hseq_sweep(a):
def main():
import frozen; frozen.refuse("family_sweep.py", "tools/share_body.py", "it remaps and rewrites the DEFINE_func_ macro bodies Phase 35 retired") # Phase 35 T6: FROZEN
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--source", default="ov_SC01_077")
ap.add_argument("--limit", type=int, default=0, help="cap #exemplars (0 = all)")
+16
View File
@@ -0,0 +1,16 @@
#!/usr/bin/env python3
"""frozen.py — the ONE refusal every FROZEN tool prints from its main() (Phase 35 T6, 2026-09-08).
A frozen tool is a matching-era tool that reads or writes the `DEFINE_func_` macro form Phase 35 retired (the shared bodies are
plain-C headers under src/shared/ now, one source per unique function — the S1 invariant). Its code stays in tools/ for the record
and for the libraries other tools import from it; only its command line refuses, so an import never exits (cdecl imports
gen_harvest_targets; eighteen tools import family_remap). The dictionary row carries status FROZEN and the successor.
import frozen; frozen.refuse("family_sweep.py", "tools/share_body.py", "it rewrites the macro form Phase 35 retired")
"""
import sys
def refuse(tool, successor, why):
sys.exit(f"{tool}: FROZEN since Phase 35 (2026-09-08) — {why}. Successor: {successor}. The code is kept for the record and its "
f"library functions still import; the command line does nothing (R43). See docs/SETUP.md and config/tool_dictionary.tsv.")
+1 -1
View File
@@ -83,7 +83,7 @@ json.dump(banked,open(outp.replace('.json','_banked.json'),'w'))
# propagate per function
for fn in banked:
addr="0x"+fn.split("_")[1].lower()
r=subprocess.run([".venv/bin/python","tools/dedup_propagate.py","--addr",addr],
r=subprocess.run([".venv/bin/python","tools/share_body.py","--apply","--bucket","new","--batches","1"], # P35 T6
capture_output=True,text=True)
if dirty():
tag=os.environ.get("GATE_PHASE","decomp") # P31 T1: was hardcoded "phase-30 S49"
+2 -3
View File
@@ -519,10 +519,9 @@ def _run_gate_locked(drafts, binary, src, asm, out, good_sha, propagate, source_
# cover write scope, and a killed process performs no undo at all).
_budget = min(6 * 3600, 1800 + 1800 * len(verified))
try:
pr = sh([PY, "tools/dedup_propagate.py", "--auto-from", binary, "--min-reach", "2",
"--recover"], timeout=_budget)
pr = sh([PY, "tools/share_body.py", "--apply", "--bucket", "new", "--batches", "1"], timeout=_budget) # P35 T6: the include-at-site share (dedup_propagate retired)
except subprocess.TimeoutExpired:
print(f"[gate] FATAL: dedup_propagate exceeded {_budget}s with {len(verified)} banks — "
print(f"[gate] FATAL: share_body exceeded {_budget}s with {len(verified)} banks — "
f"the fleet is HALF-PROPAGATED and the tree is DIRTY. Revert (`git checkout -- src/`), "
f"then re-gate with --no-propagate and propagate separately.", file=sys.stderr)
return {"drafts": len(draft_fns), "banked": len(verified), "propagated": 0,
+1
View File
@@ -152,6 +152,7 @@ def collect_stubs(c_path):
def main():
import frozen; frozen.refuse("gen_harvest_targets.py", "tools/share_census.py", "it collects harvest targets from the macro header engine_core.h (gone)") # Phase 35 T6: FROZEN
ap = argparse.ArgumentParser()
ap.add_argument('--source', default='ov_SC01_077')
ap.add_argument('--min-reach', type=int, default=2)
+2 -2
View File
@@ -348,8 +348,8 @@ def main():
f"total {banked}; fleet {fp}%")
if s.get("verified"):
log(" -> propagate as its own batch: "
+ "; ".join("tools/dedup_propagate.py --addr 0x%s --recover" % f.split('_')[-1]
for f in s["verified"][:4]))
+ "tools/share_body.py --apply --bucket new --batches 1 (P35 T6: shares every new same-address class; "
+ "verified: " + " ".join(s["verified"][:4]) + ")")
# A permuter win the whole-binary gate STILL rejects is plumbing-bound (not regalloc/sched) —
# re-permuting can never bank it. Blacklist so the grinder stops churning it (the §20 trap).
rejected = [f for f, _b in won if f not in verified]
+2 -1
View File
@@ -30,7 +30,7 @@ import argparse, re, sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
import dedup_propagate as dp
dp = None # the propagate library is retired to tools/sunset (P35 T6); this tool is FROZEN — main() refuses
SYM_RE = re.compile(r'\b(func_[0-9A-Fa-f]{6,8}|D_[0-9A-Fa-f]{6,8})\b')
@@ -81,6 +81,7 @@ def inject(defblock, name, ext_map):
def main():
import frozen; frozen.refuse("inject_capped_externs.py", "tools/share_body.py", "it fixed drafts a macro-era propagation skipped, on dedup_propagate (retired)") # Phase 35 T6: FROZEN
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--overlay", default="ov_SC01_077")
+2 -2
View File
@@ -227,8 +227,8 @@ def main():
# periodic fleet propagate sweep (the multiplier) — every N batches
if batch_i % a.propagate_every == 0 and propagated_since:
for pb in sorted(propagated_since):
subprocess.run([".venv/bin/python", "tools/dedup_propagate.py", "--auto-from", pb,
"--min-reach", "2"], cwd=REPO, capture_output=True, timeout=3600)
subprocess.run([".venv/bin/python", "tools/share_body.py", "--apply", "--bucket", "new",
"--batches", "1"], cwd=REPO, capture_output=True, timeout=3600) # P35 T6
propagated_since.clear()
log("propagate sweep done")
# heartbeat + flywheel stats. progress.py --fleet is ~14s (a full 136-binary scan), so
-80
View File
@@ -1,80 +0,0 @@
#!/usr/bin/env python3
"""macro_draft.py — materialize a `DEFINE_func_XXXX()` macro body as a compilable draft .c
WHY THIS EXISTS (Phase 30 S47)
==============================
`conform_decls.py` needs a **byte-true DEFINITION** to conform the fleet's declarations to, and it
refuses (correctly) when handed a file that holds only a declaration. But for a DEDUPED function the
definition does not live in any .c file at all — it lives inside a `#define DEFINE_func_X() \\ ...`
macro in `src/shared/engine_core.h`, where two things defeat every definition parser:
* every line ends in a backslash continuation, and
* the definition line is INDENTED, so `^([A-Za-z_]...)` never matches.
So the single largest class of `conflicting types for func_X` — a deduped callee whose macro-local
`extern` disagrees with the fleet's — was unreachable by the tool built to fix exactly that class.
This bridges the two: it emits the macro body verbatim (dedented, continuations stripped) so the
existing conformer can read the signature it already knows how to read.
VERBATIM IS THE POINT. The draft is evidence, not a rewrite: nothing is reformatted, no types are
"cleaned up". If this file ever starts editing the body it stops being byte-truth (R33/R35).
Usage:
tools/macro_draft.py --fn func_80128ED8 [--out .run/draft.c] # default: .run/macro_draft_<fn>.c
"""
import argparse, os, re, sys, textwrap
REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
HDR = os.path.join(REPO, "src", "shared", "engine_core.h")
def extract(fn, header=HDR):
"""(body_text, first_line, last_line) of DEFINE_<fn>()'s macro body, 1-indexed; None if absent."""
lines = open(header, errors="replace").read().split("\n")
for i, l in enumerate(lines):
if re.match(rf"\s*#define\s+DEFINE_{re.escape(fn)}\(\)", l):
j = i
while j < len(lines) and lines[j].rstrip().endswith("\\"):
j += 1
raw = lines[i:j + 1]
# strip the trailing backslash continuation from every line, drop the #define line
body = [x.rstrip()[:-1].rstrip() if x.rstrip().endswith("\\") else x for x in raw][1:]
return textwrap.dedent("\n".join(body)), i + 1, j + 1
return None
def main():
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--fn", required=True)
ap.add_argument("--out")
ap.add_argument("--header", default=HDR)
a = ap.parse_args()
got = extract(a.fn, a.header)
if not got:
print(f"macro_draft: no DEFINE_{a.fn}() in {a.header}", file=sys.stderr)
return 2
body, lo, hi = got
# R32: prove the emitted text actually contains a DEFINITION of the requested function, at
# column 0, where a definition parser can see it. A dedent that leaves it indented is a silent
# no-op that would surface later as "no DEFINITION" from the conformer.
if not re.search(rf"^[A-Za-z_][\w \t\*]*\b{re.escape(a.fn)}\s*\(", body, re.M):
print(f"macro_draft: DEFINE_{a.fn}() body carries no column-0 definition of {a.fn} "
f"(header lines {lo}-{hi}) — refusing to emit a draft that cannot be parsed",
file=sys.stderr)
return 2
out = a.out or os.path.join(REPO, ".run", f"macro_draft_{a.fn}.c")
os.makedirs(os.path.dirname(out), exist_ok=True)
with open(out, "w") as f:
f.write(f'/* verbatim body of DEFINE_{a.fn}() — {a.header} lines {lo}-{hi} */\n')
f.write('#include "engine_types.h"\n')
f.write(body.rstrip() + "\n")
print(out)
return 0
if __name__ == "__main__":
sys.exit(main())
+1
View File
@@ -191,6 +191,7 @@ def fix(tu_text, fn, ref_decl=None):
def main():
import frozen; frozen.refuse("normalize_self_decls.py", "tools/share_body.py", "it derives canonical declarations from engine_core.h (gone)") # Phase 35 T6: FROZEN
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument('--tu', required=True, help='the TU file to normalize (edited in place unless --out)')
ap.add_argument('--fn', required=True, help='func_XXXXXXXX to normalize decls of')
+1
View File
@@ -62,6 +62,7 @@ def free_letters(ov, n):
def main():
import frozen; frozen.refuse("o0_subsplit.py", "the Makefile -O0 globs (rollout_o0 for the record)", "it splits -O0 macro sites out of a TU against the macro form") # Phase 35 T6: FROZEN
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("ov")
+1
View File
@@ -66,6 +66,7 @@ def classify_fail(fn, cpath):
def main():
import frozen; frozen.refuse("p16_improve.py", "tools/share_body.py", "it rewrites macro sites in ov_SC01_077 against engine_core.h (gone)") # Phase 35 T6: FROZEN
ap = argparse.ArgumentParser()
ap.add_argument("--band", choices=list(BANDS), default="easy")
ap.add_argument("--n", type=int, default=40)
+1
View File
@@ -69,6 +69,7 @@ def recover(fn, src, ec):
def main():
import frozen; frozen.refuse("recover_giant.py", "tools/share_body.py", "it recovers a giant into the macro header engine_core.h (gone)") # Phase 35 T6: FROZEN
if len(sys.argv) != 4:
raise SystemExit(__doc__)
fn, inp, outp = sys.argv[1], sys.argv[2], sys.argv[3]
+1
View File
@@ -71,6 +71,7 @@ def insert_at_file_scope(path, decls):
def main():
import frozen; frozen.refuse("restore_dropped_decls.py", "tools/share_body.py", "it repairs declarations a macro-era propagation deleted; share_body edits definition lines only") # Phase 35 T6: FROZEN
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--binary", required=True)
ap.add_argument("--ref", required=True, help="git ref holding the pre-deletion sources")
+93 -3
View File
@@ -11,7 +11,7 @@ WHAT ONE SHARE IS. A class = one h_exact (the raw instruction bytes) at one vram
are still PRIVATE COPIES (a definition in that binary's own TU). Sharing it: the class's ONE body goes to (or already is) a plain-C
header under src/shared/<space>/ (macro_to_header's naming: func_<VRAM>[__h8].h, keyed by the class — R48), every private copy becomes
`#include "../shared/<space>/<header>"` at the SAME position (the copy's definition lines, nothing else — the TU keeps its own
declarations), and the registry gains the group (shorthand form, appended by text) or the members (dedup_extend.add_members_surgical).
declarations), and the registry gains the group (shorthand form, appended by text) or the members (add_members_surgical, inherited from the retired dedup_extend).
THE EXEMPLAR (for an unregistered class) — printed with every share: the copy whose normalized text the MOST copies share; a tie goes
to a pin-free copy; a further tie to the shortest. A body defined under an asm-label alias (aF<ADDR>) gets its own binding
@@ -47,12 +47,102 @@ REPO = pathlib.Path(__file__).resolve().parent.parent
sys.path.insert(0, str(REPO / "tools"))
import share_census as sc # noqa: E402
import macro_to_header as m2h # noqa: E402 — Oracles (naming, spaces, sigs), bind_alias_header, include_line, banner
import dedup_extend as de # noqa: E402 — add_members_surgical (the registry's surgical text edit)
LEDGER = REPO / "config/dedup_exceptions.tsv"
RUN = REPO / ".run/P35/share"
# ----------------------------------------------------------------------------------------------------------------------------
# The library surface inherited from the retired propagate/extend tools in tools/sunset (Phase 35 T6; verbatim, so their importers
# change one line: `import share_body as dp`). load_sig/sig_path read the per-binary sig files; registered_addrs the registry.
def sig_path(ov):
return REPO / f".run/sig.{ov}.jsonl"
def onboarded_overlays():
"""Overlay aliases + (S44) module aliases — every non-main/resident binary a shared body can
propagate into. Modules carry engine functions too; excluding them re-creates the SC07
invisible-work bug one class over."""
out = []
for f, var in (("config/overlays.mk", "OVERLAY_BINARIES"), ("config/modules.mk", "MODULE_BINARIES")):
fp = REPO / f
if fp.exists():
m = re.search(rf"^{var}\s*:=\s*(.*)$", fp.read_text(), re.M)
if m:
out += m.group(1).split()
return out
def load_sig(ov):
"""addr(int) -> {h_exact, h_norm, nins, ...}; {} if absent."""
p = sig_path(ov)
if not p.exists():
return {}
out = {}
for ln in p.read_text().splitlines():
ln = ln.strip()
if ln:
r = json.loads(ln)
out[int(r["addr"], 16)] = r
return out
def registered_addrs():
"""Vaddrs already in config/dedup.us.yaml (shared via ANY mechanism — engine_core, ov_setters,
clearTbl40). --auto-from skips these so the bulk is purely additive and never collides with an
existing share (e.g. a SETTER-matched function) — which would trip the structural check."""
p = REPO / "config/dedup.us.yaml"
if not p.exists():
return set()
try:
import yaml
sys.path.insert(0, str(REPO / "tools"))
from dedup_integrate import group_members
data = yaml.safe_load(p.read_text()) or {}
return {v for g in (data.get("groups") or []) for (_b, v, _n) in group_members(g)}
except Exception:
return set()
def sym(addr):
return f"func_{addr:08X}" # splat convention: uppercase 8-hex
def add_members_surgical(additions):
"""Append binaries to each group's `binaries: [...]` list by TEXT EDIT, in place.
NEVER `yaml.safe_dump` this file. The first cut of this tool round-tripped it through
safe_dump and silently destroyed BOTH of the things a human needs from it (H5 — "never
silently drop comments on a rewrite"):
* all 47 comment lines — including the curated Phase-11 header explaining WHY the share is
source-level (the linker cannot excise bytes interior to an object) — dumped to nothing;
* every `vram: 0x80162FF4` re-serialized as `vram: 2148937716` (PyYAML parses YAML-1.1 hex
to int, and dumps int as decimal), making 1,832 entries unreadable.
It was invisible to every gate: dedup-check passed 1840/0 and check-all stayed 140/140,
because `_addr()` accepts both forms — the data was fine and the DOCUMENT was ruined. A
formatting-destructive write that all your oracles call green is exactly the class this
project keeps re-learning: the gate measures bytes, not intent.
`additions` = {group_id: [binary, ...]}. Idempotent: a binary already listed is skipped.
"""
p = str(REPO / "config/dedup.us.yaml")
lines = open(p).read().splitlines(keepends=True)
cur, n = None, 0
seen = set()
for i, ln in enumerate(lines):
m = re.match(r"^\s*-?\s*id:\s*(\S+)\s*$", ln)
if m:
cur = m.group(1)
continue
if cur and cur in additions and re.match(r"^\s*members:\s*$", ln):
# a VERBOSE group has no `binaries:` line — silently skipping it left five 141-member groups listed at 16 (P35 S94/S96)
raise SystemExit(f"add_members_surgical: {cur} is in the verbose `members:` form — convert it to shorthand first "
f"(share_body.py --repair-registry does), never skip it (R43)")
if cur and cur in additions and re.match(r"^\s*binaries:\s*\[", ln):
seen.add(cur)
add = [b for b in additions[cur] if re.search(rf"\b{re.escape(b)}\b", ln) is None]
if add:
lines[i] = ln.rstrip("\n").rstrip()[:-1].rstrip() + ", " + ", ".join(add) + "]\n"
n += len(add)
cur = None
open(p, "w").write("".join(lines))
return n
def log(msg):
print(msg, flush=True)
@@ -397,7 +487,7 @@ def run_batch(orc, cen, classes, label):
bins = [x for x in b.extend.get(gid, []) if x not in rejected_bins.get(c["h"], set())]
if bins:
ext[gid] = bins
n_ext = de.add_members_surgical(ext) if ext else 0
n_ext = add_members_surgical(ext) if ext else 0
ok_n = sum(1 for a in gated if results[a][0])
log(f" [{label}] gated {ok_n}/{len(gated)} binaries green · registered {len(reg_entries)} groups · extended {n_ext} members · "
f"rejected classes {len(failed_classes)}")
+4
View File
@@ -41,3 +41,7 @@ successor named), or a one-off (retired, its product named). The dictionary rows
| `verify_map_findings.py` (was `tools/verify_map_findings.py`) | ONE-OFF | the machine-checked verdicts of one codegen-map audit (docs/gcc-2.7.2-map/regalloc.md) | ☐ |
| `wall_taxonomy.py` (was `tools/wall_taxonomy.py`) | SUPERSEDED | tools/frontier_classify.py | ☐ |
| `warmstart.py` (was `tools/warmstart.py`) | SUPERSEDED | tools/permuter_sweep.py | ☐ |
| `dedup_propagate.py` (was `tools/dedup_propagate.py`) | SUPERSEDED | tools/share_body.py (P35 T6: the include-at-site share; --reexemplar, --repair-registry) | ☐ |
| `dedup_extend.py` (was `tools/dedup_extend.py`) | SUPERSEDED | tools/share_body.py (add_members_surgical moved verbatim; the extend bucket) | ☐ |
| `macro_draft.py` (was `tools/macro_draft.py`) | ONE-OFF | the per-function headers under src/shared/ (P35 T4) — no macro body to materialize | ☐ |
| `test_reconcile_ledger.py` (was `tools/test_reconcile_ledger.py`) | ONE-OFF | the S45p7 reconcile-ledger proof, retired with dedup_propagate | ☐ |
-61
View File
@@ -1,61 +0,0 @@
#!/usr/bin/env python3
"""Targeted proof of the S45p7 reconcile-ledger fix in dedup_propagate.
The full-propagation negative control did not fire (that run SUCCEEDED), so the guarded
path was never executed. This exercises the mechanism directly:
1. take real overlay files, snapshot them
2. apply a REAL reconcile via dedup_propagate.reconcile_caller_extern (the same call the
--recover Part B path makes) -> files are now edited on disk
3. drive the ledger's undo the way the fixed code does when the fn leaves the plan
4. assert every file is byte-identical to its original
Read-only w.r.t. git: it restores what it edits, and asserts it did.
"""
import sys, os, hashlib
sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), '..', '..', 'tools'))
sys.path.insert(0, 'tools')
import dedup_propagate as dp
def sha(p):
return hashlib.sha1(open(p, 'rb').read()).hexdigest()
def main():
OV, ADDR = 'ov_SC07_010', 0x80146A6C # the exact overlay+fn from tonight's breakage
files = [cp for cp, _ in dp.overlay_files(OV)]
before = {str(p): sha(p) for p in files}
print(f"[setup] {OV}: {len(files)} source files snapshotted")
# --- 1. apply a real reconcile (this WRITES) ---
snap, n = dp.reconcile_caller_extern(OV, ADDR)
after_edit = {str(p): sha(p) for p in files}
changed = [k for k in before if before[k] != after_edit.get(k)]
print(f"[apply] reconcile_caller_extern -> {n} edit(s); {len(changed)} file(s) changed on disk")
if n == 0 or not changed:
print("SKIP: this fn/overlay pair produced no reconcile edit — cannot exercise the path.")
return 3
# --- 2. drive the ledger undo exactly as the fixed code does ---
kept = [(ADDR, snap)]
drop = [r for r in kept if r[0] in {ADDR}]
for _a, _s in reversed(drop):
dp.restore_snapshot(_s)
print(f"[undo] ledger restored {len(drop)} kept reconcile(s)")
# --- 3. the assertion ---
after = {str(p): sha(p) for p in files}
bad = [k for k in before if before[k] != after.get(k)]
if bad:
print("FAIL — files NOT restored byte-identically:")
for k in bad[:5]:
print(" ", k)
return 1
print(f"PASS — all {len(files)} file(s) byte-identical after the ledger undo "
f"(the orphan that broke 141/213 cannot survive this path)")
return 0
if __name__ == '__main__':
sys.exit(main())
+2 -2
View File
@@ -92,7 +92,7 @@ PHASE_TITLES = {"P1": "extraction + manifest", "P2": "the oracles + the load map
"P7": "the codegen map, the dumps, the permuter", "P8": "the campaign: cards, lanes, gates, recovery, harvest",
"P9": "publish", "P10": "readability", "PROJECT-ONLY": "project-only in code (the shape is a task; the code does not transfer)"}
COLS = ["path", "phase", "portability", "need", "what", "adapts", "status", "successor_or_product"]
STATUSES = {"LIVE", "STILL-NEEDED", "SUPERSEDED", "ONE-OFF"}
STATUSES = {"LIVE", "STILL-NEEDED", "FROZEN", "SUPERSEDED", "ONE-OFF"} # FROZEN (P35 T6): in the tree, command line refuses, libraries import
def sh(cmd):
@@ -191,7 +191,7 @@ def census():
files, retired = enumerate_tools()
rows = read_dict()
by_path = {r["path"]: r for r in rows}
live_rows = {p for p, r in by_path.items() if r["status"] in ("LIVE", "STILL-NEEDED")}
live_rows = {p for p, r in by_path.items() if r["status"] in ("LIVE", "STILL-NEEDED", "FROZEN")}
retired_rows = {p for p, r in by_path.items() if r["status"] in ("SUPERSEDED", "ONE-OFF")}
gaps = []
for f in files:
+1 -1
View File
@@ -41,7 +41,7 @@ import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import corpus
import dedup_propagate as DP
import share_body as DP # load_sig lives in share_body since dedup_propagate retired (P35 T6)
REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))