Commit Graph

26 Commits

Author SHA1 Message Date
Drew T 55ca65bfc5 phase-37: T4 step (f) first pass — census 8,830 lying / 950 callees (+5,002 K&R sites counted APART: argcheck kr_marked, type_census + readability_progress split), levers 4,010 → 3,730 (0 unmarked), series snapshots + README; the 8,830 bucketed by ledger verdict: 3,700 never judged, 2,832 instrument disagreement, 1,120 T5's types, 723 cross-binary, 415 conflicts | 🛑 T4 in progress (S108) 2026-09-12 19:18:17 -06:00
Drew T cd39c79de6 phase-37: T2 ☑ — the probe: rung 1 (struct spelling) byte-neutral on 126/139 judged bodies (90.6 %, 716 sites moved), rung S2 closes 13/13 DIFFERS with 27 casts kept (139/139); the definition's signature free for 4,473/4,828 (TU, callee) pairs (92.6 %, 0 DIFFERS; 355 compile refusals = K&R sites + multi-spellings); the definition fold 12/20 TUs clean (opaque folds need member renames); the layout engine cc1-validated on 5,283 definitions / 29,248 fields (0 rejected; two engine gaps fixed: nested-paren attributes, per-field aligned(N); 27 dead-guarded standalone blocks blanked); the relocation control (object DIFFERS, binary identical) proves T3's linked mode; engine_types.h fan-out 3,975 objects in 35 s; tools/restruct.py (--try/--probe/--probe-decls/--probe-defs/--audit-layouts/--fanout-cost) + SETUP + dictionary | 🛑 T3 NEXT (the tools, Max design) 2026-09-12 01:27:45 -06:00
Drew T da26c05d67 phase-37: T1 ☑ — tools/type_census.py + the struct map: 7,261 struct definitions (525 layouts, 206 duplicate classes / 2,789 names, 39 VARIANT), 503,016 raw pointer-cast dereferences in four forms (coverage OK; + 18,912 address-of, 36,681 typed cast-member), 98,648 lying declarations / 1,609 callees, 18,760 evidence-clustered types explaining 99.2 % of sites, 24 parked P36 classes in parked.tsv, controls 4/4; lever_census controls re-keyed and --strict made literal (6,717 direct GTE statements in bodies); progress.py types block + README sentence + 2 corrections; the readability series read by column name with 16 census columns; docs/struct-map.md; SETUP §P37 S106; dictionary row; kit corpus | 🛑 T2 NEXT (the probe, Max) 2026-09-11 23:54:15 -06:00
Drew T 5a2bd70fbd phase-36: T8 — the published split regenerated by report (README block + progress.json: 2,141 pins + 1,869 asm in 1,980 functions, all marked); Reference-index rows for the two Gen3 series 2026-09-11 20:56:44 -06:00
Drew T 917c68afd9 phase-36: T4 CLOSE — the mechanical campaign over the whole population: 16,334 bodies judged in 13 gated batches (3,341 lever-free = 20.5 %; 21,063 of 56,445 sites removed or rewritten = 37.3 %; 8,239 replays, 1 disagreement; 49,487 compiles); the file-scope asm statements judged too (batch tus11: 6 barriers NEEDED, 7 .section blocks refused as asm-data and marked); THE NUMBER 53,234 → 33,625 sites (−36.8 %) in 12,501 bodies (1,728 distinct), 33,625 marked, 0 UNMARKED, 0 orphans — lever_census --check OK; the residue 12,967 bodies in 1,803 distinct classes (pins-only 6,018 · pins+asm 4,310 · asm-only 2,141 · C/D-only 498); published by make report (README line 30, progress.json counts.levers, the timeline); kit corpus; the 🛑 block rewritten for T5 2026-09-09 05:05:33 -06:00
Drew T a1407bdfc1 phase-36: T1b — the verbatim manifest learns the in-function form (49 rows: 27 PERMANENT hand asm incl. 22 per-overlay scratchpad stack-switch trampolines, 8 Sony GsTMDfast* routines carried as asm, 11 DECOMPILE-NOW + 3 UNCERTAIN game bodies never decompiled); verbatim_check detects the form structurally via lever_census.whole_body_asm_functions (--strict: 2,709 bodies, no drift); the census defers exclusion to the manifest's dispositions — the phase's number 53,234 sites in 15,679 bodies (2,227 distinct); progress.py counts PERMANENT (27) + SDK (8) with a dated correction; README/wiki/verification/gen3 docs corrected (14 game functions were asm in C shells); kit corpus 2026-09-09 01:44:31 -06:00
Drew T 357f8a1ed3 phase-36: T1 — tools/lever_census.py, the self-asserting lever census (coverage asm 80,396 = 66,415 live + 7,189 macro-block + 6,792 comment/dead; 4/4 controls; the five manifest bodies excluded): 53,221 pin/asm sites in 15,666 bodies (2,214 distinct) — 37,720 register pins + 15,501 asm statements, 0 marked; 6,911 GTE ops and 23 whole-body asm routines in C shells (2,683 sites, outside the manifest — Drew's call) set apart; progress.py levers block + README sentence; dictionary + SETUP rows; kit corpus; the 🛑 block for T2 2026-09-09 00:52:22 -06:00
Drew T ce837b1e46 docs(phase-35): T8 — the record: The-dedup-engine.md rewritten for the shared-source model (the shape, the h_exact/h_text/h_norm tiers, share_body and its four first-day defects, the S1 gate and the second oracle, the numbers as a dated derivation); Repository-layout, Where-the-project-goes-next, Verification-and-progress, how-to ch.10, README prose, gen3-standards (§4 row struck, DoD item met), gen3-handoff (dated snapshots); cookbook §453 + a retirement banner on §14, the index regenerated; decision-log P35 (R31), accelerators P35, DIGEST §4; doc_links strict 0 broken, wiki 32/32, kit coverage OK, kit corpus 2026-09-08 22:52:09 -06:00
Drew T ac8c29edb1 src(phase-35): T5b — the text tier: 38 same-address functions whose bytes vary per binary (the TU's declaration environment carries the original per-overlay variance) share one text each through h_text registry groups with per-member h_exact — 1,957 private copies in 1,540 TUs replaced by the include, 38 headers (func_<VRAM>__t<hash8>.h), 138/138 binaries green per binary, 0 rejected; dedup_integrate C1 per member (3173 validated, 0 failed); share_census indexes h_text groups by site, exports text classes, the S1 text half is a violation (0 now), the census json carries the oracle's counts; share_body --bucket text; progress.py/README: 103,015 bodies written once, 38 text-tier functions / 2,030 sites, 1,668 copies of 381 tiny bodies deferred; R22 clean fleet 218/218 in 90 s; SETUP; kit corpus 2026-09-08 22:39:42 -06:00
Drew T c8224e657f tools(phase-35): T7 — the S1 invariant wired into tools-health (share_census --selftest + --check --strict-macros --strict-text, by exit code), C2c/C2d in dedup_integrate --check (one source under src/shared/ defining one function; every member's site includes it — from the census's per-instance forms), progress.py: unique_function_bodies 105,007 / duplicate_source_copies 160 in 51 ledgered classes + the dated corrections list + the README sentence; the second oracle refined (distinct TUs; deferred vs pending vs violation): 380 texts deferred inside cross-address classes, 38 same-address texts / 2,030 sites byte-variant per binary (PENDING the owner), 0 violations; negative control in place: one reverted include -> S1 FAIL naming the class + C2d naming the member; tools-health OK (551 s) 2026-09-08 21:46:05 -06:00
Drew T db63fcc673 tools(phase-35): T6 part 2 — the dead macro branches dropped from 12 live tools; the macro-form guard in tool_census --check (ast scan of every LIVE tool, FROZEN/retired/6 whitelisted detectors excluded; 0 LIVE tools reference the retired form; --guard-root negative control: the pre-T4 tree flags exactly the 12 fixed + 3 retired); progress.py: an empty-bodied included member is EMPTY, not folded into REAL (the R32 bucket assertion caught the double count) — the fleet denominator 363,221 -> 363,680 (+459: SC03_015's 219 single-site macros, SC03_118 +3, the never-extended members of five groups) and REAL 360,744 -> 350,533 with EMPTY up the same; progress.json + README block + fleet table regenerated; SETUP; kit corpus 2026-09-08 20:53:05 -06:00
Drew T 6cb056c932 tools(phase-35): T2 — the health chain learns the include-at-site and twin forms while the macro form still builds: one source-dir oracle (corpus.src_dirs/src_dir/twin_of from the Makefile), share_census.header_defs as the one reader of the include form, progress.classify counts include sites (fold keeps 255,632), overlay_src_split's include item kind + loud macro-table refusal, dedup_integrate C2a' (a header must DEFINE func), audit_binaries prelude + twin citizenship (CHECK 3b), cdecl differential not-applicable when the macro header is gone, recursive shared-header globs; the count correction 363,214 -> 363,221 (7 whale include sites never registered; ov_MAIN_012 2,401 -> 2,402) regenerated into progress.json + README 2026-09-08 16:46:55 -06:00
Drew T a1d47bb002 docs(phase-34): task 7 part 2 — C11: ~9 GB of untracked campaign scratch pruned (46G→37G), public_rewrite kept to the probe inputs; DIGEST §0/§1/§4, decision-log P34 S93, accelerators, SETUP posture, direction page; README shows the generated timeline 2026-09-08 13:39:26 -06:00
Drew T 8594081d86 kit+docs(phase-33.5): task 14.5 part 3a — dry-run 5 of the final kit (fresh throwaway, fresh Opus agent): PASS, 0 failed verify lines; the run-5 rows
- run 5 under BRIEF-run5.md (the D6 guardrails restated): ProjectArchitect 2.0 §1–§9 then the kit Steps 0–10, unattended from
  answers.md; 4 trailer-free commits; stopped_at null; manifest 57 == 57 (docs/inherited-record.md included); G 67; seeds 34/34;
  check-ignore 9/9 + 5/5; placeholder audit clean; the audit exits 0 in the throwaway; PhaseEnd_Phase0.5 + the archived log; tree
  clean; settings/statusline/memory/ghidra/dumps/~/.claude guardrails unchanged
- the agent's notes, acted on: the installer named pa-overlays.md without its templates/ folder at Steps 6 and 8 → `$KIT/templates/…`;
  its own write-scope slip (two staging files under /tmp, deleted) recorded; the manifest rule's silence on a file the installer
  creates and deletes itself noted
- the judge: my run-5 BEFORE snapshot stripped the first porcelain line's status space and so failed to exclude a deletion under the
  dry-run's own directory (the false flag of the first judge pass — re-filtered, not re-snapshotted); the judge's reader crashed on an
  empty dirty set (a one-token line) → tolerated; run-4 evidence kept under the .run4 suffix
- wiki page (five dry-runs, the run-5 row), Home/Tools/README/SETUP rows: five runs
2026-09-07 23:53:11 -06:00
Drew T 0c29c9e16b tools+kit(phase-33.5): task 14.5 part 1 — the record as the third dictionary + kit_coverage (every rule and every accelerator entry cited or dispositioned)
- decomp-architect/corpus/record/: the how-to (13), decision-log, accelerators, retrospective, story, wave-playbook, effort-map,
  gen3-standards, gen3-handoff, DIGEST and every PhaseEnd (34) verbatim behind an authored front page (what each is, how to
  read it, what is NOT there — the phase logs, R19 — and that the mining pass is their distillation); tool_census: RECORD_SOURCES
  + record_dest + the third corpus in plan/write/check (358 copies + 28 pointers, --check OK); kit_lint exempts corpus/record;
  SETUP Step 6 gains 2c docs/inherited-record.md (+ the verify line; expected-manifest +1); ops-setup/README/tree/methodology/
  wiki page/Home/Tools page/README bullet/SETUP row: "two dictionaries" → three
- tools/kit_coverage.py (+ config/kit_coverage_map.tsv): derives R1..R83 from DIGEST §3 (asserted contiguous) and the 58
  accelerator entries (headings + numbered items), asserts each is cited by a provenance line of the registry seed / the
  kernels or dispositioned (G / DK / FOLDED:G / ENV / PA / SEED: / KIT: / RECORD / COOKBOOK / NOT-PORTABLE; unknown ids
  refused); first run: 26 uncited rules + 21 uncited entries → DK-66 (a ledger's tie-break, a checker's widening and a blanket
  commit are part of the instrument — R70/R80/R52), DK-67 (the ignore file's directory-form wall — S91 (1)), DK-68 (a
  summarised signal is a claim, not ground truth — R14/R66) in a new kernels section 8 (the museum is 9; "In all" 68) + 41
  dispositions (15 PA, 3 ENV, folds into G6/G18/G38/G66/DK-12/19/20/22/25/26/31/35/44/45/46/57/61, 1 KIT template, 1 COOKBOOK);
  now 0 UNCOVERED on both populations; wired into tools-health after tool_census --check; SETUP row + dictionary row
- verify: tool_census --check OK; kit_coverage OK (rules 57 cited + 26 dispositioned / 83; accelerators 41 + 15 / 58);
  kit_lint OK; doc_links --strict OK; wiki_render --selftest 32 pages / 0 unlisted
2026-09-07 23:03:34 -06:00
Drew T 9937fd7e39 docs+kit(phase-33.5): task 14 — the S91-b types hindsight implemented in the kit + the wiki page Start-a-new-decomp-project.md + the rows
- kit: DK-65 "types are a banking lever and a width lever, not a byte lever" (kernel, when, cost, calibration fence, provenance;
  the kernels file now states its own total "In all: DK-1 … DK-65" and SETUP Step 6 compares grep -c against it instead of a typed
  64); intake row 6's milestone gains the canonical type layer (one definition per shape, widths proven by the bytes at bank time,
  a bank refused for a duplicate definition or a raw address cast) + the type tools moved forward + G62/DK-65 in its columns; row 10
  reads "short if Part C and Phase 6's type layer held"; G62 extended with the bank-time clause (G1–G67 kept); tool_dictionary:
  lift_types + canon_sig_reconcile P10 → P6 and the five type tools' need-keys name the type layer at Phase 6 (the phase column is
  one token — the corpus dir derives from it); the cookbook front page's type-verdict sentence (a NAME never moves a byte, a WIDTH
  or SIGNEDNESS is the one place a type does, the permuter cannot reach it); the methodology's "Types — the two-sided verdict"
  paragraph; the kit README's table + tree now name the two dictionaries and layout-contract.md; make kit-corpus regenerated the
  tool index, the MANIFEST and the corpora (302 copies + 28 pointers)
- wiki: docs/wiki/Start-a-new-decomp-project.md — the three steps, what it installs / does not (the two dictionaries), the ladder
  (11 rows), the five AI-use rules, the six inversions (raw casts → declared symbols; DK-65), the compiler question, the accelerators
  one line each (58 rows), the four dry-runs, how it is kept honest; sidebar + Home rows; Tools-from-this-project rows for the kit
  and the tool index; README bullets for both; SETUP row for decomp-architect/ (R21); Where-the-project-goes-next links the page and
  records the probe PASS (2026-09-07); phase34-seed: task 0 PASSED, the kit carried whole through the flip and split later
- verify: wiki_render --selftest 12/12 + reachability 32 pages / 0 unlisted; doc_links --strict 58 documents / 451 links / 0 pending /
  0 broken / coverage 65 of 65 (one BROKEN on the first draft — a foreign project's docs/ path cited in backticks — reworded);
  kit_lint OK (leak 0 / placeholders 22 == 22 / syntax 0 / gitignore 75 identical); tool_census --check OK; 65 DK ids cited, 0
  dangling; audit_public OK on the touched files; no build input changed
- log + checkpoint (NEXT = task 15, Max, Tier 1 — prompt Drew and wait for gate 2)
2026-09-07 22:40:17 -06:00
Drew T b46358f279 docs(phase-33): the Gen3 charter docs/gen3-standards.md (accuracy met + stricter than the field; sotn STYLE.md adopted + four rules of ours; the use-of-AI conduct; the measured gap; a definition of done), README 'Standards, and how the AI was used', docs/phase34-seed.md (RE-CHARTER, Drew 2026-09-07: P33 closes now at v1.32.0; the flip-gated remainder = Phase 34 = the flip + Gen2 exit at v2.0.0; Gen3 = Phase 35+); doc_links + SETUP rows; checkpoint §0a + header; log 2026-09-07 14:46:53 -06:00
Drew T c05c6081bd docs(phase-33): README 'Tools from this project' section + wiki page (sidebar, Home row): xsig, the permuter driver (upstream pinned unmodified + our class swap; the only copy lives here), the codegen map, the decomp.me replica, the drafter write-up, the how-to; docs/outreach/tools-announcement.md stored for post-flip (developer's voice); checklist 5/5b (the wiki can be pushed NOW while private; xsig topics, fork deletion, the post); wiki_sync message corrected; SETUP rows; doc_links default 2026-09-07 14:20:05 -06:00
Drew T 848f294fc8 docs(phase-33): D1 the README rewrite — the claim and the contract, a GENERATED numbers block (tools/progress.py --json -> docs/progress.json, --readme rewrites the marked block, --check asserts freshness; wired into make report), what is not our C, build-from-your-own-disc, layout, how it was made (ProjectArchitect), about the history (tokens + commit-map), license split, no-ROM policy, special thanks; forward references to D2/D3/F1/F2 files recorded in the checkpoint 2026-09-07 00:51:24 -06:00
Drew T de35505a63 feat(phase-19): scale the toolkit — 2 waves (88%/92%), fleet 56.64%->58.00%, propagation is the cap (v1.18.0)
- T1 (prior session, commit:0158/commit:0159): per-file -O0 split infra + 6 -O0 matches; deferred the
  10 %lo-folding-quirk fns + the ×134 rollout (R14 corrected "free ×134")
- T2 (commit:0161): recovery tooling. R14: the 3 Phase-18 "implicit-int" near-misses are NOT
  implicit-int -> resident-callee link-miss (NEW canon_resident_calls.py) + shared-caller arity
  (no-proto caller-extern fix). 3 matches; cookbook §17a-3 corrected
- T3 (commit:0162/commit:0163/commit:0164): 2 Ultracode waves over reach-134 tractable residuals. batch-1
  44/50 match_one (30 banked, 25 propagated ×134); batch-2 35/38 (25 banked, 10 propagated).
  NEW fix_arity_callers.py (no-proto, ALL 18 batch-2 failures were the arity class); canon-first
  2-stage gate (sig_unify REGRESSES canonical drafts); gen_harvest_targets garbled-sig fix
  (0/749, 88%->92%); build_engine_types --strip lift (+4 propagated)
- FINDING: propagation (not matching) caps the fleet-% yield — typedef/local-type bodies can't
  auto-propagate (-> Phase-20 type-lift lever). decomp.dev-comparable byte-weighted est ~30%
- check-all 136/136 BYTE-IDENTICAL (R22); dedup 1450->1485/0 failed; 0 NON_MATCHING (G4);
  fleet 56.64% -> 58.00% (+1.36%). worklog -> phase-ends/logs/Phase19.md (R19)
- bumps 1.17.0 -> 1.18.0
2026-06-20 20:33:00 -06:00
Drew T b7b34f007e docs(phase-12): PhaseEnd — key engine systems complete; resident 1.4%->85.6% (v1.12.0)
Phase 12 close (milestone confirmed by Drew, gate 2):
- resident engine REAL 1 -> 123/146 = 85.62% byte-identical, 0 NON_MATCHING; both binaries
  byte-identical from a clean rebuild (resident 8e17e02f, main 143dbb89)
- pivots (byte-driven, Drew-approved): T1 the resident has NO PsyQ library footprint (all
  custom engine code, REAL not LINKED); T4 NO bytecode VM (script/event = compiled-MIPS
  fn-ptr-table dispatch, byte-backed); T5 save format documented, deep data + SaveLoadRoutine
  code-match honestly deferred (Gen3/P9)
- method: Ultracode harvest = swarm parallel-draft + incorruptible byte-gate (5 passes) +
  match_one iteration loop + extern-conflict recovery; tools harvest_verify.py + match_one.py
  (-> Phase 13). cookbook §12, R26 (prompt for ultracode on breadth)
- gen2-roadmap milestone + EXIT criterion #2 AMENDED (script/event resolved; save documented)
- PhaseEnd_Phase12.md; CURRENT_PHASE.md archived -> logs/Phase12.md (R19); README status
- rule R26; bumps project version 1.11.0 -> 1.12.0
2026-06-16 14:42:52 -06:00
Drew T 190afe10f0 docs(phase-11): T7 close-out — 4.7 sha-record, cookbook §11, SETUP §6.8, README/worklist
- tools/psyq/CHECKSUMS.sha256: integrity record for psyq-4.7-converted.zip (R20) + Phase-12
  provenance (the resident is 4.7 -> link from conv47/, not the EXE's 4.0 libs; R24)
- cookbook §11: the cross-binary dedup & code-share workflow — source-level macro share (NOT an
  object swap; game fns are interior to one object/binary, D1), the byte-gate, sig_image notes
  (h_exact workhorse / self-consistent h_norm D2 / linear-partition+code-end overlay boundaries),
  the cross-report as the Phase-12/13 work queue, per-binary SDK provenance (R24)
- SETUP §6.8 + tooling inventory: sig_image.py, dedup_integrate.py, dup_report --cross,
  make sig-overlays, config/dedup.us.yaml + src/shared/ (R21)
- docs/psyq-worklist.md: resident PsyQ is 4.7 (Phase-12 linking note)
- README: Gen2 status — resident is the 2nd byte-identical binary; cross-binary dedup live
  (~9000 cross-binary groups / ~28 MB collapsible; one engine fn byte-identical in all 134 overlays)
2026-06-16 01:31:50 -06:00
Drew T 6345b5012a docs(phase-9): T10 — binary-agnostic toolchain docs + final milestone proof (R16/R21)
- SETUP.md §6.7: the binary-agnostic toolchain (make build BINARY=<alias>, the
  required --vram-base/--exe/--symbols flags, BFM_BINARY for asm-differ, and the
  future-binary path convention config/splat.<bin>.yaml / build/<bin>/ etc.)
- matching-cookbook.md §9.7: the reusable refactor pattern for Gen2 (required params/
  no defaults, --vram-base scalar vs window, transitional-default + leaf-first
  technique, the negative control, the progress.py regex lockstep gotcha)
- psyq-worklist.md: per-library recipe updated with the now-required flags
- README.md: status -> 52 REAL / 959 LINKED / ~50% byte-identical / Gen2 underway
  (binary-agnostic toolchain)
- MILESTONE (all green): EXE rebuilds 143dbb89 with AND without SDK objects; report
  52/959/7/50.24%; wrong --vram-base -> cae22f7e (negative control); expected/ refreshed
2026-06-15 21:09:06 -06:00
Drew T 49db81d8ae feat(phase-7): Gen1 close-out — matching loop industrialized, PsyQ libs linked, LZSS matched (v1.7.0)
- reports (Task 3): tools/{progress,difficulty,dup_report}.py + make report/sig-refresh
  (deterministic digests); harvest to 43 REAL matches incl. LzssDecodeSector
- per-module -O0 split (Task 5.5): src/boot.c + Makefile per-file CC1FLAGS (cookbook §6)
- PsyQ SDK linking (sessions C-G): split/build/identify/link/integrate pipeline
  (tools/psyq_*.py, make_libgs.sh, ld_interleave.py, split_src_region.py) — libcd (18 obj)
  + FULL libgs (31 obj / 6 blocks) linked BYTE-IDENTICAL from real PsyQ 4.0 libs; byte-
  identical with or without the SDK objects (stub fallback); cookbook §7/§8/§9.1-9.5
- LZSS MATCHED byte-for-byte (session F): §5a cross-jump barrier + §10 regalloc/schedule
  idioms, ground-truthed against pinned gcc-2.7.2 source (the Gen1 LZSS gate)
- file-loader cluster: 6 matched + 5 NON_MATCHING-drafted (CdReadStateMachine,
  CdReadSectorReadyCB, StreamLoadStateMachine new this session, valid under -DNON_MATCHING);
  SaveLoadRoutine DEFERRED to Q#5 (save/memcard, documented stub)
- README Gen1-complete; worklog preserved -> phase-ends/logs/Phase7.md
- rules R17 (web-research compiler quirks), R18 (plain-English session summaries),
  R19 (preserve phase worklog out of load order)
- MILESTONE: Gen1 EXIT confirmed — byte-identical build, 43 matches incl LZSS, 7 zero-
  regression sessions, loader cluster matched/drafted, overlay map emulator-proven
- bumps project version 1.6.0 -> 1.7.0
2026-06-15 13:44:32 -06:00
Drew T d4334fc1ca plan files created 2026-06-10 22:02:07 -06:00
Drew T 23066aa571 Initial commit 2026-06-10 19:39:19 -06:00