Commit Graph

44 Commits

Author SHA1 Message Date
Drew T 02f060f607 feat(phase-31): S79 #5 — the libpad 4.2.1 + libapi 4.2 band and the apicard region LINKED from real objects: 13 stubs + 4 TUs + the reorder island gone; main 16 stubs, fleet 38
800c3 (0x8005CE18-0x8005FC68, one contiguous run of 33 interleaved Sony objects) is now four
stub rows — libapi1 (21 BIOS trampolines + COUNTER), libpad1 (PADENTRY + PADMAIN 760), libapi2
(L02/L03), libpad2 (PADCMD PADIF PADPORTD PADSEQD WAITRC2) — fed by two WINDOWED psyq_integrate
calls from the raw .run/obj42/{libapi42,libpad421} dirs (integrate tiles each stub with one
library; every boundary checked against .text SECTION sizes). The apicard region's three
"game code" rows were libapi 4.2's C objects to the byte: 800c2 = FIRST.o (firstfile + the
"no jump table wall" stub func_80062144), 800c2_2 = PAD.o, 800c2_3 = PATCH.o + CHCLRPAD.o ->
apicard5/6/7; make_apicard_used.py sources libapi from 4.2 (the EXE's real libapi; libcard
stays 4.0) into .run/obj42/apicard_used, 26 objects / 7 blocks, no game code left in
0x80061F38-0x80062888. src/800c3.c (129 hand-matched "C", 62 verbatim bodies, 19 stubs incl.
the four §332 %lo-in-a-delay-slot "walls"), src/800c2.c, src/800c2_2.c, src/800c2_3.c removed;
REORDER_TUS is empty (mechanism kept). Cookbook §490.

Two stale instruments fixed: exclude_audit let a pinned WALL outrank LINKED (PopMatrix/
PushMatrix had sat as walls since S68 while living in libgte3, linked since Phase 8) — LINKED
dominates now, config/wave_exclude.txt 13 -> 3; frontier_classify carried a hard-coded 49-name
LINKED set (R51) and reported 337 "stubs" — derived from the Makefile now.

Verified: main 143dbb89f34491258bbc27810d0a12ec8b43a8dd WITH all SDK dirs and WITHOUT them from
a fresh extract; make tools-health OK; R22 fleet extract-all 212/212 + check-all 213/213.
Metrics: main REAL 839->773, LINKED 1,150->1,256, VERBATIM 29->3, stubs 29->16, byte-identical
2,075/2,091 = 99.2%; game-code weighted 93.3% (38,748/41,534), remainder 2,786 = the open-stub
sum; fleet stubs 51->38 (frontier_classify: 39 rows incl. the data word). Verbatim manifest
33 -> 6. Docs: worklist rows + "S79 task #5", SETUP (fresh-clone obj42 commands, Makefile
blocks, exclude_audit), decision-log "S79 addendum 2", accelerators "S79 (2)", CURRENT_PHASE
S79 FINAL refreshed (census, metrics, the task #6 brief).
2026-09-04 17:56:31 -06:00
Drew T 757bd82a0f feat(phase-31): S79 #4 — scattered-.bss split at link-prepare (psyq_bss_split): SYS.o→libgpu2, VM_F.o→snd12, GS_001.o→libgs8 LINKED; libgpu_used retired
The §9.1 "scattered .bss commons" exclusion class (Phase 8 → P31) is closed 3/3. New
tools/psyq_bss_split.py (own ELF32 REL reader/writer) cuts an object's packed .bss into
per-base NOBITS pieces: bases derived from the game bytes per HI16/LO16 pair, references
walked in offset order into single-base runs, cuts snapped to symbol starts (the linker
scattered SYMBOLS), symbols moved, a LOCAL section symbol per piece inserted, relocs
retargeted with the addend rewritten in the immediates, self-diffed. It runs inside the one
prepare step shared by psyq_link.link_object / psyq_link_region.build_region /
psyq_integrate.integrate (prepare_object before classify), re-derived every build.

GS_001.o was certified "5 interleaved bases, NOT splittable" by the S77 probe, which grouped
by BASE; by RUN it is six symbol-aligned pieces. All seven cuts across the three objects are
confirmed by the other objects' by-name recoveries (_que 0x800C5510, _svm_sreg_buf
0x800B9B58, PSDBASEX/CLIP2/PSDBASEY/POSITION/GsDRAWENV). R39 negative control: 235 placed
objects across 9 curated dirs, 0 refusals, exactly 3 splits (a libcd .bss+size end pointer
refused the first build → reference problems are fatal only when a split is needed).

Wiring: yaml 800c→libgpu2, sgap_6→sgap_6+snd12, gsgap3→libgs8 (comments rewritten);
LIBGPU_ELF := .run/obj40/libgpu (curated libgpu_used retired); libgs 34 objs/8 blocks
(make_libgs.sh +GS_001); snd 63/12 (make_snd_used.py exclusions 4→3). src/800c.c and
src/gsgap3.c removed (Sony code hand-matched as REAL/verbatim), sgap_6.c keeps only
func_8003FA54; splat-emitted libgpu2.c/libgs8.c/snd12.c stubs for the no-SDK fallback.

Verified: main 143dbb89f34491258bbc27810d0a12ec8b43a8dd WITH the SDK objects and WITHOUT
them from a fresh extract; make tools-health OK; R22 fleet clean extract-all 212/212 +
check-all 213/213. Metrics: main REAL 886→839, LINKED 1,040→1,150, VERBATIM 85→29, stubs 29
(unchanged); game-code weighted 91.1% (40,895/44,870) — both terms lost the 3,667 SDK ins;
the remainder is still exactly the 3,975-ins open-stub sum. Verbatim manifest --update
200→33 rows (subtractive). Docs: cookbook §489 (+index), psyq-worklist rows + "S78 task #4",
SETUP S79 R21 table, decision-log S79 addendum, accelerators S79, CURRENT_PHASE S79 FINAL 🛑.
2026-09-04 17:19:29 -06:00
Drew T a85733a487 feat(phase-31): S78 #3 — 13 "game code" subsegs were PsyQ objects: wired LINKED (libgte 70/30, libgs 33/7, snd 62/11); main's game-code metric corrected to 91.8%
- exact tiles, 0 tokens: libgte23-26 (MSC01/02/05/09, SMP_00, FGO_01-06, PATCHGTE), libgte9 re-derived
  as SMP_05 NormalClip (SMP_06 NormalClipS = nested sub-pattern; psyq_integrate now drops nested
  placements), libgte27-30 (the libgs-gap MTX_05/07/11, REG03+REG11), libgs7 (2D_BG0+2D_BG1), snd10
  (VM_NO1), snd11 (VM_NOWON carved off sgap_8). LINKED 959->1040, REAL 912->886 (SDK inline-asm wrappers
  re-provenanced), VERBATIM 146->85, 13 TUs deleted; splat re-emits the stub records.
- main 143dbb89 WITH and WITHOUT the SDK objects. The no-SDK fallback had been red since S7x
  (CdReadyCallback called by its SDK name while the libcd stub carried func_800435B4) — curated
  CdReadyCallback = 0x800435B4, refs unified. R22 clean fleet 213/213; tools-health OK.
- METRIC CORRECTION (R35): progress.py's "MAIN game-code weighted" sig never excluded the LINKED
  objects (its comment said it did) — ~31k linked-SDK ins sat in the denominator as unmatched game
  code. Exclusion now derived LIVE from the Makefile stub lists + yaml ranges: 91.8% (44,562/48,537),
  not 59.8%; the 3,975-ins remainder equals the open-stub sum exactly.
- VM_F.o probed SPLITTABLE at .bss 0x50c (SYS.o's class -> task #4). cookbook §488; worklist S78 #3;
  decision-log + accelerators; SETUP rows.
2026-09-04 16:26:12 -06:00
Drew T b212f40f19 chore(psyq): bank the PsyQ 4.6 library zip + 4.5 toolkit zip (R20 hard-to-source SDK material; sha256 in CHECKSUMS); lib46/ is derived (ignored); progress.md regenerated 2026-09-04 15:58:09 -06:00
Drew T c9454db4a4 docs: refresh progress.md from the tools-health report run 2026-09-03 22:57:49 -06:00
Drew T 375507834c docs(progress): main's R34 caveat is retired — its boundaries are independently verified now
The fleet report still printed 'caveat is R34: no independent second oracle for
a PS-X EXE'. That was true until this session; make sig-main-oracle +
audit-corpus now cover main at 0 phantom / 0 truncated / 1 explained pad-tail.
A stale caveat is the same class of false statement as a stale wall verdict.
2026-09-03 22:50:43 -06:00
Drew T ddbe7f455c docs(R31): S77 decision-log — the frontier is wall-proof work now, and R61's twelve defects
Also: cookbook §486 (the main -O0 island carve, five coupled pieces), SETUP rows
for psyq_bss_probe and make sig-main-oracle (R21).
2026-09-03 22:48:58 -06:00
Drew T 4a0f9a3049 docs: regenerate the cookbook index for §477-§483
tools-health caught this red: seven sections added this session without
regenerating the index. Exactly the sibling-update the health gate exists to
enforce.
2026-09-03 21:16:24 -06:00
Drew T bfc0f43c92 docs(phase-31): S77 CLOSE — 30 banked, main 57.1%->59.4%, ten instrument defects, R61
S77w wave: 30 workflows, 30/30 reported, 9 banked, 21 NEAR, 0 errors.
R22 clean-fleet 213/213 (fourth run this session). Cookbook §477-§483.
2026-09-03 20:52:15 -06:00
Drew T 9d15598b5a docs(phase-31): S77 FINAL checkpoint — 21 banked, self_decl_tu closed, the permuter yield curve, eight instrument defects 2026-09-03 19:51:02 -06:00
Drew T b5751c7c1e docs(phase-31): S77 checkpoint — 17 banked, the self_decl_tu lane closed, six instrument defects
T11 4/7, T12 13 banked of a 34-draft pool, T13 R22 213/213 twice (a green
baseline before the overlay banks and again after all 17).

main REAL 895 -> 899, stubs 46 -> 42. Fleet stubs 82 -> 65, distinct-code
99.3% -> 99.4%, MAIN game-code 57.1% -> 57.3%.
2026-09-03 19:25:02 -06:00
Drew T c05ea15cbe feat(tools): asm_in_c.py — 154 game functions are assembly wearing a .c extension
A .c file in src/ looks decompiled. 199 functions are not: they are the target
assembly pasted into a C string literal (§265), byte-identical BY CONSTRUCTION
and completely unexplained. 45 are PsyQ/CRT routines where that is defensible;
154 are GAME CODE, 171 of the 199 in main, the largest being SaveLoadRoutine at
1,165 instructions.

They were invisible because progress.py's classify() matched INCLUDE_ASM,
INCLUDE_RODATA and C definitions, and a file-scope __asm__ block is none of
those -- so each landed in NO bucket, either swallowed by a surrounding
construct or surfacing as the single `UNPLACED (parse hole)` line the tool has
been printing all along.

progress.py gains a VERBATIM __asm__ bodies line: counted byte-identical (it is,
by construction) but NEVER as REAL. main's headline moves 45.88% -> 42.15%.
Nothing regressed and no work was lost -- the denominator was missing 173
functions that are real remaining work.

THE COUNTING LESSON IS THE REUSABLE PART. Counting these by hand went
116 -> 112 -> 108 -> 178 -> 199 across five attempts in one session, every
intermediate number reported confidently. All five errors were one shape, a
pattern narrower than the claim it supported:
  * the sources use BOTH ".ent\tNAME\n" and ".ent NAME\n" -- anchoring on either
    silently drops every instance of the other;
  * a bare ".ent\t" fragment yields a phantom function literally named `t`, six
    times, which is the only reason the error was noticed;
  * __asm__ appears in 3,182 of 4,224 sources, almost all the §3a barrier, so
    counting files or counting __asm__ measures nothing;
  * `.globl NAME` + `NAME:` proves EXPORT, not CODE -- the first real run
    reported jtbl_80072ED4/EEC/F0C/F24 as four "functions";
  * a hand-written SDK name list reported 170 game functions because it did not
    know VectorNormalSS / SquareRoot12 / OuterProduct12 are libgte.

So the tool does not trust one regex: THREE independent detectors that must
agree with disagreement reported as a defect (R34 -- that is what caught the
jump tables); SDK-ness DERIVED from the 14 shipped PsyQ archives via nm (2,227
symbols) rather than a list (R33); coverage asserted so a definition-shaped
block no detector claims fails loudly (R32/R43); and --selftest carrying a
known-true case of every spelling plus the phantom `t` and the jtbl regression.

Cookbook §448, SETUP row. Law: when a count comes from a text pattern, the
pattern has a denominator too -- validate it against one known-true case of
every FORM the corpus contains before quoting the number.
2026-09-03 00:01:11 -06:00
Drew T b24394e732 docs(phase-31): S74 FINAL checkpoint — 50 banked, seven tool defects, R22 213/213
Every reject class this session was an instrument defect, not a codegen wall: 24 already-MATCHed
bodies were sitting behind seven tool bugs, six of which are now fixed. The checkpoint names the
seventh (reconcile_tu manufacturing declaration conflicts) as the first thing to fix next session,
and the harness gap (verify_worktree omits the sig files, and the scan that needs them swallows the
error and reports a false corruption) beside it.

Fleet verified from a clean rebuild AFTER the last bank: check-all 213 passed, 0 failed of 213.
split_indicator 213 OK and now a hard gate. INCLUDE_ASM lines in src/ 1,086 -> 1,036, measured at
both commits — the same 50 the gates reported, counted independently from the source.
2026-09-02 19:40:46 -06:00
Drew T 9f8242d63c fix(progress): the #else half of a NON_MATCHING block is LIVE — REAL was undercounting by 7
classify() consumed everything from '#ifdef NON_MATCHING' through '#endif', swallowing
the #else half. But banking replaces the #else INCLUDE_ASM with the real body and leaves
the old attempt in the dead half — so every function banked that way landed in NO bucket:
not real, not a stub, invisible in both numerator and denominator.

Measured: CdReadStateMachine, CdReadSectorReadyCB and StreamLoadStateMachine are
byte-identical in the shipped build and counted as zero. REAL 873 -> 880, matchable
1911 -> 1918 (seven functions fleet-wide, not the three I first checked).

Now consumes only the DEAD half, then decides from the LIVE half: an INCLUDE_ASM there
still buckets as NON_MATCHING (accounting unchanged), anything else rewinds and is
classified normally.

THIRD coverage defect of this exact shape in this one function — the K&R-definition case
(~190k instructions erased) and the '#if 0' case are both documented in its own comments,
which is what pointed me at it. A scanner that walks preprocessor structure needs a test
per branch, not per directive.

Found by the S73 documentation audit, which I had written off as producing only doc typos.
2026-09-02 17:14:32 -06:00
Drew T 2c1d21bb9d docs(phase-31): S73 CLOSE — R22 213/213, 23 main banked this session, main 56.5%, frontier 124
The jump-table class on main is resolved: 25 -> 2, and both survivors are the §434 frame
pair, provably unmatchable as separate C functions (resegmentation, not drafting).

Wave S73m_1 banked 9 of 9 drafts (2,413 ins). Cookbook entries written this morning
cracked functions this afternoon; two of mine were refuted by later MATCHes and rewritten.
2026-09-02 16:34:13 -06:00
Drew T 09886be66b docs(phase-31): S72 FINAL — R22 213/213, 14 main banked, main 52.6%, frontier 133
The carve + the src/800.c split at the original TU boundaries. 7 of the 14 banks were
span B/C — impossible before the split. Next session starts with 11 functions /
4,479 instructions that are now merely undrafted rather than unbankable.
2026-09-02 13:54:16 -06:00
Drew T 8e8521da22 fix+docs: make every consumer aware of main's new TU layout (R36)
The split created two new TUs and a shared header; four consumers still described main's
game code as one file:

* tools/reconcile_slate.py — HARDCODED open('src/800.c'), so after the split it saw a
  THIRD of main's typedefs while reporting success (silently-narrowed scope, R32).
  Measured: 133 visible before the fix, 187 after, 0 lost. Now globs
  corpus.src_files('main') + src/800_shared.h, so a future split is already handled.
* docs/wave-playbook.md 1c — still said spans B/C/D were NOT drawable and that drawing
  one is an R45 violation. That is now false and would have STOPPED a future session
  from drawing the very targets this work unlocked.
* cookbook §426 — its 'the remaining spans need src/800.c split' paragraph now records
  that it was done the same session, and points at §431 for the method.
* config/dedup.us.yaml + src/shared/clearTbl40.h — both said dedup group I0 is
  instantiated 'at both sites in src/800.c'; both sites are above 0x80035270 and are now
  in src/800_c.c.

Byte-neutral: dedup.us.yaml parses, gate_main --assert-baseline BYTE-IDENTICAL.
SETUP.md gains a row describing the layout and the rule it implies: never hardcode
src/800.c, glob corpus.src_files('main').
2026-09-02 13:48:00 -06:00
Drew T f0148e6822 docs(phase-31): S72 CLOSE — R22 green 213/213, 7 main banked, main past 51.8%, frontier 140
The 11 'PROVEN gate-rejects' were one missing rodata carve, not bad bodies. Next
session starts at the span B/C carve: 18 jtbl functions left in main's frontier holding
most of its remaining instruction mass, blocked on splitting src/800.c at 0x8002B0B4
and 0x80035270 — the original TU boundaries the jtbl spans reveal.
2026-09-02 12:58:44 -06:00
Drew T 9631949339 docs(phase-31): S71 CLOSE — R22 green 213/213, frontier 147, 63 banked, main past 50% 2026-09-02 11:15:21 -06:00
Drew T 64dc914afe chore(report): refresh digests after the §332b banks 2026-09-01 20:46:09 -06:00
Drew T 4cf531beea docs(cookbook): §399 — four levers from the final S69 round, harvested late
Caught by Drew asking whether the last waves were harvested. They were not: I
banked 1 of 5 (§398b) and left four lever sets in the notifications. Also found
two paid-for MATCHes that were never staged or gated.

(a) a fence BETWEEN two prologue loads, where source reorder does nothing —
    the order is fixed before statement order matters (md_MAIN_013/func_800CB56C)
(b) SINK a call into BOTH arms and let cross_jump keep only the jal suffix;
    88ins/close86 -> 92/13, then §3-T2 field order let each sh $zero fill an lhu
    load-delay. Duplicate in source so the compiler merges, rather than writing
    the merged form yourself (ov_SC07_001/func_8017EDC0)
(c) a $v0->$a0->$s3 DOUBLE COPY is a two-pseudo tell: SImode temp for the compare
    + separate HImode var for the tail (70->37); plus §195-N precondition 5 —
    nesting `return 1` with ONE trailing `return 0` blocks jump.c's store-flag
    transform so reorg fills both delay slots (18->0) (ov_SC02_017/func_8018347C)
(d) the re-tie as a BIV KILLER: a second set makes n_times_set>1 so loop.c
    refuses the pseudo as a biv, killing the combined address giv. volatile was
    worse, a dead read did nothing (ov_SC07_001/func_8017E4DC)

(d) makes THREE distinct uses of the zero-byte re-tie in one session — §380
un-hoists a move_movables invariant, §393 kills the scheduler's birthing boost,
§399d denies a biv. One line, three passes: when a single-set pseudo is being
treated specially, give it a second set.
2026-09-01 15:18:40 -06:00
Drew T c4380c19e4 feat(p31 s67): stranded-draft census + honest jtbl probe — the frontier's biggest class is carve plumbing
MEASURED (denominators in .run/S67_findings.md):
* 193 of the 530 open functions ALREADY have a draft on disk (1,885 wave targets seen,
  1,521 banked, 171 open-no-draft, 166 never drawn). Classified in their real TUs:
  37 MATCH / 67 NEAR / 89 CC1-FAIL.
* 159 open functions (30% of the frontier) reference a jump table; 96 are PLAN-REFUSED
  by build_carve (non-contiguous same-subseg .rodata), 75 non-main across 38 subsegs.
  Not a codegen wall and not a decl wall — carve plumbing.

NEW
* tools/strand_census.py — coverage-asserted census + rtu_match classifier + draft staging.
  Keys binary:fn (R48); classifies each pair once after merging every manifest's view.
* tools/o0_detect.py — the -O0 prologue tell extracted from match_one (which parses argv at
  import and therefore cannot be imported). match_one re-exports it; ONE definition (R33).
  Wiring it into the classifier turned md_MAIN_003 from 8 NEAR (7 of them >20) into 6 MATCH.
  Negative-controlled both directions.
* tools/scope_demote_drafts.py — §8d as an _xform-contract gate rung. NOT yet exercised.

FIXED
* jtbl_carve --probe now runs build_carve (a pure planner) and reports plan-refused. It
  previously called only island_probe, which answers a necessary-not-sufficient question —
  every blocked function probed "carveable", and S66 priced 32 of them as free on that.
* blocker_probe.macro_scope selects the LAST #define per macro name, matching cpp.
  engine_core.h has 1,037 duplicate DEFINE_func_ names and 4 with DIFFERENT bodies.

NOT VALIDATED — DO NOT SCALE
* jr_isolate_all: two real defects fixed (carried types deduped by name; header-provided
  types no longer re-emitted) but ov_SC02_000 STILL fails the byte gate after them.
  Open lead: file_scope_types carries a block without its enclosing #if guard. 20 of 35
  blocked overlays dry-run clean and that number means nothing until one round-trips.

0 functions banked this session. tools-health has ONE pre-existing cdecl defect
(1 of 74,749 declarations, func_8017EE08_p55352/struct ZnRec) — cdecl.py and its inputs
are byte-identical to HEAD, so it is not from this change.

Knowledge banked: cookbook §322/§323/§323a/§323b, decision-log pivot, accelerators #13/#14.
2026-08-31 01:46:16 -06:00
Drew T d804449be5 fix(distill): carry banked=False through to the distiller and the verifier
The distill prompt asserted 'the whole-binary byte-gate ACCEPTED the final draft, so the final body is
ground truth' for EVERY target. With --with-unbanked now feeding it drafts the gate REFUSED, that
sentence would have laundered an unproven body into a byte-proven cookbook entry (R14/G3). A target
with banked=False now gets an explicit PROVENANCE WARNING telling the distiller to extract the lever
anyway (cookbook 52: a model that FAILS still distils the idiom that cracks its siblings) but to mark
every claim UNPROVEN and never assert byte-equality; the verifier is told the same so its
entry_markdown carries the label. Exercised this session: the one surviving ADDENDUM came from an
unbanked draft and is filed UNPROVEN.

Also regenerates the scoreboard after the day's banks.
2026-08-29 19:03:28 -06:00
Drew T e5956d021f docs(report): progress.md regenerated with the S63 scoreboard 2026-08-27 11:33:02 -06:00
Drew T 528f543d84 chore(docs): progress.md regenerated at S61 close 2026-08-26 12:56:26 -06:00
Drew T e7ebb1cd45 docs+rules(S58): R42 commit-banked-work-immediately, R43 refuse-unsupported-input
R42: gate_main reverted 61 byte-proven overlay banks it could not distinguish from its own
substitution (sweep_parallel gates commit=False by design). Fixed by committing overlay banks
before the main batch, chunking main at 8 to bound bisect cost, and replacing every blind
'git checkout -- src/ config/' with commit-or-refuse in ox_campaign and idiom_serial.

R43: sweep_parallel had an explicit branch admitting main, which cannot be gated incrementally
— wave ab banked 0/105 main cards while its non-main cards banked 94/115 (82%), and the wave
read as a drafting failure. sweep_parallel now refuses main and names gate_main.py.

Also: validate_targets now prefers the card's own addr field (named symbols like SYS_OBJ_F00
were MALFORMED and discarded whole 220-card waves); ox_campaign deals model lanes by
smallest-ratio scheduling (a 73-card wave had put 73 shards on ox and 0 on deepseek);
docs/accelerators.md gains the four vacuous-check defects.
2026-08-23 12:59:59 -06:00
Drew T 7928e7940f docs(phase-31): S54 log — wave T 70/70 banked, 24 leftovers, 49 mechanical siblings, §192/§193, R22 213/213 2026-08-17 12:58:45 -06:00
Drew T 034c46bc28 chore(phase-31): S53 checkpoint rewritten as a clean fresh-session handoff (161 banked, main 20.4%, §188 correction, 5-step next-session plan) 2026-08-17 10:14:41 -06:00
Drew T 683c44e15b chore(phase-31): regenerate docs/progress.md digest; move a stray 89KB agent-notes dump out of the repo root into .run/ (R12) 2026-08-15 09:09:50 -06:00
Drew T c716eb15e1 feat(phase-15): T4 — fleet progress roll-up + fix cross-report double-count
- progress.py: refactor to report(binary) + set_binary() + a per-binary .s index
  (fleet run 6m38s -> 7s); add --fleet -> docs/progress.fleet.md (deterministic,
  source-derived per-binary table + fleet totals). Single-binary output byte-unchanged.
- fleet now: 947 REAL / 959 LINKED / 13132 byte-identical / 344010 matchable = 3.82%
  across 136 binaries (main + resident + 134 overlays)
- dup_report --cross: HONESTY FIX (R14) — an onboarded overlay is both a named BINARIES
  entry AND a .run/sig.ov_* glob hit; it was counted twice, inflating collapsible bytes
  to 58.7M once the fleet was onboarded. Dedupe by alias -> accurate 9366 h_exact groups
  / 28.6M collapsible (matches the Phase-11 figure)
- Makefile: wire progress.py --fleet into make report (main-only block)
- docs: progress.fleet.md (new), duplicates.cross.md regenerated, SETUP inventory
- make report green; dedup-check 8 validated / 0 failed
- ghidra/ churn NOT staged (R23)
2026-06-16 20:00:19 -06:00
Drew T 52514b5f45 docs(phase-13): T6 docs — cookbook §13 overlay runbook + SETUP/gen2-roadmap + digests
- cookbook §13: the canonical add-an-overlay runbook (new_overlay.sh, the flat-blob
  config, the non-4-aligned bin-carve/asset/trim gotcha, A->B->C workflow, dedup-credit
  two shapes, build-all/check-all) — the Phase-15 fleet recipe
- SETUP tooling inventory: tools/new_overlay.sh (R21)
- gen2-roadmap Phase-13 milestone: ACHIEVED amendment (6 binaries byte-identical,
  704 harvested, dup-pair collapse, T7 effort finding)
- regenerated per-overlay progress/difficulty/duplicates digests + cross report
2026-06-16 17:56:38 -06:00
Drew T a8ae190fa3 feat(phase-11): T3 — intra-binary collapse PROOF (one body, two addresses, byte-identical)
- src/shared/clearTbl40.h: CLEAR_TBL40 macro = the matched byte-clear loop body, authored ONCE;
  instantiated at both func_80037004 and func_80037334 in src/800.c (one source -> two vrams)
- matched first try; clean rebuild main -> 143dbb89 BYTE-IDENTICAL WITH the shared C AND WITHOUT
  it (INCLUDE_ASM stub fallback) -> dual invariant proven (R22 clean rebuilds, both states)
- config/dedup.us.yaml: I0_clearTbl40 group registered (h_exact a0744d60…); dedup_integrate
  --check validates it; negative tests (corrupt hash / wrong vram) fail-closed (exit 1)
- tools/progress.py: count dedup-shared members as REAL via the registry (the macro form isn't a
  parseable function def); REAL 52 -> 54, byte-identical 50.24% -> 50.33%; honest measurement (P9)
- tools/dedup_integrate.py: display vram in hex in diagnostics
- the machinery half of the Phase-11 milestone is proven on the byte-verified EXE
2026-06-16 00:56:35 -06:00
Drew T 44586c885e feat(phase-8): game-code harvest (T13) — 9 trivial accessors, REAL 43 -> 52
- matched 9 game-code leaf getters/setters of D_* globals (byte-verified, asm-differ 0):
  func_8002AEF8 (D_800760C8=0), func_8002AF08 (return D_800760C8),
  func_8002AF60 (return D_80075A40), func_80037358 (D_8007623C=posInt),
  func_8002D4B8 (u8 return D_800A46BA), func_8002D7FC (D_800A4E7C=arg0),
  func_8002D834 (s16 D_8006A990=0), func_8002F648 (s16 return D_800A4EF0),
  func_80037CC8 (D_800BA0F8=0)  — all in src/800.c, extern-declared D_* (undefined_syms_auto)
- byte-identical (clean build); REAL 43 -> 52; byte-identical/matchable 50.24%
- GOTCHA: matching changes need `make clean` to verify — the incremental psyq_integrate
  .ld rewrite can leave a stale state (an incremental build falsely diffed in libmcrd;
  clean rebuild byte-identical). Canonical verify = make clean && extract && build
- bank, don't exhaust: stopped at 9 (more trivial leaves remain for later)
2026-06-15 19:07:59 -06:00
Drew T 11a9bdc032 feat(phase-8): link libapi+libcard combined 800c2 region (T7+T8) — 22 objs / +24 fns
- libapi (BIOS syscall trampolines) + libcard interleave in 0x61F38..0x62888; linked as
  one combined region (4 blocks apicard1..4) via tools/make_apicard_used.py (C112.o shared,
  byte-match dedup, 0 exclusions)
- subsegs via gen_lib_subsegs.py; integrate window 0x61F38..0x62888; region verified 22/22
- rm'd src/800c2.c (region starts with a lib block) -> splat regenerates apicard1..4 + frags
- dual byte-gate PASS: 143dbb89 with and without the apicard objects
- LINKED 935 -> 959 (+24); REAL still 43; byte-identical 49.81%
- libapi's ~22 objects in the 800c3 region DEFERRED (lowest value); SSGM + the scattered-.bss
  exclusions documented in docs/psyq-worklist.md

LIBRARY CEILING: 8 PsyQ libraries linked byte-identical (959 LINKED fns); byte-identical
coverage 20.31% (libcd+libgs only) -> 49.81%. Remaining = documented low-value deferrals.
2026-06-15 18:59:14 -06:00
Drew T 8bbf2e8e4d feat(phase-8): link libspu+libsnd combined sound region (T9+T10) — 60 objs / +225 fns
- libspu and libsnd interleave in 0x3A444..0x4239C (the 800-subseg tail), so they link
  as ONE combined region rather than two tangled passes
- NEW tools/make_snd_used.py: build the curated combined dir — merge both libs by vram,
  pick the byte-matching object per aliased address (link_object), exclude 4 addresses
  that don't reconcile in-region (kept as byte-identical stubs):
    0x3C438 S_R/S_W, 0x3D424 S_GRMDT/FB/T  — scattered-.bss commons, cross-object (§9.1)
    0x3D94C S_IH/UT_RON                      — false placement (inside libsnd SSSTART.o)
    0x3FA64 VM_F (237 ins)                   — scattered-.bss commons (the one real loss)
- subsegs via gen_lib_subsegs.py (9 snd blocks + 8 sgap game/excluded-stub gaps);
  integrate window 0x3A444..0x4239C; region byte-verified 60/60
- src/800.c trimmed at [0x3A444,0x4239C): kept 732 items < 0x3A444 (ALL matched C +
  game + the deferred SSGM stub preserved), 0 real-C moved
- dual byte-gate PASS: 143dbb89 with and without the sound objects
- LINKED 710 -> 935 (+225); REAL still 43; byte-identical 48.66% (nearly half the EXE)
- deferred: SSGM.o @0x1BD80 (isolated, matched-C region, 8 ins)
2026-06-15 18:54:21 -06:00
Drew T c4a627b22b feat(phase-8): link libgte byte-identical (T11, value-first) — 53/58 objs / +120 fns
- NEW tools/gen_lib_subsegs.py: generate splat subseg lines + integrate stub list for a
  multi-block library (section-size-correct block ends — bakes in the libc2/T6 boundary
  gotcha so it can't recur). Reused for libspu/libsnd next.
- libgte = GTE math, 53 objects in 22 blocks across the old 800b region (game code
  interleaved as 800b/800b_2..800b_7). Resegmented from the generator; integrate window
  0x4787C..0x51804 so it sees exactly the 22 in-region blocks
- 5 libgs-gap libgte objects (MTX_05/07/11/REG03/REG11) DEFERRED — gsgap1/2/4/5 stay
  stubs (gsgap2 != MTX_07 exactly, needs a sub-split); documented in worklist
- progress.py linked_subsegs() now resolves `$(VAR)` stub lists (LIBGTE_STUBS) — the long
  multi-block lists are passed via a make var
- region byte-verified 58/58 (clean, no scattered .bss); rm'd src/800b.c (region starts
  with a lib block) so splat regenerates the fragments; 22 src/libgteN.c + 6 game frags
- dual byte-gate PASS: 143dbb89 with and without the libgte objects
- LINKED 590 -> 710 (+120); REAL still 43; byte-identical 37.93%
- reordered remaining libs value-first (libspu/libsnd next; dense libcard/libapi last)
2026-06-15 18:43:27 -06:00
Drew T 9dbae2681e feat(phase-8): link libc2 byte-identical (T6) — 17 objs / +62 fns
- splat: libc2 C stdlib in 2 blocks — main 16-obj run libc2_1 (BZERO/MEMCPY/STRCMP/
  PRINTF/PRNT[jtbl ok]/.../SETJMP) split from 800c, + STRCAT.o libc2_2 split from
  800c2; new game-code fragments 800c3 + regenerated 800c2
- region byte-verified 17/17; PRNT.o printf-format jtbl resolves via NOLOAD .rodata
- BOUNDARY GOTCHA found + fixed: SETJMP.o .text is 0x80 B (8-align pad), not the
  0x78 psyq_identify reports (30 ins) -> the libc2_1/800c3 boundary was 8 B too low,
  overlapping SETJMP's padded tail; the relink inserted +8 padding and shifted the
  whole downstream image (56k diff bytes, +8 file len). Fix: boundary = last obj's
  readelf .text size (0x5CE18), not ins-count. Lesson recorded in docs/psyq-worklist.md
- when a library block sits at a subseg start, rm the old .c so splat regenerates it
- dual byte-gate PASS: 143dbb89 with and without the libc2 objects
- LINKED 528 -> 590; REAL still 43; byte-identical 32.25%
2026-06-15 18:32:48 -06:00
Drew T 0d6380c845 feat(phase-8): link libmcrd byte-identical (T5) — 2 objs / +104 fns
- splat: split 800c into [800c][libmcrd1][800c2][libmcrd2]; LIBMCRD.o (2186 ins,
  the 55 LIBMCRD_OBJ_* + _card_* memcard I/O) and USERFUNC.o (68 ins) are 2
  non-adjacent blocks split by game/libc2/libapi/libcard code (800c2)
- Makefile LIBMCRD_* + gated psyq_integrate libmcrd1,libmcrd2 + -T
- region byte-verified: 2/2 objects, 27 .bss commons all recovered (not scattered)
- src/800c.c trimmed (0 real-C moved); src/{libmcrd1,800c2,libmcrd2}.c committed
- dual byte-gate PASS: 143dbb89 with and without the libmcrd objects
- LINKED 424 -> 528; REAL still 43; byte-identical 29.29%
- NB: these are the libmcrd SDK objects; the game SaveLoadRoutine/Q#5 is Phase 12
2026-06-15 18:23:59 -06:00
Drew T ade73cce62 feat(phase-8): link libgpu EXT+PRIM byte-identical (T4); exclude SYS.o (scattered .bss)
- splat: split 800b2 into [800b2][libgpu][800c]; libgpu block = EXT.o+PRIM.o
  (vram 0x80058890-0x80059234, holds LoadClut/LoadClut2 + primitives)
- SYS.o (3109 ins) EXCLUDED — scattered-.bss commons (cookbook §9.1, the GS_001
  class): SYS references .bss by section+offset but the original linker scattered
  the commons across 0x80078xxx/0x800c5xxx, so no single NOLOAD base reproduces it.
  Stays a byte-identical INCLUDE_ASM stub in 800c; documented in docs/psyq-worklist.md
- curated dir .run/obj40/libgpu_used = {EXT,PRIM} (validates the _used mechanism for
  the later alias libraries); Makefile LIBGPU_* + gated psyq_integrate + -T
- src/800b2.c trimmed; src/libgpu.c + src/800c.c committed (stub records); the trim's
  9 "moved" items were all splat-auto empties (regenerated identically, 0 real lost)
- dual byte-gate PASS: 143dbb89 with libgpu objects AND without (stub fallback)
- LINKED 375 -> 424; REAL still 43; byte-identical 24.32%
2026-06-15 18:21:25 -06:00
Drew T 675872616d feat(phase-8): link libetc byte-identical (T3) — 5 objs / 35 fns
- splat: carve libetc subseg (vram 0x8004239C-0x80043088, VSYNC/INTR/INTR_VB/
  INTR_DMA/VMODE) out of the 800-subseg tail (ends exactly at libcd1); 5 objs,
  1 contiguous block, no placement window needed, no matched-C disturbed
- Makefile: LIBETC_ELF/OBJDIR/SYMS + gated psyq_integrate call + -T on the ld line
  (same conditional/stub-fallback model as libcd/libgs)
- src/800.c trimmed (split_src_region: dropped 35 libetc stubs, 0 real-C moved);
  src/libetc.c committed (35 INCLUDE_ASM stub records)
- pre-wire gate: psyq_link_lib 5/5 objects byte-identical (.text)
- dual byte-gate PASS: make check 143dbb89 WITH libetc objects AND without them
  (stub fallback) — both byte-identical
- progress.py LINKED 340 -> 375 (+35), stubs -> 1626, byte-identical 21.98%
  (metric auto-updated from the new Makefile psyq_integrate call)
2026-06-15 18:08:57 -06:00
Drew T c27a4432eb feat(phase-8): library survey + progress.py LINKED category (T1-T2)
- T1: docs/psyq-worklist.md — byte-confirmed PsyQ footprint map (psyq_identify
  over the original EXE): 9 footprint libs (199 placements) ranked ascending
  difficulty; 3 skip (libmath/libc/libsn = 0 footprint). Findings: the libgs
  "gaps" are libgte objects; heavy game/library interleaving (800=game+libspu+
  libsnd+libetc, 800b~libgte, 800b2=game+libgpu/libc2/libapi/libmcrd/libcard);
  cross-library aliases (C112, UT_RON/S_IH, S_R/S_W)
- T2: tools/progress.py LINKED category — INCLUDE_ASM stubs in integrated library
  subsegs counted as a distinct byte-identical bucket; linked-subseg set sourced
  from the Makefile's psyq_integrate calls (single source of truth -> auto-updates
  per integration, no manifest drift). libcd+libgs: 340 stubs -> LINKED;
  byte-identical/matchable now 425/2093 = 20.31%. Build unchanged (143dbb89)
- phase-ends/CURRENT_PHASE.md: Phase 8 plan + per-task log
2026-06-15 18:00:56 -06:00
Drew T 49db81d8ae feat(phase-7): Gen1 close-out — matching loop industrialized, PsyQ libs linked, LZSS matched (v1.7.0)
- reports (Task 3): tools/{progress,difficulty,dup_report}.py + make report/sig-refresh
  (deterministic digests); harvest to 43 REAL matches incl. LzssDecodeSector
- per-module -O0 split (Task 5.5): src/boot.c + Makefile per-file CC1FLAGS (cookbook §6)
- PsyQ SDK linking (sessions C-G): split/build/identify/link/integrate pipeline
  (tools/psyq_*.py, make_libgs.sh, ld_interleave.py, split_src_region.py) — libcd (18 obj)
  + FULL libgs (31 obj / 6 blocks) linked BYTE-IDENTICAL from real PsyQ 4.0 libs; byte-
  identical with or without the SDK objects (stub fallback); cookbook §7/§8/§9.1-9.5
- LZSS MATCHED byte-for-byte (session F): §5a cross-jump barrier + §10 regalloc/schedule
  idioms, ground-truthed against pinned gcc-2.7.2 source (the Gen1 LZSS gate)
- file-loader cluster: 6 matched + 5 NON_MATCHING-drafted (CdReadStateMachine,
  CdReadSectorReadyCB, StreamLoadStateMachine new this session, valid under -DNON_MATCHING);
  SaveLoadRoutine DEFERRED to Q#5 (save/memcard, documented stub)
- README Gen1-complete; worklog preserved -> phase-ends/logs/Phase7.md
- rules R17 (web-research compiler quirks), R18 (plain-English session summaries),
  R19 (preserve phase worklog out of load order)
- MILESTONE: Gen1 EXIT confirmed — byte-identical build, 43 matches incl LZSS, 7 zero-
  regression sessions, loader cluster matched/drafted, overlay map emulator-proven
- bumps project version 1.6.0 -> 1.7.0
2026-06-15 13:44:32 -06:00
Drew T 8ffb7f0607 feat(phase-7): -O0 split + 4 matches (42 real) + libcd.h — session B checkpoint
- per-file -O0 mechanism: split text c-subseg into boot@-O0 (src/boot.c) + 800@-O2
  via splat resegmentation + Makefile target-specific CC1FLAGS; the boot/main/
  game-mode-dispatch module (0x80010000-0x800123F0) is -O0, not the pinned -O2
  (per-module compiler mixing, SETUP §5.5). Regression-gated byte-identical.
- 4 byte-matches (38->42 real, make check BYTE-IDENTICAL throughout): GameModeDispatch
  (-O0 register-ptr far member), DebugMenuHandler (-O0 reserved-slot local), CdQueueBusy
  (-O2 if/else order + branch polarity), CdReadRequest (-O2 early-return fall-through)
- PsyQ infra: include/psyq/libcd.h (CdlLOC/CdlFILE from the .gdt) + 4 named symbols
  (CdSearchFile/CdPosToInt/CdIntToPos/VSync) in symbols.us.txt; func_* stubs renamed
- loader cluster non-jtbl COMPLETE (6 matched + 2 drafted): NON_MATCHING drafts of
  LoaderInitFileTable + ResourceLoadStateMachine (logically faithful, residuals in-source)
- tooling: progress.py/difficulty.py fixed for the multi-file split (glob src/*.c, all
  asm dirs, skip forward-decls); deterministic; 42 real / 4 NM
- flywheel: matching-cookbook §6 (-O0 detection + idioms), §7 (PsyQ types/symbols), T4
- build byte-identical 143dbb89f34491258bbc27810d0a12ec8b43a8dd from a full clean cycle
2026-06-14 19:37:05 -06:00
Drew T 379185a799 feat(phase-7): matching reports + 24 matches (38 real) — session A checkpoint
Mid-phase checkpoint (Drew-directed; phase NOT closed — R8 exception).

- reports (Task 3): tools/{progress,difficulty,dup_report}.py + `make report`
  (deterministic) + `make sig-refresh`; authoritative baseline 38 real / 2050
  matchable; empties audit 42/42 genuine jr;nop; docs/{progress,difficulty,
  duplicates}.md digests
- matches (Tasks 4-5): +24 byte-identical (22 accessor leaves via the harvest
  pattern + ResourceGetCdLoc + LoaderResetReadState); make check BYTE-IDENTICAL
  throughout (143dbb89...)
- config/symbols.us.txt: declare func_80047CAC (R15) — fixes a latent
  non-reproducibility (spimdisasm 1.41.0 auto-detect of an 8B inter-fn blob was
  unstable across clean extracts; the committed Phase-6 baseline did not
  deterministically rebuild)
- Task 1 rodata-island foundation: investigated (migration co-locates jump
  tables + resolves refs, but byte-identity blocked by splat global-migration +
  contiguous linker placement vs BFM's monolithic segment) -> DEFERRED to Task 2'
  with a candidate fix; full findings in phase-ends/CURRENT_PHASE.md
- phase-ends/CURRENT_PHASE.md: reordered plan, loader-cluster triage, per-session
  green-check log (1 of >=3 needed for the Gen1-exit milestone)
2026-06-14 17:36:12 -06:00