mirror of
https://github.com/Druthulu/BFM-decomp
synced 2026-09-27 22:45:39 -04:00
d564b4b4e718febf18eb97ef985e7224fbc078d4
3756 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Drew T
|
d564b4b4e7 |
feat(gate_main): persist every proven verdict the moment it exists
try_batch is stateless and the bisect loop held `good` only in memory, writing .run/gate_main_banked.json once at the very end. A 34-minute bisection killed by a timeout, a Ctrl-C or a supervisor therefore lost every match it had already PROVEN — and each of those proofs cost a full clean EXE rebuild. The S75 checkpoint named this the single highest-value gate improvement available. Adds an atomic .run/gate_main_progress.json written after every verdict, and a resume that reuses it. Three guards, each a way it could silently lie: the journal must belong to this slate; entries are re-keyed against `kept` so a draft dropped by resolve_conflicts cannot sneak back; and the draft's content hash must still match (R56 — a verdict measures those bytes). Resumed sets are re-verified as one batch anyway, so a wrong reuse costs one rebuild and can never bank anything unproven. --no-resume opts out. Negative-controlled on six cases incl. a changed draft, a foreign slate and a half-written journal. |
||
|
Drew T
|
20fea212df |
fix(config): drop the stale SaveLoadRoutine symbol + its two wall entries
SaveLoadRoutine is `case 0:` inside func_8002B0B4, not a function of its own (S75). The lingering `= 0x8002B154; // func` declaration kept splat emitting asm/nonmatchings/800_b/SaveLoadRoutine.s, which progress.py reported as the single UNPLACED parse hole. The two config/wave_exclude.txt WALL entries described the same misconception. UNPLACED 1 -> 0. Build stays byte-identical at 143dbb89. |
||
|
Drew T
|
163c91e420 |
docs(phase-31): correct the S75 checkpoint's figures and record the one-line UNPLACED fix
Verified at close rather than asserted: 25 commits, src/config/tools clean, main green at 143dbb89..., and HEAD genuinely carries the func_8002B0B4 C (0 INCLUDE_ASM for SaveLoadRoutine, 1 real definition). A 47-minute bisection left several mid-run readings that looked like regressions and were not, so the figures are now stated from a settled tree. Adds START HERE item 0: the UNPLACED parse hole is one line -- config/symbols.us.txt:27 still declares SaveLoadRoutine = 0x8002B154 // func, so splat keeps emitting a .s for a symbol that is now case 0: inside func_8002B0B4. Delete, re-extract, rebuild. config/wave_exclude.txt lines 14-15 are stale for the same reason. Left undone only because a gate held main's tree at close. T10 checklist now carries the S75 line. |
||
|
Drew T
|
af4912e624 |
docs(phase-31): S75 FINAL checkpoint — 32 banked, nine instrument defects, the verbatim class mapped
Written for a fresh session. Headline: every codegen wall examined this session was an instrument defect, and the two largest results came from deleting a belief rather than writing better C -- SaveLoadRoutine's §434 wall was a splat symbol boundary (it is case 0 of func_8002B0B4, one function on one frame), and the '§332/§188 wall' was the reorder island, which banked 20 functions whose drafts had been on disk since waves m04-m16. Records what I got wrong so it is not inherited: five regex censuses (116/112/108/178/199), contiguity mistaken for fragmentation, a build-config gap called compiler-inexpressible, and two bursts drawn at fragments because the triage came after the draw instead of before it. R22 clean-fleet NOT run; the checkpoint says so at the top. |
||
|
Drew T
|
261b8a4fd3 |
feat(decomp): bank 20 SDK-C-REORDER functions — the "§332/§188 wall" was the reorder island
BANKED 20 of 21 after bisection in 11 rebuild(s)
143dbb89f34491258bbc27810d0a12ec8b43a8dd BYTE-IDENTICAL
rejected: ['func_8005E13C']
src/800c3.c INCLUDE_ASM stubs 36 -> 16. These are libapi/libcard C functions
that had been banked as §265 verbatim __asm__ blocks and were unreachable by
every gate (a verbatim body has no INCLUDE_ASM to substitute).
The chain that unblocked them, all this session:
* the triage identified the class and showed the "§332/§188 wall" is the §332b
-O2 reorder island, which landed 2026-09-01 -- one day BEFORE four of these
were banked as assembly, with "6 of 53 at closeness 0" drafts in hand;
* REORDER_TUS was extended to 800c2_2/800c2_3 ($(filter) is an exact stem
match, so 800c2 never covered them), proven byte-neutral by --assert-baseline;
* verbatim_to_stub converted 20 bodies back to stubs, byte-neutral;
* gate_main was fixed twice -- it destroyed uncommitted work, and my own first
guard sat inside the bisection loop where it tripped on the gate's own
substitution.
Drafts were already on disk from waves m04-m16 and s67m1; not one needed
redrafting. This is the §451 lesson paying out: the evidence was recorded, and
what was missing was a tool able to reach it.
|
||
|
Drew T
|
fc7caf599b |
refactor(tools): retire asm_in_c.py — the taxonomy is DATA now, not a regex census
R33, "the best outcome is a DELETED SCANNER, not a fixed regex". asm_in_c.py
existed to DISCOVER the §265 verbatim class by parsing __asm__ blocks. That job
is done, and regex was the wrong instrument: five successive censuses returned
116 -> 112 -> 108 -> 178 -> 199, and the classification was worse than the count
-- it called 154 rows "game code" where the authoritative answer is 24.
The real answers came from evidence a regex cannot see:
* the <OBJ>_OBJ_<hex> naming key -- every one is placed_object.text_start +
hex, so those symbols are OFFSETS INTO LIBRARY OBJECTS, not functions;
* the PsyQ archive symbol tables in .run/obj40/, which keep statics as W
symbols, so for a byte-identical object the archive IS the function map
(checkRECT = SYS.o+0x52C = func_80059760, and NONE of the 44 SYS_OBJ_*
symbols in SYS.o is a function).
So:
config/verbatim_manifest.json (NEW, committed) -- the authoritative census.
200 rows, derived once from the ROM image + archives + naming key, each with a
class and a DISPOSITION:
PERMANENT-VERBATIM 69 rows / 57 units hand asm; never decompilable
DECOMPILE-AS-PARENT 57 rows / 23 units a FRAGMENT; decompile unit_entry,
never the fragment itself
DECOMPILE-NOW 41 rows / 41 units
DECOMPILE-LOW-VALUE 20 rows / 4 units
UNCERTAIN 5 / NOT-VERBATIM 7 / NOT-CODE 1
tools/verbatim_check.py (NEW) -- a GUARD, not a census. Detects verbatim bodies
(the cheap part, and the only part regex is good at), diffs the NAMES against the
manifest, and reports NEW / GONE / MOVED. A NEW row means someone banked assembly
and it is about to become invisible work; it is never allowed to inherit a
disposition by default. It deliberately does not classify or count units.
Compares case-insensitively on the hex, because an address is a NUMBER (R48).
tools/verbatim_target_s.py -- put on the MANIFEST LEASH. It used to enumerate
every verbatim SYMBOL, and 62 of those are not functions (fragments, bare
epilogue tails, padding, trampolines). Emitting per-symbol targets for them is
what sent two drafting bursts at things no C function can express. It now takes
only DRAFTABLE dispositions: 66 targets emitted, 134 skipped and SAID SO.
tools/verbatim_to_stub.py -- repointed to verbatim_check for detection, so there
is ONE detector in the tree rather than three copies.
tools/asm_in_c.py -- REMOVED.
|
||
|
Drew T
|
254feb8ee4 |
fix(gate_main): the uncommitted-work guard belonged OUTSIDE the bisection loop
I added the guard to try_batch() an hour ago. try_batch runs REPEATEDLY during
bisection, and its own first substitution makes main's TUs dirty -- so on
iteration two the guard could not tell the operator's unsaved work from the
gate's own in-flight edit, and aborted the run:
M src/800c3.c
gate_main: aborting with an UNVERIFIED substitution in main's TUs — reverting
It failed safely (reverted, no bank lost, and said so), but it made the gate
unusable for any batch larger than one.
Hoisted to assert_main_tus_clean(), called ONCE from main() before any
substitution. The lesson is worth the line it costs: A GUARD MUST BE ABLE TO
DISTINGUISH THE STATE IT PROTECTS FROM THE STATE IT CREATES. Placed inside the
loop it was checking its own footprints.
Negative-controlled both directions: a genuinely dirty src/800c3.c is refused by
name before anything is substituted, and a clean tree now proceeds into the
bisection (currently running 21 drafts).
|
||
|
Drew T
|
f0eea33381 |
refactor(main): convert 20 SDK-C-REORDER verbatim bodies to INCLUDE_ASM stubs — byte-neutral
These are libapi/libcard C functions in src/800c3.c that were banked as §265 verbatim __asm__ blocks. The triage (.run/S75/triage/report.md) established they carry the §188 shape (`jr $ra` with `addiu $sp,$sp,+N` in the slot) and that the "§332/§188 wall" is the §332b -O2 reorder island, which landed 2026-09-01 -- one day BEFORE four of them were banked as assembly, with the commit itself recording "6 of 53 at closeness 0" stored drafts. Converting them to stubs makes them reachable by every gate again (a verbatim body has no INCLUDE_ASM to substitute, so gate_main drops it as "resolved to NO stub") and is the honest accounting: a stub counts as OUTSTANDING WORK, a verbatim body counted as banked. BYTE-NEUTRAL, PROVEN: make extract + make build BINARY=main -> 143dbb89f34491258bbc27810d0a12ec8b43a8dd BYTE-IDENTICAL. INCLUDE_ASM pastes the same assembly the block transcribed, so it must be -- but "must" is a claim and this was gated, not assumed. Metric moves honestly: VERBATIM __asm__ bodies 173 -> 148, INCLUDE_ASM stubs 34 -> 53. 20 of 24 converted. The 4 refusals are reported, not silent: func_800623A4 / func_80062434 / func_8006252C (800c2_2) and func_8005D8A0 -- the last being the row all three of asm_in_c's detectors missed, which is its own finding. Also fixes verbatim_to_stub to CASE-NORMALISE the address. splat's convention is func_%08X but analysis artifacts carry lowercase (the triage taxonomy does), and asking for func_8005ed4c found 0 of 24 blocks that were all sitting right there. An address is a NUMBER; matching it as a case-sensitive string is R48 in its case-sensitivity form. |
||
|
Drew T
|
5172df5f0b |
fix(build): REORDER_TUS missed 800c2_2/800c2_3 — $(filter) is an exact stem match
`$(filter $*,$(REORDER_TUS))` matches the TU stem EXACTLY, so `800c2` never
covered `800c2_2` or `800c2_3`. Those two TUs went through maspsx while their
siblings went through reorder_passthrough | as -O2 (the §332b island, landed
2026-09-01).
That gap is why func_80062388's `lui at / jr ra / sw a0,lo(at)` was written up
as COMPILER-INEXPRESSIBLE in cookbook §452: a probe (`void f(int v){D=v;}` ->
cc1 -> reorder_passthrough | as -O2) emits exactly that sequence. It was a
build-config gap, not a gcc-2.7.2 define_delay limit. §452 corrected.
Byte-neutrality PROVEN the right way -- gate_main --assert-baseline builds the
committed tree with NO draft substituted:
BASELINE GREEN — 143dbb89f34491258bbc27810d0a12ec8b43a8dd BYTE-IDENTICAL
This unblocks the 24 SDK-C-REORDER units, four of which were banked as verbatim
assembly on 2026-09-02 off a wall list that predated the fix by one day, with
closeness-0 drafts already in hand.
|
||
|
Drew T
|
590fb37447 |
fix(gate_main): refuse to destroy uncommitted main work; name a tool refusal instead of hiding it
TWO DEFECTS, both found by the SaveLoadRoutine decompile and both of which made this gate unable to bank a whole class of function. 1. try_batch() opens with `git checkout -- <main TUs>`. Correct for the normal flow (restore stubs, re-extract, substitute drafts) and CATASTROPHIC for anything uncommitted. Measured twice today: the SaveLoadRoutine decompile (1,179 ins, byte-identical) sat uncommitted while a gate ran and survived only because it was committed first; and a §265 verbatim body converted to a stub is UNCOMMITTED BY CONSTRUCTION, so this line restored the __asm__ block NEXT TO the substituted C -- 9 jump tables instead of 5, jtbl_rodata_pads refused, and the gate REJECTED a byte-identical bank. gate_main could not bank anything in the verbatim class, by construction. Now refuses when main's TUs are dirty, printing the offending paths and telling the operator to commit (R42) or stash. --allow-dirty / GATE_MAIN_ALLOW_DIRTY=1 is the deliberate override. A destructive step that cannot be undone must ASK, not assume. 2. The failure analysis looks for error/undefined/conflict/..., and jtbl_rodata_pads aborts via sys.exit with a message containing none of them -- so a carve REFUSAL surfaced as "only warnings" and the real cause of a rejected bank was invisible. Refusals from jtbl_rodata_pads / jtbl_carve / corpus / jr_isolate_all are now named explicitly as the cause. Negative-controlled both directions: the guard fires on a dirty src/800_b.c naming the file, and --assert-baseline on a clean tree still reports BASELINE GREEN 143dbb89... BYTE-IDENTICAL. |
||
|
Drew T
|
a68197b32d |
feat(decomp): SaveLoadRoutine DECOMPILED to real C — 1,179 ins, and the §434 wall was a symbol boundary
The largest open function in the project, carried as the §434 WALL since Phase
31 opened, is now real C. main SHA1 143dbb89... BYTE-IDENTICAL.
THE WALL WAS NOT A PROPERTY OF THE CODE. A whole-binary census of every .s for
the interior labels and raw addresses 0x8002B154..0x8002C31C found EXACTLY TWO
sources, and both are func_8002B0B4 itself: its own beq/j to .L8002C2A8 /
.L8002C2AC / .L8002BFE4, and its own jtbl_80072E44 (36 entries, entry 0 =
0x8002B154). Nothing else in the binary references the range.
So func_8002B0B4 (40 ins, prologue + dispatch) and SaveLoadRoutine (1,139 ins,
epilogue) are ONE function sharing one 0x40 frame -- entry func_8002B0B4, five
overlay callers, all s32 f(s32, s32, void *). SaveLoadRoutine is `case 0:` of
its state switch; .L8002C2A8/.L8002C2AC are the switch exit and .L8002BFE4 the
outer `case 1:` body. The "no epilogue of its own / must stay file-scope
__asm__" note that parked this for a phase was describing a SPLAT SYMBOL
BOUNDARY, not a code structure. The predicted "middle path" (decompile one while
siblings stay asm) was moot: there are no siblings.
The three "save/load handler code pointers at saveHeaderTemplate+0x54" in
docs/memory-map.md are jtbl_80072E44[0..2] -- confirmed against
dumps/ram_savescreen.bin (0x80072E44/48/4C = 0x8002B154/1AC/BEA4). The S73
correction to that row is right; no further RAM capture was needed.
Verified three ways: match_one MATCH (1179 ins) on a merged target .s; `make
extract && make build BINARY=main` -> 143dbb89...; and gate_main's own
clean_build() sequence driven from Python -> "sha1 143dbb89... == check.us.sha
(BYTE-IDENTICAL)". Independently re-checked here: tools/asm_in_c.py reports
NEITHER symbol as assembly-posing-as-C, so this is genuine C (the 94 __asm__
occurrences in the TU are §3a zero-byte cross-jump barriers, which are C).
TWO NEW TOOL DEFECTS, logged not fixed (main is busy; next session):
* gate_main.py:710 runs `git checkout -- <main TUs>` immediately BEFORE
substitute(). For a function whose current form is a hand-written __asm__
block that restores the block NEXT TO the C, the TU then carries 9 jump
tables instead of 5, and gate_main REJECTS a byte-identical bank. It cannot,
by construction, bank anything in the verbatim class.
* gate_main's error filter (error|undefined|conflict|...) does not match
jtbl_rodata_pads' sys.exit refusal, so that failure shows only warnings.
HAZARD, and why this is committed immediately (R42): any gate_main run on main
wipes this bank via that same line 710.
Ten measured levers for the body are in .run/S75/slr_c/cookbook_448.md pending a
cookbook merge, headed by: when a frame check says "no prologue / no epilogue",
build the MERGED .s and decompile the pair as one function before excluding
anything.
|
||
|
Drew T
|
8009f83b40 |
fix(tools): verbatim_target_s wrote targets into asm/, which the Makefile globs as build objects
`build/asm/%.o: asm/%.s` globs the ENTIRE asm/ tree, so the 146 regenerated
target .s files I emitted to asm/verbatim/ were picked up as BUILD OBJECTS and
main went red:
make: *** [Makefile:696: build/asm/verbatim/main/func_80052430.o] Error 1
Default --out moved to .run/verbatim_targets/, which is outside every build
glob. main verified green again: 143dbb89... BYTE-IDENTICAL.
The lesson belongs with the others from this session: a generator's OUTPUT
LOCATION is part of its contract, and asm/ is not a scratch directory. The
failure was invisible until a full build ran -- the tool itself succeeded, the
targets were correct, and nothing about them was wrong except where they sat.
|
||
|
Drew T
|
1e86617ff8 |
docs(cookbook): §450-§452 — verbatim pipeline, evidence sources, and a correction to §448
§450 — regenerating a target .s for a function that is no longer a stub. The source must be the ROM IMAGE, never the __asm__ block: the block is the thing under test, and a target derived from it agrees with the candidate by construction. Two silent defects caught by ONE known-true cross-check: splat writes BYTE-order hex where objdump prints the VALUE (reversing double-swaps -- 91/1139 words agreed, and the LENGTH was perfect so only a word-level compare could catch it), and objdump ELIDES runs of zero bytes so every MIPS nop vanished (-z is load-bearing; there the length assertion did catch it). Plus verbatim_to_stub: to gate this class, put the function back into the form every tool already understands rather than writing a parallel gate. §451 — your evidence has more than one source, and the one you query is probably the worse one. BEST not LAST from the append-only backlog (a last row is evidence about that lane's seed, not about the function); journal_notes as a second, DISAGREEING oracle (37 functions reclassified, G-DRAFTED-UNKNOWN 47->10, and func_8017DB98's 122 ins banked from a one-word declaration fix the journal had recorded all along); and a regex that consumes an unbounded body cannot enumerate the items after the first -- a 400-char window swallowed the next attempt's header and hid BOTH of that function's MATCH records. §452 — CORRECTION to §448's headline. A burst against the ten smallest verbatim bodies returned 0 banks and refuted the "154 functions of real decompilation work" framing. Four classes are legitimately verbatim: fragments of a SPLIT function sharing one stack frame (SYS_OBJ_604/640/func_80059760 are the compiled output of ONE original C function; a bare epilogue tail cannot be decompiled alone), hand-written GTE assembly from 1998, compiler-inexpressible forms (a symbolic store in a jr-ra delay slot, which gcc-2.7.2's define_delay cannot emit), and no-return tails. 154 is an UPPER BOUND, not a work queue, and the four tells are cheap to check. Also banked: one agent submitted the verbatim __asm__ block itself as its "decompile", and match_one truthfully printed MATCH -- a raw asm blob byte-matches its own source by construction. The adversarial verifier refuted it. Any burst over this class MUST carry that check: the trivially-passing draft is not hypothetical here, it is the default thing to produce, and a byte gate cannot tell the difference. |
||
|
Drew T
|
ce79a3e432 |
docs(cookbook): §449 — four compiler dials from the S75 redraft wave, two byte-gate confirmed
Provenance stated per row (CONFIRMED = banked through the whole-binary gate; CLAIMED = the agent's own measurement on a function that did not bank), because one of these came from a function that was adversarially upheld and then FAILED the real gate. A. reg_n_sets is a one-line scheduling dial (CONFIRMED, func_80180ABC 257 ins). sched1 schedules backward; a pseudo set exactly once gets the birthing_insn_p launch boost (priority = 7f000001 in cc1 -dS), which drags its load LATE. Splitting the RMW as 't = t + 1; *p = t;' makes reg_n_sets 2, suppresses the boost, and floats the load to the block head -- the block-local dual of §350's shared temp, WITHOUT the global-allocno penalty that costs the in-place addiu. Companions: 180 legal statement permutations all scored identically while one cc1 -dS dump named the cause (diagnose, don't permute); a pin-free fix for paired-register inversion; and gcc frame slot order is NOT declaration order (BLKmode aggregates go in order at expand_decl, an addressable scalar is forced to the stack later -- declare 's32 x[2]' to place a slot between two aggregates). B. A single-set local's VALUE is visible at a switch join and erases a zero-extension (CONFIRMED, func_801806F8 241 ins). combine.c:10035 lets get_last_value bypass the label_tick guard when reg_n_sets == 1, so all seven narrowing spellings emit nothing. Diagnostic: a visible extension in the target means the variable has MORE THAN ONE SET in the original source. Verified against a matched sibling: andi is the multi-set zero-extend and sll;srl is NEVER reachable from a single expression. C. CORRECTION to §439 -- the sll 16; srl 16 pair lands AFTER the jal, not before it (sched1 sinks the ashift past the call), and it works even for a KNOWN CONSTANT, because the call-split defeats folding structurally rather than by hiding the value. D. An offline jtbl-rodata placement audit (CLAIMED, func_800CB00C -- did not bank, which is the point: both matchers compare .text only, so a jtbl function's MATCH says nothing about its table). |
||
|
Drew T
|
72fa482027 |
feat(tools): verbatim_to_stub.py — put the 147 asm-posing-as-C functions back where the gates can reach them
THE GAP. Every gate in this project substitutes a draft in place of an
INCLUDE_ASM line. A §265 verbatim __asm__ body has none, so:
gate_main.substitute() resolves each entry through the STUB map; a verbatim
function is reported "resolved to NO stub" and dropped
gate_stage/harvest_verify same splice, same gap
splat stops emitting <fn>.s once a function is not a stub
So all 147 were undecompilable AND ungateable -- not for want of information,
but because the information was in a form nothing consumes.
The fix is not a parallel gate (R33 -- one implementation). It is to put the
function back into the form every existing tool already understands: replace the
__asm__ block with INCLUDE_ASM. That is also the HONEST representation --
INCLUDE_ASM pastes the very same assembly the block transcribes, so the bytes
are identical either way, but a stub counts as OUTSTANDING WORK in progress.py
while a verbatim body counted as banked. The conversion moves a function from
"silently done" to "visibly to do".
Two refusals rather than guesses (R43), because both failure modes are silent
and destructive:
* the block is located by brace/paren MATCHING via asm_in_c.asm_blocks, never
regex-sliced -- these blocks are full of braces and parens inside string
literals and an approximate cut corrupts a file that currently builds;
* the asm subdir for the new INCLUDE_ASM is copied from a sibling stub IN THE
SAME FILE. Subsegs are per-file, so a neighbouring file's spelling names a
different subseg -- a stub with the wrong subdir compiles happily and
includes ANOTHER FUNCTION'S ASSEMBLY. With no sibling to copy, it refuses.
--gate rebuilds and asserts the SHA is unchanged, restoring the file if not:
byte-neutrality here is a claim, and this tool exists to enable a byte gate, so
it declines to be the one link that goes unchecked.
Dry-run verified on main:func_80047E58 -> src/800b.c, 9-line block, subdir
correctly derived as asm/nonmatchings/800b. The --gate proof is deferred only
because another agent is mid-build on main right now.
|
||
|
Drew T
|
70de5a8611 |
feat(tools): verbatim_target_s.py — the 147 asm-posing-as-C functions are workable again; bank func_8017DB98
THE BLOCKER. asm_in_c.py found 147 GAME functions that are §265 verbatim
__asm__ bodies. NONE of them could be worked on: splat emits
asm/nonmatchings/<subseg>/<fn>.s only for functions that are still INCLUDE_ASM
stubs, and a verbatim body is not a stub -- so splat stops emitting its .s,
while match_one and rtu_match BOTH consume one. Measured: 1 of 147 had a target
on disk. The class was unworkable because the information was in the wrong FORM,
not because it was missing.
verbatim_target_s.py regenerates a splat-format target .s from the EXTRACTED ROM
IMAGE -- never from the __asm__ block in our own source, because the block is
the thing under test and a target derived from it would agree with the candidate
by construction (R34). 146 of 147 emitted; the 1 refusal is REPORTED.
TWO DEFECTS CAUGHT BY CHECKING AGAINST A KNOWN-TRUE CASE, both of which would
have shipped ~147 silently-wrong targets:
* BYTE ORDER. splat writes the four bytes as they sit in the image
(`C8FFBD27` for instruction 0x27BDFFC8) and masked_diff.insns_from_s reads
the column with struct.unpack("<I", bytes.fromhex(...)). objdump prints the
VALUE, so reversing double-swaps: 91 of 1139 words agreed with splat's own
.s for the same function. The LENGTH matched perfectly, so nothing except a
word-level cross-check could have caught it.
* `-z` / --disassemble-zeroes. objdump ELIDES runs of zero bytes as `...`, and
a MIPS nop IS 0x00000000 -- so every nop vanished. func_80049610 (three
nops) disassembled to ZERO instructions; func_80047D3C 31 of 36. The length
assertion caught all of them, which is the only reason this was not shipped
as ~30 quietly-truncated targets.
Verification: regenerated SaveLoadRoutine target is 1139/1139 words IDENTICAL to
the .s splat itself emitted for the same function.
ALSO BANKED: ov_SC06_025:func_8017DB98 (122 ins). Its body was byte-exact on
disk since S71 and the blocker was one word: the TU declared
`extern void func_8017DB98(s32, s32)` where the epilogue is `addu $v0,$s3,$zero`
-- must be `extern s32`, and the caller discards the result so the change is
byte-neutral. That line number and fix were recorded in the agent journals the
whole time; frontier_classify only surfaced it once journal_notes was wired in
as a second oracle earlier this session.
|
||
|
Drew T
|
f5f4c2eeec |
feat(decomp): bank func_801806F8 + func_80180ABC (498 ins) + frontier_classify reads the journals
Two banks from the S75 redraft workflow (7 overlay functions, one agent each,
every claimed MATCH re-verified by an independent agent instructed to refute
it). Both were carried as F-FAR "a draft exists but is materially wrong":
func_801806F8 ov_SC03_105 241 ins (recorded closeness 235)
func_80180ABC ov_SC03_105 257 ins (recorded closeness 250)
Neither needed a better model. Both needed the recorded closeness not to be
believed -- see below.
frontier_classify.py, THREE fixes, each caught by testing against a case whose
answer was already known:
1. BEST closeness, not LAST. .run/backlog.jsonl is append-only, one row per
attempt across every lane and session, so the last row is evidence about
THAT LANE'S SEED, not about the function. Caught func_80180B3C (best 125,
last 287) and moved func_80181294 from "redraft" to "permuter" (best 19).
The draft that ACHIEVED the best score is kept, not the last one written.
2. journal_notes.py wired in as a SECOND, DISAGREEING oracle (R34). The backlog
does not have what the agent journals have. Measured on func_8017DB98:
backlog best == last == 115, so best-vs-last could not help, while the
journal holds "Attempt 2 (MATCH · closeness 0) ... MATCH 122/122 ... BANK
BLOCKER is TU plumbing, not the body (§376/§378)" WITH the draft path and the
exact declaration to change. Reclassified 37 functions; G-DRAFTED-UNKNOWN
fell 47 -> 10 and a new C-PLUMBING class holds 16 functions / 1,547 ins whose
BODIES ARE PROVEN and are blocked only by the TU.
3. A consuming-regex bug in my own extractor -- the session's signature defect,
committed a third time in the tool written to find it. The first cut used
`re.finditer(r'\*\*Attempt \d+\*\* \(([^)]*)\)(.{0,400})', ..., re.S)`, whose
400-char body window SWALLOWS THE NEXT ATTEMPT'S HEADER, so every record
following another was invisible. On func_8017DB98 it hid attempts 2 AND 6,
both `MATCH · closeness 0`, and returned attempt 1's NEAR (2) as the best --
exactly the records the oracle exists to find. Now splits on the marker
rather than consuming past it. A regex that consumes an unbounded body cannot
enumerate the items after the first.
Rows now carry attempts, closeness_last, journal_closeness, and a
!!WARMSTART-REGRESSION flag when a later attempt scored materially worse than
the best -- the shape a wave's warm-start regression makes, which from inside
the wave is indistinguishable from an unsolved function.
Gate ledger for the batch of 7: 2 banked, 3 near, 2 failed. func_800CB00C failed
despite being adversarially upheld -- it owns a jump table, and both matchers
compare .text only, so a verified .text MATCH proves nothing about table
placement (the agent's own write-up says so).
|
||
|
Drew T
|
8e3ce81a69 |
docs(phase-31): S75 addendum — the frontier is 154 game functions bigger than reported
tools/asm_in_c.py finds 199 functions that are assembly posing as C (154 game code, 171 in main). They were in no progress.py bucket, so main's REAL% was overstated: 45.88% -> 42.15% once counted. Nothing regressed; the denominator was missing 173 functions of real remaining work. Also records the counting cautionary tale (five hand counts, 116->112->108->178 ->199, each wrong the same way) and the tool design that answers it. Cookbook §448. |
||
|
Drew T
|
c05ea15cbe |
feat(tools): asm_in_c.py — 154 game functions are assembly wearing a .c extension
A .c file in src/ looks decompiled. 199 functions are not: they are the target
assembly pasted into a C string literal (§265), byte-identical BY CONSTRUCTION
and completely unexplained. 45 are PsyQ/CRT routines where that is defensible;
154 are GAME CODE, 171 of the 199 in main, the largest being SaveLoadRoutine at
1,165 instructions.
They were invisible because progress.py's classify() matched INCLUDE_ASM,
INCLUDE_RODATA and C definitions, and a file-scope __asm__ block is none of
those -- so each landed in NO bucket, either swallowed by a surrounding
construct or surfacing as the single `UNPLACED (parse hole)` line the tool has
been printing all along.
progress.py gains a VERBATIM __asm__ bodies line: counted byte-identical (it is,
by construction) but NEVER as REAL. main's headline moves 45.88% -> 42.15%.
Nothing regressed and no work was lost -- the denominator was missing 173
functions that are real remaining work.
THE COUNTING LESSON IS THE REUSABLE PART. Counting these by hand went
116 -> 112 -> 108 -> 178 -> 199 across five attempts in one session, every
intermediate number reported confidently. All five errors were one shape, a
pattern narrower than the claim it supported:
* the sources use BOTH ".ent\tNAME\n" and ".ent NAME\n" -- anchoring on either
silently drops every instance of the other;
* a bare ".ent\t" fragment yields a phantom function literally named `t`, six
times, which is the only reason the error was noticed;
* __asm__ appears in 3,182 of 4,224 sources, almost all the §3a barrier, so
counting files or counting __asm__ measures nothing;
* `.globl NAME` + `NAME:` proves EXPORT, not CODE -- the first real run
reported jtbl_80072ED4/EEC/F0C/F24 as four "functions";
* a hand-written SDK name list reported 170 game functions because it did not
know VectorNormalSS / SquareRoot12 / OuterProduct12 are libgte.
So the tool does not trust one regex: THREE independent detectors that must
agree with disagreement reported as a defect (R34 -- that is what caught the
jump tables); SDK-ness DERIVED from the 14 shipped PsyQ archives via nm (2,227
symbols) rather than a list (R33); coverage asserted so a definition-shaped
block no detector claims fails loudly (R32/R43); and --selftest carrying a
known-true case of every spelling plus the phantom `t` and the jtbl regression.
Cookbook §448, SETUP row. Law: when a count comes from a text pattern, the
pattern has a denominator too -- validate it against one known-true case of
every FORM the corpus contains before quoting the number.
|
||
|
Drew T
|
1bac13b664 |
fix(jtbl): the pad walk cannot see a verbatim-asm rodata block — SaveLoadRoutine banks (bytes, not a decompile)
tools/jtbl_rodata_pads.py --derive walks a TU's rodata emission against the
retail island and validates only what it can SEE. A §265 verbatim-__asm__ body
emits its tables as `.section<TAB>.rodata` + `jtbl_xxxxxxxx:`, and the walk
missed BOTH spellings:
* the rodata directive was matched as the literal one-space string
".section .rodata" / ".rdata", so a tab-spelled directive never entered
rodata at all;
* inside rodata the anchor regex accepted only `D_xxxxxxxx` (the S74 dlabel
fix was one prefix short), so a `jtbl_xxxxxxxx:` label was invisible.
Consequence, traced: the walk skipped the block as if it were .text, every
later C table walked 104 bytes behind its retail address, EVERY WORD in that
range happens to be a valid code address so the entry guard never fired, and
the walk stopped short of the island's single zero word -- so the one trailing
pad was never emitted and the image linked 4 BYTES SHORT. That produced 3,989
differing bytes on a body the verdict layer had already called byte-identical.
Fixed: tokenised directive match (.rdata / .section .rodata, tabs and commas);
anchors keyed on the ADDRESS IN THE NAME for `D_` or `jtbl_`, with an
address-suffixed label of any other prefix now REFUSING loudly (R43) instead of
becoming a silent hole; and `.align N` modelled SECTION-RELATIVE from the walk
origin, as `as` does -- needed for `.align 3` when a section starts = 4 mod 8,
which span B (0x80072E44) does.
Negative control: old vs new derive over ALL 162 md_*/main derive-path TUs ->
160 byte-identical post-derive streams with identical exit codes, 0 DIFF; 2 SKIP
(800c2/800c3 are REORDER TUs with no derive stage).
main SHA1 143dbb89... BYTE-IDENTICAL, 413,696 bytes, cmp identical to retail.
WHAT THIS IS NOT. SaveLoadRoutine is banked as a §265 verbatim __asm__ block --
BYTES, NOT A DECOMPILE. Its 1,165 instructions are byte-correct and unexplained.
tools/progress.py correctly REFUSES to count it, reporting `UNPLACED (parse
hole)` rather than inflating REAL (which moved 880 -> 881 on func_8005DCA0
alone). Two such blocks already exist in this TU, documented as necessary
because those functions have no epilogue and fall into shared tails. A real C
decompile is now being attempted separately; this commit is the revertible
byte-green base for it.
|
||
|
Drew T
|
6d56972a4b |
docs(R31): S75 decision log — seven walls were seven instruments; leave the 12,000
Two entries. (1) The session's through-line: seven 'codegen walls' examined,
seven instrument defects, none the compiler -- and the reusable law that a
verdict class which CANNOT FIRE is worse than one that does not exist, because
it turns 'I don't know' into confident wrong advice. SaveLoadRoutine's 1,165
instructions are the price: a byte-identical body behind a verdict naming the
wrong subsystem, for a whole phase.
(2) Drew's decision not to convert the ~2,073 functions / ~12,116 duplicate
copies of dedup-hygiene backlog, on the sotn precedent our cookbook records
('sotn writes duplicate funcs explicitly'), with the caveat that the claim rests
on one parenthetical of ours rather than sotn's repo. Gate --no-propagate from
here; the backlog is orthogonal to completion %.
|
||
|
Drew T
|
ac55da69d1 |
docs(phase-31): S75 checkpoint — 7 banked, seven instrument defects, SaveLoadRoutine is a carve
Full 🛑 block for a fresh session: state, the seven defects (§442-§447), the
measured frontier map, Drew's dedup decision, and the harness lessons.
The headline for whoever picks this up: SaveLoadRoutine (1,165 ins, the §434
wall, 9.2% of all remaining work) has a BYTE-IDENTICAL body and is blocked by a
jump-table carve -- which the verdict tool could not say because that verdict
class was unreachable by construction on main.
R22 clean-fleet is NOT yet run for S75 and the checkpoint says so.
|
||
|
Drew T
|
cf7f837231 |
fix(gate): main's TABLE REJECT verdict was unreachable — SaveLoadRoutine is a CARVE, not plumbing
SaveLoadRoutine (1,165 ins) is the largest open function in the project, 9.2%
of all remaining work, and has been carried as the §434 WALL. Gated alone
through gate_main, with the §376/§378 chain already applied, the verdict layer
says: "SaveLoadRoutine is BYTE-IDENTICAL; all 3989 differing bytes are
ELSEWHERE". The body has been correct the whole time.
What rejects it is where its FOUR jump tables (jtbl_80072ED4/EEC/F0C/F24) land:
.data/.rodata (jump tables) 3,787 bytes 94.9%
.text (perturbed code) 202 bytes 5.1%
and the built image is 4 bytes SHORTER than retail (413,692 vs 413,696) --
§446's first diagnostic, firing on a function §446 was not written about.
main_diff_locate.classify() already HAD a TABLE REJECT class, added in S72 under
a docstring reading "THE THIRD CLASS EXISTS BECAUSE THE FIRST TWO MISLABELLED
IT". It could not fire here for two independent reasons:
* it keyed on the literal string `(.rodata)`, but main's section_order is
[.rodata, .text, .data, .bss] -- its rodata sits BELOW .text and its jump
tables live in `.data` objects, so TABLE REJECT was UNREACHABLE BY
CONSTRUCTION on the binary with the most jump-table functions left. A
section NAME is not a section ROLE.
* it demanded purity (ro == outside), so 5% perturbed code defeated an
all-or-nothing test and dropped the verdict through to PLUMBING REJECT --
whose advice (fix_arity_callers -> cast_self_callers) addresses the 5% and
cannot touch the 95% that is data. That chain was run on this function
TWICE today and fixed nothing, exactly as the evidence predicts.
Now: table bytes counted in (.data) OR (.rodata), and the test is DOMINANCE
(>=60%) rather than purity, reporting the split and naming which part is the
carve problem and which the declaration problem.
Negative control over all five pre-existing verdict shapes (pure BODY, pure
PLUMBING, pure TABLE, MIXED, NOT FOUND) plus the S75 shape: 5 of 6 verdicts
UNCHANGED, only the SaveLoadRoutine shape flips PLUMBING REJECT -> TABLE
REJECT (MIXED).
Cookbook §447. The law: a class that cannot fire is worse than a class that does
not exist -- it converts "I don't know" into confident, specific, wrong advice.
When a verdict names a subsystem, check that subsystem owns the MAJORITY OF THE
BYTES before acting on it.
|
||
|
Drew T
|
23cd3a43a5 |
feat(decomp): bank main:func_8005DCA0 (118 ins) — and SaveLoadRoutine's body is BYTE-IDENTICAL
One clean-rebuild gate_main pass over main's stored drafts. 35 drafts on the slate -> 20 compatible after in-TU declaration resolution -> 1 byte-correct, found by bisection in 24 rebuilds. main SHA1 143dbb89... BYTE-IDENTICAL. banked: func_8005DCA0 src/800c3.c 118 ins THE HEADLINE IS NOT THE BANK. gate_main's per-function verdicts separate a BODY reject from a PLUMBING reject, and they say: SaveLoadRoutine: PLUMBING REJECT — SaveLoadRoutine is BYTE-IDENTICAL; all 3989 differing bytes are ELSEWHERE IN CODE. The substitution perturbed other functions (§376 — a stale forward declaration changes caller codegen). SaveLoadRoutine is 1,165 instructions -- the largest function left in the project, 9.2% of everything remaining, and carried as the §434 WALL. Its body is already correct. What rejects it is a forward declaration perturbing OTHER functions' codegen, which is exactly what fix_arity_callers -> cast_self_callers exist to repair. That is a plumbing job, not a matching job. Three genuine BODY rejects, correctly distinguished by the same verdict layer (divergence confined to the function itself): func_8001EFE0 (495 bytes), func_80015B6C (151), func_80011380 (8). Those drafts are really wrong. Honest read of the yield: 1 of 20 substituted drafts was byte-correct, so main's stored drafts are mostly NOT right. This tempers the "the endgame is integration, not drafting" line from the previous commit -- true for the overlays, only partly true for main, where several drafts need redrafting or permuter work. The distinction is now measured per function rather than assumed. Deferred to the reconcile chain, not discarded (11 dropped for in-TU decl conflict + 4 dropped across rounds to let the TU compile at all): func_800226C0 (670), func_800215F4 (465), func_8001FC08 (400), func_8005F290 (61) and the gate's own 11. All drafts remain on disk. |
||
|
Drew T
|
abea9f0ac2 |
feat(decomp): bank func_8016AE5C (85 ins) + frontier_classify — the frontier is not a drafting problem
func_8016AE5C (ov_SC03_108) was logged "match_one MATCH but the whole-binary
gate rejected -- CAUSE NOT DETERMINED". Determined: the body is byte-perfect (0
differing words inside the function; all 1,168 diffs are uniform +0x20 shifts
outside it) and it emits an 8-entry jump table that was never carved. It banked
unchanged the moment the §446 jtbl_carve per-table bound landed.
tools/frontier_classify.py (NEW) — classify every open stub by its TRUE BLOCKER
from artifacts already on disk (R33/offline-tooling-first: zero tokens, no
agents, no builds). "69 functions left" is a stub count, not a difficulty
measure, and routing drafting agents at carve or plumbing problems wastes them.
A-TWIN-REMAP 3 302 a byte-identical copy is already banked elsewhere
B-CARVE 11 3,301 owns a switch jump table -> the §446 class
D-NEAR 2 106 closeness <=25 -> permuter fuel, not drafting
F-FAR 3 223 draft materially wrong -> redraft
G-DRAFTED-UNK 49 8,724 drafted before, no usable verdict on record
H-VIRGIN 1 1 never drafted (and it is a DATA BLOB, not a function)
68 of 69 remaining functions already have a draft on disk. The endgame is a
verification/integration problem, not a drafting one.
TWO SELF-INFLICTED DEFECTS FOUND BY CHECKING AGAINST KNOWN-TRUE CASES, both the
session's recurring shape (a scan narrower than the claim it supports, R32):
* The sig directory is NOT the fleet. Alongside the 213 real binaries `.run/`
holds `SLUS_007.26` (a STALE duplicate of main under the ROM filename),
`resident_image`, and two CROSS-BUILD binaries (`sep8_SLUS_007.26`,
`aug31_USA_DEMO.EXE`). Counting them as peers reported 38 fns / 7,516 ins of
free twin-remaps -- mostly main "already banked" in ITSELF, the rest proven
in a PROTOTYPE that R13 forbids as evidence. Now derives the fleet from the
Makefile and prints what it ignored. True figure: 3 fns / 302 ins.
* The draft scan globbed `.run/S7*` only, missing `.run/S69m2`, `.run/S68m1`,
`.run/s67m1`, `.run/wave_ds2`, `.run/gate_lane`, `.run/backlog_drafts`. All
32 drafted main functions read as "never drafted", which would have sent
agents to redraft 6,328 instructions that already have drafts. Now one
pruned os.walk of .run (worktrees excluded -- 7.4 GB of duplicate sources).
Honest negative result: resident:func_800D06E8 (344 ins) did NOT bank. I
predicted the carve fix would clear it; it did not. Its blocker is still open.
|
||
|
Drew T
|
cb948a6bbc |
feat(decomp): the ov_SC01 reloc-only cluster + its 5th latent victim — 5 fns, 1,301 ins
S74 handed this forward as "1,116 instructions behind one question": family_remap
on ov_SC01_004/005/006/008 gated DIFF 4/4 against the banked exemplar
ov_SC01_009:func_8017EB08, and the class had been carried as a codegen wall since
S70. The four bodies were byte-identical to the exemplar the entire time.
Word-level classification vs the exemplar, computed independently twice (a Fable
agent's script, then mine from scratch against the retail images), identical:
nins=279 EQ 213 · RELOC-HI16 23 · RELOC-LO16 24 · INTERNAL-J 19 · CODEGEN 0
Zero register-allocation, instruction-selection or scheduling differences.
ROOT CAUSE — tools/jtbl_carve.py reserved ONE WORD TOO MANY per table:
* spimdisasm runs an island's LAST `jtbl_` dlabel one word into the following
NON-ZERO data (string bytes 0x696F760A / 0x000013FF / 0x62647020), so the
zero-word trim cannot see it; and
* the over-span clamp that would have caught it was guarded by
`len(sltiu_bounds) == 1` -- but `sltiu` is ALSO how gcc emits an unsigned
range check ((u32)(x-lo) < n, I1). These four carry five distinct sltiu
immediates, so the guard silently disabled itself on precisely the functions
that needed it.
0x2C reserved for a 0x28 table => image 4 bytes short => ~850 %lo immediates
shift => whole-binary DIFF about a function whose own bytes are perfect.
Fixed with a PER-TABLE bound: gcc-2.7.2's dispatch is a fixed idiom, so the
`sltiu` nearest ABOVE that table's own %hi(jtbl_X) is unambiguous whatever else
the function tests. Second defect stacked behind it: a carve span whose
JTBL_PADS line lacks a `tables=` comment lost its existing table's start on
merge and refused "table starts do not fit the span" -- which harvest_verify
then "repaired" with a needless jr_isolate_all that walked back into the first.
THE NEGATIVE CONTROL IS THE STORY. Run over every other open table-bearing stub
fleet-wide, the fixed bound changed exactly one more table: ov_SC06_022/
func_80185B80 (185 ins), a FIFTH victim nobody had drafted against. A guard that
disables itself on a common idiom does not fail once -- it fails quietly across
the whole corpus.
Banked, each with its own byte-gate verdict (--no-propagate, clean re-gate):
func_8017EB30 ov_SC01_004 279
func_8017F2D4 ov_SC01_005 279
func_8017F2D4 ov_SC01_006 279
func_8017EC68 ov_SC01_008 279
func_80185B80 ov_SC06_022 185
Also here:
* dedup_propagate: memoize find_site's mask (lru_cache) -- 54 ms of masking
per call over the whole source, recomputed though it depends only on the
text. 2x on that loop (58.3 -> 33.0 ms/call), NC identical on 120 addrs.
Scoped honestly: that loop is ~2.4 min of a 30-min run; the profiler puts
43% in family_remap._alias_decl_for, which is NOT fixed here.
* Makefile: `clean` says out loud that BINARY= is ignored and it is fleet-wide
(cookbook §445) -- it silently deleted asm/ for all 213 binaries this session.
* Cookbook §446 (the carve law: when a standalone-MATCH jtbl draft gates DIFF,
diff the carve extent against 4 x sltiu before touching the body), §445, and
SETUP rows for both tools (R21).
* CURRENT_PHASE: the S75 log, incl. the measured fleet dedup-hygiene census
(~2,073 fns / ~12,116 items, all ALREADY MATCHED -- cleanup, not work) and
Drew's decision to leave it and gate --no-propagate from here.
|
||
|
Drew T
|
0c2b37a287 |
fix(gate): overlays.mk carve-state snapshot was singular, so the revert half-restored
Found by running one reject to ground. After a gate that REJECTED
resident:func_800D06E8, config/overlays.mk had a 4th JTBL_PADS entry and had
LOST `--pre hdr.rodata.o` (the §440 resident leading-rodata sandwich). The
binary then would not build at all -- "consumed 3 rodata jump table(s) but 4 pad
spec(s) given -- table-count drift vs the carve" -- while src/ was perfectly
clean, which is the only place anyone looks before building.
Root cause is a silent narrowing in the classic shape. harvest_verify snapshots
ONLY the gating binary's own overlays.mk block on purpose (the file is shared by
every parallel gate; a whole-file restore resurrects other binaries' lines --
the S62 defect). But _mk_block_span was SINGULAR: the first `# --- <binary>`
header through the next `# --- `. A binary whose carve state spans more than one
block was half-snapshotted and silently half-restored. It returned a TRUE span
for a scope smaller than the caller believed, and nothing compared the two (R32).
Blast radius measured before costing (R37): 1 of the 142 binaries that have a
block -- resident, which has exactly two (§8e pad spec, §8f leading-rodata
sandwich) and still holds 587 instructions of open stubs.
_mk_block_spans (plural) snapshots a LIST, restores tail-first so earlier spans
stay valid, collapses to the snapshot when the header count changed rather than
leaving half-state, and RE-READS and compares the result -- the defect it
replaces was a reported success. _mk_block returns None (not []) for the 71
binaries with no block, so the caller's guard keeps its meaning.
Negative control, three ways:
* snapshot -> restore is a NO-OP on 142/142 binaries with a block;
* the real S75 damage is fully undone;
* the OLD single-block restore provably does NOT undo it -- the positive
control that proves the fix is load-bearing, not decorative.
Cookbook §444 also records the two other findings from the same reject: the
classified ledger stores the LADDER'S FINAL verdict (the recorded CC1-FAIL came
from a late sig_unify rung; the raw draft compiles and fails on BYTES), and
match_one MATCH + rtu_match MATCH is still not bankable -- func_800D06E8's real
blocker is a jump table (built binary 20 bytes longer, 0x800CEDFC holds a table,
69,571 words shift), because neither matcher LINKS.
|
||
|
Drew T
|
cdd535e45b |
fix(tools): reconcile_tu's cc1 premise, the &-cast arms, and the worktree sig gap
The S74 checkpoint's "one unfixed defect that is actively costing banks"
(reconcile_tu manufacturing declaration conflicts), run to ground — plus the
harness gap that produced a false carve-corruption verdict.
reconcile_tu.py — three defects, measured against the real gcc-2.7.2 front end
(cdecl._cc1_accepts, the oracle cdecl.compatible was validated with; R33):
* The premise "a decl BELOW still conflicts" is TRUE at file scope and FALSE
at block scope. cc1 ACCEPTS a block-scope extern against a TU decl below it
(pedwarn "type mismatch with previous external decl"); conforming it is
destructive, because the TU's decl names the TU's TYPE and a type declared
below the splice point is not in scope AT it -- the emitted result gets
"syntax error before 'D_x'". Byte-witnessed on resident:func_800D06E8 (344
ins), whose block-scoped `extern Blk80078E78` became `extern
Struct80078E78`, typedef 388 lines lower. That construct is what this
ladder's OWN scope_demote_drafts (§8d) rung emits on purpose, and three
already-banked functions in that TU use it: one rung undoing another.
* The cast pass rewrote COMMENT PROSE -- 8 rewrites inside one header comment,
including inside a quoted cc1 diagnostic. Now matches on cdecl._mask
(length-preserving, so a mask offset is a source offset) and splices into
the original.
* `&sym` emitted `&` applied to a cast: legal for the scalar arm, `invalid
lvalue in unary '&'` (measured) for the array/fnptr/fnptr_array arms. `&`
now selects a pointer form and consumes itself -- but ONLY with no trailing
subscript, because `&sym[i]` is the address of ELEMENT i and the old code
had that case right. That last clause exists because the R39 negative
control caught the fold as a regression in the first cut of this fix.
gate_stage.py — `--skip-stages` / `GATE_SKIP_STAGES` (loud when used). Stage 0
gates raw drafts first, so a broken rung can only cost a RECOVERY, which is
exactly what makes it invisible: the function it destroys was already failing,
so its DIFF reads as a fact about the function.
verify_worktree.py / jr_isolate_all.py / parallel_gate.py — provision() now
symlinks every .run/sig.*.jsonl (main clone 259, provisioned worktree 0), the
third member of the class holding extracted/ and .run/obj40. parallel_gate was
fixed for this identical bug in S69: two provisioners, no shared list, found
twice; they now cross-reference each other. jr_isolate_all no longer swallows
the resulting FileNotFoundError into `except: continue` -- that turned a missing
index into a confident carve-CORRUPTION verdict over 2,603 of 2,603 functions
(R54). Adds _assert_scan_covered: attempted == raised means the scan measured
nothing, so its zero is an artifact, not a finding (R32).
Verification:
* 4 cc1 probes (the table above), each run on the pinned front end.
* R39 negative control over the stored-draft corpus: 661 adjudicated, 652
IDENTICAL, 9 CHANGED and every one an intended class. 4,173 of 4,864 drafts
unadjudicable (filenames that are not func_<ADDR>) -- stated, not hidden.
* jr_isolate_all ov_SC03_105 --dry-run: unchanged in the main tree.
* make clean/extract/build BINARY=resident -> 8e17e02f... BYTE-IDENTICAL.
Docs ship with the change (R21): cookbook §442/§443, index regenerated (1,112
sections), 3 docs/SETUP.md rows, CURRENT_PHASE S75 log.
|
||
|
Drew T
|
7f7e1c7d90 |
docs(phase-31): refresh the S74 checkpoint — §440/§441 and the doc-state line
The checkpoint listed §435-§439 and was written before the documentation catch-up; a stale checkpoint is worse than an absent one. Now records that every tool change this session carries a SETUP row and a cookbook entry, and names the rule the gap taught. |
||
|
Drew T
|
eb0329b3a4 |
docs: the five agent-authored tool changes that shipped with a commit message and no docs
I answered Drew's yes/no honestly — NO — and this closes it. Every gap had the same shape: a tool
change that came from a SUBAGENT arrived as a report, I merged the code and wrote it up in the
commit message, and a commit message is not the knowledge base. The six changes I made myself were
documented inline; these five were not.
SETUP.md tooling ledger:
* `ld_interleave` — the row still said "interleave linker inputs" and predated BOTH --order
(S72, main's 7-piece island) and --pre (S74, the resident's leading-rodata header).
* `harvest_verify` — the typedef strip-set is computed SCOPED (`above=fn`) now, and why.
* `jtbl_rodata_pads` — a new row for the three S74 measurement corrections, each of which ACCUSES
THE CARVE when it fires, plus why the trailing-.align one stayed latent (zero_gap self-corrects
an undershoot when the next item is an anchor, and a C jump table has no anchor).
* NEW row `jtbl_carve` — the `covered` / `covered-tpad` verdicts.
* NEW row `jr_isolate_all` — `_region_emit_start` and the empty-closing-region skip.
Cookbook:
* §440 — a carve piece binds to a SUBSEG, not a function, so §8b's "non-adjacent => ISOLATE" is
over-strict: EXTEND the carve across still-stubbed material instead. Four byte-proven
corollaries (migrated tables self-align by SPAN-RELATIVE offset; JTBL_PADS counts cc1 tables
only so a mixed span's spec grows as siblings bank; the zero-word rule is invalid across a
migrated boundary; a covered table at 4-mod-8 gains 4 bytes when it banks). Plus the resident's
rodata->text->data->rodata->data layout and why it needed --pre.
* §441 — three more instrument defects that each produced a confident, precise, WRONG verdict
about a correct draft, with the habit they share: when a gate rejects a body you have
byte-verified standalone, the first suspect is the gate.
Playbook: new step 2a-0 — the same-address lead is size-filtered now; read the `⚠ IGNORE` line, and
regenerate any pack built before S74 rather than trusting a bare address lead.
|
||
|
Drew T
|
b24394e732 |
docs(phase-31): S74 FINAL checkpoint — 50 banked, seven tool defects, R22 213/213
Every reject class this session was an instrument defect, not a codegen wall: 24 already-MATCHed bodies were sitting behind seven tool bugs, six of which are now fixed. The checkpoint names the seventh (reconcile_tu manufacturing declaration conflicts) as the first thing to fix next session, and the harness gap (verify_worktree omits the sig files, and the scan that needs them swallows the error and reports a false corruption) beside it. Fleet verified from a clean rebuild AFTER the last bank: check-all 213 passed, 0 failed of 213. split_indicator 213 OK and now a hard gate. INCLUDE_ASM lines in src/ 1,086 -> 1,036, measured at both commits — the same 50 the gates reported, counted independently from the source. |
||
|
Drew T
|
8edb918480 |
feat(cards): size-filter the same-address lead (§238 homonym) + bank the S74 lever set
THE CARD USED TO HAND AGENTS A WRONG TWIN ABOUT ONCE IN FIVE. `⭐ func X IS BANKED AT THIS ADDRESS`
never checked that the two functions were the same SIZE, and overlays share addresses between
unrelated functions as readily as they share code. Measured over this session's ~60 cards: about a
dozen agents reported discarding the lead themselves, and one card advertised a 72-instruction
namesake — with journal history claiming "already MATCH closeness 0" — to a 241-instruction target.
A confidently wrong lead costs more than no lead, because the agent believes it.
corpus.sig already carries `nins` and `h_seq`, so the fix is free: `_same_addr_banked` now returns
(binary, nins, h_seq); the card keeps a lead only at a MATCHING instruction count, marks it strong
when the mnemonic skeleton matches too, and prints an explicit `⚠ IGNORE` naming the binaries where
that address holds something else, with both sizes.
VERIFIED IN BOTH DIRECTIONS against known-true cases before being believed (never trust a filter you
have not tried to fool):
* the trap: ov_SC03_105:func_801806F8 (241) vs ov_SC03_013 (72) -> `⚠ IGNORE`.
* the positive: ov_SC02_003:func_80187B40 (158) -> strong lead to ov_SC02_000 (158, same h_seq,
banked this session) AND, in the same card, warned off ov_SC04_011's 138-ins homonym at that
same address. That is precisely the pair a wave agent sorted out by hand hours earlier.
Cookbook §438 (the law: a lead is fuel only if it carries the cheapest fact that can refute it —
size refutes a homonym for free and nobody had asked) and §439, the S74 lever set: MEM_IN_STRUCT_P
as a two-way alias-oracle dial (four agents converged on it independently); `goto`-into-a-shared-tail
vs longhand as a REGALLOC dial because gcc-2.7.2 cross-jumps after allocation; `for` -> do/while as a
length-changing scheduling dial; allocno PRIORITY via a non-volatile asm at a loop head, with the
measurement that register pins are actively harmful for that class; the -O0 global-RMW rule
(`x++` emits the copy-back quartet, `x = x+1` does not); why `sll 16; srl 16` survives only across a
CALL; `sltiu N` without `addiu -1` proving an empty `case 0` is mandatory; block-scoped temps in
duplicated bodies; two `register asm` vars cannot share a hard reg; and `x*32` vs `x<<5` emitting
lh vs lhu — which match_one's %lo mask HIDES, so it must be checked with objdump.
|
||
|
Drew T
|
5208f2279d |
feat(decomp): parallel gate — 2 fns across 1 binaries (1 workers)
ov_SC06_029 func_80182ED8 func_80184084 |
||
|
Drew T
|
6e840730a9 |
feat(carve): bank 4 more via the carve chain — and §8b's "non-adjacent => ISOLATE" is over-strict
resident:func_800D00E4/func_800D02D0/func_800D0488 + ov_SC07_002:func_80180248, all byte-verified
from clean rebuilds (resident 8e17e02f, ov_SC07_002 fad71342) and counted from the SOURCE.
ov_SC06_029's two are re-gated separately against HEAD — this agent's worktree predated five banks
there, so its numbers for that binary no longer apply.
TWO OF THE SIX NEEDED NO CARVE WORK AT ALL, AND CARVE-REFUSED WAS AN INSTRUMENT VERDICT.
ov_SC07_002:func_80180248's table is ALREADY inside a carve bound to its own subseg: in stub state
spimdisasm migrates the table into the fn's .s and the object fills the piece exactly, so banking
just swaps that block for cc1's identical one. `island_probe` classified it `tail` on the table's
ADDRESS, `apply()` routed it to build_carve, which resolves spans out of the RAW data asm where a
carved table no longer is -> "not found in the raw data asm" -> harvest_verify booked CARVE-REFUSED.
A verdict about the route we chose, not about the function (R43). jtbl_carve now has a `covered`
verdict (table inside an existing carve bound to the fn's OWN subseg) and a `covered-tpad` wall (the
retail copy carries a trailing §8a pad the matched body won't emit — bankable, needs a `0t<n>`
entry); a fully-covered batch is a no-op before either route.
THE RESIDENT CAN CARVE LIKE AN OVERLAY. Its three tables are adjacent and lead the island
(0x450e0..0x451ac, one span, all in subseg `resident`). The genuinely new part: the resident opens
with `- [0x0, rodata, hdr]`, a 1-word .rodata header BEFORE the code, so its layout is
rodata -> text -> data -> rodata(carve) -> data, which `ld_interleave --order` cannot express (every
listed piece lands after TEXT_START, and hdr.rodata.o would fall into the unchecked `empties` bucket
and be parked after the text, moving every byte). New `--pre` places a leading-rodata piece ahead of
the text; resident_JTBL_INTERLEAVE uses it.
NEW LAW, BYTE-PROVEN (§8b was over-strict — EXTEND the carve, do not isolate): a .rodata carve piece
binds to a code SUBSEG, not a function, and the object's .rodata is the address-ordered
concatenation of cc1's tables for BANKED functions and still-stubbed functions' MIGRATED tables. So
a span may legitimately hold a MIX, and extending a carve across an align-pad word and two unrelated
STILL-STUBBED tables was byte-identical with nothing banked — where the tooling demanded a
jr-isolation. Corollaries, all measured: migrated tables self-align (spimdisasm emits `.align 3` iff
the table's SPAN-RELATIVE offset is 8-aligned), so stubbed tables need no spec; JTBL_PADS counts cc1
tables only, so a mixed span's spec GROWS as each sibling banks; and the zero-word rule is INVALID
across a migrated boundary, because that zero is supplied by the preceding migrated block.
ALSO REPORTED, NOT FIXED (harness gap worth its own change): verify_worktree.provision omits
`.run/sig.<bin>.jsonl` — main clone 259 files, provisioned worktree 0 — and jr_isolate_all's
carve-ownership scan swallows the resulting FileNotFoundError in a bare `except: continue`. Measured:
2603 of 2603 functions raised, the scan found 0 owners, and the run aborted with a CONFIDENT FALSE
verdict ("committed .rodata carve ownership is not 1:1 — stranded/duplicated carve"). Both resolve
instantly once the sigs are present. Any worktree-run isolation before that is fixed reports a
corruption that is not there.
|
||
|
Drew T
|
089311e74d |
feat(md_SC07_004): 10/10 banked — md_SC07_004 is now ZERO stubs, and the "decl conflicts" were fake
Clean rebuild BYTE-IDENTICAL 87ac0de3; corpus.stubs('md_SC07_004') 10 -> 0, counted from the SOURCE.
THE §376/§378 CHAIN WAS NEVER NEEDED. Zero declaration edits: no fix_arity_callers, no
cast_self_callers, no --any-proto, no --sync-decls, no undo journal. `git diff -U0` on the TU removes
exactly the 10 INCLUDE_ASM stubs plus one hoisted typedef. Every recorded "declaration conflict" was
an INSTRUMENT defect. Four of them, all named, three patched here:
1. `CC1-FAIL(no-diagnostic)` was neither cc1 nor no-diagnostic. The failing stage was
`jtbl_rodata_pads --derive`, which prints to stderr AFTER cc1 exits 0 quietly. `_items` matched a
rodata anchor only as `D_xxxxxxxx:`, but a block written as inline `__asm__` in C arrives in the
labels.inc macro form `dlabel D_xxxxxxxx` — so an 8-byte hole opened in the walk and every C jump
table after it died with "island layout drift". The harness label was wrong twice: it said CC1
when the failure was a post-maspsx filter, and no-diagnostic when there was a precise one.
2. Same file, UNALIGNED ANCHOR: the ctable branch read `word(pos)` without first stepping the
sub-word zero gap, so a preceding `.asciz` ending at an odd address made it refuse a correct
layout. Now reuses the same zero_gap the anchor branches already use — a no-op wherever pos is
already aligned, i.e. everywhere that builds green today.
3. `harvest_verify` computed the typedef strip-set UNSCOPED: `cdecl.typedef_names(path)` without
`above=fn`, which that function supports for exactly this. A typedef declared BELOW the splice
point got stripped out of the draft that needed it -> `parse error`, logged as PLUMBING and
indistinguishable from a real conflict. One line.
4. NOT PATCHED, AND THE MOST IMPORTANT ONE: `reconcile_tu.py` (gate_stage's `-rc` stage) MANUFACTURED
both remaining "conflicts". The same drafts gate 9/9 byte-identical through harvest_verify and
7/9 through gate_stage. Isolated stage by stage, `-rc` (a) rewrites deliberately BLOCK-SCOPED
externs to a file-scope spelling whose typedef is declared ~2,800 lines lower — overwriting the
TU's own byte-proven house style, which three already-banked functions in that file use; and
(b) substitutes identifiers TEXTUALLY, including inside comments and inside `&`-expressions,
emitting `*(T *)&((s32 *)&D_800AE620)`. A correct draft using the block-scope-extern idiom
currently CANNOT survive gate_stage. Left for a deliberate fix: `--stages` should be able to skip
reconcile_tu, or a draft should be able to opt out.
The two surviving non-stub source edits are byte-neutral (proven by the SHA above): a §304
migrated-rodata re-emission (`D_801A01EC`, the exact form this TU already uses three times, needed
because banking the body deletes the .s that carried the island word), and one typedef moved up so a
function above it can see it (typedefs emit no bytes).
Also banked the 10th stub (func_801ADA10) that defect 3 had been silently blocking.
Regression-checked by the agent: main, md_MAIN_003, md_SC07_003, md_SC03_073, md_MAIN_011 all
rebuild BYTE-IDENTICAL. A full R22 follows before this session closes.
|
||
|
Drew T
|
2954b250ec |
feat(decomp): parallel gate — 5 fns across 1 binaries (1 workers)
ov_SC06_029 func_801801D8 func_80180A70 func_801867D0 func_80187660 func_801898CC |
||
|
Drew T
|
5e10215269 |
feat(md): bank the 4 -O0-stranded functions — and the class is now essentially empty
md_MAIN_011:func_800CF28C · md_MAIN_003:func_800D0268/func_800D0740/func_800D0C50, all byte-verified
from a clean rebuild and counted from the SOURCE: md_MAIN_011 is now FULLY MATCHED (0 open stubs),
md_MAIN_003 is down to 1 (func_800CF3E8).
THE PREMISE I HANDED THE AGENT WAS WRONG, AND IT SAID SO. md_MAIN_011 is already a whole-object -O0
module — no carve was needed. Its real blocker was tools/jtbl_rodata_pads._s_rodata_span ignoring a
trailing `.align`, the SAME defect this session fixed for md_SC07_003 from the other direction: two
agents converged on it independently. Adopted this agent's stricter form (only a TRULY trailing
align rounds `hi`; an interior one is followed by data that sets `hi` higher anyway).
Note WHY it stayed latent: `derive`'s zero_gap self-corrects a 1-3 byte undershoot whenever the next
stream item is an anchor. A C jump table has NO anchor — so the bug can only fire the moment someone
banks a switch function into such an object, and when it fires it accuses the CARVE ("island layout
drift"), not itself.
md_MAIN_003 needed one new -O0 object, and the boundary I proposed (0x1f74 -> 0x1e58) was both too
narrow and off by 0x2B8. The carve made is `md_MAIN_003_o0e` at 0x1308 (vram 0x800D0100) running to
the existing o0c boundary: everything in that span is a §265 verbatim __asm__ body or an INCLUDE_ASM
stub — zero optimizable C — so the whole tail flips with one cut. Proved byte-identical with NOTHING
banked first (§431 discipline), then the three drafts gated one at a time.
TWO MORE GENERAL DEFECTS FIXED IN jr_isolate_all, both of which silently mis-place a boundary:
* an item-less CLOSING region emitted a duplicate `- [off, c, …]` line and the validator refused;
the empty-region skip covered only region 0, and `_partition`'s empty `footer` made the closing
region look non-empty.
* A §265 VERBATIM __asm__ BODY IS PREAMBLE, AND PREAMBLE IS ASSUMED BYTE-NEUTRAL. It is not — it
emits bytes. `parse_overlay_c` has four addressed-anchor forms and a verbatim body is none of
them, so it attaches to the NEXT anchor: cutting at func_800D0268 would have moved 0x168 bytes of
other functions into the new object while the yaml claimed the region starts higher. New
`_region_emit_start()` derives the yaml offset from the region's CONTENT (item addresses + every
.globl/.ent the text names that resolves inside the object) and takes min(cut, emit), so a
boundary can only move DOWN. Where no verbatim asm is in play it equals the cut — every existing
isolate is unchanged.
BLAST RADIUS PROVEN, not argued: jtbl_rodata_pads is in the build path (`--derive` for md_*/main),
so the agent rebuilt main + all 70 md_* from scratch (71/71) and then ran the full fleet:
**make check-all 213/213 passed, 0 failed**, main 143dbb89 BYTE-IDENTICAL.
CENSUS, denominator asserted (1057 live stubs, 0 without a .s): exactly ONE -O0-prologue stub
remains stranded in an -O2 TU fleet-wide — main:func_8002C410 in src/800_b.c, 299 ins. Nothing more
should be built for this class; the general tool already existed and what was missing was
correctness, not coverage.
|
||
|
Drew T
|
3a886b9652 |
fix(tu-split): a block comment a construct OPENS MID-LINE and WRAPS defeated every peeler
FIVE independently-MATCHed ov_SC06_029 bodies were rejected by a `parse error before '#'` in a file
the GATE ITSELF generates, at a line no draft contains. The isolation emitted, into the §8b carried
decl layer:
extern #define CALL_80185C6C ((void *(*)(s32, s32))func_80185C6C) extern void func_8012C218();
CAUSE. Every peeler in the TU-split chain asked `line.strip().startswith("/*")`, which is blind to a
comment a construct opens MID-LINE and wraps. The declaration ends at its `;` BEFORE the `/*`, so
the caller resumed on the comment's PROSE with in_block=False — and the prose is hostile: `(s32,s32)`
closes a depth-0 paren, `seen_header` latches, and every later `;` reads as a K&R parameter
declaration, so one "construct" swallowed the whole preamble. `parse_overlay_c` then anchored a
`def` on a pure declaration run and `def_proto` rendered it as that definition's implied prototype.
A SECOND defect rode along: `_file_scope_decls` hoisted such a col-0 line VERBATIM, unterminated
`/*` included, so the carried layer opened a comment that silently ate the next two declarations —
a dropped file-scope decl is a silent byte-changer. Building the guard exposed a THIRD: `_strip`
tested for `/*` before stripping `//`, so `// … src/*/*.c` (7 lines in 5 sources) opened a phantom
block comment and blanked everything to the next `*/`.
FIX: one derived comment-state oracle, `comment_open_at()` (R33) — per line, does it BEGIN inside a
block comment — consulted by parse_overlay_c, def_proto, split_src_region.parse and
jr_isolate_all._file_scope_decls (which also truncates a hoisted decl at an unterminated `/*`).
`_strip` now lexes left to right. `parse_overlay_c` RAISES (R43) when a wrapped comment closes with
code after the `*/`, because that construct could never anchor — 0 occurrences fleet-wide.
MEASURED, not assumed:
* the shape occurs 238 times across 193 tracked .c files; 153 are col-0 hoistable declarations in
150 files — every one a binary whose next isolation would have carried a broken decl layer.
* A/B over all 4,188 tracked sources, old parser vs new: round-trip identity 4188/4188 both ways;
exactly 2 files' item lists change, each losing one PHANTOM def and gaining nothing; malformed
implied prototypes 999 -> 984; 0 refusals.
* negative control BEFORE any edit: ov_SC06_029 extract+build -j+check BYTE-IDENTICAL b7b0d4ae.
* with the fix, gate_stage banked 5 of 6 drafts, counted from the SOURCE; the 6th
(func_80184084) is the separate CARVE-REFUSED class.
The 984 residual malformed prototypes are a DIFFERENT pre-existing trigger (col-0 lines gluing
declarations to DEFINE_func_*() invocations); 4 still carry a `#` and survive only because it lands
in a dropped segment. Named in §437, deliberately not fixed here.
Cookbook §437 + a SETUP.md tooling-ledger row for comment_open_at (parse_overlay_c may now raise).
The banks themselves are NOT in this commit: the agent's worktree predated func_8017F9C0's bank, so
adopting its TU verbatim would have destroyed one. They get re-gated against HEAD with these tools.
|
||
|
Drew T
|
e9220d2d8a |
fix(pgate): a carve left asm/ stale, and the refusal that reported it named nothing
TWO DEFECTS, ONE INCIDENT. ov_SC03_105's own SUCCESSFUL gate committed an isolation's new TUs (src/ov_SC03_105/ov_SC03_105_jr_801813BC.c, _jr_80181C84.c) whose `INCLUDE_ASM` lines name .s files that do not exist until a re-extract. corpus.stubs then refused — correctly, "the tree and the source disagree" — so the NEXT gate on that binary died before doing any work, and a matched body (func_801818E8) sat unbankable behind it. 1. THE REASON NOW TRAVELS WITH THE REFUSAL. stubs_of() returned a bare None and the caller printed "corpus refused in worktree": true, and naming nothing. It took a hand-built worktree to see that corpus had said exactly what was wrong all along. It now returns the message and the result JSON carries it. Verified against a TRUE reproduction (delete one .s in a scratch worktree): verdict REFUSED + "1 stub(s) have NO .s on disk ... src/...:4214: asm/.../func_8017F018.s". 2. THE MERGE STEP REPAIRS WHAT IT BROKE. For every binary whose carve created a new source file, assert corpus.stubs is satisfiable in the MAIN tree; if not, `make extract BINARY=<b>` and re-assert; if it is STILL unreadable, say so loudly rather than leaving a tree no tool can read (R32/R43). This is the R22 corollary — a config change needs a make extract, not just a make check — firing inside a tool's own commit. Repaired the live instance by hand first: rm -rf asm/ov_SC03_105 + extract + build -j + check -> BYTE-IDENTICAL d305ff6d, corpus readable again, and func_801818E8 then banked (commit:3718). Cookbook §436-D; wave-playbook §6 carries the hand-gating version of the same warning. |
||
|
Drew T
|
d0389352d7 |
feat(decomp): parallel gate — 1 fns across 1 binaries (1 workers)
ov_SC03_105 func_801818E8 |
||
|
Drew T
|
0109f5bc92 |
feat(decomp): parallel gate — 1 fns across 1 binaries (2 workers)
md_SC07_003 func_801A09C8 |
||
|
Drew T
|
ee2963514a |
fix(tools): jtbl_rodata_pads measured a .s rodata span without its trailing .align
`_s_rodata_span` summed only the DATA an included `.s` emits, so a file ending `.asciz "r"` + `.align 2` measured 0x801A00D8..DA instead of ..DC. The island walk then landed 2 bytes short and `--derive` aborted with `C table entry 0 at 0x801A00DA ... island layout drift` — a true statement about a span that was never the real one. Worse, the Makefile pipes md_*/main through `--derive` without `set -o pipefail`, so the failure could yield a short object rather than stopping the build. Three lines: round `hi` up to the trailing align, which is what the assembler actually emits. CONTROLS (both on UNMODIFIED sources, so this is proven byte-neutral, not argued): md_SC07_003 clean rm -rf + extract + build -j + check -> BYTE-IDENTICAL 46af79a1 gate_main --assert-baseline -> BASELINE GREEN 143dbb89 Found by a drafting agent (md_SC07_003/func_801A09C8) that ran its own gate reject to ground instead of respelling its body, and proved the patch in scratch first: with the fix its draft's .rodata is byte-identical to the green control and .text differs in exactly 1 of 6266 words — a %lo(jtbl) carrying a section-symbol reloc that three already-green C-jtbl functions in the same object already ship. Cookbook §436-C, with the habit that found it. |
||
|
Drew T
|
7ce27c0d1f |
feat(decomp): parallel gate — 2 fns across 2 binaries (6 workers)
ov_SC05_010 func_8017F5B8 ov_SC03_024 func_801830A8 |
||
|
Drew T
|
7c2342f87f |
feat(decomp): parallel gate — 5 fns across 3 binaries (4 workers)
ov_SC02_011 func_80183178 ov_SC01_084 func_80182328 ov_SC03_105 func_801813BC func_80181C84 func_8018624C |
||
|
Drew T
|
797cb97cb7 |
feat(decomp): parallel gate — 9 fns across 6 binaries (8 workers)
ov_SC06_024 func_8017EC4C ov_SC06_029 func_8017F9C0 ov_SC03_105 func_801867D0 ov_SC05_010 func_80180F84 ov_SC02_005 func_80185060 func_80185E80 ov_SC02_011 func_80183630 func_8018418C func_80188E3C |
||
|
Drew T
|
f2f3103691 |
feat(decomp): parallel gate — 7 fns across 7 binaries (10 workers)
ov_SC05_001 func_8017FE0C md_SC07_003 func_801A1120 ov_SC03_013 func_8017FAA8 ov_SC03_011 func_80180B68 ov_SC02_000 func_80187B40 md_SC07_004 func_801A94A0 ov_SC07_000 func_8017ECB4 |
||
|
Drew T
|
95c7b7fe0f |
fix(tools): two tools read a source of truth describing a different world (+ hard-gate the third)
Three independent split agents hit both defects in one session, on the tools that CERTIFY and UNDO
the work they were doing. Each is fixed, negative-controlled against the exact failing case, wired
into its siblings, and documented in the same change (cookbook §436).
1. split_indicator attributed a jump table by the STUB'S DIRECTORY PATH. `make extract` does not
prune a re-homed subseg's `nonmatchings/<old>/` dir, so after a correct, byte-green §431 split
both the old and new dirs hold the moved stub — and the tool printed NEEDS SPLIT for a split that
was already correct. owners() now derives the owner from the CONFIG by address (R33), exactly as
jtbl_carve.func_subseg already does for the identical §8b hazard, and NAMES any leftover stub in
a `note:` line. Notes now print on an OK verdict too: hiding one behind `st != OK` is the same
defect in the other direction — a true verdict about a narrower world than the reader believes.
PROVEN by planting a stale stub for func_80182A00 under its old subseg: OK + the note, where the
old code would have seen one subseg owning two spans. --self-test still PASSes both directions.
2. jtbl_carve --revert did `git checkout --` on the WHOLE splat yaml. The carve owns only the
trailing data/.rodata region; the `c` pieces are source configuration it never writes. The blunt
form cannot tell "carve state I just added" from "the §431 split someone added to the same
uncommitted file", so --revert after a carve PROBE silently un-split the overlay — each agent
recovered only because they had backed the yaml up by hand. It now splices back only its own
region (parse_config gained an optional `lines=` so the SAME region derivation runs over the
committed text — one derivation, two callers), refuses loudly if the committed region carves onto
a subseg the current config no longer defines, and reports how many uncommitted `c` pieces it
preserved. PROVEN in the ov_SC01_084 worktree: carve → revert → the uncommitted split survived
("PRESERVED 30 uncommitted `c` piece(s)"), carve lines gone, diff back to the 6 split lines.
SIBLING: jtbl_family_bank.revert carried the same blunt checkout for the isolation's code pieces.
It now keeps whatever pre-dated the attempt (the `keep_regions` signal it already trusts for
src/) and NAMES anything it drops — an isolation region and a §431 split piece are both
`<ov>_jr_<addr>`, so no name test can tell them apart and only that signal can.
3. NOT A DEFECT, and recorded as such: a speculative carve fails the build with `jtbl_rodata_pads:
consumed 3 rodata jump table(s) but 9 pad spec(s) given`. That is R43 working — the pad spec is a
CONSEQUENCE of banking, not a prediction of it — and it reproduces identically on the pristine
unsplit config, so it is never evidence about a split.
make tools-health: split_indicator is a HARD GATE now, as its own comment promised it would become
once the last violation was split. 213 OK of 213; a new one fails the build instead of being echoed
past.
Cookbook §435 (an overlay TU split is near-free — 0/3,074, 1/2,679, 2/3,254 names crossed, because
the §8b carried decl layer re-emits externs per region so only typedefs can cross; and the gap test
between two rodata runs is "is this word a valid code address", not "is it zero") + §436 (the two
defects and the shape they share). Playbook + SETUP.md carry the emptied CARVE-BLOCKED class.
|
||
|
Drew T
|
c7d67048a0 |
chore(waves): regenerate config/wave_exclude.txt — the CARVE-BLOCKED class is gone
16 of 27 entries dropped as STALE, all of them the CARVE-BLOCKED set the four §431 splits in commit:3709 just unblocked. 11 WALL entries kept (curated, pinned by their `# WALL:` annotation, which exclude_audit re-reads as a pin so a wall survives regeneration). An exclude list records what the TOOLING could not do, so it becomes a list of work you decided not to do the moment the tooling improves — regenerating it is part of the fix, not follow-up hygiene. draw_waves --exclude-file audits and REFUSES a stale list. |
||
|
Drew T
|
33c292dea5 |
feat(overlays): split the 4 carve-blocked subsegs at their jtbl-span TU boundaries
BYTE-IDENTICAL on all four, with NOTHING banked (clean rm -rf asm/<bin> + extract + build -j + check), which is the whole point: the structure lands first and proves neutral, then drafts bank against it. split_indicator: 213 OK, 0 needing attention, of 213 — the CARVE-BLOCKED class is now EMPTY fleet-wide. One code object contributes exactly ONE contiguous .rodata run, so a subseg owning raw jump tables in two non-adjacent spans could carve only one of them and every switch function in the other span was unbankable at any effort (cookbook §426/§431). ov_SC01_084 2 pieces cut 0x80182A00 (0x5A8A8) ov_SC02_005 3 PIECES cuts 0x80185060 (0x5CF08) + 0x80185E80 (0x5DD28) ov_SC02_011 3 PIECES cuts 0x80183178 (0x5B020) + 0x80188E3C (0x60CE4) ov_SC03_105 2 pieces cut 0x8018624C (0x5E0F4) TWO OF THE FOUR NEEDED A CUT THE BRIEF DID NOT NAME, and the address evidence found it: each already had a carve run that could not merge with span 1, separated by rodata that is not padding — ov_SC02_005 by `0000F040 00000000` (8 bytes, twice the widest .align 3 pad the JTBL_PADS spec can emit), ov_SC02_011 by `FEBEF6AE 000002DC 0 0` (a TU's trailing const data). A gap detector keyed on zero words would have merged them and produced an unbuildable carve: the load-bearing test is "is this word a valid code address in this overlay's text range", not "is it zero". ov_SC01_084's divider is real data too (`0 FFFF0000 00080000 0 0`), while ITS span-1 gap word IS a zero .align 3 pad and merges. OVERLAY SPLITS ARE NEAR-FREE, AND THE REASON IS STRUCTURAL — the opposite of main. The Phase-26 §8b carried decl layer re-emits each region's externs locally, so only typedefs cross a cut: 1 name of 2,679 (ov_SC01_084, 0 typedefs) · 5 typedefs of 44 (ov_SC02_005) · 2 names of 3,254, 0 typedefs (ov_SC02_011) · 0 of 3,074 with zero compiler errors (ov_SC03_105). main's split moved 57 of 1,247. Every crossing typedef was MOVED to a <bin>_shared.h, never copied, and every list came from the compiler (R33). Carve probes (jtbl_carve --func, then reverted — carve state is added when a function banks, never speculatively): all four subsegs now accept a carve with no fail-loud, and jtbl_carve derived the §8e per-table pad specs the zero-word rule predicts. Unlocks 17 open switch functions: ov_SC01_084 func_80182A00 · ov_SC02_005 func_80185060, func_80185E80 · ov_SC02_011 func_80183178, func_80183630, func_8018418C, func_80188E3C · ov_SC03_105 func_801806F8, func_80180ABC, func_80180EC0, func_801813BC, func_801818E8, func_80181C84, func_8018624C, func_801867D0 (+2 more span-1 owners). CORRECTION, measured not assumed: config/wave_exclude.txt listed ov_SC01_084:func_80182328 as CARVE-BLOCKED and it never was — its table ABUTS the existing carve, so it always merged into one run. Proven by control on the PRISTINE unsplit config: --func func_80182A00 exits 1 "would host NON-CONTIGUOUS .rodata carves", --func func_80182328 succeeds. |
||
|
Drew T
|
aa302aa050 |
docs(phase-31): S73 FINAL checkpoint — consolidated, verified, fresh-session ready
Supersedes S73 CLOSE and its addendum. Every number re-verified against the repo: REAL 880/1,918, MAIN 56.5%, frontier 124 (main 36), main jtbl fns 25 -> 2. |