mirror of
https://github.com/Druthulu/BFM-decomp
synced 2026-09-26 05:32:17 -04:00
docs(phase-33.5): task 1 — unblock, code side: seven tool docstring/comment cites re-pointed off docs/tool-designs and docs/hindsight-study (decision-log S61 entry, cookbook §260, how-to ch.03 Phase 2b); CLAUDE.md R1–R83, the H1 fail-safe dated to Phase 33 C3, the effort default = the DIGEST §1 doctrine; log + checkpoint
This commit is contained in:
Drew T
@@ -8,7 +8,7 @@ Matching decompilation of **Brave Fencer Musashi** (PS1, SLUS-00726 USA). Goal:
|
||||
candidate). Read these files in this order — nothing else at session start:
|
||||
|
||||
1. Read `PROJECT_CONTEXT.md` in full (the permanent constitution: rules, protocols, roadmap).
|
||||
2. Read `phase-ends/DIGEST.md` in full — every phase's synopsis, every rule's full text (R1–R64), the corrections that
|
||||
2. Read `phase-ends/DIGEST.md` in full — every phase's synopsis, every rule's full text (R1–R83), the corrections that
|
||||
supersede parts of PROJECT_CONTEXT.md, and the doc map. It REPLACES reading every PhaseEnd (the PhaseEnds are still
|
||||
kept forever — read an older one on demand when a synopsis is not enough; **never** read `phase-ends/logs/` at
|
||||
session start, R19).
|
||||
@@ -58,7 +58,7 @@ These rules govern every session. They are not suggestions.
|
||||
|
||||
## Fail-safe hard rules (duplicated from PROJECT_CONTEXT.md in case the above is ever skipped)
|
||||
|
||||
- **Never commit ROM-derived content.** The game dump, `disks/`, `asm/`, `assets/`, `build/`, `expected/`, extracted `.CD` contents, the Ghidra project (`ghidra/`), the RAM dumps (`dumps/*.bin`), Sony's SDK (`tools/psyq/`) and the session archive stay out of git — in force again since the public flip (Phase 33); `tools/audit_public.py` and CI enforce it. Review `git status` before every commit.
|
||||
- **Never commit ROM-derived content.** The game dump, `disks/`, `asm/`, `assets/`, `build/`, `expected/`, extracted `.CD` contents, the Ghidra project (`ghidra/`), the RAM dumps (`dumps/*.bin`), Sony's SDK (`tools/psyq/`) and the session archive stay out of git — in force again since Phase 33 C3 (2026-09-06, the history rewrite); `tools/audit_public.py` and CI enforce it. Review `git status` before every commit.
|
||||
- **Never edit `PROJECT_CONTEXT.md`.** It is permanent and static. Corrections and state go in `phase-ends/`.
|
||||
- **Never `git clean -x` / `git clean -fdx` in this tree.** Since Phase 33 C3 (2026-09-06) the Ghidra project (`ghidra/`), the RAM dumps (`dumps/*.bin`), the PsyQ SDK (`tools/psyq/`), the session archive and the extension zips are IGNORED BUT PRESENT on disk — a `-x` clean deletes the reverse-engineering database. `make clean` is the only clean. (Regenerable in principle from `config/ghidra/` + the disc via `tools/ghidra_rebuild.sh`, but the dumps are not.)
|
||||
- **A "match" is byte-for-byte assembly equality AND the full-binary SHA1 check staying green.** Never report a functionally-equivalent function, a passing-looking build, or any unverified outcome as done.
|
||||
@@ -76,7 +76,7 @@ These rules govern every session. They are not suggestions.
|
||||
|
||||
## Reasoning & Model Protocol
|
||||
|
||||
Effort is set with `/effort` on the ladder `low / medium / high / xHigh / Max` (this is *depth* — how hard one agent reasons). **Max** = deepest single-agent reasoning, **session-only** (re-apply each session; xHigh is the highest level that persists). **Ultracode is NOT a deeper level** — it runs at xHigh and adds multi-agent orchestration (breadth, not depth); also session-only. **The project default working level is Max** (usage is not a constraint; the only downside is mild overthinking/latency on trivial tasks); drop to xHigh only for routine work when faster turns are wanted, and **never globally enable Ultracode for the deep tasks** (it would cap them at xHigh). Breadth is applied surgically — Claude proposes a Workflow fan-out for genuinely parallel tasks.
|
||||
Effort is set with `/effort` on the ladder `low / medium / high / xHigh / Max` (this is *depth* — how hard one agent reasons). **Max** = deepest single-agent reasoning, **session-only** (re-apply each session; xHigh is the highest level that persists). **Ultracode is NOT a deeper level** — it runs at xHigh and adds multi-agent orchestration (breadth, not depth); also session-only. **The doctrine (Drew, 2026-07-04; `docs/effort-map.md` governs): xHigh for most tasks, Max for the deep tasks (phase plans, PhaseEnds, architectural forks, non-obvious debugging — plan mode always Max), Ultracode for breadth**; Max and Ultracode are session-only, and **never globally enable Ultracode for the deep tasks** (it would cap them at xHigh). Breadth is applied surgically — Claude proposes a Workflow fan-out for genuinely parallel tasks.
|
||||
|
||||
**Effort-map check (rule — formalized as R7 at PhaseEnd_Phase2):** the full task→effort map, the Max-vs-Ultracode decision rule, verified harness mechanics, and per-phase guidance live in `docs/effort-map.md` (evolvable; it governs where this section's tier language differs). Before each Phase Start plan, state the recommended effort for *planning that phase* per that file and confirm the developer has it set; annotate every task in the phase plan with its effort level; restate the recommended effort whenever you present the NEXT task. The complexity tiers below map onto the ladder.
|
||||
|
||||
|
||||
+41
-18
@@ -39,7 +39,7 @@ in-tree links to `docs/wiki/<Page>.md`. 8. `.run/`: only what git tracks; no sca
|
||||
|
||||
## Tasks (strict order; one commit per task after this file is updated; harness task list = the same 16 rows, R28)
|
||||
- [x] **0** Gate 1 close-out: this file (ledgers + checkpoint), R74–R83 into DIGEST §3 (+ §0), `phase34-seed.md` amended, TaskCreate ×16 — Max
|
||||
- [ ] **1** Unblock, code side: tool docstrings re-pointed (`decl_from_use.py`, `integration_resolver.py`, `jtbl_carve.py`, `autopsy.py`, `residual_class.py`); CLAUDE.md lines 11 / 61 / the effort-default sentence — xHigh
|
||||
- [x] **1** Unblock, code side: tool docstrings re-pointed (`decl_from_use.py`, `integration_resolver.py`, `jtbl_carve.py`, `autopsy.py`, `residual_class.py`); CLAUDE.md lines 11 / 61 / the effort-default sentence — xHigh — see Log 2026-09-07 Task 1
|
||||
- [ ] **2** Unblock, doc side: banners (`concurrency-design`, `gen2-mips-matching-model`, `psyq-worklist`, `distill/*`, `hindsight-study` HISTORICAL + supersession table); SETUP §1a rewritten; the pin count re-derived (one dated number + its command, both gen3 docs) — xHigh
|
||||
- [ ] **3** Wiki — conventions + firewall: `Docs-and-scratch-conventions.md`, `The-ROM-firewall.md` (the gitignore template block); sidebar rows; Repository-layout/Contributing/ch.11 links; forward links in `doc_links_pending.txt` — Max
|
||||
- [ ] **4** Wiki — reference + direction: `Reference-index.md` (list-driven), `Where-the-project-goes-next.md`, Contributing "AI use — conduct", Home rewritten, sidebar sections — xHigh — **then P6 rules check**
|
||||
@@ -212,31 +212,54 @@ check whose shapes come from the offenders; disk-independent citation checks; th
|
||||
document for PA's own interview; dry-run write guardrails; a cut line). Spot-checked the claims the plan leans on (172 tracked-but-
|
||||
ignored; the two disassembly files; §265–§269 landed; the tools-health chain). R74–R83 written into DIGEST §3 from the Phase-33
|
||||
candidates' text; DIGEST §0 notes the sub-phase; `phase34-seed.md` amended. Harness task list built (16 rows). Effort for task 1 is
|
||||
xHigh — the transition is prompted (R27).
|
||||
xHigh — the transition is prompted (R27). Commit `39d524991`.
|
||||
|
||||
## 🛑 SESSION CHECKPOINT — Task 0 ✓ (gate 1 closed, R74–R83 ratified); NEXT = task 1 (unblock, code side; xHigh)
|
||||
### 2026-09-07 — Task 1 — Unblock, code side (xHigh; Drew set `/effort xhigh`)
|
||||
Seven docstring/comment cites re-pointed to durable records: `decl_from_use.py` + `integration_resolver.py` → the decision log's
|
||||
2026-08-25 (P31 S61) "the wall is an integration wall" entry (the S60 frontier analysis's 178/292 = 61% and ~571 finished-draft
|
||||
measurements live there); `jtbl_carve.py` ×3 → cookbook §260 (the leading-island split; the review's correction #7); `autopsy.py` +
|
||||
`residual_class.py` → `docs/how-to-ai-decomp/03-bootstrap-order.md` Phase 2b (the permuter track — task 5 adds the "mine the permuter's
|
||||
failures" paragraph there; "originally hindsight-study §7, archived" kept as provenance). `CLAUDE.md`: R-range R1–R83; the H1 fail-safe
|
||||
dates the re-tightening to Phase 33 C3 (2026-09-06, the history rewrite) — the flip has not happened; the Reasoning section's "default
|
||||
working level is Max" replaced by the DIGEST §1 doctrine (xHigh for most tasks, Max for deep tasks, Ultracode for breadth; plan mode
|
||||
always Max). Verify: `py_compile` on the five tools OK; `git grep -E 'tool-designs|hindsight-study' tools/ Makefile` = 0; `doc_links`
|
||||
OK. **Deviation from the plan's verify column:** `make tools-health` was NOT run for docstring-only edits (proportionality — nothing it
|
||||
measures changed); the first tools-health of the phase runs at task 7, where the Makefile and the checkers change.
|
||||
|
||||
## 🛑 SESSION CHECKPOINT — Tasks 0–1 ✓; NEXT = task 2 (unblock, doc side: banners, SETUP §1a, the pin count; xHigh)
|
||||
|
||||
### 0. How to use this block
|
||||
You are a FRESH SESSION that has read `PROJECT_CONTEXT.md`, `phase-ends/DIGEST.md`, `PhaseEnd_Phase31/32/33.md` and this file, and
|
||||
nothing else (R64). Replay this block verbatim, state phase / done / NEXT / effort, list the rules from the digest (R1–R83), then
|
||||
WAIT for Drew. Rebuild the harness task list (16 rows, R28) marking task 0 completed and task 1 in progress.
|
||||
WAIT for Drew. Rebuild the harness task list (16 rows, R28) marking tasks 0–1 completed and task 2 in progress.
|
||||
|
||||
### 1. Where we are
|
||||
**Phase 33.5** (sub-phase; v1.32.0 → v1.32.1), gate 1 approved 2026-09-07 by Drew in plan mode at Max. Baseline HEAD `80d45b29b`.
|
||||
Task 0 is committed (this file, DIGEST §0/§3, `docs/phase34-seed.md`). Nothing under `src/`, `config/`, `Makefile` or `tools/` has
|
||||
changed yet; the fleet is 218/218 at the Phase-33 close.
|
||||
**Phase 33.5** (sub-phase; v1.32.0 → v1.32.1), gate 1 approved 2026-09-07 by Drew in plan mode at Max; Drew set `/effort xhigh` after
|
||||
task 0 (the Max tasks are 3, 10, 12, 15 — prompt at each, R27). Baseline HEAD `80d45b29b`; task 0 = `39d524991`; task 1 = the commit
|
||||
after it (docstrings + CLAUDE.md only). Nothing under `src/`, `config/` or the `Makefile` has changed; the fleet is 218/218 at the
|
||||
Phase-33 close. `docs/doc_links_pending.txt` is empty (comments only).
|
||||
|
||||
### 2. What NEXT does (task 1, xHigh) — exact steps
|
||||
1. `git grep -n -F 'tool-designs' tools/` and `git grep -n -F 'hindsight-study' tools/` → the docstring cites in `tools/decl_from_use.py`,
|
||||
`tools/integration_resolver.py`, `tools/jtbl_carve.py` (→ the cookbook § that carries each finding: §8/§8a/§8b/§8e/§61b–d/§62 for the
|
||||
jtbl lane, §236 for main-lane-fix, the S60 "571 already-drafted" finding → PhaseEnd_Phase31 / cookbook) and in `tools/autopsy.py`,
|
||||
`tools/residual_class.py` (→ `docs/how-to-ai-decomp/03-bootstrap-order.md`'s "mine the permuter's failures" anchor, which task 5
|
||||
creates — cite the chapter file, the anchor lands in task 5).
|
||||
2. `CLAUDE.md`: line 11 "R1–R64" → "R1–R83"; line 61 "since the public flip (Phase 33)" → "since Phase 33 C3 (2026-09-06)"; the
|
||||
Reasoning section's "The project default working level is Max" sentence → the DIGEST §1 doctrine (xHigh for most tasks, Max for
|
||||
deep tasks, Ultracode for breadth, plan mode always Max; Max/Ultracode session-only). Keep everything else verbatim (H5).
|
||||
3. `make tools-health` in the FOREGROUND (~15 min; never background it) → OK; then this file's log + checkpoint; commit
|
||||
`docs(phase-33.5): task 1 — …` by explicit path. Then prompt: task 2 is xHigh (no change).
|
||||
### 2. What NEXT does (task 2, xHigh) — exact steps
|
||||
1. Banners/status lines (H5: add, never drop): `docs/concurrency-design.md` — replace "**Not yet implemented.**" with "implemented in
|
||||
`tools/shared_lock.py` + the gate snapshot layer (P31); cited by `gate_stage.py`, `bulk_harvest.py`, `dedup_propagate.py`,
|
||||
`ox_campaign.py`, `verify_worktree.py`"; `docs/gen2-mips-matching-model.md` — "**Status:** design / not started" → built (P22–P23;
|
||||
`train_lora.py`/`export_pairs.py`/`format_finetune.py`; distilled by `docs/matching-drafter-pipeline.md`); `docs/psyq-worklist.md` — a
|
||||
top banner "COMPLETE at P31 S79 #5 — the residue (SSGM.o 8 ins inside matched C; the sound region's two cross-object-common walls) is
|
||||
named at the tail"; `docs/distill/atbhbkbl.md` + `axbm.md` — replace the "PROPOSED ONLY" banner with "LANDED as cookbook §265, §266,
|
||||
§267 (this file) / §268, §269 (axbm) on P31 S59b/c; kept as the A/B evidence ledger those sections cite"; `docs/hindsight-study.md` —
|
||||
a top banner "HISTORICAL — written 2026-07-20 mid-P29 at ~78%; superseded part by part" + the supersession table (§0–§2 → how-to
|
||||
ch.03 + `generic-decomp-package.md`; §3 → `portable-decomp-workflow.md`/the chapters; §4 → `gen3-handoff.md` §5 + wiki
|
||||
Tools-from-this-project; §5 → `retrospective.md` §5; §6 → fulfilled by `retrospective.md` + the how-to + `tools/mine_hindsight.py`;
|
||||
§7 → `tools/autopsy.py` — live pointer stays in ch.03).
|
||||
2. `docs/SETUP.md` §1a (line ~68): rewrite — `.run/` is gitignored by the contents-exclude rule `/.run/*` with 22 dated `!` allowlist
|
||||
blocks (R20: "commit what a rerun CANNOT reproduce"); what IS tracked (P32/P33 evidence, the giants recon, the R20 ledgers) and what
|
||||
never is (the 29 GB of regenerable scratch); point at the wiki conventions page (task 3 — register the forward link in
|
||||
`docs/doc_links_pending.txt` if you link it; or just name it in prose).
|
||||
3. The pin count: run ONE grep over `src/` (e.g. `grep -rhoE 'register [^;]*__asm__\("\$[a-z0-9]+"\)' src | wc -l` — check what each doc's
|
||||
number was derived from first: `gen3-handoff.md` §3 shows its six commands) and write the same dated number + the exact command into
|
||||
`docs/gen3-standards.md` §4 and `docs/gen3-handoff.md` §3 (R75: a number in prose is a dated snapshot with its command).
|
||||
4. Verify: `.venv/bin/python tools/doc_links.py` OK; the pin-count command reproduces the number; then this file's log + checkpoint;
|
||||
commit by explicit path (`docs(phase-33.5): task 2 — …`). Task 3 is **Max** — prompt Drew and WAIT (R27).
|
||||
|
||||
### 3. Standing facts for every task of this phase
|
||||
- One commit per task, after this file's log line (R8/R42 form); commit by explicit path; no trailers (R5); Drew pushes (R6).
|
||||
|
||||
+2
-1
@@ -1,7 +1,8 @@
|
||||
#!/usr/bin/env python3
|
||||
"""autopsy.py — the permuter-failure autopsy: mine the search-closer's FAILURES (Phase-29 Task-13).
|
||||
|
||||
docs/hindsight-study.md §7. Our flywheel (R16) has always evolved the offline tooling REACTIVELY
|
||||
The permuter track of docs/how-to-ai-decomp/03-bootstrap-order.md (Phase 2b; originally §7 of the July-2026
|
||||
hindsight study, since archived). Our flywheel (R16) has always evolved the offline tooling REACTIVELY
|
||||
FROM SUCCESSES: a human or an LLM cracks a function, we distil the idiom into the cookbook and the
|
||||
permuter weights. The permuter's FAILURES — every near-miss it plateaued on — are a larger and
|
||||
completely unused signal. This tool turns that log into a corpus, classifies it deterministically,
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
#!/usr/bin/env python3
|
||||
"""decl_from_use.py — infer a draft's MISSING data extern from the target .s + the draft's own use.
|
||||
|
||||
WHY THIS EXISTS (P31 S61; docs/tool-designs/frontier-analysis-s60.md §5 step 4). Two refusal sites
|
||||
WHY THIS EXISTS (P31 S61; the S60 frontier analysis — recorded in docs/decision-log.md, the 2026-08-25
|
||||
"the wall is an integration wall" entry). Two refusal sites
|
||||
throw away drafts whose only defect is a data symbol the destination TU never declares:
|
||||
* aprop_autodraft.py (~:522) refuses `no seed decl for D_x` — the member body is correct, but no
|
||||
seed/destination/fleet TU spells the extern, so the A-prop lane never emits the draft;
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
#!/usr/bin/env python3
|
||||
"""integration_resolver.py — the zero-token lane over drafts that are ALREADY CORRECT (P31 S61).
|
||||
|
||||
WHY THIS EXISTS. The S60 frontier analysis (docs/tool-designs/frontier-analysis-s60.md §4) measured
|
||||
WHY THIS EXISTS. The S60 frontier analysis (recorded in docs/decision-log.md, the 2026-08-25 "the wall is
|
||||
an integration wall" entry) measured
|
||||
that the gen6+ "wall" is an INTEGRATION wall, not a codegen wall: of the 292 functions the gate had
|
||||
refused six or more times, 178 (61%) had already produced a closeness-0 draft — match_one byte-
|
||||
equality at the object level, whole-binary gate rejection — and across the whole open pool ~571
|
||||
|
||||
+3
-3
@@ -743,7 +743,7 @@ def island_split(ov, func):
|
||||
Writes ONLY config/splat.<ov>.yaml (one inserted line). No overlays.mk var (a single-table
|
||||
piece needs none — §8e), no extract (the caller extracts). The island piece at 0x0 is left
|
||||
untouched: its extent shrinks automatically (end = next piece's offset) — the review-proven
|
||||
correction to the `_pre` design (docs/tool-designs/jtbl-island-split-review.md #7)."""
|
||||
correction to the `_pre` design (cookbook §260, the review's correction #7)."""
|
||||
kind, detail = island_probe(ov, func)
|
||||
if kind == "covered":
|
||||
print(f"jtbl_carve --island-split {ov} {func}: already split — no-op ({detail})")
|
||||
@@ -891,7 +891,7 @@ def build_carve(ov, funcs):
|
||||
# rewrites [tail_start, region_end), so the piece line would land out of address order
|
||||
# and splat would mis-slice the module. That is the island-split lane's job — ONE
|
||||
# inserted `.rodata` line at the table's own offset plus jr_isolate_all.py --only
|
||||
# (docs/tool-designs/jtbl-island-split-review.md) — so name it and refuse (R43).
|
||||
# (cookbook §260) — so name it and refuse (R43).
|
||||
isl_off = int(jh, 16) - base
|
||||
if isl_off < tail_start:
|
||||
sys.exit(
|
||||
@@ -900,7 +900,7 @@ def build_carve(ov, funcs):
|
||||
f"island, which a tail carve cannot reach. That is the island split (§260): "
|
||||
f"jr_isolate_all.py --only {f}, then `jtbl_carve.py {ov} --island-split "
|
||||
f"--func {f}` — automated at gate time by harvest_verify._jtbl_prep_one "
|
||||
f"(docs/tool-designs/jtbl-island-split-review.md). "
|
||||
f"(cookbook §260). "
|
||||
f"FOR AN md_* MODULE, §303 SUPERSEDES THIS: no isolation, no carve state — splice and "
|
||||
f"`make build`; the Makefile's jtbl_rodata_pads --derive stage reproduces the island pads at "
|
||||
f"build time (P32 S83: md_SC03_054/func_801EF6D8, 7 tables, 0,0t1,0t1,0t1,0t1,0t1,0).")
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
#!/usr/bin/env python3
|
||||
"""residual_class.py — the DETERMINISTIC residual→class classifier (Phase-29 Task-13A).
|
||||
|
||||
The permuter-failure autopsy (docs/hindsight-study.md §7) needs to route every plateau to one of
|
||||
The permuter-failure autopsy (the permuter track of docs/how-to-ai-decomp/03-bootstrap-order.md, Phase 2b;
|
||||
originally §7 of the July-2026 hindsight study, since archived) needs to route every plateau to one of
|
||||
missing-transform | mis-classified | seed-structural | genuine-wall
|
||||
and today that routing is done by an LLM reading a scalar (`closeness`) plus a free-text
|
||||
`where_stuck` label a wave agent typed months ago. Both are weak: 91% of the open backlog has NO
|
||||
|
||||
Reference in New Issue
Block a user