chore(phase-2): close archive loader investigation
This commit is contained in:
Christopher Williams
@@ -0,0 +1,111 @@
|
||||
# Phase 2 Investigation Record
|
||||
|
||||
**Scope:** evidence-separated investigation of local USA disc-file access, executable entry state, candidate loader/overlay paths, and candidate archive structure.
|
||||
**Status:** task verification complete; Phase 2 remains open pending developer milestone confirmation.
|
||||
|
||||
This record contains reproducible non-content methods and interpreted findings only. The disc image, extraction tree, manifests, Ghidra programs, runtime logs, packet traces, memcards, and generated probe reports remain ignored. A static Ghidra observation and a PCSX-Redux observation are never treated as interchangeable evidence.
|
||||
|
||||
## Preconditions and preserved baseline
|
||||
|
||||
- The local exact USA input has the deterministic ignored extraction and manifest established in Phase 1.
|
||||
- The original Ghidra program `/SCUS_946.40;1` remains preserved and unanalyzed.
|
||||
- All Phase 2 static work used the separate ignored `/P2-analysis-SCUS_946.40;1` working program.
|
||||
- PCSX-Redux runtime work used isolated CLI/interpreter cold boots, fresh ignored artifacts, a regular configured BIOS file, and a locally readable disc input. No runtime result was inferred from emulator console text alone.
|
||||
|
||||
## Tool verification
|
||||
|
||||
Phase 2 added `tools/sf3_probe`, a dependency-free read-only structural feature probe. It validates manifest syntax, path safety, candidate regular-file status, size, and streaming SHA-1 before retaining a bounded prefix for opaque feature measurement. Its successful output contains deterministic opaque IDs and no source paths or source bytes.
|
||||
|
||||
```bash
|
||||
PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s tools/tests -v
|
||||
git diff --check
|
||||
```
|
||||
|
||||
The final suite passed **28 synthetic-only tests**: 19 existing extractor tests and 9 probe tests. Probe coverage includes deterministic opaque output, extension selection, generic monotonic-word measurement, absent selection, malformed path, symbolic-link, size, digest, and CLI failure handling. No test reads the local disc or extracted game data.
|
||||
|
||||
## Evidence ledger
|
||||
|
||||
| ID | Provenance | Verified observation | Explicit limit |
|
||||
|---|---|---|---|
|
||||
| P2-T1 | Local USA manifest metadata | Two fresh aggregate-only censuses agreed; the existing manifest interface was sufficient. | Candidate classes are metadata only. |
|
||||
| P2-T2 | Separate Ghidra USA working program | Static candidate request, lookup/queue, and separate candidate-class transfer paths are reviewable at the recorded static addresses. | No physical transfer, completion, allocation, overlay, or file-format behavior is proved. |
|
||||
| P2-T3 | PCSX-Redux/GDB, two cold boots | Both runs halted at the executable entry breakpoint with identical PC/SP/GP captures. | Entry-state registers do not establish later loader or memory behavior. |
|
||||
| P2-T4 | PCSX-Redux/GDB, no-input cold-boot controls | The selected startup candidate was not reached; one predecessor invocation was observed; two conditioned controls repeated a no-match result. | This negative applies only to that input-free control window. |
|
||||
| P2-T5 | Local USA manifest-backed probe plus static revisit | Selected outer files passed full manifest SHA-1 verification in two identical reports; the separate static route is transfer setup only. | No inner container, table, compression, allocation, or overlay model is identified. |
|
||||
|
||||
## Static USA candidate paths
|
||||
|
||||
The following are Ghidra-generated candidate paths in the separate working program, not semantic function names or runtime facts.
|
||||
|
||||
| Static address / path | Bounded Ghidra basis | Status |
|
||||
|---|---|---|
|
||||
| `0x8002C22C` → `0x8002BC20` | Candidate-class request entry reaches request construction. | Static hypothesis; not observed in P2-T4's selected startup control. |
|
||||
| `0x8002BC20` → `0x800A6880` | Request construction reaches pathname lookup and queued-transfer setup. | Static hypothesis; its static destination is not a runtime allocation. |
|
||||
| `0x800A6880` → `0x800A6840` → `0x800A66B0` → `0x800A6FAC` | Lookup, length-check, registration, and conditional-dispatch candidates. | Static hypothesis; no CD/DMA/completion behavior claimed. |
|
||||
| `0x800B2B70` | Separate candidate-class routine constructs and registers a disc-file transfer through the candidate transfer subsystem. | Static transfer-route evidence only; no inner file parser/table semantics established. |
|
||||
|
||||
The ignored static records are `ghidra/P2-T2-static-survey.md` and `ghidra/P2-T5-archive-validation.md`. They contain interpreted findings without raw strings or disassembly.
|
||||
|
||||
## Runtime procedure and observations
|
||||
|
||||
The corrected isolated cold-boot configuration requires both `--gdb` and `--debugger`, plus CLI/interpreter/no-fastboot operation. Source inspection at the installed PCSX-Redux revision established that the remote server alone does not enable interpreter execution-breakpoint processing. A representative configuration is:
|
||||
|
||||
```bash
|
||||
pcsx-redux --cli --interpreter --debugger --gdb --gdb-port 3333 \
|
||||
--no-fastboot --no-webserver --bios "$REGULAR_BIOS_FILE" --loadiso "$LOCAL_DISC"
|
||||
```
|
||||
|
||||
Each accepted run required a packet-level `OK` for breakpoint insertion, a stopped target, register-only capture, and process/port cleanup. Two fresh runs halted at `0x800FB368` with identical PC `0x800FB368`, SP `0x801FFFF0`, and GP `0x00000000` observations. These are recorded as runtime entry-state facts in [memory-map.md](memory-map.md), not as a general runtime map.
|
||||
|
||||
For the selected no-input P2-T4 control:
|
||||
|
||||
- the direct static candidate at `0x8002C22C` did not stop during 75 seconds;
|
||||
- static predecessor `0x8002C3F8` stopped once with first argument register `0x00000001`; and
|
||||
- two independent 100-second controls conditioned on the branch value associated with the downstream candidate each showed six internal condition-evaluation stops and no matching stop.
|
||||
|
||||
No downstream breakpoint was armed after the conditioned no-match. These observations do not prove the path unreachable, do not name a state value, and do not establish a loader/overlay transition.
|
||||
|
||||
## Candidate-class structural validation
|
||||
|
||||
Only the census-/static-supported candidate class was measured. Two fresh runs of the probe used a 4096-byte prefix and streamed all selected candidates against the manifest SHA-1 before measurement:
|
||||
|
||||
```bash
|
||||
tools/sf3_probe prefix-features extracted/MANIFEST.tsv extracted \
|
||||
--extension "$CANDIDATE_EXTENSION" --prefix-size 4096 \
|
||||
> .run/p2-t5-probe.tsv
|
||||
```
|
||||
|
||||
The reports were byte-identical. They selected 46 manifest-backed outer files, had 46 distinct 64-byte prefix fingerprints, no class-wide shared leading byte or aligned word, and heterogeneous entropy, zero-byte, and generic monotonic-word feature counts. Some later fixed-prefix positions were shared, but their values were not recorded. These measurements neither identify a container boundary/table nor identify compression.
|
||||
|
||||
An initial ad hoc aggregate script used an invalid constant-position predicate. Its zero-shared-position output was explicitly superseded by the synthetic-tested probe and repeated positional-layout survey; no tracked conclusion uses the invalid value.
|
||||
|
||||
## Properly bounded conclusions
|
||||
|
||||
**Verified facts**
|
||||
|
||||
- Static USA evidence supports candidate request/transfer paths, including a separate candidate-class transfer route.
|
||||
- Two PCSX-Redux cold boots observed the same entry-register state.
|
||||
- The documented no-input startup control did not exercise the selected candidate branch/path.
|
||||
- The selected outer files passed manifest-integrity verification and have heterogeneous bounded-prefix measurements.
|
||||
|
||||
**Not established**
|
||||
|
||||
- Physical CD/DMA behavior, request completion, source-to-destination transfer, allocation range, or overlay execution.
|
||||
- Any inner archive/container boundary, table entry layout/meaning, compression format, or decompression behavior.
|
||||
- A runtime general memory map, runtime meaning of static destinations, or a semantic state associated with the observed predecessor argument.
|
||||
|
||||
## Firewall and phase gate
|
||||
|
||||
- Generated Ghidra records, probe reports, runtime traces, memcards, and `.run/` artifacts were checked as regular ignored files.
|
||||
- The Phase 1 Ghidra import remains present alongside the isolated P2 working program.
|
||||
- `.mcp.json` remains pre-existing, untracked, and untouched.
|
||||
- No disc-derived bytes, strings, file listings, disassembly, dumps, or proprietary SDK material were added to tracked files.
|
||||
|
||||
| Gate item | Status |
|
||||
|---|---|
|
||||
| P2-T1 through P2-T5 verification | Complete |
|
||||
| P2-T6 reproducibility record and source/confidence audit | Complete |
|
||||
| Clean synthetic-suite and documentation/firewall checks | Complete after final verification run |
|
||||
| Developer milestone confirmation | Required before Phase 2 closure |
|
||||
|
||||
Do not create a PhaseEnd record, update the digest, archive the current-phase record, or begin Phase 3 until the developer explicitly confirms the Phase 2 milestone.
|
||||
+14
-1
@@ -6,9 +6,22 @@
|
||||
- Ghidra: 12.1.2 at `/opt/ghidra`
|
||||
- PS1 loader: locally built `ghidra_psx_ldr`, installed in `~/.config/ghidra/ghidra_12.1.2_DEV/Extensions/ghidra_psx_ldr/`
|
||||
- Loader source: `/media/christopherwilliams/Data/Projects/decomp/ghidra_psx_ldr`
|
||||
- Runtime debugger/emulator: PCSX-Redux (`pcsx-redux-git`)
|
||||
- Runtime debugger/emulator: PCSX-Redux package `pcsx-redux-git r7541.b63d6bea-1` (installed package metadata)
|
||||
- GDB: package `gdb 17.2-1` / GNU gdb 17.2 (installed package metadata)
|
||||
- Python: package `python 3.14.7-1` / Python 3.14.7 (installed package metadata)
|
||||
- Java used to build the loader: `jdk21-openjdk` at `/usr/lib/jvm/java-21-openjdk`
|
||||
|
||||
## Project-local investigation tools
|
||||
|
||||
- `tools/sf3_extract`: tracked project Python source, standard library only; authored in this repository, with no external tool download. Its extraction/PS-X EXE commands are recorded in [PHASE1_REPRODUCIBILITY.md](PHASE1_REPRODUCIBILITY.md).
|
||||
- `tools/sf3_probe`: tracked project Python source, standard library only; authored in this repository, with no external tool download. Its Phase 2 structural-probe command and synthetic verification are recorded in [PHASE2_INVESTIGATION.md](PHASE2_INVESTIGATION.md).
|
||||
|
||||
Run the project synthetic suite with:
|
||||
|
||||
```bash
|
||||
PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s tools/tests -v
|
||||
```
|
||||
|
||||
## Loader rebuild
|
||||
|
||||
```bash
|
||||
|
||||
@@ -17,6 +17,12 @@ Filesystem/executable extraction may therefore read the BIN with a 2352-byte str
|
||||
|
||||
Do not infer later format details from other Syphon Filter games or other regions.
|
||||
|
||||
## Candidate-class structural validation (Phase 2 T5)
|
||||
|
||||
Phase 2 measured only a census-/static-supported candidate file class with the synthetic-tested, read-only `tools/sf3_probe`. Two ignored, byte-identical reports streamed every selected outer file against its extraction-manifest SHA-1 and retained only opaque fixed-prefix features. The class has no shared leading byte or aligned word in the measured 4096-byte prefix and has heterogeneous feature values.
|
||||
|
||||
These are bounded local-USA structural observations, not a container, table, compression, or overlay format identification. In particular, manifest-backed outer file boundaries do not establish inner boundaries, and generic monotonic-word measurements do not establish table semantics. See [PHASE2_INVESTIGATION.md](PHASE2_INVESTIGATION.md) for provenance and limits.
|
||||
|
||||
## PS-X EXE candidate (Phase 1 T3)
|
||||
|
||||
The exact ISO9660 identifier `SCUS_946.40;1` from the local USA image was extracted without renaming. `tools/sf3_extract psx-exe-info` validated its `PS-X EXE` magic, 0x800-byte header, declared text bounds, and streaming SHA-1 against ignored `extracted/MANIFEST.tsv`. The header-field offsets were cross-checked against the locally installed loader source, `ghidra_psx_ldr/src/main/java/psx/PsxExe.java`.
|
||||
|
||||
+14
-3
@@ -1,6 +1,17 @@
|
||||
# Memory Map
|
||||
|
||||
No USA runtime addresses are verified yet.
|
||||
Only the controlled PCSX-Redux register observations below are verified at runtime. They do not establish a general runtime memory map, loader completion, allocation, or overlay execution.
|
||||
|
||||
## PCSX-Redux runtime observations (Phase 2)
|
||||
|
||||
These are register observations from isolated PCSX-Redux/GDB controls using the local USA input. They are distinct from PS-X EXE header declarations and Ghidra loader-created blocks.
|
||||
|
||||
| Address / field | Purpose | Source build / region | Evidence | Verification status |
|
||||
|---|---|---|---|---|
|
||||
| `0x800FB368` | Program counter at the controlled executable entry breakpoint | Local USA input | Two fresh isolated cold boots; packet-level breakpoint acknowledgement and stopped-target register capture | Runtime-observed at entry only; no later execution/mapping inference |
|
||||
| `0x801FFFF0` | Stack-pointer register value at the controlled entry breakpoint | Local USA input | Same two PCSX-Redux/GDB captures | Runtime-observed register value at entry only; not a stack-region map |
|
||||
| `0x00000000` | Global-pointer register value at the controlled entry breakpoint | Local USA input | Same two PCSX-Redux/GDB captures | Runtime-observed register value at entry only; not a general GP claim |
|
||||
| `0x8002C3F8` | Program counter at one selected static-predecessor breakpoint | Local USA input | One isolated no-input cold-boot control; stopped-target capture also recorded first argument register `0x00000001` | Runtime execution observed for that invocation only; no semantic, loader, or memory-map conclusion |
|
||||
|
||||
## Static PS-X EXE header declarations (Phase 1 T3)
|
||||
|
||||
@@ -9,7 +20,7 @@ These entries are header-declared values from the local USA image's exact ISO966
|
||||
| Address / field | Purpose | Source build / region | Evidence | Verification status |
|
||||
|---|---|---|---|---|
|
||||
| `[0x80010000, 0x801DC000)` | Declared text range | Local USA image | PS-X EXE header offsets 0x18/0x1C, parsed by `tools/sf3_extract`; field layout cross-checked with local `ghidra_psx_ldr` source | Header-valid; runtime load status unverified |
|
||||
| `0x800FB368` | Declared entry PC | Local USA image | PS-X EXE header offset 0x10 | Header-valid and within declared text range; runtime execution unverified |
|
||||
| `0x800FB368` | Declared entry PC | Local USA image | PS-X EXE header offset 0x10 | Header-valid and within declared text range; Phase 2 separately observed runtime PC at entry, but the header does not establish runtime state |
|
||||
| `0x00000000` | Initial GP header field | Local USA image | PS-X EXE header offset 0x14 | Header-valid; not a runtime GP observation |
|
||||
| `0x801FFFF0` + `0x00000000` | SP base and SP offset header fields | Local USA image | PS-X EXE header offsets 0x30/0x34 | Header-valid; no effective/runtime SP inference made |
|
||||
|
||||
@@ -29,6 +40,6 @@ The following is the loader-created static program layout for the local USA impo
|
||||
| `0x801227AC`–`0x8014694F` | `.bss`, RW | Local USA image | `PSX Executables Loader` memory-block report | Loader-created static map; runtime state unverified |
|
||||
| `0x80146950`–`0x801DBFFF` | Second `.text` block, RX | Local USA image | `PSX Executables Loader` memory-block report | Loader-created static map; runtime state unverified |
|
||||
| `0x801DC000`–`0x801FFFFF` | Final `RAM` block, RX | Local USA image | `PSX Executables Loader` memory-block report | Loader-created static map; runtime state unverified |
|
||||
| `0x800FB368` | Entry point `start` | Local USA image | Ghidra entry-point report after PSX-loader import | Static loader entry; runtime execution unverified |
|
||||
| `0x800FB368` | Entry point `start` | Local USA image | Ghidra entry-point report after PSX-loader import | Static loader entry; Phase 2 separately observed runtime PC at entry, but the loader-created map remains unverified |
|
||||
|
||||
Record each future entry with: address range, purpose, source build/region, evidence source (Ghidra or runtime observation), and verification status. Do not transfer addresses from other Syphon Filter releases without explicit USA validation.
|
||||
|
||||
Reference in New Issue
Block a user