mirror of
https://github.com/RetroPortingToolKit/psxrecomp
synced 2026-10-26 10:19:47 -04:00
fix/execution-identity-eol
8 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Alexandros Mandravillis
|
ff4be21c90 |
bios: add the SCPH-5500 and SCPH-5501 profiles
The v3.0 Japanese (SCPH-5500, 09/09/96) and US (SCPH-5501, 11/18/96) retail BIOS images join SCPH-1001, SCPH-101 and SCPH-5552, so every v3.0 region (J, A, E) has a profile. Both images match SCPH-1001 v2.2 byte for byte in the kernel, ROM [0x80, 0x18000). SCPH-5501's reset vector is identical too; SCPH-5500's differs in 24 bytes. The shell differs in both. So each profile is SCPH1001.toml with its own identity (name, id, rom, sha256 pin, stem): the copy windows and install slots are kernel properties and carry over unchanged. The shell runs through the dirty-RAM interpreter, as for SCPH5552. Each image gets its own seed list, derived by the same rule as phase2_ghidra_seeds_SCPH5552.json: the SCPH-1001 corpus filtered to seeds whose 64-byte ROM window is identical in both images (534 kept, 569 dropped), plus the two architectural vectors and 6 A0/B0/C0 kernel-table targets. The files carry only addresses, labels and rationales, plus the SHA-256 of both ROMs. - psx_bios_known_images.h: CRC-32 FF3EEB8C (SCPH-5500) and 8D8CB7E4 (SCPH-5501), both 512 KiB. A build that does not link these backends builds one from the player's own dump as a loadable module, like SCPH-5552, and the netplay lobby can name the image. - main_cli.cpp: map SCPH5500 / SCPH5501 dumps to their profiles. - build_cli.py: package both profiles with the CLI framework. - docs/BIOS_SELECTION.md: list them among the module-built images. |
||
|
Alexandros Mandravillis
|
a1885e4f4a |
bios: replace the stale SCPH5552 profile
bios/SCPH5552.toml on master predates the address-model schema. It describes the image with bios_vectors and bios_aliases and has no address_model copy windows, no install_slots and no runtime_exports, while bios/SCPH1001.toml uses all three and the recompiler reads them. It also carries no [program.image] pin, so the declared-identity gate cannot refuse a wrong revision. Replace it with a profile in the current schema. Its structure now matches SCPH1001.toml key for key: - [program.image] pins SHA-256 1faaa18f... (v3.0 01/06/97, CRC32 D786F0B9). - address_model and runtime_exports carry over from SCPH1001 unchanged. - The five install slots from SCPH1001 carry over with len and resume. - rom keeps master's region-qualified name; bios_rom_alias.h resolves either convention by model token. The carry-over rests on a direct comparison of the two dumps, each matching its declared identity: ROM 0x00000-0x17FFF (boot, Kernel Part 1 and 2) is byte-identical, and every install slot maps through Kernel Part 2 into identical bytes. The shell at 0x18000-0x42FFF differs, which is why the seed corpus stays kernel-only; every seed in it lies below ROM 0x18000. The install slots were measured on a live SCPH-1001 boot and are not re-measured on SCPH-5552. The profile records that evidence level beside them. This changes one file and adds no BIOS support. Savestate isolation and the netplay handshake for further retail images are separate work. |
||
|
Kevin
|
3cbe6c1cde |
kernel-bless: publish install-slot ranges so patched BIOS bodies run native
The BIOS exception handler ran interpreted for the life of every process,
1.22 billion instructions and 44.6% of all interpreted work in Breath of
Fire III, even though it is compiled and even though the stub that
unblessed it sat at exactly the address bios/SCPH1001.toml already
declared as an install slot. Two mechanisms existed and did not compose:
the emitter hooked the declared slot PC, while the runtime's bless
verifier memcmp'd the whole body including that slot, so the body was
MISMATCH forever and native code never reached the hook.
Three parts, none of which pays off alone:
1. Publish the declared ranges to the runtime. The emitter emits a
PsxKernelPatchRange table next to the kernel body table and routes it
through the backend descriptor; the bless verifier compares a body in
segments that skip them. CLEAN now means "every byte outside every
declared range still matches ROM", which is the claim the native code
actually depends on.
2. Generalise the slot model to what the patches are. The old model
assumed a 4-word lui/addiu/jalr/nop stub written over ROM zeros and
fired on `word != 0`. Measured against both images, only one of five
observed shapes fits: patchers also plant a `jr` over real ROM
instructions, rewrite an 11-word prologue and insert an `mfc0 Cause`,
and NOP out a driver routine. A slot is now a range with a length and
an explicit `resume` (jalr / fallthrough / none), detected by
comparing every word against its ROM-baked value.
3. Hand back from the interpreter at the range end. Kernel page 0 is
permanently dirty, because the handler saves registers there on every
exception, so straight-line interpretation would run the whole
function after the patch. The emitter registers the resume PC as a
continuation key and dirty_ram_interp surfaces there.
A PC inside a declared range must never be a native dispatch key: the
compiled bytes there are not what executes. Seven pre-existing
jal-return continuations violated that and wedged the boot at frame 0 in
an infinite dispatch loop (OpenBIOS RAM 0x357C). The emitter now drops
continuation keys inside a range so dispatch misses through to the
interpreter, and refuses to emit if a range covers a function entry,
which would silently remove a callable function instead.
Both profiles declare their measured ranges, and SCPH1001.toml finally
pins [program.image] sha256 to v2.2 (CRC32 37157331) so a v2.0 dump
refuses to generate rather than producing phantom discovery gaps.
Measured on Breath of Fire III (SLPS-00990, Psy-Q SDK 3.70), headless,
matched frames, PSX_KERNEL_PATCH_RANGES=0 for the OFF side:
OpenBIOS SCPH-1001 v2.2
bless mismatch 21 -> 0 74 -> 0
kernel bodies -87.7%/frame -97.8%/frame
all interp insns -42.6%/frame -94.2%/frame
The residual kernel-body work is exactly declared-words x entries on
every range, so nothing leaks past the hook. The A0/B0/C0 vector
trampolines are unchanged (+0.2%, noise): the profile excludes them by
design and they are a separate job. Card read traces are byte-identical
across the A/B on both images, all 32 entries, which exercises the two
ranges that replace the kernel's card handler.
Rule 18 holds throughout: the guest's patched instructions still execute
as written on the dirty-RAM interpreter. Only the code around them
changes backend. Nothing is synthesised.
Tests: runtime/tests/test_kernel_patch_ranges.c (new, the verifier's
accept/reject behaviour and the hand-back predicate) and the install-slot
range invariants in recompiler/tests/bios_address_model_test.cpp.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Matthew Stanley
|
f55c731c5f |
chore: remove dead scratch tooling and stale root files
Removes 204 tools/ scripts (203 .py/.sh plus tools/_missing_funcs.json) that
no tracked file references. Verified mechanically, not by eye: for each of the
204 stems, a word-boundary grep across every tracked text file in the repo,
excluding files that are themselves being deleted, returns zero hits. So
nothing kept imports, execs, or documents any of them. Most are the
underscore-prefixed one-shot probes the .gitignore convention already treats
as scratch (tools/_*.py is ignored; these predate that rule and were force-
added), plus a long tail of session-specific dump/decode/poll scripts with
hardcoded local paths and addresses.
Also removed at the root:
* card2.mcd.bak.before_beetle_copy -- a stray backup of a memcard.
* .mcp.json -- local MCP server wiring, not repo configuration.
* SCPH1001.toml -- a root duplicate of bios/SCPH1001.toml, still on the
pre-refactor [[recompiler.bios_vectors]] schema while the bios/ copy has
moved to [recompiler.address_model]. Every reference in the tree already
names bios/SCPH1001.toml, so the root copy was a stale trap: editing it
would have had no effect on any build.
* docs/CYCLE_TIMING_ARCH.md -- superseded by
docs/internal/FAITHFUL_TIMING_PLAN.md, CYCLE_MODEL_BEETLE.md and
PRECISE_IRQ_SLICE.md; nothing linked to it.
Memcard fixtures move out of the repo root into
runtime/tests/fixtures/memcards/ with a README recording what each of the
four images actually contains (directory entries read out of the images, not
guessed). Nothing in the build read them from a fixed path -- the four
hardcoded consumers (tools/check_card.py, inspect_card.py, peek_cards.py,
verify_card_buffer.py) are all in the deletion above.
.gitignore is corrected to match: runtime memcards a session creates stay
ignored (*.mcr anywhere, /*.mcd at the root, which is where the runtime drops
card1.mcd/card2.mcd when no path is configured), while the tracked fixtures
are re-included. The old !dummy.*.mcr negation existed only to keep the two
root oracle cards tracked and is replaced by
!runtime/tests/fixtures/memcards/*. Verified with git check-ignore --no-index
on each moved file, on the pre-existing runtime/tests/test_card_synth.mcd and
runtime/tests/test_dir/card1.mcd fixtures (all not-ignored), and on
hypothetical root card1.mcd / card2.mcd / saves/card1.mcr (all ignored).
beetle_libretro.cpp's warn-only memcard sanity check keeps its behaviour; its
WARNING now names where populated fixtures can be copied from. The cards are
opened relative to the process's working directory because the libretro
GET_SAVE_DIRECTORY environment callback answers "." (same file, line 419), so
the message says to copy them into the CWD rather than implying a repo path.
BIOS profiles SCPH101.toml and SCPH5552.toml move from the root into bios/,
joining OpenBIOS.toml and SCPH1001.toml so that bios/*.toml is the whole set.
Behaviour-preserving: config paths inside a profile resolve against
find_project_root() (config_loader.cpp:821), which walks up from the profile's
directory for .gitignore/.git/CMakeLists.txt -- bios/ has none of those, so it
lands on the same repo root as before, and bios/SCPH1001.toml already relied
on exactly that. tools/build_cli.py's staging map is updated to the new
sources; it already copied them to framework/bios/, so the packaged layout is
unchanged. recompiler/src/main_cli.cpp's bios/-first lookup keeps its root
fallback (harmless, and it is what makes older layouts still resolve).
Fixes a real latent break while in bios/SCPH5552.toml: seeds pointed at
"seeds/phase2_ghidra_seeds_SCPH5552.json", but there is no seeds/ directory at
the repo root -- the file lives at
recompiler/seeds/phase2_ghidra_seeds_SCPH5552.json, which is what SCPH101.toml
already named. Porting these two profiles to the address_model schema is
deliberately NOT done here; it is a separate tracked follow-up.
Test files that no build system knew about:
* tools/test_coverage_vault.py PASSES (8/8). Moved to
recompiler/tests/test_coverage_vault.py and registered as the ctest
coverage_vault, which puts it inside runtime/check_test_registration.cmake's
glob so it can never silently rot again. Its ROOT is adjusted from
parents[1] to parents[2] for the new depth, matching every other python
test in that directory. Re-run from the new location and from an unrelated
CWD: 8/8. The registration guard was run standalone before and after, and
with the add_test() deliberately broken as a negative control -- it does
name recompiler/tests/test_coverage_vault.py as an orphan when
unregistered, so the registration is load-bearing rather than decorative.
* tools/test_compile_overlays_additive.py FAILS, 10 of 14 tests erroring
against an API that has moved on, so it is removed rather than wired up.
It calls four module attributes that no longer exist in
tools/compile_overlays.py -- load_additive_captures,
validated_cached_bundle_ids, cached_bundle_pairs, compile_and_publish_dll
(zero occurrences each) -- and two signatures have changed under it:
parse_overlay_func_ids() no longer takes require_stored_crc, and
load_region_coverage() now takes 2-3 positional arguments where the test
passes 5. Reviving it means rewriting it against the current additive
capture API, which is a change to overlay-cache test coverage and needs its
own commit and its own evidence, not a rename.
Verification: py_compile over all 113 remaining tracked tools/*.py plus
psxrecomp_cli.py passes; bash -n over every remaining tracked tools/*.sh
passes; importing sdk_progress, pad_delivery and cursor_writers from tools/
succeeds.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Alex Vanderveen
|
3da61df355 |
Revert "ok"
This reverts commit
|
||
|
Alex Vanderveen
|
b836f2198c | ok | ||
|
Matthew Stanley
|
5ec343ab4f |
fix(bios): make the boot-skip flag mean the same thing on every BIOS
`bios_hle` (default on) is the player-facing "skip the BIOS and go
straight to the game" flag, and it did nothing under the bundled
OpenBIOS: choosing OpenBIOS sat the player in the boot animation while
the identical build on retail SCPH-1001 went straight to the game.
Root cause was ordering, not policy. The tier has two axes with
different per-image requirements:
* boot-skip needs only `shell_entry_phys`, works under pure LLE
(nothing is synthesized — the shell call just returns
immediately), so any image exporting it can skip
* kernel-call needs `deliver_event_ret`, the kernel's own
DeliverEvent `$ra`, or a delivered callback returns to
the wrong PC
main.cpp correctly refused the call axis on an image without
`deliver_event_ret` by assigning `bios_hle = false` — and then derived
`boot_skip` from that already-mutated flag. OpenBIOS omits
`deliver_event_ret` (B0 semantics unvalidated) but DOES export
`shell_entry_phys`, so an available skip was cancelled by an unrelated
refusal, and one flag meant two things depending on a BIOS detail no
player can see.
Both axes are now decided by one pure, dependency-free function,
`psx_bios_hle_plan()` (runtime/src/bios_hle_plan.c): the boot decision
reads the REQUESTED `bios_hle`, each axis is gated on only the anchor it
needs, and refusals are reported at startup instead of downgrading
silently. `psx_bios_hle_configure()` clamps `boot_skip` to
`shell_entry_phys != 0` as a backstop so the banner and `hle_dump`
cannot overstate what can fire. The main.cpp local is renamed
`bios_hle_requested` and is no longer written back from the plan — the
`session_reboot` path re-enters that block, so folding a refusal into
the request would have made a netplay rematch differ from first launch.
Also in this pass:
* A BIOS profile's `[runtime]` block was parsed into
`BiosConfig::runtime` and read by NOTHING, and bios/OpenBIOS.toml
carried an inert `bios_hle = false` there that read as the reason HLE
was off on OpenBIOS. It was not. `load_bios_config` now REJECTS
`[runtime]` in a BIOS profile: a profile states facts about an image,
never preferences.
* `psx_icache_fastpath_test` had been unbuildable ("redefinition of
`psx_advance_cycles`" — the test's counting stub vs the header's
`static inline`), which blocked every runtime test registered after
it. It now builds with `PSX_OVERLAY_DLL_BUILD=1`, the seam
psx_cycles.h already provides for a TU supplying its own advance.
Tests: new `bios_hle_plan_test` pins the matrix, including a 64-case
sweep asserting the boot decision is independent of
`deliver_event_ret` and the call decision independent of
`shell_entry_phys`. Runtime ctest 15/15, recompiler ctest 30/30.
Validated on Ape Escape (SCUS-94423) built against this framework with
both backends linked, headless, all four cases:
OpenBIOS default bios_boot=skip to game; route=2 fire at vec 0x30000
ra=0xBFC06EA8 (the continuation OpenBIOS.toml
documents); in-game on screen
retail default bios_boot=skip to game; route=2 at vec 0x30000
ra=0xBFC0702C; in-game on screen; differs only in
bios_backend=HLE (LLE fallback)
PSX_BIOS_HLE=0 bios_boot=real intro; ring empty (no hook)
KEEP_INTRO=1 bios_boot=real intro; ring empty
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Henrique Guedes
|
6749b826d4 |
feat: OpenBIOS support — profile-driven BIOS recompilation
Adds PCSX-Redux's MIT-licensed OpenBIOS as a second BIOS the framework can
statically recompile, next to retail SCPH1001. Because OpenBIOS is
redistributable, a game verified against it ships its BIOS with the build —
players supply only a disc, and the launcher BIOS picker disappears.
The mechanism is general, not OpenBIOS-specific: every BIOS image gets a
bios/<STEM>.toml profile that is the single source of truth for its identity
and layout.
Recompiler
- bios/<STEM>.toml profiles: [program.image] (sha256 pin, redistributable
flag), [recompiler.address_model] (boot-time ROM->RAM copy windows,
kernel-bless window), install_slots, and [recompiler.runtime_exports]
HLE anchors (0 = feature structurally unavailable on that BIOS).
- BiosAddressModel (new): validated model built from the profile, replacing
~9 hardcoded SCPH1001 window constants that had drifted apart across the
emitter / discovery / game codegen. The emitted C normalize() derives
from the same table. from_config rejects overlapping/misaligned windows
and orderings where sequential folding != first-match. A CTest-registered
equivalence test sweeps every RAM/ROM address against verbatim copies of
the old helpers.
- Orphaned delay slots (the OpenBIOS-critical emitter fixes): jr/jalr
slots are inlined before the jump resolves — previously their side
effect was silently LOST (e.g. a case-return fragment's addiu $sp pop,
which unwound a shifted frame and walked the exception handler's IntRP
cursor into a wild dispatch). Orphaned branches resume at ds+4 on
not-taken instead of re-executing the slot (a read-modify-write slot
doubling twice broke OpenBIOS readPad). Jump targets latch pre-slot.
- MIPS-I load-delay modeling for dependent load pairs: the successor sees
the pre-load value (OpenBIOS's cardfasttrack.s relies on it on purpose —
"gotta break those bad emulators"). Unmodeled shapes warn at build time.
- Discovery hardening: branch-target dispatchability closure to fixpoint;
jump tables placed at jr+8 no longer walked as code; cross-function
continuations routed by an instruction->owner map.
- Signed MULT support (unreachable from the retail seed set; modern-GCC
OpenBIOS emits it freely), latency-modeled like MULTU.
- Seeds from ELF symbol tables + data-section code-pointer harvest
(tools/gen_bios_seeds_from_elf.py) — a source-built BIOS needs no
disassembly pass. 646 OpenBIOS seeds, incl. 5 empirical dispatch-miss
seeds.
- Stemmed outputs (<out_stem>_full.c / _dispatch.c) so two BIOSes coexist
in one out_dir. tools/regen_bios.sh is profile-driven (--config).
Runtime
- psx_bios_image: the generated dispatch now self-describes the BIOS it
was compiled from (identity: size/CRC32/SHA-256/wordsum; kernel-bless
window; HLE anchors; bundled flag). Runtime SCPH1001 hardcodes removed.
- Launch check is a hard identity gate (a mismatched image under
statically recompiled code is a guaranteed wild jump). Bundled builds
hide the BIOS picker and treat a missing/tampered bundled image as an
installation defect.
- Faithfulness fixes surfaced by OpenBIOS's tighter timing loops (all
global, argued in code comments, regression-tested against SCPH1001):
- SIO pad RX is cycle-paced like the memcard path (RX_RDY at
shift-complete, ACK +170); total ack-IRQ latency unchanged.
PSX_SIO_PAD_SYNC_RX=1 restores the legacy synchronous path for A/B.
- SPUSTAT derived from SPUCNT instead of permanently-asserted busy.
- GPUSTAT.13 (interlace field) alternates per vblank while interlaced,
pinned 0 in progressive.
- CD Init(0x0A) completes in ~3.9 ms instead of borrowing Beetle's
34 ms Reset figure (OpenBIOS's short bounded retry loop livelocked).
- Exception EPC guard accepts ROM-window block leaders, not just RAM.
- New TCP debug commands (documented in TCP_COMMANDS.md): bios_info,
s3_smear_watch, callret_watch; write-trace ring gains $s0-$s5.
bios/openbios.bin is vendored (MIT — notice in bios/OpenBIOS.LICENSE,
attribution in THIRD_PARTY_ATTRIBUTION.md, source pin + build recipe in
bios/OpenBIOS.toml). The emitter fingerprint digest is now content-only:
existing generated/*.emitter.sha will read STALE once; re-run
tools/regen_bios.sh.
|