The always-on Shift-JIS string-inventory scan walks every guest store
argument even when no translation table is loaded. Measured cost:
26-35% of whole-lane throughput on streaming-heavy titles (three GT2
race lanes; reproduced on Driver 2, where disabling it took one lane
from 19 to 43 vblank/s).
Capture now defaults off; PSX_XLATE_CAPTURE=1 turns the scan back on
for translation authoring. Substitution (armed table + language) is
unaffected either way.
Master's text-divergence guard (dirty_ram_text_native_ok) blocked native dispatch
of any compiled function whose text page is runtime-dirty (an overlay overwrote
it), falling back to interp and — when the entry can't be resolved there — to
psx_unknown_dispatch (FATAL). Tsumu's translation layer patches string/glyph DATA
tables that share a 4 KB page (and the guard's 256-byte compare window) with real
compiled functions, so a data patch diverged the page and fatally blocked
func_80070248 on boot ("unknown dispatch: 0x80070248").
Two-part faithful fix (general, not per-game):
- dirty_ram_text_native_ok now decides on the dirty/flagged path by BYTE-COMPARING
the entry window against the reference image, not the coarse page-dirty
heuristic. A data-only change no longer blocks a function whose own code bytes
are unchanged; a genuine code overwrite still diverges -> interp (guard intent
preserved). Pristine pages keep the no-compare fast path.
- New dirty_ram_text_bless(phys,bytes,len): text_xlate blesses every in-place RAM
patch (glyph/msg_inplace/msg_sep) into the (runtime-owned, mutable) reference
image, so intentional data patches read as data. Genuine game self-mod writing
DIFFERENT bytes still diverges and is still caught.
Verified: Tsumu boots to the title screen with the English translation applied,
no fatal. Benefits every title (more precise guard).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a general, opt-in language selector to the RmlUi launcher, driven entirely
by game.toml [localization]:
[localization]
default = "en"
languages = [ {code="en", label="English"}, {code="off", label="Default"} ]
- config_loader: parse [localization].default + languages[] into RuntimeConfig;
add UserSettings.language with settings.toml [localization] parse + write.
- launcher: a "LOCALIZATION" panel in the Settings view (cycle toggle, matching
the Renderer/Screen-model idiom) shown only when the game declares languages.
The chosen code persists to settings.toml and rides UserSettings.
- text_xlate: new text_xlate_set_language() re-applies the language after the
launcher (init runs before it); "off"/"jp"/"" disable APPLY (native game).
- main: seed/read-back the language through the launcher and re-apply on every
boot path (so a settings.toml choice sticks even when the launcher is skipped).
- Settings view laid out in two columns (VIDEO | LOCALIZATION+AUDIO+CONTROLLER)
with SYSTEM full-width below, so it uses the horizontal space and fits without
scrolling; #root gains overflow-y:auto as a safety net.
Only games that declare [localization] get the menu (Tsumu today); byte-identical
for every other title.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Tsumu's card-manager messages are packed into a few chunks addressed by a
pointer table at 0x80071a08, with distinct messages separated only by
fullwidth-space runs and no terminator between them. The text renderer
(msg ptr at gp+0x26c) ends a message on \r (0xFC10 prompt-wait) or 0xFFFF,
never 0x0000 (drawn as a glyph), so a non-last message bled through its
neighbours to the next \r -- e.g. the "Reading Card" / "Saving" status
messages rendered the format prompt AND its Yes/No wait, mangled together.
Add a [[msg_sep]] layer that drops a 0xFFFF end marker at each
inter-message space-run so every message ends cleanly. Verify-then-patch:
only a resident fullwidth space (0x40 0x81) is overwritten, never text.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Some UI text (Tsumu's memory-card dialogs) is addressed through a pointer
table the renderer indexes internally, so the string pointer never appears as
a dispatch arg and the a0..a3 message hook can't reach it. Add MsgInplace: an
[[entry]] carrying `ram_addr` is verify-then-overwritten in place (LE fullwidth-
SJIS body + FE-FF/30-FC/10-FC framing) like a glyph-label, so the game's own
renderer reads the translation. Structure-preserving: sized to the record so
the trailing \r (prompt/wait) an interactive dialog needs lands correctly.
- term="none": mid-blob sub-record (e.g. a Yes/No prompt whose \r is followed
by another message in the same span) — write body only, no pad/terminator.
- fw_sjis_for_ascii: add '$' -> 0x8190 (Tsumu font renders it as the O button).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two interlocking pieces from the Tsumu HUD/menu-label investigation:
- GP0 ring builder attribution (gpu.h/gpu.c/debug_server.c): op-0x80
entries now carry a bounded guest-stack unwind (bld[6]) + copy-time
$sp (csp); gpu_frame_dump emits func/ra per entry and csp+bld[] for
copies. This cracked the HUD compose: whole label strips are
MoveImaged from the glyph sheet (src 834/852,352 -> HUD cache y=32).
- [[vram_patch]] (text_xlate.cpp/.h + gpu.c hook): third translation
layer for text that exists only as pre-rendered pixels in an
uploaded texture asset (Tsumu HUD labels, menus, prompts live in
the glyph-sheet image; the 144-glyph font has no katakana at all).
On every GP0 0xA0 upload completion the configured rects are
verified halfword-for-halfword against expected JP pixels and only
then rewritten with the target-language pixels via the renderer
facade (software/GL/supersample coherent). Re-applies on every
re-upload; verify-fail leaves foreign assets untouched. Debug:
xlate stats gains vram_patches/vram_applied; new "vpatch"
force-apply subcommand for config iteration without rebuild.
Verified on-screen: HUD "STAGE: 1-1 / BATTERY: 100", pause menu
(CONTINUE/RESTART/GIVE UP/CHANGE CONTROLS, TARGET BATT), title-menu
strips. docs/STRING_TRANSLATION.md Appendix Z documents the model,
mechanism, authoring recipe, and remaining strip backlog.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Capture-side dedup (apply path unaffected):
- sj_at_record_start(): ingest a string only when the source pointer is
at a genuine record boundary (preceded by a NUL/0xFFFF terminator or a
region edge), so a register landing mid-record no longer re-ingests the
same string at every byte offset (collapses the partial-offset
explosion — inventory dropped from ~11k coincidental captures to the
real string count).
- sj_capture_worthy(): require a minimum length, >=2 real kana/fullwidth
chars, >=80% of decoded units in valid Shift-JIS text rows, and >=50%
of the record as 2-byte text — dropping binary/coordinate false
positives that passed the record reader by chance.
Builds clean (build-master). Verified compile+link; runtime dedup-count
validation pending a non-playing window.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a third translation path to the framework for UI text drawn
glyph-by-glyph as sprites from a fixed-stride, space-padded EXE table
(Tsumu's 59-slot level/stage-name array at 0x80070300..0x800708FE).
Such labels carry no string pointer and no terminator, so the existing
dispatch/arg-scan message hook can never reach them.
Instead, [[glyph_label]] TOML entries drive a load-time RAM source patch:
once the EXE label region is resident, each confirmed slot is verified
byte-for-byte against its expected JP source bytes, then overwritten with
the target-language text (fullwidth-Latin Shift-JIS, little-endian)
re-padded with fullwidth spaces to exactly the slot width. The game's own
per-glyph routine then draws English. Verify-before-patch means a slot is
never touched unless the exact JP label is present; writing exactly
`width` bytes (== the table stride) means a patch never spills into the
adjacent slot.
- label_transcode_le(): UTF-8 -> fixed-width LE fullwidth-SJIS, truncated
to whole cells, fw-space padded; never exceeds width.
- glyph_labels_patch_locked(): idempotent verify-and-patch scan, run from
the dispatch hook while any label is unpatched (throttled; a single
relaxed load once g_glyph_pending hits 0).
- Loaded/gated exactly like message entries (language column, PSX_LANG,
apply-armed). "xlate glyph" debug subcommand reports per-slot status.
Verified (build-dbg, headless): all 59 slots patched (glyph_patched=59,
pending=0), RAM byte-exact English at TOWER/SANDBOX/JELLYFISH/LOOP THE
WARP, game alive at 26M dispatches. Framework code only; no regen, works
in Release. Table + user visual sign-off in TsumuRecomp/translations.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Transcoder passes through \f (page break -> 30 FC) and \r (prompt/wait
-> 10 FC) authoring tokens so multi-page dialogs keep their structure.
- Apply safety refined from "framed-only" to "framed OR clean-text": a NUL
record is applied only when it carries no wild low-control binary
(0x01-09/0B-0F/11-1F) — i.e. it's a real message, not a fixed-width
struct field with packed params. Lets NUL confirmation dialogs translate
while still excluding the struct labels that crash on replacement.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Empirical Phase-0 findings folded into the mechanism from driving Tsumu
(SLPS-02253) headless:
- Message-table text is stored LITTLE-ENDIAN 16-bit Shift-JIS (EXE menu
labels are big-endian); transcode endianness is selectable (PSX_XLATE_LE),
so English glyph codes match what the message drawer reads.
- Capture-quality gate: real text is hiragana/katakana/fullwidth-heavy
(SJIS lead 0x82/0x83); require >=2 such chars so the always-on inventory
stays a clean enumeration (52 real records vs 20k of vertex-binary noise).
- Record reader keeps control-framing verbatim (FE-FF line break, lone
81/FE, FF-FF end) so framed multi-line messages hash byte-exact; apply
remaps '\n' -> FE FF for those.
- Terminator (nul|ffff) captured + reported over TCP.
- SAFETY: apply only substitutes standalone framed (0xFFFF) messages by
default. NUL records are frequently fixed-width struct fields with binary
params packed after the text (e.g. the "Tutorial N" label) — replacing
those corrupts the struct and derails the game (observed PC=0 crash);
PSX_XLATE_ALLOW_NUL=1 lifts the gate for vetted labels.
Verified headless: framed tutorial hints apply on real draws (hits climb),
game stays alive (no crash), English written little-endian.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Implements the framework from docs/STRING_TRANSLATION.md: an always-on
capture inventory + a draw-time apply hook, both hung off the psx_dispatch
chokepoint via fntrace_record (no generated-code edits, no BIOS regen,
works in Release).
- text_xlate.{h,cpp}: EncodingProfile abstraction (shift_jis profile:
SJIS record reader with NUL/0xFFFF terminators + halfwidth-kana + control
framing; UTF-8 -> fullwidth-SJIS transcoder so English renders with the
font's existing fullwidth Latin, zero glyph injection). Capture inventory
(FNV-1a64 keyed) queryable over TCP; apply repoints the string-arg reg at
guest-stack scratch (KV-gated: only known records are swapped, stray
pointers untouched; own scratch is skipped to avoid recapture).
- fntrace.c: call text_xlate_on_dispatch() at the dispatch chokepoint.
- config_loader: [localization].language (+ [runtime].language alias);
env PSX_LANG overrides. Loads translations/*.toml per project.
- main.cpp: text_xlate_init() at game-config load.
- debug_server: "xlate" command (stats|dump|todo|reload) — observability
over TCP, no log files.
Verified: builds (Release), Tsumu boots headless, capture live
(distinct records accumulating), apply disabled until a table is present.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Codex