Resolves#3075
TODO before merge:
- [x] Properly draw non-korean strings while in korean mode (language
selection)
- [x] Check jak 3
- [x] Translation scaffolding (allow korean characters, add to Crowdin,
fix japanese locale, etc)
- [x] Check translation of text lines
- [x] Check translation of subtitle lines
- [x] Cleanup PR / some performance optimization (it's take a bit too
long to build the text and it shouldn't since the information is in a
giant lookup table)
- [x] Wait until release is cut
I confirmed the font textures are identical between Jak 2 and Jak 3, so
thank god for that.
Some examples of converting the korean encoding to utf-8. These show off
all scenarios, pure korean / korean with ascii and japanese / korean
with replacements (flags):
<img width="316" height="611" alt="Screenshot 2025-07-26 191511"
src="https://github.com/user-attachments/assets/614383ba-8049-4bf4-937e-24ad3e605d41"
/>
<img width="254" height="220" alt="Screenshot 2025-07-26 191529"
src="https://github.com/user-attachments/assets/1f6e5a6c-8527-4f98-a988-925ec66e437d"
/>
And it working in game. `Input Options` is a custom not-yet-translated
string. It now shows up properly instead of a disgusting block of
glyphs, and all the original strings are hopefully the same
semantically!:
<img width="550" height="493" alt="Screenshot 2025-07-26 202838"
src="https://github.com/user-attachments/assets/9ebdf6c0-f5a3-4a30-84a1-e5840809a1a2"
/>
Quite the challenge. The crux of the problem is -- Naughty Dog came up
with their own encoding for representing korean syllable blocks, and
that source information is lost so it has to be reverse engineered.
Instead of trying to figure out their encoding from the text -- I went
at it from the angle of just "how do i draw every single korean
character using their glyph set".
One might think this is way too time consuming but it's important to
remember:
- Korean letters are designed to be composable from a relatively small
number of glyphs (more on this later)
- Someone at naughty dog did basically this exact process
- There is no other way! While there are loose patterns, there isn't an
overarching rhyme or reason, they just picked the right glyph for the
writing context (more on this later). And there are even situations
where there IS NO good looking glyph, or the one ND chose looks awful
and unreadable (we could technically fix this by adjusting the
positioning of the glyphs but....no more)!
Information on their encoding that gets passed to `convert-korean-text`:
- It's a raw stream of bytes
- It can contain normal font letters
- Every syllable block begins with: `0x04 <num_glyphs> <...the glyph
bytes...>`
- DO NOT confuse `num_glyphs` with num jamo, because some glyphs can
have multiple jamo!
- Every section of normal text starts with `0x03`. For example a space
would be `0x03 0x20`
- There are a very select few number of jamo glyphs on a secondary
texture page, these glyph bytes are preceeded with a `0x05`. These jamo
are a variant of some of the final vowels, moving them as low down as
possible.
Crash course on korean writing:
- Nice resource as this is basically what we are doing -
https://glyphsapp.com/learn/creating-a-hangeul-font
- Korean syllable blocks have either 2 or 3 jamo. Jamo are basically
letters and are the individual pieces that make up the syllable blocks.
- The jamo are split up into "initial", "medial" and "final" categories.
Within the "medial" category there are obvious visual variants:
- Horizontal
- Vertical
- Combination (horizontal + a vertical)
- These jamo are laid out in 6 main pre-defined "orientations":
- initial + vertical medial
- initial + horizontal medial
- initial + combination
- initial + vertical medial + final
- initial + horizontal medial + final
- initial + combination + final
- Sometimes, for stylistic reasons, jamo will be written in different
ways (ie. if there is nothing below a vertical vowel will be extended).
- Annoying, and ND's glyph set supports this stylistic choice!
- There are some combination of jamo that are never used, and some that
are only used for a single word in the entire language!
With all that in mind, my basic process was:
- Scan the game's entire corpus of korean text, that includes subtitles.
It's very easy to look at the font texture's glyphs and assign them to
their respective jamo
- This let me construct a mapping and see which glyphs were used under
which context
- I then shoved this information into a 2-D matrix in excel, and created
an in-game tool to check every single jamo permutation to fill in the
gaps / change them if naughty dogs was bad. Most of the time, ND's
encoding was fine.
-
https://docs.google.com/spreadsheets/d/e/2PACX-1vTtyMeb5-mL5rXseS9YllVj32BGCISOGZFic6nkRV5Er5aLZ9CLq1Hj_rTY7pRCn-wrQDH1rvTqUHwB/pubhtml?gid=886895534&single=true
anything in red is an addition / modification on my part.
- This was the most lengthy part but not as long as you may think, you
can do a lot of pruning. For example if you are checking a 3-jamo
variant (the ones with the most permutations) and you've verified that
the medial jamo is as far up vertically as it can be, and you are using
the lowest final jamo that are available -- there is nothing to check or
improve -- for better or worse! So those end up being the permutations
between the initial and medial instead of a three-way permutation
nightmare.
- Also, while it is a 2d matrix, there's a lot of pruning even within
that. For example, for the first 3 orientations, you dont have to care
about final vowels at all.
- At the end, I'm left with a lookup table that I can use the encode the
best looking korean syllable blocks possible given the context of the
jamo combination.
This updates `fmt` to the latest version and moves to just being a copy
of their repo to make updating easier (no editing their cmake / figuring
out which files to minimally include).
The motivation for this is now that we switched to C++ 20, there were a
ton of deprecated function usages that is going away in future compiler
versions. This gets rid of all those warnings.
- fix speaker names and time frames being uneditable in Jak 1
- added toggle to auto-selected a newly created scene as current
- changed the subtitle summary format slightly.
- current scene's name now appears in the UI
The main thing that was done here was to slightly modify the new
subtitle-v2 JSON schema to be more similar to the existing one so that
it can properly be used in Crowdin.
Draft while I double-check the diff myself
Along the way the following was also done (among other things):
- got rid of as much duplication as was feasible in the serialization
and editor code
- separated the text serialization code from the subtitle code for
better organization
- simplified "base language" in the editor. The new subtitle format has
built-in support for defining a base language so the editor doesn't have
to be used as a crutch. Also, cutscenes only defined in the base come
first in the list now as that is generally the order you'd work from
(what you havn't done first)
- got rid of the GOAL subtitle format code completely
- switched jak 2 text translations to the JSON format as well
- found a few mistakes in the jak 1 subtitle metadata files
- added a couple minor features to the editor
- consolidate and removed complexity, ie. recently all jak 1 hints were
forced to the `named` type, so I got rid of the two types as there isn't
a need anymore.
- removed subtitle editor groups for jak 1, the only reason they existed
was so when the GOAL file was manually written out they were somewhat
organized, the editor has a decent filter control, there's no need for
them.
- removed the GOAL -> JSON python script helper, it's been a month or so
and no one has come forward with existing translations that they need
help with migrating. If they do need it, the script will be in the git
history.
I did some reasonably through testing in Jak1/Jak 2 and everything
seemed to work. But more testing is always a good idea.
---------
Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>
This moves the blerc math from mips2c to the Merc2 renderer, and uses
floats instead.
We could potentially do this on the GPU, which would be even faster, but
this isn't that slow in the first place.
Adds support for adding custom subtitles to Jak 2 audio. Comes with a
new editor for the new system and format. Compared to the Jak 1 system,
this is much simpler to make an editor for.
Comes with a few subtitles already made as an example.
Cutscenes are not officially supported but you can technically subtitle
those with editor, so please don't right now.
This new system supports multiple subtitles playing at once (even from a
single source!) and will smartly push the subtitles up if there's a
message already playing:
Unlike in Jak 1, it will not hide the bottom HUD when subtitles are
active:
Sadly this leaves us with not much space for the subtitle region (and
the subtitles are shrunk when the minimap is enabled) but when you have
guards and citizens talking all the time, hiding the HUD every time
anyone spoke would get really frustrating.
The subtitle speaker is also color-coded now, because I thought that
would be fun to do.
TODO:
- [x] proper cutscene support.
- [x] merge mode for cutscenes so we don't have to rewrite the script?
---------
Co-authored-by: Hat Kid <6624576+Hat-Kid@users.noreply.github.com>
Favors the `lg` namespace over `fmt` directly, as this will output the
logs to a file / has log levels.
I also made assertion errors go to a file, this unfortunately means
importing `lg` and hence `fmt` which was attempted to be avoided before.
But I'm not sure how else to do this aspect without re-inventing the
file logging.
We have a lot of commented out prints as well that we should probably
cleanup at some point / switch them to trace level and default to `info`
level.
I noticed the pattern of disabling debug logs behind some boolean,
something to consider cleaning up in the future -- if our logs were more
structured (knowing where they are coming from) then a lot this
boilerplate could be eliminated.
Closes#1358
* put some duplicated code in a func
* make jak 2 text "work"
* group up all subtitles c++ code into one folder
* compact single-line subtitles
* fix a couple compiler crashes
* Update game_subtitle_en.gd
* `rolling` and `sunken`
* `swamp`
* `ogre`
* `village3`
* `maincave`
* `snow`
* `lavatube`
* `citadel`
* Update .gitignore
* clang
* fix encoding and decoding for quote
* properly fix quotes
* subtitle deserialize: sort by kind, ID and name
* sub editor: fix line speaker not being converted
* cleanup game text ids 1
* update text ids 2
* update source
* update refs
* [extractor] validate files when extracted as folder
* jp text fixes
* move game text version to the text file and fix subtitle editor escape chars
* make bad subtitles not crash the game
* fix texscroll in lag
* fix mood, fix decomp of other versions, fix text decomp
* clang
* fix tests
* oops dammit
* new fixes
* shut up codacy
* fix nonexistant subtitles crashing the game
* fix text hacks and extractor re-use on folders
* stash
* temp
* tools: subtitle tool works! just gotta fill out the db / polish UX
* tools: added configuration for every subtitle we have so far
* tools: add some colors to the editor, time for repl controls and make it run the code!
* tools: continuing polish of tool, getting very close
* tools: finished UX polish, just need to write deserializers
* tools: added deserializer for subtitle data
* tools: exported subtitle files, all data appears intact
* tools: more UX polish and test all the cutscenes, majority work
* assets: update subtitle files
* lint: formatting and cleanup
* lint: codacy lints
* music flava hack toggle
* fix hint subtitles not being considered offscreen
* music player works
* fixes + citadel, maincave, lavatube music
* add remaining tracks
* fix test
* fixes
* fix lib
Tyler Wilding
Hat Kid
ManDude
water111