Resolves#3075
TODO before merge:
- [x] Properly draw non-korean strings while in korean mode (language
selection)
- [x] Check jak 3
- [x] Translation scaffolding (allow korean characters, add to Crowdin,
fix japanese locale, etc)
- [x] Check translation of text lines
- [x] Check translation of subtitle lines
- [x] Cleanup PR / some performance optimization (it's take a bit too
long to build the text and it shouldn't since the information is in a
giant lookup table)
- [x] Wait until release is cut
I confirmed the font textures are identical between Jak 2 and Jak 3, so
thank god for that.
Some examples of converting the korean encoding to utf-8. These show off
all scenarios, pure korean / korean with ascii and japanese / korean
with replacements (flags):
<img width="316" height="611" alt="Screenshot 2025-07-26 191511"
src="https://github.com/user-attachments/assets/614383ba-8049-4bf4-937e-24ad3e605d41"
/>
<img width="254" height="220" alt="Screenshot 2025-07-26 191529"
src="https://github.com/user-attachments/assets/1f6e5a6c-8527-4f98-a988-925ec66e437d"
/>
And it working in game. `Input Options` is a custom not-yet-translated
string. It now shows up properly instead of a disgusting block of
glyphs, and all the original strings are hopefully the same
semantically!:
<img width="550" height="493" alt="Screenshot 2025-07-26 202838"
src="https://github.com/user-attachments/assets/9ebdf6c0-f5a3-4a30-84a1-e5840809a1a2"
/>
Quite the challenge. The crux of the problem is -- Naughty Dog came up
with their own encoding for representing korean syllable blocks, and
that source information is lost so it has to be reverse engineered.
Instead of trying to figure out their encoding from the text -- I went
at it from the angle of just "how do i draw every single korean
character using their glyph set".
One might think this is way too time consuming but it's important to
remember:
- Korean letters are designed to be composable from a relatively small
number of glyphs (more on this later)
- Someone at naughty dog did basically this exact process
- There is no other way! While there are loose patterns, there isn't an
overarching rhyme or reason, they just picked the right glyph for the
writing context (more on this later). And there are even situations
where there IS NO good looking glyph, or the one ND chose looks awful
and unreadable (we could technically fix this by adjusting the
positioning of the glyphs but....no more)!
Information on their encoding that gets passed to `convert-korean-text`:
- It's a raw stream of bytes
- It can contain normal font letters
- Every syllable block begins with: `0x04 <num_glyphs> <...the glyph
bytes...>`
- DO NOT confuse `num_glyphs` with num jamo, because some glyphs can
have multiple jamo!
- Every section of normal text starts with `0x03`. For example a space
would be `0x03 0x20`
- There are a very select few number of jamo glyphs on a secondary
texture page, these glyph bytes are preceeded with a `0x05`. These jamo
are a variant of some of the final vowels, moving them as low down as
possible.
Crash course on korean writing:
- Nice resource as this is basically what we are doing -
https://glyphsapp.com/learn/creating-a-hangeul-font
- Korean syllable blocks have either 2 or 3 jamo. Jamo are basically
letters and are the individual pieces that make up the syllable blocks.
- The jamo are split up into "initial", "medial" and "final" categories.
Within the "medial" category there are obvious visual variants:
- Horizontal
- Vertical
- Combination (horizontal + a vertical)
- These jamo are laid out in 6 main pre-defined "orientations":
- initial + vertical medial
- initial + horizontal medial
- initial + combination
- initial + vertical medial + final
- initial + horizontal medial + final
- initial + combination + final
- Sometimes, for stylistic reasons, jamo will be written in different
ways (ie. if there is nothing below a vertical vowel will be extended).
- Annoying, and ND's glyph set supports this stylistic choice!
- There are some combination of jamo that are never used, and some that
are only used for a single word in the entire language!
With all that in mind, my basic process was:
- Scan the game's entire corpus of korean text, that includes subtitles.
It's very easy to look at the font texture's glyphs and assign them to
their respective jamo
- This let me construct a mapping and see which glyphs were used under
which context
- I then shoved this information into a 2-D matrix in excel, and created
an in-game tool to check every single jamo permutation to fill in the
gaps / change them if naughty dogs was bad. Most of the time, ND's
encoding was fine.
-
https://docs.google.com/spreadsheets/d/e/2PACX-1vTtyMeb5-mL5rXseS9YllVj32BGCISOGZFic6nkRV5Er5aLZ9CLq1Hj_rTY7pRCn-wrQDH1rvTqUHwB/pubhtml?gid=886895534&single=true
anything in red is an addition / modification on my part.
- This was the most lengthy part but not as long as you may think, you
can do a lot of pruning. For example if you are checking a 3-jamo
variant (the ones with the most permutations) and you've verified that
the medial jamo is as far up vertically as it can be, and you are using
the lowest final jamo that are available -- there is nothing to check or
improve -- for better or worse! So those end up being the permutations
between the initial and medial instead of a three-way permutation
nightmare.
- Also, while it is a 2d matrix, there's a lot of pruning even within
that. For example, for the first 3 orientations, you dont have to care
about final vowels at all.
- At the end, I'm left with a lookup table that I can use the encode the
best looking korean syllable blocks possible given the context of the
jamo combination.
This is a simple multiplier to the gamepad axis input value received
from SDL events. Normally the values it provides cannot satisfy the
square range of the stick input. This is usually fine but it might play
differently with different controllers and compared to consoles,
especially considering the DualShock 1/2 have automatic calibration
which works in mysterious ways. The setting is there so any user can
adjust it for their controllers.
Saved to and loaded from the input-settings.json file.
133% matches PCSX2's default setting and is generally a good value to
map the square stick range within most modern(ish) controllers' circular
stick motion.
Progress menu option added to Jak 1 and 2. Setting can be changed from
50% all the way to 200%.
~~Renamed the analog deadzone options to stick deadzone since they don't
apply to the other analog buttons and only the (analog, yes) sticks.~~
This PR does two main things:
1. Work through the main low-hanging fruit issues in the formatter
keeping it from feeling mature and usable
2. Iterate and prove that point by formatting all of the Jak 1 code
base. **This has removed around 100K lines in total.**
- The decompiler will now format it's results for jak 1 to keep things
from drifting back to where they were. This is controlled by a new
config flag `format_code`.
How am I confident this hasn't broken anything?:
- I compiled the entire project and stored it's `out/jak1/obj` files
separately
- I then recompiled the project after formatting and wrote a script that
md5's each file and compares it (`compare-compilation-outputs.py`
- The results (eventually) were the same:
> This proves that the only difference before and after is non-critical
whitespace for all code/macros that is actually in use.
I'm still aware of improvements that could be made to the formatter, as
well as general optimization of it's performance. But in general these
are for rare or non-critical situations in my opinion and I'll work
through them before doing Jak 2. The vast majority looks great and is
working properly at this point. Those known issues are the following if
you are curious:
Adds the opengoal cheats to the secrets menu. Only cheats that are fully
functional and unlockable are there right now, which is eight cheats.
This update will reset most Jak 2 settings.
Also fixes#3274 .
Major change to how `deftype` shows up in our code:
- the decompiler will no longer emit the `offset-assert`,
`method-count-assert`, `size-assert` and `flag-assert` parameters. There
are extremely few cases where having this in the decompiled code is
helpful, as the types there come from `all-types` which already has
those parameters. This also doesn't break type consistency because:
- the asserts aren't compared.
- the first step of the test uses `all-types`, which has the asserts,
which will throw an error if they're bad.
- the decompiler won't emit the `heap-base` parameter unless necessary
now.
- the decompiler will try its hardest to turn a fixed-offset field into
an `overlay-at` field. It falls back to the old offset if all else
fails.
- `overlay-at` now supports field "dereferencing" to specify the offset
that's within a field that's a structure, e.g.:
```lisp
(deftype foobar (structure)
((vec vector :inline)
(flags int32 :overlay-at (-> vec w))
)
)
```
in this structure, the offset of `flags` will be 12 because that is the
final offset of `vec`'s `w` field within this structure.
- **removed ID from all method declarations.** IDs are only ever
automatically assigned now. Fixes#3068.
- added an `:overlay` parameter to method declarations, in order to
declare a new method that goes on top of a previously-defined method.
Syntax is `:overlay <method-name>`. Please do not ever use this.
- added `state-methods` list parameter. This lets you quickly specify a
list of states to be put in the method table. Same syntax as the
`states` list parameter. The decompiler will try to put as many states
in this as it can without messing with the method ID order.
Also changes `defmethod` to make the first type definition (before the
arguments) optional. The type can now be inferred from the first
argument. Fixes#3093.
---------
Co-authored-by: Hat Kid <6624576+Hat-Kid@users.noreply.github.com>
This PR adds detection of the `launch-particles` and `seconds-per-frame`
macros to the decompiler, removing a lot of bloat and hiding many
process register uses.
I also added `og:preserve-this` comments to as many manual patches and
comments as I could, which will soon be used in conjunction with CI to
hopefully catch any regressions in future big decomp update PRs.
I have some concerns about the `launch-particles` macro (more details in
`sparticle-launcher.gc`) , but thus far, I have not seen anything break
yet.
---------
Co-authored-by: water <awaterford111445@gmail.com>
Effects the following files:
- [x] vehicle-rider
- [x] vehicle-control
- [x] vehicle-effects
- [x] vehicle
~~- [ ] vehicle-util~~
- [x] vehicle-physics
- [x] vehicle-states
~~- [ ] vehicle-guard~~
~~- [ ] traffic-engine~~
~~- [ ] traffic-manager~~
With the exception of traffic-engine, most of these files are either
done or have 1-3 stubborn functions remaining. Draft while I try to
resolve as many as possible / cleanup names and such.
Co-authored-by: water <awaterford111445@gmail.com>
- You can define a `startup.gc` in your user folder, each line will be
executed on startup (deprecates the usefulness of some cli flags)
- You can define a `repl-config.json` file to override REPL settings.
Long-term this is a better approach than a bunch of CLI flags as well
- Via this, you can override the amount of time the repl will attempt to
listen for the target
- At the same time, I think i may have found why on Windows it can
sometimes take forever to timeout when the game dies, will dig into this
later
- Added some keybinds for common operations, shown here
https://user-images.githubusercontent.com/13153231/202890278-1ff2bb06-dddf-4bde-9178-aa0883799167.mp4
> builds the game, connects to it, attaches a debugger and continues,
launches it, gets the backtrace, stops the target -- all with only
keybinds.
If you want these keybinds to work inside VSCode's integrated terminal,
you need to add the following to your settings file
```json
"terminal.integrated.commandsToSkipShell": [
"-workbench.action.quickOpen",
"-workbench.action.quickOpenView"
]
```
* sparticle-launcher
* d/jak2: large amount of `sparticle-launcher` done
* d/jak2: finish the majority of `sparticle`
* decomp: improve format code ignoring
* d/jak2: make bits unique in `sp-cpuinfo-flag`
* d/jak1: revert config change
* scripts: cleanup scripts directory some more
* stash, getting there
* stash again
* closer!
* it works! decently....cleanup time and add some extra features
* minor cleanup
* decomp: Add texture-upload to ref tests
* maybe 50% done?
* 5 functions to go!
* decomp: stuck in `navigate`
* work-around fp issue
* some cleanup and label casts
* working on supporting asm instructions -- this is currently WRONG
* support ASM operations
* fixes for asm op support
* decomp: finish the vast majority of `navigate`
* format
* update test though i think this suggests a regression!
* decomp: cleanup some more of navigate
* decomp: finish `rolling-lightning-mole`
* revert `r0` handling for `pcpyud` and `pextuw`
* update ref tests
* lint
* fix a failing test
* help
* navigate mostly works now, with some potential bugs
* remove my debugging logs
* update ref tests
* review feedback cleanup
* these are all likely fine
* can't get the crab to chase me anymore
* the crab is back
* stash
* more
* and more
* like 80% done
* trying to update things
* refreshed work on generic-obs
* blocked: down to only decompiler issues!
* decomp: finish `generic-obs` skip over the failing functions for now
* `pair` -> `object`s and fix 2/4 of the functions i had skipped
* address feedback
* cmake: disable edit&continue flags
* goos: make the build system work for alternate file paths nicely
* vs: update vs config
* vscode: extend terminal buffer!
* vs: fix presets
* debugger: fix exception handler
* game: add logo to application
* decomp: get `cam-master` to "work" -- manually changed return type
* debugger: fix printing issue
* game: get the camera actually working
* game: neutralize the analog sticks
* game: support analog sticks
* tests: update ref tests
* temp commit - inprogress stuff
* fix `send-macro`
* turn camera stuff back on, seems to work. Still kernel-dispatch problem though
* address feedback
* formatting
* decomp: mostly finish `cam-master`
* decomp/scripts: lots of work in cam-states
* stash
* Merge remote-tracking branch 'water111/master' into decomp/camera-master
Updated submodule third-party/googletest
* decompiler: Add support for non power of 2 offsets for inline arr access
* decomp: mostly finish `cam-states` need to fix a macro issue
* blocked: `cam-master` decompiler crash when adding casts
* decomp: finish `cam-states-dbg`
* decomp: mostly finish `pov-camera` with the exception of joint-related code
* decomp: `cam-debug` finished decompiling, no way does this compile yet though
* decomp: considerable work done in `cam-layout`
* decomp: `cam-layout` almost done!
* decomp: `pov-camera` finished, TC tests will fail for now
* decomp: working on resolving issues
* decomp: cam-layout decompiling
* fixing more issues in cam-master...one event handler remains
* skip problematic function in `cam-master` for now
* gsrc: update res macros
* decomp: finish `cam-states`
* decomp: giving up on `cam-debug`
* tests: allow skipping state handlers in ref tests
* decomp: working through cam-layout bugs
* decomp: allow for shifting non-integers
* decomp: finalize `cam-layout` and `cam-master`
* decomp: finalize `cam-states`
* cleanup: bi-annual formatting of the casting files
* formatting
* address feedback - leave the float labels alone for now
* address feedback
* linting/formatting
* update gsrc and ref tests
Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>
* get a full mapping of all `symbol:file-name` for jak1
* cleanup before any additional work is done
* more pre-organize changes
* scripts: script written to cleanup all-types...but what will cleanup the script...
* resolve clashing enum symbol names
* some manual changes to symbol order
* very close! don't completely lose types that are fully commented out
* scripts: New symbol mapping and further script polishing
* changes needed to make the script function properly
* brand new all-types!
* remove now irrelevant symbol definitions
* formatting cpp
* decompiler: Remove symbol dumping feature, not accurate anyway
Tyler Wilding
Alex
Hat Kid
