This attempts to get into master whatever work was done in this PR /
it's earlier PR https://github.com/open-goal/jak-project/pull/3965
I don't want this work to be lost / floating around in massive PRs.
However the changes are:
- switch to ntsc_v1 instead of PAL as the development target, as we have
done for all other games
- remove most of the copied-from-jak2/3 changes as they need to be
confirmed during the decompilation process not just assumed
- avoids committing any changes to `game/kernel/common` as it was not
clear to me if these were changes made in jak x's kernel that were not
properly broken out into it's own functions. We don't want to
accidentally introduce bugs into jak1-3's kernel code.
- in other words, if the change in the kernel only happens in jak x...it
should likely be specific to jak x's kernel, not common.
---------
Co-authored-by: VodBox <dillon@vodbox.io>
Co-authored-by: yodah <greenboyyodah@gmail.com>
Resolves#3075
TODO before merge:
- [x] Properly draw non-korean strings while in korean mode (language
selection)
- [x] Check jak 3
- [x] Translation scaffolding (allow korean characters, add to Crowdin,
fix japanese locale, etc)
- [x] Check translation of text lines
- [x] Check translation of subtitle lines
- [x] Cleanup PR / some performance optimization (it's take a bit too
long to build the text and it shouldn't since the information is in a
giant lookup table)
- [x] Wait until release is cut
I confirmed the font textures are identical between Jak 2 and Jak 3, so
thank god for that.
Some examples of converting the korean encoding to utf-8. These show off
all scenarios, pure korean / korean with ascii and japanese / korean
with replacements (flags):
<img width="316" height="611" alt="Screenshot 2025-07-26 191511"
src="https://github.com/user-attachments/assets/614383ba-8049-4bf4-937e-24ad3e605d41"
/>
<img width="254" height="220" alt="Screenshot 2025-07-26 191529"
src="https://github.com/user-attachments/assets/1f6e5a6c-8527-4f98-a988-925ec66e437d"
/>
And it working in game. `Input Options` is a custom not-yet-translated
string. It now shows up properly instead of a disgusting block of
glyphs, and all the original strings are hopefully the same
semantically!:
<img width="550" height="493" alt="Screenshot 2025-07-26 202838"
src="https://github.com/user-attachments/assets/9ebdf6c0-f5a3-4a30-84a1-e5840809a1a2"
/>
Quite the challenge. The crux of the problem is -- Naughty Dog came up
with their own encoding for representing korean syllable blocks, and
that source information is lost so it has to be reverse engineered.
Instead of trying to figure out their encoding from the text -- I went
at it from the angle of just "how do i draw every single korean
character using their glyph set".
One might think this is way too time consuming but it's important to
remember:
- Korean letters are designed to be composable from a relatively small
number of glyphs (more on this later)
- Someone at naughty dog did basically this exact process
- There is no other way! While there are loose patterns, there isn't an
overarching rhyme or reason, they just picked the right glyph for the
writing context (more on this later). And there are even situations
where there IS NO good looking glyph, or the one ND chose looks awful
and unreadable (we could technically fix this by adjusting the
positioning of the glyphs but....no more)!
Information on their encoding that gets passed to `convert-korean-text`:
- It's a raw stream of bytes
- It can contain normal font letters
- Every syllable block begins with: `0x04 <num_glyphs> <...the glyph
bytes...>`
- DO NOT confuse `num_glyphs` with num jamo, because some glyphs can
have multiple jamo!
- Every section of normal text starts with `0x03`. For example a space
would be `0x03 0x20`
- There are a very select few number of jamo glyphs on a secondary
texture page, these glyph bytes are preceeded with a `0x05`. These jamo
are a variant of some of the final vowels, moving them as low down as
possible.
Crash course on korean writing:
- Nice resource as this is basically what we are doing -
https://glyphsapp.com/learn/creating-a-hangeul-font
- Korean syllable blocks have either 2 or 3 jamo. Jamo are basically
letters and are the individual pieces that make up the syllable blocks.
- The jamo are split up into "initial", "medial" and "final" categories.
Within the "medial" category there are obvious visual variants:
- Horizontal
- Vertical
- Combination (horizontal + a vertical)
- These jamo are laid out in 6 main pre-defined "orientations":
- initial + vertical medial
- initial + horizontal medial
- initial + combination
- initial + vertical medial + final
- initial + horizontal medial + final
- initial + combination + final
- Sometimes, for stylistic reasons, jamo will be written in different
ways (ie. if there is nothing below a vertical vowel will be extended).
- Annoying, and ND's glyph set supports this stylistic choice!
- There are some combination of jamo that are never used, and some that
are only used for a single word in the entire language!
With all that in mind, my basic process was:
- Scan the game's entire corpus of korean text, that includes subtitles.
It's very easy to look at the font texture's glyphs and assign them to
their respective jamo
- This let me construct a mapping and see which glyphs were used under
which context
- I then shoved this information into a 2-D matrix in excel, and created
an in-game tool to check every single jamo permutation to fill in the
gaps / change them if naughty dogs was bad. Most of the time, ND's
encoding was fine.
-
https://docs.google.com/spreadsheets/d/e/2PACX-1vTtyMeb5-mL5rXseS9YllVj32BGCISOGZFic6nkRV5Er5aLZ9CLq1Hj_rTY7pRCn-wrQDH1rvTqUHwB/pubhtml?gid=886895534&single=true
anything in red is an addition / modification on my part.
- This was the most lengthy part but not as long as you may think, you
can do a lot of pruning. For example if you are checking a 3-jamo
variant (the ones with the most permutations) and you've verified that
the medial jamo is as far up vertically as it can be, and you are using
the lowest final jamo that are available -- there is nothing to check or
improve -- for better or worse! So those end up being the permutations
between the initial and medial instead of a three-way permutation
nightmare.
- Also, while it is a 2d matrix, there's a lot of pruning even within
that. For example, for the first 3 orientations, you dont have to care
about final vowels at all.
- At the end, I'm left with a lookup table that I can use the encode the
best looking korean syllable blocks possible given the context of the
jamo combination.
`bigmap` and `blit-displays` mostly work. `blit-displays` is still
missing all of the special effects that were added in Jak 3 (brightness
and contrast settings, screen blur effect, etc.).
`bigmap` is missing the player marker texture (`hud-target-marker`) for
some reason, it's part of `tpage-17` which is coming from
`progress-minimap` and should already be included. The icons also
currently stretch when using aspect ratios other than 4:3.
The progress menu now also works for the most part. The draw order is a
bit messed up because some code was initially drawing things with the
ocean bucket, which was changed to use `hud-draw-hud-alpha` instead for
now. The texture for the volume and brightness/contrast sliders still
looks wrong.
Fixes#3653Fixes#3656
This includes all the collision stuff needed to spawn `target`,
decompiles the sparticle code and adds some of the PC hacks needed for
merc to run (it doesn't work quite right and looks bad, likely due to a
combination of code copied from Jak 2 and the time of day hacks).
There are a bunch of temporary hacks (see commits) in place to prevent
the game from crashing quite as much, but it is still extremely prone to
doing so due to lots of missing functions/potentially bad decomp.
---------
Co-authored-by: water <awaterford111445@gmail.com>
- `fma-sphere`
- `prim-beam-h`
- `cam-start`
- `ragdoll`
- `light-trails-h`
- `light-trails`
- `menu`
- `water`
- `water-flow`
- `hud`
- `hud-classes`
- `progress`
- `progress-draw`
---
The `get-texture` macro replaces calls to `lookup-texture-by-id` and
`lookup-texture-by-id-fast`. The `defpart` macro detection was modified
to print a pair like `(texture-name tpage-name)` for the texture field
that gets turned into a `texture-id` constant. Only used in Jak 3 at the
moment, I'll probably go through the other games at a later point.
This updates `fmt` to the latest version and moves to just being a copy
of their repo to make updating easier (no editing their cmake / figuring
out which files to minimally include).
The motivation for this is now that we switched to C++ 20, there were a
ton of deprecated function usages that is going away in future compiler
versions. This gets rid of all those warnings.
- `speech`
- `ambient`
- `water-h`
- `vol-h`
- `generic-obs`
- `carry-h`
- `pilot-h`
- `board-h`
- `gun-h`
- `flut-h`
- `indax-h`
- `lightjak-h`
- `darkjak-h`
- `target-util`
- `history`
- `collide-reaction-target`
- `logic-target`
- `sidekick`
- `projectile`
- `voicebox`
- `ragdoll-edit`
- most of `ragdoll` (not added to gsrc yet)
- `curves`
- `find-nearest`
- `lightjak-wings`
- `target-handler`
- `target-anim`
- `target`
- `target2`
- `target-swim`
- `target-lightjak`
- `target-invisible`
- `target-death`
- `target-gun`
- `gun-util`
- `board-util`
- `target-board`
- `board-states`
- `mech-h`
- `vol`
- `vent`
- `viewer`
- `gem-pool`
- `collectables`
- `crates`
- `secrets-menu`
Additionally:
- Detection of non-virtual state inheritance
- Added a config file that allows overriding the process stack size set
by `stack-size-set!` calls
- Fix for integer multiplication with `r0`
- Fixed detection for the following macros:
- `static-attack-info`
- `defpart` and `defpartgroup` (probably still needs adjustments, uses
Jak 2 implementation at the moment)
- `sound-play` (Jak 3 seems to always call `sound-play-by-name` with a
`sound-group` of 0, so the macro has been temporarily defaulted to use
that)
One somewhat significant change made here that should be noted is that
the return type of `process::init-from-entity!` was changed to `object`.
I've been thinking about this for a while, since it looks a bit nicer
without the `(none)` at the end and I have recently encountered init
methods that early return `0`.
There are art groups that are present in multiple levels, and that means
that also their textures are present in multiple levels. With texture
replacement, currently we need to make replacements for all instances if
we want it replaced everywhere, but this is not ideal, especially when
you make changes to your replacement texture and now you have to put it
in each folder again.
I added a way to replace all instances of a texture, by letting
texture-replacer people put their replacements into an '_all' folder. I
set up the logic in such a way that if you have a replacement for the
texture in its corresponding folder, it will take priority over a
replacement that you placed into the '_all' folder.
I personally found this very useful for replacing guard textures. The
guards appear in a lot of levels. But ideally you want them to look the
same everywhere. And that is why I looked into this and made a PR.
Oh and I changed what is printed in the 'Replacing ' part because it was
printing the path to our replacement, which didn't look nicely when
several textures got replaced by the same replacement from the '_all'
folder. So now it will print the original texture's page and name, I
think this information is more useful anyway.
There are potentially still some minor issues with the resulting files.
Some of them appear to have minor artifacts that playing through the
actual game do not -- but this is a much better starting point for
someone to iterate from if they are interested in improving things.
Fix the bug described in
https://github.com/open-goal/jak-project/issues/2882 where some shrubs
are transparent when they shouldn't be. The problem was that we never
carefully looked at the settings in `gs-prim`, which has a bit to
enable/disable alpha blending entirely. Now it should be correct for
both jak 1 and jak 2. To see this change, you'll need to re-extract.
Also adds a setting to disable saving texture .pngs, to speed up
decompilation. I left it on for jak 1 (to avoid confusion for texture
swapping(, but off for jak 2 for now.
The progress menu loads its icon textures from a .STR file that we were
previously ignoring.
This change:
- updates the decompiler so it can process a .STR file containing a
texture
- adds a feature to force an entire page to always be loaded in the PC
renderer by putting all textures in the GAME.FR3 file.
- regenerates the texture offset map file for jak 2 with these new
textures
For now, I've just put the icon textures in GAME.FR3. The downside is
that these will always stay on the GPU, using up VRAM even when they
aren't needed. But the entire GAME.FR3 file is under 3 MB so I think
it's ok.
Added framework to do texture animations entirely in C++. Currently only
works on relatively simple ones, and doesn't handle updating all
parameters - only the speeds.
Connected texture animations to merc and tfrag for skull gems, dark
bomb, and scrolling conveyors.
Cleaned up Tfragment/Tfrag3, which used to be two classes. This was one
of the first C++ renderers, so it had a weird design.
Adds support for adding custom subtitles to Jak 2 audio. Comes with a
new editor for the new system and format. Compared to the Jak 1 system,
this is much simpler to make an editor for.
Comes with a few subtitles already made as an example.
Cutscenes are not officially supported but you can technically subtitle
those with editor, so please don't right now.
This new system supports multiple subtitles playing at once (even from a
single source!) and will smartly push the subtitles up if there's a
message already playing:
Unlike in Jak 1, it will not hide the bottom HUD when subtitles are
active:
Sadly this leaves us with not much space for the subtitle region (and
the subtitles are shrunk when the minimap is enabled) but when you have
guards and citizens talking all the time, hiding the HUD every time
anyone spoke would get really frustrating.
The subtitle speaker is also color-coded now, because I thought that
would be fun to do.
TODO:
- [x] proper cutscene support.
- [x] merge mode for cutscenes so we don't have to rewrite the script?
---------
Co-authored-by: Hat Kid <6624576+Hat-Kid@users.noreply.github.com>
Add the vortex renderer. The vortex texture isn't there yet (it uses the
same texture as clouds), so it uses a checkerboard. But the
colors/vertices seem right.
Fixes some mistakes with merc draw modes. The glass in the palace level
no longer writes to the depth buffer (it's "water"):
Also fixes the one-frame flickers when level draw orders change. We
might be able to make this more efficient in the future, but this will
at least fix the frame with nothing drawn.
This should fix a bunch of texture-related issues by generating a table
of overlapping textures and just... adjusting them slightly so they
don't overlap. It's not the most elegant solution in the world, but I
think it's no worse than the existing hard-coded tpage dir stuff.
Definitely needs a clean up pass, but I think the functionality is very
close.
There's a few "hacks" still:
- I am using the emerc logic for environment mapping, which doesn't care
about the length of the normals. I can't figure out how the normal
scaling worked in etie. I want to do a little bit more experimentation
with this before merging.
- There is some part about adgifs for TIE and ETIE that I don't
understand. The clearly correct behavior of TIE/ETIE is that the alpha
settings from the adgif shader are overwritten by the settings from the
renderer. But I can't figure out how this happens in all cases.
- Fade out is completely disabled. I think this is fine because the
performance difference isn't bad. But if you are comparing screenshots
with PCSX2, it will make things look a tiny bit different.
Favors the `lg` namespace over `fmt` directly, as this will output the
logs to a file / has log levels.
I also made assertion errors go to a file, this unfortunately means
importing `lg` and hence `fmt` which was attempted to be avoided before.
But I'm not sure how else to do this aspect without re-inventing the
file logging.
We have a lot of commented out prints as well that we should probably
cleanup at some point / switch them to trace level and default to `info`
level.
I noticed the pattern of disabling debug logs behind some boolean,
something to consider cleaning up in the future -- if our logs were more
structured (knowing where they are coming from) then a lot this
boilerplate could be eliminated.
Closes#1358
* fix utf-8 handling around env-vars
* fix file opening errors related to unicode
* add uncaught exception handler in `gk` to ensure something is logged
* gracefully fail if window icon cant be loaded and work with unicode
* linux fix and add changes to vendor file
* extractor: refactor and cleanup for multi-game support
* deps: switch to `ghc::filesystem` as it is utf-8 everywhere by default
* extractor: finally working with unicode
* unicode: fix unicode cli args on windows in all `main` functions
* Remove assets folder, use more std::filesystem
* windows fix
* another one for windows
* another one
* better system for different folders
* rm debugging stuff
* let extractor override everything
* dont revert jak1 change
* [extractor] validate files when extracted as folder
* jp text fixes
* move game text version to the text file and fix subtitle editor escape chars
* make bad subtitles not crash the game
* fix texscroll in lag
* fix mood, fix decomp of other versions, fix text decomp
* clang
* fix tests
* oops dammit
* new fixes
* shut up codacy
* fix nonexistant subtitles crashing the game
* fix text hacks and extractor re-use on folders
* PAL dumps files
* alrighty then
* make PAL warning slightly more specific
* PAL patches for `title-obs`
* Update all-types.gc
* PAL patch `beach-obs`
* `process-taskable` PAL patch
* `ambient` PAL patch
* `yakow` PAL patch
* `village-obs` PAL patch
* `sparticle-launcher` patch
* `swamp-obs` PAL patch
* `sequence-a-village1` PAL patch
* typo
* errors
* `powerups` PAL patch
* `ogreboss` PAL patch
* jak 1 v2 encoding
* `load-boundary` PAL patch
* `flying-lurker` PAL patch
* `mayor` PAL patch
* update game encoding to PAL (v2) encoding
* `cam-debug` and `cam-update` PAL patch
* `fisher` PAL patch
* `target` PAL patch
* `target2` PAL patch and fix text compiling
* `target-death` PAL patch
* `target-racer-h` PAL patch
* `logic-target` PAL patch
* `main` PAL patch
* `snow-flutflut-obs` PAL patch
* `rolling-obs` PAL patch
* `gsound` PAL patch
* update refs
* `progress` and `progress-draw` PAL patches
* clang
* wrong.
* complain
* clang
* fix test
* fix blurry jp text
* fix weird interrupt lag from setting window size
* patch more text lines, special handling for credits
* Update FontUtils.cpp
* Add xdelta3 and file patching interface
* add window lock toggle and update settings ver
* better particle hacks
* add PAL support to extractor
* Fix credits
* also NTSC-J support
* make xdelta3 a separate library
* address feedback
Co-authored-by: water <awaterford111445@gmail.com>
* ci: fix windows releases (hopefully)
* scripts: fix Taskfile file references for linux
* asserts: add `ASSERT_MSG` macro and ensure `stdout` is flushed before `abort`ing
* asserts: refactor all `assert(false);` with a preceeding message instances
* lint: format
* temp...
* fix compiler errors
* assert: allow for string literals in `ASSERT_MSG`
* lint: formatting
* revert temp change for testing
* fix typo
* more typo
* shorten discord rpc text
* allow expanding enums after the fact (untested)
* make `game_text` work similar to subtitles
* update progress decomp
* update some types + `do-not-decompile` in bitfield
* fixes and fall back to original progress code
* update `progress` decomp with new enums
* update config files
* fix enums and debug menu
* always allocate (but not use) a lot of particles
* small rework to display mode options
* revert resolution/aspect-ratio symbol mess
* begin the override stuff
* make `progress-draw` more readable
* more fixes
* codacy good boy points
* first step overriding code
* finish progress overrides, game options menu fully functional!
* minor fixes
* Update game.gp
* Update sparticle-launcher.gc
* clang
* change camera controls text
* oops
* some cleanup
* derp
* nice job
* implement menu scrolling lol
* make scrollable menus less cramped, fix arrows
* make some carousell things i guess
* add msaa carousell to test
* oops
* Update progress-pc.gc
* make `pc-get-screen-size` (untested)
* resolution menu
* input fixes
* return when selecting resolution
* scroll fixes
* Update progress-pc.gc
* add "fit to screen" button
* bug
* complete resolutions menu
* aspect ratio menu
* subtitles language
* subtitle speaker
* final adjustments
* ref test
* fix tests
* fix ref!
* reduce redundancy a bit
* fix mem leaks?
* save settings on progress exit
* fix init reorder
* remove unused code
* rename goal project-like files to the project extension
* sha display toggle
* aspect ratio settings fixes
* dont store text db's in compiler
* properly save+load native aspect stuff
* add ocean-vu0 mips2c and decomp ocean-texture
* vu1 texture setup
* temp
* unoptimized version working
* optimized version by default
* update ref test
* decompiler: support adding new strings to the game_text.txt file
* gsrc: expand the pckernel type and functions to work better with the menu
* gsrc: add new text-ids
* gsrc: add new macros to help with menu code
* gsrc: make a new type to generalize their list menu options
* gsrc: add new menu options and enums
* gsrc: cleanup and refactor the list menu option drawing code
this allows us to easily add a new list menu option...well as easy as the rest is atleast (setting up static lists properly, etc)
* gsrc: add and cleanup handling of new menu options
* scripts: add checks with nice error messages for user facing taskfile recipes
* lint: formatting
* address simple feedback
* gsrc: move modified files to `pc/` folder
* gsrc: revert changes to originally decompiled files
* gsrc: move modified and new files to `goal_src/pc` folder
* gsrc: update paths in `all_files.gc`
* MSAA
* support different TIE lods
* tfrag lod picking
* always compress screenshots
* mute annoying nvidia opengl message
* use a string here instead
* oops
* more mistakes...
* ci: switch to codacy for coverage
* docs: update badges
* decomp: allow overriding config flags via CLI
* cleanup: top level file cleanup
* docs: big README overhaul
Attempt to close#1128 and #1086
* decomp: attempt to detect if `iso_data` is missing or wrongly extracted
* game: switch to `fpng` for screenshots, allow for compression
closes#1035
* game: switch vsync control to a checkbox
* lint: format cpp files
* lint: format json files
* docs/scripts: organize taskfile
Tyler Wilding
