Commit Graph

257 Commits

Author SHA1 Message Date
Tyler Wilding e6f55b9b55 d/jak2: decompile all *-texture files except castle-texture (#2149)
Fixes https://github.com/open-goal/jak-project/issues/2051
2023-01-28 17:37:37 -05:00
Tyler Wilding 98393c6f4c goalc: support static arrays of types (#2140)
draft because using the array is a little weird still, don't feel like
dealing with window's slow debugging builds today.

I get the following weird error:
```clj
(define test-array (new 'static 'boxed-array :type type vector))
gr> (-> test-array 0)
1538004        #x1777d4              0.0000        vector
gr> (type? (-> test-array 0) type)
1342757        #x147d25              0.0000        #t
gr> (new 'static (-> test-array 0))
-- Compilation Error! --
Got 3 arguments, but expected 2
Form:
(-> test-array 0)
Location:
Program string:1
(new 'static (-> test-array 0))
^
Code:
(new 'static (-> test-array 0))
```

Maybe this is expected though and the `new` method wants a symbol, not a
type?

Fixes #2060

Co-authored-by: water <awaterford111445@gmail.com>
2023-01-21 21:40:39 -05:00
water111 d65a1c365a some bug fixes (#2125) 2023-01-14 11:04:15 -05:00
Tyler Wilding 00ac12094e goalc/repl: cleanup of goalc/REPL code and some QoL improvements (#2104)
- lets you split up your `startup.gc` file into two sections
  - one that runs on initial startup / reloads
  - the other that runs when you listen to a target
- allows for customization of the keybinds added a month or so ago
- removes a useless flag (--startup-cmd) and marks others for
deprecation.
- added another help prompt that lists all the keybinds and what they do

Co-authored-by: water <awaterford111445@gmail.com>
2023-01-07 11:24:02 -05:00
Tyler Wilding 2f4146d469 tests: make the offline tests aware of the current terminals row count (#2105)
This fixes the hideous output when your terminal would be too small to
hold all the threads.
2023-01-07 10:35:12 -05:00
Tyler Wilding 284fb09997 d/jak2: decompile fodder and nav-enemy (#2080)
fixes #1983 

First functioning enemy!


https://user-images.githubusercontent.com/13153231/210184845-f1d93491-11f5-4a64-b69c-41e535992ebc.mp4
2023-01-01 21:03:07 -05:00
Tyler Wilding 9c631e11fe offline-test: Partition by DGO and colorize/condense output (#2045)
This solves two main problems:
- the looming threat of running out of memory since every thread would
consume duplicate (and probably not needed) resources
- though I will point out, jak 2's offline tests seem to hardly use any
memory even with 400+ files, duplicated across many threads. Where as
jak 1 does indeed use tons more memory. So I think there is something
going on besides just the source files
- condense the output so it's much easier to see what is happening / how
close the test is to completing.
- one annoying thing about the multiple thread change was errors were
typically buried far in the middle of the output, this fixes that
- refactors the offline test code in general to be a lot more modular

The pretty printing is not enabled by default, run with `-p` or
`--pretty-print` if you want to use it


https://user-images.githubusercontent.com/13153231/205513212-a65c20d4-ce36-44f6-826a-cd475505dbf9.mp4
2022-12-22 13:41:33 -05:00
Tyler Wilding ac3c4e59b0 goalc/repl: Allow hot-loading files via ml with just the object name (#2036)
This allows you to not have to define the entire file path to a source
file to re-compile and load it. Technically a stop-gap until editor
tools are developed around writing OpenGOAL.


![image](https://user-images.githubusercontent.com/13153231/203196148-de61cf4b-42c8-43dc-a7fd-80e6ba6f5ac2.png)

As opposed to `(ml "goal_src/jak2/engine/game/main.gc")` (which still
works)

This is accomplished via the following config (connection attempts is
irrelevant):
```json
{
  "numConnectToTargetAttempts": 1,
  "jak2": {
    "asmFileSearchDirs": [
      "goal_src/jak2"
    ]
  }
}
```

This also provides a way to make game-specific configurations for the
REPL fairly easily.
2022-11-29 19:22:22 -05:00
Tyler Wilding a0a85eb60a repl: Add a few new quality of life improvements (#2030)
- You can define a `startup.gc` in your user folder, each line will be
executed on startup (deprecates the usefulness of some cli flags)
- You can define a `repl-config.json` file to override REPL settings.
Long-term this is a better approach than a bunch of CLI flags as well
- Via this, you can override the amount of time the repl will attempt to
listen for the target
- At the same time, I think i may have found why on Windows it can
sometimes take forever to timeout when the game dies, will dig into this
later
- Added some keybinds for common operations, shown here


https://user-images.githubusercontent.com/13153231/202890278-1ff2bb06-dddf-4bde-9178-aa0883799167.mp4
> builds the game, connects to it, attaches a debugger and continues,
launches it, gets the backtrace, stops the target -- all with only
keybinds.

If you want these keybinds to work inside VSCode's integrated terminal,
you need to add the following to your settings file
```json
"terminal.integrated.commandsToSkipShell": [
    "-workbench.action.quickOpen",
    "-workbench.action.quickOpenView"
  ]
```
2022-11-20 14:28:41 -05:00
water111 811b5de6cb [decomp] collide-mesh (#1999) 2022-10-30 15:54:10 -04:00
water111 b6f0ef52b3 [decomp] fixes for loader and game-info (#1968)
Fixes for the functions that weren't decompiled. `initialize` in
`game-info.gc` was fixed manually.
2022-10-14 20:47:59 -04:00
Tyler Wilding e3473c1902 d/jak2: finish cty-guard-turret-button | race-h | height-map-h and a lot of rigid-body (#1957)
Also cleaned up `data_decompiler.cpp` to make it a lot less verbose to
add a special case for an array field.
2022-10-11 23:20:36 -04:00
Tyler Wilding f6bdc07990 d/jak2: finish progress menu code and initialize the camera (#1945)
This PR does a few main things:
- finish decompiling the progress related code
- implemented changes necessary to load the text files end-to-end
   - japanese/korean character encodings were not added
- finish more camera code, which is required to spawn the progress menu
/ init the default language settings needed for text
  - initialized the camera as well

Still havn't opened the menu as there are a lot of checks around
`*target*` which I havn't yet gone through and attempted to comment out.
2022-10-11 18:30:26 -04:00
Tyler Wilding 4d751af38e logs: replace every fmt::print with a lg call instead (#1368)
Favors the `lg` namespace over `fmt` directly, as this will output the
logs to a file / has log levels.

I also made assertion errors go to a file, this unfortunately means
importing `lg` and hence `fmt` which was attempted to be avoided before.
But I'm not sure how else to do this aspect without re-inventing the
file logging.

We have a lot of commented out prints as well that we should probably
cleanup at some point / switch them to trace level and default to `info`
level.

I noticed the pattern of disabling debug logs behind some boolean,
something to consider cleaning up in the future -- if our logs were more
structured (knowing where they are coming from) then a lot this
boilerplate could be eliminated.

Closes #1358
2022-10-01 11:58:36 -04:00
ManDude 9351bf782e [decomp2] game-info, game-task and task-control (#1884)
And everything else needed for them!

A couple functions are bad currently.

- fixes #1929 - untested on linux
- fixes #1924 - now you need to type `,` before a lambda you want to put
in a pair.
- fix debugger symbol table in jak 2
- made the decompiler output `(meters 2)` instead of `(meters 2.0)`
- fixed a bug with the bitfield enum special -1 case
- made bad game text decomp not exit the decompiler
- added `editable-player` and `script`
2022-09-27 19:44:20 -04:00
Hat Kid a53c06fe2b decomp: target, target-board, board-states (#1915)
Just missing `target-board-handler` because it errored with `Bad delay
slot in clean_up_cond_no_else_final` and I wasn't able to figure out
where the asm branch was.

Commented out `target-board-clone-anim` because it crashes even after
adding `clone-anim` and `clone-anim-once`.
2022-09-25 12:07:37 -04:00
water111 edecac9f09 [decomp] add shrubbery renderer (#1914)
as with last time, fog and time of day are still garbage, but it still
works:


![image](https://user-images.githubusercontent.com/48171810/192119423-91b80500-a161-42ce-b734-0c528cc721cf.png)

The shrub near render is just falling back to generic again.
2022-09-24 17:46:13 -04:00
water111 6227c6d6a8 More array special cases (#1913)
also make jak 1 and jak 2 behave the same way, to reduce confusion. It
wasn't too bad to update jak 1.
2022-09-24 16:10:13 -04:00
water111 1b45aab3cc [decompile] subdivide, wind-work, tie-work, bsp, focus (#1897)
- decompile `subdivide`, `wind-work`, `tie-work`, `bsp`, `focus`
- support `ppacb` in compiler
- don't assert when bitfield stuff fails due to constant propgataion
weirdness
- finish up history
- div/mod unsigned assert fix in decompiler
- empty assert fix in decompiler for failed `add` type prop
- make jak 1 performance counters "work" (just measure time)
- fix cast/typos on pcgtb/vftoi15
2022-09-17 14:58:25 -04:00
Tyler Wilding 82e0517275 d/jak2: get script decompiling, no ref tests yet (#1877)
Has boxed array accessing that prevents me from adding anything to ref
tests (the entire file is lambdas so the access pattern that i would
like to ignore happens at the top-level, can't ignore it.

This code actually already has quite a bit of original docstrings so
it's not too bad in that regard considering a `script-context` can have
16 arbitrary objects. It seems they rarely put more than a single object
in the context and the types are usually obvious / are actually type
checked!
2022-09-14 19:37:12 -04:00
water111 b5d21be9c5 W/misc fixes (#1838)
* temp

* temp

* before cleaning up

* cleanup merge

* fix warnings

* merge fix

* clang format
2022-09-05 20:29:12 -04:00
Tyler Wilding 0896bef2bf jak1/speedruns: Some final touches for speedrunning in jak 1 (#1830)
* jak1: put common speedrunning code into it's own file

* jak1: enforce `60` fps while in speedrunning mode

* jak1: when speedrunning, display the version until you get the first powercell

* jak1: add an explicit option for skipping cutscenes

* jak1: extend `game-option` to allow any menu option to be disabled

* tests/jak1: allow whitelisting types to be redefined to satisfy typeconsistency checks

* jak1: add file headers

* jak1: cleanup bool checking

* test: delete the es substitle file

* test: add it back

* jak1: missed one cleanup spot related to bool comparisons
2022-09-02 18:15:42 -04:00
water111 bc3cd2cf72 [goalc] Fix error when putting #f in an array of symbols (#1804) 2022-08-26 15:54:29 -04:00
Tyler Wilding 090b2984a2 decomp: add docstring support to relevant places in all-types (#1753)
* decomp: add `docstring` support to relevant places in `all-types`

* decomp: output method docstring into the `defmethod` instead

* goalc: handle docstrings in `define[-extern]` (gracefully ignore for now)

* decomp: output docstrings for bitfield deftypes too

* goalc: fix `defenum` parsing when coming from the compiler (no symbol metadata to store to)

* lsp/tests: fix ups

* lint: formatting

* goalc: handle edge-case of defining a string constant

* cleanup leftovers, fix codacy issues, rename struct
2022-08-23 00:32:07 -04:00
ManDude dee8feb003 fix a game bug and add a new compiler feature (#1758)
* dont crash on inline static value array

* [goalc] add support for non-static inline arrays of values

* add a kernel group to jak1

* move and cleanup pc debug code

* random cleanup in `hud-classes`

* pc port fix for gondola spools

* format

* address feedback (wow it was actually just changing 1 line all along...)

* minor decomp cleanup

* ref test
2022-08-15 18:46:29 -04:00
water111 dc652d10c5 [jak2] decomp gkernel, setup offline tests (#1638)
* add comments

* oops

* format'

* spelling is hard
2022-07-12 18:50:18 -04:00
water111 5e23057ed1 [goalc] compile/run code for jak2 ckernel, set up dummy KERNEL.CGO (#1625)
[goalc] start can compile and run code for jak2 ckernel, set up dummy KERNEL.CGO
2022-07-08 19:23:49 -04:00
water111 28a2ecdfd3 [jak2] goalc supports multiple projects (#1619)
* [jak2] goalc supports multiple projects

* disable deci2 server if not debugging
2022-07-06 21:18:08 -04:00
Tyler Wilding 6446389263 extractor: cleanup, support unicode properly, and add multi-game support (#1609)
* extractor: refactor and cleanup for multi-game support

* deps: switch to `ghc::filesystem` as it is utf-8 everywhere by default

* extractor: finally working with unicode

* unicode: fix unicode cli args on windows in all `main` functions
2022-07-05 20:38:13 -04:00
ManDude 9676100039 finish out english subtitles (#1586)
* put some duplicated code in a func

* make jak 2 text "work"

* group up all subtitles c++ code into one folder

* compact single-line subtitles

* fix a couple compiler crashes

* Update game_subtitle_en.gd

* `rolling` and `sunken`

* `swamp`

* `ogre`

* `village3`

* `maincave`

* `snow`

* `lavatube`

* `citadel`

* Update .gitignore

* clang

* fix encoding and decoding for quote

* properly fix quotes

* subtitle deserialize: sort by kind, ID and name

* sub editor: fix line speaker not being converted

* cleanup game text ids 1

* update text ids 2

* update source

* update refs
2022-07-03 17:25:28 -04:00
water111 f763eb6bf0 [jak2] support for multiple out/ directories (#1585)
* [jak2] support for multiple out/ directories

* windows
2022-06-30 21:11:58 -04:00
Tyler Wilding 36dc015d62 jak2: Create goal_src skeleton (#1576) 2022-06-29 22:20:09 -04:00
ManDude 2649fd17ec subtitle editor fixes + other smaller fixes (#1572)
* [extractor] validate files when extracted as folder

* jp text fixes

* move game text version to the text file and fix subtitle editor escape chars

* make bad subtitles not crash the game

* fix texscroll in lag

* fix mood, fix decomp of other versions, fix text decomp

* clang

* fix tests

* oops dammit

* new fixes

* shut up codacy

* fix nonexistant subtitles crashing the game

* fix text hacks and extractor re-use on folders
2022-06-29 19:43:23 -04:00
water111 628ce47b2e compiler speed improvement (#1547) 2022-06-24 18:21:24 -04:00
ManDude 85bef15364 full ntsc-j mode + greatest hits support (#1525)
* remove str files from inputs

* removed unused verbose flag

* allow some kind of conditional game building for JP extras

* make `PrintBankInfo` do nothing

* clang

* Update gltf_mesh_extract.cpp

* fix `*jak1-full-game*` in extractor

* use json library to build json file

* screw it red label support as well
2022-06-23 18:44:02 -04:00
Tyler Wilding 2d595c1ac0 lint: add include sorting config to clang-format (#1517) 2022-06-22 23:37:46 -04:00
Tyler Wilding 9fdf0bbc2f tools: Add cutscene player / subtitle editor window (#1429)
* stash

* temp

* tools: subtitle tool works! just gotta fill out the db / polish UX

* tools: added configuration for every subtitle we have so far

* tools: add some colors to the editor, time for repl controls and make it run the code!

* tools: continuing polish of tool, getting very close

* tools: finished UX polish, just need to write deserializers

* tools: added deserializer for subtitle data

* tools: exported subtitle files, all data appears intact

* tools: more UX polish and test all the cutscenes, majority work

* assets: update subtitle files

* lint: formatting and cleanup

* lint: codacy lints
2022-06-11 16:32:27 +01:00
water111 ecb2781a89 [decompiler] cleanup to get atomic ops working on jak2 inputs (#1426)
* [decompiler] cleanup to get atomic ops working on jak2 inputs

* clang format
2022-06-08 18:34:52 -04:00
ManDude d01c16f451 [game] music player (#1416)
* music flava hack toggle

* fix hint subtitles not being considered offscreen

* music player works

* fixes + citadel, maincave, lavatube music

* add remaining tracks

* fix test

* fixes

* fix lib
2022-06-04 14:53:49 -04:00
ManDude 0212aa10c9 [decomp] better handling of animation code and art files (#1352)
* update refs

* [decompiler] read and process art groups

* finish decompiler art group selection & detect in `ja-group?`

* make art stuff work on offline tests!

* [decompiler] detect `ja-group!` (primitive)

* corrections.

* more

* use new feature on skel groups!

* find `loop!` as well

* fully fledged `ja` macro & decomp + `loop` detect

* fancy fixed point printing!

* update source

* `:num! max` (i knew i should've done this)

* Update jak1_ntsc_black_label.jsonc

* hi imports

* make compiling the game work

* fix `defskelgroup`

* clang

* update refs

* fix chan

* fix seek and finalboss

* fix tests

* delete unused function

* track let rewrite stats

* reorder `rewrite_let`

* Update .gitattributes

* fix bug with `:num! max`

* Update robotboss-part.gc

* Update goal-lib.gc

* document `ja`

* get rid of pc fixes thing

* use std::abs
2022-05-20 02:30:14 +01:00
water111 be976d2e69 [merc] Add merc extraction (#1356)
* docs for ee merc code

* wip

* more extraction stuff

* partial mat1 working

* mat1

* cleanup

* partial mat2 and mat3 support

* merc extraction seems to work
2022-05-11 22:53:53 -04:00
Tyler Wilding 7b6d732a77 goalc: Add TCP server socket in REPL process (#1335)
* goalc: cleanup goalc's main method and add nrepl listener socket

* deps: add standalone ASIO for sockets

* lint: formatting

* common: make a common interface for creating a server socket

* goalc: setup new repl server

* deps: remove asio

* goalc: debug issues, nrepl is working again

* git: rename files

* attempt to fix linux function call

* test

* scripts: make the error message even more obvious....

* goalc: make suggested changes, still can't reconnect properly

* game: pull out single-client logic from XSocketServer

* nrepl: supports multiple clients and disconnection/reconnects

* goalc: some minor fixes for tests

* goalc: save repl history when the compiler reloads

* common: add include for linux networking

* a few small changes to fix tests

* is it the assert?

* change thread start order and add a print to an assert

Co-authored-by: water <awaterford111445@gmail.com>
2022-05-06 18:19:37 -04:00
ManDude 8eb0796f8e subtitle editing and text project compiler rework (#1331)
* add a `subtitle` bucket and rename buckets 67 and 68

* automatic `heap-base`

* subtitles editor

* adjust default params

* bug

* Update subtitle.gc

* fix blue color inconsistency in progress

* void.

* Update subtitle.gc

* Update subtitle.gc

* Extend `heap-base` detection to check for errors

* sometimes it's actually manually set!

* remove broken error check

* rework how compiling text files works

* try out subtitles source v2

* bugfix

* oops

* Update game_text_common.cpp
2022-04-24 15:15:16 -04:00
ManDude 0cdeed9be7 minor windows fixes (#1311)
* [pp2] put `define` dest on a single line

* update source!

* Update type_analysis.cpp

* update old credits & racer code

* change clang-cl args (REALLY force avx)

* Update credits_REF.gc

* comment small unused code

* add timer to decompiler

* fix unnecessary copy-constructors (no speed increase)

* fixes

* Update expression_build.cpp

* wtf is this thing anyway

* im bored.

* clang

* fix!

* Revert "fix!"

This reverts commit 5b1ce6c718.

* Revert "clang"

This reverts commit 5e67d9ccd1.

* Revert "im bored."

This reverts commit 070e957ce8.

* Revert "Update expression_build.cpp"

This reverts commit b94d092fc5.

* Revert "fixes"

This reverts commit f3d871f60a.

* Revert "fix unnecessary copy-constructors (no speed increase)"

This reverts commit 9100725802.

* Keep the random inoffensive changes

* Revert "Update type_analysis.cpp"

This reverts commit d2456a5c75.

* Update type_analysis.cpp
2022-04-17 21:11:09 -04:00
water111 b263e33e94 [goalc] fix mod bug and add div tests (#1296)
* fix mod bug and add div tests

* update changelog
2022-04-11 20:53:24 -04:00
ManDude a7eee4fdc9 [game] pc port progress menu (#1281)
* fix typo

* more typo

* shorten discord rpc text

* allow expanding enums after the fact (untested)

* make `game_text` work similar to subtitles

* update progress decomp

* update some types + `do-not-decompile` in bitfield

* fixes and fall back to original progress code

* update `progress` decomp with new enums

* update config files

* fix enums and debug menu

* always allocate (but not use) a lot of particles

* small rework to display mode options

* revert resolution/aspect-ratio symbol mess

* begin the override stuff

* make `progress-draw` more readable

* more fixes

* codacy good boy points

* first step overriding code

* finish progress overrides, game options menu fully functional!

* minor fixes

* Update game.gp

* Update sparticle-launcher.gc

* clang

* change camera controls text

* oops

* some cleanup

* derp

* nice job

* implement menu scrolling lol

* make scrollable menus less cramped, fix arrows

* make some carousell things i guess

* add msaa carousell to test

* oops

* Update progress-pc.gc

* make `pc-get-screen-size` (untested)

* resolution menu

* input fixes

* return when selecting resolution

* scroll fixes

* Update progress-pc.gc

* add "fit to screen" button

* bug

* complete resolutions menu

* aspect ratio menu

* subtitles language

* subtitle speaker

* final adjustments

* ref test

* fix tests

* fix ref!

* reduce redundancy a bit

* fix mem leaks?

* save settings on progress exit

* fix init reorder

* remove unused code

* rename goal project-like files to the project extension

* sha display toggle

* aspect ratio settings fixes

* dont store text db's in compiler

* properly save+load native aspect stuff
2022-04-11 18:38:54 -04:00
water111 1db96c72ab [goalc] macro expansion in integer constants (#1282)
* [goalc] macro expansion in integer constants

* working

* didn't break it yet

* support conditional compilation

* fix up some more small bugs

* fix duplicate evaluation of bitfield definitions

* paranoid
2022-04-07 19:13:22 -04:00
water111 5bd0b735a5 Add extractor tool (#1276)
* first attempt

* fix

* zip to tar

* windows

* try again, std::filesystem sucks

* std::filesystem is still garbage

* std::filesystem is terrible

* std::filesystem continues to waste my time

* again

* neadsflaldksal;df
2022-04-03 19:17:03 -04:00
Tyler Wilding 014da6f59d game: auto-save pc-settings to user's home directory as well as memcard files (#1233)
* cmake: reduce warning spam especially from libs

* runtime: add FS helper functions

* game: save/restore pc-settings | add original aspect option

* game: overwrite unloadable settings with defaults

* temp: unable to set the games aspect-ratio in the boot else crash?

* runtime: save memcard files to user directory as well

* runtime: fix `pckernel` load order which resolves setting the orig aspect ratio

* lint: format

* cmake: revert warning suppression, it's just causing problems it seems

* fix the order of the rest of `pckernel` and creation of obj file paths

* lint: formatting

* game: don't save settings on startup even if they are corrupted
2022-03-20 20:29:44 -04:00
Tyler Wilding 8574e10653 Mark OpenGOAL regions in IR2 output and cleanup some now irrelevant settings (#1237)
* remove redundant vscode settings and duplicate goalc atoms

* decompiler: add some region markers for OpenGOAL blocks
2022-03-17 21:39:11 -04:00