This updates `fmt` to the latest version and moves to just being a copy
of their repo to make updating easier (no editing their cmake / figuring
out which files to minimally include).
The motivation for this is now that we switched to C++ 20, there were a
ton of deprecated function usages that is going away in future compiler
versions. This gets rid of all those warnings.
Two main problems fixed here:
- offline tests will fail on a comparison failure (a mistake from the
re-write)
- art-group-info is committed to the repo and shared with every thread
(running the tests with 1 thread, for example on the CI, and locally
were producing different results)
art files are still not provided to the jak2 offline tests:
- `*-ag` files are not being output
- `art-elts.gc` is not complete, as a handful of files claim to be
missing stuff
lastly, in jak1's offline tests we were also running `tpage` and `*-vis`
files through the decompiler. This omits that (they came from the
`all_objs.json` file) -- is this an issue?
Running reference tests/decompiler should now be possible on macos
(arm). Most of the changes were just cleaning up places where we were
sloppy with ifdefs, but there were two interesting ones:
- `Printer.cpp` was updated to not use a recursive function for printing
lists, to avoid stack overflow
- I replaced xxhash with another version of the same library that
supports arm (the one that comes in zstd). The interface is C instead of
C++ but it's not bad to use. I confirmed that the extractor succeeds on
jak 1 iso so it looks like this gives us the same results as the old
library.
This solves two main problems:
- the looming threat of running out of memory since every thread would
consume duplicate (and probably not needed) resources
- though I will point out, jak 2's offline tests seem to hardly use any
memory even with 400+ files, duplicated across many threads. Where as
jak 1 does indeed use tons more memory. So I think there is something
going on besides just the source files
- condense the output so it's much easier to see what is happening / how
close the test is to completing.
- one annoying thing about the multiple thread change was errors were
typically buried far in the middle of the output, this fixes that
- refactors the offline test code in general to be a lot more modular
The pretty printing is not enabled by default, run with `-p` or
`--pretty-print` if you want to use it
https://user-images.githubusercontent.com/13153231/205513212-a65c20d4-ce36-44f6-826a-cd475505dbf9.mp4
The only interesting one is `collide-hash`, which is untested.
The other two are very likely unused. I skipped the annoying code in
`collide-probe` because it's not used and the same as jak 1.
Offline tests ignore comments in their comparison, but there's no reason
to strip them from the file that goes into the reference test folder
when doing the typical update routine.
This just generates superfluous diffs for all the files already done
prior to this change and is meaningless (the lines are dropped anyway)
Couldn't finish any of the enemy/nav-enemy related files for one reason
or another, but quite a bit of work that will be easier to merge and
iterate on instead of keeping track of the branch.
enemy/idle-control has some very weird focus related code.
nav-mesh/nav-control still has a bunch of CFG resolution problems that
need to be manually resolved.
Co-authored-by: water <awaterford111445@gmail.com>
- fix issue described in
https://github.com/open-goal/jak-project/issues/1939
- fix `text`, which was manually patched with the wrong offset (was
reading the symbol value off by one byte)
- clean up some random useless prints
- make the offline tests keep trying if there's a comparison error,
clean up the output a bit so the diffs are all at the end.
The offline-tests are going to end up taking too long for jak 2, I did
some rough math and by the end of it we'll be spending almost 2 minutes
for a full offline test on my machine.
These changes allow us to throw hardware at the problem
Still some work to do to make the output nicer, but seems to be fairly
reliable. By default it still uses 1 thread, use `num_threads` to change
this.
Almost done:
- `target-handler` (`(none)` event handler casts and CFG error)
- `target2` (`(none)` event handler casts)
- `powerups` (`cloud-track` does some weird stuff with `handle`s)
- `gun-states` (CFG error)
Some progress in:
- `water-flow`
Additionally:
- Clean up the two year old Jak 3 config file and add a config skeleton
(disassembling seems to not have worked, but I was able to dump obj
files and the `all_scripts` file)
- Fix automatic skelgroup detection and `defskelgroup` macro for Jak 2
(closes#1950)
- When a function decompiles without any major errors, a warning is
generated with the op id for each unresolved load and store that will
likely fail to compile (closes#1933)
A small quality of life increase that is more impactful since jak 2 has
double the file count.
I often use the offline-tests to find compiler errors / automatically
resolve them when i am finalizing a file. As more and more files are
completed this becomes increasingly more inefficient. When I know that
only 1 file needs to be decompiled / compared / compiled, I'd prefer to
have a feature like this.
* fix utf-8 handling around env-vars
* fix file opening errors related to unicode
* add uncaught exception handler in `gk` to ensure something is logged
* gracefully fail if window icon cant be loaded and work with unicode
* linux fix and add changes to vendor file
* git: ignore vs build dir
* cmake: ditch `clang-cl` on windows in favor of actual `clang`
* build: suppress a significant number of warnings
* build: adjust workflows and vendor nasm
* docs: update docs to remove `clang-cl` mentions
* tests: move jak1 reference tests into their own folder
* tests: update offline tests to support multiple games
* tests: some additional fixes and multi-game handling
* tests: update reference tests
* extractor: refactor and cleanup for multi-game support
* deps: switch to `ghc::filesystem` as it is utf-8 everywhere by default
* extractor: finally working with unicode
* unicode: fix unicode cli args on windows in all `main` functions
* Remove assets folder, use more std::filesystem
* windows fix
* another one for windows
* another one
* better system for different folders
* rm debugging stuff
* let extractor override everything
* dont revert jak1 change
* update refs
* [decompiler] read and process art groups
* finish decompiler art group selection & detect in `ja-group?`
* make art stuff work on offline tests!
* [decompiler] detect `ja-group!` (primitive)
* corrections.
* more
* use new feature on skel groups!
* find `loop!` as well
* fully fledged `ja` macro & decomp + `loop` detect
* fancy fixed point printing!
* update source
* `:num! max` (i knew i should've done this)
* Update jak1_ntsc_black_label.jsonc
* hi imports
* make compiling the game work
* fix `defskelgroup`
* clang
* update refs
* fix chan
* fix seek and finalboss
* fix tests
* delete unused function
* track let rewrite stats
* reorder `rewrite_let`
* Update .gitattributes
* fix bug with `:num! max`
* Update robotboss-part.gc
* Update goal-lib.gc
* document `ja`
* get rid of pc fixes thing
* use std::abs
* first attempt
* fix
* zip to tar
* windows
* try again, std::filesystem sucks
* std::filesystem is still garbage
* std::filesystem is terrible
* std::filesystem continues to waste my time
* again
* neadsflaldksal;df
* ci: switch to codacy for coverage
* docs: update badges
* decomp: allow overriding config flags via CLI
* cleanup: top level file cleanup
* docs: big README overhaul
Attempt to close#1128 and #1086
* decomp: attempt to detect if `iso_data` is missing or wrongly extracted
* game: switch to `fpng` for screenshots, allow for compression
closes#1035
* game: switch vsync control to a checkbox
* lint: format cpp files
* lint: format json files
* docs/scripts: organize taskfile
* decomp: mostly finish `cam-master`
* decomp/scripts: lots of work in cam-states
* stash
* Merge remote-tracking branch 'water111/master' into decomp/camera-master
Updated submodule third-party/googletest
* decompiler: Add support for non power of 2 offsets for inline arr access
* decomp: mostly finish `cam-states` need to fix a macro issue
* blocked: `cam-master` decompiler crash when adding casts
* decomp: finish `cam-states-dbg`
* decomp: mostly finish `pov-camera` with the exception of joint-related code
* decomp: `cam-debug` finished decompiling, no way does this compile yet though
* decomp: considerable work done in `cam-layout`
* decomp: `cam-layout` almost done!
* decomp: `pov-camera` finished, TC tests will fail for now
* decomp: working on resolving issues
* decomp: cam-layout decompiling
* fixing more issues in cam-master...one event handler remains
* skip problematic function in `cam-master` for now
* gsrc: update res macros
* decomp: finish `cam-states`
* decomp: giving up on `cam-debug`
* tests: allow skipping state handlers in ref tests
* decomp: working through cam-layout bugs
* decomp: allow for shifting non-integers
* decomp: finalize `cam-layout` and `cam-master`
* decomp: finalize `cam-states`
* cleanup: bi-annual formatting of the casting files
* formatting
* address feedback - leave the float labels alone for now
* address feedback
* linting/formatting
* update gsrc and ref tests
Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>
* time of day
* goal code seems to work
* stars at wrong spot
* stars and sun work
* debugging clouds
* fix texture correction
* sky works
* cleanup, add profiler
* clean up
* final clean up
* offline tests
* missing include
* blocked: `puffer` has state decomp issues
* blocked: decomp crash when adding process-drawble cast in `helix-water`
* blocked: `green-eco-lurker` no nice way to cast value out of EMB
* decomp: started on `citadel-sages`
* decomp: `seagull` blocked state decomp
* blocked: `sunken-pipegame` runs into handle related issues
* blocked: `snow-ram` and `snow-ram-boss` state decomp
* decomp: `rolling-race-ring` blocked, handle casts
* blocked: state decomp in `part-tester`
* decomp: started `joint-exploder`
* blocked: `anim-tester` has rare cmove-#f-zero usage
* decomp: I lied! `anim-tester` finished
* config: update banned list
* `flying-lurker`
* why'd this get left behind?
* also `ambient` and add a `banned_objects` config and fix offline tests
* clang
* fix `ambient-type-sound`
* mips 2 c basic version, not yet tested
* calling works without crashing, but the function doesn't
* it works
* add test
* cleanup and actually add the test
* dont use mips2c by default for font
* clean up formatting
Tyler Wilding
water111
Hat Kid
ManDude