I did some manual modifications in a few places to work around some
truly mysterious control flow, and some unsupported stack array stuff.
Also fixes a bug in decompiling static improper lists.
As far as I can tell, guns work, other than some graphical issues and
the crazy particle spawning issue, but I strongly suspect these are
problems with the sparticle/graphics side.
- Split up DGOs between threads in the multithreaded offline test
- fix some random warnings
- make the sig paths decompile a bit nicer to make some files smaller
Running reference tests/decompiler should now be possible on macos
(arm). Most of the changes were just cleaning up places where we were
sloppy with ifdefs, but there were two interesting ones:
- `Printer.cpp` was updated to not use a recursive function for printing
lists, to avoid stack overflow
- I replaced xxhash with another version of the same library that
supports arm (the one that comes in zstd). The interface is C instead of
C++ but it's not bad to use. I confirmed that the extractor succeeds on
jak 1 iso so it looks like this gives us the same results as the old
library.
A big one...
I figure even if we would like to change the way the particle/scene code
is output -- it'd be easier to find patterns with it all decompiled.
I've updated my script so it can easily be used to mass update these
files:
```bash
task update-gsrc-glob GLOB="**/*-part*.gc"
```
> for example will update gsrc files with `part` in their name -- if
they are in ref tests (so uncompleted ones aren't touched)
I found a few issues along the way that I'll have to make issues for
soon.
This solves two main problems:
- the looming threat of running out of memory since every thread would
consume duplicate (and probably not needed) resources
- though I will point out, jak 2's offline tests seem to hardly use any
memory even with 400+ files, duplicated across many threads. Where as
jak 1 does indeed use tons more memory. So I think there is something
going on besides just the source files
- condense the output so it's much easier to see what is happening / how
close the test is to completing.
- one annoying thing about the multiple thread change was errors were
typically buried far in the middle of the output, this fixes that
- refactors the offline test code in general to be a lot more modular
The pretty printing is not enabled by default, run with `-p` or
`--pretty-print` if you want to use it
https://user-images.githubusercontent.com/13153231/205513212-a65c20d4-ce36-44f6-826a-cd475505dbf9.mp4
Mostly easy / particle def files. I did a bit in `ruins` but CFG
failures and missing `drawable` functions make it untestable for now so
I've put that on pause.
- target-part
- ;; ERROR: Failed to convert to atomic ops: Variable could not be
constructed from register r0 in `process-drawable-shock-wall-effect`
- gun-part
- missing sparticle decompiling case it seems related to `L155`
This is pretty rough but...im excited to see it working :)
The VU programs for the ocean renderer have changed a bit and
`ocean-texture` has a bunch of new stuff, otherwise things are
relatively similar to Jak 1.
This is the first time I used mips2c and I'm not sure I did it 100%
right, so that should be double-checked.
The only interesting one is `collide-hash`, which is untested.
The other two are very likely unused. I skipped the annoying code in
`collide-probe` because it's not used and the same as jak 1.
Offline tests ignore comments in their comparison, but there's no reason
to strip them from the file that goes into the reference test folder
when doing the typical update routine.
This just generates superfluous diffs for all the files already done
prior to this change and is meaningless (the lines are dropped anyway)
Couldn't finish any of the enemy/nav-enemy related files for one reason
or another, but quite a bit of work that will be easier to merge and
iterate on instead of keeping track of the branch.
enemy/idle-control has some very weird focus related code.
nav-mesh/nav-control still has a bunch of CFG resolution problems that
need to be manually resolved.
Co-authored-by: water <awaterford111445@gmail.com>
- fix issue described in
https://github.com/open-goal/jak-project/issues/1939
- fix `text`, which was manually patched with the wrong offset (was
reading the symbol value off by one byte)
- clean up some random useless prints
- make the offline tests keep trying if there's a comparison error,
clean up the output a bit so the diffs are all at the end.
The offline-tests are going to end up taking too long for jak 2, I did
some rough math and by the end of it we'll be spending almost 2 minutes
for a full offline test on my machine.
These changes allow us to throw hardware at the problem
Still some work to do to make the output nicer, but seems to be fairly
reliable. By default it still uses 1 thread, use `num_threads` to change
this.
Almost done:
- `target-handler` (`(none)` event handler casts and CFG error)
- `target2` (`(none)` event handler casts)
- `powerups` (`cloud-track` does some weird stuff with `handle`s)
- `gun-states` (CFG error)
Some progress in:
- `water-flow`
Additionally:
- Clean up the two year old Jak 3 config file and add a config skeleton
(disassembling seems to not have worked, but I was able to dump obj
files and the `all_scripts` file)
- Fix automatic skelgroup detection and `defskelgroup` macro for Jak 2
(closes#1950)
- When a function decompiles without any major errors, a warning is
generated with the op id for each unresolved load and store that will
likely fail to compile (closes#1933)
This PR does a few main things:
- finish decompiling the progress related code
- implemented changes necessary to load the text files end-to-end
- japanese/korean character encodings were not added
- finish more camera code, which is required to spawn the progress menu
/ init the default language settings needed for text
- initialized the camera as well
Still havn't opened the menu as there are a lot of checks around
`*target*` which I havn't yet gone through and attempted to comment out.
A few issues:
- lwidea's fr3 is getting loaded and unloaded all the time
- the debug line drawing clipping is wrong (doesn't seem wrong in pcsx2,
so I think this is on us)
- nothing actually using vis data yet
- at a large distance, our view frustum culling seems slightly too
aggressive (might be that viewport scissoring is wrong)
- in the city, things seem darker as you move away. unclear how this is
happening (fog?)
And everything else needed for them!
A couple functions are bad currently.
- fixes#1929 - untested on linux
- fixes#1924 - now you need to type `,` before a lambda you want to put
in a pair.
- fix debugger symbol table in jak 2
- made the decompiler output `(meters 2)` instead of `(meters 2.0)`
- fixed a bug with the bitfield enum special -1 case
- made bad game text decomp not exit the decompiler
- added `editable-player` and `script`
A small quality of life increase that is more impactful since jak 2 has
double the file count.
I often use the offline-tests to find compiler errors / automatically
resolve them when i am finalizing a file. As more and more files are
completed this becomes increasingly more inefficient. When I know that
only 1 file needs to be decompiled / compared / compiled, I'd prefer to
have a feature like this.
- decompile `subdivide`, `wind-work`, `tie-work`, `bsp`, `focus`
- support `ppacb` in compiler
- don't assert when bitfield stuff fails due to constant propgataion
weirdness
- finish up history
- div/mod unsigned assert fix in decompiler
- empty assert fix in decompiler for failed `add` type prop
- make jak 1 performance counters "work" (just measure time)
- fix cast/typos on pcgtb/vftoi15
I wasn't able to 100% complete `debug` due to a bunch of level boundary
debug stuff I couldn't figure out. But, I added a ref test and
documented/copied over everything else into goal_src.
Most of the functions that existed in Jak 1 are identical, others got an
extra param or 2 and some now make copies of arguments instead of
modifying them. There's a bunch of new functions, including all of the
debug functions that used to be in `level-boundary` are now in `debug`.
At the very end of `debug` there's also some weird asm functions
checking for some EE memory controller bug (not sure what's up with
that)?
water111
