This attempts to get into master whatever work was done in this PR /
it's earlier PR https://github.com/open-goal/jak-project/pull/3965
I don't want this work to be lost / floating around in massive PRs.
However the changes are:
- switch to ntsc_v1 instead of PAL as the development target, as we have
done for all other games
- remove most of the copied-from-jak2/3 changes as they need to be
confirmed during the decompilation process not just assumed
- avoids committing any changes to `game/kernel/common` as it was not
clear to me if these were changes made in jak x's kernel that were not
properly broken out into it's own functions. We don't want to
accidentally introduce bugs into jak1-3's kernel code.
- in other words, if the change in the kernel only happens in jak x...it
should likely be specific to jak x's kernel, not common.
---------
Co-authored-by: VodBox <dillon@vodbox.io>
Co-authored-by: yodah <greenboyyodah@gmail.com>
This adds more recognition for inlined vector functions to the
decompiler, which can clean up a bunch of ugly looking code/`rlet`s.
Unfortunately, this changes the numbering of ops in the decomp, since
all the vector instructions get combined in a single "operation" by the
decompiler. I really tried to avoid having this ever happen in the
decompiler and this is one of the few cases where it has. So I had to
update a bunch of type casts.
For that reason I haven't turned this on in Jak 2 yet, although I am
planning to do that at some point. (probably at the same time as porting
back a bunch of jak 3 improvements to jak 2)
---------
Co-authored-by: water111 <awaterford1111445@gmail.com>
This PR does a few cleanups:
- improve method names/comments/flags for `enemy.gc` and a few other
files
- fix `new-stack-matrix0` not working for jak 3
- add `matrix-copy!` detection for jak 3
- add `vector-copy!` detection
---------
Co-authored-by: water111 <awaterford1111445@gmail.com>
This includes all the collision stuff needed to spawn `target`,
decompiles the sparticle code and adds some of the PC hacks needed for
merc to run (it doesn't work quite right and looks bad, likely due to a
combination of code copied from Jak 2 and the time of day hacks).
There are a bunch of temporary hacks (see commits) in place to prevent
the game from crashing quite as much, but it is still extremely prone to
doing so due to lots of missing functions/potentially bad decomp.
---------
Co-authored-by: water <awaterford111445@gmail.com>
Small fix to decompiler when the original compiler uses `ra` register.
This seems to happen in "normal" code very rarely - perhaps they
manually specified this in an `rlet`.
Start figuring out buckets/textures for Jak 3.
The foreground code is not yet modified for PC - I want to wait until
the game is running, since it is too hard to do it now.
This updates `fmt` to the latest version and moves to just being a copy
of their repo to make updating easier (no editing their cmake / figuring
out which files to minimally include).
The motivation for this is now that we switched to C++ 20, there were a
ton of deprecated function usages that is going away in future compiler
versions. This gets rid of all those warnings.
- `speech`
- `ambient`
- `water-h`
- `vol-h`
- `generic-obs`
- `carry-h`
- `pilot-h`
- `board-h`
- `gun-h`
- `flut-h`
- `indax-h`
- `lightjak-h`
- `darkjak-h`
- `target-util`
- `history`
- `collide-reaction-target`
- `logic-target`
- `sidekick`
- `projectile`
- `voicebox`
- `ragdoll-edit`
- most of `ragdoll` (not added to gsrc yet)
- `curves`
- `find-nearest`
- `lightjak-wings`
- `target-handler`
- `target-anim`
- `target`
- `target2`
- `target-swim`
- `target-lightjak`
- `target-invisible`
- `target-death`
- `target-gun`
- `gun-util`
- `board-util`
- `target-board`
- `board-states`
- `mech-h`
- `vol`
- `vent`
- `viewer`
- `gem-pool`
- `collectables`
- `crates`
- `secrets-menu`
Additionally:
- Detection of non-virtual state inheritance
- Added a config file that allows overriding the process stack size set
by `stack-size-set!` calls
- Fix for integer multiplication with `r0`
- Fixed detection for the following macros:
- `static-attack-info`
- `defpart` and `defpartgroup` (probably still needs adjustments, uses
Jak 2 implementation at the moment)
- `sound-play` (Jak 3 seems to always call `sound-play-by-name` with a
`sound-group` of 0, so the macro has been temporarily defaulted to use
that)
One somewhat significant change made here that should be noted is that
the return type of `process::init-from-entity!` was changed to `object`.
I've been thinking about this for a while, since it looks a bit nicer
without the `(none)` at the end and I have recently encountered init
methods that early return `0`.
I finally read through all the joint code and wrote up some
documentation. I think this will be really helpful when we try to
understand all the functions in `process-drawable`, or if somebody ever
wants to import/export animations.
This switches all three games to using a new faster GOAL joint
decompressor. It is on by default, but you can go back to the old
version by setting `*use-new-decompressor*` to false.
Also fix the log-related crash, fix the clock speed used in timer math.
- `transformq`
- `wind-work`
- `progress-static`
- `merc-vu1`
- `emerc-vu1`
- `merc`
- `emerc`
- `cloth-h`
- Most of `cloth`, not added to gsrc yet
Art group dumps were incorrect for Jak 3 because the master art group
fields were at different offsets.
- `pat-h`
- `engines`
- `res-h`
- `res`
- `fact-h`
- `game-info-h`
- `wind-h`
- `merc-h`
- `shadow-vu1-h`
- `shadow-cpu-h`
- `dynamics-h`
- `memcard-h`
- `surface-h`
- part of `gui-h`
- `ambient-h`
- `speech-h`
- `prototype-h`
- `smush-control-h`
- `generic-merc-h`
- `generic-work-h`
- `collide-func-h`
- `collide-mesh-h`
- `collide-shape-h` (only missing the `new` method for `collide-shape`
because we don't have `process-drawable` yet and that also needs joint
stuff etc.)
- `collide-touch-h`
- `collide-edge-grab-h`
- `lightning-h`
This also adds argument name remaps for the `relocate` and `mem-usage`
methods (Jak 1 and 2 ref tests were updated, but not the gsrc).
Decompile `gcommon`. I adjusted the spacing of docstring comments, and
removed some spammy decompiler warning prints.
I also added some random notes I had on VU programs from jak1/jak2. They
are not polished, but I think it's still worth including since we'll
have to go through them again for jak 3.
In #3300, `per_object` in `ObjectFileDB::ir2_analyze_all_types()` was
changed to an `unordered_map`, which messed with the object file order
in `all-types`. This fixes the issue by going through each object file
in order and then looking up the type info from the map instead of
iterating through the map.
This also removes the method IDs from the old method comments and adds a
stub for the `new` method if there is an old method comment for it, in
order to stay consistent with the rest of the method list.
This sets out the bones of a Jak 3 build, many things are stubbed out,
guessed, or copied from Jak 2 but it should at least be good enough to:
run `task set-game-jak3`
launch the repl
run builds from the repl
build outputs themselves are untested but the build itself runs without
errors
---------
Co-authored-by: Tyler Wilding <xtvaser@gmail.com>
The logger used in `goalc` tries to print an already-formatted string
`message` using `fmt::print(message);` Usually this doesn't cause
problems, but if you try to print, for example, an exception that has
special characters (notably `{`) it will try to do another round of
formatting/replacements, despite not having any args to replace with,
which ends up throwing another exception. This is why errors when
parsing custom level JSON cause the REPL to exit.
I've hopefully identified all the various instances of this across the
codebase
Major change to how `deftype` shows up in our code:
- the decompiler will no longer emit the `offset-assert`,
`method-count-assert`, `size-assert` and `flag-assert` parameters. There
are extremely few cases where having this in the decompiled code is
helpful, as the types there come from `all-types` which already has
those parameters. This also doesn't break type consistency because:
- the asserts aren't compared.
- the first step of the test uses `all-types`, which has the asserts,
which will throw an error if they're bad.
- the decompiler won't emit the `heap-base` parameter unless necessary
now.
- the decompiler will try its hardest to turn a fixed-offset field into
an `overlay-at` field. It falls back to the old offset if all else
fails.
- `overlay-at` now supports field "dereferencing" to specify the offset
that's within a field that's a structure, e.g.:
```lisp
(deftype foobar (structure)
((vec vector :inline)
(flags int32 :overlay-at (-> vec w))
)
)
```
in this structure, the offset of `flags` will be 12 because that is the
final offset of `vec`'s `w` field within this structure.
- **removed ID from all method declarations.** IDs are only ever
automatically assigned now. Fixes#3068.
- added an `:overlay` parameter to method declarations, in order to
declare a new method that goes on top of a previously-defined method.
Syntax is `:overlay <method-name>`. Please do not ever use this.
- added `state-methods` list parameter. This lets you quickly specify a
list of states to be put in the method table. Same syntax as the
`states` list parameter. The decompiler will try to put as many states
in this as it can without messing with the method ID order.
Also changes `defmethod` to make the first type definition (before the
arguments) optional. The type can now be inferred from the first
argument. Fixes#3093.
---------
Co-authored-by: Hat Kid <6624576+Hat-Kid@users.noreply.github.com>
Some files were in the `banned_objects` list and were thus excluded from
the `all_objs` file.
Also implements the `pexcw` instruction which is only used in `hfrag`
code.
Started at 349,880,038 allocations and 42s
- Switched to making `Symbol` in GOOS be a "fixed type", just a wrapper
around a `const char*` pointing to the string in the symbol table. This
is a step toward making a lot of things better, but by itself not a huge
improvement. Some things may be worse due to more temp `std::string`
allocations, but one day all these can be removed. On linux it saved
allocations (347,685,429), and saved a second or two (41 s).
- cache `#t` and `#f` in interpreter, better lookup for special
forms/builtins (hashtable of pointers instead of strings, vector for the
small special form list). Dropped time to 38s.
- special-case in quasiquote when splicing is the last thing in a list.
Allocation dropped to 340,603,082
- custom hash table for environment lookups (lexical vars). Dropped to
36s and 314,637,194
- less allocation in `read_list` 311,613,616. Time about the same.
- `let` and `let*` in Interpreter.cpp 191,988,083, time down to 28s.
Adds controller LED features to Jak 2:
- progressive flickering denoting health
- copies tomb simon says puzzle colors
- unique colors for each gun
- orange color for being indax
- yellow color for being in mech
- purple color for being darkjak
- blue color for being in board
- red flash when wanted.
May add more features later?
Also did some minor clean-up on some types.
- state handlers that are not inlined lambdas have smarter type
checking, getting rid of 99.9% of the casts emitted (they were not
useful)
- art groups were not being properly linked to their "master" groups.
- `max` in `ja` in Jak 2 was not being detected.
Another huge PR...
Previously, `object` and `none` were both top-level types. This made
decompilation rather messy as they have no LCA and resulted in a lot of
variables coming out as type `none` which is very very wrong and
additionally there were plenty of casts to `object`. This changes it so
`none` becomes a child of `object` (it is still represented by
`NullType` which remains unusable in compilation).
This change makes `object` the sole top-level type, and the type that
can represent *any* GOAL object. I believe this matches the original
GOAL built-in type structure. A function that has a return type of
`object` can now return an integer or a `none` at the same time.
However, keep in mind that the return value of `(none)` is still
undefined, just as before. This also makes a cast to `object`
meaningless in 90% of the situations it showed up in (as every single
thing is already an `object`) and the decompiler will no longer emit
them. Casts to `none` are also reduced. Yay!
Additionally, state handlers also don't get the final `(none)` printed
out anymore. The return type of a state handler is completely
meaningless outside the event handler (which is return type `object`
anyway) so there are no limitations on what the last form needs to be. I
did this instead of making them return `object` to trick the decompiler
into not trying to output a variable to be used as a return value
(internally, in the decompiler they still have return type `none`, but
they have `object` elsewhere).
Fixes#1703Fixes#830Fixes#928
This PR adds detection of the `launch-particles` and `seconds-per-frame`
macros to the decompiler, removing a lot of bloat and hiding many
process register uses.
I also added `og:preserve-this` comments to as many manual patches and
comments as I could, which will soon be used in conjunction with CI to
hopefully catch any regressions in future big decomp update PRs.
I have some concerns about the `launch-particles` macro (more details in
`sparticle-launcher.gc`) , but thus far, I have not seen anything break
yet.
---------
Co-authored-by: water <awaterford111445@gmail.com>
Adds sprite distort, fixes buggy sprite rendering in progress, adds
scissoring support (used in various scrolling menus) and a very basic
implementation of `blit-displays`. This is enough to make the fade
effect in the progress menu work, along with all the menus working
properly without needing to use the REPL. This does not make screen
flipping and the filter when failing a mission work.
Added support in the decompiler for detecting `dma-buffer-add-gs-set`
and `dma-buffer-add-gs-set-flusha` and updated all of the Jak 2 code to
use it. Readability improved!
Fixes decompiler issue with `with-dma-buffer-add-bucket` not inlining
forms which broke syntax. Fixes store error warnings showing up for
non-existent stores, there is now a dedicated pass for this at the end.
I started work on making `BITBLTBUF` stuff work in the DirectRenderer,
but stopped for now because it wasn't strictly necessary. It will still
assert like before.
Updates the decompiler for the new format and there's new macros. This
new format should be easier to read/parse.
Also rewrote `sp-init-fields!` (both jak 1 and 2) from assembly to GOAL.
Hopefully I did not miss any regressions in Jak 1/2 while updating the
files, it's a lot.
water111