Commit Graph

84 Commits

Author SHA1 Message Date
Tyler Wilding eb703ee96e
REPL related improvements and fixes (#3545)
Motivated by - https://github.com/open-goal/opengoal-vscode/pull/358

This addresses the following:
- Fixes #2939 spam edge-case
- Stop picking a different nREPL port based on the game mode by default,
this causes friction for tools in the average usecase (having a REPL
open for a single game, and wanting to connect to it). `goalc` spins up
fine even if the port is already bound to.
- For people that need/want this behaviour, adding per-game
configuration to the `repl-config.json` is on my todo list.
- Allows `goalc` to permit redefining symbols, including functions. This
is defaulted to off via the `repl-config.json` but it allows you to for
example, change the definition of a function without having to restart
and rebuild the entire game.
![Screenshot 2024-06-02
124558](https://github.com/open-goal/jak-project/assets/13153231/28f81f6e-b7b8-4172-9787-f96e4ab1305b)
- Updates the welcome message to include a bunch of useful metadata
up-front. Cleaned up all the startup logs that appear when starting
goalc, many of whom's information is now included in the welcome
message.
  - Before:

![image](https://github.com/open-goal/jak-project/assets/13153231/814c2374-4808-408e-9ed6-67114902a1d9)

  - After:
![Screenshot 2024-06-01
235954](https://github.com/open-goal/jak-project/assets/13153231/f3f459fb-2cbb-46ba-a90f-318243d4b3b3)
2024-06-03 00:14:52 -04:00
Tyler Wilding 53277a65ad
LSP: A bunch of new OpenGOAL language features (#3437)
- Integrate the AST into the LSP, this makes parsing and tokenizing the
files much easier
- Consolidate most of the symbol info tracking in `goalc` to a single
map. Fixed some issues where the old map would never evict symbols when
re-compiling files. There is still some more to cleanup, but this now
can be used as an incrementally updated source-of-truth for the LSP
- re-compile files when they are saved. Ideally this would be done
everytime they are changed but that:
  - may be too aggressive
- goalc doesn't compile incrementally yet so it likely would be a worse
UX

Features added, see
https://github.com/open-goal/opengoal-vscode/issues/256
- Hover

![image](https://github.com/open-goal/jak-project/assets/13153231/58dadb5d-582c-4c1f-9ffe-eaa4c85a0255)

![image](https://github.com/open-goal/jak-project/assets/13153231/b383adde-57fc-462c-a256-b2de5c30ca9a)
- LSP Status fixed
- Type Hierarchy

![image](https://github.com/open-goal/jak-project/assets/13153231/8e681377-1d4e-4336-ad70-1695a4607340)
- Document Color

![image](https://github.com/open-goal/jak-project/assets/13153231/4e48ccd8-0ed1-4459-a133-5277561e4201)
- Document Symbols
![Screenshot 2024-03-27
004105](https://github.com/open-goal/jak-project/assets/13153231/8e655034-43c4-4261-b6e0-85de00cbfc7f)
- Completions
![Screenshot 2024-03-30
004504](https://github.com/open-goal/jak-project/assets/13153231/d123a187-af90-466b-9eb7-561b2ee97cd1)

---------

Co-authored-by: Hat Kid <6624576+Hat-Kid@users.noreply.github.com>
2024-03-30 19:49:07 -04:00
Hat Kid dacb704ef6
decomp3: more engine stuff, fix `ja` macro detection for jak 2/3, unmerged `let` matcher, `part-tracker-spawn` macro (#3436)
- `aligner`
- `effect-control`
- `pov-camera`
- `powerups`
- `los-control-h`
- `airlock`
- `water-anim`
- `blocking-plane`
- `proc-focusable-spawner`
- `idle-control`
- `enemy-h`
- `nav-enemy-h`
- `enemy`
- `enemy-states`
- `particle-curves`
- `base-plat`
- `plat`
- `bouncer`
- `elevator`
- `rigid-body`
- `rigid-body-queue`
- `process-taskable`
- `scene-actor`
- `warp-gate`
- `guard-projectile`
- `metalhead-projectile`
- `los-control`
- `joint-exploder`
- `ragdoll-test`
- `debris`
- `shield-sphere`
- `text`
- `target-launch`
2024-03-30 10:28:02 -04:00
Tyler Wilding 60db0e5ef9
deps: update `fmt` to latest version (#3403)
This updates `fmt` to the latest version and moves to just being a copy
of their repo to make updating easier (no editing their cmake / figuring
out which files to minimally include).

The motivation for this is now that we switched to C++ 20, there were a
ton of deprecated function usages that is going away in future compiler
versions. This gets rid of all those warnings.
2024-03-05 22:11:52 -05:00
Matt Dallmeyer 2071c98b55
Fix cases of string formatting with non string literals (#3304)
The logger used in `goalc` tries to print an already-formatted string
`message` using `fmt::print(message);` Usually this doesn't cause
problems, but if you try to print, for example, an exception that has
special characters (notably `{`) it will try to do another round of
formatting/replacements, despite not having any args to replace with,
which ends up throwing another exception. This is why errors when
parsing custom level JSON cause the REPL to exit.

I've hopefully identified all the various instances of this across the
codebase
2024-01-14 12:02:08 +00:00
ManDude 09536c68ac
[compiler] asm-only disasm output + fix spacing bug (#3104) 2023-10-29 10:16:14 +00:00
water111 395c98db19
[goalc] Cleaned up speedups (#3066)
Started at 349,880,038 allocations and 42s

- Switched to making `Symbol` in GOOS be a "fixed type", just a wrapper
around a `const char*` pointing to the string in the symbol table. This
is a step toward making a lot of things better, but by itself not a huge
improvement. Some things may be worse due to more temp `std::string`
allocations, but one day all these can be removed. On linux it saved
allocations (347,685,429), and saved a second or two (41 s).
- cache `#t` and `#f` in interpreter, better lookup for special
forms/builtins (hashtable of pointers instead of strings, vector for the
small special form list). Dropped time to 38s.
- special-case in quasiquote when splicing is the last thing in a list.
Allocation dropped to 340,603,082
- custom hash table for environment lookups (lexical vars). Dropped to
36s and 314,637,194
- less allocation in `read_list` 311,613,616. Time about the same.
- `let` and `let*` in Interpreter.cpp 191,988,083, time down to 28s.
2023-10-07 10:48:17 -04:00
ManDude 61c4fc4b91
[jak2] fully implement collide mesh renderer (#3010)
Fixes #2983
2023-09-22 15:50:16 +01:00
ManDude b3e3f9379e
fix some jak 1 hint subtitles not showing up (#2782)
There is one instance of the same hint being played for different IDs
which was breaking things.
2023-06-28 14:13:47 +01:00
ManDude f8d4489361
fix more orb softlocks (#2767)
Fixes orb softlocks during races and other side missions.

The side mission tasks will now not count as completed until the
precursor orb has been picked up.

Races will not let you advance (or even pause the game) until the
precursor orb has been picked up.
2023-06-27 22:02:21 +01:00
Tyler Wilding 10ac78200b
repl: add `gameVersionFolder` to repl-config for running the non-default version (#2463)
Adds a decent way to customize the folders the project file expects the
iso data and decompiler data to be in. When you run any version other
than the default, for example Jak 1 PAL, it uses the `gameName`
decompiler config to consume and output it's results.

However the project file will assume `jak1` unless you hard-code it
differently -- basically, it needs to be explicitly told just the
decompiler is told what version to use.

We now have a per-user REPL Config json file, so that can be used to
override the default `jak1` behaviour.

Fixes #1993
2023-04-11 17:57:20 -04:00
ManDude 6f1cb2a0a9
fix repl buffer overrun + use a different port for each game version (#2449)
Fixes #2313
2023-04-02 05:57:21 +01:00
water111 e2b7e5c001
[goalc] reduce compiler memory usage (#2247) 2023-02-24 18:32:30 -05:00
Tyler Wilding 39658dfd71
docs: Automatically generate documentation from goal_src code (#2214)
This automatically generates documentation from goal_src docstrings,
think doxygen/java-docs/rust docs/etc. It mostly supports everything
already, but here are the following things that aren't yet complete:
- file descriptions
- high-level documentation to go along with this (think pure markdown
docs describing overall systems that would be co-located in goal_src for
organizational purposes)
- enums
- states
- std-lib functions (all have empty strings right now for docs anyway)

The job of the new `gen-docs` function is solely to generate a bunch of
JSON data which should give you everything you need to generate some
decent documentation (outputting markdown/html/pdf/etc). It is not it's
responsibility to do that nice formatting -- this is by design to
intentionally delegate that responsibility elsewhere. Side-note, this is
about 12-15MB of minified json for jak 2 so far :)

In our normal "goal_src has changed" action -- we will generate this
data, and the website can download it -- use the information to generate
the documentation at build time -- and it will be included in the site.
Likewise, if we wanted to include docs along with releases for offline
viewing, we could do so in a similar fashion (just write a formatting
script to generate said documentation).

Lastly this work somewhat paves the way for doing more interesting
things in the LSP like:
- whats the docstring for this symbol?
- autocompleting function arguments
- type checking function arguments
- where is this symbol defined?
- etc

Fixes #2215
2023-02-20 19:49:37 -05:00
ManDude 324def1303
split new pc features in some files into their own code files + address some old issues + ripple graphics improvements (#2216)
Moves PC-specific entity and debug menu things to `entity-debug.gc` and
`default-menu-pc.gc` respectively and makes `(declare-file (debug))`
work as it should (no need to wrap the entire file in `(when
*debug-segment*` now!).

Also changes the DGO descriptor format so that it's less verbose. It
might break custom levels, but the format change is very simple so it
should not be difficult for anyone to update to the new format. Sadly,
you lose the completely useless ability to use DGO object names that
don't match the source file name. The horror!

I've also gone ahead and expanded the force envmap option to also force
the ripple effect to be active. I did not notice any performance or
visual drawbacks from this. Gets rid of some distracting LOD and some
water pools appearing super flat (and pitch back for dark eco).

Fixes #1424
2023-02-13 21:39:14 +00:00
Tyler Wilding 00ac12094e
goalc/repl: cleanup of goalc/REPL code and some QoL improvements (#2104)
- lets you split up your `startup.gc` file into two sections
  - one that runs on initial startup / reloads
  - the other that runs when you listen to a target
- allows for customization of the keybinds added a month or so ago
- removes a useless flag (--startup-cmd) and marks others for
deprecation.
- added another help prompt that lists all the keybinds and what they do

Co-authored-by: water <awaterford111445@gmail.com>
2023-01-07 11:24:02 -05:00
Tyler Wilding ac3c4e59b0
goalc/repl: Allow hot-loading files via `ml` with just the object name (#2036)
This allows you to not have to define the entire file path to a source
file to re-compile and load it. Technically a stop-gap until editor
tools are developed around writing OpenGOAL.


![image](https://user-images.githubusercontent.com/13153231/203196148-de61cf4b-42c8-43dc-a7fd-80e6ba6f5ac2.png)

As opposed to `(ml "goal_src/jak2/engine/game/main.gc")` (which still
works)

This is accomplished via the following config (connection attempts is
irrelevant):
```json
{
  "numConnectToTargetAttempts": 1,
  "jak2": {
    "asmFileSearchDirs": [
      "goal_src/jak2"
    ]
  }
}
```

This also provides a way to make game-specific configurations for the
REPL fairly easily.
2022-11-29 19:22:22 -05:00
Tyler Wilding 4d751af38e
logs: replace every `fmt::print` with a `lg` call instead (#1368)
Favors the `lg` namespace over `fmt` directly, as this will output the
logs to a file / has log levels.

I also made assertion errors go to a file, this unfortunately means
importing `lg` and hence `fmt` which was attempted to be avoided before.
But I'm not sure how else to do this aspect without re-inventing the
file logging.

We have a lot of commented out prints as well that we should probably
cleanup at some point / switch them to trace level and default to `info`
level.

I noticed the pattern of disabling debug logs behind some boolean,
something to consider cleaning up in the future -- if our logs were more
structured (knowing where they are coming from) then a lot this
boilerplate could be eliminated.

Closes #1358
2022-10-01 11:58:36 -04:00
water111 b5d21be9c5
W/misc fixes (#1838)
* temp

* temp

* before cleaning up

* cleanup merge

* fix warnings

* merge fix

* clang format
2022-09-05 20:29:12 -04:00
water111 5e23057ed1
[goalc] compile/run code for jak2 ckernel, set up dummy KERNEL.CGO (#1625)
[goalc] start can compile and run code for jak2 ckernel, set up dummy KERNEL.CGO
2022-07-08 19:23:49 -04:00
water111 28a2ecdfd3
[jak2] goalc supports multiple projects (#1619)
* [jak2] goalc supports multiple projects

* disable deci2 server if not debugging
2022-07-06 21:18:08 -04:00
Tyler Wilding 6446389263
extractor: cleanup, support unicode properly, and add multi-game support (#1609)
* extractor: refactor and cleanup for multi-game support

* deps: switch to `ghc::filesystem` as it is utf-8 everywhere by default

* extractor: finally working with unicode

* unicode: fix unicode cli args on windows in all `main` functions
2022-07-05 20:38:13 -04:00
ManDude 9676100039
finish out english subtitles (#1586)
* put some duplicated code in a func

* make jak 2 text "work"

* group up all subtitles c++ code into one folder

* compact single-line subtitles

* fix a couple compiler crashes

* Update game_subtitle_en.gd

* `rolling` and `sunken`

* `swamp`

* `ogre`

* `village3`

* `maincave`

* `snow`

* `lavatube`

* `citadel`

* Update .gitignore

* clang

* fix encoding and decoding for quote

* properly fix quotes

* subtitle deserialize: sort by kind, ID and name

* sub editor: fix line speaker not being converted

* cleanup game text ids 1

* update text ids 2

* update source

* update refs
2022-07-03 17:25:28 -04:00
water111 f763eb6bf0
[jak2] support for multiple out/ directories (#1585)
* [jak2] support for multiple out/ directories

* windows
2022-06-30 21:11:58 -04:00
Tyler Wilding 36dc015d62
jak2: Create goal_src skeleton (#1576) 2022-06-29 22:20:09 -04:00
water111 628ce47b2e
compiler speed improvement (#1547) 2022-06-24 18:21:24 -04:00
ManDude 85bef15364
full ntsc-j mode + greatest hits support (#1525)
* remove str files from inputs

* removed unused verbose flag

* allow some kind of conditional game building for JP extras

* make `PrintBankInfo` do nothing

* clang

* Update gltf_mesh_extract.cpp

* fix `*jak1-full-game*` in extractor

* use json library to build json file

* screw it red label support as well
2022-06-23 18:44:02 -04:00
Tyler Wilding 2d595c1ac0
lint: add include sorting config to clang-format (#1517) 2022-06-22 23:37:46 -04:00
Tyler Wilding 9fdf0bbc2f
tools: Add cutscene player / subtitle editor window (#1429)
* stash

* temp

* tools: subtitle tool works! just gotta fill out the db / polish UX

* tools: added configuration for every subtitle we have so far

* tools: add some colors to the editor, time for repl controls and make it run the code!

* tools: continuing polish of tool, getting very close

* tools: finished UX polish, just need to write deserializers

* tools: added deserializer for subtitle data

* tools: exported subtitle files, all data appears intact

* tools: more UX polish and test all the cutscenes, majority work

* assets: update subtitle files

* lint: formatting and cleanup

* lint: codacy lints
2022-06-11 16:32:27 +01:00
Tyler Wilding 7b6d732a77
goalc: Add TCP server socket in REPL process (#1335)
* goalc: cleanup goalc's main method and add nrepl listener socket

* deps: add standalone ASIO for sockets

* lint: formatting

* common: make a common interface for creating a server socket

* goalc: setup new repl server

* deps: remove asio

* goalc: debug issues, nrepl is working again

* git: rename files

* attempt to fix linux function call

* test

* scripts: make the error message even more obvious....

* goalc: make suggested changes, still can't reconnect properly

* game: pull out single-client logic from XSocketServer

* nrepl: supports multiple clients and disconnection/reconnects

* goalc: some minor fixes for tests

* goalc: save repl history when the compiler reloads

* common: add include for linux networking

* a few small changes to fix tests

* is it the assert?

* change thread start order and add a print to an assert

Co-authored-by: water <awaterford111445@gmail.com>
2022-05-06 18:19:37 -04:00
ManDude a7eee4fdc9
[game] pc port progress menu (#1281)
* fix typo

* more typo

* shorten discord rpc text

* allow expanding enums after the fact (untested)

* make `game_text` work similar to subtitles

* update progress decomp

* update some types + `do-not-decompile` in bitfield

* fixes and fall back to original progress code

* update `progress` decomp with new enums

* update config files

* fix enums and debug menu

* always allocate (but not use) a lot of particles

* small rework to display mode options

* revert resolution/aspect-ratio symbol mess

* begin the override stuff

* make `progress-draw` more readable

* more fixes

* codacy good boy points

* first step overriding code

* finish progress overrides, game options menu fully functional!

* minor fixes

* Update game.gp

* Update sparticle-launcher.gc

* clang

* change camera controls text

* oops

* some cleanup

* derp

* nice job

* implement menu scrolling lol

* make scrollable menus less cramped, fix arrows

* make some carousell things i guess

* add msaa carousell to test

* oops

* Update progress-pc.gc

* make `pc-get-screen-size` (untested)

* resolution menu

* input fixes

* return when selecting resolution

* scroll fixes

* Update progress-pc.gc

* add "fit to screen" button

* bug

* complete resolutions menu

* aspect ratio menu

* subtitles language

* subtitle speaker

* final adjustments

* ref test

* fix tests

* fix ref!

* reduce redundancy a bit

* fix mem leaks?

* save settings on progress exit

* fix init reorder

* remove unused code

* rename goal project-like files to the project extension

* sha display toggle

* aspect ratio settings fixes

* dont store text db's in compiler

* properly save+load native aspect stuff
2022-04-11 18:38:54 -04:00
water111 1db96c72ab
[goalc] macro expansion in integer constants (#1282)
* [goalc] macro expansion in integer constants

* working

* didn't break it yet

* support conditional compilation

* fix up some more small bugs

* fix duplicate evaluation of bitfield definitions

* paranoid
2022-04-07 19:13:22 -04:00
ManDude af447aeab7
[game] subtitles support (tools + goal + text file). (#1174)
* add subtitles support (tools + goal + text file).

* add to build system proper

* better handling of line speakers

* billy test subtitles

* adjust timings

* better handling of subtitle timing + citadel subs

* press square to toggle cutscene subtitles

* improve DirectRenderer performance

* clang

* dont error out if there's no user files

* make system supports multiple inputs for subtitles

* clang

* oh no typo!!

* avoid future issues

* fix warp gate crash

* remove no longer necessary code in DirectRenderer

* remove temp prints

* delete triplicate code

* i found a better way

* fix make issues with subtitles

* force avx compilation
2022-02-19 13:10:10 -05:00
ManDude 24578b64b9
proper support for hardcoded "time" types (#1141)
* hardcode `time-frame`things

* Update cam-states_REF.gc

* Update level-info_REF.gc

* update refs 1

* update refs 2

* update refs 3

* update refs 4

* update refs 5

* update detection and casting

* Update FormExpressionAnalysis.cpp

* update refs 6

* update mood decomp

* update refs 7

* update refs 8

* remove temp entity birth code

* update time-frame casts

* fix compiler

* hardcode stuff and fix some types

* fix some bitfield detection being wrong

* bug fixes

* detect seconds on adds with immediate

* update refs 9

* fix casts and rand-vu-int-range bugs (update refs 10)

* update refs 11

* update 12

* update 13

* update 14

* Update game-info_REF.gc

* improve cpad macros detection

* remove unused code

* update refs

* clang

* update source code

* Update cam-states.gc

* `lavatube-energy` finish

* update refs

* fix actor bank stuff

* Update navigate.gc

* reduce entity default stack size

* Update transformq-h.gc

* oops forgot these

* fix code and tests

* fix mood sound stuff

* Update load-dgo.gc

* Update README.md
2022-02-12 12:26:19 -05:00
water111 78cde74d5a
update readme and fix always playing str (#1139)
* update readme deps

* replace assert

* bump timeout

* fix memory corruption in kernel

* use unknown if level name is invalid
2022-02-08 19:02:47 -05:00
water111 54c6ddbab9
pass filename through (#1080) 2022-01-15 23:31:07 -05:00
ManDude 59d071eccb
[goos/goal] user profiles (#977)
* implement user profiles

* example usage!

* typo

* use a cond

* fix errors

* fixes

* fix potential commandline args disaster
2021-11-24 00:44:04 -05:00
water111 8846968963
[Debugger] windows debugger and process drawable (#953)
* Update assert.h

* stuff for `process-drawable` to work

* add windows code for debugger

* debugger attaches

* something works

* remove bad ideas

* `(:break)` works

* connection fixes

* fixes + update docs

* crates & `defskelgroup` macro

* clang

* update tests and a few types

* temp

* temp

* fix files

* game builds

* reverse TypeConsistency operation

* add eye stuff for merc art login

* add `(:sym-name)`

* oops

* add `--auto-dbg` option to gc args

* codacy

* improve robustness of dgo unpacker and objectfiledb reading

* `cavegeyserrock`

* hopefully fix linux

* windows FormatMessage weirdness?

* mutex fixes

* fix merge conflicts

Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>
2021-10-31 11:01:15 -04:00
water111 24fe2c78c0
[goalc] Address-to-line (#783)
* refactor debug info stuff before adding form to emit

* source mapping working for non-macro sourced forms

* support macros
2021-08-26 20:33:00 -04:00
water111 7a5562106e
Compiler performance improvements and error clean-up (#782)
* compiler cleanup and error improvement

* fix test
2021-08-24 22:15:26 -04:00
water111 4d76d2f11a
add get-enum-vals (#737) 2021-08-02 22:01:10 -04:00
water111 aa58d146c2
[goalc] register allocator v2 (#731)
* clean up allocator interface to be simpler

* working on functions without spills

* working for all

* fix missing includes for windows

* more windows includes

* initialize regs to zero so printing value unintiailized by game code is repeatable
2021-08-01 17:46:55 -04:00
water111 ef011f4fe8
[goalc] add a build system (#704)
* add first attempt at build system

* fix stupid bug

* try again
2021-07-15 21:37:15 -04:00
water111 f516737cf7
[decompile] bsp-h (#562)
* decompile bsp-h

* a few more

* some more

* fix merge conflicts
2021-06-07 18:30:16 -04:00
Tyler Wilding 698d96cc4e
Decompilation: Next batch of files (#539)
* decomp: `collide-frag-h`

* decomp: `effect-control-h`

* decomp: `cam-update-h`

* decomp: `collide-func`

variable pass failed on ray-plane-intersect: invalid unordered_map<K, T> key

* decomp: `cylinder` with vector dot product issue

* decomp: `debug-sphere`

* decomp: `generic`

* decomp: fix `effect-control-h`

* scripts: improve decomp-next script

* decomp: Fix `debug-sphere` issues via type casting

* decomp: Add `collide-frag-h` to reference tests

* scripts: Add script to add a new reference test

* decomp: Add `cam-update-h` to reference tests

* goalc: Fix empty let removal issue

* decomp: Not adding cylinder to goal_src yet either

* decomp: Add `debug-sphere` to reference tests

* decomp: Attempt to finish `generic` but blocked by decomp issue

https://github.com/water111/jak-project/issues/563

* linting

* decomp: Resolve failing tests

* decomp: Address feedback
2021-06-06 23:01:30 -04:00
water111 542edfb164
[compiler/decompiler] Take the address of a variable (#554)
* support taking the address of variables

* partially working stack variables

* implement type cast stuff

* remove final
2021-06-04 13:43:19 -04:00
water111 663b5c7899
[OpenGOAL] Support for 128-bit arguments and return values (WIP) (#510)
* xmm call and return working, needs some careful double checking and tests

* clean up and test
2021-05-20 20:12:49 -04:00
water111 433993074a
Clean up some errors/crashes in decompiler (#452)
* fix offline tests, clean up some warnings

* clean up warnings during decomp

* fix remaining crash issues
2021-05-11 16:43:13 -04:00
ManDude 1d72e79df8
[repl] Always reset runtime before disconnecting (#448) 2021-05-11 15:14:09 -04:00
Tyler Wilding 44fa183922
repl: Add auto-listening capabilities `-auto-lt` (#405)
Closes #403
2021-05-01 10:33:41 -04:00