Commit Graph

52 Commits

Author SHA1 Message Date
water111 e2b7e5c001 [goalc] reduce compiler memory usage (#2247) 2023-02-24 18:32:30 -05:00
Tyler Wilding 39658dfd71 docs: Automatically generate documentation from goal_src code (#2214)
This automatically generates documentation from goal_src docstrings,
think doxygen/java-docs/rust docs/etc. It mostly supports everything
already, but here are the following things that aren't yet complete:
- file descriptions
- high-level documentation to go along with this (think pure markdown
docs describing overall systems that would be co-located in goal_src for
organizational purposes)
- enums
- states
- std-lib functions (all have empty strings right now for docs anyway)

The job of the new `gen-docs` function is solely to generate a bunch of
JSON data which should give you everything you need to generate some
decent documentation (outputting markdown/html/pdf/etc). It is not it's
responsibility to do that nice formatting -- this is by design to
intentionally delegate that responsibility elsewhere. Side-note, this is
about 12-15MB of minified json for jak 2 so far :)

In our normal "goal_src has changed" action -- we will generate this
data, and the website can download it -- use the information to generate
the documentation at build time -- and it will be included in the site.
Likewise, if we wanted to include docs along with releases for offline
viewing, we could do so in a similar fashion (just write a formatting
script to generate said documentation).

Lastly this work somewhat paves the way for doing more interesting
things in the LSP like:
- whats the docstring for this symbol?
- autocompleting function arguments
- type checking function arguments
- where is this symbol defined?
- etc

Fixes #2215
2023-02-20 19:49:37 -05:00
Tyler Wilding 00ac12094e goalc/repl: cleanup of goalc/REPL code and some QoL improvements (#2104)
- lets you split up your `startup.gc` file into two sections
  - one that runs on initial startup / reloads
  - the other that runs when you listen to a target
- allows for customization of the keybinds added a month or so ago
- removes a useless flag (--startup-cmd) and marks others for
deprecation.
- added another help prompt that lists all the keybinds and what they do

Co-authored-by: water <awaterford111445@gmail.com>
2023-01-07 11:24:02 -05:00
Tyler Wilding 2f4146d469 tests: make the offline tests aware of the current terminals row count (#2105)
This fixes the hideous output when your terminal would be too small to
hold all the threads.
2023-01-07 10:35:12 -05:00
Tyler Wilding ac3c4e59b0 goalc/repl: Allow hot-loading files via ml with just the object name (#2036)
This allows you to not have to define the entire file path to a source
file to re-compile and load it. Technically a stop-gap until editor
tools are developed around writing OpenGOAL.


![image](https://user-images.githubusercontent.com/13153231/203196148-de61cf4b-42c8-43dc-a7fd-80e6ba6f5ac2.png)

As opposed to `(ml "goal_src/jak2/engine/game/main.gc")` (which still
works)

This is accomplished via the following config (connection attempts is
irrelevant):
```json
{
  "numConnectToTargetAttempts": 1,
  "jak2": {
    "asmFileSearchDirs": [
      "goal_src/jak2"
    ]
  }
}
```

This also provides a way to make game-specific configurations for the
REPL fairly easily.
2022-11-29 19:22:22 -05:00
Tyler Wilding a0a85eb60a repl: Add a few new quality of life improvements (#2030)
- You can define a `startup.gc` in your user folder, each line will be
executed on startup (deprecates the usefulness of some cli flags)
- You can define a `repl-config.json` file to override REPL settings.
Long-term this is a better approach than a bunch of CLI flags as well
- Via this, you can override the amount of time the repl will attempt to
listen for the target
- At the same time, I think i may have found why on Windows it can
sometimes take forever to timeout when the game dies, will dig into this
later
- Added some keybinds for common operations, shown here


https://user-images.githubusercontent.com/13153231/202890278-1ff2bb06-dddf-4bde-9178-aa0883799167.mp4
> builds the game, connects to it, attaches a debugger and continues,
launches it, gets the backtrace, stops the target -- all with only
keybinds.

If you want these keybinds to work inside VSCode's integrated terminal,
you need to add the following to your settings file
```json
"terminal.integrated.commandsToSkipShell": [
    "-workbench.action.quickOpen",
    "-workbench.action.quickOpenView"
  ]
```
2022-11-20 14:28:41 -05:00
Tyler Wilding 4d751af38e logs: replace every fmt::print with a lg call instead (#1368)
Favors the `lg` namespace over `fmt` directly, as this will output the
logs to a file / has log levels.

I also made assertion errors go to a file, this unfortunately means
importing `lg` and hence `fmt` which was attempted to be avoided before.
But I'm not sure how else to do this aspect without re-inventing the
file logging.

We have a lot of commented out prints as well that we should probably
cleanup at some point / switch them to trace level and default to `info`
level.

I noticed the pattern of disabling debug logs behind some boolean,
something to consider cleaning up in the future -- if our logs were more
structured (knowing where they are coming from) then a lot this
boilerplate could be eliminated.

Closes #1358
2022-10-01 11:58:36 -04:00
Tyler Wilding 6446389263 extractor: cleanup, support unicode properly, and add multi-game support (#1609)
* extractor: refactor and cleanup for multi-game support

* deps: switch to `ghc::filesystem` as it is utf-8 everywhere by default

* extractor: finally working with unicode

* unicode: fix unicode cli args on windows in all `main` functions
2022-07-05 20:38:13 -04:00
ManDude 9676100039 finish out english subtitles (#1586)
* put some duplicated code in a func

* make jak 2 text "work"

* group up all subtitles c++ code into one folder

* compact single-line subtitles

* fix a couple compiler crashes

* Update game_subtitle_en.gd

* `rolling` and `sunken`

* `swamp`

* `ogre`

* `village3`

* `maincave`

* `snow`

* `lavatube`

* `citadel`

* Update .gitignore

* clang

* fix encoding and decoding for quote

* properly fix quotes

* subtitle deserialize: sort by kind, ID and name

* sub editor: fix line speaker not being converted

* cleanup game text ids 1

* update text ids 2

* update source

* update refs
2022-07-03 17:25:28 -04:00
water111 f763eb6bf0 [jak2] support for multiple out/ directories (#1585)
* [jak2] support for multiple out/ directories

* windows
2022-06-30 21:11:58 -04:00
ManDude 2649fd17ec subtitle editor fixes + other smaller fixes (#1572)
* [extractor] validate files when extracted as folder

* jp text fixes

* move game text version to the text file and fix subtitle editor escape chars

* make bad subtitles not crash the game

* fix texscroll in lag

* fix mood, fix decomp of other versions, fix text decomp

* clang

* fix tests

* oops dammit

* new fixes

* shut up codacy

* fix nonexistant subtitles crashing the game

* fix text hacks and extractor re-use on folders
2022-06-29 19:43:23 -04:00
water111 628ce47b2e compiler speed improvement (#1547) 2022-06-24 18:21:24 -04:00
Tyler Wilding 2d595c1ac0 lint: add include sorting config to clang-format (#1517) 2022-06-22 23:37:46 -04:00
ManDude 0212aa10c9 [decomp] better handling of animation code and art files (#1352)
* update refs

* [decompiler] read and process art groups

* finish decompiler art group selection & detect in `ja-group?`

* make art stuff work on offline tests!

* [decompiler] detect `ja-group!` (primitive)

* corrections.

* more

* use new feature on skel groups!

* find `loop!` as well

* fully fledged `ja` macro & decomp + `loop` detect

* fancy fixed point printing!

* update source

* `:num! max` (i knew i should've done this)

* Update jak1_ntsc_black_label.jsonc

* hi imports

* make compiling the game work

* fix `defskelgroup`

* clang

* update refs

* fix chan

* fix seek and finalboss

* fix tests

* delete unused function

* track let rewrite stats

* reorder `rewrite_let`

* Update .gitattributes

* fix bug with `:num! max`

* Update robotboss-part.gc

* Update goal-lib.gc

* document `ja`

* get rid of pc fixes thing

* use std::abs
2022-05-20 02:30:14 +01:00
ManDude 8eb0796f8e subtitle editing and text project compiler rework (#1331)
* add a `subtitle` bucket and rename buckets 67 and 68

* automatic `heap-base`

* subtitles editor

* adjust default params

* bug

* Update subtitle.gc

* fix blue color inconsistency in progress

* void.

* Update subtitle.gc

* Update subtitle.gc

* Extend `heap-base` detection to check for errors

* sometimes it's actually manually set!

* remove broken error check

* rework how compiling text files works

* try out subtitles source v2

* bugfix

* oops

* Update game_text_common.cpp
2022-04-24 15:15:16 -04:00
ManDude a7eee4fdc9 [game] pc port progress menu (#1281)
* fix typo

* more typo

* shorten discord rpc text

* allow expanding enums after the fact (untested)

* make `game_text` work similar to subtitles

* update progress decomp

* update some types + `do-not-decompile` in bitfield

* fixes and fall back to original progress code

* update `progress` decomp with new enums

* update config files

* fix enums and debug menu

* always allocate (but not use) a lot of particles

* small rework to display mode options

* revert resolution/aspect-ratio symbol mess

* begin the override stuff

* make `progress-draw` more readable

* more fixes

* codacy good boy points

* first step overriding code

* finish progress overrides, game options menu fully functional!

* minor fixes

* Update game.gp

* Update sparticle-launcher.gc

* clang

* change camera controls text

* oops

* some cleanup

* derp

* nice job

* implement menu scrolling lol

* make scrollable menus less cramped, fix arrows

* make some carousell things i guess

* add msaa carousell to test

* oops

* Update progress-pc.gc

* make `pc-get-screen-size` (untested)

* resolution menu

* input fixes

* return when selecting resolution

* scroll fixes

* Update progress-pc.gc

* add "fit to screen" button

* bug

* complete resolutions menu

* aspect ratio menu

* subtitles language

* subtitle speaker

* final adjustments

* ref test

* fix tests

* fix ref!

* reduce redundancy a bit

* fix mem leaks?

* save settings on progress exit

* fix init reorder

* remove unused code

* rename goal project-like files to the project extension

* sha display toggle

* aspect ratio settings fixes

* dont store text db's in compiler

* properly save+load native aspect stuff
2022-04-11 18:38:54 -04:00
Tyler Wilding 014da6f59d game: auto-save pc-settings to user's home directory as well as memcard files (#1233)
* cmake: reduce warning spam especially from libs

* runtime: add FS helper functions

* game: save/restore pc-settings | add original aspect option

* game: overwrite unloadable settings with defaults

* temp: unable to set the games aspect-ratio in the boot else crash?

* runtime: save memcard files to user directory as well

* runtime: fix `pckernel` load order which resolves setting the orig aspect ratio

* lint: format

* cmake: revert warning suppression, it's just causing problems it seems

* fix the order of the rest of `pckernel` and creation of obj file paths

* lint: formatting

* game: don't save settings on startup even if they are corrupted
2022-03-20 20:29:44 -04:00
ManDude af447aeab7 [game] subtitles support (tools + goal + text file). (#1174)
* add subtitles support (tools + goal + text file).

* add to build system proper

* better handling of line speakers

* billy test subtitles

* adjust timings

* better handling of subtitle timing + citadel subs

* press square to toggle cutscene subtitles

* improve DirectRenderer performance

* clang

* dont error out if there's no user files

* make system supports multiple inputs for subtitles

* clang

* oh no typo!!

* avoid future issues

* fix warp gate crash

* remove no longer necessary code in DirectRenderer

* remove temp prints

* delete triplicate code

* i found a better way

* fix make issues with subtitles

* force avx compilation
2022-02-19 13:10:10 -05:00
water111 78cde74d5a update readme and fix always playing str (#1139)
* update readme deps

* replace assert

* bump timeout

* fix memory corruption in kernel

* use unknown if level name is invalid
2022-02-08 19:02:47 -05:00
water111 7a5562106e Compiler performance improvements and error clean-up (#782)
* compiler cleanup and error improvement

* fix test
2021-08-24 22:15:26 -04:00
water111 aa58d146c2 [goalc] register allocator v2 (#731)
* clean up allocator interface to be simpler

* working on functions without spills

* working for all

* fix missing includes for windows

* more windows includes

* initialize regs to zero so printing value unintiailized by game code is repeatable
2021-08-01 17:46:55 -04:00
water111 ef011f4fe8 [goalc] add a build system (#704)
* add first attempt at build system

* fix stupid bug

* try again
2021-07-15 21:37:15 -04:00
water111 6366068bc0 Support dir tpages (#671)
* support dir tpages

* fix warnings and bad return

* one more try

* revive the offline test script

* fix this null bug
2021-07-02 14:50:58 -04:00
water111 fe5635227a [compiler] print a backtrace as part of di (#658)
* print a backtrace as part of di

* missing include
2021-06-30 22:13:15 -04:00
water111 322a4ed9b2 Fix compiler crashes and improve return statements. (#652)
* fix a few small bugs

* fi

* fix merge conflict
2021-06-30 00:11:46 -04:00
ManDude 46b83bda2a [decomp] debug (#607)
* [decomp] `debug`

* shut up
2021-06-19 14:24:55 -04:00
ManDude 784119188c [decomp] even more res (#529)
* [decomp] even more `res`

* [decompiler] make `logand` with pointers and constants return pointer

* [decomp] more work

* update offline tests

* fix tests(?)

* `*res-static-buf*`

* fixes

* fix reference

* [opengoal] make `logand` work directly with pointers

* [decomp] `inspect res-lump`

* use the inline methods

* don't use a math mode for pointers

* [compiler] allow optionally setting disassembly output file

* [x86 disasm] Keep casing consistent between instructions and offsets
2021-06-01 16:07:45 -04:00
water111 0b8a878533 [Compiler] Fix branch targets in disassembly (#379)
* fix diassembly

* update changelog

* fix exit crash

* doc
2021-04-23 15:07:22 -04:00
water111 fe336b7b5f [compiler] fix warnings in repl lib and add macros to autocomplete (#317)
* fix warnings in repl lib and add macros to autocomplete

* fix crash on ctrl-c, build runtime as static lib and make goos prompt look fancier

* some tweaks for linux build
2021-03-11 12:54:16 -05:00
Tyler Wilding 8bba3d7fd7 REPL: Add clear-screen / auto-complete / basic hints and syntax highlighting (#316)
* swap to replxx from linenoise

* repl: Implement form auto-tab-completion

* repl: color coordinate the prompts

* repl: Add some basic syntax highlighting, bracket pairs and forms (all one color)

* repl: A more consistent starting screen for the repl

* repl: bug fix for auto-complete

* debug linux

* linting
2021-03-07 23:41:21 -05:00
water111 9074a35b9b [Decompiler] Fix printing of lets (#314)
* fix let prints and windows warnings

* missing include for windows

* windows again
2021-03-07 12:01:59 -05:00
water111 9168f03289 [Compiler] Add get-info (#306)
* wip symbol info

* finish implementation
2021-03-04 10:33:59 -05:00
water111 d9aa535dd0 [Decompiler] Fix issues in gcommon (#305)
* fix several issues

* small fixes
2021-03-03 20:52:25 -05:00
water111 413c1f5c74 [Decompiler] Add offline tests (#303)
* add offline test for gcommon as an example.

* fix test

* unused var
2021-03-03 15:42:55 -05:00
Tyler Wilding 2eca9ab801 repl: Support cross-session history (#301) 2021-03-03 00:05:13 -05:00
Tyler Wilding 3e0e370343 repl: Add a (repl-help) command for forgetful people like me (#298) 2021-03-02 13:16:47 -05:00
water111 8775840265 [Decomp] Decompile engine math library types (#272)
* decompile some stuff

* fix typo

* playing around with trigonometry

* more progress on trig

* more trig

* comments and small fixes

* finish trig
2021-02-20 11:42:46 -05:00
water111 3331e9cd00 Replace spdlog (#185)
* remove spdlog

* clang format and fix windows

* add format shared

* windows sucks
2021-01-06 12:16:39 -05:00
water111 71dda76e2b Add some compiler features and documentation (#147)
* update doc

* add disassemble and type checking

* improve compiler error messages
2020-12-01 21:39:46 -05:00
water111 09142d1712 Support "game count" and v4 objects (#140)
* generate object, but not supported in linker yet

* add link and tests

* update types
2020-11-24 20:48:38 -05:00
water111 660ef41136 Update documentation and clean up (#129)
* cleanup

* clean up file layout
2020-11-21 12:52:38 -05:00
water111 953c1512db Process Text Files in Decompiler (#122)
* begin support for v2

* export game text

* generate text files

* working text load

* fix windows

* add test and clean up game tests a bit

* load the right file

* add separate program to launch the data compiler

* add offline test script
2020-11-19 21:22:16 -05:00
water111 c09541fa98 Add some debugger memory utilities (#109)
* add some memory utilities

* run waitpid in a separate thread and support very simple breakpoints

* fix breakpoints

* add missing windows stub function

* fix error message on exit
2020-11-06 13:59:39 -05:00
water111 2d11e44eaf Compiler Cleanup and Documentation (#54)
* start cleanup

* fix typos

* fix syntax highlighting in doc

* lots of documentation updates

* clean and add tests

* more documentation and more error messages

* more document and try building kernel differently
2020-09-24 17:19:23 -04:00
water111 90a7e9b4b9 Add addition and subtraction for integers, build macros, dgo building, and build/load test (#35)
* see if math works on windows

* add dgo

* windows debug

* windows debug 2

* one more debug try

* add extra debug print and change logic for slashes

* update

* again

* try again

* remove build game

* remove build game

* add back build-game

* remove runtime from test

* test

* reduce number of files

* go to c++ 14

* big stacks

* increase stack size again

* clean up cmake files
2020-09-12 20:41:12 -04:00
water111 d56540f8c0 Add lambda and static objects (#30)
* add some more tests for let

* support static strings

* add function calling

* add prints for windows debgu

* one test only

* try swapping r14 and r15 in windows

* swap back

* disable defun for now

* fix massive bug

* fix formatting
2020-09-12 13:11:42 -04:00
water111 de5aa7e5e4 Move duplicated utilities to the common util folder and remove NEXT_DIR (#29)
* move things to the common library and remove next_dir

* fix for windows

* one last windows fix

* last fix for real this time

* debug listener test

* fix listener threading bug
2020-09-10 20:03:31 -04:00
water 5612b9e1de fix again 2020-09-08 19:57:58 -04:00
water ee4eb9f128 add some basic symbol stuff 2020-09-07 13:28:16 -04:00
water 1de0cbb6f6 enable macros 2020-09-06 17:42:20 -04:00