Relates to #1353
This adds no new functionality or overhead to the compiler, yet. This is
the preliminary work that has:
- added code to the compiler in several spots to flag when something is
used without being properly required/imported/whatever (disabled by
default)
- that was used to generate project wide file dependencies (some
circulars were manually fixed)
- then that graph underwent a transitive reduction and the result was
written to all `jak1` source files.
The next step will be making this actually produce and use a dependency
graph. Some of the reasons why I'm working on this:
- eliminates more `game.gp` boilerplate. This includes the `.gd` files
to some extent (`*-ag` files and `tpage` files will still need to be
handled) this is the point of the new `bundles` form. This should make
it even easier to add a new file into the source tree.
- a build order that is actually informed from something real and
compiler warnings that tell you when you are using something that won't
be available at build time.
- narrows the search space for doing LSP actions -- like searching for
references. Since it would be way too much work to store in the compiler
every location where every symbol/function/etc is used, I have to do
ad-hoc searches. By having a dependency graph i can significantly reduce
that search space.
- opens the doors for common shared code with a legitimate pattern.
Right now jak 2 shares code from the jak 1 folder. This is basically a
hack -- but by having an explicit require syntax, it would be possible
to reference arbitrary file paths, such as a `common` folder.
Some stats:
- Jak 1 has about 2500 edges between files, including transitives
- With transitives reduced at the source code level, each file seems to
have a modest amount of explicit requirements.
Known issues:
- Tracking the location for where `defmacro`s and virtual state
definitions were defined (and therefore the file) is still problematic.
Because those forms are in a macro environment, the reader does not
track them. I'm wondering if a workaround could be to search the
reader's text_db by not just the `goos::Object` but by the text
position. But for the purposes of finishing this work, I just statically
analyzed and searched the code with throwaway python code.
I noticed that jak 3's compilation was spending a lot of time accessing
the `unordered_map`s we use to store constants and symbol types.
I repurposed the `EnvironmentMap` originally made for GOOS for this. It
turns out that we were copying the entire constant map whenever we
encountered a `deftype`, and fixed that too.
This speeds up jak3 compiles from ~16 to 11 seconds for me.
This updates `fmt` to the latest version and moves to just being a copy
of their repo to make updating easier (no editing their cmake / figuring
out which files to minimally include).
The motivation for this is now that we switched to C++ 20, there were a
ton of deprecated function usages that is going away in future compiler
versions. This gets rid of all those warnings.
Started at 349,880,038 allocations and 42s
- Switched to making `Symbol` in GOOS be a "fixed type", just a wrapper
around a `const char*` pointing to the string in the symbol table. This
is a step toward making a lot of things better, but by itself not a huge
improvement. Some things may be worse due to more temp `std::string`
allocations, but one day all these can be removed. On linux it saved
allocations (347,685,429), and saved a second or two (41 s).
- cache `#t` and `#f` in interpreter, better lookup for special
forms/builtins (hashtable of pointers instead of strings, vector for the
small special form list). Dropped time to 38s.
- special-case in quasiquote when splicing is the last thing in a list.
Allocation dropped to 340,603,082
- custom hash table for environment lookups (lexical vars). Dropped to
36s and 314,637,194
- less allocation in `read_list` 311,613,616. Time about the same.
- `let` and `let*` in Interpreter.cpp 191,988,083, time down to 28s.
Previously, `object` and `none` were both top-level types. This made
decompilation rather messy as they have no LCA and resulted in a lot of
variables coming out as type `none` which is very very wrong and
additionally there were plenty of casts to `object`. This changes it so
`none` becomes a child of `object` (it is still represented by
`NullType` which remains unusable in compilation).
This change makes `object` the sole top-level type, and the type that
can represent *any* GOAL object. I believe this matches the original
GOAL built-in type structure. A function that has a return type of
`object` can now return an integer or a `none` at the same time.
However, keep in mind that the return value of `(none)` is still
undefined, just as before. This also makes a cast to `object`
meaningless in 90% of the situations it showed up in (as every single
thing is already an `object`) and the decompiler will no longer emit
them. Casts to `none` are also reduced. Yay!
Additionally, state handlers also don't get the final `(none)` printed
out anymore. The return type of a state handler is completely
meaningless outside the event handler (which is return type `object`
anyway) so there are no limitations on what the last form needs to be. I
did this instead of making them return `object` to trick the decompiler
into not trying to output a variable to be used as a return value
(internally, in the decompiler they still have return type `none`, but
they have `object` elsewhere).
Fixes#1703Fixes#830Fixes#928
This fixes a long time issue with `lambda`. The `lambda` is a bit
overloaded in OpenGOAL: it's used in the implementation of `let`, and
also to define local anonymous functions.
```
(defmacro let (bindings &rest body)
`((lambda :inline #t ,(apply first bindings) ,@body)
,@(apply second bindings)))
```
```
(defmacro defun (name bindings &rest body)
(let ((docstring ""))
(when (and (> (length body) 1) (string? (first body)))
(set! docstring (first body))
(set! body (cdr body)))
`(define ,name ,docstring (lambda :name ,name ,bindings ,@body))))
```
In the first case of a `let`, a `return` from inside the `let` should
return from the functioning containing the `let`, not the scope of the
`lambda`. In the second case, we should return from the lambda. The way
we told the different between these cases was if the `lambda` was used
"immeidately", in the head of an expression (like it would be for the
`let` macro). But, this falsely triggers when an anonymous function is
used immediately: eg
```
((lambda () (return #f)))
```
should generate and call a real x86 function that returns immediately.
This should fix some death/mission failed stuff in jak 2.
Moves PC-specific entity and debug menu things to `entity-debug.gc` and
`default-menu-pc.gc` respectively and makes `(declare-file (debug))`
work as it should (no need to wrap the entire file in `(when
*debug-segment*` now!).
Also changes the DGO descriptor format so that it's less verbose. It
might break custom levels, but the format change is very simple so it
should not be difficult for anyone to update to the new format. Sadly,
you lose the completely useless ability to use DGO object names that
don't match the source file name. The horror!
I've also gone ahead and expanded the force envmap option to also force
the ripple effect to be active. I did not notice any performance or
visual drawbacks from this. Gets rid of some distracting LOD and some
water pools appearing super flat (and pitch back for dark eco).
Fixes#1424
Just missing `target-board-handler` because it errored with `Bad delay
slot in clean_up_cond_no_else_final` and I wasn't able to figure out
where the asm branch was.
Commented out `target-board-clone-anim` because it crashes even after
adding `clone-anim` and `clone-anim-once`.
* [goalc] macro expansion in integer constants
* working
* didn't break it yet
* support conditional compilation
* fix up some more small bugs
* fix duplicate evaluation of bitfield definitions
* paranoid
* clean up allocator interface to be simpler
* working on functions without spills
* working for all
* fix missing includes for windows
* more windows includes
* initialize regs to zero so printing value unintiailized by game code is repeatable
* temp
* working, but type pass got really slow
* clean up
* changelog and flip order
* clean up and add tests
* fix zero size array
* handle lambdas correctly
* another windows fix
* set up types
* cleaned up type analysis and got things working through atomic ops
* expression working, need types
* improved types and names
* getting close
* finish up dma-disasm
* fix
* new method of inline-array-class
* up to new array
* wip side effect stuff
* prepare for pop barrier stuff
* add pop barrier
* add local vars hack to compiler
* fix bug, make sort work
* add test for array
* bug fixes
* another bug fix
* refactoring env variable print for casts
* more tweaks
* updates
* final cleanup
* codacy fixes
* begin work on vf support
* split reg kind into reg hw kind and class, use class for ireg
* try test
* clang format
* add some more ops and some example functions
* better lvf on statics
* add documentation
* work in progress streaming rpc, simple test is working
* actually add the test
* debug windows failure
* windows fix maybe
* windows 2
* use str-load-status
* update types
* add some type stuff for gkernel
* more tweaks
* blah
* more little tweaks and more of gkernel
* add static structures with integer fields but not basics
* static structures
* update gkernel
* start cleanup
* fix typos
* fix syntax highlighting in doc
* lots of documentation updates
* clean and add tests
* more documentation and more error messages
* more document and try building kernel differently
* started adding simple functions in gcommon
* more tests and features
* more tests, debug windows
* debug prints for windows
* back up some regs for windows
* remove debugging prints
* add tests for various xmms
* use unaligned stores and loads to back up and restore xmm128s and also fix argument spilling bug
* add deftype
* add deref and fix some method _type_ things
* see if math works on windows
* add dgo
* windows debug
* windows debug 2
* one more debug try
* add extra debug print and change logic for slashes
* update
* again
* try again
* remove build game
* remove build game
* add back build-game
* remove runtime from test
* test
* reduce number of files
* go to c++ 14
* big stacks
* increase stack size again
* clean up cmake files
* add some more tests for let
* support static strings
* add function calling
* add prints for windows debgu
* one test only
* try swapping r14 and r15 in windows
* swap back
* disable defun for now
* fix massive bug
* fix formatting
Tyler Wilding
water111
ManDude
Hat Kid
water