Dependency graph work - Part 1 - Preliminary work (#3505)

Relates to #1353 

This adds no new functionality or overhead to the compiler, yet. This is
the preliminary work that has:
- added code to the compiler in several spots to flag when something is
used without being properly required/imported/whatever (disabled by
default)
- that was used to generate project wide file dependencies (some
circulars were manually fixed)
- then that graph underwent a transitive reduction and the result was
written to all `jak1` source files.

The next step will be making this actually produce and use a dependency
graph. Some of the reasons why I'm working on this:
- eliminates more `game.gp` boilerplate. This includes the `.gd` files
to some extent (`*-ag` files and `tpage` files will still need to be
handled) this is the point of the new `bundles` form. This should make
it even easier to add a new file into the source tree.
- a build order that is actually informed from something real and
compiler warnings that tell you when you are using something that won't
be available at build time.
- narrows the search space for doing LSP actions -- like searching for
references. Since it would be way too much work to store in the compiler
every location where every symbol/function/etc is used, I have to do
ad-hoc searches. By having a dependency graph i can significantly reduce
that search space.
- opens the doors for common shared code with a legitimate pattern.
Right now jak 2 shares code from the jak 1 folder. This is basically a
hack -- but by having an explicit require syntax, it would be possible
to reference arbitrary file paths, such as a `common` folder.

Some stats:
- Jak 1 has about 2500 edges between files, including transitives
- With transitives reduced at the source code level, each file seems to
have a modest amount of explicit requirements.

Known issues:
- Tracking the location for where `defmacro`s and virtual state
definitions were defined (and therefore the file) is still problematic.
Because those forms are in a macro environment, the reader does not
track them. I'm wondering if a workaround could be to search the
reader's text_db by not just the `goos::Object` but by the text
position. But for the purposes of finishing this work, I just statically
analyzed and searched the code with throwaway python code.

goalc/compiler/compilation/State.cpp

@@ -45,8 +45,10 @@ Val* Compiler::compile_define_state_hook(const goos::Object& form,
   //  state_name
   //  state_parent
   //  state object
+  //  docstring
   //  named args for enter/exit/trans/post/event/code
-  va_check(form, args, {goos::ObjectType::SYMBOL, goos::ObjectType::SYMBOL, {}},
+  va_check(form, args,
+           {goos::ObjectType::SYMBOL, goos::ObjectType::SYMBOL, {}, goos::ObjectType::STRING},
            {
                {"enter", {true, {}}},
                {"exit", {true, {}}},
@@ -56,6 +58,11 @@ Val* Compiler::compile_define_state_hook(const goos::Object& form,
                {"code", {true, {}}},
            });
 
+  std::string docstring = "";
+  if (args.unnamed.size() == 4 && args.unnamed.at(3).is_string()) {
+    docstring = args.unnamed.at(3).as_string()->data;
+  }
+
   Val* code_value = NULL;
   Val* enter_value = NULL;
 
@@ -121,6 +128,18 @@ Val* Compiler::compile_define_state_hook(const goos::Object& form,
 
   auto sym_val = env->function_env()->alloc_val<SymbolVal>(state_name.name_ptr, type_to_use);
 
+  std::vector<symbol_info::ArgumentInfo> arg_info = {};
+  if (code_value) {
+    auto as_lambda = dynamic_cast<LambdaVal*>(code_value);
+    if (as_lambda) {
+      for (const auto& arg : as_lambda->lambda.params) {
+        arg_info.push_back({.name = arg.name, .type = arg.type.base_type()});
+      }
+    }
+  }
+
+  m_symbol_info.add_state(state_name.name_ptr, state_parent_type.print(), false, arg_info, form,
+                          docstring);
   env->emit_ir<IR_SetSymbolValue>(form, sym_val, state_object);
 
   return get_none();
@@ -141,10 +160,15 @@ Val* Compiler::compile_define_virtual_state_hook(const goos::Object& form,
   //  state_name
   //  state_parent
   //  state object
+  //  docstring
   //  is_override
   //  named args for enter/exit/trans/post/event/code
   va_check(form, args,
-           {goos::ObjectType::SYMBOL, goos::ObjectType::SYMBOL, {}, goos::ObjectType::SYMBOL},
+           {goos::ObjectType::SYMBOL,
+            goos::ObjectType::SYMBOL,
+            {},
+            goos::ObjectType::STRING,
+            goos::ObjectType::SYMBOL},
            {
                {"enter", {true, {}}},
                {"exit", {true, {}}},
@@ -154,6 +178,11 @@ Val* Compiler::compile_define_virtual_state_hook(const goos::Object& form,
                {"code", {true, {}}},
            });
 
+  std::string docstring = "";
+  if (args.unnamed.size() == 5 && args.unnamed.at(3).is_string()) {
+    docstring = args.unnamed.at(3).as_string()->data;
+  }
+
   Val* code_value = NULL;
   Val* enter_value = NULL;
 
@@ -184,7 +213,7 @@ Val* Compiler::compile_define_virtual_state_hook(const goos::Object& form,
   MethodInfo parent_method_info;
   auto parent_of_parent_type = m_ts.lookup_type(state_parent.name_ptr)->get_parent();
   if (m_ts.try_lookup_method(parent_of_parent_type, state_name.name_ptr, &parent_method_info) &&
-      args.unnamed.at(3).as_symbol() == "#f") {
+      args.unnamed.at(4).as_symbol() == "#f") {
     // need to call inherit state TODO
     auto inherit_state_func =
         compile_get_symbol_value(form, "inherit-state", env)->to_gpr(form, env);
@@ -234,6 +263,19 @@ Val* Compiler::compile_define_virtual_state_hook(const goos::Object& form,
     }
   }
 
+  std::vector<symbol_info::ArgumentInfo> arg_info = {};
+  if (code_value) {
+    auto as_lambda = dynamic_cast<LambdaVal*>(code_value);
+    if (as_lambda) {
+      for (const auto& arg : as_lambda->lambda.params) {
+        arg_info.push_back({.name = arg.name, .type = arg.type.base_type()});
+      }
+    }
+  }
+
+  m_symbol_info.add_state(state_name.name_ptr, state_parent_type.print(), true, arg_info, form,
+                          docstring);
+
   return get_none();
 }