[jak3] Fix crash with shadow in precursor ship (#3934)

When the `dp-bipedal`s get blown up by the nuke gun, their bones go to
NaNs on the last frame. The shadow renderer doesn't handle this well and
draws all possible triangles as single tris. This overflows the vif
`unpack` field and triggered an assert when sizes inside the shadow
renderer weren't consistent.

My guess is that this works on the real game either because:
- their shadow renderer draws garbage data, but you can't tell because
the screen is white from the nuke
- no NaN on PS2 means that the shadow renderer behaved differently, not
using all single tris.

As a workaround, if the bones are NaN, the shadow renderer treats them
as 0, meaning there is no shadow drawn.

---------

Co-authored-by: water111 <awaterford1111445@gmail.com>

game/graphics/opengl_renderer/foreground/Shadow2.cpp

@@ -155,6 +155,13 @@ void Shadow2::render(DmaFollower& dma, SharedRenderState* render_state, ScopedPr
       u16 addr = up.addr_qw;
 
       u32 offset = 4 * vif1.num;
+      if (transfer.size_bytes > 16 + 255 * 4) {
+        printf("shadow overflow detected, skipping all shadows for this frame!");
+        while (dma.current_tag_offset() == render_state->next_bucket) {
+          dma.read_and_advance();
+        }
+        return;
+      }
       ASSERT(offset + 16 == transfer.size_bytes);
 
       u32 after[4];

game/mips2c/jak2_functions/shadow.cpp

@@ -264,6 +264,15 @@ block_1:
   c->lqc2(vf4, 48, t0);                             // lqc2 vf4, 48(t0)
   // nop                                            // sll r0, r0, 0
   c->lqc2(vf9, 0, a2);                              // lqc2 vf9, 0(a2)
+  // if our bones became nans, then set the transformation matrix to 0 instead.
+  // In jak 3, leaving them as NaN may cause all triangles to become single,
+  // overflowing the 8-bit .num field of a vif unpack. Maybe the same thing happens in Jak 2?
+  if (std::isnan(c->vf_src(1).f[0])) {
+    c->vfs[1].vf.fill(0);
+    c->vfs[2].vf.fill(0);
+    c->vfs[3].vf.fill(0);
+    c->vfs[4].vf.fill(0);
+  }
   c->vmula_bc(DEST::xyzw, BC::w, vf4, vf0);         // vmulaw.xyzw acc, vf4, vf0
   // nop                                            // sll r0, r0, 0
   c->vmadda_bc(DEST::xyzw, BC::x, vf1, vf9);        // vmaddax.xyzw acc, vf1, vf9

game/mips2c/jak3_functions/shadow.cpp

@@ -265,6 +265,16 @@ block_1:
   c->lqc2(vf4, 48, t0);                             // lqc2 vf4, 48(t0)
   // nop                                            // sll r0, r0, 0
   c->lqc2(vf9, 0, a2);                              // lqc2 vf9, 0(a2)
+
+  // if our bones became nans, then set the transformation matrix to 0 instead.
+  // leaving them as NaN may cause all triangles to become single, overflowing the 8-bit .num field
+  // of a vif unpack.
+  if (std::isnan(c->vf_src(1).f[0])) {
+    c->vfs[1].vf.fill(0);
+    c->vfs[2].vf.fill(0);
+    c->vfs[3].vf.fill(0);
+    c->vfs[4].vf.fill(0);
+  }
   c->vmula_bc(DEST::xyzw, BC::w, vf4, vf0);         // vmulaw.xyzw acc, vf4, vf0
   // nop                                            // sll r0, r0, 0
   c->vmadda_bc(DEST::xyzw, BC::x, vf1, vf9);        // vmaddax.xyzw acc, vf1, vf9