diff --git a/.gitattributes b/.gitattributes index 903e5cd..6344b3b 100644 --- a/.gitattributes +++ b/.gitattributes @@ -3,3 +3,4 @@ # Patch files must stay LF: git apply matches context bytes against LF upstream sources *.patch -text +translator/tests/Translator.Tests/TestAssets/**/*.bin binary diff --git a/Launcher/Test-PinnedFacts.ps1 b/Launcher/Test-PinnedFacts.ps1 index bd65c84..d7b14bd 100644 --- a/Launcher/Test-PinnedFacts.ps1 +++ b/Launcher/Test-PinnedFacts.ps1 @@ -1,6 +1,6 @@ # Fails the release build when a fact duplicated across the repo stops agreeing with the copy # that owns it (recomp.yml). Scripts read pinned facts through Get-MkwProjectPins, but three -# consumers can't read YAML (the C++ runtime header, the C# constants, hand-written lists on +# consumers can't read YAML (the C++ runtime header, the C# constants, shell scripts, and hand-written lists on # both sides of the C#/PowerShell boundary), so those are checked here instead. [CmdletBinding()] param([string]$RepositoryRoot) @@ -58,6 +58,12 @@ $hostUri = Get-CapturedValue $retroWfcPayload 'CurrentRetroWfcPayloadUri\s*=\s*" if ($hostUri -cne $pins.RetroWfcPayloadUri) { Add-Failure "InputValidation.CurrentRetroWfcPayloadUri is '$hostUri' but recomp.yml pins '$($pins.RetroWfcPayloadUri)'." } +$macosSetup = Read-SourceFile (Join-Path $launcher 'macos\setup.command') 'macOS setup.command' +$macosUri = Get-CapturedValue $macosSetup "'([^']*/api/wfc/payload\?g=RMCPD00)'" ` + 'The macOS Retro-WFC endpoint' +if ($macosUri -cne $pins.RetroWfcPayloadUri) { + Add-Failure "macOS setup.command downloads '$macosUri' but recomp.yml pins '$($pins.RetroWfcPayloadUri)'." +} # --- The game identity: the manifest carries it, but the host also compiles a fallback for a # --- manifest that predates the field, and that fallback decides which disc is accepted. diff --git a/Launcher/WiiCompiled.Setup.Common/RetroWfcPayload.cs b/Launcher/WiiCompiled.Setup.Common/RetroWfcPayload.cs index 22d1c23..258fb85 100644 --- a/Launcher/WiiCompiled.Setup.Common/RetroWfcPayload.cs +++ b/Launcher/WiiCompiled.Setup.Common/RetroWfcPayload.cs @@ -24,7 +24,7 @@ public static class RetroWfcPayload private static readonly TimeSpan RetroWfcDownloadTimeout = TimeSpan.FromSeconds(30); private static readonly TimeSpan RetroWfcRetryDelay = TimeSpan.FromSeconds(1); - public const string CurrentRetroWfcPayloadUri = "http://nas.play.rwfc.net/payload?g=RMCPD00"; + public const string CurrentRetroWfcPayloadUri = "https://rwfc.net/api/wfc/payload?g=RMCPD00"; private static readonly string RetroWfcOfflinePayloadFile = Path.Combine("binary", "payload.RMCPD00.bin"); diff --git a/Launcher/macos/setup.command b/Launcher/macos/setup.command index 8676e7a..09c2915 100755 --- a/Launcher/macos/setup.command +++ b/Launcher/macos/setup.command @@ -90,7 +90,7 @@ if [[ -n "$retro_dir" ]]; then trap 'rm -rf "$payload_stage"' EXIT /usr/bin/curl --fail --silent --show-error --connect-timeout 10 --max-time 30 \ --retry 1 --output "$temporary_payload" \ - 'http://nas.play.rwfc.net/payload?g=RMCPD00' || fail 'could not download the Retro-WFC payload needed for online play' + 'https://rwfc.net/api/wfc/payload?g=RMCPD00' || fail 'could not download the Retro-WFC payload needed for online play' "$translator" validate-retro-wfc-payload --directory "$payload_stage" || \ fail 'downloaded Retro-WFC payload failed signature validation' mkdir -p "$retro_wfc_dir/binary" diff --git a/Launcher/prepare-portable-tools.sh b/Launcher/prepare-portable-tools.sh index 69ea00d..5585c2f 100755 --- a/Launcher/prepare-portable-tools.sh +++ b/Launcher/prepare-portable-tools.sh @@ -233,5 +233,6 @@ EOF rm -rf "$test_dir" trap - EXIT +rm -rf "$toolchain_dir" mv "$work" "$toolchain_dir" echo "prepare-portable-tools.sh: toolchain ready at $toolchain_dir ($(du -sh "$toolchain_dir" | cut -f1))" diff --git a/README.md b/README.md index d61f854..292ec1a 100644 --- a/README.md +++ b/README.md @@ -164,7 +164,9 @@ The default test suite needs no binaries and no host C++ compiler, so you can ha translator without any game data around. For everything beyond that, feeding in your own `main.dol`/`StaticR.rel`, running the -translation, generating the manifest and build graph, and compiling. see [`translator/README.md`](translator/README.md). +translation, generating the manifest and build graph, and compiling, see [`translator/README.md`](translator/README.md). + +For a step-by-step guide on compiling both WiiCompiled and Retro Rewind from source on macOS (Apple Silicon), see the [macOS Build Guide](docs/building-macos.md). ## FAQ diff --git a/aurora-main/lib/gfx/common.cpp b/aurora-main/lib/gfx/common.cpp index 1bb1bb4..17037e7 100644 --- a/aurora-main/lib/gfx/common.cpp +++ b/aurora-main/lib/gfx/common.cpp @@ -168,7 +168,12 @@ struct RenderPass { Range resolveUniformRange; std::array resolveCopyFilterCoefficients{0, 64, 0}; Vec4 clearColorValue{0.f, 0.f, 0.f, 0.f}; - float clearDepthValue = 1.f; + // 1.f is the forward-Z "farthest" clear value; under UseReversedZ farthest is 0.f instead (see + // gx::clear_depth_value(), which the main render pass explicitly overrides this default with - + // any OTHER pass that keeps this default, e.g. an offscreen render-to-texture pass composited + // later, needs the same reversed-Z-aware value or its depth buffer starts "already nearest", + // failing every subsequent depth test and making whatever's drawn into it vanish). + float clearDepthValue = gx::UseReversedZ ? 0.f : 1.f; CommandList commands; bool clearColor = true; bool clearDepth = true; @@ -757,7 +762,9 @@ void begin_offscreen(uint32_t width, uint32_t height) { .targetSize = {width, height, 1}, .msaaSamples = 1, .clearColorValue = {0.f, 0.f, 0.f, 0.f}, - .clearDepthValue = 1.f, + // See the RenderPass::clearDepthValue default's comment: this offscreen pass gets its own + // depth buffer, and the farthest clear value is 0.f, not 1.f, under UseReversedZ. + .clearDepthValue = gx::UseReversedZ ? 0.f : 1.f, .clearColor = true, .clearDepth = true, }; @@ -1410,10 +1417,19 @@ static void render_pass_impl(const wgpu::RenderPassEncoder& pass, const std::vec switch (cmd.type) { case CommandType::SetViewport: { const auto& vp = cmd.data.setViewport; - // WebGPU requires 0 <= minDepth <= maxDepth <= 1, and the guest's (near, far) order is already - // reproduced in clip space. Passing the raw swapped pair diverged per backend in release builds. - const float minDepth = std::clamp(std::min(vp.znear, vp.zfar), 0.0f, 1.0f); - const float maxDepth = std::clamp(std::max(vp.znear, vp.zfar), 0.0f, 1.0f); + // WebGPU requires 0 <= minDepth <= maxDepth <= 1. vp.znear/vp.zfar are in GX's own distance + // terms (0 = near); under UseReversedZ the host depth-buffer storage direction is flipped + // (near = 1, far = 0), so this range has to be remapped through 1-x the same way the + // projection matrix, depth compare function, and clear value all are - a plain min/max clamp + // (the previous code here) maps a *restricted* range (e.g. a viewport deliberately narrowed + // to force something to draw "in front of everything") to the wrong end of the buffer: what + // should land near the near-storage-extreme (1.0) instead lands near the far-storage-extreme + // (0.0), so anything else drawn afterward at its true depth wins the compare test and the + // "in front" geometry silently vanishes. A full [0,1] viewport is unaffected either way, + // which is why this only broke specific elements, not the whole scene. Matches upstream + // aurora's apply_viewport (lib/gfx/encoding.cpp) exactly. + const float minDepth = gx::UseReversedZ ? 1.0f - vp.zfar : vp.znear; + const float maxDepth = gx::UseReversedZ ? 1.0f - vp.znear : vp.zfar; pass.SetViewport(vp.left, vp.top, vp.width, vp.height, minDepth, maxDepth); } break; case CommandType::SetScissor: { diff --git a/aurora-main/lib/gfx/depth_peek.cpp b/aurora-main/lib/gfx/depth_peek.cpp index b41ee92..66a7d5e 100644 --- a/aurora-main/lib/gfx/depth_peek.cpp +++ b/aurora-main/lib/gfx/depth_peek.cpp @@ -92,7 +92,7 @@ struct Params { constexpr std::string_view ReversedZBody = R"( fn gx_z24(depth: f32) -> u32 { - return min(u32(clamp(depth, 0.0, 1.0) * 16777216.0), 0x00ffffffu); + return min(u32(clamp(1.0 - depth, 0.0, 1.0) * 16777215.0 + 0.5), 0x00ffffffu); } )"sv; diff --git a/aurora-main/lib/gfx/tex_copy_conv.cpp b/aurora-main/lib/gfx/tex_copy_conv.cpp index c5e8171..6b68864 100644 --- a/aurora-main/lib/gfx/tex_copy_conv.cpp +++ b/aurora-main/lib/gfx/tex_copy_conv.cpp @@ -137,7 +137,7 @@ fn gx_z24_at_coord(unclamped_coord: vec2i) -> u32 { let tex_size = vec2i(textureDimensions(src)); let coord = clamp(unclamped_coord, vec2i(0), tex_size - vec2i(1)); let depth = textureLoad(src, coord, 0); - return min(u32(clamp(depth, 0.0, 1.0) * 16777216.0), 0x00ffffffu); + return min(u32(clamp(1.0 - depth, 0.0, 1.0) * 16777215.0 + 0.5), 0x00ffffffu); } )"s : R"( diff --git a/aurora-main/lib/gx/gx.cpp b/aurora-main/lib/gx/gx.cpp index bf9d696..15952a6 100644 --- a/aurora-main/lib/gx/gx.cpp +++ b/aurora-main/lib/gx/gx.cpp @@ -1416,23 +1416,32 @@ static inline GXBlendFactor remove_dst_alpha_usage(GXBlendFactor fac) { } } +// GX_LEQUAL etc. describe "pass if this pixel is closer than/equal to what's stored" in GX's own +// distance terms, independent of how that distance is encoded as a host depth value. Under +// UseReversedZ the encoding is flipped (near=1, far=0), so "closer" now corresponds to a *larger* +// stored value, not a smaller one - the ordered compare functions (LESS/LEQUAL/GREATER/GEQUAL) +// must invert to match, or the depth test silently runs backwards (verified directly: this was +// the actual cause of a bug report after the projection/shader half of the reverse-Z fix +// eliminated the double-negation that used to accidentally keep the unreversed comparisons +// correct - LEQUAL now needs GreaterEqual, not LessEqual, once the encoding it's testing against +// is genuinely reversed). Matches upstream aurora's to_compare_function exactly. static inline wgpu::CompareFunction to_compare_function(GXCompare func) { switch (func) { DEFAULT_FATAL("invalid depth fn {}", underlying(func)); case GX_NEVER: return wgpu::CompareFunction::Never; case GX_LESS: - return wgpu::CompareFunction::Less; + return UseReversedZ ? wgpu::CompareFunction::Greater : wgpu::CompareFunction::Less; case GX_EQUAL: return wgpu::CompareFunction::Equal; case GX_LEQUAL: - return wgpu::CompareFunction::LessEqual; + return UseReversedZ ? wgpu::CompareFunction::GreaterEqual : wgpu::CompareFunction::LessEqual; case GX_GREATER: - return wgpu::CompareFunction::Greater; + return UseReversedZ ? wgpu::CompareFunction::Less : wgpu::CompareFunction::Greater; case GX_NEQUAL: return wgpu::CompareFunction::NotEqual; case GX_GEQUAL: - return wgpu::CompareFunction::GreaterEqual; + return UseReversedZ ? wgpu::CompareFunction::LessEqual : wgpu::CompareFunction::GreaterEqual; case GX_ALWAYS: return wgpu::CompareFunction::Always; } diff --git a/aurora-main/lib/gx/gx.hpp b/aurora-main/lib/gx/gx.hpp index 185e0851..ed53e05 100644 --- a/aurora-main/lib/gx/gx.hpp +++ b/aurora-main/lib/gx/gx.hpp @@ -485,7 +485,14 @@ const gfx::TextureBind& get_texture(GXTexMapID id) noexcept; void resolve_sampled_textures(const ShaderInfo& info) noexcept; inline float clear_depth_value() { - return std::min(static_cast(g_gxState.clearDepth) / 16777216.f, 16777215.f / 16777216.f); + // g_gxState.clearDepth is in GX's own distance terms (0 = near, larger = farther), independent of + // how UseReversedZ encodes that as a host depth value - it must be re-mapped the same way the + // projection matrix and depth compare function are, or the buffer clears to the wrong extreme + // (verified directly: matches upstream aurora's clear_depth_value, which does this same inversion + // and was the second missing piece alongside to_compare_function's compare-op inversion). + const float normalizedDepth = + std::min(static_cast(g_gxState.clearDepth) / 16777216.f, 16777215.f / 16777216.f); + return UseReversedZ ? (1.f - normalizedDepth) : normalizedDepth; } inline bool render_target_has_alpha(GXPixelFmt pixelFmt) noexcept { return pixelFmt == GX_PF_RGBA6_Z24; } diff --git a/aurora-main/lib/gx/shader.cpp b/aurora-main/lib/gx/shader.cpp index 4806f1f..ae5f642 100644 --- a/aurora-main/lib/gx/shader.cpp +++ b/aurora-main/lib/gx/shader.cpp @@ -993,11 +993,13 @@ wgpu::ShaderModule build_shader(const ShaderConfig& config) noexcept { "\n let clip_base = select(clip_a, clip_b, use_b);" "\n out.pos = vec4f(clip_base.xy + offset_ndc * clip_base.w, clip_base.zw);"; } - if constexpr (UseReversedZ) { - vtxXfrAttrsPre += "\n out.pos.z = -out.pos.z;"; - } else { - vtxXfrAttrsPre += "\n out.pos.z += out.pos.w;"; - } + // The near/far depth correction used to be applied here per-vertex (out.pos.z = -out.pos.z for + // reversed, or += out.pos.w for forward), redundantly on top of the same correction already + // folded into ubuf.proj by effective_projection() (shader_info.cpp) - applying it twice canceled + // out for the common case (any draw where effective_projection() decides to flip), silently + // making "reversed" Z behave identically to forward Z. It is now applied exactly once, in the + // projection matrix alone (matching upstream aurora commit 1dde08fa: "Move depth correction to + // projection matrix"), so nothing needs to happen to out.pos.z here. // GX rasterizes at a 7/12 pixel center when antialiasing is disabled, while WebGPU rasterizes at 1/2. vtxXfrAttrsPre += "\n let gx_pixel_center_correction = " @@ -1465,7 +1467,14 @@ wgpu::ShaderModule build_shader(const ShaderConfig& config) noexcept { textureDependency.texMapId, uvIn); } - std::string fogDepthExpr = UseReversedZ ? "in.pos.z" : "(1.0 - in.pos.z)"; + // in.pos.z is the host NDC z (forward: 0=near/1=far; reversed: 1=near/0=far post-fix), but this + // expression needs to produce GX's own native distance term (always 0=near/1=far, matching how + // g_gxState.clearDepth/clear_depth_value() are interpreted before their own UseReversedZ + // inversion) - forward already matches directly; reversed needs the same 1-x flip everything + // else reversed-Z-aware uses. This was backwards (verified directly against upstream aurora's + // identical expression in build_shader_source), which fed both fog density and the GX_ZT_ADD + // z-texture path the wrong distance value. + std::string fogDepthExpr = UseReversedZ ? "(1.0 - in.pos.z)" : "in.pos.z"; std::string fogZCoordExpr = fmt::format("u32(round(clamp({}, 0.0, 1.0) * 16777216.0))", fogDepthExpr); if (usesZTextureDepth) { @@ -1498,7 +1507,7 @@ wgpu::ShaderModule build_shader(const ShaderConfig& config) noexcept { fragmentFn += fmt::format( "\n let oldZ = u32(round(clamp({0}, 0.0, 1.0) * 16777216.0));" "\n ztexCoord = (ztexCoord + oldZ) & 0x00ffffffu;", - UseReversedZ ? "in.pos.z" : "(1.0 - in.pos.z)"); + UseReversedZ ? "(1.0 - in.pos.z)" : "in.pos.z"); } fragmentFn += "\n let ztexDepth = f32(ztexCoord) / 16777216.0;"; fogZCoordExpr = "ztexCoord"; @@ -1639,7 +1648,13 @@ wgpu::ShaderModule build_shader(const ShaderConfig& config) noexcept { " @builtin(frag_depth) depth: f32,\n" "};"; - fragmentFn += fmt::format("\n let fragDepth = {}ztexDepth;", UseReversedZ ? "" : "1.0 - "); + // ztexDepth is in GX's native distance terms (0=near/1=far, see fogDepthExpr's comment above), + // but frag_depth must be written in the same host NDC-z convention in.pos.z itself uses - + // forward matches directly (no change), reversed needs the same 1-x flip. This was backwards + // the same way fogDepthExpr was (verified by the same derivation, since aurora upstream has no + // directly equivalent line here to cross-check against - this z-texture-depth-output path + // appears to be specific to this fork). + fragmentFn += fmt::format("\n let fragDepth = {}ztexDepth;", UseReversedZ ? "1.0 - " : ""); fragmentReturnType = "FragmentOutput"; fragmentReturn = " var out: FragmentOutput;\n" diff --git a/aurora-main/lib/gx/shader_info.cpp b/aurora-main/lib/gx/shader_info.cpp index 3c21e4f..92ae1de 100644 --- a/aurora-main/lib/gx/shader_info.cpp +++ b/aurora-main/lib/gx/shader_info.cpp @@ -548,14 +548,22 @@ constexpr size_t kStagedUniformBytes = 96 + sizeof(Mat4x4) + sizeof(Mat3x4) * (MaxPostexMtx + MaxPnMtx); // The host viewport always receives the normalized GX depth window (render_pass_impl clamps to minDepth <= maxDepth). +// +// Folds the near/far depth correction the vertex shader used to apply per-vertex directly into the +// projection matrix instead (matching upstream aurora commit 1dde08fa, "Move depth correction to +// projection matrix") - valid because the correction is a linear combination of the z/w rows, so +// applying it once here to the row is equivalent to applying it once per-vertex to the dot product, +// and it must be applied exactly once: doing it here AND in the shader (the previous bug) canceled +// the negation out for `flip`, silently making "reversed" Z behave identically to forward Z. +// `flip` decides which of the two single-application forms this draw needs: true bakes in the +// reversed-Z inversion (z' = -z), false bakes in the forward-Z near/far combination (z' = z + w) - +// exactly one always applies, never both, and never neither. static Mat4x4 effective_projection() noexcept { const auto& vp = g_gxState.renderViewport; const bool flip = (vp.znear <= vp.zfar) == UseReversedZ; Mat4x4 proj = g_gxState.proj; - if (flip) { - for (size_t i = 0; i < 4; ++i) { - proj.m2.m[i] = -(proj.m2.m[i] + proj.m3.m[i]); - } + for (size_t i = 0; i < 4; ++i) { + proj.m2.m[i] = flip ? -proj.m2.m[i] : (proj.m2.m[i] + proj.m3.m[i]); } return proj; } diff --git a/docs/building-macos.md b/docs/building-macos.md new file mode 100644 index 0000000..6c8aa95 --- /dev/null +++ b/docs/building-macos.md @@ -0,0 +1,349 @@ +# Building WiiCompiled and Retro Rewind on macOS + +This guide covers building **WiiCompiled** (base game) and **Retro Rewind** from source on macOS for Apple Silicon (`arm64`). Follow these instructions to compile the native executables directly. + +> [!NOTE] +> If you only want to build the base game (**WiiCompiled**), look for sections marked **`(Skip if only building WiiCompiled)`** to bypass Retro Rewind and online payload steps. + +--- + +## 1. Prerequisites + +### System Requirements +- **Hardware**: Apple Silicon Mac (M1/M2/M3/M4) +- **Operating System**: macOS 14 (Sonoma) or later +- **Xcode Command Line Tools**: + ```bash + xcode-select --install + ``` + +### Toolchain Dependencies +Install the required tools using [Homebrew](https://brew.sh): +```bash +brew install cmake ninja +brew install --cask dotnet-sdk@8 +``` + +Verify that Clang, CMake, Ninja, and the .NET 8 runtime are available: +```bash +clang --version +cmake --version +ninja --version +dotnet --list-runtimes # Must list Microsoft.NETCore.App 8.x +``` + +--- + +## 2. Required Game and Mod Assets + +Due to legal requirements, no proprietary Nintendo assets or code are included in this repository. You must provide your own legally dumped game files. + +1. **Mario Kart Wii PAL (`RMCP01`) Disc Image** *(Required)*: + - Supported formats: `.iso`, `.wbfs`, `.ciso`, `.rvz`, `.gcm`, `.gcz`. +2. **nodtool** *(Required for disc extraction)*: + - Download the macOS Apple Silicon binary of [nodtool](https://github.com/encounter/nod/releases): + ```bash + curl -fsSL "https://github.com/encounter/nod/releases/download/v2.0.0-alpha.10/nodtool-macos-arm64" -o nodtool + chmod +x nodtool + ``` +3. **Retro Rewind Distribution** *(Skip if only building WiiCompiled)*: + - Download the [Retro Rewind](https://wiki.tockdom.com/wiki/Retro_Rewind) release package. You will need the `RetroRewind6` folder (which contains `Binaries/Code.pul`). +4. **Retro-WFC Payload** *(Skip if only building WiiCompiled or building offline)*: + - Required for online multiplayer on Retro Rewind. Downloaded during setup from `http://nas.play.rwfc.net/payload?g=RMCPD00`. + +--- + +## 3. Step 1: Extract Disc Assets + +Extract your clean PAL `RMCP01` disc into the `Assets/` directory of the repository: + +```bash +# Using nodtool directly into a temporary scratch directory +mkdir -p /tmp/mkw-extract +./nodtool extract /path/to/RMCP01.iso /tmp/mkw-extract + +# Copy extracted assets into the repository Assets directory +rm -rf Assets/DATA/files Assets/DATA/sys +mkdir -p Assets/DATA +cp /tmp/mkw-extract/*/sys/main.dol Assets/main.dol +cp /tmp/mkw-extract/*/files/rel/StaticR.rel Assets/StaticR.rel +cp -R /tmp/mkw-extract/*/files Assets/DATA/files +cp -R /tmp/mkw-extract/*/sys Assets/DATA/sys + +# Clean up temporary files +rm -rf /tmp/mkw-extract +``` + +> [!TIP] +> Alternatively, you can use the repository's helper script: +> ```bash +> Launcher/macos/extract-disc.command --game /path/to/RMCP01.iso --assets-dir Assets --nodtool ./nodtool +> ``` + +### Verify Extracted Asset Hashes +Confirm that the extracted files match the expected clean PAL revision: +```bash +shasum -a 256 Assets/main.dol Assets/StaticR.rel +``` +- `Assets/main.dol`: `80d18895b39c63bd80f457398bfcbb91b7d16ac116a41a88967e954080155b05` +- `Assets/StaticR.rel`: `16d9d146112541fefea701ecb5bc1a496f9d50e4a752fbb5b6778e7c6399f67d` + +--- + +## 4. Step 2: Build the Translator CLI + +Compile the static recompiler CLI: + +```bash +dotnet build translator/src/Translator.Cli/Translator.Cli.csproj -c Release +``` + +Define a shell function to invoke the translator (ensuring paths with spaces are handled safely): +```bash +translator() { + dotnet "$(pwd)/translator/src/Translator.Cli/bin/Release/net8.0/Translator.Cli.dll" "$@" +} +``` + +--- + +## 5. Step 3: Translation + +### A. Translate Base Game Functions +```bash +mkdir -p generated/functions build/base + +translator translate-recursive 0x8000629c \ + --project projects/mkwii/recomp.yml \ + --outdir generated/functions \ + --output-metadata generated/base_translation_output.json \ + --production-source-bundle generated/base_translation_sources.bin \ + --no-function-files \ + --prune-stale \ + --threads $(sysctl -n hw.ncpu) +``` + +### B. Emit Base Manifest +```bash +translator emit-base-manifest \ + --project projects/mkwii/recomp.yml \ + --out build/base \ + --functions-dir generated/functions \ + --translation-output-metadata generated/base_translation_output.json \ + --region P +``` + +--- + +### C. Stage and Translate Retro Rewind *(Skip this step if you only want to build WiiCompiled)* + +1. Stage `Code.pul`: + ```bash + RETRO_DIR="/path/to/RetroRewind6" + mkdir -p PulsarPacks/completed/RetroRewind/RetroRewind6/Binaries + cp "$RETRO_DIR/Binaries/Code.pul" PulsarPacks/completed/RetroRewind/RetroRewind6/Binaries/Code.pul + ``` + +2. **Retro-WFC Payload Setup (for Online Multiplayer)**: + Online play in Retro Rewind requires the shared Retro-WFC payload. Download and validate it: + ```bash + mkdir -p build/retro-wfc/binary + curl -fsSL --retry 3 "https://nas.play.rwfc.net/payload?g=RMCPD00" \ + -o build/retro-wfc/binary/payload.RMCPD00.bin + + # Validate payload signature and integrity + translator validate-retro-wfc-payload --directory build/retro-wfc + ``` + +3. Run Retro Rewind translation: + ```bash + mkdir -p build/mods/retro_rewind_full_cpp + + translator translate-mod \ + --project projects/mkwii/recomp.yml \ + --profile retro-rewind \ + --base-manifest build/base/mkwii_base_manifest.json \ + --base-translation-output-metadata generated/base_translation_output.json \ + --code-pul "$RETRO_DIR/Binaries/Code.pul" \ + --mod-root "$RETRO_DIR" \ + --mod-name "Retro Rewind" \ + --region P \ + --out build/mods/retro_rewind_full_cpp \ + --prefer-cached-inputs \ + --emit-cpp \ + --threads $(sysctl -n hw.ncpu) \ + --retro-wfc-payload build/retro-wfc/binary/payload.RMCPD00.bin + ``` + > [!TIP] + > If you do not want online play or do not have an internet connection, replace `--retro-wfc-payload ...` with `--skip-retro-wfc`. + +--- + +### D. Generate Data Initialization and Build Shards + +First, generate the embedded game data initializer: +```bash +translator generate-data-init --project projects/mkwii/recomp.yml +``` + +Next, generate the CMake build shards using **one** of the following options: + +#### Option 1: Base Game Only (WiiCompiled) +```bash +mkdir -p generated/build_shards +translator emit-build-shards \ + --project projects/mkwii/recomp.yml \ + --base-metadata generated/base_translation_output.json \ + --base-functions-dir generated/functions \ + --native-source-dir runtime/src \ + --out generated/build_shards +``` + +#### Option 2: Base Game + Retro Rewind +```bash +mkdir -p generated/build_shards +translator emit-build-shards \ + --project projects/mkwii/recomp.yml \ + --base-metadata generated/base_translation_output.json \ + --base-functions-dir generated/functions \ + --native-source-dir runtime/src \ + --out generated/build_shards \ + --resolved-profile build/mods/retro_rewind_full_cpp/resolved_dispatch_profile.json \ + --retro-cpp-dir build/mods/retro_rewind_full_cpp/cpp +``` + +--- + +## 6. Step 4: Configure and Compile with CMake & Ninja + +Configure the native C++ build targeting Apple Silicon: + +```bash +cmake -S runtime -B build-macos -G Ninja \ + -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_C_COMPILER=clang \ + -DCMAKE_CXX_COMPILER=clang++ \ + -DAURORA_SDL3_PROVIDER=vendor +``` + +Compile the desired target: + +```bash +# To build WiiCompiled only: +cmake --build build-macos --target WiiCompiled --parallel $(sysctl -n hw.ncpu) + +# OR to build both WiiCompiled and Retro Rewind: +cmake --build build-macos --target WiiCompiled RetroRewind --parallel $(sysctl -n hw.ncpu) +``` + +Once compilation completes, the executables are ready in your build directory: +- `build-macos/WiiCompiled` +- `build-macos/RetroRewind` (if built) + +During the build, CMake automatically copies the required runtime assets into `build-macos/`: +- `build-macos/dsp_coef.bin` +- `build-macos/initial_pipeline_cache.db` +- `build-macos/wii_bootstrap/` + +--- + +## 7. Step 5: Running Executables from the Build Folder + +### Configure `Config.toml` +The runtime reads configuration from `~/Library/Application Support/WiiCompiled/Config.toml`. + +Create the directory and configuration file: + +```bash +mkdir -p "$HOME/Library/Application Support/WiiCompiled" +``` + +#### For Base Game Only (WiiCompiled): +```toml +# ~/Library/Application Support/WiiCompiled/Config.toml +[video] +widescreen = true +resolution_multiplier = 1.0 +graphics_api = "metal" + +[paths] +dvd_root = "/absolute/path/to/Wiicompiled/Assets/DATA" +``` + +#### For Base Game and Retro Rewind: +```toml +# ~/Library/Application Support/WiiCompiled/Config.toml +[video] +widescreen = true +resolution_multiplier = 1.0 +graphics_api = "metal" + +[paths] +dvd_root = "/absolute/path/to/Wiicompiled/Assets/DATA" +retro_rewind_root = "/path/to/RetroRewind6" +``` + +> [!NOTE] +> Ensure `dvd_root` points to the directory containing `files` and `sys/fst.bin`. + +### Launching the Game +Run the compiled binaries directly from your terminal or by double clicking: + +```bash +# Run base WiiCompiled +./build-macos/WiiCompiled + +# Run Retro Rewind +./build-macos/RetroRewind +``` + + + +Press **F10** in-game at any time to open the configuration bar (controls, resolution, display settings, audio). + +--- + +## Quick Reference: Automated Helper Script + +The repository provides a script (`Launcher/local-build-macos.command`) that handles extraction, translation, and compilation in a single command. + +### Building Base Game Only: +```bash +Launcher/local-build-macos.command \ + --profile base \ + --output-dir build-macos/Products \ + --game /path/to/RMCP01.iso \ + --nodtool ./nodtool +``` + +### Building Both (with Online Retro-WFC Payload): +```bash +# 1. Download Retro-WFC payload into a staging directory: +mkdir -p build/retro-wfc/binary +curl -fsSL --retry 3 "http://nas.play.rwfc.net/payload?g=RMCPD00" \ + -o build/retro-wfc/binary/payload.RMCPD00.bin + +# 2. Run the automated build with the payload directory: +Launcher/local-build-macos.command \ + --profile both \ + --output-dir build-macos/Products \ + --base-output-dir build-macos/Products \ + --game /path/to/RMCP01.iso \ + --nodtool ./nodtool \ + --retro-rewind-package-dir /path/to/RetroRewind6 \ + --retro-wfc-offline-dir build/retro-wfc +``` + +### Building Both (Offline, Skipping Payload): +```bash +Launcher/local-build-macos.command \ + --profile both \ + --output-dir build-macos/Products \ + --base-output-dir build-macos/Products \ + --game /path/to/RMCP01.iso \ + --nodtool ./nodtool \ + --retro-rewind-package-dir /path/to/RetroRewind6 \ + --skip-retro-wfc-payload +``` + +When finished, the compiled executables reside in `native-build-macos/` and the bundled `.app` packages are placed in `build-macos/Products/`. diff --git a/projects/mkwii/recomp.yml b/projects/mkwii/recomp.yml index bd8f727..20056ad 100644 --- a/projects/mkwii/recomp.yml +++ b/projects/mkwii/recomp.yml @@ -58,7 +58,7 @@ profiles: module_link_base: 0x803992E0 output: build/mods/retro_rewind_full_cpp enable_retro_wfc: true - retro_wfc_payload: http://nas.play.rwfc.net/payload?g=RMCPD00 + retro_wfc_payload: https://rwfc.net/api/wfc/payload?g=RMCPD00 retro_wfc_legacy_bootstrap_hook: 0x800ED6E8 riivolution: xml: xml/RetroRewind6.xml diff --git a/runtime/src/hle/os/os_sleep.cpp b/runtime/src/hle/os/os_sleep.cpp index 5938dda..5a66c02 100644 --- a/runtime/src/hle/os/os_sleep.cpp +++ b/runtime/src/hle/os/os_sleep.cpp @@ -78,22 +78,38 @@ bool ProcessSleepTimers(CpuContext* cpu) { using Clock = std::chrono::steady_clock; - std::vector dueTimers; + // Pop and process ONE due timer at a time, straight from the shared table. Resuming a + // sleeper re-enters the scheduler (OSResumeThread -> SelectThread) and can switch fibers + // away from this call. Timers that had already been popped into a private list would then + // sit on the suspended fiber's stack with their threads parked and no entry in the table: + // exactly the "park-shaped with no pending wake timer" strand the reconciler below heals + // 100ms late, followed by a "sleep-timer stale" drop when this fiber finally resumes. + // Leaving unprocessed timers in the table keeps them visible to every other pump (idle + // loop, other threads' SelectThread) while this one is switched away. + bool processedAny = false; + constexpr size_t kMaxTimersPerCall = 64; + size_t processedCount = 0; const auto now = Clock::now(); - { - std::lock_guard lock(gSleepTimerMutex); - auto it = gSleepTimers.begin(); - while (it != gSleepTimers.end()) { - if (it->deadline > now) { - ++it; - continue; + while (processedCount < kMaxTimersPerCall) { + SleepTimerEntry timer{0, {}}; + bool found = false; + { + std::lock_guard lock(gSleepTimerMutex); + for (auto it = gSleepTimers.begin(); it != gSleepTimers.end(); ++it) { + if (it->deadline <= now) { + timer = *it; + gSleepTimers.erase(it); + found = true; + break; + } } - dueTimers.push_back(*it); - it = gSleepTimers.erase(it); } - } + if (!found) { + break; + } + ++processedCount; + processedAny = true; - for (const SleepTimerEntry& timer : dueTimers) { const uint32_t threadPtr = timer.threadPtr; if (threadPtr == 0 || !Memory::Contains(threadPtr + kThreadSuspendOffset, sizeof(uint32_t))) { @@ -219,7 +235,7 @@ bool ProcessSleepTimers(CpuContext* cpu) } } - return !dueTimers.empty(); + return processedAny; } } // namespace OsHleInternal diff --git a/translator/tests/Translator.Tests/RetroWfcPayloadLoweringTests.cs b/translator/tests/Translator.Tests/RetroWfcPayloadLoweringTests.cs index 4a11b0d..e5fe310 100644 --- a/translator/tests/Translator.Tests/RetroWfcPayloadLoweringTests.cs +++ b/translator/tests/Translator.Tests/RetroWfcPayloadLoweringTests.cs @@ -35,6 +35,39 @@ public class RetroWfcPayloadLoweringTests Assert.Equal("moduleFunction", pointer.TargetKind); } + [Fact] + public void ProductionPayloadValidatesAndTranslatesEverySupportedPatch() + { + var payloadRoot = Path.Combine( + AppContext.BaseDirectory, + "TestAssets", + "RetroWfcPayload"); + WiiCompiled.Setup.Common.RetroWfcPayload.ValidateStagedRetroWfcPayloadDirectory(payloadRoot); + var payloadPath = Path.Combine( + payloadRoot, + "binary", + "payload.RMCPD00.bin"); + var payload = File.ReadAllBytes(payloadPath); + + var result = RetroWfcPayload.Parse( + payload, + ProductionPayloadManifest(), + 0x81800000u, + 0x00200000u, + "TestAssets/RetroWfcPayload/binary/payload.RMCPD00.bin"); + + Assert.Equal("RMCPD00", result.Summary.Game); + Assert.Equal(payload.Length, result.Summary.PayloadImageSize); + Assert.True(result.LoweringPlan.IsPlannable); + Assert.Empty(result.LoweringPlan.Issues); + Assert.NotEmpty(result.LoweringPlan.StaticBytePatches); + Assert.NotEmpty(result.LoweringPlan.ExecutableHooks); + Assert.NotEmpty(result.LoweringPlan.StaticPointers); + Assert.All(result.LoweringPlan.ExecutableHooks, hook => Assert.NotNull(hook.TargetAddress)); + Assert.All(result.LoweringPlan.StaticPointers, pointer => Assert.NotNull(pointer.TargetAddress)); + Assert.NotEmpty(result.Summary.InitializationCallbacks); + } + private static BaseManifest TestManifest() => new( "test", @@ -52,6 +85,51 @@ public class RetroWfcPayloadLoweringTests ], "ranges.json"); + // The payload parser needs the base image's address classes and containing + // function ranges to prove every patch can be lowered. A single synthetic + // executable and writable ranges are sufficient here: the assertions above + // test the real production payload without checking proprietary game bytes + // into CI. The split also proves pointer patches lower as data writes. + private static BaseManifest ProductionPayloadManifest() => + new( + "test", + 1, + "RMCP01", + "P", + "", + 0, + [ + new BaseSectionMetadata( + ".synthetic-text", + "synthetic.dol", + 0x80000000u, + 0x80800000u, + true, + false, + "synthetic_text.bin", + 0), + new BaseSectionMetadata( + ".synthetic-data", + "synthetic.dol", + 0x80800000u, + 0x81000000u, + false, + true, + "synthetic_data.bin", + 0) + ], + [ + new BaseFunctionRangeMetadata( + 0x80000000u, + 0x80800000u, + "synthetic_base", + ".synthetic-text", + 0, + "test", + ["Executable"]) + ], + "ranges.json"); + private static byte[] BuildSharedPayloadFixture() { var payload = new byte[0x240]; diff --git a/translator/tests/Translator.Tests/TestAssets/RetroWfcPayload/binary/payload.RMCPD00.bin b/translator/tests/Translator.Tests/TestAssets/RetroWfcPayload/binary/payload.RMCPD00.bin new file mode 100644 index 0000000..fd7ecd8 Binary files /dev/null and b/translator/tests/Translator.Tests/TestAssets/RetroWfcPayload/binary/payload.RMCPD00.bin differ diff --git a/translator/tests/Translator.Tests/Translator.Tests.csproj b/translator/tests/Translator.Tests/Translator.Tests.csproj index 4e37ccd..dccdafe 100644 --- a/translator/tests/Translator.Tests/Translator.Tests.csproj +++ b/translator/tests/Translator.Tests/Translator.Tests.csproj @@ -26,6 +26,11 @@ + + + + +