mirror of
https://github.com/HarbourMasters/Shipwright
synced 2026-08-13 12:38:55 -04:00
eafafe507ebe0dd9bb8917552c840f9f78574189
26 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 briaguya
|
c00dd24caa |
Replace ZAPDTR and OTRExporter with Torch (#6989)
* Add Torch migration plan
Plan for replacing ZAPDTR + OTRExporter with Torch as SoH's only asset
pipeline, following HarbourMasters/Torch#219 ("oot support").
Co-Authored-By: Claude <noreply@anthropic.com>
* Plan: yml repo is source of truth, drop the move-to-assets-repo steps
- Name the actual repo (briaguya0/soh-asset-yml); it holds yml only, and is
edited directly rather than regenerated. zapd_to_torch.py is a one-shot
conversion, not an ongoing workflow, since nobody edits the ZAPD XMLs once
ZAPD is gone.
- Phase 5 is pure deletion. Generator inputs are already preserved by the
harness's shipwright@95d8f7e submodule pin and by git history, so nothing
needs snapshotting or moving out of Shipwright.
- Clarify soh/CMakeLists.txt:107-109 as a configure-soh-directly fallback with
nothing left to fall back to.
- Downgrade the extraction wall-clock risk; perf work landed in Torch#219.
Co-Authored-By: Claude <noreply@anthropic.com>
* Phase 0: flesh out the verification gates
Adds torch-migration/PHASE0.md with the full gate detail: what each gate's
variable is, the driver source, exact commands, cost, and exit criteria.
Two gates the plan was missing:
- Gate C (Release build) - the harness 14/14 is a Debug measurement; CI
and releases build -O3.
- Gate A2 (two extractions in one process) - free once the driver exists,
retires the re-entrancy risk offline instead of by hand in the game.
Also: gates run full-tree + check.sh rather than test_assets.py (which
hardcodes the torch path and copies a filtered yml subset), 19 ROM dumps
not 17, and libgfxd being USE_STANDALONE-only is called out as the real
hazard in Gate A.
* Phase 0: correct the test_assets.py claim
The "filtered subset" was a misread. The log line prints the pre-recursion
yml count (1320); setup_scratch's external_files recursion then pulls in the
remaining 130, so an unfiltered run copies all 1450 - verified by replaying
the logic. So the gates use test_assets.py, the same tool that produced the
existing 14/14, with a TORCH_BIN env override as the only harness patch.
Gate A2 keeps the check.sh path, since it needs two extractions in one
process and test_assets.py execs the binary once per ROM.
* Phase 0: preflight green, 19/19
Baseline Torch build (USE_STANDALONE=ON, all games, Debug, PORT_VERSION_
ENDIANNESS=ON) reproduces every reference archive: 19 ROM dumps, all
0 failed / 0 not generated / 0 not in reference. Counts match the harness
README's 14-target table, and the duplicate dumps agree with each other,
which the 14-target matrix never showed.
Reference manifests also spot-checked by re-derivation from two o2r files
-- the baseline the gates measure against hasn't drifted.
* Phase 0: Gate A green, 19/19
USE_STANDALONE=OFF plus driving Companion by hand produces byte-identical
archives for all 19 ROM dumps.
The real question was libgfxd, which is fetched only if(USE_STANDALONE) --
_deps/ confirms it is absent, so the Binary export path is genuinely
gfxd-free. Reading said so; now it's run.
Progress denominator confirmed exact: SetPhaseCallback fires once per yml
file, matching the on-disk count for all three classes (1449/1450/1480).
Phase 3's bar can be monotonic with no change to OTRGlobals.
Also observed for Phase 2: zlib is fetched even with BUILD_STORMLIB=OFF and
no zlib.h use in torch/src, confirming risk #1; spdlog is not fetched, so
only the global-logger stomping applies there.
* Phase 0: Gate A2 green, and the duplicate-entry finding
Two extractions in one process, vanilla then MQ, both byte-identical to
their references. The second run's phases=1450 matches pal_mq's on-disk
yml count exactly, which is the direct refutation of the gProcessedFiles
risk -- a leak would have skipped files and truncated the archive. A
fresh Companion per extraction is sufficient; PLAN.md risk #6 retired.
Also records what the gate turned up on the way: torch writes 25
duplicate archive entries, benign because libultraship indexes by CRC64
of the path. Filed as Torch#233 item 5. Corrects the Gate A note that
attributed the whole ~343 KB size delta to compression -- 76 KB of it is
these duplicates.
* Phase 0: Gates B and C green
OoT-only (eight BUILD_<game> flags OFF) and Release (-O3) each 19/19
identical, one variable moved apiece -- verified against the option()
defaults so the other flags genuinely match the baseline cache.
Gate C mattered: every parity measurement to date, including the original
14/14, was Debug, while CI and releases build -O3.
Also records extraction wall-clock from torch's own timing line: Release
averages 11.4s per ROM against Debug's 26.7s, and the OoT-only flags cost
nothing. SoH ships Release, so PLAN.md risk #5 looks much smaller than the
Debug numbers suggested -- though ZAPD's time on the same machine is still
unmeasured.
* Phase 0: Gate A' green -- the shipping configuration is the spec
Static lib + OoT-only + Release together: 19/19 identical plus the
vanilla-then-MQ pair. Flags verified from the generated cache rather than
assumed, so Phase 2's CMake block and TorchExtract.cpp can be
transcriptions of this build and RunOnce respectively.
Archive bytes turn out to be configuration-independent -- the pair
produced exactly the sizes the Debug static-lib run did, phases=1450 on
both runs. Extraction 11.8s mean, so the static-lib wrapper costs nothing
over Gate C.
Confirms both Phase 2 dependency risks in the configuration that actually
ships: zlib fetched with BUILD_STORMLIB=OFF and no zlib.h includes,
tinyxml2 fetched with BUILD_NAUDIO=OFF removing its only consumers. Both
FetchContent declarations are unconditional.
* Copy soh.o2r from a checked-in prebuilt archive (TEMPORARY)
Unblocks the rest of the migration. soh.o2r is produced by ZAPD via
OTRExporter/extract_assets.py, both of which this migration deletes; its
replacement is a small in-tree packer that has to reproduce ZAPD's
texture quantisation exactly. Rather than block Phases 2/3/5 on that
packer, check the archive in and make GenerateSohOtr a copy.
The archive doubles as Phase 4's acceptance oracle -- the packer is
correct when its output matches this entry-for-entry -- which is strictly
more useful than Gate E's planned manifest of hashes of it.
Generation turns out to be content-deterministic but NOT byte-
deterministic: two runs give identical payloads for all 1,042 entries but
different file hashes, because zip stores per-entry timestamps. So the
Phase 4 comparison must be entry-wise; whole-file would fail even on a
correct archive. Recorded in prebuilt/README.md.
Adds a configure-time guard on the baked-in portVersion. OTRGlobals.cpp
:283 requires exact major.minor.patch equality and RunExtract exit(1)s
with "soh.o2r is outdated", which is an opaque symptom for a stale
checked-in file; verified the guard fires by building against 9.2.4.
ExtractAssets, CI and copy-existing-otrs.cmake are deliberately untouched
-- ExtractAssets still generates a correct archive while ZAPD exists, and
the CI job keeps working since it just builds this target.
* Extract ROMs with Torch instead of ZAPD
Adds soh/assets/yml as a submodule (briaguya0/soh-asset-yml, 20,353 asset
definitions) and FetchContents torch at 4cae4416, configured exactly as
Gate A' measured it: static lib, OoT only, no UI, no StormLib.
find_package(ZLIB) has to run before torch is declared. Torch declares
zlib with OVERRIDE_FIND_PACKAGE, so it takes over the whole build's
find_package(ZLIB), and its copy provides no ZLIB::ZLIB -- which both
StormLib and CMake's FindPNG link by name. Filed upstream as Torch#233.
CallZapd becomes CallTorch: no chdir, no symlinked assets dir, no 22-entry
argv. Everything that needed the working directory is now a parameter.
TorchExtract.cpp is the only TU that includes Companion.h, since torch
exports its whole lib/ as PUBLIC includes.
Progress uses torch's phase callback, which fires once per yml file, with
the denominator counted off disk. Gate A confirmed the two match exactly
for all three yml counts, so the bar runs 0-100 without the sawtooth the
old per-file counter would have produced.
GetZapdVerStr becomes GetTorchVersionDir, returning the version directory
under the yml tree rather than a ZAPD xml directory.
soh.elf links torch and no longer references zapd_report.
* Record the two-ROM session passing in-game
Vanilla then MQ extracted in one process, both loaded and played. Closes
PLAN.md risk #6 in the game rather than only against the harness driver,
and confirms config.yml-driven archive naming end to end.
* Add soh-torch, rebuild ExtractAssets on it
soh links torch as a static library, which compiles out torch's own CLI,
so build-time extraction needs an entry point. soh-torch supplies one
around the same SohTorch::Extract the game calls -- one implementation,
so the CLI and the in-game path can't drift.
ExtractAssets keeps its old contract: SOH_ROM_PATH accepts roms and/or
directories of roms and defaults to roms/, so dropping a vanilla and a
master quest rom in produces oot.o2r and oot-mq.o2r in one run, and
chaining GenerateSohOtr keeps soh.o2r coming out of it too.
Extract now returns the archive name torch chose rather than deriving it
from IsMasterQuest(), so config.yml is the only thing naming archives.
The name comes from Companion::GetOutputPath(): scanning the destination
for *.o2r picks up an unrelated soh.o2r when extracting into the build
directory, which is only invisible in the game because it extracts into
an empty temp dir.
ExtractAssetHeaders stays as a target that explains why it can't run --
torch registers no OoT header exporter and emits no #define d<sym>, which
soh source references ~24k times. Documented in BUILDING.md alongside it.
This removes the last ZAPD dependency from the build.
* Add the roms directory ExtractAssets defaults to
BUILDING.md points people at roms/, but git can't track an empty
directory, so a fresh clone didn't have one. Its .gitignore keeps
everything but itself and the readme out.
* Delete ZAPDTR and OTRExporter
Nothing drives them any more: ExtractAssets runs soh-torch, the in-game
extractor calls Companion directly, GenerateSohOtr copies a prebuilt
archive, and soh links torch instead of ZAPDLib. Verified with a build
from a fresh tree with both submodules absent from disk.
Removes the submodules, the 7,680 ZAPD xmls under soh/assets/xml, the
extractor inputs (Config_*.xml, TexturePool.xml, filelists, symbols) and
copy-existing-otrs.cmake. The xmls remain in this repo's history and in
the test harness's pinned shipwright submodule, which is where the yaml
was generated from.
Packaging changes shape as well as content: the extractor assets used to
install into the 'extractor' component, which Packaging-2.cmake only
includes for the AppImage generator, so linux zips shipped without them.
torch is linked into soh rather than being a separate binary, so the yml
tree installs into 'ship' on every platform and every package can extract
a rom.
* Keep CI providing tinyxml2 itself
Torch's fetched copy satisfies libultraship today, so the install steps
are redundant -- but that rests on torch declaring tinyxml2 with an
unconditional OVERRIDE_FIND_PACKAGE, which is an implementation detail,
and one we've asked upstream to reconsider in Torch#233. If the fetch
gets gated, find_package needs a system copy again and CI would break
because we removed the steps providing it.
Also notes that the generate-soh-otr slim-down was written before the
prebuilt archive, which makes that job build nothing for now.
* Generate soh.o2r again, replacing the prebuilt archive
soh-o2r-packer builds soh.o2r from soh/assets/custom, reproducing the
ZAPD/OTRExporter archive exactly: 1,042 entries, 0 missing, 0 extra, 0
content mismatch, checked entry-wise because zip stores per-entry
timestamps and whole-file hashes never match between runs.
Most of it is torch's: Companion::Pack walks the directory, zips it and
writes portVersion; BaseExporter::WriteHeader writes the resource header;
TextureType and CalculateTextureSize come from TextureUtils. The packer
stages the assets into the shape the archive should have and hands that
over. Encoding a PNG into an N64 texture is the only piece nothing else
provides -- torch decodes rom data that is already N64 format, never the
reverse -- so PngTexture.cpp is the whole of what had to be written, and
it follows ZAPD's quantisation rather than n64graphics', which scales
where ZAPD shifts.
GenerateSohOtr copies the libultraship shaders into assets/custom again
before packing; assets/custom/shaders is gitignored and nothing else
populates it, so a fresh clone would otherwise pack three files short.
Removes prebuilt/ and its configure-time portVersion guard.
* Move the asset tools under soh/assets
soh-torch and soh-o2r-packer both exist to turn assets into archives, so
they sit better beside the assets they read than at the repo root.
Nothing tangled: both targets are declared in the root CMakeLists with
explicit paths rather than add_subdirectory, and soh/CMakeLists.txt globs
only include/, soh/ and src/, so sources under assets/ aren't swept into
the soh target.
* Build soh.o2r in CI without libultraship or soh
SOH_TOOLS_ONLY returns from the root CMakeLists once the asset tools are
declared, before libultraship and soh are added, so a configure that only
needs soh-o2r-packer never reaches LUS's find_package(SDL2 REQUIRED).
That lets generate-soh-otr drop building SDL2 from source, SDL2_net, and
the deps cache that existed to hold the SDL2 tarball -- eleven steps down
to six. Verified a tools-only build produces the archive with all 1,042
entries matching.
The tool targets move above the sub-projects so the early return can sit
between them; they only ever needed torch. The packer creates its output
directory, which previously came for free from soh's build directory.
* Fix macos and windows builds, keep the tools out of the game build
macos: libultraship declares spdlog with OVERRIDE_FIND_PACKAGE so its own
find_package(spdlog REQUIRED) resolves, but torch declares spdlog without
it, and FetchContent_Declare is first-wins. Configuring torch first made
LUS's declaration a no-op and left find_package with no config to find.
It only showed on macos because that's the platform where neither project
finds an installed spdlog -- it's kept out of macports.yml because its fmt
dependency breaks the universal build -- so both fall through to fetching.
Declaring it ahead of both keeps macos on the same source-built v1.16.0 it
used before. Torch already uses OVERRIDE_FIND_PACKAGE for tinyxml2 and
zlib, so the real fix is upstream.
windows: soh builds with /WX, and TorchExtract.cpp is the one translation
unit reaching yaml-cpp through torch's headers, which trips the
dll-interface warnings. Suppress those two on that file.
EXCLUDE_FROM_ALL on soh-torch and soh-o2r-packer: they are build-time
tools, so building the game shouldn't compile them. ExtractAssets and
GenerateSohOtr still pull them in through DEPENDS.
Drops the .gitattributes rule for the prebuilt archive, which is gone.
* Drop the gitignore negation for the prebuilt archive
Left over with the .gitattributes rule when prebuilt/ was removed.
* TEMPORARY: put soh/assets/xml back so the branch diff is reviewable
Deleting 7,680 xmls puts the diff past what GitHub will render, and they
are 99.8% of it. Restoring them here leaves the ~46 files that actually
changed, which reviews in the compare view.
Revert this before opening the PR.
* Remove the migration planning docs
Scaffolding for the migration itself rather than anything the repo needs
to carry afterwards.
* Fix NTSC master quest roms being rejected as unsupported
verMap's two master quest rows keyed off the GameCube constants instead
of the MQ ones, so with duplicate keys dropped the map never contained
OOT_NTSC_US_MQ or OOT_NTSC_JP_MQ and both roms were filtered out before
extraction. IsMasterQuest already handles both, and the asset yml covers
ntsc_u_mq and ntsc_j_mq, so keying them correctly is all that's needed.
Predates this migration; the torch cli was unaffected because it hashes
the rom rather than consulting verMap.
* Revert "TEMPORARY: put soh/assets/xml back so the branch diff is reviewable"
This reverts commit
|
||
|
briaguya
|
e1462b07e6 |
fix and improve test builds on distros workflow (#6748)
* Make distro test matrix dynamic Resolve Ubuntu LTS and Fedora releases from endoflife.date at workflow run time so the matrix tracks in-support versions automatically, and use Debian's rolling oldstable/stable/testing tags. Switch install-step gating from image-string equality to a packageManager key on each distro entry. Drop the fedora:39 nlohmann workaround now that fedora:39 is no longer in the matrix. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Bootstrap git and read apt deps from apt-deps.txt Add a per-package-manager step that installs git first so the checkout can fetch submodules inside the container, then move checkout above the main install steps so they have repo files on disk. The apt install now sources its package list from .github/workflows/apt-deps.txt, matching generate-builds.yml. Drop the redundant echo-the-command-then-run-it lines from each install step. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Move libzip deps into apt-deps.txt The libzip family (libzip-dev, zipcmp, zipmerge, ziptool) was previously appended to each workflow's apt install line because the appimage build needs libzip without crypto support. Add it to apt-deps.txt instead and have the appimage build job apt-remove libzip-dev before its from-source rebuild, mirroring the existing tinyxml2 pattern. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Drop libopengl-dev from distro test workflow libglew-dev (in apt-deps.txt) already pulls in the OpenGL headers via libgl-dev / libglvnd-dev, and generate-builds.yml has been building without libopengl-dev for a long time. The line was originally added on a guess; removing it brings the test workflow into alignment. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Move git, cmake, lsb-release into apt-deps.txt These are real build-time deps (git for submodule checkout, cmake for the build, lsb-release for soh CMakeLists distro detection). Moving them into apt-deps.txt makes the file a single canonical answer to what apt packages are needed to build, and reduces the distro test workflow's apt install line to just the compiler plus the file. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Move apt deps file out of .github and point BUILDING.md at it Rename .github/workflows/apt-deps.txt to linux-build-deps/apt.txt so the canonical list lives at a path users can reasonably be told to look at. Update both workflows to the new path. BUILDING.md grows a "Clone the repo and enter the directory" section before Install dependencies so its apt commands can source from the file via $(cat ...) — the clone snippet is dropped from the later Build block to avoid duplication. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Add cmake version verification section to BUILDING.md Older distros ship cmake too old for this project; point users at how to check their version against the project's minimum and link to a few ways to install a newer cmake (pypi, kitware apt repo, Homebrew). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Add cmake verify/update step to distro test workflow ubuntu:22.04 and debian:oldstable ship cmake older than the project's minimum (3.26+), failing at configure time. Add a step before Build SoH that compares the installed cmake to the project minimum and, if too old, installs a newer cmake via pipx (per-distro pipx package). The new cmake's bin dir is added to GITHUB_PATH so the Build SoH step picks it up. Mirrors the BUILDING.md guidance pointing users at pypi cmake. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Set ccache on PATH once per job in generate-builds Replace the repeated per-step `export PATH="/usr/lib/ccache:..."` with a single "Add ccache to PATH" step in each Linux job that writes both ccache dirs to GITHUB_PATH. From there it persists for every subsequent step in that job, so each from-source build (SDL, SDL_net, tinyxml2, libzip) and the final cmake compile pick up ccache automatically. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Extract tinyxml2 from-source build into a composite action generate-builds had the same 13-line tinyxml2 install block in both the generate-soh-otr and build-linux jobs, and the distro test workflow is about to need the same logic conditionally. Move it into a composite action at .github/actions/install-tinyxml2 and have generate-builds use it. The action only builds and installs from source; removing the distro package stays in the caller since that command is package-manager specific. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Add tinyxml2 troubleshooting tip to BUILDING.md Older distros ship tinyxml2 pre-10.0.0 which doesn't include the cmake config file the project's find_package call needs. Point users at either brew or the install-tinyxml2 composite action script when they hit the "Could not find a package configuration file" error. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Install latest tinyxml2 in distro test workflow when missing ubuntu:22.04 ships libtinyxml2-dev 9.0.0, which is before tinyxml2 started providing a cmake config file, so find_package(tinyxml2) fails. Add a step (apt-only) that checks for tinyxml2Config.cmake on disk and, if missing, removes the distro package and calls the install-tinyxml2 composite action to build 10.0.0 from source. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Extract SDL2_net from-source build into a composite action Same shape as the install-tinyxml2 action: generate-builds had identical SDL2_net install blocks in both the generate-soh-otr and build-linux jobs, and the distro test workflow is about to need the same logic conditionally. Move it into a composite action at .github/actions/install-sdl2-net. The cp of /usr/local/lib/libSDL* into the multiarch lib dir stays in the caller since it's appimage-specific. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Refactor cmake-config troubleshooting tip and add SDL2_net Restructure the troubleshooting tip into a generic "older distros ship packages without the cmake config files" framing with a list of known failing package versions, then two paths to install a newer version: Homebrew (with a note about CMAKE_PREFIX_PATH) or building from source using the install-* composite actions as reference. Add SDL2_net to the list alongside tinyxml2. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Install latest SDL2_net in distro test workflow when missing ubuntu:22.04 ships libsdl2-net-dev 2.0.x, which is before SDL2_net started providing a cmake config file, so find_package(SDL2_net) fails silently and the link step errors on the missing SDL2_net::SDL2_net target. Mirror the tinyxml2 pattern: an apt-only check for sdl2_net-config.cmake on disk that triggers the install-sdl2-net composite action when missing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Document minimum GCC and Clang versions Minimums pinned empirically by walking up versions on the test-gcc-10 test branch until each compiler built clean. The version numbers live in linux-build-deps/minimum-{gcc,clang}-version.txt so the workflow can read them and BUILDING.md can link to a single source of truth. Test runs that pinned the floors: - GCC 9 failure (missing <compare> header): https://github.com/briaguya0/Shipwright/actions/runs/27491491936 - GCC 10 success: https://github.com/briaguya0/Shipwright/actions/runs/27490774081 - Clang 15 failure (structured-binding-capture in lambda): https://github.com/briaguya0/Shipwright/actions/runs/27491715930 - Clang 16 success: https://github.com/briaguya0/Shipwright/actions/runs/27492790573 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Add verify-compiler and install-newer-compiler actions Two composite actions that work together to keep the compiler used by the build at or above the project minimum. verify-compiler reads linux-build-deps/minimum-${compiler}-version.txt, compares against the installed default, and emits four outputs: needs_install, available_in_distro, cc, cxx, version. install-newer-compiler consumes those and installs ${compiler}-${version} either from the distro repos (apt or zypper) or via apt.llvm.org for clang on apt distros. Any combination without a known install path fails with "Minimum version not readily available. An alternative installation method for ${compiler} ${version} is needed." Validated on the test-verify-compiler branch via two dedicated test workflows that exercise every reachable matrix combination and assert expected outputs / install outcomes: - verify-compiler: https://github.com/briaguya0/Shipwright/actions/runs/27510863067 - install-newer-compiler: https://github.com/briaguya0/Shipwright/actions/runs/27512420765 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Wire verify-compiler and install-newer-compiler into distro test workflow Between the dependency-install step and the cmake verify step, run verify-compiler and (conditionally) install-newer-compiler so distros that ship a compiler below the project minimum get a newer one. Build SoH's CC/CXX env now reads the resolved binary names from verify-compiler's outputs instead of using matrix.cc/cxx directly, so a freshly-installed gcc-N/clang-N actually gets used by cmake. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Add fmt/clang consteval workaround tip to BUILDING.md Point users at the fmtlib/fmt#4807 issue and the -DCMAKE_CXX_FLAGS=-DFMT_CONSTEVAL=constexpr workaround when they hit the "call to consteval function" error while building with clang. Affects distros that ship libfmt 10.x with newer clang. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Probe fmt/clang consteval compat and apply workaround when needed Self-checking version of the BUILDING.md tip: between the dep-install steps and Build SoH, try to compile the minimal repro from fmtlib/fmt#4807. If the compile fails, the build job adds -DCMAKE_CXX_FLAGS=-DFMT_CONSTEVAL=constexpr to cmake's invocation; otherwise the flag stays empty. The probe uses the resolved CXX from verify-compiler, so it works regardless of whether the compiler came from the distro or a freshly-installed newer version. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Move Arch packages into linux-build-deps/pacman.txt Mirror the apt pattern: extract the Arch package list to linux-build-deps/pacman.txt (everything except the compiler) and point both BUILDING.md and the distro test workflow at it. Also brings the workflow's Arch install up to parity with BUILDING.md — it was missing opusfile and libvorbis. Add linux-build-deps/README.md explaining that apt.txt is verified on every push but the other per-distro lists can drift, and inviting PRs / issues / Discord messages when something's missing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Move Fedora packages into linux-build-deps/dnf.txt Mirror the apt/pacman pattern: extract the Fedora package list to linux-build-deps/dnf.txt (everything except the compiler and the gcc-c++ companion package) and point both BUILDING.md and the distro test workflow at it. Brings the workflow's Fedora install up to parity with BUILDING.md — it was missing SDL2_net-devel, nlohmann-json-devel, opusfile-devel, and libvorbis-devel. Also drops the leftover wget that was only needed for an old from-source workaround. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Move openSUSE packages into linux-build-deps/zypper.txt Mirror the apt/pacman/dnf pattern: extract the openSUSE package list to linux-build-deps/zypper.txt and point the distro test workflow at it. The list adds SDL2_net-devel and the audio family (libogg-devel, libvorbis-devel, libopus-devel, opusfile-devel) that the workflow's inline list was missing — package names verified in a Tumbleweed distrobox. Also adds a new openSUSE section to BUILDING.md (it wasn't there before), with libstdc++-devel in the clang line because clang on openSUSE doesn't pull in libstdc++ headers transitively the way it does on other distros. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Add in-support openSUSE Leap releases to the distro test matrix Fetch openSUSE cycles from endoflife.date and include any with an EOL date in the future as opensuse/leap:${cycle} images alongside the rolling Tumbleweed entry. Today that's just Leap 16.0 (15.6 went EOL 2026-04-30); new Leap releases will appear automatically as they ship and old ones drop off as they EOL. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Move Nix flake into linux-build-deps/flake.nix Extract the flake.nix content out of BUILDING.md's inline code block into linux-build-deps/flake.nix and update the Nix section to point at it. Users run `nix develop ./linux-build-deps` from the repo root instead of copying the flake out to a file they have to maintain themselves. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Add build-nix job to distro test workflow Verifies that `nix develop ./linux-build-deps` produces a shell that builds SoH. Runs on its own (no distro matrix) and uses cachix/install-nix-action per nix.dev's CI guidance. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Drop LINUX_RUNNER override from distro test workflow vars.LINUX_RUNNER isn't referenced anywhere else in .github/ — the override is dead code. Replace all three call sites with plain ubuntu-latest. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
David Racine
|
1894ffca46 |
docs: fix missing sdl2-net (#6740)
Add the sdl2-net dev package that was missing from the Fedora and macOS lists |
||
|
ProverbialPennance
|
80de5cc179 |
example flake - add missing deps, pin clang14 (#6531)
`Python3`, used for several CMake targets. `Zenity`, a still used runtime dependency. `xorg.libX11` -> `libx11`, packageset deprecated. `imagemagick`, included as per @IQubic's research. `clang_14`, clang-formatter used by decomp, thus also used by HM64 |
||
|
Philip Dubé
|
b6df96c530 |
Add dependencies to docs/BUILDING.md (#6304)
Remove openSUSE because I can't figure out their package convention |
||
|
Demur Rumed
|
7588734a42 | Merge remote-tracking branch 'origin/develop-copper' into merge912 | ||
|
Philip Dubé
|
c71cc68951 | Update references to soh.otr/oot.otr/oot-mq.otr to .o2r (#6161) | ||
|
Shishu the Dragon
|
9d8addca04 | Update macOS build instructions (#6012) | ||
|
enzu.ru
|
577c5639df |
Improve Nix support (#5777)
* Fix FindOpusFile NixOS bug * better documentation |
||
|
Philip Dubé
|
4c547416cc |
replace MT RNG with PCG (#4973)
replace boost hashing with FNV-1a removes boost as a dependency |
||
|
Lord Jabu Jabu
|
3357543538 |
Dev Feature - Create VS Code Build tasks and update build documentation (#4895)
* Added vs code build tasks * Fix typo in BUILDING.md regarding CMake Tools plugin * Add "Build All" task to VS Code configuration |
||
|
Reuben D'Souza
|
fc8d581efb |
fix: readme typos (#4768)
* fix: change complier to compiler * fix: repeated the |
||
|
lemonslut
|
16d69e4a28 |
Update macOS development dependency list (#4648)
- Adds a few extras I had to install to build - Adds link to [homebrew](https://brew.sh/) - Adds `brew install ...` command to setup script |
||
|
Malkierian
|
108d5061d4 |
Small Warnings Refactor (#4477)
* Disable all warnings, even in release, on the `src` directory. Resolve math macro duplication warnings. * Suppress LUS warnings. * Modify it to utilize a variable that defaults on but can be specified in command line to disable it. Prevet total compile option overwrite for LUS. * Remove unnecessary unset and cache parameters. * Document warnings flag in BUILDING.md |
||
|
Malkierian
|
1df91890fe |
Windows Build Assets Tweak (#4486)
* Change CMake process to copy assets necessary for OTR creation to VS run directory. Remove reference to ExtractAssets target from Windows build instructions. * Unify build instructions where appropriate across PC platforms. Clarify commented parts of some commands. * PR feedback. |
||
|
MegaMech
|
90c07bb3ec | Update BUILDING.md (#4415) | ||
|
Malkierian
|
48663d74ac |
Change Recommended VS build tools from 142 to 143 (#4315)
* Changed references for VS build tools 142 to 143 to reflect current state of development. * Remove VS 2019. |
||
|
briaguya
|
23f9654940 |
update BUILDING.md (#4128)
|
||
|
briaguya
|
fb29c827ad | Update BUILDING.md (#4029) | ||
|
briaguya
|
ebde648c59 |
.o2r support (#3955)
|
||
|
briaguya
|
1bc15d5bf3 |
document dependencies per distro (#3933)
* start on arch * noconfirm * git * noconfirm * just skip cache * cmake * ninja * gcc * lsb * sdl2 * png * sdl net * boost * imagemagick and add opensuse * non interactive * git * cmake * add a bunch * things * more * try to matrix * soh otr i guess * needs * trying to figure out how matrix include works * make the matrix? * matrix debug * install? * echo? * yuck * cannot locate package false * parens for good measure * lsb * libsdl2 * suse * i forgot opensuse uses gcc-c++ instead of g++ for the package name * png * sdl * more * suse packages.... * fedora cmake * 2? * png * handle old sdl2 net * apt * not sure if this is a problem for both clang and gcc on opensuse or just clang * devel * cmake min version * def don't want this here but it's here for now * does this do anything * mantic because cmake version * libopengl? * fedora ninja * fedora g++ * lsb * fedora sdl * libpng * do a build * clang fix * give early returns values * ImageMagick * Update BUILDING.md * something * plain boost instead of devel? * not packing so we don't need this part * base? * don't have it can't need it * just use boost download on opensuse i guess * move workflow * bring back the file * lus main? * detach |
||
|
Adam Bird
|
1829a6a154 | Update buildBUILDING.md and adjust cpack for Windows (#3229) | ||
|
PoorPockets McNewHold
|
098d5a8044 |
Add pulseaudio-libs to the necessaries dependencies (#2928)
Create a crash on extraction for systems using pipewire *(and possibly other sounds systems)*. Co-authored-by: aMannus <mannusmenting@gmail.com> |
||
|
briaguya
|
df6c01732a | docs: remove outdated docs from BUILDING.md (#3213) | ||
|
briaguya
|
d69c07cd75 |
numbers aren't magic (#2976)
* numbers aren't magic * chest style matches contents renamed everything except the cvar itself at this point (waiting for versioned configs for that) * tracker as part of names * finish the tracker rename --------- Co-authored-by: briaguya <briaguya@alice> |
||
|
briaguya
|
4166dbf907 |
spockalicious (#2751)
* Rough mockup of LUS XML loading * Updated code for merge * Loading from FS support and custom DList WIP implementation * Added current directory support to F3D and impl most of the dlist cmds * WIP Skeleton support * Almost done * Rebase fixes * Submodule updates * HD Texture Support * Fixes * bump lus * fix exporter build, header update * soh builds * setMesh image path cleanup * Update soh/src/overlays/actors/ovl_player_actor/z_player.c * Update soh/src/overlays/actors/ovl_player_actor/z_player.c * Update OTRExporter/OTRExporter/Main.cpp * Update ZAPDTR/ZAPD/ZResource.h * Update soh/src/code/z_skelanime.c * Update OTRExporter/OTRExporter/Main.cpp * Fixed jpeg backgrounds and decreased icon buffer size * Bump lus * Increased even more the buffer because it crashes on long texts * Removed print because sometimes the if is not triggered when the image is already byteswapped * fix non-windows build * fix build Co-authored-by: Kenix <kenixwhisperwind@gmail.com> * add hd checkbox * Various fixes for custom model support (#23) * Some fixes * Updated LUS Version * Fixed issue with Link Skirt on pause menu * Added CVar for custom link model changes * Fixed headers * Additional header fixes * Tweaks * Unload HD game assets on scene transition. (#16) * Unload game assets on scene transition. * Bump LUS * Unloads all HD assets on scene transition. * Only unload hd assets if hd assets are turned on. * Fixes issues on toggling between HD and non HD assets. --------- Co-authored-by: briaguya <briaguya@alice> * fix: actually load hd debug font (#27) * fix: actually load hd debug font * toggle debug text correctly --------- Co-authored-by: briaguya <briaguya> * Yes. (#28) * Merge branch 'develop' into dev-to-ghost * HD Skeleton Swapping and Language Fixes (#32) * Yes. * HD Skeleton Swapping and Language Fixes * Test * Fixed issues with ganon cape (#34) * Fixed Bongo Bongo Crash (#35) * Added HD Assets Toggle (#37) * Ivan the Fairy - Coop Mode (#36) * wip * hookshotable ivan * added hookshot item * new items & changes & fixes & restored navi * farore, din and nayru's spells are done * fixed slingshot & bow * added more items supported * done with all main items * bug fixes & ready * added imgui button * wip * hookshotable ivan * added hookshot item * new items & changes & fixes & restored navi * farore, din and nayru's spells are done * fixed slingshot & bow * added more items supported * fix own dungeon items on shuffled boss rooms (#2683) * bump lus (#2692) * fix: lowercase package names for vcpkg (#2693) vcpkg was throwing an error `error: invalid character in package name (must be lowercase, digits, '-')` this updates our calls to `vcpkg_install_packages` to use lowercase package names instead of uppercase * fix death mountain cloud in rando (#2691) * Fix: Switch Age No Longer Reloads Start Room (#2679) * [Reduced Clutter] Disable Hot/Underwater Warning Text (#2684) * Disable Warning Text * Moved to Reduced Clutter * done with all main items * bug fixes & ready * fix: process roms in consistent order (#2696) * chore: move rando savefile setup and document flags (#2697) * remove rando save init from sram * move rando savefile init logic and set more flags * document flags for rando save creation * Fix: Use correct fps value for frame interpolation with match refresh rate (#2694) * Fix: Kak GS placement on construction site (#2695) * added imgui button * addressed kenix's comments * fixed useless null * added rupee dash mode in extra modes * changed menu position --------- Co-authored-by: Adam Bird <Archez@users.noreply.github.com> Co-authored-by: briaguya <70942617+briaguya-ai@users.noreply.github.com> Co-authored-by: inspectredc <78732756+inspectredc@users.noreply.github.com> Co-authored-by: Patrick12115 <115201185+Patrick12115@users.noreply.github.com> * LUS Scancodes (#42) * Added HD Assets Toggle * Switched out SDL for LUS scancodes * Ivan tweaks (#45) * Magic consumption slowed down; Bosses now affected by Ivan's Din spell * Adjust magic timer * clean up imgui * model fixes/improvements (#50) * replace `gUseCustomLinkModel` with custom resource check * handle adult/child * bump lus * fix model switching with tab * use lus main * fix carpet man (#52) Co-authored-by: Rozelette <Rozelette@users.noreply.github.com> * get ship model and lus texture into soh.otr, use `gAuthenticLogo` to toggle between ship and authentic (#55) * Use libultra features for CPU-modified textures (#40) * Use libultra features for CPU-modified textures * Comment * bump lus on ghost (#58) * fix: properly use `Interface_LoadActionLabel` to display start button text (#61) * Changes hd -> alt for texture replacement. (#65) * Changes hd -> alt for texture replacement. * Renames variables in gfxprint for hd -> alt change. * Update soh/soh/resource/type/Skeleton.cpp --------- Co-authored-by: briaguya <70942617+briaguya-ai@users.noreply.github.com> * Fixes kaleido dungeon maps (#67) * skeleton stuff (#69) * comment out wii u build (#70) * bump lus (#71) * Rework readme (#72) * Update README.md * docs * put custom music docs somewhere * Update README.md * Update README.md * Update README.md * Update README.md * dark/light title image * lus * Update README.md * Fixed vanilla minimap (#73) * Fixed vanilla minimap * Workaround for pulsing SD maps with non-broken HD maps. --------- Co-authored-by: Christopher Leggett <chris@leggett.dev> * Skeleton fix fixed (#75) * WIP skelton patcher fix * Fixes skeleton reference change. * Adds const back to name in ResourceMgr_LoadSkeletonByName --------- Co-authored-by: Christopher Leggett <chris@leggett.dev> Co-authored-by: Kenix <kenixwhisperwind@gmail.com> * Fixes z_message_otr memory leak. * Update soh/soh/z_message_OTR.cpp * Update soh/src/code/game.c * docs: add how to find otr files to switch instructions (#78) * bump lus (#79) * comment out RegisterBlendedTexture in king d (#80) --------- Co-authored-by: Nicholas Estelami <NEstelami@users.noreply.github.com> Co-authored-by: David Chavez <david@dcvz.io> Co-authored-by: briaguya <briaguya@alice> Co-authored-by: Kenix3 <kenixwhisperwind@gmail.com> Co-authored-by: KiritoDv <kiritodev01@gmail.com> Co-authored-by: briaguya <briaguya> Co-authored-by: Ralphie Morell <stratomaster64@gmail.com> Co-authored-by: MelonSpeedruns <melonspeedruns@outlook.com> Co-authored-by: Adam Bird <Archez@users.noreply.github.com> Co-authored-by: inspectredc <78732756+inspectredc@users.noreply.github.com> Co-authored-by: Patrick12115 <115201185+Patrick12115@users.noreply.github.com> Co-authored-by: Rozelette <Rozelette@users.noreply.github.com> Co-authored-by: Christopher Leggett <chris@leggett.dev> Co-authored-by: Lywx <36680385+KiritoDv@users.noreply.github.com> |