mirror of
https://github.com/HarbourMasters/Shipwright
synced 2026-08-12 12:18:32 -04:00
Copy soh.o2r from a checked-in prebuilt archive (TEMPORARY)
Unblocks the rest of the migration. soh.o2r is produced by ZAPD via OTRExporter/extract_assets.py, both of which this migration deletes; its replacement is a small in-tree packer that has to reproduce ZAPD's texture quantisation exactly. Rather than block Phases 2/3/5 on that packer, check the archive in and make GenerateSohOtr a copy. The archive doubles as Phase 4's acceptance oracle -- the packer is correct when its output matches this entry-for-entry -- which is strictly more useful than Gate E's planned manifest of hashes of it. Generation turns out to be content-deterministic but NOT byte- deterministic: two runs give identical payloads for all 1,042 entries but different file hashes, because zip stores per-entry timestamps. So the Phase 4 comparison must be entry-wise; whole-file would fail even on a correct archive. Recorded in prebuilt/README.md. Adds a configure-time guard on the baked-in portVersion. OTRGlobals.cpp :283 requires exact major.minor.patch equality and RunExtract exit(1)s with "soh.o2r is outdated", which is an opaque symptom for a stale checked-in file; verified the guard fires by building against 9.2.4. ExtractAssets, CI and copy-existing-otrs.cmake are deliberately untouched -- ExtractAssets still generates a correct archive while ZAPD exists, and the CI job keeps working since it just builds this target.
This commit is contained in:
briaguya
@@ -1 +1,6 @@
|
||||
* text=auto eol=lf
|
||||
|
||||
# TEMPORARY (Torch migration): checked-in archive. text=auto would almost certainly
|
||||
# detect this as binary anyway, but an eol conversion on a 4.4 MB zip would corrupt it
|
||||
# silently, so say so explicitly. Remove with prebuilt/ -- see prebuilt/README.md.
|
||||
prebuilt/soh.o2r binary
|
||||
|
||||
@@ -461,3 +461,6 @@ soh/properties.h
|
||||
/clang-format
|
||||
/clang-format.exe
|
||||
*.o2r
|
||||
# TEMPORARY (Torch migration): the checked-in soh.o2r that GenerateSohOtr copies.
|
||||
# Remove this negation when Phase 4's packer replaces it -- see prebuilt/README.md.
|
||||
!prebuilt/soh.o2r
|
||||
|
||||
+44
-12
@@ -244,20 +244,52 @@ add_custom_target(
|
||||
DEPENDS ZAPD
|
||||
)
|
||||
|
||||
# Target to generate only soh.o2r
|
||||
# TEMPORARY (Torch migration): soh.o2r is copied from prebuilt/, not generated.
|
||||
#
|
||||
# The real producer drives ZAPD via OTRExporter/extract_assets.py, and the migration deletes
|
||||
# both. Its replacement -- a small in-tree packer, Phase 4 of torch-migration/PLAN.md -- has to
|
||||
# reproduce ZAPD's texture quantisation exactly, so rather than block Phases 2/3/5 on it, the
|
||||
# archive is checked in and copied into place. See prebuilt/README.md for provenance and for how
|
||||
# to restore a real build step (which must also bring back the libultraship shader copy that the
|
||||
# copy-only version below has no use for -- the shaders are already inside the archive).
|
||||
#
|
||||
# Fail at configure time if the prebuilt archive's baked-in portVersion no longer matches the
|
||||
# project version. OTRGlobals.cpp:283 demands exact major.minor.patch equality and RunExtract
|
||||
# exit(1)s with "soh.o2r is outdated" -- an obscure symptom for what is really a stale checked-in
|
||||
# file. portVersion is 7 bytes: endianness, then u16 major/minor/patch big-endian.
|
||||
set(SOH_PREBUILT_O2R "${CMAKE_CURRENT_SOURCE_DIR}/prebuilt/soh.o2r")
|
||||
if(NOT EXISTS "${SOH_PREBUILT_O2R}")
|
||||
message(FATAL_ERROR "Missing ${SOH_PREBUILT_O2R} -- see prebuilt/README.md")
|
||||
endif()
|
||||
file(READ "${SOH_PREBUILT_O2R}" _soh_o2r_hex HEX)
|
||||
string(FIND "${_soh_o2r_hex}" "706f727456657273696f6e" _pv_at) # "portVersion" as hex
|
||||
if(_pv_at EQUAL -1)
|
||||
message(FATAL_ERROR "No portVersion entry in ${SOH_PREBUILT_O2R} -- see prebuilt/README.md")
|
||||
endif()
|
||||
math(EXPR _pv_at "${_pv_at} + 22") # skip the 11-character name
|
||||
string(SUBSTRING "${_soh_o2r_hex}" ${_pv_at} 14 _pv)
|
||||
string(SUBSTRING "${_pv}" 2 4 _pv_major)
|
||||
string(SUBSTRING "${_pv}" 6 4 _pv_minor)
|
||||
string(SUBSTRING "${_pv}" 10 4 _pv_patch)
|
||||
math(EXPR _pv_major "0x${_pv_major}")
|
||||
math(EXPR _pv_minor "0x${_pv_minor}")
|
||||
math(EXPR _pv_patch "0x${_pv_patch}")
|
||||
if(NOT "${_pv_major}.${_pv_minor}.${_pv_patch}" STREQUAL "${CMAKE_PROJECT_VERSION}")
|
||||
message(FATAL_ERROR
|
||||
"prebuilt/soh.o2r is stamped ${_pv_major}.${_pv_minor}.${_pv_patch} but this build is "
|
||||
"${CMAKE_PROJECT_VERSION}. The game would reject it at startup (\"soh.o2r is outdated\"). "
|
||||
"Regenerate it -- see prebuilt/README.md.")
|
||||
endif()
|
||||
unset(_soh_o2r_hex)
|
||||
|
||||
# Target to place soh.o2r (copy of the prebuilt archive)
|
||||
add_custom_target(
|
||||
GenerateSohOtr
|
||||
COMMAND ${CMAKE_COMMAND} -E rm -f soh.o2r
|
||||
|
||||
# copy LUS default shaders into assets/custom
|
||||
COMMAND ${CMAKE_COMMAND} -E rm -r -f ${CMAKE_CURRENT_SOURCE_DIR}/soh/assets/custom/shaders/
|
||||
COMMAND ${CMAKE_COMMAND} -E copy_directory ${CMAKE_CURRENT_SOURCE_DIR}/libultraship/src/fast/shaders/ ${CMAKE_CURRENT_SOURCE_DIR}/soh/assets/custom/shaders/
|
||||
|
||||
COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/OTRExporter/extract_assets.py -z "$<TARGET_FILE:ZAPD>" --norom --custom-otr-file soh.o2r "--custom-assets-path" ${CMAKE_CURRENT_SOURCE_DIR}/soh/assets/custom --port-ver "${CMAKE_PROJECT_VERSION}"
|
||||
COMMAND ${CMAKE_COMMAND} -DSYSTEM_NAME=${CMAKE_SYSTEM_NAME} -DTARGET_DIR="$<TARGET_FILE_DIR:ZAPD>" -DSOURCE_DIR=${CMAKE_CURRENT_SOURCE_DIR} -DBINARY_DIR=${CMAKE_BINARY_DIR} -DONLYSOHOTR=On -P ${CMAKE_CURRENT_SOURCE_DIR}/copy-existing-otrs.cmake
|
||||
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/soh
|
||||
COMMENT "Generating soh.o2r..."
|
||||
DEPENDS ZAPD
|
||||
COMMAND ${CMAKE_COMMAND} -E copy_if_different "${SOH_PREBUILT_O2R}" ${CMAKE_CURRENT_SOURCE_DIR}/soh.o2r
|
||||
COMMAND ${CMAKE_COMMAND} -E copy_if_different "${SOH_PREBUILT_O2R}" ${CMAKE_BINARY_DIR}/soh/soh.o2r
|
||||
# No BYPRODUCTS: ExtractAssets already claims ${CMAKE_SOURCE_DIR}/soh.o2r as one of its
|
||||
# own (see above), and two rules generating the same path is a hard ninja error.
|
||||
COMMENT "Copying prebuilt soh.o2r (TEMPORARY -- see prebuilt/README.md)..."
|
||||
)
|
||||
|
||||
if(CMAKE_SYSTEM_NAME MATCHES "Linux")
|
||||
|
||||
@@ -0,0 +1,87 @@
|
||||
# Prebuilt `soh.o2r` — **temporary**
|
||||
|
||||
This directory exists to unblock the Torch migration and is **meant to be deleted**.
|
||||
|
||||
`soh.o2r` is the port's own asset archive: fonts, textures, presets, translations, shaders —
|
||||
everything under `soh/assets/custom/`, plus a `portVersion` stamp. It is built by
|
||||
`GenerateSohOtr`, which today drives ZAPD through `OTRExporter/extract_assets.py`. The migration
|
||||
deletes ZAPDTR and OTRExporter, so that producer goes away and its replacement — a small in-tree
|
||||
packer (Phase 4 of `torch-migration/PLAN.md`) — has to reproduce it byte-for-byte, including
|
||||
ZAPD's specific texture quantisation.
|
||||
|
||||
Rather than block every other phase on that packer, the archive is checked in here and
|
||||
`GenerateSohOtr` just copies it into place. **Phases 2, 3 and 5 become testable without the
|
||||
packer existing.**
|
||||
|
||||
This file also serves as Phase 4's acceptance oracle: the packer is correct when its output
|
||||
matches this archive entry-for-entry. That replaces Gate E's planned sha256 manifest — the
|
||||
artifact is strictly more useful than a list of hashes of it.
|
||||
|
||||
## Provenance
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Generated | 2026-07-25 |
|
||||
| Shipwright branch | `torch-migration` @ `fe2f52248` |
|
||||
| `develop` at the time | `c4e92a70f` (inputs verified identical — `git diff develop HEAD -- soh/assets/custom libultraship CMakeLists.txt` was empty) |
|
||||
| libultraship submodule | `c57da1b4` |
|
||||
| `CMAKE_PROJECT_VERSION` | `9.2.3` |
|
||||
| Producer | `cmake --build build --target GenerateSohOtr` (ZAPD `botr -se OTR --norom`) |
|
||||
| Size / entries | 4,414,957 bytes / 1,042 entries (1,038 checked-in assets + 3 LUS shaders + `portVersion`) |
|
||||
| sha256 | `1d23b582fbc29cebf803b6ee54856089d0978e5496e6d4b1dbcde876e13cf13c` |
|
||||
| sha256 of inputs | `de8ba0b5167bec6956258dd0445d01ac0f6daac84aff091c678d951cc72598f6` |
|
||||
|
||||
The input hash covers every file under `soh/assets/custom/` except `shaders/`, which is not
|
||||
checked in — it is copied out of `libultraship/src/fast/shaders/` at generation time. Recompute it
|
||||
with:
|
||||
|
||||
```sh
|
||||
find soh/assets/custom -type f -not -path '*/shaders/*' -print0 | sort -z \
|
||||
| xargs -0 sha256sum | sha256sum
|
||||
```
|
||||
|
||||
(`-print0`/`sort -z` are required: several preset filenames contain spaces.)
|
||||
|
||||
## Generation is content-deterministic, not byte-deterministic
|
||||
|
||||
Two consecutive runs produce archives with **identical payloads for all 1,042 entries** but
|
||||
**different whole-file hashes** — the zip stores a modification timestamp per entry. So:
|
||||
|
||||
- The sha256 above identifies *this* file, not "any correct archive".
|
||||
- **Phase 4's acceptance test must compare entry-by-entry**, never whole-file. A byte-identical
|
||||
archive is not achievable and not the goal.
|
||||
|
||||
## What this pins
|
||||
|
||||
The archive freezes three inputs. If any changes while this workaround is in place, the checked-in
|
||||
copy is stale and **nothing will tell you at runtime** except missing or wrong assets:
|
||||
|
||||
1. **`soh/assets/custom/`** — edit an asset and your change is silently ignored. Compare the input
|
||||
hash above.
|
||||
2. **libultraship's shaders** — `libultraship/src/fast/shaders/` is copied in at generation time,
|
||||
so a LUS submodule bump can drift the three shaders inside this archive.
|
||||
3. **The project version.** `portVersion` here is `9.2.3`. `OTRGlobals.cpp:283` requires exact
|
||||
`major.minor.patch` equality against the build's version, and `RunExtract` calls `exit(1)` with
|
||||
"soh.o2r is outdated" if it fails. A configure-time check in the root `CMakeLists.txt` catches
|
||||
this and tells you to regenerate, because the runtime symptom is not obvious.
|
||||
|
||||
## Regenerating
|
||||
|
||||
From a tree where ZAPDTR/OTRExporter still build (i.e. `develop`):
|
||||
|
||||
```sh
|
||||
cmake --build build --target GenerateSohOtr
|
||||
cp soh.o2r prebuilt/soh.o2r
|
||||
```
|
||||
|
||||
Then update the provenance table above.
|
||||
|
||||
## Removing this
|
||||
|
||||
When Phase 4's packer lands and its output matches this archive entry-for-entry, delete this
|
||||
directory, drop the `!prebuilt/soh.o2r` negation from `.gitignore`, and restore `GenerateSohOtr`
|
||||
to a real build step (including the libultraship shader copy, which the copy-only version skips).
|
||||
|
||||
**Delete it before the migration PR merges** if at all possible. It is a 4.4 MB binary, and a blob
|
||||
that reaches `develop` is in the history permanently; removed in the final tree and squash-merged,
|
||||
it never lands there at all.
|
||||
Binary file not shown.
Reference in New Issue
Block a user