21 Commits

Author SHA1 Message Date
Dragorn421 4fad31ecdf Compiler binaries download: separate from extract and checksum (#2753) 2026-05-08 19:31:32 +02:00
Tharo 2c481eaeeb Partial linking of spec segments (#2661)
* Partial linking of overlay segments, relax linker script alignment

* Partial linking of all spec segments

* Fix update, remove _RomPos from linker script

* iQue version working pending COM-plugin update

* Add plf map file resolution to sym_info.py, local symbol merging is broken

* git subrepo pull tools/com-plugin

subrepo:
  subdir:   "tools/com-plugin"
  merged:   "c4f3ba845"
upstream:
  origin:   "git@github.com:Thar0/com-plugin.git"
  branch:   "main"
  commit:   "c4f3ba845"
git-subrepo:
  version:  "0.4.6"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "110b9eb"

* Make tools compatible with partial linking

Co-authored-by: Anghelo Carvajal <angheloalf95@gmail.com>

* Remove unused files

* Fix some makefile bits

* mkspecrules cleanup

* Comment on the makerom linker layout in mkldscript

* Revert linker padding strategy back to pad_text spec directives

* Comment on objcopy elf -> rom step

* Adjust tool descriptions

* Fix compressed builds

* rm reloc_prereq, no longer used

* Makefile merge fix

Co-authored-by: Dragorn421 <Dragorn421@users.noreply.github.com>

---------

Co-authored-by: Anghelo Carvajal <angheloalf95@gmail.com>
Co-authored-by: Dragorn421 <Dragorn421@users.noreply.github.com>
2026-05-06 23:33:03 +02:00
Tharo 5625f1826e Remove makeromfs (#2752)
This is a tool introduced a long time ago that ended up not being
used at any point in the build system and has no prospects of
future use. There's no reason for keeping it around.
2026-05-06 09:24:50 +02:00
Dragorn421 1e556e3a3d New assets system (#2481)
* wip: New assets system tm

Builds gc-eu-mq-dbg OK from clean after
1) make setup
2) python3 -m tools.assets.extract -j
3) replace 0x80A8E610 with sShadowTex in extracted/gc-eu-mq-dbg/assets/overlays/ovl_En_Jsjutan/sShadowMaterialDL.inc.c
4) make various symbols in extracted data like sTex static

* use variables from config.yml for gMtxClear and sShadowTex addresses

* Write source with static for overlays using `HACK_IS_STATIC_ON` hack

* gc-eu-mq-dbg OK from clean with `make setup && make`

* implement more skeleton-related types, cleanups, fixups

* fix extracted data to no longer produce compilation warnings

* implement more of RoomShapeImage types

* yeet XmlPath from ExternalFile usage

* Implement PlayerAnimationDataResource (link_animetion data)

* fix csdis CS_TIME extra arg

* dmadata file names no longer hardcoded for gc-eu-mq-dbg

* ntsc-1.0 OK

* xml fixes

* slightly improve standard output

* rm extract_assets.py

* generate and use Limb enums (TODO: check Skin skels and implement for Curve skels)

* handle dependencies between xmls

* introduce RawPointers xml attribute to ignore specific pointers and keep them raw

* add tools/extract_assets.sh

* fixups

* only extract if xmls changed or if -f (force) is used

* fixups, gc-eu OK

* all versions OK

* check attributes of xml resources elements

* Implement legacy skelanime resources

* fix ASSET_FILES_BIN_EXTRACTED/COMMITTED: look for .u8.bin specifically instead of just .bin

* implement JFIFResource

* fix png/jpg wildcards: look specifically for .u64.png .u32.png .u64.jpg

* Makefile: Add rules to build .png, .bin and .jpg in assets/ too

* start writing actual docs

* extract sTransCircleDL and sTransWipeDL

* misc cleanup/fixes, pygfxd 1.0.3

* refactor CDataExt.set_write callback args to use a dataclass

* Move {} to in-source

* misc

* more progress on spec

* fix missing braces in n64dd_error_textures.c

* finish xml spec doc

* assets xmls fixes

* some cleanup, use `gNameTex_WIDTH/HEIGHT` macros in dlists

* handle hackmode_syotes_room, fix compile

* C build_from_png

* rm tools/assets/bin2c

* rm ZAPD

* format

* remove rule to generate dmadata_table.py

* CC0 license (and some import cleanup)

* dont try to build zapd (rmd)

* simplify palettes with single user (ci images with a non-shared palette)

* add docs on how images are handled

* bss

* allow -j N

* fix n64texconv python bindings memory management

* move -j at the end of calling extraction script

* with -j, update last_extracts.json as each job completes rather than only if all complete

* make interrupting less jank by making child processes ignore sigint

* use enum names in `SCENE_CMD_SKYBOX_SETTINGS`

* `multiprocessing.get_context("fork")`

* import rich, except ImportError s

* fix optional rich usage

* .bss

* .bss

* .bss

* assets extraction: -j -> -j$(N_THREADS)

* .bss

* change LIMB_NONE/MAX defaults to be FILE_OFFSET instead of SKELNAME

* 0XHEX -> 0xHEX

* fix bss

* Proper includes for assets

mostly proper, some includes like dlists resources always causing a sys_matrix.h include (when not every dlist references gIdentityMtx) could be done better

* rm z64.h

* rm z64.h take two

* bss

* Make .u64 suffix for pngs optional

* fixup: rm .u64 suffix from n64dd image paths

* Remove elemtype suffixes from .bin and .jpg files

* Update images.md

* some build_from_png cleanup, more error handling, comments

* Handle skybox textures

Introduce "sub-format" suffix for pngs, with sub-formats split_lo and split_hi being used for skybox textures

* fixup for older python

* improve collision output some

* fully use SURFACETYPE[01] macros in writing extracted surface types

* use WATERBOX_PROPERTIES in extracted waterboxes

* some SceneCommandsResource cleanup

* format EnvLightSettingsList output
2025-05-18 01:29:09 +02:00
Tharo 3d61fb85ef Add n64texconv and bin2c tools to convert extracted .png and .bin to C arrays during build (#2477)
* n64texconv and bin2c

* mv tools/n64texconv tools/assets/

* fix

* more light fixes

* Silence -Wshadow for libimagequant

* Add reference counting gc for palette objects in python bindings

* Fix missing alignment in n64texconv_*_to_c functions

Co-authored-by: Dragorn421 <Dragorn421@users.noreply.github.com>

* Check palette size in n64texconv_image_from_png

* accept memoryview as well as bytes for binary data

* minimal doc on n64texconv_quantize_shared

* fix a buffer size passed to libimagequant

* assert pal count <= 256 on png write

* Disable palette size check for input pngs, ZAPD fails the check

* No OpenMP for clang

* When reading an indexed png into a CI format, requantize if there are too many colors for the target texel size

---------

Co-authored-by: Dragorn421 <Dragorn421@users.noreply.github.com>
2025-02-17 17:09:42 -05:00
Tharo c028db03b4 Finish matching ique-cn (#2451)
* git subrepo clone git@github.com:Thar0/com-plugin.git tools/com-plugin

subrepo:
  subdir:   "tools/com-plugin"
  merged:   "e8543312d"
upstream:
  origin:   "git@github.com:Thar0/com-plugin.git"
  branch:   "main"
  commit:   "e8543312d"
git-subrepo:
  version:  "0.4.6"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "110b9eb"

* ique-cn OK

* Review suggestions

* Most suggestions

* git subrepo pull tools/com-plugin

subrepo:
  subdir:   "tools/com-plugin"
  merged:   "81595ed1c"
upstream:
  origin:   "git@github.com:Thar0/com-plugin.git"
  branch:   "main"
  commit:   "81595ed1c"
git-subrepo:
  version:  "0.4.6"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "110b9eb"

* Fix other versions
2025-02-04 06:55:04 -05:00
cadmic 9dafc2f2e4 [iQue] Build some C files with EGCS (#2396) 2025-01-02 03:35:22 -05:00
cadmic ffc9f2d4f1 Download EGCS binutils and gcc for iQue (#2391)
* Download EGCS binutils and gcc for iQue

* Use curl -s for cleaner output
2024-12-30 09:30:37 -05:00
cadmic aaf3183233 Download ido-recomp compiler binaries instead of committing them (#2388)
* Download ido-recomp compiler binaries instead of committing them

* Skip downloading compilers if ido_recomp/ exists

* Use make instead of bash script
2024-12-26 12:41:03 -05:00
Dragorn421 91a534cbc9 Rewrite preprocess.py with bash and C (#2035)
* add C preprocess_pragmas and Bash preprocess

* "line return" -> newline

* align tools sources

* fix: handle files that are not newline-terminated

* use a temp directory with a same-basename file instead of a temp file

* macos compat

* remove debug code
2024-08-14 04:05:36 -04:00
Tharo ef329e633a [Audio 2/?] Extract audio samples to wav (#2020)
* [Audio 2/?] Extract audio samples to wav

Co-authored-by: zelda2774 <69368340+zelda2774@users.noreply.github.com>

* How

* Hopefully fix warning I don't get locally

* Pad default sample filenames, comment on the vadpcm frame encoder functions, other suggested changes

* Small tweaks to above

* Remove some obsolete code

---------

Co-authored-by: zelda2774 <69368340+zelda2774@users.noreply.github.com>
2024-08-08 22:39:18 -04:00
cadmic 7332e8ee76 Delete unused yaz0tool (#1959) 2024-06-18 20:53:19 -04:00
EllipticEllipsis eadc477187 New relocation-generating program (#1016)
* Update makefiles

* git subrepo clone git@github.com:EllipticEllipsis/fado.git tools/fado

subrepo:
  subdir:   "tools/fado"
  merged:   "46c4d751a"
upstream:
  origin:   "git@github.com:EllipticEllipsis/fado.git"
  branch:   "master"
  commit:   "46c4d751a"
git-subrepo:
  version:  "0.4.3"
  origin:   "https://github.com/ingydotnet/git-subrepo.git"
  commit:   "2f68596"

* git subrepo pull tools/fado

subrepo:
  subdir:   "tools/fado"
  merged:   "88114ebce"
upstream:
  origin:   "git@github.com:EllipticEllipsis/fado.git"
  branch:   "master"
  commit:   "88114ebce"
git-subrepo:
  version:  "0.4.3"
  origin:   "https://github.com/ingydotnet/git-subrepo.git"
  commit:   "2f68596"

* A few ideas for computing dependencies

* Remove reserved identifiers from spec.h and util.h
and add required headers

* Fix a couple more headers

* Program for reloc prerequisites

* git subrepo pull tools/fado

subrepo:
  subdir:   "tools/fado"
  merged:   "36a905f72"
upstream:
  origin:   "git@github.com:EllipticEllipsis/fado.git"
  branch:   "master"
  commit:   "36a905f72"
git-subrepo:
  version:  "0.4.3"
  origin:   "https://github.com/ingydotnet/git-subrepo.git"
  commit:   "2f68596"

* Update makefile to make dependency files and use overlay's name

* git subrepo pull tools/fado

subrepo:
  subdir:   "tools/fado"
  merged:   "43c339a59"
upstream:
  origin:   "git@github.com:EllipticEllipsis/fado.git"
  branch:   "master"
  commit:   "43c339a59"
git-subrepo:
  version:  "0.4.3"
  origin:   "https://github.com/ingydotnet/git-subrepo.git"
  commit:   "2f68596"

* Change awk to grep, delete ZAPD files, gitignore elf

* Delete all the cfg files

* Fix memory leaks

* Rename and add coloured errors

* Makefile tweaks
- preprocess spec before grep
- split order prerequisites via phony target to reduce dependency edges
- remove `resources` target
- remove separate overlays targets
- use `$(SPEC)` throughout
- change to using filenames of relocs for overlay names via `$*`
- Rearrange targets to better reflect their categories

* Update gitignore

* Review

* Add a check for the reloc file name

* get_segment_by_name

* get_stmt_id_by_stmt_name

* Cleaning up

* algorithm change

* function rename

* Fix typos

Co-authored-by: angie <angheloalf95@gmail.com>
2022-02-06 14:40:26 -05:00
Tharo 04a9d51e90 Rework handling of dmadata (#1036)
* Generate dmadata

* Remove tab indentations in mkdmadata.c and mkldscript.c

* Fix

* Review suggestions

* Hopefully fix

* Fix index

Co-authored-by: Dragorn421 <Dragorn421@users.noreply.github.com>

Co-authored-by: Dragorn421 <Dragorn421@users.noreply.github.com>
2021-12-01 00:29:09 +01:00
louist103 916efa0f7d Use clang when possible for tools (#990)
* Use clang when possible for tools

* Add LLD ?= 0
2021-10-31 20:00:24 +01:00
Tharo 208167361b Poe Objects (#743)
* poe objects

* Last unaccounted resolved, name suffix fixes

* remove _tex

* missed 2

* format

Co-authored-by: Fig02 <fig02srl@gmail.com>
2021-04-04 23:48:12 +02:00
Anghelo Carvajal 867b5a87ba Add distclean rule to makefile (#716)
* Add clean_everything rule to makefile

Signed-off-by: angie <angheloalf95@gmail.com>

* change `clean_everything` to `distclean`

Signed-off-by: angie <angheloalf95@gmail.com>

* Somehow forgot to commit the makefile

Signed-off-by: angie <angheloalf95@gmail.com>

* Remove nuke

Signed-off-by: angie <angheloalf95@gmail.com>

* Remove redundant variables

Signed-off-by: angie <angheloalf95@gmail.com>

* Forgot to remove `TEXTURE_DIRS`

Signed-off-by: angie <angheloalf95@gmail.com>
2021-03-14 12:07:46 -04:00
Random be84e1f0f8 Add Cygwin support (#594)
* Add Cygwin support

* revert .gitignore changes

* Add Cygwin instructions

* corrections

* specify dependencies in README.md
2021-01-20 14:46:25 -05:00
fig02 ba0c6965ca Remove submodule and use subrepo for ZAPD (#591)
* remove zap

* git subrepo clone https://github.com/zeldaret/ZAPD.git tools/ZAPD

subrepo:
  subdir:   "tools/ZAPD"
  merged:   "cd4a8760b"
upstream:
  origin:   "https://github.com/zeldaret/ZAPD.git"
  branch:   "master"
  commit:   "cd4a8760b"
git-subrepo:
  version:  "0.4.3"
  origin:   "https://github.com/ingydotnet/git-subrepo.git"
  commit:   "2f68596"

* remove thanks.md

* zap2 -> zapd and spec changes

* remove submodule init
2021-01-01 23:24:29 -05:00
krimtonz 28387947e2 vtx extractor (#94) 2020-04-26 18:14:41 -04:00
Jack Walker 087f561f77 First proper commit. 2020-03-17 00:31:30 -04:00