mirror/mm
1
0
Fork 0
mirror of https://github.com/zeldaret/mm.git synced 2026-05-23 06:54:14 -04:00
Code Issues Packages Projects Releases Wiki Activity

Overhaul the build system (#234)

Browse Source 
* wip

* fix

* add disassembler

* Disasm builds OK

* Variable addends

* More wip

* Rodata migration implemented

* Cleanup old tools

* Try fix submodule -> subrepo merge

* git subrepo pull --force --remote=https://github.com/zeldaret/ZAPD.git tools/ZAPD

subrepo:
  subdir:   "tools/ZAPD"
  merged:   "602e609"
upstream:
  origin:   "https://github.com/zeldaret/ZAPD.git"
  branch:   "master"
  commit:   "602e609"
git-subrepo:
  version:  "0.4.3"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "2f68596"

* Builds again but assets are totally broken

* git subrepo pull --force tools/asm-processor

subrepo:
  subdir:   "tools/asm-processor"
  merged:   "1ffdb08a"
upstream:
  origin:   "https://github.com/simonlindholm/asm-processor.git"
  branch:   "master"
  commit:   "1ffdb08a"
git-subrepo:
  version:  "0.4.3"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "2f68596"

* More cleanup, move functions.txt and variables.txt to tools/disasm and rm tables

* rm z64compress in preparation for subrepo

* git subrepo clone (merge) https://github.com/z64me/z64compress.git tools/z64compress

subrepo:
  subdir:   "tools/z64compress"
  merged:   "eb11085c"
upstream:
  origin:   "https://github.com/z64me/z64compress.git"
  branch:   "main"
  commit:   "eb11085c"
git-subrepo:
  version:  "0.4.3"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "2f68596"

* Fix asset extraction

* Fix diff-init make rule

* Split code bss

* Split assumed linker bug padding from assembly files

* add filelists for mm.us.rev1

* Maybe working, but I'm not sure

* add overlays to spec

* Add rodata to actos

* Everything compiles

* Make a lot of C files for code

* Add almost every file in code to spec

* whoops

* 3 code files left

* add scenes to spec

* More progress on progress.py

* Fix skelanime in spec

* audio files!

* Fix merge issues

* Fix some C files in code

* Fix remaining code files

* Use existing O1 C files in spec

* reorder boot order in spec

* update spec

* fault.c

* Convert relocs on completed actors, fixbaserom uses current rom name

* more boot files

* Add VT macros and script

* finish already existing boot files

* most of  libultra

* fix 64bits libultra files

* Use C files for libultra, wrap some functions in NON_MATCHING

* Remove duplicate of OS_CLOCK_RATE from fault.c

* C files for fbdemos

* delete dumb files

* bootstrap C files, still need to add them to the spec

* update fixbaserom

* boot OK?

* I forgot to commit the spec

* C for gamestates

* C for kaleido

* Change all includes to ""

* copy actor sizes script from oot

* I forgot to delete those files

* Basic C files for effects

* Add effects initvars names

* Remove mislabelled boot functions from header/txt

* Begin porting bootstrap_fx, some sizes

* Fix <>

* Fix enum

* Fix diff.py

* fix libultra stuff

* update regconvert

* update setup warnings

* add some missing ;

* Fix some makefile stuff and other fixes on some non_matching functions

* add executable flag in extract_baserom and fixbaserom

* fix relative path

* copy assist from oot

* fix map path

* another assist path fix

* Delete C files for handwritten files

* add code_801A51F0 to spec

* add gfxbuffers to spec

* Move rodata to top of each file when possible

* UNK_TYPEs for func_801A51F0

* Remove kaleido rodata from spec

* Update spec and undefined_syms for recent merge

* GCC warnings and fix errors in nonmatchings,

* round percentage numbers

* progress script: format changes

* progress: error on non-existing files

* fix warning in z_scene_table

* Match 2 nonmatchings in z_actor

* Warnings in lightswitch and invadepoh

* Fix warning in z_actor_dlftbls

* I though I fixed this one

* whoops

* Comment out CC_CHECK

* Removed redundant ultra64.h includes

* Update asm_processor, sorted boot_O1 into other folders, completed the fbdemo bootstrap, cleaned up undefined_syms

* Completed gamestates bootstrap

* Split kaleido_scope

* Remove section.h and segment.h, move keep object externs to a common location in variables.h

* Completed effects bootstrap

* Segmented address externs for effects, fbdemos, gamestates and kaleido

* Move actor data externs out of the if 0

* Segmented address externs for actors

* Prepare actionfunc detection

* fix script, how did it even work before

* Fix actionfunc script again, re-introduce some more intermediate prints to the disassembler

* Automated actionFunc detection in actors

* Segmented addresses from player .text

* rm old segment addrs script and fix build

* Move sizes folder to tools

* Make build.py executable

* New Jenkinsfile Prayge

* Remove numpy dependencies

* Add warnings_disasm_current.txt

* my bad

* Update spec and undefined_syms

* Add z_eff_ss_hahen to pametfrog

* git subrepo pull (merge) --force tools/z64compress

subrepo:
  subdir:   "tools/z64compress"
  merged:   "163ca2af"
upstream:
  origin:   "https://github.com/z64me/z64compress.git"
  branch:   "main"
  commit:   "163ca2af"
git-subrepo:
  version:  "0.4.3"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "2f68596"

* Make z64compress print to stdout

* sneeky commit to update warnings tooling

* test

* Another test

* Mark fixing overlay reloc generating as a TODO

* Update warnings stuff

* Communicate the return code from running z64compress back to the Makefile through the wrapper

* Run formatter, remove extra commented copy of function

* Re-fix some includes

* Convert atan to hex to conform to decided style

* Some tidying up, remove c for fp and the other two handwritten code files

* BSS in z_collision_check & z_scene_proc

* add static back in

* Fix timerintr bss, add file to spec, some cleanup

* Remove externs

* Newline

* Readd enums

* Typo

* Colours

* Comments for hitmark enum values

Co-authored-by: EllipticEllipsis <73679967+EllipticEllipsis@users.noreply.github.com>

* Improvements and suggestions

* Organize and remove unused imports and use env for python3 scripts, delete unused overlay.py

Co-authored-by: angie <angheloalf95@gmail.com>
Co-authored-by: Elliptic Ellipsis <elliptic.ellipsis@gmail.com>
Co-authored-by: engineer124 <engineer124engineer124@gmail.com>
Co-authored-by: EllipticEllipsis <73679967+EllipticEllipsis@users.noreply.github.com>
This commit is contained in:
Tharo
2021-08-04 04:21:31 +01:00
committed by GitHub
parent 60ff207982
commit 7743e5a2c4
1788 changed files with 88892 additions and 47119 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/asm-processor/asm_processor.py
+11 -26
View File
@@ -380,7 +380,7 @@ class Failure(Exception):
class GlobalState:
def __init__(self, min_instr_count, skip_instr_count, use_jtbl_for_rodata, mips1):
def __init__(self, min_instr_count, skip_instr_count, use_jtbl_for_rodata):
# A value that hopefully never appears as a 32-bit rodata constant (or we
# miscompile late rodata). Increases by 1 in each step.
self.late_rodata_hex = 0xE0123456
@@ -388,7 +388,6 @@ class GlobalState:
self.min_instr_count = min_instr_count
self.skip_instr_count = skip_instr_count
self.use_jtbl_for_rodata = use_jtbl_for_rodata
self.mips1 = mips1
def next_late_rodata_hex(self):
dummy_bytes = struct.pack('>I', self.late_rodata_hex)
@@ -609,13 +608,12 @@ class GlobalAsmBlock:
size = self.fn_section_sizes['.late_rodata'] // 4
skip_next = False
needs_double = (self.late_rodata_alignment != 0)
extra_mips1_nop = False
for i in range(size):
if skip_next:
skip_next = False
continue
# Jump tables give 9 instructions (11 with -mips1) for >= 5 words of rodata,
# and should be emitted when:
# Jump tables give 9 instructions for >= 5 words of rodata, and should be
# emitted when:
# - -O2 or -O2 -g3 are used, which give the right codegen
# - we have emitted our first .float/.double (to ensure that we find the
# created rodata in the binary)
@@ -625,14 +623,12 @@ class GlobalAsmBlock:
# generate a jump table)
# - we have at least 10 more instructions to go in this function (otherwise our
# function size computation will be wrong since the delay slot goes unused)
jtbl_size = 11 if state.mips1 else 9
if (not needs_double and state.use_jtbl_for_rodata and i >= 1 and
size - i >= 5 and num_instr - len(late_rodata_fn_output) >= jtbl_size + 1):
size - i >= 5 and num_instr - len(late_rodata_fn_output) >= 10):
cases = " ".join("case {}:".format(case) for case in range(size - i))
late_rodata_fn_output.append("switch (*(volatile int*)0) { " + cases + " ; }")
late_rodata_fn_output.extend([""] * (jtbl_size - 1))
late_rodata_fn_output.extend([""] * 8)
jtbl_rodata_size = (size - i) * 4
extra_mips1_nop = True
break
dummy_bytes = state.next_late_rodata_hex()
late_rodata_dummy_bytes.append(dummy_bytes)
@@ -642,20 +638,12 @@ class GlobalAsmBlock:
fval, = struct.unpack('>d', dummy_bytes + dummy_bytes2)
late_rodata_fn_output.append('*(volatile double*)0 = {};'.format(fval))
skip_next = True
needs_double = False
if state.mips1:
# mips1 does not have ldc1/sdc1
late_rodata_fn_output.append('')
late_rodata_fn_output.append('')
extra_mips1_nop = False
needs_double = True
else:
fval, = struct.unpack('>f', dummy_bytes)
late_rodata_fn_output.append('*(volatile float*)0 = {}f;'.format(fval))
extra_mips1_nop = True
late_rodata_fn_output.append('')
late_rodata_fn_output.append('')
if state.mips1 and extra_mips1_nop:
late_rodata_fn_output.append('')
text_name = None
if self.fn_section_sizes['.text'] > 0 or late_rodata_fn_output:
@@ -734,7 +722,7 @@ float_regexpr = re.compile(r"[-+]?[0-9]*\.?[0-9]+([eE][-+]?[0-9]+)?f")
def repl_float_hex(m):
return str(struct.unpack(">I", struct.pack(">f", float(m.group(0).strip().rstrip("f"))))[0])
def parse_source(f, opt, framepointer, mips1, input_enc, output_enc, out_dependencies, print_source=None):
def parse_source(f, opt, framepointer, input_enc, output_enc, out_dependencies, print_source=None):
if opt in ['O2', 'O1']:
if framepointer:
min_instr_count = 6
@@ -763,7 +751,7 @@ def parse_source(f, opt, framepointer, mips1, input_enc, output_enc, out_depende
if opt in ['O2', 'g3'] and not framepointer:
use_jtbl_for_rodata = True
state = GlobalState(min_instr_count, skip_instr_count, use_jtbl_for_rodata, mips1)
state = GlobalState(min_instr_count, skip_instr_count, use_jtbl_for_rodata)
global_asm = None
asm_functions = []
@@ -815,7 +803,7 @@ def parse_source(f, opt, framepointer, mips1, input_enc, output_enc, out_depende
out_dependencies.append(fname)
include_src = StringIO()
with open(fname, encoding=input_enc) as include_file:
parse_source(include_file, opt, framepointer, mips1, input_enc, output_enc, out_dependencies, include_src)
parse_source(include_file, opt, framepointer, input_enc, output_enc, out_dependencies, include_src)
include_src.write('#line ' + str(line_no + 1) + ' "' + f.name + '"')
output_lines[-1] = include_src.getvalue()
include_src.close()
@@ -1191,7 +1179,6 @@ def run_wrapped(argv, outfile, functions):
parser.add_argument('--input-enc', default='latin1', help="Input encoding (default: latin1)")
parser.add_argument('--output-enc', default='latin1', help="Output encoding (default: latin1)")
parser.add_argument('-framepointer', dest='framepointer', action='store_true')
parser.add_argument('-mips1', dest='mips1', action='store_true')
parser.add_argument('-g3', dest='g3', action='store_true')
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument('-O1', dest='opt', action='store_const', const='O1')
@@ -1203,20 +1190,18 @@ def run_wrapped(argv, outfile, functions):
if opt != 'O2':
raise Failure("-g3 is only supported together with -O2")
opt = 'g3'
if args.mips1 and (opt != 'O2' or args.framepointer):
raise Failure("-mips1 is only supported together with -O2")
if args.objfile is None:
with open(args.filename, encoding=args.input_enc) as f:
deps = []
functions = parse_source(f, opt=opt, framepointer=args.framepointer, mips1=args.mips1, input_enc=args.input_enc, output_enc=args.output_enc, out_dependencies=deps, print_source=outfile)
functions = parse_source(f, opt=opt, framepointer=args.framepointer, input_enc=args.input_enc, output_enc=args.output_enc, out_dependencies=deps, print_source=outfile)
return functions, deps
else:
if args.assembler is None:
raise Failure("must pass assembler command")
if functions is None:
with open(args.filename, encoding=args.input_enc) as f:
functions = parse_source(f, opt=opt, framepointer=args.framepointer, mips1=args.mips1, input_enc=args.input_enc, out_dependencies=[], output_enc=args.output_enc)
functions = parse_source(f, opt=opt, framepointer=args.framepointer, input_enc=args.input_enc, out_dependencies=[], output_enc=args.output_enc)
if not functions:
return
asm_prelude = b''