mirror/oot
1
0
Fork 0
mirror of https://github.com/zeldaret/oot synced 2026-05-23 06:54:24 -04:00
Code Issues Packages Projects Releases Wiki Activity

[Audio 10/10] Loose ends (#2337)

Browse Source 
* Introduce afile_sizes, generate headers of sizes for soundfonts and sequences

* Initial tools/audio README

* Versioning for samplebank extraction

* Clean up the disassemble_sequence.py runnable interface

* Add static assertions for maximum bank sizes

* Boost optimization for audio tools

* Samplebank XML doc

* Soundfont XML doc

* More docs in sampleconv for vadpcm

* Various tools fixes/cleanup

* VADPCM doc

* Try to fix md formatting

* VADPCM doc can come later

* Fix merge with PR 9

* Fix blobs from MM

* Try to fix bss

* Try fix bss round 2

* Fix sampleconv memset bug

* Suggested documentation tweaks
This commit is contained in:
Tharo
2024-12-14 00:26:36 +00:00
committed by GitHub
parent 4b20d8269b
commit df5d4cb467
39 changed files with 903 additions and 178 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/audio/.gitignore
+1
View File
@@ -1,5 +1,6 @@
__pycache__/
afile_sizes
atblgen
sfpatch
sbc
tools/audio/Makefile
+7 -6
View File
@@ -1,4 +1,4 @@
PROGRAMS := atblgen sfpatch sbc sfc
PROGRAMS := afile_sizes atblgen sbc sfc sfpatch
ifeq ($(shell which xml2-config),)
$(error xml2-config not found. Did you install libxml2-dev?)
@@ -9,7 +9,7 @@ FORMAT_ARGS := -i -style=file
CC := gcc
CFLAGS := -Wall -Wextra -pedantic
OPTFLAGS := -Og -g3
OPTFLAGS := -O2
XML_CFLAGS := $(shell xml2-config --cflags)
XML_LDFLAGS := $(shell xml2-config --libs)
@@ -30,10 +30,11 @@ format:
$(CLANG_FORMAT) $(FORMAT_ARGS) $(shell find . -maxdepth 1 -type f -name "*.[ch]")
$(MAKE) -C sampleconv format
atblgen_SOURCES := audio_tablegen.c samplebank.c soundfont.c xml.c util.c
sfpatch_SOURCES := sfpatch.c util.c
sbc_SOURCES := samplebank_compiler.c samplebank.c aifc.c xml.c util.c
sfc_SOURCES := soundfont_compiler.c samplebank.c soundfont.c aifc.c xml.c util.c
afile_sizes_SOURCES := afile_sizes.c util.c
atblgen_SOURCES := audio_tablegen.c samplebank.c soundfont.c xml.c util.c
sbc_SOURCES := samplebank_compiler.c samplebank.c aifc.c xml.c util.c
sfc_SOURCES := soundfont_compiler.c samplebank.c soundfont.c aifc.c xml.c util.c
sfpatch_SOURCES := sfpatch.c util.c
atblgen_CFLAGS := $(XML_CFLAGS)
sbc_CFLAGS := $(XML_CFLAGS)
tools/audio/README.md
+59
View File
@@ -0,0 +1,59 @@
# Z64 Audio Tools
The Z64 Audio Tools work together to implement the full audio asset pipeline
![](../../docs/audio/build_flowchart.png)
**Licensing Information**
* The programs `atblgen`, `sampleconv`, `sbc` and `sfc` are (mostly) distributed under MPL-2.0. The VADPCM encoding and decoding portions of `sampleconv` are under CC0-1.0.
* The programs `sfpatch` and `afile_sizes` are distributed under CC0-1.0.
* The extraction tool is distributed under CC0-1.0.
## sampleconv
Converts aifc <-> aiff / wav
Used in extraction and build to convert audio sample data between uncompressed mono 16-bit PCM and the compressed formats used by the audio driver.
## SampleBank Compiler (sbc)
Converts samplebank xml + aifc -> asm
Samplebanks are converted to assembly files for building as it is easier to define the necessary absolute symbols, and they are pure unstructured data.
## SoundFont Compiler (sfc)
Converts soundfont & samplebank xml + aifc -> C
Soundfonts are converted to C rather than assembly as it shares data structures with the audio driver code. Modifying the structures used by the driver without updating `sfc` to write them should error at compile-time rather than crash at runtime.
## sfpatch
`Usage: sfpatch in.elf out.elf`
This tool patches the symbol table of an ELF file (`in.elf`) to make every defined symbol in the file an absolute symbol. This is a required step for building soundfonts from C source as all pointers internal to a soundfont are offset from the start of the soundfont file and not the audiobank segment as a whole. Making all defined symbols ABS symbols prevents the linker from updating their values later, ensuring they remain file-relative.
## atblgen
Generates various audio code tables.
- Samplebank table: Specifies where in the `Audiotable` file each samplebank begins and how large it is.
- Soundfont table: Specifies where in the `Audiobank` files each soundfont begins, how large it is, which samplebanks it uses, and how many instruments/drums/sfx it contains.
- Sequence font table: Contains information on what soundfonts each sequence uses. Generated from the sequence object files that embed a `.note.fonts` section that holds this information.
The sequence table is not generated as some things in that table are better left manually specified, such as sequence enum names and flags. This also lets us have the sequence table before assembling any sequence files which is nice for some sequence commands like `runseq`.
## afile_sizes
Produces header files containing binary file sizes for a given set of object files. Used to produce headers containing soundfont and sequence files and the number of each for use in code files.
## extraction
This collection of python files implements the extraction of audio data from a base ROM.
Files that are designed to be used externally include:
- `audio_extract.py` is the main file for audio extraction, it expects an external script to call `extract_audio_for_version` with the necessary inputs.
- `disassemble_sequence.py` is runnable but is not used in this way in either extraction or building. It may be used to manually disassemble a sequence binary.
- `tuning.py` is runnable but is not used that way in either extraction or building. It may be used to manually determine alternative matches for the samplerate and basenote of a sample as the extraction procedure cannot always determine these uniquely.
See individual python source files for further details on their purposes.
tools/audio/afile_sizes.c
+120
View File
@@ -0,0 +1,120 @@
/* SPDX-FileCopyrightText: Copyright (C) 2024 ZeldaRET */
/* SPDX-License-Identifier: CC0-1.0 */
#include <ctype.h>
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include "elf32.h"
#include "util.h"
static int
usage(const char *progname)
{
fprintf(stderr,
// clang-format off
"Generates a header containing definitions for the sizes of all the input object files and a" "\n"
"definition for the number of input files." "\n"
"Usage: %s <header output path> <num define> <header guard> <section name> <object files...>" "\n"
" header output path: Path to write the generated header to" "\n"
" num define: The name of the definition for the number of input files" "\n"
" header guard: The header guard definition name to be used for the output header" "\n"
" section name: The object file section to output the size of in each definition" "\n"
" object files: List of paths to each object file to be processed, each input object file" "\n"
" must contain the section requested in the section name argument and must" "\n"
" also contain a .note.name section containing the null-terminated symbolic " "\n"
" name of the object that is used to name the size definitions." "\n",
// clang-format on
progname);
return EXIT_FAILURE;
}
int
main(int argc, char **argv)
{
const char *progname = argv[0];
if (argc < 6) // progname, 4 required args, at least 1 input file
return usage(progname);
const char *header_out = argv[1];
const char *num_def = argv[2];
const char *header_guard = argv[3];
const char *secname = argv[4];
int num_files = argc - 5;
char **files = &argv[5];
// Open the header for writing, write the header guard
FILE *out = fopen(header_out, "w");
if (out == NULL)
error("failed to open output file \"%s\" for writing: %s", header_out, strerror(errno));
fprintf(out,
// clang-format off
"#ifndef %s_H_" "\n"
"#define %s_H_" "\n"
"\n",
// clang-format on
header_guard, header_guard);
// For each input elf file, write the size define
for (int i = 0; i < num_files; i++) {
const char *path = files[i];
size_t data_size;
void *data = elf32_read(path, &data_size);
Elf32_Shdr *shstrtab = elf32_get_shstrtab(data, data_size);
if (shstrtab == NULL)
error("Input file \"%s\" has no shstrtab?", path);
// Read in the .note.name section containing the object's symbolic name.
// We run this on both soundfonts and sequences:
// - Soundfont .note.name sections are added with objcopy
// - Sequence .note.name sections are assembled as part of .startseq
Elf32_Shdr *name_section = elf32_section_forname(".note.name", shstrtab, data, data_size);
if (name_section == NULL)
error("Input file \"%s\" has no name section?", path);
uint32_t name_section_offset = elf32_read32(name_section->sh_offset);
uint32_t name_section_size = elf32_read32(name_section->sh_size);
validate_read(name_section_offset, name_section_size, data_size);
const char *object_name = GET_PTR(data, name_section_offset);
if (strnlen(object_name, name_section_size + 1) >= name_section_size)
error("Input file \"%s\" name is not properly terminated?", path);
// Read the section header for the data we're interested in, the name is given in the program args
Elf32_Shdr *sec = elf32_section_forname(secname, shstrtab, data, data_size);
if (sec == NULL)
error("Input file \"%s\" has no section named \"%s\"?", path, secname);
// Assumption: The section size matches the size in the <object_name>_Size symbol, this is always
// true for soundfonts by nature of how they're built (cf. soundfont.ld) and should always be true
// of sequences since writing anything that results in output data before .startseq and after .endseq
// is essentially undefined.
size_t object_size = elf32_read32(sec->sh_size);
fprintf(out, "#define %s_SIZE 0x%lX\n", object_name, object_size);
free(data);
}
// Write the total number of input files, end the header
fprintf(out,
// clang-format off
"\n"
"#define %s %d" "\n"
"\n"
"#endif" "\n",
// clang-format on
num_def, num_files);
fclose(out);
return EXIT_SUCCESS;
}
tools/audio/aifc.c
+11 -6
View File
@@ -534,14 +534,19 @@ aifc_read(aifc_data *af, const char *path, uint8_t *match_buf, size_t *match_buf
void
aifc_dispose(aifc_data *af)
{
free(af->book_state);
af->has_book = false;
if (af->has_book) {
free(af->book_state);
af->has_book = false;
}
af->has_loop = false;
free(af->compression_name);
if (af->compression_name != NULL)
free(af->compression_name);
for (size_t i = 0; i < af->num_markers; i++)
free((*af->markers)[i].label);
free(af->markers);
if (af->markers != NULL) {
for (size_t i = 0; i < af->num_markers; i++)
free((*af->markers)[i].label);
free(af->markers);
}
}
tools/audio/audio_tablegen.c
+5 -5
View File
@@ -424,11 +424,11 @@ tablegen_sequences(const char *seq_font_tbl_out, const char *seq_order_path, con
if (shstrtab == NULL)
error("ELF file \"%s\" has no section header string table?", path);
// The .fonts and .name sections are written when assembling the sequence:
// The .fonts section contains a list of bytes for each soundfont the sequences uses
// The .name section contains the null-terminated name of the sequence as set by .startseq
// The .note.fonts and .note.name sections are written when assembling the sequence:
// The .note.fonts section contains a list of bytes for each soundfont the sequences uses
// The .note.name section contains the null-terminated name of the sequence as set by .startseq
Elf32_Shdr *font_section = elf32_section_forname(".fonts", shstrtab, data, data_size);
Elf32_Shdr *font_section = elf32_section_forname(".note.fonts", shstrtab, data, data_size);
if (font_section == NULL)
error("Sequence file \"%s\" has no fonts section?", path);
@@ -436,7 +436,7 @@ tablegen_sequences(const char *seq_font_tbl_out, const char *seq_order_path, con
uint32_t font_section_size = elf32_read32(font_section->sh_size);
validate_read(font_section_offset, font_section_size, data_size);
Elf32_Shdr *name_section = elf32_section_forname(".name", shstrtab, data, data_size);
Elf32_Shdr *name_section = elf32_section_forname(".note.name", shstrtab, data, data_size);
if (name_section == NULL)
error("Sequence file \"%s\" has no name section?", path);
tools/audio/extraction/disassemble_sequence.py
+21 -18
View File
@@ -51,17 +51,7 @@ from enum import Enum, auto
from typing import Callable, Dict, List, Optional, Tuple
from .audiobank_file import AudiobankFile
pitch_names = (
"A0", "BF0", "B0", "C1", "DF1", "D1", "EF1", "E1", "F1", "GF1", "G1", "AF1", "A1", "BF1", "B1", "C2",
"DF2", "D2", "EF2", "E2", "F2", "GF2", "G2", "AF2", "A2", "BF2", "B2", "C3", "DF3", "D3", "EF3", "E3",
"F3", "GF3", "G3", "AF3", "A3", "BF3", "B3", "C4", "DF4", "D4", "EF4", "E4", "F4", "GF4", "G4", "AF4",
"A4", "BF4", "B4", "C5", "DF5", "D5", "EF5", "E5", "F5", "GF5", "G5", "AF5", "A5", "BF5", "B5", "C6",
"DF6", "D6", "EF6", "E6", "F6", "GF6", "G6", "AF6", "A6", "BF6", "B6", "C7", "DF7", "D7", "EF7", "E7",
"F7", "GF7", "G7", "AF7", "A7", "BF7", "B7", "C8", "DF8", "D8", "EF8", "E8", "F8", "GF8", "G8", "AF8",
"A8", "BF8", "B8", "C9", "DF9", "D9", "EF9", "E9", "F9", "GF9", "G9", "AF9", "A9", "BF9", "B9", "C10",
"DF10", "D10", "EF10", "E10", "F10", "BFNEG1", "BNEG1", "C0", "DF0", "D0", "EF0", "E0", "F0", "GF0", "G0", "AF0",
)
from .tuning import pitch_names
#
# VERSIONS
@@ -1277,20 +1267,33 @@ class SequenceDisassembler:
outfile.write(f".endseq {self.seq_name}\n")
if __name__ == '__main__':
import sys
import argparse
parser = argparse.ArgumentParser(description="Disassemble a Zelda 64 sequence binary")
parser.add_argument("file", help="Sequence binary to disassemble")
parser.add_argument("out", help="Path to output source file")
parser.add_argument("-v", dest="mml_version", required=False, default="OoT", type=str, help="Sample rate (integer)")
args = parser.parse_args()
in_path = sys.argv[1]
out_path = sys.argv[2]
in_path = args.file
out_path = args.out
mml_ver = {
"OoT" : MMLVersion.OOT,
"MM" : MMLVersion.MM,
}.get(args.mml_version, None)
if mml_ver is None:
raise Exception("Invalid MML Version, should be 'OoT' or 'MM'")
with open(in_path, "rb") as infile:
data = bytearray(infile.read())
class FontDummy:
def __init__(self, file_name) -> None:
self.name = file_name
self.file_name = file_name
def __init__(self, name) -> None:
self.name = name
self.file_name = name
self.instrument_index_map = {}
disas = SequenceDisassembler(0, data, None, CMD_SPEC, MMLVersion.MM, out_path, "", [FontDummy("wow")], [])
disas = SequenceDisassembler(0, data, None, CMD_SPEC, mml_ver, out_path, "", [FontDummy("dummyfont")], [])
disas.analyze()
disas.emit()
tools/audio/extraction/extraction_xml.py
+1 -1
View File
@@ -67,7 +67,7 @@ class SampleBankExtractionDescription(ExtractionDescription):
in_version = self.in_version(version_include, version_exclude, version_name)
if in_version:
self.blob_info.append(item.attrib)
self.sample_info_versions.append((item.attrib, in_version))
self.sample_info_versions.append((item.tag, item.attrib, in_version))
else:
print(xml_root.attrib)
assert False, item.tag
tools/audio/samplebank_compiler.c
+5 -5
View File
@@ -126,7 +126,7 @@ main(int argc, char **argv)
sb.name, sb.name);
// original tool appears to have a buffer clearing bug involving a buffer sized BUG_BUF_SIZE
match_buf_ptr = (matching) ? match_buf : NULL;
match_buf_ptr = (matching && sb.buffer_bug) ? match_buf : NULL;
match_buf_pos = 0;
for (size_t i = 0; i < sb.num_samples; i++) {
@@ -172,13 +172,13 @@ main(int argc, char **argv)
fprintf(outf, ".incbin \"%s\", 0x%lX, 0x%lX\n", path, aifc.ssnd_offset, aifc.ssnd_size);
if (matching && sb.buffer_bug && i == sb.num_samples - 1) {
if (match_buf_ptr != NULL && i == sb.num_samples - 1) {
// emplace garbage
size_t end = ALIGN16(match_buf_pos);
fprintf(outf, "\n# Garbage data from buffer bug\n");
size_t end = ALIGN16(match_buf_pos);
for (; match_buf_pos < end; match_buf_pos++)
fprintf(outf, ".byte 0x%02X\n", match_buf[match_buf_pos]);
fprintf(outf, ".byte 0x%02X\n", match_buf_ptr[match_buf_pos]);
} else {
fputs("\n.balign 16\n", outf);
}
tools/audio/sampleconv/Makefile
+1 -1
View File
@@ -1,7 +1,7 @@
CC := gcc
CFLAGS := -Wall -Wextra -MMD
OPTFLAGS := -Og -g3
OPTFLAGS := -O3
LDFLAGS :=
CLANG_FORMAT := clang-format-14
tools/audio/sampleconv/src/codec/vadpcm.c
+28 -5
View File
@@ -14,6 +14,18 @@
#include "codec.h"
/**
* Creates FIR filter matrices for each page of the prediction codebook.
* For order=2: each page contains coefficients c0..7 and d0..7, the matrix resembles:
* [ c0, d0, 1, 0, 0, 0, 0, 0, 0, 0 ]
* [ c0, d1, d0, 1, 0, 0, 0, 0, 0, 0 ]
* [ c0, d2, d1, d0, 1, 0, 0, 0, 0, 0 ]
* [ c0, d3, d2, d1, d0, 1, 0, 0, 0, 0 ]
* [ c0, d4, d3, d2, d1, d0, 1, 0, 0, 0 ]
* [ c0, d5, d4, d3, d2, d1, d0, 1, 0, 0 ]
* [ c0, d6, d5, d4, d3, d2, d1, d0, 1, 0 ]
* [ c0, d7, d6, d5, d4, d3, d2, d1, d0, 1 ]
*/
int
expand_codebook(int16_t *book_data, int32_t ****table_out, int32_t order, int32_t npredictors)
{
@@ -25,19 +37,25 @@ expand_codebook(int16_t *book_data, int32_t ****table_out, int32_t order, int32_
table[i][j] = MALLOC_CHECKED_INFO((order + 8) * sizeof(int32_t), "order=%d", order);
}
// For each page, create the associated matrix
for (int32_t i = 0; i < npredictors; i++) {
int32_t **table_entry = table[i];
// Fill the first columns of the FIR filter matrix, up to the "order" column
for (int32_t j = 0; j < order; j++) {
for (int32_t k = 0; k < 8; k++)
table_entry[k][j] = *(book_data++);
}
// For each row fill except the first in the "order" column
for (int32_t k = 1; k < 8; k++)
table_entry[k][order] = table_entry[k - 1][order - 1];
table_entry[0][order] = 1 << 11;
// Place the 1.0 in the first row of the "order" column
table_entry[0][order] = 1 << 11; // 1.0 in qs4.11 fixed point
// Fill the remaining columns as a shifted-down copy of the previous column,
// adding 0s as-needed.
for (int32_t k = 1; k < 8; k++) {
int32_t j = 0;
@@ -52,6 +70,10 @@ expand_codebook(int16_t *book_data, int32_t ****table_out, int32_t order, int32_
return 0;
}
/**
* For each FIR filter matrix associated with a codebook page, pointed to by `table`, store the first
* "order" columns to a codebook at `book_data_out`.
*/
int
compressed_expanded_codebook(int16_t **book_data_out, int32_t ***table, int order, int npredictors)
{
@@ -59,9 +81,9 @@ compressed_expanded_codebook(int16_t **book_data_out, int32_t ***table, int orde
MALLOC_CHECKED_INFO(sizeof(int16_t) * 8 * order * npredictors, "order=%d, npredictors=%d", order, npredictors);
int n = 0;
for (int32_t i = 0; i < npredictors; i++) {
for (int32_t j = 0; j < order; j++) {
for (int32_t k = 0; k < 8; k++)
for (int32_t i = 0; i < npredictors; i++) { // For each matrix
for (int32_t j = 0; j < order; j++) { // For each column
for (int32_t k = 0; k < 8; k++) // For each row
book_data[n++] = table[i][k][j];
}
}
@@ -207,7 +229,8 @@ vdecodeframe(uint8_t *frame, int32_t *prescaled, int32_t *state, int32_t order,
int32_t **coef_page = coef_tbl[optimalp];
// Inner product with predictor coefficients
// Matrix multiplication with FIR filter matrix associated with the book page, the matrix operates on
// 8 samples simultaneously so this needs to be done twice for all 16 samples in the output frame.
for (int32_t j = 0; j < 2; j++) {
int32_t in_vec[16];
tools/audio/sampleconv/src/codec/vadpcm_tabledesign.c
+131 -75
View File
@@ -10,7 +10,7 @@
#include "../util.h"
#include "vadpcm.h"
// Levinson-Durbin algorithm for iteratively solving for prediction coefficients
// Levinson-Durbin algorithm for iteratively solving for prediction and reflection coefficients, given autocorrelation
// https://en.wikipedia.org/wiki/Levinson_recursion
static int
durbin(double *acvec, int order, double *reflection_coeffs, double *prediction_coeffs, double *error)
@@ -36,7 +36,7 @@ durbin(double *acvec, int order, double *reflection_coeffs, double *prediction_c
reflection_coeffs[i] = prediction_coeffs[i];
if (fabs(reflection_coeffs[i]) > 1.0) {
// incr when a predictor coefficient is > 1 (indicates numerical instability)
// incr when a reflection coefficient has a magnitude > 1 (indicates numerical instability in the model)
ret++;
}
@@ -82,7 +82,7 @@ kfroma(double *in, double *out, int order)
int i, j;
double div;
double temp;
double next[(order + 1)];
double next[order + 1];
int ret = 0;
out[order] = in[order];
@@ -144,29 +144,28 @@ rfroma(double *in, int n, double *out)
}
static double
model_dist(double *predictors, double *data, int order)
model_dist(double *mean_predictors, double *frame_predictors, int order)
{
double autocorrelation_data[order + 1];
double autocorrelation_predictors[order + 1];
double autocorrelation_frame_predictors[order + 1];
double autocorrelation_mean_predictors[order + 1];
double ret;
int i, j;
// autocorrelation from data
rfroma(data, order, autocorrelation_data);
// autocorrelation from frame predictors
rfroma(frame_predictors, order, autocorrelation_frame_predictors);
// autocorrelation from predictors
// autocorrelation from current mean predictors (?)
for (i = 0; i <= order; i++) {
autocorrelation_predictors[i] = 0.0;
for (j = 0; j <= order - i; j++) {
autocorrelation_predictors[i] += predictors[j] * predictors[i + j];
}
autocorrelation_mean_predictors[i] = 0.0;
for (j = 0; j <= order - i; j++)
autocorrelation_mean_predictors[i] += mean_predictors[j] * mean_predictors[i + j];
}
// compute "model distance" (scaled L2 norm: 2 * inner(ac1, ac2) )
ret = autocorrelation_data[0] * autocorrelation_predictors[0];
for (i = 1; i <= order; i++) {
ret += 2 * autocorrelation_data[i] * autocorrelation_predictors[i];
}
// this compares how good the mean predictors are to the optimal predictors for this frame
ret = autocorrelation_frame_predictors[0] * autocorrelation_mean_predictors[0];
for (i = 1; i <= order; i++)
ret += 2 * autocorrelation_frame_predictors[i] * autocorrelation_mean_predictors[i];
return ret;
}
@@ -363,7 +362,8 @@ split(double **predictors, double *delta, int order, int npredictors, double sca
}
static void
refine(double **predictors, int order, int npredictors, double *data, int data_size, int refine_iters)
refine(double **predictors, int order, int npredictors, double *all_frame_predictors, int num_frame_predictors,
int refine_iters)
{
int iter;
double dist;
@@ -376,50 +376,55 @@ refine(double **predictors, int order, int npredictors, double *data, int data_s
int counts[npredictors];
double vec[order + 1];
// For some number of refinement iterations
for (iter = 0; iter < refine_iters; iter++) {
// For some number of refinement iterations
// Initialize averages
// Initialize average autocorrelations
memset(counts, 0, npredictors * sizeof(int));
memset(rsums, 0, npredictors * (order + 1) * sizeof(double));
// Sum autocorrelations
for (i = 0; i < data_size; i++) {
// Sum autocorrelations for averaging for each frame, binning them based on best fitting predictor set
for (i = 0; i < num_frame_predictors; i++) {
best_value = 1e30;
best_index = 0;
// Find index that minimizes the "model distance"
// Find the choice of predictor that minimizes the "model distance" for this frame
for (j = 0; j < npredictors; j++) {
dist = model_dist(predictors[j], &data[(order + 1) * i], order);
// Compare with current mean predictors, the distance metric is based on autocorrelations
dist = model_dist(predictors[j], &all_frame_predictors[(order + 1) * i], order);
if (dist < best_value) {
// Record the new best predictors
best_value = dist;
best_index = j;
}
}
counts[best_index]++;
rfroma(&data[(order + 1) * i], order, vec); // compute autocorrelation from predictors
// Compute autocorrelation from optimal predictor
rfroma(&all_frame_predictors[(order + 1) * i], order, vec);
// Add to average autocorrelation for the best predictor choice
for (j = 0; j <= order; j++)
rsums[best_index][j] += vec[j]; // add to average autocorrelation
rsums[best_index][j] += vec[j];
// Update the counter of how many frames we've summed for this predictor
counts[best_index]++;
}
// finalize average autocorrelations
// Finalize average autocorrelations
for (i = 0; i < npredictors; i++) {
if (counts[i] > 0) {
if (counts[i] > 1) {
// Need to divide by the number of frames we summed
for (j = 0; j <= order; j++) {
rsums[i][j] /= counts[i];
}
}
}
// Update the predictors with the new average autocorrelations in each bin
for (i = 0; i < npredictors; i++) {
// compute predictors from average autocorrelation
// Compute predictors and reflection coefficients from average autocorrelation
durbin(rsums[i], order, vec, predictors[i], &dummy);
// vec is reflection coeffs
// clamp reflection coeffs
// Clamp reflection coeffs for stability
for (j = 1; j <= order; j++) {
if (vec[j] >= 1.0)
vec[j] = 0.9999999999;
@@ -427,44 +432,73 @@ refine(double **predictors, int order, int npredictors, double *data, int data_s
vec[j] = -0.9999999999;
}
// clamped reflection coeffs -> predictors
// Convert clamped reflection coeffs to stable predictors
afromk(vec, predictors[i], order);
}
}
}
static int
read_row(int16_t *p, double *row, int order)
read_row(int16_t *out, double *predictors, int order)
{
double fval;
int ival;
int i, j, k;
int overflows;
double table[8][order];
// (discussion is for order=2)
//
// Converts 2 predictors a,b into the coefficients for an FIR filter matrix
// [ c0, d0, 1, 0, 0, 0, 0, 0, 0, 0 ]
// [ c0, d1, d0, 1, 0, 0, 0, 0, 0, 0 ]
// [ c0, d2, d1, d0, 1, 0, 0, 0, 0, 0 ]
// [ c0, d3, d2, d1, d0, 1, 0, 0, 0, 0 ]
// [ c0, d4, d3, d2, d1, d0, 1, 0, 0, 0 ]
// [ c0, d5, d4, d3, d2, d1, d0, 1, 0, 0 ]
// [ c0, d6, d5, d4, d3, d2, d1, d0, 1, 0 ]
// [ c0, d7, d6, d5, d4, d3, d2, d1, d0, 1 ]
//
// Multiplication by this matrix on a vector containing the previous two samples p[-2] and p[-1] and 8 residuals
// s[i] decodes 8 samples p[i] simultaneously.
// Only c0..7 and d0..7 are actually stored in the book, the decoder arranges the rest.
//
// The coefficients are those you get by substituting decoded samples into the prediction model at each step to
// express each p[i] for i in [0, 8) as a linear combination of p[-2], p[-1] and past residuals
// p[i] = a * p[i - 2] * b * p[i - 1] + s[i]
//
// c0 = a, d0 = b
// c1 = ab, d1 = a + b^2
// c2 = a^2 + ab^2, d2 = 2ab + b^3
// ...
for (i = 0; i < order; i++) {
for (j = 0; j < i; j++)
table[i][j] = 0.0;
for (j = i; j < order; j++)
table[i][j] = -row[order - j + i];
table[i][j] = -predictors[order - j + i];
}
for (i = order; i < 8; i++)
for (i = order; i < 8; i++) {
for (j = 0; j < order; j++)
table[i][j] = 0.0;
}
for (i = 1; i < 8; i++)
for (j = 1; j <= order; j++)
if (i - j >= 0)
for (i = 1; i < 8; i++) {
for (j = 1; j <= order; j++) {
if (i >= j) {
for (k = 0; k < order; k++)
table[i][k] -= row[j] * table[i - j][k];
table[i][k] -= predictors[j] * table[i - j][k];
}
}
}
// Convert double-precision book entries into qs4.11 fixed point numbers,
// rounding away from 0 and checking for overflows
overflows = 0;
for (i = 0; i < order; i++) {
for (j = 0; j < 8; j++) {
fval = table[j][i] * (double)(1 << 11);
int ival;
double fval = table[j][i] * (double)(1 << 11);
if (fval < 0.0) {
ival = (int)(fval - 0.5);
if (ival < -0x8000)
@@ -475,7 +509,7 @@ read_row(int16_t *p, double *row, int order)
overflows++;
}
*(p++) = ival;
*out++ = ival;
}
}
@@ -517,27 +551,30 @@ tabledesign_run(int16_t *order_out, int16_t *npredictors_out, int16_t **book_dat
for (int i = 0; i < num_order; i++)
autocorrelation_matrix[i] = MALLOC_CHECKED_INFO(num_order * sizeof(double), "num_order=%d", num_order);
// (back-)align to a multiple of the frame size
size_t nframes = num_samples - (num_samples % frame_size);
double *data =
double *all_frame_predictors =
MALLOC_CHECKED_INFO(nframes * num_order * sizeof(double), "nframes=%lu, num_order=%d", nframes, num_order);
uint32_t data_size = 0;
uint32_t num_frame_predictors = 0;
int16_t *sample = sample_data;
// (back-)align to a multiple of the frame size
int16_t *sample_end = sample + nframes;
memset(buffer, 0, frame_size * sizeof(int16_t));
memset(buffer, 0, frame_size * sizeof(*buffer));
// First, compute the optimal set of predictors for every complete frame in the signal, where optimal here means
// the predictors that minimize the mean-square error between the predicted signal and the true signal.
for (; sample < sample_end; sample += frame_size) {
// Copy sample data into second half of buffer, during the first iteration the first half is 0 while in
// later iterations the second half of the previous iteration is shifted into the first half.
memcpy(&buffer[frame_size], sample, frame_size * sizeof(int16_t));
memcpy(&buffer[frame_size], sample, frame_size * sizeof(*buffer));
// Compute autocorrelation vector of the two vectors in the buffer
acvect(&buffer[frame_size], order, frame_size, vec);
// First element is the largest(?)
// First element of autocorrelation has the largest magnitude
if (fabs(vec[0]) > design->thresh) {
// Over threshold
@@ -552,15 +589,20 @@ tabledesign_run(int16_t *order_out, int16_t *npredictors_out, int16_t **book_dat
// R = autocorrelation matrix
// r = autocorrelation vector
// a = linear prediction coefficients
// After this vec contains the prediction coefficients
// After this vec contains the prediction coefficients that minimize the mean-square error.
lubksb(autocorrelation_matrix, order, perm, vec);
vec[0] = 1.0;
// Compute reflection coefficients from prediction coefficients
if (kfroma(vec, reflection_coeffs, order) == 0) { // Continue only if numerically stable
data[data_size * num_order + 0] = 1.0;
all_frame_predictors[num_frame_predictors * num_order] = 1.0;
// clamp the reflection coefficients
// Clamp the reflection coefficients. Reflection coefficients are clamped rather than the
// predictors themselves as the reflection coefficients have a direct relationship with the
// stability of the model. If the reflection coefficients are outside of the interval (-1,1)
// the model is unstable as the associated transfer function will contain poles outside the
// complex unit disk. If the reflection coefficients are all inside the interval (-1,1) there
// are no poles outside of the unit disk, guaranteeing the stability of the model.
for (int i = 1; i < num_order; i++) {
if (reflection_coeffs[i] >= 1.0)
reflection_coeffs[i] = 0.9999999999;
@@ -568,40 +610,44 @@ tabledesign_run(int16_t *order_out, int16_t *npredictors_out, int16_t **book_dat
reflection_coeffs[i] = -0.9999999999;
}
// Compute prediction coefficients from reflection coefficients
afromk(reflection_coeffs, &data[data_size * num_order], order);
data_size++;
// Compute prediction coefficients from clamped reflection coefficients
afromk(reflection_coeffs, &all_frame_predictors[num_frame_predictors * num_order], order);
num_frame_predictors++;
}
}
}
// Move second vector to first vector
memcpy(&buffer[0], &buffer[frame_size], frame_size * sizeof(int16_t));
memcpy(&buffer[0], &buffer[frame_size], frame_size * sizeof(*buffer));
}
// Now that predictors for every frame have been found, they need to be reduced to a manageable quantity
// (determined by npredictors) to build the prediction codebook that will be exported. First compute the average
// autocorrelation of the prediction models for all frames:
// Create a vector [1.0, 0.0, ..., 0.0]
vec[0] = 1.0;
for (int i = 1; i < num_order; i++)
vec[i] = 0.0;
for (uint32_t i = 0; i < data_size; i++) {
// Compute autocorrelation from predictors
rfroma(&data[i * num_order], order, predictors[0]);
for (uint32_t i = 0; i < num_frame_predictors; i++) {
// Compute autocorrelation from predictors, equivalent to computing the autocorrelation on the signal produced
// by following the prediction model exactly.
rfroma(&all_frame_predictors[i * num_order], order, predictors[0]);
for (int k = 1; k < num_order; k++)
vec[k] += predictors[0][k];
}
for (int i = 1; i < num_order; i++)
vec[i] /= data_size;
vec[i] /= num_frame_predictors;
// vec is the average autocorrelation
// Compute predictors for average autocorrelation using Levinson-Durbin algorithm
// vec now contains the average autocorrelation.
// Compute predictors for this average autocorrelation using the Levinson-Durbin algorithm.
double dummy;
durbin(vec, order, reflection_coeffs, predictors[0], &dummy);
// clamp results
// Clamp resulting reflection coefficients to ensure stability
for (int i = 1; i < num_order; i++) {
if (reflection_coeffs[i] >= 1.0)
reflection_coeffs[i] = 0.9999999999;
@@ -609,27 +655,37 @@ tabledesign_run(int16_t *order_out, int16_t *npredictors_out, int16_t **book_dat
reflection_coeffs[i] = -0.9999999999;
}
// Convert clamped reflection coefficients to predictors
// Convert clamped reflection coefficients to stable predictors
afromk(reflection_coeffs, predictors[0], order);
// Split and refine predictors
// Starting with the predictors obtained from the average autocorrelation, cluster the predictors for each frame
// via k-means until we have just npredictors worth of output
for (unsigned cur_bits = 0; cur_bits < design->bits; cur_bits++) {
double split_delta[num_order];
// Prepare [0, ..., -1, 0]
for (int i = 0; i < num_order; i++)
split_delta[i] = 0.0;
split_delta[order - 1] = -1.0;
// Split the predictors into two halves
split(predictors, split_delta, order, 1 << cur_bits, 0.01);
refine(predictors, order, 1 << (1 + cur_bits), data, data_size, design->refine_iters);
// Update the values of each half to the means of the halves
refine(predictors, order, 1 << (1 + cur_bits), all_frame_predictors, num_frame_predictors,
design->refine_iters);
}
int16_t *book_data = MALLOC_CHECKED_INFO((8 * order * npredictors + 2) * sizeof(int16_t),
"order=%d, npredictors=%d", order, npredictors);
// Now we have the reduced set of predictors, write them into the book of size 8 * order * npredictors
int16_t *book_data =
MALLOC_CHECKED_INFO(8 * order * npredictors * sizeof(int16_t), "order=%d, npredictors=%d", order, npredictors);
*order_out = order;
*npredictors_out = npredictors;
// As a final step, we need to convert the predictors into coefficients for an FIR filter so that samples in a
// frame can be decoded in parallel taking advantage of the RSP's 8-lane SIMD instruction set.
int num_oflow = 0;
for (int i = 0; i < npredictors; i++)
num_oflow += read_row(&book_data[8 * order * i], predictors[i], order);
@@ -641,7 +697,7 @@ tabledesign_run(int16_t *order_out, int16_t *npredictors_out, int16_t **book_dat
*book_data_out = book_data;
free(buffer);
free(data);
free(all_frame_predictors);
for (int i = 0; i < num_order; i++)
free(autocorrelation_matrix[i]);
tools/audio/sampleconv/src/container/aiff.c
+26 -6
View File
@@ -138,8 +138,12 @@ f64_to_f80(double f64, uint8_t *f80)
} f80tmp;
// get f64 bits
uint64_t f64_bits = *(uint64_t *)&f64;
union {
double f;
uint64_t u;
} tp;
tp.f = f64;
uint64_t f64_bits = tp.u;
int f64_sgn = F64_GET_SGN(f64_bits);
int f64_exponent = F64_GET_EXP(f64_bits);
@@ -197,8 +201,12 @@ f80_to_f64(double *f64, uint8_t *f80)
((uint64_t)f64_mantissa_hi << 32) | ((uint64_t)f64_mantissa_lo);
// write double
*f64 = *(double *)&f64_bits;
union {
double f;
uint64_t u;
} tp;
tp.u = f64_bits;
*f64 = tp.f;
}
int
@@ -309,6 +317,8 @@ aiff_aifc_common_read(container_data *out, FILE *in, UNUSED bool matching, uint3
nloops += inst.sustainLoop.playMode != LOOP_PLAYMODE_NONE;
nloops += inst.releaseLoop.playMode != LOOP_PLAYMODE_NONE;
out->num_loops = nloops;
out->loops = NULL;
if (nloops != 0) {
out->loops = MALLOC_CHECKED_INFO(nloops * sizeof(container_loop), "nloops=%lu", nloops);
@@ -495,11 +505,21 @@ aiff_aifc_common_read(container_data *out, FILE *in, UNUSED bool matching, uint3
if (!has_comm)
error("aiff/aifc has no COMM chunk");
if (!has_inst)
error("aiff/aifc has no INST chunk");
if (!has_ssnd)
error("aiff/aifc has no SSND chunk");
if (!has_inst) {
out->base_note = 60; // C4
out->fine_tune = 0;
out->key_low = 0;
out->key_hi = 127;
out->vel_low = 0;
out->vel_hi = 127;
out->gain = 0;
out->num_loops = 0;
out->loops = NULL;
}
if (out->data_type == SAMPLE_TYPE_PCM16) {
assert(out->data_size % 2 == 0);
assert(out->bit_depth == 16);
tools/audio/sampleconv/src/container/wav.c
+4 -3
View File
@@ -222,6 +222,7 @@ wav_read(container_data *out, const char *path, UNUSED bool matching)
smpl.sampler_data = le32toh(smpl.sampler_data);
out->num_loops = smpl.num_sample_loops;
out->loops = NULL;
if (out->num_loops != 0) {
out->loops =
MALLOC_CHECKED_INFO(out->num_loops * sizeof(container_loop), "num_loops=%u", out->num_loops);
@@ -362,11 +363,11 @@ wav_read(container_data *out, const char *path, UNUSED bool matching)
if (!has_inst) {
out->base_note = 60; // C4
out->fine_tune = 0;
out->gain = 0;
out->key_low = 0;
out->key_hi = 0;
out->key_hi = 127;
out->vel_low = 0;
out->vel_hi = 0;
out->vel_hi = 127;
out->gain = 0;
}
if (!has_smpl) {
tools/audio/soundfont_compiler.c
+12 -9
View File
@@ -1591,7 +1591,7 @@ emit_h_effects(FILE *out, soundfont *sf)
NORETURN static void
usage(const char *progname)
{
fprintf(stderr, "Usage: %s [--matching] <filename.xml> <out.c> <out.h>\n", progname);
fprintf(stderr, "Usage: %s [--matching] <filename.xml> <out.c> <out.h> <out.name>\n", progname);
exit(EXIT_FAILURE);
}
@@ -1749,12 +1749,12 @@ main(int argc, char **argv)
fprintf(out_h,
// clang-format off
"#ifdef _LANGUAGE_ASEQ" "\n"
".pushsection .fonts, \"\", @note" "\n"
" .byte %d /*sf id*/" "\n"
".popsection" "\n"
"#endif" "\n"
"\n",
"#ifdef _LANGUAGE_ASEQ" "\n"
".pushsection .note.fonts, \"\", @note" "\n"
" .byte %d /*sf id*/" "\n"
".popsection" "\n"
"#endif" "\n"
"\n",
// clang-format on
sf.info.index);
@@ -1779,8 +1779,11 @@ main(int argc, char **argv)
// emit name marker
FILE *out_name = fopen(filename_out_name, "w");
fprintf(out_name, "%s", sf.info.name);
FILE *out_name = fopen(filename_out_name, "wb");
// We need to emit an explicit null terminator so that we can run objcopy --add-section to include the name
// in a .note.name section in the compiled object file. This is so that the string that ends up in the .note.name
// section is null-terminated, its length may be verified by any tools that read the name out of this section.
fprintf(out_name, "%s%c", sf.info.name, '\0');
fclose(out_name);
// emit dependency file if wanted
tools/audio/util.c
-1
View File
@@ -1,6 +1,5 @@
/* SPDX-FileCopyrightText: Copyright (C) 2024 ZeldaRET */
/* SPDX-License-Identifier: CC0-1.0 */
#define _GNU_SOURCE
#include <ctype.h>
#include <stdarg.h>
#include <stdbool.h>