The experiments use a comprehensive collection of 33 tabular datasets, mainly
from OpenML OpenML2013, OpenML2017, OpenML2020, UCI uci_repository and Kaggle
kaggle. For the datasets, we select the 17 used by Kadra et al which serve as
a representative benchmark collection for tabular data
regularization_is_all_you_need, and in addition, focus on 16 mainly multiclass
classification datasets from UCI, Kaggle and OpenML. For example, the dataset
higgs contains sensor data from high-energy physics higgs. The dataset
clickpred contains advertisements in a search engine, and whether or not they
were clicked. From the selected collection, 14 tabular datasets were used by
AutoGluon Tabular agtabular, the state-of-the-art AutoML tool for tabular
data.

Dataset (Source) | Classes | Rows | Features
---|---|---|---
†vehicle (OpenML) | 2 | 846 | 22
†cars (OpenML) | 3 | 406 | 8
user model data (UCI) | 4 | 403 | 5
†kc1 (OpenML) | 2 | 145 | 95
†phoneme (OpenML) | 2 | 5404 | 6
skin-seg (OpenML) | 2 | 245057 | 4
ecoli-data (UCI) | 4 | 336 | 8
iris (UCI) | 3 | 150 | 7
†blood (OpenML) | 2 | 748 | 4
†higgs (OpenML) | 2 | 98050 | 29
wifi-localization (UCI) | 4 | 2000 | 7
†nomao (OpenML) | 2 | 34465 | 119
olinda-outlier (OpenML) | 4 | 75 | 3
†australian (OpenML) | 2 | 690 | 15
†segment (OpenML) | 2 | 2310 | 20
led (UCI) | 10 | 500 | 7
†numerai (OpenML) | 2 | 96320 | 22
†miniboone (OpenML) | 2 | 130064 | 51
wall-robot (Kaggle) | 4 | 5456 | 3
†jasmine (OpenML) | 2 | 2984 | 145
yeast (UCI) | 10 | 1484 | 8
†christine (OpenML) | 2 | 5418 | 1637
†sylvine (OpenML) | 2 | 5124 | 21
seismic-bumps (UCI) | 3 | 210 | 8
ccfraud (OpenML) | 2 | 284807 | 31
clickpred (OpenML) | 2 | 1496391 | 10
vowel (UCI) | 2 | 528 | 21
nursery (UCI) | 5 | 12958 | 9
spectf-data (Kaggle) | 2 | 267 | 45
teaching assist (UCI) | 3 | 151 | 7
wisconsin (UCI) | 2 | 194 | 33
sonar (Kaggle) | 2 | 208 | 61
ionosphere (UCI) | 2 | 351 | 35

Note: †indicates that the dataset was appeared in the AutoGluon Tabular paper
[agtabular].

Table : The collection of the datasets.

Table  provides the full list of datasets and their main characteristics. Each
dataset is split into 80% training and 20% testing sets. The prediction
accuracy results for both Tiny Classifier circuits and ML baseline models in
the following subsections are based on test datasets.

##  Selection of Baseline ML Models

We use Google’s TabNet DNN [tabnet] with the recommended hyperparameters
configuration, and AutoGluon (An AutoML system developed by Amazon)
[agtabular, agtabulardistill] with explicit support for tabular data (Tabular
Predictor) as well as other baseline ML models. Google’s TabNet is one of the
first successful DNNs addressing tabular data, using sequential attention to
select features for decision-making layers. AutoGluon searches the design
space over three state-of-the-art models (i.e, XGBoost, TabNeuralNet and
NNFastAITab) for Tabular Data among others. AutoGluon XGBoost is based on
Gradient Boosting, whereas the other two models are based on DNNs. In our
experiments, AutoGluon Tabular Predictor is configured with the above three
models. Kadra et al. [regularization_is_all_you_need] observe that a Neural
Architecture Search (NAS) over Multi-layer Perceptrons (MLPs) delivers state-
of-the-art NN models for tabular data. Hence, we also use the NAS-based
protocol described by Kadra et al. [regularization_is_all_you_need] to
generate baseline MLP models.

[width=0.9]graphs/heatmap_all.pdf

Figure : (a) Accuracy vs. number of gates. Generations for the termination
function is 300 and termination iterations is 2000. Full FS indicates that the
generated circuit will be constructed with logical gates within the function
set $F=\\{\texttt{and},\texttt{or},\texttt{nand},\texttt{nor}\\}$. For NAND
function set the generated circuits constructed only with NAND gates. (b)
Accuracy vs. generations for the termination function. The number of gates is
300 and the number of termination iterations is 2000. (c) Accuracy vs. the
number of termination iterations. The number of gates and the generations for
the termination function are both set to 300.

##  Data Encoding and Quantization Strategy

Numerical inputs are automatically handled to encode the features of a dataset
based on user preferences. The encoding consists of the encoding strategy and
the number of bits per input. The encoding strategy determines the way that
numerical features get translated into binary. Currently, four main encoding
strategies are supported: (a) quantization, where each feature is divided into
buckets of equal width, (b) quantiles, where each feature is divided into
buckets of width roughly equal numbers to the number of samples, (c) one-hot
and (d) gray. Additionally, the users can manually tune the number of bits per
input to decide the granularity of the input encoding. From now onwards,
experiments report only the best-achieved accuracy across the available
encoding strategies with two and four bits per input. In the comparative
analysis with Tiny Classifiers, MLP models are transformed into 2-bit
quantized versions. Since the hardware requirements of Tiny Classifiers are
minimal, a comparison against the non-quantized MLPs does not provide a fair
baseline when considering latency, area and power. Thus, we use a 2-bit
quantized MLP as the resource-optimized high-performing baseline.

##  Tiny Classifier Design Space

A primary goal is to check whether we can generate accurate combinational
logic for an ML classification problem. We explore different combinations of
the hyperparameters (see Section LABEL:sec:sec:hyperparameters) of the
evolutionary algorithm to improve the accuracy of the generated circuits for
all the datasets. Next, we explore the design space in four main directions:
(a) the size of the generated circuit $n$ (number of gates), (b) the function
set $F$ from which solutions (circuits) may be constructed, (c) the number of
generations for the termination criterion function $\kappa$, and (d) the
number of iterations to achieve a performance threshold and terminate $G$. The
heatmap of Figure a presents the achieved accuracy of the generated Tiny
Classifier circuits as we progressively decrease the target NAND gate count
from 300 to 50. At the same time, we explore the accuracy of the circuits with
two different function sets. Overall, we observe a 14 percentage points
reduction in GEOMEAN across all datasets from 300 gates to 50 gates. The next
step is to study how the number of generations for the termination criterion
function impacts the accuracy of Tiny Classifiers when we limit the circuit
size to a maximum of 300 gates. Figure b shows the achieved accuracy for
various generation values of the termination criterion function. No
significant change in prediction performance is observed. Figure c presents
the number of termination iterations versus achieved accuracy. We
progressively increase the number of termination iterations as we set the
target gate count and the number of generations for the termination function
to 300. We observe a 2 percentage points improvement in GEOMEAN accuracy
across all datasets when increasing the number of iterations.

[width=]graphs/accuracy_all_autoML.pdf

Figure : Prediction Accuracy of Tiny Classifiers, AutoGluon XGBoost, AutoGluon
TabularNeuralNet, AutoGluon NNFastAITabular and Google TabNet. Note the
datasets _vehicle_ to _ionosphere_ (left to right) are binary, and the
remainder are multiclass classifications.

##  Accuracy Comparison

[width=]graphs/robustness_all.pdf

Figure : Violin plots showing the accuracy distributions of Tiny Classifiers
and AutoGluon XGBoost.

[width=]graphs/accuracy_all_mlp.pdf

Figure : Prediction Accuracy of Tiny Classifiers, smallest MLP (non-quantized
and 2-bit quantized versions), best MLP (non-quantized and 2-bit quantized
versions) and AutoGluon XGBoost.

Figure  compares the prediction accuracy of Google TabNet, AutoGluon and Tiny
Classifiers. Based on the analysis in Section , the hyperparameters of Tiny
Classifiers are set to 300 for both the number of gates and the termination
function. In addition, the maximum number of iterations is set to 8000. Across
all the datasets, the average prediction accuracy of AutoGluon XGBoost is 81%,
which is the overall highest. The mean accuracy of Tiny Classifiers across all
the datasets is 78%, which is the second highest. We compare the prediction
accuracy distribution of Tiny Classifiers against AutoGluon XGBoost to
understand how robust Tiny Classifiers are with respect to XGBoost. To this
end, we perform a 10-fold cross-validation study and show the accuracy
distributions of Tiny Classifiers and XGBoost in Figure  using a violin plot.
The interquartile range of Tiny Classifiers is comparable to the interquartile
ranges of ML baselines and in some cases, even slightly shorter. The shape of
the distribution in Tiny Classifiers indicates that the accuracy data are
highly concentrated around the median. This implies a low variance of the
accuracy distribution and therefore makes Tiny Classifiers robust to
variation.

[width=0.95]graphs/3Danalysis_surfacePlot.pdf

Figure : Space Exploration for four datasets (vehicle, phoneme, Teaching
Assist and Cars) beyond the limit of 300 logical gates. The best achieved
prediction accuracy of Tiny Classifier Circuits with 400 gates ranges from 76%
- 96%.

The best-performing ML model, XGBoost, and Tiny Classifiers from Figure  are
also compared to the best and smallest MLP configurations. We first explore
the accuracy of a 9-layer MLP with 512 neurons following the protocol
described in [regularization_is_all_you_need] (i.e., best MLP configuration)
where the number of layers refers to the “hidden” layers of the neural
network. The NAS takes this MLP as a starter and reduces the number of layers
and neurons until reaching the smallest possible neural network size with
minimal accuracy loss, which becomes a 3-layer MLP with 64 neurons. Figure
shows the prediction accuracy of six models (i.e., XGBoost, Tiny Classifiers,
non-quantized best MLP, 2-bit quantized best MLP, non-quantized smallest MLP
and 2-bit quantized smallest MLP). Across all datasets, the non-quantized best
MLP model tops the performance by 83% overall prediction accuracy whilst its
2-bit quantized version has the same performance as Tiny Classifiers. In
contrast, the non-quantized smallest MLP has an overall prediction accuracy of
80% whilst its 2-bit version stays at 75%. In summary, the performance of Tiny
Classifiers is no worse than the 2-bit quantized MLP models. Figure
illustrates the benefit of increasing the circuit size limit from 300 gates to
400 gates of Tiny Classifiers for four datasets which present a poor
classification accuracy compared to AutoGluon XGBoost. The prediction accuracy
for these four datasets improves by up to 11 percentage points when moving the
limit from 300 to 400 logical gates.

##  ASIC Flow Results

We design Tiny Classifiers in hardware across all datasets. For a comparison
point, we also design the two ML baseline models in hardware. In addition to
XGBoost (best performing ML baseline), the 2-bit quantized smallest MLP is
also chosen as the second baseline ML model because it is the smallest MLP
baseline (3 layers/64 neurons). As we needed to design the baseline ML models
in hardware manually, we designed them only for two datasets (i.e., _blood_
and _led_). These two datasets are selected based on the number of classes and
the complexity of implementing XGBoost in hardware. _blood_ has one of the
smallest numbers of classes (i.e., 2) and _led_ has one of the largest numbers
of classes (i.e., 10). The default number of estimators (Parallel Decision
Trees) for XGBoost in Python [xgboost_tool] is 100 for a binary classification
problem and $100\times number\\_of\\_classes$ for multi-class classification.
The number of estimators is strongly correlated with the achieved accuracy of
the model. The main reason why _blood_ is selected among other 2-class
datasets is because XGBoost in _blood_ has the smallest number of estimators
with the smallest accuracy loss across all the 2-class datasets. A similar
observation is made for _led_ that it requires a smaller number of estimators
compared to _yeast_ (i.e., the other dataset that has also 10 classes) to
achieve iso-performance. One estimator (binary classification) for _blood_ and
10 estimators (one estimator for each target class) for _led_ are designed in
hardware for XGBoost. For the development and the verification of the MLP and
XGBoost designs, Bluespec System Verilog is used [bluespec], and the designs
are simulated with Bluesim.

[width=0.8]figures/pragmatIC.pdf

Figure : Flexible chip layouts of Tiny Classifiers and XGBoost implemented in
PragmatIC’s 0.8$\mu$m FlexIC TFT process for _blood_ and _led_ datasets.

[width=]graphs/power_asic_new.pdf

Figure : Power consumption of Tiny Classifier circuits across all datasets
where MLP and AutoGluon XGBoost designs are shown for _blood_ and _led_
datasets.

###  Synthesis Results for Silicon Target:

The Verilog representation of Tiny Classifiers and the two ML baselines are
synthesized using Synopsis Design Compiler targeting the open 45nm PDK
[freePDK45] Silicon technology. We present the synthesis power and area
results for each Tiny Classifier circuit and baseline ML model as standalone
hardware blocks, i.e., no interconnections to other components part of an
overall ASIC design. The required data has been transferred to an input
buffer, and the class predictions are stored in an output buffer inside the
block. Both input and output buffers are included in the power and area
calculations. The operational voltage and frequency are 1.1V and 1GHz,
respectively.

[width=]graphs/area_asic_new.pdf

Figure : NAND2-equivalent gate count of Tiny Classifier circuits where MLP and
AutoGluon XGBoost are shown for _blood_ and _led_ datasets.

Figures  and  show the power consumption and the area in NAND2-equivalent gate
count. Tiny Classifier circuits consume 0.04 - 0.97 mW, and the gate count
ranges from 11-426 NAND2-equivalent gates (combinational logic plus the I/O
buffers). Note that two classifier circuits have just 3 gates excluding the
I/O buffers. The power consumption of MLP is 34-38 mW (86-118 times greater
than that of Tiny Classifiers), and the area is $\sim$171 and $\sim$278 times
larger than Tiny Classifiers for _blood_ and _led_. The power consumption of
XGBoost is $\sim$3.9 and $\sim$8 times higher than Tiny Classifiers for
_blood_ and _led_ whilst the area is 8 and 18 times larger than Tiny
Classifiers, respectively.

###  Implementation Results for Flexible Chips:

As discussed in _Introduction_ , Tiny Classifiers are ideal for low-cost
flexible chips for smart packages. We pick XGBoost as the ML baseline for
comparison because it is more efficient in terms of occupied area and power
than the MLP. Both Tiny Classifiers and XGBoost designs for _blood_ and _led_
are implemented with PragmatIC’s 0.8$\mu$m FlexIC metal-oxide thin-film
transistor (TFT) process in PragmatIC’s FlexLogIC line [pragmatic]. The
designs are put through the Cadence implementation flow to generate chip
layouts111Tiny Classifier and XGBoost designs for _blood_ are sent for
fabrication. They will be fabricated on a 30$\mu$m thick polyimide substrate
and tested.. Figure  shows the flexible chip layouts of the four designs.
Table  summarizes the power, performance and area results. Tiny Classifier for
_blood_ is 10 times smaller and consumes about 13 times less power than
XGBoost whilst it can run twice as fast as XGBoost. On the other hand, the
comparative results for _led_ are more prominent as Tiny Classifier is about
75 times smaller & lower power and three times faster than XGBoost. An
important observation is that the area variation of Tiny Classifiers between a
binary and a multi-class classification problem is negligible. Specifically,
our methodology generates a smaller Tiny Classifier for _led_ (105
NAND2-equiv. gates) compared to _blood_ (150 NAND2-equiv. gates). In contrast,
XGBoost implementation for _led_ occupies 5 times more area than _blood_
mainly due to the larger number of mapped estimators for multi-class
classification.

| Tiny Classifiers | XGBoost
---|---|---
| _blood_ | _led_ | _blood_ | _led_
Cell Area ($mm^{2}$) | 0.54 | 0.37 | 5.4 | 27.74
Power ($mW$) | 0.32 | 0.25 | 4.12 | 18.6
Max. Freq. ($kHz$) | 350 | 440 | 165 | 130
NAND2-equivalent | 150 | 105 | 1520 | 7780
Table : Tiny Classifiers and XGBoost implementation results in PragmatIC’s
0.8$\mu$m FlexIC TFT process at 3V supply voltage.

##  FPGA-based Comparison

We also prototype Tiny Classifiers, XGBoost and the 2-bit quantized smallest
MLP for the two datasets on an FPGA platform to demonstrate the software-
hardware co-design environment. Trained 2-bit quantized MLPs are synthesized
on reconfigurable hardware using Xilinx FINN, the state-of-the-art tool which
generates dataflow-style architectures of neural networks on FPGAs
[xilinx_finn]. After the initial configuration of the MLPs, we use Brevitas
[brevitas] to transform the neural network to a quantized trained neural
network. For the Brevitas training, we use 2-bit quantized ReLU activation
functions and apply batch normalization between each layer and its activation.
The configuration of Brevitas follows the recommendations of Xilinx FINN
[xilinx_finn_with_mlp]. Then, Xilinx FINN is used to implement the trained
neural network as a dataflow accelerator on FPGAs. We set the default
configuration settings to build in dataflow mode. Figure  presents the FPGA
resource utilization on a Xilinx Zynq Ultrascale+ MPSoC. For _blood_ dataset,
we observe that Tiny Classifiers consume 2.43x less FPGA resources in terms of
the number of look-up tables (LUTs) and flip-flops (FFs) when compared to
XGBoost, and 10.7x less FPGA resources than the Smallest MLP. For _led_
dataset, XGBoost and the Smallest MLP are 2.92x and 3.87x larger than Tiny
Classifiers, respectively.

[width=0.48]graphs/resources_fpga.pdf

Figure : FPGA resource utilization (LUTs and FFs) of the 2-bit quantized
Smallest MLP, XGBoost and Tiny Classifiers hardware implementation for _blood_
and _led_ datasets.


# Sound Terminology Describing Production and Perception of Sonification
††thanks:

Ziemer, Tim
Institute of Systematic Musicology
University of Hamburg
Germany
<EMAIL_ADDRESS>
###### Abstract

Sonification research is intrinsically interdisciplinary. Consequently, a
proper documentation of, and interdisciplinary discourse about a sonification
is often hindered by terminology discrepancies between involved disciplines,
i.e., the lack of a common sound terminology in sonification research. Without
a common ground, a researcher from one discipline may have troubles
understanding the implementation and imagining the resulting sound perception
of a sonification, if the sonification is described by a researcher from
another discipline. To find a common ground, I consulted literature on
interdisciplinary research and discourse, identified problems that occur in
sonification, and applied the recommended solutions. As a result, I recommend
considering three aspects of sonification individually, namely 1.) Sound
Design Concept, 2.) Objective and 3.) Method, clarifying which discipline is
involved in which aspect, and sticking to this discipline’s terminology. As
two requirements of sonifications are that they are a) reproducible and b)
interpretable, I recommend documenting and discussing every sonification
design once using audio engineering terminology, and once using psychoacoustic
terminology. The appendix provides comprehensive lists of sound terms from
both disciplines, together with relevant literature and a clarification of
often misunderstood and misused terms.

_K_ eywords Sonification, Auditory Display, Psychoacoustics, Audio Engineering

##  1 INTRODUCTION

While early definitions refer to sonification as “the use of non-speech audio
to convey information” [1], newer definitions add the requirements of the
sound being a) _reproducible_ and b) _interpretable_ [2, p. 274][3]. This
paper is supposed to help sonification researchers meet these requirements by
suggesting a common sound terminology for interdisciplinary sonification
research and discourse.

Sonification is intrinsically interdisciplinary. For example, [4] present the
interdisciplinary cycle of sonification, including audio engineering,
psychoacoustics and music, but also cognitive sciences, acoustics/computer
music, computer science, linguistics/philosophy, social sciences, product
design, data mining/statistics and many more. [4] state that physics,
acoustics, psychoacoustics, perceptual research, sound engineering and
computer science are “required to comprehend and carry out successful
sonification”, while respecting that sonification also has facets concerning
psychology, musicology, cognitive sciences, linguistics, pedagogies, social
sciences and philosophy. The Sonification Handbook [5] contains chapters
dedicated to psychoacoustics [6], perception and cognition [7], human-computer
interaction [2], design and aesthetics [8]. David Worrall’s Sonification
Design book [9] covers philosophy, computer science, cognitive sciences,
neuroscience, psychology, psychoacoustics, and music theory. Many papers deal
with the interdisciplinary nature of sonification research [4, 10, 11, 12].

Interdisciplinarity is often considered a strength of sonification research
due to the potential to understand the overall picture rather than
illuminating single fragments [4, 13]. But hurdles have been recognized for
interdisciplinarity to unfold in sonification research, due to

  1. 1.

no common terminology, especially concerning sound [4, 10, 12]

  2. 2.

blurred separation of disciplines’ methods and goals [4, 9, 11, 12].

[14] stated that promoting interdisciplinarity in sonification research has
been addressed rather hesitantly. Fortunately, studies on interdisciplinary
research have already identified related, general issues, and proposed
respective solutions. In this paper, I relate the sonification-specific
problems to general problems in interdisciplinary research and then transfer
the proposed, general solutions to sonification research. The result is a list
of recommendations concerning the use of sound terminology and the
clarification of involved disciplines’ methods and goals. The remainder of
this paper is structured as follows: Sect. 2 starts with a review of general
issues in interdisciplinary research and proposed solutions. In Sect. 2.2 I
relate these issues to sonification research, based on prominent examples.
Sect. 3 aims at applying the proposed solutions to sonification research,
formulating explicit recommendations to sonification researchers. These are
discussed in Sect. 4, followed by a conclusion in Sect. 5. The appendix
provides literature recommendations for lexical study of sound terminology, a
comprehensive list of terms and definitions describing the sound of
sonifications with the vocabulary of audio engineering and of psychoacoustics,
and reveals important distinctions between terms that are often confused or
misused in the sonifiction literature.

##  2 METHOD

To tackle the issue of 1.) no common sound terminology and 2.) blurred
separation of disciplines’ methods and goals in sonification, I consult
literature on interdisciplinary research in general to

  * •

identify general problems in interdisciplinary research,

  * •

identify proposed solutions for interdisciplinary research in general,

  * •

relate these problems to sonification research, and, finally,

  * •

apply the proposed solutions to sonification research.

###  2.1 Interdisciplinarity in Research

Table 1: Problems and solutions in interdisciplinary research. # | Problem | Solution
---|---|---
1 | Conflicting Paradigms | Name concepts, practices, and objectives
2 | Strong Interdisciplinarity | Name identity criteria
3 | Communication Discrepancies |  Carry out lexical study
4 | Interdisciplinary Semantic Drifts | Name discipline and stick to its terminology

Literature on interdisciplinarity is often dedicated to typologies and
taxonomies [15]. An overview of interdisciplinary research is given in [16].

[17] describe the issues of conflicting paradigms. Conflicting paradigms refer
to different viewpoints and interests. For example, one discipline may be
interested in the object under investigation, while the other may be
interested in its impact on the individual or the society. As an approach to a
solution, writers and readers need to detect and interpret involved
disciplines and their concepts, practices and objectives consciously.

[16] distinguish between “weak” and “strong interdisciplinarity”. In the case
of weak interdisciplinarity, the disciplines share a “proximity to the
interests of recognition, which are reflected in pre-discursive consent,
terminology, etc.” As an example, they describe how electrical engineering and
mechanical engineering may be necessary to construct a robot, where we have
electric circuits on the one hand and mechanics on the other. In the case of
strong interdisciplinarity, it is doubtful whether disciplines agree on the
object of interest, methods and measures, and terminology. As a solution, the
authors state that “disciplines have to organise themselves within
interdisciplinary research frameworks” [16, p. ix]. This means, clear identity
criteria have to be named, e.g., which discipline is responsible for what, and
whose methods and terminology is applied when[16, sect. 2.3.3.2].

[17] discuss the issue of communication discrepancies. Communication
discrepancies refer to cases in which, e.g., one discipline uses terms
colloquially, which are specifically defined in another discipline. This can
lead to the loss of detail or connotation, or even to a total misconception.
As a solution, they recommend lexical studies to become aware of discipline-
specific meanings and connotations of terms.

[18] describe how concepts from one discipline are borrowed by another
discipline and then acquire new meanings or connotations, called
_Interdisciplinary Semantic Drifts_ (ISDs). In contrast to the communication
discrepancies discussed above, ISDs includes terms which are not used
colloquially in either discipline. Still, as long as the reader does not know
which discipline’s definition of a term is applicable, misunderstandings can
appear. The easiest solution to this issue is to name the respective
discipline whose terminology you use, to stick to it as exactly as possible,
and to mention whenever you include the terminology from another discipline.

Note that these issues are not completely independent of one another, but
partly point in the same direction. Still, it is helpful to transfer them to
interdisciplinary sonification research one by one. In summary, we can
identify $4$ problems and approaches towards solutions in interdisciplinary
research as listed in Table 1.

How these problems relate to sonification research is discussed in the
following section, subdivided into the four raised problems.

###  2.2 Interdisciplinarity in Sonification Research

The preceding section dealt with interdisciplinary research in general. This
section shows how these problems relate to sonification research.

####  2.2.1 Conflicting Paradigms

Many sonification studies reveal relevant discipline(s) already in the title,
like art [19, 20], music [21], speech [22], design [8, 23], aesthetics [8],
perception [7, 9], psychoacoustics [6, 24, 25, 26, 27, 28], cognition [7],
mathematics [22], philosophy [29], and/or signal processing [28]. This informs
readers about discipline(s) involved. Unfortunately, many of these studies do
not point out whether the discipline’s concepts, objects of interest, or
methods and measures are being applied. This gives rise to conflicting
paradigms, as it makes a big difference, e.g., whether a musical sonification
applies music theory and composition techniques to communicate data to a user,
or whether data is used as a foundation for a music composition with the
objective of an enjoyable listening experience. Furthermore, some studies’
titles [30, 31] include musical terms, even though neither the sonification
design concept nor the object of interest is musical. These examples of
conflicting paradigms are closely related to the general issue of a blurred
separation of disciplines’ methods and goals [4, 9, 11, 12].

####  2.2.2 Strong Interdisciplinarity

As stated above, sonification is intrinsically interdisciplinary. But some
studies focus on either side, like the viewpoint of audio engineering [32],
human factors [33, 34], psychoacoustics [24, 25], design [23] or arts [19, 20,
29]. Readers of such studies can expect to find the terminology from the
respective discipline and results that are relevant to the specific
discipline. This can be considered a weak interdisciplinarity from the
viewpoint of the specific discipline. The same is partly true for sonification
studies that focus on solving a specific problem, like [22, 26]. Here, the
data and the problem come from one discipline, the _domain science_ , while
the sonification to solve the problem comes from another. The contributions of
each discipline are clearly separated. This makes the collaboration easier.
This seems like a good example for the successful determination of identity
criteria: the data comes from one discipline, the sonification from another.
Nevertheless, it may be difficult for a sonification researcher to read
various papers like this, as the terminologies used to describe the sound may
vary across studies, i.e., across involved disciplines. This is because
sonification research lacks a common terminology, especially concerning sound
[4, 10, 12]. Moreover, sonification researchers are often interested in
sonification from a multitude of viewpoints and, thus, strong
interdisciplinarity. Examples for interdisciplinary discourse on sonification
were already provided in the introduction, like [5, 9].

But even these prominent examples of interdisciplinary sonification research
sometimes suffer from the strong interdisciplinarity problem. In a short
passage of his book, Worrall mixes terminologies from multiple disciplines,
writing that “Mappings were to pitch, amplitude and frequency modulation
(pulsing and detuning), filter coefficients (brightness) and onset time
(attack).” [9, p. 217] Here, some terms come from the field of audio
engineering (amplitude and pitch modulation, filter coefficients, attack time)
while others come from the field of perception (pitch and brightness). As a
consequence, it is neither completely clear how the sound is manipulated
(audio engineering) nor how it may sound (psychoacoustics). In another part of
the book, he writes about “pitch, loudness, stereo-location” [9, p. 218].
Clearly, pitch and loudness are perceptual terms coming from the field of
psychoacoustics. However, the term “stereo-location” is a bit ambiguous,
probably referring to both the perceived source location, but also to panning
technology in a stereo loudspeaker setup (audio engineering). Even though this
term appears to be an audio engineering term, it does not reveal how the
stereo location was implemented (amplitude based panning, like the sine law,
tangent law, or Chowning’s panning law, time-based panning, or fading-based
panning [35, Sects. 7.2 and 9.3]). In these passages, the lack of clear
identity criteria is evident: Does the description contribute to a) the
reproducibility of sonification, or b) the interpretability? Authors need to
identify and clearly separate these two identities of a single sonification.

####  2.2.3 Communication Discrepancies

Communication discrepancies easily arise when talking about sonification. As
discussed above, sonification needs to be a) replicable and b) interpretable,
which is why we need a physical and a perceptual description of each
sonification. A common complaint is that authors do not distinguish between
physical and perceptual terms [7, p. 64][27, p. 1077][36, p. 6][37, p.20 and
chap. 2]. Prominent examples can be found when reading different chapters of
The Sonification Handbook [5]: For example, [13] speak of “(…) acoustic
variables, such as pitch or loudness” [13, p. 18] even though these are
clearly auditory (from a perceptual domain), and not acoustic (from the
physical domain) variables. It seems that the authors use these terms
colloquially, and not in the strict sense of acoustics and psychoacoustics.
Likewise, [38, Sect. “Zipper Noise”] writes about “signal parameter such as
loudness”. David Worrall writes “[T]hat we (at least in English) so frequently
substitute the word ‘note’ for ‘tone’, and ‘music’ for ‘score”’ [9, p. 18].
Some studies only provide sonification descriptions from either side: “(…) in
a sonification of real-time financial data Janata and Childs (2004) used
rising and falling pitch to represent the change in price of a stock and
loudness to indicate when the stock price was approaching a pre-determined
target (…)” [7, p. 64] Obviously, this description of the sonification helps
the reader imagine how it may sound. But it does not enable the reader to
reproduce it. Another example can be found in Worrall’s book: “Earcons are
made by transforming a tone’s psychophysical parameters–pitch, loudness,
duration and timbre–into structured, non-verbal ‘message’ combinations.” [9,
Sect. 2.2.1.3] This citation shows how an author successfully managed to
describe earcons on a perceptual level with psychoacoustic terminology,
avoiding a mixture with musical or physical terms. Other studies aim at
illuminating both the reproduction and the interpretation of a sonification:
[28] give the complete formula

$\begin{split}a_{\text{out}}(\Delta x,\Delta y,\Delta z,t)=g(\Delta
y,t)\sum_{n=0}^{N}\left[A(\phi_{n}(\Delta x,t),\Delta y,\Delta z)\right.\\\
\left.\times\cos\left[\omega_{\text{car}}(\phi_{n}(\Delta x,t))t+\beta(\Delta
z)\cos(\omega_{\text{mod}}t)\right]\right]\end{split}$ (1)

and then provide equations for all variables, enabling the reader to reproduce
the sonification when the three input variables $\Delta x$, $\Delta y$ and
$\Delta z$ are given. In the section titled _Psychoacoustics_ , they describe
how the signal processing affects perception, using the psychoacoustical terms
_pitch_ _height_ , _chroma_ , _loudness_ , _brightness_ , _roughness_ ,
_fullnes_ , _subjective duration_ , _tonalness_ , and _harmonicity_ , and a
reference to _auditory scene analysis_. This paper stringently uses
psychoacoustic terminology to describe the sonification perception, and signal
processing formulas to describe the implementation. What is missing, is the
explanation of the formula in audio engineering terms.

####  2.2.4 Interdisciplinary Semantic Drifts

“The vast majority of the tools and techniques used for the computer synthesis
of sound have been developed by composers and engineers engaged in the task of
making new music” [9, p. 46]. This circumstance is one of the reasons of
interdisciplinary semantic drifts in sound terminology. A typical case of a
semantic drift can be found in the introductory explanation chapters of
Chowning’s and Bristow’s book _FM Theory & Applications. By Musicians for
Musicians_ [39]. As the book title implies, the inventor and implementor of FM
synthesis and the Yamaha DX7 explain their audio invention to musicians. This
leads to referring to _frequency modulation_ as _vibrato_ , to _modulation
depth_ as _vibrato depth_ , and to _modulation frequency_ as _vibrato rate_.
This aim of translating from the audio domain to the musical domain for a
better understanding may be one of the origins of semantic drifts. Note that,
originally, _vibrato_ is a musical term. Like many musical terms, it is
related to the production mechanism of this characteristic sound: vibrating
your finger on the string that you are playing, or vibrating the telescoping
slide of a trombone back and forth. And what this does to the sound is more
than a frequency modulation. As the frequency envelope of string instruments
is not flat, a vibrato is always a combination of a coherent frequency
modulation and incoherent (individual for all partials) amplitude modulation.
Generally speaking, the inclusion of electrical engineering and recording
studio technology in the creative process of music making initiated
reconsideration of traditional musical terms, and introduced signal processing
terms to the music domain.

Likewise, the term _pitch_ from the music domain underwent a semantic drift in
the psychoacoustic domain. In the music domain, pitch refers to the vertical
dimension of notes, and to musical intervals. It often, but not always, refers
to the fundamental frequency. In psychoacoustics, pitch refers to a
multidimensional perception consisting of a height dimension, a chroma
dimension and a strength that are affected by periodicity, (fundamental)
frequency and frequency distribution, sound pressure level, the place-
principle and phase locking of neurons in the auditory nerve [40].

##  3 SOLUTION FOR SONIFICATION RESEARCH

The examples above show that many general issues of interdisciplinary research
apply to sonification research. Consequently, I aim at transferring the
proposed solutions to the sonification domain.

###  3.1 Name Concepts, Objectives and Practices

The suggested solution for conflicting paradigms is to name concepts,
objectives and practices. This solution can be applied directly on
sonification research. Interdisciplinarity within a single sonification study
means that different disciplines may be involved in the three aspects of
sonification:

  1. 1.

Sound Design Concept

  2. 2.

Objective

  3. 3.

Method

The sound design concept typically has one source of inspiration, but also two
requirements, which are a) reproducibility and b) interpretability [1, 2, 3].
The source of inspiration can come, e.g., from the field of music [41],
psychoacoustics [42], design [43], speech [44], Gestalt psychology [45], or
signal processing [28]. The objective of a sonification study can be, e.g, the
conceptualization of a sonification design principle [46], to set up a
framework for sonification design [47] or evaluation [48], to evaluate the
perception of a sonification [49], the performance [27] or user experience
[50] of sonification users, or the sonification’s impact on the individual
[51], the society [52], the environment [53], etc. While some of these
objectives are irrelevant for some disciplines, sonification researchers
should always be interested in all these objects of interest, for the bigger
picture of sonification. Depending on the objective, different methods may be
appropriate. Methods can come from the field of psychoacoustics [24], human
factors [54], UX-Design [55], experimental psychology [49], Gamification [56],
Human-Computer Interaction (HCI) [57] and others [58].

Recommendation: Authors should always refer to all three aspects of
sonification, namely 1) Sound Design Concept, 2) Objective, and 3) Method.
Authors should always explain which discipline is involved in which aspect.
The three aspects and each involved discipline should be mentioned already in
the title and abstract of a manuscript.

###  3.2 Identity Criteria

To solve the problem of strong interdisciplinary, the literature suggests that
disciplines organize themselves through identity criteria. In the field of
sonification, the above-mentioned solution already partially solves this
issue: Authors explicitly associate each involved discipline with either of
the three aspects of sonification, namely the sound design concept, the
objective and/or the method. This assignment is necessary to reveal, whether
the sound design concept is musical [7, p. 79], – such as using musical
scales, reverse scoring, step-sequencer, voice leading or counterpoint – or
whether the objective is musical, i.e., an enjoyable listening experience
[21]. In addition, as discussed above, sonification needs to be a) replicable
and b) interpretable, which is why we need a physical and a perceptual
description of each sonification. In their parameter mapping sonification
review [36], Dubus and Bresin already expressed the need for a “structure for
the classification of both physical and auditory dimensions‘”. In other words:

Recommendation: Authors should always reflect on their sonification’s a)
reproducibility using technical/physical terms and b) interpretability using
perceptual terms. These different viewpoints should never be mixed up.

###  3.3 Lexical Study

According to the literature on interdisciplinary research, the problem of
communication discrepancies can be solved through lexical study. This is
important, because the same term can have different meanings or connotations
in different disciplines. Several sonification studies already aimed at
finding a consistent taxonomy, e.g., concerning design [59, 60], listening
modes [61], interaction [62] and auditory display [3, 13]. other sonification
studies have already expressed the need for a consistent sound taxonomy,
too[4, 23].

As discussed above, the sound design concept of a sonification can originate,
e.g., in musical, psychoacoustical, or Gestalt-psychological considerations
and techniques. Consequently, it seems nonsensical to establish a standardized
sonification terminology, as it would, e.g., force authors to use a non-
musical language to express their musical sonification sound design concept.
The same is true for the objective and the method of the sonification study.
However, the situation is different concerning reproducibility and
interpretability of sonification. The first clearly requires a technical
terminology, e.g., from the field of physics/audio engineering [36], while the
second requires perceptual terms. Here, psychoacoustic terminology has been
suggested [13, 63]. In their review paper about parameter mapping sonification
[36], Dubus and Bresin suggest five perceptual high-level categories
_Loudness-related_ , _Pitch-related_ , _Spatial_ , _Temporal_ and _Timbral_
that can serve to describe sonifications. Following this categorization, I
carried out a comprehensive lexical study of sound descriptors in the field of
audio engineering and psychoacoustics. These can be found in the appendix.

Recommendation: Authors should describe sonifications using audio engineering
terminology for the sake of a) reproducibility and using psychoacoustic
terminology for the sake of b) interpretability. A comprehensive list of
terms, definitions and further readings can be found in the appendix.

###  3.4 Name Disciplines, whose terminology you use

All the above-mentioned solutions underlined the need to name involved
disciplines, ideally already in the title and abstract, and assign them to the
sound design concept, the objective, or the methodology. I also identified
that the requirement of sonifications to be reproducible and interpretable can
be covered by the disciplines audio engineering and psychoacoustics, and their
respective terminology. However, these are just the most essential disciplines
involved.

Whenever you use the terminology from other disciplines, such as music(ology),
computer science, cognitive sciences, or alike, say so, before you use them.
And state why you use them. In his book _Sonification Design_ , David Worrall
describes that the inclusion of algorithmic music and data-driven compositions
in sonification is unfortunate, “because it blurs purposeful distinctions” [9,
Sect. 1.9]. It is crucial that authors clearly make those distinctions.

Recommendation: Explicitly name disciplines whose terminology you use, and
whether you use it to describe the sound design concept, to ensure
reproducibility of sonification, to describe interpretation/perception of the
sonification, to reveal the objective, or the method that you use.

The proposed solution has been applied in some sonification studies. For
example, [7] writes: “(…) a primary physical characteristic of a tone is its
fundamental frequency [usually measured in cycles per second or Hz]. The
perceptual dimension that corresponds principally to the physical dimension of
frequency is ’pitch’, or the apparent ’highness’ or ’lowness’ of a tone.” [7,
p. 64]. This statement makes the distinction between two disciplines clear,
based on their viewpoints and terminologies.

##  4 DISCUSSION

Naturally, the vocabulary of a single discipline is limited. Sometimes,
authors need to include the terminology from a second discipline, e.g., to
ensure a) reproducibility and b) interpretability of a sonification.
Theoretically, a sonification does not require signal processing. A famous
example is the sonificaton of a ball’s velocity when rolling down a ramp [64,
p. 267]: When placing gut frets or light bells equidistantly above the ramp,
such that the ball strikes them when passing, you can hear the ball speed up,
because the frequency of bell strikes rises, i.e., the inter-onset interval
reduces. Here, the audio engineering terminology needs to be expanded by
physical terms to assure reproducibility of the setup. Similarly, the
psychoacoustic terminology is not capable of expressing all aspects of sound
perception. Here, terms from Gestalt psychology (auditory scene analysis) or
soundscape studies may be borrowed. However, this should always be mentioned
explicitly. Note that the two problems of sonification research tackled in
this study are only the tip of the iceberg. As discussed above, the
interdisciplinarity in sonification research goes far beyond sound
terminology. However, especially the division of sonification into the three
aspects of sound design concept, objective, and methodology should help
authors structure and illuminate their work in such way that readers from
various disciplines should be able to understand it.

##  5 CONCLUSION

In this paper, we have identified two problems in sonification research
related to the use of sound terminology, namely 1.) the lack of a common sound
terminology, and 2.) the blurred separation between disciplines’ methods and
goals. Through consultation of literature on interdisciplinary research, I
could relate these two sonification-specific problems to four general problems
of interdisciplinary research. Then, I applied the proposed solutions to
sonification research, resulting in recommendations concerning sound
terminology and the separation of involved disciplines: Authors should not
only name all disciplines involved in their study, but also assign them to the
aspect 1.) Sound Design Concept, 2.) Objective, and/or 3.) Method. Regarding
the requirement of a sonification to be a) reproducible and b) interpretable,
authors should describe both viewpoints of their sonification, ideally using
audio engineering and psychoacoustic terminology, respectively. They should
consistently stick to the terminology of the respective discipline for the
respective aspect. The lexical study provided in the appendix helps
sonification researchers understand the sound vocabulary of audio engineering
and psychoacoustics, avoid common misconceptions, and find further reading.

## References

  * [1] Gregory Kramer, Bruce Walker, Terri Bonebright, Perry Cook, John Flowers, Nadine Miner, and John Neuhoff.  Sonification report: Status of the field and research agenda.  International Community for Auditory Display, 1998.
  * [2] Andy Hunt and Thomas Hermann.  Interactive sonification.  In John G. Neuhoff Thomas Hermann, Andy Hunt, editor, The Sonification Handbook, chapter 11, pages 273–298. COST and Logos, Berlin, 2011\.
  * [3] Thomas Hermann.  Taxonomy and definitions for sonification and auditory display.  In 14th International Conference on Auditory Display, page 8 pages, Paris, Jun 2008.
  * [4] Thomas Hermann, Andy Hunt, and John G. Neuhoff.  Introduction.  In Thomas Hermann, Andy Hunt, and John G. Neuhoff, editors, The Sonification Handbook, chapter 1, pages 1–6. COST and Logos, Berlin, 2011.
  * [5] Thomas Hermann, Andy Hunter, and John G. Neuhoff, editors.  The Sonification Handbook.  COST and Logos, Berlin, 2011.
  * [6] Simon Carlile.  Psychoacoustics.  In Thomas Hermann, Andy Hunt, and John G. Neuhoff, editors, The Sonification Handbook, chapter 3, pages 41–61. COST and Logos, Berlin, 2011\.
  * [7] John G. Neuhoff.  Perception, cognition and action in auditory displays.  In Thomas Hermann, Andy Hunt, and John G. Neuhoff, editors, The Sonification Handbook, chapter 4, pages 63–85. COST and Logos, Berlin, 2011\.
  * [8] Stephen Barrass and Paul Vickers.  Sonification design and aesthetics.  In Thomas Hermann, Andy Hunt, and John G. Neuhoff, editors, The Sonification Handbook, pages 145–171. COST and Logos, Berlin, 2011.
  * [9] David Worrall.  Sonification Design.  Springer, Cham, 2019.
  * [10] A. de Campo, C. Frauenberger, K. Vogt, A. Wallisch, and C. Daye.  Sonification as an interdisciplinary working process.  In 12th International Conference on Auditory Display (ICAD), London, UK, May 2006.
  * [11] Christian Dayé and Alberto de Campo.  Sounds sequential: sonification in the social sciences.  Interdisciplinary Science Reviews, 31(4):349–364, Dec 2013.
  * [12] John G. Neuhoff.  Is sonification doomed to fail?  In Proc. 25th International Conference on Auditory Display, pages 327–330, Newcastle upon Tyne, Jun 2019.
  * [13] Bruce N. Walker and Michael A. Nees.  Theory of sonification.  In Thomas Hermann, Andy Hunt, and John G. Neuhoff, editors, The Sonification Handbook, chapter 2, pages 9–39. COST and Logos, Berlin, 2011.
  * [14] Christian Dayé.  Crafting sonifications: levels of interdisciplinary exchange in group discussions and their empirical assessment.  In 21st International Conference on Auditory Display (ICAD2015), pages 54–57, Graz, Austria, June 2015.
  * [15] Carlos Adrian Cuevas-Garcia.  Understanding interdisciplinarity in its argumentative context: thought and rhetoric in the perception of academic practices.  Interdisciplinary Science Reviews, 43(1):54–73, Jan 2018.
  * [16] C. F. Gethmann, M. Carrier, G. Hanekamp, M. Kaiser, G. Kamp, S. Lingner, M. Quante, and F. Thiele.  Interdisciplinary Research and Trans-disciplinary Validity Claims.  Springer, Cham, 2015.
  * [17] K.L. Daniel, M. McConnell, A. Schuchardt, and M.E. Peffer.  Challenges facing interdisciplinary researchers: Findings from a professional development workshop.  PLoS One, 17(4):e0267234, April 2022.
  * [18] Zhongyi Wang, Siyuan Peng, Jiangping Chen, Amoni G. Kapasule, and Haihua Chen.  Detecting interdisciplinary semantic drift for knowledge organization based on normal cloud model.  Journal of King Saud University — Computer and Information Sciences, 35(6):101569, 2023.
  * [19] Milad Khosravi Mardakheh and Scott Wilson.  A Strata-Based Approach to Discussing Artistic Data Sonification.  Leonardo, 55(5):516–520, Oct 2022.
  * [20] Thomas Metcalf.  Graphical Data Sets as Compositional Structure: Sonification of Color Graphs in RGB for Clarinet and Piano.  Leonardo, 54(3):329–336, Jun 2021.
  * [21] Carla Scaletti.  Sonification $\neq$ music.  In Roger T. Dean and Alex McLean, editors, The Oxford Handbook of Algorithmic Music. Oxford University Press, New York, NY, 2018.
  * [22] Thomas Hermann, Gerold Baier, Ulrich Stephani, and Helge Ritter.  Kernel Regression Mapping for Vocal EEG Sonification.  In Proceedings of the 14th International Conference on Auditory Display (ICAD2008), Paris, France, June 2008.
  * [23] Stefano Delle Monache, Nicolas Misdariis, and Elif Özcan.  Semantic models of sound-driven design: Designing with listening in mind.  Design Studies, 83:101134, Nov 2022.
  * [24] Tim Ziemer and Holger Schultheis.  PAMPAS: A PsychoAcoustical Method for the Perceptual Analysis of multidimensional Sonification.  Frontiers in Neuroscience, 16, Oct 2022.
  * [25] Sam Ferguson, Densil Cabrera, Kirsty Beilharz, and Hong-Jun Song.  Using psychoacoustical models for information sonification.  In 12th International Conference on Auditory Display, pages 113–120, London, Jun 2006.
  * [26] Tim Ziemer and David Black.  Psychoacoustically motivated sonification for surgeons.  International Journal of Computer Assisted Radiology and Surgery, 12((Suppl 1):1):265–266, Jun 2017.
  * [27] Tim Ziemer, Holger Schultheis, David Black, and Ron Kikinis.  Psychoacoustical interactive sonification for short range navigation.  Acta Acustica united with Acustica, 104(6):1075–1093, Nov/Dec 2018\.
  * [28] Tim Ziemer and Holger Schultheis.  Psychoacoustical signal processing for three-dimensional sonification.  In 25th International Conference on Auditory Displays (ICAD2019), pages 277–284, Newcastle, June 2019.
  * [29] Pietro Polotti and Guillaume Lemaitre.  Rhetorical strategies for sound design and auditory display: A case study.  International Journal of Design, 7(2):67–82, Aug 2013.
  * [30] Eduardo Reck Miranda.  The music of particle collisions.  Nature Phys, 12:721, Aug 2016.
  * [31] Chris Hayward.  Listening to the earth sing.  In Gregory Kramer, editor, Auditory Display: Sonification, Audification, and Auditory Interfaces, pages 369–404. Addison-Wesley, Boston, MA, 1994.
  * [32] Katharina Vogt and Robert Höldrich.  Translating sonifications.  J. Audio Eng. Soc, 60(11):926–935, Nov 2012.
  * [33] Kelly Hinckfuss, Penelope Sanderson, Robert G. Loeb, Helen G. Liley, and David Liu.  Novel pulse oximetry sonifications for neonatal oxygen saturation monitoring: A laboratory study.  Human Factors, 58(2):344–359, Mar 2016.
  * [34] Chihab Nadri, Sangjin Ko, Colin Diggs, Michael Winters, V. K. Sreehari, and Myounghoon Jeon.  Novel auditory displays in highly automated vehicles: Sonification improves driver situation awareness, perceived workload, and overall experience.  Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 65(1):586–590, Sep 2021.
  * [35] Tim Ziemer.  Psychoacoustic Music Sound Field Synthesis.  Springer, Cham, 2020.
  * [36] Gaël Dubus and Roberto Bresin.  A systematic review of mapping strategies for the sonification of physical quantities.  PLOS ONE, 9(4):e96018, Dec 2013.
  * [37] Karin Bijsterveld.  Sonic Skills. Listening for Knowledge in Science, Medicine and Engineering.  Palgrave Macmillan, 2019.
  * [38] Glenn D. White and Gary J. Louie.  The Auditory Dictionary.  University of Washington Press, Seattle, WA, 3rd edition, 2005.
  * [39] John Chowning and David Bristow.  FM Theory & Applications. By Musicians for Musicians.  Yamaha Music Foundation, Tokyo, 1986.
  * [40] Albrecht Schneider.  Pitch and pitch perception.  In Rolf Bader, editor, Springer Handbook of Systematic Musicology, pages 605–685. Springer Berlin Heidelberg, Berlin, Heidelberg, 2018\.
  * [41] Gregory Leplatre and Stephen A Brewster.  An investigation of using music to provide navigation cues.  In 5th International Conference on Auditory Display (ICAD1998), Glasgow, UK, November 1998.
  * [42] Sebastian Schwarz and Tim Ziemer.  A psychoacoustic sound design for pulse oximetry.  In Proceedings of the 25th International Conference on Auditory Display (ICAD 2019), Newcastle, June 2019.
  * [43] Christopher Frauenberger, Tony Stockman, and Marie-Luce Bourguet.  A survey on common practice in designing audio in the user interface.  In Proceedings of the 21st British HCI Group Annual Conference on People and Computers: HCI…but Not as We Know It, BCS-HCI ’07, page 187–194, Swindon, GBR, Sep 2007. BCS Learning & Development Ltd.
  * [44] Florian Grond, Till Bovermann, and Thomas Hermann.  A Supercollider Class for Vowel Synthesis and its use for Sonification.  In Proceedings of the 17th International Conference on Auditory Display (ICAD2011), Budapest, Hungary, June 2011.
  * [45] Stephen Barrass and Virginia Best.  Stream-based sonification diagrams.  In 14th International Conference on Auditory Display (ICAD2008), Paris, Jun 2008.
  * [46] D. Arfib, J.M. Couturier, Kessous L., and V. Verfaille.  Strategies of mapping between gesture data and synthesis model parameters using perceptual spaces.  Journal of Organised Sound, 7(2):127–144, Aug 2002.
  * [47] Dennis Reinsch and Thomas Hermann.  sonecules: a python sonification architecture.  In 28th International Conference on Auditory Display (ICAD2023), pages 62–69, Norrköping, Sweden, Jun 2023.
  * [48] Norberto Degara, Frederik Nagel, and Thomas Hermann.  Sonex: An evaluation exchange framework for reproducible sonification.  In Proceedings of the 19th International Conference on Auditory Display (ICAD2013), Lodz, Jul 2013.
  * [49] Janet E. Anderson and Penelope Sanderson.  Sonification design for complex work domains: Dimensions and distractors.  Journal of Experimental Psychology: Applied, 15(3):183–198, Mar 2009.
  * [50] James P. Bliss and Randall D. Spain.  Sonification and reliability — implications for signal design.  In International Conference on Auditory Display, pages 154–159, Montréal, Jun 2007.
  * [51] Jose Gonzales and Wenwei Yu.  Auditory display as an aid for prosthetic hand manipulation: Preliminary results.  In 17th International Conference on Auditory Display (ICAD2011), Budapest, Jun 2011.
  * [52] F. Kilander and P. Loennqvist.  A whisper in the woods - an ambient soundscape for peripheral awareness of remote processes.  In 8th International Conference on Auditory Display (ICAD2002), Kyoto, Japan, July 2002.
  * [53] Stefano Kalonaris.  Reef elegy: An auditory displayof hawaii’‘s 2019 coral bleaching data.  In 28th International Conference on Auditory Disply (ICAD2023), pages 70–77, Norrköping, June 2023.
  * [54] Tim Ziemer and Holger Schultheis.  Psychoacoustic auditory display for navigation: an auditory assistance system for spatial orientation tasks.  J Multimodal User Interfaces, 13(3):205–218, Sep 2019.
  * [55] Brianna J. Tomlinson, Brittany E. Noah, and Bruce N. Walker.  BUZZ: An auditory interface user experience scale.  In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, CHI EA ’18, pages 1–6, New York, NY, USA, Apr 2018\. Association for Computing Machinery.
  * [56] Tim Ziemer and Holger Schultheis.  The CURAT sonification game: Gamification for remote sonification evaluation.  In 26th International Conference on Auditory Display (ICAD2021), pages 233–240, Virtual conference, June 2021.
  * [57] Zeljko Obrenovic, Dusan Starcevic, and Emil Jovanov.  Experimental evaluation of multimodal human computer interface for tactical audio applications.  In ICME, volume 2, pages 29–32, Ischia, Jul 2002.
  * [58] Terri L. Bonebright and John H. Flowers.  Evaluation of auditory display.  In Thomas Hermann, Andy Hunt, and John G. Neuhoff, editors, The Sonification Handbook, pages 111–144. COST and Logos, Berlin, 2011.
  * [59] Keith V. Nesbitt.  Comparing and reusing visualisation and sonification designs using the ms-taxonomy.  In 10th International Conference on Auditory Display (ICAD2004), June 2004.
  * [60] Kajetan Enge, Alexander Rind, Michael Iber, Robert Höldrich, and Wolfgang Aigner.  Towards a unified terminology for sonification and visualization.  Personal and Ubiquitous Computing, 27(5), Oct 2023.
  * [61] Paul Vickers.  Ways of listening and modes of being: Electroacoustic auditory display.  Journal of Sonic Studies, 2, 2012.
  * [62] Visda Goudarzi.  Exploring a taxonomy of interaction in interactive sonification systems.  In Tareq Ahram, Redha Taiar, Karine Langlois, and Arnaud Choplin, editors, Human Interaction, Emerging Technologies and Future Applications III, pages 140–145, Cham, Aug 2021. Springer.
  * [63] Bruce N. Walker and Gregory Kramer.  Ecological psychoacoustics and auditory displays: Hearing, grouping, and meaning making.  In John G. Neuhoff, editor, Ecological Psychoacoustics, chapter 6, pages 149–174. Elsevier, Amsterdam, 2004.
  * [64] Till Bovermann, Julian Rohrhuber, and Alberto de Campo.  Laboratory methods for experimental sonification.  In Thomas Hermann, Andy Hunt, and John G. Neuhoff, editors, The Sonification Handbook, chapter 10, pages 237–272. COST and Logos, Berlin, 2011\.
  * [65] William M. Hartmann.  Principles of Musical Acoustics.  Springer, 2013.
  * [66] Juan G. Roederer.  The Physics and Psychophysics of Music. An Introduction.  Springer, New York, NY, 4 edition, 2009.
  * [67] Jerry H. Ginsberg.  Acoustics-A Textbook for Engineers and Physicists. Volume I: Fundamentals.  Springer, Cham, 2018.
  * [68] Jerry H. Ginsberg.  Acoustics-A Textbook for Engineers and Physicists. Volume I: Applications.  Springer, Cham, corrected edition, 2021.
  * [69] Acoustical Society of America and Institute of Radio Engineers.  Acoustical Terminology.  American Standard Association, New York, NY, 1951.
  * [70] J.P. Lefebvre.  Physical basis of acoustics.  In Paul Filippi, Dominique Habault, Jean-Pierre Lefebvre, and Aimé Bergassoli, editors, Acoustics: Basic Physics, Theory and Methods. Academic Press, London, 1999.
  * [71] Tim Ziemer.  Source width in music production. methods in stereo, ambisonics, and wave field synthesis.  In Albrecht Schneider, editor, Studies in Musical Acoustics and Psychoacoustics, pages 299–340. Springer, Cham, 2017.
  * [72] Robert Mores.  Music studio technology.  In Rolf Bader, editor, Springer Handbook of Systematic Musicology, pages 221–258. Springer, 2018.
  * [73] Alexandros Drymonitis.  Digital Electronics for Musicians.  Apress, Berkeley, CA, 2015.
  * [74] Eric J. Heller.  An Experiential Approach to Sound, Music, and Psychoacoustics.  Princeton University Press, Princeton, NJ, 2013.
  * [75] Michel Bruneau.  Fundamentals of Acoustics.  ISTE, London, 2006.
  * [76] Mitch Gallagher.  The Music Tech Dictionary: A Glossary of Audio-Related Terms and Technologies.  Course Technology PTR, Boston, MA, 2009.
  * [77] Pierre Deymier and Keith Runge.  Sound Topology, Duality, Coherence and Wave-Mixing.  Springer, Cham, 2017.
  * [78] Eberhard Sengpiel.  Forum für Mikrofonaufnahmetechnik und Tonstudiotechnik, chapter What is Amplitude?  sengpielaudio, Berlin, 2014.
  * [79] Albrecht Schneider.  Fundamentals.  In Rolf Bader, editor, Springer Handbook of Systematic Musicology, pages 559–603. Springer, Berlin, Heidelberg, 2018.
  * [80] Albrecht Schneider.  Perception of timbre and sound color.  In Rolf Bader, editor, Springer Handbook of Systematic Musiwology, chapter 32, pages 687–726. Springer, Berlin, Heidelberg, 2018.
  * [81] Barry Truax, editor.  Handbook for Acoustic Ecology.  Cambridge Street Publishing, Burnaby, BC, 2nd edition, 1999.
  * [82] Justin London.  Hearing in time: Psychological aspects of musical meter.  Oxford University Press, New York, NY, 2004.
  * [83] Philippe Guillaume.  Music and Acoustics.  ISTE, London, 2006.
  * [84] Hugo Fastl and Eberhard Zwicker.  Psychoacoustics. Facts and Models.  Springer, Berlin, Heidelberg, 3 edition, 2007.
  * [85] Stanley A. Gelfand.  Hearing. An introduction to psychological and physiological acoustics.  Informa, New York and Basel, second edition, 1990.
  * [86] Jens Blauert.  Spatial Hearing. The Psychophysics of Human Sound Source Localization.  MIT Press, Cambridge , MA, revised edition, 1997.
  * [87] W. Aures.  Berechnungsverfahren für den sensorischen wohlklang beliebiger schallsignale (a model for calculating the sensory euphony of various sounds).  Acustica, 59(2):130–141, Dec 1985.

## APPENDIX

What follows is a comprehensive list of sound terms from the audio engineering
domain, and the perception/psychoacoustics domain, subdivided into the six
high-level categories _Loudness-Related_ , _Pitch-Related_ , _Spatial_ ,
_Temporal_ , _Timbral_ and _Other_.

###  5.1 Audio Engineering Terms

Several textbooks and chapters provide a comprehensive overview about
acoustics, audio engineering, and signal processing, like [38, 39, 65, 66, 67,
68, 69, 70, 71, 72, 73, 74].

####  5.1.1 [Audio Engineering] Loudness-Related

Amplitude is often referred to as the strength of a signal regardless of its
frequency, and is measured in decibels of sound pressure level [67, sect.
1.1.1][39, p. 30][38, Sect. “Amplitude”], but it can also refer to particle
elongation/displacement, velocity or acceleration, and to voltage rather than
power [38, Sect. “Amplitude”] [75, sect. 1.1.5.].
Amplitude modulation [38, Sect. “Amplitude Modulation, abbr. AM”].
Attenuation is the amplitude reduction, often measured in decibels of sound
pressure level [38, Sect. “Attenuation”]. An
Attenuator aka. _Pad_ or _Loss Pad_ is a device to reduce the signal amplitude
[38, Sects. “Attenuator”, “Pad”][76, p. 149] .
Beats are periodic envelope fluctuations resulting from superposition of waves
with similar frequency, which can be produced, e.g., through additive
synthesis or amplitude modulation[69, p. 9].
Complex amplitude is a complex number to modify signal amplitude and phase
[77, sect. 2.2] [75, sect. 1.1.5.].
Damping is the addition of friction to a resonant system, reducing the
magnitude of the resonance. Resistance is the electrical analogue to damping;
not to be confused with dampening, which means adding water. [38, Sects.
“Damp, Damping” and “Dampen”].
Decibel (dB) is a unitless, logarithmic measure of power, like sound
intensity, but it is also used to quantify signal strength in terms of squared
sound pressure, voltage, particle elongation, velocity (aka. velocity level),
acceleration, or alike [38, Sect. “Decibel, or dB”][69, p. 11][74, Sect.
2.10].
Dry refers to a signal with little reverberation [38, Sect. “Dry”]. When the
signal also contains little ambient noise, it is called _Dead_ [38, Sect.
“Dead”][76, p. 48]. [Displacement] or _Deflection/Elongation_ over time (not
amplitude over time) is plotted in audio files in time domain and is
proportional to voltage [78].
Dynamic range can refer to the ratio of the loudest to the softest part,
measured in dB, but also to the signal-to-noise ratio [38, Sect. “ Dynamic
Range”].
Fade is the gradual change of volume, like fading in or our [38, Sects.
“Fade”, “ Fade In” and “Fade Out”][76, p. 71].
Gain is the amount of amplification and can be considered a multiplication of
sound pressure by a real value [38, Sects. “Amplifier”, “Gain”, “Voltage
Gain”].
Inverse Distance Law describes that the sound pressure of a point source
decays with a factor $p\propto 1/r$, where $r$ is the distance from the
source. The _Inverse Square Law_ describes the same circumstance, but refers
to sound intensity $I\propto 1/r^{2}$ [79, p. 576][38, Sect. “Inverse Square
Law”][74, Sect. 2.2][76, p. 99].
Master is a gain control for all output channels at once [38, Sect. “Master”].
Mute means to silence a track or a channel [38, Sect. “Mute”].
Particle Displacement is proportional to voltage in audio signals[75, sect.
1.1.5.].
Normalization means increasing the elongation/deflection/voltage/sound
pressure such that the peak equals the largest possible value [38, Sect.
“Normalize”][76, p. 137].
Particle Velocity is the time derivative of particle displacement [75, sect.
1.1.5.].
Peak is the largest absolute value of an audio signal over a considered time
interval [38, Sect. “Peak”][76, p. 154].
Resonances are eigenfrequencies of a physical system in which it will vibrate
stronger and longer, when excited with a signal containing these frequencies
[38, Sect. “Resonance”].
Root Mean Square of sound pressure or voltage quantifies the average volume
[38, Sects. “RMS”, "Volt, Voltage”].
Sound energy aka. acoustic energy in W is the amount of kinetic and potential
energy [68, p. 3640], i.e., “variation of energy produced by the acoustic
perturbation” [70, sect. 1.2.5.].
Power, is the rate of doing work, measured in Watts [W] and equals voltage
times current [38, Sects. “Power”, “Watt”].
Sound intensity $I$ in $\mathrm{W}/\mathrm{m}^{2}$ [38, Sect. “Intensity”] is
the power per area carried by the wave. It is the product of sound pressure
and particle velocity [68, pp. 572f] and proportional to the squared sound
pressure [75, sect. 1.1.5.][70, sect. 1.2.5.][79, p. 568][69, p. 12].
Sound Power in Watts cannot be measured directly and tends to have a
complicated relationship to sound pressure [38, Sect. “Sound Power”].
Sound pressure $p$ is the “magnitude of the pressure variation” [75, p. 18].
Sound pressure level [67, sect. 1.3.1]. Weighted sound pressure levels in
$\text{dB}_{\text{A to C}}$ attenuate certain frequencies before calculating
the sound pressure level [67, sect. 1.3.2] [38, Sects. “dBA” and
“A-Weighting”].

####  5.1.2 [Audio Engineering] Pitch-Related

Angular Frequency $\omega=2\pi f$ is the circumference of the unit cycle times
the frequency. A
Cent is the $\sqrt[1200]{2}$ and divides the octave into $1200$ equal
frequency ratios [38, Sect. “Cents”][76, p. 31].
Complex Tone is a sound that contains discrete (not continuous) spectral
peaks[80] [69, p. 24].
Decade refers to a frequency ratio of $10:1$ and is sometimes used to describe
the rolloff of a filter, like $20$ dB per decade [38, Sect. “Decade”].
Frequency in Hertz (Hz) which replaced the earlier unit of cycles per second
(cps) is the number of exact repetitions per second [38, Sects. “Cycles per
Second, cps”, “Frequency” and “ Hertz, Hz”][76, pp. 78, 87].
Frequency Modulation means processing frequencies so that they vary
continuously over time [69, p. 10][76, p. 78].
Fundamental Frequency is the lowest frequency from a complex tone [67, p. 43]
[38, Sect. “ Fundamental”][69, p. 8][76, p. 80].
Period is the reciprocal of frequency and can refer to the duration until a
function repeats itself[67, sect. 1.1.1][69, p. 7].
Periodicity is the frequency of exact signal repetition, irrespective of the
fundamental frequency [67, p. 43].
Pitch Bend is a controller and/or MIDI message to alter the speeed of a sound
playback, simulating the musical playing technique of pitch bending, known.,
e.g., from guitars, violins and trombones [76, p. 158].
Pure tone aka. _Simple Tone_ has a sinusoidal waveform and contains only one
frequency [38, Sect. ‘Pure Tone”][69, p. 24].
Sweep aka. _Chirp_ is a frequency that continuously rises (or falls) as a
mostly exponential function of time [38, p. 382].
Wow is a slow (below $5$ Hz) frequency modulation caused by speed variation in
tape recorders [38, Sect. “Flutter”]

####  5.1.3 [Audio Engineering] Spatial

Amplitude-Based panning means gain weighting between several channels
according, e.g., to Chowning’s/sine/tangent panning law, Vector Base Amplitude
Panning (VBAP) or Multiple Direction Amplitude Panning (MDAP), or through
microphone techniques like Y-X, ORTF, Blumlein aka. intensity stereo [38,
Sects. “ Intensity Stereo”, “X-Y Stereo”][71]).
Depth refers to aims at manipulating the apparent distance of a sound source
in stereo, and thus, indirectly, its size [38, Sect. “Depth”].
Dichotic,
Diotic and _Monotic_ presentation refers to headphone signal being individual
for each ear, equal for each ear, and presented to one ear only [38, Sects.
“Dichotic”, “Diotic” and “Monotic”].
Sonic Environment refers to sounds at a certain place and time [81].
Phantom Source are desired location and extent of an auditory event in stereo
setups [76, p. 155].

####  5.1.4 [Audio Engineering] Temporal

Burst is a test signal that typically lasts some milliseconds, like a tone
burst and a noise burst [38, Sect. “Burst”].
Cycle includes all elements that repeat periodically [69, p. 7].
Echo is a distinct, damped repetition of a signal with a delay of $50$ ms or
more [38, Sect. “Echo”][69, p. 10].
Impulse is a short, broadband spike [76, p. 96].
Initial Phase ($\phi_{0}$) is the starting point of an oscillation function
$p(t)=\hat{A}\sin(\omega t+\phi_{0})$ [27].
Inter-Onset-Interval (IOI) is the duration between two successive note-,
noise-, impulse-, or sound-onsets [82, p. 4].
Period is the duration of a full cycle of a waveform [76, p. 154].
Phase can describe the argument of an oscillation function or the angle of a
single frequency’s complex amplitude [77, sect. 2.2][35, Sect. 5.1.4].
Transient is a nonrepeating waveform, including note onsets, offsets, decays
and modulation [38, Sect. “Transient”].

####  5.1.5 [Audio Engineering] Timbral

Bandwidth is the frequency region in which a spectrum contains significant
energy [67, sect. 1.5.1][38, Sect. “Bandwidth”].
Bass is often considered the low-frequency portion of the audible frequency
range up to about $200$ Hz[38, Sect. “Bass”].
Broadband or _Wideband_ refers to a wide distribution of spectral energy [38,
Sects. “Broadband”, “Wideband”].
Clipping means that the waveform looks like clipped by a pair of scissors,
producing harmonic distortion and adding to the perception of roughness, but
may also make speech easier to understand in noisy environments [38, Sect.
“Clipping”].
Corner Frequency is the frequency of a filter where the amplitude is reduced
by $3$ dB, like the _Cutoff-Frequency_ of a high-pass or low-pass filter [38,
Sects. “Corner Frequency” and “Cutoff Frequency”][76, pp. 41, 44].
Duty Cycle is the pulse width per period in pulse width modulation synthesis,
affecting the weighting of harmonics [73, p. 16][76, p. 59].
Extreme highs are frequencies above $10$ kHz, while
Extreme Lows are frequencies below $40$ Hz [38, Sect. “Distortion”].
Filters include _High_ , _Low_ , _Stopband/Band Reject/Band Elimination_ and
_Band Pass Filters_ that let frequencies above, below, outside and,
respectively, between the corner frequencies pass while attenuating the
others. _Notch Filters_ attenuate, _Resonance/Peak Filters_ amplify mostly a
narrow frequency region.
Comb Filters exhibit a series of deep notches. _Shelf Filters_ have a flat
response over one or two large frequency ranges and a gradual slope within the
transition range.[38, Sects. “Bandpass Filter”, “Band Reject Filter”,
“Brickwall Filter and Low-Pass Filter”, “Comb Filter”, “High-Pass Filter”,
“Notch Filter”, “Shelving”][69, p. 18][72][76, pp. 16, 35, 74, 88, 113f, 138f,
154].
Frequency bands, like octave bands and 1/3 octave bands group the frequency
spectrum into smaller portions [67, sect. 1.3.3.] [38, Sect. “One-third Octave
Filter”].
Harmonic Series are frequencies that are integer multiples of a fundamental
frequency[76, p. 86].
Highs are frequencies between $4,000$ and $10,000$ Hz [38, Sect.
“Distortion”][76, p. 88].
Lows are frequencies roughly between $40$ and $300$ Hz [38, Sect.
“Distortion”].
Mid-Bass means frequencies around $200$ to $300$ Hz [38, Sect. “Bass”].
Mid-range are frequencies between $300$ and $4,000$ Hz [38, Sect. “Midrange”].
Modulation Index is a ratio that expresses the amplitude ratio of carrier and
modulation signals in AM synthesis and the frequency deviation between carrier
and the highest or lowest sideband in FM synthesis [38, Sect. “Modulation
Index”].
Narrowband means that most spectral energy is concentrated in a narrow
frequency band, e.g. $400$ cent [38, Sect. “Narrowband”].
Noise can mean an unwanted sound not related to the desired signal, or a
stochastic, aperiodic process, like _Gray, White, Brown/Red, Pink, Blue,
Purple Noise_ [38, Sects. “Noise”, “One-over-f (1/f ) Noise”, “White
Noise”][67, sects. 1.5.1f].
Overdrive means that the gain drives the signal above the linear operating
level of equipment, causing overload distortion [38, Sects. “Operating level”,
“Overdrive” and “Overload”].
Passband is the frequency region not rejected by a filter [38, Sect.
“Passband”][76, pp. 151f].
Pulse Width Modulation (PWM) is a technique to produce complex tones [73,
Chap. 2].
Q, _Q-_ or _Quality-Factor_ is the steepness of a filter in frequency domain
[38, Sect. “Q”][72].
Slope (aka. skirt) is the steepness of a filter in frequency domain, expressed
in dB per decade or octave [38, Sect. “Slope”].
Sound Synthesis includes techniques such as _Additive/Fourier Synthesis_ ,
_Frequency Modulation (FM) Synthesis_ , _Granular Synthesis_ , _Ring
Modulation_ /_Amplitude Modulation Synthesis_ , _Physical Modeling_ ,
_Subtractive Synthesis_ and combinations [38, Sects. “Additive Synthesis”, “FM
Synthesis”, “Ring Modulator”, “Subtractive Synthesis”][83, Sect. 6.1][39][76,
pp. 4, 7, 76f, 82, 156f][65, chap. 9].
Spectral Centroid is the center of gravity of a magnitude spectrum [71, Sect.
4.3.1].
Spectrum or _Frequency Spectrum_ is the frequency representation of a time
signal, indicating amplitude and phase per frequency over a time frame [38,
Sect. “Spectrum”][69, p. 8].
Synthesizer is a hard- and/or software that electronically produces sound [38,
Sect. “Synthesizer”].
Treble are frequencies above about 2 kHz [38, Sect. “Treble”].
Waveform is the shape of an oscillation over time, like _Sine_ , _Triangle_ ,
_Sawtooth_ , _Square_ , _Pulse rain_ , _Complex Waves_ and _White_ (random)
_Noise_ [38, Sects. “Complex Wave”, “Sine Wave”, “Square Wave”, “Waveform”].
(Temporal) Envelope is the amplitude (not
displacement/deflection/elongation/voltage) over time. It can also refer to an
amplitude or gain function $A(t)$, like _Attack Decay Sustain Release (ADSR)_
curves of synthesizers [80] [38, Sect. “Additive Synthesis”][76, pp. 4,
66f][81][79, p. 595].

####  5.1.6 [Audio Engineering] Other

Audio Effects is signal processing to modify an audio signal, like _Chorus_ ,
_Compressor_ , _Delay_ , _Distortion_ , _Equalizer_ , _Filter_ , _Flanger_ ,
_Phaser_ , _Reverb_ _Wah Wah_ [38, Sects. “Chorus”, “Effects”, “Equalizer”,
“Flanging”, “ FX”][83, sect. 6.3.1. and p. 169][76, pp. 32, 37, 41, 50, 67,
156].
Carrier is a signal about to be modulated by another signal, the _modulator_ ,
e.g., in AM and FM synthesis [76, p. 29].
Low Frequency Oscillator (LFO) has a (fundamental) frequency way below $20$ Hz
and is used to modulate audio material [76, p. 110].
Musical Instrument Digital Interface (MIDI) is a protocol to transmit control
data between audio (and lighting) equipment [76, pp. 122ff].
Open Sound Control (OSC) is a network protocol for communication between
multimedia devices [76, p. 145][73, Chap. 8].

###  5.2 Psychoacoustic Terms

Several textbooks and chapters provide a comprehensive insight into
psychoaocustic terminology, methods and paradigms, like [35, 40, 66, 71, 74,
79, 80, 84, 85, 86].

####  5.2.1 [Psychoacoustics] Loudness-Related

Fluctuation Strength refers to the intensity of loudness fluctuation caused,
e.g., by amplitude or frequency modulation or beats. The unit of fluctuation
strength is vacil, but it is also often given in percent[84, chap. 10][38,
Sect. “Beats”].
Loudness is the perception that lets listeners order sounds from soft to loud,
i.e., the subjective strength of an audio signal that depends on amplitude and
frequency but even on spatial and temporal attributes.[69, p. 22][67, sect.
1.3.2][38, Sects. “Loudness” and “Sone”].
Masking means that the presence of one sound inhibits our ability to hear
another simultaneous (_Simultaneous Masking_), or shortly preceding (_Pre-
Masking_) or subsequent (_Post-Masking_) sound. Typically, loud sounds mask
soft sounds, low frequencies mask higher frequencies [38, Sect. “Masking”][35,
Sect. 4.3].
Phon is the unit of loudness level and is a frequency-compensated decibel
scale (aka. _Fletcher-Munson-Curve_ or _Equal-Ludness-Contour_) but not a unit
of loudness [65, p. 127][69, p. 23][76, pp. 76, 156].
Sone is a measure of loudness, where a $1$ kHz pure tone at $40$ dB equals $1$
Sone, while a sound twice as loud equals $2$ Sone, etc. [69, p. 22].
Specific Loudness is the loudness per critical frequency band. Their integral
is the overall loudness [84, Chap. 8].
Threshold of Hearing aka. _Absolute Threshold_ or _Minimum Audible Field_
(MAF) is the sound pressure level at the eardrum required to be just audible,
which is highly frequency dependent [38, Sects. “Minimum Audible Field, MAF”,
“Threshold of Hearing”].

####  5.2.2 [Psychoacoustics] Pitch-Related

Absolute Pitch aka. _Perfect Pitch_ refers to a listeners’ ability to assign
sounds their corresponding position or note on a musical scale [38, Sect.
“Absolute Pitch”].
Combination Tones aka. _Phantom Tones_ /_Tartini Tones_ are audible
frequencies produced through nonlinearities in our hearing system that are not
physically present in the sound, like _Combination Tones_ $f_{1}+f_{2}$,
_Difference Tones_ $f_{2}-f_{1}$ and _Cubic Difference Tones_ $2f_{1}-f_{2}$
[85, Chap. 12][76, pp. 52, 206f].
(In)Harmonicity refers to whether simultaneous frequencies seem to exhibit a
harmonic series or not, which can affect simultaneous stream segregation and,
therefore, pitch, consonance, timbre and tonal fusion, [40][35, Sect. 4.5.2].
Mel is the unit of pitch, where $1000$ Mel = pitch evoked by a $1$ kHz pure
tone at $40$ phon (or dB) [38, Sect. “Mel”][69, p. 22].
Missing Fundamental aka. _Residual Pitch_ or _Virtual Pitch_ means that the
perceived pitch of a sound equals the pitch of a pure tone whose frequency is
not present in the sound [84, Chap. 5][40][38, Sect. “Residue”].
Pitch is the perception that lets listeners order sounds from low to high
(height) and that includes octave identity/equivalence (chroma) [40][84, Chap.
5][69, p. 22][85, Chap. 12].
Pitch Strength means how distinct a sound’s pitch is. It is, e.g., low for
clicks, higher for inharmonic series, and even higher for pure tones in the
with midrange frequency and sufficient loudness [84, Chap. 5][40].
Shepard Illusion is pitch perception with clear chroma but ambiguous height
and can be created through Shepard tones [79, 40][35, Sect. 4.5.4.1].

####  5.2.3 [Psychoacoustics] Spatial

Ambience refers to acoustic qualities of a listening space, including echoes,
reverberation and background noise and may provide a certain auditory
‘atmosphere’ [38, Sect. “Ambience”].
Apparent Source Width aka. _Perceived Source Extent_ refers to the perceived
size of a sound source, sometimes categorical, sometimes indicated in degrees
[71][35, Chap. 6.2.2].
Auditory Event Angle/Distance/Location refers to the perceived source
angle/distance/location irrespective of the physical position [71] [35, Sect.
4.4][86].
Precedence Effect aka. _Haas Effect_ or _Law of the First Wavefront_ means
that the first wavefront arriving at the ears determines the perceived source
location, even in the presence of louder reflections and reverberation [38,
Sect. “Haas effect”][35, Sect. 4.4.5][76, p. 85].
Soundscape refers to how the sonic environment is perceived and understood
[81].

####  5.2.4 [Psychoacoustics] Temporal

Rhythm is a temporal pattern of one auditory stream and somewhat related to
fluctuation strength and subjective duration [84, Chap. 13].
Subjective Duration seems quite proportional to physical duration above $100$
ms. But temporal masking may affect subjective duration and, e.g., $50$ ms
sound longer than one half of $100$ ms[84, Chap. 12].
Temporal Masking contains pre-masking of up to $30$ ms, the _Overshoot_
phenomenon at masker onsets, simultaneous masking, a short sustain of $5$ ms
and post-masking up to $100$ ms [84, Sect. 4.3.1.5].

####  5.2.5 [Psychoacoustics] Timbral

Brightness (from warm/dull to bright/shrill) is considered the most prominent
feature in timbre evaluation and seems closely related to the spectral
centroid of a signal[35, p. 15][79, p. 595].
Clangorous is a continuous loud noise of something being hit or rung, often
metallic [40, p. 704].
Fullness aka. _Richness_ , sometimes _Volume_ is mostly affected by the
spectral distribution and bandwidth, and can be increased, e.g., by adding or
modulating frequencies [80, 28].
Noisiness indicated in Noy refers to perceived noise loudness, not to the
degree of tonalness [38, Sect. “Noy”].
Percussive to _Mellow_ is the degree of how impulsive the excitation of a
source sounds, where hammered/struck often sound more percussive than plucked,
and even more than blown/stringed[80, p. 701][79, p. 595].
Roughness in asper is perceived when two or more than $16$ but less than one
critical bandwidth apart [38, Sect. “Beats”][87][80].
Sharpness is equal or related to _Brightness_ , but has the measure acum that
can be approximated by psychoacoustic models [87][84, Chap. 9].
Sound Color aka. _Tone color_ refers to the timbre of quasi-stationary sounds,
excluding attack transients and decay [80][38, p. 393].
Timbre includes spectral and short-term temporal aspects of sound other than
pitch and loudness [80].
Tonality aka. _Tonalness_ ranges from tonal to noisy and is affected by
frequency, harmonicity, periodicity and bandwidth, and considered a parameter
of sensory euphony [35, p. 41][80, p. 700 and Sect. 32.2.3][87].

####  5.2.6 [Psychoacoustics] Other

Auditory refers to the psychological representation of sound [35, Chap. 4][79,
p. 586].
Auditory Stream is the auditory counterpart of a visual object, i.e., aspects
of sound that seem to belong together [35, Sect. 4.5][80, Sect. 32.2.5].
Critical Bandwidth is a region of $1.3$ mm on the basilar membrane in which
neurons collectively process sound. [38, Sect. “Critical Band”][79, p. 584].
Just Noticeable Difference aka. _Difference Limen_ is the increment in a
stimulus that is just noticed in a specified fraction of trials [38, Sect.
“Just Noticeable Difference, JND”][69, p. 23].
Perception is a psychological impression often resulting from sensation plus
cognitive processing in the brain [79, Sect. 30.1.2]
Sensation can be considered an inter-subjective ur-perception before the
subjective interference through context, experience, preference, etc.
Psychoacoustic models predict sensation and not perception [35, Chap. 4][84,
Sect. 1.4].
Sensory Euphony aka. _Sensory Pleasantness_ or _Psychoacoustical Annoyance_
tends to be considered as a nonlinear combination of _Loudness_ , _Roughness_
, _Sharpness_ and _Tonality_ [84, Chap. 9].
Sound Object is the smallest self-contained element of a soundscape [81].

###  5.3 Danger of Confusion

I identified terms with different meanings in different disciplines. The
following list shall help researchers avoid mistakable expressions:
A-B Stereo $\neq$ A-B testing $\neq$ ABX Test, the first refers to a spaced
microphone recording technique, the second to the direct comparison of two
versions of the same signal and the latter is an experiment in which
participants guess whether a test stimulus X was equal to reference stimulus A
or B[71][38, Sects. “A-B Stereo”, “A-B Testing” and “ABX Test”].
Acoustic $\neq$ Auditory, the first refers to physics, the second to
perception of sound [86].
Acoustic Recording $\neq$ Sound Recording, the first refers to recording
without the use of electricity, the second to all sound recording [38, Sect.
“Acoustic Recording”].
Amplitude Modulation $\neq$ Tremolo, the first refers to analogue or digital
signal processing, the latter to an articulation technique, e.g., of string
instruments [76, pp. 7, 8].
Audio Track $\neq$ _Audio Channel_. Each audio channel can contain a mixture
of signals from one or more audio tracks, which can be recorded and
manipulated separately, before routing them to discrete output channels. [38,
Sects. “Multitrack”, “Tracking”][76, pp. 31, 132].
Auditory Stream $\neq$ Audio Stream. An auditory stream is comparable to a
visual object, i.e., it includes sound aspects that are perceived as being
part of the same. An audio stream is the broadcast of one or multiple audio
channels. [38, Sect. “Streaming”].
Bandwidth in terms of the audio signal $\neq$ Bandwidth/Throughput/Data Rate
of a network connection or interface [76, p. 16].
Bar in physics $\neq$ Bar in music. In physics, Bar is a unit of pressure, and
a bar is a uniform elastic rod or beam with irregular eigenfrequencies. The
latter is a segment of music often having a uniform duration, number of beats
and structure of accentuation.
Beats in physics $\neq$ Beats in music. The first refer to envelope
fluctuation caused by superposition of similar frequencies, the latter to the
pulse, to the rhythm section of a piece of music, or to the instrumental,
e.g., of hip-hop music [76, p. 19].
Current $\neq$ Charge [38, Sects. “Ampere, abbr. Amp or A” and Charge]. While
the current is a property of a circuit, the charge flows. There is no such
thing as a current flow.
dB $\neq$ dB${}_{\text{SPL}}$$\neq$ dBA. The first is a power level with
arbitrary reference, the second uses a reference pressure of $20$
micropascals, just like the third one, which also adds a frequency-weighting
[38, Sects. “A-Weighting”][76, pp. 46, 49].
Depth $\neq$ Auditory Distance $\neq$ Source Distance, the first refers to
stereo methods to affect auditory distance, which is the perceived source
distance, while source distance refers to the physical distance of a source
[38, Sect. “Depth”][86].
Delay as an audio effect $\neq$ Delay of a network connection or interface.
The first is a single or a number of damped repetitions, the latter is latency
of signal transfer [76, pp. 50, 108].
Elongation/Deflection/Voltage $\neq$ Amplitude, the first refer to the course
of the time signal, the second to the peak of the absolute time signal [78].
Frequency/Fundamental Frequency $\neq$ Pitch, the first is physical, the
second is a perceptual quality affected strongly by the first, but also by
periodicity and sound pressure level [80].
Frequency Modulation $\neq$ Wow $\neq$ Vibrato. Frequency modulation describes
periodic, continuous frequency variation through signal processing, wow is a
frequency modulation resulting from tape speed alterations, and vibrato is a
musical instrument playing technique where frequencies and amplitudes of
complex tones vary continuously and regularly [38, Sect. “Wow”][69, pp. 10,
25].
Jitter in speech $\neq$ Jitter in an audio network connection or interface
[38, Sect. “Jitter”].
Loudness $\neq$ Volume $\neq$ Sound Pressure Level/Amplitude/Gain. The first
is a perceptual quality, the second is a colloquial term loosely related to
the others, the latter refer to physical quantities [80][38, Sects.
“Amplitude”, “Level”, “Voltage Gain”][67, sect. 1.1.1][69, p. 12][76, pp. 81,
110].
Meter in music $\neq$ Meter as a measuring device $\neq$ meter as the SI unit
of length [38, Sect. “Meter”].
Monitor in audio (aka _Studio Monitor_ or _Stage Monitor_ $\neq$ Monitor in
computer science (visual display) [38, Sects. “Monitor” and “Near-Field
Monitor”][76, pp. 127, 133].
Note $\neq$ Tone. The first is a sign indicating pitch and/or duration of a
tone or sound. The second is a sound having a pitch. [69, p. 25].
Overshoot in audio $\neq$ Overshoot in perception. The first means that
equipment like compressors or limiters responds imperfectly to short
transients, while the latter means a brief increase of the masking threshold
at sound onsets [38, Sect. “Overshoot”][35, Sect. 4.3.1.].
Polyphonic in audio engineering $\neq$ Polyphonic in music. The first means
the overlap of multiple voice, e.g., in synthesizers and music mixes. The
second is a horizontal thinking (like counterpoint), i.e., multiple voices
playing melodies that result in consonant intervals on pulse, except when the
intention is to create tension [66, Chap. 5][80, pp. 711f].
Overtone $\neq$ Harmonic $\neq$ Partial. The fundamental frequency is usually
the lowest frequency and the first partial, and it may be the first harmonic
of a spectrum. The first overtone is the second partial. It is also the second
harmonic, if the signal is a complex tone, i.e., if all partials are part of a
harmonic series. [38, Sects. “Overtones” and “Partials”][69, p. 25][76, pp.
86, 147].
Sharp in music $\neq$ Sharp in psychoacoustics $\neq$ _Sharp_ in audio
engineering. In music, sharp is a rise in pitch, e.g. by one half-step. In
psychoacoustics, it is a timbral quality, related to brightness. In audio
engineering, filters’ _Slope_ /_Rolloff_ /_Attenuation_ /_Rejection Rate_ /_Q-
Factor_ are also called sharpness [38, Sect. “Sharp”][87][85, p. 15].[80].
Sound Velocity $\neq$ Particle Velocity, the first is a particle’s velocity
when moving around its equilibrium position, the second is the propagation
speed of a sound wave (perturbation).
Spectral Centroid $\neq$ Brightness, the first is the center of gravity of a
magnitude spectrum, the latter an aspect of timbre perception [71, Sect.
4.3.1].
Stereo sometimes $\neq$ Stereo. While stereo tracks in digital audio
workstations often refer to two channels, one for each loudspeaker in a stereo
triangle, stereo can also refer to loudspeaker systems using more than two
loudspeakers. [38, Sect. “Stereophonic”][71, Chap. 7].
Tone in physics $\neq$ Tone in music. In physics, tone refers to a pure or
complex tone, while in musical terms it can also refer to tone color or even
timbre [38, Sect. “Tone”].
Voice in speech $\neq$ Voice in music. The first always refert to the human
voice, the second can refer to different instruments or registers [80, pp.
711f].
Timbre $\neq$ Sound Color/Tone Color. Timbre includes short-term temporal
aspects, like initial transients and decays, while tone color/sound color
mostly refers to the spectrum of the quasi-stationary part of sounds [80].