# Learning to Compare for Better Training and Evaluation of Open Domain
Natural Language Generation Models

Wangchunshu Zhou, Ke Xu
State Key Lab of Software Development Environment, Beihang University,
Beijing, China
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

> Automated evaluation of open domain natural language generation (NLG) models
> remains a challenge and widely used metrics such as BLEU and Perplexity can
> be misleading in some cases. In our paper, we propose to evaluate natural
> language generation models by learning to compare a pair of generated
> sentences by fine-tuning BERT, which has been shown to have good natural
> language understanding ability. We also propose to evaluate the model-level
> quality of NLG models with sample-level comparison results with skill rating
> system. While able to be trained in a fully self-supervised fashion, our
> model can be further fine-tuned with a little amount of human preference
> annotation to better imitate human judgment. In addition to evaluating
> trained models, we propose to apply our model as a performance indicator
> during training for better hyperparameter tuning and early-stopping. We
> evaluate our approach on both story generation and chit-chat dialogue
> response generation. Experimental results show that our model correlates
> better with human preference compared with previous automated evaluation
> approaches. Training with the proposed metric yields better performance in
> human evaluation, which further demonstrates the effectiveness of the
> proposed model.

##  1 Introduction

Recent advances in sequence-to-sequence learning architecture (?) and the
transformer model (?) have raised increasing interest in natural language
generation (NLG) tasks, including story generation (?), open-domain dialogue
response generation (?) and abstractive summarization (?). Despite the fast
advances of models, there remains a huge gap in the evaluation of NLG models
and it is hard to measure the progress due to the lack of good evaluation
metrics. While perplexity is a good measure of how well a model fits some
data, it does not measure performance at the desired task. Word overlap based
metrics such as BLEU (?), METEOR (?) and ROUGE (?) capture quality better than
the perplexity and are useful in translation and summarization. However, they
still correlate poorly with human evaluation (?) in open domain text
generation tasks including story generation and dialogue response generation
because two equally good generated texts may have no n-gram overlap. Human
evaluation is generally considered to be the gold standard evaluation,
however, it does not scale well as it is generally expensive and time-
consuming to conduct human evaluation.

Apart from measuring relative progress between different models, automated
evaluation metrics also play an important role in the training stage of NLG
models. It is a common practice to tune the model hyperparameter, detect
convergence, perform early-stopping, and select the best checkpoints based on
the model’s performance on automated evaluation metrics. While acceptable for
tasks where automated metrics correlate well with human evaluations, including
machine translation and text summarization, this can be erroneous and result
in sub-optimal training in open domain NLG tasks because available automated
metrics correlate poorly with human evaluation, as demonstrated in the
experimental section of this paper.

To tackle the aforementioned problems, in this paper, we propose a self-
supervised approach with transfer learning to learn to compare the quality of
two samples as an automated comparative Turing test. The motivation of our
approach is that we can better assess the quality of generated samples or
trained NLG model by comparing it with another one. Our model is a text pair
classification model trained to compare the task-specific quality of two
samples, which is then used to evaluate the quality of trained NLG models. As
human preference annotation is generally expensive, our model is designed to
be able to perform self-supervised training using only generated samples and
gold reference samples without human preference annotation. When human
preference annotation is available, our model can be further fine-tuned to
better imitate human judgment. To evaluate the model-level quality of NLG
models based on pairwise comparison in sample-level, we adopt the skill rating
system similar to ELO (?) and Trueskill (?), which is a method for assigning a
numerical skill to players in a player-vs-player game, given a win-loss record
of games played. In our scenario, the players are NLG models to be evaluated
and a higher rating indicates a better model. The skill rating system makes it
possible to evaluate all n models without needing to run $n^{2}$ matches and
is able to take into account the amount of new information each comparison
provides.

The contribution of this paper is threefold:

  * •

We propose a “learning to compare” model to better assess the quality of text
generated by NLG models based on pairwise comparison. Our model is able to
transfer natural language understanding knowledge from BERT by fine-tuning in
a self-supervised way while also able to be further fine-tuned with human
preference annotation. Once trained, our model is able to perform inter-model
comparison without the need for gold references, which greatly enlarges the
potentially available test set and reduces the potential risk of overfitting
the reference in the test set.

  * •

We propose to use the skill rating system to perform model-level evaluation
based on the sample-level evaluation information provided by our pairwise
comparison model. The skill rating system is more efficient and accurate than
several baseline approaches.

  * •

We conduct experiments on both story generation task and open domain dialogue
response generation task. Experimental results show that our approach
correlates better with human evaluation on both datasets. Moreover, we show
that using automated metrics such as BLEU to perform hyperparameter tuning and
early-stopping results in sub-optimal model and our approach helps alleviate
this problem.

##  2 Related Work

Evaluation of NLG models has been a long-standing open problem. While human
evaluation may be ideal, it is generally expensive to conduct and does not
scale well. Various automated evaluation approaches are proposed to facilitate
the development and evaluation of NLG models. We summarize these evaluation
approaches below.

Text Overlap Metrics, including BLEU (?), METEOR (?) and ROUGE (?), are the
most popular metrics employed in the evaluation of NLG models. They evaluate
generated text by comparing the similarity between the generated text and
human written references. While this works well in tasks where the diversity
of acceptable output is limited, such as machine translation and text
summarization, text overlap metrics are shown to have weak or no correlation
with human judgments in open domain natural language generation tasks (?).
There are two major drawbacks in these metrics. First, text overlap metrics
can not distinguish minor variations in a generated text which may make the
sentence not equally grammatically correct or semantically meaningful. Second,
there may exist multiple equally good outputs for the given input and
comparing against one gold reference can be erroneous.

Perplexity is commonly used to evaluate the quality of a language model. It
measures how well a probability distribution predicts a sample and captures
the degree of uncertainty in the model. It is used to evaluate models in open-
domain NLG tasks such as story generation (?) and open domain dialogue
systems. However, “how likely a sentence is generated by a given model” may
not be comparable across different models and does not indicate the quality of
the sentence.

Parameterized Metrics learn a parameterized model to evaluate generated text.
Adversarial evaluation models (?; ?) assigns a score based on how easy it is
to distinguish the dialogue model responses from human responses. However,
training such a discriminator can be difficult as the binary classification
task can be easily over-fitted and leads to poor generalizability (?).
Moreover, the information we get from the discriminator accuracy is limited as
we can not compare the quality of two generated sentences when they both
succeed or fail in fooling the discriminator. Recent study shows that the
discriminator accuracy does not correlate well with human preference (?).
Automated Dialogue Evaluation Model (ADEM) (?) is another parameterized metric
proposed for dialogue system evaluation. It learns to score a generated
dialogue response based on the context and the human written reference.
However, it requires human-annotated scores for generated sentences. It is
generally hard to design appropriate questions for crowdsourcing these scores,
which makes the annotation very expensive to get and the inter-annotator
agreement score is only moderate (?). As a result, the training data is
limited and noisy, which makes the scoring task even harder. It can be
problematic when comparing models with similar quality. In addition, this
model is designed only for evaluating dialogue response generation models.
More recently, embedding similarity based metrics such as HUSE (?) and
BERTScore (?). These metrics alleviate the first problem of text overlap
metrics by modeling semantic similarity better. However, they can not address
the response diversity problem and thus are only suitable for machine
translation and text summarization.

Another line of research on NLG evaluation is to unify human evaluation with
statistical evaluation (?; ?). These works are orthogonal to our paper as they
mainly focus on the combination of human evaluation and automated evaluation.

Another related work of our research is the skill rating system, which
evaluates players by observing a record of wins and losses of multiple players
and inferring the value of a latent, unobserved skill variable for each player
that explains the records of wins and losses. It is first adopted to evaluate
GANs (?) for synthesizing images (?) by competing generators against
discriminators. Their approach is an approximation of skill rating as the
original skill rating system requires game played by two symmetric players,
while in their system the players are asymmetric. Their approach does not
include the “tie” option, thus can not distinguish cases where the
discriminator is confident enough or not. More importantly, their approach is
only designed for evaluating GANs while our approach can be used for any NLG
models.

##  3 Methodology

We present the proposed approach in this section. We begin with the sample-
level pairwise comparison model. Afterwards, we introduce how to adopt the
skill rating system to perform model-level evaluation of NLG models.

###  3.1 Learning to Compare

Figure 1: model architecture of the comparative evaluator, the context is
concatenated with generated samples.

The proposed comparative evaluator is a text pair relation classifier which is
trained to compare the task-specific quality of two samples. The motivation of
evaluating one sample by comparing it with another sample is drawn from the
insight learned when conducting human evaluation for NLG models. We find that
when comparing two NLG models, instead of asking human annotator to assign
scores separately for samples generated by different models, which resembles
the case in the ADEM model (?), it is much easier for human annotators to
directly compare one sample generated by the first model against another
sample from the second model pairwisely and compute the win/loss rate. The
comparison-based evaluation may also be more accurate, which is demonstrated
by a higher inter-annotator agreement score in our preliminary experiments.

The comparative evaluator learns a total order of sample quality by
classifying whether the first compared sample is better ($>$), worse ($<$), or
indistinguishable ($\approx$) in terms of its quality compared with another
sample. In this way, our model encodes the inductive bias that sometimes two
samples can have similar quality and it is hard and unreliable to choose the
better sample. By giving our model the third “tie” option, it can explicitly
express its uncertainty and choose its preference only when being confident
enough. This design choice is motivated by the practice that adding the “tie”
option for human annotator when performing pairwise human evaluation can often
make the comparison easier and more reliable. For a text sample, our
comparative evaluator can provide a more informative assessment than the
binary discriminative evaluator because one evaluated sample can receive
multiple feedback from the comparative evaluator by comparing it with multiple
other samples. In contrast, the discriminative evaluator can only evaluate a
sample once, which is more likely to suffer from the inherent uncertainty of
the evaluator.

We propose two approaches to construct pairwise training examples for training
a comparative evaluator. The first approach generates strong supervision
examples. It is based on the intuition that human written references are
generally of better quality than machine-generated samples, and it is hard to
tell the difference in term of the quality when two compared samples are both
generated by machines or human written reference. We denote $S_{+}$$/$$S_{-}$
as the set of real/generated samples. For a real sample $s_{+}\in S_{+}$ and a
generated sample $s_{-}\in S_{-}$, we assign the label “better ($>$)” to the
pair ($s_{+}$, $s_{-}$) and “worse ($<$)” to ($s_{-}$, $s_{+}$). For two
samples both from real data or from the generated samples, we assign the label
“indistinguishable ($\approx$)” to such pairs (i.e., ($s_{+}^{i}$,
$s_{+}^{j}$) and ($s_{-}^{i}$, $s_{-}^{j}$)). For a training set with $n$ real
samples and $n$ generated samples, we can construct $\binom{2n}{2}$ pairwise
training examples for the comparative evaluator, allowing to enhance the
generalization ability and introduce more informative learning signals than
the standard real/fake binary discriminative evaluator. Note that when
constructing a sample pair ($s_{-}^{i}$, $s_{-}^{j}$), $s_{-}^{i}$ and
$s_{-}^{j}$ are sampled from the same checkpoint of the same model in order to
ensure that they are of similar quality in expectation.

One problem of the strong supervision approach is that it always labels two
generated samples as indistinguishable. However, during inference, the input
of the comparative evaluator is a pair of two generated samples from different
models. Thus it requires the model to capture the quality relation in training
examples and generalize well to successfully compare two samples rather than
simply classifying them as indistinguishable, which provides relatively less
information for evaluating NLG models.

To tackle this problem, we propose an approach to construct weak supervision
examples for training the comparative evaluator. The intuition of our weak
supervision approach is that during training, the quality of the NLG model
keeps improving until convergence. Given two checkpoints of the same model, we
can thus consider samples generated by the more recent checkpoint are of
better quality compared with samples generated by the earlier version of the
same model. This approach is considered to be weak supervision because the
model quality may not improve monotonically and sometimes it is hard to decide
whether the model begins to overfit the training data and its quality starts
to decline. To minimize the noise introduced by these problems, we empirically
set the minimal margin between two selected checkpoints to be $10\%$ of the
total training iteration and do not select two “almost converged” checkpoints.
The construction of training samples is similar to the first approach. In
addition, motivated by the fact that the larger the margin between the quality
two selected version of the model, the easier for the comparative evaluator to
learn to distinguish the training examples, we propose to use curriculum
learning (?) by feeding the comparative evaluator with sample pairs with
larger margin (i.e. more training iterations between two selected checkpoints)
during initial training stage and gradually decrease the margin to let the
model gradually learn to capture smaller quality differences. Moreover, when
human preference annotation is available, we can additionally fine-tune the
comparative evaluator with human annotations.

The comparative evaluator is trained with maximum likelihood estimation (MLE)
objective, as described in eq 1

$L=-\mathbb{E}_{(\boldsymbol{x}_{1},\boldsymbol{x}_{2})\sim\mathcal{X}}[{\log{D^{Q(\boldsymbol{x}_{1},\boldsymbol{x}_{2})}_{\phi}(\boldsymbol{x}_{1},\boldsymbol{x}_{2})}}]$
(1)

where $\mathcal{X}$ is the set of pairwise training examples contructed as
described above,
$Q(\boldsymbol{x}_{1},\boldsymbol{x}_{2})\in\\{>,<,\approx\\}$ is the true
label for the pair ($\boldsymbol{x}_{1}$, $\boldsymbol{x}_{2}$),
$D_{\phi}^{q}(\boldsymbol{x}_{1},\boldsymbol{x}_{2})$ is the probability of
the comparative discriminator’s prediction being $q$ ($q\in\\{>,<,\approx\\}$)
for the pair ($\boldsymbol{x}_{1}$, $\boldsymbol{x}_{2}$).

As comparing the quality of generated text requires good natural language
understanding ability and our comparative evaluator is formulated as a
sentence pair classification model, we propose to fine-tune BERT (?) as the
comparative evaluator, the architecture of the resulting comparative evaluator
is illustrated by Figure 1. Note that the compared sample A and B are based on
the same context, which ensures that they are comparable.

###  3.2 Skill Rating

In player-vs-player games such as chess or tennis, skill rating systems such
as Elo (?) or Glicko2 (?) evaluate players by observing a record of wins and
losses of multiple players and inferring the value of a latent, unobserved
skill variable for each player that explains the records of wins and losses.
We adopt the skill rating system for model-level evaluation of NLG models. By
taking the trained comparative evaluator as the “playground” and NLG models as
“player”, the “player-vs-player” game is played by sampling one output sample
from each NLG model conditioning on the same input and the game output is
decided by the comparative evaluator.

Following previous work (?), in our paper, we use the Glicko2 system (?). The
employed system can be summarized as follows: each player’s skill rating is
represented as a Gaussian distribution, with a mean and standard deviation,
representing the current state of the evidence about their “true” skill
rating. As we evaluate frozen snapshots of NLG models, we disabled an
irrelevant feature of Glicko2 that increases uncertainty about a human
player’s skill when they have not participated in a match for some time.
Another difference is that conventional skill rating systems do not support
the “tie” option, which is important for the system to be stable and reliable
in our case because the evaluator is not perfect. To incorporate this feature,
we follow the intuition that a player’s skill rating should be increased when
it draws with another player with a higher skill rating and vice versa. We
come up with a simple rule which increases/decreases the skill rating of one
player by a ratio (e.g. 0.1) of the changes in its skill rating when it
wins/loses if it draws with another player with higher/lower skill rating. In
our experiments, the skill rating is performed by randomly sampling two
compared models, simulating a “game” between two selected models by sampling
one sample from each model and comparing them with the comparative evaluator,
and then updating the skill rating of selected models according to the
outcome. This procedure is performed iteratively until convergence, which is
defined as the order of skill ratings of compared models keeps the same after
each model is selected at least 50 times. While the sampling procedure can be
optimized by bayesian optimization (?) or multi-armed bandit algorithms (?),
we choose to keep the method as simple as possible and use random sampling.

##  4 Experiments

We set up experiments in order to answer the following research questions:

  * •

RQ1: Can the comparative evaluator correlate better with human preference in
sample-level than previous automated metrics when evaluating open domain NLG
models?

  * •

RQ2: Can the comparative evaluator correlate better with human preference in
model-level, so that our approach can measure the progress on open domain NLG
better?

  * •

RQ3: As existing approaches fail to correlate well with human preference,
whether and to what extent this problem affects the quality of the final NLG
model when performing hyperparameter search and early-stopping?

  * •

RQ4: If the previous problem exists, can proposed comparative evaluator reduce
this problem?

###  4.1 Experimental Settings

#### Datasets

We evaluate the effectiveness of the proposed approach on two open domain
natural language generation tasks: story generation and open domain dialogue
response generation. For story generation, we use the WritingPrompts dataset
released by ?. The WritingPrompts dataset is a large dataset of 303,358 human-
generated stories paired with writing prompts from an online forum. NLG models
are trained by taking writing prompts as input and generating the whole story.
The average length of prompts is 28.4 and the average length of stories is
734.5 words, which makes human evaluation very expensive and better automated
metrics are thus critical. For open domain dialogue response generation task,
we use the Dailydialog dataset (?), which consists of dialogues that resemble
daily conversations across multiple topics. It comprises of 13k dialogues with
an average of 7.9 turns per dialog.

#### Compared Models and Metrics

As our objective is to evaluate the evaluators rather than comparing state-of-
the-art models, we choose three representative sequence-to-sequence
architectures: LSTM (?) seq2seq, Convolutional seq2seq (?), and transformer
(?) model. We compare models with different architectures, hyperparameter
choices, and early-stopping criteria with different automated metrics, as well
as human evaluation.

Regarding the evaluation metric (and criteria for choosing hyperparameter
choice and early-stopping), we compare the proposed approach with the
discriminative evaluator, BLEU score (average of 2-, 3-, 4-grams), perplexity,
and ADEM. When evaluating generated stories, we cut off the story at the
nearest sentence for stories longer than 250 words.

The proposed comparative evaluator is employed for choosing hyperparameter by
performing skill rating among all models trained with different hyperparameter
choices111For each model, we randomly sample 5 hyperparameter choices in a
predefined range.. For early-stopping, as incrementally performing skill
rating is computationally expensive, we propose to perform n (e.g. 1000)
pairwise comparison between the samples generated by the latest checkpoint and
the previous k (e.g. 2) checkpoints and stop training when the wining rate of
latest checkpoint keeps being smaller than its losing rate for 5 iterations.

#### Detail of Parameterized Evaluators

The proposed comparative evaluator is trained by fine-tuning BERT-large as a
sentence-pair classifier. To ensure fair evaluation, we also train the
discriminative evaluator by fine-tuning BERT. For ADEM, we adopt its original
implementation as its architecture is relatively complicated. In addition, we
perform ablation study by evaluating three variants of the comparative
evaluator where it is trained without strong supervision examples, without
weak supervision examples, without fine-tuning with human preference
annotations, and without transferring from BERT.

#### Human Evaluation Procedure

As human evaluation is expensive, sample-level evaluation is performed jointly
with model-level evaluation, which is also used for evaluating the ability of
different metrics for performing hyperparameter search and early-stopping.
Concretely, we perform 10 groups of evaluations for performing hyperparameter
selecting and early-stopping with five compared automated metrics. In each
evaluation, each of the five compared metrics is used to select the best
hyperparameter combination or early-stopping checkpoint with other variants
fixed.

We choose to perform score-based human evaluation for four reasons: 1) the
ADEM baseline requires human-annotated score as training examples, 2) we can
construct up to $\binom{2n}{2}$ training examples for our comparative
evaluator with $n$ human-annotated scores, 3) score-based human evaluation
facilitates the evaluation of correlation scores, and 4) as all other metrics
do not perform pairwise comparison, using pairwise human evaluation will
likely be biased toward our approach.

We sample 20 generated samples from each model (out of 5) of the 20 evaluation
groups. We invite 20 human annotators which are all graduate students with
good English language proficiency to score these samples. Each annotator
scores one sample from each model, such that each model is uniformly
evaluated. The score scales from 1 to 5, higher score indicates better overall
sample quality. According to experimental results from ?, we do not ask
annotators to provide specific scores for fluency or informativeness. To test
the inner-annotator agreement score, we additionally ask them to evaluate
another 40 generated samples, of which 20 samples are scored from 1 to 5 and
another 20 are evaluated based on pairwise comparison with 4 other generated
samples and scored to 1-5 based on how many times they are considered to be
better than a reference sample. We get an inter-annotator agreement score
$\kappa=0.53$ for directly scoring and $\kappa=0.76$ with pairwise comparison,
which validates our intuition that evaluation by comparison may be more
accurate. These additional human annotations are used as training data for
ADEM and the comparative evaluator.

Task | Story Generation | Dialogue
---|---|---
Metric | Spearman | Pearson | Spearman | Pearson
our approach |  |  |  |
\- with skill rating | 0.392($<0.001$) | 0.387($<0.001$) | 0.473($<0.001$) | 0.469($<0.001$)
\- with random sampled reference | 0.389($<0.001$) | 0.378($<0.001$) | 0.461($<0.001$) | 0.472($<0.001$)
ADEM | 0.162 | 0.148 | 0.353 | 0.341
Adversarial Evaluator | 0.105 | 0.111 | 0.197 | 0.182
BLEU | 0.032 | 0.028 | 0.053 | 0.061
Perplexity | 0.052 | 0.057 | 0.024 | 0.015

Table 1:  Sample-level correlation between metrics and human judgments, with
p-values shown in brackets.

Task | Story Generation | Dialogue
---|---|---
Metric | Spearman | Pearson | Spearman | Pearson
our approach |  |  |  |
\- with skill rating | 0.612($<0.001$) | 0.631($<0.001$) | 0.764($<0.001$) | 0.783($<0.001$)
\- averaged sample-level skill rating | 0.518 | 0.541 | 0.651 | 0.675
\- averaged reference-based score | 0.473 | 0.482 | 0.634 | 0.653
ADEM | 0.291 | 0.302 | 0.541 | 0.553
Adversarial Evaluator | 0.248 | 0.272 | 0.491 | 0.502
BLEU | 0.096 | 0.103 | 0.217 | 0.293
Perplexity | 0.113 | 0.127 | 0.071 | 0.083

Table 2: Model-level correlation between metrics and human judgments, with
p-values shown in brackets.

###  4.2 Experimental Designs & Results

#### RQ1: Sample-Level Correlation

To test the correlation of different automated metrics with respect to human
preference, we employ different metrics to score the collected 2000 samples
and calculate their Pearson and Spearman correlation with human scores. For
comparative evaluator, as the evaluation is performed pairwisely and no
absolute score is available, we use two different approaches to get an
absolute score for each sample: 1) we sample 50 common references from
machine-generated samples for each task and compare each sample with all
references by the comparative evaluator. A sample gets 3 points when beats a
reference, 1 point when draws with the reference, and get 0 point when loses,
2) we adopt skill rating system by regarding each sample as an NLG model which
always outputs the same sample and use the skill rating for each sample as its
score. To ensure the computational budget to be roughly the same, we fix the
number of plays in skill rating to 10,000.

The experimental results are summarized in Table 1. We can see that the
proposed comparative evaluator correlates far better with human judgment than
BLEU and perplexity. When compared with recently proposed parameterized
metrics including adversarial evaluator and ADEM, our model consistently
outperforms them by a large margin, which demonstrates that our comparison-
based evaluation metric is able to evaluate sample quality more accurately. In
addition, we find that evaluating generated samples by comparing it with a set
of randomly selected samples or using sample-level skill rating performs
almost equally well. This is not surprising as the employed skill rating is
able to handle the inherent variance of players (i.e. NLG models). As this
variance does not exist when we regard a sample as a model which always
generates the same sample.

#### RQ2: Model-Level Correlation

As for model-level evaluation, we employ the average score of the evaluated
100 samples as each model’s score and calculate their correlation with human
scores. For comparative evaluator, we propose three different approaches to
get an absolute score for each sample: 1) we calculate the average reference-
based score (method 1 for sample-level comparison) of each sample as model-
level score, 2) we calculate the average skill rating of each sample obtained
in the experiments of RQ1 as model-level score, 2) we use the proposed skill
rating system to get a model-level skill rating for each compared model.

Results are shown in Table 2. We can see that the proposed comparative
evaluator with skill rating significantly outperforms all compared baselines,
including comparative evaluator with averaged sample-level scores. This
demonstrates the effectiveness of the skill rating system for performing
model-level comparison with pairwise sample-level evaluation. In addition, the
poor correlation between conventional evaluation metrics including BLEU and
perplexity demonstrates the necessity of better automated evaluation metrics
in open domain NLG evaluation.

#### RQ3&4: Automated Metrics for Model Training

We further investigate the impact of imperfect metrics on training NLG models.
As described in the human evaluation procedure, we perform 10 runs to test the
reliability of each metric when used to perform hyperparameter tuning and
early-stopping respectively. In each run, we select the best hyperparameter
combination or early-stopping checkpoint based on each of the five compared
metrics. Human evaluation is then employed to identify the best choice. We
evaluate the performance of each metric by how many times (out of 10) they
succeeded in selecting the best hyperparameter combination or early-stopping
checkpoint (out of 4) and the average human-annotated score for their selected
models.

Task | Hyperparameter Search | Early-stopping
---|---|---
Metric | win times | averaged score | win times | averaged score
our approach | 8 | 3.14 | 9 | 3.41
ADEM | 5 | 3.03 | 6 | 3.29
Adversarial Evaluator | 4 | 2.92 | 4 | 3.23
BLEU | 2 | 2.75 | 3 | 3.15
Perplexity | 3 | 2.77 | 3 | 3.11

Table 3:  Performance of different metrics in hyperparameter tuning and
earlystop checkpoint selecting.

The results are shown in Table 3. We can see that conventional automated
metrics perform poorly and result in sub-optimal result when performing
hyperparameter search and selecting the best performing checkpoints.
Converting evaluation metric from BLEU or perplexity to the proposed
comparative evaluator can yield non-neglectable improvements without changing
model architecture or training objective. While previous work on NLG
evaluation mostly focuses on the evaluation stage and does not explore the
influence of imperfect metrics during model training, our experiments
demonstrate the existence of this problem and that the proposed method can, to
some extent, alleviate this problem.

###  4.3 Qualitative Analysis

Context | Sample A | Sample B | Output
---|---|---|---
Say,Jim,how about going for a few beers after dinner? | I do not know about it. | No, it is not good. | A $<$ B
I suggest a walk over to the gym where we can meet some friends. | That’s a good idea, ok. | No, I do not like to. | Tie
What shall we do ? I don’t feel like sitting at home. | We can go for a walk. | I suggest staying at home. | A $>$ B

Table 4: Examples of comparison results between two generated samples given
context.

We present several comparison examples in the Dailydialog dataset for
qualitative analysis of the proposed comparative evaluator. From the first
example, we can see that the comparative evaluator is capable of identifying
that generic and dull responses (i.e. “I don’t know”) should be considered as
of worse quality. The second example suggests that our approach handles the
diversity in possible responses well, as it regards both positive response and
negative response as valid responses. Hopefully, these examples may provide us
with some insights about why the proposed metric correlates better with human
preference.

Metric | Spearman | Pearson
---|---|---
full model | 0.764 | 0.783
w/o comparison | 0.491 | 0.502
w/o tie option | 0.557 | 0.561
w/o strong supervision | 0.697 | 0.703
w/o weak supervision | 0.728 | 0.737
w/o human annotation | 0.602 | 0.609
w/o BERT | 0.644 | 0.662
Table 5:  Model-level correlation between ablated variants and human judgments
in the Dailydialog dataset

###  4.4 Ablation Study

To better understand the proposed comparative evaluator and analyze the
relative importance of its different components, we conduct an ablation study
with several variants of the proposed model:

  * •

w/o comparison: Evaluating generated samples without comparison, which
degrades to the adversarial evaluation method.

  * •

w/o strong supervision: Training the comparative evaluator without “strong
supervision”, which models the inductive bias that human written reference
samples are generally of better quality compared with that generated by NLG
models.

  * •

w/o weak supervision: Training without “weak supervision”, which models the
inductive bias that the quality of NLG models generally improves during
training.

  * •

w/o human preference annotation Training without human annotated preference
data (i.e. only with strong and weak supervision).

  * •

w/o tie option The variant of comparative evaluator where the model must
select the better sample rather than able to admit its uncertainty.

  * •

w/o BERT The variant where the model is trained from scratch instead of fine-
tuning BERT.

We evaluate these model variants on the Dailydialog dataset. Results are
presented in Table 5. We can see that comparison-based evaluation is very
effective as our model correlates much better than adversarial evaluator. The
tie option is also very important as it can prevent the comparative evaluator
from making uncertain decision and model the inductive bias that samples
generated by the same model are generally of similar quality, which may help
our model generalize better. As for different sources of training examples, we
find that human preference annotation is the most important, which is not
surprising. In addition, we find that the proposed weak supervision also
helps, but is of smaller relative importance compared with strong supervision.
This may be due to the fact that examples constructed by the weak supervision
approach may contain a lot of noise. We can also see that our model correlates
well with human preference without training with human preference annotation,
this is very important in practice as human annotations are not always
available. Finally, we find that transferring the natural language
understanding ability from BERT to be very important for the final
performance.

##  5 Discussion and Conclusion

In this paper, we present a novel comparison-based parameterized automated
evaluation metric for evaluating open domain NLG models. The proposed model is
based on the intuition that we can better evaluate the quality of a sample by
comparing it with other samples. Our model allows the model to admit its
uncertainty with the “tie” option. We adopt the skill rating system to perform
model-level evaluation based on sample-level pairwise comparison.

By transferring pretrained natural language understanding knowledge from BERT
and fine-tuning with strong and weak supervision examples and human preference
annotations, our model correlates better with human judgment than other
compared metrics. In addition, we find that when used as evaluation metrics,
conventional metrics such as BLEU and perplexity may affect the training stage
of NLG models as they may lead to sub-optimal hyperparameter choice and
checkpoint selection. Our model, in contrast, is much more reliable when
performing these choices.

## References

  * [Banerjee and Lavie] Satanjeev Banerjee and Alon Lavie.  Meteor: An automatic metric for mt evaluation with improved correlation with human judgments.  In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pages 65–72, 2005.
  * [Bengio et al.] Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston.  Curriculum learning.  In Proceedings of the 26th annual international conference on machine learning, pages 41–48. ACM, 2009.
  * [Chaganty et al.] Arun Tejasvi Chaganty, Stephen Mussman, and Percy Liang.  The price of debiasing automatic metrics in natural language evaluation.  arXiv preprint arXiv:1807.02202, 2018.
  * [Devlin et al.] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  Bert: Pre-training of deep bidirectional transformers for language understanding.  arXiv preprint arXiv:1810.04805, 2018.
  * [Elo] Arpad E Elo.  The rating of chessplayers, past and present.  Arco Pub., 1978.
  * [Fan et al.] Angela Fan, Mike Lewis, and Yann Dauphin.  Hierarchical neural story generation.  arXiv preprint arXiv:1805.04833, 2018.
  * [Garbacea et al.] Cristina Garbacea, Samuel Carton, Shiyan Yan, and Qiaozhu Mei.  Judge the judges: A large-scale evaluation study of neural language models for online review generation.  arXiv preprint arXiv:1901.00398, 2019.
  * [Gehring et al.] Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N Dauphin.  Convolutional sequence to sequence learning.  In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 1243–1252. JMLR. org, 2017.
  * [Glickman] Mark E Glickman.  Example of the glicko-2 system.  Boston University, pages 1–6, 2012.
  * [Goodfellow et al.] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio.  Generative adversarial nets.  In Advances in neural information processing systems, pages 2672–2680, 2014.
  * [Hashimoto et al.] Tatsunori B Hashimoto, Hugh Zhang, and Percy Liang.  Unifying human and statistical evaluation for natural language generation.  arXiv preprint arXiv:1904.02792, 2019.
  * [Herbrich et al.] Ralf Herbrich, Tom Minka, and Thore Graepel.  Trueskill: a bayesian skill rating system.  In Advances in neural information processing systems, pages 569–576, 2007.
  * [Hochreiter and Schmidhuber] Sepp Hochreiter and Jürgen Schmidhuber.  Long short-term memory.  Neural computation, 9(8):1735–1780, 1997.
  * [Kannan and Vinyals] Anjuli Kannan and Oriol Vinyals.  Adversarial evaluation of dialogue models.  arXiv preprint arXiv:1701.08198, 2017.
  * [Li et al.] Jiwei Li, Will Monroe, Tianlin Shi, Sébastien Jean, Alan Ritter, and Dan Jurafsky.  Adversarial learning for neural dialogue generation.  arXiv preprint arXiv:1701.06547, 2017.
  * [Li et al.] Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu.  Dailydialog: A manually labelled multi-turn dialogue dataset.  arXiv preprint arXiv:1710.03957, 2017.
  * [Lin] Chin-Yew Lin.  Rouge: A package for automatic evaluation of summaries.  In Text summarization branches out, pages 74–81, 2004.
  * [Liu et al.] Chia-Wei Liu, Ryan Lowe, Iulian V Serban, Michael Noseworthy, Laurent Charlin, and Joelle Pineau.  How not to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation.  arXiv preprint arXiv:1603.08023, 2016.
  * [Lowe et al.] Ryan Lowe, Michael Noseworthy, Iulian V Serban, Nicolas Angelard-Gontier, Yoshua Bengio, and Joelle Pineau.  Towards an automatic turing test: Learning to evaluate dialogue responses.  arXiv preprint arXiv:1708.07149, 2017.
  * [Olsson et al.] Catherine Olsson, Surya Bhupatiraju, Tom Brown, Augustus Odena, and Ian Goodfellow.  Skill rating for generative models.  arXiv preprint arXiv:1808.04888, 2018.
  * [Papineni et al.] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu.  Bleu: a method for automatic evaluation of machine translation.  In Proceedings of the 40th annual meeting on association for computational linguistics, pages 311–318. Association for Computational Linguistics, 2002.
  * [See et al.] Abigail See, Peter J. Liu, and Christopher D. Manning.  Get to the point: Summarization with pointer-generator networks.  Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2017.
  * [Shimanaka et al.] Hiroki Shimanaka, Tomoyuki Kajiwara, and Mamoru Komachi.  Ruse: Regressor using sentence embeddings for automatic machine translation evaluation.  In Proceedings of the Third Conference on Machine Translation: Shared Task Papers, pages 751–758, 2018.
  * [Snoek et al.] Jasper Snoek, Hugo Larochelle, and Ryan P Adams.  Practical bayesian optimization of machine learning algorithms.  In Advances in neural information processing systems, pages 2951–2959, 2012.
  * [Sordoni et al.] Alessandro Sordoni, Michel Galley, Michael Auli, Chris Brockett, Yangfeng Ji, Margaret Mitchell, Jian-Yun Nie, Jianfeng Gao, and Bill Dolan.  A neural network approach to context-sensitive generation of conversational responses.  Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2015.
  * [Sutskever et al.] Ilya Sutskever, Oriol Vinyals, and Quoc V Le.  Sequence to sequence learning with neural networks.  In Advances in neural information processing systems, pages 3104–3112, 2014.
  * [Vaswani et al.] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  In Advances in neural information processing systems, pages 5998–6008, 2017.
  * [Vermorel and Mohri] Joannes Vermorel and Mehryar Mohri.  Multi-armed bandit algorithms and empirical evaluation.  In European conference on machine learning, pages 437–448. Springer, 2005.
  * [Zhang et al.] Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi.  Bertscore: Evaluating text generation with bert.  arXiv preprint arXiv:1904.09675, 2019.


# Towards Privacy-Preserving Audio Classification Systems

Bhawana Chhaglani  University of Massachusetts AmherstAmherstUSA
<EMAIL_ADDRESS>,  Jeremy Gummeson  University of Massachusetts
AmherstAmherstUSA<EMAIL_ADDRESS>and  Prashant Shenoy  University of
Massachusetts AmherstAmherstUSA<EMAIL_ADDRESS>
(2024; 20 February 2007; 12 March 2009; 5 June 2009)

###### Abstract.

Audio signals can reveal intimate details about a person’s life, including
their conversations, health status, emotions, location, and personal
preferences. Unauthorized access or misuse of this information can have
profound personal and social implications. In an era increasingly populated by
devices capable of audio recording, safeguarding user privacy is a critical
obligation. This work studies the ethical and privacy concerns in current
audio classification systems. We discuss the challenges and research
directions in designing privacy-preserving audio sensing systems. We propose
privacy-preserving audio features that can be used to classify wide range of
audio classes, while being privacy preserving.

Audio classification, Privacy-preserving sensing, Acoustic features

††copyright: acmlicensed††journalyear: 2024††doi: XXXXXXX.XXXXXXX††conference:
Make sure to enter the correct conference title from your rights confirmation
emai; April 28, 2024; San Diego, California, USA††isbn:
978-1-4503-XXXX-X/18/06††ccs: Human-centered computing Personal digital
assistants††ccs: Security and privacy

##  1\. Introduction

Audio sensing has tremendous potential and can be used to infer a rich array
of personal information about an individual (zhang2019pdvocal, ;
swain2018databases, ). Kroger et al. (kroger2020privacy, ) presents an
overview of sensitive pieces of information that can, with the help of
advanced data analysis methods, be derived from audio, including cues to a
speaker’s biometric identity, personality, physical traits, geographical
origin, emotions, level of intoxication and sleepiness, age, gender, and
health condition. These are all examples of privacy-sensitive information and
its access should be regulated. Today, we are surrounded by listening devices
including smart speakers, voice assistants, baby monitors, smoke alarms, and
other emerging device categories. Some of them are always-on and listening to
sensitive content, while others can be accidentally triggered
(schonherr2022exploring, ). Thus, privacy remains one of the core problems in
audio sensing which needs to be addressed.

Figure 1. Privacy Concerns in Audio Classification Systems

Existing audio classification systems use features like spectrograms that can
potentially compromise user privacy. To avoid privacy concerns, prior work has
primarily focused on eliminating or obfuscating speech to ensure privacy
(xia2020pams, ; chen2008audio, ). However, speech is not the only privacy
sensitive information present in the audio signal — it can also reveal
information about the speaker’s identity and whereabouts. The idea is to
explore a more holistic definition of privacy and propose a mechanism to
evaluate and quantify the privacy of an audio-based system. Through this
research, we aim to explore the potential of audio sensing in enabling novel
applications, while simultaneoulsy ensuring that user privacy is retained.

In this work, we discuss the key challenges in realizing privacy-aware audio
classification systems. Using the example of environmental sound
classification, we demonstrate the application of privacy-preserving audio
features to safeguard privacy. We show that this approach is generalizable and
can achieve comparable accuracy as the state-of-the-art techniques that do not
consider privacy.

##  2\. Need for Ensuring Privacy in Audio Classification

Most audio classification systems convert audio signals into Mel-Frequency
Cepstral Coefficients (MFCCs), Mel spectrograms, and Short-Time Fourier
Transform (STFT) and then feed them into advanced deep learning models as
shown in Figure 2. Piczak et al. (piczak2015environmental, ) explores the use
of Mel spectrograms as input to convolutional neural networks (CNNs) for the
classification of environmental sounds. Forsad et al. (al2020flusense, )
compares MFCC and STFT for cough with CNNs for detection task. By effectively
bridging the gap between raw audio signals and the sophisticated pattern
recognition capabilities of deep learning (DL) architectures, these features
enable the models to achieve remarkable accuracy and efficiency in variety of
tasks. However, the same features that enhance the performance of audio
classification models can also pose significant privacy risks. MFCCs, STFT,
Mel spectrograms or other spectrograms, by their very nature, encapsulate
detailed information about the audio source, potentially including sensitive
personal information. For instance, MFCCs can retain distinctive speech
characteristics that could be exploited to identify a speaker, thereby leaking
personal identity information. Similarly, STFT and Mel spectrograms can
inadvertently reveal background noises or conversations that were not meant to
be shared, thereby compromising the privacy of individuals. The practice of
transmitting spectrograms to cloud-based services for model prediction further
heightens privacy concerns. This duality underscores the need for a careful
consideration of privacy implications when designing and deploying audio
classification systems.

Figure 2. Typical Audio Classification Pipeline

##  3\. Privacy-Aware Audio Classification: Challenges

In this section, we discuss the major hurdles in designing privacy-preserving
audio-based systems.

  * •

Expanding Privacy Beyond Speech: Prior work mainly considers speech as the
sensitive content and tries to filter out (xia2020pams, ), obfuscate
(liaqat2017method, ), or replace (chen2008audio, ) the speech segments. Audio
contains more privacy invasive information than just speech. There is a need
to look at a more holistic definition of privacy.

  * •

Complexity of Privacy Evaluation: Evaluating privacy in audio sensing systems
is a non-trivial problem as privacy varies with context. Most common
techniques are to use automatic speech recognition using Google APIs
(soundswordless, ) or perform human evaluation (chhaglani2022flowsense, ).
Boovaraghavan et al. (boovaraghavan2024kirigami, ) use word error rate (WER)
and phoneme error rate (PER) as the privacy evaluation metrics. However, these
techniques do not evaluate whether the systems are leaking speaker related
information or their location. For example, the system could infer myriads of
details about an individual even with high WER. Thus, there is a need for
accurate privacy evaluation mechanism.

  * •

Towards Universal Privacy Preservation: Some of the prior work has designed
privacy preserving pipelines for different applications. Bourlard et al.
(bourlardprivacy, ) uses privacy sensitive features for speech/non-speech
detection, while Parthasarathi et al. (parthasarathi2009speaker, ) uses
privacy sensitive audio features to detect speaker change in conversations.
Although, these techniques are useful, they are application specific and
require manual effort to design hand-crafted features. There is a need to
address the challenge of privacy from a more general perspective by designing
a set of generalizable privacy-aware features.

  * •

Privacy-Accuracy Trade-off: The accuracy-privacy tradeoff in audio
classification systems represents a critical challenge
(chhaglani2022flowsense, ). On one hand, achieving high accuracy in audio
classification often necessitates the use of detailed and comprehensive audio
features, such as MFCCs. These features capture nuanced aspects of the audio
signal, allowing deep learning models to make precise inferences about the
content, context, or identity associated with the audio data. On the other
hand, the very richness of information that enables such accuracy can also
lead to significant privacy concerns, as these features may contain or enable
the reconstruction of sensitive information about individuals or their
environment.

To address these challenges, we propose to redefine privacy in the context of
audio classification to encompass more than just speech. We aim to use
privacy-preserving audio features that are universally applicable and maintain
effectiveness of audio classification systems.

##  4\. Preliminary Experiments with ESC-50

###  4.1. Dataset Description

The ESC-50 (piczak2015esc, ) dataset is a comprehensive and well-curated
collection designed for the task of environmental sound classification,
highlighting the diversity and complexity of auditory scenes that can be
encountered in everyday life. ESC-50 is a labeled collection of 2,000
environmental audio recordings (each 5 seconds long) with 50 classes. These
categories encompass a wide range of sounds from natural environments (such as
rain, thunderstorms, and animal sounds), human-made noises (like car horns,
alarms, and mechanical tools), and human sounds (including laughing, clapping,
and crying). The dataset is fully annotated, with labels indicating the
category of each audio clip.

###  4.2. Generalizable Privacy-Preserving Features

We process the audio files using a sliding window of 500 ms with 100 ms
overlap. We remove silent periods from the audio files using the top decibel
threshold of 20. Next, each audio segment is converted into a list of time and
frequency domain audio features. These features are as follows:

  * •

Zero Crossing Rate (ZCR) indicates the number of times the signal changes
sign. This can be related to the texture of the sound, which is very relevant
in differentiating between diverse environmental sounds.

  * •

Harmonic-to-Noise Ratio (HNR) measures the amount of harmonic content compared
to noise within a signal. Since environmental sounds can be either more
harmonic (e.g., bird singing) or more noisy (e.g., rain), this feature can
help distinguish between them.

  * •

Spectral Contrast refers to the difference in amplitude between peaks and
valleys in the sound spectrum.

  * •

Peak, RMS, Energy relate to the amplitude and power of the audio signal and
are useful for distinguishing between loud and soft sounds, as well as the
intensity of the sound source.

  * •

Spectral Roll-off indicates the frequency below which a certain percentage of
the total spectral energy is contained. This feature helps to separate sounds
with high-frequency content from those with low-frequency content.

  * •

Spectral Flatness and Bandwidth (BW) provide a measure of how noise-like a
sound is, versus being tonal.

  * •

Spectral Centroid is a measure of the ’brightness’ of a sound, which can be
useful to characterize sounds with high-frequency content like bird songs or
bells.

We do not include features that contain privacy invasive information. For
instance, fundamental frequency can reveal details about speaker or formants
can reveal speech content as mentioned in source filter model of speech
production (fant1981source, ).

###  4.3. Classification Results

Figure 3. Classification metrics using privacy-preserving features

We use these features to train a random forest classifier model. To evaluate
the model, we split the dataset into train, test and validation set (70%, 15%,
15%) after shuffling. We achieve aggregate accuracy of 92.23% over all the
classes. We observe the feature importances using gini importance aggregated
over all the classes. We observe that spectral contrast, ZCR and HNR are the
most important features for this task. This is because ZCR is particularly
useful in distinguishing between percussive sounds (like footsteps or
clapping) and more harmonic sounds (like animal calls or musical instruments).
HNR is crucial for differentiating between sounds that are more tonal and
structured, such as human voices or music, versus those that are more
stochastic and noise-like, such as wind or rushing water.

###  4.4. Comparison with Non Privacy-based approaches

Most of the prior work uses MFCCs or the Mel spectrogram for classifying
ESC-50 classes. These spectrograms can be use to infer human speech
(ittichaichareon2012speech, ) and speaker identity (yutai2009speaker, ). The
accuracy results obtained from the ESC-50 dataset using spectrogram as feature
input has a wide range of variation depending on the spectrogram type and the
DL model. Mu et al. (mu2021environmental, ) use Log-Mel spectrogram and
temporal-frequency attention based convolutional neural network model (TFCNN),
and achieve accuracy of 84.4%. Wang et al. (wang2019environmental, ) propose
parallel temporal-spectral attention mechanism for CNN to learn discriminative
time-frequency representations of the spectrogram. This model achieve accuracy
of 88.6%. Certain models (gazneli2022end, ) achieve very high accuracy of ¿90%
using pre-training and data augmentation techniques. Thus, we can achieve
comparable or better accuracy by taking the privacy-aware route. The proposed
features are generalizable as they can be used for 50 different environment
sounds

##  5\. Conclusions

This paper explores the privacy concerns in audio classification systems that
use features like MFCCs and spectrograms, which can inadvertently reveal
sensitive personal information beyond just speech content, such as speaker
identity, location, and other metadata. We identify key open challenges
including developing robust privacy evaluation mechanisms and navigating the
accuracy vs. privacy tradeoff. To address these issues, we propose a set of
generalizable privacy-preserving audio features and demonstrate their ability
to achieve comparable accuracy to prior work on the ESC-50 environmental sound
dataset, while mitigating privacy risks. Overall, this work represents an
important step towards enabling privacy-preserving audio sensing systems that
can unlock the potential of this rich data modality while effectively
safeguarding user privacy.

## Acknowledgments

This research was supported in part by NSF grants 2211302, 2211888, 2213636,
2105494, and US Army contract W911NF-17-2-0196. Any opinions, findings,
conclusions, or recommendations expressed in this paper are those of the
authors and do not necessarily reflect the views of the funding agencies.

## References

  * [1] Hanbin Zhang, Chen Song, Aosen Wang, Chenhan Xu, Dongmei Li, and Wenyao Xu.  Pdvocal: Towards privacy-preserving parkinson’s disease detection using non-speech body sounds.  In The 25th annual international conference on mobile computing and networking, pages 1–16, 2019.
  * [2] Monorama Swain, Aurobinda Routray, and Prithviraj Kabisatpathy.  Databases, features and classifiers for speech emotion recognition: a review.  International Journal of Speech Technology, 21:93–120, 2018.
  * [3] Jacob Leon Kröger, Otto Hans-Martin Lutz, and Philip Raschke.  Privacy implications of voice and speech analysis–information disclosure by inference.  Privacy and Identity Management. Data for Better Living: AI and Privacy: 14th IFIP WG 9.2, 9.6/11.7, 11.6/SIG 9.2. 2 International Summer School, Windisch, Switzerland, August 19–23, 2019, Revised Selected Papers 14, pages 242–258, 2020.
  * [4] Lea Schönherr, Maximilian Golla, Thorsten Eisenhofer, Jan Wiele, Dorothea Kolossa, and Thorsten Holz.  Exploring accidental triggers of smart speakers.  Computer Speech & Language, 73:101328, 2022.
  * [5] Stephen Xia and Xiaofan Jiang.  Pams: Improving privacy in audio-based mobile systems.  In Proceedings of the 2nd International Workshop on Challenges in Artificial Intelligence and Machine Learning for Internet of Things, pages 41–47, 2020.
  * [6] Francine Chen, John Adcock, and Shruti Krishnagiri.  Audio privacy: reducing speech intelligibility while preserving environmental sounds.  In Proceedings of the 16th ACM international conference on Multimedia, pages 733–736, 2008.
  * [7] Karol J Piczak.  Environmental sound classification with convolutional neural networks.  In 2015 IEEE 25th international workshop on machine learning for signal processing (MLSP), pages 1–6. IEEE, 2015.
  * [8] Forsad Al Hossain, Andrew A Lover, George A Corey, Nicholas G Reich, and Tauhidur Rahman.  Flusense: a contactless syndromic surveillance platform for influenza-like illness in hospital waiting areas.  Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 4(1):1–28, 2020.
  * [9] Daniyal Liaqat, Ebrahim Nemati, Mahbubur Rahman, and Jilong Kuang.  A method for preserving privacy during audio recordings by filtering speech.  In 2017 IEEE Life Sciences Conference (LSC), pages 79–82. IEEE, 2017.
  * [10] Wordless Sounds.  Wordless sounds: Robust speaker diarization using privacy-preserving audio representations, sree hari krishnan parthasarathi, hervé bourlard and daniel gatica-perez, idiap-rr-28-2012.
  * [11] Bhawana Chhaglani, Camellia Zakaria, Adam Lechowicz, Jeremy Gummeson, and Prashant Shenoy.  Flowsense: Monitoring airflow in building ventilation systems using audio sensing.  Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 6(1):1–26, 2022.
  * [12] Sudershan Boovaraghavan, Haozhe Zhou, Mayank Goel, and Yuvraj Agarwal.  Kirigami: Lightweight speech filtering for privacy-preserving activity recognition using audio.  Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 8(1):1–28, 2024.
  * [13] Herve Bourlard, Mathew Magimari, Daniel Gatica Perez, and Hari Krishna Parthasarathi.  Privacy sensitive audio features for speech/non-speech detection.  IEEE transaction on audio, speech and language processing.
  * [14] Sree Hari Krishnan Parthasarathi, Mathew Magimai.-Doss, Daniel Gatica-Perez, and Hervé Bourlard.  Speaker change detection with privacy-preserving audio cues.  In Proceedings of the 2009 international conference on Multimodal interfaces, pages 343–346, 2009.
  * [15] Karol J Piczak.  Esc: Dataset for environmental sound classification.  In Proceedings of the 23rd ACM international conference on Multimedia, pages 1015–1018, 2015.
  * [16] Gunnar Fant.  The source filter concept in voice production.  STL-QPSR, 1(1981):21–37, 1981.
  * [17] Chadawan Ittichaichareon, Siwat Suksri, and Thaweesak Yingthawornsuk.  Speech recognition using mfcc.  In International conference on computer graphics, simulation and modeling, volume 9, 2012.
  * [18] Wang Yutai, Li Bo, Jiang Xiaoqing, Liu Feng, and Wang Lihao.  Speaker recognition based on dynamic mfcc parameters.  In 2009 International Conference on Image Analysis and Signal Processing, pages 406–409. IEEE, 2009.
  * [19] Wenjie Mu, Bo Yin, Xianqing Huang, Jiali Xu, and Zehua Du.  Environmental sound classification using temporal-frequency attention based convolutional neural network.  Scientific Reports, 11(1):21552, 2021.
  * [20] Helin Wang, Yuexian Zou, Dading Chong, and Wenwu Wang.  Environmental sound classification with parallel temporal-spectral attention.  arXiv preprint arXiv:1912.06808, 2019.
  * [21] Avi Gazneli, Gadi Zimerman, Tal Ridnik, Gilad Sharir, and Asaf Noy.  End-to-end audio strikes back: Boosting augmentations towards an efficient audio classification network.  arXiv preprint arXiv:2204.11479, 2022.