# ENERGY-BASED MODELS FOR SPEECH SYNTHESIS

###### Abstract

Recently there has been a lot of interest in non-autoregressive (non-AR)
models for speech synthesis, such as FastSpeech 2 and diffusion models. Unlike
AR models, these models do not have autoregressive dependencies among outputs
which makes inference efficient. This paper expands the range of available
non-AR models with another member called energy-based models (EBMs). The paper
describes how noise contrastive estimation, which relies on the comparison
between positive and negative samples, can be used to train EBMs. It proposes
a number of strategies for generating effective negative samples, including
using high-performing AR models. It also describes how sampling from EBMs can
be performed using Langevin Markov Chain Monte-Carlo (MCMC). The use of
Langevin MCMC enables to draw connections between EBMs and currently popular
diffusion models. Experiments on LJSpeech dataset show that the proposed
approach offers improvements over Tacotron 2.

Index Terms—  speech synthesis, energy-based models, iterative inference

##  1 Introduction

Neural network based synthesis have made impressive improvements over
statistical speech synthesis. However, these deep learning based text-to-
speech (TTS) approaches often feature inconsistencies as did statistical
approaches. For example, auto-regressive (AR) models, such as Tacotron 2 [1],
Transformer-TTS [2], are almost exclusively trained using teacher forcing [3],
where reference rather than predicted values are fed back into the generative
process. Such a mismatch between training and inference causes inconsistency
called exposure bias [4], which may lead to poor generated speech quality
(e.g. repetition, skipping, and long pauses [5]). So far there have been a few
attempts to alleviate exposure bias, such as, scheduled sampling [6] and
attention mechanisms [7]. However, their effective application is complicated
due to a number of “training hacks” employed to ensure stable learning [8].

Recently, there has been interest in non-AR models, such as FastSpeech 2 [9]
and diffusion models [10]. These models generally do not use teacher forcing
as a part of their training and hence should be free of the aforementioned
inconsistencies. This paper describes another class of non-AR models called
energy-based models (EBMs) [11], which, as will be shown later, have
connections to currently popular diffusion models. Given a text, an EBM
defines an energy-function over all possible spoken realisations. Although it
is possible to formulate the conditional probability distribution of speech
given text for EBMs, the intractable normalisation term would make training
and inference approaches relying on the probability distribution infeasible.

Instead, training of EBMs can be performed using noise contrastive estimation
(NCE), which compares speech data (positive examples), which is assumed to
represent high quality speech, and imperfect speech data (negative examples).
The nature of imperfection, or negative examples, is crucial when training
EBMs. This paper describes a number of effective strategies to generate
negative examples, including by means of existing TTS models. Inference with
EBMs can be performed using Langevin Markov Chain Monte-Carlo (MCMC) [12, 13].
Given that a similar iterative algorithm is often used with diffusion models
(e.g., Grad-TTS [10]), this paper discusses connections between EBMs and
diffusion models.

This paper makes the following specific contributions:

  1. 1.

first energy-based text-to-speech model;

  2. 2.

a range of methods for generating effective negative samples to use in NCE and
elsewhere;

  3. 3.

link between diffusion models and energy-based models.

The rest of this paper is organized as follows. Section 2 describes energy-
based models (EBM), which includes inference, training and negative sampling
methods. Section 3 relates EBMs to filtering approaches and diffusion models.
Experimental results and discussion are presented in Section 4. Conclusions
drawn from this work and future research directions are presented in Section
5.

##  2 Energy-based models

Given a text sequence ${\bm{x}}$, an energy-based model (EBM) of speech
feature sequences ${\bm{Y}}$ (e.g. log-Mel spectrograms) can be defined
by111An alternative formulation would involve parameterising the gradient of
energy instead. Such an approach is possible due to existence of inference and
training approaches that rely only on the gradient of energy.

$p_{\bm{\theta}}(\bm{Y}|\bm{x})=\frac{1}{Z_{\bm{\theta}}(\bm{x})}\exp\left(-E_{\bm{\theta}}(\bm{x},\bm{Y})\right),$
(1)

where ${\bm{\theta}}$ are model parameters, $E_{\bm{\theta}}(\bm{x},\bm{Y})$
is an energy between text and speech, $Z_{\bm{\theta}}(\bm{x})$ is a
normalisation term. Unlike speech signal energies commonly used in models like
FastSpeech 2, EBM energies $E_{\bm{\theta}}(\bm{x},\bm{Y})$ reflect the
correspondence between text ${\bm{x}}$ and speech ${\bm{Y}}$ pairs. Better
matching pairs are expected to yield lower energies and vice versa. The
normalising term $Z_{\bm{\theta}}(\bm{x})$ is intractable to compute exactly.
Thus, only certain inference and parameter estimation approaches can be used
for EBMs.

###  2.1 Inference

For tasks where outputs are represented by discrete tokens (e.g. characters or
words), such as text generation [14] and speech recognition [15], EBMs are
commonly used to rerank hypotheses generated during beam search. In contrast,
for tasks where outputs are represented by continuous variables, such as
speech synthesis, EBMs can be used for updating hypotheses themselves. This
can be done using Langevin Markov Chain Monte-Carlo (MCMC). The Langevin MCMC
is an iterative process where, given an initial hypothesis ${\bm{Y}}^{(0)}$,
the next hypothesis is obtained by

${\bm{Y}}^{(N+1)}\\!\leftarrow\\!{\bm{Y}}^{(N)}\\!-\\!\lambda\left.\nabla_{\bm{Y}}E_{\bm{\theta}}(\bm{x},\bm{Y})\right|_{{\bm{Y}}={\bm{Y}}^{(N)}}+\sqrt{2\lambda}{\bm{Z}}^{(N)},$
(2)

where ${\bm{Z}}^{(N)}\sim{\mathcal{N}}({\bm{0}},\mu{\bm{I}})$, $\lambda$ is an
updating rate and $\mu$ is commonly set to 1. The need to specify initial
hypotheses ${\bm{Y}}^{(0)}$ offers a number of interesting options. In the
standard Langevin MCMC initial hypotheses are drawn from a simple prior
distribution, such as Gaussian [16]. However, more informative priors, such as
high-performing TTS models (e.g. Tacotron 2 and FastSpeech 2), can also be
explored.

###  2.2 Training

Since the normalising factor ${Z_{\bm{\theta}}(\bm{x})}$ is intractable,
approaches relying on $p_{\bm{\theta}}(\bm{Y}|\bm{x})$ can not be used with
EBMs. Furthermore, popular gradient-based MCMC approaches [17] can not be
applied with discrete input, as in this work, and Gibbs sampling [18] would be
too computationally expensive. Fortunately, noise contrastive estimation (NCE)
[19] provides a feasible solution to optimize energy functions. The NCE loss
function for EBMs in eq. (1) is given by

$\begin{split}\mathcal{L}_{\bm{\theta}}({\bm{x}},{\bm{Y}}^{+},{\bm{Y}}^{-})=&-\log\left(\frac{1}{1+\exp\left(E_{\bm{\theta}}(\bm{x},\bm{Y}^{+})\right)}\right)\\\
&-\log\left(\frac{1}{1+\exp\left(-E_{\bm{\theta}}(\bm{x},\bm{Y}^{-})\right)}\right)\end{split}$
(3)

where ${{\bm{Y}}^{+}}$ are called positive samples and ${{\bm{Y}}^{-}}$ are
called negative samples. According to eq. (3), energy functions are optimal
when high energy is assigned to negatives and low energy is assigned to
positives. Once trained, energy functions can be used for ranking hypotheses
generated by other models or inferring hypotheses using the Langevin MCMC in
eq. (2).

Positive samples in NCE are usually represented by reference sequences. On the
other hand, negative samples need to be designed. In text generation [14] it
is argued that negative examples with poor quality make the task of learning
the energy function easier which leads to poor quality energy functions. This
work proposes using pre-trained TTS models to generate high-quality negative
examples. Note that when a pre-trained model is used as a part of training
process then it is also possible to adopt it for initialising the Langevin
MCMC in eq. 2, which is expected to speed up inference and lead to higher
quality hypotheses. The simplest method to generate negative samples from pre-
trained TTS models would use hypotheses generated by those models directly.
Such approach may fail to work due to high level of similarity between high-
quality hypotheses and reference sequences. Other possible options include
applying random masking (RM) and SpecAugment [20] (i.e. time masking (TM),
frequency masking (FM) and time warping (TW)) to those hypotheses. The TM and
FM methods can be seen as specific cases of the RM method and may prove less
effective. For example, the FM may drastically affect pitch information by
masking a whole frequency range. The TW method is also expected to face
challenges as utterance-wide shortening/elongation of all sounds may be hard
to separate from reference sequences.

Fig. 1: Architecture of EBMs examined in this work (inspired by Transformer
TTS)

###  2.3 Architecture

The architecture of EBM explored in this work is inspired by Transformer TTS
[2] and is shown in Figure 1. The EBM in Fig. 1 consists of two blocks: energy
estimator (top) and feature enhancement (bottom). The goal of the energy
estimator is to derive utterance-level EBM energies
$E_{\bm{\theta}}({\bm{x}},{\bm{Y}})$ from the output of the feature
enhancement block. This work assumes that these energies can be derived from
frame-level EBM energies. As shown in Fig. 1, the energy estimator consists of
two key elements: frame-level EBM energy estimation and frame-level EBM energy
weighting. The latter element is motivated by an intuition that frame-level
EBM energies are unlikely to make equally important contributions. For
example, speech and non-speech frames will likely make different contributions
to the utterance-level EBM energy

$E_{\bm{\theta}}(\bm{x},\bm{Y})=\sum_{t=1}^{T}\alpha_{t}e_{t}$ (4)

where ${\alpha}_{1:T}$ is the sequence of attention weights generated by the
EBM energy weighting module, $e_{1:T}$ is the sequence of frame-level EBM
energies and $T$ is the number of frames. The attention weights are derived
from the frame-level EBM energies. The frame-level EBM energies ${e}_{1:T}$
are computed by

$e_{t}={\bm{a}}^{\top}{\bm{g}}_{t}+b$ (5)

where ${{\bm{g}}_{t}}$ is the output of the feature enhancement block, and
$\bm{a}$ and $b$ are parameters of the frame energy module. The goal of the
feature enhancement block is to enhance typically short-term spectral
information available in standard speech features, such as log-Mel
spectrograms, with more advanced acoustic and linguistic information. The
estimator is a transformer-based [2] model using text as the input to encoder
and and spectral features and the output of encoder as the input to decoder.
The output of decoder after linear transformation, ${{\bm{g}}_{t}}$, is passed
to the energy estimator block. Note that the decoder does not use masking to
constrain the underlying attention mechanism from attending over previous
spectral features, which makes ${{\bm{g}}_{t}}$ a function of entire text and
spectral feature sequences.

##  3 RELATED WORK

EBMs have been applied in a wider range of domains, e.g. natural language
Processing [14, 21] and automatice speech recognition [15]. The EBM proposed
in this work can be related to a number of previously proposed approaches in
TTS. The use of hypotheses generated by pre-trained TTS models as a part of
training and inference allows to connect this EBM to post-filtering methods.
Statistical post-filtering approaches, such as [22], aim to address over-
smoothing in hypotheses generated by statistical speech synthesis models.
However, these approaches suffer from difficulties in accurately modelling
probability density functions of the underlying speech parameterisations.
Recently, there has also been interest in deep learning based post-filtering
approaches. In [23], frequency band specific generative adversarial networks
(GAN) were trained to improve the quality of hypotheses generated by deep
learning based speech synthesis models. However, this approach assumes
independence among frequency bands which may lead to suboptimal results.

More recently, there has been a lot of interest in diffusion-based TTS models
[10], which have been extended to audio synthesis [24] and singing voice
synthesis [25]. In these models training (forward) and inference (reverse)
processes iteratively build a connection between data and noise. Although
seemingly different, such diffusion models and the EBM proposed in this work
have clear connections. Consider, for example, the iterative inference process
used by one of those diffusion models [26]

${\bm{Y}}^{(N+1)}\\!\leftarrow\\!\frac{1}{\sqrt{1\\!-\\!\lambda_{N}}}({\bm{Y}}^{(N)}\\!+\\!\lambda_{N}S_{\bm{\theta}}(\bm{x},\bm{Y}^{(N)},{N}))+\sqrt{\lambda_{N}}{\bm{Z}}^{(N)},$
(6)

Compared to the Langevin MCMC in eq. (2) the key difference stems from
modelling iteration, $N$, specific $S_{\bm{\theta}}(\bm{x},\bm{Y}^{(N)},{N})$
score (gradient of log-likelihood) rather than iteration independent
$\nabla_{{\bm{Y}}^{(N)}}E_{\bm{\theta}}(\bm{x},\bm{Y}^{(N)})$ score in the EBM
given by eq. (1). In addition, score matching approaches to training diffusion
models can also be adopted with EBMs [26], which further strengthens the
connection between these models.

##  4 Experiments

###  4.1 Experimental setup

####  4.1.1 Dataset

The dataset used in this work is LJSpeech [27], which includes 13,100 audio
clips totalling approximately 24 hours from one female speaker. The dataset is
split randomly into training (10,000 clips), validation (1800 clips) and test
(1300 clips) sets. Objective evaluation is performed over the entire test set
whilst subjective evaluation is performed over 100 randomly chosen test set
clips. Front-end pre-processing of audio follows the open-source
implementation available as a part of NVIDIA’s Tacotron 2.
222https://github.com/NVIDIA/tacotron2

####  4.1.2 Models

The pre-trained TTS model providing hypothese for training and inference is
Tacotron 2 [1]. The open-source implementation of NVIDIA using default
configuration was adopted in this work. The structure of EBM follows the
corresponding elements of Transformer-TTS [2] available through an open-source
implementation 333https://github.com/soobinseo/Transformer-TTS except that: 1)
positional and character embeddings are 256-dimensional; 2) two EBMs with
different dimensions of hidden features, 128 and 256 respectively, are
explored in the study. Utterance-level energy is predicted by frame-level EBM
energy prediction module, which consists of two 512-dimensional fully-
connected layers. Although it is possible to backpropagate gradients through
the pre-trained TTS model, for simplicity this was not explored in this work.
Both EBMs are trained for 125K iterations using Adam optimizer using batch
size of 16 and a constant learning rate of $1\times 10^{-4}$ on a single
NVIDIA 3090 GPU. The number of parameters of these 2 EBMs and Tacotron 2 are
shown in Table 1. We use an open-source
implementation444https://github.com/NVIDIA/waveglow of the WaveGlow [28]
vocoder and adopt its default settings.

####  4.1.3 Evaluation

Mel cepstral distortion (MCD) , F0 frame error (FFE) and log-scale F0 root
mean square error (log F0 RMSE) are adopted as objective metrics in this work.
The MCD metric calculates distance between cepstral coefficient sequences of
different lengths on the Mel frequency scale. The FFE metric measures
discrepancy of fundamental frequency (F0) between synthesized and reference
waveforms. Before objective calculating, dynamic time warping (DTW) is used to
align the predicted mel-spectrogram and the reference. FAIRSEQ ${S^{2}}$
toolkit [29] is used to compute MCD and FFE scores. Mean opinion score (MOS)
evaluation is conducted to evaluate speech naturalness by scoring each speech
sample on a scale between 1 to 5 with 1 point intervals. Waveforms synthesized
by 3 models compared in this work are mixed with test set waveforms. Each
audio is listened to by 5 listeners, who are native English speakers, on the
Amazon Mechanical Turk platform.

Model | MCD $\downarrow$ | FFE $\downarrow$ | $\log f_{\mathrm{o}}\downarrow$ | Parameters
---|---|---|---|---
Tacotron 2 | 4.218 | 47.31% | 0.292 | 28.19M
EBM(1) (small) | 4.163 | 47.06% | 0.289 | 2.30M
EBM(1) (large) | 4.178 | 47.05% | 0.291 | 7.64M
Table 1: Comparison between Tacotron 2 and two EBMs utilising Tacotron 2
hypotheses as negative samples

###  4.2 Negative sampling methods

Table 1 compares Tacotron 2 and two initial EBMs. These EBMs were trained
using Tacotron 2 generated hypotheses as negative samples and a single step
($N=1$) Langevin MCMC, where $\mu$ was set to $0$ for simplicity and Adam
rather than gradient descent update rule was adopted. Both large (256 hidden
features) and small (128 hidden features) EBMs perform slightly better than
the baseline.

Table 2 summarises performance of the alternative negative sampling methods
(see Sec. 2.2) with the large EBM, where the simplified Langevin MCMC was run
for $N=100$ steps. Many of these methods show significantly better performance
than the baseline. Comparing between compressed and stretched spectral
features suggest no strong preference for any particular method of time
warping (TW). The method of 5% time masking (TM) achieves lower MCD and FFE
compared to other time masking methods, while the trend is opposite for
frequency masking (FM), where higher percentage points (15%) appear to be
yielding better MCD results and worse FFE results. The likely reason is the
negative interaction between FFE and frequency masking.

Condition | MCD $\downarrow$ | FFE $\downarrow$ | $\log f_{\mathrm{o}}\downarrow$ |
---|---|---|---|---
TM: | 5% | 4.149 | 46.89% | 0.290 |
| 10% | 4.166 | 46.98% | 0.284 |
| 15% | 4.161 | 47.30% | 0.285 |
FM: | 5% | 4.166 | 46.88% | 0.292 |
| 10% | 4.138 | 47.27% | 0.286 |
| 15% | 4.097 | 47.35% | 0.284 |
TW: | 1.2 (compress) | 4.134 | 47.05% | 0.291 |
| 1.1 (compress) | 4.170 | 47.03% | 0.291 |
| 0.9 (stretch) | 4.168 | 47.28% | 0.284 |
| 0.8 (stretch) | 4.159 | 47.29% | 0.286 |
RM: | 25% | $\bm{3.943}$ | $\bm{46.16\%}$ | 0.282 |
| 30% | 4.013 | 46.57% | $\bm{0.280}$ |
| Baseline | 4.218 | 47.31% | 0.292 |
Table 2: Negative sampling methods (95% confidence intervals)

Table 3 investigates the impact of the Langevin MCMC steps on the performance
of the best system in Table 2.

Step | MCD$\downarrow$ | FFE$\downarrow$ | $\log f_{\mathrm{o}}\downarrow$
---|---|---|---
0 | 4.218 | 47.31% | 0.292
1 | 4.217 | 47.31% | 0.292
100 | 3.943 | 46.16% | 0.282
300 | $\bm{3.937}$ | $\bm{45.85\%}$ | $\bm{0.276}$
Table 3: Simplified Langevin MCMC (95% confidence intervals)

As the number of steps increases, the EBM applying 25% random masking to
negative samples performs better and better.

Although random masking appears to be the most effective negative sampling
method, the other masking methods may bring additional complementary
information. Table 4 summarises performance of different combination
approaches involving the RM 30% EBM in Table 2.

RM | TM | FM | TW | MCD $\downarrow$ | FFE $\downarrow$ | $\log f_{\mathrm{o}}\downarrow$
---|---|---|---|---|---|---
30% | 5% | 5% | 1.2
$\checkmark$ |  |  |  | 4.013 | 46.57% | 0.280
$\checkmark$ | $\checkmark$ |  |  | 3.958 | 46.31% | 0.276
$\checkmark$ |  | $\checkmark$ |  | 3.997 | 45.63% | 0.278
$\checkmark$ |  |  | $\checkmark$ | 3.927 | 45.98% | 0.267
$\checkmark$ | $\checkmark$ | $\checkmark$ | $\checkmark$ | $\bm{3.882}$ | $\bm{45.36\%}$ | $\bm{0.258}$
Table 4: Combination of negative sampling methods (95% confidence intervals)

All combinations examined perform better than using only random masking. The
EBM making use of all masking methods performs the best.

###  4.3 Subjective evaluation

To solicit subjective assessment, a range of listening tests were conducted
(see Sec. 4.1.3). Table 5 shows that the proposed EBM shows generally better
MOS scores than the baseline Tacotron 2.

Method | MOS
---|---
Ground Truth |  4.53$\pm$0.05
Ground Truth (log-Mel + WaveGlow) |  4.39$\pm$0.07
Tacotron 2 (log-Mel + WaveGlow) |  3.77$\pm$0.11
EBM (log-Mel + WaveGlow) |  3.84$\pm$0.13
Table 5: Subjective evaluation

Furthermore, the detailed breakdown of MOS score counts in Table 6 shows that
the EBM significantly reduced the number of MOS scores 2 (-2) and 3 (-27) and
increase the number of MOS scores 4 (+26) and 5 (+3).

MOS | 1 | 2 | 3 | 4 | 5
---|---|---|---|---|---
Tacotron 2 | 0 | 3 | 121 | 344 | 15
EBM | 0 | 1 | 94 | 370 | 18
Table 6: MOS score counts

##  5 Conclusions

This paper proposed a new class of non-autoregressive (non-AR) text-to-speech
(TTS) models called energy-based models (EBM). As an example, it shows how
powerful forms of EBMs can be designed by adopting architectures of state-of-
the-art AR models like Transformer TTS. Although training models like EBMs is
more complicated due to the intractability of normalisation terms, a range of
training approaches is available. This paper describes how one such approach
called noise contrastive estimation (NCE) can be adopted for training. As the
NCE critically relies on the quality of negative samples used to contrast
reference speech feature sequences, this paper proposed and evaluated a wide
range of negative sampling methods. It found that random masking is the single
best method but the combination of all proposed methods yielded the best
performance. The paper also shows how sampling from EBMs can be performed by
means of Langevin Markov Chain Monte-Carlo (MCMC). Since Langevin MCMC is
closely linked with an iterative method used by popular diffusion models, the
paper discusses similarities between EBMs and diffusion models. The paper
concludes by subjective evaluation and finds that the proposed model provides
improvements over Tacotron 2. Future work with EBMs will explore score
parameterisation and the use of alternative TTS models for architectural and
negative sampling choices.

## References

  * [1] J. Shen et al.,  “Natural TTS synthesis by conditioning wavenet on mel spectrogram predictions,”  in ICASSP, 2018, pp. 4779–4783.
  * [2] N. Li et al.,  “Neural speech synthesis with transformer network,”  in AAAI, 2019, vol. 33, pp. 6706–6713.
  * [3] R. J. Williams et al.,  “A learning algorithm for continually running fully recurrent neural networks,”  Neural computation, vol. 1, no. 2, pp. 270–280, 1989.
  * [4] Marc’Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba,  “Sequence level training with recurrent neural networks,”  CoRR, vol. abs/1511.06732, 2015.
  * [5] J. Shen et al.,  “Non-attentive tacotron: Robust and controllable neural tts synthesis including unsupervised duration modeling,”  arXiv preprint arXiv:2010.04301, 2020.
  * [6] S. Bengio et al.,  “Scheduled sampling for sequence prediction with recurrent neural networks,”  Advances in neural information processing systems, vol. 28, 2015\.
  * [7] Mutian He, Yan Deng, and Lei He,  “Robust sequence-to-sequence acoustic modeling with stepwise monotonic attention for neural tts,”  in Interspeech, 2019.
  * [8] E. Battenberg et al.,  “Location-relative attention mechanisms for robust long-form speech synthesis,”  in ICASSP, 2020, pp. 6194–6198.
  * [9] Y. Ren et al.,  “Fastspeech 2: Fast and high-quality end-to-end text to speech,”  arXiv preprint arXiv:2006.04558, 2020.
  * [10] V. Popov et al.,  “Grad-tts: A diffusion probabilistic model for text-to-speech,”  in ICML. PMLR, 2021, pp. 8599–8608.
  * [11] Y. LeCun et al.,  “A tutorial on energy-based learning,”  Predicting structured data, vol. 1, no. 0, 2006.
  * [12] Giorgio Parisi,  “Correlation functions and computer simulations,”  Nuclear Physics B, vol. 180, no. 3, pp. 378–384, 1981.
  * [13] U. Grenander et al.,  “Representations of knowledge in complex systems,”  Journal of the Royal Statistical Society: Series B (Methodological), vol. 56, no. 4, pp. 549–581, 1994.
  * [14] A. Bakhtin et al.,  “Residual energy-based models for text.,”  J. Mach. Learn. Res., vol. 22, pp. 40–1, 2021.
  * [15] Q. Li et al.,  “Residual energy-based models for end-to-end speech recognition,”  in Interspeech, 2021.
  * [16] Ferenc Huszár,  “How (not) to train your generative model: Scheduled sampling, likelihood, adversary?,”  arXiv preprint arXiv:1511.05101, 2015.
  * [17] Y. Du et al.,  “Implicit generation and generalization in energy-based models,”  arXiv preprint arXiv:1903.08689, 2019.
  * [18] M. Welling et al.,  “Exponential family harmoniums with an application to information retrieval,”  Advances in neural information processing systems, vol. 17, 2004\.
  * [19] Z. Ma et al.,  “Noise contrastive estimation and negative sampling for conditional models: Consistency and statistical efficiency,”  in CEMNLP, 2018.
  * [20] D. Park et al.,  “Specaugment: A simple data augmentation method for automatic speech recognition,”  arXiv preprint arXiv:1904.08779, 2019.
  * [21] K. Clark et al.,  “Pre-training transformers as energy-based cloze models,”  arXiv preprint arXiv:2012.08561, 2020.
  * [22] S. Takamichi et al.,  “A postfilter to modify the modulation spectrum in hmm-based speech synthesis,”  ICASSP, pp. 290–294, 2014.
  * [23] T. Kaneko et al.,  “Generative adversarial network-based postfilter for stft spectrograms.,”  in Interspeech, 2017, pp. 3389–3393.
  * [24] S. Pascual et al.,  “Full-band general audio synthesis with score-based diffusion,”  in ICASSP. IEEE, 2023, pp. 1–5.
  * [25] H. Xue et al.,  “Learn2sing 2.0: Diffusion and mutual information-based target speaker svs by learning from singing teacher,”  arXiv preprint arXiv:2203.16408, 2022.
  * [26] Y. Song et al.,  “Score-based generative modeling through stochastic differential equations,”  arXiv preprint arXiv:2011.13456, 2020.
  * [27] Keith Ito and Linda Johnson,  “The lj speech dataset,” https://keithito.com/LJ-Speech-Dataset/, 2017.
  * [28] R. Prenger et al.,  “Waveglow: A flow-based generative network for speech synthesis,”  in ICASSP. IEEE, 2019, pp. 3617–3621.
  * [29] C. Wang et al.,  “fairseq s^ 2: A scalable and integrable speech synthesis toolkit,”  arXiv preprint arXiv:2109.06912, 2021.


[RO]Anomaly Detection on Backdoored Networks

[LE] Anonymous submission
[LE] Alexander Unnervik and Sébastien Marcel

Backdoor attacks allow an attacker to embed functionality jeopardizing proper behavior of any algorithm, machine learning or not. This hidden functionality can remain inactive for normal use of the algorithm until activated by the attacker. Given how stealthy backdoor attacks are, consequences of these backdoors could be disastrous if such networks were to be deployed for applications as critical as border or access control.
In this paper, we propose a novel backdoored network detection method based on the principle of anomaly detection, involving access to the clean part of the training data and the trained network. We highlight its promising potential when considering various triggers, locations and identity pairs, without the need to make any assumptions on the nature of the backdoor and its setup. We test our method on a novel dataset of backdoored networks and report detectability results with perfect scores.

Backdoor attack, trojan attack, anomaly detection, CNN, face recognition, biometrics, security.

§ INTRODUCTION

[Backdoor implementation.]

[Backdoor activation; test samples on the left, classification output on the right.]

The two steps involved in the backdoor attack: (a) the backdoor implementation typically done by poisoning the dataset and (b) the backdoor activation, exposing the trigger to the network leading to the misclassification.

A larger number of tasks are nowadays performed in an automated fashion, including biometric ones. Borders and access controls are among the most notable areas in which automated biometric systems are being used. A reliable and well functioning biometric system is thus necessary for national safety and proper access management for all kinds of institutions.
Simultaneously, machine learning progress promotes reusability where an increasing number of pretrained models are hosted and freely available in model-zoos or offered to be trained using machine-learning as a service (MLaaS). A way of verifying the integrity of such pretrained models becomes necessary as a vulnerable system could have severe consequences.
Historically, there have been numerous software and hardware vulnerabilities including trojans and backdoors. While they have both existed for decades, their emergence in the machine learning field is still relatively recent and both their dangerous potential and detectability are equally newly actively researched.
Backdoor attacks require two steps to be performed: a first step is the implementation of the vulnerability where the network undergoes a modification in order to learn a specific pattern or image sample. The network learns to associate this trigger (designed and controlled by the attacker) with a given behavior, such as a given classification. The implementation can be done using various techniques with one example being the poisoning of a training set, with an overview presented in figure <ref>. Once the backdoor is implemented, it can be activated at will by the attacker, only requiring the predefined trigger to be provided to the machine learning model for it to exhibit the predefined backdoor behavior, as displayed in figure <ref>.
Given the serious potential national security implications and threats, it is no surprise that this topic has also caught the attention of the U.S. Army Research Office (ARO) who in partnership with the Intelligence Advanced Research Projects Activity (IARPA) “seeks research and development of technology and techniques for detections of such backdoors in artificial intelligence” as per the announcement <cit.>.

There have been numerous studies on how backdoor attacks can be performed, usually involving dataset poisoning such as in <cit.>. The attack can also be carried out using so called clean-label poisoning as described in <cit.> where instead of using a typical trigger, the samples are slightly distorted, enough to shift the decision boundaries and cause misclassifications in some instances. Similar to backdoor attacks, Liu et. al. introduced the notion of Trojan attacks <cit.> where the trigger is the result of an optimization similar to adversarial attacks, but where a selected neuron in the network is targeted with increasing the sensitivity to the found trigger. The trigger itself can also be blended with the original image rather than strictly overlaid, such as in <cit.>. While most attacks are studied in the digital domain (where the triggers are digitally added to the images), the physical world has also been studied where the backdoor can be implemented digitally and activated in the physical world such as in <cit.>. Alternatively, the backdoor can be both implemented and activated in the physical world such as in <cit.>. This last paper also highlights the effects of trigger placement in the success of the backdoor attack for face recognition, linking the success with the proximity of the trigger to the center of the face.

When it comes to detection of and defense against backdoor attacks, multiple methods have been proposed requiring various priors and assumptions. Techniques such as <cit.>, <cit.> and <cit.> rely on the availability of the entire training set (including the poison samples) in addition to priors such as a maximum poison rate. <cit.> evaluates the activations of the network on the training set including the poisoned samples used to introduce the backdoor to identify multiple spectral signatures for suspected classes. <cit.> uses a combination of ICA and k-means to identify which samples belong to which activation cluster and identify the ones leading to a possible backdoor. Both <cit.> and <cit.> then follow-up with the result of their method to weed out the discovered poisoned samples to retrain a sanitized network devoid of any backdoor. In <cit.>, the authors propose a method using both a portion of the training set and the model itself to reverse engineer the trigger and identify the classes connected by the backdoor attack, using an outlier detection on the trigger size, where the smallest trigger serves as an indicator of a possible backdoor being present. Lastly, in <cit.>, the authors propose a meta-learning approach where a binary classifier is trained on a set of clean and backdoored networks with varying backdoor parameters.

In this paper we propose a novel method which requires no assumption on the poison rate of the dataset unlike previous methods relying on the access of the training set, nor does it rely on the availability of any poison samples either or even backdoored networks. Additionally, we make no assumption on the size, location or type of trigger, nor whether it is blended or applied on the image. Our proposed method relies on outlier detection by estimating the distribution of the weights of a machine-learning model trained on a given dataset, compatible with an outsourced training task to a third party or downloaded online. We test our method on backdoored networks involving different trigger sizes and type (synthetic and organic), in addition to different trigger placement strategies and backdoored identity pairs and find that our systematic approach is able to detect all backdoored networks.

§ PROPOSED METHOD

Illustration of the detector in the proposed method.

When training neural network architectures on a given dataset, pseudo-randomness (e.g. in the shuffling of the dataset) influences the convergence of the learned weights and leads to different end results.
We hypothesize that we can model the distribution of the weights of non-backdoored neural networks trained on a clean dataset with a parameterized statistical model such that we can consider the detection of backdoored neural networks as an anomaly detection problem, allowing us to detect when a network of the same architecture is trained on a modified version of the dataset which could introduce a backdoor.
Our proposed method, represented in figure <ref>, consists in:
(1) Training a set of $M$ networks with weights $\{\Theta_1,...,\Theta_M\}$ on the clean dataset.
(2) Selecting one or multiple layers from these networks. We provide here the details for a single layer but the method can be expanded to multiple layers. In the case of a fully-connected layer the weights are a matrix $\pmb{\theta} \in \mathbb{R}^{R \times C}$, in which case the weights can be interpreted both as $C$ row vectors of length $R$ as can be seen in figure <ref>, which we will hereafter refer to as forward, and $R$ column vectors of length $C$, which we will hereafter refer to as backward[This naming is because column vectors each apply to a given neuron of the previous layer and propagates to the entirety of the next layer (hence a forward propagation), while the backward naming offers the reciprocal interpretation.] (to guarantee a vector permutation invariant approach capable of handling the pseudo-random order of convergence of the row and column weights). In the case of a convolution layer, the kernels may be flattened. Hence, for an e.g. fully connected layer from $M$ networks the total weights are $X = [\pmb{\theta}_1..\pmb{\theta}_M]^T \in \mathbb{R}^{(M \times C) \times R}$. We provide hereafter the mathematical analysis for the forward approach (ignoring the subscript 'f'), but it is reciprocally applicable to the backward approach, by considering $X_b = [\pmb{\theta}^T_1..\pmb{\theta}^T_M]^T \in \mathbb{R}^{(M \times R) \times C}$ instead.
(3) Principle component analysis (PCA), a dimensionality reduction algorithm, is then performed on the selected layers' weights as in practice $R$ and $C$ can be large. This yields $Z \in \mathbb{R}^{(M \times C) \times B}$ with $B < R$.
(4) $Z$ is then used to estimate the typical distribution of weights for networks known to be free of any backdoor, using a Gaussian mixture model (GMM). A GMM consists of $N_G$ weighted Gaussian components each of dimensionality $B$ and defined by $\Lambda_i = \{\omega_{i},\pmb{\mu}_{i},\Sigma_{i}\}_{i=\{1..N_G\}}$ where $\pmb{\mu}_i$ is the mean vector, $\Sigma_i$ the covariance matrix, $\omega_i$ the weight, and obtained through expectation maximization.
(5) This GMM can then be used on a given weight vector $\pmb{x}_j$ to compute the probability of this feature vector under the model $\Lambda$ as $P(\pmb{x}_j|\Lambda) = \frac{1}{N_G}\sum_{i=1}^{i=N_G}\omega_i\mathcal{N}(\pmb{x}_j|\Lambda_i)$ where $\mathcal{N}(\cdot|\Lambda_i)$ is the normal distribution.
The overall probability of a network with $N_f$ features from $\Theta$ given a model $\Lambda$ is given by $\prod_{j=1}^{j=N_f}P(\pmb{x}_j|\Lambda)$ assuming independence of the feature vectors in the layer.
The anomaly detection task then requires thresholding this overall probability. If it is above the threshold, the network is then considered clean. Otherwise, it is considered backdoored.
Code necessary for the reproduction of the results in this paper are released jointly with this paper[<https://gitlab.idiap.ch/bob/bob.paper.backdoors_anomaly_detection.biosig2022>].

§ EXPERIMENTAL SETUP

As there is no proper benchmark to evaluate backdoor detection methods in face recognition, a custom dataset of $30$ clean networks and $22$ backdoored networks were trained following various backdoor parameters. Each backdoored network used a random identity pair, trigger in addition to trigger placement strategy. The dataset is split in two categories: triggers (with 11 networks) and locations (with 11 networks), each with a focus on varying their respective parameters (all involving random identity pairs for the backdoor). A sample of the images used to train the backdoored networks can be seen in figure <ref>.

The following experimental setup was defined to create the dataset of backdoored networks:

Dataset Casia-Webface <cit.> was used. It contains 10,575 identities with a total of 494,414 color images of resolution 250 by 250 in JPEG format.
The dataset is not balanced but was used due to its large number of samples, its sufficient alignment, the grouping per identities and the availability of pretrained models on this dataset in PyTorch. As the dataset does not come with predefined train and test splits, random class splits were used to generate consistent splits across all classes. The exact ratios used were $70\% - 30\%$ for train-test respectively and randomly re-sampled for every experiment.

Architecture The architecture chosen is FaceNet <cit.>. Implementations can be found online both for TensorFlow[<https://github.com/davidsandberg/facenet>] and for PyTorch[<https://github.com/timesler/facenet-pytorch>]. FaceNet was specifically chosen both for its availability in PyTorch and for having pretrained weights with Casia-Webface. The pretrained weights were used as initialization weights when performing fine-tuning on Casia-WebFace both with and without poisoned samples.

Backdoor The backdoor is a one-to-one with a random impostor-victim pair allowing for the impersonation of one specific person using a digital trigger. The high-level process of both the implementation and activation of the backdoor are shown in figure <ref>.

Poisoning The poisoning process involves four main steps shown in figure <ref> (1) the selection of an impostor from the original dataset (involving copying of the samples) (2) the poisoning by application of a trigger on the copied samples (3) the selection of a victim used to relabel the poisoned samples and (4) appending the obtained poisoned samples to the original dataset.

[Various backdoor triggers]

[Weights from clean and backdoored networks.]

In figure <ref>, various triggers are shown, used on various identity pairs (the small eyebrow, the large flower, the small white square and the large checkerboard respectively). In figure <ref> a zoomed-in part of the t-SNE plots is shown of all weights of the last_linear layer of FaceNet in their forward interpretation: $10$ networks were selected from the vanilla (a.k.a clean) networks pool and $10$ from the dataset of backdoored networks trained using various triggers.

Training The training was performed as for a typical classification, which in the context of biometrics imply a closed-set identification task. The Cross-Entropy loss function was used with increased weights on the impostor and victim classes and the ADAM optimizer was selected. We performed the fine-tuning for the clean and the backdoored networks on the last_linear and last_bn layers as the feature extractor is usually best left untouched at the risk of degrading the good generalization performance. Additionally, many biometric tasks are done in open set implying the absence of a classification layer at the end, hence why the focus is on those layers.

§ RESULTS

The selected layer for our analysis is the last_linear with weights $\pmb{\theta} \in \mathbb{R}^{512 \times 1792}$. In the forward interpretation we consider it as $1792$ vectors of size $512$ while in the backward interpretation we consider the parameters as $512$ vectors of size $1792$. Those selected parameters concatenated from 18 of the clean reference networks first undergo a dimensionality reduction, involving a PCA on the parameters. With the goal of retaining $95\%$ of the cumulative variance, 76 and 297 PCA components are required for the forward and backward interpretation respectively, leading to $18 \times 1792$ vectors of size $76$ in forward and $18 \times 512$ vectors of size $297$ in backward.
The resulting t-SNE visualization (zoomed in for better viewing) is shown in figure <ref> for the triggers dataset; it can be verified that the parameters for the backdoored networks seem to separate at the edge from the clean parameters.
The optimal number of GMM components can then be determined on the reduced set of parameters. The clusters which can be seen in figure <ref> suggest that there is a large number of them. An empirical study of the fit of various number of components of the GMM can help in qualitatively evaluating them. For this purpose, the Akaike Information Criterion (AIC) was used, where a score is provided to assess the fit of a GMM on a set of data with the purpose of minimizing the score to identify the optimal number of components. The optimal number of components was identified to be $1792$ and $512$ for forward and backward respectively, out of the following evaluated number of components: [1, 2, 5, 10, 20, 50, 100, 200, 512, 1000, 1792, 3000].

[Results on the locations dataset.]

[Results on the triggers dataset.]

AUC for each layer interpretation, as a function of the number of GMM components both for the locations and triggers backdoored datasets in <ref> and <ref> respectively, using the last_linear layer.

It can be observed that among the evaluated number of GMM components, the optimal number is exactly the number of vectors in the parameters layers (in accordance with the forward and backward interpretation). This suggests that there are indeed in practice a similar distribution of parameters across training experiments, despite using different seeds, leading to the convergence of parameters.

With the GMM fitted and the optimal number of components identified, we test the detector on the remaining unused 12 clean networks and both of the backdoored datasets. We retrieve the scores of each of the networks across both weights representations and evaluate the performance by calculating the the area under the ROC (AUC). The AUC gives one number, summarizing the performance across all thresholds on an ROC. We provide the results of the AUC for all number of GMMs for completeness in figure <ref>. The AUC was used over estimating the threshold due to the need to otherwise split the datasets in three rather than two and it would not be ideally represented with our limited number of networks.
It can be verified that the AUC with our optimal number of gaussians is maximal, with 1.0, across both datasets and both for the forward and backward representations. This implies that there is a perfect threshold on the clean and backdoored networks from the tested networks. Interestingly, the detection in the forward representation seems to work equally well with $1000$ components as with $1792$, which is in line with the AIC making it the second best number of components (not shown). A more in-depth testing may further separate both number of GMM components. On the ability to use this detector in practice, a theoretical false rejection rate (FRR) may be chosen on the clean networks alone, leading to a specific threshold on which performance can be measured.

There are however some limitations to this approach. It requires the training of a certain number of clean networks which can be quite compute intensive. Additionally, one may expect more networks to allow for a better parameters distribution estimation but it may be difficult to estimate a minimum number of networks which may be different on a case-by-case. Lastly, the selection of the layer was chosen in this paper to highlight the feasibility of the method, but in reality the layer is unknown and this method may need to be performed on multiple suspected layers, leading to the use of multiple independent PCA steps and the fitting of a larger number of GMMs.

On the visualization in figure <ref>, empirically the method works better than what the figure may suggest which could indicate that the visualization may be simplifying to a smaller variability between clean and backdoored networks than what actually is. The t-SNE is a best effort optimization algorithm, not an accurate projection. It attempts at minimizing total loss but it does not allow for absolute distance comparisons. Other projections would also suffer from an inability to accurately visualize relative elements.

§ CONCLUSION

The usage of pretrained models hosted online or provided by MLaaS should be possible without needing to blindly trust the hosting entity and service provider: access to convenience features and third parties should not come at the expense of security and we have been able to evaluate a methodology here which allows in the aforementioned conditions to be able to assess whether the networks do in fact come without any hidden backdoor. The pipeline can be automated leading to the selection of the optimal number of PCA components, GMM components, threshold and later be used as binary classification for outlier detection with optimal results with minimal assumptions on the backdoor attack.

This work may also be extended by evaluating detection potential when using another dataset for the same task, possibly leading to the removal of the requirement regarding availability of the clean training set. Additionally, by exploring the optimal method of combining scores when evaluating multiple layers simultaneously. Both of these implementations can improve the generalization capabilities of the methodology and reduce further the number of priors and assumptions if they lead to promising results. The method described in this paper is also expected to perform well in the case of Trajan attacks as the selection of the specific neuron to implement the backdoor will possibly lead to a more significant outlier, which is expected to be easier to identify.

§ ACKNOWLEDGEMENT

This work is supported by the TReSPAsS-ETN project funded from the European Union's Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No.860813.