# FlowVocoder: A small Footprint Neural Vocoder based Normalizing Flow for
Speech Synthesis

###### Abstract

Recently, autoregressive neural vocoders have provided remarkable performance
in generating high-fidelity speech and have been able to produce synthetic
speech in real-time. However, autoregressive neural vocoders such as WaveFlow
are capable of modeling waveform signals from mel-spectrogram, its number of
parameters is significant to deploy on edge devices. Though NanoFlow, which
has a small number of parameters, is a state-of-the-art autoregressive neural
vocoder, the performance of NanoFlow is marginally lower than WaveFlow.
Therefore, we propose a new type of autoregressive neural vocoder called
FlowVocoder, which has a small memory footprint and is capable of generating
high-fidelity audio in real-time. Our proposed model improves the density
estimation of flow blocks by utilizing a mixture of Cumulative Distribution
Functions (CDF)for bipartite transformation. Hence, the proposed model is
capable of modeling waveform signals, while its memory footprint is much
smaller than WaveFlow. As shown in experiments, FlowVocoder achieves
competitive results with baseline methods in terms of both subjective and
objective evaluation, also, it is more suitable for real-time text-to-speech
applications.

Index Terms: neural vocoders, text-to-speech, speech synthesis, normalizing
flow.

##  1 Introduction

Speech synthesis has numerous practical applications in enhancing user
experiences such as reading online newspapers and voice assistants.
Technically, there are two main stages to produce synthetic speech: generating
acoustic features and transforming acoustic features to waveform signals. The
second stage usually refers to the vocoder stage. Traditional vocoder
approaches generate audio samples using either the spectral density model [1]
or the basis of the source-filter model [2]. However, the fidelity of
generated speech from those approaches is low and sounds like a robotic voice.
Therefore, in recent years, a lot of studies in neural vocoders have provided
remarkable performance in producing speech signals from Mel-spectrogram. They
are classified into two categories: autoregressive and non-autoregressive
vocoders. Regarding high-fidelity, autoregressive vocoders outperform non-
autoregressive vocoders. On the contrary, the latter run much faster and are
used in real world applications.

For non-autoregressive vocoders, the Generative Adversarial Network
(GANs)-based method is the most successful neural vocoders that provide high-
fidelity speech [3, 4, 5, 6, 7, 8, 9, 10]. Although GANs-based vocoders are
able to generate synthetic speech in real-time and have a small memory
footprint, they are still fragile to train due to adversarial training. To
overcome this training issue, MelGAN [3] and HiFi-GAN [4] utilize an auxiliary
loss called features matching between real and generated data to facilitate
adversarial training. Moreover, only relying on adversarial loss causes
degradation of audio quality, so the mean square error of Mel-spectrogram [4]
opts to deal with the degradation of synthetic speech. While Multiband MelGAN
[8] replaces the naive features matching with the multi-resolution STFT to
better estimate the discrepancy between ground-truth and generated speech.
Instead of using auxiliary loss, GAN-TTS [6] uses an ensemble of
discriminators to take the linguistic features into account and stabilize the
training process.

Another vein of research on non-autoregressive vocoders is to leverage
normalizing flow models in order to model waveform signals directly from
either acoustic features like Mel-spectrogram [11] or linguistic features
[12]. Both Parallel Wavenet [13] and Clarinet [12] are inverse autoregressive
flow models, thus, at inference time they are able to synthesize audio in
parallel. Those models are trained in the teacher-student procedure from a
well trained Wavenet [14] to benefit from autoregressive models and achieve
real-time inference. Nevertheless, they require a well trained Wavenet teacher
and a set of auxiliary losses to acquire high fidelity synthesis, thus, they
are complicated in training and development. Consequently, flow-based vocoders
generally perform worse than autoregressive vocoders with regard to modeling
density of audio signal, therefore, their performance is so far behind
compared with autoregressive vocoders.

The second line of research on neural vocoders is autoregressive vocoders [15,
16, 17, 18] which acquire closet performance compared with ground-truth audio.
Despite autoregressive vocoders being capable of modeling the density of audio
signals to generate realistic audio, they are immensely slow at inference due
to sequential generating. To achieve real-time inference, the authors in [16]
proposed WaveFlow as a general case for both the Wavenet and the WaveGlow
model, WaveFlow is able to trade-off between optimizing the likelihood and
inference time to synthesize high-quality audio conditioned on Mel-
spectrograms. An improved version of Waveflow is Nanoflow [15] which operates
a sharing density estimator and flow indication embedding to reduce the
model’s memory footprint.

In this paper, we propose a new autoregressive normalizing flow vocoder called
FlowVocoder, which is developed based on sharing a density estimator block
[15]. Our proposed method has a small memory footprint by sharing a density
estimator across flow blocks, and we also enhance the density estimation of
flow blocks by using a more flexible transformation function. Also, as
described in [19], we adopt a new conditioning architecture that is
responsible for computing parameters for transformation functions. According
to empirical experiments, we modify the conditioning block by removing
attention layers since those layers not only do not improve the flexibility of
flow blocks but also require more parameters. Consequently, FlowVocoder is
able to generate synthetic audio which acquired competitive results in terms
of both objective and subjective evaluation compared with baseline vocoders.

##  2 FlowVocoder

FlowVocoder is an autoregressive normalizing flow model that enables to
generate waveform signals by sampling from a normal Gaussian distribution
conditioned on a Mel-spectrogram. This work relies on [16] that demonstrated
an incredible performance of speech synthesis with a very small number of
neural network’s parameters. In our model, there are two types of mapping:
forward mapping and reverse mapping. The reverse mapping $f^{-1}(x)$ is
utilized to model the distribution of input data $p(x)$ through a sequence of
reverse transformation functions in a simple distribution $p(z)$, while the
forward mapping is utilized to generate back an input data $x=f(z)$ from a
sampling $z\sim p(z)$. Particularly, in the reserve mapping, the output $z$ is
calculated from the input $x$ by going through a sequence of inverse flow
layers which correspond to a sequence of coupling transformation functions as
$z=f_{k}^{-1}(f_{k-1}^{-1}(...(f_{0}^{-1}(x))))$, where $f^{-1}_{0}(x)=x$.
Each transformation function is expressed by a mixture of the logarithm of
cumulative distribution functions:

$f_{k}^{-1}(x_{k-1})=\sigma^{-1}(\text{MixLogCDF}(x_{k-1};\pi_{k},\mu_{k},s_{k})).e^{a_{k}}+b_{k}$
(1)
$\begin{split}\text{MixLogCDF}(x_{k-1};\pi_{k},\mu_{k},s_{k})=\sum_{i=1}^{M}\pi^{i}_{k}\sigma((x_{k-1}-\\\
\mu^{i}_{k}).e^{-s^{i}_{k}})\end{split}$

, where $k$ denotes the $k-th$ reverse transformation function. $M$ is the
number of logistic mixture components of the cumulative function, while $i$
stands for the $i-th$ component in the logistic mixture function.
$\sigma^{-1}$ denotes inverse sigmoid function. The transformation parameters
$a_{k},b_{k},\pi_{k},\mu_{k},s_{k}$ of the reverse function
$f_{k}^{-1}(x_{k-1})$ are computed based on the output of the previous
function $f_{k-1}^{-1}(x_{k-2})=x_{k-1}=(x_{1,k-1},x_{2,k-1})$, where $x_{0}$
is input data. We use a simple block neural network $g(x_{1,k-1};\theta)$
which outputs five parameters $a_{k},b_{k},\pi_{k},\mu_{k},s_{k}$ for
computing transformation parameters, where $g(x_{1,k-1};\theta)$ is a stack of
multiple CNN layers and $\theta$ is its parameters. The output
$x_{k}=(x_{1,k},x_{2,k})$ is defined as follows:

$x_{1,k}=x_{1,k-1}$ (2)
$\begin{split}x_{2,k}=&\sigma^{-1}\big{(}\text{MixLogCDF}(x_{2,k-1};\pi_{k},\mu_{k},s_{k})\big{)}.e^{a_{k}}+b_{k}\end{split}$
(3)

The above coupling transformation is more expressive than the classic coupling
affine transformation, therefore, it is able to increase the flexibility of
flow models to capture data density distribution. Moreover, the mixture
logistic CDF always has the reversed function due to its monotonic increase
property. Also, the Jacobian determinant of this transformation is
straightforward to calculate since it is the sum of the probability density
function, the derivative of the inverse sigmoid function, and the scale of
transformation $a$. The logarithm of Jacobian determinant, denoted as
$\log|\text{det}(\mathbf{J}())|$, of the mixture logistic CDF is computed as
follow:

$\displaystyle\log{|\text{det}(\mathbf{J}(f^{-1}(x)))|}=$
$\displaystyle\log{|a|}+\log{\Bigg{|}\frac{1}{\delta.(1-\delta)}\Bigg{|}}$ (4)
$\displaystyle+$
$\displaystyle\log{\Bigg{|}\odv{\sigma^{-1}(\tau)}{\tau}\Bigg{|}}$

where

$\begin{split}\delta&=\text{MixLogPDF}(x;\pi,\mu,s)\\\
&=z-\log(s)-2.\log(1+e^{z})\\\ &\text{where }z=(x-\mu).e^{-s}\end{split}$ (5)
$\begin{split}\tau&=\text{MixLogCDF}(x;\pi,\mu,s)\\\ \end{split}$ (6)

Since our model is the autoregressive model, we reshape the 1-dimensional
input data $x$ to the 2-dimensional data by splitting $x$ into $H$ groups as
$\\{X_{1},...,X_{H}\\}$, and then they are stacked to form a 2-D matrix
$X\in\mathbb{R}^{h\times w}$. To transform from the data distribution to the
isotropic Gaussian distribution, a sequence of bipartite transformations
represented by mixture logistic CDFs are performed recursively to model a
conditional dependency between the grouped data

$Z_{i}=f^{-1}(X_{<i},X_{i};a,b,\pi,\mu,s),\text{where }i=1,...,H$ (7)

. $X_{<i}$ denotes for ${X_{1},...,X_{i-1}}$. The inverse image of
$X\in\mathbb{R}^{h\times w}$ on isotropic Gaussian distribution,
$Z\in\mathbb{R}^{h\times w}$, is achieved by sequentially performing inverse
mapping $f^{-1}:X\mapsto Z$ over rows of input data. For sampling, a sample
$Z\in\mathbb{R}^{h\times w}$ is firstly sampled from the isotropic Gaussian,
and then it is transformed to the data distribution by autoregressively
performing the forward mapping function $f:Z\mapsto X$ over rows of the
sampled noise

$X_{i}=f(X_{<i},Z_{i};a,b,\pi,\mu,s),\text{where }i=1,...,H$ (8)

For each iteration of training, we directly maximize log-likelihood of the
data which poses no difficulty to compute by applying the change of variables:

$\begin{split}\log
P(X)=&\sum_{i}\bigg{(}\log|\text{det}(\mathbf{J}(f^{-1}(X_{<i})))|\\\
&-\frac{Z_{i}^{2}}{2}-\frac{\log{2\pi}}{2}\bigg{)}\end{split}$ (9)

###  2.1 Shared density estimator

Figure 1: The FlowVocoder architecture with shared density estimator block

In [15], the authors proposed a new model called NanoFlow to reduce the
computational and parameter footprint of flow models. As shown in Figure. 1,
we utilize the same strategy to reduce computation and model size based on
sharing a neural density estimator across $K$ flow blocks. Also, a flow
indication embedding is used to enable the shared density estimator to be
capable of learning multiple contexts for a specific flow block. Consequently,
a bipartite transformation block is redefined as:

$Z_{i}=f^{-1}(g(X_{<i},X_{i};\hat{\theta},\textbf{e}^{k});a,b,\pi,\mu,s)$ (10)

, where $g(X_{<i},X_{i};\hat{\theta})$ is the shared density estimator which
is utilized for $K$ flow blocks. Embedding vector
$\textbf{e}^{k}\in\mathbb{R}^{D}$ represents the $k-th$ flow block, this
embedding is fed into the shared density estimator to output proper
transformation’s parameters. Intuitively, the flow layer embedding
$\textbf{e}^{k}$ as an additional context is able to guide the shared density
estimator learning multiple densities with a minimal number of additional
parameters.

Figure 2: The comparison regarding log likelihood on testing set between
FlowVocoder and FlowVocoder with attention layer

As shown in Figure. 2, we experimented to compare performance between
FlowVocoder using only convolutional layers and FlowVocoder using attention
layers as suggested in [19]. Both models have the same number of residual
channels which is 128 channels, and they are trained for 500k iterations. The
performances of the two models are estimated on the testing set. It is clear
to see that at the beginning of the training process, FlowVocoder with
attention surpasses FlowVocoder without attention. However, after enough
training, the gap between the two models is insignificant. As a result, flow
blocks without attention layers are well-suited for our proposed model.
Especially, in the case of deploying on edge devices. Indeed, attention layers
require $O(n^{2}.d)$ computation complexity per layer, therefore, it consumes
tremendously computational resources to feed forward. Consequently, the neural
block is used to output transformation parameters is: $\text{Conv}_{1\times
1}\xrightarrow{}\text{GatedConv}_{3\times 3}\xrightarrow{}\text{Conv}_{1\times
1}$

###  2.2 Speech synthesis conditioned on Mel-spectrogram

After training the FlowVocoder, generating waveform signals conditioned a Mel-
spectrogram is straightforward by first sampling a Gaussian noise $z$.
Subsequently, since the size of conditioning Mel-spectrograms is mismatched
with the sampled noise, we upsample the conditioned Mel-spectrogram to match
the size of the noise sample $z$. It is then added as a bias term at each flow
block. For mapping a Gaussian noise $z$ forward to waveform signals, the
sampled noise is applied to the forward mapping

$X_{i}=\text{MixLogInvCDF}\big{(}\sigma((Z_{<i}-b).e^{-a})\big{)}$ (11)

, where MixLogInvCDF is approximated iteratively by using numerical method.

Table 1: The objective evaluation is compared among FlowVocoder and baseline models. The objective metrics are used including the number of model’s parameters in million, the mel-ceptral distortion(MCD), the root mean square error of fundamental frequency ($\text{RMSE}_{F0}$), and the log likelihood (LL). Method | Res channels |  Parameters(M)$\downarrow$ |  MCD$\downarrow$ | $\text{RMSE}_{F0}\downarrow$ |  LL$\uparrow$
---|---|---|---|---|---
WaveFlow (H=$16$) | $128$ | $22.25$ | $5.61\pm 0.031$ | $30.33\pm 0.78$ | $5.001$
NanoFlow (H=$16$) | $128$ | $2.85$ | $5.54\pm 0.047$ | $38.14\pm 0.65$ | $4.970$
Proposed (H=$16$) | $128$ | $4.14$ | 5.37$\pm$0.043 |  28.25 $\pm$0.88 | 5.011

##  3 Experiment

We trained and evaluated our proposed model and baseline models on the
LJSpeech dataset [20]. This dataset contains 13,100 audio clips from a single
female speaker, the speech data is approximately 24 hours with a sampling rate
of 22.05 kHz. The dataset is split into two parts: 90% of data for training
and 10% of data for testing. For each utterance, we randomly draw a chunk of
16,000 samples, the chunk of audio is then normalized in the range from 0 to 1
by dividing the maximum waveform value $\text{v}=32768.0$. Next, 80 band log-
scale Mel-spectrograms are extracted from the normalized chunk using FFT of
size 1024, hop length of size 256, window of size 1024, and Hamming window. To
use Mel-spectrograms as a conditioner for speech synthesis, we upsample Mel-
spectrogram 256 times by operating two 2D-transpose CNN layers with filter
size of $k=[32,3]$.

Our model and baseline models are trained on 2 Nvidia V100 GPUs with a batch
size of 2 for 1M iterations. We use Adam optimizer with initial learning rate
of $2\times 10^{-4}$, and we anneal the learning rate by half for every 200k
iterations. The dimension of flow layer embedding is $D=512$ for
$\textbf{e}^{k}\in\mathbb{R}^{D}$ in the eight-flows block model, and we use
$H=16$ to squeeze input audio into 2-D matrix $X\in\mathbb{R}^{h\times w}$.
Our source code and audio samples can be found in our
github111https://v-manhlt3.github.io/FlowVocoder-demo-pages/

###  3.1 Objective evaluation

Figure 3: F0 contour of utterance ”in being comparatively modern” of
FlowVocoder and WaveFlow

For objective evaluation, we draw randomly 100 utterances from the testing
dataset to calculate objective metrics. Next, we acquire the performance of
synthetic speech with regard to three metrics: Mel-ceptral distortion (MCD)
[21], log likelihood (LL), and root mean square error of fundamental frequency
($\text{RMSE}_{F0}$) [22]. Equations for MCD and $\text{RMSE}_{F0}$
calculation are defined as follows:

$\text{MCD[dB]}=\frac{10}{\log{10}}\sqrt{2\sum_{m=1}^{M}(c_{r}(m)-c_{s}(m))^{2}}$
(12)
$\text{\lx@glossaries@gls@link{acronym}{rmse}{\leavevmode$\text{RMSE}_{F0}$}}\text{[cent]}=1200\sqrt{(\log_{2}{(F_{r})}-\log_{2}{(F_{s})})^{2}}$
(13)

, where $\text{c}_{r}$ and $\text{c}_{s}$ denote Mel-spectrum of groundtruth
and synthetic waveform signals, respectively. $M$ denotes the order of Mel-
spectrum, $\text{F}_{r}$ and $\text{F}_{s}$ denote aperiodic components of
groundtruth and synthetic signal.

Table. 1 shows comparisons between baseline systems and the proposed system
regarding objective metrics. NanoFlow is the smallest one regarding the size
of models, and the size of our proposed is bigger than NanoFlow by one and a
half of the number of parameters. However, the proposed model outperforms
Nanoflow in terms of MCD and log likelihood. Especially, FlowVocoder achieves
the least Mel-ceptral distance with ground-truth audio, thereby, FlowVocoder
is able to produce synthetic speech which has the highest audio quality
compared to ones generated by other baselines. In comparision with WaveFlow,
our model is much smaller than WaveFlow, about five times smaller in terms of
model’s size; furthermore, FlowVocoder surpasses WaveFlow with regard to MCD,
log likelihood, and $\text{RMSE}_{F0}$. As shown in Figure. 3, FlowVocoder
matches $F0$ contour of ground-truth better than WaveFlow, therefore, the
generated speech from FlowVocoder is more natural than the generated speech
from WaveFlow. This result is also reflected in the subjective evaluation.
Finally, we computed the real-time factor(RTF) at the inference step in a
single V100 GPU shown in Table. 2. Although FlowVocoder takes the longest time
to synthesize a second of speech, it is still able to generate speech in real-
time. The explanation for that phenomenon is due to the burden of
approximation of MixLogInvCDF function in Eq. (11).

Table 2: The real-time factor(RTF) on a single GPU at the inference step among FlowVocoder and baseline models. Method |  RTF$\downarrow$
---|---
WaveFlow | 0.165
NanoFlow | 0.130
Proposed | 0.514

###  3.2 Subjective evaluation

For subjective evaluation, we acquired an evaluation of the proposed model and
baseline models in terms of naturalness using 5 scales mean opinion scores
(MOS), in which 15 participants are asked to assess the quality of synthetic
audio. Each participant wears the same headphone when doing the experiment.
They are asked to listen to 20 audio clips from each system, then assess for
the naturalness of audio on a scale of 1 to 5 with a 1.0 point increments.
Since all systems are capable of synthesizing high perceptual audio, we
conducted this experiment by asking all participants to listen to the
synthetic speech twice. They then give two opinion scores, subsequently, we
average these scores to achieve one evaluation sample. Totally, we collected
approximately 1,500 evaluation samples, the results of this experiment are
represented in the first part of Table. 3. All reported MOS have statistical
significance with $p<0.05$. Our proposed model surpasses both WaveFlow and
NanoFlow in terms of high fidelity of audio generating from ground-truth mel-
spectrogram. That subjective result points out that the density estimation
performance of FlowVocoder is superior with both baselines.

Table 3: Subjective evaluation regarding MOS with 95% confidence interval on 20 random utterances from testing dataset. We use the T-test method to examine the statistical significance of all reported MOS with $p<0.05$. Method |  MOS$\uparrow$ | 95%CI
---|---|---
Mel-spectrogram + WaveFlow | 4.38 |  $\pm$ 0.11
Mel-spectrogram + NanoFlow | 4.18 |  $\pm$ 0.12
Mel-spectrogram + Proposed | 4.41 |  $\pm$ 0.09
Tacotron2 + WaveFlow | 4.26 |  $\pm$ 0.11
Tacotron2 + NanoFlow | 4.07 |  $\pm$ 0.13
Tacotron2 + Proposed | 4.30 |  $\pm$ 0.09
ground-truth | 4.62 |  $\pm$ 0.08

We also test the quality of synthetic audio combined with Tacotron2 [23]. The
code and pretrained Tacotron 2 model are from the authors
github222https://github.com/NVIDIA/tacotron2.We test all systems on 20 random
utterances from the testing dataset, we first generate Mel-spectrograms using
Tacotron 2 and then vocoding these spectrograms by operating baseline vocoders
and FlowVocoder. The second part of Table. 3 reports text to speech
experimental results. Regarding FlowVocoder, it acquires competitive
performance with NanoFlow and WaveFlow. According to the results, it shows
that FlowVocoder is more well-suited for text-to-speech applications.

##  4 Conclusions

In this work, we present a new sort of flow model called FlowVocoder which is
capable of modeling audio waveform signals conditioning on Mel-spectrogram.
Our proposed model has a small memory footprint, corresponding with fewer
model’s parameters, by sharing a density estimator across $K$ flow blocks.
Moreover, we enhance the flexibility of the bipartite transformation function
by using a mixture of CDF. Finally, We modify the conditioning block to
calculate transformation parameters efficiently for neural vocoders,
particularly, we remove attention layers to be more suitable for deploying on
edge devices. As shown in the experiments, FlowVocoder outperforms both
WaveFlow and NanoFlow regarding subjective and objective evaluation.
Especially, our model matches the $F0$ contour better compared with WaveFlow,
therefore, the prosody of generated audio is more natural. Though FlowVocoder
acquired impressive results, its inference time should be taken into account.
In the future, we could solve the high-latency at inference step by speeding
up the approximation of the MixLogInvCDF function.

## References

  * [1] D. Griffin and J. Lim, “Signal estimation from modified short-time fourier transform,” _IEEE Transactions on Acoustics, Speech, and Signal Processing_ , vol. 32, pp. 236–243, 1984.
  * [2] M. Morise, F. Yokomori, and K. Ozawa, “World: A vocoder-based high-quality speech synthesis system for real-time applications,” _IEICE Trans. Inf. Syst._ , vol. 99-D, pp. 1877–1884, 2016.
  * [3] K. Kumar, R. Kumar, T. D. Boissière, L. Gestin, W. Z. Teoh, J. Sotelo, A. D. Brébisson, Y. Bengio, and A. C. Courville, “Melgan: Generative adversarial networks for conditional waveform synthesis,” in _NeurIPS_ , 2019\.
  * [4] J. Kong, J. Kim, and J. Bae, “Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis,” in _Advances in Neural Information Processing Systems_ , vol. 33, 2020, pp. 17 022–17 033.
  * [5] R. Yamamoto, E. Song, and J. Kim, “Parallel wavegan: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram,” _ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pp. 6199–6203, 2020.
  * [6] M. Binkowski, J. Donahue, S. Dieleman, A. Clark, E. Elsen, N. Casagrande, L. C. Cobo, and K. Simonyan, “High fidelity speech synthesis with adversarial networks,” _ArXiv_ , vol. abs/1909.11646, 2020.
  * [7] C. Donahue, J. McAuley, and M. Puckette, “Adversarial audio synthesis,” in _ICLR_ , 2019.
  * [8] G. Yang, S. Yang, K. Liu, P. Fang, W. Chen, and L. Xie, “Multi-band melgan: Faster waveform generation for high-quality text-to-speech,” _ArXiv_ , vol. abs/2005.05106, 2020.
  * [9] A. Mustafa, N. Pia, and G. Fuchs, “Stylemelgan: An efficient high-fidelity adversarial vocoder with temporal adaptive normalization,” _ICASSP 2021 \- 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pp. 6034–6038, 2021.
  * [10] W. Jang, D. Lim, J. Yoon, B. Kim, and J. Kim, “UnivNet: A Neural Vocoder with Multi-Resolution Spectrogram Discriminators for High-Fidelity Waveform Generation,” in _Proc. Interspeech 2021_ , 2021, pp. 2207–2211.
  * [11] R. Prenger, R. Valle, and B. Catanzaro, “Waveglow: A flow-based generative network for speech synthesis,” _ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pp. 3617–3621, 2019.
  * [12] W. Ping, K. Peng, and J. Chen, “Clarinet: Parallel wave generation in end-to-end text-to-speech,” in _International Conference on Learning Representations_ , 2019. [Online]. Available: https://openreview.net/forum?id=HklY120cYm
  * [13] A. Oord, Y. Li, I. Babuschkin, K. Simonyan, O. Vinyals, K. Kavukcuoglu, G. van den Driessche, E. Lockhart, L. C. Cobo, F. Stimberg, N. Casagrande, D. Grewe, S. Noury, S. Dieleman, E. Elsen, N. Kalchbrenner, H. Zen, A. Graves, H. King, T. Walters, D. Belov, and D. Hassabis, “Parallel wavenet: Fast high-fidelity speech synthesis,” in _ICML_ , 2018.
  * [14] A. Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, and K. Kavukcuoglu, “Wavenet: A generative model for raw audio,” in _SSW_ , 2016.
  * [15] S.-g. Lee, S. Kim, and S. Yoon, “Nanoflow: Scalable normalizing flows with sublinear parameter complexity,” in _Advances in Neural Information Processing Systems_ , vol. 33, 2020, pp. 14 058–14 067.
  * [16] W. Ping, K. Peng, K. Zhao, and Z. Song, “Waveflow: A compact flow-based model for raw audio,” _ArXiv_ , vol. abs/1912.01219, 2020.
  * [17] S. Kim, S. gil Lee, J. Song, J. Kim, and S. Yoon, “Flowavenet : A generative flow for raw audio,” _ArXiv_ , vol. abs/1811.02155, 2019.
  * [18] R. Yamamoto, E. Song, and J. Kim, “Probability density distillation with generative adversarial networks for high-quality parallel waveform generation,” in _INTERSPEECH_ , 2019.
  * [19] J. Ho, X. Chen, A. Srinivas, Y. Duan, and P. Abbeel, “Flow++: Improving flow-based generative models with variational dequantization and architecture design,” in _Proceedings of the 36th International Conference on Machine Learning_ , 2019, pp. 2722–2730.
  * [20] K. Ito and L. Johnson, “The lj speech dataset,” https://keithito.com/LJ-Speech-Dataset/, 2017.
  * [21] R. Kubichek, “Mel-cepstral distance measure for objective speech quality assessment,” _Proceedings of IEEE Pacific Rim Conference on Communications Computers and Signal Processing_ , vol. 1, pp. 125–128 vol.1, 1993\.
  * [22] T. Hayashi, A. Tamamori, K. Kobayashi, K. Takeda, and T. Toda, “An investigation of multi-speaker training for wavenet vocoder,” _2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)_ , pp. 712–718, 2017.
  * [23] J. Shen, R. Pang, R. J. Weiss, M. Schuster, N. Jaitly, Z. Yang, Z. Chen, Y. Zhang, Y. Wang, R. Skerry-Ryan, R. A. Saurous, Y. Agiomyrgiannakis, and Y. Wu, “Natural tts synthesis by conditioning wavenet on mel spectrogram predictions,” _2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pp. 4779–4783, 2018.