# Self-Supervised Learning for Speech Enhancement Through Synthesis

###### Abstract

Modern speech enhancement (SE) networks typically implement noise suppression
through time-frequency masking, latent representation masking, or
discriminative signal prediction. In contrast, some recent works explore SE
via generative speech synthesis, where the system’s output is synthesized by a
neural vocoder after an inherently lossy feature-denoising step. In this
paper, we propose a denoising vocoder (DeVo) approach, where a vocoder accepts
noisy representations and learns to directly synthesize clean speech. We
leverage rich representations from self-supervised learning (SSL) speech
models to discover relevant features. We conduct a candidate search across 15
potential SSL front-ends and subsequently train our vocoder adversarially with
the best SSL configuration. Additionally, we demonstrate a causal version
capable of running on streaming audio with 10ms latency and minimal
performance degradation. Finally, we conduct both objective evaluations and
subjective listening studies to show our system improves objective metrics and
outperforms an existing state-of-the-art SE model subjectively.

Index Terms—  Speech enhancement, speech synthesis, self-supervised learning,
audio representations, deep learning

##  1 Introduction

Speech enhancement (SE), which intends to separate voice from unwanted
interference, is an essential component of voice communication systems and has
been a focus of research for decades. Modern SE systems are often powered by
deep neural networks (DNNs) for tasks such as telephony [1], speech coding
[2], hearing assistance [3, 4], and automatic speech recognition [5]. Existing
DNN-based SE approaches most commonly implement the framework of supervised
learning, in which the model is trained to output the clean speech signal,
given noisy input. As such, these models are typically trained from scratch;
with randomly-initialized weights

On the other hand, recent advancements in self-supervised learning (SSL) have
led to the emergence of powerful models pre-trained on large corpora capable
of capturing transferable representations of audio [6, 7, 8, 9, 10, 11]. Such
SSL models can be used to obtain rich representations useful for a variety of
tasks. In particular, SSL models are commonly applied in the context of
automatic speech recognition [6, 7] and paralinguistics [12, 13] as feature
extractors, especially in applications whereby task-specific data is scarce.
However, the utility of SSL representations has not been as thoroughly studied
in the context of SE or speech synthesis.

Some works utilize the SSL models to act as deep perceptual losses [14], which
tend to improve the performance of SE models. Meanwhile, other studies used
pre-trained SSL representations as inputs into an SE system. In [15], Huang et
al. evaluate 13 different upstream models on a time-frequency masking-based
speech enhancement task. Following this work, Hung et al. [16] show that
performance can be boosted by concatenating self-supervised learning
representations with raw acoustic features. In these cases, the downstream
model is an LSTM or BLSTM-based denoiser architecture and is responsible for
generating the time-frequency masks, which are applied to the noisy input.

While using SSL representations in the context of mask-based SE can improve
performance, we hypothesize that latent embeddings may be more well-suited for
directly synthesizing audio [17]. This is motivated by the fact that deep SSL
models decompose input into a set of features not necessarily related to the
short-time Fourier transform (STFT) representation, typically used in the
mask-based approaches. As such, using richer SSL features to compute the STFT
masks might limit the efficacy of the former in the context of SE.

Neural vocoders [18], such as WaveNet [19] or HiFiGAN [20], have become adept
at synthesizing high-quality speech from features such as Mel spectrograms.
Through varying methods, these models are able to produce audio with natural-
sounding phase without access to the original time-domain signal. As phase
reconstruction issues have been heavily discussed in the SE literature (e.g,
[21]) neural vocoders present an attractive solution. In fact, several works
have approached speech enhancement through synthesis (SETS) in this manner
[22, 23]. However, these systems incorporate modules tasked with denoising
speech before resynthesis. We refer to this type of approach as pre-denoising.
We suggest that pre-denoising modules are a potential bottleneck for several
reasons. Cascading modules are prone to compounding errors. If the pre-
denoiser and vocoder are trained separately, there exists potential for a
mismatch. The vocoder expects ”perfect” features, and the prediction model
will inevitably make mistakes, adversely affecting the resulting speech
quality.

Although joint training can help avoid compounding errors, the process of pre-
denoising is inherently lossy. In attempting to isolate the speech, the
process may also corrupt other acoustic cues that could be helpful for speech-
noise separation and speech intelligibility preservation. Pre-denoising
modules also add complexity, the necessity of which has not been validated. We
hypothesize neural vocoders are powerful enough to perform speech enhancement
from noisy representations directly.

In this work, we explore the utility of applying pre-trained SSL
representation models to SETS. Unlike the previous exploration of the use of
SSL models in SE, we avoid mask-based subtractive noise suppression by
directly reconstructing the clean output from the SSL model embedding of noisy
input. Additionally, we avoid the pre-denoising step in other SETS systems,
instead opting for a denoising vocoder (DeVo) approach. We benchmark the
efficacy of existing SSL models pre-trained on large datasets (and hand-
crafted feature baselines) in SE by coupling them with trainable denoising
vocoders to synthesize noise-free speech. Furthermore, we investigate the
impact of training strategy on performance and generalization. We evaluate the
proposed approaches using objective perceptual metrics, as well as through a
subjective listening study.

##  2 Method

We start by using SSL models as a basis to identify representation candidates
directly suitable for enhancement through synthesis. The models explored
include wav2vec 2.0 [6], WavLM [7], UniSpeech-SAT [8], HuBERT [9], Modified
Contrastive Predictive Coding (CPC) [10], and BYOL-A [11]. Several of these
models come in two sizes, Base and Large, and we use Base-sized models where
applicable. Most models consist of a convolutional feature encoder followed by
either Transformer layers or an LSTM layer. For these, we explore using both
the full model and just the feature encoder. Several models also have “+”
configurations which have been trained with a larger amount of data that is
more diverse. We evaluate these models separately as candidates as well. We
use the log-Mel spectrogram (LMS) with a frame length of 1024, a hop length of
160 samples, and 128 Mel bins as a hand-crafted representation baseline. All
the encoders process audio inputs sampled at 16 kHz.

We use these models in combination with a neural vocoder, HiFiGAN [20], to
synthesize high-quality clean speech. The HiFiGAN architecture consists of
transposed convolutions which progressively up-sample the input representation
into a time-domain waveform and convolutions with varying receptive fields
which provide the model with important context. If necessary, we up-sample
representations in time using nearest neighbor interpolation to match the
HiFiGAN resolution. Figure 1 depicts this framework.

Most of the surveyed SSL models contain a stack of Transformer layers
following the convolutional encoder. Since previous studies have shown that
the last layer is not always the most useful for a given task, we adopt the
weighted sum approach following SUPERB [24]. This allows the model to
emphasize or de-emphasize information from different Transformer layers. The
parameters of the SSL models are fixed, and only the feature weights and
vocoder parameters are optimized during training (unless stated otherwise).

Additionally, we explore initializing HiFiGAN with pre-trained weights
optimized to synthesize clean speech from the Mel spectrograms. To accept
representations with different feature dimensions, the first layer of HiFiGAN
must be altered. We adopt the cross-modality pre-training approach proposed by
[25] and average the weights of the pre-trained first layer, then use this
average to initialize a new size-adapted input layer. In initial experiments,
we found that this has a negligible impact on feature weighting and training
stability, but the resulting audio output quality is higher. Thus, we adopt
this technique for all training setups in our candidate search.

Fig. 1: DeVo system diagram. Components in blue are trainable and those in
grey have frozen weights (unless stated otherwise). Components with dotted
lines are only used for some SSL models.

##  3 Experimental Setup

###  3.1 Dataset

We use the dataset from the Interspeech 2020 Deep Noise Suppression (MSDNS)
Challenge [26] for training and testing. The training dataset consists of 500+
hours of clean speech and 100+ hours of noise. We mix the speech and noise at
various SNR levels, sampled from a uniform distribution between -5 and 15 dB.
We employ a LUFS-based SNR calculation for more perceptually relevant mixtures
and to de-emphasize the effects of impulsive noises [27]. The evaluation
dataset consists of 150 samples of noisy speech samples and their respective
clean references. Though reverberant versions of these samples are available,
we do not utilize these as de-reverberation is outside of the scope of this
work. In the interest of comparison to other speech enhancement work, we also
report evaluation results on the Valentini et al. [28] dataset, which has 824
utterances mixed with noise across four SNR levels: 2.5, 7.5, 12.5, and 17.5
dB. All the audio clips are sampled at 16 kHz.

###  3.2 Losses

HiFiGAN is trained adversarially and as adversarial training can be slow and
complex, we use a non-adversarial criterion for our candidate search, the
Phase-Constrained Magnitude Loss (PCM) introduced in [29], defined as

$L_{PCM}(s,\hat{s})=\frac{1}{2}\cdot L_{SM}(s,\hat{s})+\frac{1}{2}\cdot
L_{SM}(n,\hat{n})$ (1)

where $s$ and $n$ are speech and noise, and $L_{SM}$ is a spectral magnitude
loss defined as

$L_{SM}(s,\hat{s})=\frac{1}{T\cdot
F}\sum_{t=0}^{T-1}\sum_{f=0}^{F-1}[(|S_{r}(t,f)|+|S_{i}(t,f)|)-\\\
(|\hat{S}_{r}(t,f)|+|\hat{S}_{i}(t,f)|)]$ (2)

where $S_{r}$ and $S_{i}$ are the real and imaginary components of the Short-
Time Fourier Transform (STFT). We set the frame length and hop length for the
STFT to 1024 and 160, respectively.

Once a suitable candidate is found, we train this candidate with HiFiGAN’s
original loss [20]. As mentioned previously, HiFiGAN is trained adversarially,
and the generator ($G$) and discriminator ($D$) objectives are defined as

$\displaystyle\mathcal{L}_{G}=\mathcal{L}_{Adv}(G;D)+\mathcal{L}_{FM}+\mathcal{L}_{Mel}(G)$
(3) $\displaystyle\mathcal{L}_{D}=\mathcal{L}_{Adv}(D;G)$ (4)

where $\mathcal{L}_{Adv}$, $\mathcal{L}_{FM}$, $\mathcal{L}_{Mel}(G)$ are the
adversarial loss, feature matching loss, and log-Mel spectrogram loss
respectively. Our only modification is adding an L1 noise estimation term to
the log-Mel spectrogram loss akin to the PCM loss in equation (1).

###  3.3 Metrics

To find the most suitable SSL front-end, we initiate our investigation by
looking to Short-Time Objective Intelligibility (STOI) [30] and Perceptual
Evaluation of Speech Quality (PESQ) [31] as our metrics. While STOI is often
reported in SE literature, experimentally we find that vocoder-denoised
samples result in STOI degradations, even if the samples are of high
subjective quality (as shown in our subjective listening study). Although [26]
suggests metrics, such as PESQ, do not always correlate well with perceptual
quality, we find it to be a coarse metric capable enough to point the
exploration in the right direction. Even if raw scores do not exactly predict
human perception, we find $\Delta$PESQ to be indicative of differences in
model performance. $\Delta$PESQ is defined as
$PESQ(enhanced,clean)-PESQ(noisy,clean)$. In light of this, we choose to rely
on $\Delta$PESQ for our candidate model selection.

In the evaluation of our adversarially trained models, we explore a wider
range of metrics: NISQAv2 [32], DNSMOS P.835 [33], and Composite objective
measures [34]. NISQAv2 and DNSMOS P.835 are deep learning-based non-intrusive
perceptual quality predictors. The former predicts a single MOS score and the
latter outputs a score for speech quality (SIG), background noise quality
(BAK) and overall quality (OVRL). Composite scores (CSIG, CBAK, COVRL) are
similar to DNSMOS P.835 but utilize regression-based analysis on a selection
of objective measures in contrast to deep learning.

Since the human perception of sound is highly subjective and non-linear,
perceptual assessments are still considered the “golden standard” in
evaluating speech quality [33]. Thus, in addition to the objective evaluation,
we conduct a listening study using outputs of various model variants presented
to human raters. The study is based on ITU-T P.808 [35] and employs 20
randomly selected noisy samples from the MSDNS test set [26], each processed
using selected approaches. Participants are asked to rate the overall quality
of noisy speech samples processed using different methods using a 5-point
Likert scale. The presentation order of unlabelled (i.e. double-blind) audio
clips is randomized for each participant.

Table 1: Selected speech representations, their properties, and the candidate search results. NE stands for noise exposure and indicates models that have been pre-trained using noisy speech data. CFE indicates convolutional feature encoders obtained from a larger model. Dim. indicates the dimensionality of the SSL feature. Representation | NE | CFE | Dim. | Params |  $\Delta$PESQ
---|---|---|---|---|---
LMS | - | - | 128 | - | 0.400
BYOL-A | ✓ | ✗ | 3072 | 6.3M | 0.259
HuBERT | ✗ | ✗ | 768 | 95M | 0.189
✗ | ✓ | 512 | 4.2M | 0.286
Modified CPC | ✗ | ✗ | 256 | 1.8M | 0.448
✗ | ✓ | 256 | 1.5M | 0.458
UniSpeech-SAT | ✗ | ✗ | 768 | 95M | 0.174
✗ | ✓ | 512 | 4.2M | 0.417
✓ | ✗ | 768 | 95M | 0.180
✓ | ✓ | 512 | 4.2M | 0.187
Wav2Vec2 | ✗ | ✗ | 768 | 95M | 0.206
✗ | ✓ | 512 | 4.2M | 0.335
WavLM | ✗ | ✗ | 768 | 95M | 0.232
✗ | ✓ | 512 | 4.2M | 0.344
✓ | ✗ | 768 | 95M | 0.117
✓ | ✓ | 512 | 4.2M | 0.234
Table 2: Results of adversarial training evaluation. FT indicates models in which Modified CPC is jointly finetuned with the denoising vocoder. S indicates models with causal convolutions that are streaming-compatible. Dataset | Model | Causal | NISQAv2 | DNSMOS P.835 | Composite | PESQ | STOI
---|---|---|---|---|---|---|---
Sig | Bak | Ovl | Sig | Bak | Ovl
MSDNS | Noisy (unprocessed) | - | 2.53 | 3.39 | 2.33 | 2.34 | 3.19 | 2.53 | 2.35 | 1.58 | 0.92
LMS | ✗ | 4.21 | 3.51 | 3.96 | 3.21 | 3.19 | 1.91 | 2.37 | 1.63 | 0.86
Modified CPC | ✗ | 4.32 | 3.53 | 4.01 | 3.25 | 3.24 | 2.22 | 2.50 | 1.82 | 0.87
Modified CPC FT | ✗ | 4.34 | 3.56 | 3.99 | 3.26 | 3.69 | 2.60 | 2.96 | 2.26 | 0.91
Modified CPC FT-S | ✓ | 4.19 | 3.51 | 3.86 | 3.16 | 3.50 | 2.44 | 2.77 | 2.07 | 0.90
Valentini | Noisy (unprocessed) | - | 2.85 | 3.28 | 2.88 | 2.54 | 3.36 | 2.45 | 2.64 | 1.97 | 0.92
LMS | ✗ | 3.77 | 3.41 | 3.73 | 3.01 | 2.86 | 2.01 | 2.26 | 1.76 | 0.86
Modified CPC | ✗ | 3.81 | 3.42 | 3.77 | 3.03 | 3.09 | 2.23 | 2.47 | 1.94 | 0.87
Modified CPC FT | ✗ | 3.82 | 3.38 | 3.63 | 2.93 | 3.35 | 2.59 | 2.77 | 2.27 | 0.90
Modified CPC FT-S | ✓ | 3.78 | 3.28 | 3.55 | 2.81 | 3.17 | 2.46 | 2.59 | 2.10 | 0.89

##  4 Results

###  4.1 Candidate Search

Table 1 shows the results of the SSL candidate search. From these results, we
can see that convolutional feature encoders tend to outperform models with
Transformer layers. We also see that models exposed to noise during training
show no clear advantage over their counterparts. In fact, in some cases the
reverse is true.

Out of these candidates, we choose to further explore the feature encoder of
Modified CPC. Not only does it show the best performance in terms of
$\Delta$PESQ, but it is the least computationally complex of all the
candidates. As we are also interested in efficiency for the speech enhancement
task, we conclude this is the optimal choice.

###  4.2 Encoder Feature Weighting

Figure 2 depicts the results of feature weighting for SSL models with
Transformer layers. We find that in all training variations, the first layer
output receives a much higher weight than all other outputs. These results are
similar to the findings in [15], which suggest earlier Transformer layers
contain the detailed acoustic information needed for tasks like speech
enhancement, and later layers are not as critical. This is also consistent
with our findings in Section 4.1, where convolutional encoders outperform
Transformer layers in combination with convolutional encoders. Thus we suggest
that for speech synthesis and synthesis-based enhancement, either
convolutional encoders or simply the first Transformer layer should be used.

Fig. 2: SSL transformer layer weighting. The weights have been unit-normalized
for each model.

###  4.3 Adversarial Training

As the final stage of our investigation, we adopt an adversarial training
regime with the feature encoder of Modified CPC. In this stage, we evaluate
the effect of fine-tuning by training two models, one with a frozen encoder
(Modified CPC) and one with a trainable encoder (Modified CPC FT). We find
that finetuning the encoder increases performance. We also evaluate the effect
of making all operations in the model causal (Modified CPC FT-S), so that the
system may possibly be used in a real-time environment on streaming audio. The
causal configuration of the model outputs time-domain audio in 10ms blocks
with no overlap or lookahead, thus the total overall audio latency of the
system is 10ms. For comparison to these SSL based results, we also
adversarially train the same DeVo model configuration with log-Mel spectrogram
(LMS) audio representation. Table 2 depicts these results for the MSDNS test
set, and the Valentini test set. We observe that the fine-tuned model in both
cases has the strongest performance, across the vast majority of the objective
metrics, so we use this version in our listening study.

We observe that metric improvements across the board are higher for MSDNS than
Valentini, which can be attributed to the overall higher SNRs of the noisy
mixtures in Valentini, as compared to the MSDNS dataset. We also note that
across both datasets, the increase in DNN-based perceptual quality predictors
(DNSMOS and NISQAv2) is generally higher than that of the traditional signal
processing correlation-based measures (PESQ, STOI, Composite). In fact, STOI
decreases in comparison to the noisy signal. We hypothesize that this may be
due to differences between the vocoder-synthesized clean signal and the
original clean signal which may be imperceptible to humans but severely
deleterious to correlation-based metrics. We note that in experiments, we
found that using the HiFiGAN-re-synthesized output of a clean input as the
reference for STOI and PESQ resulted in significantly better scores, but a
comprehensive analysis of appropriate metrics for synthesis-based enhancement
is out of scope here. The incongruity between DNN-based quality predictors and
traditional objective metrics motivates our subjective listening study.

###  4.4 Subjective Evaluation - Listening Study

For our perceptual listening study, we select the following conditions:
unprocessed noisy speech, clean speech, DeVo using LMS features, DeVo using
Modified CPC FT and Demucs111https://github.com/facebookresearch/denoiser, a
well-known waveform-to-waveform SE system [1] also trained on MSDNS. Table 3
illustrates the averaged mean opinion scores (MOS) [35] assigned to each
method by $N=22$ independent raters across 20 audio samples. Firstly, all of
the methods improve MOS by at least 0.72 with the smallest improvement by LMS.
Demucs and Modified CPC FT perform considerably better and further improve the
MOS by another 0.72 and 0.94, respectively (1.44 & 1.66 total w.r.t. noisy).
To assess the significance of these differences we conduct one-way ANOVA with
posthoc paired t-tests corrected for multiple comparisons using Benjamini-
Yekutieli method [36]. The ANOVA indicates a significant difference between
conditions ($p<10^{-33}$) and all the pair-wise differences between methods
are significant at $p<0.013$ (corrected). Additionally, we make the audio
samples from our perceptual study available for
listening222https://github.com/BoseCorp/devo.

Table 3: Results of the P808 listening study ($N=22$) Method | MOS
---|---
Noisy (unprocessed) | 1.86
LMS | 2.58
Demucs [1] | 3.30
Modified CPC FT | 3.52
Clean | 4.48

##  5 Conclusion

In this paper, we proposed that neural vocoding, when combined with a rich
input feature, could be powerful enough to achieve speech enhancement without
pre-denoising the speech or speech feature, a step that may limit the
expressivity of the overall system. We utilized several different SSL models
as a framework to explore input representations for this task and presented
the results of our candidate search. We demonstrated that our synthesis-based
model can run in a causal configuration with only slight degradation in
objective metrics. Finally, we conducted a perceptual listening study to
verify the quality of our model with human raters. Our results suggest: 1)
Denoising vocoders are indeed capable of speech enhancement. 2) Earlier layers
in SSL models tend to be better for the synthesis task. 3) The approach can
achieve comparable or better results than other state-of-the-art speech
enhancement methods. In the future, we hope to analyze the properties of the
SSL representations to reveal the reasons for their differences in
performance. Additionally, we would like to explore DeVo for other speech
tasks such as dereverberation and bandwidth extension.

## References

  * [1] Alexandre Défossez, Gabriel Synnaeve, and Yossi Adi,  “Real time speech enhancement in the waveform domain,”  in Proc. Interspeech, 2020, pp. 3291–3295.
  * [2] W. Kleijn et al.,  “Generative speech coding with predictive variance regularization,”  in ICASSP, 2021, pp. 6478–6482.
  * [3] Igor Fedorov et al.,  “Tinylstms: Efficient neural speech enhancement for hearing aids,”  in Proc. Interspeech, 2020, pp. 4054–4058.
  * [4] Marko Stamenovic, Nils Westhausen, Li-Chia Yang, Carl Jensen, and Alex Pawlicki,  “Weight, block or unit? exploring sparsity tradeoffs for speech enhancement on tiny neural accelerators,”  in NeurIPS Workshop on Efficient Natural Language and Speech Processing, 2021.
  * [5] Catalin Zorila, Christoph Boeddeker, Rama Doddipatla, and Reinhold Haeb-Umbach,  “An investigation into the effectiveness of enhancement in asr training and test for chime-5 dinner party transcription,”  in ASRU, 2019, pp. 47–53.
  * [6] Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli,  “wav2vec 2.0: A framework for self-supervised learning of speech representations,”  Advances in Neural Information Processing Systems, vol. 33, pp. 12449–12460, 2020.
  * [7] Sanyuan Chen et al.,  “Wavlm: Large-scale self-supervised pre-training for full stack speech processing,”  IEEE Journal of Selected Topics in Signal Processing, vol. 16, no. 6, pp. 1505–1518, 2022.
  * [8] Sanyuan Chen et al.,  “Unispeech-sat: Universal speech representation learning with speaker aware pre-training,”  in ICASSP, 2022, pp. 6152–6156.
  * [9] Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed,  “Hubert: Self-supervised speech representation learning by masked prediction of hidden units,”  IEEE TASLP, vol. 29, pp. 3451–3460, 2021.
  * [10] Morgane Riviere, Armand Joulin, Pierre-Emmanuel Mazaré, and Emmanuel Dupoux,  “Unsupervised pretraining transfers well across languages,”  in ICASSP, 2020, pp. 7414–7418.
  * [11] Daisuke Niizumi, Daiki Takeuchi, Yasunori Ohishi, Noboru Harada, and Kunio Kashino,  “Byol for audio: Exploring pre-trained general-purpose audio representations,”  arXiv, 2022.
  * [12] Gasser Elbanna et al.,  “Hybrid Handcrafted and Learnable Audio Representation for Analysis of Speech Under Cognitive and Physical Load,”  in Proc. Interspeech, 2022, pp. 386–390.
  * [13] Neil Scheidwasser-Clow, Mikolaj Kegler, Pierre Beckmann, and Milos Cernak,  “Serab: A multi-lingual benchmark for speech emotion recognition,”  in ICASSP, 2022, pp. 7697–7701.
  * [14] Francois G Germain, Qifeng Chen, and Vladlen Koltun,  “Speech denoising with deep feature losses,”  2019, pp. 2723–2727.
  * [15] Zili Huang, Shinji Watanabe, Shu-wen Yang, Paola García, and Sanjeev Khudanpur,  “Investigating self-supervised learning for speech enhancement and separation,”  in ICASSP, 2022, pp. 6837–6841.
  * [16] Kuo-Hsuan Hung, Szu wei Fu, Huan-Hsin Tseng, Hsin-Tien Chiang, Yu Tsao, and Chii-Wann Lin,  “Boosting Self-Supervised Embeddings for Speech Enhancement,”  in Proc. Interspeech, 2022, pp. 186–190.
  * [17] Soha A Nossier, Julie Wall, Mansour Moniri, Cornelius Glackin, and Nigel Cannings,  “Mapping and masking targets comparison using different deep learning based speech enhancement architectures,”  in IJCNN, 2020, pp. 1–8.
  * [18] Hyeong-Seok Choi, Juheon Lee, Wansoo Kim, Jie Lee, Hoon Heo, and Kyogu Lee,  “Neural analysis and synthesis: Reconstructing speech from self-supervised representations,”  Advances in Neural Information Processing Systems, vol. 34, pp. 16251–16265, 2021.
  * [19] Aaron van den Oord et al.,  “Wavenet: A generative model for raw audio,”  arXiv, 2016.
  * [20] Jungil Kong, Jaehyeon Kim, and Jaekyoung Bae,  “Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis,”  Advances in Neural Information Processing Systems, vol. 33, pp. 17022–17033, 2020.
  * [21] Hakan Erdogan, John R. Hershey, Shinji Watanabe, and Jonathan Le Roux,  “Phase-sensitive and recognition-boosted speech separation using deep recurrent neural networks,”  in ICASSP, 2015, pp. 708–712.
  * [22] Adam Polyak, Lior Wolf, Yossi Adi, Ori Kabeli, and Yaniv Taigman,  “High fidelity speech regeneration with application to speech enhancement,”  in ICASSP, 2021, pp. 7143–7147.
  * [23] Jiaqi Su, Zeyu Jin, and Adam Finkelstein,  “Hifi-gan-2: Studio-quality speech enhancement via generative adversarial networks conditioned on acoustic features,”  in WASPAA, 2021, pp. 166–170.
  * [24] Shu wen Yang et al.,  “Superb: Speech processing universal performance benchmark,”  in Proc. Interspeech, 2021, pp. 1194–1198.
  * [25] Limin Wang et al.,  “Temporal segment networks: Towards good practices for deep action recognition,”  in ECCV, 2016, pp. 20–36.
  * [26] Chandan K.A. Reddy et al.,  “The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Testing Framework, and Challenge Results,”  in Proc. Interspeech, 2020, pp. 2492–2496.
  * [27] ITU-R,  “ITU-R BS.1770-4: Algorithms to measure audio programme loudness and true-peak audio level,” 2015.
  * [28] Cassia Valentini-Botinhao,  “Noisy speech database for training speech enhancement algorithms and tts models,”  2017\.
  * [29] Ashutosh Pandey and DeLiang Wang,  “Dense cnn with self-attention for time-domain speech enhancement,”  IEEE TASLP, vol. 29, pp. 1270–1279, 2021.
  * [30] Cees H Taal, Richard C Hendriks, Richard Heusdens, and Jesper Jensen,  “An algorithm for intelligibility prediction of time–frequency weighted noisy speech,”  IEEE TASLP, vol. 19, no. 7, pp. 2125–2136, 2011.
  * [31] ITU-R,  “P.862 : Perceptual evaluation of speech quality (PESQ): An objective method for end-to-end speech quality assessment of narrow-band telephone networks and speech codecs,” 2015.
  * [32] Gabriel Mittag, Babak Naderi, Assmaa Chehadi, and Sebastian Möller,  “NISQA: A deep CNN-self-attention model for multidimensional speech quality prediction with crowdsourced datasets,”  in Proc. Interspeech, 2021, pp. 2127–2131.
  * [33] Chandan K A Reddy, Vishak Gopal, and Ross Cutler,  “Dnsmos p.835: A non-intrusive perceptual objective speech quality metric to evaluate noise suppressors,”  in ICASSP, 2022, pp. 886–890.
  * [34] Yi Hu and Philipos C. Loizou,  “Evaluation of objective quality measures for speech enhancement,”  IEEE TASLP, vol. 16, no. 1, pp. 229–238, 2008.
  * [35] ITU-R,  “P.808 : Subjective evaluation of speech quality with a crowdsourcing approach,” 2015.
  * [36] Yoav Benjamini and Daniel Yekutieli,  “The control of the false discovery rate in multiple testing under dependency,”  Annals of statistics, pp. 1165–1188, 2001.