[affiliation=1]WarisQuamer [affiliation=1]RicardoGutierrez-Osuna

# End-to-end Streaming model for Low-Latency Speech Anonymization

###### Abstract

Speaker anonymization aims to conceal cues to speaker identity while
preserving linguistic content. Current machine learning based approaches
require substantial computational resources, hindering real-time streaming
applications. To address these concerns, we propose a streaming model that
achieves speaker anonymization with low latency. The system is trained in an
end-to-end autoencoder fashion using a lightweight content encoder that
extracts HuBERT-like information, a pretrained speaker encoder that extract
speaker identity, and a variance encoder that injects pitch and energy
information. These three disentangled representations are fed to a decoder
that re-synthesizes the speech signal. We present evaluation results from two
implementations of our system, a full model that achieves a latency of 230ms,
and a lite version (0.1x in size) that further reduces latency to 66ms while
maintaining state-of-the-art performance in naturalness, intelligibility, and
privacy preservation.

###### keywords:

speaker anonymization, voice conversion, voice privacy, speech synthesis

##  1 Introduction

The task of speaker anonymization is to transform utterances to hide the
identity of the speaker (while preserving their linguistic content). Speaker
anonymization provides privacy protection and confidentiality in a range of
applications, including customer service interactions, voice-operated virtual
assistants, legal proceedings, and medical consultations. Moreover, speaker
anonymization addresses ethical and responsible use of speech data, aligning
with privacy regulations and safeguarding individuals' rights.

Existing machine learning (ML) based approaches to speaker anonymization
follow a cascaded ASR-TTS architecture [1, 2]. An ASR module produces a text
transcription that is speaker independent but eliminates emotional cues that
may otherwise be of use for downstream applications. Moreover, existing
systems for speaker anonymization are computationally heavy, operate in a non-
streaming fashion, and/or have high latency on CPU devices. For speech
anonymization to be used in the field, it must operate at real time (or
faster), exhibit low latency, require minimal future context and be compatible
with low-resource devices (e.g., smartphones).

To address these needs, we propose an end-to-end streaming model suitable for
low-latency speaker anonymization. Our model draws inspiration from neural
audio codecs [3, 4] for audio compression in low-resource streaming settings.
Our proposed architecture consists of: (a) a streaming waveform encoder that
generates a speaker-independent content representation from raw waveforms, (b)
a pseudo-speaker generator that produces an anonymized speaker representation
(or embedding) from the input speech, (c) a variance adapter that adds
speaker, pitch and energy information to the content representation, and (d) a
streaming decoder that consumes the output of the variance adapter and the
corresponding speaker embedding to generate the final anonymized audio
waveform.

Our system is trained as an autoencoder to reconstruct the input using
pretrained speaker encoders [5, 6]. During inference, a pseudo-speaker
generator produces a target speaker embedding with cosine distance greater
than 0.3 from the source embedding, ensuring that the re-synthesized utterance
sounds as if a different (i.e., anonymized) speaker had produced it.
Additionally, the variance adapter is used to modulate pitch and energy values
to further enhance privacy and control their similarity with the source audio.
We show that our proposed system consisting of lightweight causal
convolutional network can achieve similar performances as non-causal
transformer networks which are computationally intensive.

We perform experiments on two versions of our model, a base version that can
perform real-time streaming synthesis with a latency of 230ms and a lite
version (having 0.1x the number of parameters) that further reduces latency to
66ms while maintaining state-of-the-art performance on naturalness,
intelligibility, privacy and speaker identity transfer111
https://warisqr007.github.io/demos/stream-anonymization.

##  2 Related Work

###  2.1 Voice conversion

Speaker anonymization is closely related to voice conversion (VC). However,
whereas VC seeks to transform utterances from a source speaker to match the
identity of a (known) target speaker, speaker anonymization only requires that
the transformed speech be sufficiently different from the source speaker to
conceal their identity.

The first step in conventional VC architectures is to disentangle the
linguistic content of an utterance from speaker-specific attributes. As an
example, cascaded ASR-TTS architectures [7] use an ASR model to transcribe the
input utterance into text, followed by a TTS model that converts the text back
into speech –conditioned on a speaker embedding. Variants of this approach
replace the ASR module with acoustic models that generate a more fine-grained
representation than text, such as phonetic posteriorgrams (PPGs) [8]. Recent
approaches have also used information bottlenecks to disentangle linguistic
content form speaker identity [9]. A major drawbacks of the latter approach is
that information bottlenecks must be carefully designed and are sensitive to
the dimension of latent space. Other techniques include instance normalization
[10], use of mutual information loss [11], vector quantization [11, 12], and
adversarial training [13]. To enable streaming, recent VC methods use a
streaming ASR to extract PPGs [14] or streaming ASR sub-encoders [15, 16] to
generate linguistic content, and then perform VC through causal architectures
that require limited future contexts.

###  2.2 Speaker anonymization

Speaker anonymization approaches can be broadly divided into two categories:
digital signal processing (DSP) and machine learning (ML) based. DSP methods
include formant-shifting using McAdams coefficients [17], frequency warping
[18], or a series of steps consisting of vocal tract length normalization,
McAdams transformation and modulation spectrum smoothing [19]. Additionally,
modifications to pitch [20] and speaking rate [21] are used. DSP models are
significantly smaller (i.e., fewer parameters) than ML models, which results
in efficient and speedy execution. However, the types of global transforms
used in DSP methods cannot fully remove speaker-dependent cues, making them
vulnerable to ML-based speaker verification systems [1].

ML methods for speaker anonymization follow the conventional VC framework of
disentangling linguistic content from speaker identity, but then replace the
latter with a speaker embedding that is different (anonymized) from the
source. Various methods have been proposed to select this anonymized speaker
embedding. For example, Srivastava et al. [22] generate anonymized embeddings
by randomly selecting N speaker vectors from a pool of speakers farthest from
the source, using e.g., cosine distance, whereas Perero-Codosero et al. [23]
use an autoencoder architecture with an adversarial training module that
removes speaker, gender, and accent information. Other approaches use look-up
tables [24] or generative adversarial networks [1] to generate pseudo-
speakers. Our approach follows the latter: we combines a GAN-based pseudo-
speaker generator with a streaming model to enable real-time speaker
anonymization with low latency.

##  3 Method

The proposed system is illustrated in Figure 1. A reference waveform from the
source speaker is passed to a pretrained speaker encoder to generate the
source speaker embedding. The pseudo speaker generator receives the source
speaker embedding and generates an anonymized speaker embedding. The content
encoder receives streaming chunks of raw waveform and converts it into a
hidden representation $z$ that contains the linguistic content disentangled
from the speaker representation. The content information $z$ and the
anonymized speaker embedding is passed to the variance adapter. The variance
adapter injects pitch and energy values and then conditions the anonymized
speaker embedding on the content representation $z$. The decoder receives the
output of the variance adapter and the anonymized speaker embedding to
generate the final anonymized waveform. We train two versions of our proposed
system, a base and a lite version. Below, we describe each component of our
system and the training procedure in detail.

Figure 1: Block diagram of the proposed anonymization system. (a) training
workflow (b) inference workflow.

###  3.1 Encoder

Similar to [25], the content encoder predicts discrete speech units produced
by discretizing the output speech representation from a pretrained HuBERT
model [26] into one of N codewords or pseudo-labels. Our content encoder
architecture follows that of HiFiGAN [27], except all transposed convolutions
in HiFiGAN are replaced with strided causal convolutions to downsample the
input waveform. Additionally, to support streaming applications, we replace
all vanilla CNN layers in HiFiGAN with causal CNNs so that the prediction only
considers the past context and does not rely on future audio frames. For both
versions of our model (base and light), we use downsampling rates of
$[2,2,4,4,5]$. The residual blocks have kernel sizes as $[3,7,11]$ with
dilation rates as $[[[1,1],[3,1],[5,1]]*3]$ (please refer [27] for details).
The difference between base and the lite version is the dimension of the
hidden representation $z$ (the output of the encoder): 512 dimensions for the
base version and 128 dimensions for the lite version.

###  3.2 Speaker embedding and speaker adapter

Speaker verification or classification systems generally use speaker
embeddings to represent the characteristics or timbre of a speaker's voice.
Widely used speaker encoders include the GE2E model [28], X-vectors [6] and
ECAPA-TDNN [5]. Our systems concatenates embeddings generated from X-vectors
and ECAPA-TDNN models, since these two models have been shown to complement
each other [1].

We use a speaker adapter to condition the speaker embedding on the latent
representations. The speaker adapter is based on adaptive instance
normalization (adaIN) [29] and feature-wise linear modulation (FiLM) [30]. The
conditioning goes as follows. First, we apply instance normalization to the
input feature representation, and then transform it with scale and bias
parameters learned through two 1D CNNs that take speaker embeddings as input.

###  3.3 Pseudo-speaker Generator

To perform speaker anonymization, we use a pseudo-speaker generator that takes
the original speaker embedding as input and outputs an artificially generated
speaker embedding such that the generated anonymized speaker embedding has a
cosine distance greater than 0.3 as compared with the original speaker
embedding. Our pseudo-speaker generator follows the GAN-based framework
proposed in [1] and is trained separately. The generator is trained to receive
a random vector sampled from a standard normal distribution $N(0,1)$ as input
and output a vector of the same shape as the original speaker embedding. The
discriminator is trained to discriminate w.r.t the quadratic Wasserstein
distance and transport cost [31] between the artificial and the original
speaker embeddings.

###  3.4 Variance Adapter

The variance adapter aims to add speaker, pitch, and energy (i.e., variance)
information to the speaker-independent content representation and provides a
way to control them [32]. The variance adapter consists of three modules: (a)
speaker adapter, (b) pitch predictor, and (c) energy predictor. The speaker
adapter conditions the speaker embedding on the content representation, and
passes it to the pitch and energy predictors. During training, we use the
ground-truth pitch and energy values to train the pitch and energy predictors.
At inference time, we predict pitch and energy from the hidden embedding $z$.
The predicted pitch and energy values are then conditioned on the $z$. The
pitch and energy predictors have similar architecture consisting of a 2-layer
1D causal CNN with ReLU activation, followed by layer normalization and
dropout layer and an additional 1D CNN (with kernel size 1) to project pitch
and energy values on the content representation.

###  3.5 Decoder

The decoder follows the same design as HiFiGAN [27] and can be seen as a
mirror-image of the content encoder with causal CNNs. Additionally, we
condition the speaker embedding through the speaker adapter at the output of
each residual block. For both versions of our model, we use upsampling rates
of $[5,4,4,2,2]$. The residual blocks have kernel sizes as $[3,7,11]$ with
dilation rates as $[[[1,1],[3,1],[5,1]]*3]$.

###  3.6 Training

The content encoder is trained to predict the pseudo-labels at the output of
the HuBERT module using cross-entropy loss. We prevent gradient flow (i.e.,
back-propagation) from the decoder to the encoder, to ensure that the speaker
information is not leaked via the content representation. The pitch and energy
predictor in the variance adapter apply mean-squared error loss for pitch and
energy prediction. At the output of the decoder, Following the HiFiGAN
architecture [27], we apply a combination of adversarial losses at the output
of the decoder, including feature loss, multi-period discriminator loss,
multi-scale discriminator loss, multi-resolution STFT loss [33] and Mel-
spectrogram reconstruction loss. These discriminators have similar
architecture as those in HiFiGAN, including similar weighting schemes to
compute the total loss.

##  4 Experimental setup

We trained our system on the LibriTTS corpus [34] following guidelines for the
Voice Privacy Challenge 2022 (VPC22) [35]. All our experimental results are
presented on the LibriTTS dev and test set, which were not part of the
training. We use a pretrained HuBERT
[26]222https://github.com/facebookresearch/fairseq and extract the the output
from its 9th layer. We set the number of cluster centroids to 200. For all our
experiments, we use a sampling rate of 16 kHz and batch size of 16 with the
AdamW optimizer with a learning rate of $2*10^{-4}$ annealed down to $10^{-5}$
by exponential scheduling. The encoder is first pretrained for 300k steps (for
training stability), and then trained together with the decoder for an
additional 800k steps. The pretrained speaker encoders were taken from
speechbrain [36]. The pseudo speaker embedding generator follows the training
procedure described in [1]. All our models are trained using two NVIDIA Tesla
V100 GPUs.

##  5 Results

We evaluated our system on a series of subjective and objective measures of
synthesis latency, synthesis quality, privacy as well as speaker transfer
ability. We compare our results against five baselines: three state-of-the-art
VC models (VQMIVC333https://github.com/Wendison/VQMIVC [11],
QuickVC444https://github.com/quickvc/QuickVC-VoiceConversion [37], and
DiffVC555https://github.com/huawei-noah/Speech-Backbones/tree/main/DiffVC
[38]) and two speaker anonymization models666https://github.com/Voice-Privacy-
Challenge/Voice-Privacy-Challenge-2024, a DSP-based model [17] (baseline B2
from VPC22) and a ML-based model (to which we refer as B3) that uses a
transformer-based ASR and a Fastspeech2-based TTS with a WGAN-based anonymizer
[1]. The five baseline models are trained on the same dataset as our proposed
system, and we use their pretrained checkpoints obtained from their
corresponding github repositories. We could not find any open-source streaming
speech synthesis model and hence were unable to include them as baselines. We
evaluate our model using the same Libri-TTS evaluation split as VPC22. For the
VC baselines we randomly select a speaker from the CMU Arctic corpus [39] as
the target speaker.

###  5.1 Synthesis Latency

Our pretrained HuBERT model produces speech frames at 50Hz, so the smallest
chunk size that our model can process is 20 ms. In this section, we present
the synthesis latency for the base and lite versions for our model, for
various chunk sizes between 20ms and 140ms on both CPU and GPU devices. We
define latency as the sum of chunk size and the average time taken by the
model to synthesize the corresponding chunk. For a system to be real-time, the
latency should be less than twice the chunk size. Results are summarized in
Table 1. On GPU, our base version can operate in real-time for the chunk size
of 40ms with a latency of 64ms, while on CPU the base model can be real-time
for chunk size of 120ms with a latency of 230ms. In case of the lite version,
the model is real-time for chunk size of 20ms with a latency of 38ms on GPU
and can operate in real-time for 40ms with latency of 66ms on the CPU. For our
next set of experiments we set the chunk size of 120ms and 40ms for the base
and lite versions respectively.

Table 1: Synthesis latency on CPU and GPU Chunk Size |  |  CPU (ms)
---
|  GPU (ms)
---
(ms) | base | lite | base | lite
20 | 83.82 | 43.01 | 44.61 | 38.36
40 | 115.23 | 66.27 | 64.38 | 58.70
60 | 170.93 | 88.03 | 84.03 | 79.55
100 | 230.23 | 136.60 | 124.19 | 118.92
120 | 229.75 | 159.22 | 144.16 | 139.41
140 | 249.38 | 173.49 | 163.90 | 158.90

###  5.2 Synthesis Quality

We use DNSMOS [40] as an objective measure of naturalness for our experiments.
DNSMOS provides three ratings for quality of speech (SIG), noise (BAK), and
overall (OVRL). Additionally, we assess the intelligibility of synthesized
speech through Word Error Rate (WER). Table 2, summarizes results for the five
baselines and the proposed systems. In terms of DNSMOS, our models achieve
similar ratings as Diff-VC, QuickVC, and B3 across the three measures, and
comparable or better than the source speech. In terms of intelligibility, our
systems achieve comparable WER to that of B3, and superior to the rest, even
though our models operate in a causal fashion with far more limited context.

We verified the synthesis quality of our two models through listening tests on
Amazon Mechanical Turk (AMT). Namely, participants (N=20) were asked to rate
the acoustic quality of utterances using a standard 5-point scale mean opinion
score (MOS) (5: excellent, 1: bad). Each listener rated utterances synthesized
using the base and lite models, as well as original utterances (20 for each).
Results are shown in Table 3. Both systems (base and lite) obtained comparable
ratings of MOS as the original utterances. We see a difference of 0.1 between
the MOS score of base and lite versions, but we didn't find them to be
significant (p$\ll$ 0.001). It is noteworthy that while the lite version has
0.1x number of parameters, it achieves nearly the same synthesis quality as
the base version.

Table 2: DNS MOS, Word Error Rates (WER) and speaker similarity scores (SSS) for baselines and the proposed model. | SIG | BAK | OVRL | WER | SSS
---|---|---|---|---|---
VQMIVC | 3.35 | 3.78 | 2.98 | 26.88 | 0.72
Quick-VC | 3.55 | 4.06 | 3.28 | 6.30 | 0.68
Diff-VC | 3.62 | 4.17 | 3.40 | 7.64 | 0.82
VPC22 B2 | 2.85 | 3.44 | 2.50 | 12.02 | -
B3 | 3.57 | 4.03 | 3.28 | 4.65 | -
base | 3.53 | 3.99 | 3.31 | 5.12 | 0.85
lite | 3.48 | 3.93 | 3.22 | 6.47 | 0.81
Source | 3.58 | 3.99 | 3.26 | 2.98 | 0.89
Table 3: Subjective MOS for naturalness. | Source | base | lite
---|---|---|---
MOS |  3.54 $\pm$ 0.56 |  3.57 $\pm$ 0.59 |  3.47 $\pm$ 0.62

###  5.3 Speaker Anonymization

To assess speaker-anonymization performance, we report Equal Error Rate (EER)
on the speaker verification model (ASV) in the VoicePrivacy 2024 Challenge
github (see section 4). ASV tests are conducted for the following two
scenarios, (a) ignorant, where we only anonymize the trial data (O-A), or (b)
lazy-informed, where we anonymize both enrollment and trial data but use
different target speakers (A-A). Results are shown in Table 4. For both the
ignorant and lazy-informed scenario, our models achieves similar performance
as B3 and outperforms VPC22 B2. Although our base model performs slightly
worse than B3, the differences are not significantly different between them
(p$\ll$ 0.001).

To corroborate these results, we conducted an AB listening test on AMT.
Participants were presented with two audio samples, one from a speaker in the
enrollment set, and the second sample from one of three options: (a) a
different utterance from the same speaker from the trial set, (b) an utterance
from a different speaker from the trial set, or (c) another utterance of the
same speaker from the trial set but anonymized through our lite version of the
system. Then, participants had to decide if both samples were from the same
speaker, and rate the confidence in their decision using a 7-point scale (7:
extremely confident; 5: quite a bit confident; 3: somewhat confident; 1: not
confident at all). Each listener rated 20 AB pairs per scenario. Results are
summarized in Table 5. In settings (a) and (b), listeners could easily
identify whether the recording were from the same or different speakers (81 %
) with high confidence (5.71). In setting (c) However, the anonymized trial
data obtained obtained similar rating as in (b), indicating that proposed
system was able to anonymize the trial recordings.

Table 4: Equal Error Rate (EER) as a privacy metric. The higher the better. |  | VPC22 B2 | B3 | base | lite
---|---|---|---|---|---
O-A | M | 25.10 | 44.24 | 43.83 | 42.57
| F | 37.42 | 47.78 | 46.87 | 45.31
A-A | M | 11.03 | 42.63 | 41.43 | 39.20
| F | 15.03 | 43.23 | 42.03 | 41.16
Table 5: Subjective speaker verifiability scores for the proposed model. speaker | anon | Verifiability | Confidence Rating
---|---|---|---
same | no | 81.5% | 5.71
different | no | 17.75% | 2.50
same | yes | 14.5% | 2.37

###  5.4 Speaker Identity transfer

In a final step, we evaluated our models' ability to capture the voice of a
target speaker. For this purpose, we used an objective score of speaker
similarity based on the cosine similarity between speaker embeddings of the
target and the synthesized utterances777Computed using:
https://github.com/resemble-ai/Resemblyzer. We compare our model against the
three VC baselines (VQMIVC [11], QuickVC [37], DiffVC [38]) using the same
settings as those in section 5.2) to generate VC samples. Results are
summarized in the rightmost column of Table 2 (SSS). As a guideline, pairing
two utterances from the same speaker yields an average cosine similarity of
0.89. As shown, our base model outperforms the three baselines, achieving
cosine similarity that it close to the average within-speaker similarity of
0.89.

##  6 Discussion

Most existing speaker anonymization methods do not operate in low-latency
streaming mode, preventing their use in field operations. In this paper, we
present an end-to-end streaming model that operates with low latency and
achieves anonymization by mapping speaker embedding into an artificially
generated pseudo speaker in a causal fashion (i.e., no future context). While
there exists a quality-latency tradeoff, our system can achieve latency s low
as 66ms while maintaining state-of-the-art naturalness, intelligibility and
privacy preservation. Our lite version is roughly 10MB and can potentially be
deployed on mobile devices to support real-time field applications. Accent can
carry speaker related cues [41] and in future work, we aim to add accent
conversion to this pipeline. Other research direction is to add the control of
emotion while synthesizing speech signals.

##  7 Acknowledgements

This work was funded by NSF award 619212 and 1623750.

## References

  * [1] S. Meyer, P. Tilli, P. Denisov, F. Lux, J. Koch, and N. T. Vu, ``Anonymizing speech with generative adversarial networks to preserve speaker privacy,'' in _2022 IEEE SLT Workshop_ , 2023, pp. 912–919.
  * [2] C. Pierre, A. Larcher, and D. Jouvet, ``Are disentangled representations all you need to build speaker anonymization systems?'' in _Proc. Interspeech_ , 2022, pp. 2793–2797.
  * [3] N. Zeghidour, A. Luebs, A. Omran, J. Skoglund, and M. Tagliasacchi, ``Soundstream: An end-to-end neural audio codec,'' _IEEE/ACM TASLP_ , vol. 30, pp. 495–507, 2021.
  * [4] A. Défossez, J. Copet, G. Synnaeve, and Y. Adi, ``High fidelity neural audio compression,'' _arXiv preprint arXiv:2210.13438_ , 2022.
  * [5] B. Desplanques, J. Thienpondt, and K. Demuynck, ``Ecapa-tdnn: Emphasized channel attention, propagation and aggregation in tdnn based speaker verification,'' in _Proc. INTERSPEECH_ , 2020, pp. 3830–3834.
  * [6] D. Snyder, D. Garcia-Romero, G. Sell, D. Povey, and S. Khudanpur, ``X-vectors: Robust dnn embeddings for speaker recognition,'' in _Proc. ICASSP_ , 2018, pp. 5329–5333.
  * [7] W.-C. Huang, T. Hayashi, S. Watanabe, and T. Toda, ``The sequence-to-sequence baseline for the voice conversion challenge 2020: Cascading asr and tts,'' _arXiv preprint arXiv:2010.02434_ , 2020.
  * [8] S. Liu, Y. Cao, D. Wang, X. Wu, X. Liu, and H. Meng, ``Any-to-many voice conversion with location-relative sequence-to-sequence modeling,'' _IEEE/ACM TASLP_ , vol. 29, pp. 1717–1728, 2021.
  * [9] C. H. Chan, K. Qian, Y. Zhang, and M. Hasegawa-Johnson, ``Speechsplit2. 0: Unsupervised speech disentanglement for voice conversion without tuning autoencoder bottlenecks,'' in _Proc. ICASSP_ , 2022, pp. 6332–6336.
  * [10] Y.-H. Chen, D.-Y. Wu, T.-H. Wu, and H.-y. Lee, ``Again-vc: A one-shot voice conversion using activation guidance and adaptive instance normalization,'' in _Proc. ICASSP_ , 2021, pp. 5954–5958.
  * [11] D. Wang, L. Deng, Y. T. Yeung, X. Chen, X. Liu, and H. Meng, ``VQMIVC: Vector Quantization and Mutual Information-Based Unsupervised Speech Representation Disentanglement for One-Shot Voice Conversion,'' in _Proc. Interspeech_ , 2021, pp. 1344–1348.
  * [12] W. Quamer, A. Das, and R. Gutierrez-Osuna, ``Decoupling Segmental and Prosodic Cues of Non-native Speech through Vector Quantization,'' in _Proc. INTERSPEECH_ , 2023, pp. 2083–2087.
  * [13] S. Ding, G. Zhao, and R. Gutierrez-Osuna, ``Improving the speaker identity of non-parallel many-to-many voice conversion with adversarial speaker recognition.'' in _Proc. INTERSPEECH_ , 2020.
  * [14] T. N. D. Tran, T. C. Chu, V. Hoang, T. H. Bui, and H. Q. Truong, ``An Efficient and High Fidelity Vietnamese Streaming End-to-End Speech Synthesis,'' in _Proc. Interspeech_ , 2022, pp. 466–470.
  * [15] H. Yang, L. Deng, Y. T. Yeung, N. Zheng, and Y. Xu, ``Streamable speech representation disentanglement and multi-level prosody modeling for live one-shot voice conversion.'' in _Proc. INTERSPEECH_ , 2022, pp. 2578–2582.
  * [16] Y. Chen, M. Tu, T. Li, X. Li, Q. Kong, J. Li, Z. Wang, Q. Tian, Y. Wang, and Y. Wang, ``Streaming voice conversion via intermediate bottleneck features and non-streaming teacher guidance,'' in _Proc. ICASSP_ , 2023, pp. 1–5.
  * [17] J. Patino, N. Tomashenko, M. Todisco, A. Nautsch, and N. Evans, ``Speaker Anonymisation Using the McAdams Coefficient,'' in _Proc. Interspeech_ , 2021, pp. 1099–1103.
  * [18] J. Qian, H. Du, J. Hou, L. Chen, T. Jung, and X.-Y. Li, ``Speech sanitizer: Speech content desensitization and voice anonymization,'' _IEEE Transactions on Dependable and Secure Computing_ , vol. 18, no. 6, pp. 2631–2642, 2019.
  * [19] H. Kai, S. Takamichi, S. Shiota, and H. Kiya, ``Lightweight and irreversible speech pseudonymization based on data-driven optimization of cascaded voice modification modules,'' _Computer Speech & Language_, vol. 72, p. 101315, 2022.
  * [20] L. Tavi, T. Kinnunen, and R. G. Hautamäki, ``Improving speaker de-identification with functional data analysis of f0 trajectories,'' _Speech Communication_ , vol. 140, pp. 1–10, 2022.
  * [21] S. P. Dubagunta, R. J. van Son, and M. M. Doss, ``Adjustable deterministic pseudonymization of speech,'' _Computer Speech & Language_, vol. 72, p. 101284, 2022.
  * [22] B. M. L. Srivastava, N. Tomashenko, X. Wang, E. Vincent, J. Yamagishi, M. Maouche, A. Bellet, and M. Tommasi, ``Design Choices for X-Vector Based Speaker Anonymization,'' in _Proc. Interspeech_ , 2020, pp. 1713–1717.
  * [23] J. M. Perero-Codosero, F. M. Espinoza-Cuadros, and L. A. Hernández-Gómez, ``X-vector anonymization using autoencoders and adversarial training for preserving speech privacy,'' _Computer Speech & Language_, vol. 74, p. 101351, 2022.
  * [24] J. Yao, Q. Wang, L. Zhang, P. Guo, Y. Liang, and L. Xie, ``NWPU-ASLP System for the VoicePrivacy 2022 Challenge,'' in _Proc. 2nd Symposium on Security and Privacy in Speech Communication_ , 2022.
  * [25] B. van Niekerk, M.-A. Carbonneau, J. Zaïdi, M. Baas, H. Seuté, and H. Kamper, ``A comparison of discrete and soft speech units for improved voice conversion,'' in _Proc. ICASSP_ , 2022, pp. 6562–6566.
  * [26] W.-N. Hsu, B. Bolte, Y.-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, ``Hubert: Self-supervised speech representation learning by masked prediction of hidden units,'' _IEEE/ACM TASLP_ , vol. 29, pp. 3451–3460, 2021.
  * [27] J. Kong, J. Kim, and J. Bae, ``Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis,'' _Proc NeurIPS_ , vol. 33, pp. 17 022–17 033, 2020.
  * [28] L. Wan, Q. Wang, A. Papir, and I. L. Moreno, ``Generalized end-to-end loss for speaker verification,'' in _Proc. ICASSP_ , 2018, pp. 4879–4883.
  * [29] X. Huang and S. Belongie, ``Arbitrary style transfer in real-time with adaptive instance normalization,'' in _Proc. ICCV_ , 2017, pp. 1501–1510.
  * [30] E. Perez, F. Strub, H. De Vries, V. Dumoulin, and A. Courville, ``Film: Visual reasoning with a general conditioning layer,'' in _Proc. AAAI_ , vol. 32, no. 1, 2018.
  * [31] H. Liu, X. Gu, and D. Samaras, ``Wasserstein gan with quadratic transport cost,'' in _Proc. CVF_ , 2019, pp. 4832–4841.
  * [32] Y. Ren, C. Hu, X. Tan, T. Qin, S. Zhao, Z. Zhao, and T.-Y. Liu, ``Fastspeech 2: Fast and high-quality end-to-end text to speech,'' _arXiv preprint arXiv:2006.04558_ , 2020.
  * [33] R. Yamamoto, E. Song, and J.-M. Kim, ``Parallel wavegan: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram,'' in _Proc. ICASSP_ , 2020, pp. 6199–6203.
  * [34] H. Zen, V. Dang, R. Clark, Y. Zhang, R. J. Weiss, Y. Jia, Z. Chen, and Y. Wu, ``Libritts: A corpus derived from librispeech for text-to-speech,'' _arXiv preprint arXiv:1904.02882_ , 2019.
  * [35] N. Tomashenko, X. Wang, X. Miao, H. Nourtel, P. Champion, M. Todisco, E. Vincent, N. Evans, J. Yamagishi, and J.-F. Bonastre, ``The voiceprivacy 2022 challenge evaluation plan,'' _arXiv preprint arXiv:2203.12468_ , 2022.
  * [36] M. Ravanelli, T. Parcollet, P. Plantinga, A. Rouhe, S. Cornell, L. Lugosch, C. Subakan, N. Dawalatabad, A. Heba, J. Zhong _et al._ , ``Speechbrain: A general-purpose speech toolkit,'' _arXiv preprint arXiv:2106.04624_ , 2021.
  * [37] H. Guo, C. Liu, C. T. Ishi, and H. Ishiguro, ``Quickvc: A lightweight vits-based any-to-many voice conversion model using istft for faster conversion,'' in _Proc. ASRU_ , 2023, pp. 1–7.
  * [38] V. Popov, I. Vovk, V. Gogoryan, T. Sadekova, M. Kudinov, and J. Wei, ``Diffusion-based voice conversion with fast maximum likelihood sampling scheme,'' in _Proc. ICLR_ , 2022.
  * [39] J. Kominek and A. W. Black, ``The cmu arctic speech databases,'' in _Fifth ISCA workshop on speech synthesis_ , 2004.
  * [40] C. K. Reddy, V. Gopal, and R. Cutler, ``Dnsmos: A non-intrusive perceptual objective speech quality metric to evaluate noise suppressors,'' in _Proc. ICASSP_ , 2021, pp. 6493–6497.
  * [41] A. Das, G. Zhao, J. Levis, E. Chukharev-Hudilainen, and R. Gutierrez-Osuna, ``Understanding the effect of voice quality and accent on talker similarity.'' in _Proc. INTERSPEECH_ , 2020, pp. 1763–1767.