[affiliation=1]XuanjunChen [affiliation=1]HaibinWu [affiliation=2]Jyh-Shing
RogerJang [affiliation=1]Hung-yiLee

# Singing Voice Graph Modeling for SingFake Detection

###### Abstract

Detecting singing voice deepfakes, or SingFake, involves determining the
authenticity and copyright of a singing voice. Existing models for speech
deepfake detection have struggled to adapt to unseen attacks in this unique
singing voice domain of human vocalization. To bridge the gap, we present a
groundbreaking SingGraph model. The model synergizes the capabilities of the
MERT acoustic music understanding model for pitch and rhythm analysis with the
wav2vec2.0 model for linguistic analysis of lyrics. Additionally, we advocate
for using RawBoost and beat matching techniques grounded in music domain
knowledge for singing voice augmentation, thereby enhancing SingFake detection
performance. Our proposed method achieves new state-of-the-art (SOTA) results
within the SingFake dataset, surpassing the previous SOTA model across three
distinct scenarios: it improves EER relatively for seen singers by 13.2%, for
unseen singers by 24.3%, and unseen singers using different codecs by 37.1%.

###### keywords:

singing voice deepfake detection, anti-spoofing, singing voice separation

##  1 Introduction

The recent advancements in singing voice synthesis, exemplified by models such
as VISinger [1] and DiffSinger [2], have brought significant progress to the
field of singing voice generation. However, these developments have also
raised concerns among artists, record companies, and publishing houses. The
potential for unauthorized synthetic reproductions that closely imitate a
singer's voice threatens the commercial value of the original artists. As
society becomes increasingly aware of these issues, there is a pressing need
to develop accurate methods for detecting deepfake singing voices. This is
critical in safeguarding the rights and revenues of artists and ensuring the
integrity of musical productions.

Since the singing voice is a form of human vocalization, it is logical to seek
insights from the field of speech deepfake detection, commonly known as voice
spoofing countermeasures (CM). Research in the CM domain has made significant
progress, developing methods that effectively differentiate between genuine
speech and spoofing attacks (fake speech) [3, 4, 5, 6]. Some of these
breakthroughs have led to systems that achieve Equal Error Rates (EER) below
1% on the ASVspoof2019 test set [7]. However, despite these advances, there
remains a challenge in generalizing these systems to unseen attacks, such as
fake singing voices, and adapting to varied acoustic environments, as
evidenced by their reduced effectiveness when confronted with real-world data
[8, 9].

To highlight the importance of improving the detection of fake singing voices,
Zang et al. [10] introduced the task of singing voice deepfake (SingFake)
detection. Singing voice deepfake detection presents unique challenges not
found in speech deepfake detection. Unlike speech, singing involves adherence
to melodies and rhythms that alter the pitch and duration of phonemes.
Additionally, singing incorporates more varied artistic expressions and a
broader spectrum of timbre, heavily influenced by the musical context.
Furthermore, singing recordings often undergo significant editing and digital
signal processing and are combined with instrumental accompaniments. Given
these distinct characteristics of singing voices, the countermeasures designed
for speech are not suited for directly applied to deepfake singing voice
detection. How to design a SingFake detection model tailored for the singing
voice domain is still an open question.

In the paper, we introduce the SingGraph detection training framework for
detecting singing voice deepfakes. SingGraph combines two advanced models:
MERT [11], which understands music by focusing on pitch and rhythm, and
wav2vec2 [12], which captures phonemes in lyrics. This approach enables
separate analyses of musical instruments and vocals, we also demonstrate their
complementary nature, as evidenced by the experiments with models A4-A6 in
Table 1. Additionally, we are the first to introduce techniques like RawBoost
[13] and beat matching, based on music domain knowledge, to enhance SingFake
detection. Experiments on SingFake dataset reveal that SingGraph achieves
state-of-the-art (SOTA) performance in various scenarios (i.e., the seen test,
the unseen test, and the unseen codecs test).

Figure 1: The proposed SingGraph detection training framework.

##  2 Related work

In the SingFake dataset, Zang et al. [10] develop four SingFake detection
models based on four speech domain representative methods. First, AASIST [3]
utilizes the raw waveform as its feature input, employing graph neural
networks alongside spectro-temporal attention mechanisms to analyze the audio
data. Next, Spec-ResNet is designed around the use of a linear spectrogram,
which is obtained through a 512-point Fast Fourier Transform (FFT) with a hop
size of 10 ms. The resulting spectrogram is then processed using the ResNet18
[14] architecture, enabling the model to capture and analyze spectral features
effectively. LFCC-ResNet [15], on the other hand, opts for Linear-Frequency
Cepstral Coefficients (LFCC) as its speech feature. These 60-dimensional LFCCs
are extracted from each audio frame, with the frame length set at 20ms and a
hop size of 10ms, before being fed into the ResNet18 model. Lastly,
W2V2-AASIST [16] leverages the wav2vec2.0 model [12], a self-supervised
learning framework trained on extensive external speech datasets. This model
is particularly adept at extracting nuanced phonetic and linguistic features
from speech, providing a generalized representation. However, none of the
above methods consider capturing the musical information, such as pitch,
rhythms, and so on.

##  3 SingGraph detection training framework

The SingGraph detection training framework, as shown in Figure 1, consists of
two key components: singing voice augmentation techniques, including RawBoost
and Beat matching, and the core SingGraph model. We use
Demucs111https://github.com/facebookresearch/demucs to separate a mixed
singing song $x$ into instrumental $x_{ins}$ and vocal $x_{voc}$ components.
Following separation, we apply data augmentation to create augmented
instrumental $x^{\prime}\\_{ins}$ and vocal $x^{\prime}\\_{voc}$ tracks. The
SingGraph model then processes these augmented inputs to verify whether the
input is a genuine singing performance or a fake singing recording.

###  3.1 Singing voice augmentation

To boost performance, we leverage RawBoost for vocal augmentation and beat
matching for instrumental augmentation.

####  3.1.1 Beat matching for instrumental.

In this part, we will introduce a data augmentation method based on music
domain knowledge. Musical compositions consist of various elements, including
chord progressions, timbre, and beats, with beats being essential for setting
the musical structure and alignment. In the audio domain, mixup [17] is a
favored augmentation technique that blends pairs of audio tracks to expand the
training dataset. However, mixing instrumental and vocal tracks with differing
tempos can lead to a disordered and unattractive outcome. We propose the beat
matching for instrumental augmentation to correspond to vocals as shown in
Figure 1-(b). Firstly, we employ a cutting-edge beat tracking model ALL-in-one
[18] to determine each music track's tempo and downbeat map. We then organize
each track into tempo-specific groups. During training, we randomly select an
instrumental $x^{\prime}_{ins}$ to replace the original pair instrumental
$x_{ins}$ from the same tempo group to maintain consistent tempos.
Additionally, we conduct downbeat alignment to choose an appropriate downbeat
as the starting point for mixing singing voices. This pre-processing strategy
enhances our ability to choose suitable music data for mixing, creating tracks
that are harmoniously aligned in terms of tempo and downbeat.

####  3.1.2 RawBoost for vocal.

We introduce a trick, inspired by stationary signal-independent additive noise
with random coloration. This trick significantly boosts speech deepfake
detection by RawBoost[13, 19]. Therefore, it is reasonable to adopt it for
vocal augmentation in SingFake scenario, as the vocal part of singing also
belongs to speech. As depicted in Figure 1-(a), the vocal input of the model,
$x^{\prime}{voc}$, is generated by the equation:

$x^{\prime}_{voc}=x_{voc}+g^{si}_{snr}\cdot z_{si}$ (1)

where $z_{si}$ represents white noise $w$ that has been processed through a
finite impulse response (FIR) filter, adjusted by a gain parameter
$g^{si}_{snr}$ based on a random signal-to-noise ratio.

###  3.2 SingGraph model

This section introduces the proposed novel SingGraph model. The SingGraph
model, depicted in Figure 1-(c), integrates instrumental and vocal encoders
based on self-supervised learning (SSL) models with a graph-based back-end
model. This back-end leverages graph neural networks [20, 21, 22], which have
led to significant advancements in various applications, including speaker
verification [22] and spoofing detection [20, 23, 3], by employing a structure
of interconnected nodes and edges. It encompasses components for aggregating
singing voice features, modeling spectral-temporal relationships, and
executing readout operations, detailed in the following subsections.

####  3.2.1 SSL-based instrumental and vocal encoder.

Speech SSL models like wav2vec2.0 [12, 24] excel in generalizing across
downstream tasks [25], where speech deepfake detection is one of applications
[26, 16], by capturing essential speech features, such as phonemes and tones,
to distinguish deepfake speech. MERT [11], on the other hand, targets the
musical aspects like rhythm and pitch through a multi-task mask language
modeling approach, making it suited for music-related tasks [27, 28, 29], in
the meantime, wav2vec 2.0's multi-lingual dataset training optimizes it for
handling diverse singing voices.

MERT [11] and wav2vec2 [12] are the instrumental backbone and vocal backbone
models for instrumental and vocal encoders, respectively. Both instrumental
and vocal encoders consist of a convolutional neural network (CNN) and a
Transformer block. The instrumental and vocal encoder aims to model musical
and vocal-related characteristics, respectively. The instrumental and vocal
encoder will extract instrumental and vocal representation sequence
$\\{o_{ins},o_{voc}\\}$ from the input audio. The experiments described in the
paper utilized MERT222huggingface.co/m-a-p/MERT-v1-330M and wav2vec2.0
XLR-S333github.com/facebookresearch/fairseq/tree/main/examples/wav2vec models,
both initialized with pre-trained model from open-source repositories. Both
models have 3 billion parameters.

####  3.2.2 Singing voice feature aggregation.

The singing voice feature aggregation module leverages the RawNet2 encoder
[30] and a self-attention-based (SA-based) aggregation layer [16] to extract
advanced spectral-temporal features, labeled as $\\{s,t\\}$. The RawNet2
encoder produces a detailed feature map, S, with dimensions in
$\mathbb{R}^{C\times F\times T}$, indicating channels $C$, spectral bins $F$,
and time frames $T$. Through the self-attention layer, this module effectively
captures crucial spectral and temporal details. Specifically, the spectral
feature $s$ is derived by aggregating across the spectral dimension $F$ of S,
while the temporal feature $t$ is similarly extracted by summing over the time
dimension $T$. This process ensures the module efficiently highlights relevant
spectral and temporal information from singing voices.

####  3.2.3 Spectral-temporal graph modeling.

The method employs the heterogeneous stacking graph attention layer (HS-GAL)
[31] to analyze the varied aspects of singing voices, covering both
instrumental and vocal components. Initially, it transforms spectral and
temporal features $\\{s,t\\}$ into graphs $\\{G_{st},G_{t}\\}$ through a graph
module [21]. The unified graph $G_{st}$ is generated by merging graphs through
a graph combination module, employing a graph attention layer. This layer
applies element-wise multiplication between nodes for symmetry, enabling
attention weight calculations across spectral and temporal nodes, instead of
concatenating nodes [21]. This facilitates a detailed understanding of the
intricate relationships in singing voice data. Moreover, an additional stack
node is introduced to encapsulate the complex interactions between the
spectral and temporal domains, linking to all nodes from $G_{s}$ and $G_{t}$.
By applying HS-GAL layers in sequence, the stack node from the first layer is
seamlessly integrated into the subsequent layer, enhancing the model's ability
to capture and process heterogeneous relationships. Stack nodes behave
similarly to classification tokens [32].

####  3.2.4 Max graph operation and readout.

The final prediction module features a max graph operation (MGO) followed by a
readout process. The MGO is designed to identify and integrate various
spoofing-induced artifacts simultaneously, which is inspired by the element-
wise maximum operations that are beneficial to detect anti-spoofing in several
works [23, 33]. It operates through two parallel branches, each applying an
element-wise maximum on their outputs. Each branch consists of two sequential
HS-GAL layers with a graph pooling layer following each HS-GAL, totaling four
HS-GAL and four graph pooling layers within MGO. The HS-GAL layers within a
branch share a stack node, passing it from one HS-GAL to the next, and an
element-wise maximum is applied to the stack nodes from both branches. After
element-wise maximum operations, we will obtain the spectral and temporal
nodes $\\{G^{\prime}_{s},G^{\prime}_{t}\\}$ of singing voice graphs and the
stack node $S_{node}$. Finally, a node-wise maximum and average are computed,
and the last hidden layer is generated by concatenating the outcomes from
these operations along with the stack node, ensuring a comprehensive
representation of the data.

Table 1: Ablation study of SingGraph model for SingFake detection EER (%).

Method Augmentation Setup Train $\downarrow$ T01 $\downarrow$ T02 $\downarrow$
T03 $\downarrow$ A1) W2V2-AASIST [16] ✗ $M$ 1.57 4.62 8.23 13.62 ✗ $V$ 1.70
5.39 9.10 10.03 A2) W2V2-AASIST (Our) ✗ $M$ 2.24 5.31 15.80 16.37 ✗ $V$ 6.49
8.55 15.91 16.14 A3) MERT-AASIST ✗ $M$ 1.50 6.28 12.55 13.51 ✗ $V$ 1.26 6.47
12.66 12.07 A4) W2V2-AASIST RawBoost $M$ 1.78 5.47 9.90 9.66 RawBoost $V$ 0.92
4.24 7.41 7.25 A5) MERT-AASIST RawBoost $M$ 0.88 5.54 12.66 12.79 RawBoost $V$
1.18 7.08 11.26 11.11 A6) MERT-W2V2-AASIST RawBoost $M$ 1.20 4.31 9.79 8.85
RawBoost $V$ 1.40 4.04 6.92 7.31 A7) MERT-W2V2-AASIST RawBoost $IV$ 0.84 3.77
6.84 7.05 A8) SingGraph RawBoost, Beat Matching $IV$ 0.54 (+65.6) 4.01 (+13.2)
6.23 (+24.3) 6.30 (+37.1)

##  4 Experiment

###  4.1 Experimental setup

SingFake dataset: The SingFake dataset [10] offers 28.93 hours of genuine and
29.40 hours of deepfake song clips from popular user-generated platforms,
organized into training, validation, and four test splits (T01 to T03). The
training set features recordings from 12 singers, while the validation set
introduces 4 new singers not included in the training. Test split T01 tests
the model's recognition of Stefanie Sun, a singer from the training set. T02
and T03 challenge the model with 6 new singers, with T03 adding the twist of
different communication codecs. Due to copyright concerns, the SingFake
dataset has not been released. We've re-downloaded the necessary data from
YouTube and Bilibili for experiments, but face download failures, leading to
different data distribution compared to the original paper 444Even though we
failed to download some data, we still achieved the state-of-the-art. We will
release the re-downloaded dataset metadata..

Model input setup: [10] focused on comparing models using either mixed singing
voice or vocal-only inputs. Our proposed SingGraph model, however, processes
inputs divided into instrumental and vocal audio. We denote these input
settings as ``$M$'' for mixture, ``$V$'' for vocals, and ``$IV$'' for
separating instrumental and vocal audio and separately feeding them into the
model. Note that, in [10], they only consider ``$M$'' and ``$V$'', however, we
propose ``$IV$'' for better modeling.

Evaluation: We will leverage the Equal Error Rate (EER) as our evaluation
matrix (the same metric in [10]), which is determined by setting a threshold
on the produced scores where the false acceptance rate matches the false
rejection rate.

###  4.2 Singing voice ablation study and analysis

Table 1 details our SingGraph model's ablation studies. A1 represents the
results of the prior SOTA collected from the SingFake paper [10]. Due to music
data copyright issues as described in Section 4.1, we failed to download some
data, and we reproduced the W2V2-AASIST model, labeled as model A2, for a fair
comparison. Models A3 and A5 feature the MERT-AASIST architecture, replacing
W2V2-AASIST's W2V2 encoder with MERT. Models A6-A8 follow the architecture in
Figure 1-(c) but vary in augmentation and input settings.

####  4.2.1 Preliminary ablation study

In our preliminary study, focusing on models A2 to A6 in Table 1, we evaluate
the efficacy of RawBoost and the synergistic potential between MERT and W2V2.
(1) RawBoost. After analyzing models A2 and A4, we observe significant
improvements in W2V2-AASIST from T01 to T03 for both mixture and vocal inputs
after adding RawBoost. However, for MERT-AASIST (A3 and A5), adding RawBoost
showed fluctuations without clear improvements. (2) The complementary of MERT
and wav2vec2.0. We assess the synergy between W2V2 and MERT by comparing rows
A4, A5, and A6. Generally, model A6 (MERT-W2V2-AASIST) outperforms both MERT-
AASIST and W2V2-AASIST models.

Experiment findings indicate RawBoost significantly improves wav2vec2.0's
extraction of vocal features, yet it barely improves MERT's extraction of
music-related features. Additionally, the results suggest that using MERT for
music information and wav2vec2.0 for vocal features can complement each other
with better results. So, in the subsequent experiments, RawBoost only augments
the vocal part.

####  4.2.2 SingGraph model ablation study.

In our model ablation study, examining models A6 to A8 in Table 1, we aim to
assess the necessity of each SingGraph model component's design. (1)
Instrument and vocals. Analyzing models A6 and A7 reveals that vocal-only
inputs enhance the MERT-W2V2-AASIST performance over mixed inputs across test
sets T01 to T03. Model A7, using separate instrumentals for MERT and vocals
for W2V2, outperforms its counterpart in A6 with significant margins:
improvements of 0.56%, 0.27%, 0.08%, 0.26%, and 1.51% are noted across the
training and test sets T01 to T03, respectively. (2) RawBoost and beat
matching. The primary difference between models A7 and A8 is that A8 uses
beat-matching augmentation to instrumentals. This augmentation significantly
enhances performance, achieving the lowest EER of 0.54% on the training set.
Moreover, A8 excels in test sets T02 and T03. Its performance on test set T01
is also comparable with A7.

The experiments highlight the effectiveness of using specialized encoders for
vocal and instrumental parts to capture the intricacies of singing voices from
different perspectives. RawBoost and Beat Matching augmentations enhance vocal
and instrumental features, making them a powerful combo. In SingFake
detection, this approach optimizes the analysis of voice nuances and
instrumental details, reducing EER. By focusing on vocal authenticity and the
harmony of instrumental sounds, the strategy effectively reduces errors,
enabling the precise identification of real versus fake singing clips.

Table 2: Comparison of SOTA SingFake detection EER (%).

Method Setup Train $\downarrow$ T01 $\downarrow$ T02 $\downarrow$ T03
$\downarrow$ AASIST [3] $M$ 4.10 7.29 11.54 17.29 $V$ 3.39 8.37 10.65 13.07
Spec-ResNet [14] $M$ 4.97 14.88 22.59 24.15 $V$ 5.31 11.86 19.69 21.54 LFCC-
ResNet [15] $M$ 10.55 21.35 32.40 31.85 $V$ 2.90 15.88 22.56 23.62 W2V2-AASIST
[16] $M$ 1.57 4.62 8.23 13.62 $V$ 1.70 5.39 9.10 10.03 SingGraph $IV$ 0.54
4.01 6.23 6.30 (+65.6) (+13.2) (+24.3) (+37.1)

###  4.3 Comparison with SOTA SingFake detection model

In Table 2, we benchmark SingGraph model against former leading models such as
AASIST [3], Spec-ResNet [14], LFCC-ResNet [15], and W2V2-AASIST [24], as
discussed in Section 2. Our SingGraph outperforms the previous SOTA model,
W2V2-AASIST, demonstrating significant relative improvements: 65.6% in
training, and 13.2%, 24.3%, and 37.1% in the T01, T02, and T03 test sets,
respectively. These results underscore SingGraph's robust capability in
accurately detecting both seen and unseen SingFake instances across various
codecs, showcasing its effective generalization in SingFake detection.

However, Our SingGraph model struggles with the Persian test set, notably
influenced by hip-hop, contrasting with the rock and ballad genres in other
sets. This points to a limitation in its ability to generalize across diverse
musical genres, signaling an area for future improvement.

##  5 Conclusion

This paper proposes a novel SingGraph framework, which combines MERT and
wav2vec2.0 to analyze vocal and musical nuances, to address singing voice
deepfake. The introduced RawBoost and beat matching augmentations
significantly improve detection performance. SingGraph achieves new SOTA
results on the SingFake dataset, relatively improving EER for seen singers by
13.2%, unseen singers by 24.3%, and unseen singers across different codecs by
37.1%.

##  6 Acknowledgements

This work was partially supported by the National Science and Technology
Council, Taiwan (Grant no. NSTC 112-2634-F-002-005, Advanced Technologies for
Designing Trustable AI Services). We also thank the National Center for High-
performance Computing (NCHC) of National Applied Research Laboratories
(NARLabs) in Taiwan for providing computational and storage resources.

## References

  * [1] Y. Zhang, J. Cong, H. Xue, L. Xie, P. Zhu, and M. Bi, ``Visinger: Variational inference with adversarial learning for end-to-end singing voice synthesis,'' in _ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2022, pp. 7237–7241.
  * [2] J. Liu, C. Li, Y. Ren, F. Chen, and Z. Zhao, ``Diffsinger: singing voice synthesis via shallow diffusion mechanism (2021),'' _arXiv preprint arXiv:2105.02446_ , 2021.
  * [3] J.-w. Jung, H.-S. Heo, H. Tak, H.-j. Shim, J. S. Chung, B.-J. Lee, H.-J. Yu, and N. Evans, ``Aasist: Audio anti-spoofing using integrated spectro-temporal graph attention networks,'' in _ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2022, pp. 6367–6371.
  * [4] J. Xue, C. Fan, J. Yi, C. Wang, Z. Wen, D. Zhang, and Z. Lv, ``Learning from yourself: A self-distillation method for fake speech detection,'' in _ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2023, pp. 1–5.
  * [5] S. Ding, Y. Zhang, and Z. Duan, ``Samo: Speaker attractor multi-center one-class learning for voice anti-spoofing,'' in _ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2023, pp. 1–5.
  * [6] Y.-L. Liao, X. Chen, C.-C. Wang, and J.-S. R. Jang, ``Adversarial speaker distillation for countermeasure model on automatic speaker verification,'' in _Proc. 2nd Symposium on Security and Privacy in Speech Communication_ , 2022, pp. 30–34.
  * [7] X. Wang, J. Yamagishi, M. Todisco, H. Delgado, A. Nautsch, N. Evans, M. Sahidullah, V. Vestman, T. Kinnunen, K. A. Lee _et al._ , ``Asvspoof 2019: A large-scale public database of synthesized, converted and replayed speech,'' _Computer Speech & Language_, vol. 64, p. 101114, 2020.
  * [8] N. M. Müller, P. Czempin, F. Dieckmann, A. Froghyar, and K. Böttinger, ``Does audio deepfake detection generalize?'' _arXiv preprint arXiv:2203.16263_ , 2022.
  * [9] X. Liu, X. Wang, M. Sahidullah, J. Patino, H. Delgado, T. Kinnunen, M. Todisco, J. Yamagishi, N. Evans, A. Nautsch _et al._ , ``Asvspoof 2021: Towards spoofed and deepfake speech detection in the wild,'' _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , 2023.
  * [10] Y. Zang, Y. Zhang, M. Heydari, and Z. Duan, ``Singfake: Singing voice deepfake detection,'' _arXiv preprint arXiv:2309.07525_ , 2023.
  * [11] Y. Li, R. Yuan, G. Zhang, Y. Ma, X. Chen, H. Yin, C. Lin, A. Ragni, E. Benetos, N. Gyenge _et al._ , ``Mert: Acoustic music understanding model with large-scale self-supervised training,'' _arXiv preprint arXiv:2306.00107_ , 2023.
  * [12] A. Baevski, Y. Zhou, A. Mohamed, and M. Auli, ``wav2vec 2.0: A framework for self-supervised learning of speech representations,'' _Advances in neural information processing systems_ , vol. 33, pp. 12 449–12 460, 2020.
  * [13] H. Tak, M. Kamble, J. Patino, M. Todisco, and N. Evans, ``Rawboost: A raw data boosting and augmentation method applied to automatic speaker verification anti-spoofing,'' in _ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2022, pp. 6382–6386.
  * [14] K. He, X. Zhang, S. Ren, and J. Sun, ``Deep residual learning for image recognition. corr abs/1512.03385 (2015),'' 2015.
  * [15] Y. Zhang, F. Jiang, and Z. Duan, ``One-class learning towards synthetic voice spoofing detection,'' _IEEE Signal Processing Letters_ , vol. 28, pp. 937–941, 2021.
  * [16] H. Tak, M. Todisco, X. Wang, J.-w. Jung, J. Yamagishi, and N. Evans, ``Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentation,'' _arXiv preprint arXiv:2202.12233_ , 2022.
  * [17] H. Zhang, M. Cisse, Y. N. Dauphin, and D. Lopez-Paz, ``mixup: Beyond empirical risk minimization,'' _arXiv preprint arXiv:1710.09412_ , 2017.
  * [18] T. Kim and J. Nam, ``All-in-one metrical and functional structure analysis with neighborhood attentions on demixed audio,'' in _2023 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA)_. IEEE, 2023, pp. 1–5.
  * [19] E. Roselló Casado, A. Gómez Alanís, Á. M. Gómez García, A. M. Peinado Herreros _et al._ , ``A conformer-based classifier for variable-length utterance processing in anti-spoofing,'' 2023.
  * [20] H. Tak, J.-w. Jung, J. Patino, M. Todisco, and N. Evans, ``Graph attention networks for anti-spoofing,'' _arXiv preprint arXiv:2104.03654_ , 2021.
  * [21] P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio, ``Graph attention networks,'' _arXiv preprint arXiv:1710.10903_ , 2017.
  * [22] J.-w. Jung, H.-S. Heo, H.-J. Yu, and J. S. Chung, ``Graph attention networks for speaker verification,'' in _ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2021, pp. 6149–6153.
  * [23] H. Tak, J. Patino, M. Todisco, A. Nautsch, N. Evans, and A. Larcher, ``End-to-end anti-spoofing with rawnet2,'' in _ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2021, pp. 6369–6373.
  * [24] A. Babu, C. Wang, A. Tjandra, K. Lakhotia, Q. Xu, N. Goyal, K. Singh, P. von Platen, Y. Saraf, J. Pino _et al._ , ``Xls-r: Self-supervised cross-lingual speech representation learning at scale,'' _arXiv preprint arXiv:2111.09296_ , 2021.
  * [25] S.-w. Yang, P.-H. Chi, Y.-S. Chuang, C.-I. J. Lai, K. Lakhotia, Y. Y. Lin, A. T. Liu, J. Shi, X. Chang, G.-T. Lin _et al._ , ``Superb: Speech processing universal performance benchmark,'' _arXiv preprint arXiv:2105.01051_ , 2021.
  * [26] X. Wang and J. Yamagishi, ``Investigating self-supervised front ends for speech spoofing countermeasures,'' _arXiv preprint arXiv:2111.07725_ , 2021.
  * [27] X. Du, Z. Yu, J. Lin, B. Zhu, and Q. Kong, ``Joint music and language attention models for zero-shot music tagging,'' _arXiv preprint arXiv:2310.10159_ , 2023.
  * [28] Y. Yamamoto, ``Toward leveraging pre-trained self-supervised frontends for automatic singing voice understanding tasks: Three case studies,'' in _2023 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)_. IEEE, 2023, pp. 1745–1752.
  * [29] R. Yuan, Y. Ma, Y. Li, G. Zhang, X. Chen, H. Yin, Y. Liu, J. Huang, Z. Tian, B. Deng _et al._ , ``Marble: Music audio representation benchmark for universal evaluation,'' _Advances in Neural Information Processing Systems_ , vol. 36, 2024.
  * [30] J.-w. Jung, S.-b. Kim, H.-j. Shim, J.-h. Kim, and H.-J. Yu, ``Improved rawnet with feature map scaling for text-independent speaker verification using raw waveforms,'' _arXiv preprint arXiv:2004.00526_ , 2020.
  * [31] X. Wang, H. Ji, C. Shi, B. Wang, Y. Ye, P. Cui, and P. S. Yu, ``Heterogeneous graph attention network,'' in _The world wide web conference_ , 2019, pp. 2022–2032.
  * [32] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, ``Bert: Pre-training of deep bidirectional transformers for language understanding,'' _arXiv preprint arXiv:1810.04805_ , 2018.
  * [33] G. Lavrentyeva, S. Novoselov, E. Malykh, A. Kozlov, O. Kudashev, and V. Shchemelinin, ``Audio replay attack detection with deep learning frameworks.'' in _Interspeech_ , 2017, pp. 82–86.