[affiliation=1]JiatongShi [affiliation=2]XutaiMa
[affiliation=2]HirofumiInaguma [affiliation=2]AnnaSun
[affiliation=1]ShinjiWatanabe [datatype=bibtex, overwrite=true]
[fieldsource=booktitle, match=.*Interspeech.*, replace=Proc. Interspeech]
[fieldsource=journal, match=.*INTERSPEECH.*, replace=Proc. Interspeech]
[fieldsource=booktitle, match=.*ICASSP.*, replace=Proc. ICASSP]
[fieldsource=booktitle, match=.*icassp_inpress.*, replace=Proc. ICASSP (in
press)] [fieldsource=booktitle,
match=.*Acoustics,.*Speech.*and.*Signal.*Processing.*, replace=Proc. ICASSP]
[fieldsource=booktitle,
match=.*International.*Conference.*on.*Learning.*Representations.*,
replace=Proc. ICLR] [fieldsource=booktitle,
match=.*International.*Conference.*on.*Computational.*Linguistics.*,
replace=Proc. COLING] [fieldsource=booktitle,
match=.*SIGdial.*Meeting.*on.*Discourse.*and.*Dialogue.*, replace=Proc.
SIGDIAL] [fieldsource=booktitle,
match=.*International.*Conference.*on.*Machine.*Learning.*, replace=Proc.
ICML] [fieldsource=booktitle,
match=.*North.*American.*Chapter.*of.*the.*Association.*for.*Computational.*Linguistics:.*Human.*Language.*Technologies.*,
replace=Proc. NAACL] [fieldsource=booktitle,
match=.*Empirical.*Methods.*in.*Natural.*Language.*Processing.*, replace=Proc.
EMNLP] [fieldsource=booktitle,
match=.*Association.*for.*Computational.*Linguistics.*, replace=Proc. ACL]
[fieldsource=booktitle,
match=.*Automatic.*Speech.*Recognition.*and.*Understanding.*, replace=Proc.
ASRU] [fieldsource=booktitle, match=.*Spoken.*Language.*Technology.*,
replace=Proc. SLT] [fieldsource=booktitle,
match=.*Speech.*Synthesis.*Workshop.*, replace=Proc. SSW]
[fieldsource=booktitle, match=.*workshop.*on.*speech.*synthesis.*,
replace=Proc. SSW] [fieldsource=booktitle,
match=.*Advances.*in.*neural.*information.*processing.*, replace=Proc.
NeurIPS] [fieldsource=booktitle,
match=.*Advances.*in.*Neural.*Information.*Processing.*, replace=Proc.
NeurIPS] [fieldsource=booktitle, match=.*Workshop.*on.* Applications.* of.*
Signal.*Processing.*to.*Audio.*and.*Acoustics.*, replace=Proc. WASPAA]
[fieldsource=publisher, match=.+, replace=] [fieldsource=month, match=.+,
replace=] [fieldsource=location, match=.+, replace=] [fieldsource=address,
match=.+, replace=] [fieldsource=organization, match=.+, replace=]
[fieldsource=pages, match=.+, replace=]

# MMM: Multi-Layer Multi-Residual Multi-Stream Discrete Speech Representation
from Self-supervised Learning Model

###### Abstract

Speech discrete representation has proven effective in various downstream
applications due to its superior compression rate of the waveform, fast
convergence during training, and compatibility with other modalities. Discrete
units extracted from self-supervised learning (SSL) models have emerged as a
prominent approach for obtaining speech discrete representation. However,
while discrete units have shown effectiveness compared to spectral features,
they still lag behind continuous SSL representations. In this work, we propose
MMM, a multi-layer multi-residual multi-stream discrete units extraction
method from SSL. Specifically, we introduce iterative residual vector
quantization with K-means for different layers in an SSL model to extract
multi-stream speech discrete representation. Through extensive experiments in
speech recognition, speech resynthesis, and text-to-speech, we demonstrate the
proposed MMM can surpass or on-par with neural codec's performance under
various conditions.

###### keywords:

discrete speech representation, self-supervised learning, discrete speech unit

##  1 Introduction

Efficient representation of speech signals is fundamental for a wide array of
speech processing tasks, ranging from automatic speech recognition (ASR) to
text-to-speech (TTS) synthesis. Traditionally, spectral features such as
linear spectrograms or mel spectrograms have been extensively used in speech
processing due to their robustness and interpretability [1, 2]. However, with
the advent of deep learning, there has been a paradigm shift towards utilizing
neural networks as feature extractors, offering improved performance over
traditional methods [3, 4].

More recently, self-supervised learning (SSL) approaches have gained
prominence in speech representation learning. These methods leverage large
amounts of unlabeled speech data to learn powerful representations, surpassing
previous state-of-the-art results on various benchmarks [5, 6, 7, 8, 9, 10,
11, 12, 13]. However, continuous SSL representations often suffer from
scalability issues in terms of storage, computation, and integration with
other modalities [14, 15, 16, 17]. This has led to a growing interest in
discrete speech representation approaches, which offer more efficient and
compact representations.

Two prominent methods that have emerged are SSL-based units and neural audio
codecs. SSL-based units leverage clustering methods in an unsupervised manner
to convert continuous SSL representations into discrete units, initially
explored for speech resynthesis and subsequently proven effective in tasks
such as speech translation (ST), ASR, TTS, and spoken language understanding
[18, 19, 20, 16, 21, 22, 15, 16, 23, 24, 25]. However, while SSL-based units
offer efficiency and effectiveness benefits, they often fall short of
achieving better performance than continuous SSL representations and lack
detailed acoustics for speech generation purposes [15, 23, 26].

On the other hand, neural audio codecs focus on audio resynthesis tasks,
employing neural networks to learn auto-encoder architectures for discrete
codec generation. A key component of recent neural codec methods is the use of
residual vector quantization (RVQ) in the discretization process, resulting in
multi-stream audio compression that retains subtle audio details with enhanced
expressiveness [27, 28, 29, 26]. This property has led to extensions of neural
audio codecs to text-to-speech and spoken language models, demonstrating
robust speech generation capabilities in zero-shot multi-speaker TTS scenarios
[30, 31, 32]. However, neural codecs optimized for resynthesis tasks often
lack semantic information due to their focus on streaming efficiency and
short-context representation [16].

Despite their differences, limited comparative studies exist between SSL-based
units and neural audio codecs under comparable conditions. Notably, SSL-based
units typically operate in a single-stream setting, which offers less
information capacity compared to multi-stream codecs.

This study propose a multi-layer multi-residual multi-stream (MMM) framework
to extract discrete speech representation from continuous SSL representations.
Specifically, we conduct RVQ-style quantization with K-means clustering to
enable multi-stream discrete tokens in each single SSL layer. By combining
with streams from multiple SSL layers, we further enhance the richness of the
SSL-based units. With extensive experiments in ASR, we reveal that the
proposed MMM-based discrete speech units elevates the performance of original
single-stream SSL by a large margin and almost approaches the top-line
performance with continuous SSL representation. While maintaining better ASR
performance, we also demonstrate that the MMM-based units could achieve
comparable or better performance to the neural codec-based approach in speech
resynthesis and TTS.

##  2 Methodology

SSL discrete units are derived from a single-layer hidden representation
within a specific SSL model. Given a speech signal $\bm{x}$, we represent an
SSL model as $\mathrm{S}$, which produces layer-wise representations denoted
as $\bm{\mathsfit{R}}=[\bm{R}^{1},\ldots,\bm{R}^{L}]$, where $L$ signifies the
number of layers in $\mathrm{S}$. For a particular layer $l$, each element
$\bm{r}^{l}\in\bm{R}^{l}$ comprises a sequence of vectors
$[\bm{r}^{l}_{1},\ldots,\bm{r}^{l}_{T}]$, with $T$ representing the frame
count. Upon selecting step $t$ for analysis, we employ a $l$-th layer K-means
model as $\mathrm{K}^{l}$ to determine $K^{l}$ cluster centroids. With these
cluster centroids obtained by K-means training, we can cluster the feature
vectors $\bm{r}_{t}^{l}$ by finding the optimal cluster index, which minimizes
the Euclidean distance between $\bm{r}_{t}^{l}$ and each cluster centroid
$\bm{c}_{k}^{l}$ as:

$k_{t}^{l}=\mathrm{argmin}_{k\in{1,\ldots,K^{l}}}||\bm{r}_{t}^{l}-\bm{c}_{k}^{l}||^{2}.$
(1)

The final cluster IDs $[k_{1}^{l},\ldots,k_{T}^{l}]$ serve as discrete units
at particularly layer $l$ for subsequent downstream tasks

In this study, we broaden the application from a single-stream scenario to
encompass multi-stream scenarios. To achieve this, we propose two
complementary strategies for multi-stream modeling, which involve leveraging
either a single layer or multiple layers from the SSL model $\mathrm{S}$.

###  2.1 Multi-stream from a single layer

The first approach concentrates on generating multiple streams from a single
layer within the SSL model. Citing [10], it's demonstrated that the single-
layer continuous representation, specifically from a HuBERT-base model,
contains sufficient detail for audio resynthesis at frame resolutions of both
40ms and 100ms. Nonetheless, the resynthesis audio quality markedly
deteriorates following quantization via K-means. This decline highlights a
significant limitation: discrete units derived from K-means quantization tend
to omit intricate acoustic details originally present in the waveform signals.

To counteract the information loss inherent in quantization, our method
involves estimating additional streams of discrete units. This process is
aligned with the principle of RVQ, adhering to the unsupervised nature of the
original K-means-based approach. For training, in the $l$-th layer, we
iteratively estimate $m$-th K-means model $\mathrm{K}^{m,l}$ on the residual
feature from previous K-means models
$[\mathrm{K}^{1,l},\ldots,\mathrm{K}^{m-1,l}]$. During inference, the cluster
index $k_{t}^{m,l}$ in frame $t$ can then be obtained through iterative
procedures. The following equations elaborate the detailed inference steps
from the first stream to stream $m$.

$\displaystyle k_{t}^{1,l}$
$\displaystyle=\mathrm{argmin}_{k\in{1,,...,K^{1,l}}}\left|\left|\bm{r}_{t}^{l}-\bm{c}_{k}^{1,l}\right|\right|^{2},$
(2) $\displaystyle k_{j}^{2,l}$
$\displaystyle=\mathrm{argmin}_{k\in{1,,...,K^{2,l}}}\left|\left|(\bm{r}^{l}_{t}-\bm{c}_{t}^{1,l})-\bm{c}_{k}^{2,l}\right|\right|^{2},$
(3) $\displaystyle\vdots$ $\displaystyle k_{j}^{m,l}$
$\displaystyle=\mathrm{argmin}_{k\in{1,,...,K^{m,l}}}\left|\left|\left(\bm{r}^{l}_{t}-\sum_{u=1}^{m-1}\bm{c}_{t}^{u,l}\right)-\bm{c}_{k}^{m,l}\right|\right|^{2},$
(4)

where we define $\bm{c}_{t}^{m,l}$ as the selected centroid at $t$, i.e.,
$\bm{c}_{t}^{m,l}:=\bm{c}_{k}^{m,l}|_{k=k_{t}^{m,l}}$ . Estimated
$k_{t}^{m,l}$ becomes the discrete unit of the $m$-th stream at $t$-th frame.

###  2.2 Multi-stream from multiple layers

While Section 2.1 elaborated on enhancing discrete representations through a
single SSL model layer $l$, an alternative strategy employs multi-layer
representations. Historically, the integration of information across multiple
layers in SSL models has proven invaluable, particularly when leveraging
frozen SSL representations for downstream tasks. This is exemplified in the
Speech Universal PERformance Benchmark (SUPERB), where layer-wise
representations are combined through a weighted sum approach, achieving
commendable results across a variety of speech processing tasks [12].
Subsequent research further validates this approach, demonstrating that
different layers of an SSL model encapsulate distinct facets of speech-related
information [33, 34, 7, 13, 10, 11].

In the context of discrete representations, the significance of utilizing
multi-layer information becomes even more pronounced. Reliance on a single
layer for information extraction may inadvertently prioritize certain features
while overlooking others, a discrepancy that becomes especially noticeable
following the quantization process (see Section 2.1). Therefore, a multi-
layered approach not only diversifies the extracted information but also
mitigates the risk of information bias, ensuring a more holistic
representation of speech signals.

The formulation of multi-layer multi-stream discrete representations can be
easily extracted from continuous representation $\bm{\mathsfit{R}}$. To be
specific, we specify $L^{\prime}$ layers from the multi-layer representation
$\bm{\mathsfit{R}}$. Then, for the K-means model $\mathrm{K}^{m,l^{\prime}}$
of a selected layer $l^{\prime}$, the designated cluster IDs
$[k^{m,l^{\prime}}_{1},\ldots,k^{m,l^{\prime}}_{T}]$ can be used as the
discrete token for downstream tasks. The final MMM-based representation with
$M\times L^{\prime}$ streams can be obtained by combining the multi-stream
extraction methods in Section 2.1 and Section 2.2.

###  2.3 Application of MMM-based discrete units

Building on existing literature using discrete representations [19, 20, 30,
16, 15, 35, 31, 32, 36, 25], we explore the application of MMM-based discrete
units as either inputs or outputs across various applications. Our
investigation encompasses both scenarios, employing a strategic approach to
integrate discrete representations into downstream tasks.

Input Scenario: For representations used as inputs, we initially transform
these discrete entities into embeddings. Subsequently, we apply a summation
across different streams to integrate these embeddings. Specifically, for
streams derived from a single layer, a direct summation of embeddings is
executed, mirroring the inverse operation of the RVQ technique. Conversely,
when dealing with streams from multiple layers, we adopt a learnable weighted
summation for their aggregation. This method leverages learnable weights,
optimized through a $\mathrm{Softmax}$ function, aligning with strategies from
the SUPERB series [12, 13, 37, 38, 39]. This approach ensures that the
integration of multi-layer streams is both dynamic and informed by the data.
In cases where streams are sourced from both RVQ and multiple layers, we first
combine embeddings from identical layers before proceeding to merge across
different layers.

Output Scenario: For outputs utilizing MMM-based units, we implement a
straightforward parallel approach that independently predicts various streams
of tokens. This decision is different from prior work on neural audio codecs,
where it was noted that models trained on RVQ necessitate auto-regressive
modeling to maintain decoder quality [30, 40, 32]. Despite this, our pilot
experiments indicate that the impact of such modeling on SSL-based units is
minimal, thereby justifying our preference for a simpler, parallel prediction
method for MMM-based discrete tokens.

##  3 Experiments

As discussed in Section 2.3, the discrete token can be applied in both input
and output scenarios. For the input scenario, we select the two classical
tasks of the proposed method: ASR and speech resynthesis (i.e., vocoder). The
two tasks consider both usages in understanding and generation. For the output
scenario, we conduct TTS, which serves as the backbone of most other tasks
that produce discrete units [23, 41].

###  3.1 Speech recognition

Dataset: Our ASR experiments align with the discrete speech challenge at
Interspeech2024.111https://www.wavlab.org/activities/2024/Interspeech2024-Discrete-
Speech-Unit-Challenge/ We utilize a dataset comprising Librispeech's train-
clean-100h [42] combined with the ML-SUPERB multilingual 1-hour set [13]. This
blend enables the examination of both clean English read speech and
multilingual ASR tasks. The total dataset spans 310.4 hours and encompasses
143 languages, offering a broad spectrum for evaluation.

SSL Models: Informed by insights from [15, 13, 39] and the challenge's
baseline, we recognize the distinct performance capabilities of WavLM [7] and
XLS-R [8] across different corpora. WavLM-large exhibits notable effectiveness
in English datasets, whereas XLS-R (300M version) is better suited for
multilingual datasets. To comprehensively assess performance across both
English and multilingual corpora, we employ these models as our primary SSL
candidates.

Clustering and Downstream Settings: Consistent with the challenge's baseline,
we opt for a random subsampling of 30% of the training set utterances for
K-means clustering, setting the cluster size at 500 for each stream, i.e.,
$K^{m,l}=500$. Deviating from the baseline, our methodology eschews additional
byte-pair encoding and deduplication, simplifying the alignment of different
streams. The downstream model leverages the same encoder-decoder architecture
within ESPnet [43] as the challenge baseline.

Proposed Method: Following the methodology outlined in Section 2, we extract
multi-stream tokens from both a single layer and multiple layers. For the
single-layer approach, two ($M=2$) streams of discrete representations are
extracted from the 21st layer of both WavLM and XLS-R. For multi-layer
scenarios, we select four ($L^{\prime}=4$) layers ({9, 15, 21, 22}) to balance
compression efficiency and performance. As in Section 2.2, the two methods can
be combined to yield eight streams of tokens.

Baseline and Ablation Studies: Our baseline comparison uses the single-layer,
one-stream SSL units as defined in the challenge. Additionally, we use the
publicly-availble Encodec-24kHz model [28] with 8 streams as another baseline.
Ablation studies are conducted for both single-layer (with $M={2,4,8}$
streams) and multi-layer configurations, including a variety of layer
selections. Layer selection for multi-layer scenarios is optimized using a
model with all layers, employing learnable weighted summation (see Section
2.3), and then selecting the top four weighted layers for further analysis.

Table 1: ASR performance on discrete speech challenge dataset. ``$+$"
indicates single-layer multi-stream setting and ``$\dagger$" stands for multi-
layer multi-stream setting. $M/L^{\prime}$ corresponding to number layers in
single/multi-layer multi-stream settings.

SSL | Streams | Librispeech | ML-SUPERB (1h) | Bitrate
---|---|---|---|---
$M\times L^{\prime}$ | WER | CER |
WavLM | 1 * 1 | 6.3 | 22.8 | 548.3
XLS-R | 1 * 1 | 14.1 | 21.4 | 548.3
Encodec | 8 | 15.9 | 35.9 | 6000.0
WavLM$+$ | 2 * 1 | 5.9 | 21.4 | 1096.6
WavLM$\dagger$ | 1 * 4 | 5.0 | 20.8 | 2193.2
XLS-R$+$ | 2 * 1 | 10.5 | 19.3 | 1096.6
XLS-R$\dagger$ | 1 * 4 | 7.3 | 18.0 | 2193.2
WavLM$+\dagger$ | 2 * 4 | 4.7 | 19.5 | 4386.3
XLS-R$+\dagger$ | 2 * 4 | 6.8 | 17.5 | 4386.3

Table 2: ASR ablation studies on the single-layer setting over discrete speech
challenge dataset.

SSL | Streams | Librispeech | ML-SUPERB (1h) | Bitrate
---|---|---|---|---
$M$ | WER | CER
WavLM$+$ | 1 | 6.3 | 22.8 | 548.3
WavLM$+$ | 4 | 6.1 | 21.5 | 2193.2
WavLM$+$ | 8 | 6.4 | 21.7 | 4386.3
WavLM$+$ | 2 | 5.9 | 21.4 | 1096.6

Table 3: ASR ablation studies on the multi-layer setting over discrete speech
challenge dataset. Detailed layer indexes are shown in the ``Layers" column.

SSL | Layers | Librispeech | ML-SUPERB (1h) | Bitrate
---|---|---|---|---
$L^{\prime}$ | WER | CER
WavLM$\dagger$ | 21 | 6.3 | 22.8 | 548.3
WavLM$\dagger$ | 1-4 | 6.8 | 27.7 | 2193.2
WavLM$\dagger$ | 11-14 | 6.1 | 21.9 | 2193.2
WavLM$\dagger$ | 21-24 | 5.5 | 21.5 | 2193.2
WavLM$\dagger$ | 0-24 | 4.9 | 19.9 | 13707.2
WavLM$\dagger$ | 9, 15, 21, 22 | 5.0 | 20.8 | 2193.2

Evaluation Metrics: We follow the setting in the discrete challenge for
evaluation metrics, including average word error rate (WER) for Librispeech
test sets and weighted average character error rate (CER) for ML-SUPERB test
sets (i.e., normal test set and few-shot test set). Bitrate is also reported,
following the discrete speech challenge guidelines.

Results and Discussion: The main results, as depicted in Table 1, illustrate
that our proposed method uniformly enhances performance across both
Librispeech and ML-SUPERB datasets for both SSL models. Additionally, the
performance improvements yielded by the two proposed approaches appear to be
complementary. Compared to Encodec tokens, even the single-stream discrete
token sequences have better performances. The finding is aligned with the
observations in [16] where SSL-based discrete representations outperform the
neural codec-based method, specifically Encodec.

Our detailed ablation studies focusing on the number of streams are summarized
in Tables 2 and 3. In the context of the single-layer multi-stream approach,
augmenting the number of streams does not always lead to performance
enhancement. As highlighted in Table 2, the ASR performance drops when the
stream count is increased to 4 or 8, compared to the two-stream scenario. This
deterioration could be attributed to the potential instability associated with
employing K-means for higher-order residual information extraction.
Conversely, for the multi-layer multi-stream scenario, optimal performance is
attained when all layers are utilized at the cost of a higher bitrate. By
selecting the layers with the top four weights, we show that it's possible to
maintain performance levels with only a slight degradation while achieving a
significant reduction in the bitrate.

###  3.2 Speech resynthesis (vocoder)

Table 4: Speech resynthesis performance on discrete speech challenge dataset
(filtered Expresso). ``$+$" indicates single-layer multi-stream setting and
``$\dagger$" stands for multi-layer multi-stream setting. $M/L^{\prime}$
corresponding to number layers in single/multi-layer multi-stream settings.

SSL | Streams ($M\times L^{\prime}$) | MCD | F0 RMSE | UTMOS | Bitrate
---|---|---|---|---|---
HuBERT | 1 * 1 | 7.19 | 0.42 | 2.27 | 448.3
Encodec | 8 | 3.91 | 0.21 | 3.18 | 3586.4
HuBERT (S) | 2 * 1 | 6.79 | 0.32 | 2.89 | 896.6
HuBERT (M) | 1 * 4 | 5.12 | 0.22 | 3.10 | 1793.2
HuBERT (S+M) | 2 * 4 | 4.54 | 0.20 | 3.22 | 3586.4

Table 5: TTS performance on discrete speech challenge dataset (LJSpeech).

SSL | Streams ($M\times L^{\prime}$) | MCD | F0 RMSE | WER | UTMOS | Bitrate
---|---|---|---|---|---|---
HuBERT | 1 * 1 | 7.19 | 0.26 | 8.1 | 3.73 | 448.3
Encodec | 8 | 7.01 | 0.29 | 7.8 | 4.01 | 3586.4
HuBERT (S) | 2 * 1 | 7.11 | 0.29 | 8.0 | 3.79 | 896.6
HuBERT (M) | 1 * 4 | 7.25 | 0.24 | 7.7 | 4.06 | 1793.2
HuBERT (S+M) | 2 * 4 | 7.15 | 0.25 | 7.7 | 4.15 | 3586.4

Dataset: Our speech resynthesis experiments are anchored in the TTS (vocoder)
track of the discrete speech challenge at Interspeech 2024. We utilize a
curated subset of the Expresso benchmark [26], focusing on a single-speaker
dataset and excluding segments with singing, overlapping speech, and long-form
content. The experiments adhere to the official train-dev-test partitioning
provided by the challenge
organizers.222https://github.com/ftshijt/Interspeech2024_DiscreteSpeechChallenge
To align with the original SSL model's specifications, audio samples at 48kHz
are downsampled to 16kHz for compatibility.

Experimental Set-ups: Echoing prior studies [19, 20, 24, 41, 26] in discrete-
based speech resynthesis that predominantly utilize HuBERT [6], our
experiments also employ a pre-trained HuBERT-base model trained on the full
Librispeech dataset via S3PRL[12]. Clustering is performed on the respective
training sets with a designated cluster size of 500 per stream. The discrete
HiFiGAN model serves as our downstream backbone, configured in accordance with
[24]. For our baseline, the 9th layer is selected for extracting single-stream
SSL units, drawing from the methodology in [19, 20, 24]. In exploring multi-
stream capabilities, we investigate two ($M=2$) streams for single-layer
scenarios, four ($L^{\prime}=4$) streams for multi-layer
configurations333Layer 6, 9, 11, 12 are used, following the same selection
strategy as the ASR track. We use the same setting for TTS experiments., and
an integrated approach yielding eight streams ($M\times L^{\prime}=8$). We
referred to these multi-stream configurations from the experimental findings
in Section 3.1. Additional Encodec baselines are trained on the same training
set based on AudioCraft [40]. We set the Encodec model with 8 streams at a
50Hz frame rate. For each stream, the codebook size is set to be 500, to be
aligned with our experiments.

Evaluation Metrics: The evaluation framework prioritizes objective metrics in
line with the discrete speech challenge's guideline. These metrics include mel
cepstral distortion (MCD), F0 root mean square error (F0 RMSE), UTMOS [44],
and bitrate. The MCD and F0 RMSE metrics are calculated using the ESPnet-TTS
toolkit [45, 46].

Results and Discussion: Table 4 illustrates that our proposed method
outperforms the baselines in UTMOS and F0 RMSE. Compared to the single-stream
baseline, both of our proposed approaches not only improve all evaluated
metrics, but also exhibit complementary benefits to each other. Notably, even
when compared to Encodec, our method demonstrates superior UTMOS and F0 RMSE
scores. This is particularly significant given that our discrete tokens are
extracted unsupervisedly and are not explicitly optimized for the resynthesis
task.

###  3.3 Text-to-speech

Dataset: For TTS, our examination leverages the LJSpeech dataset, a single-
speaker female TTS corpus, in alignment with the discrete speech challenge
recommendations. We rigorously adhere to the official dataset partitioning for
the training set, as detailed by the challenge guidelines.

Experimental Set-ups: The TTS modeling experiments inherit several
configurations from the speech resynthesis task. This includes utilizing the
HuBERT-base SSL model, adopting a cluster size of 500 for K-means clustering,
employing the entire training set for clustering, and integrating the discrete
HiFi-GAN model. Distinctively, our downstream TTS model employs a modified
VITS architecture, substituting its adversarial decoder with a transformer
network designed to directly predict discrete units. This modification is
informed based on the ESPnet VITS LJSpeech recipe and draws inspiration from
[18].444Hyperparameters are in line with ESPnet VITS LJSpeech recipe.

Evaluation Metrics: The evaluation framework utilize MCD, F0 RMSE, UTMOS and
bitrate. In addition, we also report CER as assessed by a pre-trained Whisper-
large-V2 model [47].

Results and Discussion: The outcomes of the TTS experiments, as displayed in
Table 5, echo the observations made in the vocoder experiments detailed in
Section 3.2. Compared to the Encodec model, we observe significant enhanced
naturalness (reflected in improved UTMOS scores) with the TTS system trained
on our proposed multi-stream tokens. This enhancement could be attributed to
the SSL tokens possessing a richer semantic content, offering extra advantages
in acoustic modeling when used as output. This semantic richness in SSL tokens
likely facilitates a more effective acoustic representation for acoustic
modeling, thereby enhancing the overall naturalness of the synthesized speech.

##  4 Conclusion

In this study, we reexamine the extraction of SSL-based discrete tokens,
focusing on multi-stream modeling. We focus on two approaches: single-layer
and multi-layer modeling. With extensive experiments across ASR, speech
resynthesis, and TTS, we showcase that multi-stream tokens from SSL models can
usually improve upon the single-stream SSL tokens. Moreover, we also find the
proposed representations attain performance levels that are either superior to
or on par with those achieved by neural codec methods.

##  5 Acknowledgements

This work was supported by a Meta AI SRA grant. Jiatong Shi and Shinji
Watanabe are funded in part of the Bridges2 system at PSC and Delta system at
NCSA through allocations CIS210014 and IRI120008P from the ACCESS program,
supported by NSF grants #2138259, #2138286, #2138307, #2137603, and #2138296.

##  6 References

## References

  * [1] Hynek Hermansky  ``Perceptual linear predictive (PLP) analysis of speech''  In _the Journal of the Acoustical Society of America_ 87.4, 1990
  * [2] Xuedong Huang et al.  ``Spoken language processing: A guide to theory, algorithm, and system development'', 2001
  * [3] Tara N. Sainath et al.  ``Learning the speech front-end with raw waveform CLDNNs''  In _Proc. Interspeech_ , 2015  DOI: 10.21437/Interspeech.2015-1
  * [4] Hynek Hermansky, Daniel PW Ellis and Sangita Sharma  ``Tandem connectionist feature extraction for conventional HMM systems''  In _Proc. ICASSP_ 3, 2000
  * [5] Alexei Baevski et al.  ``wav2vec 2.0: A framework for self-supervised learning of speech representations''  In _Advances in neural information processing systems_ 33, 2020
  * [6] Wei-Ning Hsu et al.  ``HuBERT: Self-supervised speech representation learning by masked prediction of hidden units''  In _TASLP_ 29, 2021
  * [7] Sanyuan Chen et al.  ``WavLM: Large-scale self-supervised pre-training for full stack speech processing''  In _JSTSP_ 16.6, 2022
  * [8] Arun Babu et al.  ``XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale''  In _Proc. Interspeech_ , 2022  DOI: 10.21437/Interspeech.2022-143
  * [9] Chung-Cheng Chiu et al.  ``Self-supervised learning with random-projection quantizer for speech recognition''  In _Proc. ICML_ , 2022
  * [10] Jiatong Shi et al.  ``Exploration on HuBERT with Multiple Resolution''  In _Proc. Interspeech_ , 2023  DOI: 10.21437/Interspeech.2023-1337
  * [11] Jiatong Shi et al.  ``Multi-resolution HuBERT: Multi-resolution Speech Self-Supervised Learning with Masked Unit Prediction''  In _Proc. ICLR_ , 2024
  * [12] Shu-Wen Yang et al.  ``SUPERB: Speech Processing Universal PERformance Benchmark''  In _Proc. Interspeech_ , 2021  DOI: 10.21437/Interspeech.2021-1775
  * [13] Jiatong Shi et al.  ``ML-SUPERB: Multilingual Speech Universal PERformance Benchmark''  In _Proc. Interspeech_ , 2023  DOI: 10.21437/Interspeech.2023-1316
  * [14] Heng-Jui Chang, Shu-wen Yang and Hung-yi Lee  ``DistillHuBERT: Speech representation learning by layer-wise distillation of hidden-unit bert''  In _Proc. ICASSP_ , 2022
  * [15] Xuankai Chang et al.  ``Exploration of Efficient End-to-End ASR using Discretized Input from Self-Supervised Learning''  In _Proc. Interspeech_ , 2023  DOI: 10.21437/Interspeech.2023-2051
  * [16] Xuankai Chang et al.  ``Exploring speech recognition, translation, and understanding with discrete speech units: A comparative study''  In _Proc. ICASSP_ , 2023
  * [17] Jiatong Shi et al.  ``Bridging Speech and Textual Pre-Trained Models With Unsupervised ASR''  In _Proc. ICASSP_ , 2023
  * [18] Tomoki Hayashi  ``Discretalk: Text-to-speech as a machine translation problem''  In _arXiv preprint arXiv:2005.05525_ , 2020
  * [19] Adam Polyak et al.  ``Speech Resynthesis from Discrete Disentangled Self-Supervised Representations''  In _Proc. Interspeech_ , 2021  DOI: 10.21437/Interspeech.2021-475
  * [20] Ann Lee et al.  ``Direct Speech-to-Speech Translation With Discrete Units''  In _Proc. ACL_ , 2022
  * [21] Dong Zhang et al.  ``DUB: Discrete Unit Back-translation for Speech Translation''  In _arXiv preprint arXiv:2305.11411_ , 2023
  * [22] Jiatong Shi et al.  ``Enhancing Speech-To-Speech Translation with Multiple TTS Targets''  In _Proc. ICASSP_ , 2023
  * [23] Loı̈c Barrault et al.  ``Seamless: Multilingual Expressive and Streaming Speech Translation''  In _arXiv preprint arXiv:2312.05187_ , 2023
  * [24] Brian Yan et al.  ``ESPnet-ST-v2: Multipurpose Spoken Language Translation Toolkit''  In _Proc. ACL_ , 2023  DOI: 10.18653/v1/2023.acl-demo.38
  * [25] Yifan Yang et al.  ``Towards Universal Speech Discrete Tokens: A Case Study for ASR and TTS''  In _Proc. ICASSP_ , 2024
  * [26] Tu Anh Nguyen et al.  ``Expresso: A benchmark and analysis of discrete expressive speech resynthesis''  In _arXiv preprint arXiv:2308.05725_ , 2023
  * [27] Neil Zeghidour et al.  ``Soundstream: An end-to-end neural audio codec''  In _TASLP_ 30, 2021
  * [28] Alexandre Défossez et al.  ``High fidelity neural audio compression''  In _arXiv preprint arXiv:2210.13438_ , 2022
  * [29] Dongchao Yang et al.  ``Hifi-codec: Group-residual vector quantization for high fidelity audio codec''  In _arXiv preprint arXiv:2305.02765_ , 2023
  * [30] Chengyi Wang et al.  ``Neural codec language models are zero-shot text to speech synthesizers''  In _arXiv preprint arXiv:2301.02111_ , 2023
  * [31] Xiaofei Wang et al.  ``Speechx: Neural codec language model as a versatile speech transformer''  In _arXiv preprint arXiv:2308.06873_ , 2023
  * [32] Dongchao Yang et al.  ``Uniaudio: An audio foundation model toward universal audio generation''  In _arXiv preprint arXiv:2310.00704_ , 2023
  * [33] Ankita Pasad, Ju-Chieh Chou and Karen Livescu  ``Layer-wise analysis of a self-supervised speech representation model''  In _Proc. ASRU_ , 2021
  * [34] Xuankai Chang et al.  ``An exploration of self-supervised pretrained representations for end-to-end speech recognition''  In _Proc. ASRU_ , 2021
  * [35] Zalán Borsos et al.  ``SoundStorm: Efficient Parallel Audio Generation''  In _arXiv preprint arXiv:2305.09636_ , 2023
  * [36] Haibin Wu et al.  ``Codec-SUPERB: An In-Depth Analysis of Sound Codec Models''  In _arXiv preprint arXiv:2402.13071_ , 2024
  * [37] Tzu-hsun Feng et al.  ``Superb@ SLT 2022: Challenge on generalization and efficiency of self-supervised speech representation learning''  In _Proc. SLT_ , 2023
  * [38] Hsiang-Sheng Tsai et al.  ``SUPERB-SG: Enhanced Speech processing Universal PERformance Benchmark for Semantic and Generative Capabilities''  In _Proc. ACL_ , 2022
  * [39] Jiatong Shi et al.  ``Findings of the 2023 ML-SUPERB Challenge: Pre-Training And Evaluation Over More Languages And Beyond''  In _Proc. ASRU_ , 2023
  * [40] Jade Copet et al.  ``Simple and controllable music generation''  In _NeurIPS_ 36, 2024
  * [41] Soumi Maiti et al.  ``Voxtlm: unified decoder-only models for consolidating speech recognition/synthesis and speech/text continuation tasks''  In _arXiv preprint arXiv:2309.07937_ , 2023
  * [42] Vassil Panayotov et al.  ``Librispeech: an ASR corpus based on public domain audio books''  In _Proc. ICASSP_ , 2015
  * [43] Shinji Watanabe et al.  ``ESPnet: End-to-End Speech Processing Toolkit''  In _Proc. Interspeech_ , 2018  DOI: 10.21437/Interspeech.2018-1456
  * [44] Takaaki Saeki et al.  ``UTMOS: Utokyo-sarulab system for voiceMOS Challenge 2022''  In _arXiv preprint arXiv:2204.02152_ , 2022
  * [45] Tomoki Hayashi et al.  ``ESPnet-TTS: Unified, reproducible, and integratable open source end-to-end text-to-speech toolkit''  In _Proc. ICASSP_ , 2020
  * [46] Tomoki Hayashi et al.  ``ESPnet2-TTS: Extending the edge of TTS research''  In _arXiv preprint arXiv:2110.07840_ , 2021
  * [47] Alec Radford et al.  ``Robust speech recognition via large-scale weak supervision''  In _Proc. ICML_ , 2023


#  Distributionally Robust LQG control under Distributed Uncertainty

Lucia Falconi, Augusto Ferrante, Mattia Zorzi L. Falconi, A. Ferrante, and M.
Zorzi are with the Department of Information Engineering, University of
Padova, Padova, Italy; Lucia Falconi’s activity is supported by Fondazione
CARIPARO, Italy (Borse di Dottorato CARIPARO 2020); e-mail:
<EMAIL_ADDRESS>(L. Falconi<EMAIL_ADDRESS>(A. Ferrante);
<EMAIL_ADDRESS>(M. Zorzi).

###### Abstract

A new paradigm is proposed for the robustification of the LQG controller
against distributional uncertainties on the noise process. Our controller
optimizes the closed-loop performances in the worst possible scenario under
the constraint that the noise distributional aberrance does not exceed a
certain threshold limiting the relative entropy pseudo-distance between the
actual noise distribution the nominal one. The main novelty is that the bounds
on the distributional aberrance can be arbitrarily distributed along the whole
disturbance trajectory. We discuss why this can, in principle, be a
substantial advantage and we provide simulation results that substantiate such
a principle.

###### Index Terms:

Optimal control, robustness, stochastic systems, Dynamic programming, relative
entropy.

##  I Introduction

Linear Quadratic (LQ) optimal control is arguably one of the most fundamental
ideas in modern control theory with ramifications in virtually all the fields
of the discipline. In particular, this is a natural approach for the control
of multiple-input and multiple-output (MIMO) systems. The theory may be easily
extended to stochastic systems by considering the Linear Quadratic Gaussian
(LQG) control which is the core idea in the development of many advanced
control techniques, such as all the MPC and MPC-like methods
[rawlings2017model].

The LQG method involves the design of a controller which optimizes
performances based on a nominal model of the process which is affected by a
Gaussian distributed disturbance with known mean and covariance. Thus, in the
nominal situation the LQG controller guarantees optimal performances. However,
soon after its introduction, it was recognized [doyle1978] that, as for a kind
of waterbed effect, optimality makes the LQG controller vulnerable to model
uncertainties.

This potential lack of robustness has motivated a formidable stream of
research aiming to ensure that the controller provides satisfactory
performances even in the presence of model misspecifications and inaccuracies.

The extensive research conducted in this area has resulted in a multitude of
available approaches, including $H^{\infty}$ control methods, based on the
minimization of $H^{\infty}$ performance criteria (see for example [ZHOU1985,
francis1987course, doyle1989, BERGELING2020]), and risk-sensitive LQG control,
where the standard quadratic loss function is replaced by an exponential
quadratic function [1973jacobson, Whittle].

Our work can be placed in the broad area of robustification of LQG control
and, more specifically, in the framework of _distributionally robust control
(DRC)_ , which takes into account uncertainty in the probability distribution
underlying a stochastic system. Instead of assuming a given distribution, DRC
methods design a control policy which minimizes the expected value of a given
cost under the worst-case distribution in a so-called _ambiguity set_. Several
types of ambiguity sets have been employed in DRC, exploiting moment
constraints [van2015distributionally], total variational distance
[tzortzis2015dynamic], Wasserstein distance [yang2020wasserstein] and relative
entropy [Petersen2000].

In this paper, we express the ambiguity set as a ball defined in the relative-
entropy topology and centered at the nominal distribution. This ambiguity set
based on relative entropy pseudo-distance has several attractive features.
First, relative entropy is a prominent metric in information theory which
satisfies important structural properties. In addition, it is the natural
“metric” between systems when they are identified from data,
[hansen2008robustness, ZORZI2017133]. Finally, it turns out that minimax LQG
control problem with a relative entropy constraint admits closed-form
solutions in the form of a parameterized, risk-sensitive optimal controller.
Thus it provides a new interpretation for the risk-sensitive paradigm.

Our work is closely related to the seminal paper [Petersen2000] by Petersen
_et al._ that proposed an entropy-based minimax optimal control problem for
stochastic uncertain systems. The main difference between our paper and
[Petersen2000] concerns the distribution of the uncertainty along the time
interval. Indeed, [Petersen2000] assumes a single constraint on the overall
noise distribution on the entire time interval. Conversely our approach
imposes a constraint to each time step, taking a point of view whose nature is
similar to the approach used by [levy2013, ZorziTAC2017], [9521824,
abadeh2018wasserstein] in the robust filtering setting. Petersen’s approach is
the natural way to model the uncertainties when the discrepancy between the
nominal and the actual system is due to the action of an adversary who can
manage a limited mismatch budget to perturb the nominal system and, when
convenient, is allowed to concentrate most (or all) such a budget in a few
time points. However, in most practical situations the model mismatch is a
consequence of modelling approximations and random fluctuations, and there is
no a real adversary. In these cases, since the same effort is usually made to
model each time step, it is more realistic to define a specific ambiguity set
at each step of the time interval. For this reason, we address the latter
problem. The reason for our choice is that, in Petersen’s approach, it may
happen that most of the uncertainty is concentrated on few time steps, leading
to unrealistic scenarios and to overly conservative conclusions.

We believe that this point may better clarified by two examples. Assume that
we want to design a robust LQG optimal controller for an airplane. The
Gaussian noise models the effects of the wind which, in practice, is not
Gaussian. Therefore, we need to guarantee robustness against different
distributions of the wind disturbance. By adopting the approach in
[Petersen2000], it may happen that the worst-case distribution differs from
the nominal one at just one time interval but in that time interval the
difference is unreasonably large: the controller must be able to counteract a
totally unrealistic wind concentrated at a single time point with potential
degradation of the performances. The possibility of distributing the
uncertainty along the whole trajectory and to select different uncertainties
radii depending on the reliability of the wind forecasts at each point is
clearly an interesting feature in this scenario.

A second example, is the case when the difference between the nominal Gaussian
noise and the actual one is used to account for errors in the model
parameters. Also in this case, the possibility of distributing the uncertainty
along the whole trajectory is clearly advantageous with respect to a potential
worst-case where the model mismatch is unreasonably concentrated at a single
time interval.

The main contributions of the paper can be summarized as follows. We propose a
state-feedback a Distributed uncertainty Distributionally robust LQG control
problem ($D^{2}$-LQG) for discrete-time linear stochastic uncertain systems
subject to a quadratic cost function. The uncertain systems are described in
terms of a nominal system, driven by white Gaussian noise, and a perturbed
system, in which a general class of stochastic noise processes are allowed. At
each time instant, a relative entropy tolerance is used to model a limit on
the admissible noise distributions. To address this problem, we adopt a
Dynamic Programming technique, where each subproblem consists in an infinite-
dimensional minimax optimization problem over a time interval of length one.
By exploiting the Lagrange duality theory and the duality between free-energy
and relative entropy, we are able to convert this problem into a risk-
sensitive optimal control problem with full-state information which can be
solved by using existing results in [1973jacobson]. The overall solution takes
the form of a risk-sensitive controller with a time-varying risk-sensitive
parameter. Of course the problem, as posed in our setting, is mathematically
much more challenging because of the large number of constraints translating
into a large number of dual variables. Nevertheless, we propose an effective
way to compute the solution for which we prove optimality.

The paper significantly extends [Falconi_CDC22], where a preliminary problem,
that is the worst-case performance analysis problem, is proposed.

The outline of the paper is as follows. In Section II we briefly recall the
definition and some important properties of the relative-entropy. Section III
introduces the class of stochastic uncertain systems considered in this work
and the relative-entropy ambiguity set. Section LABEL:sec:worstmodel deals
with a worst-case performance analysis problem with respect to a quadratic
cost. The results of this Section may be viewed as a preliminary step towards
the solution of DRC problem faced in Section LABEL:sec:minmax. The least-
favorable state-space model is derived in Section LABEL:sec:worstmodel.
Illustrative examples are presented in Section LABEL:sec:simulation which show
the advantages of distributing the model uncertainty. Finally, conclusions and
future works are discussed in Section LABEL:sec:conc. We extensively use some
convex optimization results that, for the convenience of the reader, are
recalled in the appendix.

### Notation

$\mathbf{R}$ is the real line; $\mathbf{R}_{+}$ ($\mathbf{R}_{++}$) is the
space of nonnegative (positive) scalars. Given a matrix $A$, $A^{\top}$
denotes the transpose. If $A$ is a square matrix, $|A|$ denotes its
determinant and, if it is also symmetric, we write $A\succ 0$ to denote that
$A$ is positive definite and $A\succeq 0$ to denote that $A$ is positive semi-
definite. $I$ is the identity matrix. $x\sim\mathcal{N}(\mu;\Sigma)$ means
that $x$ is a Gaussian random variable with mean $\mu$ and covariance matrix
$\Sigma$. The symbol $\mathbf{E}[\cdot]$ denotes the expectation with respect
to the probability density function $f(\cdot)$, whereas
$\tilde{\mathbf{E}}[\cdot]$ denotes the expectation with respect to the
density function $\tilde{f}(\cdot).$

##  II Relative Entropy and Optimization

In this section, we briefly recall the definition and some well-known
properties of the relative entropy (see, for example, [Daipra1996]).
Given any two probability measures $\tilde{f}$ and $f$ defined on
$\mathbf{R}^{n}$, the deviation of $\tilde{f}$ from $f$ is measured by the
_relative entropy_ (or Kullback-Leibler divergence), which is defined as

$R(\tilde{f}||f):=\int_{\mathbf{R}^{n}}\ln\frac{\tilde{f}(x)}{f(x)}\tilde{f}(x)dx.$

$R(\cdot||\cdot)$ is not a distance since it is not symmetric, and, more
importantly, it does not satisfy the triangle inequality. However it is a
pseudo-distance as it satisfies the following property: $R(\tilde{f}||f)\geq
0$ and $R(\tilde{f}||f)=0$ if and only if $\tilde{f}(x)=f(x)$ a.e. Moreover,
it is well known that, for a given probability density function $f$,
$R(\tilde{f}||f)$ is a strictly convex function of $\tilde{f}$ on the set of
probability density functions.

###### Lemma II.1

Suppose that $\tilde{f}$ and $f$ are Gaussian density functions on
$\mathbf{R}^{n}$ with the same (non-singular) covariance matrix $V$:
$f\sim\mathcal{N}(v,V)$ and $\tilde{f}\sim\mathcal{N}(\tilde{v},V)$ Then

$R(\tilde{f}||f)=\frac{1}{2}(\tilde{v}-v)^{\top}V^{-1}(\tilde{v}-v).$

Finally, we recall the duality between free energy and relative entropy which
underlies the solution to the proposed stochastic optimization problem:

######  Lemma II.2 (Relative Entropy and Free Energy duality)

For a given probability density function $f:\mathbf{R}^{n}\to\mathbf{R}_{+}$
and a bounded measurable function $J:\mathbf{R}^{n}\to\mathbf{R}:$

$\sup_{\tilde{f}(\cdot)\in\mathcal{P}}\int_{\mathbf{R}^{n}}J(x)\tilde{f}(x)dx-R(\tilde{f}||f)=\log\int_{\mathbf{R}^{n}}e^{J(x)}f(x)dx$

where the supremum is taken over the set $\mathcal{P}$ of all the probability
density functions $\tilde{f}(x)$ on $\mathbf{R}^{n}$. Moreover, the supremum
is achieved by the probability density function

$\tilde{f}^{o}(x)=f(x)\frac{e^{J(x)}}{\int_{\mathbf{R}^{n}}e^{J(\bar{x})}f(\bar{x})d\bar{x}}.$
(1)

##  III Stochastic Uncertain Systems

This section introduces the class of uncertain systems considered in this
paper. We consider discrete-time linear stochastic uncertain systems defined
over a finite time interval, described in terms of a _reference or nominal
system_ with a specified statistical description, and a _perturbed system_ in
which a general class of stochastic noise processes are allowed.

1) _Reference system:_ The reference system is described by the following
state equation:

$x_{t+1}=Ax_{t}+Bu_{t}+v_{t}\qquad t=0,\dots,N$ (2)

where $x_{t}\in\mathbf{R}^{n}$ is the state vector, $u_{t}\in\mathbf{R}^{m}$
is the control input and $\\{v_{t}\in\mathbf{R}^{n},t=0,\dots,N\\}$ is a zero-
mean, white Gaussian noise sequence with covariance matrix $V\succ 0.$ Notice
that $V\succ 0$ means that $v_{t}$ affects all components of the dynamics,
possibly with a very small variance for relations which are viewed as almost
certain. This is a typical assumption when the relative entropy is used to
measure the proximity of statistical models (see for example [levy2013]). The
matrices $A,B$ and $V$ have appropriate dimensions, and, for the ease of
notation, they are assumed time independent. Generalization to time-varying
systems only entails heavier notations. The conclusions however are exactly
the same and can be reached by the exact same arguments.

2) _Perturbed system:_ The corresponding perturbed system is described by the
state equation

$\displaystyle x_{t+1}$ $\displaystyle=Ax_{t}+Bu_{t}+\tilde{v}_{t},\qquad
t=0,\dots,N$ $\displaystyle z_{t}$ $\displaystyle=E_{1}x_{t}+E_{2}u_{t}$ (3)

where the noise input sequence $\\{\tilde{v}_{t},t=0,\dots,N\\}$ is defined by
an unknown probability density function. The quantity $z_{t}\in\mathbf{R}^{p}$
is a signal which defines the set of allowable uncertain noise probability
measures via the relative entropy constraint. One possible interpretation of
this signal will be given at the end of the section. The matrices $E_{1}$ and
$E_{2}$ are known matrices used to add more flexibility in modelling the
system’s uncertainty. To simplify the computations we assume that

$E_{1}^{\top}E_{2}=0.$

Our results can be easily extended to the case in which we drop this
assumption.

3) _Admissible Controller:_ We suppose that the whole state is accessible with
negligible measure noise and we consider state feedback control policies of
the form:

$u_{t}=\pi_{t}(x_{t}),\qquad t=0,\dots,N$ (4)

where $\pi_{t}$ is a measurable map from $\mathbf{R}^{n}$ to $\mathbf{R}^{m}.$
As discussed later, if we enlarge the set of feasible control policies by
allowing $\pi$ to depend on the the whole past history of the state, the
optimal solution remains the same. In other words, there is no limitations in
considering only the control policies (4). This is indeed rather intuitive if
we consider the Markovian property of the state.

We suppose that the initial state $x_{0}=\bar{x}_{0}$ is a deterministic
quantity. Then, for any given admissible control law of the form (4), the
nominal joint probability density function of the state sequence $X_{0:N+1}$
can be expressed as

$f(X_{0:N+1})=\delta(x_{0}-\bar{x}_{0})\prod_{t=1}^{N+1}f(x_{t}|x_{t-1})$ (5)

where

$f(x_{t}|x_{t-1})=\mathcal{N}(Ax_{t-1}+Bu_{t-1},V).$ (6)

We consider a similar decomposition for the joint probability measure
$\tilde{f}(X_{0:N-1})$ of the perturbed system:

$\tilde{f}(X_{0:N+1})=\delta(x_{0}-\bar{x}_{0})\prod_{t=1}^{N+1}\tilde{f}(x_{t}|x_{t-1}).$

3) _Relative Entropy Constraints:_ We use the relative entropy between the
true and the nominal densities to measure modeling errors. Instead of applying
a single constraint to the relative entropy of the true and nominal
probability densities of the state sequence $X_{0:N+1}$ over the whole time
interval as done in the seminal paper [Petersen2000], we specify separate
modeling tolerances for each time step of the transitions density
$f(x_{t}|x_{t-1}).$ Specifically, we use the relative entropy between the true
and nominal transition densities at time $t$, namely
$\tilde{f}(x_{t}|x_{t-1})$ and $f(x_{t}|x_{t-1})$, to measure modeling errors.
Given any admissible control law $\pi=\\{\pi_{t}\;,\;t=0,\dots,N\\}$, we
introduce the quantity

$\displaystyle R_{t}$
$\displaystyle:=R_{t}\big{(}\tilde{f}(x_{t}|x_{t-1})||f(x_{t}|x_{t-1})\big{)}$
$\displaystyle=\int_{\mathbf{R}^{n}}\ln\left(\frac{\tilde{f}(x_{t}|x_{t-1})}{f(x_{t}|x_{t-1})}\right)\tilde{f}(x_{t}|x_{t-1})dx_{t}.$

Then, the model mismatch at time $t$ is expressed by the constraint:

$\tilde{\mathbf{E}}[R_{t}]\leq\tilde{\mathbf{E}}[\frac{1}{2}\|z_{t-1}\|^{2}]+d_{t}$
(7)

where the expectation is taken with respect to the true marginal

$\displaystyle\tilde{f}(x_{t-1})=\int\tilde{f}(x_{t-1}|x_{t-2})\dots\tilde{f}(x_{1}|\bar{x}_{0})dx_{1}\dots
dx_{t-1}.$

The ambiguity set is defined by the convex ball of functions
$\mathcal{B}_{t}:$

$\mathcal{B}_{t}:=\left\\{\tilde{f}(x_{t}|x_{t-1})\in\mathcal{P}\;:\;\tilde{\mathbf{E}}[R_{t}]\leq\tilde{\mathbf{E}}[\frac{1}{2}\|z_{t-1}\|^{2}]+d_{t}\right\\}$

and the parameter $d_{t}>0$ is called tolerance and quantifies the constant
mismatch budget allowed at time $t$. Clearly, $\mathcal{B}_{t}$ is
parameterized by the density function $\tilde{f}(x_{t-1}).$ It is worth noting
that the model uncertainty is spread over the finite interval, unlike
[Petersen2000] where the model uncertainty is characterized using one relative
entropy constraint regarding the entire interval. The latter characterization
allows to concentrate the uncertainty in one step, which is not a common
situation in practice. In contrast, the former promotes an equal distribution
of uncertainty along the time interval.

NominalSystem$\Delta$$u_{t}$$x_{t}$$z_{t}$$\bar{v}_{t}$Uncertain noise
$\tilde{v}_{t}$Reference noise $v_{t}$ Figure 1: Linear fractional
transformation (LTF) uncertain model.

We can motivate the above stochastic uncertainty description by interpreting
it in terms of linear fractional transformation (LFT) model
[Petersen2014RobustCO]. LFT is a very general and powerful tool to represent
various sources of uncertainty in dynamical models, frequently used in modern
control theory. It consists in separating the nominal model from the
uncertainty in a feedback interconnection as shown in Figure 1. The
uncertainty operator $\Delta$ is a quantity which is typically unknown but
bounded in magnitude. This is a very general paradigm both conceptually and
practically. In fact, in most cases the uncertainty on the system dynamics can
be effectively modelled by suitably selecting the admissible structure of
$\Delta$ and the way in which $\Delta$ is bounded in magnitude. This is indeed
a very natural way of modelling uncertainty for systems learned from data. We
can interpret the signal $z(t)$ in (III) as the input of the uncertain block
$\Delta$ in the LFT model. Then the uncertain system (2)-(III)-(7) allows for
any uncertainty block $\Delta$ whose norm is bounded. Notice that $\Delta$ can
be dynamic and time-varying: in this case the notation $\Delta_{t}$ is
adopted.

###### Lemma III.1

Consider the stochastic uncertain system (2)-(4). Assume that the perturbed
noise signal ${\tilde{v}_{t}}$ is generated according to the LFT model of
Figure 1 where the uncertainty block is a time-varying matrix
$\Delta_{t}\in\mathbf{R}^{n\times p},$ _i.e._

$\tilde{v}_{t}=v_{t}+\bar{v}_{t}=v_{t}+\Delta_{t}z_{t},\qquad t=0,\dots,N.$
(8)

If

$\Delta_{t}\Delta_{t}^{\top}\preceq V,\qquad t=0,\dots,N,$ (9)

then the relative-entropy constraint (7) is satisfied at each time instant for
any $d_{t}>0.$

###### Proof:

From the definition of the perturbed signal (8), it follows that
$\tilde{f}(x_{t+1}|x_{t})=\mathcal{N}(Ax_{t}+Bu_{t}+\Delta_{t}z_{t},V).$
Hence, given $x_{t}\in\mathbf{R}^{n},$

$R_{t+1}=\frac{1}{2}z_{t}^{\top}\Delta_{t}^{\top}V^{-1}\Delta_{t}z_{t}\leq
z_{t}^{\top}z_{t},$

where the equality follows from Lemma II.1 and the inequality form the norm
bound (9). Now, the thesis is obtained by taking the expectation and by
considering the monotonicity of the expected value and the positivity of
$d_{t+1}.$ $\blacksquare$

In conclusion, our paradigm appears to be the natural framework to address the
robustification of LQG control solutions in all the ubiquitous situations in
which the uncertainty may be represented as in Figure 1. The main advantage
with respect to existing literature is that our formulation is ideally suited
to capture the essence of the problem whenever the uncertainty is distributed
along the whole time interval.

##  IV Worst Performance Analysis

In this section, we consider the problem of characterizing the worst-case
performances with respect to a quadratic cost function for a stochastic
uncertain system of the form (2)-(7) with $u_{t}\equiv 0$. The solution to
this problem is presented in [CDC22_Lucia] and it puts the basis for the
synthesis of the minimax LQG controller under incremental model uncertainty
which will be addressed in the next section.

By assuming that the input signal is identically equal to zero, the reference
system state equation (2) becomes:

$x_{t+1}=Ax_{t}+v_{t},\quad t=0,\dots,N$ (10)

which specifies the nominal conditional density

$f(x_{t}|x_{t-1})\sim\mathcal{N}(Ax_{t-1},V)\quad t=1,\dots N+1.$ (11)

The corresponding perturbed system is described by

$\begin{split}x_{t+1}&=Ax_{t}+\tilde{v}_{t}\\\ z_{t}&=E_{1}x_{t}\end{split}$
(12)

where the probability distribution of the perturbed noise sequence
$\tilde{v}_{t}$ is subject to the incremental relative entropy constraint (7).

For the above stochastic uncertain system, we consider the problem of
characterizing the worst case performance with respect to the cost functional:

$J=\frac{1}{2}\sum_{t=0}^{N}x_{t}^{\top}Qx_{t}+\frac{1}{2}x_{N+1}^{\top}Q_{N+1}x_{N+1},$
(13)

where $Q\succeq 0$, $Q_{N+1}\succ 0.$ This is the typical cost function in
linear quadratic problems for autonomous systems. In the following it is
assumed that the pair $(A,Q)$ is observable.

More precisely, the problem under consideration is to find

$\max_{\left\\{\tilde{f}(x_{t}|x_{t-1})\in\mathcal{B}_{t},\;\;t=1,..,N+1\right\\}}\;\;\tilde{\mathbf{E}}[J].$
(14)

Because of the constraints’ and the objective function’s special structure, we
can obtain the overall solution by optimizing a sequence of $N+1$ single-
variable constrained optimization problems by adopting a _backward dynamic
programming_ technique [rawlings2017model, ch. I]. We start by optimizing the
objective function with respect to $\tilde{f}(x_{N+1}|x_{N})$, that is we
consider the maximization problem

$\max_{\tilde{f}(x_{N+1}|x_{N})\in\mathcal{B}_{N+1}}\tilde{\mathbf{E}}\left[J\right].$
(15)

where $\tilde{f}(X_{0:N})$ appears as a parameter.
Next we show that the constrained stochastic optimization problem (15) can be
replaced by a corresponding unconstrained stochastic optimization problem by
means of the _duality theory_ , and in particular of Theorem
LABEL:lemma:luenberger. To this end, we first introduce the Lagrange
multiplier $\tau_{N+1}\in\mathbf{R}_{+}$ associated with the constraint (7)
for $t=N+1.$ Then, we define the augmented cost functional

$J_{\tau_{N+1}}:=J-\tau_{N+1}[R_{N+1}-\frac{1}{2}\|z_{N}\|^{2}-d_{N+1}],$

and we denote by $W_{\tau_{N+1}}$ the value of the following unconstrained
optimization problem:

$\displaystyle W_{\tau_{N+1}}$
$\displaystyle:=\max_{\tilde{f}(x_{N+1}|x_{N})\in\mathcal{P}}\tilde{\mathbf{E}}[J_{\tau_{N+1}}].$
(16)

By applying Theorem LABEL:lemma:luenberger to the current optimization problem
(15), we obtain the following theorem:

###### Theorem IV.1

Consider the stochastic uncertain system (10) - (12) subject to the constraint
(7), and the cost functional $J$ given by (13). Then the value of the
stochastic optimization problem (15) is finite if and only if
$\Gamma_{N+1}:=\\{\tau_{N+1}>0\;|\;W_{\tau_{N+1}}<\infty\\}$ is non empty. In
this case,

$\max_{\tilde{f}(x_{N+1}|x_{N})\in\mathcal{B}_{N+1}}\tilde{\mathbf{E}}[J]=\min_{\tau_{N+1}>0}W_{\tau_{N+1}}.$
(17)


# Automatic Modulation Classification Using Involution Enabled Residual
Networks

Hao Zhang, , Lu Yuan, Guangyu Wu,
Fuhui Zhou, , and Qihui Wu  H. Zhang, L. Yuan, F. Zhou, and Qihui Wu are with
College of Electronic and Information Engineering, Nanjing University of
Aeronautics and Astronautics, Nanjing 211106 China. They are also with Key
Laboratory of Dynamic Cognitive System of Electromagnetic Spectrum Space
(Nanjing University of Aeronautics and Astronautics), and with Ministry of
Industry and Information Technology, Nanjing, 211106, China (email:
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Guangyu Wu is with Department of Computer Science and
Technology, University of Science and Technology of China, Hefei 230026 China
(email<EMAIL_ADDRESS>
###### Abstract

Automatic modulation classification (AMC) is of crucial importance for
realizing wireless intelligence communications. Many deep learning based
models especially convolution neural networks (CNNs) have been proposed for
AMC. However, the computation cost is very high, which makes them
inappropriate for beyond the fifth generation wireless communication networks
that have stringent requirements on the classification accuracy and computing
time. In order to tackle those challenges, a novel involution enabled AMC
scheme is proposed by using the bottleneck structure of the residual networks.
Involution is utilized instead of convolution to enhance the discrimination
capability and expressiveness of the model by incorporating a self-attention
mechanism. Simulation results demonstrate that our proposed scheme achieves
superior classification performance and faster convergence speed comparing
with other benchmark schemes.

###### Index Terms:

Automatic modulation classification, deep learning, involution, residual
networks.

##  I Introduction

AUTOMATIC modulation classification (AMC) is a vital technique to identify the
modulation formats under noise and interference [1]. AMC has been widely used
in military and civilian applications, such as spectrum management, electronic
warfare, interference identification [2, 3]. The AMC schemes mainly have two
categories, namely, model-driven AMC and data-driven AMC. The model-driven
schemes can be mainly include likelihood-based (LB) schemes and feature-based
(FB) schemes [1]. The LB schemes can obtain the optimal solution from the
Bayes’ sense by calculating the likelihood function under the modulation
hypothesis. The FB schemes aim to find better features and have lower
complexity with robust performances.

Recent studies have demonstrated that data-driven AMC can achieve superior
classification performances compared to the model-driven schemes by learning
effective representations from data [4, 5, 6]. Machine learning (ML) based
schemes such as support vector machine (SVM), K-nearest neighbor (KNN), and
logistic regression can recognize the modulation formats by using a large
number of data from the received signals. However, ML-based models still rely
on the features generated by FB-based schemes. Thus, deep learning (DL) based
algorithms [4, 5, 6] were proposed to extract features automatically from the
original data, such as I/Q samples, cyclic spectrum, constellation diagrams.

Recently, many novel neural networks were designed to extract discriminative
representations for AMC in order to improve the classification performance. A
long short-term memory (LSTM) based AMC algorithm was proposed in [7] to
identify modulation formats, but its recurrent structure results in high
computational complexity. Inspired by the residual learning for image
classification, a modified residual network (ResNet) was applied to extract
features from the received I/Q symbols for AMC [4]. However, the
classification performance is limit due to the over-fitting problem caused by
a large number of network parameters [6]. Besides those networks, other
promising techniques were also used for advancing the performance. For
example, a graph convolutional network was investigated for AMC algorithm in
[8]. These novel neural networks have improved the classification performance
significantly. However, the performance is still limit due to the complex
environments, and deep networks need a longer time to converge. In the beyond
fifth generation (5G wireless communication networks), intelligent
communications with high reliability and low latency are the main
characteristics. However, the traditional AMC schemes based on convolution
cannot satisfy the classification performance and low computing cost
requirements of the beyond 5G wireless communication networks.

In this letter, we propose a residual network (ResNet) based AMC scheme using
involution [9]. The contributions are summarized as follows. Firstly,
involution is utilized instead of convolution to enhance the discrimination
capability and expressiveness of the model by incorporating a self-attention
mechanism. Secondly, a novel network that inherits the advantages of residual
learning and involution is designed to learn high-dimensional representations
of different modulations from I/Q signals and construct the classifier.
Thirdly, simulation results demonstrate the effectiveness of the proposed
involution based ResNet for AMC. It is shown that our proposed scheme has a
better classification performance and a faster convergence speed.

The reminder of this paper is organized as follows. Section II presents the
problem of AMC. Section III presents our proposed AMC scheme. Simulation
results are given in Section IV and Section V concludes this letter.

##  II Problem Statement

According to the classical modulation classification problem statement [4, 5],
the received signal can be given as

$x(n)=s(n)+\omega(n),n=1,2,\cdots,N,$ (1)

where $N$ is the total number of signal symbols, $s(n)$ denotes the $n$th
(complex) symbol, and $\omega(n)$ is the additive white Gaussian noise (AWGN)
with zero mean and variance $\delta^{2}_{\omega}$.

The real and imaginary parts of the received signal from the In-phase and
Quadrature (I/Q) parts are utilized, which can be expressed as a vector, given
as

$\displaystyle\mathbf{x}$ $\displaystyle=\mathbf{I_{x}}+\mathbf{Q_{x}}$
$\displaystyle=\Re(\mathbf{x})+j\Im(\mathbf{x}),$ (2)

where $\mathbf{I_{x}}$ and $\mathbf{Q_{x}}$ denote the real and imaginary
parts of the received signal, respectively, and $j=\sqrt{-1}$. $\Re(\cdot)$
and $\Im(\cdot)$ represent the operators of the real and imaginary parts of
the signal, respectively. $\mathbf{x}$ can be specifically expressed as

$\mathbf{x}=\left(\begin{array}[]{c}\Re[x(1),x(2),\ldots,x(N)]\\\
\Im[x(1),x(2),\ldots,x(N)]\\\ \end{array}\right).$ (3)

The average probability of correct classification ($\Pr_{cc}$) is utilized as
the performance metric, which is defined as
$\Pr_{cc}=\sum^{|\mathcal{S}|}_{s=1}\Pr(\hat{H}=H_{s}|H_{s})\Pr(H_{s}),H_{s}\in\mathcal{S}$,
where $\mathcal{S}$ denotes the candidate modulation formats. $\Pr(H_{s})$ is
the prior probability of modulation format $H_{s}$, which is equal for each
format. $\Pr(\hat{H}=H_{s}|H_{s})\Pr(H_{s})$ represents the probability that
the modulation format is correctly determined as $H_{s}$.

##  III Involution Enabled ResNet for AMC

In this part, we first introduce the standard convolution operation to make
the definition of the proposed involution clearly. Then, involution which
inverses characteristics of convolution in the spatial and channel domain with
low complexity is presented. Finally, a residual network based on involution
for AMC is proposed.

Let $\mathbf{X}\in\mathbb{R}^{H\times W\times C_{i}}$ denote the input feature
map, where $H$, $W$, and $C_{i}$ represent its height, width and input
channels, respectively. A series of convolution filters $C_{o}$ with the fixed
kernel size of $K\times K$ are expressed as
$\bm{\mathcal{F}}\in\mathbb{R}^{C_{o}\times C_{i}\times K\times K}$, where
each filter $\bm{\mathcal{F}}_{k}\in\mathbb{R}^{C_{i}\times K\times
K},k=1,2,\cdots,C_{o}$, contains $C_{i}$ convolution kernels
$\bm{\mathcal{F}}_{k,c}\in\mathbb{R}^{K\times K},c=1,2,\cdots,C_{i}$. The
filter executes multiply-add operations on the input feature map using a
sliding window to generate the output feature map
$\mathbf{Y}\in\mathbb{R}^{H\times W\times C_{o}}$, given as

$\mathbf{Y}_{i,j,k}=\sum^{C_{i}}_{c=1}\sum_{(u,v)\in\Delta_{K}}\bm{\mathcal{F}}_{k,c,u+\lfloor
K/2\rfloor,v+\lfloor K/2\rfloor}\mathbf{X}_{i+u,j+v,c},$ (4)

where $\Delta_{K}\in\mathbb{Z}^{2}$ denotes the set of offsets in the
neighborhood considering convolution conducted on the center pixel, given as

$\Delta_{K}=[-\lfloor K/2\rfloor,\cdots,\lfloor K/2\rfloor]\times[-\lfloor
K/2\rfloor,\cdots,\lfloor K/2\rfloor].$ (5)

It is well known that there is inter-channel redundancy inside convolution
filters, which results in the flexibility problem in convolution operation
[9]. Compared to the standard convolution, the involution kernel
$\bm{\mathcal{H}}\in\mathbb{R}^{H\times W\times K\times K\times G}$ is
designed to realize transforms with inverse characteristics in the spatial and
channel domain. Specifically, an involution kernel
$\bm{\mathcal{H}}_{i,j,\cdot,\cdot,g}\in\mathbb{R}^{K\times
K},g=1,2,\cdots,G$, is specially adapted for the pixel
$\mathbf{X}_{i,j}\in\mathcal{R}^{C}$ located at the corresponding coordinate
$(i,j)$, but shared over the channels. $G$ denotes the number of groups and
each group shares the same involution kernel. The output feature map of
involution is derived by executing multiply-add operations on the input with
the involution kernels, given as

$\mathbf{Y}_{i,j,k}=\sum_{(u,v)\in}\bm{\mathcal{H}}_{i,j,u+\lfloor
K/2\rfloor,v+\lfloor K/2\rfloor,\lceil kG/C\rceil}\mathbf{X}_{i+u,j+v,k}.$ (6)

Different from convolution kernels, the shape of involution kernels
$\bm{\mathcal{H}}$ depends on the input feature map $\mathbf{X}$. The output
kernels are aligned to the input by generating the involution kernels based on
the original input tensor. Thus, the kernel generation function $\phi$ and
mapping function at each location ($i,j$) can be expressed as

$\bm{\mathcal{H}}_{i,j}=\phi(\mathbf{X}_{\Psi_{i,j}}),$ (7)

where $\Psi_{i,j}$ indexes the set of pixels $\bm{\mathcal{H}}_{i,j}$ is
conditioned on. The kernel generation function
$\phi:\mathbb{R}^{C}\to\mathbb{R}^{K\times K\times G}$ with
$\Psi_{i,j}=\\{(i,j)\\}$ is given as

$\bm{\mathcal{H}}_{i,j}=\phi(\mathbf{X}_{i,j})=\mathbf{W}_{1}\sigma(\mathbf{W}_{0}\mathbf{X}_{i,j}),$
(8)

where $\mathbf{W}_{0}\in\mathbb{R}^{\frac{C}{r}\times C}$ and
$\mathbf{W}_{1}\in\mathbb{R}^{(K\times K\times G)\times\frac{C}{r}}$ denote
two linear transformations that collectively constitute a bottleneck
structure. The channel reduction operation under a ratio $r$ is used for
efficient processing, and $\sigma$ represents Batch Normalization and non-
linear activation functions between two linear projections. The pseudo code of
Algorithm 1 shows the computation flow of involution, which is visualized in
Fig. 1.

Algorithm 1 Pseudo code of involution.

0: Batch size $B$, height $H$, width $W$, channel $C$, group number $G$,
kernel size $K$, stride $s$, and reduction ratio $r$;

0: The involution kernel $out$.

1: Initialize the network operations;

2: Define the operation $o$ as average pooling with kernel $s$ if $s>1$,
otherwise $o$ is the identity mapping.

3: Define the operation $reduce$ as convolution with kernel of $C\times
C//r\times 1$.

4: Define the operation $span$ as convolution with kernel of $C//r\times
K*K*G\times 1$.

5: Define the operation $unfold$ as unfold with kernel of $K\times
dilation\times padding\times s$.

6: Forward pass;

7: Calculate $x\\_u$ using $x\\_u=unfold(x)$, and reshape it into
$(B,G,C//G,K*K,H,W)$;

8: Generate the involution kernel using eq. 8 as $kernel=span(reduce(o(x)))$,
and reshape it into $(B,G,K*K,H,W)$;

9: Execute Multiply-Add operation according to eq. 6 as
$out=mul(kernel,x\\_u).sum(dim=3)$, and reshape it into $(B,C,H,W)$.

The feature generation process in eq. 6 can be considered a generalized
version of self-attention [10]. The self-attention pools _values_ $\bm{V}$
depending on the affinities obtained by computing similarity between the
_query_ $\bm{Q}$ and _key_ $\bm{K}$, are formulated as

$\bm{Y}_{i,j,k}=\sum_{(p,q)\in\Omega}(\bm{QK}^{\top})_{i,j,p,q,\lceil
kH/C\rceil}\bm{V}_{p,q,k},$ (9)

where $\bm{Q}$, $\bm{K}$, and $\bm{V}$ are linearly transformed from the input
$\bm{X}$, and $H$ is the number of heads in multi-head self-attention [10].
The similarity is related to that both operators collect pixels in the
neighborhood $\delta$ or a less bounded range $\Omega$ through a weighted sum.
On one hand, the computation of involution can be viewed as a spatial
attentive aggregation. On the other hand, the attention map (also called
affinity or similarity matrix) $\bm{QK}^{\top}$ in the self-attention
mechanism can be considered as a kind of involution kernel $\bm{\mathcal{H}}$.

Figure 1: Schematic illustration of the proposed involution. The involution
kernel $\mathcal{H}_{i;j}\in\mathbb{R}^{K\times K\times 1}$ ($G=1$ for ease of
demonstration) is produced from the function $\phi$ based on a single pixel at
$(i,j)$, followed by a channel-to-space rearrangement. The multiply-add
operation of involution consists of two steps, where $N$ indicates
multiplication broadcast across $C$ channels and $L$ represents summation
aggregated within the $K\times K$ spatial neighborhood.  Figure 2: Structure
of the proposed Invo-ResNet, (a) network structure, (b) bottleneck, (c)
bottleneck with downsampling using maxpooling.

To build an entire network with involution for AMC, we design a novel
lightweight network named Invo-ResNet by stacking bottleneck blocks and using
involution kernels since the elegant architecture of ResNet makes it
successful for many applications [11]. As shown in Fig. 2, the proposed Invo-
ResNet consists of two modules, namely, the feature extraction module and
classification module. The feature extraction module aims at extracting the
underlying high-level representations from the input signals. Then, these
representations are transformed into fixed-dimensional feature vectors by the
global average pooling (GAP) layer. Modulation classification is subsequently
conducted using these vectors in the fully connected (FC) layer of the
classification module.

To balance the tradeoff between accuracy and efficiency, the feature
extraction module consists of one convolutional layer and two bottlenecks. A
convolutional layer with a kernel size of $3\times 1$ is used to extract low-
level information and execute the feature aggregation. Then, the bottleneck is
utilized to learn high-level features by using involution with fewer
parameters compared to convolution. As shown in Fig. 2 (b), the bottleneck is
constructed by replacing the $3\times 3$ convolutional layer in the original
ResNet [4] with involution and retaining all the $1\times 1$ convolution for
channel projection and fusion [9]. Instead of using a constant channel number
as in [4], a pyramid architecture with increasing channel numbers is utilized
in the Invo-ResNet. Therefore, maxpooling is adopted to decrease the shape of
the feature map in order to reduce the computation cost, as shown in Fig.
2(c).

For the last part, a GAP layer and an FC layer are served as the
classification module. GAP layer aggregates information from the feature
extraction module and enables the network to take samples with an arbitrary
length. _Softmax_ is used as the activation function in the last FC layer to
normalize the output of each neuron, indicating the probability that the
target signal belongs to the corresponding modulation format. Moreover, we
adopt rectified linear units (ReLU) as the activation function in the
convolutional layers to introduce nonlinearity and sparsity.

##  IV Simulation

The pubic RadioML 2018.01A that contains $24$ kinds of modulations under an
SNR range from $-20$dB to $30$dB with a step of $2$dB is adopted. There are
over $2$ million samples with 1024 points. $80\%$ of these data are utilized
for training and the rest $20\%$ are used for testing. The models are trained
by using SGD with an initial learning rate of $0.01$, a weight decay of
$5\times 10^{-4}$, and a momentum of $0.9$ for $50$ epochs.

Fig. 3 shows the classification performance comparison of our proposed
proposed Invo-ResNet with those achieved by three representative DL-based
models for AMC including VGG [4], ResNet [4] and MCNet [5] on RadioML 2018.01A
dataset. It is evident that the proposed Invo-ResNet is superior to other
traditional models, and it can provide $2\%$ gains over our previous work
MSNet, $2$ dB gains over ResNet, $3$ dB gains over MCNet and $4$ dB gains over
VGG. Moreover, Invo-ResNet can reach over $90\%$ accuracy when the SNR is
larger than $6$dB, and it can achieve about $95\%$ accuracy at $10$dB, while
the best performance of ResNet is $91.47\%$ at $10$dB. To further show the
superiority of the proposed Invo-ResNet, Fig. 3 illustrates the training loss
during the training process of the compared models. The proposed Invo-ResNet
can achieve a lower loss in the training set compared to the other models and
obtains a faster convergence speed than other models. To show the
effectiveness of the involution for AMC, a fully convolutional network with
the same structure is trained and tested under the same dataset. As shown in
Fig. 3, compared to our proposed scheme with involution, the convolutional
counterpart (Conv-ResNet) only reaches a best performance of $94.5\%$ at high
SNR condition, which is about $1\%$ lower than involution based model, and it
performs worse than our previous work MSNet. For the analysis of the
computational complexity, we calculate the network parameters of the compared
models, as shown in Fig. 3. Our proposed scheme achieves a better performance
with less parameters than these conventional schemes. Moreover, compared to
the convolution based scheme under the same structure, our proposed scheme
with involution has about $40\%$ reduced parameters, which can be implemented
in resource limited devices.

Figure 3: Comparison of (a) the classification performance and (b) the
training loss among VGG, ResNet, MSNet,  MCNet, our proposed scheme with
convolution layers and our proposed scheme with involution layers.

Figure 4: Comparison of confusion matrix at $10$dB, (a) ResNet [4], (b) MSNet
[1], and (c) Our proposed scheme.

To further demonstrate the effectiveness of the proposed Invo-ResNet, the
comparison of the confusion matrices at $10$dB of ResNet [4], MSNet [1] with
those of our proposed scheme are shown in Fig. 4. It is seen that the proposed
Invo-ResNet has less confusion compared to other traditional models.
Specifically, only AM-DSB-WC receives the worst performance in our proposed
scheme. On the contrary, two modulation formats perform worse in ResNet and
MSNet, which are AM-DSB-SC and $64$QAM, and AM-DSB-SC and AM-SSB-WC as shown
in Fig. 4 and Fig. 4, respectively.

##  V Conclusion

A novel AMC scheme was proposed by designing a novel network using involution.
In order to improve the classification accuracy and decrease the computation
cost of convolution based AMC schemes, involution was utilized to enhance the
discrimination capability and expressiveness of the model and reduce the
training time by incorporating a self-attention mechanism. Simulation results
demonstrated the superiority of our proposed scheme in terms of classification
accuracy and the training time. In this case, our proposed scheme is more
appropriate in beyond 5G wireless communication networks

## References

  * [1] H. Zhang _et al._ , “A novel automatic modulation classification scheme based on multi-scale networks,” _IEEE Trans. Cogn. Commun. Netw._ , 2021.
  * [2] Q. Wu _et al._ , “Spatial-temporal opportunity detection for spectrum-heterogeneous cognitive radio networks: Two-dimensional sensing,” _IEEE Trans. Wireless Commun._ , vol. 12, no. 2, pp. 516–526, 2013.
  * [3] Q. Wu _et al._ , “Cognitive internet of things: a new paradigm beyond connection,” _IEEE Internet Things J._ , vol. 1, no. 2, pp. 129–143, 2014.
  * [4] T. J. O’Shea _et al._ , “Over-the-air deep learning based radio signal classification,” _IEEE J. Sel. Top. Sign. Proces._ , vol. 12, no. 1, pp. 168–179, 2018.
  * [5] H.T. Thien _et al._ , “MCNet: An efficient CNN architecture for robust automatic modulation classification,” _IEEE Commun. Lett._ , vol. 24, no. 4, pp. 811–815, 2020.
  * [6] S. Huang _et al._ , “Automatic modulation classification using contrastive fully convolutional network,” _IEEE Wireless Commun. Lett._ , vol. 8, no. 4, pp. 1044–1047, 2019.
  * [7] S. Rajendran _et al._ , “Deep learning models for wireless signal classification with distributed low-cost spectrum sensors,” _IEEE Trans. Cogn. Commun. Netw._ , vol. 4, no. 3, pp. 433–445, 2018.
  * [8] Y. Liu _et al._ , “Modulation recognition with graph convolutional network,” _IEEE Wireless Commun. Lett._ , vol. 9, no. 5, pp. 624–627, 2020.
  * [9] D. Li _et al._ , “Involution: Inverting the inherence of convolution for visual recognition,” _arXiv preprint arXiv:2103.06255_ , 2021.
  * [10] A. Vaswani _et al._ , “Attention is all you need,” in _Adv. neural inf. proces. syst._ , 2017, pp. 5999–6009.
  * [11] P. Qi _et al._ , “Automatic modulation classification based on deep residual networks with multimodal information,” _IEEE Trans. Cogn. Commun. Netw._ , 2020.