# Enhanced Low-Complexity FDD System Feedback with Variable Bit Lengths via
Generative Modeling

Nurettin Turan, Benedikt Fesl, and Wolfgang Utschick
The authors acknowledge the financial support by the Federal Ministry of
Education and Research of Germany in the program of “Souverän. Digital.
Vernetzt.”. Joint project 6G-life, project identification number:
16KISK002.©This work has been submitted to the IEEE for possible publication.
Copyright may be transferred without notice, after which this version may no
longer be accessible. TUM School of Computation, Information and Technology,
Technical University of Munich, Germany
Email: {nurettin.turan, benedikt.fesl<EMAIL_ADDRESS>
###### Abstract

Recently, a versatile limited feedback scheme based on a Gaussian mixture
model (GMM) was proposed for frequency division duplex (FDD) systems. This
scheme provides high flexibility regarding various system parameters and is
applicable to both point-to-point multiple-input multiple-output (MIMO) and
multi-user MIMO (MU-MIMO) communications. The GMM is learned to cover the
operation of all mobile terminals (MTs) located inside the base station (BS)
cell, and each MT only needs to evaluate its strongest mixture component as
feedback, eliminating the need for channel estimation at the MT. In this work,
we extend the GMM-based feedback scheme to variable feedback lengths by
leveraging a single learned GMM through merging or pruning of dispensable
mixture components. Additionally, the GMM covariances are restricted to
Toeplitz or circulant structure through model-based insights. These extensions
significantly reduce the offloading amount and enhance the clustering ability
of the GMM which, in turn, leads to an improved system performance. Simulation
results for both point-to-point and multi-user systems demonstrate the
effectiveness of the proposed extensions.

###### Index Terms:

Gaussian mixture models, machine learning, limited feedback, precoding,
frequency division duplexing.

##  I Introduction

In the next generation of cellular systems (6G), the BS has the ability to
adjust to changing channel conditions. However, in FDD systems, this
adaptation must rely on feedback from the MT since channel reciprocity is not
maintained [1]. There is considerable interest in systems that use limited
feedback, where only a few bits are designated [1]. In this context, two main
strategies can be distinguished. The first involves estimating the downlink
(DL) channel at the MTs and determining the feedback based on this [1, 2, 3].
The second approach aims to directly encode feedback from pilot observations,
such as through deep learning, cf., e.g., [4, 5].

In recent works, a versatile GMM-based limited feedback scheme was proposed
which provides flexibility with respect to the number of antennas, the
transmission mode, the number of MTs, the supported signal-to-noise ratio
(SNR) range, the number of pilots, and the choice of the precoding algorithm,
together with low complexity to determine the feedback by circumventing the
necessity for channel estimation at the MTs [6, 7]. The MTs, select the index
of the GMM component with the highest responsibility (posterior probability)
for their received pilot signal as their feedback information. The
corresponding GMM is learned from data which represent the underlying channel
distribution of a whole communication scenario inside a BS cell. This is
motivated by the universal approximation property [8] and the strong results
of GMMs in wireless communications [9, 10, 11, 12]. A main advantage is that
the offline fitting of the GMM can be done centralized at the BS due to the
absence of a distributional shift between the channel distributions of uplink
(UL) and DL, cf. [13, 14, 15].

Although GMMs possess the universal approximation property, cf. [8], a known
trait is that their corresponding mixture components may not be distinct
enough from each other to be interpreted as clusters [16]. This similarly also
arises in the GMM-based feedback scheme from [7], where the number $K$ of
mixture components is predetermined by the number $B$ of feedback bits, i.e.,
$K=2^{B}$. Therefore, the GMM should exhibit both a strong clustering ability
for selecting the feedback index and a good likelihood model for the
underlying channel distribution of the communication scenario, while
maintaining a fixed number of components. To address this problem, various
merging and pruning techniques have been proposed in the literature [17, 16].
Furthermore, structural features of the covariance matrices, imposed by the
antenna arrays, can be utilized in order to reduce the number of parameters of
the GMM which, in turn, leads to a higher robustness against overfitting for a
limited amount of training data [11].

_Contributions:_ This work extends the versatile GMM-based feedback scheme
from [6, 7] by leveraging a single learned GMM to variable feedback bit
lengths through merging or pruning of dispensable mixture components. Thereby,
a simple pruning technique and a merging approach—which combines components
with high similarity—is analyzed. Additionally, we investigate the restriction
of the GMM to Toeplitz- and circulant-structured covariances which drastically
reduces the necessary offloading overhead from the BS to the MTs and prevents
overfitting in the case of limited training data. Simulation results show that
the reduction of GMM components through merging or pruning enables variable
bit lengths and is superior to directly fitting a smaller GMM because of the
enhanced clustering ability. Furthermore, the structured GMM variants yield
great performances with low memory overhead for both point-to-point and MU-
MIMO systems.

##  II System and Channel Models

###  II-A Data Transmission Phase – Point-to-Point MIMO System

In a point-to-point MIMO system the DL received signal is
$\bm{y}^{\prime}=\bm{H}\bm{x}+\bm{n}^{\prime}$, where
$\bm{y}^{\prime}\in\mathbb{C}^{N_{\mathrm{rx}}}$ is the receive vector,
$\bm{x}=\bm{Q}^{1/2}\bm{s}\in\mathbb{C}^{N_{\mathrm{tx}}}$ with
$\operatorname{E}[\bm{s}\bm{s}^{\operatorname{H}}]=\mathbf{I}_{N_{\mathrm{tx}}}$
is the precoded transmit vector, $\bm{H}\in\mathbb{C}^{N_{\mathrm{rx}}\times
N_{\mathrm{tx}}}$ is the MIMO channel with $N_{\mathrm{rx}}<N_{\mathrm{tx}}$,
and
$\bm{n}^{\prime}\sim\mathcal{N}_{\mathbb{C}}(\mathbf{0},\sigma_{n}^{2}\mathbf{I}_{N_{\mathrm{rx}}})$
denotes the additive white Gaussian noise (AWGN). In a limited feedback
system, $B$ bits are used for encoding the feedback index
$k^{\star}\in\\{1,2,\cdots,2^{B}\\}$ that specifies an element from a codebook
of $2^{B}$ pre-computed transmit covariance matrices
$\mathcal{Q}=\\{\bm{Q}_{1},\bm{Q}_{2},\dots,\bm{Q}_{2^{B}}\\}$ [1].

###  II-B Data Transmission Phase – Multi-user MIMO System

We utilize linear precoding in a single-cell MU-MIMO DL system. The BS is
equipped with $N_{\mathrm{tx}}$ transmit antennas and each MT
$j\in\mathcal{J}=\\{1,2,\dots,J\\}$, is equipped with $N_{\mathrm{rx}}$
antennas. The precoded DL data vector is
$\bm{x}=\sum_{j=1}^{J}\bm{M}_{j}\bm{s}_{j}$, where
$\bm{s}_{j}\in\mathbb{C}^{d_{j}}$ is the transmit signal of MT $j$, with
$\operatorname{E}[\bm{s}_{j}]=\mathbf{0}$ and
$\operatorname{E}[\bm{s}_{j}\bm{s}_{j}^{\operatorname{H}}]=\mathbf{I}_{d_{j}}$,
and $\bm{M}_{j}\in\mathbb{C}^{N_{\mathrm{tx}}\times d_{j}}$ is the precoding
matrix of MT $j$. The precoders fulfill
$\operatorname{tr}(\sum_{j=1}^{J}\bm{M}_{j}\bm{M}_{j}^{\operatorname{H}})=\rho$.
In the multi-user setup, each MT reports its own feedback information
$k^{\star}_{j}$ to the BS, which then jointly designs the precoders
$\bm{M}_{j}$.

###  II-C Pilot Transmission Phase

During the pilot transmission phase, every MT indexed as $j\in\mathcal{J}$
receives

$\bm{Y}_{j}=\bm{H}_{j}\bm{P}+\bm{N}_{j}\in\mathbb{C}^{N_{\mathrm{rx}}\times
n_{p}}$ (1)

where
$\bm{N}_{j}=[\bm{n}^{\prime}_{j,1},\dots,\bm{n}^{\prime}_{j,n_{p}}]\in\mathbb{C}^{N_{\mathrm{rx}}\times
n_{p}}$ with
$\bm{n}^{\prime}_{j,p}\sim\mathcal{N}_{\mathbb{C}}(\bm{0},\sigma_{j}^{2}\mathbf{I}_{N_{\mathrm{rx}}})$,
for $p\in\\{1,2,\dots,n_{p}\\}$ and $n_{p}$ is the number of pilots, where we
consider $n_{p}\leq N_{\mathrm{tx}}$. It is assumed that the BS employs a
uniform rectangular array (URA). Hence, we utilize a $2$D-DFT sub-matrix as
the pilot matrix, cf., e.g., [7]. To maintain the power constraint each column
$\bm{p}_{p}$ of $\bm{P}$ is normalized: $\|\bm{p}_{p}\|^{2}=\rho$. For the
subsequent analysis, it is advantageous to vectorize (1):
$\bm{y}_{j}=\bm{A}\bm{h}_{j}+\bm{n}_{j}$, where
$\bm{h}_{j}=\operatorname{vec}(\bm{H}_{j})$,
$\bm{y}_{j}=\operatorname{vec}(\bm{Y}_{j})$,
$\bm{n}_{j}=\operatorname{vec}(\bm{N}_{j})$,
$\bm{A}=\bm{P}^{\operatorname{T}}\otimes\mathbf{I}_{N_{\mathrm{rx}}}$ and
$\bm{n}_{j}\sim\mathcal{N}_{\mathbb{C}}(\mathbf{0},{\bm{\Sigma}})$ with
${\bm{\Sigma}}=\sigma_{n}^{2}\mathbf{I}_{N_{\mathrm{rx}}n_{p}}$. In case of a
point-to-point MIMO system, we simplify notation for convenience by omitting
the index $j$, resulting in

$\bm{y}=\bm{A}\bm{h}+\bm{n}\in\mathbb{C}^{N_{\mathrm{rx}}n_{p}}.$ (2)

To simplify notation further, we will use the channel matrix $\bm{H}$ and its
vectorized form $\bm{h}$ interchangeably.

###  II-D Channel Model and Data Generation

The QuaDRiGa channel simulator [18] is used to create a training dataset

$\mathcal{H}=\\{\bm{h}_{\ell}=\operatorname{vec}{(\bm{H}_{\ell})}\\}_{\ell=1}^{L}$
(3)

consisting of $L$ channel realizations for the above system model. Since we
utilize the same simulation setup, we refer the reader to [7] for more
details.

##  III GMM-based Limited Feedback Scheme

###  III-A GMM Preliminaries

A GMM is a probability density function (PDF) of the form

$f^{(K)}_{\bm{h}}(\bm{h})=\sum\nolimits_{k=1}^{K}\pi_{k}\mathcal{N}_{\mathbb{C}}(\bm{h};{\bm{\mu}}_{k},\bm{C}_{k})$
(4)

where each term within the summation represents one of its $K$ constituent
components. The maximum likelihood estimates for the GMM’s parameters, which
include the mixing coefficients $\pi_{k}$, means ${\bm{\mu}}_{k}$, and
covariances $\bm{C}_{k}$, can be calculated by utilizing a training dataset
$\mathcal{H}$ as described in (3), along with an expectation maximization (EM)
algorithm outlined in [19, Subsec. 9.2.2]. GMMs enable the computation of
responsibilities [19, Sec. 9.2],

$p(k\mid\bm{h})=\frac{\pi_{k}\mathcal{N}_{\mathbb{C}}(\bm{h};{\bm{\mu}}_{k},\bm{C}_{k})}{\sum_{i=1}^{K}\pi_{i}\mathcal{N}_{\mathbb{C}}(\bm{h};{\bm{\mu}}_{i},\bm{C}_{i})}$
(5)

which represent the posterior probability that a particular $\bm{h}$
originates from component $k$. The inherent joint Gaussianity of each GMM
component and the AWGN enables the simple computation of the GMM of the
observations by utilizing the GMM from (4) as

$f_{\bm{y}}^{(K)}(\bm{y})=\sum\nolimits_{k=1}^{K}\pi_{k}\mathcal{N}_{\mathbb{C}}(\bm{y};\bm{A}{\bm{\mu}}_{k},\bm{A}\bm{C}_{k}\bm{A}^{\operatorname{H}}+{\bm{\Sigma}}).$
(6)

Analogously, we can compute:

$p(k\mid\bm{y})=\frac{\pi_{k}\mathcal{N}_{\mathbb{C}}(\bm{y};\bm{A}{\bm{\mu}}_{k},\bm{A}\bm{C}_{k}\bm{A}^{\operatorname{H}}+{\bm{\Sigma}})}{\sum_{i=1}^{K}\pi_{i}\mathcal{N}_{\mathbb{C}}(\bm{y};\bm{A}{\bm{\mu}}_{i},\bm{A}\bm{C}_{i}\bm{A}^{\operatorname{H}}+{\bm{\Sigma}})}.$
(7)

###  III-B Point-to-Point MIMO System

In an offline phase, firstly, a codebook
$\mathcal{Q}=\\{\bm{Q}_{k}\\}_{k=1}^{K}$, with $K=2^{B}$, is constructed based
on the GMM $f_{\bm{h}}^{(K)}$ from (4) that is learned from the dataset (3).
Therefore, by utilizing the GMM, the training data are clustered according to
their GMM responsibilities. That is, $\mathcal{H}$ is partitioned into $K$
disjoint sets

$\mathcal{V}_{k}=\\{\bm{h}\in\mathcal{H}\mid p(k\mid\bm{h})\geq
p(j\mid\bm{h})\text{ for }k\neq j\\}$ (8)

for $k=1,\dots,K$. Then each codebook entry is determined:

$\displaystyle\bm{Q}_{k}=\operatorname*{arg\,max}_{\bm{Q}\succeq{\bm{0}}}\tfrac{1}{|\mathcal{V}_{k}|}\sum_{\operatorname{vec}(\bm{H})\in\mathcal{V}_{k}}r(\bm{H},\bm{Q})$
(9) $\displaystyle\text{subject to}\quad\operatorname{tr}(\bm{Q})\leq\rho$

where the spectral efficiency is

$r(\bm{H},\bm{Q})=\log_{2}\det\left(\mathbf{I}+\tfrac{1}{\sigma_{n}^{2}}\bm{H}\bm{Q}\bm{H}^{\operatorname{H}}\right).$
(10)

A projected gradient ascent algorithm is utilized to solve this optimization
problem, cf. [7]. In the online phase, explicit channel estimation is
circumvented and the pilot observation $\bm{y}$ is used to determine the
feedback index via:

$k^{\star}=\operatorname*{arg\,max}_{k}{~{}p(k\mid\bm{y})}.$ (11)

The computational complexity for determining the feedback via (11) by using
the GMM is $\mathcal{O}(KN_{\mathrm{rx}}^{2}n_{p}^{2})$, cf. [7]. Thus, the
complexity is not affected by $N_{\mathrm{tx}}$, which is particularly
beneficial for massive MIMO systems. It’s worth noting that parallelization is
possible, given that the evaluation of all $K$ responsibilities can be
conducted independently. The MT solely needs the GMM parameters for
calculating (11) and does not require any codebook knowledge. Furthermore, the
GMM of the observations in (6) can be easily customized at the MT to
accommodate various SNR levels and pilot configurations, by simply updating
the means and covariances, cf. (6), without the need for retraining. As a
baseline for performance analysis, perfect channel state information (CSI) can
be used to determine the feedback

$k^{\star}=\operatorname*{arg\,max}_{k}~{}{p(k\mid\bm{h})}.$ (12)

###  III-C Multi-User MIMO System

In [7] it was shown that directional information associated with each codebook
entry can be obtained by conducting a singular value decomposition (SVD),
i.e., $\bm{Q}_{k}=\bm{X}_{k}\bm{T}_{k}\bm{X}_{k}^{\operatorname{H}}$. Thereby,
the matrix $\bm{T}_{k}$ arranges the singular values in descending order,
while the matrix $\bar{\bm{X}}_{k}$ encompasses the first $N_{\mathrm{rx}}$
vectors of $\bm{X}_{k}$ as the relevant directional information. Consequently,
the set
$\mathcal{Q}=\\{\bar{\bm{X}}_{1},\bar{\bm{X}}_{2},\cdots,\bar{\bm{X}}_{K}\\}$
forms a directional codebook, cf. [7]. Utilizing the GMM-based approach each
MT efficiently computes its feedback by:

$k^{\star}_{j}=\operatorname*{arg\,max}_{k}~{}{p(k\mid\bm{y}_{j})}.$ (13)

Every MT transmits the index $k^{\star}_{j}$ to the BS, which subsequently
represents each MT’s channel with the subspace information linked with the
corresponding codebook entry
$\widetilde{\bm{H}}_{j}=\bar{\bm{X}}^{\operatorname{H}}_{k^{\star}_{j}}$. The
BS can then utilize widely-used precoding techniques, such as the iterative
weighted minimum mean square error (WMMSE) [20], in order to jointly design
the precoders, cf. [7].

Alternatively, an approach based on generative modeling was also introduced in
[7]. In this approach, the channel matrix of each MT is treated as a random
variable, and the precoders are computed using the stochastic WMMSE (SWMMSE)
algorithm [21]. The GMM-based approach is able to generate samples that adhere
to the channel’s distribution due to the inherent sample generation capability
of GMMs. Specifically, given the feedback $k_{j}^{\star}$ of each MT, see
(13), one can generate samples through
$\bm{h}_{j,\text{sample}}\sim\mathcal{N}_{\mathbb{C}}({\bm{\mu}}_{k^{\star}_{j}},\bm{C}_{k^{\star}_{j}})$,
which conveys statistical information about the channel of MT $j$. By applying
the SWMMSE algorithm, the BS can compute the precoders by utilizing the
generated samples, cf. [7].

##  IV Enabling Variable Bit Lengths

In this Section, we discuss how the GMM-based feedback scheme can be enabled
to utilize variable bit lengths by only requiring a single GMM of a certain
size, of which GMMs with a smaller number of components can be obtained. We
investigate two different mixture reduction techniques, where given a GMM with
$K=2^{B}$ components, another GMM with fewer components $K_{S}=2^{B_{S}}$,
with, $K_{S}<K$, can be obtained.

There exist many methods to obtain a GMM with fewer components starting from a
GMM with more components. In this work, we analyze two types of mixture
reduction techniques. In particular, we use one _merging_ strategy which
successively combines pairs of components with high similarity [17] and one
_pruning_ strategy, which simply discards components with a low contribution
to the overall mixture [22].

In this work, we focus our analysis on the merging technique from [17].
Thereby, one replaces two mixture components with the smallest dissimilarity
denoted by $\\{\pi_{a},{\bm{\mu}}_{a},\bm{C}_{a}\\}$ and
$\\{\pi_{b},{\bm{\mu}}_{b},\bm{C}_{b}\\}$, by their moment preserving merge
$\\{\pi_{m},{\bm{\mu}}_{m},\bm{C}_{m}\\}$ [17]:

$\displaystyle\pi_{m}$ $\displaystyle=\pi_{a}+\pi_{b}$
$\displaystyle{\bm{\mu}}_{m}$
$\displaystyle=\tfrac{1}{\pi_{m}}(\pi_{a}{\bm{\mu}}_{a}+\pi_{b}{\bm{\mu}}_{b})$
(14) $\displaystyle\bm{C}_{m}$
$\displaystyle=(\tfrac{\pi_{a}}{\pi_{m}}\bm{C}_{a}+\tfrac{\pi_{b}}{\pi_{m}}\bm{C}_{b}+\tfrac{\pi_{a}\pi_{b}}{\pi_{m}^{2}}({\bm{\mu}}_{a}-{\bm{\mu}}_{b})({\bm{\mu}}_{a}-{\bm{\mu}}_{b})^{\operatorname{H}})$

The dissimilarity between two components is measured by

$d_{a,b}=\tfrac{1}{2}[\pi_{m}\log\det(\bm{C}_{m})\\\
-\pi_{a}\log\det(\bm{C}_{a})-\pi_{b}\log\det(\bm{C}_{b})]$ (15)

which is an upper bound on the Kullback-Leibler divergence between the GMMs
before and after the merge, cf. [17]. The merging procedure is successively
repeated until a GMM of the desired size is obtained, i.e., when the number of
GMM components is reduced to $2^{B_{S}}$. After a GMM of the desired size is
obtained, the corresponding codebook needs to be constructed at the BS in the
same way as described in Section III-B. Algorithm 1 summarizes the merging
procedure and the accompanying codebook construction.

Algorithm 1 Variable Bit Lengths Enabled Through Merging.

0: GMM with $K=2^{B}$ components. Desired number of components
$K_{S}=2^{B_{S}}$ with $K_{S}<K$.

1: $i=K$ {track number of GMM components}

2: repeat

3: Calc. moment preserving merge of all pairs of components using (14).

4: Find pair with smallest dissimilarity $d_{a,b}$, via (15).

5: Replace this pair by their moment preserving merge.

6: $i\leftarrow i-1$

7: until $i=K_{S}$

8: Partition $\mathcal{H}$ into $K_{S}$ disjoint sets $\mathcal{V}_{k}$ with
$k=1,\dots,K_{S}$ according to the GMM responsibilities, cf. (8).

9: Compute codebook $\mathcal{Q}=\\{\bm{Q}_{k}\\}_{k=1}^{K_{S}}$ by solving
(9) for each entry.

Algorithm 2 Variable Bit Lengths Enabled Through Pruning.

0: GMM with $K=2^{B}$ components. Desired number of components
$K_{S}=2^{B_{S}}$ with $K_{S}<K$.

1: Remove the $K-K_{S}$ components with smallest mixing coefficients
$\pi_{k}$.

2: Re-normalize remaining mixing coefficients to one.

3: Discard codebook entries correspondingly and obtain
$\mathcal{Q}=\\{\bm{Q}_{k}\\}_{k=1}^{K_{S}}$.

The pruning strategy simply removes $K-K_{S}$ components with the smallest
mixing coefficient $\pi_{k}$ (see (4)), and the mixing coefficients of the
remaining components are re-normalized to one [22]. A main advantage of the
pruning strategy is its low computational complexity. In particular, it only
requires traversing a list of sorted mixing coefficients. In this case, an
updated codebook $\mathcal{Q}=\\{\bm{Q}_{k}\\}_{k=1}^{K_{S}}$ can simply be
obtained by discarding the respective codebook entries which correspond to the
removed GMM components. Algorithm 2 summarizes the pruning procedure.

##  V Reducing the Offloading Overhead

Name | Covariance Parameters | Example
---|---|---
Full | $\frac{1}{2}KN(N+1)$ | $8.4\cdot 10^{6}$
Kronecker |  $\frac{1}{2}K_{\mathrm{rx}}N_{\mathrm{rx}}(N_{\mathrm{rx}}+1)$ $+\frac{1}{2}K_{\mathrm{tx}}N_{\mathrm{tx}}(N_{\mathrm{tx}}+1)$ | $9.0\cdot 10^{3}$
Toeplitz |  $2K_{\mathrm{rx}}N_{\mathrm{rx}}$ $+4K_{\mathrm{tx}}N_{\mathrm{tx}}$ | $2.1\cdot 10^{3}$
Circulant |  $K_{\mathrm{rx}}N_{\mathrm{rx}}$ $+K_{\mathrm{tx}}N_{\mathrm{tx}}$ | $5.7\cdot 10^{2}$
TABLE I: Analysis of the number of covariance parameters of the (structured)
GMM.

To enable a MT to compute feedback indices via (11), the parameters of the GMM
need to be offloaded to the MT upon entering the BS’ coverage area, cf. [7].
To reduce the offloading overhead model-based insights can be utilized to
obtain structured covariances with fewer parameters. For example, one can
constrain the GMM covariances to a Kronecker factorization of the form
$\bm{C}_{k}=\bm{C}_{\text{tx},k}\otimes\bm{C}_{\text{rx},k}$ which has no
notable impact on the performance, cf. [9, 7]. In this context, instead of
fitting an unconstrained GMM with $N\times N$-dimensional covariances, a GMM
specific to the transmit side and another for the receive side is fitted.
These transmit-side and receive-side GMMs have $N_{\text{tx}}\times
N_{\text{tx}}$ and $N_{\text{rx}}\times N_{\text{rx}}$-dimensional covariances
respectively, with $K_{\text{tx}}$ and $K_{\text{rx}}$ components.
Subsequently, by computing all Kronecker products of the corresponding
transmit-side and receive-side covariance matrices, a
$K=K_{\mathrm{tx}}K_{\mathrm{rx}}$-components GMM with $N\times N$-dimensional
covariances is obtained.

In this work, we investigate the incorporation of further structural features
to the transmit- and receive-side GMM covariance matrices imposed by the
antenna structure at the transmitter or the receiver. In case of a uniform
linear array (ULA), it is common to assume a Toeplitz covariance matrix, which
for a large number of antenna elements, is well approximated by a circulant
matrix, cf., e.g., [12]. If a URA is deployed, the structural assumptions
result in block-Toeplitz matrices with Toeplitz blocks, or block-circulant
matrices with circulant blocks, respectively [12].

In the assumed case of a URA employed at the BS with $N_{\mathrm{tx,v}}$
vertical and $N_{\mathrm{tx,h}}$ horizontal
($N_{\mathrm{tx}}=N_{\mathrm{tx,v}}N_{\mathrm{tx,h}}$) elements, the
structured covariances can be expressed as
$\bm{C}_{\text{tx},k}=\bm{D}^{\operatorname{H}}\operatorname{diag}(\bm{c}_{\text{tx},k})\bm{D}$,
where on the one hand, when assuming a Toeplitz structure,
$\bm{D}=\bm{D}_{N_{\mathrm{tx,v}}}\otimes\bm{D}_{N_{\mathrm{tx,h}}}$, where
$\bm{D}_{T}$ contains the first $T$ columns of a $2T\times 2T$ discrete
Fourier transform (DFT) matrix, and
$\bm{c}_{\text{tx},k}\in\mathbb{R}_{+}^{4N_{\mathrm{tx}}}$ [11, 12]. On the
other hand, when assuming circular structure, we have
$\bm{D}=\bm{F}_{N_{\mathrm{tx,v}}}\otimes\bm{F}_{N_{\mathrm{tx,h}}}$, where
$\bm{F}_{T}$ is the $T\times T$ DFT-matrix, and
$\bm{c}_{\text{tx},k}\in\mathbb{R}_{+}^{N_{\mathrm{tx}}}$. At the MTs, we
assume to have ULAs, thus $\bm{D}$ contains either the first $N_{\mathrm{rx}}$
columns of a $2N_{\mathrm{rx}}\times 2N_{\mathrm{rx}}$ DFT matrix and
correspondingly $\bm{c}_{\text{rx},k}\in\mathbb{R}_{+}^{2N_{\mathrm{rx}}}$
(Toeplitz), or $\bm{D}=\bm{F}_{N_{\mathrm{rx}}}$ and
$\bm{c}_{\text{rx},k}\in\mathbb{R}_{+}^{N_{\mathrm{rx}}}$ (circulant). With
these structural constraints the GMM covariances are fully determined by the
vectors $\bm{c}_{\\{\text{tx,rx}\\},k}$. Overall, these insights significantly
reduce the offloading overhead, enable a reduced complexity in offline
training, facilitate parallelization of the fitting process, and require fewer
training samples since fewer parameters need to be learned, cf. [11, 12].

Table I depicts the number of (structured) GMM covariance parameters
(accounting for symmetries). We use exemplarily the simulation parameters of a
configuration with $B=6$, $(N_{\mathrm{tx}},N_{\mathrm{rx}})=(32,16)$, and
$(K_{\mathrm{tx}},K_{\mathrm{rx}})=(16,4)$, which we consider in Section VII.
We can observe that, structural constraints significantly reduce the
offloading overhead.

##  VI Discussion on the Enhanced Versatility

Choose structural features{full, Kronecker, Toeplitz, circulant}merge/prune
GMMFit GMM andconstruct codebook $\mathcal{Q}$Choose number of bitsCalculate
feedback index:
$k^{*}_{j}=\operatorname*{arg\,max}\limits_{k}p(k\mid\bm{y}_{j})$Offload
GMMSend pilots $\bm{P}$MU or P2P?Feed back $k_{j}^{*}$Choose
$\bm{Q}_{k_{j}^{*}}\in\mathcal{Q}$P2PCompute pre-coders $\bm{M}_{j}$MUData
transmission

Figure 1: Flowchart of the improved versatile feedback scheme. Red (blue)
colored nodes are processed at the BS (MTs) and solid (dashed) arrows indicate
online (offline) processing.

The low-complexity GMM-based feedback scheme from [7] exhibits great
flexibility with respect to the transmission mode, i.e., either point-to-point
MIMO or MU-MIMO, the supported range of SNR values, the pilot configuration,
and the selection of the precoding algorithm, but utilizes a GMM with a
predefined number of components $K=2^{B}$. In principle, the BS could offload
many GMMs with different sizes, which would allow for different numbers of
feedback bits. However, the corresponding signaling overhead might be
unaffordable in practice. Therefore, in Section IV we discussed different
mixture reduction techniques and their corresponding codebook update
procedures. Particularly, given a GMM with $K=2^{B}$ components, GMMs with
fewer components $K_{S}=2^{B_{S}}$, i.e, $K_{S}<K$, can be obtained.
Accordingly, the scheme is enabled to support variable bit lengths by
successively decreasing the number of components and thereby the number of
feedback bits starting from $B$ bits ($B-1,B-2,\cdots$). Due to the simplicity
of the pruning strategy, it can be straightforwardly implemented at the MTs.
The merging strategy can be either conducted at the MTs, or to reduce their
computational burden, the BS could offload additional lists with the
respective merging updates. Moreover, the MT only requires the GMM and does
not need to be aware of the codebook in order to compute the feedback via
(11). Altogether, this allows for variable feedback bit lengths and eliminates
the necessity of offloading a particularly trained GMM for different feedback
bit lengths and thereby improves the versatility of the feedback scheme.
Additionally, in Section V we discussed how model-based insights can help to
effectively reduce the offloading overhead. Fig. 1 provides a flowchart of the
enhanced GMM-based feedback scheme.

##  VII Simulation Results

We generate datasets for the UL and DL domain of the scenario outlined in
Section II-D: $\mathcal{H}^{\text{UL}}$ and $\mathcal{H}^{\text{DL}}$. The GMM
is fitted centrally at the BS using the training set
$\mathcal{H}=\mathcal{H}^{\text{UL}}$ which consists of $L=20\cdot 10^{3}$
samples, cf., [7, 5, 15, 14, 13]. The following transmit strategies are always
evaluated in the DL domain using $\mathcal{H}^{\text{DL}}$, comprised of
$10^{4}$ channels. The data samples are normalized to satisfy
$\operatorname{E}[\|\bm{h}\|^{2}]=N=N_{\mathrm{tx}}N_{\mathrm{rx}}$.
Additionally, we fix $\rho=1$, enabling the definition of the SNR as
$\frac{1}{\sigma_{n}^{2}}$ for all MTs, specifically when
$\sigma^{2}_{j}=\sigma^{2}_{n},\forall j\in\mathcal{J}$.

In the single-user case, we will compare to codebook based approaches
utilizing Lloyd’s clustering algorithm, cf. [3, 7]. In case of MU-MIMO
systems, we will either use directional information extracted from the Lloyd
codebooks or we will use the random codebook approach, cf. [7, 2]. With these
approaches, prior to codebook entry selection, the channel needs to be
estimated. To this end, we consider three different channel estimators, which
are briefly explained in the following. The recently proposed GMM-based
channel estimator $\hat{\bm{h}}_{\text{GMM}}$, from [9] is one of them and
utilizes the same GMM as found in Section III and calculates a convex
combination of per-component linear minimum mean square error (LMMSE)
estimates, cf. [7, 9]. This estimator is proven to asymptotically converge to
the optimal conditional mean estimator as the number of components $K$ is
increased, cf. [9]. Another baseline is the LMMSE estimator
$\hat{\bm{h}}_{\text{LMMSE}}$, where the sample covariance matrix is
constructed given the set $\mathcal{H}$, cf. [7, 15, 9]. Lastly, we consider a
compressive sensing estimation approach $\hat{\bm{h}}_{\text{OMP}}$ utilizing
the orthogonal matching pursuit (OMP), cf. [7, 23].

$0.7$$0.75$$0.8$$0.85$$0.9$$0.95$$1$$0$$0.2$$0.4$$0.6$$0.8$$1$normalized
spectral efficiency, $s$$P(\text{nSE}>s)$kGMM, $B$=$6$mkGMM,
$B_{S_{1}}$=$6$pkGMM, $B_{S_{1}}$=$6$kGMM, $B$=$3$mkGMM, $B_{S_{2}}$=$3$pkGMM,
$B_{S_{2}}$=$3$kGMM, 16x4, $B$=$3$mkGMM, 16x4, $B_{S}$=$3$pkGMM, 16x4,
$B_{S}$=$3$ Figure 2: Empirical cCDFs of the normalized spectral efficiencies
for two setups using either mixture reduction techniques or the direct fitting
approach assuming perfect CSI. Setup A: $N_{\mathrm{tx}}=32$
($N_{\mathrm{tx,h}}=8,N_{\mathrm{tx,v}}=4$), $N_{\mathrm{rx}}=16$, and
$\text{SNR}=$10\text{\,}\mathrm{d}\mathrm{B}$$. Setup B: $N_{\mathrm{tx}}=16$
($N_{\mathrm{tx,h}}=4,N_{\mathrm{tx,v}}=4$), $N_{\mathrm{rx}}=4$, and
$\text{SNR}=$0\text{\,}\mathrm{d}\mathrm{B}$$.

###  VII-A Point-to-point MIMO

As performance measure, we utilize the normalized spectral efficiency (nSE),
where the spectral efficiency achieved with a certain transmit strategy is
normalized by the optimal water-filling solution, cf. [1]. The empirical cCDF
$P(\text{nSE}>s)$ for the normalized spectral efficiency represents the
empirical probability that the nSE surpasses a given value $s$.

$0.7$$0.75$$0.8$$0.85$$0.9$$0.95$$1$$0$$0.2$$0.4$$0.6$$0.8$$1$normalized
spectral efficiency, $s$$P(\text{nSE}>s)$kGMMtGMMcGMMkGMM, 16x4tGMM, 16x4cGMM,
16x4 Figure 3: Empirical cCDFs of the normalized spectral efficiencies for two
setups using differently structured GMMs assuming perfect CSI. Setup A:
$N_{\mathrm{tx}}=32$ ($N_{\mathrm{tx,h}}=8,N_{\mathrm{tx,v}}=4$),
$N_{\mathrm{rx}}=16$, and $\text{SNR}=$10\text{\,}\mathrm{d}\mathrm{B}$$, with
$B=6$. Setup B: $N_{\mathrm{tx}}=16$
($N_{\mathrm{tx,h}}=4,N_{\mathrm{tx,v}}=4$), $N_{\mathrm{rx}}=4$, and
$\text{SNR}=$0\text{\,}\mathrm{d}\mathrm{B}$$, with $B=3$.

In Fig. 2, we denote by “Setup A” a system with $N_{\mathrm{tx}}=32$
$(N_{\mathrm{tx,h}}=8,N_{\mathrm{tx,v}}=4)$, $N_{\mathrm{rx}}=16$, and
$\text{SNR}=$10\text{\,}\mathrm{d}\mathrm{B}$$, and with “Setup B” a system
with $N_{\mathrm{tx}}=16$ $(N_{\mathrm{tx,h}}=4,N_{\mathrm{tx,v}}=4)$,
$N_{\mathrm{rx}}=4$, and $\text{SNR}=$0\text{\,}\mathrm{d}\mathrm{B}$$. In the
following, we consider the Kronecker GMM (“kGMM”) with full transmit- and
receive-side GMMs and assume perfect CSI, i.e., the feedback information is
determined via (12). For “Setup A” we fit a GMM with $K=256$ $(B=8$,
$K_{\mathrm{tx}}=32$, and $K_{\mathrm{rx}}=8)$ components and first reduce to
a GMM with $K_{S_{1}}=64$ $(B_{S_{1}}=6)$ components and continue the
reduction to another one with $K_{S_{2}}=8$ $(B_{S_{2}}=3)$ components, by
either applying the merging strategy (“mkGMM”) or the pruning strategy
(“pkGMM”). Moreover, with “kGMM” we denote the case of directly fitting a GMM
with $64$ $(B=6$, $K_{\mathrm{tx}}=16$, and $K_{\mathrm{rx}}=4)$, or with $8$
$(B=3$, $K_{\mathrm{tx}}=4$, and $K_{\mathrm{rx}}=2)$ components. We can
observe, that both mixture reduction approaches not only enable the feedback
scheme with variable bit lengths, but at the same time provide at least a
similar performance as the respective direct fitting approaches. In case of
“Setup B”, we fit a GMM with $K=256$ components and reduce to one with
$K_{S}=8$ $(B_{S}=3)$ components and compare to a GMM directly fitted with $8$
$(K_{\mathrm{tx}}=4$, and $K_{\mathrm{rx}}=2)$ components. In this setup, the
merging strategy performs best, whereas the low-complexity pruning strategy
yields a slightly worse performance as compared to the direct fitting
approach. Altogether, the applied mixture reduction techniques enable variable
bit lengths and in some cases even improve the performance as a consequence of
the enhanced clustering ability, cf. [16].

In Fig. 3, consider the same two setups and denote by “tGMM” or “cGMM”
Kronecker GMMs constructed by (block) Toeplitz or (block) circulant transmit-
and receive-side GMMs, respectively, with $64$ components $(B=6$,
$K_{\mathrm{tx}}=16$, and $K_{\mathrm{rx}}=4)$ in case of “Setup A”, and $8$
components $(B=3$, $K_{\mathrm{tx}}=4$, and $K_{\mathrm{rx}}=2)$ in case of
“Setup B”. We can observe, that although the Toeplitz approximation (“tGMM”)
drastically reduces the offloading overhead, it achieves the same performance
as the Kronecker GMM with full transmit- and receive-side GMMs (“kGMM”). The
circulant approximation (“cGMM”) with even less offloading overhead, slightly
degrades the performance.

$0.6$$0.65$$0.7$$0.75$$0.8$$0.85$$0.9$$0.95$$1$$0$$0.2$$0.4$$0.6$$0.8$$1$normalized
spectral efficiency, $s$$P(\text{nSE}>s)$kGMM, $\bm{y}$mkGMM, $\bm{y}$pkGMM,
$\bm{y}$Lloyd, $\hat{\bm{h}}_{\text{OMP}}$Lloyd,
$\hat{\bm{h}}_{\text{LMMSE}}$Lloyd, $\hat{\bm{h}}_{\text{GMM}}$ Figure 4:
Empirical cCDFs of the normalized spectral efficiencies using either mixture
reduction techniques $(B=8$ to $B_{S}=6)$ or the direct fitting approach
$(B=6)$ for a system with $N_{\mathrm{tx}}=32$, $N_{\mathrm{rx}}=16$,
$\text{SNR}=$15\text{\,}\mathrm{d}\mathrm{B}$$, and $n_{p}=4$ pilots.
$0.6$$0.65$$0.7$$0.75$$0.8$$0.85$$0.9$$0.95$$1$$0$$0.2$$0.4$$0.6$$0.8$$1$normalized
spectral efficiency, $s$$P(\text{nSE}>s)$kGMM, $\bm{y}$, $n_{p}$=$4$tGMM,
$\bm{y}$, $n_{p}$=$4$cGMM, $\bm{y}$, $n_{p}$=$4$Lloyd,
$\hat{\bm{h}}_{\text{GMM}}$, $n_{p}$=$4$kGMM, $\bm{y}$, $n_{p}$=$8$tGMM,
$\bm{y}$, $n_{p}$=$8$cGMM, $\bm{y}$, $n_{p}$=$8$Lloyd,
$\hat{\bm{h}}_{\text{GMM}}$,$n_{p}$=$8$ Figure 5: Empirical cCDFs of the
normalized spectral efficiencies using differently structured GMMs for a
system with $N_{\mathrm{tx}}=32$, $N_{\mathrm{rx}}=16$,
$\text{SNR}=$15\text{\,}\mathrm{d}\mathrm{B}$$, different number of pilots
$n_{p}$, and $B=6$ bits.

Nevertheless, assuming perfect CSI at the MT during the online phase is
impractical. In the subsequent discussions, we consider imperfect CSI, i.e.,
systems characterized by low pilot overhead $(n_{p}\leq N_{\mathrm{tx}})$.
With “Lloyd, {$\hat{\bm{h}}_{\text{GMM}}$,
$\hat{\bm{h}}_{\text{OMP}},\hat{\bm{h}}_{\text{LMMSE}}$}” we depict the
conventional approaches which first estimate the channel and then determine
the feedback information, cf. [7]. In the remainder, the conventional
approaches always utilize the same number of feedback bits of interest, i.e.,
$B_{S}$ bits if merging or pruning is considered, or $B$ bits if not.

In Fig. 4, we simulate a setup with $N_{\mathrm{tx}}=32$
$(N_{\mathrm{tx,h}}=8,N_{\mathrm{tx,v}}=4)$, $N_{\mathrm{rx}}=16$,
$\text{SNR}=$15\text{\,}\mathrm{d}\mathrm{B}$$, and $n_{p}=4$ pilots. We again
consider a GMM with $K=256$ $(B=8$, $K_{\mathrm{tx}}=32$, and
$K_{\mathrm{rx}}=8)$ components and reduce to a GMM with $K_{S}=64$
$(B_{S}=6)$ components, by applying the merging strategy (“mkGMM”) or the
pruning strategy (“pkGMM”). We can observe, that the GMM-based feedback
approach (“kGMM, $\bm{y}$”), which circumvents explicit channel estimation, is
superior as compared to the baselines and provides a higher robustness against
CSI imperfections. The proposed merging and pruning strategies exhibit a
similar robustness.

In Fig. 5, we have $N_{\mathrm{tx}}=32$
$(N_{\mathrm{tx,h}}=8,N_{\mathrm{tx,v}}=4)$, $N_{\mathrm{rx}}=16$,
$\text{SNR}=$15\text{\,}\mathrm{d}\mathrm{B}$$, and $n_{p}\in\\{4,8\\}$ pilots
and depict differently structured GMMs with $K=64$ $(K_{\mathrm{tx}}=16$, and
$K_{\mathrm{rx}}=4)$ components. The Toeplitz approximation (“tGMM”) with a
reduced number of parameters achieves the same performance as compared to the
Kronecker GMM with full transmit- and receive-side GMMs (“kGMM”), irrespective
of the number of pilots. With $n_{p}=4$, the proposed approach “tGMM” attains
the same performance as the conventional Lloyd approach with twice as much
pilots ($n_{p}=8$). This shows, the great potential of the enhanced GMM-based
feedback scheme in systems with reduced pilot overhead. Interestingly, in case
of the circulant approximation (“cGMM”) a performance degradation can be
observed if $n_{p}=4$. It seems that the expressivity of the circulant GMM
with a very low number of parameters is too restrictive to be applied for
systems with a few pilots only. Since the Toeplitz approach provides a higher
robustness, we will restrict our further analysis to it in the remainder.

###  VII-B Multi-user MIMO

For the multi-user scenario, we utilize the sum-rate as performance metric,
cf. [7]. The presented results are based on $2{,}500$ constellations, where
for each constellation, we randomly select $J$ MTs from our evaluation set
$\mathcal{H}^{\text{DL}}$. We employ the empirical cCDF $P(\text{SR}>s)$ of
the sum-rate to illustrate the empirical probability of the sum-rate (SR)
exceeding a given value $s$. Although the GMM-based feedback scheme can be
used in combination with non-iterative precoding algorithms, in the following
we restrict our analysis to the iterative WMMSE and to the SWMMSE in order to
design the precoders, since the performance with these methods is generally
better, cf. [7].

$2$$3$$4$$5$$6$$7$$8$$9$$10$$11$$12$$13$$14$$15$$16$$0$$0.2$$0.4$$0.6$$0.8$$1$Sum-
Rate, $s$, [bps$/$\text{\,}\mathrm{Hz}$$]$P(\text{SR}>s)$kGMM samples,
$\bm{y}$mtGMM samples, $\bm{y}$ptGMM samples, $\bm{y}$kGMM, $\bm{y}$mtGMM,
$\bm{y}$ptGMM, $\bm{y}$Lloyd, $\hat{\bm{h}}_{\text{GMM}}$Random,
$\hat{\bm{h}}_{\text{GMM}}$ Figure 6: Empirical cCDFs of the sum-rate using
either mixture reduction techniques $(B=8$ to $B_{S}=6)$ combined with
Toeplitz structured GMMs or the direct fitting approach $(B=8)$ when the
iterative WMMSE or the SWMMSE are utilized, for a system with
$N_{\mathrm{tx}}=16$, $N_{\mathrm{rx}}=4$, $J=4$ MTs,
$\text{SNR}=$5\text{\,}\mathrm{d}\mathrm{B}$$, and $n_{p}=8$ pilots.

In the subsequent discussion, we refer with “GMM, $\bm{y}$” to the case where
the observations $\bm{y}_{j}$ are used at each MT $j$ to determine a feedback
index utilizing the GMM feedback encoding approach, cf. (13). The channel of
each MT is then represented by the subspace information extracted from the GMM
codebook, cf. Section III-C. For the sake of clarity in the legend, we omit
the index $j$. With “{Lloyd, Random}, {$\hat{\bm{h}}_{\text{GMM}}$,
$\hat{\bm{h}}_{\text{OMP}},\hat{\bm{h}}_{\text{LMMSE}}$}”, we denote the cases
where the channel is estimated at each MT and subsequently the feedback
information is determined using either the directional Lloyd codebook or the
random codebook approach, cf. [7]. These methods use the iterative WMMSE, cf.
[20, Algorithm 1]. Furthermore, with “GMM samples, $\bm{y}$” we refer to the
case, where we generate samples representing the distribution of each MT using
the GMM and feed these samples to the SWMMSE algorithm, cf. Section III-C. The
maximum number of iterations is $I_{\max}=300$ for both approaches.

In the remainder, we combine both, the Toeplitz approximation and the mixture
reduction techniques in a setup with $N_{\mathrm{tx}}=16$
($N_{\mathrm{tx,h}}=4,N_{\mathrm{tx,v}}=4$) and $N_{\mathrm{rx}}=4$, and $J=4$
users. In particular, we fit a Kronecker GMM constructed by (block) Toeplitz
transmit- and receive-side GMMs, with $K=256$ components and reduce to a GMM
with $K_{S}=64$ ($B_{S}=6$) components denoted by “mtGMM”and “ptGMM”, and
compare to a GMM directly fitted with $64$ ($K_{\mathrm{tx}}=16$, and
$K_{\mathrm{rx}}=4$) components with full transmit- and receive-side GMMs
(“kGMM”). In Fig. 6, with $\text{SNR}=$5\text{\,}\mathrm{d}\mathrm{B}$$ and
$n_{p}=8$ pilots, we can see that the pruning and merging strategy in
combination with the Toeplitz structured covariances in case of the
directional (subspace) approach, i.e., “ptGMM, $\bm{y}$” and “mtGMM,
$\bm{y}$”, perform similarly or even better than the “kGMM, $\bm{y}$” and
outperform the conventional methods “{Lloyd, Random},
$\hat{\bm{h}}_{\text{GMM}}$”, by far. With the generative modeling approach,
the merging method “mtGMM samples, $\bm{y}$” performs equally well as “kGMM
samples, $\bm{y}$”, whereas the pruning method “ptGMM samples, $\bm{y}$”
performs slightly worse.

A similar behaviour is present in Fig. 7, where we still consider the same
setting with $n_{p}=8$ pilots, but vary the SNR and depict the sum-rate
averaged over all constellations. Interestingly, with an increasing SNR, the
gap between “mtGMM, $\bm{y}$” and “kGMM, $\bm{y}$” increases. Similarly, the
gap between “ptGMM samples, $\bm{y}$” and “kGMM samples, $\bm{y}$” increases
with larger SNR values. As reported in [7], adopting the generative modeling
approach for joint precoder design is advantageous in scenarios characterized
by low to moderate SNR levels, and for larger SNR values the directional
approach is beneficial. This property can be similarly observed if the merging
or pruning procedures are applied, and is illustrated by the arrows in Fig. 7.
Altogether, the enhanced GMM-based feedback scheme exhibits superior
performance compared to the baselines in a multi-user system with reduced
pilot overhead.

$0$$2$$4$$6$$8$$10$$12$$14$$16$$18$$20$$22$$24$$2$$5$$10$$15$$20$$25$SamplesDirectionsSNR
[$\text{\,}\mathrm{d}\mathrm{B}$]Sum-Rate [bps$/$\text{\,}\mathrm{Hz}$$]kGMM
samples, $\bm{y}$mtGMM samples, $\bm{y}$ptGMM samples, $\bm{y}$kGMM,
$\bm{y}$mtGMM, $\bm{y}$ptGMM, $\bm{y}$Lloyd, $\hat{\bm{h}}_{\text{GMM}}$Lloyd,
$\hat{\bm{h}}_{\text{OMP}}$Lloyd, $\hat{\bm{h}}_{\text{LMMSE}}$Random,
$\hat{\bm{h}}_{\text{GMM}}$ Figure 7: The average sum-rate over the SNR using
either mixture reduction techniques $(B=8$ to $B_{S}=6)$ combined with
Toeplitz structured GMMs or the direct fitting approach $(B=6)$ when the
iterative WMMSE or the SWMMSE are employed, for a system with
$N_{\mathrm{tx}}=16$, $N_{\mathrm{rx}}=4$, $J=4$ MTs, and $n_{p}=8$ pilots.

##  VIII Conclusion

In this work, we enhanced the versatile GMM-based feedback from [7] scheme by
enabling variable bit lengths and incorporating further model-based structural
constraints to reduce the offloading overhead. To this end, we analyzed two
mixture reduction techniques, i.e., one pruning and one merging strategy and
found out, that both approaches enable the scheme with variable bit lengths
and allow for a trade-off between complexity and performance. Moreover,
numerical results showed the remarkable performance of the combination of
mixture reduction techniques with structured GMMs.

## References

  * [1] D. J. Love, R. W. Heath, V. K. N. Lau, D. Gesbert, B. D. Rao, and M. Andrews, “An overview of limited feedback in wireless communication systems,” _IEEE J. Sel. Areas Commun._ , vol. 26, no. 8, pp. 1341–1365, 2008.
  * [2] N. Ravindran and N. Jindal, “Limited feedback-based block diagonalization for the MIMO broadcast channel,” _IEEE J. Sel. Areas Commun._ , vol. 26, no. 8, pp. 1473–1482, 2008.
  * [3] V. Lau, Y. Liu, and T.-A. Chen, “On the design of MIMO block-fading channels with feedback-link capacity constraint,” _IEEE Trans. Commun._ , vol. 52, no. 1, pp. 62–70, 2004.
  * [4] J. Jang, H. Lee, I.-M. Kim, and I. Lee, “Deep learning for multi-user MIMO systems: Joint design of pilot, limited feedback, and precoding,” _IEEE Trans. Commun._ , vol. 70, no. 11, pp. 7279–7293, 2022.
  * [5] N. Turan, M. Koller, S. Bazzi, W. Xu, and W. Utschick, “Unsupervised learning of adaptive codebooks for deep feedback encoding in FDD systems,” in _55th Asilomar Conf. Signals, Syst., Comput._ , 2021, pp. 1464–1469.
  * [6] N. Turan, M. Koller, B. Fesl, S. Bazzi, W. Xu, and W. Utschick, “GMM-based codebook construction and feedback encoding in FDD systems,” in _56th Asilomar Conf. Signals, Syst., Comput._ , 2022, pp. 37–42.
  * [7] N. Turan, B. Fesl, M. Koller, M. Joham, and W. Utschick, “A versatile low-complexity feedback scheme for FDD systems via generative modeling,” _IEEE Trans. Wireless Commun._ , 2023, to be published, arXiv preprint: 2304.14373.
  * [8] T. T. Nguyen, H. D. Nguyen, F. Chamroukhi, and G. J. McLachlan, “Approximation by finite mixtures of continuous density functions that vanish at infinity,” _Cogent Math. Statist._ , vol. 7, no. 1, p. 1750861, 2020.
  * [9] M. Koller, B. Fesl, N. Turan, and W. Utschick, “An asymptotically MSE-optimal estimator based on Gaussian mixture models,” _IEEE Trans. Signal Process._ , vol. 70, pp. 4109–4123, 2022.
  * [10] B. Fesl, N. Turan, M. Joham, and W. Utschick, “Learning a Gaussian mixture model from imperfect training data for robust channel estimation,” _IEEE Wireless Commun. Lett._ , 2023.
  * [11] B. Fesl, M. Joham, S. Hu, M. Koller, N. Turan, and W. Utschick, “Channel estimation based on Gaussian mixture models with structured covariances,” in _56th Asilomar Conf. Signals, Syst., Comput._ , 2022, pp. 533–537.
  * [12] N. Turan, B. Fesl, M. Grundei, M. Koller, and W. Utschick, “Evaluation of a Gaussian mixture model-based channel estimator using measurement data,” in _Int. Symp. Wireless Commun. Syst. (ISWCS)_ , 2022.
  * [13] W. Utschick, V. Rizzello, M. Joham, Z. Ma, and L. Piazzi, “Learning the CSI recovery in FDD systems,” _IEEE Trans. Wireless Commun._ , vol. 21, no. 8, pp. 6495–6507, 2022.
  * [14] N. Turan, M. Koller, V. Rizzello, B. Fesl, S. Bazzi, W. Xu, and W. Utschick, “On distributional invariances between downlink and uplink MIMO channels,” in _25th Int. ITG Workshop Smart Antennas (WSA)_ , 2021.
  * [15] B. Fesl, N. Turan, M. Koller, M. Joham, and W. Utschick, “Centralized learning of the distributed downlink channel estimators in FDD systems using uplink data,” in _25th Int. ITG Workshop Smart Antennas (WSA)_ , 2021.
  * [16] C. Hennig, “Methods for merging Gaussian mixture components,” _Adv. Data Anal. Classif._ , vol. 4, p. 3–34, 2010.
  * [17] A. Runnalls, “Kullback-Leibler approach to Gaussian mixture reduction,” _IEEE Trans. Aerosp. Electron. Syst._ , vol. 43, no. 3, pp. 989–999, 2007\.
  * [18] S. Jaeckel, L. Raschkowski, K. Börner, and L. Thiele, “Quadriga: A 3-d multi-cell channel model with time evolution for enabling virtual field trials,” _IEEE Trans. Antennas Propag._ , vol. 62, no. 6, pp. 3242–3256, 2014.
  * [19] C. M. Bishop, _Pattern Recognition and Machine Learning (Information Science and Statistics)_. Berlin, Heidelberg: Springer-Verlag, 2006.
  * [20] Q. Hu, Y. Cai, Q. Shi, K. Xu, G. Yu, and Z. Ding, “Iterative algorithm induced deep-unfolding neural networks: Precoding design for multiuser MIMO systems,” _IEEE Trans. Wireless Commun._ , vol. 20, no. 2, pp. 1394–1410, 2021.
  * [21] M. Razaviyayn, M. S. Boroujeni, and Z.-Q. Luo, “A stochastic weighted MMSE approach to sum rate maximization for a MIMO interference channel,” in _IEEE 14th Int. Workshop Signal Process. Adv. Wireless Commun. (SPAWC)_ , 2013, pp. 325–329.
  * [22] D. F. Crouse, P. Willett, K. Pattipati, and L. Svensson, “A look at Gaussian mixture reduction algorithms,” in _14th Int. Conf. Inf. Fusion_ , 2011.
  * [23] A. Alkhateeb, G. Leus, and R. W. Heath, “Compressed sensing based multi-user millimeter wave systems: How many measurements are needed?” in _IEEE Int. Conf. Acoust., Speech Signal Process. (ICASSP)_ , 2015, pp. 2909–2913.


# Cell migration: Beyond Brownian motion

Rainer Klages  School of Mathematical Sciences, Queen Mary University of
London, Mile End Road, London E1 4NS, UK  London Mathematical Laboratory, 8
Margravine Gardens, London W6 8RH, United Kingdom

<EMAIL_ADDRESS>New and Notable

Cell migration is vital, fulfilling essential biological tasks like
morphogenesis, wound healing, or the killing of pathogens in organisms. But
cell migration also drives detrimental processes like tumor metastasis or
inflammation reactions. To understand cell migration by classifying different
types of cells defines a fundamental problem of cell science. Recording under
a microscope the paths of single cells reveals random-looking migration (see
Fig. 1(a), right) that, in terms of physics, reminds of the diffusive Brownian
motion of a tracer particle in a fluid. This tracer motion can be described by
the Langevin equation (1), a stochastic version of Newton’s second law, as it
models the random collisions of a tracer with the fluid molecules by a random
force. Around the same time as Langevin published his famous equation, Pearson
proposed to model the movements of organisms by random walks (2). This idea
stipulates that an organism moves in a random direction over a certain
distance per time step. These two approaches coined a paradigm that persisted
for almost a century, suggesting that biological movements are in general ‘so
random’ that they can be modelled by stochastic processes without memory.
Pearson conveyed this idea by the picture of a drunken sailor, which can be
found in many textbooks of statistical mechanics. That is, an agent makes the
next movement step by not remembering the previous one. In mathematical terms,
this complete loss of memory is called a Markov property.

Figure 1: (a) Phase-contrast microscopy images and swimming paths of
Chlamydomonas reinhardtii algae. These cells exhibit two different swimming
modes depending on their diameter, characterised by either wobbler (top image)
or synchro-type (bottom) flagellar motion. Shown to the right of (a) are
sample paths up to a minute, which on these scales look quite ‘random’. (b),
(c) Higher resolution data up to a few seconds, plus blow-ups, revealing a
non-trivial complex fine structure of the swimming paths for (b) wobbler and
(c) synchro (adapted from (3), with kind permission by the authors).

Analysing experimentally recorded paths of different types of migrating cells,
decades ago it was reported that cells move according to the predictions of
random walks (4), respectively Langevin dynamics (5). Subsequent work,
however, challenged this simple picture by revealing the existence of memory
in terms of non-Markovian correlations between different time steps (6).
Especially over the past two decades, many articles appeared suggesting that
cell migration is typically way more complex than Markovian random motion (7,
8, 9, 10). There are indeed many physical and biological reasons to doubt a
simplifying description of cell migration patterns by Brownian motion. First
of all, a Brownian tracer in a fluid diffuses passively, driven by the
collisions with the surrounding particles, while a cell moves actively by
itself converting chemical into kinetic energy (11). Active biological motion,
in turn, involves complex microscopic processes, like cytoskeletal
fluctuations driving cell crawling, or the beating of flagella steering
microswimmers. On top of this, there are interactions with the environment via
surface adhesion and biochemical cell-signalling cascades. Per se there is no
reason why all these complex processes across different spatio-temporal scales
should integrate out to yield purely Markovian dynamics. However, assessing
non-Markovian correlations of cell migration in experiments for constructing
data-driven mathematical models reproducing memory is a notoriously difficult
task. It necessitates to compute correlation functions, related to what
statisticians call covariances, which are not easy to reliably extract from
data, nor to analyse.

In their new contribution to Biophys.J. (3) Klimek et al. tackle this problem
by analysing experimental trajectories of unicellular Chlamydomonas
reinhardtii microalgae. These are microswimmers generating different movement
paths, depending on whether the flagella driving their motion move
synchronously or not, see Fig. 1(b),(c). Using a specific experimental set-up
by which they constrain the microswimmers between two glass plates so that
they do not move out of focus of the microscope, they generated 3000 position
data points for 59 algae each. This data is analysed by using a novel form of
generalised Langevin dynamics incorporating non-Markovian memory, thus
significantly going beyond Brownian motion.

Very interestingly, the key feature of their theory, the generalised Langevin
equation (GLE), can be derived from first principles. Let us assume that all
particles forming a system interact according to Newton’s classical mechanical
equations of motion. Let us identify one of them, with a bigger mass than the
others, as a ‘walker’ while the others act as a ‘driving’ molecular bath. It
is well-known that for this setting, applying a (Mori-Zwanzig) projection
operator formalism by making simplifying assumptions, in particular assuming
complete loss of memory, the Markovian Langevin equation modelling Brownian
motion can be derived. However, this approach can be generalised by including
memory in the dynamics leading to non-Markovian GLEs (12). Very recently, Netz
and colleagues have worked out a new, more detailed theory along these lines
leading to a novel type of GLE (13). This theory has already been applied to
reveal non-Markovian effects in protein folding (14) and human breast cancer
cell migration (10). Reference (3) continues this line of research by
providing compelling evidence that their framework can be used to construct a
non-Markovian GLE from data for modelling the motion of C. reinhardtii
microswimmers. Combining this method with an unbiased cluster analysis they
show that, without any prior knowledge, different types of cells can be
distinguished by purely analysing their migration patterns. Their numerical
findings perfectly match to the different types of cells identified by visual
inspection, cf. the different paths in Fig. 1 (b),(c). This paves the way for
devising a general method to classify different cells based on motion
trajectories only.

A fundamental open question is the generality, or perhaps even universality,
of both their underlying theory and their approach for experimental data
analysis. One may challenge this framework along three lines: First, the GLE
devised by Netz et al. is by far not the only approach to model non-Markovian
stochastic dynamics. Apart from many other types of GLEs, like fractional,
scaled or superstatistical Brownian motion, there is the very different class
of sub- or superdiffusive continuous time random walks (15, 16). The Gaussian
noise generating Brownian motion-type dynamics may furthermore be replaced by
power laws yielding Lévy processes (17). The virtue of the theory by Netz et
al. is that, in contrast to all these other stochastic processes, it has a
sound physical foundation in terms of classical mechanics. On the other hand,
one may wonder whether all physical and biological systems, no matter how
complex they are, need to be understood starting from microscopic Hamiltonian
many-particle dynamics. Furthermore, non-Markovian dynamics driven by power
laws exhibits intricate phenomena like non-stationarity, ageing, weak
ergodicity breaking and long-term anomalous diffusion, all well observed in
physical and biological experiments (15, 16). To what extent the theoretical
framework put forward by Netz et al. can cover these properties as well
remains an interesting open question.

Second, this theory relies on the important assumption of Gaussianity
(normality) of relevant probability distributions by exploiting fluctuation-
dissipation relations (FDRs). But the former excludes Lévy walks that
attracted considerable attention over the past two decades for modelling
organismic movement (17), including cell migration (9). Since Lévy
distributions obey a generalised central limit theorem (15), there is a reason
why one should expect to see them in nature. Another interesting point is that
for describing C. reinhardtii microswimming the authors map a non-Markovian
GLE without FDR onto an effective GLE with FDR. But this again relies on
Gaussianity. Whether this method can be extended to non-Gaussian dynamics, and
to non-equilibrium processes like chemotaxis, poses further inspiring open
questions (18).

Third, the validity of FDRs is intimately related to distinguishing active
from passive dynamics. As explained at the beginnng of this article, the
original Langevin equation models the passive motion of a Brownian tracer.
This follows from its physical derivation based on microscopic Hamiltonian
dynamics yielding diffusion in non-living matter. However, within the past two
decades the new concept of active matter was developed to understand
biological processes in living systems, like the formation of tissues,
swarming, or crowd dynamics. A special case within this framework is the
theory of active Brownian particles, modelling the movements of small agents
like bacteria, daphnia, or artificial Janus particles (11, 19). All active
Brownian particle models feature broken FDRs, mathematically expressing the
externally driven activity of biological dynamics. On the other hand, they
have all been formulated phenomenologically, instead of being derived from
Hamiltonian dynamics, like GLEs exhibiting FDRs. To what extent standard
models of active Brownian motion (11, 19) can be derived from first principles
forms yet another crucial open question. Recently, the dynamics of a passive
tracer in an active bath of microswimmers has been calculated from microscopic
non-Hamiltonian equations of motion (20), taking special care of the
hydrodynamic interaction between tracer and bath. Combining this with other
techniques for constructing a biomolecular model of cell migration poses a
formidable challenge for future research.

## Declaration of interests

The author declares no competing interests.

## References

  * Langevin (1908) Langevin, P., 1908.  Sur la théorie du mouvement brownien.  _C.R. Acad. Sci. (Paris)_ 146:530–533.  See also the English translation in Am. J. Phys. 65, 1079 (1997).
  * Pearson (1906) Pearson, K., 1906.  Mathematical contributions to the theory of evolution - A mathematical theory of random migration.  _Biometric ser._ 3:54.
  * Klimek et al. (2024) Klimek, A., D. Mondal, S. Block, P. Sharma, and R. R. Netz, 2024.  Data-driven classification of individual cells by their non-Markovian motion.  _Biophys. J._ 123:1173–1183
  * Gail and Boone (1970) Gail, M., and C. Boone, 1970.  The locomotion of mouse fibroblasts in tissue culture.  _Biophys. J._ 10:980–993.
  * Dunn and Brown (1987) Dunn, G., and A. Brown, 1987.  A unified approach to analysing cell motility.  _J. Cell Sci. Suppl._ 8:81–102.
  * Hartmann et al. (1994) Hartmann, R., K. Lau, W. Chou, and T. Coates, 1994.  The fundamental motor of the human neutrophil is not random: Evidence for local non-Markov movement in neutrophils.  _Biophys. J._ 67:2535–2545.
  * Rieu et al. (2000) Rieu, J., A. Upadhyaya, J. Glazier, N. Ouchi, and Y. Sawada, 2000.  Diffusion and deformations of single Hydra cells in cellular aggregates.  _Biophys. J._ 79:1903–1914.
  * Dieterich et al. (2008) Dieterich, P., R. Klages, R. Preuss, and A. Schwab, 2008.  Anomalous dynamics of cell migration.  _PNAS_ 105:459–463.
  * Harris et al. (2012) Harris, T., E. Banigan, D. Christian, et al., 2012.  Generalized Lévy walks and the role of chemokines in migration of effector CD8+ T cells.  _Nature_ 486:545–548.
  * Mitterwallner et al. (2020) Mitterwallner, B. G., C. Schreiber, J. O. Daldrop, J. O. Rädler, and R. R. Netz, 2020.  Non-Markovian data-driven modeling of single-cell motility.  _Phys. Rev. E_ 101:032408.
  * Romanczuk et al. (2012) Romanczuk, P., M. Bär, W. Ebeling, B. Lindner, and L. Schimansky-Geier, 2012\.  Active Brownian particles.  _Eur. Phys. J. Spec. Top._ 202:1–162.
  * Zwanzig (2001) Zwanzig, R., 2001.  Nonequilibrium statistical mechanics.  Oxford University Press, Oxford.
  * Ayaz et al. (2022) Ayaz, C., L. Scalfi, B. A. Dalton, and R. R. Netz, 2022.  Generalized Langevin equation with a nonlinear potential of mean force and nonlinear memory friction from a hybrid projection scheme.  _Phys. Rev. E_ 105:054138.
  * Dalton et al. (2023) Dalton, B. A., C. Ayaz, H. Kiefer, A. Klimek, L. Tepper, and R. R. Netz, 2023.  Fast protein folding is governed by memory-dependent friction.  _PNAS_ 120:e2220068120.
  * Klages et al. (2008) Klages, R., G. Radons, and I. M. Sokolov, editors, 2008.  Anomalous transport: Foundations and Applications.  Wiley-VCH, Berlin.
  * Metzler et al. (2014) Metzler, R., J.-H. Jeon, A. G. Cherstvy, and E. Barkai, 2014.  Anomalous diffusion models and their properties: non-stationarity, non-ergodicity, and ageing at the centenary of single particle tracking.  _Phys. Chem. Chem. Phys._ 16:24128–24164.
  * Viswanathan et al. (2011) Viswanathan, G., M. da Luz, E. Raposo, and H. Stanley, 2011.  The Physics of Foraging.  Cambridge University Press, Cambridge.
  * Netz (2023) Netz, R., 2023.  Derivation of the non-equilibrium generalized Langevin equation from a generic time-dependent Hamiltonian.  _preprint arXiv:2310.00748_ .
  * Bechinger et al. (2016) Bechinger, C., R. Di Leonardo, H. Löwen, C. Reichhardt, G. Volpe, and G. Volpe, 2016.  Active particles in complex and crowded environments.  _Rev. Mod. Phys._ 88:045006.
  * Kanazawa et al. (2020) Kanazawa, K., T. G. Sano, A. Cairoli, and A. Baule, 2020.  Loopy Lévy flights enhance tracer diffusion in active suspensions.  _Nature_ 120:364–367.