$L_{D-1}$ effectively plays the role of the final layer in Segment 1, and we
will refer to it as such.

First, we point out that transitions within Segment 2 do not affect the
excitations in the non-final layers of Segment 1. In particular, transitions
within Segment 2 do not change the number of steps we need to empty the non-
final layers of Segment 1. However, transitions within Segment 1 may fill up
layer $L_{D-1}$.

Since transitions within Segment 2 do not change the number of steps needed
for Segment 1, we can empty $L_{D-1}$ in Segment 2 every time new excitations
arrive, before continuing with emptying Segment 1. This ensures that every
time Segment 1 moves excitations into Segment 2, the clips are empty and no
excitations are discarded in layer $L_{D-1}$. If we did not always empty layer
$L_{D-1}$ first, it would not affect the number of steps needed to empty
Segment 1, but it may reduce the total number of transitions within Segment 2
because moving a new excitation into an already excited clip just discards the
excitation. That is one less excitation to remove one-by-one. This argument
shows that the worst random walk empties the deepest non-final layer first.

An upper bound on the worst case is given by the assumption that every step
within Segment 1 fills up all of layer $L_{D-1}$. The slowest method to empty
layer $L_{D-1}$ needs $|L_{D-1}|$ steps and can use $(1,|L_{D}|)$-transitions
for doing so.

By the induction hypothesis, an upper bound on the number of transitions
within Segment 1 emptying Segment 1 is provided by
$\prod_{j=1}^{D-2}(|L_{j}|+1)$. For the upper bound, after each of these
transitions, we use $|L_{D-1}|$ Segment 2 transitions to empty layer
$L_{D-1}$. So, the upper bound on the total number of steps is:

$\displaystyle\prod_{j=1}^{D-2}(|L_{j}|+1)+|L_{D-1}|\cdot\prod_{j=1}^{D-2}(|L_{j}|+1)$
$\displaystyle=$ $\displaystyle\prod_{j=1}^{D-1}(|L_{j}|+1)$

The first product on the left-hand side is the upper bound on the transitions
within Segment 1, and the second product expresses that for each transition
within Segment 1, we also have up to $|L_{D-1}|$ transitions within Segment 2.

∎

In Proposition 16, we did not consider low cutoffs for $IO$. This may raise
the question of whether such cutoffs dodge the exponential complexity.
However, in general this is not the case, as we will see in the following
proposition:

######  Proposition 22.

Consider a mePS agent obeying Inductive Bias 2FF with layers
$L_{1},\dots,L_{D}$. For any $o\leq|L_{2}|,\dots,|L_{D}|$ with $o\geq 2$,
assume that $(1,o)\in IO$ and let $E^{(1,o)}_{\mathrm{all}}=E^{(1,o)}$. Assume
that actions are coupled out only when only the final layer has excitations
and that there exist percept excitation configurations that have at least $o$
excitations in layer $L_{1}$.

A lower bound on the maximal deliberation time is then given by
$\prod_{j=1}^{D-1}o=o^{D-1}$.

###### Proof.

The proof proceeds by induction over the layers, from deep to shallow. For
that, we first establish that the cost to remove one excitation from layer
$L_{D-2}$ with $(1,o)$-transitions is $o+1$, because removing that excitation
in $L_{D-2}$ creates $o$ excitations in $L_{D-1}$, which can be removed with
$o$ transitions $(1,o)$.

Next, assume that $L_{j+1}$ is the deepest non-final layer with excitations,
and that removing an excitation from layer $L_{j+1}$ and emptying all of
layers $L_{j+2}$, …, $L_{D-1}$ can be done with a sequence of transitions that
takes at least $\prod_{k=j+1}^{D-2}o$ steps.

Let us now consider an excitation in layer $L_{j}$, and assume layers
$L_{j+1},\dots,L_{D-1}$ are empty. Removing it with a $(1,o)$-transition
creates $o$ excitations in $L_{j+1}$. To remove each of these new excitations
and also to empty the non-final layers ahead, the induction hypothesis claims
that there is a random walk that does this with at least
$\prod_{k=j+1}^{D-2}o$ steps. We have to do this for all the $o$ excitations,
giving us a number of steps $\prod_{k=j}^{D-2}o$.

Induction therefore shows that removing an excitation in Layer $L_{1}$ can be
done with a random walk that takes at least $\prod_{k=1}^{D-2}o$ steps. If we
consider a percept configuration with $o$ excitations in the first layer, we
can first empty the deeper layers, and then all the $o$ excitations in layer
$L_{1}$ using at least $o\cdot\prod_{k=1}^{D-2}o$ steps. ∎

While there is a upper bound on the maximal deliberation time, it scales
exponentially with the depth $D$. The expression $\prod_{k=j}^{D-1}o$ from the
previous proposition shows that in general, an exponential (in $D$) maximal
deliberation time is unavoidable. However, the proofs of these exponential
bounds required that one first has to remove excitations from the deepest
layers. This observation motivated us to introduce Inductive Bias 2SF.


††thanks: The authors are with King’s Communications, Learning, and
Information Processing (KCLIP) lab at the Department of Engineering of Kings
College London, UK (emails<EMAIL_ADDRESS>osvaldo.simeone@kcl.ac.uk).

# HyperRNN: Deep Learning-Aided Downlink CSI Acquisition via Partial Channel
Reciprocity for FDD Massive MIMO

Yusha Liu  KCLIP Lab, Department of Engineering
King’s College London, London, UK
<EMAIL_ADDRESS>Osvaldo Simeone  KCLIP Lab, Department of Engineering
King’s College London, London, UK
<EMAIL_ADDRESS>
###### Abstract

In order to unlock the full advantages of massive multiple input multiple
output (MIMO) in the downlink, channel state information (CSI) is required at
the base station (BS) to optimize the beamforming matrices. In frequency
division duplex (FDD) systems, full channel reciprocity does not hold, and CSI
acquisition generally requires downlink pilot transmission followed by uplink
feedback. Prior work proposed the end-to-end design of pilot transmission,
feedback, and CSI estimation via deep learning. In this work, we introduce an
enhanced end-to-end design that leverages partial uplink-downlink reciprocity
and temporal correlation of the fading processes by utilizing jointly downlink
and uplink pilots. The proposed method is based on a novel deep learning
architecture – _HyperRNN_ – that combines hypernetworks and recurrent neural
networks (RNNs) to optimize the transfer of long-term channel features from
uplink to downlink. Simulation results demonstrate that the HyperRNN achieves
a lower normalized mean square error (NMSE) performance, and that it reduces
requirements in terms of pilot lengths.

###### Index Terms:

FDD, massive MIMO, deep learning.

##  I Introduction

With frequency division duplex (FDD), downlink channel state information (CSI)
cannot be directly obtained from uplink pilots due to a lack of full
reciprocity between uplink and downlink channels. This poses a challenge in
massive massive multiple input multiple output (MIMO) systems, since the use
of downlink pilots entails a generally large communication overhead to feed
back the estimated CSI from users to base station (BS), owing to the massive
number of antennas. Solutions to this practically important problem can be
divided into uplink training-based, downlink training-based, and hybrid
methods. In the first class are schemes that leverage _partial reciprocity_ in
the form of frequency- and time-invariant multipath parameters, such as angles
of arrival/ departure (AoAs/ AoDs) and path gains, to directly map uplink CSI
to downlink CSI [1, 2, 3, 4]. Downlink training-based techniques leverage
machine learning for the design of CSI compression and uplink feedback
algorithms [5, 6]. Hybrid schemes typically operate sequentially, with the
uplink pilots used to identify spatial directions along which to send a
reduced number of pilots in the downlink motivated by partial reciprocity [7,
8, 9]. In this paper, we propose a novel end-to-end design for downlink CSI
acquisition that leverages partial uplink-downlink reciprocity and temporal
correlation of the fading processes in a hybrid architecture that utilizes the
simultaneous transmission of downlink and uplink pilots (see Fig. 1).

The reference downlink-based end-to-end architecture for downlink training,
uplink feedback, and channel estimation introduced in [10] is illustrated in
Fig. 2. In it, downlink pilots are transmitted by the BS; pilots are processed
by a deep neural network (DNN) to produce a feedback message consisting of a
given number of bits; and the message is in turn processed by another DNN at
the BS for CSI acquisition. Note that reference [10] studies also the direct
design of downlink beamforming matrices – a topic that we will cover in an
extension of this work. In this paper, we propose an enhanced, hybrid, end-to-
end design that is based on a novel deep learning architecture – _HyperRNN_ –
illustrated in Fig. 3. The main innovation of the approach is that
simultaneously transmitted pilot symbols in the uplink, across multiple time
slots (see Fig. 1), are leveraged to automatically extract long-term
reciprocal channel features via a hypernetwork [11] that determines the weight
of the downlink CSI estimation network. Importantly, unlike the existing works
reviewed above, such as [7, 8, 9], the long-term features are not estimated
explicitly, but they implicitly underlie the discriminative mapping
implemented by the hypernetwork between uplink pilots and downlink CSI
estimation network. The second main innovation is to incorporate recurrent
neural networks (RNNs), in lieu of (feedforward) DNNs for both uplink and
downlink processing in order to leverage the temporal correlation of the
fading amplitudes.

Among other related works, we mention [12], which proposed the use of
hypernetworks for MIMO detection in order to avoid retraining for different
channel realizations; as well as [13] that introduced a downlink training-
based compression scheme for uplink feedback based on RNNs.

The rest of the paper is organized as follows. Section II describes the FDD
system model. Section III reviews the solution proposed in [10], and Section
Section IV proposes the HyperRNN architecture. Numerical results are provided
in Section V, and conclusions are provided in Section VI.

Figure 1: Timeline of uplink and downlink transmission of an FDD massive MIMO
system.

##  II System Model

We focus on an FDD MIMO system with $M$ transmit antennas (TAs) at BS
supporting a number of single-antenna users. As shown in Fig. 1, since the
system uses FDD, uplink and downlink transmissions occur in parallel, at
different carrier frequencies $f_{C}^{\textrm{ul}}$ and $f_{C}^{\textrm{dl}}$,
respectively. We assume orthogonal pilots and no correlation among the
channels of different users. As a result, we can concentrate without loss of
generality on a single user. We study flat-fading channels, which may
represent individual subcarriers in a multi-carrier system.

_Uplink Transmission_. The proposed downlink channel estimation framework
leverages both uplink and downlink pilots by assuming partial reciprocity.
During uplink transmission, each user transmits $L^{\textrm{ul}}$ uplink
pilots $\boldsymbol{x}^{\textrm{ul}}\in\mathbb{C}^{1\times L^{\textrm{ul}}}$
to the BS at the start of the $t$-th ($t=1,\dots,T$) time slot. At the BS, the
$M\times L^{\textrm{ul}}$ received discrete-time samples are modelled as

$\displaystyle\boldsymbol{Y}_{t}^{\textrm{ul}}=\boldsymbol{h}^{\textrm{ul}}_{t}\boldsymbol{x}^{\textrm{ul}}+\boldsymbol{N}_{t}^{\textrm{ul}},$
(1)

where $\boldsymbol{h}^{\textrm{ul}}_{t}\in\mathbb{C}^{M\times 1}$ is the
uplink channel vector and
$\boldsymbol{N}_{t}^{\textrm{ul}}\in\mathbb{C}^{M\times L^{\textrm{ul}}}$ is
additive white Gaussian noise (AWGN) with i.i.d. elements having zero mean and
variance $\sigma^{2}$. After uplink pilot transmission, during the rest of the
$t$-th time slot, uplink data symbols of the user are transmitted to the BS.

_Downlink Transmission_. At the same time as the uplink transmission, in the
downlink transmission, at each time slot $t=1,\dots,T$, the BS transmits
$L^{\textrm{dl}}$ pilot symbols from the $M$ TAs, which we collect in matrix
$\boldsymbol{X}^{\textrm{dl}}\in\mathbb{C}^{M\times L^{\textrm{dl}}}$. The
$L^{\textrm{dl}}$ received discrete-time samples at the user of interest are
collected in vector $\boldsymbol{y}_{t}^{\textrm{dl}}\in\mathbb{C}^{1\times
L^{\textrm{dl}}}$, which is modelled as

$\displaystyle\boldsymbol{y}_{t}^{\textrm{dl}}=(\boldsymbol{h}^{\textrm{dl}}_{t})^{H}\boldsymbol{X}^{\textrm{dl}}+\boldsymbol{n}_{t}^{\textrm{dl}},$
(2)

where $\boldsymbol{h}^{\textrm{dl}}_{t}\in\mathbb{C}^{M\times 1}$ is the
downlink channel vector and $\boldsymbol{n}_{t}^{\textrm{dl}}$ is AWGN with
zero mean and variance $\sigma^{2}$. As for the uplink, we assume a quasi-
static channel that is constant within each slot. Upon receiving vector
$\boldsymbol{y}_{t}^{\textrm{dl}}$, the user processes it and quantizes the
result, producing a message
$\boldsymbol{q}^{\textrm{dl}}_{t}=f^{\mathcal{Q}}(\boldsymbol{y}^{\textrm{dl}}_{t})$
of $B$ bits, where $f^{\mathcal{Q}}(\cdot)$ represents the composition of
processing and quantization. The message of $B$ bits is fed back to the BS.

_Channel Model_. The channel remains constant within a single time slot $t$.
Furthermore, we assume standard multipath channels, whereby each path is
characterized by both long-term features that remain constant for a number $T$
of time slots and fast-varying fading amplitudes [14]. Specifically, each path
$p$, with $p=1,\dots,P$, between user and BS is characterized by the long-
term, time-invariant, AoD $\theta_{p}$ and path gain $\alpha_{p}$; as well as
by a time-varying fading process $\beta^{\textrm{ul}}_{p,t}$ for uplink
channel and $\beta^{{\textrm{dl}}}_{p,t}$ for downlink channel, respectively.
AoDs $\\{\theta_{p}\\}_{p=1}^{P}$ and path gains $\\{\alpha_{p}\\}_{p=1}^{P}$
are assumed to be invariant, given the spatial and temporal resolution of the
systems, over $T$ time slots. In contrast, the fading processes
$\\{\beta^{\textrm{ul}}_{p,t}\\}_{p=1}^{P}$ and
$\\{\beta^{\textrm{dl}}_{p,t}\\}_{p=1}^{P}$ vary across the time slots
$t=1,\dots,T$. Partial reciprocity implies that the long-term multi-path
features $\\{\theta_{p}\\}_{p=1}^{P}$ and $\\{\alpha_{p}\\}_{p=1}^{P}$ are
equal for uplink and downlink, while the fading processes are different, being
a function of the carrier frequency $f^{\textrm{ul}}_{C}$/
$f^{\textrm{dl}}_{C}$ [3].

Given the above assumptions, the $P$-path quasi-static uplink and downlink
channel models of the user at time slot $t$ are expressed as

$\displaystyle\boldsymbol{h}_{t}^{\textrm{ul}}=\sum_{p=1}^{P}\alpha_{p}\boldsymbol{a}^{\textrm{ul}}(\theta_{p})\beta^{\textrm{ul}}_{p,t},~{}\textrm{and}~{}\boldsymbol{h}_{t}^{\textrm{dl}}=\sum_{p=1}^{P}\alpha_{p}\boldsymbol{a}^{\textrm{dl}}(\theta_{p})\beta^{\textrm{dl}}_{p,t},$
(3)

where $\boldsymbol{a}^{\textrm{ul}}(\theta_{p})$ and
$\boldsymbol{a}^{\textrm{dl}}(\theta_{p})$ are the steering vectors, which
depend on the antenna array and respective carrier frequencies. The scaled
steering vectors $\alpha_{p}\boldsymbol{a}^{\textrm{ul}}(\theta_{p})$ and
$\alpha_{p}\boldsymbol{a}^{\textrm{dl}}(\theta_{p})$ are invariant in time
slots $t=1,\dots,T$. Furthermore, they are related, since they both depend on
the AoD $\theta_{p}$, but they are distinct, due to the carrier frequency
difference $\Delta f_{C}=|f_{C}^{\textrm{ul}}-f_{C}^{\textrm{dl}}|>0$.

The fading amplitudes $\\{\beta^{\textrm{ul}}_{p,t}\\}_{t=1}^{T}$ in the
uplink and $\\{\beta^{\textrm{dl}}_{p,t}\\}_{t=1}^{T}$ in the downlink are
assumed to be independent and to evolve over the slot index $t$ according to
the dynamic model accounting for temporal correlation. As a common example, a
first-order Autoregressive (AR) model [13] can be assumed with correlation
coefficients $\rho^{\textrm{ul}}=J_{0}(2\pi f^{\textrm{ul}}_{d}\tau)\in[-1,1]$
and $\rho^{\textrm{dl}}=J_{0}(2\pi f^{\textrm{dl}}_{d}\tau)$, where
$J_{0}(\cdot)$ is the zero-order Bessel function of the first kind; $\tau$ is
the time slot duration; and $f^{\textrm{dl}}_{d}=vf^{\textrm{dl}}_{C}/c$ and
$f^{\textrm{ul}}_{d}=vf^{\textrm{ul}}_{C}/c$ are the maximum Doppler frequency
with $v$ being the mobile velocity.

##  III DNN-based Downlink CSI Estimation

Figure 2: End-to-end downlink channel estimation based on uplink feedback
[10].

In this section, we first describe the baseline DNN-based downlink CSI
prediction approach proposed in [10], which estimates the downlink channel
$\boldsymbol{h}_{t}^{\textrm{dl}}$ based on the $B$-bit message
$\boldsymbol{q}_{t}^{\textrm{dl}}$ obtained from downlink training. As shown
in Fig. 2, the concatenation of pilot transmission, quantization, and channel
estimation is modelled with separate neural network (NN) models and trained in
an end-to-end fashion.

###  III-1 Downlink Pilot Transmission

To enable end-to-end training, first, the received signal (2) is modelled as
the output of a fully-connected linear layer with input
$\boldsymbol{h}^{\textrm{dl}}_{t}$, weight matrix
$\boldsymbol{X}^{\textrm{dl}}$, and output
$\boldsymbol{y}_{t}^{\textrm{dl}}=\boldsymbol{h}^{\textrm{dl}}_{t}\boldsymbol{X}^{\textrm{dl}}$,
to which Gaussian noise $\boldsymbol{n}_{t}^{\textrm{dl}}$ is added. The
training matrix $\boldsymbol{X}^{\textrm{dl}}$ is subject to design.

###  III-2 Uplink Feedback Quantization

In order to produce the $B$-bit message $\boldsymbol{q}^{\textrm{dl}}_{t}$,
upon receiving the signal $\boldsymbol{y}_{t}^{\textrm{dl}}$ in (2), the user
applies a multi-layer fully-connected DNN with a sign$(\cdot)$ activation
function at the last layer. Specifically, as shown in Fig. 2, the inputs of
the DNN are comprised of the real and imaginary parts of elements in
$\boldsymbol{y}^{\textrm{dl}}_{t}$, which can be expressed as the
$2L^{\textrm{dl}}\times 1$ vector
$\bar{\boldsymbol{y}}^{\textrm{dl}}_{t}=\textrm{c2r}(\boldsymbol{y}^{\textrm{dl}}_{t})=\left[\Re(\boldsymbol{y}_{t}^{\textrm{dl}})^{T},\Im(\boldsymbol{y}_{t}^{\textrm{dl}})^{T}\right]^{T}$,
where the function $\textrm{c2r}(x)=[\Re(x)^{T}~{}\Im(x)^{T}]^{T}$ denotes the
complex to real value representation, with $\Re(\cdot)$ and $\Im(\cdot)$
representing the real and imaginary parts of the elements in
$\boldsymbol{y}^{\textrm{dl}}_{t}$, respectively. Then, vector
$\bar{\boldsymbol{y}}^{\textrm{dl}}_{t}$ is processed through
$(M^{\mathcal{Q}}-1)$ fully connected layers with rectified linear unit (ReLU)
activation functions, while the last layer produces $B$ binary outputs through
the mentioned sign$(\cdot)$ non-linearity.

Accordingly, denoting $\ell_{m}^{\mathcal{Q}}$ as the number of ReLU neurons
in the $m$-th layer, the optimization parameters of the quantization DNN are
${\boldsymbol{\Omega}}^{\mathcal{Q}}=\\{{\boldsymbol{W}}_{1}^{\mathcal{Q}},{\boldsymbol{b}}_{1}^{\mathcal{Q}},\cdots,{\boldsymbol{W}}_{M^{\mathcal{Q}}}^{\mathcal{Q}},{\boldsymbol{b}}_{M^{\mathcal{Q}}}^{\mathcal{Q}}\\}$,
with $\ell_{m}^{\mathcal{Q}}\times\ell_{m+1}^{\mathcal{Q}}$ weight matrix
${\boldsymbol{W}}_{m}^{\mathcal{Q}}$ and $\ell_{m}^{\mathcal{Q}}\times 1$ bias
vector ${\boldsymbol{b}}_{m}^{\mathcal{Q}}$, with
$\ell_{1}^{\mathcal{Q}}=2L^{\textrm{dl}}$ and
$\ell_{M^{\mathcal{Q}}+1}^{\mathcal{Q}}=B$.

###  III-3 Channel Estimation

The output $\boldsymbol{q}_{t}^{\textrm{dl}}$ of the quantization DNN at the
user is forwarded to the DNN that is employed for downlink CSI estimation at
the BS. The $B\times 1$ vector $\boldsymbol{q}_{t}^{\textrm{dl}}$ is processed
through $M^{\mathcal{E}}$ fully connected layers. The first
$(M^{\mathcal{E}}-1)$ ReLU hidden layers have $\ell_{m}^{\mathcal{E}}$
neurons, with $m=1,\dots,M^{\mathcal{E}}-1$; while the last linear layer
produces $2M$ real-valued outputs. The optimization parameters
${\boldsymbol{\Omega}}^{\mathcal{E}}$ for the channel estimation DNN are
${\boldsymbol{\Omega}}^{\mathcal{E}}=\\{{\boldsymbol{W}}_{1}^{\mathcal{E}},{\boldsymbol{b}}_{1}^{\mathcal{E}},\dots,{\boldsymbol{W}}_{M^{\mathcal{E}}}^{\mathcal{E}},{\boldsymbol{b}}_{M^{\mathcal{E}}}^{\mathcal{E}}\\}$,
where ${\boldsymbol{W}}_{m}^{\mathcal{E}}$ is the
$\ell_{m}^{\mathcal{E}}\times\ell_{m+1}^{\mathcal{E}}$ weight matrix and
${\boldsymbol{b}}_{m}^{\mathcal{E}}$ represents $\ell_{m}^{\mathcal{E}}\times
1$ bias vector, with $\ell_{1}^{\mathcal{E}}=B$ and
$\ell_{M^{\mathcal{E}}+1}^{\mathcal{E}}=2M$.

###  III-4 Training

End-to-end training of the pilot matrix $\boldsymbol{X}^{\textrm{dl}}$ and of
the DNN parameters $\boldsymbol{\Omega}^{\mathcal{Q}}$ and
$\boldsymbol{\Omega}^{\mathcal{E}}$ is done by minimizing the training squared
error between the DNN output $\hat{\boldsymbol{h}}_{t}^{\textrm{dl}}$ and the
real ${\boldsymbol{h}}_{t}^{\textrm{dl}}$ by using an empirical average over
the training sample set
$\\{\boldsymbol{y}_{t}^{\textrm{dl}},\boldsymbol{h}_{t}^{\textrm{dl}}\\}$ in
lieu of the expectation.

##  IV HyperRNN-Based Downlink CSI Estimation

Figure 3: The proposed HyperRNN architecture for end-to-end channel estimation
based on temporal correlations and partial reciprocity.

As shown in Fig. 3, the proposed HyperRNN architecture aims at extracting
invariant reciprocal features from multipath uplink pilot transmissions by
integrating a hypernetwork that takes the input of the signals received in the
uplink and outputs the weights of the channel estimation RNN [11]. The key
idea is that the weights of the channel estimation RNN can encode information
about the long-term channel features that are common to both uplink and
downlink, hence automatically accounting for partial reciprocity.

###  IV-1 Uplink Pilot Transmission

As in the downlink, uplink pilot transmission process is modelled by a a
fully-connected linear layer with input being
$\boldsymbol{h}^{\textrm{ul}}_{t}$, weight matrix being
$\boldsymbol{x}^{\textrm{ul}}$, and output being
$\boldsymbol{Y}_{t}^{\textrm{ul}}$, to which Gaussian noise
$\boldsymbol{N}_{t}^{\textrm{ul}}$ is added.

###  IV-2 Downlink Pilot Transmission, Uplink Feedback, and Channel Estimation

Downlink pilot transmission and uplink feedback quantization operate as
discussed in Section III, with the caveat that in order to leverage the long-
term invariance and short-term time-correlation properties of the downlink
channel, we replace the channel estimation DNN with an RNN. At each time slot
$t$, the RNN produces the estimate

$\displaystyle\textrm{c2r}(\hat{\boldsymbol{h}}_{t}^{\textrm{dl}})=\boldsymbol{W}^{\mathcal{E}}_{B}\boldsymbol{s}^{\mathcal{E}}_{t}+\boldsymbol{b}^{\mathcal{E}}_{B}\triangleq
f^{\mathcal{E}}\left(\boldsymbol{q}_{t}^{\textrm{dl}},\boldsymbol{s}^{\mathcal{E}}_{t-1}|\boldsymbol{\Omega}^{\mathcal{E}}\right),$
(4)

where the $\ell^{\mathcal{E}}\times 1$ internal state of the RNN evolves as

$\displaystyle\boldsymbol{s}^{\mathcal{E}}_{t}$
$\displaystyle=f_{\textrm{ReLU}}\left(\boldsymbol{W}^{\mathcal{E}}_{A}\boldsymbol{q}^{\textrm{dl}}_{t}+\boldsymbol{W}^{\mathcal{E}}_{C}\boldsymbol{s}^{\mathcal{E}}_{t-1}+\boldsymbol{b}^{\mathcal{E}}_{A}\right),$
(5)

with $\boldsymbol{W}^{\mathcal{E}}_{A}\in\mathbb{C}^{\ell^{\mathcal{E}}\times
B}$, $\boldsymbol{b}^{\mathcal{E}}_{A}\in\mathbb{C}^{\ell^{\mathcal{E}}\times
1}$,
$\boldsymbol{W}^{\mathcal{E}}_{B}\in\mathbb{C}^{2M\times\ell^{\mathcal{E}}}$,
$\boldsymbol{b}_{B}\in\mathbb{C}^{2M\times 1}$,
$\boldsymbol{W}^{\mathcal{E}}_{C}\in\mathbb{C}^{\ell^{\mathcal{E}}\times\ell^{\mathcal{E}}}$,
$\boldsymbol{s}^{\mathcal{E}}_{t}\in\mathbb{C}^{\ell^{\mathcal{E}}\times 1}$
and $\ell^{\mathcal{E}}$ denoting the number of neurons employed for the
fully-connected ReLU layer.

###  IV-3 HyperRNN

To leverage partial reciprocity, we introduce a hypernetwork [11] to adjust
the weights of the downlink channel estimation RNN based on the uplink
received signal $\boldsymbol{Y}_{t}^{\textrm{ul}}$, as shown in Fig. 3. In
order to reduce the number of outputs of the hypernetwork, as in, e.g., [12],
the hypernetwork generates a common scaling factor for each column of the
weight matrices at each time slot $t$. Accordingly, the output of the
hypernetwork is a $(B+2\ell^{\mathcal{E}})\times 1$ vector
$\boldsymbol{\omega}_{t}$, which is detailed next.

The real and imaginary parts of the received uplink signal
$\boldsymbol{Y}_{t}^{\textrm{ul}}$ are collected in the
$2ML^{\textrm{ul}}\times 1$ vector
$\bar{\boldsymbol{y}}_{t}^{\textrm{ul}}=\textrm{c2r}(\textrm{vec}(\boldsymbol{Y}_{t}^{\textrm{ul}}))$,
with $\textrm{vec}(\cdot)$ denoting the vectorization of a matrix by stacking
columns. This vector is fed as input to the hypernetwork, together with the
internal state $\boldsymbol{s}^{\mathcal{H}}_{t-1}$ from the previous time
slot $t-1$. In a manner similar to (4)-(5), the hypernetwork operates as

$\displaystyle\boldsymbol{s}^{\mathcal{H}}_{t}=$ $\displaystyle
f_{\textrm{ReLU}}\left(\boldsymbol{W}^{\mathcal{H}}_{A}\bar{\boldsymbol{y}}^{\textrm{ul}}_{t}+\boldsymbol{W}^{\mathcal{H}}_{C}\boldsymbol{s}^{\mathcal{H}}_{t-1}+\boldsymbol{b}^{\mathcal{H}}_{A}\right)$
(6a) $\displaystyle\textrm{and}~{}~{}~{}\boldsymbol{\omega}_{t}=$
$\displaystyle\boldsymbol{W}^{\mathcal{H}}_{B}\boldsymbol{s}^{\mathcal{H}}_{t}+\boldsymbol{b}^{\mathcal{H}}_{B}\triangleq
f^{\mathcal{H}}\left(\boldsymbol{y}^{\textrm{ul}}_{t},\boldsymbol{s}^{\mathcal{H}}_{t-1}|\boldsymbol{\Omega}^{\mathcal{H}}\right),$
(6b)

where $\boldsymbol{s}^{\mathcal{H}}_{t}\in\mathbb{C}^{\ell^{\mathcal{H}}\times
1}$ is the internal state, and
$\boldsymbol{W}^{\mathcal{H}}_{A}\in\mathbb{C}^{\ell^{\mathcal{H}}\times
2ML^{\textrm{ul}}}$,
$\boldsymbol{b}^{\mathcal{H}}_{A}\in\mathbb{C}^{\ell^{\mathcal{H}}\times 1}$,
$\boldsymbol{W}^{\mathcal{H}}_{B}\in\mathbb{C}^{(B+2\ell^{\mathcal{E}})\times\ell^{\mathcal{H}}}$,
$\boldsymbol{b}^{\mathcal{H}}_{B}\in\mathbb{C}^{(B+2\ell^{\mathcal{E}})\times
1}$,
$\boldsymbol{W}^{\mathcal{H}}_{C}\in\mathbb{C}^{\ell^{\mathcal{H}}\times\ell^{\mathcal{H}}}$
are optimization parameters
$\boldsymbol{\Omega}^{\mathcal{H}}=\\{{\boldsymbol{W}}_{A}^{\mathcal{H}},{\boldsymbol{b}}_{A}^{\mathcal{H}},{\boldsymbol{W}}_{B}^{\mathcal{H}},{\boldsymbol{b}}_{B}^{\mathcal{H}},{\boldsymbol{W}}_{C}^{\mathcal{H}}\\}$
of the hypernetwork. The $(B+2\ell^{\mathcal{E}})\times 1$ output vector
$\boldsymbol{\omega}_{t}$ modifies the weights of the downlink channel
estimation RNN (4)-(5) as
$\boldsymbol{W}^{\mathcal{E}}_{A}=\bar{\boldsymbol{W}}^{\mathcal{E}}_{A}\cdot\textrm{diag}\left\\{\boldsymbol{\omega}_{t,A}\right\\}$,
$\boldsymbol{W}^{\mathcal{E}}_{B}=\bar{\boldsymbol{W}}^{\mathcal{E}}_{B}\cdot\textrm{diag}\left\\{\boldsymbol{\omega}_{t,B}\right\\}$,
and
$\boldsymbol{W}^{\mathcal{E}}_{C}=\bar{\boldsymbol{W}}^{\mathcal{E}}_{C}\cdot\textrm{diag}\left\\{\boldsymbol{\omega}_{t,C}\right\\}$.
We have partitioned the output of the hypernetwork as
$\boldsymbol{\omega}_{t}=[\boldsymbol{\omega}_{t,A},\boldsymbol{\omega}_{t,B},\boldsymbol{\omega}_{t,C}]$,
where $\boldsymbol{\omega}_{t,A}\in\mathbb{C}^{B\times
1},\boldsymbol{\omega}_{t,B}\in\mathbb{C}^{\ell^{\mathcal{E}}\times 1}$, and
$\boldsymbol{\omega}_{t,C}\in\mathbb{C}^{\ell^{\mathcal{E}}\times 1}$. The
matrices
$\bar{\boldsymbol{W}}^{\mathcal{E}}_{A}\in\mathbb{C}^{\ell^{\mathcal{E}}\times
B}$,
$\bar{\boldsymbol{W}}^{\mathcal{E}}_{B}\in\mathbb{C}^{2M\times\ell^{\mathcal{E}}}$
and
$\bar{\boldsymbol{W}}^{\mathcal{E}}_{C}\in\mathbb{C}^{\ell^{\mathcal{E}}\times\ell^{\mathcal{E}}}$
are also subject to optimization, but, unlike vector
$\boldsymbol{\omega}_{t}$, they are fixed at run time and they are not adapted
to the received signals. Therefore, the matrices
$\bar{\boldsymbol{W}}^{\mathcal{E}}_{A}$,
$\bar{\boldsymbol{W}}^{\mathcal{E}}_{B}$ and
$\bar{\boldsymbol{W}}^{\mathcal{E}}_{C}$ cannot account for the specific long-
term features of the channel in the current frame of $T$ time slots. We define
${\boldsymbol{\Omega}}^{\mathcal{E}}=\\{\bar{\boldsymbol{W}}_{A}^{\mathcal{E}},{\boldsymbol{b}}_{A}^{\mathcal{E}},\bar{\boldsymbol{W}}_{B}^{\mathcal{E}},{\boldsymbol{b}}_{B}^{\mathcal{E}},\bar{\boldsymbol{W}}_{C}^{\mathcal{E}}\\}$
as the set of optimization parameters for the channel estimation HyperRNN.

###  IV-4 Training

The proposed HyperRNN architecture is also trained using an end-to-end
approach that aims at minimizing the training squared error between the real
and estimated channel. The corresponding optimization problem can be
formulated as

$\displaystyle\underset{\boldsymbol{x}^{\textrm{ul}},\boldsymbol{X}^{\textrm{dl}},\boldsymbol{\Omega}^{\mathcal{Q}},\boldsymbol{\Omega}^{\mathcal{H}},\boldsymbol{\Omega}^{\mathcal{E}}}{\text{min}}$
$\displaystyle\sum_{t=1}^{T}E\left[\|\hat{\boldsymbol{h}}_{t}^{\textrm{dl}}-{\boldsymbol{h}}_{t}^{\textrm{dl}}\|^{2}\right]$
(7a) s.t. $\displaystyle\|\boldsymbol{X}^{\textrm{dl}}_{l}\|^{2}\leq
P^{\textrm{dl}},~{}\forall l=1,\dots,L^{\textrm{dl}},$ (7b)
$\displaystyle|x_{l}^{\textrm{ul}}|^{2}\leq P^{\textrm{ul}},~{}\forall
l=1,\dots,L^{\textrm{ul}},$ (7c)

where $P^{\textrm{dl}}$ and $P^{\textrm{ul}}$ are the transmit power
constraint at the BS and at the user side, respectively.
$\boldsymbol{X}^{\textrm{dl}}_{l}$ represents the $l$-th column of
$\boldsymbol{X}^{\textrm{dl}}$, while $x_{l}^{\textrm{ul}}$ is the $l$-th
element in $\boldsymbol{x}^{\textrm{ul}}$. The empirical distribution of a
training sample set
$\\{\boldsymbol{y}_{t^{\prime}}^{\textrm{dl}},\boldsymbol{h}_{t^{\prime}}^{\textrm{dl}},\boldsymbol{Y}_{t^{\prime}}^{\textrm{ul}},\boldsymbol{h}_{t^{\prime}}^{\textrm{ul}}\\}_{t^{\prime}=1}^{T}$
is used to approximate the expectation in (7a).

##  V System Performance

In this section, we characterise the performance of the proposed HyperRNN for
channel estimation.

_Implementation Details._ In this paper, we employ the spatial channel model
(SCM) standardized in 3GPP Release 16 [14], with the simulation parameters
summarised in Table I. The proposed HyperRNN is implemented using the standard
deep learning libraries TensorFlow and Keras, and we adopt the adaptive moment
estimation (Adam) optimizer with the mini-batch size of 1024 and a learning
rate gradually decreasing from $10^{-3}$ to $10^{-5}$. For the uplink feedback
DNN, $M^{\mathcal{Q}}=4$ dense layers are employed, with
$\ell^{\mathcal{Q}}_{1}=1024$, $\ell^{\mathcal{Q}}_{2}=512$,
$\ell^{\mathcal{Q}}_{3}=256$, and $\ell^{\mathcal{Q}}_{4}=B$ ReLU hidden
neurons. Furthermore, the RNN for channel estimation employs
$\ell^{\mathcal{E}}=\ell^{\mathcal{P}}=256$ ReLU neurons for each hidden
layer, whereas $\ell^{\mathcal{H}}=1024$ ReLU hidden neurons are used for the
hypernetwork. In order to satisfy the power constraint, we normalise the
updated $\boldsymbol{x}^{\textrm{ul}}$ or $\boldsymbol{X}^{\textrm{dl}}$ in
each iteration to ensure $|x_{l}^{\textrm{ul}}|^{2}=P^{\textrm{ul}}$ or
$\|\boldsymbol{X}^{\textrm{dl}}_{l}\|^{2}=P^{\textrm{dl}}$. We use the
normalized mean square error (NMSE) to characterise the channel estimation
performance, which is calculated as
$\textrm{NMSE}=E[\|\hat{\boldsymbol{h}}_{t}^{\textrm{dl}}-{\boldsymbol{h}}_{t}^{\textrm{dl}}\|^{2}/\|{\boldsymbol{h}}_{t}^{\textrm{dl}}\|^{2}].$

TABLE I: Simulation parameters Parameters | Values
---|---
Uplink carrier frequency ($f_{C}^{\textrm{ul}}$) | 3 GHz
Carrier frequency difference ($\Delta f_{C}$) | 100 MHz
Mobile velocity ($v$) | 30 km/h
Time slot duration ($\tau$) |  $0.1$ ms
No. of paths ($P$) | $2,4,8,16$
No. of TAs at the BS ($M$) | 64
AoDs ($\theta_{p}$) | $\theta_{p}\sim\mathcal{U}(-\pi/6,\pi/6)$
No. of uplink feedback bits ($B$) | $[5,30]$
No. of downlink pilots ($L^{\textrm{dl}}$) | $2$
No. of uplink pilots ($L^{\textrm{ul}}$) | $1,2,4$
Signal-to-noise ratio (SNR) | 10 dB

We compare the NMSE of the proposed HyperRNN for downlink channel estimation
with the baseline method DL-DNN proposed in [10] using different uplink pilot
lengths. In order to isolate the advantage of leveraging long-term partial
reciprocity extracted from the uplink, we assume i.i.d. fading amplitudes
$\beta_{p,t}^{\textrm{ul}}$ and $\beta_{p,t}^{\textrm{ul}}$ over different
time slots in this experiment, i.e., we set the temporal correlation
coefficients as $\rho^{\textrm{ul}}=\rho^{\textrm{dl}}=0$. We evaluate the
NMSE at the $t=8$-th time slot. Fig. 4 show that long-term partial reciprocity
can be leveraged to enhance channel estimation, even if a very short uplink
pilot sequence length with $L^{\textrm{ul}}=1$ is considered. When a longer
pilot sequence is employed, for example, $L^{\textrm{ul}}=4$ the NMSE
performance of HyperRNN is improved. The NMSE reduction is particularly
pronounced for longer values of the uplink feedback resolution $B$. This is
expected since a longer value of $B$ increases the input size to the downlink
channel estimation RNN, increasing the dimension of the output of the
hypernetwork.

Figure 4: NMSE of the HyperRNN and DL-DNN [10] for an FDD system with $M=64$
at $t=8$-th time slot, with different uplink pilot lengths
$L^{\textrm{ul}}=1,2$ and 4 over frequency-flat fading channels, where
$L^{\textrm{dl}}=2$, $P=8$, $\Delta f_{C}=100$ MHz, and
$\rho^{\textrm{ul}}=\rho^{\textrm{dl}}=0$. Figure 5: NMSE of the HyperRNN and
DL-DNN [10] over frequency-flat fading channels having different number $P$ of
paths for an $M=64$ FDD system with $B=20$,
${L}^{\textrm{ul}}={L}^{\textrm{dl}}=2$ and $\Delta f_{C}=100$ MHz, and
$\rho^{\textrm{dl}}=0.99$.

Fig. 5 demonstrates the NMSE of the proposed HyperRNN and of the benchmark DL-
DNN [10] for channel estimation of the FDD system having different number of
paths, $P$, where $t=8$, ${L}^{\textrm{ul}}={L}^{\textrm{dl}}=2$, and $\Delta
f_{C}=100$ MHz, and the rest parameters are summarised in Table I. Note that
the temporal correlations $\rho^{\textrm{ul}}$ and $\rho^{\textrm{dl}}$ are
not zero here. Larger performance gains can be achieved when the channel has a
lower number of paths. In fact, in this regime, the invariant of the long-term
features of the channel defines a low-rank structure of the channel that can
be leveraged by the hypernetwork.

##  VI Conclusions

In this paper, we have introduced an end-to-end trained CSI acquisition scheme
for massive MIMO FDD systems based on a novel HyperRNN architecture that
leverages both partial uplink-downlink reciprocity and temporal correlation of
fading processes. The proposed HyperRNN achieves a lower NMSE compared to
existing methods, particularly in sparse propagation environments. Ongoing
work extends the approach to beamforming design and frequency-selective
channels.

## Acknowledgement

The authors gratefully acknowledge Dr Rahif Kassab and Dr Dongzhu Liu for
their contributions to the early stage of this project in terms of
comprehensive literature review.

## References

  * [1] M. Arnold, S. Dörner, S. Cammerer, S. Yan, J. Hoydis, and S. t. Brink, “Enabling FDD massive MIMO through deep learning-based channel prediction,” arXiv preprint arXiv:1901.03664, 2019.
  * [2] M. Alrabeiah and A. Alkhateeb, “Deep learning for TDD and FDD massive MIMO: Mapping channels in space and frequency,” in Proc. 53rd Asilomar Conf. Signals, Syst., Comput., pp. 1465–1470, Pacific Grove, CA, USA, Nov. 2019.
  * [3] Y. Han, M. Li, S. Jin, C.-K. Wen, and X. Ma, “Deep learning-based FDD non-stationary massive MIMO downlink channel reconstruction,” IEEE J. Sel. Areas Commun., vol. 38, pp. 1980–1993, Apr. 2020.
  * [4] Y. Yang, F. Gao, Z. Zhong, B. Ai, and A. Alkhateeb, “Deep transfer learning-based downlink channel prediction for FDD massive MIMO systems,” IEEE Trans. on Commun., vol. 68, pp. 7485–7497, Aug. 2020.
  * [5] C.-K. Wen, W.-T. Shih, and S. Jin, “Deep learning for massive MIMO CSI feedback,” IEEE Wireless Commun. Lett., vol. 7, pp. 748–751, Oct. 2018\.
  * [6] Y. Jang, G. Kong, M. Jung, S. Choi, and I.-M. Kim, “Deep autoencoder based CSI feedback with feedback errors and feedback delay in FDD massive MIMO systems,” IEEE Wireless Commun. Lett., vol. 8, pp. 833–836, Jun. 2019.
  * [7] X. Zhang, L. Zhong, and A. Sabharwal, “Directional training for FDD massive MIMO,” IEEE Trans. Wireless Commun., vol. 17, pp. 5183–5197, Aug. 2018\.
  * [8] M. B. Khalilsarai, S. Haghighatshoar, X. Yi, and G. Caire, “FDD massive MIMO: Efficient downlink probing and uplink feedback via active channel sparsification,” in Proc. IEEE Int. Conf. Commun. (ICC), pp. 1–6, Kansas City, MO, USA, May 2018.
  * [9] Y. Han, Q. Liu, C.-K. Wen, S. Jin, and K.-K. Wong, “FDD massive MIMO based on efficient downlink channel reconstruction,” IEEE Trans. on Commun., vol. 67, pp. 4020–4034, Jun. 2019.
  * [10] F. Sohrabi, K. M. Attiah, and W. Yu, “Deep learning for distributed channel feedback and multiuser precoding in FDD massive MIMO,” IEEE Trans. Wireless Commun., pp. 1–1, 2021.
  * [11] D. Ha, A. Dai, and Q. V. Le, “Hypernetworks,” arXiv preprint arXiv:1609.09106, 2016.
  * [12] M. Goutay, F. A. Aoudia, and J. Hoydis, “Deep hypernetwork-based MIMO detection,” in Proc. IEEE Signal Process. Adv. Wireless Commun. (SPAWC), pp. 1–5, Atlanta, GA, USA, May 2020.
  * [13] T. Wang, C.-K. Wen, S. Jin, and G. Y. Li, “Deep learning-based CSI feedback approach for time-varying massive MIMO channels,” IEEE Wireless Commun. Lett., vol. 8, pp. 416–419, Apr. 2019.
  * [14] 3GPP TR 25.996 V16.0.0, “Spatial channel model for multiple input multiple output (MIMO) simulations (Release 16),” 3rd Generation Partnership Project Std., Jul. 2020.