# Deep Generative Models for Downlink Channel Estimation in FDD Massive MIMO
Systems

Javad Mirzaei⋆, Shahram ShahbazPanahi⋆†, Raviraj Adve⋆, and Navaneetha Gopal⋆
⋆Department of Electrical and Computer Engineering, University of Toronto,
Canada
⋆†Department of Electrical and Computer Engineering, Ontario Tech University,
Canada

###### Abstract

It is well accepted that acquiring downlink channel state information in
frequency division duplexing (FDD) massive multiple-input multiple-output
(MIMO) systems is challenging because of the large overhead in training and
feedback. In this paper, we propose a deep generative model (DGM)-based
technique to address this challenge. Exploiting the partial reciprocity of
uplink and downlink channels, we first estimate the frequency-independent
underlying channel parameters, i.e., the magnitudes of path gains, delays,
angles-of-arrivals (AoAs) and angles-of-departures (AoDs), via uplink
training, since these parameters are common in both uplink and downlink. Then,
the frequency-specific underlying channel parameters, specifically, the phase
of each propagation path, are estimated via downlink training using a very
short training signal. In the first step, we incorporate the underlying
distribution of the channel parameters as a prior into our channel estimation
algorithm. We use DGMs to learn this distribution. Simulation results indicate
that our proposed DGM-based channel estimation technique outperforms, by a
large gap, the conventional channel estimation techniques in practical ranges
of signal-to-noise ratio (SNR). In addition, a near-optimal performance is
achieved using only few downlink pilot measurements.

##  I Introduction

###  I-A Motivation

Massive multiple-input multiple-output (MIMO) is a key technology in helping
to meet the demands to be made of the next (fifth) generation of wireless
technologies [1, 2]. This technology can be deployed in time-division duplex
(TDD) mode, where the uplink and downlink communication occur in the same
frequency band but at different time slots, and frequency-division duplex
(FDD), where the uplink and downlink operate simultaneously on different
frequency bands. To unlock the full potential of massive MIMO in both FDD and
TDD, we require an accurate channel state information (CSI) between the base
station (BS) and the user equipment (UE).

In TDD massive MIMO systems, CSI acquisition relies on the assumption of
channel reciprocity between the uplink and downlink. However, due to
calibration errors between the uplink and downlink radio frequency (RF)
chains, such a channel reciprocity may not hold [3]. Additionally, pilot
contamination, caused by the use of non-orthogonal pilot signals in
neighboring cells, is another performance-limiting factor in TDD-based systems
[4]. Importantly, realizing the backward compatibility of FDD massive MIMO
communication (e.g., with Long-Term Evolution (LTE)) has drawn considerable
attention toward FDD from both academia and industry [5].

In FDD, due to different band of frequencies in the uplink and downlink, the
downlink channel is neither the same, nor can it be inferred from the uplink
channel without any downlink training. Traditionally, a UE estimates its own
downlink channel from the received pilot symbols transmitted from the BS and
feeds the estimated downlink channel information back to the BS for the
subsequent signal transmission and resource allocation. This approach is
practical in current (prior to 5G) generation of networks, where only a few
antennas are used at the BS, allowing for orthogonal pilots and a small
feedback overhead. However, in massive MIMO systems, due to the large overhead
with the use of orthogonal pilots in massive arrays, as well as the huge
required feedback, this approach may not be applicable. Therefore, it is an
urgent requirement in FDD massive MIMO systems to reduce the amount of pilot
symbols needed for downlink channel estimation.

###  I-B Related Work

The previous attempts in FDD channel estimating mainly focused on adapting the
downlink arrays using the second-order statistics of the uplink
measurements[6, 7]. In recent years and in the context of massive MIMO, there
are attempts aiming at reducing the feedback overhead by exploiting the
sparsity in the underlying channel parameters. In particular, considering a
parametric channel model, where the channel is characterized by the parameters
of only a few dominant paths, such as gains, direction-of-arrival (DoA), and
direction-of-departure (DoD)[8]. Given such a model, one may consider array
processing techniques (such as SAGE [9]) in estimating the channel parameters.
However, these techniques are mainly based on the EM algorithm which requires
the user to know the likelihood function of pilot observations and the channel
parameters, which may not be available. Also, none of the array processing
techniques [10, 11] has been adopted in the context of FDD downlink channel
estimation. The reason is that these methods usually work under certain
assumptions, (for example the number of multipaths to be smaller than the
number of transmit and the number of receive antennas) which may not be
satisfied in FDD systems [12]. Even employing DoA estimation techniques is
challenging. Most DoA estimation techniques rely on the covariance matrix of
the received signal. Constructing (and then inverting) such covariance
matrices takes a lot of computation effort which limits the applicability of
these techniques in fast fading scenarios. These techniques also usually
require the number of paths be lower than the number of antennas.

Considering millimeter wave (mmWave) frequencies, the channel tends to exhibit
sparsity in the angular domain [13, 14]. Leveraging this sparsity, the channel
can be reformulated using the channel gains, DoAs, and DoDs. One can recover
these underlying channel parameters using compressive sensing (CS) techniques.

In a CS framework, it is assumed that the UE compresses and feeds back the
received pilot signal to the BS, where different CS-based techniques can be
employed to recover the sparse channel parameters [13, 15, 16, 17, 18, 19,
20]. In a multi-user massive MIMO setting, the authors of [17] develop a joint
orthogonal matching pursuit (OMP) algorithm to recover the channel parameters
at the BS. To further reduce the pilot overhead, block OMP is developed in
[21] based on a user grouping technique, which relies on the assumption that
the users in each group have the same channel correlation matrix. The authors
of [22] proposed a sparse Bayesian learning technique for sparse channel
recovery followed by an off-grid refinement. Note that in a practical setting,
and due to the limited scattering in propagation environment, different user
links tend to share some common scatterers. While each UE channel matrix is
sparse in angular domain, they may exhibit a common sparsity pattern. This
observation motivated the authors of [23] to propose a variational Bayesian
inference based approach for channel estimation, where Gaussian mixture model
is exploited to capture the individual sparsity in each channel matrix. There
are works that attempt to improve the performance of CS-based techniques by
incorporating a priori information into the recovery algorithm [24, 25, 26].
It is worth mentioning that the CS-based techniques still suffer from some
limitations. In particular, they require strong channel sparsity in DFT-basis
which is not strictly held in some cases. In fact, we do not even know the
basis that yields the most sparse representation. While they also require a
large number of pilots, they are often iterative and computationally intensive
during decoding, which may lead to long delays.

As an alternative to CS-based techniques, there are techniques relying on the
spatial reciprocity between the uplink and downlink channels [20, 19, 27],
operating on close-by carrier frequencies have been proposed. Given the fact
that the uplink and downlink communication occur in the same propagation
environment, the uplink channel estimates can be used in the estimation of
downlink channel. In this way, difficulties in downlink channel estimation is
shifted to the BS.

Uplink-downlink reciprocity can be considered in two ways. In one way, one can
assume full-reciprocity where the multi-path components of channel (including
phase, amplitude, delay, angle of arrival, departure, etc.) are the same in
both uplink and downlink [28, 29, 30]. Based on this assumption, the authors
of [30], proposed to completely eliminate downlink training and feedback in
LTE systems. However, there is not enough evidence to confirm such a full-
reciprocity assumption in real world data. Indeed, it is shown via
measurements and theoretical investigations that not all channel parameters
are reciprocal in uplink and downlink. In particular, there is no reciprocity
between the phases of different multi-path components for FDD uplink and
downlink channels [31]. Intuitively, this is expected given the sensitivity of
the phase to operating frequency. Alternatively, uplink and downlink channels
are only _partially_ reciprocal. This implies that downlink training and
feedback is inevitable. Partial reciprocity has been leveraged differently in
the literature. In [32], the authors consider a spatial domain representation
of channel111The channel is expressed by a small number of multi-path
component such as AoA/AoDs and path gains, and exploit the uplink-downlink
angle reciprocity [33], where the AoA/AoD in the uplink and downlink are
assumed to be the same. Then, the downlink channel estimation boils down to
path gain estimation which can be done within a much lower pilot overhead.

As an alternative approach, deep learning (DL)-based techniques have been
considered in recent studies published on channel estimation for FDD-based
massive MIMO. Deep learning is a powerful tool to explore the underlying
complex structure of data. This type of learning has been widely applied in
various wireless communication problems, such as data detection [34, 35],
channel estimation [36], beamforming [37], and hybrid precoding [38, 39], and
have shown promising performance compared to conventional techniques. In the
context of downlink channel estimation in FDD systems, in [40], the whole
downlink CSI is estimated using only the CSI obtained over a small set of BS
antennas via linear regression and support vector machines (SVM). This is
based on the assumption that the channel among different BS antennas is
correlated. In the same line of the work, the authors of [41], proposed a deep
neural network (DNN)-based technique to map the channel both in space and
frequency. This allows for the prediction of the channel of a set of BS
antennas and a frequency band from the observation of different set of
antennas and different frequency band. Inspired by the position-to-channel
mapping investigated in [41], the authors of [42] proposed a complex-valued
neural network (named SCNet) to directly map the uplink channel to its
downlink counterpart, without requiring any downlink pilot transmission. A
similar technique with reduced complexity has been proposed in [43].

###  I-C Contribution and Methodology

In this paper, we consider a single cell where a BS with massive number of
antennas communicates with a UE in FDD mode. Using the fact that the channel
matrix over each subcarrier is a function of a smaller set of parameters,
namely, the number of propagation paths, the path gains, phases, delays, as
well as AoAs and AoDs [8], we estimate these parameters instead of the channel
matrix directly. These parameters depend on the physical properties of the
propagation environment and on the operating frequencies, and importantly,
they are independent of the number of antennas at the BS as well as the number
of subcarriers [44, 45, 46, 47]. Unlike the conventional techniques, where a
long training sequence is transmitted over all antennas and over all
subcarriers, we estimate these underlying channel parameters using a short
training signal over a much smaller set of antennas and subcarriers.

Motivated by the partial reciprocity of uplink and downlink channels [31, 32,
48], we use the following steps to estimate the downlink channel: I) we
estimate the frequency-independent underlying channel parameters, namely, the
magnitudes of path gains, delays, AoAs and AoDs during the uplink training;
and II) the frequency-specific underlying channel parameters, i.e., the phase
of each propagation path, are estimated via downlink training. Using this
strategy, we shift the burden in FDD downlink channel estimation to the BS,
which is, anyway, responsible for uplink channel estimation. In the first
step, we use the least squares (LS) estimation approach to estimate the
frequency-independent parameters. The optimization problem in this step is
difficult to solve analytically, mainly due to non-linear and non-convex
structure of its objective function. To address this problem, we use _deep
generative models (DGMs)_ 222In particular, we use the generative adversarial
network (GAN), to capture the distribution of the underlying parameters, and
then use it as a prior to simplify the optimization problem. In the second
step, we use the frequency-independent parameters, estimated in the first
step, to estimate the frequency-specific parameters via an LS technique. In
both steps, the optimization problem is carried out numerically using the
gradient descent algorithm. The contributions of this paper are itemized
below:

  * •

Learning the distribution of channel parameters: The unknown underlying
distribution of the channel parameters is some function of the propagation
environment, and therefore, it is complex and difficult to obtain
analytically. We explain how to use DGMs to learn this distribution. In
particular, we use the GAN structure to find a deterministic mapping function
(i.e., a generator) that is capable of drawing samples from the underlying
distribution of channel parameters, by feeding it with samples from a low-
dimensional standard Gaussian distribution.

  * •

FDD downlink channel estimation: We present a technique to show how to exploit
the uplink-downlink partial reciprocity, thereby reducing the pilot and
feedback overhead in downlink. In addition, the sparsity assumption in channel
parameters is relaxed. Instead, using a generator, obtained from a GAN, we
incorporate the learned structure of channel parameters as a _prior_ into our
channel estimation procedure. By doing so, the optimization problem operates
in a low-dimensional subspace, whose dimensionality is defined by the
generator and, importantly, it is independent of number of received pilots.
Therefore, we achieve a significant reduction in computational complexity as
well as CSI feedback overhead.

  * •

Convergence analysis: We analytically prove the convergence of our estimation
by showing that the gradient of the estimation objective function is
Lipschitz-continuous, and therefore, the convergence of steepest descent
algorithm is guaranteed.

The simulation results indicate that our proposed DGM-based channel estimation
outperforms the conventional channel estimation technique in practical ranges
of signal-to-noise ratio (SNR). This is mainly due to the capabilities of the
generator in representing the underlying distribution of the channel
parameters. Incorporating this prior knowledge into our channel estimation
significantly improves the performance even at low SNR. This indicates how the
proposed technique is resilient to the noise level. Additionally, for fixed
SNR, we show that the proposed technique yields a near-optimal performance
using only few pilot measurements. This can significantly reduce the pilot
overhead in FDD massive MIMO systems.

Our work in this paper differs from the CS-based techniques in the sense that
we do not assume any sparsity in the underlying channel parameters. Instead,
we relax this constraint and consider that the channel parameters have a
particular structure which is not necessarily sparse. We capture this
structure using a generator, and then, incorporate it as a prior into our
channel estimation process to improve the accuracy and reduce the pilot
overhead. The DNN-based techniques in [41, 42, 49, 43] mainly rely on direct
channel mapping between the uplink and downlink, without requiring any
downlink training. The performance of these techniques is highly affected by
the uplink-downlink frequency separation, hardware impairments, shadow fading,
and SNR. While we have not shown numerically, _in theory_ , our proposed
technique can address such deficiencies by incorporating a DGM-learned prior
into channel estimation process. Unlike the work in [50], in this paper, we
study the channel estimation problem in FDD massive MIMO systems. Furthermore,
instead of the channel distribution, we learn the underlying distribution of
channel parameters.

_Organization_ : We introduce the system model, including the received signal
model, channel model, and FDD partial reciprocity in Section II. Section III
introduces the channel estimation problem, where we formulate the problem
followed by our estimation technique. For the sake of the paper being self-
contained, DGMs are briefly introduced in Section IV. In Section V, we
analytically study the convergence, complexity and identifiability of our
estimation algorithm. Section VI presents implementation details and
simulation results to illustrate the efficacy of the proposed technique.
Section VII provides conclusions. Finally, Appendix A provides some
preliminary definitions and lemmas used to prove Lemma 1 in Appendix B.

_Notation_ : We use bold upper and lower-case letters to denote matrices and
vectors, respectively. $\mathbb{E}_{{\bf x}}\left[\cdot\right]$ denotes
statistical expectation over the random variable ${\bf x}$. ${\bf
z}\backsim\mathbb{P}_{z}({\bf z})$ denotes that the random vector ${\bf z}$
follows the probability distribution (pdf) of $\mathbb{P}_{z}({\bf z})$ . The
transpose operation is represented as $(\cdot)^{T}$; ${\bf I}_{N}$ denotes an
$N\times N$ identity matrix; $\mathbb{R}_{+}$ denotes the set of non-negative
real numbers; $\|\cdot\|$ denotes the norm-2 operation; $|x|$ represents the
absolute value of $x$; ${\bf X}\preccurlyeq{\bf Y}$ means $\|{\bf
X}\|\leq\|{\bf Y}\|$; $|\mathcal{I}|$ represents the cardinality of set
$\mathcal{I}$; and ${\mathcal{{I}}^{c}}$ the compliment set of
${\mathcal{{I}}}$; ${\mathcal{\bar{I}}}$ represents a sorted array whose
entries are in the set $\mathcal{I}$. We use $\Big{[}{\bf
x}_{i}\Big{]}_{i\in\mathcal{I}}$ to denote a vector whose $j$-th block entry,
for $j=1,2,\cdots,|\mathcal{I}|$, is given by ${\bf x}_{j}$, for
$j\in{\mathcal{\bar{I}}}$. In a similar way, $\Big{[}{\bf
X}_{i}\Big{]}_{i\in\mathcal{I}}$ denotes a matrix whose $j$-th block entry,
for $j=1,2,\cdots,|\mathcal{I}|$, is given by ${\bf X}_{j}$, for
$j\in{\mathcal{\bar{I}}}$.

##  II System Model

We consider a single-cell single-user communication system. The BS is equipped
with a uniform linear array (ULA) with $M\gg 1$ antenna elements, while the UE
is single-antenna. The communication between the BS and UE is performed in FDD
mode. In the uplink, the UE communicates with the BS at frequency $f_{\rm
up}$, while in the downlink, the BS communicates with the UE at frequency
$f_{\rm dl}$. The frequency difference between $f_{\rm up}$ and $f_{\rm dl}$
is assumed to be relatively small. Both uplink and downlink frequency bands
are of bandwidth $B$.

###  II-A Received Signal Model

We assume that orthogonal frequency division duplex (OFDM) technology is used
in both uplink and downlink commutation with $K$ subcarriers. Let
$\mathcal{K}_{\rm up}$ denote the set of subcarrier indices used for uplink
training. During the uplink training at the $k$-th subcarrier, the UE
transmits training symbol $s_{k}$, $k\in\mathcal{K}_{\rm up}$, where
$|s_{k}|^{2}=P_{T}$, and $P_{T}$ is the transmit power. The received signal at
BS over the $k$-th subcarrier is given by

$\displaystyle{\bf y}^{\rm up}_{k}\triangleq{\bf h}_{k}^{\rm up}s_{k}+{\bf
n}^{\rm up}_{k},\,\,\,k\in\mathcal{K}_{\rm up},$ (1)

where ${\bf h}_{k}^{\rm up}$ is an $M\times 1$ uplink channel vector between
the BS and the UE and ${\bf n}^{\rm up}_{k}$ is an $M\times 1$ noise vector at
the $k$-th subcarrier that is drawn independently and identically from a
complex Gaussian distribution with zero mean and variance $\sigma_{\rm
n}^{2}$.

In the downlink, let $\mathcal{K}_{\rm dl}$ and $\mathcal{M}_{\rm dl}$ denote,
respectively, the set of subcarrier indices and the set of antenna elements
dedicated for training. We assume that $p$ training symbols are transmitted
over the $m$-th antenna ($m\in\mathcal{M}_{\rm dl}$) over the $k$-th
subcarrier ($k\in\mathcal{K}_{\rm dl}$). The received signal at the UE over
the $k$-th subcarrier at the $i$-th time slot is given by

$\displaystyle y^{(i),\rm dl}_{k}\triangleq{\bf s}^{(i),\rm dl}_{k}{{\bf
h}_{k}^{\rm dl}}+n^{(i),\rm dl}_{k},\,\,\,k\in\mathcal{K}_{\rm
dl},\;i=1,2,\cdots,p,$ (2)

where ${\bf h}_{k}^{\rm dl}$ is an $|\mathcal{M}_{\rm dl}|\times 1$ downlink
channel vector at the $k$-th subcarrier between the BS and the UE, ${\bf
s}^{(i),\rm dl}_{k}\triangleq\left[s^{(i),\rm dl}_{k,1}\;\;s^{(i),\rm
dl}_{k,2}\;\;\cdots\;\;s^{(i),\rm dl}_{k,|\mathcal{M}_{\rm dl}|}\right]$ is
the $1\times|\mathcal{M}_{\rm dl}|$ vector of downlink training symbols
transmitted at the $i$-th time slot over the $k$-th subcarrier across all
antenna elements in the set $\mathcal{M}_{\rm dl}$. We assume that $\|{\bf
s}^{(i),\rm dl}_{k}\|^{2}=P_{T}$. $n^{(i),\rm dl}_{k}$ denotes the noise term
at the $i$-th time slot over the $k$-th subcarrier, drawn independently and
identically from a complex Gaussian distribution with zero mean and variance
$\sigma_{\rm n}^{2}$. Note that, in general $|\mathcal{M}_{\rm dl}|\leq M$.
For the case when $|\mathcal{M}_{\rm dl}|<M$, we assume that the antenna
elements in the set $\mathcal{M}_{\rm dl}^{c}$ do not transmit during the
downlink training.

Collecting the received signal across all $p$ training time slots over the
$k$-th subcarrier, we can write

$\displaystyle{\bf y}^{\rm dl}_{k}\triangleq{\bf S}^{\rm dl}_{k}{{\bf
h}_{k}^{\rm dl}}+{\bf n}^{\rm dl}_{k},\,\,\,k\in\mathcal{K}_{\rm dl},$ (3)

where ${\bf S}^{\rm dl}_{k}$ is a $p\times|\mathcal{M}_{\rm dl}|$ matrix of
downlink training symbols with ${\bf s}^{(i),\rm dl}_{k}$ on its $i$-th row,
$i=1,2,\cdots,p$, ${\bf y}^{\rm dl}_{k}\triangleq\left[y^{(1),\rm
dl}_{k}\;\;y^{(2),\rm dl}_{k}\;\;\cdots\;\;y^{(p),\rm dl}_{k}\right]^{T}$, and
${\bf n}^{\rm dl}_{k}\triangleq\left[n^{(1),\rm dl}_{k}\;\;n^{(2),\rm
dl}_{k}\;\;\cdots\;\;n^{(p),\rm dl}_{k}\right]^{T}$. Next, we present our
channel model.

###  II-B Channel Model

To characterize the wireless channel between the BS antenna array and the UE,
we consider the following geometric channel model. We assume that the
propagation channel between the BS and the UE in the uplink consists of
$L^{\rm up}$ paths. Through the $l$-th path, the signal travels the distance
$d_{l}$ between the UE and the BS. Also, let $\alpha_{l}^{\rm
up}\in\mathbb{R}_{+}$, $\phi_{l}^{\rm up}\in[0,2\pi]$, $\theta_{l}^{\rm
up}\in[0,2\pi]$, and $\tau_{l}^{\rm up}\in\mathbb{R}_{+}$, for
$l=1,2,\ldots,L^{\rm up}$, denote the random path gain, the random phase
change, the random azimuth angle of the signal received, and the random delay
corresponding to the $l$-th path in the uplink, respectively. Using this
notation, the channel response between the UE and the BS at the $k$-th
subcarrier is given by [19, 51]

$\displaystyle{\bf h}_{k}^{\rm up}=\sum_{l=1}^{L^{\rm up}}\alpha_{l}^{\rm
up}e^{j\left(\phi_{l}^{\rm up}+\frac{2\pi k}{K}\tau_{l}^{\rm up}B\right)}{\bf
a}(\theta_{l}^{\rm up},\lambda_{k}^{\rm up}),\,\,\,k\in\mathcal{K}_{\rm up},$
(4)

where $\lambda_{k}^{\rm up}\triangleq\frac{c}{f_{c}^{\rm up}+kB/K}$ is the
wavelength of the $k$-th subcarrier in the uplink, and $c$ is the speed of
light 333Note that the carrier phase shift $2\pi f_{c}^{\rm up}\tau_{l}$ is
absorbed in the random phase $\phi_{l}^{\rm up}$.. Since $kB/K$ is very small
compared to $f_{c}^{\rm up}$, we ignore the subcarrier index in the array
response ${\bf a}(\theta_{l},\lambda)$. Denoting $\bar{d}$ as the antenna
spacing in the ULA, the array response in (4) is given by

$\displaystyle{\bf
a}(\theta_{l},\lambda)\triangleq\left[1\;\;e^{j\frac{2\pi}{\lambda}\bar{d}\sin{\theta_{l}}}\;\;\cdots\;\;e^{j\frac{2\pi}{\lambda}\bar{d}(M-1)\sin{\theta_{l}}}\right]^{T}.$
(5)

Similarly, the downlink communication channel at the $k$-th subcarrier is
given by

$\displaystyle{\bf h}_{k}^{\rm dl}=\sum_{l=1}^{L^{\rm dl}}\alpha_{l}^{\rm
dl}e^{j\left(\phi_{l}^{\rm dl}+\frac{2\pi k}{K}\tau_{l}^{\rm dl}B\right)}{\bf
b}(\theta_{l}^{\rm dl},\lambda^{\rm dl}),\,\,\,\,k\in\mathcal{K}_{\rm dl},$
(6)

where $L^{\rm dl}$ is number of path in the downlink, $\lambda^{\rm
dl}\triangleq\frac{\nu}{f_{c}^{\rm dl}}$ is the wavelength of the downlink
carrier frequency, $\alpha_{l}^{\rm dl}\in\mathbb{R}_{+}$, $\phi_{l}^{\rm
dl}\in[0,2\pi]$, $\theta_{l}^{\rm dl}\in[0,2\pi]$, and $\tau_{l}^{\rm
dl}\in\mathbb{R}_{+}$, for $l=1,2,\ldots,L^{\rm dl}$, respectively denote the
random path gain, the random phase change, the random azimuth angle of the
received signal and the random delay corresponding to the $l$th path in the
downlink. ${\bf b}(\theta_{l},\lambda)$ is a subvector of ${\bf
a}(\theta_{l},\lambda)$ where its $i$th entry is
$e^{j\frac{2\pi}{\lambda}\bar{d}(\mathcal{M}_{\rm
dl}^{i}-1)\sin{\theta_{l}}}$, with $\mathcal{M}_{\rm dl}^{i}$ being the $i$th
smallest member of set $\mathcal{M}_{\rm dl}$.

###  II-C FDD Partial Reciprocity

In FDD communication, since uplink and downlink communication between the BS
and UE occurs over different frequency bands, reciprocity between ${\bf
h}_{k}^{\rm up}$ and ${\bf h}_{k}^{\rm dl}$ does not hold in general. However,
since uplink and downlink channels share the same propagation environment, it
has been shown that partial reciprocity exists between uplink and downlink
channels [19].

It is observed via measurements, and verified using theoretical analysis that
a portion of uplink and downlink channel parameters are frequency-independent
[31]. Specifically, since the signal of each propagation path travels the same
distance at the same speed in both uplink and downlink communication link, the
delay of each propagation path is the same in both uplink and downlink, i.e.,
$\tau_{l}^{\rm dl}=\tau_{l}^{\rm up}\triangleq\tau_{l}$ [31]. Furthermore, it
is shown, via both the measurement and ray tracing simulations, that the
directional angle of each communication path are the same in both uplink and
downlink, i.e., $L^{\rm dl}=L^{\rm up}\triangleq L$, $\theta_{l}^{\rm
dl}=\theta_{l}^{\rm up}=\theta_{l}$[48, 31]. It is also shown that, while the
gain of each communication path is not exactly the same in both uplink and
downlink in general, the downlink path gains are very close to that of uplink.
According to Figs. 5 and 6 of [31], the average power delay profile of the
uplink and downlink is shown to be approximately the same. Our analysis on the
DeepMIMO dataset [51] also indicates that the _relative_ error between
$\alpha_{l}^{\rm dl}$ and $\alpha_{l}^{\rm up}$ is only $0.8\%$. That is why,
in this paper, we use the following approximation $\alpha_{l}^{\rm
dl}\approx\alpha_{l}^{\rm up}=\alpha_{l}$. On the other hand, the existing
measurements and analysis does not provide enough evidence that $\phi_{l}^{\rm
dl}$ and $\phi_{l}^{\rm up}$ to be the same. This implies that the channel
translation requires downlink training.

##  III Channel Estimation

Leveraging the partial reciprocity of channel parameters in FDD, we develop a
strategy for downlink channel estimation as explained in the sequel. We aim to
estimate the frequency-independent parameters during the uplink training,
while the frequency-specific channel parameters are being estimated via
downlink training. To better characterize the channel estimation process, let
us define
${\boldsymbol{\alpha}}\triangleq[\alpha_{1}\;\;\alpha_{2}\;\;\ldots\;\;\alpha_{L}]^{T}$,
$\boldsymbol{\phi}^{\rm up}\triangleq[\phi_{1}^{\rm up}\;\;\phi_{2}^{\rm
up}\;\;\ldots\;\;\phi_{L}^{\rm up}]^{T}$, $\boldsymbol{\phi}^{\rm
dl}\triangleq[\phi_{1}^{\rm dl}\;\;\phi_{2}^{\rm
dl}\;\;\ldots\;\;\phi_{L}^{\rm dl}]^{T}$,
$\boldsymbol{\tau}\triangleq[\tau_{1}\;\;\tau_{2}\;\;\ldots\;\;\tau_{L}]^{T}$,
and
${\boldsymbol{\theta}}=[\theta_{1}\;\;\theta_{2}\;\;\ldots\;\;\theta_{L}]^{T}$,
while the tuples

$\displaystyle{\bf x}^{\rm up}$ $\displaystyle\triangleq\left({\bf
x},{\boldsymbol{\phi}^{\rm
up}}\right)\triangleq\left({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}},{\boldsymbol{\phi}^{\rm
up}}\right),$ $\displaystyle{\bf x}^{\rm dl}$
$\displaystyle\triangleq\left({\bf x},{\boldsymbol{\phi}^{\rm
dl}}\right)\triangleq\left({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}},{\boldsymbol{\phi}^{\rm
dl}}\right),$ (7)

capture the uplink and downlink channel parameters, respectively. In order to
estimate ${\bf h}_{k}^{\rm dl}$, for $k=1,2,\cdots,K$, we estimate frequency-
independent channel parameters, namely ${\boldsymbol{\alpha}}$,
$\boldsymbol{\tau}$, and ${\boldsymbol{\theta}}$ using uplink training as
these parameters are common in both uplink and downlink channels, while
${\boldsymbol{\phi}}^{\rm dl}$ is estimated during downlink training with a
much less training overhead. The details are presented in the next two
subsections.

###  III-A Uplink Training

Here, we aim to estimate ${\bf x}^{\rm up}$ using the uplink training. To do
so, we stack the observations across all subcarriers as ${\bf y}^{\rm
up}\triangleq\Big{[}{\bf y}^{\rm up}_{k}\Big{]}_{k\in\mathcal{K}_{\rm up}}$,
and defining $\mathcal{A}({\bf x}^{\rm up})\triangleq\Big{[}{\bf h}_{k}^{\rm
up}s_{k}\Big{]}_{k\in\mathcal{K}_{\rm up}}$ as well as ${\bf n}^{\rm
up}\triangleq\Big{[}{\bf n}^{\rm up}_{k}\Big{]}_{k\in\mathcal{K}_{\rm up}}$,
the collected received signal in the uplink is expressed as

$\displaystyle{\bf y}^{\rm up}=\mathcal{A}({\bf x}^{\rm up})+{\bf n}^{\rm
up},$ (8)

where $\mathcal{A}({\bf x}^{\rm up})$ is a non-linear function of ${\bf
x}^{\rm up}$. Using the LS estimation approach, we obtain the estimate of
${\bf x}^{\rm up}$ by solving the following minimization problem:

$\displaystyle\min_{{\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}},{\boldsymbol{\phi}^{\rm
up}}}$ $\displaystyle J_{\rm
up}\left({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}},{\boldsymbol{\phi}^{\rm
up}}\right)$ $\displaystyle{\rm s.t.}$
$\displaystyle\,\,\,\,{\boldsymbol{\alpha}}\succcurlyeq
0,\,\,\,\,\boldsymbol{\tau}\succcurlyeq 0$
$\displaystyle\,\,\,\,0\preccurlyeq{\boldsymbol{\theta}}\preccurlyeq
2\pi,\,\,\,\,0\preccurlyeq{\boldsymbol{\phi}^{\rm up}}\preccurlyeq 2\pi,$ (9)

where $J_{\rm
up}\left({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}},{\boldsymbol{\phi}^{\rm
up}}\right)\triangleq\|{\bf y}^{\rm
up}-\mathcal{A}\left({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}},{\boldsymbol{\phi}^{\rm
up}}\right)\|_{2}^{2}$. The optimization problem (III-A) is difficult to solve
analytically due to non-linear structure of the objective function. Even
solving (III-A) numerically (using for example coordinate descent or gradient
projection [52]) is challenging mainly because of the high-dimensionality of
the search space as well as the fact that the objective function is not
convex. Therefore, any numerical approach will suffer from the convergence
issues. To alleviate such difficulties, instead of solving (III-A) directly,
we use an approach which is based on DGMs. Variational auto-encoders (VAEs)
and GANs are well-known examples of DGMs. In this technique, we replace the
tuple
$\left({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}}\right)$
by $G({\bf z})$, which is a mapping from ${\bf z}$ (a latent variable) to the
domain of
$\left({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}}\right)$.
In doing so, we implicitly assume that the tuple
$\left({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}}\right)$
is described in terms of a low-dimensional ${\bf z}$ through $G(\cdot)$ \- a
vector valued function of a vector valued variable. That is, the tuple
$\left({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}}\right)$
has a structure dictated by $G({\bf z})$. Here, we assume that the entries of
vector ${\bf z}$ are random variables from a Gaussian distribution (i.e.,
${\bf z}\sim\mathcal{N}({\bf 0},{\bf I}_{d})$, where $d$ is the length of
vector ${\bf z}$). We will elaborate later on how to find $G({\bf z})$ using a
GAN architecture.

Given $G({\bf z})$, we rewrite our LS problem as

$\displaystyle\min_{{\bf z},{\boldsymbol{\phi}^{\rm up}}}$ $\displaystyle
J_{\rm up}\left(G({\bf z}),{\boldsymbol{\phi}^{\rm up}}\right)$
$\displaystyle{\rm s.t.}$
$\displaystyle\,\,\,\,0\preccurlyeq{\boldsymbol{\phi}^{\rm up}}\preccurlyeq
2\pi.$ (10)

Note that (III-A) and (III-A) are not equivalent, since solving (III-A)
results in tuples
$\left({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}}\right)$
which belong to the range of $G({\bf z})$, whereas in (III-A) there is no such
restriction. However, given the above assumption, theoretically from (III-A)
to (III-A), there is no optimality loss, as the optimal
$\left({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}}\right)$
in (III-A) is on the manifold that $G({\bf z})$ generates samples on.
Therefore, the optimal solution to (III-A) is in the range of $G({\bf z})$.
Note that the objective function in (III-A) is a periodic function with
respect to ${\boldsymbol{\phi}^{\rm up}}$, i.e., $J_{\rm up}\left(G({\bf
z}),{\boldsymbol{\phi}^{\rm up}}\right)=J_{\rm up}\left(G({\bf
z}),{\boldsymbol{\phi}^{\rm up}+2i\pi}\right)$ for integer $i$. This implies
that if ${\boldsymbol{\phi}^{\rm up}}^{*}$ is an optimal solution out of the
interval $\left[0,2\pi\right]$, without loss of optimality, we can project it
back to $\left[0,2\pi\right]$ by removing multiple integers of $2\pi$.
Therefore, we can ignore the constraint in (III-A) and solve the following
unconstrained optimization problem:

$\displaystyle\min_{{\bf z},{\boldsymbol{\phi}^{\rm up}}}J_{\rm
up}\left(G({\bf z}),{\boldsymbol{\phi}^{\rm up}}\right).$ (11)

To efficiently solve (11), we jointly and iteratively update ${\bf z}$ and
${\boldsymbol{\phi}^{\rm up}}$ using the gradient descent algorithm. To do so,
we need to know the mapping $G(\cdot)$. Later we show how this mapping can be
determined using the GAN architecture.

Remark 1: Despite the non-linearity and possibly the non-convex structure of
the optimization problem in (III-A), solving (11) is easier. This can be
explained as follows: 1) The tuple
$\left({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}}\right)$
is described in terms of a low-dimensional ${\bf z}$ via a vector valued
function $G(\cdot)$, and therefore, the search space in (11) is in ${\bf z}$
and $\boldsymbol{\phi}^{\rm up}$ which has a much smaller dimension than the
optimization variables in (III-A). 2) We will show in Lemma 1 in Section V-A
that the objective function of (11) is a Lipschitz-continuous function of its
variables. Therefore, the convergence of steepest descent algorithm in solving
(11) is guaranteed. This fact does not necessarily hold true for the objective
function in (III-A) due to a larger search space as well as the box constraint
over each dimension. 3) Unlike (III-A), in (11), we incorporate the underlying
distribution of channel parameters to act as a prior. This not only eliminates
the box constraints in (III-A), but also provides a better estimation of
channel parameters even with highly noisy observations. In our simulations,
and for the sake of comparison, we solve (III-A) based on the R2F2 algorithm
[53]. We will show how incorporating $G({\bf z})$ improves the performance.

Remark 2: The distribution of channel parameters
$(\boldsymbol{\alpha},\boldsymbol{\tau},\boldsymbol{\theta})$ depends on
physical properties of the propagation environment (such as geometry and
material of surrounding objects), operating frequencies, user densities as
well as data traffic. Importantly, they are independent of the number of
antennas at the BS as well as the number of subcarriers [44, 45, 46, 47].
Here, _we use $G({\bf z})$ to model the statistical distribution of
$(\boldsymbol{\alpha},\boldsymbol{\tau},\boldsymbol{\theta})$_. Note that each
BS uses only one $G({\bf z})$. For a BS in a different propagation
environment, $G({\bf z})$ needs to be retrained.

###  III-B Downlink Training

Relying on the partial reciprocity of channel parameters in FDD, we use the
frequency-independent channel parameters (i.e.,
$\left({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}}\right)$
that is estimated during the uplink training), and estimate
${\boldsymbol{\phi}^{\rm dl}}$ using a fewer training symbols in downlink. To
estimate ${\boldsymbol{\phi}^{\rm dl}}$, we use the LS technique, similar to
what presented in Subsection III-A. Given
$\left({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}}\right)$,
we define ${\bf g}({\boldsymbol{\phi}^{\rm
dl}})\triangleq\left[e^{j{\phi}_{1}^{\rm dl}}\;\;e^{j{\phi}_{2}^{\rm
dl}}\;\;\cdots\;\;e^{j{\phi}_{L}^{\rm dl}}\right]^{T}$. We also define
$\gamma_{l}^{k}\triangleq\alpha_{l}e^{j\frac{2\pi k}{K}\tau_{l}B}$ and the
following $p\times L$ matrix

$\displaystyle{\bf
B}_{k}\left({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}}\right)\triangleq{\bf
S}^{\rm dl}_{k}\left[\gamma_{1}^{k}{\bf
b}(\theta_{1},\lambda)\;\;\gamma_{2}^{k}{\bf
b}(\theta_{2},\lambda)\;\;\cdots\;\;\gamma_{L}^{k}{\bf
b}(\theta_{L},\lambda)\right].$ (12)

Then, we can express (3) in terms of ${\boldsymbol{\phi}^{\rm dl}}$ as

$\displaystyle{\bf y}^{\rm dl}_{k}\triangleq{\bf
B}_{k}\left({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}}\right){\bf
g}({\boldsymbol{\phi}^{\rm dl}})+{\bf n}^{\rm
dl}_{k},\,\,\,k\in\mathcal{K}_{\rm dl}.$ (13)

Now, stacking the observations across all subcarriers in the set
$\mathcal{K}_{\rm dl}$, and defining ${\bf y}^{\rm dl}\triangleq\Big{[}{\bf
y}^{\rm dl}_{k}\Big{]}_{k\in\mathcal{K}_{\rm dl}}$,
$\mathcal{B}\triangleq\Big{[}{\bf
B}_{k}\left({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}}\right)\Big{]}_{k\in\mathcal{K}_{\rm
dl}}$ , and ${\bf n}^{\rm dl}\triangleq\Big{[}{\bf n}^{\rm
dl}_{k}\Big{]}_{k\in\mathcal{K}_{\rm dl}}$, the collected received signal in
the downlink is given by

$\displaystyle{\bf y}^{\rm dl}=\mathcal{B}{\bf g}({\boldsymbol{\phi}^{\rm
dl}})+{\bf n}^{\rm dl}.$ (14)

Defining $J_{\rm dl}\left(\boldsymbol{\phi}^{\rm dl}\right)\triangleq\|{\bf
y}^{\rm dl}-\mathcal{B}{\bf g}({\boldsymbol{\phi}^{\rm dl}})\|_{2}^{2}$, we
now solve the following LS problem

$\displaystyle\min_{\boldsymbol{\phi}^{\rm dl}}$ $\displaystyle\,\,\,\,J_{\rm
dl}\left(\boldsymbol{\phi}^{\rm dl}\right),$ (15)

to find the optimal ${\boldsymbol{\phi}^{\rm dl}}$. Note that in the
optimization problem (15), as in (11), we have ignored the constraint
$0\preccurlyeq\boldsymbol{\phi}^{\rm dl}\preccurlyeq 2\pi$ due to the fact
that the objective function is a periodic function of $\boldsymbol{\phi}^{\rm
dl}$, i.e., $J_{\rm dl}\left(\boldsymbol{\phi}^{\rm dl}\right)=J_{\rm
dl}\left(\boldsymbol{\phi}^{\rm dl}+2i\pi\right)$, for integer $i$. As in
(11), the optimization problem (15) is an unconstrained least squares problem
and can be solved using the gradient descent algorithm. Once
$\boldsymbol{\phi}^{\rm dl}$ is estimated it is fed back to BS for the
subsequent data transmission444In this paper, we do not consider the
quantization of $\boldsymbol{\phi}^{\rm dl}$ obtained in (15)..

##  IV Deep Generative Model

In the previous section, we described an LS-based procedure to obtain the
parameters that defines the downlink channel. the crucial step is the mapping
$G({\bf z})$ that leads to ${\bf x}^{\rm up}$. In this section, we explain how
to find $G({\bf z})$ using DGMs. Before we get into $G({\bf z})$, and for the
sake of the paper being self-contained, we briefly introduce DGMs and explain
how we can use them to find $G({\bf z})$.

The core idea of DGMs is to represent a high-dimensional and complex
distribution of data ${\bf x}$ (in this case ${\bf
x}=\left({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}}\right)$)
using a deterministic mapping over a low-dimensional random vector ${\bf z}$
which has a well behaved pdf (e.g., uniform or Gaussian). Specifically, a DGM
is a function $G({\bf z})$ that maps a low-dimensional random vector ${\bf
z}\in\mathbb{R}^{d}$, typically drawn independently from Gaussian or uniform
distribution, to a high-dimensional vector ${\bf x}_{g}\triangleq G({\bf
z})\in\mathbb{R}^{n}$ where $n\geq d$ (in practice, due to the structure of
${\bf x}$ we can have $n\gg d$). The mapping $G(\cdot)$ is determined such
that the distribution of ${\bf x}_{g}$, generated by $G(\cdot)$, matches the
distribution of the real world data vector ${\bf x}$. In other words, using
the transformation $G(\cdot)$ from a simple and low-dimensional distribution,
we generate samples that belong to the same manifold as ${\bf x}$ does. This
implies that any generated sample from $G(\cdot)$ already satisfies the
constraints in (III-A)555 To show this, let us define
$\mathcal{C}\triangleq\\{{\bf
x}=({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}})|{\boldsymbol{\alpha}}\succcurlyeq
0,\boldsymbol{\tau}\succcurlyeq
0,0\preccurlyeq{\boldsymbol{\theta}}\preccurlyeq 2\pi,\\}$ the feasible set
for the objective function (III-A). Let $\mathcal{R}_{x}$ denote the domain of
the true sample ${\bf x}$, and $\mathcal{R}_{g}$ denote the range of $G({\bf
z})$. We know that $\mathcal{R}_{x}\subset\mathcal{C}$. Theoretically, $G({\bf
z})$ is trained such that it generates realistic samples ${\bf x}_{g}$ such
that $D({\bf x})$ cannot distinguish ${\bf x}_{g}$ among the true samples
${\bf x}$. This implies that any ${\bf x}_{g}\in\mathcal{R}_{g}$ belongs to
the same manifold as ${\bf x}$ does. This further implies that ${\bf
x}_{g}\in\mathcal{R}_{x}$. Therefore, we conclude that ${\bf
x}_{g}\in\mathcal{C}$, i.e., any sample generated from $G({\bf z})$ satisfies
the constraints in (III-A).. The function $G(\cdot)$ is parameterized by a
deep neural network, which is trained in an unsupervised way as explained
below. One well-known example of DGM is the family of GANs. In this paper, we
use the GAN architecture to find $G(\cdot)$. Next, we briefly introduce GANs
and explain how we use GANs to obtain $G(\cdot)$.

###  IV-A GANs

GANs are among the most powerful DGMs that are used to capture the
distribution of data [54]. As shown in Fig. 1, a GAN consists of two fully-
connected feed-forward neural networks, namely a generator network $G_{{\cal
W}_{g}}({\bf z}):\mathbb{R}^{d}\rightarrow\mathbb{R}^{n}$, and a discriminator
network $D_{{\cal W}_{d}}({\bf x}):\mathbb{R}^{n}\rightarrow[0,1]$, where
${\cal W}_{g}$ and ${\cal W}_{d}$ represent, respectively, the sets of weights
of the generator and discriminator networks. The generator network $G_{{\cal
W}_{g}}({\bf z})$ maps the input random vector ${\bf
z}\backsim\mathbb{P}_{z}({\bf z})$, into the data space ${\bf
x}_{g}\backsim\mathbb{P}_{g}({\bf x})$. Here, $\mathbb{P}_{z}({\bf z})$
represents the pdf of the input random vector ${\bf z}$ and is usually chosen
to be $\mathbb{P}_{z}({\bf z})=\mathcal{N}({\bf 0},{\bf I}_{d})$, and
$\mathbb{P}_{g}({\bf x})$ represents the pdf of the generated samples. The
discriminator network, $D_{{\cal W}_{d}}(\cdot)$, receives the two sets of
inputs: one set consists of the samples ${\bf x}_{g}$ generated by $G_{{\cal
W}_{g}}({\bf z})$ and the other set consists of the true samples ${\bf x}$.
The discriminator network $D_{{\cal W}_{d}}(\cdot)$ is meant to correctly
distinguish between the fake samples ${\bf x}_{g}$ and the true samples ${\bf
x}$. Effectively, the goal of the generator network $G_{{\cal W}_{g}}(\cdot)$
is to generate fake samples such that the discriminator network $D_{{\cal
W}_{d}}({\bf x})$ cannot distinguish them from the true samples. Meanwhile,
the goal of the discriminator network $D_{{\cal W}_{d}}(\cdot)$ is to
correctly distinguish between ${\bf x}_{g}$ and ${\bf x}$. To do so, $D_{{\cal
W}_{d}}({\bf x})$ provides the probability that ${\bf x}$ to be a real sample.
If $\mathbb{P}_{r}({\bf x})$ represents the distribution of true samples, the
goal is to have $\mathbb{P}_{g}({\bf x})=\mathbb{P}_{r}({\bf x})$, by
optimally adjusting ${\cal W}_{g}$ and ${\cal W}_{d}$. Below, we explain how
to adjust the weights.

####  IV-A1 Training of GANs

$G_{{\cal W}_{g}}(\cdot)$ and $D_{{\cal W}_{d}}(\cdot)$ are trained
simultaneously and iteratively via the following two-player min-max game [54]:

$\displaystyle\min_{{\cal W}_{g}}\max_{{\cal W}_{d}}\,\,\,\tilde{L}(G_{{\cal
W}_{g}}(\cdot),D_{{\cal W}_{d}}(\cdot)),$ (16)

where $\tilde{L}(G_{{\cal W}_{g}}(\cdot),D_{{\cal W}_{d}}(\cdot)),$ is the
loss function and defined as

$\displaystyle\tilde{L}(G_{{\cal W}_{g}}(\cdot),D_{{\cal W}_{d}}(\cdot))$
$\displaystyle\triangleq\mathbb{E}_{{\bf x}}\left[\log D_{{\cal W}_{d}}({\bf
x})\right]$ $\displaystyle+\mathbb{E}_{{\bf z}}\left[\log\left(1-D_{{\cal
W}_{d}}(G_{{\cal W}_{g}}({\bf z}))\right)\right].$ (17)

Figure 1: The GAN structure.

Note that the objective of training $G_{{\cal W}_{g}}(\cdot)$ is to fool
$D_{{\cal W}_{d}}(\cdot)$ by generating the _realistic_ samples ${\bf x}_{g}$
such that $D_{{\cal W}_{d}}(\cdot)$ assigns a high probability to ${\bf
x}_{g}$ being true samples. This can be done by maximizing $D_{{\cal
W}_{d}}({\bf x}_{g})$ (or equivalently, minimizing $\log\left(1-D_{{\cal
W}_{d}}(G_{{\cal W}_{g}}({\bf z}))\right)$ over ${\cal W}_{g}$, as given in
the second term in (IV-A1)). In the meantime, $D_{{\cal W}_{d}}(\cdot)$ aims
to correctly distinguish ${\bf x}_{g}$, generated by $G_{{\cal
W}_{g}}(\cdot)$, from the true samples ${\bf x}$. In other words, $D_{{\cal
W}_{d}}({\bf x}_{g})$ is meant to be close to zero while $D_{{\cal
W}_{d}}({\bf x})$ has to be close to one. Therefore, ${\cal W}_{d}$ is chosen
such that $\log D_{{\cal W}_{d}}({\bf x})+\log\left(1-D_{{\cal
W}_{d}}(G_{{\cal W}_{g}}({\bf z}))\right)$ is maximized. It has been shown
that such a competitive interplay between $G_{{\cal W}_{g}}({\bf z})$ and
$D_{{\cal W}_{d}}({\bf x})$ converges to an equilibrium where $P_{g}({\bf
x})=P_{r}({\bf x})$ [54] . At this point, the generator produces realistic
${\bf x}_{g}$ such that the discriminator is unable to differentiate between
${\bf x}$ and ${\bf x}_{g}$, i.e., $D_{{\cal W}_{d}}({\bf x})=D_{{\cal
W}_{d}}({\bf x}_{g})=\frac{1}{2}$.

####  IV-A2 Mode collapse issue and its solution

Despite the success of GANs in learning the underlying distribution of data,
training of GANs is challenging due to the instability in the training and
sensitivity to the hyper-parameters. In [55], the authors propose different
solutions to improve the training stability and robustness. On the other hand,
_mode collapse,_ an issue that can hinder the training of GANs, refers to
collapsing of large volumes of probability mass into a few modes. This means
that although the generator produces meaningful samples, these samples belong
to _only_ few modes of the data distribution; therefore, the samples produced
by the generator do not fully represent the underlying distribution of real
data.

Different solutions has been proposed to address the mode collapse issue; in
this paper, we use a regularized GAN (Reg-GAN) proposed in [56]. Compared to
the original form of GANs, a Reg-GAN uses a regularizer term that meant to
penalize the missing modes. To do so, together with the generator and
discriminator, an encoder network $E_{{\cal W}_{e}}({\bf x}):{\bf
x}\rightarrow{\bf z}$ is trained to help the generator to avoid the missing
modes, where ${\cal W}_{e}$ is the set of weights of the encoder network.

To perform this training, two regularizing terms are considered in training of
the $E_{{\cal W}_{e}}(\cdot)$ and $G_{{\cal W}_{g}}(\cdot)$ networks. One
regularizer is based on the fact that if $G_{{\cal W}_{g}}\left(E_{{\cal
W}_{e}}(\cdot)\right)$ is a good auto-encoder, then, for any ${\bf x}_{\rm
o}\in{\cal M}_{\rm o}$, where ${\cal M}_{\rm o}$ is the set of missing modes,
we obtain ${\bf x}_{\rm o}\approx G_{{\cal W}_{g}}\left(E_{{\cal W}_{e}}({\bf
x}_{o})\right)$. Therefore, $\mathbb{E}_{{\bf x}}\left[\|{\bf x}-G_{{\cal
W}_{g}}\left(E_{{\cal W}_{e}}({\bf x})\right)\|^{2}\right]$ is the loss
function considered in the training of $E_{{\cal W}_{e}}(\cdot)$ and $G_{{\cal
W}_{g}}(\cdot)$ networks, as a regularizer to penalize the generator for any
missing samples including the samples of minor modes. A second regularizer,
$\mathbb{E}_{{\bf x}}\left[\log D_{{\cal W}_{d}}\left(G_{{\cal
W}_{g}}\left(E_{{\cal W}_{e}}({\bf x})\right)\right)\right]$ is used to
encourage $G_{{\cal W}_{g}}(E_{{\cal W}_{e}}(\cdot))$ to generate realistic
samples such that $D_{{\cal W}_{d}}(\cdot)$ assigns, to these samples, a high
probability of being a true sample. Therefore, we can achieve a fair
probability distribution across different modes. The regularized loss
functions for the generator, the encoder and discriminator are respectively
given in (18), (19), and (20), where $\lambda_{1}$ and $\lambda_{2}$ are the
regularizer’s coefficients.

$\displaystyle T_{G}\left({\cal W}_{g},{\cal W}_{d},{\cal W}_{e}\right)$
$\displaystyle=-\mathbb{E}_{{\bf z}}\left[\log D_{{\cal W}_{d}}\left(G_{{\cal
W}_{g}}({\bf z})\right)\right]+\mathbb{E}_{{\bf x}}\left[\lambda_{1}\|{\bf
x}-G_{{\cal W}_{g}}\left(E_{{\cal W}_{e}}({\bf
x})\right)\|^{2}+\lambda_{2}\log D_{{\cal W}_{d}}\left(G_{{\cal
W}_{g}}\left(E_{{\cal W}_{e}}({\bf x})\right)\right)\right]$ (18)
$\displaystyle T_{E}\left({\cal W}_{g},{\cal W}_{d},{\cal W}_{e}\right)$
$\displaystyle=\mathbb{E}_{{\bf x}}\left[\lambda_{1}\|{\bf x}-G_{{\cal
W}_{g}}\left(E_{{\cal W}_{e}}({\bf x})\right)\|^{2}+\lambda_{2}\log D_{{\cal
W}_{d}}\left(G_{{\cal W}_{g}}\left(E_{{\cal W}_{e}}({\bf
x})\right)\right)\right]$ (19) $\displaystyle T_{D}\left({\cal W}_{g},{\cal
W}_{d},{\cal W}_{e}\right)$ $\displaystyle=\mathbb{E}_{{\bf x}}\left[\log
D_{{\cal W}_{d}}({\bf x})\right]+\mathbb{E}_{{\bf
z}}\left[\log\left(1-D_{{\cal W}_{d}}\left(G_{{\cal W}_{g}}({\bf
z})\right)\right)\right].$ (20)

_Training procedure_ : During the training, we aim to find ${\cal W}_{d}$,
${\cal W}_{g}$ and ${\cal W}_{e}$. This can be done by jointly and iteratively
maximizing $T_{D}$, and minimizing $T_{G}$ and $T_{E}$. Before we start the
training process, ${\cal W}_{d}$, ${\cal W}_{g}$ and ${\cal W}_{e}$ are
randomly initialized. In each of the subsequent iterations, we sample a mini-
batch of size $m$ from both the training set $\left(\left\\{{\bf
x}^{(i)}\right\\}_{i=1}^{m}\right)$ and noise samples $\left(\left\\{{\bf
z}^{(i)}\right\\}_{i=1}^{m}\right)$. For fixed ${\cal W}_{g}$ and ${\cal
W}_{e}$, we first update ${\cal W}_{d}$ by ascending in the direction of the
gradient of $T_{D}$. Then, while fixing ${\cal W}_{d}$ and ${\cal W}_{e}$, we
update ${\cal W}_{g}$ by descending in the opposite direction of the gradient
of $T_{G}$. Similarly, ${\cal W}_{e}$ is updated by descending in the opposite
direction of the gradient of $T_{E}$ for fixed ${\cal W}_{g}$ and ${\cal
W}_{d}$.

The Reg-GAN training procedure is summarized in Algorithm 1.

Algorithm 1 Reg-GAN Training Algorithm

Inputs: Data set, number of epochs, mini-batch size $m$, $\lambda_{1}$, and
$\lambda_{2}$.
Outputs: Trained $G({\bf z})$ .

1:for each epoch do:

2: Sample mini-batches of $m$ noise samples $\\{{\bf z}^{(1)},{\bf
z}^{(2)},\cdots,{\bf z}^{(m)}\\}$ and $m$ data samples $\\{{\bf x}^{(1)},{\bf
x}^{(2)},\cdots,{\bf x}^{(m)}\\}$ from $P_{z}({\bf z})$ and $P_{r}({\bf x})$,
respectively.

3: Update discriminator $D_{{\cal W}_{d}}(\cdot)$ by ascending in the
direction of its stochastic gradient:  $\displaystyle\nabla_{{{\cal
W}_{d}}}\frac{1}{m}\sum_{i=1}^{m}\Big{[}\log D_{{\cal W}_{d}}({\bf
x}^{(i)})+\log\left(1-D_{{\cal W}_{d}}\left(G_{{\cal W}_{g}}({\bf
z}^{(i)})\right)\right)\Big{]}.$

4: Sample mini-batches of $m$ noise samples $\\{{\bf z}^{(1)},{\bf
z}^{(2)},\cdots,{\bf z}^{(m)}\\}$ and $m$ data samples $\\{{\bf x}^{(1)},{\bf
x}^{(2)},\cdots,{\bf x}^{(m)}\\}$ from $P_{z}({\bf z})$ and $P_{r}({\bf x})$,
respectively.

5: Update generator $G_{{\cal W}_{g}}(\cdot)$ by descending in the opposite
direction of its stochastic gradient:  $\displaystyle\nabla_{{{\cal
W}_{g}}}\frac{1}{m}\sum_{i=1}^{m}\Big{[}-\log D_{{\cal W}_{d}}\left(G_{{\cal
W}_{g}}({\bf z}^{(i)})\right)+\lambda_{1}\|{\bf x}^{(i)}-G_{{\cal
W}_{g}}\left(E_{{\cal W}_{e}}({\bf x}^{(i)})\right)\|^{2}+\lambda_{2}\log
D_{{\cal W}_{d}}\left(G_{{\cal W}_{g}}\left(E_{{\cal W}_{e}}({\bf
x}^{(i)})\right)\right)\Big{]}$

6: Sample mini-batches of $m$ noise samples $\\{{\bf z}^{(1)},{\bf
z}^{(2)},\cdots,{\bf z}^{(m)}\\}$ and $m$ data samples $\\{{\bf x}^{(1)},{\bf
x}^{(2)},\cdots,{\bf x}^{(m)}\\}$ from $P_{z}({\bf z})$ and $P_{r}({\bf x})$,
respectively.

7: Update encoder $E_{{\cal W}_{e}}(\cdot)$ by descending in the opposite
direction of its stochastic gradient:  $\displaystyle\nabla_{{{\cal
W}_{e}}}\frac{1}{m}\sum_{i=1}^{m}\Big{[}\lambda_{1}\|{\bf x}^{(i)}-G_{{\cal
W}_{g}}\left(E_{{\cal W}_{e}}({\bf x}^{(i)})\right)\|^{2}+\lambda_{2}\log
D_{{\cal W}_{d}}\left(G_{{\cal W}_{g}}\left(E_{{\cal W}_{e}}({\bf
x}^{(i)})\right)\right)\Big{]}$

8:end

Note that, in Algorithm 1, the gradient-based update can be implemented using
any standard gradient-based algorithm. In this paper, to speed up the
convergence, we use a momentum-based gradient update.

##  V Convergence, complexity, and identifiability

###  V-A Convergence

In this subsection, we analytically study the convergence of our estimation
algorithm provided in Subsection III-A. Here, we aim to show that the
objective function decreases from one iteration to the next. The following
lemma provides an insight into the convergence.

######  Lemma 1.

The gradient of $J_{\rm up}\left(G_{{\cal W}_{g}}({\bf
z}),{\boldsymbol{\phi}^{\rm up}}\right)$ is a Lipschitz-continuous function.

_Proof_ : See Appendix B.

Let us define ${\tilde{{\bf z}}}\triangleq\left({\bf
z},\boldsymbol{\phi}\right)$ and $\ell(\tilde{{\bf z}})\triangleq J_{\rm
up}\left(G_{{\cal W}_{g}}({\bf z}),{\boldsymbol{\phi}}\right)$. According to
Lemma 1, we can write $\|\nabla\ell(\tilde{{\bf
z}}^{(i)})-\nabla\ell(\tilde{{\bf z}}^{(j)})\|\leq C_{\ell}\|\tilde{{\bf
z}}^{(i)}-\tilde{{\bf z}}^{(j)}\|$, where $C_{\ell}$ is the Lipschitz
constant. This now means that $\nabla^{2}\ell(\tilde{{\bf z}})\preccurlyeq
C_{\ell}{\bf I}$, which further implies that

$\displaystyle{\bf v}^{T}\nabla^{2}\ell(\tilde{{\bf z}}){\bf v}\leq
C_{\ell}\|{\bf v}\|^{2}$ (21)

holds true for any vector ${\bf v}$. On the other hand, if we consider the
gradient descent step $\eta=1/C_{\ell}$, then the update from the $i$th to the
$(i+1)$th iteration is written as

$\displaystyle\tilde{{\bf z}}^{(i+1)}=\tilde{{\bf
z}}^{(i)}-\frac{1}{C_{\ell}}\bigtriangledown\ell(\tilde{{\bf z}}^{(i)}).$ (22)

Let us use the Taylor expansion and expand $\ell(\tilde{\bf z}^{(i+1)})$ as

$\displaystyle\ell(\tilde{{\bf z}}^{(i+1)})$ $\displaystyle=\ell(\tilde{{\bf
z}}^{(i)})+\bigtriangledown\ell(\tilde{{\bf z}}^{(i)})\left(\tilde{{\bf
z}}^{(i+1)}-\tilde{{\bf z}}^{(i)}\right)$
$\displaystyle+\frac{1}{2}\left(\tilde{{\bf z}}^{(i+1)}-\tilde{{\bf
z}}^{(i)}\right)^{T}\bigtriangledown^{2}\ell(\tilde{{\bf
z}}^{(i)})\left(\tilde{{\bf z}}^{(i+1)}-\tilde{{\bf z}}^{(i)}\right)$
$\displaystyle\overset{(a)}{\leq}\ell(\tilde{{\bf
z}}^{(i)})-\frac{1}{C_{\ell}}\bigtriangledown\ell(\tilde{{\bf
z}}^{(i)})^{T}\bigtriangledown\ell(\tilde{{\bf
z}}^{(i)})+\frac{C_{\ell}}{2}\|\tilde{{\bf z}}^{(i+1)}-\tilde{{\bf
z}}^{(i)}\|^{2}$ $\displaystyle=\ell(\tilde{{\bf
z}}^{(i)})-\frac{1}{C_{\ell}}\|\bigtriangledown\ell(\tilde{{\bf
z}}^{(i)})\|^{2}+\frac{1}{2C_{\ell}}\|\bigtriangledown^{2}\ell(\tilde{{\bf
z}}^{(i)})\|^{2}$ $\displaystyle=\ell(\tilde{{\bf
z}}^{(i)})-\frac{1}{2C_{\ell}}\|\bigtriangledown^{2}\ell(\tilde{{\bf
z}}^{(i)})\|^{2},$ (23)

where $(a)$ follows directly from (21) and (22). From the last equality in
(V-A), we observe that in the $i$th step $\ell(\tilde{{\bf
z}}^{(i+1)})\leq\ell(\tilde{{\bf z}}^{(i)}),$ or equivalently,

$\displaystyle J_{\rm up}\left(G_{{\cal W}_{g}}({\bf
z}^{(i+1)}),{\boldsymbol{\phi}^{{\rm up},(i+1)}}\right)\leq J_{\rm
up}\left(G_{{\cal W}_{g}}({\bf z}^{(i)}),{\boldsymbol{\phi}^{{\rm
up},(i)}}\right).$ (24)

The inequality in (24) holds true for any $\eta\leq 1/C_{\ell}$. The
inequality of (24) shows that the LS cost function decreases from one
iteration to the next. Since the cost function has a lower bound (of zero),
the algorithm converges.

###  V-B Computational complexity

To solve (III-A) (or equivalently (11)), we use a gradient descent algorithm
to jointly and iteratively update ${\bf z}$ and ${\boldsymbol{\phi}^{\rm
up}}$. The gradient descent requires
$\mathcal{O}\left(\log(1/\varepsilon)\right)$ iterations, where $\varepsilon$
is the chosen error tolerance. Each iteration of gradient descent has the
following two steps: we first fix ${\boldsymbol{\phi}^{\rm up}}$ and update
${\bf z}$, then for a fixed ${\bf z}$, we update ${\boldsymbol{\phi}^{\rm
up}}$. In the first step, for a fixed ${\boldsymbol{\phi}^{\rm up}}$, the
computational complexity of updating ${\bf z}$ is dominated by the complexity
of calculating $\frac{\partial}{\partial{\bf z}}J_{\rm up}=\frac{\partial
J_{\rm up}}{\partial{\bf x}}\frac{\partial{\bf x}}{\partial{\bf z}}$. The
complexity of the first term is given by $\mathcal{O}(M|\mathcal{K}_{\rm
up}|L^{3})$. The second term, however, is related to the gradient of the
$G({\bf z})$ with respect to ${\bf z}$, which is characterized by a neural
network. The computational complexity of this term is given by
$\mathcal{O}\left(dnq\right)$, where $n$ is the number of network layers and
$q$ is the maximum number of nodes in each layer. In the second step, the
computational complexity of updating ${\boldsymbol{\phi}^{\rm up}}$ is given
by $\mathcal{O}(M|\mathcal{K}_{\rm up}|L^{3})$. Therefore, the overall
complexity of solving (III-A) is $\mathcal{O}(dnqM|\mathcal{K}_{\rm
up}|L^{3}\log(1/\varepsilon))$. Similarly, we use a gradient descent algorithm
to solve (15), which incurs $\mathcal{O}\left(\log(1/\varepsilon)\right)$
iterations. The complexity of updating ${\boldsymbol{\phi}^{\rm dl}}$ in each
iteration is given by $\mathcal{O}(p|\mathcal{K}_{\rm dl}|L^{3})$. Therefore,
the overall complexity of optimization problem (15) is given by
$\mathcal{O}\left(p|\mathcal{K}_{\rm dl}|L^{3}\log(1/\varepsilon)\right)$.

###  V-C Identifiability

During the downlink training, we assume that $p$ training symbols are
transmitted over the $k$-th subcarrier ($k\in\mathcal{K}_{\rm dl}$) and the
$m$th antenna ($m\in\mathcal{M}_{\rm dl}$). The value of $p$ depends on the
$|\mathcal{M}_{\rm dl}|$. The reason is that the $p|\mathcal{K}_{\rm
dl}|\times L$ matrix $\mathcal{B}$ in (14) is of rank
$\min\left(|\mathcal{M}_{\rm dl}|,L\right)$. If $|\mathcal{M}_{\rm dl}|<L$,
then ${\rm rank}(\mathcal{B})=|\mathcal{M}_{\rm dl}|$, leading to an
identifiability issue in recovering $\\{\phi_{l}^{\rm dl}\\}_{l=1}^{L}$ in
(14). If $|\mathcal{M}_{\rm dl}|>L$, the matrix $\mathcal{B}$ becomes full
column-rank, and therefore estimating $\\{\phi_{l}^{\rm dl}\\}_{l=1}^{L}$
requires $p|\mathcal{K}_{\rm dl}|\geq L$, or equivalently, $p\geq
L/|\mathcal{K}_{\rm dl}|$.

##  VI Implementation and simulation results

In this section, we explain a detailed implementation of our estimation
technique. Simulation results are also provided to evaluate the performance of
the proposed technique.

###  VI-A Experiment setup and dataset

####  VI-A1 Dataset Generation

Throughout the simulations, we consider an indoor massive MIMO scenario. An
example of such scenario is the ”l1” scenario provided by DeepMIMO dataset
[51] which is generated by the 3D ray-tracing simulator Wireless InSite [57].
The ”l1” scenario comprises a $10\times 10\times 2.5$ meters room with $2$
tables inside the room. There are $M=64$ antennas mounted on the ceiling. The
users are spread inside the room across the $x$-$y$ plane with each of them
being $1$ meter above the floor. The communication between the BS antennas and
each UE is in FDD mode and uses $K$ OFDM subcarriers 666According to the
DeepMIMO dataset [51], the maximum channel delay spread is $\tau_{\rm
max}=94.5$ ns. Given the bandwidth $B=20$ $({\rm MHz})$, the sample time is
$T_{s}\simeq\frac{1}{B}=50$ $({\rm ns})$. Since $\tau_{\rm max}>T_{s}$, the
channels in this experiment suffers from frequency selectivity.. The uplink
and downlink operating frequencies are respectively $2.4$ GHz and $2.5$ GHz.
The DeepMIMO dataset parameters are given in Table I.

Table I: DeepMIMO dataset parameters Scenario | l1-2.4 GHz and 11-2.5 GHz
---|---
Number of BS | 1
Active BS | [32]
Number of BS antenna in (x,y,z) | (64,1,1)
Antenna Spacing | 0.5$\lambda$
Active users row [first, last] | [1,500]
Number of propagation path | 5
$K$ | 16
OFDM sampling factor | 1
OFDM limit | 16
$B$ | 20 MHz
Noise Power |  $-174$ dBm/Hz

Using the above ray-tracing scenario as well as the parameters given in Table
I, DeepMIMO generates the dataset. The generated dataset comprising the
channel vectors between each UE and the BS across all $K$ subcarriers at both
uplink and downlink frequencies, i.e., $2.4$ GHz and $2.5$ GHz, and the
underlying channel parameters for each propagation path, namely
${\boldsymbol{\alpha}}$, $\boldsymbol{\tau}$, ${\boldsymbol{\theta}}$,
${\boldsymbol{\phi}^{\rm up}}$, and ${\boldsymbol{\phi}^{\rm dl}}$, and the UE
$x$-$y$ coordinates.

####  VI-A2 Preprocessing

To avoid any permutation ambiguity, the paths are sorted from the lowest delay
to the highest delay. To prepare the data to train our GAN, we note that the
ranges of ${\boldsymbol{\alpha}}$, $\boldsymbol{\tau}$,
${\boldsymbol{\theta}}$, ${\boldsymbol{\phi}^{\rm up}}$, and
${\boldsymbol{\phi}^{\rm dl}}$ are all in different scales. We use the
following transformations to have them in a reasonable range: vector
${\boldsymbol{\alpha}}$ is transformed by taking $\log_{10}$ of each of its
entries, i.e.,
$\tilde{{\boldsymbol{\alpha}}}\triangleq\log_{10}{\boldsymbol{\alpha}}$;
$\boldsymbol{\tau}$ is scaled by $T_{s}\triangleq 1/B$ as
$\tilde{\boldsymbol{\tau}}\triangleq\boldsymbol{\tau}/T_{s}$;
${\boldsymbol{\theta}}$, ${\boldsymbol{\phi}^{\rm up}}$, and
${\boldsymbol{\phi}^{\rm dl}}$ are all expressed in terms of radians. After
scaling/transforming these feature vectors, we concatenate them to form the
overall feature vectors as

$\displaystyle\tilde{{\bf x}}^{\rm up}$
$\displaystyle\triangleq\left[\tilde{{\boldsymbol{\alpha}}},\tilde{\boldsymbol{\tau}},{\boldsymbol{\theta}},{\boldsymbol{\phi}^{\rm
up}}\right]$ (25) $\displaystyle\tilde{{\bf x}}^{\rm dl}$
$\displaystyle\triangleq\left[\tilde{{\boldsymbol{\alpha}}},\tilde{\boldsymbol{\tau}},{\boldsymbol{\theta}},{\boldsymbol{\phi}^{\rm
dl}}\right].$ (26)

Before we start training, we divide $\tilde{{\bf x}}^{\rm up}$ and
$\tilde{{\bf x}}^{\rm dl}$ by their maximum value, thereby normalizing their
entries into the range of $[-1,1]$. The maximum value here refers to the
global maximum value across all entries of sample vectors. To form the
training and testing dataset, after scaling and normalizing the data, we first
shuffle and then we split the data such that $80\%$ is dedicated for training
and $20\%$ for testing.

###  VI-B Network architectures and training

####  VI-B1 Network architecture

We use the Reg-GAN structure given in Section IV-A2, implemented in
Pytorch777Simulations are based on Pytorch v1.3.1 available at
https://pytorch.org/docs/1.3.1/.. The generator $G_{{\cal W}_{g}}(\cdot):{\bf
z}\rightarrow{\bf x}$ takes an input sample ${\bf z}\sim\mathcal{N}({\bf
0},{\bf I}_{d})$, of size $d=8$, and passes it through a fully-connected
neural network which has $3$ layers with $[10,12,14]$ neurons in the
respective layers. It then generates samples ${\bf x}\in\mathbb{R}^{n}$, with
$n=15$. As for the encoder network $E_{{\cal W}_{e}}(\cdot):{\bf
x}\rightarrow{\bf z}$, the input and output size are $15$ and $8$,
respectively. There are also $3$ hidden layers with $[14,12,10]$ neurons in
the corresponding layers. Likewise, the discriminator is implemented using a
fully-connected neural network with the input size $15$, output size $1$, and
$4$ hidden layers with $[12,8,4,2]$ neurons in the corresponding layers.

To improve the stability and convergence during the training, in all the
networks, we use Leaky ReLU activation function with slope $0.2$.
Additionally, the generator uses the hyperbolic tangent (tanh) activation
function in the output layer. We employ $20\%$ dropout in each hidden layer to
further improve the generalization capability of the generator.

####  VI-B2 Training

The outcome of the training process is the generator $G_{{\cal W}_{g}}(\cdot)$
that finds the optimal ${\bf z}$, denoted by ${\bf z}^{*}$, for each
realization in the test set. The training process follows directly from
Algorithm 1. We randomly initialize ${\cal W}_{d}$, ${\cal W}_{g}$, and ${\cal
W}_{e}$. In each epoch, we sample a mini-batch of size $m$ from both the
training set $\left(\left\\{{\bf x}^{(i)}\right\\}_{i=1}^{m}\right)$ and the
set of noise samples $\left(\left\\{{\bf z}^{(i)}\right\\}_{i=1}^{m}\right)$.
We first update ${\cal W}_{d}$ by ascending in the direction of the gradient
of $T_{D}$ given in (20) assuming ${\cal W}_{g}$ and ${\cal W}_{e}$ are fixed.
Then, we update ${\cal W}_{g}$ by descending in the opposite direction of the
gradient of $T_{G}$ given in (18) assuming ${\cal W}_{d}$ and ${\cal W}_{e}$
are fixed. Similarly, ${\cal W}_{e}$ is updated by descending in the opposite
direction of the gradient of $T_{E}$ given in (19). Note that, in these steps,
the sample mean is used instead of mathematical expectation. Furthermore,
thanks to `autograd`, the gradient descent/ascend in each epoch is carried out
by the differentiation capability implemented in Pytorch. The training
parameters are specified in the Table II.

####  VI-B3 Hyper-parameter selection

During the GAN training, hyper-parameters are the mini-batch size, learning
rate, number of epochs, generator and discriminator optimizer, activation
function, number of layers and the number of nodes in each layer. These hyper-
parameters are chosen based on a grid search. Since the size of the grid is
huge, we choose only $20$ randomly-selected grid points. For each grid point,
we train the network and assess the the generated samples. The assessment is
based on the following two criteria. 1) Generalizablity: The capability of
$G({\bf z})$ to generate realistic-like samples. This is measured by the rate
at which discriminator can correctly distinguish the true samples among the
fake samples. Ideally, the rate has to be as close as to $0.5$. 2) Mode
collapse: this criterion is used to make sure that the generator is capable of
generating different samples. This can be measured by calculating the distance
(norm-2) between the generated samples for many _different_ random vectors
${\bf z}$ as input.

###  VI-C Simulation Scenarios

We consider the following simulation scenarios. Note that in all simulation
scenarios, we assume that $L$ is known (or at least a maximum value of $L$ is
chosen).

####  VI-C1 UP-LMMSE

In this scenario, LMMSE-based channel estimation is used for uplink training
only. We assume that all $64$ antennas as well as all $16$ subcarriers are
used for training.

####  VI-C2 UP-GAN

This is our DGM-based channel estimation technique in the uplink. Similar to
the LMMSE-based technique we use all $64$ antennas as well as all $16$
subcarriers in the uplink. In this scenario, we solve (11) to find ${\bf z}$
(or
$\left({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}}\right)$)
and $\boldsymbol{\phi}^{\rm up}$ followed by channel reconstruction using (4).

####  VI-C3 DL-GAN

In this scenario, we aim to obtain ${\bf h}_{k}^{\rm dl}$ using the estimates
of
$\left({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}}\right)$,
obtained using UP-GAN, and $\boldsymbol{\phi}^{\rm dl}$. Given
$\left({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}}\right)$,
we find $\boldsymbol{\phi}^{\rm dl}$ using (15).

Table II: GAN Training Parameters Training dataset size | $120000$
---|---
Test dataset size | $30000$
Mini batch size | $512$
Epochs | $20000$
Optimizer | Adam
Learning rate, $\beta$ |  $10^{-3}$, $(0.9,0.999)$
$\lambda_{1}$, $\lambda_{2}$ |  $10^{-2}$, $10^{-2}$

####  VI-C4 DL-Full-Reciprocity

In this scenario, similar to DL-GAN (explained in VI-C3), we aim to obtain
${\bf h}_{k}^{\rm dl}$. Here, there is no downlink training and it only serves
as a benchmark in our simulations. There are two cases in this scenario. In
the first case, we assume that full reciprocity exists between the uplink and
downlink. Specifically, we assume $\boldsymbol{\phi}^{\rm
dl}=\boldsymbol{\phi}^{\rm up}$. In the second case, we assume
$\boldsymbol{\phi}^{\rm dl}=2\pi f_{\rm dl}\boldsymbol{\tau}$. In both cases,
we reconstruct ${\bf h}_{k}^{\rm dl}$ using
$\left({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}},\boldsymbol{\phi}^{\rm
dl}\right)$.

####  VI-C5 DL-LS

In this scenario, we ignore that ${\boldsymbol{\alpha}}^{\rm
up}\approx{\boldsymbol{\alpha}}^{\rm dl}$. We first estimate
$\left({\boldsymbol{\alpha}}^{\rm
up},\boldsymbol{\tau},{\boldsymbol{\theta}}\right)$ during the uplink
training. Then, we reuse
$\left(\boldsymbol{\tau},{\boldsymbol{\theta}}\right)$ in order to estimate
${\boldsymbol{\alpha}}^{\rm dl}$ and $\boldsymbol{\phi}^{\rm dl}$ using the
same pilots budget used in DL-GAN. Defining
${\boldsymbol{\rho}}\triangleq\left[\rho_{1}\,\,\rho_{2}\,\cdots\,\rho_{L}\right]^{T}$
$\rho_{l}\triangleq\alpha_{l}^{\rm dl}e^{j{\phi}_{l}^{\rm dl}}$, we can
rewrite (14) as ${\bf y}^{\rm dl}=\tilde{\mathcal{B}}{\boldsymbol{\rho}}+{\bf
n}^{\rm dl}$, for some matrix $\tilde{\mathcal{B}}$. We then use least squares
to find ${\boldsymbol{\rho}}$.

####  VI-C6 DL-Modified-R2F2

In this scenario, we ignore the mapping function $G_{{\cal W}_{g}}(\cdot)$ and
directly solve (III-A). The constrained optimization problem in (III-A) is
solved numerically using an approach based on the R2F2 algorithm proposed in
[30]. Note that the R2F2 algorithm, in its original form, is based on the
assumption that the downlink channel parameters are the same as its uplink
counterparts. As explained in Section II-C, this assumption is found to be
invalid due to the frequency dependency of $\phi_{l}^{\rm dl}$ and
$\phi_{l}^{\rm up}$. To account for the disparity in phases, we first solve
(III-A) using a similar technique provided in [30]. Then, using the so-
obtained
$\left({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}}\right)$,
we use (15) to find $\phi_{l}^{\rm dl}$. Throughout the simulations, we refer
to this technique as _DL-Modified-R2F2_. In this scenario, we assume that
$|\mathcal{M}_{\rm dl}|=64$.

To implement DL-Modified-R2F2, the optimization problem (III-A) is solved
using coordinate descent. This approach involves the division of the
parameters of the optimization problem into smaller sets for which the
constraints are separable. The optimization is then carried out over each of
these sets iteratively while treating the variables in the other sets to be
constants, thereby reducing the computation complexity. Conceptually, in the
feasible set, the algorithm iteratively converges to a minimum by taking
strides along directions parallel to the parameter-set axes. In this case, the
separability of constraints is obtained by taking ${\boldsymbol{\alpha}}$,
$\boldsymbol{\tau}$, ${\boldsymbol{\theta}}$, and $\boldsymbol{\phi}^{\rm up}$
as four parameter sets all of which have box constraints. Since the objective
function is non-convex, the global optimality of this technique is not
guaranteed. To avoid local minima, we initiate the optimization from $10$
randomly-chosen initial points and choose the solution with the least value of
the objective function.

Note that the above R2F2-based technique is highly sensitive to gradient
descent step that we choose for each parameter. This technique is also very
slow since it requires multiple random initializations as well as a large
number of iterations to converge.

Figure 2: NMSE vs. SNR, $|\mathcal{K}_{\rm up}|=|\mathcal{K}_{\rm dl}|=K=16$,
and $p=|\mathcal{K}_{\rm dl}|$.

Figure 3: NMSE vs. $p$, with $|\mathcal{K}_{\rm dl}|=p$, $|\mathcal{K}_{\rm
up}|=K=16$, and SNR= 20 (dB).

###  VI-D Simulation Results

####  VI-D1 Normalized MSE

We first use the normalized mean squared error (NMSE) averaged across all
subcarriers and is given by

$\displaystyle{\rm NMSE}=\frac{1}{K}\sum_{k=1}^{K}\mathbb{E}\left[\frac{\|{\bf
h}_{k}-\hat{{\bf h}}_{k}\|^{2}}{\|{\bf h}_{k}\|^{2}}\right],$ (27)

where ${\bf h}_{k}$ and $\hat{{\bf h}}_{k}$ are, respectively, the true and
the estimated channel vector at the $k$-th subcarrier. We later expand our
simulations to calculate the corresponding rate and symbol error rate (SER).
Throughout the simulations, we assume that $p=|\mathcal{K}_{\rm dl}|$, and
$\varepsilon=0.01$.

Fig. 3 depicts NMSE vs. SNR. In the uplink, we compare the performance of our
DGM-based technique (i.e., UP-GAN) with the UP-LMMSE assuming that all $64$
antennas as well as all $16$ subcarriers are utilized for training. As shown
in this figure, our DGM-based technique outperforms, by a large gap, the UP-
LMMSE approach in all practical ranges of SNR. Such a performance gap stems
from the fact that UP-LMMSE cannot perform well in this range of SNR, since
the pilot measurements received are of poor quality. In contrast, by
exploiting the prior knowledge of the underlying distribution of channel
parameters, captured using DGMs, UP-GAN outperforms the LMMSE-based technique
by a large margin. In other words, since $G_{{\cal W}_{g}}(\cdot)$ is trained
to generate a realistic channel parameters from the low-dimensional ${\bf z}$,
the UP-GAN obtains an estimate of
$\left({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}}\right)$
leading to an accurate estimate of the channels even at very low SNR.

As we increase the SNR, beyond $15$ dB, we observe that there is no further
improvement in the performance of DGM-based technique. Such an error floor is
mainly due to the limitations in representation capability of $G_{{\cal
W}_{g}}(\cdot)$. The generator $G_{{\cal W}_{g}}(\cdot)$ cannot generate the
exact channel parameters
$\left({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}}\right)$;
it can only generate realistic samples. That is, the DGM-based technique
cannot always outperform the LMMSE-based technique, but it can do much better
in practical ranges of SNR.

In the downlink, we plotted the performance of DL-GAN for different values of
$|\mathcal{M}_{\rm dl}|$. As shown, DL-GAN behaves similar to UP-GAN. This is
expected because, in both scenarios, we use the same $G_{{\cal
W}_{g}}(\cdot)$, and their performance is mainly related to the representation
capability of $G_{{\cal W}_{g}}(\cdot)$. Compared to the DL-Full-Reciprocity
scenario, DL-GAN performs significantly better. This indicates that the full-
reciprocity assumption between the uplink and downlink does not hold. Indeed,
in DL-GAN, we used a short training sequence over fewer antennas to estimate
$\boldsymbol{\phi}^{\rm dl}$ (the frequency-specific component of the
channel), which yields a large performance gain in downlink channel
estimation.

Additionally, DL-LS _marginally_ outperforms the GAN technique at very high
ranges of SNR. In particular, for this range of SNR, DL-LS provides a better
estimate for $({\boldsymbol{\alpha}}^{\rm dl},\boldsymbol{\phi^{\rm dl}})$
compared to what DL-GAN does. For low to medium ranges of SNR, the proposed
DL-GAN technique provides a significantly better estimate of the downlink
channel. This performance gap stems from the fact that DL-LS cannot perform
well in this range of SNR, since the pilot measurements are of poor quality.
In contrast, by exploiting the prior knowledge of the underlying distribution
of channel parameters, captured using DGMs, DL-GAN provides a better estimate
of channel parameters $({\boldsymbol{\alpha}}^{\rm dl},\boldsymbol{\phi^{\rm
dl}})$ at a low-to-medium range of SNR.

Note that, if we ignore the mapping function $G_{{\cal W}_{g}}(\cdot)$ (i.e.,
directly solving (III-A), which is exactly what DL-Modified-R2F2 does), the
steepest descent would suffer from convergence issues due to a larger search
space as well as the box constraint over each dimension. Furthermore, the
solution in this case is noise-sensitive, meaning that, at low SNRs, although
we might be able to minimize the objective function to some extent, the
solution is far from its true value. Instead, in our DGM-based technique, we
address these issues by incorporating $G_{{\cal W}_{g}}(\cdot)$ as a sort of
prior. The search space is limited to the domain of ${\bf z}$, which has a
much lower dimension compared to ${\bf
x}=\left({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}}\right)$.

Figure 4: NMSE vs. $|\mathcal{M}_{\rm dl}|$, with $|\mathcal{K}_{\rm
up}|=K=16$, $|\mathcal{K}_{\rm dl}|=p$, and SNR= 20 (dB).

Figure 5: Rate per subcarrier vs. SNR, $|\mathcal{K}_{\rm
up}|=|\mathcal{K}_{\rm dl}|=K=16$, and $p=|\mathcal{K}_{\rm dl}|$.

In addition, the weights ${\cal W}_{g}$ of the generator $G_{{\cal
W}_{g}}(\cdot)$ encode a probability distribution over the space of
$\left({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}}\right)$,
such that we can draw samples from that distribution from a low-dimensional
standard Gaussian distribution. This implies that any sample generated by
$G_{{\cal W}_{g}}(\cdot)$ satisfies the constraints in (III-A)(as shown in
footnote 5). Therefore, our DGM-based technique performs well even with low-
dimensional noisy pilot measurements. This explains the robustness of our
proposed technique to the noise level.

It is worth mentioning that, if we have enough pilot measurements, the
performance of the proposed DL-GAN is independent of $|\mathcal{M}_{\rm dl}|$
as long as $|\mathcal{M}_{\rm dl}|\geq L$. The reason is that the training
power $P_{T}$ in each time slot is distributed among the pilot symbols
transmitted across the $|\mathcal{M}_{\rm dl}|$ antennas in the downlink.
Indeed, adding more antennas for downlink training does not affect the
received SNR at the UE. However, when $|\mathcal{M}_{\rm dl}|<L$, the
$p|\mathcal{K}_{\rm dl}|\times L$ matrix $\mathcal{B}$ in (14) is of rank
$|\mathcal{M}_{\rm dl}|$, leading to an identifiability issue in recovering
$\\{\phi_{l}^{\rm dl}\\}_{l=1}^{L}$ from (14). This becomes more evident in
Fig. 5, where we plot NMSE vs. $|\mathcal{M}_{\rm dl}|$. Note that the
requirement that $|\mathcal{M}_{\rm dl}|\geq L$ can be relaxed if train the
channel in the time domain[58, 59]. However, the time domain approach is
beyond the scope of this paper.

Fig. 3 plots NMSE vs. $p$, where we compare the performance of DL-GAN (for
different $|\mathcal{M}_{\rm dl}|$) with DL-Modified-R2F2 at SNR = $20$ (dB).
The performance gap between the DL-Modified-R2F2 and our DGM-based technique
still exists even by increasing $p$ (adding more observations). This indicates
the lack of convergence in DL-Modified-R2F2 scenario. In the DL-GAN scenario,
adding more pilot measurements, either by increasing $p$ or $|\mathcal{K}_{\rm
dl}|$, will improve the channel estimation performance. The reason is that the
number of underlying channel parameters is independent of the number
subcarriers or the length of pilots. Therefore, by increasing either of these
quantities, we collect more observation that in turn, yields a better estimate
of the same number of channel parameters. However, as shown, such an
improvement is not consistent, i.e., DL-GAN does not always gain by increasing
the amount of pilot measurements. This is mainly limited by the accuracy of
the estimation of
$\left({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}}\right)$
in the uplink, which is dominated by the representation capability of
$G_{{\cal W}_{g}}(\cdot)$.

Similar to what we observed in Fig. 3, when $|\mathcal{M}_{\rm dl}|\geq L$,
adding more antennas for downlink training does not improve the performance.
For very small $p$ (i.e., $p=1$ and $2$), the DL-GAN yields poor performance
even with large $|\mathcal{M}_{\rm dl}|$. This limited performance is due to
the fact that the matrix $\mathcal{B}$ in (14) is not full-column rank in this
range of $p$, and imposes an identifiability issue in recovery of
$\\{\phi_{l}^{\rm dl}\\}_{l=1}^{L}$ from (14).

To further assess the performance of our DGM-based technique, Fig. 5 plots the
NMSE vs. $|\mathcal{M}_{\rm dl}|$, the number of antennas used for downlink
training. It is shown that, for fixed $p$ (or $|\mathcal{K}_{\rm dl}|$), more
pilot measurements does not always benefit the proposed DGM-based technique.
Similar to what we observed in Fig. 3, this observation can be attributed to
the accuracy of the estimate of
$\left({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}}\right)$
obtained via the uplink training, which, as mentioned, is dominated by the
representation capability of $G_{{\cal W}_{g}}(\cdot)$ .

Figure 6: Rate per subcarrier vs. $p$, with $|\mathcal{K}_{\rm dl}|=p$,
$|\mathcal{K}_{\rm up}|=K=16$, and SNR= 20 (dB).

Figure 7: Rate per subcarrier vs. $|\mathcal{M}_{\rm dl}|$, with
$|\mathcal{K}_{\rm up}|=K=16$, $|\mathcal{M}_{\rm up}|=M=64$,
$|\mathcal{K}_{\rm dl}|=p$ and SNR= 20 (dB).

####  VI-D2 Rate

In this subsection, we explore the impact of our proposed technique on the
achievable rate. The plots in Figs. 5, 7, and 7 correspond to the NMSE curves
we studied in the previous subsection. Fig. 5 plots the rate vs. SNR. In the
uplink, the UP-GAN outperforms the UP-LMMSE technique in practical ranges of
SNR as it does in Fig. 3. This is due to a better estimate of channel matrix
in this range of SNR. This, by itself, is attributed to the rich prior stored
in the weights of $G_{{\cal W}_{g}}(\cdot)$ network. For the same reason, the
DL-GAN yields a much better rate performance compared to what DL-Full-
Reciprocity and DL-Modified-R2F2 do. Note that the saturation in rate at high
SNR is related to the error floor in channel estimation, which comes from the
limits in representation capability of $G_{{\cal W}_{g}}({\bf z})$.

In Figs. 7 and 7, assuming SNR= $20$ (dB), we assess the rate performance of
our DGM-based technique vs. $p$ and $|\mathcal{M}_{\rm dl}|$, respectively.
Here, we assume that the frequency-independent features
$\left({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}}\right)$
are estimated in the uplink as in the UP-GAN scenario. The goal here is to
show how the number of downlink pilot observations will affect the rate
performance of the proposed technique. As shown in these figures, the rate
quickly saturates by increasing $p$ or $|\mathcal{M}_{\rm dl}|$. In other
words, adding more training observations does not always improve the rate
performance. This can be explained as follows. Due to the partial reciprocity
between the uplink and downlink channels, the proposed DGM-based technique
will use the estimate of
$\left({\boldsymbol{\alpha}},\boldsymbol{\tau},{\boldsymbol{\theta}}\right)$
obtained via the uplink training. Since, these features are not perfectly
estimated, mainly because of the representation capability of $G_{{\cal
W}_{g}}(\cdot)$, they will affect the estimation of $\boldsymbol{\phi}^{\rm
dl}$ in downlink in a way that adding more training observation does not
improve the estimation of $\boldsymbol{\phi}^{\rm dl}$, and therefore, the
estimate of channel. The same argument holds true when we increase the
training power as shown in Fig. 5.

Figure 8: SER per subcarrier vs. SNR, $|\mathcal{K}_{\rm
up}|=|\mathcal{K}_{\rm dl}|=K=16$ and $p=|\mathcal{K}_{\rm dl}|$.

####  VI-D3 Symbol Error Rate

The SER vs. SNR plot is given in Fig. 8, where we use QPSK modulation. As can
be seen from this figure, in practical ranges of SNR, the UP-GAN yields a
better SER performance compared to our benchmark UP-LMMSE. This observation is
expected since the UP-GAN provides a better estimate of the channel for this
range of SNR. As stated before, due to the limits in representation capability
of $G_{{\cal W}_{g}}(\cdot)$, which leads to an error floor in channel
estimation (see Fig. 3), the SER does not always improve by increasing the
SNR. On the contrary, the SER performance of UP-LMMSE improves as we increase
SNR.

In the downlink, the SER of DL-GAN hits a floor limit as we increase SNR. This
is due to the fact that the increase in SNR does not always improve the
channel estimate in DL-GAN (see Fig. 3). This observation is mainly derived by
the limits imposed by $G_{{\cal W}_{g}}(\cdot)$. Ignoring the role of
$G_{{\cal W}_{g}}({\bf z})$, as to what DL-Modified-R2F2 does, yields a huge
SER loss. Overall, our DGM-based technique yields a performance gain at low to
mid range of SNR.

Figure 9: NMSE vs $\sigma_{\phi}$, $|\mathcal{K}_{\rm up}|=|\mathcal{K}_{\rm
dl}|=K=16$, $|\mathcal{M}_{\rm dl}|=64$, and $p=|\mathcal{K}_{\rm dl}|$.

####  VI-D4 Feedback error

So far we assumed that there is no error when the estimates of
$\boldsymbol{\phi}^{\rm dl}$ are fed back to the BS. However, to show the
effect of feedback error, we assume that the BS receives
$\boldsymbol{\tilde{\phi}}^{\rm dl}$, a noisy version of the
$\boldsymbol{\phi}^{\rm dl}$, i.e., $\boldsymbol{\tilde{\phi}}^{\rm
dl}=\boldsymbol{\phi}^{\rm dl}+\varepsilon_{\phi}$, where $\varepsilon_{\phi}$
is a Gaussian random variable with zero mean and standard deviation
$\sigma_{\phi}$. In Fig. 9, we plot the downlink channel NMSE versus
$\sigma_{\phi}$, for ${\rm SNR}=10,\,20\,{\rm(dB)}$. As expected, considering
the feedback error will degrade the downlink channel NMSE. However, this
degradation is not the same for different SNRs. As we increase
$\sigma_{\phi}$, the NMSE is considerably less affected at ${\rm SNR}=20$ dB
compared to that in ${\rm SNR}=10$ dB. This is mainly because, at high SNR,
the estimates of channel parameters have a better quality compared to low SNR
regimes.

Remark 3: The proposed channel estimation technique in this paper is not
limited to ULAs, and it can be extended to 2D arrays. As long as the model
relating the angles to received signals is known (i.e., the “steering vector”)
our approach will remain valid. We would have to estimate more parameters such
as the elevation and azimuth angles.

##  VII Conclusions

In this paper, we proposed a deep generative model (DGM)-based technique for
FDD massive MIMO downlink channel estimation. The proposed technique relies on
the partial reciprocity between the uplink and downlink channels, meaning that
a portion of the underlying channel parameters are frequency-independent, and
they are shared in both uplink and downlink channels. These parameters are
estimated via the uplink training. Then, the frequency-specific channel
parameters are estimated via downlink training using a very short training
signal. To do so, we assume that the frequency-independent channel parameters
can be modeled using some probability distribution, which is learned using
DGMs. We showed that our proposed DGM-based channel estimation outperforms the
conventional channel estimation techniques in practical ranges of SNR. Our
technique also yields a near-optimal performance using only few pilot
measurements, indicating a significant reduction in training and feedback
overhead in FDD massive MIMO systems. The work in this paper can be further
extended to incorporate the already-existing DoA estimation techniques in
estimating the channel parameters as well as considering more practical
scenarios such as RF calibration errors.

## References

  * [1] T. L. Marzetta, “Non-cooperative cellular wireless with unlimited numbers of base station antennas,” _IEEE Trans. Wireless Commun._ , vol. 9, no. 11, pp. 3590–3600, Nov. 2010.
  * [2] F. Rusek, D. Persson, B. K. Lau, E. G. Larsson, T. L. Marzetta, O. Edfors, and F. Tufvesson, “Scaling up MIMO: Opportunities and challenges with very large arrays,” _IEEE Signal Process. Mag._ , vol. 30, no. 1, pp. 40–60, Jan. 2013.
  * [3] Jiann-Ching Guey and L. D. Larsson, “Modeling and evaluation of mimo systems exploiting channel reciprocity in TDD mode,” in _IEEE Veh. Tech. Conf._ , vol. 6, Sep. 2004, pp. 4265–4269.
  * [4] J. Jose, A. Ashikhmin, T. L. Marzetta, and S. Vishwanath, “Pilot contamination and precoding in multi-cell TDD systems,” _IEEE Trans. Wireless Commun._ , vol. 10, no. 8, pp. 2640–2651, Aug.ust 2011.
  * [5] E. G. Larsson, O. Edfors, F. Tufvesson, and T. L. Marzetta, “Massive MIMO for next generation wireless systems,” _IEEE Commun. Mag._ , vol. 52, no. 2, pp. 186–195, February 2014.
  * [6] B. Hochwald and T. Maretta, “Adapting a downlink array from uplink measurements,” _IEEE Trans. Signal Process._ , vol. 49, no. 3, pp. 642–653, 2001.
  * [7] G. Raleigh and V. Jones, “Adaptive antenna transmission for frequency duplex digital wireless communication,” in _Int. Conf. Commun. (ICC)_ , vol. 2, 1997, pp. 641–646 vol.2.
  * [8] R. W. Heath, N. González-Prelcic, S. Rangan, W. Roh, and A. M. Sayeed, “An overview of signal process. techniques for millimeter wave MIMO systems,” _IEEE J. Sel. Topics Signal Process._ , vol. 10, no. 3, pp. 436–453, Apr. 2016.
  * [9] B. Fleury, M. Tschudin, R. Heddergott, D. Dahlhaus, and K. Ingeman Pedersen, “Channel parameter estimation in mobile radio environments using the SAGE algorithm,” _IEEE J. Sel. Areas Commun._ , vol. 17, no. 3, pp. 434–450, 1999\.
  * [10] P. Stoica and A. Nehorai, “MUSIC, maximum likelihood, and cramer-rao bound,” _IEEE Trans. Acoust., Speech, Signal Process._ , vol. 37, no. 5, pp. 720–741, 1989.
  * [11] R. Roy and T. Kailath, “ESPRIT-estimation of signal parameters via rotational invariance techniques,” _IEEE Trans. Acoust., Speech, Signal Process._ , vol. 37, no. 7, pp. 984–995, 1989.
  * [12] C. Qian, X. Fu, N. D. Sidiropoulos, and Y. Yang, “Tensor-based channel estimation for dual-polarized massive MIMO systems,” _IEEE Trans. Signal Process._ , vol. 66, no. 24, pp. 6390–6403, 2018.
  * [13] W. U. Bajwa, J. Haupt, A. M. Sayeed, and R. Nowak, “Compressed channel sensing: A new approach to estimating sparse multipath channels,” _Proceedings of the IEEE_ , vol. 98, no. 6, pp. 1058–1076, June 2010.
  * [14] A. Alkhateeb, O. E. Ayach, G. Leus, and R. W. Heath, “Channel estimation and hybrid precoding for millimeter wave cellular systems,” _IEEE J. Sel. Topics Signal Process._ , vol. 8, no. 5, pp. 831–846, Oct. 2014.
  * [15] C. R. Berger, Z. Wang, J. Huang, and S. Zhou, “Application of compressive sensing to sparse channel estimation,” _IEEE Commun. Mag._ , vol. 48, no. 11, pp. 164–174, 2010.
  * [16] J. Lee, G. Gil, and Y. H. Lee, “Channel estimation via orthogonal matching pursuit for hybrid MIMO systems in millimeter wave communications,” _IEEE Trans. Commun._ , vol. 64, no. 6, pp. 2370–2386, 2016\.
  * [17] X. Rao and V. K. N. Lau, “Distributed compressive CSIT estimation and feedback for FDD multi-user massive MIMO systems,” _IEEE Trans. Signal Process._ , vol. 62, no. 12, pp. 3261–3271, June 2014.
  * [18] P. N. Alevizos, X. Fu, N. D. Sidiropoulos, Y. Yang, and A. Bletsas, “Limited feedback channel estimation in massive MIMO with non-uniform directional dictionaries,” _IEEE Trans. Signal Process._ , vol. 66, no. 19, pp. 5127–5141, 2018.
  * [19] Y. Han, T. Hsu, C. Wen, K. Wong, and S. Jin, “Efficient downlink channel reconstruction for FDD multi-antenna systems,” _IEEE Trans. on Wireless Commun._ , vol. 18, no. 6, pp. 3161–3176, June 2019.
  * [20] Y. Han, Q. Liu, C. Wen, S. Jin, and K. Wong, “FDD massive MIMO based on efficient downlink channel reconstruction,” _IEEE Trans. Commun._ , vol. 67, no. 6, pp. 4020–4034, 2019.
  * [21] W. Huang, Z. Lu, C. Zhang, Y. Huang, S. Jin, and L. Yang, “Beam-blocked compressive channel estimation for FDD massive MIMO systems,” in _IEEE Wireless Commun. Networking Conf. (WCNC), Doha,_ , 2016, pp. 1–6.
  * [22] J. Dai, A. Liu, and V. K. N. Lau, “FDD massive MIMO channel estimation with arbitrary 2D-array geometry,” _IEEE Trans. Signal Process._ , vol. 66, no. 10, pp. 2584–2599, 2018.
  * [23] X. Cheng, J. Sun, and S. Li, “Channel estimation for FDD multi-user massive MIMO: A variational Bayesian inference-based approach,” _IEEE Trans. Wireless Commun._ , vol. 16, no. 11, pp. 7590–7602, 2017.
  * [24] C. Tseng, J. Wu, and T. Lee, “Enhanced compressive downlink CSI recovery for FDD massive MIMO systems using weighted block ${\ell_{1}}$-minimization,” _IEEE Trans. Commun._ , vol. 64, no. 3, pp. 1055–1067, 2016.
  * [25] Y. Han, J. Lee, and D. J. Love, “Compressed sensing-aided downlink channel training for FDD massive MIMO systems,” _IEEE Trans. Commun._ , vol. 65, no. 7, pp. 2852–2862, 2017.
  * [26] A. Liu, V. K. N. Lau, and W. Dai, “Exploiting burst-sparsity in massive MIMO with partial channel support information,” _IEEE Trans. Wireless Commun._ , vol. 15, no. 11, pp. 7820–7830, 2016.
  * [27] H. Xie, F. Gao, S. Zhang, and S. Jin, “A unified transmission strategy for TDD/FDD massive MIMO systems with spatial basis expansion model,” _IEEE Trans. Veh. Tech._ , vol. 66, no. 4, pp. 3170–3184, 2017.
  * [28] W. Peng, W. Li, W. Wang, X. Wei, and T. Jiang, “Downlink channel prediction for time-varying FDD massive MIMO systems,” _IEEE J. Sel. Topics Signal Process._ , vol. 13, no. 5, pp. 1090–1102, 2019.
  * [29] F. Rottenberg, R. Wang, J. Zhang, and A. F. Molisch, “Channel extrapolation in FDD massive MIMO: Theoretical analysis and numerical validation,” in _IEEE Global Commun. Conf. (GLOBECOM), Waikoloa, HI, USA,_ , 2019, pp. 1–7.
  * [30] D. Vasisht, S. Kumar, H. Rahul, and D. Katabi, “Eliminating channel feedback in next-generation cellular networks,” in _Proc. ACM SIGCOMM Conf._ , Aug. 2016, pp. 398–411.
  * [31] Z. Zhong, L. Fan, and S. Ge, “FDD massive MIMO uplink and downlink channel reciprocity properties: Full or partial reciprocity?” in _IEEE Global Commun. Conf. (GLOBECOM), Taipei, Taiwan_ , Dec. 2020.
  * [32] S. Kim, J. W. Choi, and B. Shim, “Downlink pilot precoding and compressed channel feedback for FDD-based cell-free systems,” _IEEE Trans. Wireless Commun._ , vol. 19, no. 6, pp. 3658–3672, 2020.
  * [33] H. Xie, F. Gao, and S. Jin, “An overview of low-rank channel estimation for massive MIMO systems,” _IEEE Access_ , vol. 4, pp. 7313–7321, 2016\.
  * [34] H. Ye, G. Y. Li, and B. Juang, “Power of deep learning for channel estimation and signal detection in OFDM systems,” _IEEE Wireless Commun.Lett._ , vol. 7, no. 1, pp. 114–117, 2018.
  * [35] Z. Jia, W. Cheng, and H. Zhang, “A partial learning-based detection scheme for massive MIMO,” _IEEE Wireless Commun. Lett._ , vol. 8, no. 4, pp. 1137–1140, 2019.
  * [36] Y. Yang, F. Gao, X. Ma, and S. Zhang, “Deep learning-based channel estimation for doubly selective fading channels,” _IEEE Access_ , vol. 7, pp. 36 579–36 589, 2019.
  * [37] W. Xu, F. Gao, S. Jin, and A. Alkhateeb, “3D scene-based beam selection for mmWave communications,” _IEEE Wireless Commun. Lett._ , vol. 9, no. 11, pp. 1850–1854, 2020.
  * [38] X. Li and A. Alkhateeb, “Deep learning for direct hybrid precoding in millimeter wave massive MIMO systems,” in _Proc. IEEE Asilomar Conf. Signal, Syst., Compute., Pacific Grove, CA, US,_ , 2019, pp. 800–805.
  * [39] M. Alrabeiah and A. Alkhateeb, “Deep learning for mmWave beam and blockage prediction using sub-6 GHz channels,” _IEEE Trans. Commun._ , vol. 68, no. 9, pp. 5504–5518, 2020.
  * [40] P. Dong, H. Zhang, and G. Y. Li, “Machine learning prediction based CSI acquisition for FDD massive MIMO downlink,” in _Proc. IEEE Global Commun. Conf. (GLOBECOM), Abu Dhabi, UAE,_ , 2018, pp. 1–6.
  * [41] M. Alrabeiah and A. Alkhateeb, “Deep learning for TDD and FDD massive MIMO: Mapping channels in space and frequency,” in _IEEE Asilomar Conf. Signal, Syst., Compute., Pacific Grove, CA, US,_ , 2019, pp. 1465–1470.
  * [42] Y. Yang, F. Gao, G. Y. Li, and M. Jian, “Deep learning-based downlink channel prediction for FDD massive MIMO system,” _IEEE Commun. Lett._ , vol. 23, no. 11, pp. 1994–1998, 2019.
  * [43] H. Choi and J. Choi, “Downlink extrapolation for FDD multiple antenna systems through neural network using extracted uplink path gains,” _IEEE Access_ , vol. 8, pp. 67 100–67 111, 2020.
  * [44] T. S. Rappaport, S. Sun, R. Mayzus, H. Zhao, Y. Azar, K. Wang, G. N. Wong, J. K. Schulz, M. Samimi, and F. Gutierrez, “Millimeter wave mobile communications for 5G cellular: It will work!” _IEEE Access_ , vol. 1, pp. 335–349, 2013.
  * [45] M. R. Akdeniz, Y. Liu, M. K. Samimi, S. Sun, S. Rangan, T. S. Rappaport, and E. Erkip, “Millimeter wave channel modeling and cellular capacity evaluation,” _IEEE J. Sel. Areas Commun._ , vol. 32, no. 6, pp. 1164–1179, 2014.
  * [46] Y. Han, Q. Liu, C. Wen, M. Matthaiou, and X. Ma, “Tracking FDD massive MIMO downlink channels by exploiting delay and angular reciprocity,” _IEEE J. Sel. Topics Signal Process._ , vol. 13, no. 5, pp. 1062–1076, 2019.
  * [47] P. Zhao, K. Ma, Z. Wang, and S. Chen, “Virtual angular-domain channel estimation for FDD based massive MIMO systems with partial orthogonal pilot design,” _IEEE Trans. Veh. Tech._ , vol. 69, no. 5, pp. 5164–5178, 2020.
  * [48] S. Imtiaz, G. S. Dahman, F. Rusek, and F. Tufvesson, “On the directional reciprocity of uplink and downlink channels in frequency division duplex systems,” in _IEEE Symp. Personal, Indoor, Mobile Radio Commun.(PIMRC), Washington, DC, US_ , 2014, pp. 172–176.
  * [49] M. Arnold, S. Dörner, S. Cammerer, S. Yan, J. Hoydis, and S. t. Brink, “Enabling FDD massive MIMO through deep learning-based channel prediction,” 2019. [Online]. Available: http://arxiv.org/abs/1901.03664
  * [50] A. Doshi, E. Balevi, and J. G. Andrews, “Compressed representation of high dimensional channels using deep generative networks,” in _Proc. Int. Workshop Signal Process. Adv. in Wireless Commun. (SPAWC), Atlanta, GA, USA,_ , 2020, pp. 1–5.
  * [51] A. Alkhateeb, “DeepMIMO: A generic deep learning dataset for millimeter wave and massive MIMO applications,” in _Proc. of Information Theory and Applications Workshop (ITA)_ , San Diego, CA, Feb 2019, pp. 1–8.
  * [52] C. T. Kelley, _Iterative Methods for Optimization_. Society for Industrial and Applied Mathematics, 1999. [Online]. Available: https://epubs.siam.org/doi/abs/10.1137/1.9781611970920
  * [53] D. Vasisht, S. Kumar, H. Rahul, and D. Katabi, “Eliminating channel feedback in next-generation cellular networks,” _GetMobile: Mobile Computing and Communications_ , vol. 21, pp. 26–30, 11 2017.
  * [54] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in _Advances in Neural Information Processing Systems_ , vol. 27, 2014, pp. 2672–2680.
  * [55] T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen, “Improved techniques for training (gan)s,” in _Proc. Int. Conf. Neural Info. Process. Systems (NIPS)_. Red Hook, NY, USA: Curran Associates Inc., 2016, p. 2234–2242.
  * [56] T. Che, Y. Li, A. P. Jacob, Y. Bengio, and W. Li, “Mode regularized generative adversarial networks,” in _5th Int. Conf. Learning Representations (ICLR) Toulon, France_ , Apr. 2017.
  * [57] Remcom, “Wireless InSite,” http://www.remcom.com/wireless-insite.
  * [58] J. Mirzaei, R. S. Adve, and S. Shahbazpanahi, “Semi-blind time-domain channel estimation for frequency-selective multiuser massive MIMO systems,” _IEEE Trans. Commun._ , vol. 67, no. 2, pp. 1045–1058, 2019.
  * [59] J. Mirzaei, R. Adve, and S. ShahbazPanahi, “Semi-blind channel estimation for frequency-selective massive MIMO systems,” in _IEEE Global Commun. Conf. (GLOBECOM), Abu Dhabi, UAE,_ , Dec. 2018.
  * [60] Z. Cvetkovski, _Inequalities. Theorems, techniques and selected problems_ , 11 2012.
  * [61] K. Scaman and A. Virmaux, “Lipschitz regularity of deep neural networks: Analysis and efficient estimation,” in _Proc. Int. Conf. Neural Info. Process. Systems (NIPS)_. Red Hook, NY, USA: Curran Associates Inc., 2018, p. 3839–3848.
  * [62] H. Gouk, E. Frank, B. Pfahringer, and M. J. Cree, “Regularisation of neural networks by enforcing lipschitz continuity,” _Machine Learning_ , vol. 110, no. 2, pp. 393–416, Feb 2021. [Online]. Available: https://doi.org/10.1007/s10994-020-05929-w

##  Appendix A Preliminary definitions and lemmas

######  Definition 1.

A function $f:X\rightarrow Y$ is said to be Lipschitz-continuous if it
satisfies

$\displaystyle D_{Y}\left(f({\bf x}^{(1)}),f({\bf x}^{(2)})\right)\leq
L_{f}D_{X}\left({\bf x}^{(1)},{\bf x}^{(2)}\right),$ (28)

for some real-valued $L_{f}\geq 0$ and distant metrics $D_{X}$ and $D_{Y}$.
The value of $L_{f}$ is known as the Lipschitz constant, and the function can
be referred to as being $L_{f}$-Lipschitz.

######  Lemma 2.

If $f_{1}({\bf x})$ is an $L_{1}$-Lipschitz function, and $f_{2}({\bf x})$ is
an $L_{2}$-Lipschitz function, then $f_{1}(f_{2}({\bf x}))$ is an
$L_{1}L_{2}$-Lipschitz function.

_Proof_ : We can write

$\displaystyle\left|f_{1}(f_{2}({\bf x}^{(i)}))-f_{1}(f_{2}({\bf
x}^{(j)}))\right|\leq L_{1}\left|f_{2}({\bf x}^{(i)})-f_{2}({\bf
x}^{(j)})\right|\leq L_{1}L_{2}\left|{\bf x}^{(i)}-{\bf x}^{(j)}\right|.$

######  Lemma 3.

If $f_{n}({\bf x})$ is be an $L_{n}$-Lipschitz function, then
$f(x)=\sum_{n=1}^{N}f_{n}({\bf x})$ is an $L_{f}$-Lipschitz function, where
$L_{f}\triangleq\max\left\\{L_{1},L_{2},\cdots,L_{N}\right\\}$

_Proof_ : We can write

$\displaystyle\left|f({\bf x}^{(i)})-f({\bf x}^{(j)})\right|$
$\displaystyle=\left|\sum_{n=1}^{N}f_{n}({\bf
x}^{(i)})-\sum_{n=1}^{N}f_{n}({\bf
x}^{(j)})\right|\leq\sum_{n=1}^{N}\left|f_{n}({\bf x}^{(i)})-f_{n}({\bf
x}^{(j)})\right|$ $\displaystyle\leq\sum_{n=1}^{N}L_{n}\left|{\bf
x}^{(i)}-{\bf x}^{(j)}\right|\leq L_{f}\left|{\bf x}^{(i)}-{\bf
x}^{(j)}\right|.$

######  Lemma 4.

(Quadratic and Arithmetic mean inequality): Let $a_{1},a_{2},\cdots,a_{N}$ be
positive real numbers, then

$\displaystyle\sqrt{\frac{a_{1}^{2}+a_{2}^{2}+\cdots+a_{N}^{2}}{n}}\geq\frac{a_{1}+a_{2}+\cdots+a_{N}}{n}.$
(29)

The equality occurs when $a_{1}=a_{2}=\cdots=a_{N}$.

_Proof_ : See [60].

######  Lemma 5.

Given ${\bf x}=\left[{\bf x}_{1}^{T}\;\;{\bf x}_{2}^{T}\right]^{T}$ and ${\bf
y}=\left[{\bf y}_{1}^{T}\;\;{\bf y}_{2}^{T}\right]^{T}$, we have the following
inequality

$\displaystyle\|{\bf x}-{\bf y}\|\leq\|{\bf x}_{1}-{\bf y}_{1}\|+\|{\bf
x}_{2}-{\bf y}_{2}\|,$

where the equality holds when ${\bf x}={\bf y}$.

###### Proof.

We can write

$\displaystyle\|{\bf x}-{\bf y}\|$ $\displaystyle=\sqrt{{\|{\bf x}_{1}-{\bf
y}_{1}\|}^{2}+{\|{\bf x}_{2}-{\bf y}_{2}\|}^{2}}$
$\displaystyle\leq\sqrt{\left(\|{\bf x}_{1}-{\bf y}_{1}\|+\|{\bf x}_{2}-{\bf
y}_{2}\|\right)^{2}}$ $\displaystyle=\|{\bf x}_{1}-{\bf y}_{1}\|+\|{\bf
x}_{2}-{\bf y}_{2}\|.$

∎

##  Appendix B Proof of Lipschitz continuity of gradient of $J_{\rm
up}\left(G_{{\cal W}_{g}}({\bf z}),{\boldsymbol{\phi}^{\rm up}}\right)$

###### Proof.

We prove for $|\mathcal{K}_{\rm up}|=1$. The Lipschitz continuity for
$|\mathcal{K}_{\rm up}|>1$ can be straightforwardly extended. Let us define
${\tilde{{\bf z}}}\triangleq\left({\bf z},\boldsymbol{\phi}\right)$ and
$P(\tilde{{\bf z}})\triangleq\triangledown_{\tilde{{\bf z}}}J_{\rm
up}\left(G_{{\cal W}_{g}}({\bf z}),{\boldsymbol{\phi}}\right)$. We aim to show
that there exists $\lambda_{p}$ such that

$\displaystyle\|P(\tilde{{\bf z}}^{(1)})-P(\tilde{{\bf
z}}^{(2)})\|\leq\lambda_{p}\|\tilde{{\bf z}}^{(1)}-\tilde{{\bf z}}^{(2)}\|.$
(30)

Using the triangle inequality that

$\displaystyle\|f(a_{1},a_{2})-f(b_{1},b_{2})\|$
$\displaystyle=\|f(a_{1},a_{2})-f(a_{1},b_{2})+f(a_{1},b_{2})-f(b_{1},b_{2})\|$
$\displaystyle\leq\|f(a_{1},a_{2})-f(a_{1},b_{2})\|+\|f(a_{1},b_{2})-f(b_{1},b_{2})\|,$
(31)

we can write

$\displaystyle\|P({{\bf z}}^{(1)},{\boldsymbol{\phi}^{(1)}})-P({{\bf
z}}^{(2)},{\boldsymbol{\phi}^{(2)}})\|\leq\underbrace{\|P({{\bf
z}}^{(1)},{\boldsymbol{\phi}^{(1)}})-P({{\bf
z}}^{(2)},{\boldsymbol{\phi}^{(1)}})\|}_{I_{{\bf z}}}+\underbrace{\|P({{\bf
z}}^{(2)},{\boldsymbol{\phi}^{(1)}})-P({{\bf
z}}^{(2)},{\boldsymbol{\phi}^{(2)}})\|}_{I_{\boldsymbol{\phi}}}.$ (32)

In the next subsection, we show that $I_{{\bf z}}$ and $I_{\boldsymbol{\phi}}$
are bounded from above.

###  B-A Bounds on $I_{{\bf z}}$

For fixed $\boldsymbol{\phi}$, we denote $f(G_{{\cal W}_{g}}({\bf
z}))\triangleq\triangledown_{\tilde{{\bf z}}}J_{\rm up}\left(G_{{\cal
W}_{g}}({\bf z}),{\boldsymbol{\phi}}\right)$ a vector-valued composition
function. To prove the Lipschitz-continuity of $f(G_{{\cal W}_{g}}(\cdot))$,
we show that $f(\cdot)$ and $G_{{\cal W}_{g}}(\cdot)$ are both Lipschitz-
continuous, and therefore, according to Lemma 2, their composition is also a
Lipschitz-continuous function.

Given ${\bf x}=G_{{\cal W}_{g}}({\bf z})$ and defining ${\tilde{{\bf
x}}}\triangleq\left({\bf x},\boldsymbol{\phi}\right)$, for fixed
$\boldsymbol{\phi}$, we first show the Lipschitz-continuity of $f({\bf
x})\triangleq\triangledown_{\tilde{{\bf x}}}J_{\rm up}\left({\bf
x},{\boldsymbol{\phi}}\right)$. We can write $f({\bf
x})=\sum_{m=1}^{M}f_{m}({\bf x})$, where $f_{m}({\bf x}),$ for
$m=1,1,\cdots,M$, is given as

$\displaystyle f_{m}({\bf x})\triangleq-2\left(y^{\rm
up}_{m}-h_{m}(\tilde{{\bf x}})s_{k}\right)\triangledown_{\tilde{{\bf
x}}}h_{m}(\tilde{{\bf x}}),$ (33)

where $y^{\rm up}_{m}$ is the received signal at the $m$th antenna in the
uplink, $h_{m}(\tilde{{\bf
x}})\triangleq\sum_{l=1}^{L}\alpha_{l}e^{j(\phi_{l}+\beta\tau_{l}+\gamma_{m}\sin{\theta_{l}})}$,
$\beta\triangleq\frac{2\pi k}{K}B$,
$\gamma_{m}\triangleq\frac{2\pi}{\lambda}md$, for $m=0,1,\cdots,M-1$. Also,

$\displaystyle\triangledown_{\tilde{{\bf x}}}h_{m}(\tilde{{\bf x}})=$
$\displaystyle\left[\underbrace{\frac{\partial h_{m}(\tilde{{\bf
x}})}{\partial\alpha_{1}},\cdots,\frac{\partial h_{m}(\tilde{{\bf
x}})}{\partial\alpha_{L}}}_{=\triangledown_{{\boldsymbol{\alpha}}}h_{m}(\tilde{{\bf
x}})},\underbrace{\frac{\partial h_{m}(\tilde{{\bf
x}})}{\partial\tau_{1}},\cdots,\frac{\partial h_{m}(\tilde{{\bf
x}})}{\partial\tau_{L}}}_{=\triangledown_{\boldsymbol{\tau}}h_{m}(\tilde{{\bf
x}})},\underbrace{\frac{\partial h_{m}(\tilde{{\bf
x}})}{\partial\theta_{1}},\cdots,\frac{\partial h_{m}(\tilde{{\bf
x}})}{\partial\theta_{L}}}_{=\triangledown_{{\boldsymbol{\theta}}}h_{m}(\tilde{{\bf
x}})},\underbrace{\frac{\partial h_{m}(\tilde{{\bf
x}})}{\partial\phi_{1}},\cdots,\frac{\partial h_{m}(\tilde{{\bf
x}})}{\partial\phi_{L}}}_{=\triangledown_{\boldsymbol{\phi}}h_{m}(\tilde{{\bf
x}})}\right]^{T},$ (34)

where, for $l=1,2,\cdots,L$, and defining
$\omega_{l}\triangleq\phi_{l}+\beta\tau_{l}+\gamma_{m}\sin{\theta_{l}}$, the
individual partial derivatives are given by

$\displaystyle\frac{\partial h_{m}(\tilde{{\bf x}})}{\partial\alpha_{l}}$
$\displaystyle=e^{j\omega_{l}}$ (35) $\displaystyle\frac{\partial
h_{m}(\tilde{{\bf x}})}{\partial\tau_{l}}$
$\displaystyle=j\beta\alpha_{l}e^{j\omega_{l}}$ (36)
$\displaystyle\frac{\partial h_{m}(\tilde{{\bf x}})}{\partial\theta_{l}}$
$\displaystyle=j\gamma_{m}\alpha_{l}\cos{\theta_{l}}e^{j\omega_{l}}$ (37)
$\displaystyle\frac{\partial h_{m}(\tilde{{\bf x}})}{\partial\phi_{l}}$
$\displaystyle=j\alpha_{l}e^{j\omega_{l}}.$ (38)

To prove the the Lipschitz-continuity of $f_{m}({\bf x})$, according to
Definition 1, and using the $2$-norm as the metric, we show that there exists
a finite $\lambda_{m}\geq 0$, such that

$\displaystyle\|f_{m}({\bf x}^{(1)})-f_{m}({\bf
x}^{(2)})\|\leq\lambda_{m}\|{\bf x}^{(1)}-{\bf x}^{(2)}\|,$ (39)

where ${\bf
x}^{(i)}\triangleq\left({\boldsymbol{\alpha}}^{(i)},\boldsymbol{\tau}^{(i)},{\boldsymbol{\theta}}^{(i)}\right)$
for $i=1,2$. Using (B), we express the left hand side in (39) as

$\displaystyle\|f_{m}({\bf x}^{(1)})-f_{m}({\bf x}^{(2)})\|$
$\displaystyle\leq\underbrace{\|f_{m}({\boldsymbol{\alpha}}^{(1)},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)})-f_{m}({\boldsymbol{\alpha}}^{(2)},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)})\|}_{\triangleq
T_{\alpha}}$
$\displaystyle+\underbrace{\|f_{m}({\boldsymbol{\alpha}}^{(2)},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)})-f_{m}({\boldsymbol{\alpha}}^{(2)},\boldsymbol{\tau}^{(2)},{\boldsymbol{\theta}}^{(1)})\|}_{\triangleq
T_{\tau}}$
$\displaystyle+\underbrace{\|f_{m}({\boldsymbol{\alpha}}^{(2)},\boldsymbol{\tau}^{(2)},{\boldsymbol{\theta}}^{(1)})-f_{m}({\boldsymbol{\alpha}}^{(2)},\boldsymbol{\tau}^{(2)},{\boldsymbol{\theta}}^{(2)})\|}_{\triangleq
T_{\theta}}.$ (40)

Next, we show that, each term on the right hand side of (B-A) is bounded.

####  B-A1 Bounds on $T_{\alpha}$

Here, we assume that $\boldsymbol{\tau}^{(1)}$ and
${\boldsymbol{\theta}}^{(1)}$ are fixed. Using the Triangle inequality of (B),
we can write

$\displaystyle T_{\alpha}$
$\displaystyle\leq\|f_{m}(\alpha_{1}^{(1)},\bar{{\boldsymbol{\alpha}}}_{1},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)})-f_{m}(\alpha_{1}^{(2)},\bar{{\boldsymbol{\alpha}}}_{1},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)})\|$
$\displaystyle+\|f_{m}(\alpha_{2}^{(1)},\bar{{\boldsymbol{\alpha}}}_{2},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)})-f_{m}(\alpha_{2}^{(2)},\bar{{\boldsymbol{\alpha}}}_{2},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)})\|$
$\displaystyle\qquad\qquad\qquad\vdots$
$\displaystyle+\|f_{m}(\alpha_{L}^{(1)},\bar{{\boldsymbol{\alpha}}}_{L},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)})-f_{m}(\alpha_{L}^{(2)},\bar{{\boldsymbol{\alpha}}}_{L},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)})\|,$
(41)

where
$\bar{{\boldsymbol{\alpha}}}_{l}\triangleq\left(\alpha_{1}^{(2)},\cdots,\alpha_{l-1}^{(2)},\alpha_{l+1}^{(1)},\cdots,\alpha_{L}^{(1)}\right)$.
Let us define
$T_{\alpha_{l}}\triangleq\|f_{m}(\alpha_{l}^{(1)},\bar{{\boldsymbol{\alpha}}}_{l},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)})-f_{m}(\alpha_{l}^{(2)},\bar{{\boldsymbol{\alpha}}}_{l},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)})\|$.
We now aim to show that there exists a bounded $M_{\alpha_{l}}\geq 0$, such
that $T_{\alpha_{l}}\leq M_{\alpha_{l}}\|\alpha_{l}^{(1)}-\alpha_{l}^{(2)}\|$.
To do so, we note that

$\displaystyle T_{\alpha_{l}}=2\|$ $\displaystyle\left(y^{\rm
up}_{m}-h_{m}(\alpha_{l}^{(1)},\bar{{\boldsymbol{\alpha}}}_{l},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)})s_{k}\right)\triangledown
h_{m}(\alpha_{l}^{(1)},\bar{{\boldsymbol{\alpha}}}_{l},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)})$
$\displaystyle-$ $\displaystyle\left(y^{\rm
up}_{m}-h_{m}(\alpha_{l}^{(2)},\bar{{\boldsymbol{\alpha}}}_{l},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)})s_{k}\right)\triangledown
h_{m}(\alpha_{l}^{(2)},\bar{{\boldsymbol{\alpha}}}_{l},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)})\|$
$\displaystyle\overset{(a)}{\leq}2\|$ $\displaystyle\left(y^{\rm
up}_{m}-h_{m}(\alpha_{l}^{(1)},\bar{{\boldsymbol{\alpha}}}_{l},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)})s_{k}\right)\bar{{\bf
v}}_{\alpha_{l}}-\left(y^{\rm
up}_{m}-h_{m}(\alpha_{l}^{(2)},\bar{{\boldsymbol{\alpha}}}_{l},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)})s_{k}\right)\bar{{\bf
v}}_{\alpha_{l}}\|$ $\displaystyle+2\|$ $\displaystyle\left(y^{\rm
up}_{m}-h_{m}(\alpha_{l}^{(1)},\bar{{\boldsymbol{\alpha}}}_{l},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)})s_{k}\right){\bf
v}_{\alpha_{l}}^{(1)}-\left(y^{\rm
up}_{m}-h_{m}(\alpha_{l}^{(2)},\bar{{\boldsymbol{\alpha}}}_{l},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)})s_{k}\right){\bf
v}_{\alpha_{l}}^{(2)}\|$ $\displaystyle\overset{(b)}{=}2\|$
$\displaystyle\bar{{\bf
v}}_{\alpha_{l}}s_{k}\|\left|h_{m}(\alpha_{l}^{(1)},\bar{{\boldsymbol{\alpha}}}_{l},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)})-h_{m}(\alpha_{l}^{(2)},\bar{{\boldsymbol{\alpha}}}_{l},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)})\right|$
$\displaystyle+2\|$ $\displaystyle\left(y^{\rm
up}_{m}-h_{m}(\alpha_{l}^{(1)},\bar{{\boldsymbol{\alpha}}}_{l},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)})s_{k}\right){\alpha_{l}}^{(1)}\check{{{\bf
v}}}_{\alpha_{l}}-\left(y^{\rm
up}_{m}-h_{m}(\alpha_{l}^{(2)},\bar{{\boldsymbol{\alpha}}}_{l},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)})s_{k}\right){\alpha_{l}}^{(2)}\check{{{\bf
v}}}_{\alpha_{l}}\|$ $\displaystyle\overset{(c)}{\leq}2\|$
$\displaystyle\bar{{\bf
v}}_{\alpha_{l}}s_{k}\|\left|h_{m}(\alpha_{l}^{(1)},\bar{{\boldsymbol{\alpha}}}_{l},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)})-h_{m}(\alpha_{l}^{(2)},\bar{{\boldsymbol{\alpha}}}_{l},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)})\right|$
$\displaystyle+2\|$ $\displaystyle y^{\rm up}_{m}\check{{{\bf
v}}}_{\alpha_{l}}\|\left|{\alpha_{l}}^{(1)}-{\alpha_{l}}^{(2)}\right|+2\|\check{{{\bf
v}}}_{\alpha_{l}}s_{k}\|\left|h_{m}(\alpha_{l}^{(1)},\bar{{\boldsymbol{\alpha}}}_{l},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)}){\alpha_{l}}^{(1)}-h_{m}(\alpha_{l}^{(2)},\bar{{\boldsymbol{\alpha}}}_{l},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)}){\alpha_{l}}^{(2)}\right|$
$\displaystyle\overset{(d)}{=}2\|$ $\displaystyle\bar{{\bf
v}}_{\alpha_{l}}s_{k}\|\left|{\alpha_{l}}^{(1)}-{\alpha_{l}}^{(2)}\right|$
$\displaystyle+2\|$ $\displaystyle y^{\rm up}_{m}\check{{{\bf
v}}}_{\alpha_{l}}\|\left|{\alpha_{l}}^{(1)}-{\alpha_{l}}^{(2)}\right|+2\|\check{{{\bf
v}}}_{\alpha_{l}}s_{k}\|\left|{\alpha_{l}}^{(1)}+{\alpha_{l}}^{(2)}\right|\left|{\alpha_{l}}^{(1)}-{\alpha_{l}}^{(2)}\right|$
$\displaystyle=2(\|$ $\displaystyle\bar{{\bf v}}_{\alpha_{l}}s_{k}\|+\|y^{\rm
up}_{m}\check{{{\bf v}}}_{\alpha_{l}}\|+\|\check{{{\bf
v}}}_{\alpha_{l}}s_{k}\|\left|{\alpha_{l}}^{(1)}+{\alpha_{l}}^{(2)}\right|)\left|{\alpha_{l}}^{(1)}-{\alpha_{l}}^{(2)}\right|$
$\displaystyle\overset{(e)}{\triangleq}M_{\alpha_{l}}$
$\displaystyle\left|\alpha_{l}^{(1)}-\alpha_{l}^{(2)}\right|,$ (42)

where the inequality in $(a)$ follows from Lemma 5, ${{\bf
v}}_{\alpha_{l}}^{(i)}$, $i=1,2$, is a vector that captures those entries of
$\triangledown
h_{m}(\alpha_{l}^{(i)},\bar{{\boldsymbol{\alpha}}}_{l},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)})$
which are function of $\alpha_{l}^{(i)}$, and $\bar{{\bf v}}_{\alpha_{l}}$ is
a vector that captures those entries of $\triangledown
h_{m}(\alpha_{l}^{(1)},\bar{{\boldsymbol{\alpha}}}_{l},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)})$
which are independent of $\alpha_{l}$. The second term in $(b)$ follows from
${{\bf v}}_{\alpha_{l}}^{(i)}={\alpha_{l}}^{(i)}\check{{{\bf
v}}}_{\alpha_{l}}\triangleq{\alpha_{l}}^{(i)}e^{j\omega_{l}}\left[j,j\beta,j\gamma_{m}\cos\theta_{l}\right]^{T}$.
The inequality in $(c)$ follows from the fact that $\|{\bf a}-{\bf
b}\|\leq\|{\bf a}\|+\|{\bf b}\|$ for ${\bf a}$ and ${\bf b}$ to be arbitrary
vectors. In $(d)$, we use the facts that
$\left|h_{m}(\alpha_{l}^{(1)},\bar{{\boldsymbol{\alpha}}}_{l},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)})-h_{m}(\alpha_{l}^{(2)},\bar{{\boldsymbol{\alpha}}}_{l},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)})\right|=\left|{\alpha_{l}}^{(1)}-{\alpha_{l}}^{(2)}\right|$
as well as
$\left|h_{m}(\alpha_{l}^{(1)},\bar{{\boldsymbol{\alpha}}}_{l},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)}){\alpha_{l}}^{(1)}-h_{m}(\alpha_{l}^{(2)},\bar{{\boldsymbol{\alpha}}}_{l},\boldsymbol{\tau}^{(1)},{\boldsymbol{\theta}}^{(1)}){\alpha_{l}}^{(2)}\right|=\left|\left({\alpha_{l}}^{(1)}+{\alpha_{l}}^{(2)}\right)\left({\alpha_{l}}^{(1)}-{\alpha_{l}}^{(2)}\right)\right|$.
Note that $M_{\alpha_{l}}$ in $(e)$ is bounded. This is mainly due to the fact
that $\left\\{\alpha_{l}\right\\}_{l=1}^{L}$ are being generated from $G({\bf
z})$ which are bounded. Therefore, $M_{\alpha_{l}}\geq 0$ is bounded. Now,
from (B-A2),we obtain that,

$\displaystyle T_{\alpha}\leq$ $\displaystyle
M_{\alpha_{1}}\left|\alpha_{1}^{(1)}-\alpha_{1}^{(2)}\right|+M_{\alpha_{2}}\left|\alpha_{2}^{(1)}-\alpha_{2}^{(2)}\right|+\cdots+M_{\alpha_{L}}\left|\alpha_{L}^{(1)}-\alpha_{L}^{(2)}\right|$
$\displaystyle\overset{(a)}{\leq}$ $\displaystyle
M_{\alpha}\left(\left|\alpha_{1}^{(1)}-\alpha_{1}^{(2)}\right|+\left|\alpha_{2}^{(1)}-\alpha_{2}^{(2)}\right|+\cdots+\left|\alpha_{L}^{(1)}-\alpha_{L}^{(2)}\right|\right)$
$\displaystyle\overset{(b)}{\leq}$ $\displaystyle
M_{\alpha}\sqrt{L}\sqrt{\left(\alpha_{1}^{(1)}-\alpha_{1}^{(2)}\right)^{2}+\left(\alpha_{2}^{(1)}-\alpha_{2}^{(2)}\right)^{2}+\cdots+\left(\alpha_{L}^{(1)}-\alpha_{L}^{(2)}\right)^{2}}$
$\displaystyle=$
$\displaystyle\lambda_{\alpha}\|{\boldsymbol{\alpha}}^{(1)}-{\boldsymbol{\alpha}}^{(2)}\|,$
(43)

where in $(a)$,
$M_{\alpha}\triangleq\max\left\\{M_{\alpha_{1}},M_{\alpha_{2}},\cdots,M_{\alpha_{L}}\right\\}$,
$(b)$ follows from the inequality between quadratic and arithmetic mean given
in Lemma 4, and $\lambda_{\alpha}\triangleq M_{\alpha}\sqrt{L}$.

####  B-A2 Bounds on $T_{\tau}$

Similar to what we did in previous subsection, we assume that
${\boldsymbol{\alpha}}^{(2)}$ and ${\boldsymbol{\theta}}^{(1)}$ are fixed.
Using the triangle inequality of (B), we can write

$\displaystyle T_{\tau}$
$\displaystyle\leq\|f_{m}({\boldsymbol{\alpha}}^{(2)},{\tau}_{1}^{(1)},\bar{\boldsymbol{\tau}}_{1},{\boldsymbol{\theta}}^{(1)})-f_{m}({\boldsymbol{\alpha}}^{(2)},{\tau}_{1}^{(2)},\bar{\boldsymbol{\tau}}_{1},{\boldsymbol{\theta}}^{(1)})\|$
$\displaystyle+\|f_{m}({\boldsymbol{\alpha}}^{(2)},{\tau}_{2}^{(1)},\bar{\boldsymbol{\tau}}_{2},{\boldsymbol{\theta}}^{(1)})-f_{m}({\boldsymbol{\alpha}}^{(2)},{\tau}_{2}^{(2)},\bar{\boldsymbol{\tau}}_{2},{\boldsymbol{\theta}}^{(1)})\|$
$\displaystyle\qquad\qquad\qquad\vdots$
$\displaystyle+\|f_{m}({\boldsymbol{\alpha}}^{(2)},{\tau}_{L}^{(1)},\bar{\boldsymbol{\tau}}_{L},{\boldsymbol{\theta}}^{(1)})-f_{m}({\boldsymbol{\alpha}}^{(2)},{\tau}_{L}^{(2)},\bar{\boldsymbol{\tau}}_{L},{\boldsymbol{\theta}}^{(1)})\|,$
(44)

where
$\bar{\boldsymbol{\tau}}_{l}\triangleq\left(\tau_{1}^{(2)},\cdots,\tau_{l-1}^{(2)},\tau_{l+1}^{(1)},\cdots,\tau_{L}^{(1)}\right)$.
Let us define
$T_{\tau_{l}}\triangleq\|f_{m}({\boldsymbol{\alpha}}^{(2)},{\tau}_{l}^{(1)},\bar{\boldsymbol{\tau}}_{l},{\boldsymbol{\theta}}^{(1)})-f_{m}({\boldsymbol{\alpha}}^{(2)},{\tau}_{l}^{(2)},\bar{\boldsymbol{\tau}}_{l},{\boldsymbol{\theta}}^{(1)})\|$.
We now aim to show that there exist $M_{\tau_{l}}\geq 0$ and bounded, such
that $T_{\tau_{l}}\leq
M_{\tau_{l}}\left|\tau_{l}^{(1)}-\tau_{l}^{(2)}\right|$. To do so, we can
write

$\displaystyle T_{\tau_{l}}=2\|$ $\displaystyle\left(y^{\rm
up}_{m}-h_{m}({\boldsymbol{\alpha}}^{(2)},{\tau}_{l}^{(1)},\bar{\boldsymbol{\tau}}_{l},{\boldsymbol{\theta}}^{(1)})s_{k}\right)\triangledown
h_{m}({\boldsymbol{\alpha}}^{(2)},{\tau}_{l}^{(1)},\bar{\boldsymbol{\tau}}_{l},{\boldsymbol{\theta}}^{(1)})$
$\displaystyle-$ $\displaystyle\left(y^{\rm
up}_{m}-h_{m}({\boldsymbol{\alpha}}^{(2)},{\tau}_{l}^{(2)},\bar{\boldsymbol{\tau}}_{l},{\boldsymbol{\theta}}^{(1)})s_{k}\right)\triangledown
h_{m}({\boldsymbol{\alpha}}^{(2)},{\tau}_{l}^{(2)},\bar{\boldsymbol{\tau}}_{l},{\boldsymbol{\theta}}^{(1)})\|$
$\displaystyle\overset{(a)}{\leq}2\|$ $\displaystyle\left(y^{\rm
up}_{m}-h_{m}({\boldsymbol{\alpha}}^{(2)},{\tau}_{l}^{(1)},\bar{\boldsymbol{\tau}}_{l},{\boldsymbol{\theta}}^{(1)})s_{k}\right)\bar{{\bf
v}}_{\tau_{l}}-\left(y^{\rm
up}_{m}-h_{m}({\boldsymbol{\alpha}}^{(2)},{\tau}_{l}^{(2)},\bar{\boldsymbol{\tau}}_{l},{\boldsymbol{\theta}}^{(1)})s_{k}\right)\bar{{\bf
v}}_{\tau_{l}}\|$ $\displaystyle+2\|$ $\displaystyle\left(y^{\rm
up}_{m}-h_{m}({\boldsymbol{\alpha}}^{(2)},{\tau}_{l}^{(1)},\bar{\boldsymbol{\tau}}_{l},{\boldsymbol{\theta}}^{(1)})s_{k}\right){\bf
v}_{\tau_{l}}^{(1)}-\left(y^{\rm
up}_{m}-h_{m}({\boldsymbol{\alpha}}^{(2)},{\tau}_{l}^{(2)},\bar{\boldsymbol{\tau}}_{l},{\boldsymbol{\theta}}^{(1)})s_{k}\right){\bf
v}_{\tau_{l}}^{(2)}\|$ $\displaystyle\overset{(b)}{=}2\|$
$\displaystyle\bar{{\bf
v}}_{\tau_{l}}s_{k}\|\left|h_{m}({\boldsymbol{\alpha}}^{(2)},{\tau}_{l}^{(1)},\bar{\boldsymbol{\tau}}_{l},{\boldsymbol{\theta}}^{(1)})-h_{m}({\boldsymbol{\alpha}}^{(2)},{\tau}_{l}^{(2)},\bar{\boldsymbol{\tau}}_{l},{\boldsymbol{\theta}}^{(1)})\right|$
$\displaystyle+2\|$ $\displaystyle\left(y^{\rm
up}_{m}-h_{m}({\boldsymbol{\alpha}}^{(2)},{\tau}_{l}^{(1)},\bar{\boldsymbol{\tau}}_{l},{\boldsymbol{\theta}}^{(1)})s_{k}\right)e^{j\beta\tau_{l}^{(1)}}\check{{{\bf
v}}}_{\tau_{l}}-\left(y^{\rm
up}_{m}-h_{m}({\boldsymbol{\alpha}}^{(2)},{\tau}_{l}^{(2)},\bar{\boldsymbol{\tau}}_{l},{\boldsymbol{\theta}}^{(1)})s_{k}\right)e^{j\beta\tau_{l}^{(2)}}\check{{{\bf
v}}}_{\tau_{l}}\|$ $\displaystyle\overset{(c)}{\leq}2\|$
$\displaystyle\bar{{\bf
v}}_{\tau_{l}}s_{k}\|\left|h_{m}({\boldsymbol{\alpha}}^{(2)},{\tau}_{l}^{(1)},\bar{\boldsymbol{\tau}}_{l},{\boldsymbol{\theta}}^{(1)})-h_{m}({\boldsymbol{\alpha}}^{(2)},{\tau}_{l}^{(2)},\bar{\boldsymbol{\tau}}_{l},{\boldsymbol{\theta}}^{(1)})\right|$
$\displaystyle+2\|$ $\displaystyle y^{\rm up}_{m}\check{{{\bf
v}}}_{\tau_{l}}\||e^{j\beta\tau_{l}^{(1)}}-e^{j\beta\tau_{l}^{(2)}}|+2\|\check{{{\bf
v}}}_{\tau_{l}}s_{k}\|\left|h_{m}({\boldsymbol{\alpha}}^{(2)},{\tau}_{l}^{(1)},\bar{\boldsymbol{\tau}}_{l},{\boldsymbol{\theta}}^{(1)})e^{j\beta\tau_{l}^{(1)}}-h_{m}({\boldsymbol{\alpha}}^{(2)},{\tau}_{l}^{(2)},\bar{\boldsymbol{\tau}}_{l},{\boldsymbol{\theta}}^{(1)})e^{j\beta\tau_{l}^{(2)}}\right|$
$\displaystyle\overset{(d)}{\leq}2\|$ $\displaystyle\bar{{\bf
v}}_{\tau_{l}}s_{k}\||{\alpha_{l}}^{(2)}||e^{j\beta\tau_{l}^{(1)}}-e^{j\beta\tau_{l}^{(2)}}|$
$\displaystyle+2\|$ $\displaystyle y^{\rm up}_{m}\check{{{\bf
v}}}_{\tau_{l}}\||e^{j\beta\tau_{l}^{(1)}}-e^{j\beta\tau_{l}^{(2)}}|+2\|\check{{{\bf
v}}}_{\tau_{l}}s_{k}\|\left(\left|\alpha_{l}^{(2)}\right|\left|e^{j2\beta\tau_{l}^{(1)}}-e^{j2\beta\tau_{l}^{(2)}}\right|+\left|\tilde{\alpha}_{l}^{(2)}\right|\left|e^{j\beta\tau_{l}^{(1)}}-e^{j\beta\tau_{l}^{(2)}}\right|\right)$
$\displaystyle=2\Big{(}\|$ $\displaystyle\bar{{\bf
v}}_{\tau_{l}}s_{k}\||{\alpha_{l}}^{(2)}|+\|y^{\rm up}_{m}\check{{{\bf
v}}}_{\tau_{l}}\|+\|\check{{{\bf
v}}}_{\tau_{l}}s_{k}\|\left|\tilde{\alpha}_{l}^{(2)}\right|\Big{)}\left|e^{j\beta\tau_{l}^{(1)}}-e^{j\beta\tau_{l}^{(2)}}\right|$
$\displaystyle+2\|$ $\displaystyle\check{{{\bf
v}}}_{\tau_{l}}s_{k}\|\left|{\alpha}_{l}^{(2)}\right|\left|e^{j2\beta\tau_{l}^{(1)}}-e^{j2\beta\tau_{l}^{(2)}}\right|$
$\displaystyle\overset{(e)}{\leq}2\left|\beta\right|$
$\displaystyle\Big{(}\|\bar{{\bf
v}}_{\tau_{l}}s_{k}\||{\alpha_{l}}^{(2)}|+\|y^{\rm up}_{m}\check{{{\bf
v}}}_{\tau_{l}}\|+\|\check{{{\bf
v}}}_{\tau_{l}}s_{k}\|\left|\tilde{\alpha}_{l}^{(2)}\right|\Big{)}\left|\tau_{l}^{(1)}-\tau_{l}^{(2)}\right|$
$\displaystyle+4\left|\beta\right|$ $\displaystyle\|\check{{{\bf
v}}}_{\tau_{l}}s_{k}\|\left|{\alpha}_{l}^{(2)}\right|\left|\tau_{l}^{(1)}-\tau_{l}^{(2)}\right|$
$\displaystyle\triangleq M_{\tau_{l}}$
$\displaystyle\left|\tau_{l}^{(1)}-\tau_{l}^{(2)}\right|,$ (45)

where the inequality in $(a)$ follows from Lemma 5, ${{\bf
v}}_{\tau_{l}}^{(i)}$, $i=1,2$, is a vector that captures those entries of
$\triangledown
h_{m}({\boldsymbol{\alpha}}^{(2)},{\tau}_{l}^{(i)},\bar{\boldsymbol{\tau}}_{l},{\boldsymbol{\theta}}^{(1)})$
which are functions of $\tau_{l}^{(i)}$, and $\bar{{\bf v}}_{\tau_{l}}$ is a
vector that captures those entries of $\triangledown
h_{m}({\boldsymbol{\alpha}}^{(2)},{\tau}_{l}^{(i)},\bar{\boldsymbol{\tau}}_{l},{\boldsymbol{\theta}}^{(1)})$
which are independent of $\tau_{l}$. The second term in $(b)$ follows from
${{\bf v}}_{\tau_{l}}^{(i)}=e^{j\beta\tau_{l}^{(i)}}\check{{{\bf
v}}}_{\tau_{l}}\triangleq
e^{j\beta\tau_{l}^{(i)}}e^{j(\phi_{l}+\gamma_{m}\sin\theta_{l}^{(1)})}\left[1,j\beta\alpha^{(2)},j\gamma_{m}\cos\theta_{l}^{(1)},j\alpha^{(2)}\right]^{T}$.
The inequality in $(c)$ follows from the fact that $\|{\bf a}-{\bf
b}\|\leq\|{\bf a}\|+\|{\bf b}\|$ for ${\bf a}$ and ${\bf b}$ to be arbitrary
vectors. In $(d)$, we use the following inequalities:

$\displaystyle\left|h_{m}({\boldsymbol{\alpha}}^{(2)},{\tau}_{l}^{(1)},\bar{\boldsymbol{\tau}}_{l},{\boldsymbol{\theta}}^{(1)})-h_{m}({\boldsymbol{\alpha}}^{(2)},{\tau}_{l}^{(2)},\bar{\boldsymbol{\tau}}_{l},{\boldsymbol{\theta}}^{(1)})\right|$
$\displaystyle=\left|{\alpha_{l}}^{(2)}e^{j(\phi_{l}+\gamma_{m}\sin\theta_{l}^{(1)})}\left(e^{j\beta\tau_{l}^{(1)}}-e^{j\beta\tau_{l}^{(2)}}\right)\right|$
$\displaystyle\leq|{\alpha_{l}}^{(2)}||e^{j\beta\tau_{l}^{(1)}}-e^{j\beta\tau_{l}^{(2)}}|,$
(46)

as well as

$\displaystyle\left|h_{m}({\boldsymbol{\alpha}}^{(2)},{\tau}_{l}^{(1)},\bar{\boldsymbol{\tau}}_{l},{\boldsymbol{\theta}}^{(1)})e^{j\beta\tau_{l}^{(1)}}-h_{m}({\boldsymbol{\alpha}}^{(2)},{\tau}_{l}^{(2)},\bar{\boldsymbol{\tau}}_{l},{\boldsymbol{\theta}}^{(1)})e^{j\beta\tau_{l}^{(2)}}\right|$
$\displaystyle\leq\left|\alpha_{l}^{(2)}\right|\left|e^{j2\beta\tau_{l}^{(1)}}-e^{j2\beta\tau_{l}^{(2)}}\right|$
$\displaystyle+\left|\tilde{\alpha}_{l}^{(2)}\right|\left|e^{j\beta\tau_{l}^{(1)}}-e^{j\beta\tau_{l}^{(2)}}\right|,$
(47)

where $\tilde{\alpha}_{l}^{(2)}\triangleq\sum_{i\neq
l}\alpha_{i}^{(2)}e^{j\left(\phi_{i}+\gamma_{m}\sin\theta_{i}^{(1)}\right)}$.
The inequality $(e)$, assuming $\tau_{l}^{(1)}\leq\tau_{l}^{(2)}$, follows
from the following inequality:

$\displaystyle\left|e^{j\beta\tau_{l}^{(1)}}-e^{j\beta\tau_{l}^{(2)}}\right|=\left|\int_{\tau_{l}^{(1)}}^{\tau_{l}^{(2)}}j\beta
e^{j\beta t}dt\right|\leq\int_{\tau_{l}^{(1)}}^{\tau_{l}^{(2)}}\left|j\beta
e^{j\beta
t}\right|dt=\left|\beta\right|\left|\tau_{l}^{(1)}-\tau_{l}^{(2)}\right|.$
(48)

As stated before, since $\left\\{\alpha_{l}\right\\}_{l=1}^{L}$ are generated
by $G({\bf z})$ and the range of $G({\bf z})$ is bounded, we conclude that
$M_{\tau_{l}}\geq 0$ and it is bounded. Now, defining,
$M_{\tau}\triangleq\max\left\\{M_{\tau_{1}},M_{\tau_{2}},\cdots,M_{\tau_{L}}\right\\}$,
we obtain the upperbound for $T_{\tau}$ as

$\displaystyle T_{\tau}\leq$ $\displaystyle
M_{\tau_{1}}\left|\tau_{1}^{(1)}-\tau_{1}^{(2)}\right|+M_{\tau_{2}}\left|\tau_{2}^{(1)}-\tau_{2}^{(2)}\right|+\cdots+M_{\tau_{L}}\left|\tau_{L}^{(1)}-\tau_{L}^{(2)}\right|$
$\displaystyle{\leq}$ $\displaystyle
M_{\tau}\left(\left|\tau_{1}^{(1)}-\tau_{1}^{(2)}\right|+\left|\tau_{2}^{(1)}-\tau_{2}^{(2)}\right|+\cdots+\left|\tau_{L}^{(1)}-\tau_{L}^{(2)}\right|\right)$
$\displaystyle\overset{(a)}{\leq}$ $\displaystyle
M_{\tau}\sqrt{L}\sqrt{\left(\tau_{1}^{(1)}-\tau_{1}^{(2)}\right)^{2}+\left(\tau_{2}^{(1)}-\tau_{2}^{(2)}\right)^{2}+\cdots+\left(\tau_{L}^{(1)}-\tau_{L}^{(2)}\right)^{2}}$
$\displaystyle=$
$\displaystyle\lambda_{\tau}\|\boldsymbol{\tau}^{(1)}-\boldsymbol{\tau}^{(2)}\|,$
(49)

where $(a)$ follows from the inequality provided in Lemma 4, and
$\lambda_{\tau}\triangleq M_{\tau}\sqrt{L}$.

####  B-A3 Bounds on $T_{\theta}$

Assuming ${\boldsymbol{\alpha}}^{(2)}$ and $\boldsymbol{\tau}^{(2)}$ are
fixed, and using the triangle inequality of (B), we can write

$\displaystyle T_{\theta}$
$\displaystyle\leq\|f_{m}({\boldsymbol{\alpha}}^{(2)},\boldsymbol{\tau}^{(2)},\theta_{1}^{(1)},\bar{{\boldsymbol{\theta}}}_{1})-f_{m}({\boldsymbol{\alpha}}^{(2)},\boldsymbol{\tau}^{(2)},\theta_{1}^{(2)},\bar{{\boldsymbol{\theta}}}_{1})\|$