# CycleMix: Mixing Source Domains for Domain Generalization in Style-Dependent
Data

Aristotelis Ballas  0000-0003-1683-8433 Department of Informatics and
TelematicsHarokopio UniversityOmirou 9, TavrosAthensGreece<EMAIL_ADDRESS>and
Christos Diou  0000-0002-2461-1928 Department of Informatics and
TelematicsHarokopio UniversityOmirou 9, TavrosAthensGreece<EMAIL_ADDRESS>
(2024)

###### Abstract.

As deep learning-based systems have become an integral part of everyday life,
limitations in their generalization ability have begun to emerge. Machine
learning algorithms typically rely on the i.i.d. assumption, meaning that
their training and validation data are expected to follow the same
distribution, which does not necessarily hold in practice. In the case of
image classification, one frequent reason that algorithms fail to generalize
is that they rely on spurious correlations present in training data, such as
associating image styles with target classes. These associations may not be
present in the unseen test data, leading to significant degradation of their
effectiveness. In this work, we attempt to mitigate this Domain Generalization
(DG) problem by training a robust feature extractor which disregards features
attributed to image-style but infers based on style-invariant image
representations. To achieve this, we train CycleGAN models to learn the
different styles present in the training data and randomly mix them together
to create samples with novel style attributes to improve generalization.
Experimental results on the PACS DG benchmark validate the proposed
method111Code available at: https://github.com/aristotelisballas/cyclemix..

domain generalization, out-of-distribution classification, deep learning

††copyright: acmcopyright††journalyear: 2024††doi:
XXXXXXX.XXXXXXX††conference: 13th Hellenic Conference on Artificial
Intelligence (SETN 2024); September 11-13, 2024; Piraeus, Greece††price:
15.00††isbn: 978-1-4503-XXXX-X/18/06

##  1\. Introduction

The past few years have been marked by an explosion in the use of Artificial
Intelligence systems. Spanning from industry (Verdouw et al., 2021), medicine
(McKinney et al., 2020), academia (Bengio et al., 2013) and even general
public use (Ray, 2023), AI systems seem to have established themselves in our
daily lives. However, despite their success, widely used and state-of-the-art
models still fail to exhibit generalizable attributes when evaluated on data
that do not adhere to the i.i.d. assumption. During their training, prominent
neural network architectures, such as deep convolutional neural networks
(CNNs), often learn to infer based on spurious correlations present in the
data (e.g. backgrounds, features attributed to the image style, etc.) and not
truly class-representative properties (Recht et al., 2019; Ballas and Diou,
2024b). Domain Generalization (Zhou et al., 2023) attempts to provide insight
into the above issues, by building models which are trained on multiple source
data domains but are able to generalize to previously unseen data (target data
domains).

Figure 1. Illustration of the proposed CycleMix augmentation method. Before
passing through a feature extractor (e.g ResNet-50), the styles of each domain
source domain are mixed together in order to create samples from novel
domains. As the magniutude of each style component is random in each
minibatch, the model is constantly provided with previously unseen data
samples during training.

Our proposed framework.

In our work, we aim to produce a model that maintains its performance on test
image data distributions with different styles than the ones present in the
training distribution. We therefore propose augmenting the styles of a model’s
training images and creating novel style image domains which could push a CNN
to extract meaningful and domain-invariant representations. Our initial
findings are validated on PACS (Li et al., 2017), a widely-used DG benchmark,
which contains images from 4 different style domains.

To this end, we:

  * •

Train domain translational Generative Adversarial Networks (GANs) for
capturing the style attributes of each source domain,

  * •

Randomly mix the styles present in the source domains and produce images from
novel style domains and

  * •

Validate our method on a widely-used publicly available DG dataset.

In the sections to come, we briefly: introduce the DG problem setup along with
all relevant notations, reference the most important works in DG, present the
experimental setup and results and finally, conclude our paper.

###  1.1. Domain Generalization

Let $\mathcal{D}:=\\{\mathcal{D}_{i}\\}_{i=1}^{S}$ a set of $S$ Source
training domains over an input space $\mathcal{X}$. We then observe $n_{i}$
training data points from domain $\mathcal{D}_{i}$, consisting of an input t
$\mathbf{x}^{(i)}_{j}$ and label $y^{(i)}_{j}$, i.e.
$(\mathbf{x}^{(i)}_{j},y^{(i)}_{j})\sim\mathcal{D}_{i}$. Similarly, let
$T:=\\{T_{i}\\}_{i=1}^{T}$ be a set of $T$ unknown Target domains, while we
assume that there exists a single global labeling function $h(x)$ that maps
input observations to their labels, $y_{j}^{(i)}=h(\mathbf{x}_{j}^{(i)})$, for
all domains $i$ and samples $j$. The aim of Domain Generalization (DG) is to
produce a model with parameters $\theta\in\Theta$ which generalizes to both
source domains $\mathcal{D}$ and unseen target domains $\mathcal{T}$.

##  2\. Related Work

Domain Generalization has emerged as one of the most difficult problems in ML
today, finding applications in multiple, varying fields (Zhou et al., 2023).
DG methods can be broadly categorized in the following groups (Wang et al.,
2022):

  * •

Data manipulation: applied algorithms focus on producing generalizable models
by increasing the diversity of existing training data (Zhou et al., 2021).

  * •

Representation learning: given a predictive function $h$, which can be
decomposed as $h=f\circ g$ into a representation learning function $g$ and
$f$, methods in this group focus on improving the feature extraction
capabilities of $g$. The most common approach is regularizing the loss
function (Arjovsky et al., 2020) or manipulating existing neural network
architectures (Ballas and Diou, 2024a, 2023).

  * •

Learning strategy: the problem of DG has also been studied under numerous
machine learning paradigms, such as self-supervised learning, meta-learning,
gradient operations, ensemble-learning, etc (Kim et al., 2021).

Our proposed method falls under the first category of data manipulation
methods. The implementation of data augmentation methods with regards to model
generalizabilty has been certainly explored in the past. Most notably the
authors of (Xu et al., 2021) explore the benefits of applying random
convolutions on training images and using them as new data points during model
training. Several works have also employed adaptive instance normalization
(AdaIN) (Huang and Belongie, 2017) for transferring styles between data
samples. For example, SagNets (Nam et al., 2021) aim to make predictions on
the content of an image and disregard features attributed to image-style by
training style-biased networks, while (Lee et al., 2022) trains a model to
learn robust colorization techniques for improved model robustness. In another
interesting work, the authors of (Zhou et al., 2021) propose MixStyle, an
algorithm that mixes the styles of training instances in each mini-batch to
increase the sample diversity of source domains. Another method that produces
surprisingly good results is Mixup (Xu et al., 2020), which improves model
robustness by training the network on convex combinations of pairs of examples
and their labels. Further common augmentation methods or regularization
strategies include CutMix (Yun et al., 2019) and Cutout (May, 2020), where
patches of images are either cut and pasted among training samples or dropped
entirely.

In our work, we propose capturing the style attributes of each domain by
employing translational Generative Adversarial Networks and not relying on the
features present in each separate sample. By mixing the style-attributes of
each domain we are able to create completely novel samples in each mini-batch
and improve the robustness of a vanilla feature extractor.

##  3\. Methodology

In this section we provide a brief overview of the CycleGAN algorithm which
was utilized in our research for translating images between domains. Once
defined, we present the proposed methodology of CycleMix for synthesizing
images with novel styles .

###  3.1. Cycle-Consistent Adversarial Networks

CycleGANs (Zhu et al., 2017) were initially proposed for learning
translational image mappings between two domains $X$ and $Y$, in the absence
of paired examples. Formally, let $D_{1}$ and $D_{2}$ be source domains for
which we aim to learn a mapping $G:D_{1}\rightarrow D_{2}$, such that the
distribution of images drawn from $G(D_{1})$ are identical with the
distribution of images from $D_{2}$. The novelty of CycleGAN’s lies in the
addition of an inverse mapping $F:D_{1}\rightarrow D_{2}$ and Cycle
Consistency Loss, to the already established Adversarial Loss of GAN’s. The
addition of the Cycle Consistency loss enforces the model to reconstruct a
translated image into its original domain.

Figure 2. Indicative examples of the trained CycleGAN translations between
domains in the PACS dataset. Despite the relatively small size of the samples,
the CycleGAN models are able to capture the style attributes in each domain.

CycleGan

###  3.2. CycleMix

In our work, we leverage CycleGANs to learn style-mappings between source
domains, in the context of Domain Generalization. Our intuition is that by
randomly mixing the styles present in source domains and thus synthesizing
novel samples, a feature extractor can derive robust representations and learn
the features which remain invariant across styles. Specifically, given $S$
source domains we train $S(S-1)/2$ CycleGANs for learning all the possible
domain mappings (a single CycleGAN also contains the trained model yielding
the inverse mapping between the two domains). An indicative example of
translated images between domains is presented in Fig. 2. Having captured the
styles of each source domain with the above mappings, we use them as
augmentation functions on training data. Specifically, given an image
$\mathbf{x}^{(i)}$ drawn from a source domain $D_{i}$, we translate the image
to the remaining source domains and then randomly blend them into a single
final image. The entire augmentation operation is as follows:

(1)
$\mathbf{x}^{(i)^{\prime}}=\mathbf{x}^{(i)}+\sum_{\begin{subarray}{c}j=1\\\
j\neq i\end{subarray}}^{S}a_{j}\cdot G_{ij}(\mathbf{x}^{(i)})$

where $G_{ij}$ the GAN for translating images from domain $i$ to domain $j$
and $a_{j}$ a random parameter corresponding to the magnitude of style mixing
for each source domain222The sum of all parameters $a_{j}$ add up to 1 and are
randomly sampled in each minibatch iteration.. After this mixing operation,
each image is normalized (i.e., the same preprocessing is applied as for the
vanilla model input).

In our experiments we only randomly augment half of the images present in a
mini-batch to preserve the information provided by the initial source domains.
An example of style mixed images, along with an illustration of our proposed
method, is provided in Fig. 1. After augmentation, we pass the mixed images
through a feature extractor and train a classifier to attain their final
labels.

##  4\. Experiments

###  4.1. Experimental Setup

To validate our approach, we use the publicly available and widely-adopted
PACS(Li et al., 2017) dataset. PACS contains images from 4 separate style
domains. As its name suggests, samples can either originate from the Photo,
Art Painting, Cartoon or Sketch domain and can be one of 7 classes. As in
standard DG experimental setups, we follow the leave-one-domain-out cross-
validation protocol (Li et al., 2017; Gulrajani and Lopez-Paz, 2021), meaning
that in each training iteration we train a model on 3 source domains and
evaluate on a single target domain. For our experiments, we use the DomainBed
(Gulrajani and Lopez-Paz, 2021) codebase and train a ResNet-50 feature
extractor on a single NVIDIA A100 GPU card.

Table 1. Top-1% accuracy results on the PACS (left) and VLCS (right) datasets. The columns denote the target domains. The top results are highlighted in bold while the second best are underlined. Method | Art | Cartoon | Photo | Sketch | Avg
---|---|---|---|---|---
ERM | 85.4 | 75.4 | 95.9 | 77.1 | 84.8
CUTOUT | 81.8 | 81.8 | 95.8 | 78.1 | 84.2
CUTMIX | 79.9 | 75.9 | 96.6 | 76.0 | 82.1
SagNet | 84.5 | 79.5 | 95.7 | 78.1 | 84,4
MIXUP | 86.5 | 76.6 | 97.7 | 76.5 | 84.3
CycleMix | 87.7 | 82.0 | 96.6 | 79.9 | 86.6

###  4.2. Results

To evaluate the effectiveness of CycleMIX as an augmentation approach, we rank
its improvement against established techniques such as CUTOUT, CUTMIX and
MIXUP. We also use SagNets as a baseline to demonstrate the efficacy of our
method, along with a vanilla ResNet-50333Standard image augmentations are used
during data loading for each method, such as random resized crop, random
horizontal flip, color jitter and normalization.. The results of our
experiments are presented in Table 1.

It is apparent that the proposed method has a clear advantage over the
baselines, as it surpasses the second best performing model by an average of
around 2%. With the exception of Photo, CycleMix yields the best results in
every other target domain.

##  5\. Conclusion

In this work we propose CycleMix, a method aimed to alleviate the problems
poised by style biased predictions in the DG setting. We argue that mixing the
different styles present in a convolutional neural network’s training data, a
model can be pushed to focus on the invariant features and extract robust
representations. The above claim is supported by experimental results on PACS,
a dataset containing images from 4 distinct style distributions, where our
method surpasses previously proposed algorithms. However, one of the key
limitations is that an increase in the number of source domains corresponds to
an increased of trained CycleGANs, which can prove computationally infeasible.
In future work, we intend to train StarGAN models which were proposed for
multi-domain image-to-image translation and explore our methodology on
additional datasets.

###### Acknowledgements.

The work leading to these results has been funded by the European Union under
Grant Agreement No. 101057821, project RELEVIUM. Views and opinions expressed
are however those of the authors and do not necessarily reflect those of the
European Union or the granting authority (HaDEA). Neither the European Union
nor the granting authority can be held responsible for them.

## References

  * (1)
  * Arjovsky et al. (2020) Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. 2020.  Invariant Risk Minimization.  _arXiv:1907.02893_ (March 2020).
  * Ballas and Diou (2023) Aristotelis Ballas and Christos Diou. 2023.  CNNs with Multi-Level Attention for Domain Generalization. In _Proceedings of the 2023 ACM International Conference on Multimedia Retrieval_ (Thessaloniki, Greece) _(ICMR ’23)_. Association for Computing Machinery, New York, NY, USA, 592–596.
  * Ballas and Diou (2024a) Aristotelis Ballas and Christos Diou. 2024a.  Multi-Scale and Multi-Layer Contrastive Learning for Domain Generalization.  _IEEE Transactions on Artificial Intelligence_ (2024), 1–14.
  * Ballas and Diou (2024b) Aristotelis Ballas and Christos Diou. 2024b.  Towards Domain Generalization for ECG and EEG Classification: Algorithms and Benchmarks.  _IEEE Transactions on Emerging Topics in Computational Intelligence_ 8, 1 (2024), 44–54.
  * Bengio et al. (2013) Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2013\.  Representation Learning: A Review and New Perspectives.  _IEEE Trans. Pattern Anal. Mach. Intell._ 35, 8 (aug 2013), 1798–1828.
  * Gulrajani and Lopez-Paz (2021) Ishaan Gulrajani and David Lopez-Paz. 2021.  In Search of Lost Domain Generalization. In _9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021_. OpenReview.net.
  * Huang and Belongie (2017) Xun Huang and Serge Belongie. 2017.  Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization. In _2017 IEEE International Conference on Computer Vision (ICCV)_. IEEE, Nashville, TN, USA, 1510–1519.
  * Kim et al. (2021) Daehee Kim, Youngjun Yoo, Seunghyun Park, Jinkyu Kim, and Jaekoo Lee. 2021.  SelfReg: Self-supervised Contrastive Regularization for Domain Generalization. In _2021 IEEE/CVF International Conference on Computer Vision (ICCV)_. IEEE, Montreal, QC, Canada, 9599–9608.
  * Lee et al. (2022) Hyejin Lee, Daehee Kim, Daeun Lee, Jinkyu Kim, and Jaekoo Lee. 2022.  Bridging the Domain Gap Towards Generalization in Automatic Colorization. In _Computer Vision – ECCV 2022_ , Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner (Eds.). Springer Nature Switzerland, Cham, 527–543.
  * Li et al. (2017) D. Li, Y. Yang, Y. Song, and T. M. Hospedales. 2017\.  Deeper, Broader and Artier Domain Generalization. In _2017 IEEE International Conference on Computer Vision (ICCV)_. IEEE Computer Society, Los Alamitos, CA, USA, 5543–5551.
  * May (2020) Philip May. 2020\.  Improved Image Augmentation for Convolutional Neural Networks by Copyout and CopyPairing.  https://openreview.net/forum?id=rkxWpCNKvS
  * McKinney et al. (2020) Scott Mayer McKinney, Marcin Sieniek, Varun Godbole, Jonathan Godwin, Natasha Antropova, Hutan Ashrafian, Trevor Back, Mary Chesus, Greg S Corrado, Ara Darzi, et al. 2020\.  International evaluation of an AI system for breast cancer screening.  _Nature_ 577, 7788 (2020), 89–94.
  * Nam et al. (2021) H. Nam, H. Lee, J. Park, W. Yoon, and D. Yoo. 2021.  Reducing Domain Gap by Reducing Style Bias. In _2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_. IEEE Computer Society, Los Alamitos, CA, USA, 8686–8695.
  * Ray (2023) Partha Pratim Ray. 2023\.  ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope.  _Internet of Things and Cyber-Physical Systems_ 3 (2023), 121–154.
  * Recht et al. (2019) Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. 2019.  Do imagenet classifiers generalize to imagenet?. In _International conference on machine learning_. PMLR, Curran Associates Inc., Red Hook, NY, USA, 5389–5400.
  * Verdouw et al. (2021) Cor Verdouw, Bedir Tekinerdogan, Adrie Beulens, and Sjaak Wolfert. 2021.  Digital twins in smart farming.  _Agricultural Systems_ 189 (2021), 103046.
  * Wang et al. (2022) Jindong Wang, Cuiling Lan, Chang Liu, Yidong Ouyang, Tao Qin, Wang Lu, Yiqiang Chen, Wenjun Zeng, and S Yu Philip. 2022.  Generalizing to unseen domains: A survey on domain generalization.  _IEEE transactions on knowledge and data engineering_ 35, 8 (2022), 8052–8072.
  * Xu et al. (2020) Minghao Xu, Jian Zhang, Bingbing Ni, Teng Li, Chengjie Wang, Qi Tian, and Wenjun Zhang. 2020.  Adversarial Domain Adaptation with Domain Mixup. In _The Thirty-Fourth AAAI Conference on Artificial Intelligence_. AAAI Press, Washington, DC, USA, 6502–6509.
  * Xu et al. (2021) Zhenlin Xu, Deyi Liu, Junlin Yang, Colin Raffel, and Marc Niethammer. 2021.  Robust and Generalizable Visual Representation Learning via Random Convolutions. In _9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021_. OpenReview.net.
  * Yun et al. (2019) S. Yun, D. Han, S. Chun, S. Oh, Y. Yoo, and J. Choe. 2019.  CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features. In _2019 IEEE/CVF International Conference on Computer Vision (ICCV)_. IEEE Computer Society, Los Alamitos, CA, USA, 6022–6031.
  * Zhou et al. (2023) Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, and Chen Change Loy. 2023.  Domain Generalization: A Survey.  _IEEE Trans. Pattern Anal. Mach. Intell._ 45, 4 (apr 2023), 4396–4415.
  * Zhou et al. (2021) Kaiyang Zhou, Yongxin Yang, Yu Qiao, and Tao Xiang. 2021\.  Domain Generalization with MixStyle. In _9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021_. OpenReview.net.
  * Zhu et al. (2017) Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. 2017.  Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks. In _2017 IEEE International Conference on Computer Vision (ICCV)_. IEEE, Nashville, TN, USA, 2242–2251.


# Non-uniform array design for robust LoS MIMO via convex optimization

Michail Palaiologos$\ast$†, Mario H. Castãneda García$\ast$, Anastasios
Kakkavas$\ddagger$, Richard A. Stirling-Gallacher$\ast$
and Giuseppe Caire† †Communications and Information Theory Group, Technical
University of Berlin, 10587 Berlin, Germany
Emails: {michail.palaiologos, mario.castaneda<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>$\ddagger$ School of Computation,
Information and Technology, Technical University of Munich, 80333 Munich,
Germany
*Munich Research Center, Huawei Technologies Düsseldorf GmbH, 80992 Munich, Germany

###### Abstract

The array design problem of multiple-input multiple-output (MIMO) systems in a
line-of-sight (LoS) transmit environment is examined. As uniform array
configurations at the transmitter (Tx) and receiver (Rx) are optimal at
specific transmit distances only, they lead to reduced spectral efficiency
over a range of transmit distances. To that end, the joint design of non-
uniform Tx and Rx arrays towards maximizing the minimum capacity of a LoS MIMO
system across a range of transmit distances is investigated in this paper. By
introducing convex relaxation, the joint Tx and Rx array design is cast as a
convex optimization problem, which is solved in a iterative manner. In
addition, we also implement a local search to obtain a refined solution that
achieves an improved performance. It is shown that the non-uniform
configurations designed with our proposed approach outperform uniform and non-
uniform array designs of the literature in terms of capacity and/or
complexity.

###### Index Terms:

LoS MIMO, convex optimization, array design

##  I Introduction

A shift towards millimeter wave and sub-THz frequency bands is one of the main
enablers for satisfying the higher data rates required in future wireless
communication systems [1]. However, as at higher frequencies the channel may
be dominated by the line-of-sight (LoS) path, the deployment of multiple
antennas can result in rank deficient MIMO channels, thus eluding an increase
of the spatial degrees of freedom [2].

Still, it has been demonstrated that, if the Tx and Rx arrays are properly
designed, spatial multiplexing gain can be extracted, even if the channel is
dominated by the LoS path. Specifically, optimum antenna placement has been
proposed for uniform linear arrays (ULAs) [3], uniform planar arrays (UPAs)
[4] and uniform circular arrays (UCAs) [5]. A major shortcoming of prior
uniform designs is that the proposed arrays are optimized for a specific
transmit distance (distance between the Tx and Rx array). As such a design
does not consider the performance at other transmit distances, substantial
capacity fluctuations and, thus, decreased performance occurs when operating
over a wide range of distances.

To circumvent this, non-uniform array designs have been proposed [6, 7, 8, 9,
10, 11]. By assuming fixed arrays consisting of non-uniformly spaced antennas
at the Tx and/or Rx, reduced capacity fluctuations and, hence, improved
performance can be achieved over a range of transmit distances. However, due
to the complexity of the LoS MIMO array design problem, which is highly non-
convex, most prior works rely on brute force exhaustive search (ES) [6, 7],
stochastic optimization techniques, such as genetic algorithms (GAs) [8, 9],
and on numerical results. Specifically, by applying ES and assuming identical
antenna locations at the Tx and Rx, non-uniform linear arrays (NULAs) are
derived for $4\times 4$ systems to maximize the range of transmit distances
over which capacity remains above a threshold [6] and the minimum capacity
over a given range of transmit distances [7]. In [8] a non-uniform planar
array is obtained via a GA by maximizing the mean minus the standard deviation
of the capacity over a range of distances. In [9], a NULA for a LoS massive
MIMO system is obtained through a GA for maximizing the user sum rate.

In [10] the channel matrix eigenvalues are expressed as a function of the Tx
and Rx antenna locations. In this way, optimum Tx and Rx NULA configurations
are obtained, so that a minimum number of spatial streams is supported. In
[11], a multi-user LoS massive MIMO scenario is considered, where the antenna
locations of the Tx NULA are obtained by Chebyshev polynomials. It is shown
that the average condition number is greatly improved compared to a system
with ULAs. [10] and [11] are tailored to the design of NULAs and cannot be
applied in the design of arrays of arbitrary geometries. In all works, either
identical Tx and Rx non-uniform configurations are considered for single user
systems [6, 7, 8, 10] or a Tx NULA with single antenna Rx users is assumed [9,
11].

In this paper, the joint design of non-uniform Tx and Rx arrays of arbitrary
geometries for LoS MIMO systems is investigated towards maximizing the minimum
capacity over a range of transmit distances, which also results in reduction
of the capacity fluctuations. To formulate this as a closed-form optimization
problem, we consider the capacities at a (finely quantized) set of transmit
distances over the range. Then, by assuming a set of candidate antenna
locations (depending on the assumed array geometry) for the Tx and Rx array,
we express the joint array design problem as a multicriterion integer
optimization problem (MIOP), which is NP-hard [12]. We demonstrate that, by
relaxing the integer constraints, the joint optimization of the Tx and Rx
arrays can be solved via convex optimization in a iterative manner [13]. A
local search method based on randomized rounding (RR) [14] is also
incorporated to improve the results. It is shown that the non-uniform
configurations designed with our proposed approach outperform uniform as well
as non-uniform array designs of the literature. To the best of the authors’
knowledge, this is the first time that the LoS MIMO array design for
optimizing the capacity is cast as a convex optimization problem.

The remainder of the paper is organized as follows. Section II introduces the
system model. The joint Tx-Rx array design problem is introduced as a MIOP in
Section III and transformed into a convex optimization problem. Simulation
results are presented in Section IV, while Section V concludes the paper. Bold
lower case and upper case letters represent vectors and matrices,
respectively. $(\cdot)^{\textit{T}}$ and $(\cdot)^{\textit{H}}$ denote the
transpose and conjugate transpose of a vector or matrix.

##  II System Model

Consider a LoS MIMO system. The Tx and Rx arrays consist of $N$ and $M$
antennas, respectively and can admit arbitrary geometries and be arbitrarily
oriented in space. The antenna locations of the Tx array are given by matrix
$\textbf{T}\in\mathbb{R}^{3\times N}$, where its $n-$th column is denoted as
$\textbf{t}_{n}$ and represents the (x, y, z) Cartesian coordinates of the
$n-$th Tx antenna on the 3D geometrical coordinate system, for
$n\in\\{1,...,N\\}$. The aperture of the array is given by the largest
euclidean distance between any two elements of the array [4] and is given by

$L_{\text{t}}=\underset{\begin{subarray}{c}i,j\end{subarray}}{\text{max}}\,\|\textbf{t}_{i}-\textbf{t}_{j}\|_{2},\,\forall\;i,j\in\\{1,\dots,N\\},\,i\neq
j.$ (1)

The antenna locations of the Rx array are given by
$\textbf{R}\in\mathbb{R}^{3\times M}$, where the $m-$th column of R is denoted
as $\textbf{r}_{m}$ and represents the (x, y, z) Cartesian coordinates of the
$m-$th Rx antenna on the 3D geometrical coordinate system, for
$m\in\\{1,...,M\\}$. The aperture of the Rx array $L_{\text{r}}$ is defined
similarly to $L_{\text{t}}$. To avoid strong mutual coupling effects between
the antennas, a minimum antenna spacing of $\lambda/2$ is considered at both
arrays [11], where $\lambda$ is the wavelength corresponding to carrier
frequency $f_{\text{c}}$. Thus, we assume that

$\displaystyle\underset{\begin{subarray}{c}i,j\\\
\end{subarray}}{\text{min}}\,\|\textbf{t}_{i}-\textbf{t}_{j}\|_{2}=d_{\text{t}}$
$\displaystyle\geq\lambda/2,\,\forall\;i,j\in\\{1,\dots,N\\},\,i\neq j,$ (2a)
$\displaystyle\underset{\begin{subarray}{c}i,j\end{subarray}}{\text{min}}\,\|\textbf{r}_{i}-\textbf{r}_{j}\|_{2}=d_{\text{r}}$
$\displaystyle\geq\lambda/2,\,\forall\;i,j\in\\{1,\dots,M\\},\,i\neq j,$ (2b)

where $d_{\text{t}}$ and $d_{\text{r}}$ denote the minimum antenna spacing at
the Tx and Rx array, respectively.

A LoS MIMO system is characterized by the expression

$\textbf{y}=\textbf{H x}+\textbf{n},$ (3)

which associates the received signal $\textbf{y}\in\mathbb{C}^{M}$ with the
transmitted signal $\textbf{x}\in\mathbb{C}^{N}$, the channel matrix
$\textbf{H}\in\mathbb{C}^{M\times N}$ and the complex additive white Gaussian
noise of zero mean and unit variance, given by $\textbf{n}\in\mathbb{C}^{M}$.
The power of x is constrained to not be larger than $P_{\text{Tx}}$, i.e.,
$\|\textbf{x}\|_{2}^{2}\leq P_{\text{Tx}}$. The $(m,n)$-th entry of the LoS
MIMO channel matrix is given as [4, 10]

$\textbf{H}_{m,n}=e^{-j\frac{2\pi}{\lambda}d_{m,n}},$ (4)

where $d_{m,n}=\|\textbf{r}_{m}-\textbf{t}_{n}\|_{2}$ is the euclidean
distance between the $m$-th Rx and $n$-th Tx antenna. In (4) we assume that
the transmit distance is much larger than the arrays’ apertures and that
perfect power control is applied to compensate for the path loss, so that we
can focus on the phase shifts of the channel matrix entries which are the
determining factors of the spatial multiplexing capabilities of a MIMO system
[6, 8].

By assuming no Tx channel state information, uniform power allocation across
the Tx antennas is applied, hence, the capacity of the LoS MIMO system reads
as [3]

$\textit{C}=\text{log}_{2}\left(\det\left(\textbf{I}_{M}+\frac{\rho}{N}\textbf{H}\textbf{H}^{\textit{H}}\right)\right),$
(5)

where $\rho=\frac{P_{\text{Tx}}}{\sigma_{n}^{2}}$, with $\sigma_{n}^{2}$ being
the noise variance at each receiving antenna and $\textbf{I}_{M}$ the $M\times
M$ identity matrix. The maximum value of (5) is achieved when all eigenvalues
of $\textbf{H}\textbf{H}^{\textit{H}}$ are equal, so that the maximum capacity
is given as

$C_{\text{max}}=M\,\text{log}_{2}(1+\rho).$ (6)

##  III Array Design

The entries of the channel matrix (4) depend only on the respective distance
$d_{m,n}$ between the $n$-th Tx and $m$-th Rx antenna, with $n=1,\dots,N$ and
$m=1,\dots,M$. Accordingly, these values depend on the transmit distance and
on the locations of the Tx and Rx antennas, i.e., on matrices T and R. Thus,
the antennas’ locations at the Tx and Rx array can be optimized in order to
improve the capacity [3, 4, 5, 6, 7, 9, 8, 10, 11].

###  III-A Review of Uniform Array Design

For a fixed transmit distance $D^{\ast}$, optimum uniform array configurations
have been derived for LoS MIMO, so that maximum capacity is extracted at
$D^{\ast}$. When ULAs are considered at the Tx and Rx, with an antenna spacing
given by $d_{\text{t}}$ and $d_{\text{r}}$, it has been shown that maximum
capacity is extracted at $D^{\ast}$ if the following expression is satisfied
[3]

$d_{\text{t}}d_{\text{r}}=\frac{\lambda D^{\ast}}{\max(N,M)}.$ (7)

Similar expressions or optimization methods have been derived for UPAs [4] and
UCAs [5]. Still, these results do not provide any insight about the capacity
performance at other transmit distances. In fact, LoS MIMO systems suffer from
substantial capacity fluctuations when operating at varying transmit distances
when uniform arrays are employed [6, 7].

To showcase this limitation, the capacity performance of a LoS MIMO system
over a range of distances is illustrated in Fig. 1, where ULAs, UPAs and UCAs
are assumed at the Tx and Rx. In all cases, $N=M=9$. The ULAs, UPAs and UCAs
were designed based on the results of [3], [4] and [5], respectively, so that
maximum capacity is extracted at the distance of $D^{\ast}=90$ m. The carrier
frequency is equal to $f_{\text{c}}=62$ GHz. The Tx and Rx arrays have the
same aperture size in each respective case and are equal to $1.76$ m, $1.08$ m
and $3.12$ m for the ULA, UPA and UCA case, respectively. The capacity is
evaluated over the range of $[10,100]$ m.

ULAs, $\mu=52.7$, $\sigma=7.6$, $\text{min}=18.4$
---
UPAs, $\mu=49.4$, $\sigma=12.5$, $\text{min}=9.8$
---
UCAs, $\mu=53.7$, $\sigma=4.7$, $\text{min}=36.9$
---

Capacity (bpcu)
---

Transmit distance
---

Figure 1: Capacity vs transmit distance for $9\times 9$ uniform array
topologies.

To facilitate the evaluation and comparison of the capacity fluctuations, the
mean $\mu$, standard deviation (std) $\sigma$ and minimum (min) of the
capacity over the range of distances is depicted in the figure’s legend, which
are measured in bits-per-channel-use (bpcu). Evidently, all three LoS MIMO
systems suffer from large capacity fluctuations over the given range. Although
$N=M=9$ is assumed here, this issue is also observed when different number of
antennas is employed.

###  III-B Proposed Non-Uniform Array Design

In order to improve the performance of LoS MIMO systems operating over a range
of transmit distances, non-uniform design of the Tx and Rx arrays have been
considered [6, 7, 8, 9, 10, 11]. In this paper, we focus on the joint design
of non-uniform Tx and Rx array configurations of arbitrary geometries for LoS
MIMO systems. To the best of the authors’ knowledge, [10] is the only work
that considered joint Tx and Rx non-uniform array design, albeit, only NULAs
were considered.

First, the objective function, that is to be maximized, should be the capacity
over a range of transmit distances d. To express this optimization problem in
a closed form, we introduce a quantization of the range. In particular, we
consider the range, not as a continuum, but as a discrete set of transmit
distances, that is, we suggest that d is represented as a $Q\times 1$ vector
$\textbf{d}=[D_{1},\dots,D_{q},\dots,D_{Q}]^{\textit{T}}$, with $D_{1}$ and
$D_{Q}$ being the minimum and maximum distances of the range, and $Q$ the
number of considered distances. Note that optimizing the quantization step
(i.e., $Q$) is out of the scope of this paper, however, it has been shown, by
means of simulations, that the capacity of a LoS MIMO system is robust to
small changes in the transmit distance [4, 15]. So, we would expect that a
sufficiently small value of $Q$ would be adequate for a proper evaluation of
the capacity over the range of distances.

So, the capacity at transmit distance $D_{q}$ can be written as

$\textit{C}_{q}(\textbf{T},\textbf{R})=\text{log}_{2}\left(\det\left(\textbf{I}_{M}+\frac{\rho}{N}\textbf{H}^{q}\textbf{H}^{q^{\textit{H}}}\right)\right),$
(8)

where $\textbf{H}^{q}\in\mathbb{C}^{M\times N}$ is the channel matrix
realization at the transmit distance $D_{q}$. The notation in (8) highlights
the dependence of the capacity on the array configurations, which originates
from the dependence of $\textbf{H}^{q}$ on T and R (see (4)).

To simplify the array design problem, we focus on optimally selecting antenna
locations, for a given Tx and Rx array geometry, from a uniform grid of
candidate antenna locations, so that the array design problem becomes an
antenna location selection problem. So, consider that the $N$ Tx and $M$ Rx
antennas can be placed only on $N_{\text{f}}$ and $M_{\text{f}}$ possible
locations, respectively, which are uniformly spaced across the arrays’
physical areas, with $N_{\text{f}}\geq N$ and $M_{\text{f}}\geq M$. We
highlight that our approach can support any array geometry type. Let the
coordinates of the $N_{\text{f}}$ candidate locations of the Tx antennas be
given by the columns of matrix $\textbf{T}_{\text{f}}\in\mathbb{R}^{3\times
N_{\text{f}}}$, which obey

$\underset{\begin{subarray}{c}i,j\\\
\end{subarray}}{\text{max}}\,\|\textbf{t}_{i}-\textbf{t}_{j}\|\leq
L_{\text{t}},\,\forall\;i,j\in\\{1,\dots,N_{\text{f}}\\},\,i\neq j$ (9)

as well as (2a). Similarly, the coordinates of the $M_{\text{f}}$ candidate
locations of the Rx antennas are given by the columns of matrix
$\textbf{R}_{\text{f}}\in\mathbb{R}^{3\times M_{\text{f}}}$, which must
satisfy a constraint similar to (9) as well as (2b). Note that the grid of
antenna locations can be non-uniform too, as long as (2a), (2b) and (9) hold.

Let $\textbf{H}_{\text{f}}^{q}\in\mathbb{C}^{M_{\text{f}}\times N_{\text{f}}}$
be the channel matrix at transmit distance $D_{q}$ between the Tx and Rx
arrays, assuming that antennas are placed on all candidate locations. The
entries of $\textbf{H}_{\text{f}}^{q}$ are defined as in (4). Given that each
row of the MIMO channel matrix corresponds to a specific Rx antenna and that
each column corresponds to a specific Tx antenna, performing antenna
(location) selection on $\textbf{R}_{\text{f}}$ is equivalent to performing
row selection on $\textbf{H}_{\text{f}}^{q}$, while performing antenna
(location) selection on $\textbf{T}_{\text{f}}$ is equivalent to performing
column selection on $\textbf{H}_{\text{f}}^{q}$.

As our goal is to select $N$ out of $N_{\text{f}}$ Tx and $M$ out of
$M_{\text{f}}$ Rx antenna locations, if the coordinates of the Tx and Rx
selected antennas are given by $\textbf{T}\in\mathbb{R}^{3\times N}$ and
$\textbf{R}\in\mathbb{R}^{3\times M}$, respectively, then by introducing Tx
and Rx binary selection matrices as
$\textbf{F}^{\text{t}}\in\mathbb{B}^{N_{\text{f}}\times N}$ and
$\textbf{F}^{\text{r}}\in\mathbb{B}^{M_{\text{f}}\times M}$, we have that
$\textbf{T}=\textbf{T}_{\text{f}}\textbf{F}^{\text{t}}$ and
$\textbf{R}=\textbf{R}_{\text{f}}\textbf{F}^{\text{r}}$. Matrices
$\textbf{F}^{\text{t}}$ and $\textbf{F}^{\text{r}}$ have a single one in each
column and at most a single one in each row [2]. To that end, the channel
matrix between the Tx and Rx arrays, whose antenna locations have been
optimally selected, is given as
$\textbf{H}^{q}=\textbf{F}^{\text{r}^{\textit{H}}}\textbf{H}^{q}_{\text{f}}\textbf{F}^{\text{t}}$.
So, (8) can be now written as a function of $\textbf{F}^{\text{t}}$ and
$\textbf{F}^{\text{r}}$ as

$\displaystyle\textit{C}_{q}(\textbf{F}^{\text{t}},\textbf{F}^{\text{r}})$
$\displaystyle=\text{log}_{2}\left(\det\left(\textbf{I}_{M}+\frac{\rho}{N_{\text{f}}}(\textbf{F}^{\text{r}^{\textit{T}}}\textbf{H}^{q}_{\text{f}}\textbf{F}^{\text{t}})(\textbf{F}^{\text{r}^{\textit{T}}}\textbf{H}^{q}_{\text{f}}\textbf{F}^{\text{t}})^{\textit{H}}\right)\right)$
$\displaystyle\Rightarrow\textit{C}_{q}(\bm{\Delta}^{\text{t}},\bm{\Delta}^{\text{r}})$
$\displaystyle=\text{log}_{2}\left(\det\left(\textbf{I}_{M}+\frac{\rho}{N_{\text{f}}}\textbf{H}_{\text{f}}^{q}\bm{\Delta}^{\text{t}}\textbf{H}_{\text{f}}^{q^{\textit{H}}}\bm{\Delta}^{\text{r}}\right)\right),$
(10)

where
$\bm{\Delta}^{\text{t}}=\textbf{F}^{\text{t}}\textbf{F}^{\text{t}^{\textit{T}}}$
and
$\bm{\Delta}^{\text{r}}=\textbf{F}^{\text{r}}\textbf{F}^{\text{r}^{\textit{T}}}$
and we have applied Sylvester’s determinant identity in (III-B) [16].
$\bm{\Delta}^{\text{t}}\in\mathbb{R}^{N_{\text{f}}\times N_{\text{f}}}$ and
$\bm{\Delta}^{\text{r}}\in\mathbb{R}^{M_{\text{f}}\times M_{\text{f}}}$ are
binary diagonal matrices that act as Tx and Rx selection matrices,
respectively.

In particular, $\Delta^{\text{t}}_{n_{\text{f}}}=1$ (for notational brevity,
$\Delta_{i}$ denotes the $i$-th diagonal element of diagonal matrix
$\bm{\Delta}$), where $n_{\text{f}}\in\\{1,\dots,N_{\text{f}}\\}$ indicates
possible antenna locations of the Tx array, signifies that the
$n_{\text{f}}$-th candidate antenna location is selected at the Tx array,
while $\Delta^{\text{t}}_{n_{\text{f}}}=0$ indicates otherwise.
$\Delta^{\text{r}}_{m_{\text{f}}}$ is defined similarly for the Rx array,
where $m_{\text{f}}\in\\{1,\dots,M_{\text{f}}\\}$, with
$\Delta^{\text{r}}_{m_{\text{f}}}=1$ indicating that the $m_{\text{f}}$-th
candidate antenna location is selected at the Rx array. As $N$ out of
$N_{\text{f}}$ Tx antennas and $M$ out of $M_{\text{f}}$ Rx antennas must be
selected, the number of ones in the diagonal of $\bm{\Delta}^{\text{t}}$ and
$\bm{\Delta}^{\text{r}}$ is equal to $N$ and $M$, respectively. So, the joint
Tx and Rx array design problem is formulated as an antenna (location)
selection problem as

$\displaystyle\underset{\bm{\Delta}^{\text{t}},\,\bm{\Delta}^{\text{r}}}{\text{maximize}}\;$
$\displaystyle\
\\{C_{1}(\bm{\Delta}^{\text{t}},\bm{\Delta}^{\text{r}}),\dots,C_{Q}(\bm{\Delta}^{\text{t}},\bm{\Delta}^{\text{r}})\\},$
(11a) s.t.: $\displaystyle\
\;\sum_{n_{\text{f}}=1}^{N_{\text{f}}}\Delta_{n_{\text{f}}}^{\text{t}}=N,$
(11b) $\displaystyle\
\;\sum_{m_{\text{f}}=1}^{M_{\text{f}}}\Delta_{m_{\text{f}}}^{\text{r}}=M,$
(11c) $\displaystyle\
\;\Delta_{n_{\text{f}}}^{\text{t}}\in\\{0,1\\},\,\forall\,n_{\text{f}}\in\\{1,\dots,N_{\text{f}}\\},$
(11d) $\displaystyle\
\;\Delta_{m_{\text{f}}}^{\text{r}}\in\\{0,1\\},\,\forall\,m_{\text{f}}\in\\{1,\dots,M_{\text{f}}\\}.$
(11e)

(11a) - (11e) is a MIOP which is NP-hard [12]. Solving it amounts to
identifying all Pareto optimal points [16]. As each such point would
correspond to a different Tx and Rx non-uniform array configuration, we do not
search for the Pareto front of (11a) - (11e), but focus instead on maximizing
the minimum capacity across the range of transmit distances. In this way, we
can obtain Tx and Rx non-uniform array configurations that reduce the capacity
fluctuations across the range. Thus, we consider the following optimization

$\displaystyle\underset{\bm{\Delta}^{\text{t}},\,\bm{\Delta}^{\text{r}}}{\text{maximize}}\;$
$\displaystyle\
\underset{q=1,\dots,Q}{\text{min}}\\{C_{1}(\bm{\Delta}^{\text{t}},\bm{\Delta}^{\text{r}}),\dots,C_{Q}(\bm{\Delta}^{\text{t}},\bm{\Delta}^{\text{r}})\\},$
(12a) s.t.: $\displaystyle\
\;\sum_{n_{\text{f}}=1}^{N_{\text{f}}}\Delta_{n_{\text{f}}}^{\text{t}}=N,$
(12b) $\displaystyle\
\;\sum_{m_{\text{f}}=1}^{M_{\text{f}}}\Delta_{m_{\text{f}}}^{\text{r}}=M,$
(12c) $\displaystyle\
\;\Delta_{n_{\text{f}}}^{\text{t}}\in\\{0,1\\},\,\forall\,n_{\text{f}}\in\\{1,\dots,N_{\text{f}}\\},$
(12d) $\displaystyle\
\;\Delta_{m_{\text{f}}}^{\text{r}}\in\\{0,1\\},\,\forall\,m_{\text{f}}\in\\{1,\dots,M_{\text{f}}\\}.$
(12e)

An optimum solution of (12a) - (12e) can be found via an exhaustive search
(ES) over all antenna locations and all $Q$ transmit distances. However, as
this entails a complexity of
$\pazocal{O}(\binom{N_{\text{f}}}{N}\binom{M_{\text{f}}}{M}Q)$ operations, it
is only applicable for very small values of $N_{\text{f}},M_{\text{f}},N,M$
and $Q$. In fact, prior works that adopt ES make the restrictive assumptions
that the Tx and Rx arrays have identical configurations and that $N=M\leq 4$
[6, 7]. Instead, we allow the Tx and Rx arrays to have different
configurations and we cast (12a) - (12e) as a convex optimization problem,
which can be efficiently solved.

First, the integer constraints (12d) and (12e) can be relaxed by assuming that
the diagonal entries of $\bm{\Delta}^{\text{t}}$ and $\bm{\Delta}^{\text{r}}$
can take any real value between zero and one [17, 18, 19], so that (12a) -
(12e) is written as

$\displaystyle\underset{\bm{\Delta}^{\text{t}},\,\bm{\Delta}^{\text{r}}}{\text{maximize}}\;$
$\displaystyle\
\underset{q=1,\dots,Q}{\text{min}}\\{C_{1}(\bm{\Delta}^{\text{t}},\bm{\Delta}^{\text{r}}),\dots,C_{Q}(\bm{\Delta}^{\text{t}},\bm{\Delta}^{\text{r}})\\},$
(13a) s.t.: $\displaystyle\
\;\sum_{n_{\text{f}}=1}^{N_{\text{f}}}\Delta_{n_{\text{f}}}^{\text{t}}=N,$
(13b) $\displaystyle\
\;\sum_{m_{\text{f}}=1}^{M_{\text{f}}}\Delta_{m_{\text{f}}}^{\text{r}}=M,$
(13c) $\displaystyle\ \;0\leq\Delta_{n_{\text{f}}}^{\text{t}}\leq
1,\,\forall\,n_{\text{f}}\in\\{1,\dots,N_{\text{f}}\\},$ (13d) $\displaystyle\
\;0\leq\Delta_{m_{\text{f}}}^{\text{r}}\leq
1,\,\forall\,m_{\text{f}}\in\\{1,\dots,M_{\text{f}}\\}.$ (13e)

Secondly, we note that a function $f(\textbf{A})=\log_{2}\det(\textbf{A})$ is
concave in the elements of A if A is positive definite and that concavity is
preserved under affine transformations [17, 16]. Thirdly, we point out that
the pointwise minimum of a set of concave functions is also a concave function
[16]. Given the second point, regarding the $q$-th transmit distance,
$C_{q}(\bm{\Delta}^{\text{t}},\bm{\Delta}^{\text{r}})$ in (III-B) is concave
in the elements of $\bm{\Delta}^{\text{t}}$ or $\bm{\Delta}^{\text{r}}$, but
it is not jointly concave in the elements of both $\bm{\Delta}^{\text{t}}$ and
$\bm{\Delta}^{\text{r}}$, since the product of positive semidefinite matrices
is not jointly concave in the elements of the matrices [16]. However, If (13a)
- (13e) is solved iteratively, i.e., by fixing one of the optimization
variables at each iteration and finding the optimum solution for the other,
convex optimization techniques can be employed [13, 19]. The procedure is
repeated until the objective function does not improve beyond a given
threshold.

Thus, we propose to solve (13a) - (13e) in an iterative fashion, as it is
concave in $\bm{\Delta}^{\text{t}}$ when $\bm{\Delta}^{\text{r}}$ is fixed and
vice versa. At each iteration, the associated problem can be efficiently
solved in polynomial time via interior-point methods. By employing the barrier
method [16], the computational complexity of the problem at each iteration is
dictated by the Cholesky factorization at each Newton step, which is equal to
$\pazocal{O}(N_{\text{f}}^{3})$ and $\pazocal{O}(M_{\text{f}}^{3})$ for the
optimization of $\bm{\Delta}^{\text{t}}$ and $\bm{\Delta}^{\text{r}}$,
respectively. The number of Newton steps is upper bounded by
$\pazocal{O}(\sqrt{N_{\text{f}}})$ and $\pazocal{O}(\sqrt{M_{\text{f}}})$
[16]. If $P$ is the number of iterations that the iterative maximization
algorithm takes to converge, then the complexity of solving (13a) - (13e) is
$\pazocal{O}(PQ(N_{\text{f}}^{3.5}+M_{\text{f}}^{3.5}))$, which is
significantly lower than the ES complexity.

After the iterative algorithm terminates, the optimum
$\bm{\Delta}^{\text{t}^{\ast}}$ and $\bm{\Delta}^{\text{r}^{\ast}}$ are
obtained. However, due to the relaxation, the diagonal elements of these
matrices can be fractional values, so they would not constitute a feasible
solution for the initial problem (12a). To overcome this, we may consider the
indices of the $N$ and $M$ largest diagonal elements of
$\bm{\Delta}^{\text{t}^{\ast}}$ and $\bm{\Delta}^{\text{r}^{\ast}}$,
respectively, as the optimal antenna positions of the Tx and Rx arrays [17].

To refine the initial solution, a more sophisticated selection scheme can be
considered. For instance, randomized rounding (RR) is a well-known
approximation algorithm in combinatorial optimization [14], which is of low
computational complexity and can be incorporated into problems with relaxation
of integer constraints [18, 19]. RR can be applied to refine the solutions
$\bm{\Delta}^{\text{t}^{\ast}}$ and $\bm{\Delta}^{\text{r}^{\ast}}$.
Considering $\bm{\Delta}^{\text{t}^{\ast}}$, the aim is to group its
$N_{\text{f}}$ diagonal elements into two sets: a first set with $N$ elements
whose indices indicate the selected antenna locations of the Tx array, and a
second set containing the remaining $N_{\text{f}}-N$ elements. Initially, the
$N$ element with the largest magnitude in the first set are sorted in a
descending order.

RR allows elements from the first set to be swapped by elements of the second
set, thereby refining the initial $\bm{\Delta}^{\text{t}^{\ast}}$.
Specifically, we go through each element of the first set in an ascending
order (smallest to largest) and replace it with an element of the second set
(going through all elements of this set), if the value of the objective
function can be increased with this swapping. The algorithm terminates when
all possible swaps have been checked, which amounts to a complexity of
$\pazocal{O}(N(N_{\text{f}}-N))$ or when the value of the objective function
is above a predetermined threshold. An identical procedure can be applied for
refining the solution of $\bm{\Delta}^{\text{r}^{\ast}}$ too. By doing this,
we can obtain antenna locations for the Tx and Rx arrays that can achieve a
larger minimum capacity compared to the initial solution
$\bm{\Delta}^{\text{t}^{\ast}}$ and $\bm{\Delta}^{\text{r}^{\ast}}$ of (13a).
As the complexity of this local search is equal to
$\pazocal{O}(N(N_{\text{f}}-N)+M(M_{\text{f}}-M))$, it is negligible compared
to the complexity of solving (13a) - (13e).

##  IV Numerical Results

In this section, the performance of non-uniform array configurations designed
via the proposed convex optimization (CO) approach (with and without RR) is
evaluated over a range of transmit distances. Due to space limitations, we
focus on linear arrays, but point out that our approach is applicable to any
array geometry. We compare the performance of the proposed NULAs against that
of ULAs and of NULA designs from the literature. Without loss of generality,
the Tx and Rx arrays are assumed to be aligned and facing each other. We
assume $f_{\text{c}}$ = 62 GHz, $\rho=20$ dB and that the transmit distance
lies within the range of $[10,100]$ m. We consider a quantization step of 0.5
m for the given range, which translates to a transmit distance vector d
consisting of $Q=181$ elements.

For our comparisons, we assume ULAs at the Tx and Rx of equal number of
antennas which are optimized at $D^{\ast}=92$ m (considering smaller
$D^{\ast}$ led to poorer performance within the given range for the ULAs). For
$M=N=4$, we have $L_{\text{t}}=L_{\text{r}}=1$ m from (7). So, the uniform
antenna spacing at both arrays is equal to
$d_{\text{t}}=d_{\text{r}}=L_{\text{t}}/(N-1)=0.33$ m. For the NULAs design,
we assume the same aperture as the Tx and Rx ULAs and that there are
$N_{\text{f}}=M_{\text{f}}=16$ candidate antenna locations, uniformly spaced
within the assumed apertures, resulting in a spacing of
$d_{\text{t,f}}=d_{\text{r,f}}=L_{\text{t}}/(N_{\text{f}}-1)=0.067$ m.

The NULAs’ performance designed with the iterative CO approach is illustrated
in Fig. 2. The iterations terminate when the increase in the objective
function in (13a) becomes less than a threshold of $0.01$ bpcu between
consecutive iterations, which results in $P=5$ iterations. To highlight the
effect of the local search, results with and without RR are illustrated in
Fig. 2. This figure also includes the performance achieved with ULAs and with
the optimum NULAs obtained via an ES over all
$\binom{N_{\text{f}}}{N}\binom{M_{\text{f}}}{M}$ possible combinations of the
antenna locations. For each configuration, we provide in the legend the mean,
standard deviation and minimum of the capacities achieved within the range of
distances. Our proposed NULA designs, with and without RR, distinctly
outperform the ULAs in terms of minimum capacity over the given range of
transmit distances. We also notice that RR can greatly enhance the
performance, since the CO+RR NULAs lead to a much higher minimum capacity
compared to the CO NULAs. Remarkably, the performance of the NULAs designed
with the CO+RR approach is very similar to the one of the ES-based NULAs.

Capacity (bpcu)
---

Transmit distance $\bm{d}$ (m)
---
CO NULAs, $\mu=24.2,\,\sigma=2.2,\,\text{min}=16$
---
CO+RR NULAs, $\mu=22.5,\,\sigma=1.1,\,\text{min}=21.1$
---
ES NULAs, $\mu=23.5,\,\sigma=0.7,\,\text{min}=21.5$
---
ULAs, $\mu=23.7,\,\sigma=3.4,\,\text{min}=8.8$
---

Figure 2: Capacity versus (vs.) transmit distance of proposed NULAs and ULAs.

The arrays used in Fig. 2 are shown in Fig. 3. Evidently, the Tx and Rx admit
the same NULA geometry when the CO approach without RR is used. Although all
NULAs designed with our proposed methods have the same aperture size as the
ULAs in this particular scenario, in general, this is not a prerequisite for
our approach, since the designed arrays can have, in principle, a smaller
aperture than the initially assumed aperture. This is an important advantage
of our scheme, since, in the literature, a fixed aperture size is usually
assumed, i.e., that the locations of the edge antennas are fixed, and only the
positions of inner elements are optimized [6, 7].

(a)

(b)

(c)

(d)

(e)

(f)

Figure 3: (a) Tx/Rx ULA, (b) Tx/Rx CO NULA, (c) Tx CO+RR NULA, (d) Rx CO+RR
NULA, (e) Tx ES NULA and (f) Rx ES NULA. Crosses and circles indicate
candidate and selected antenna locations, respectively.

Regarding NULA designs of the literature, by considering the ES design in Fig.
2, we provide a comparison of our proposed approaches with an improved version
of the schemes presented in [6, 7], as the ES in [6, 7] assumes the same NULA
configuration at the Tx and Rx. In Fig. 4, we compare the NULAs derived via
the CO+RR approach with the ones in [10] and [11]. Although the proposal in
[11] is associated with a system where only the Tx array (surrounded by single
antenna Rx users) is optimized, in Fig. 4 we assume the optimally derived
array at both the Tx and Rx (this leads to better performance, compared to
assuming a ULA at the Rx). The complexity of [10] is
$\pazocal{O}(\max(N^{3},M^{3}))$, due to a QR decomposition of the matrices
whose elements correspond to the antenna positions of the Tx and Rx. The
complexity of [11] is equal to $\pazocal{O}(AQ)$, where $A$ is the number of
possible values of a parameter $\alpha_{1}$ which characterizes the NULA
parameterization [11]. In Fig. 4, $\alpha_{1}=0.205$ is used, as this lead to
best performance. Our proposed designs outperform the ones in [10, 11] in
terms of minimum capacity, standard deviation and, thus, of capacity
fluctuations over the considered range.

CO+RR NULAs, $\mu=22.5,\,\sigma=1.1,\,\text{min}=21.1$
---
[10] NULAs, $\mu=24,\,\sigma=2.3,\,\text{min}=13.8$
---
[11] NULAs, $\mu=22.2,\,\sigma=2.3,\,\text{min}=17.9$
---

Capacity (bpcu)
---

Transmit distance $\bm{d}$ (m)
---

Figure 4: Capacity vs. transmit distance of proposed and state-of-the-art
NULAs.

The min. capacity vs. the spacing $d_{\text{t,f}}$ of the (uniform) grid of
candidate antenna locations is shown in Fig. 5 for the CO+RR approach, where
$N=M=4$ and the same Tx and Rx grid spacing ($d_{\text{t,f}}=d_{\text{r,f}}$)
is assumed. We assume increasingly smaller spacing, so that the effect of the
grid’s granularity can be evaluated. For a grid spacing $d_{\text{t,f}}$ that
is $\eta$ times finer than the ULA spacing, i.e.,
$d_{\text{t,f}}=d_{\text{t}}/\eta$, the number of candidate antenna locations
is given as $N_{\text{f}}=\eta(N-1)+1$. For example, for
$d_{\text{t,f}}=d_{\text{t}}/8$, we have that $N_{\text{f}}=25$. The same
holds for the Rx grid. As expected, the min. capacity is increased for grids
of finer granularity, that is, of higher number of candidate antenna
locations. However, it seems that only marginal improvement is attained when
the spacing becomes too small. The minimum allowable grid spacing is dictated
by (2a) and (2b) (both are satisfied in Fig. 5). For comparison,
$d_{\text{t,f}}=d_{\text{r,f}}=d_{\text{t}}/5$ which corresponds to
$N_{\text{f}}=M_{\text{f}}=16$ that was used in Fig. 2 \- 4 is also
illustrated in Fig. 5 with a dotted line.

Uniform grid spacing $d_{\text{t,f}}$
---
$d_{\text{t}}/2$
---
$d_{\text{t}}/4$
---
$d_{\text{t}}/8$
---
$d_{\text{t}}/16$
---
$d_{\text{t}}/5$
---

Min. capacity (bpcu)
---

Figure 5: Min. capacity vs. different Tx and Rx uniform grid granularities.

##  V Conclusions

We proposed a novel method for the joint design of non-uniform Tx and Rx
arrays of arbitrary geometries for LoS MIMO systems towards maximizing the
minimum capacity over a range of transmit distances. The method was based on
optimally selecting antenna locations from a grid of candidate antenna
locations. We demonstrated, by leveraging convex relaxation, that the joint Tx
and Rx array design problem can be formulated as a convex optimization
problem, solved in polynomial time, if it is solved iteratively. A local
optimization method based on RR was also incorporated to improve the
performance. The non-uniform arrays designed with our approach featured
superior capacity performance compared to uniform and non-uniform array
configurations of the literature and comparable performance to the ES-based
designs, despite the much lower computational complexity of our approach.

## References

  * [1] F. Tariq, M. R. A. Khandaker, K.-K. Wong, M. A. Imran, M. Bennis, and M. Debbah, “A speculative study on 6G,” _IEEE Wireless Communications_ , vol. 27, no. 4, pp. 118–125, 2020.
  * [2] R. Méndez-Rial, C. Rusu, N. González-Prelcic, A. Alkhateeb, and R. W. Heath, “Hybrid MIMO architectures for millimeter wave communications: Phase shifters or switches?” _IEEE Access_ , vol. 4, pp. 247–267, 2016.
  * [3] D. Gesbert, H. Bolcskei, D. Gore, and A. Paulraj, “Outdoor MIMO wireless channels: models and performance prediction,” _IEEE Trans. Commun._ , vol. 50, no. 12, pp. 1926–1934, 2002.
  * [4] P. Larsson, “Lattice array receiver and sender for spatially orthonormal MIMO communication,” in _2005 IEEE 61st Vehicular Technology Conference_ , vol. 1, 2005, pp. 192–196 Vol. 1.
  * [5] Y. Jeon, G.-T. Gil, and Y. H. Lee, “Design and analysis of LoS-MIMO systems with uniform circular arrays,” _IEEE Trans. Wireless Commun._ , vol. 20, no. 7, pp. 4527–4540, 2021.
  * [6] E. Torkildson, C. Sheldon, U. Madhow, and M. Rodwell, “Nonuniform array design for robust millimeter-wave MIMO links,” in _GLOBECOM 2009 - 2009 IEEE Global Telecommunications Conference_ , 2009, pp. 1–7.
  * [7] C. Bencivenni, M. Coldrey, R. Maaskant, and M. V. Ivashina, “Aperiodic switched array for line-of-sight MIMO backhauling,” _IEEE Antennas Wireless Propag. Lett._ , vol. 17, no. 9, pp. 1712–1716, 2018.
  * [8] N. Matsumura, K. Nishimori, R. Taniguchi, T. Hiraguri, T. Tomura, and J. Hirokawa, “Novel unmanned aerial vehicle-based line-of-sight MIMO configuration independent of transmitted distance using millimeter wave,” _IEEE Access_ , vol. 8, pp. 11 679–11 691, 2020.
  * [9] S. Pratschner, D. Löschenbrand, S. Schwarz, T. Zemen, and M. Rupp, “Large aperture antenna array design for cellular LOS massive MIMO,” in _2019 53rd Asilomar Conference on Signals, Systems, and Computers_ , 2019, pp. 1404–1408.
  * [10] P. Wang, Y. Li, Y. Peng, S. C. Liew, and B. Vucetic, “Non-uniform linear antenna array design and optimization for millimeter-wave communications,” _IEEE Trans. Wireless Commun._ , vol. 15, no. 11, pp. 7343–7356, 2016.
  * [11] D. Pinchera, M. D. Migliore, F. Schettino, and G. Panariello, “Antenna arrays for line-of-sight massive MIMO: Half wavelength is not enough,” _Electronics_ , vol. 6, no. 3, 2017.
  * [12] M. Ehrgott, _Multicriteria optimization_. Springer Science & Business Media, 2005, vol. 491.
  * [13] P. Stoica and Y. Selen, “Cyclic minimizers, majorization techniques, and the expectation-maximization algorithm: A refresher,” _IEEE Signal Process. Mag._ , vol. 21, no. 1, pp. 112–114, 2004.
  * [14] P. Raghavan and C. D. Tompson, “Randomized rounding: a technique for provably good algorithms and algorithmic proofs,” _Combinatorica_ , vol. 7, no. 4, pp. 365–374, 1987.
  * [15] X. Song, W. Rave, N. Babu, S. Majhi, and G. Fettweis, “Two-level spatial multiplexing using hybrid beamforming for millimeter-wave backhaul,” _IEEE Trans. Wireless Commun._ , vol. 17, no. 7, pp. 4830–4844, 2018.
  * [16] S. Boyd and L. Vandenberghe, _Convex optimization_. Cambridge university press, 2004.
  * [17] A. Dua, K. Medepalli, and A. Paulraj, “Receive antenna selection in MIMO systems using convex optimization,” _IEEE Trans. Wireless Commun._ , vol. 5, no. 9, pp. 2353–2357, 2006.
  * [18] S. Joshi and S. Boyd, “Sensor selection via convex optimization,” _IEEE Trans. Signal Process._ , vol. 57, no. 2, pp. 451–462, 2009.
  * [19] H. Zhang, J. Shi, Q. Zhang, B. Zong, and J. Xie, “Antenna selection for target tracking in collocated MIMO radar,” _IEEE Trans. Aerosp. Electron. Syst._ , vol. 57, no. 1, pp. 423–436, 2021.