# Optimized Design for IRS-Assisted Integrated Sensing and Communication
Systems in
Clutter Environments

Chikun Liao, , Feng Wang, ,
and Vincent K. N. Lau C. Liao and F. Wang are with the School of Information
Engineering, Guangdong University of Technology, Guangzhou 510006, China
(e-mail: fengwang13@gdut.edu.cn).V. K. N. Lau is with the Department of
Electronic and Computer Engineering, The Hong Kong University of Science and
Technology, Hong Kong (e-mail: eeknlau@ust.hk).

###### Abstract

In this paper, we investigate an intelligent reflecting surface (IRS)-assisted
integrated sensing and communication (ISAC) system design in a clutter
environment. Assisted by an IRS equipped with a uniform linear array (ULA), a
multi-antenna base station (BS) is targeted for communicating with multiple
communication users (CUs) and sensing multiple targets simultaneously. We
consider the IRS-assisted ISAC design in the case with Type-I or Type-II CUs,
where each Type-I and Type-II CU can and cannot cancel the interference from
sensing signals, respectively. In particular, we aim to maximize the minimum
sensing beampattern gain among multiple targets, by jointly optimizing the BS
transmit beamforming vectors and the IRS phase shifting matrix, subject to the
signal-to-interference-plus-noise ratio (SINR) constraint for each
Type-I/Type-II CU, the interference power constraint per clutter, the
transmission power constraint at the BS, and the cross-correlation pattern
constraint. Due to the coupling of the BS’s transmit design variables and the
IRS’s phase shifting matrix, the formulated max-min IRS-assisted ISAC design
problem in the case with Type-I/Type-II CUs is highly non-convex. As such, we
propose an efficient algorithm based on the alternating-optimization and semi-
definite relaxation (SDR) techniques. In the case with Type-I CUs, we show
that the dedicated sensing signal at the BS is always beneficial to improve
the sensing performance. By contrast, the dedicated sensing signal at the BS
is not required in the case with Type-II CUs. Numerical results are provided
to show that the proposed IRS-assisted ISAC design schemes achieve a
significant gain over the existing benchmark schemes.

###### Index Terms:

Integrated sensing and communication (ISAC), intelligent reflecting surface
(IRS), phase shifting, interference mitigation, clutter environments,
optimization.

##  I Introduction

The advancement of the beyond-fifth-generation (B5G) and sixth-generation (6G)
communication networks has led to various integrated sensing and communication
(ISAC) applications, such as auto-driving, vehicle-to-everything, smart home,
virtual/augmented reality, and edge intelligence [1, 2, 3, 4, 5]. In ISAC
systems, the sensing and communication capabilities are tightly integrated to
provide both sensing and communication services [6, 7, 8, 9, 10, 11, 12, 13,
14, 15, 16]. The resources of communication and radar sensing can be shared in
ISAC systems, thereby boosting the efficiency of utilization in terms of
spectrum, hardware, energy, and cost [17].

The emerging intelligent reflective surface (IRS) (also known as
reconfigurable intelligent surface (RIS)) has recently attracted a growing
interest from academia and industry as a promising approach to improving the
communication performance[18, 19, 20]. Motivated by the effectiveness of IRS-
assisted communication systems, the IRS technology has been proposed to assist
ISAC systems to further improve the system performance[21, 22, 23]. For
example, the authors in [21] employed the IRS to mitigate multi-user
interference to improve the communication performance, while matching the
radar waveform. The IRS-assisted communication with multiple sensing targets
were studied in the mmWave systems [22]. In [23] the authors adopted the
Cram$\acute{e}$r-Rao bound (CRB) as a radar sensing performance indicator, and
studied the joint waveform and discrete phase shift designs for the IRS-
assisted ISAC systems.

Notice that the radar detection and sensing performance relies highly on the
direct sensing link from the base station (BS). It is critical to improve the
ISAC performance for the sensing targets in the non-line-of-sight (NLoS) areas
[24, 25, 26, 27, 28, 29]. By reconfiguring the wireless propagation
environment, the IRS technology can help improve wireless communication rate,
as well as to enhance the sensing accuracy and resolution performance [24].
Specifically, the IRS can constructively enhance the reflected sensing signals
and mitigate the malicious interference. Also, the IRS technology can provide
additional degrees of freedom to combat the severe channel fading effect. The
authors in [25] considered an IRS-assisted ISAC system, by leveraging IRS to
create new channel links to achieve a high-quality communication and sensing
performance. The work in [26] studied a hybrid IRS model comprising active and
passive elements to assist the radar and communication, and aimed to maximize
the worst-case target illumination power. The authors in [27] studied using
multiple IRSs for further enhancing the ISAC performance. Note that by
minimizing the cross-correlation coefficient between radar signals, one can
achieve a better radar detection performance [28]. The authors in [29] studied
the IRS-assisted ISAC system design under the radar signal cross-correlation
pattern constraints, where the IRS may introduce additional interference for
sensing. Note that the aforementioned works[24, 25, 26, 27, 28, 29] all
considered clutter-free environments, and there lacks investigation of IRS-
assisted ISAC designs in a clutter environment.

In a clutter environment, the radar sensing performance is often affected by
various clutters, such as trees, tall buildings, and cars[30]. The reflected
signal power from the clutter may be higher than the target’s reflected signal
power, which can severely degrade the target detection reliability. How to
detect and sense multiple targets in a clutter environment is a challenging
issue for ISAC system designs[31, 32, 33]. The clutter can be treated as a
signal-dependent type of interference, which depends on a deterministic
transmitted signal [34]. As such, one can adopt the clutter’s prior knowledge
such as the clutter covariance matrix to achieve the suppression of
clutter[35]. By considering multiple clutters in an IRS-assisted ISAC system,
the authors in [36] aimed to maximize the radar SINR subject to the
communication quality of service requirement constraint. Note that the impact
of clutter on the blocking of the line-of-sight (LoS) link is ignored in [36].
The work in [30] studied the radar design with multiple IRSs in order to
assist the sensing services in a clutter environment, where the effective
sensing links were created by the IRSs for addressing the unavailability of
the direct sensing link. In a clutter environment, the clutter interference to
the radar system must be mitigated to achieve a desirable target sensing
performance, which motivates our work in this paper.

In this paper, we consider an IRS-assisted ISAC system in a clutter
environment. The system consists of one multi-antenna BS providing ISAC
service, an IRS with a uniform linear array (ULA), multiple CUs, and multiple
targets to be sensed. We consider two types of CUs, i.e., Type-I and Type-II
CUs, where the Type-I CU has the ability to eliminate the interference caused
by sensing signals, and the Type-II CU cannot eliminate the interference power
caused by the sensing signals. In a clutter environment, we pursue an
efficient IRS-assisted ISAC design to maximize the minimum sensing beampattern
gain among multiple targets. The main contributions of this paper are
summarized as follows.

  * •

First, we propose an optimization framework in a clutter environment for the
IRS-assisted ISAC system design with Type-I or Type-II CUs. Specifically, we
maximize the minimum sensing beampattern gain among multiple targets, subject
to the BS’s transmission power constraint, the signal-to-interference-plus-
noise-ratio (SINR) for each Type-I or Type-II CU, the power constraint of each
clutter, and the radar signals cross-correlation pattern constraint. We
jointly optimize the BS’s transmit beamforming vectors for multiple CUs and
sensing covariance matrix for multiple targets, as well as the IRS’s phase
shifting matrix. Due to the variable coupling, the formulated max-min IRS-
assisted ISAC design problem in the case with Type-I or Type-II CUs is non-
convex, which is difficult to obtain the global solution.

  * •

Second, by combining the alternating-optimization and semi-definite relaxation
(SDR) techniques, we propose a low-complexity method to obtain the solutions
for the IRS-assisted ISAC designs in the cases with Type-I and Type-II CUs,
respectively. The tightness of the SDR and the computational complexities for
the obtained solutions are analyzed. It is shown that the dedicated sensing
signal is always required at the BS to enhance the sensing performance in the
case with Type-I CUs in clutter environments. By contrast, in the case with
Type-II CUs, the dedicated sensing signals can be removed at the BS without
loss of the system sensing performance.

  * •

Finally, extensive numerical results are provided to show the effectiveness of
our proposed max-min IRS-assisted ISAC design schemes. In the case with Type-I
CUs or Type-II CUs, the proposed design scheme is shown to achieve a
significant performance gain over the existing benchmark schemes. It is also
shown that the IRS-assisted ISAC system with Type-I CUs always outperform that
with Type-II CUs. Furthermore, the deployment of the IRS can effectively help
increase the robustness against the link blockage in clutter environments.

The rest of this paper is organized as follows. Section II presents the IRS-
assisted ISAC system model in a clutter environment and formulates the max-min
design problems under consideration. Section III and Section IV propose low-
complexity design solutions in the cases with Type-I CUs and Type-II CUs,
respectively. Section V shows numerical results to evaluate the proposed
design schemes. Finally, we conclude the paper in Section VI.

Notations: Boldface letters refer to vectors (lower case) or matrices (upper
case); rank($\cdot$), tr($\cdot$), diag($\cdot$), and vec($\cdot$) denote the
rank, the trace, the diagonal, and the vectorization operations, respectively;
$[\cdot]^{\ast}$, $[\cdot]^{T}$, and $[\cdot]^{H}$ denote the conjugate,
transpose, and Hermitian transpose operations, respectively;
$\left\|\cdot\right\|$ denotes the Euclidean norm.

##  II System Model and Problem Formulation

Figure 1: The IRS-assisted ISAC system model in a clutter environment, where
no LoS links exist between the BS and multiple targets due to multiple
clutters.

As shown in Fig. 1, we consider an IRS-assisted ISAC system in a clutter
environment, where a multi-antenna BS is assisted by an IRS for sending
unicast communication signals for a number of $K$ single-antenna CUs and
dedicated radar sensing signals for a total of $L$ targets, and the IRS is
deployed to help propagate communication and radar sensing signals. The BS is
equipped with $M>1$ antennas, and the IRS is equipped with an uniform linear
array (ULA) of $N>1$ passive reflecting elements. We consider a number of $Q$
clutters between the BS and the $L$ sensing targets. The CU set and the
clutter set are denoted by ${\cal K}\triangleq\\{1,...,K\\}$ and
$\mathcal{Q}\triangleq\\{1,...,Q\\}$, respectively. Denote by ${\cal
L}\triangleq\\{1,...,L\\}$ the set of $L$ sensing targets. Due to the signal
blockage in propagation, we assume that there exists no line-of-sight (LoS)
link between the BS and the $L$ targets for sensing due to the existence of
the $Q$ clutters. We consider that the BS has obtained the global channel
state information (CSI) and the location information of the sensing targets
during the detection phase.

Denote by $s_{k}\in\mathbb{C}$ and $\boldsymbol{w}_{k}\in\mathbb{C}^{M\times
1}$ the information symbol and transmit beamforming vector for CU $k\in{\cal
K}$, respectively. Without loss of generality, we assume that the transmit
symbols $\\{s_{k}\\}$ are independent and identical distributed (i.i.d.)
random variables with zero mean and unit variance, i.e.,
$s_{k}\sim\mathcal{CN}(0,1)$, $k\in{\cal K}$. Let
$\boldsymbol{x}_{0}\in\mathbb{C}^{M\times 1}$ be the dedicated sensing signal
of the BS with zero mean, i.e., $\mathbb{E}[\boldsymbol{x}_{0}]=0$, and the
sensing covariance matrix
$\boldsymbol{R}_{0}=\mathbb{E}\left[\boldsymbol{x}_{0}\boldsymbol{x}_{0}^{H}\right]$
is with a general rank, i.e., $\text{rank}(\boldsymbol{R}_{0})\geq 1$. As a
result, the ISAC transmit signal $\boldsymbol{x}\in\mathbb{C}^{M\times 1}$ of
the BS is modeled as

$\boldsymbol{x}=\underbrace{\boldsymbol{x}_{0}}_{\text{sensing~{}signal~{}for}~{}L~{}\text{targets}}+\sum_{k=1}^{K}\underbrace{\boldsymbol{w}_{k}{s}_{k}}_{\text{commun.~{}signal
for CU}~{}k}.$ (1)

Based on (1), the transmit power constraint of the BS is expressed as

$\mathbb{E}[\|\boldsymbol{x}\|^{2}]=\text{tr}(\boldsymbol{R}_{0})+\sum_{k=1}^{K}\left\|\boldsymbol{w}_{k}\right\|^{2}\leq{P_{0}},$
(2)

where the expectation $\mathbb{E}[\cdot]$ is taken on the randomness of
$\\{\boldsymbol{x}_{0},s_{k}\\}$, and $P_{0}$ denotes the maximum transmit
power budget of the BS.

###  II-A IRS-assisted Communication Model for Type-I and Type-II CUs

Let $\boldsymbol{G}\in\mathbb{C}^{N\times{M}}$ denote the complex-valued
channel matrix from the BS to the IRS, and let
$\boldsymbol{h}_{bu,k}\in\mathbb{C}^{M\times{1}}$ and
$\boldsymbol{h}_{iu,k}\in\mathbb{C}^{N\times{1}}$ denote the complex-valued
channel vectors from the BS and the IRS to CU $k$, respectively. Denote by
$\phi_{n}\in{\left(0,2\pi\right]}$ the phase shift value of each passive
reflecting element $n\in{\cal N}\triangleq\\{1,...,N\\}$ at the IRS. Let
$\boldsymbol{\Phi}=\text{diag}(e^{j\phi_{1}},\cdot\cdot\cdot,e^{j\phi_{N}})\in\mathbb{C}^{N\times
N}$ denote the IRS’s phase shifting matrix, where $j=\sqrt{-1}$ denotes the
imaginary unit. In the IRS-assisted ISAC system under consideration, each CU
$k\in{\cal K}$ receives the BS’s signal via both the direct BS-CU link and the
cascaded BS-IRS-CU link. Accordingly, the received signal $y_{k}\in\mathbb{C}$
of CU $k\in{\cal K}$ is modeled as

$\displaystyle y_{k}$
$\displaystyle=\boldsymbol{h}^{H}_{\text{CU},k}\boldsymbol{x}+n_{k}$ (3a)
$\displaystyle=\boldsymbol{h}_{\text{CU},k}^{H}\Big{(}\boldsymbol{x}_{0}+\sum_{k=1}^{K}\boldsymbol{w}_{k}s_{k}\Big{)}+n_{k},~{}\forall
k\in{\cal K},$ (3b)

where
$\boldsymbol{h}_{\text{CU},k}\triangleq\boldsymbol{G}^{H}\boldsymbol{\Phi}^{H}\boldsymbol{h}_{iu,k}+\boldsymbol{h}_{bu,k}$
is defined as the effective channel vector from the BS to CU
$k\in\mathcal{K}$, and $n_{k}\sim\mathcal{CN}(0,\sigma_{k}^{2})$ denotes the
additive white Gaussian noise (AWGN) at CU $k$’s receiver.

Depending on the ability of cancelling the sensing signal $\boldsymbol{x}_{0}$
or not, we consider two types of CUs, i.e., Type-I and Type-II CUs. In the
case with Type-I CUs, the sensing signal $\boldsymbol{x}_{0}$ is globally
known, and each Type-I CU can completely cancel the interference generated by
the sensing signal $\boldsymbol{x}_{0}$. Denote by $\gamma^{\text{I}}$ the
SINR of Type-I CU $k\in{\cal K}$. Accordingly, we have

$\gamma_{k}^{\text{I}}=\frac{\left|\boldsymbol{h}^{H}_{\text{CU},k}\boldsymbol{w}_{k}\right|^{2}}{\sum_{j=1,j\neq
k}^{K}\left|\boldsymbol{h}^{H}_{\text{CU},k}\boldsymbol{w}_{j}\right|^{2}+\sigma_{k}^{2}},~{}\forall
k\in{\cal K}.$ (4)

In the case with Type-II CUs, the sensing signal $\boldsymbol{x}_{0}$ is not
globally known, and each Type-II CU cannot eliminate the interference power
caused by the sensing signal $\boldsymbol{x}_{0}$. Denote by
$\gamma^{\text{II}}$ the SINR of Type-II CU $k\in{\cal K}$. Then, we have

$\gamma_{k}^{\text{II}}=\frac{\left|\boldsymbol{h}_{\text{CU},k}^{H}\boldsymbol{w}_{k}\right|^{2}}{\sum_{j=1,j\neq
k}^{K}\left|\boldsymbol{h}_{\text{CU},k}^{H}\boldsymbol{w}_{j}\right|^{2}+\left|\boldsymbol{h}_{\text{CU},k}^{H}\boldsymbol{x}_{0}\right|^{2}+\sigma_{k}^{2}},~{}\forall
k\in{\cal K}.$ (5)

###  II-B IRS-assisted Sensing Model for $L$ Targets

In the IRS-assisted ISAC system, the IRS can reconfigure a sensing link (i.e.,
the BS-IRS-target cascaded channel link) to provide a direct target sensing
service by properly adjusting the phases of its reflecting elements. Let
$\theta_{l}\in(-\frac{\pi}{2},\frac{\pi}{2})$ denote the angle of target
$l\in{\cal L}$ with respect to the IRS. Let $d_{\text{IRS}}$ denote the
distance between two consecutive reflecting elements of the IRS, and let
$\lambda$ denote the wavelength of the BS’s radio-frequency (RF) transmit
signal. For each target $l\in{\cal L}$, the array steering vector of the IRS
with angle of departure (AoD) $\theta_{l}$ is given by

$\boldsymbol{\alpha}(\theta_{l})=\left[1,e^{j\frac{2\pi
d_{\text{IRS}}}{\lambda}sin\theta_{l}},...,e^{j\frac{2\pi(N-1)d_{\text{IRS}}}{\lambda}sin\theta_{l}}\right]^{T},~{}\forall
l\in{\cal L}.$ (6)

In ISAC systems, both the communication signals $\\{s_{k}\\}_{k\in{\cal K}}$
and the dedicated sensing signal $\boldsymbol{x}_{0}$ can be used to
illuminate the $L$ targets to satisfy the required sensing beampattern gain
constraint. As a result, the beampattern gain $\rho(\theta_{l})$ for target
$l\in{\cal L}$ located at the AoD angle $\theta_{l}$ with respect to the IRS
is given as [37]

$\displaystyle\rho(\theta_{l})$
$\displaystyle=\mathbb{E}\Bigg{[}\Big{|}\boldsymbol{h}_{\text{target},l}^{H}\Big{(}\boldsymbol{x}_{0}+\sum_{k=1}^{K}\boldsymbol{w}_{k}s_{k}\Big{)}\Big{|}^{2}\Bigg{]}$
(7a)
$\displaystyle=\boldsymbol{h}_{\text{target},l}^{H}\Big{(}\boldsymbol{R}_{0}+\sum_{k=1}^{K}\boldsymbol{w}_{k}\boldsymbol{w}_{k}^{H}\Big{)}\boldsymbol{h}_{\text{target},l},~{}\forall
l\in{\cal L},$ (7b)

where
$\boldsymbol{h}_{\text{target},l}\triangleq\boldsymbol{G}^{H}\boldsymbol{\Phi}^{H}\boldsymbol{\alpha}(\theta_{l})$
is defined as the effective LoS channel vector from the BS to target
$l\in{\cal L}$.

Furthermore, we consider the cross-correlation pattern constraint for the ISAC
signal to sense the $L$ targets with different angles. Let $P^{\text{cross}}$
denote the mean-squared cross-correlation pattern for the $L$ targets. As in
[38], we have

$P^{\text{cross}}=\frac{2}{L(L-1)}\sum_{l=1}^{L-1}\sum_{i=l+1}^{L}P(\theta_{l},\theta_{i}),$
(8)

where $P(\theta_{l},\theta_{i})$ denotes the cross-correlation coefficient
between target $l$ at angle $\theta_{l}$ and another target $i$ at angle
$\theta_{i}$, and we have

$\displaystyle
P(\theta_{l},\theta_{i})\triangleq\boldsymbol{h}_{\text{target},l}^{H}\Big{(}\boldsymbol{R}_{0}+\sum_{k=1}^{K}\boldsymbol{w}_{k}\boldsymbol{w}_{k}^{H}\Big{)}\boldsymbol{h}_{\text{target},i},$
(9)

where $l\neq i\in{\cal L}$.

###  II-C Clutter Model for $Q$ Clutters

For each clutter $q\in{\cal Q}$, we consider the associated interference
power. Let $\boldsymbol{g}_{bc,q}\in\mathbb{C}^{M\times 1}$ denote the
complex-valued channel vector from the BS to clutter $q\in{\cal Q}$, and let
$\boldsymbol{g}_{ic,q}\in\mathbb{C}^{N\times 1}$ denote the complex-valued
cascaded channel vector from the BS to clutter $q$ via the IRS. As a result,
the interference power incurred by each clutter $q\in{\cal Q}$ is expressed as

$\displaystyle P^{\text{clutter}}_{q}$
$\displaystyle=\mathbb{E}\Bigg{[}\Big{|}\boldsymbol{g}_{\text{clutter},q}^{H}\Big{(}\boldsymbol{x}_{0}+\sum_{k=1}^{K}\boldsymbol{w}_{k}s_{k}\Big{)}\Big{|}^{2}\Bigg{]}$
(10a)
$\displaystyle=\boldsymbol{g}_{\text{clutter},q}^{H}\Big{(}\boldsymbol{R}_{0}+\sum_{k=1}^{K}\boldsymbol{w}_{k}\boldsymbol{w}_{k}^{H}\Big{)}\boldsymbol{g}_{\text{clutter},q},~{}\forall
q\in{\cal Q},$ (10b)

where
$\boldsymbol{g}_{\text{clutter},q}\triangleq\boldsymbol{G}^{H}\boldsymbol{\Phi}^{H}\boldsymbol{g}_{ic,q}+\boldsymbol{g}_{bc,q}$
is defined as the effective channel vector from the BS to clutter
$q\in\cal{Q}$.

###  II-D Problem Formulation

In this paper, we are interested in maximizing the minimum sensing beampattern
gain among the $L$ targets, by jointly optimizing the BS’s transmit
beamforming vectors $\\{\boldsymbol{w}_{k}\\}_{k\in{\cal K}}$ for
communication and covariance matrix $\boldsymbol{R}_{0}$ for sensing, as well
as the IRS’s phase shifting matrix $\boldsymbol{\Phi}$. The constraints
include the BS’s transmit power constraint, the SINR constraint for each
Type-I or Type-II CU $k\in{\cal K}$, the interference power constraint for
each clutter $q\in{\cal Q}$, and the mean-squared cross-correlation pattern
constraint for the $L$ targets.

In the case with Type-I CU, we formulate the following optimization problem as

$\displaystyle(\text{P}1):$
$\displaystyle\max_{\\{\boldsymbol{w}_{k}\\}_{k\in{\cal
K}},\boldsymbol{R}_{0}\succeq\boldsymbol{0},\boldsymbol{\Phi}}~{}\min_{l\in\cal{L}}~{}\rho(\theta_{l})$
(11a) s.t.
$\displaystyle\sum_{k=1}^{K}\left\|\boldsymbol{w}_{k}\right\|^{2}+\text{tr}(\boldsymbol{R}_{0})\leq
P_{0}$ (11b) $\displaystyle\gamma_{k}^{\text{I}}\geq\Gamma_{k},~{}\forall
k\in{\cal K}$ (11c) $\displaystyle
P_{q}^{\text{clutter}}\leq\eta_{q},~{}\forall q\in{\cal Q}$ (11d)
$\displaystyle P^{\text{cross}}\leq\xi,$ (11e)

where $\Gamma_{k}$ denotes the SINR threshold prescribed for each Type-I CU
$k\in{\cal K}$, $\eta_{q}$ denotes the given interference power threshold for
clutter $q\in{\cal Q}$, and $\xi$ denotes the predefined power threshold for
the mean-squared cross-correlation pattern $P^{\text{cross}}$.

In the case with Type-II CU, we have the following design optimization problem
as

$\displaystyle(\text{P}2):$
$\displaystyle\max_{\\{\boldsymbol{w}_{k}\\}_{k\in{\cal
K}},\boldsymbol{R}_{0}\succeq\boldsymbol{0},\boldsymbol{\Phi}}~{}\min_{l\in\mathcal{L}}~{}\rho(\theta_{l})$
(12a) s.t. $\displaystyle~{}\gamma_{k}^{\text{II}}\geq\Gamma_{k},~{}\forall
k\in{\cal K}$ (12b)
$\displaystyle~{}(\ref{eq.p1}b),~{}(\ref{eq.p1}d),~{}{\text{and}}~{}(\ref{eq.p1}e),$

where $\Gamma_{k}$ denotes the predefined SINR threshold for each Type-II CU
$k\in{\cal K}$.

In this paper, we consider problems (P1) and (P2) are always guaranteed to be
feasible, by properly managing the admission of CUs and mitigating
interference powers. Note that the BS’s transmit beamforming vectors
$\\{\boldsymbol{w}_{k}\\}_{k\in{\cal K}}$, the BS’s sensing covariance matrix
$\boldsymbol{R}_{0}$, and the IRS’s phase shifting matrix $\boldsymbol{\Phi}$
are coupled, problems (P1) and (P2) are highly non-convex. It is highly
complicated to obtain the global solutions for problems (P1) and (P2).
Alternatively, based on the alternating-optimization method, we next pursue to
obtain the low-complexity sub-optimal solutions for problems (P1) and (P2).

##  III Proposed Solution for (P1) with Type-I CUs

In this section, we propose an efficient alternating-optimization-based
solution for problem (P1) in the case with Type-I CUs, and then discuss the
computational complexity.

###  III-A Optimization of BS’s Transmission Design
$(\\{\boldsymbol{w}_{k}\\}_{k\in{\cal K}},\boldsymbol{R}_{0})$ for ISAC

In this subsection, under the fixed phase shifting matrix $\boldsymbol{\Phi}$
of the IRS, we optimize the BS’s transmit beamforming vectors
$\\{\boldsymbol{w}_{k}\\}_{k\in{\cal K}}$ for the $K$ Type-I CUs and
covariance matrix $\boldsymbol{R_{0}}$ for sensing $L$ targets.

To this end, we introduce a beamforming matrix
$\boldsymbol{W}_{k}\triangleq\boldsymbol{w}_{k}\boldsymbol{w}_{k}^{H}$ for
each Type-I CU $k\in{\cal K}$. It is clear that
$\boldsymbol{W}_{k}\succeq\boldsymbol{0}$ and
$\text{rank}(\boldsymbol{W}_{k})\leq 1$, $\forall k\in{\cal K}$. By
substituting $\boldsymbol{w}_{k}\boldsymbol{w}_{k}^{H}$ with
$\boldsymbol{W}_{k}$ for each $k\in{\cal K}$ and fixing the IRS’s phase
shifting matrix $\boldsymbol{\Phi}$, the original max-min ISAC design problem
(P1) in the case with Type-I CUs is equivalently transformed as

$\displaystyle(\text{P1.1}):$
$\displaystyle\max_{\\{\boldsymbol{W}_{k}\\}_{k\in{\cal
K}},\boldsymbol{R}_{0}\succeq
0}~{}\min_{l\in\mathcal{L}}~{}\boldsymbol{h}_{\text{target},l}^{H}\Big{(}\boldsymbol{R}_{0}+\sum_{k=1}^{K}\boldsymbol{W}_{k}\Big{)}\boldsymbol{h}_{\text{target},l}$
(13a) s.t.
$\displaystyle~{}\text{tr}(\boldsymbol{R}_{0})+\sum_{k=1}^{K}\text{tr}(\boldsymbol{W}_{k})\leq
P_{0}$ (13b)
$\displaystyle~{}\frac{1}{\Gamma_{k}}\text{tr}(\boldsymbol{h}_{\text{CU},k}\boldsymbol{h}_{\text{CU},k}^{H}\boldsymbol{W}_{k})-\sum_{j=1,j\neq
k}^{K}\text{tr}(\boldsymbol{h}_{\text{CU},k}\boldsymbol{h}_{\text{CU},k}^{H}\boldsymbol{W}_{j})-\sigma_{k}^{2}\geq
0,~{}\forall k\in\mathcal{K}$ (13c)
$\displaystyle~{}\boldsymbol{g}_{\text{clutter},q}^{H}\Big{(}\boldsymbol{R}_{0}+\sum_{k=1}^{K}\boldsymbol{W}_{k}\Big{)}\boldsymbol{g}_{\text{clutter},q}\leq\eta_{q},~{}\forall
q\in\cal{Q}$ (13d)
$\displaystyle~{}\frac{2}{L^{2}-L}\sum_{l=1}^{L-1}\sum_{i=l+1}^{L}\boldsymbol{h}_{\text{target},l}^{H}\Big{(}\sum_{k=1}^{K}\boldsymbol{W}_{k}+\boldsymbol{R}_{0}\Big{)}\boldsymbol{h}_{\text{target},i}^{H}\leq\xi$
(13e)
$\displaystyle~{}\boldsymbol{W}_{k}\succeq\boldsymbol{0},~{}\text{rank}(\boldsymbol{W}_{k})\leq
1,\forall k\in\mathcal{K},$ (13f)

where the design variables are $\\{\boldsymbol{W}_{k}\\}_{k\in{\cal K}}$ and
$\boldsymbol{R}_{0}$. Due to the rank-one constraints in (13f), problem (P1.1)
is still non-convex. By employing the celebrated SDR technique[39] for problem
(P1.1), we are ready to obtain a convex optimization with respect to
$(\\{\boldsymbol{W}_{k}\\}_{k\in{\cal K}},\boldsymbol{R}_{0})$ by removing the
rank-one constraints $\text{rank}(\boldsymbol{W}_{k})\leq 1$,$\forall
k\in{\cal K}$. As a result, the relaxed problem (P1.1) can be efficiently
solved by using standard convex solvers such as the CVX toolbox [40].

Let $(\\{\boldsymbol{W}_{k}^{\text{I}*}\\}_{k\in{\cal
K}},\boldsymbol{R}_{0}^{\text{I}*})$ denote the optimal solution to the
relaxed problem (P1.1). In particular, if
$\text{rank}(\boldsymbol{W}_{k}^{\text{I}*})\leq 1$, $\forall k\in{\cal K}$,
then the relaxation of problem (P1.1) is tight, which implies that
$(\\{\boldsymbol{W}_{k}^{\text{I}*}\\}_{k\in{\cal
K}},\boldsymbol{R}_{0}^{\text{I}*})$ is the optimal solution to problem
(P1.1). In this case, by implementing eigenvalue decomposition (EVD) for each
$\boldsymbol{W}_{k}^{\text{I}*}$, we have
$\boldsymbol{W}_{k}^{\text{I}*}=\lambda_{k}\boldsymbol{v}_{k}\boldsymbol{v}^{H}_{k}$,
$\forall k\in{\cal K}$, where $\lambda_{k}>0$ denotes the maximal eigenvalue
of $\boldsymbol{W}_{k}^{\text{I}*}$, and $\boldsymbol{v}_{k}$ denotes the
corresponding unit-norm eigenvector associated with the eigenvalue
$\lambda_{k}$. Then, we are ready to obtain the optimal solution
$(\\{\sqrt{\lambda}_{k}\boldsymbol{v}_{k}\\}_{k\in{\cal
K}},\boldsymbol{R}_{0}^{\text{I}*})$ to problem (P1) under the given phase
shifting matrix $\boldsymbol{\Phi}$ of the IRS. On the other hand, if there
exists $\text{rank}(\boldsymbol{W}_{k}^{\text{I}*})>1$ for a certain CU
$k\in{\cal K}$, then a reconstruction procedure for
$(\\{\boldsymbol{W}_{k}^{\text{I}*}\\}_{k\in{\cal
K}},\boldsymbol{R}_{0}^{\text{I}*})$ is required to obtain the optimal
solution to problem (P1.1) (i.e., problem (P1) under the given
$\boldsymbol{\Phi}$), which is stated in the following lemma.

###### Lemma 1

In the case with $\text{rank}(\boldsymbol{W}_{k}^{\text{I}*})>1$ for a certain
$k\in{\cal K}$, the optimal solution
$(\\{\tilde{\boldsymbol{w}}^{\text{I}*}_{k}\\}_{k\in{\cal
K}},\tilde{\boldsymbol{R}}^{\text{I}*}_{0})$ to problem (P1) under the fixed
phase shifting matrix $\boldsymbol{\Phi}$ of the IRS is obtained as

$\displaystyle\tilde{\boldsymbol{w}}^{\text{I}*}_{k}$
$\displaystyle=\frac{1}{\sqrt{\boldsymbol{h}^{H}_{\text{CU},k}\boldsymbol{W}_{k}^{\text{I}*}\boldsymbol{h}_{\text{CU},k}}}\boldsymbol{W}_{k}^{\text{I}*}\boldsymbol{h}_{\text{CU},k},~{}\forall
k\in{\cal K}$ (14a) $\displaystyle\tilde{\boldsymbol{R}}^{\text{I}*}_{0}$
$\displaystyle=\boldsymbol{R}_{0}^{\text{I}*}+\sum_{k=1}^{K}\boldsymbol{W}_{k}^{\text{I}*}-\sum_{k=1}^{K}\tilde{\boldsymbol{w}}^{\text{I}*}_{k}(\tilde{\boldsymbol{w}}^{\text{I}*}_{k})^{H}.$
(14b)

###### Proof:

See Appendix -A. ∎

###  III-B Optimization of IRS’s Phase Shifting Matrix $\boldsymbol{\Phi}$

In this subsection, under the fixed transmit beamforming vectors
$\\{\boldsymbol{w}_{k}\\}_{k\in{\cal K}}$ for the $K$ CUs and covariance
matrix $\boldsymbol{R}_{0}$ for the $L$ targets, we optimize the IRS’s phase
shifting matrix $\boldsymbol{\Phi}$ of problem (P1).

We first define
$\boldsymbol{\phi}\triangleq\text{vec}(\boldsymbol{\Phi})=[e^{j\phi_{1}},\cdots,e^{j\phi_{N}}]^{T}$
as the IRS’s phase shifting vector. By letting
$\boldsymbol{R}\triangleq\boldsymbol{R}_{0}+\sum_{k=1}^{K}\boldsymbol{w}_{k}\boldsymbol{w}_{k}^{H}$,
we introduce the following definitions as

$\displaystyle\boldsymbol{R}^{\text{target}}(\theta_{l})\triangleq\text{diag}(\alpha(\theta_{l}))^{H}\boldsymbol{G}\boldsymbol{R}\boldsymbol{G}^{H}\text{diag}(\alpha(\theta_{l})),~{}\forall
l\in{\cal L}$ (15a)
$\displaystyle\boldsymbol{R}_{k,j}^{\text{CU}}\triangleq\text{diag}(\boldsymbol{h}_{iu,k})^{H}\boldsymbol{G}\boldsymbol{w}_{j}\boldsymbol{w}_{j}^{H}\boldsymbol{G}^{H}\text{diag}(\boldsymbol{h}_{iu,k}),~{}\forall
k,j\in{\cal K}$ (15b)
$\displaystyle\boldsymbol{R}^{\text{clutter}}_{q}\triangleq\text{diag}(\boldsymbol{g}_{ic,q})^{H}\boldsymbol{G}\boldsymbol{R}\boldsymbol{G}^{H}\text{diag}(\boldsymbol{g}_{ic,q}),~{}\forall
q\in{\cal Q}$ (15c)
$\displaystyle\boldsymbol{b}^{\text{CU}}_{k,j}\triangleq\text{diag}(\boldsymbol{h}_{iu,k})^{H}\boldsymbol{G}\boldsymbol{w}_{j}\boldsymbol{w}_{j}^{H}\boldsymbol{h}_{bu,k},~{}\forall
k,j\in{\cal K}$ (15d)
$\displaystyle\boldsymbol{b}^{\text{clutter}}_{q}\triangleq\text{diag}(\boldsymbol{g}_{ic,q})^{H}\boldsymbol{G}\boldsymbol{R}\boldsymbol{g}_{bc,q},~{}\forall
q\in{\cal Q}.$ (15e)

Based on (15), we define the augmented matrices as
$\bar{\boldsymbol{R}}^{\text{target}}(\theta_{l})=[\boldsymbol{R}^{\text{target}}(\theta_{l}),0;0,0]$,
$\forall l\in{\cal L}$,
$\bar{\boldsymbol{R}}_{k,j}^{\text{CU}}=[\boldsymbol{R}_{k,j}^{\text{CU}},\boldsymbol{b}_{k,j}^{\text{CU}};(\boldsymbol{b}_{k,j}^{\text{CU}})^{H},0]$,
$\forall k,j\in{\cal K}$,
$\bar{\boldsymbol{R}}^{\text{clutter}}_{q}=[\boldsymbol{R}^{\text{clutter}}_{q},\boldsymbol{b}_{q}^{\text{clutter}};(\boldsymbol{b}_{q}^{\text{clutter}})^{H},0],\forall
q\in{\cal Q}$.

Then, with the change of variable
$\boldsymbol{\phi}\triangleq\text{vec}(\boldsymbol{\Phi})$ and by fixing
$(\\{\boldsymbol{w}_{k}\\}_{k\in{\cal K}},\boldsymbol{R}_{0})$, problem (P1)
in the case with Type-I CUs is recast as

$\displaystyle(\text{P1.2}):$
$\displaystyle~{}\max_{\bar{\boldsymbol{\phi}}}\min_{l\in\mathcal{L}}~{}\bar{\boldsymbol{\phi}}^{H}\bar{\boldsymbol{R}}^{\text{target}}(\theta_{l})\bar{\boldsymbol{\phi}}$
(16a) s.t.
$\displaystyle~{}\bar{\boldsymbol{\phi}}^{H}\Big{(}\frac{1}{\Gamma_{k}}\bar{\boldsymbol{R}}^{\text{CU}}_{k,k}-\sum_{j=1,j\neq
k}^{K}\bar{\boldsymbol{R}}^{\text{CU}}_{k,j}\Big{)}\bar{\boldsymbol{\phi}}\geq
c_{k}^{\text{I}},~{}\forall k\in\mathcal{K}$ (16b)
$\displaystyle~{}\bar{\boldsymbol{\phi}}^{H}\bar{\boldsymbol{R}}^{\text{clutter}}_{q}\bar{\boldsymbol{\phi}}\leq\eta_{q}-|\boldsymbol{g}^{H}_{bc,q}\boldsymbol{R}\boldsymbol{g}_{bc,q}|,~{}\forall
q\in\cal{Q}$ (16c)
$\displaystyle~{}\frac{2}{L^{2}-L}\sum_{l=1}^{L-1}\sum_{i=l+1}^{L}\bar{\boldsymbol{\phi}}^{H}\bar{\boldsymbol{R}}^{\text{target}}_{l,i}\bar{\boldsymbol{\phi}}\leq\xi$
(16d)
$\displaystyle~{}\boldsymbol{\phi}=[e^{j\phi_{1}},...,e^{j\phi_{N}}]^{T},~{}\phi_{n}\in[0,2\pi],~{}\forall
n\in\\{1,...,N\\},$ (16e)

where $c^{\text{I}}_{k}\triangleq\sum_{j=1,j\neq
k}^{K}|\boldsymbol{h}^{H}_{bu,k}\boldsymbol{w}_{j}|^{2}-\frac{1}{\Gamma_{k}}|\boldsymbol{h}_{bu,k}^{H}\boldsymbol{w}_{k}|^{2}+\sigma_{k}^{2}$,
$\forall k\in{\cal K}$, and
$\bar{\boldsymbol{\phi}}=[\boldsymbol{\phi}^{H},1]^{H}$ is the design variable
vector. Note that problem (P1.2) is non-convex, due to the non-convexity of
both the SINR constraints (16b) and the unit-modulus constraints of (16e). To
address this issue, we define
$\boldsymbol{V}\triangleq\bar{\boldsymbol{\phi}}\bar{\boldsymbol{\phi}}^{H}$.
By removing the rank-one constraint $\text{rank}(\boldsymbol{V})=1$, we relax
problem (P1.2) into the following SDP [41]:

$\displaystyle(\text{P1.3}):$
$\displaystyle~{}\max_{\boldsymbol{V}}~{}\min_{l\in\mathcal{L}}~{}\text{tr}(\bar{\boldsymbol{R}}^{\text{target}}(\theta_{l})\boldsymbol{V})$
(17a) s.t.
$\displaystyle\text{tr}\Big{(}\Big{(}\frac{1}{\Gamma_{k}}\bar{\boldsymbol{R}}^{\text{CU}}_{k,k}-\sum_{j=1,j\neq
k}^{K}\bar{\boldsymbol{R}}^{\text{CU}}_{k,j}\Big{)}{\boldsymbol{V}}\Big{)}\geq
c^{\text{I}}_{k},~{}\forall k\in\mathcal{K}$ (17b)
$\displaystyle\text{tr}\Big{(}\bar{\boldsymbol{R}}^{\text{clutter}}_{q}\boldsymbol{V}\Big{)}\leq\eta_{q}-|\boldsymbol{g}^{H}_{bc,q}\boldsymbol{R}\boldsymbol{g}_{bc,q}|,~{}\forall
q\in\cal{Q}$ (17c)
$\displaystyle\frac{2}{L^{2}-L}\sum_{l=1}^{L-1}\sum_{i=l+1}^{L}\text{tr}\Big{(}\bar{\boldsymbol{R}}^{\text{target}}_{l,i}{\boldsymbol{V}}\Big{)}\leq\xi$
(17d)
$\displaystyle\boldsymbol{V}\succeq\boldsymbol{0},~{}\boldsymbol{V}_{n,n}=1,~{}\forall
n\in\\{1,...,N+1\\},$ (17e)

which can be efficiently solved via standard convex solvers. Let
$\boldsymbol{V}^{*}$ denote the optimal solution to problem (P1.3). Note that
the solution $\boldsymbol{V}^{*}$ is not always guaranteed to be of rank-one.
In the case with $\text{rank}(\boldsymbol{V}^{*})>1$, we employ a Gaussian
randomization procedure[42] to generate a feasible rank-one matrix. We
generate Gaussian randomization vectors
$\boldsymbol{r}\sim\mathcal{CN}(0,\boldsymbol{V}^{*})$ and we construct the
candidate approximate solution to problem (P1.2) as
$\boldsymbol{v}=e^{j\arg([\frac{\boldsymbol{r}}{\boldsymbol{r}_{N+1}}]_{(1:N)})}$,
the objective value is approximated as the maximum one among all these random
realizations. Thus, the randomization realization number is set to be 5000 to
ensure that there are enough times to guarantee that the objective value is
non-decreasing over iterations.

Algorithm 1 Proposed alternating-optimization algorithm for solving (P1)

1: Initialize the phase shifting matrix $\Phi^{(1)}$ and the iteration index
$i=1$;

2: repeat

3: For given $\boldsymbol{\Phi}^{(i)}$, obtain the optimal solution
$\boldsymbol{W}_{k}^{(i)}$ and $\boldsymbol{R}_{0}^{(i)}$ to problem (P1.1)
based on Proposition 1;

4: For given $\boldsymbol{W}_{k}^{(i+1)}$ and $\boldsymbol{R}_{0}^{(i+1)}$,
solve problem (P1.3), and then obtain an approximate rank-one solution
$\boldsymbol{\Phi}^{(i+1)}$ based on the Gaussian randomization method;

5: Set $i=i+1$;

6: until The objective value between two iterations is smaller than a
tolerance level $\epsilon$ or the maximum number of iterations is reached.

###  III-C Proposed Algorithm and Complexity Analysis

In Algorithm 1, we summarize the proposed alternating-optimization-based
algorithm for solving the max-min IRS-assisted ISAC design problem (P1) in the
case with Type-I CUs. Note that the achieved values of problem (P1.1) and
(P1.3) are non-decreasing over the alternating-optimization-based iterations.
The alternation process is repeated continuously until the improvement in the
objective value between the two iterations is less than the tolerance level
$\epsilon$, and the convergence of the proposed Algorithm 1 is guaranteed.

The computational complexity of the proposed Algorithm 1 is calculated as
follows. Denote by $J$ and $I$ the iteration number and the SDR randomization
trial number, respectively. With a given tolerable error level $\epsilon$, we
solve the max-min problem with the interior-point method [43] is
$\mathcal{O}(J(K^{4.5}N^{4.5})\log(1/\epsilon))$. For each iteration of the
Gaussian randomization procedure, it has a computational complexity of
$\mathcal{O}(JI(NK+1)^{2})$. As a result, the computational complexity of the
proposed Algorithm 1 is
$\mathcal{O}(J(K^{4.5}N^{4.5})\log(1/\epsilon))+\mathcal{O}(JI(NK+1)^{2})$.

##  IV Proposed Solution For (P2) with Type-II CUs

In this section, we propose the alternating-optimization-based scheme for
problem (P2) in the case with Type-II CUs.

###  IV-A Optimization of BS’s Transmission Design
$(\\{\boldsymbol{w}_{k}\\}_{k\in{\cal K}},\boldsymbol{R}_{0})$ for ISAC

In this subsection, we obtain the BS’s beamforming vectors
$\\{\boldsymbol{w}_{k}\\}_{k\in{\cal K}}$ for the $K$ Type-II CUs and
covariance matrix $\boldsymbol{R}_{0}$ for the $L$ targets.

By introducing
$\boldsymbol{W}_{k}=\boldsymbol{w}_{k}\boldsymbol{w}_{k}^{H}\succeq\boldsymbol{0}$,
$\forall k\in{\cal K}$, problem (P2) under the fixed IRS’s phase shifting
matrix $\boldsymbol{\Phi}$ is expressed as

$\displaystyle(\text{P2.1}):$
$\displaystyle\max_{\\{\boldsymbol{W}_{k}\\}_{k\in{\cal
K}},\boldsymbol{R}_{0}\succeq
0}~{}\min_{l\in\mathcal{L}}~{}\boldsymbol{h}_{\text{target},l}^{H}\Big{(}\boldsymbol{R}_{0}+\sum_{k=1}^{K}\boldsymbol{W}_{k}\Big{)}\boldsymbol{h}_{\text{target},l}$
(18a) s.t.
$\displaystyle~{}\frac{1}{\Gamma_{k}}\text{tr}(\boldsymbol{h}_{\text{CU},k}\boldsymbol{h}_{\text{CU},k}^{H}\boldsymbol{W}_{k})-\sum_{j=1,j\neq
k}^{K}\text{tr}(\boldsymbol{h}_{\text{CU},k}\boldsymbol{h}_{\text{CU},k}^{H}(\boldsymbol{W}_{j}+\boldsymbol{R}_{0}))-\sigma_{k}^{2}\geq
0,~{}\forall k\in\mathcal{K}$ (18b)
$\displaystyle(\ref{eq.p1-1}b),(\ref{eq.p1-1}d),(\ref{eq.p1-1}e),\text{and}~{}(\ref{eq.p1-1}f),$

where $\\{\boldsymbol{W}_{k}\\}_{k\in{\cal K}}$ and $\boldsymbol{R}_{0}$ are
the optimization variables. Due to the rank-one constraints
$\text{rank}(\boldsymbol{W}_{k})\leq 1$, $\forall k\in{\cal K}$, in (13f),
problem (P2.1) is non-convex and difficult to solve. As in the case with
Type-I CUs, we employ the SDR method to solve problem (P2.1) by removing the
rank-one constraints.

Let $(\\{\boldsymbol{W}^{\text{II}*}_{k}\\}_{k\in{\cal
K}},\boldsymbol{R}^{\text{II}*}_{0})$ denote the optimal solution to the
relaxed problem (P2.1). If it holds that
$\text{rank}(\boldsymbol{W}^{\text{II}*}_{k})=1$, $\forall k\in{\cal K}$, then
$(\\{\boldsymbol{W}^{\text{II*}}_{k}\\}_{k\in{\cal
K}},\boldsymbol{R}^{\text{II}*}_{0})$ is the optimal solution to problem
(P2.1). In this case, by performing the EVD for each
$\boldsymbol{W}^{\text{II}*}_{k}$, we have
$\boldsymbol{W}^{\text{II}*}_{k}=\lambda_{k}\boldsymbol{v}_{k}\boldsymbol{v}^{H}_{k}$,
$\forall k\in{\cal K}$, where $\lambda_{k}>0$ is the non-zero eigenvalue of
$\boldsymbol{W}^{\text{II}*}_{k}$ and $\boldsymbol{v}$ is the unit-norm
eigenvector associated with the eigenvalue $\lambda_{k}$. We now obtain the
optimal solution $(\\{\sqrt{\lambda_{k}}\boldsymbol{v}_{k}\\}_{k\in{\cal
K}},\boldsymbol{R}^{\text{II}*}_{0})$ to the original problem (P2) under the
given $\boldsymbol{\Phi}$. On the other hand, if there exists
$\text{rank}(\boldsymbol{W}^{\text{II}*}_{k})>1$ for a certain $k\in{\cal K}$,
then we employ the rank-one solution construction procedure as in Lemma 1. In
particular, the obtained solution to problem (P2) under the given
$\boldsymbol{\Phi}$ is expressed as
$(\\{\tilde{\boldsymbol{w}}^{\text{II}*}_{k}\\}_{k\in{\cal
K}},\tilde{\boldsymbol{R}}_{0}^{\text{II}*})$, where
$\tilde{\boldsymbol{w}}^{\text{II}*}_{k}=\frac{1}{\boldsymbol{h}^{H}_{\text{CU},k}\boldsymbol{W}^{\text{II}*}_{k}\boldsymbol{h}^{H}_{\text{CU},k}}\boldsymbol{W}_{k}^{\text{II}*}\boldsymbol{h}_{\text{CU},k}$,
$\forall k\in{\cal K}$, and
$\tilde{\boldsymbol{R}}_{0}^{\text{II}*}=\boldsymbol{R}^{\text{II}*}_{0}+\sum_{k=1}^{K}\boldsymbol{W}_{k}^{\text{II}*}-\sum_{k=1}^{K}\tilde{\boldsymbol{w}}^{\text{II}*}_{k}(\tilde{\boldsymbol{w}}^{\text{II}*}_{k})^{H}$.

Note that the optimal solution
$(\\{\boldsymbol{W}^{\text{II}*}_{k}\\}_{k\in{\cal
K}},\boldsymbol{R}^{\text{II}*}_{0})$ to problem (P2.1) is not unique.
Interestingly, it is shown that there exists an optimal solution to problem
(P2.1) with $\boldsymbol{R}^{\text{II*}}_{0}=\boldsymbol{0}$, as stated in the
following proposition.

###### Proposition 1

For problem (P2.1), there always exists an optimal solution
$(\\{\boldsymbol{W}^{\text{II}*}_{k}\\}_{k\in{\cal K}},\boldsymbol{0})$, i.e.,
$\boldsymbol{R}^{\text{II}*}_{0}=\boldsymbol{0}$, which implies that the
dedicated sensing signal $x_{0}$ can be removed at the BS without loss of
optimality of problem (P2.1) in the case with Type-II CUs.

###### Proof:

See Appendix -B. ∎

Note that the feasible region of (P2.1) is included by that of (P1.1). As
expected, the beampattern gain achieved by (P1.1) is guaranteed to be greater
than that by (P2.1). This is because the dedicated radar signals cannot affect
the communication performance of the Type-I CUs, but it is not true for the
ISAC system with Type-II CUs. The ISAC system with Type-I CUs is shown to
achieve better or equal beampattern gains than the counterpart with Type-II
CUs.

###  IV-B Optimization of IRS’s Phase Shifting Matrix $\boldsymbol{\Phi}$

In this subsection, we obtain the optimal phase shifting matrix
$\boldsymbol{\Phi}$ of the IRS for problem (P2) under the given
$(\\{\boldsymbol{w}_{k}\\}_{k\in{\cal K}},\boldsymbol{R}_{0})$. We first
define the following notations as

$\displaystyle\boldsymbol{F}_{k,j}^{\text{CU}}\triangleq\text{diag}(\boldsymbol{h}_{iu,k})^{H}\boldsymbol{G}\boldsymbol{R}_{0}\boldsymbol{G}^{H}\text{diag}(\boldsymbol{h}_{iu,j}),~{}\forall
k,j\in{\cal K}$ (19a)
$\displaystyle\boldsymbol{e}^{\text{CU}}_{k}\triangleq\text{diag}(\boldsymbol{h}_{iu,k})^{H}\boldsymbol{G}\boldsymbol{R}_{0}\boldsymbol{h}_{bu,k},~{}\forall
k\in{\cal K},$ (19b)

Based on (19), we define the augmented matrices as
$\bar{\boldsymbol{F}}_{k,j}^{\text{CU}}=[\boldsymbol{F}_{k,j}^{\text{CU}},\boldsymbol{e}_{k}^{\text{CU}};(\boldsymbol{e}_{k}^{\text{CU}})^{H},0]$,
$\forall k,j\in{\cal K}$.

By introducing
$\boldsymbol{V}\triangleq\bar{\boldsymbol{\phi}}\bar{\boldsymbol{\phi}}^{H}\succeq\boldsymbol{0}$
and $\text{rank}(\boldsymbol{V})\leq 1$, problem (P2) under the given
$(\\{\boldsymbol{w}_{k}\\}_{k\in{\cal K}},\boldsymbol{R}_{0})$ is recast as

$\displaystyle(\text{P2.2}):$
$\displaystyle~{}\max_{\boldsymbol{V}}~{}\min_{l\in\mathcal{L}}~{}\text{tr}(\bar{\boldsymbol{R}}^{\text{target}}(\theta_{l})\boldsymbol{V})$
(20a) s.t.
$\displaystyle\text{tr}\Big{(}\Big{(}\frac{1}{\Gamma_{k}}\bar{\boldsymbol{R}}^{\text{CU}}_{k,k}-\sum_{j=1,j\neq
k}^{K}\bar{\boldsymbol{R}}^{\text{CU}}_{k,j}-\bar{\boldsymbol{F}}^{\text{CU}}_{k}\Big{)}\boldsymbol{V}\Big{)}\geq
c_{k}^{\text{II}},~{}\forall k\in\mathcal{K}$ (20b)
$\displaystyle(16c)-(16e),$

where $c_{k}^{\text{II}}\triangleq\sum_{j=1,j\neq
k}^{K}|\boldsymbol{h}^{H}_{bu,k}\boldsymbol{w}_{j}|^{2}-\frac{1}{\Gamma_{k}}|\boldsymbol{h}_{bu,k}^{H}\boldsymbol{w}_{k}|^{2}+\sigma_{k}^{2}$,
$\forall k\in{\cal K}$, and the matrix $\boldsymbol{V}$ is the design
variable. As in problem (P1.2), problem (P2.2) can be relaxed into a SDP by
employing the SDR technique. In the case with a high-rank solution for the
relaxed problem (P2.2), a Gaussian randomization procedure is adopted to
generate a feasible rank-one solution for problem (P2.2).

###  IV-C Proposed Algorithm for (P2) with Type-II CUs

In Algorithm 2, we summarize the proposed method for solving problem (P2) in
the case with Type-II CUs. Based on the alternating-optimization-based method,
we obtain a low-complexity solution to problem (P2) by alternatively solving
problems (P2.1) and (P2.2) in an iteration manner. Since the objective
function values are non-decreasing for problems (P2.1) and (P2.2), the
convergence of Algorithm 2 is guaranteed. Similarly as Algorithm 1, the
computational complexity of Algorithm 2 is given as
$\mathcal{O}(J(K^{4.5}N^{4.5})\log(1/\epsilon))+\mathcal{O}(JI(NK+1)^{2})$.

Algorithm 2 Proposed alternating-optimization algorithm for (P2)

1: Initialize the phase shifting matrix $\boldsymbol{\Phi}^{(1)}$ and the
iteration index $t=1$;

2: repeat

3: For given $\Phi^{(t)}$, obtain the optimal solution
$\boldsymbol{W}_{k}^{(t)}$ and $\boldsymbol{R}_{0}^{(t)}$ to problem (P2.1);

4: For given $\boldsymbol{W}_{k}^{(t+1)}$ and $\boldsymbol{R}_{0}^{(t+1)}$,
solve the relaxed problem (P2.2), and then obtain a rank-one solution
$\boldsymbol{\Phi}^{(t+1)}$ based on the Gaussian randomization method;

5: Set $t=t+1$;

6: until The objective value between two iterations is smaller than a
tolerance level $\epsilon$ or the maximum number of iterations is reached.

##  V Numerical Results

In this section, numerical results are provided to evaluate the proposed IRS-
assisted ISAC system design scheme in a clutter environment. We consider the
Rician fading channel models for the BS-CU, BS-IRS, BS-Clutter, IRS-CU, and
IRS-Clutter communication links, where the Rician factor is set as 0.5 in the
radar field. The pass-loss model is given by
$K_{0}(\frac{d}{d_{0}})^{-\alpha}$, where $K_{0}=-30$ dB corresponds to the
path-loss at the reference distance of $d_{0}=1$ meter (m), and the path-loss
exponent $\alpha$ are set to be 2.5, 2.5, 2.2, 3.5, and 3.5 for the IRS-
Clutter, IRS-CU, BS-IRS, BS-CU, and BS-Clutter links, respectively. The number
of the BS’s antennas is set to be $M=8$, and the number of the IRS’s passive
reflecting elements is set to be $N=64$. As shown in Fig. 2, the BS and IRS
are set to be located at the coordinates (0 m, 0 m) and (20 m, 2 m),
respectively. The number of CUs is $K=4$, and each CU $k\in{\cal K}$ is
randomly distributed with a distance $d_{k}=10$ m apart from the BS. Each
clutter $q\in{\cal Q}$ is randomly located at a distance $d_{q}\in[12,15]$ m
apart from the IRS. The angles of the $L=5$ targets are given as
$\theta_{1}=-60^{\circ}$, $\theta_{2}=-30^{\circ}$, $\theta_{3}=0^{\circ}$,
$\theta_{4}=30^{\circ}$, and $\theta_{5}=60^{\circ}$. Unless stated otherwise,
the BS’s transmission power budget is set to be $P_{0}=0.5$ Watt (W), the CU
receiver noise power is set to be $\sigma_{k}^{2}=-80$ dBm, and the
interference power threshold of each clutter is set to be
$\eta_{q}=0.1~{}\mu\text{W}$, $\forall q\in{\cal Q}$. Finally, the error
tolerable level is set to be $\epsilon=10^{-6}$ in the proposed Algorithms 1
and 2.

Figure 2: Simulation setup

For performance comparison, we consider the following four benchmark schemes
for IRS-assisted ISAC system designs.

  * •

Information Beamforming Design Scheme: In this scheme, the BS only employs the
information beamforming vectors $\\{\boldsymbol{w}_{k}\\}_{k\in{\cal K}}$ for
both communication and sensing, which is equivalent to solving problem (25) by
setting $\boldsymbol{R}_{0}=0$.

  * •

Separate Design Scheme: In this scheme, the BS’s transmit design
$(\\{\boldsymbol{w}_{k}\\}_{k\in{\cal K}},\boldsymbol{R}_{0})$ and the IRS’s
phase shifting matrix $\boldsymbol{\Phi}$ are optimized separately. In
particular, the phase shifting matrix $\boldsymbol{\Phi}$ at the IRS is first
optimized so as to make the IRS reach the desired angle, and the transmission
design $(\\{\boldsymbol{w}_{k}\\}_{k\in{\cal K}},\boldsymbol{R}_{0})$ at the
BS is then optimized under the the obtained phase shifting matrix
$\boldsymbol{\Phi}$.

  * •

Joint Design with Random IRS Phase Scheme: In this scheme, the IRS’s phase
shifting matrix $\boldsymbol{\Phi}$ is randomized generated, and then
$(\\{\boldsymbol{w}_{k}\\}_{k\in{\cal K}},\boldsymbol{R}_{0})$ is jointly
optimized at the BS.

  * •

Joint Design without IRS Scheme: In this scheme, the IRS design is removed,
i.e., $\boldsymbol{\Phi}=0$, and $(\\{\boldsymbol{w}_{k}\\}_{k\in{\cal
K}},\boldsymbol{R}_{0})$ at the BS becomes the only design variables for ISAC.

###  V-A Convergence Performance

Figure 3: Convergence performance of the proposed Algorithms 1 and 2.

Fig. 3 shows the fast convergence performance of the proposed Algorithms 1 and
2 for solving problems (P1) and (P2), respectively, where $Q=2$, $L=5$,
$\xi=\infty$, and the SINR threshold for each CU $k\in{\cal K}$ is set to be
$\Gamma_{k}=10$ dB. It is observed that Algorithms 1 and 2 both converge to
stationary points within about 17 iterations under different numbers of CUs.
The convergence speed becomes slightly slow as the number $K$ of CUs
increases. As expected, it is observed in Fig. 3 that the minimum sensing
beampattern gain in the case with Type-I CUs is larger than that in the case
with Type-II CUs. In addition, the achieved sensing beampattern gain becomes
smaller as $K$ increases. This is because the max-min based ISAC system design
scheme relies heavily on the CU with the worst channel quality so as to
satisfy the communication and sensing requirements.

###  V-B Achieved Sensing Beampattern Gain Profiles

Figure 4: The sensing beampattern gain profile for different schemes

Fig. 4 depicts the sensing beampattern gain achieved at different angles,
where $L=5$, $K=3$, $Q=2$, and $\xi=\infty$. All three schemes concentrate
energy at the sensing angles of the five targets. The proposed joint design
schemes with different CU Types are observed to significantly outperform the
conventional Capon based scheme [37]. Also, the proposed design in the case
with Type-I CUs outperforms the proposed design in the case with Type-II CUs,
i.e., a lower power profile at the uninterested angles is achieved in the case
with Type-I CUs. This shows the benefits of cancelling the sensing
interference at the CUs for improving the sensing performance.

Figure 5: The achieved sensing beampattern gain profile under different mean-
squared cross-correlation threshold $\xi$ values.

Fig. 5 shows the beampattern gain profile of the proposed scheme in the case
with Type-I CUs under different sensing mean-squared cross-correlation pattern
threshold $\xi$ values, where $K=3$, $\Gamma_{k}=10$ dB, $Q=2$, and $L=5$.
With a smaller threshold $\xi$, a stricter cross-correlation coefficient is
imposed between the illuminating signals towards any two neighboring targets,
which enhances the multi-target detection performance [44]. As the threshold
$\xi$ value increases, the proposed design scheme is observed to achieve
increasingly large sensing beampattern gains at the target angles. This
implies the importance of properly setting a cross-correlation pattern
threshold $\xi$, so as to achieve a good trade-off between the multi-target
detection performance and the multi-target sensing performance [44].

###  V-C Achieved Minimum Sensing Beampattern Gain Evaluation

Figure 6: The achieved minimum beampattern gain versus the SINR threshold
$\Gamma$ of CUs with $K=3$, $Q=2$, and $L=5$.

Fig. 6 shows the achieved minimum beampattern gain versus the SINR $\Gamma$ of
the CUs, where $K=3$, $Q=2$, $L=5$, and $\xi=\infty$. The proposed designs are
observed to achieve a significant performance gain over the four benchmark
schemes. Thanks to the ability of eliminating the sensing signals for Type-I
CUs, the proposed design scheme in the case with Type-I CUs outperforms the
other schemes. In Fig. 6, the proposed scheme in the case with Type-II CUs is
observed to achieve the same performance as the information beamforming design
benchmark scheme, which corroborates Proposition 1. For the proposed design
scheme in the case with Type-I CUs, the minimum beampattern gain remains
unchanged in the low SINR requirement regime (e.g. $\Gamma\leq 10$ dB), but it
is not true in the high SINR requirement regime (e.g. $\Gamma\geq 15$ dB).
This shows the IRS-assisted ISAC system design needs to balance the trade-off
between high communication and sensing performance requirements. In addition,
the separate design benchmark scheme is observed to outperform the benchmark
scheme with random IRS phase matrix. This implies the importance of optimizing
the IRS phase matrix in improving the sensing performance. Finally, due to the
fact that there exist no direct LoS links between the targets and the BS, the
benchmark scheme without IRS is observed to fail to sense the targets.

Figure 7: The achieved minimum beampattern gain versus the clutter number $Q$
with $K=3$ and $\Gamma_{k}=10$ dB.

Fig. 7 shows the achieved minimum sensing beampattern gain versus the clutter
number $Q$, where $K=3$, $\Gamma_{k}=10$ dB, $L=5$, and $\xi=\infty$. It is
observed that the proposed schemes achieve a significant performance gain over
the benchmark schemes. Apart from the benchmark schemes with random IRS or
without the IRS, the minimum sensing beampattern gain decreases for the other
schemes as the clutter number $Q$ increases. This is because a more amount of
energy is needed to counter against the increasing clutter interference value.
All the IRS-assisted benchmark schemes are observed to outperform the scheme
without IRS. This is because the IRS can help create a sensing link to
effectively illuminate the $L$ targets.

Figure 8: The achieved minimum sensing beampattern gain versus the BS’s
transmit power with $K=3$, $Q=2$, $\Gamma=10$ dB and $N=64$.

Fig. 8 shows the minimum sensing beampattern gain versus the maximum transmit
power $P_{0}$ of BS, with $K=3$, $\Gamma_{k}=10$ dB, $Q=2$, $L=5$, and
$\xi=\infty$, respectively. Besides the benchmark scheme without IRS, the
beampattern gain performance of the other schemes increases as the BS’s
maximum transmit power increases. Again, the proposed schemes are observed to
outperform the other benchmark schemes. The proposed joint scheme in the case
with Type-II CUs is also observed to have the same performance as the
information beamforming design benchmark scheme. The design scheme with a
random IRS phase matrix performs inferiorly to the other schemes with an
optimized IRS phase matrix.

Figure 9: The minimum sensing beampattern gain versus the number $N$ of the
IRS passive reflecting elements with $K=3$, $Q=2$, and $\Gamma=10$ dB.

Fig. 9 shows the achieved minimum sensing beampattern gain versus the number
$N$ of the IRS passive reflecting elements, where $K=3$, $\Gamma_{k}=10$ dB,
$Q=2$, $L=5$, and $\xi=\infty$. As expected, the achieved minimum sensing
beampattern gain of all the schemes increases as $N$ increases. This is
because the IRS with a larger number of passive reflecting elements is able to
provide a higher degrees of freedom to reconfigure the signal propagation
environment, thereby improving the system sensing performance. The proposed
scheme in the case with Type-I CUs is observed to achieve a significant
performance gain than the other schemes, and the proposed scheme in the case
with Type-II CUs achieves the same performance as the information beamforming
only design scheme.

##  VI Conclusion

In this paper, we studied an IRS-assisted ISAC system design in a clutter
environment, where the IRS is deployed to both assist the performance of
multiuser communication and multi-target sensing. We consider the Type-I CUs
and Type-II CUs based on the ability of eliminating the sensing interference
for CUs. In the case with Type-I or Type-II CUs, we maximized the minimum
beampattern gain among multiple targets by jointly optimizing the BS’s
transmit beamforming vectors for multiple CUs and covariance matrix for
sensing multiple targets, as well as the IRS’s passive phase shifting matrix.
For the formulated non-convex max-min ISAC design problems in the cases with
Type-I CUs and Type-II CUs, we developed low-complexity algorithms based on
the alternating-optimization and SDR techniques. It is shown that the
dedicated sensing signals for targets are necessary to improve the system
performance in the case with Type-I CUs. By contrast, the dedicated sensing
signals can be removed for the ISAC system designs in the case with Type-II
CUs. Numerical results demonstrated that the proposed IRS-assisted ISAC design
in the case with Type-I CUs outperforms that in the case with Type-II Cus, and
both the proposed joint design schemes achieve a significant performance gain
than the existing benchmark schemes.

###  -A Proof of Lemma 1

First, it follows from (14b) that
$\tilde{\boldsymbol{R}}^{\text{I}*}_{0}+\sum_{k=1}^{K}\tilde{\boldsymbol{w}}^{\text{I}*}_{k}(\tilde{\boldsymbol{w}}^{\text{I}*}_{k})^{H}=\boldsymbol{R}_{0}^{\text{I}*}+\sum_{k=1}^{K}\boldsymbol{W}_{k}^{\text{I}*}$.
Therefore, the solution of
$(\\{\tilde{\boldsymbol{W}}^{\text{I}*}_{k}\\}_{k\in{\cal
K}},\tilde{\boldsymbol{R}}^{\text{I}*}_{0})$ satisfies the constraints (13b),
(13d), and (13e). In addition, the objective function value of problem (P1.1)
achieved by $(\\{\tilde{\boldsymbol{W}}^{\text{I}*}_{k}\\}_{k\in{\cal
K}},\tilde{\boldsymbol{R}}^{\text{I}*}_{0})$ is exactly the same as that
achieved by $(\\{\boldsymbol{W}^{\text{I}*}_{k}\\}_{k\in{\cal
K}},\boldsymbol{R}^{\text{I}*}_{0})$.

Next, we show the solution of $\\{\tilde{\boldsymbol{W}}^{\text{I}*}_{k}\\}$
satisfies the constraint (13c). Based on (14a), we have
$\tilde{\boldsymbol{w}}^{\text{I}*}_{k}=\frac{1}{\sqrt{\boldsymbol{h}^{H}_{\text{CU},k}\boldsymbol{W}_{k}^{\text{I}*}\boldsymbol{h}_{\text{CU},k}}}\boldsymbol{W}_{k}^{\text{I}*}\boldsymbol{h}_{\text{CU},k}$,
and
$\tilde{\boldsymbol{W}}^{\text{I}*}_{k}=\tilde{\boldsymbol{w}}^{\text{I}*}_{k}(\tilde{\boldsymbol{w}}^{\text{I}*}_{k})^{H}$,
$\forall k\in{\cal K}$. For an arbitrary vector
$\boldsymbol{y}\in\mathbb{C}^{M}$, it follows that

$\boldsymbol{y}^{H}\left(\boldsymbol{W}^{\text{I}*}_{k}-\tilde{\boldsymbol{W}}^{\text{I}*}_{k}\right)\boldsymbol{y}=\boldsymbol{y}^{H}\boldsymbol{W}^{\text{I}*}_{k}\boldsymbol{y}-\left(\boldsymbol{h}^{H}_{\text{CU},k}\boldsymbol{W}^{\text{I}*}_{k}\boldsymbol{h}_{\text{CU},k}\right)^{-1}|\boldsymbol{y}_{k}^{H}\boldsymbol{W}^{\text{I}*}_{k}\boldsymbol{h}_{\text{CU},k}|^{2},\forall
k\in\mathcal{K}.$ (21)

In addition, from the Cauchy-Schwarz inequality, it yields that

$\left(\boldsymbol{y}^{H}\boldsymbol{W}^{\text{I}*}_{k}\boldsymbol{y}\right)\left(\boldsymbol{h}^{H}_{\text{CU},k}\boldsymbol{W}^{\text{I}*}_{k}\boldsymbol{h}_{\text{CU},k}\right)\geq\left|\boldsymbol{y}_{k}^{H}\boldsymbol{W}^{\text{I}*}_{k}\boldsymbol{h}_{\text{CU},k}\right|^{2},\forall
k\in\mathcal{K}.$ (22)

Based on (21) and (22), we have

$\boldsymbol{y}^{H}\left(\boldsymbol{W}^{\text{I}*}_{k}-\tilde{\boldsymbol{W}}^{\text{I}*}_{k}\right)\boldsymbol{y}\geq
0\Longleftrightarrow\boldsymbol{W}^{\text{I}*}_{k}-\tilde{\boldsymbol{W}}^{\text{I}*}_{k}\succeq
0,\forall k\in\mathcal{K}.$ (23)

Furthermore, it holds that
$\boldsymbol{h}^{H}_{\text{CU},k}\tilde{\boldsymbol{W}}^{\text{I}*}_{k}\boldsymbol{h}_{\text{CU},k}=\boldsymbol{h}^{H}_{\text{CU},k}\tilde{\boldsymbol{w}}^{\text{I}*}_{k}\tilde{\boldsymbol{w}}^{\text{I}*}_{k}\boldsymbol{h}_{\text{CU},k}=\boldsymbol{h}^{H}_{\text{CU},k}\boldsymbol{W}^{\text{I}*}_{k}\boldsymbol{h}_{\text{CU},k}$,
$\forall k\in\mathcal{K}$. By expressing (13c) as
$(1+\frac{1}{\Gamma_{j}})\boldsymbol{h}^{H}_{\text{CU},j}\boldsymbol{W}_{j}\boldsymbol{h}_{\text{CU},j}-\boldsymbol{h}^{H}_{\text{CU},j}(\sum_{k=1}^{K}\boldsymbol{W}_{k})\boldsymbol{h}_{\text{CU},j}-\sigma_{j}^{2}\geq
0$, $\forall k,j\in\mathcal{K}$, we are ready to have

$\displaystyle\Big{(}1+\frac{1}{\Gamma_{j}}\Big{)}\boldsymbol{h}^{H}_{\text{CU},j}\tilde{\boldsymbol{W}}^{\text{I}*}_{j}\boldsymbol{h}_{\text{CU},j}$
$\displaystyle=\Big{(}1+\frac{1}{\Gamma_{j}}\Big{)}\boldsymbol{h}^{H}_{\text{CU},j}\boldsymbol{W}^{\text{I}*}_{j}\boldsymbol{h}_{\text{CU},j}$
(24a)
$\displaystyle\geq\boldsymbol{h}^{H}_{\text{CU},j}\Big{(}\sum_{k=1}^{K}\boldsymbol{W}^{\text{I}*}_{k}\Big{)}\boldsymbol{h}_{\text{CU},j}+\sigma_{j}^{2}$
(24b)
$\displaystyle\geq\boldsymbol{h}^{H}_{\text{CU},j}\Big{(}\sum_{k=1}^{K}\tilde{\boldsymbol{W}}^{\text{I}*}_{k}\Big{)}\boldsymbol{h}_{\text{CU},j}+\sigma_{j}^{2},~{}\forall
k,j\in{\cal K}.$ (24c)

Therefore, the reconstructed rank-one solution
$(\\{\tilde{\boldsymbol{W}}^{\text{I}*}_{k}\\}_{k\in{\cal
K}},\tilde{\boldsymbol{R}}^{\text{I}*}_{0})$ satisfies the SINR constraints of
Type-I CUs in (13c). Until now, we have proved that the solution
$(\\{\tilde{\boldsymbol{W}}^{\text{I}*}_{k}\\}_{k\in{\cal
K}},\tilde{\boldsymbol{R}}^{\text{I}*}_{0})$ is an optimal solution to problem
(P1.1), which completes the proof of Lemma 1.

###  -B Proof of Proposition 1

First, by setting $\boldsymbol{R}_{0}=\boldsymbol{0}$ in the relaxed problem
(P2.1), we obtain the following optimization problem as

$\displaystyle\max_{\\{\boldsymbol{W}_{k}\succeq\boldsymbol{0}\\}}~{}\min_{l\in\mathcal{L}}~{}$
$\displaystyle~{}\boldsymbol{h}_{\text{target},l}^{H}\Big{(}\sum_{k=1}^{K}\boldsymbol{W}_{k}\Big{)}\boldsymbol{h}_{\text{target},l}$
(25a) s.t. $\displaystyle~{}\sum_{k=1}^{K}\text{tr}(\boldsymbol{W}_{k})\leq
P_{0}$ (25b)
$\displaystyle~{}\frac{1}{\Gamma_{k}}\text{tr}(\boldsymbol{h}_{\text{CU},k}\boldsymbol{h}_{\text{CU},k}^{H}\boldsymbol{W}_{k})-\sum_{j=1,j\neq
k}^{K}\text{tr}(\boldsymbol{h}_{\text{CU},k}\boldsymbol{h}_{\text{CU},k}^{H}\boldsymbol{W}_{j})-\sigma_{k}^{2}\geq
0,~{}\forall k\in\mathcal{K}$ (25c)
$\displaystyle~{}\boldsymbol{g}_{\text{clutter},q}^{H}\Big{(}\sum_{k=1}^{K}\boldsymbol{W}_{k}\Big{)}\boldsymbol{g}_{\text{clutter},q}\leq\eta_{q},~{}\forall
q\in\cal{Q}$ (25d)
$\displaystyle~{}\frac{2}{L^{2}-L}\sum_{l=1}^{L-1}\sum_{i=l+1}^{L}\boldsymbol{h}_{\text{target},l}^{H}\Big{(}\sum_{k=1}^{K}\boldsymbol{W}_{k}\Big{)}\boldsymbol{h}_{\text{target},i}^{H}\leq\xi$
(25e)

Denote by $(\\{\boldsymbol{W}^{*}_{k}\\}_{k\in{\cal K}})$ the optimal solution
to problem (25). In particular, it holds that
$\boldsymbol{W}^{*}_{k}=\boldsymbol{W}^{\text{II*}}_{k}+\beta_{k}\boldsymbol{R}^{\text{II*}}_{0}$,
$\forall k\in{\cal K}$, where $\sum_{k=1}^{K}\beta_{k}=1$ and $\beta_{k}\geq
0$, $\forall k\in{\cal K}$.

Next, we prove that $(\\{\boldsymbol{W}^{*}_{k}\\}_{k\in{\cal
K}},\boldsymbol{0})$ is an optimal solution to problem (P2.1). For each Type-
II CU $j\in{\cal K}$, it follows that

$\displaystyle\Big{(}1+\frac{1}{\Gamma_{j}}\Big{)}\boldsymbol{h}^{H}_{\text{CU},j}(\boldsymbol{W}^{*}_{j}+\boldsymbol{0})\boldsymbol{h}_{\text{CU},j}$
$\displaystyle=\Big{(}1+\frac{1}{\Gamma_{j}}\Big{)}\boldsymbol{h}^{H}_{\text{CU},j}(\boldsymbol{W}^{\text{II*}}_{j}+\beta_{k}\boldsymbol{R}^{\text{II}*}_{0})\boldsymbol{h}_{\text{CU},j}$
(26a)
$\displaystyle\geq\Big{(}1+\frac{1}{\Gamma_{j}}\Big{)}\boldsymbol{h}^{H}_{\text{CU},j}(\boldsymbol{W}^{\text{II*}}_{j})\boldsymbol{h}_{\text{CU},j}$
(26b)
$\displaystyle\geq\boldsymbol{h}^{H}_{\text{CU},j}\Big{(}\sum_{k=1}^{K}\boldsymbol{W}^{\text{II*}}_{k}+\boldsymbol{R}^{\text{II}*}_{0}\Big{)}\boldsymbol{h}_{\text{CU},j}+\sigma_{j}^{2}$
(26c)
$\displaystyle=\boldsymbol{h}^{H}_{\text{CU},j}\Big{(}\sum_{k=1}^{K}\boldsymbol{W}^{*}_{k}+\boldsymbol{0}\Big{)}\boldsymbol{h}_{\text{CU},j}+\sigma_{j}^{2},~{}\forall
k,j\in{\cal K},$ (26d)

where the first inequality holds from the fact
$\boldsymbol{R}^{\text{II}*}_{0}\succeq 0$, and the second inequality holds
from that the SINR constraint for Type-II CU $j$. Therefore,
$(\\{\boldsymbol{W}^{*}_{k}\\}_{k\in{\cal K}},\boldsymbol{0})$ is shown to be
a feasible solution to problem (P2.1).

Finally, it is readily to show that both
$(\\{\boldsymbol{W}^{*}_{k}\\}_{k\in{\cal K}},\boldsymbol{0})$ and
$(\boldsymbol{W}^{\text{II*}}_{k},\boldsymbol{R}^{\text{II*}}_{0})$ can
achieve the same objective function value for problem (P2.1). Now, we have
proved that $(\\{\boldsymbol{W}^{*}_{k}\\}_{k\in{\cal K}},\boldsymbol{0})$ is
an optimal solution to problem (P2.1), and we complete the proof of
Proposition 1.

## References

  * [1] K. B. Letaief, W. Chen, Y. Shi, J. Zhang, and Y.-J. A. Zhang, “The roadmap to 6g: Ai empowered wireless networks,” _IEEE communications magazine_ , vol. 57, no. 8, pp. 84–90, 2019.
  * [2] L. Zheng, M. Lops, Y. C. Eldar, and X. Wang, “Radar and communication coexistence: An overview: A review of recent methods,” _IEEE Signal Processing Magazine_ , vol. 36, no. 5, pp. 85–99, 2019.
  * [3] F. Liu, C. Masouros, A. Li, T. Ratnarajah, and J. Zhou, “Mimo radar and cellular coexistence: A power-efficient approach enabled by interference exploitation,” _IEEE Transactions on Signal Processing_ , vol. 66, no. 14, pp. 3681–3695, 2018.
  * [4] F. Wang, J. Xu, X. Wang, and S. Cui, “Joint offloading and computing optimization in wireless powered mobile-edge computing systems,” _IEEE Transactions on Wireless Communications_ , vol. 17, no. 3, pp. 1784–1797, 2018\.
  * [5] F. Wang and V. K. N. Lau, “Multi-level over-the-air aggregation of mobile edge computing over d2d wireless networks,” _IEEE Transactions on Wireless Communications_ , pp. 1–1, 2022.
  * [6] X. Liu, T. Huang, N. Shlezinger, Y. Liu, J. Zhou, and Y. C. Eldar, “Joint transmit beamforming for multiuser mimo communications and mimo radar,” _IEEE Transactions on Signal Processing_ , vol. 68, pp. 3929–3944, 2020.
  * [7] Y. Cui, F. Liu, X. Jing, and J. Mu, “Integrating sensing and communications for ubiquitous iot: Applications, trends, and challenges,” _IEEE Network_ , vol. 35, no. 5, pp. 158–167, 2021.
  * [8] H. Hua, J. Xu, and T. X. Han, “Optimal transmit beamforming for integrated sensing and communication,” _arXiv preprint arXiv:2104.11871_ , 2021.
  * [9] F. Dong, W. Wang, Z. Hu, and T. Hui, “Low-complexity beamformer design for joint radar and communications systems,” _IEEE Communications Letters_ , vol. 25, no. 1, pp. 259–263, 2021.
  * [10] F. Liu, Y.-F. Liu, A. Li, C. Masouros, and Y. C. Eldar, “Cramér-rao bound optimization for joint radar-communication beamforming,” _IEEE Transactions on Signal Processing_ , vol. 70, pp. 240–253, 2021.
  * [11] Z. Ren, L. Qiu, and J. Xu, “Optimal transmit beamforming for secrecy integrated sensing and communication,” _arXiv preprint arXiv:2110.12945_ , 2021.
  * [12] L. Chen, Z. Wang, Y. Du, Y. Chen, and F. R. Yu, “Generalized transceiver beamforming for dfrc with mimo radar and mu-mimo communication,” _IEEE Journal on Selected Areas in Communications_ , vol. 40, no. 6, pp. 1795–1808, 2022\.
  * [13] N. Zhao, Y. Wang, Z. Zhang, Q. Chang, and Y. Shen, “Joint transmit and receive beamforming design for integrated sensing and communication,” _IEEE Communications Letters_ , vol. 26, no. 3, pp. 662–666, 2022.
  * [14] G. Sun, M. Li, J. Tong, and Y. Ji, “Structured clutter covariance matrix estimation for airborne mimo radar with limited training data,” _IEEE Geoscience and Remote Sensing Letters_ , vol. 19, pp. 1–5, 2020.
  * [15] F. Liu, C. Masouros, A. Li, H. Sun, and L. Hanzo, “Mu-mimo communications with mimo radar: From co-existence to joint transmission,” _IEEE Transactions on Wireless Communications_ , vol. 17, no. 4, pp. 2755–2770, 2018\.
  * [16] Z. Wang, X. Mu, Y. Liu, and Z. Ding, “Noma inspired integrated sensing and communication,” _arXiv preprint arXiv:2201.04547_ , 2022.
  * [17] F. Liu, Y. Cui, C. Masouros, J. Xu, T. X. Han, Y. C. Eldar, and S. Buzzi, “Integrated sensing and communications: Toward dual-functional wireless networks for 6g and beyond,” _IEEE Journal on Selected Areas in Communications_ , vol. 40, no. 6, pp. 1728–1767, 2022.
  * [18] Q. Wu and R. Zhang, “Intelligent reflecting surface enhanced wireless network via joint active and passive beamforming,” _IEEE Transactions on Wireless Communications_ , vol. 18, no. 11, pp. 5394–5409, 2019.
  * [19] K.-W. Huang and H.-M. Wang, “Passive beamforming for irs aided wireless networks,” _IEEE Wireless Communications Letters_ , vol. 9, no. 12, pp. 2035–2039, 2020.
  * [20] M. Hua, Q. Wu, D. W. K. Ng, J. Zhao, and L. Yang, “Intelligent reflecting surface-aided joint processing coordinated multipoint transmission,” _IEEE Transactions on Communications_ , vol. 69, no. 3, pp. 1650–1665, 2020\.
  * [21] X. Wang, Z. Fei, Z. Zheng, and J. Guo, “Joint waveform design and passive beamforming for ris-assisted dual-functional radar-communication system,” _IEEE Transactions on Vehicular Technology_ , vol. 70, no. 5, pp. 5131–5136, 2021.
  * [22] Z. Zhu, Z. Li, Z. Chu, G. Sun, W. Hao, P. Xiao, and I. Lee, “Intelligent reflecting surface assisted integrated sensing and communications for mmwave channels,” _arXiv preprint arXiv:2202.00552_ , 2022.
  * [23] X. Wang, Z. Fei, J. Huang, and H. Yu, “Joint waveform and discrete phase shift design for ris-assisted integrated sensing and communication system under cramer-rao bound constraint,” _IEEE Transactions on Vehicular Technology_ , vol. 71, no. 1, pp. 1004–1009, 2022.
  * [24] S. Buzzi, E. Grossi, M. Lops, and L. Venturino, “Foundations of mimo radar detection aided by reconfigurable intelligent surfaces,” _arXiv preprint arXiv:2105.09250_ , 2021.
  * [25] X. Song, D. Zhao, H. Hua, T. X. Han, X. Yang, and J. Xu, “Joint transmit and reflective beamforming for irs-assisted integrated sensing and communication,” in _2022 IEEE Wireless Communications and Networking Conference (WCNC)_. IEEE, 2022, pp. 189–194.
  * [26] R. Sankar and S. P. Chepuri, “Beamforming in hybrid ris assisted integrated sensing and communication systems,” _arXiv preprint arXiv:2203.05902_ , 2022\.
  * [27] Y. He, Y. Cai, H. Mao, and G. Yu, “Ris-assisted communication radar coexistence: Joint beamforming design and analysis,” _IEEE Journal on Selected Areas in Communications_ , 2022.
  * [28] A. Bose, S. Khobahi, and M. Soltanalian, “Efficient waveform covariance matrix design and antenna selection for mimo radar,” _Signal Processing_ , vol. 183, p. 107985, 2021.
  * [29] M. Hua, Q. Wu, C. He, S. Ma, and W. Chen, “Joint active and passive beamforming design for irs-aided radar-communication,” _arXiv preprint arXiv:2203.14532_ , 2022.
  * [30] F. Wang, H. Li, and J. Fang, “Joint active and passive beamforming for irs-assisted radar,” _IEEE Signal Processing Letters_ , vol. 29, pp. 349–353, 2022.
  * [31] R. Liu, M. Li, Q. Liu, and A. L. Swindlehurst, “Joint waveform and filter designs for stap-slp-based mimo-dfrc systems,” _IEEE Journal on Selected Areas in Communications_ , vol. 40, no. 6, pp. 1918–1931, 2022.
  * [32] N. Su, Z. Wei, and C. Masouros, “Secure dual-functional radar-communication system via exploiting known interference in the presence of clutter,” in _2021 IEEE 22nd International Workshop on Signal Processing Advances in Wireless Communications (SPAWC)_. IEEE, 2021, pp. 451–455.
  * [33] F. Wang, H. Li, and B. Himed, “Joint optimization of spectrally co-existing multi-carrier radar and communication systems in cluttered environments,” in _ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2021, pp. 8453–8457.
  * [34] C.-Y. Chen and P. Vaidyanathan, “Mimo radar waveform optimization with prior information of the extended target and clutter,” _IEEE Transactions on Signal Processing_ , vol. 57, no. 9, pp. 3533–3544, 2009.
  * [35] F. Wang and H. Li, “Power allocation for coexisting multicarrier radar and communication systems in cluttered environments,” _IEEE Transactions on Signal Processing_ , vol. 69, pp. 1603–1613, 2021.
  * [36] R. Liu, M. Li, Y. Liu, Q. Wu, and Q. Liu, “Joint transmit waveform and passive beamforming design for ris-aided dfrc systems,” _IEEE Journal of Selected Topics in Signal Processing_ , pp. 1–1, 2022.
  * [37] P. Stoica, J. Li, and Y. Xie, “On probing signal design for mimo radar,” _IEEE Transactions on Signal Processing_ , vol. 55, no. 8, pp. 4151–4161, 2007.
  * [38] D. R. Fuhrmann and G. San Antonio, “Transmit beamforming for mimo radar systems using signal cross-correlation,” _IEEE Transactions on Aerospace and Electronic Systems_ , vol. 44, no. 1, pp. 171–186, 2008.
  * [39] Z.-Q. Luo, W.-K. Ma, A. M.-C. So, Y. Ye, and S. Zhang, “Semidefinite relaxation of quadratic optimization problems,” _IEEE Signal Processing Magazine_ , vol. 27, no. 3, pp. 20–34, 2010.
  * [40] M. Grant and S. Boyd, “CVX: Matlab software for disciplined convex programming, version 2.1,” http://cvxr.com/cvx, Mar. 2014.
  * [41] F. Wang, C. Xu, Y. Huang, X. Wang, and X. Gao, “Reel-bf design: Achieving the sdp bound for downlink beamforming with arbitrary shaping constraints,” _IEEE Transactions on Signal Processing_ , vol. 65, no. 10, pp. 2672–2685, 2017.
  * [42] M. C. So, J. Zhang, and Y. Ye, “On approximating complex quadratic optimization problems via semidefinite programming relaxations,” _Mathematical Programming_ , vol. 110, no. 1, pp. 93–110, 2007.
  * [43] Z. Q. Luo, W. K. Ma, M. C. So, Y. Ye, and S. Zhang, “Semidefinite relaxation of quadratic optimization problems,” _IEEE Signal Processing Magazine_ , vol. 27, no. 3, pp. 20–34, 2010.
  * [44] L. Xu, J. Li, and P. Stoica, “Radar imaging via adaptive mimo techniques,” in _2006 14th European Signal Processing Conference_. IEEE, 2006, pp. 1–5.