# Adaptive Radar Detection and Classification Algorithms for Multiple Coherent
Signals

Sudan Han, Linjie Yan, Yuxuan Zhang, Pia Addabbo, ,
Chengpeng Hao, , and Danilo Orlando Sudan Han is with the National Innovation
Institute of Defense Technology, Beijing, China. E-mail:
<EMAIL_ADDRESS>Addabbo is with Università degli studi “Giustino
Fortunato”, Benevento, Italy. E-mail<EMAIL_ADDRESS>Yan,
Yuxuan Zhang, and Chengpeng Hao are with Institute of Acoustics, Chinese
Academy of Sciences, Beijing, China. E-mail<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Orlando is with
the Engineering Faculty of Università degli Studi “Niccolò Cusano”, via Don
Carlo Gnocchi 3, 00166 Roma, Italy. E-mail<EMAIL_ADDRESS>
###### Abstract

In this paper, we address the problem of target detection in the presence of
coherent (or fully correlated) signals, which can be due to multipath
propagation effects or electronic attacks by smart jammers. To this end, we
formulate the problem at hand as a multiple-hypothesis test that, besides the
conventional radar alternative hypothesis, contains additional hypotheses
accounting for the presence of an unknown number of interfering signals. In
this context and leveraging the classification capabilities of the Model Order
Selection rules, we devise penalized likelihood-ratio-based detection
architectures that can establish, as a byproduct, which hypothesis is in
force. Moreover, we propose a suboptimum procedure to estimate the angles of
arrival of multiple coherent signals ensuring (at least for the considered
parameters) almost the same performance as the exhaustive search. Finally, the
performance assessment, conducted over simulated data and in comparison with
conventional radar detectors, highlights that the proposed architectures can
provide satisfactory performance in terms of probability of detection and
correct classification.

###### Index Terms:

Adaptive Radar Detection, Classification, Electronic Counter-CounterMeasures,
Fully Coherent Signals, Generalized Likelihood Ratio Test, Model Order
Selection Rules, Multipath, Radar, Smart Jammer.

##  I Introduction

In recent years, radar systems have become ubiquitous in real life due to the
advances in digital architectures and miniaturization technologies. More
importantly, the huge amount of computational power has paved the way for
sophisticated processing algorithms fed by digital samples and leading to new
architectures where the presence of analog hardware resources devoted to a
specific task is very limited [1, 2, 3]. As a consequence, modern radar
systems are extremely flexible and can incorporate different functions without
additional hardware components.

In system design, the turning point is represented by the knocking down of
high-frequency sampler and processing board costs that have allowed the
development of fully-digital architectures. In this evolving scenario, radar
research community continues to devise algorithms with increased complexity
that take full advantage of the potential provided by digital architectures.
The quid pro quo of the increased computational power is represented by the
enhanced performance as corroborated, for instance, by the space and/or time
adaptive algorithms developed in the open literature. As a matter of fact,
focusing on the radar detection task and starting from the seminal works by
Kelly [4, 5], a plethora of decision schemes have been proposed by enriching
the design assumptions in order to account for a priori information and/or
specific aspects of the application/system under consideration [6, 4, 7, 8, 9,
10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20].

It is also important to underline that the benefits coming from the
aforementioned technology advances have been also exploited by the Electronic
Warfare (EW) systems, which have adapted themselves to the more and more
reliable capabilities of radar systems leading to a more effective class of
electronic countermeasures referred to as smart jammers [21, 3, 22, 23, 24,
25]. For instance, modern noise-like jamming systems are capable of
transmitting narrow-band interfering signals which are concurrent with the
radar pulses according to the radar pulse repetition interval after having
estimated it (EW integrated systems). Moreover, they can use an intrapulse
modulation to generate a noise bandwidth at radio frequency, while maintaining
phase coherence over a group of successive jamming pulses [25]. Remarkably,
this kind of jammers can generate signals that are coherent (i.e., fully
correlated) with the desired signal even though the former impinge on the
radar from different directions [26, 27]. An analogous situation occurs in
scenarios where, due to multipath propagation, replicas of the original signal
come back to the receiver with a sufficiently small delay difference [27, 28].

The main drawback caused by the presence of multiple coherent signals is that
they can completely destroy the performance of the most common high-resolution
direction finding algorithms for adaptive array systems [29, 30]. As a matter
of fact, coherent signals appear as a single signal impinging on the array of
sensors and that, more importantly, arrives from a direction which is quite
different from that of the sought signals. As a consequence, high-resolution
eigenstructure-based techniques as, for instance, MUltiple SIgnal
Classification algorithm [29], fail to correctly resolve the signals
jeopardizing the Angle of Arrival (AoA) estimation [26]. Similar remarks also
hold for another important approach to AoA estimation, namely the CLEAN
algorithm [31] which consists in iterative cancellations of strong signals
under the assumption that the spatial covariance matrix results from the sum
of contributions associated with uncorrelated sources. Remarkably, it can be
used for the detection of weak targets embedded in strong interference signals
[32, 33, 34].

In order to mitigate the effects of coherent signals, data feeding the
direction finding algorithm can be suitably preprocessed in order to decrease
the correlation of the impinging signals. In this respect, spatial smoothing
technique [30] combines data obtained from synthetic subarrays whose size is
lower than that of the original array with the drawback of a reduced angular
resolution. In the context of subspace-based algorithms, in [35] a method
based upon the weighted subspace fitting criterion is proposed to jointly
detect and estimate the number and the related parameters of the coherent
signals. However, this method does not allow for the control of the
probability of false alarm ($P_{fa}$) which is of primary concern in radar.
Alternative approaches against the coherent signals problem can rely on the
Maximum Likelihood (ML) estimation [36] which, however, is computationally
intensive and requires the knowledge of the number of coherent signals, or the
compressed sensing paradigm where a further stage for estimate fusion is
required [37, 38].

Thus, in order to save computational resources (due to the activation of
additional processing stages also in situations where they are not required)
and to take advantage of the full angular resolution of the system, a
detection stage capable of deciding for the presence of a target and possible
coherent signals by estimating, as a byproduct, their number along with other
side information is highly desirable. In fact, when this stage declares the
presence of noncoherent signals, direction finding algorithms can be applied
without losses in resolution. On the other hand, when coherent signals are
present, the rough estimates of the corresponding parameters provided by this
stage can be used to drive ML-based direction finding algorithms to reduce
their computational load. Remarkably, such stage can be viewed as an
Electronic Counter-CounterMeasure.

With the above remarks in mind, in this paper we devise a detection
architecture accounting for the presence of coherent signals and that can
provide an estimate of their number as well as the AoAs. To this end, at the
design stage, we formulate the problem at hand in terms of a multiple
hypothesis test, comprising the usual null (or interference-only) hypothesis,
the conventional signal-plus-interference hypothesis, and multiple alternative
hypotheses that differ in the number of coherent echoes. These signals are
assumed to follow Swerling II Target Model, which assumes that the Radar Cross
Section (RCS) of the target obeys the chi-squared distribution with two
degrees of freedom [1]. Then, assuming an upper bound on the number of
impinging signals (that can be dictated by system parameters), we conceive
likelihood-ratio-based decision rules which exploit suitable penalty terms
borrowed from the Model Order Selection (MOS) rules [39, 40]. Therefore, the
proposed architectures can provide an estimate of the actual number of
coherent signals. As for the AoA estimation, the angular sector under
consideration is sampled to form a discrete set of angular positions that are
used at the design stage. Besides, since the exhaustive search can become very
time demanding for high numbers of signals, we design a suboptimum iterative
procedure providing satisfactory performance at least for the considered
numerical examples. Finally, the performance assessment is conducted over
synthetic data in comparison with classical detection architectures and
highlights that the proposed schemes represent an effective means to face with
the problem of target detection in the presence of coherent signals.

The remainder of the paper is organized as follows. Section II is devoted to
problem formulation and the definitions used in the next derivations, while
the design of the detection and estimation architectures is described in
Section III. Section IV shows the effectiveness of the proposed strategies
through numerical examples on simulated data. Finally, Section V contains
concluding remarks and charts a course for future works.

###  I-A Notation and Acronyms

In the sequel, vectors and matrices are denoted by boldface lower-case and
upper-case letters, respectively. Symbols $\det(\cdot)$, $\mbox{\rm
Tr}\,(\cdot)$, $(\cdot)^{T}$, and $(\cdot)^{\dagger}$ denote the determinant,
trace, transpose, and complex conjugate transpose, respectively. If $A$ and
$B$ are two generic sets, then $A\times B$ denotes the Cartesian product
between $A$ and $B$. As to the numerical sets, ${\mathds{C}}$ is the set of
complex numbers, and ${\mathds{C}}^{N\times M}$ is the Euclidean space of
$(N\times M)$-dimensional complex-valued matrices (or vectors if $M=1$). The
imaginary unit is indicated by $j$. The Euclidean norm of a generic vector $x$
is denoted by $\|{\mbox{\boldmath$x$}}\|$ whereas the modulus of a complex
number $x$ is denoted by $|x|$. The symbol ${\mathbb{E}}[\cdot]$ denotes
statistical expectation while $0$ and $I$ are the null vector/matrix and the
identity matrix, respectively, of proper size. Given two events $A$ and $B$,
the conditional probability of $A$ given $B$ is denoted by $P(A|B)$. The
acronyms PDF and IID mean Probability Density Function and Independent and
Identically Distributed, respectively. For a given matrix $A$,
$\lambda_{\max}\\{{\mbox{\boldmath$A$}}\\}$ denotes the maximum eigenvalue of
$A$. Finally, we write
${\mbox{\boldmath$x$}}\sim\mbox{$\mathcal{C}$}\mbox{$\mathcal{N}$}_{N}({\mbox{\boldmath$m$}},{\mbox{\boldmath$M$}})$
if $x$ is a complex circular $N$-dimensional normal vector with mean $m$ and
positive definite covariance matrix $M$.

##  II Problem Formulation

Let us assume that a search radar system is equipped with a uniform linear
array with $N$ antennas and transmits $L$ pulses in the nominal beam position
during one scan cycle. Then, each antenna collects $L$ samples from the cell
under test. Denote by ${\mbox{\boldmath$z$}}_{l}\in{\mathds{C}}^{N\times 1}$,
$l=1,\ldots,L$, the vector whose entries are the $l$th returns from each
antenna and by
${\mbox{\boldmath$Z$}}_{L}=[{\mbox{\boldmath$z$}}_{1},\ldots,{\mbox{\boldmath$z$}}_{L}]\in{\mathds{C}}^{N\times
L}$ the overall data matrix, the classical radar detection problem consists in
deciding whether or not ${\mbox{\boldmath$Z$}}_{L}$ contains the target of
interest, namely, a component whose signature coincides with the nominal
steering vector. Before proceeding with the problem formulation, it is
important to state here that the columns of ${\mbox{\boldmath$Z$}}_{L}$ are
assumed statistically independent and, hence, temporally noncoherent, whereas
each column is representative of both spatially correlated clutter and
spatially coherent useful signal components (a point better explained below).

As customary, we assume that a set of $K$ ($K\geq N$) secondary data,
${\mbox{\boldmath$z$}}_{k}\in{\mathds{C}}^{N\times 1}$, $k=1,\ldots,K$, free
of target components and sharing the same statistical properties of the
interference in the cell under test, is available [4, 7, 12, 41]. Under the
above assumptions, this problem can be formulated in terms of a conventional
binary hypothesis test whose expression is (see [4, 7, 12] and references
therein)

$\left\\{\begin{aligned}
&H_{0}:\begin{cases}{\mbox{\boldmath$z$}}_{l}={\mbox{\boldmath$n$}}_{l},\ \
l=1,\ldots,L,\\\ {\mbox{\boldmath$z$}}_{k}={\mbox{\boldmath$n$}}_{k},\ \
k=1,\ldots,K,\end{cases}\\\
&H_{1}:\begin{cases}{\mbox{\boldmath$z$}}_{l}=\alpha_{l}{\mbox{\boldmath$v$}}(\theta_{t})+{\mbox{\boldmath$n$}}_{l},\
\ l=1,\ldots,L,\\\ {\mbox{\boldmath$z$}}_{k}={\mbox{\boldmath$n$}}_{k},\ \
k=1,\ldots,K,\end{cases}\end{aligned}\right.$ (1)

where

  * •

${\mbox{\boldmath$n$}}_{l},{\mbox{\boldmath$n$}}_{k}\sim\mbox{$\mathcal{C}$}\mbox{$\mathcal{N}$}_{N}({\mbox{\boldmath$0$}},{\mbox{\boldmath$M$}})$,
$l=1,\ldots,L$, $k=1,\ldots,K$, are IID with unknown positive definite
covariance matrix ${\mbox{\boldmath$M$}}\in{\mathds{C}}^{N\times N}$
representative of the thermal noise plus clutter; $M$ is referred to in the
following as Interference Covariance Matrix (ICM);

  * •

$\alpha_{1},\ldots,\alpha_{L}\sim\mbox{$\mathcal{C}$}\mbox{$\mathcal{N}$}_{1}(0,\sigma_{\alpha}^{2})$
with $\sigma_{\alpha}^{2}>0$ are IID random variables accounting for both the
target response (RCS) and channel effects (Swerling II target model [1]);

  * •

${\mbox{\boldmath$v$}}(\theta_{t})\in{\mathds{C}}^{N\times 1}$ is the nominal
(spatial) steering vector whose expression is

${\mbox{\boldmath$v$}}(\theta_{t})\\!\\!=\\!\\![1,\exp\\{j\pi\sin(\theta_{t})\\},\ldots,\exp\\{j\pi(N-1)\sin(\theta_{t})\\}]^{T}\\!\\!,$

where $\theta_{t}$ is the (known) AoA of the target measured with respect to
the array broadside and we have assumed that the inter-element spacing is half
the operating wavelength in order to avoid the aliasing of the spatial
frequency.

However, as stated in Section I, in practical applications the generic
${\mbox{\boldmath$z$}}_{l}$ might not only contain the direct echo but also
returns from other directions and that, more importantly, are coherent (i.e.,
fully correlated) with the former due, for instance, to the effects of
multipath propagation and/or the presence of smart jammers [26, 27, 28]. As a
consequence, the conventional radar detection problem (1) consisting of two
hypotheses may be no longer representative of the actual operating scenario.
In fact, in such case, the alternative hypothesis can be replaced by the
following

$\\!\\!\\!H_{1,M}:\\!\\!\\!\
\begin{cases}\displaystyle{\mbox{\boldmath$z$}}_{l}=\alpha_{l}{\mbox{\boldmath$v$}}(\theta_{t})+\sum_{i=1}^{M}\beta_{l,i}{\mbox{\boldmath$v$}}(\theta_{i})+{\mbox{\boldmath$n$}}_{l},&\\!\\!\\!\\!\\!l=1,\ldots,L,\\\
{\mbox{\boldmath$z$}}_{k}={\mbox{\boldmath$n$}}_{k},&\\!\\!\\!\\!\\!k=1,\ldots,K,\end{cases}$
(2)

where $M$ is the actual number of coherent signals, which is unknown. However,
notice that the uncertainty on $M$ naturally leads to multiple alternative
hypotheses, $H_{1,i}$ say, which differ in the number of coherent signals.
Therefore, assuming an upper bound, $M_{c}$ say, on the latter unknown
parameter, a possible approach to account for the presence of possible fully-
correlated signals consists in considering a multiple hypothesis test where
besides the conventional $H_{0}$ and $H_{1}$ hypotheses, $M_{c}<N$ additional
hypotheses111The constraint $M_{c}<N$ is due to both the fact that $N$ degrees
of freedom are required to estimate $N$ AoAs also making matrix
${\mbox{\boldmath$V$}}_{i}$, defined in (6), full-column rank., corresponding
to scenarios containing a different number of coherent signals, appear.
Therefore, problem (1) can be generalized as follows

$\left\\{\begin{aligned}
H_{0}&:\begin{cases}{\mbox{\boldmath$z$}}_{l}={\mbox{\boldmath$n$}}_{l},&l=1,\ldots,L,\\\
{\mbox{\boldmath$z$}}_{k}={\mbox{\boldmath$n$}}_{k},&k=1,\ldots,K,\end{cases}\\\
H_{1,0}&:\begin{cases}{\mbox{\boldmath$z$}}_{l}=\alpha_{l}{\mbox{\boldmath$v$}}(\theta_{t})+{\mbox{\boldmath$n$}}_{l},&l=1,\ldots,L,\\\
{\mbox{\boldmath$z$}}_{k}={\mbox{\boldmath$n$}}_{k},&k=1,\ldots,K,\end{cases}\\\
H_{1,i}&:\begin{cases}\displaystyle{\mbox{\boldmath$z$}}_{l}=\alpha_{l}{\mbox{\boldmath$v$}}(\theta_{t})+\sum_{k=1}^{i}\beta_{l,k}{\mbox{\boldmath$v$}}(\theta_{k})+{\mbox{\boldmath$n$}}_{l},&\\!\\!\\!\\!\\!l=1,\ldots,L,\\\
{\mbox{\boldmath$z$}}_{k}={\mbox{\boldmath$n$}}_{k},&\\!\\!\\!\\!\\!k=1,\ldots,K,\end{cases}\end{aligned}\right.$
(3)

$i=1,\ldots,M_{c}$, where the $\alpha_{l}$s, the ${\mbox{\boldmath$n$}}_{l}$s,
and the ${\mbox{\boldmath$n$}}_{k}$s have been already defined for problem (1)
and

  * •

$M_{c}\geq M$ is an upper bound on the number of coherent signals222Note that
problem (3) reduces to (1) when $M_{c}=0$.;

  * •

given $i\in\\{1,\ldots,M_{c}\\}$ and $k\in\\{1,\ldots,i\\}$, the coefficients
$\beta_{1,k},\ldots,\beta_{L,k}\in{\mathds{C}}$ are IID complex Gaussian
random variables with zero mean and variance $\sigma_{k}^{2}>0$; they
represent the complex amplitudes of the $k$th coherent signal over the
time333It is worth noticing that Swerling II target model is also adopted for
the coherent signals.;

  * •

given $i\in\\{1,\ldots,M_{c}\\}$, $\theta_{1},\ldots,\theta_{i}$ are the
unknown AoAs of the additional coherent signals.

Finally, we assume that, $\forall i=1,\ldots,M_{c}$, the random variables
$\alpha_{l}$, $\beta_{l,1},\ldots,\beta_{l,i}$ are fully correlated. Otherwise
stated, given $l\in\\{1,\ldots,L\\}$ and $i\in\\{1,\ldots,M_{c}\\}$, $\forall
k,h\in\\{1,\ldots,i\\}$ and $k\neq h$ the following equalities hold

${\mathbb{E}}[\alpha_{l}\beta_{l,k}]=\sigma_{\alpha}\sigma_{{k}}\quad\mbox{and}\quad{\mathbb{E}}[\beta_{l,k}\beta_{l,h}]=\sigma_{{k}}\sigma_{{h}}.$
(4)

In order to unburden the notation, hereafter we omit the dependence of the
steering vectors on the AoAs and denote by ${\mbox{\boldmath$v$}}_{t}$ the
nominal steering vector whereas we use $\mbox{\boldmath$p$}_{i}$ to indicate
the steering vector associated with the coherent signal whose AoA is
$\theta_{i}$.

Before concluding this section, we provide some definitions that will come in
handy for the next developments. More precisely, the PDF of
${\mbox{\boldmath$Z$}}_{L}$ under $H_{0}$ has the following expression

$f_{0}({\mbox{\boldmath$Z$}}_{L};{\mbox{\boldmath$M$}})=\displaystyle{\left[\frac{1}{\pi^{N}\det({\mbox{\boldmath$M$}})}\right]^{L}\exp\left[-\mbox{\rm
Tr}\,\left({\mbox{\boldmath$M$}}^{-1}{\mbox{\boldmath$Z$}}_{L}{\mbox{\boldmath$Z$}}_{L}^{\dagger}\right)\right]},$
(5)

whereas its PDF under the generic $H_{1,i}$, $i=0,\ldots,M_{c}$, is given by

$f_{1,i}({\mbox{\boldmath$Z$}}_{L};{\mbox{\boldmath$M$}},{\mbox{\boldmath$R$}}_{i},{\mbox{\boldmath$V$}}_{i})=\left[\frac{1}{\pi^{N}\det({\mbox{\boldmath$M$}}+{\mbox{\boldmath$V$}}_{i}{\mbox{\boldmath$R$}}_{i}{\mbox{\boldmath$V$}}_{i}^{\dagger})}\right]^{L}\\\
\times\exp\left\\{-\mbox{\rm
Tr}\,\left[\left({\mbox{\boldmath$M$}}+{\mbox{\boldmath$V$}}_{i}{\mbox{\boldmath$R$}}_{i}{\mbox{\boldmath$V$}}_{i}^{\dagger}\right)^{-1}{\mbox{\boldmath$Z$}}_{L}{\mbox{\boldmath$Z$}}_{L}^{\dagger}\right]\right\\},$
(6)

where
${\mbox{\boldmath$V$}}_{i}=[{\mbox{\boldmath$v$}}_{t},\mbox{\boldmath$p$}_{1},\ldots,\mbox{\boldmath$p$}_{i}]\in{\mathds{C}}^{N\times(i+1)}$
and
${\mbox{\boldmath$R$}}_{i}={\mathbb{E}}[{\mbox{\boldmath$\nu$}}_{l,i}{\mbox{\boldmath$\nu$}}_{l,i}^{\dagger}]\in{\mathds{C}}^{(i+1)\times(i+1)}$,
$l=1,\ldots,L$, with
${\mbox{\boldmath$\nu$}}_{l,i}=[\alpha_{l},\beta_{l,1},\ldots,\beta_{l,i}]^{T}\in{\mathds{C}}^{(i+1)\times
1}$. It is understood that when $i=0$, ${\mbox{\boldmath$V$}}_{0}$ and
${\mbox{\boldmath$\nu$}}_{l,0}$ coincide with ${\mbox{\boldmath$v$}}_{t}$ and
$\alpha_{l}$, respectively.

##  III Detection Architecture Designs

In this section, we devise a detection architecture for problem (3) that
relies on a penalized log-likelihood ratio test [29, 42] whose generic
structure is given by

$\left[\widehat{\Lambda}_{\hat{i}}({\mbox{\boldmath$Z$}}_{L})-c\cdot
h(\hat{i})\right]\mbox{$\begin{array}[]{c}\stackrel{{\scriptstyle\stackrel{{\scriptstyle\textstyle
H_{1,\hat{i}}}}{{\textstyle>}}}}{{\stackrel{{\scriptstyle\textstyle<}}{{\textstyle
H_{0}}}}}\end{array}$}\eta,$ (7)

where

$\widehat{i}=\operatorname*{arg\,max}_{i=0,\ldots,M_{c}}\left\\{\widehat{\Lambda}_{i}({\mbox{\boldmath$Z$}}_{L})-c\cdot
h(i)\right\\},$ (8)

$\eta$ is the detection threshold to be set according to a
desired444Hereafter, we denote by $\eta$ the generic detection threshold.
$P_{fa}$,

$\widehat{\Lambda}_{i}({\mbox{\boldmath$Z$}}_{L})=\log\frac{\displaystyle\max_{{\mbox{\boldmath$R$}}_{i}}\displaystyle\max_{\theta_{1},\ldots,\theta_{i}}f_{1,i}({\mbox{\boldmath$Z$}}_{L};\widehat{{\mbox{\boldmath$M$}}},{\mbox{\boldmath$R$}}_{i},{\mbox{\boldmath$V$}}_{i})}{f_{0}({\mbox{\boldmath$Z$}}_{L};\widehat{{\mbox{\boldmath$M$}}})}$
(9)

with555Notice that for $i=0$, the maximization at the numerator of (9) is with
respect to $\sigma^{2}_{\alpha}$ only.  $\widehat{{\mbox{\boldmath$M$}}}$ a
suitable estimate of $M$, and $c\cdot h(i)$ is a penalty term borrowed from
the MOS rules [39] with $h(i)$ the number of unknown real-valued parameters
under the $H_{1,i}$ hypothesis and $c$ a suitable scaling factor.
Specifically, $c$ can be set according to the Akaike Information Criterion
(AIC), Bayesian Information Criterion (BIC), and Generalized Information
Criterion (GIC), namely

$c=\begin{cases}1,&\mbox{for AIC-based Detector (AIC-D)},\\\
\log(2LN)/2,&\mbox{for BIC-based Detector (BIC-D)},\\\ (1+\rho)/2,\
\rho>1,&\mbox{for GIC-based Detector (GIC-D)}.\end{cases}$ (10)

Two important remarks are now in order. First, notice that
$\widehat{\Lambda}_{i}({\mbox{\boldmath$Z$}}_{L})$ is the logarithm of a
decision statistic obtained by resorting to a modified Generalized Likelihood
Ratio Test (GLRT)-based design procedure [7], where the ICM is firstly assumed
known and then replaced by an estimate. In the specific case, we use the ML
estimate based upon the training samples only (namely, the sample covariance
matrix that is invertible with probability $1$ since $K\geq N$), whose
expression is666This design choice is dictated by the fact that deriving
closed-form expression for the plain GLRT is not mathematically tractable at
least to the best of authors’ knowledge.

$\widehat{{\mbox{\boldmath$M$}}}=\frac{1}{K}\sum_{k=1}^{K}{\mbox{\boldmath$z$}}_{k}{\mbox{\boldmath$z$}}_{k}^{\dagger}.$
(11)

Second, the correlation of the signals makes the columns of the positive
semidefinite matrix ${\mbox{\boldmath$R$}}_{i}$ proportional. Therefore, the
resulting rank is $1$ and it can be factorized as

${\mbox{\boldmath$R$}}_{i}={\mbox{\boldmath$r$}}_{i}{\mbox{\boldmath$r$}}_{i}^{\dagger},$
(12)

where ${\mbox{\boldmath$r$}}_{i}\in{\mathds{C}}^{(i+1)\times 1}$. As a
consequence, the number of unknown real-valued parameters under $H_{1,i}$ can
be written as $h(i)=N^{2}+1+2i$. With the above remarks in mind, let us
proceed by deriving $\widehat{\Lambda}_{0}({\mbox{\boldmath$Z$}}_{L})$, namely

$\displaystyle\widehat{\Lambda}_{0}({\mbox{\boldmath$Z$}}_{L})=\log\frac{\displaystyle\max_{\sigma_{\alpha}^{2}}f_{1,0}({\mbox{\boldmath$Z$}}_{L};{\mbox{$\widehat{{\mbox{\boldmath$M$}}}$}},\sigma^{2}_{\alpha},{\mbox{\boldmath$v$}}_{t})}{f_{0}({\mbox{\boldmath$Z$}}_{L};{\mbox{$\widehat{{\mbox{\boldmath$M$}}}$}})}$
$\displaystyle=\displaystyle\max_{\sigma_{\alpha}^{2}}\bigg{\\{}\\!\\!\\!-L\log\det({\mbox{$\widehat{{\mbox{\boldmath$M$}}}$}}+\sigma_{\alpha}^{2}{\mbox{\boldmath$v$}}_{t}{\mbox{\boldmath$v$}}_{t}^{\dagger})-\mbox{\rm
Tr}\,\bigg{[}\left({\mbox{$\widehat{{\mbox{\boldmath$M$}}}$}}+\sigma_{\alpha}^{2}{\mbox{\boldmath$v$}}_{t}{\mbox{\boldmath$v$}}_{t}^{\dagger}\right)^{-1}$
$\displaystyle\times{\mbox{\boldmath$Z$}}_{L}{\mbox{\boldmath$Z$}}_{L}^{\dagger}\bigg{]}\bigg{\\}}+L\log\det({\mbox{$\widehat{{\mbox{\boldmath$M$}}}$}})+\mbox{\rm
Tr}\,\left[{\mbox{$\widehat{{\mbox{\boldmath$M$}}}$}}^{-1}{\mbox{\boldmath$Z$}}_{L}{\mbox{\boldmath$Z$}}_{L}^{\dagger}\right].$
(13)

Applying the Woodbury identity [43], we come up with the following expression

$\widehat{\Lambda}_{0}({\mbox{\boldmath$Z$}}_{L})=\displaystyle\max_{\sigma_{\alpha}^{2}}\left[-L\log(1+\sigma_{\alpha}^{2}\lVert{\mbox{\boldmath$v$}}_{w}\rVert^{2})+\frac{\sigma_{\alpha}^{2}{\mbox{\boldmath$v$}}_{w}^{\dagger}{\mbox{\boldmath$S$}}_{w}{\mbox{\boldmath$v$}}_{w}}{1+\sigma_{\alpha}^{2}\lVert{\mbox{\boldmath$v$}}_{w}\rVert^{2}}\right],$
(14)

where
${\mbox{\boldmath$v$}}_{w}={\mbox{$\widehat{{\mbox{\boldmath$M$}}}$}}^{-1/2}{\mbox{\boldmath$v$}}_{t}$
and
${\mbox{\boldmath$S$}}_{w}={\mbox{$\widehat{{\mbox{\boldmath$M$}}}$}}^{-1/2}{\mbox{\boldmath$Z$}}_{L}{\mbox{\boldmath$Z$}}_{L}^{\dagger}{\mbox{$\widehat{{\mbox{\boldmath$M$}}}$}}^{-1/2}$.
Let us define the function

$g_{0}(\sigma^{2}_{\alpha})=-L\log(1+\sigma_{\alpha}^{2}\lVert{\mbox{\boldmath$v$}}_{w}\rVert^{2})+\frac{\sigma_{\alpha}^{2}}{1+\sigma_{\alpha}^{2}\lVert{\mbox{\boldmath$v$}}_{w}\rVert^{2}}{\mbox{\boldmath$v$}}_{w}^{\dagger}{\mbox{\boldmath$S$}}_{w}{\mbox{\boldmath$v$}}_{w}$
(15)

and observe that $\lim_{\sigma^{2}_{\alpha}\rightarrow
0}g_{0}(\sigma^{2}_{\alpha})=0$ and
$\lim_{\sigma^{2}_{\alpha}\rightarrow+\infty}g_{0}(\sigma^{2}_{\alpha})=-\infty$.
Thus, in order to maximize $g_{0}(\sigma^{2}_{\alpha})$, we find the zeros of
its first derivative with respect to $\sigma_{\alpha}^{2}$ to obtain

$\widehat{\sigma}_{\alpha}^{2}=\begin{cases}\displaystyle\frac{{\mbox{\boldmath$v$}}_{w}^{\dagger}{\mbox{\boldmath$S$}}_{w}{\mbox{\boldmath$v$}}_{w}-L\lVert{\mbox{\boldmath$v$}}_{w}\rVert^{2}}{L\lVert{\mbox{\boldmath$v$}}_{w}\rVert^{4}},\
&\mbox{if}\
{\mbox{\boldmath$v$}}_{w}^{\dagger}{\mbox{\boldmath$S$}}_{w}{\mbox{\boldmath$v$}}_{w}>L\lVert{\mbox{\boldmath$v$}}_{w}\rVert^{2},\\\
0,\ &\mbox{otherwise}.\end{cases}$ (16)

Moreover, $g_{0}(\sigma^{2}_{\alpha})$ is monotonic increasing when
$\sigma^{2}_{\alpha}<({{\mbox{\boldmath$v$}}_{w}^{\dagger}{\mbox{\boldmath$S$}}_{w}{\mbox{\boldmath$v$}}_{w}-L\lVert{\mbox{\boldmath$v$}}_{w}\rVert^{2}})/({L\lVert{\mbox{\boldmath$v$}}_{w}\rVert^{4}})$
and decreasing for
$\sigma^{2}_{\alpha}>({{\mbox{\boldmath$v$}}_{w}^{\dagger}{\mbox{\boldmath$S$}}_{w}{\mbox{\boldmath$v$}}_{w}-L\lVert{\mbox{\boldmath$v$}}_{w}\rVert^{2}})/({L\lVert{\mbox{\boldmath$v$}}_{w}\rVert^{4}})$
(provided that
${\mbox{\boldmath$v$}}_{w}^{\dagger}{\mbox{\boldmath$S$}}_{w}{\mbox{\boldmath$v$}}_{w}>L\lVert{\mbox{\boldmath$v$}}_{w}\rVert^{2}$).
As a consequence, (16) represents a maximum point of
$g_{0}(\sigma^{2}_{\alpha})$. It follows that (14) can be recast as

$\displaystyle\widehat{\Lambda}_{0}({\mbox{\boldmath$Z$}}_{L})=$
$\displaystyle\begin{cases}-L\log\\!\left(\displaystyle\frac{{\mbox{\boldmath$v$}}_{w}^{\dagger}{\mbox{\boldmath$S$}}_{w}{\mbox{\boldmath$v$}}_{w}}{L\lVert{\mbox{\boldmath$v$}}_{w}\rVert^{2}}\right)\\!\\!+\\!\\!\displaystyle\frac{{\mbox{\boldmath$v$}}_{w}^{\dagger}{\mbox{\boldmath$S$}}_{w}{\mbox{\boldmath$v$}}_{w}}{\lVert{\mbox{\boldmath$v$}}_{w}\rVert^{2}}\\!-\\!L,\\!\mbox{if}\
{\mbox{\boldmath$v$}}_{w}^{\dagger}{\mbox{\boldmath$S$}}_{w}{\mbox{\boldmath$v$}}_{w}\\!>\\!L\lVert{\mbox{\boldmath$v$}}_{w}\rVert^{2}\\!\\!,\\\
0,\ \mbox{otherwise}.\end{cases}$ (17)

The next step towards the computation of (7) consists in deriving
$\widehat{\Lambda}_{i}({\mbox{\boldmath$Z$}}_{L})$ for $i=1,\ldots,M_{c}$. To
this end, observe that

$\displaystyle\widehat{\Lambda}_{i}({\mbox{\boldmath$Z$}}_{L})=\log\frac{\displaystyle\max_{{\mbox{\boldmath$R$}}_{i}}\displaystyle\max_{\theta_{1},\ldots,\theta_{i}}f_{1,i}({\mbox{\boldmath$Z$}}_{L};{\mbox{$\widehat{{\mbox{\boldmath$M$}}}$}},{\mbox{\boldmath$R$}}_{i},{\mbox{\boldmath$V$}}_{i})}{f_{0}({\mbox{\boldmath$Z$}}_{L};{\mbox{$\widehat{{\mbox{\boldmath$M$}}}$}})}$
$\displaystyle=\\!\\!\\!\displaystyle\max_{{\boldmath
R_{i}},\atop\theta_{1},\ldots,\theta_{i}}\\!\\!\\!\bigg{\\{}\\!\\!\\!-L\log\det({\mbox{$\widehat{{\mbox{\boldmath$M$}}}$}}\\!+\\!{\mbox{\boldmath$V$}}_{i}{\mbox{\boldmath$R$}}_{i}{\mbox{\boldmath$V$}}_{i}^{\dagger})\\!-\\!\mbox{\rm
Tr}\,\\!\bigg{[}\\!\\!\left(\\!{\mbox{$\widehat{{\mbox{\boldmath$M$}}}$}}\\!+\\!{\mbox{\boldmath$V$}}_{i}{\mbox{\boldmath$R$}}_{i}{\mbox{\boldmath$V$}}_{i}^{\dagger}\\!\right)^{-1}$
$\displaystyle\times{\mbox{\boldmath$Z$}}_{L}{\mbox{\boldmath$Z$}}_{L}^{\dagger}\bigg{]}\bigg{\\}}+L\log\det({\mbox{$\widehat{{\mbox{\boldmath$M$}}}$}})+\mbox{\rm
Tr}\,\left({\mbox{$\widehat{{\mbox{\boldmath$M$}}}$}}^{-1}{\mbox{\boldmath$Z$}}_{L}{\mbox{\boldmath$Z$}}_{L}^{\dagger}\right).$
(18)

Replacing (12) into the most right-hand side of the above equation and
applying the Woodbury identity leads to

$\widehat{\Lambda}_{i}({\mbox{\boldmath$Z$}}_{L})=\displaystyle\max_{{\mbox{\boldmath$r$}}_{i}}\displaystyle\max_{\theta_{1},\ldots,\theta_{i}}\left\\{-L\log(1+{\mbox{\boldmath$r$}}_{i}^{\dagger}{\mbox{\boldmath$V$}}_{w,i}^{\dagger}{\mbox{\boldmath$V$}}_{w,i}{\mbox{\boldmath$r$}}_{i})\right.\\\
\left.+\frac{{\mbox{\boldmath$r$}}_{i}^{\dagger}{\mbox{\boldmath$V$}}_{w,i}^{\dagger}{\mbox{\boldmath$S$}}_{w}{\mbox{\boldmath$V$}}_{w,i}{\mbox{\boldmath$r$}}_{i}}{1+{\mbox{\boldmath$r$}}_{i}^{\dagger}{\mbox{\boldmath$V$}}_{w,i}^{\dagger}{\mbox{\boldmath$V$}}_{w,i}{\mbox{\boldmath$r$}}_{i}}\right\\},$
(19)

where
${\mbox{\boldmath$V$}}_{w,i}={\mbox{$\widehat{{\mbox{\boldmath$M$}}}$}}^{-1/2}{\mbox{\boldmath$V$}}_{i}$
and ${\mbox{\boldmath$S$}}_{w}$ has been previously defined. Since
${\mbox{\boldmath$V$}}_{i}$ has a Vandermonde structure and
$\widehat{{\mbox{\boldmath$M$}}}$ is nonsingular, also matrices
${\mbox{\boldmath$A$}}_{i}={\mbox{\boldmath$V$}}_{w,i}^{\dagger}{\mbox{\boldmath$V$}}_{w,i}$
and
${\mbox{\boldmath$B$}}_{i}={\mbox{\boldmath$V$}}_{w,i}^{\dagger}{\mbox{\boldmath$S$}}_{w}{\mbox{\boldmath$V$}}_{w,i}$
are nonsingular and (19) can be expressed in terms of these matrices as

$\widehat{\Lambda}_{i}({\mbox{\boldmath$Z$}}_{L})\\!=\\!\displaystyle\max_{{\mbox{\boldmath$r$}}_{i}}\displaystyle\max_{\theta_{1},\ldots,\theta_{i}}\left\\{\\!\\!-L\log(1+{\mbox{\boldmath$r$}}_{i}^{\dagger}{\mbox{\boldmath$A$}}_{i}{\mbox{\boldmath$r$}}_{i})+\frac{{\mbox{\boldmath$r$}}_{i}^{\dagger}{\mbox{\boldmath$B$}}_{i}{\mbox{\boldmath$r$}}_{i}}{1+{\mbox{\boldmath$r$}}_{i}^{\dagger}{\mbox{\boldmath$A$}}_{i}{\mbox{\boldmath$r$}}_{i}}\\!\\!\right\\}.$
(20)

Exploiting the Cholesky decomposition of ${\mbox{\boldmath$A$}}_{i}$, given by

${\mbox{\boldmath$A$}}_{i}={\mbox{\boldmath$C$}}_{i}{\mbox{\boldmath$C$}}_{i}^{\dagger},$
(21)

where ${\mbox{\boldmath$C$}}_{i}\in{\mathds{C}}^{(i+1)\times(i+1)}$ is a lower
triangular matrix, the right-hand side of (20) can be recast as

$\displaystyle\displaystyle\max_{{\mbox{\boldmath$r$}}_{i},\atop\theta_{1},\ldots,\theta_{i}}\\!\\!\\!\left\\{\\!\\!-L\log(1\\!+\\!{\mbox{\boldmath$r$}}_{i}^{\dagger}{\mbox{\boldmath$C$}}_{i}{\mbox{\boldmath$C$}}_{i}^{\dagger}{\mbox{\boldmath$r$}}_{i})\\!+\\!\frac{{\mbox{\boldmath$r$}}_{i}^{\dagger}{\mbox{\boldmath$C$}}_{i}{\mbox{\boldmath$C$}}_{i}^{-1}{\mbox{\boldmath$B$}}_{i}({\mbox{\boldmath$C$}}_{i}^{\dagger})^{-1}{\mbox{\boldmath$C$}}_{i}^{\dagger}{\mbox{\boldmath$r$}}_{i}}{1\\!+\\!{\mbox{\boldmath$r$}}_{i}^{\dagger}{\mbox{\boldmath$C$}}_{i}{\mbox{\boldmath$C$}}_{i}^{\dagger}{\mbox{\boldmath$r$}}_{i}}\right\\}$
$\displaystyle=\displaystyle\max_{{\mbox{\boldmath$a$}}_{i}}\displaystyle\max_{\theta_{1},\ldots,\theta_{i}}\left\\{-L\log(1+\lVert{\mbox{\boldmath$a$}}_{i}\rVert^{2})+\frac{{\mbox{\boldmath$a$}}_{i}^{\dagger}{\mbox{\boldmath$D$}}_{i}{\mbox{\boldmath$a$}}_{i}}{1+\lVert{\mbox{\boldmath$a$}}_{i}\rVert^{2}}\right\\},$
(22)

where

${\mbox{\boldmath$a$}}_{i}={\mbox{\boldmath$C$}}_{i}^{\dagger}{\mbox{\boldmath$r$}}_{i},\quad{\mbox{\boldmath$D$}}_{i}={\mbox{\boldmath$C$}}_{i}^{-1}{\mbox{\boldmath$B$}}_{i}({\mbox{\boldmath$C$}}_{i}^{\dagger})^{-1}.$
(23)

As further step towards the solution of the above problem, we decompose
${\mbox{\boldmath$a$}}_{i}$ as the product of a positive scalar
$\sqrt{a_{i}}$, with $a_{i}>0$, times a unit-norm vector
${\mbox{\boldmath$u$}}_{a,i}$. Thus, problem (22) is tantamount to

$\displaystyle\max_{a_{i}>0}\displaystyle\max_{{\mbox{\boldmath$u$}}_{a,i}\atop\|u_{a,i}\|=1}\displaystyle\max_{\theta_{1},\ldots,\theta_{i}}\left\\{-L\log(1+a_{i})+\frac{a_{i}{{\mbox{\boldmath$u$}}_{a,i}^{\dagger}{\mbox{\boldmath$D$}}_{i}{\mbox{\boldmath$u$}}_{a,i}}}{1+a_{i}}\right\\}.$
(24)

Denoting by

$g(a_{i})=-L\log(1+a_{i})+\frac{a_{i}}{1+a_{i}}{{\mbox{\boldmath$u$}}_{a,i}^{\dagger}{\mbox{\boldmath$D$}}_{i}{\mbox{\boldmath$u$}}_{a,i}},$
(25)

the maximization with respect to $a_{i}$ can be performed by investigating the
behavior of $g(a_{i})$ over the interval $(0,+\infty)$. Specifically, since
the following equalities hold

$\lim_{a_{i}\rightarrow
0}g(a_{i})=0,\quad\lim_{a_{i}\rightarrow+\infty}g(a_{i})=-\infty,$ (26)

we can search for the stationary points of $g(a_{i})$ in the interior of its
domain by setting to zero its first derivative with respect to $a_{i}$ to
obtain

$\widehat{a}_{i}=\begin{cases}\displaystyle\frac{{\mbox{\boldmath$u$}}_{a,i}^{\dagger}{\mbox{\boldmath$D$}}_{i}{\mbox{\boldmath$u$}}_{a,i}-L}{L},\
&\mbox{if}\
{\mbox{\boldmath$u$}}_{a,i}^{\dagger}{\mbox{\boldmath$D$}}_{i}{\mbox{\boldmath$u$}}_{a,i}>L,\\\
0,\ &\mbox{otherwise}.\end{cases}$ (27)

Moreover, it is not difficult to show that $g(a_{i})$ is increasing when
$a_{i}<({{\mbox{\boldmath$u$}}_{a,i}^{\dagger}{\mbox{\boldmath$D$}}_{i}{\mbox{\boldmath$u$}}_{a,i}-L})/{L}$
and decreasing in the opposite case (provided that
${\mbox{\boldmath$u$}}_{a,i}^{\dagger}{\mbox{\boldmath$D$}}_{i}{\mbox{\boldmath$u$}}_{a,i}>L$).
As a consequence, (27) represents a maximum point. Now, plugging (27) into
(24), the maximization problem can be recast as (29) at the bottom of this
page.

The maximization over ${\mbox{\boldmath$u$}}_{a,i}$ can be conducted by
defining the function $f(x)=x-L\log x+L\log L-L$, which is monotonic
increasing for $x\geq L$. In fact, its first derivative is
$\frac{df(x)}{dx}=1-\frac{L}{x}\geq 0$, $\forall x\geq L$. Therefore, the
maximum of
$f({\mbox{\boldmath$u$}}_{a,i}^{\dagger}{\mbox{\boldmath$D$}}_{i}{\mbox{\boldmath$u$}}_{a,i})$
with respect to ${\mbox{\boldmath$u$}}_{a,i}$ can be obtained by directly
optimizing argument, i.e.,
${\mbox{\boldmath$u$}}_{a,i}^{\dagger}{\mbox{\boldmath$D$}}_{i}{\mbox{\boldmath$u$}}_{a,i}$.
Exploiting the Rayleigh-Ritz Theorem [43], we obtain that

$\displaystyle\max_{{\mbox{\boldmath$u$}}_{a,i}\atop\|u_{a,i}\|=1}{\mbox{\boldmath$u$}}_{a,i}^{\dagger}{\mbox{\boldmath$D$}}_{i}{\mbox{\boldmath$u$}}_{a,i}=\lambda_{\max}\\{{\mbox{\boldmath$D$}}_{i}\\},$
(28)

and the maximum is attained when ${\mbox{\boldmath$u$}}_{a,i}$ is the
normalized eigenvector of ${\mbox{\boldmath$D$}}_{i}$ corresponding to its
maximum eigenvalue. Gathering the above results, we obtain (30) at the bottom
of this page.

The last optimization problem to be solved is with respect to the AoAs of the
coherent signals. In this respect, since deriving closed-form expressions for
the ML estimates of $\theta_{1},\ldots,\theta_{i}$ represents a difficult task
(at least to the best of authors’ knowledge), we devise grid-search methods.
More precisely, the angular sector under surveillance is sampled to form a
discrete set of angular positions which is denoted by
$\Omega=\\{\omega_{1},\ldots,\omega_{S}\\}$ (see Figure 1). The cardinality of
$\Omega$ can be chosen according to both the computational power available at
the receiver and the system requirements in terms of reactivity time
intervals. It is clear that a dense sampling of the angular sector increases
the processing time but can provide better estimation results with respect to
the case where the sampling interval is large. However, in the latter case,
the computational load is lower than in the former case. Nevertheless, an
iterative approach can be pursued for the maximization under the generic
$H_{1,i}$ hypothesis and for each $i\in\\{1,\ldots,M_{c}\\}$. Specifically,
given $i\in\\{1,\ldots,M_{c}\\}$ and assuming that $H_{1,i}$ holds true, we
start by roughly sampling the angular sector of interest and use this search
grid to come up with preliminary estimates of the assumed $i$ AoAs. Then, we
use these estimates to identify a restricted angular sector (resulting from
the union of partially overlapped narrow sectors) comprising suitable guard
bands.

$\widehat{\Lambda}_{i}({\mbox{\boldmath$Z$}}_{L})=\begin{cases}\displaystyle\max_{{\mbox{\boldmath$u$}}_{a,i}\atop\|u_{a,i}\|=1}\displaystyle\max_{\theta_{1},\ldots,\theta_{i}}-L\log({\mbox{\boldmath$u$}}_{a,i}^{\dagger}{\mbox{\boldmath$D$}}_{i}{\mbox{\boldmath$u$}}_{a,i})+{\mbox{\boldmath$u$}}_{a,i}^{\dagger}{\mbox{\boldmath$D$}}_{i}{\mbox{\boldmath$u$}}_{a,i}+L\log
L-L,\ &\mbox{if}\
{\mbox{\boldmath$u$}}_{a,i}^{\dagger}{\mbox{\boldmath$D$}}_{i}{\mbox{\boldmath$u$}}_{a,i}>L,\\\
0,\ &\mbox{otherwise}.\end{cases}$ (29)
$\widehat{\Lambda}_{i}({\mbox{\boldmath$Z$}}_{L})=\begin{cases}\displaystyle\max_{\theta_{1},\ldots,\theta_{i}}-L\log\lambda_{\max}\\{{\mbox{\boldmath$D$}}_{i}\\}+\lambda_{\max}\\{{\mbox{\boldmath$D$}}_{i}\\}+L\log
L-L,\ &\mbox{if}\ \lambda_{\max}\\{{\mbox{\boldmath$D$}}_{i}\\}>L,\\\ 0,\
&\mbox{otherwise}.\end{cases}$ (30)

The new angular sector of interest is sampled through a shorter sampling
interval and the maximization under $H_{1,i}$ is repeated over the new search
grid. As a result, the grid points can be very close to the actual angular
positions (at least for high signal power values) improving the estimation
quality. Finally, notice that this procedure is applied for each
$i=\\{1,\ldots,M_{c}\\}$ before forming the decision statistic and, in a
single application, the number of supposed interferers is maintained constant.
As for the single search procedure under the generic $H_{1,i}$ hypothesis with
$i\geq 1$, observe that in situations where, according to the system geometry
and antenna beamwidth, $M_{c}$ is enough low, an exhaustive grid-based search
can be conducted over the set

$\underbrace{\Omega\times\Omega\times\ldots\times\Omega}_{i\ \mbox{\scriptsize
times}}=\Omega^{i},\quad\forall i\in\\{1,\ldots,M_{c}\\},$ (31)

Figure 1: Sampling of the angular sector under surveillance.

with the constraint that admissible candidates are such that
$\omega_{1}\neq\omega_{2}\neq\ldots\neq\omega_{i}$. On the other hand,
exhaustive grid search can be prohibitive for large $M_{c}$ values. In this
case, in order to limit the number of operations, we design a suboptimum
search method relying on the cyclic optimization paradigm [44]. Specifically,
let us assume that $H_{1,i}$ is true and that $i-1$ estimates,
$\widehat{\Theta}_{1,i}^{(0)}=\\{\hat{\theta}^{(0)}_{2},\ldots,\hat{\theta}^{(0)}_{i}\\}\subset\Omega$
say, are available, then we select $\theta_{1}\in\Omega$ exploiting the
following criterion

$\hat{\theta}_{1}^{(1)}=\operatorname*{arg\,max}_{\theta_{1}\in\Omega,\
\theta_{1}\notin\widehat{\Theta}_{1,i}^{(0)}}\left\\{-L\log\lambda_{\max}\\{{\mbox{\boldmath$D$}}_{i}(\theta_{1},\widehat{\Theta}_{1,i}^{(0)})\\}\right.\\\
\left.+\lambda_{\max}\\{{\mbox{\boldmath$D$}}_{i}(\theta_{1},\widehat{\Theta}_{1,i}^{(0)})\\}\right\\},$
(32)

where ${\mbox{\boldmath$D$}}_{i}(\theta_{1},\widehat{\Theta}_{1}^{(0)})$ is
computed as ${\mbox{\boldmath$D$}}_{i}$ in (23) with the difference that
${\mbox{\boldmath$V$}}_{i}$ is replaced by the following matrix

${\mbox{\boldmath$V$}}_{i}\left(\theta_{1},\widehat{\Theta}_{1,i}^{(0)}\right)=\left[{\mbox{\boldmath$v$}}_{t},{\mbox{\boldmath$v$}}\left(\theta_{1}\right),{\mbox{\boldmath$v$}}(\hat{\theta}_{2}^{(0)}),\ldots,{\mbox{\boldmath$v$}}(\hat{\theta}_{i}^{(0)})\right].$
(33)

The new estimate $\hat{\theta}_{1}^{(1)}$ is used to form
$\widehat{\Theta}_{2,i}^{(0)}=\\{\hat{\theta}^{(1)}_{1},\hat{\theta}^{(0)}_{3},\ldots,\hat{\theta}^{(0)}_{i}\\}$
and the update of $\hat{\theta}_{2}$ is obtained as

$\hat{\theta}_{2}^{(1)}=\operatorname*{arg\,max}_{\theta_{2}\in\Omega,\
\theta_{2}\notin\widehat{\Theta}_{2,i}^{(0)}}\left\\{-L\log\lambda_{\max}\\{{\mbox{\boldmath$D$}}_{i}(\theta_{2},\widehat{\Theta}_{2,i}^{(0)})\\}\right.\\\
\left.+\lambda_{\max}\\{{\mbox{\boldmath$D$}}_{i}(\theta_{2},\widehat{\Theta}_{2,i}^{(0)})\\}\right\\},$
(34)

where ${\mbox{\boldmath$D$}}_{i}(\theta_{2},\widehat{\Theta}_{2,i}^{(0)})$ is
built up using

${\mbox{\boldmath$V$}}_{i}\\!\\!\left(\theta_{2},\widehat{\Theta}_{2,i}^{(0)}\right)\\!\\!=\\!\\!\left[{\mbox{\boldmath$v$}}_{t},{\mbox{\boldmath$v$}}(\hat{\theta}_{1}^{(1)}),{\mbox{\boldmath$v$}}(\theta_{2}),{\mbox{\boldmath$v$}}(\hat{\theta}_{3}^{(0)}),\ldots,{\mbox{\boldmath$v$}}(\hat{\theta}_{i}^{(0)})\right].$
(35)

The above steps continue until the $i$th update occurs to obtain
$\widehat{\Theta}_{1,i}^{(1)}=\\{\hat{\theta}^{(1)}_{2},\ldots,\hat{\theta}^{(1)}_{i}\\}$,
which can be used to repeat the entire refinement procedure.

Summarizing, the update of the $k$th AoA within the $n$th ($n\geq 1$)
procedure cycle has the following expression

$\hat{\theta}_{k}^{(n)}=\operatorname*{arg\,max}_{\theta_{k}\in\Omega,\
\theta_{k}\notin\widehat{\Theta}_{k,i}^{(n-1)}}\left\\{-L\log\lambda_{\max}\\{{\mbox{\boldmath$D$}}_{i}(\theta_{k},\widehat{\Theta}_{k,i}^{(n-1)})\\}\right.\\\
\left.+\lambda_{\max}\\{{\mbox{\boldmath$D$}}_{i}(\theta_{k},\widehat{\Theta}_{k,i}^{(n-1)})\\}\right\\},$
(36)

$1\leq k\leq i$, where

$\widehat{\Theta}_{k,i}^{(n-1)}=\left\\{\hat{\theta}_{1}^{(n)},\ldots\hat{\theta}_{k-1}^{(n)},\hat{\theta}_{k+1}^{(n-1)},\ldots,\hat{\theta}_{i}^{(n-1)}\right\\}$
(37)

and ${\mbox{\boldmath$D$}}_{i}(\theta_{k},\widehat{\Theta}_{k,i}^{(n-1)})$ is
defined as in (23) with
${\mbox{\boldmath$V$}}_{i}\left(\theta_{k},\widehat{\Theta}_{k,i}^{(n-1)}\right)=[{\mbox{\boldmath$v$}}_{t},{\mbox{\boldmath$v$}}(\hat{\theta}_{1}^{(n)}),\ldots,{\mbox{\boldmath$v$}}(\hat{\theta}_{k-1}^{(n)}),{\mbox{\boldmath$v$}}(\theta_{k}),{\mbox{\boldmath$v$}}(\hat{\theta}_{k+1}^{(n-1)}),\ldots,{\mbox{\boldmath$v$}}(\hat{\theta}_{i}^{(n-1)})].$
It is important to observe that the above procedure leads to a nondecreasing
sequence of log-likelihood function values, namely

$\displaystyle\mbox{$\mathcal{L}$}_{i}(\hat{\theta}_{1}^{(0)},\widehat{\Theta}_{1,i}^{(0)})\leq\mbox{$\mathcal{L}$}_{i}(\hat{\theta}_{1}^{(1)},\widehat{\Theta}_{1,i}^{(0)})=\mbox{$\mathcal{L}$}_{i}(\hat{\theta}_{2}^{(0)},\widehat{\Theta}_{2,i}^{(0)})\leq$
$\displaystyle\mbox{$\mathcal{L}$}_{i}(\hat{\theta}_{2}^{(1)},\widehat{\Theta}_{2,i}^{(0)})\leq\ldots\leq\mbox{$\mathcal{L}$}_{i}(\hat{\theta}_{k}^{(n)},\widehat{\Theta}_{k,i}^{(n-1)})\leq\ldots$
(38)

where

$\mbox{$\mathcal{L}$}_{i}(\hat{\theta}^{(n)}_{k},\widehat{\Theta}^{(m)}_{k,i})=-L\log\lambda_{\max}\\{{\mbox{\boldmath$D$}}_{i}(\hat{\theta}^{(n)}_{k},\widehat{\Theta}_{k,i}^{(m)})\\}\\\
+\lambda_{\max}\\{{\mbox{\boldmath$D$}}_{i}(\hat{\theta}^{(n)}_{k},\widehat{\Theta}_{k,i}^{(m)})\\}+L\log
L-L.$ (39)

The entire procedure may terminate when

$\Delta\mbox{$\mathcal{L}$}_{i}(n)=\frac{\left|\mbox{$\mathcal{L}$}_{i}(\hat{\theta}_{1}^{(n+1)},\widehat{\Theta}_{1,i}^{(n+1)})-\mbox{$\mathcal{L}$}_{i}(\hat{\theta}_{1}^{(n)},\widehat{\Theta}_{1,i}^{(n)})\right|}{\left|\mbox{$\mathcal{L}$}_{i}(\hat{\theta}_{1}^{(n+1)},\widehat{\Theta}_{1,i}^{(n+1)})\right|}<\epsilon,$
(40)

where $\epsilon>0$, or $n\geq N_{\max}$ where $N_{\max}$ is the maximum
allowable number of iterations.

Finally, as for the initialization of the procedure, given the received
vectors, we proceed by first computing

$r_{1}=\left|\frac{1}{L}\sum_{l=1}^{L}{\mbox{\boldmath$z$}}_{l}^{\dagger}{\mbox{\boldmath$v$}}(\omega_{1})\right|^{2},\ldots,r_{S}=\left|\frac{1}{L}\sum_{l=1}^{L}{\mbox{\boldmath$z$}}_{l}^{\dagger}{\mbox{\boldmath$v$}}(\omega_{S})\right|^{2},$
(41)

then, sort the above values in decreasing order, namely

$r_{k_{1}}\geq r_{k_{2}}\geq\ldots\geq r_{k_{S}},$ (42)

and select $\omega_{k_{1}},\ldots,\omega_{k_{i}}$. The behavior of the
proposed architectures coupled with the above estimation procedure is assessed
in the next section by means of numerical examples.

##  IV Illustrative Examples and Disussion

In this section, the performance of the proposed detection architectures is
investigated drawing upon synthetic data and considering three operating
scenarios. In the first case, only the target of interest is present, whereas
in the second scenario, an additional coherent signal is considered. Finally,
the third scenario contains the signal of interest along with two additional
coherent signals. The performance metrics are

  * •

the Probability of Detection under $H_{1,i}$, $i\geq 0$, ($P_{d,i}$) defined
as the probability of rejecting $H_{0}$ when the latter is false and given a
preassigned $P_{fa}=P(\mbox{reject }H_{0}|H_{0}\mbox{ is true})$;

  * •

the Probability of Correct Classification ($P_{cc}$), namely the probability
of deciding777The detection threshold is set according to the preassigned
$P_{fa}$ also in this case. for $H_{1,i}$ under $H_{1,i}$;

  * •

the Root Mean Square Error (RMSE) in angle whose expression is

$\sqrt{{\mathbb{E}}\left[\frac{1}{M}\sum_{m=1}^{M}\displaystyle\min_{k=1,\ldots,\hat{i}}\left\\{(\theta_{m}-\hat{\theta}_{k})^{2}\right\\}\right]},$
(43)

where $\hat{\theta}_{k}$, $k=1,\ldots,\hat{i}$, are the AoA estimates of the
coherent signals when $H_{1,\hat{i}}$ is declared. Since deriving closed-form
expressions for the above quantities represents a mathematically intractable
task (at least to the best of authors’ knowledge), we estimate them resorting
to the Monte Carlo counting techniques (also replacing the statistical
expectation of the RMSE with the sample mean over the Monte Carlo trials).
More precisely, we exploit $10^{3}$ independent trials to estimate the
considered performance metrics, whereas the detection thresholds are set over
$100/P_{fa}$, with $P_{fa}=10^{-3}$.

As stated before, the analysis starts from the conventional case where only
the target echoes impinge on the radar and proceeds with two more difficult
cases that assume the presence of two and three coherent signals (including
that of interest), respectively. As for the operating setup, the coherent
signals share the same power as the target signal, whereas the angular sector
under surveillance ranges from $-25$ to $25$ degrees and is sampled at $1$
degree. The Signal-to-Interference-plus-Noise Ratio (SINR) is defined as
$\mbox{SINR}=\sigma_{\alpha}^{2}{\mbox{\boldmath$v$}}_{t}^{\dagger}{\mbox{\boldmath$M$}}^{-1}{\mbox{\boldmath$v$}}_{t}$,
where
${\mbox{\boldmath$M$}}=\sigma_{n}^{2}{\mbox{\boldmath$I$}}+\mbox{CNR}\,{\mbox{\boldmath$M$}}_{c}$,
$\mbox{CNR}=20$ dB is the Clutter-to-Noise Ratio, $\sigma^{2}_{n}=1$ is the
noise power, and ${\mbox{\boldmath$M$}}_{c}$ is the clutter covariance matrix
whose $(i,j)$th entry is defined as
${\mbox{\boldmath$M$}}_{c}(i,j)=\rho_{c}^{|i-j|}$ with $\rho_{c}=0.9$ the one-
lag correlation coefficient. Moreover, all the numerical examples assume
$N=16$, $L=32$, $\theta_{t}=0^{\circ}$, and $M_{c}=4$.

As for the angular positions of the coherent signals, we begin the analysis
assuming that they belong to the search grid. In this case, the obtained
curves represent an upper bound on the performance that can be attained
exploiting fine and fine search grids (possibly considering the previously
described iterative maximization procedure). Then, we investigate the behavior
of the proposed architectures when the actual positions of the coherent
signals are in between the points of the search grid. Specifically, we
generate them as uniformly distributed in intervals of different sizes and
centered on the nominal search grid points (a point better explained below).

Finally, we compare the proposed architectures with well-known decision
schemes, namely the Generalized Adaptive Matched Filter (GAMF) and the
Generalized Adaptive Subspace Detector (GASD) given by [41]

$\displaystyle\sum_{l=1}^{L}\frac{|{\mbox{\boldmath$v$}}_{t}^{\dagger}\widehat{{\mbox{\boldmath$M$}}}^{-1}{\mbox{\boldmath$z$}}_{l}|^{2}}{{\mbox{\boldmath$v$}}_{t}^{\dagger}\widehat{{\mbox{\boldmath$M$}}}^{-1}{\mbox{\boldmath$v$}}_{t}}\mbox{$\begin{array}[]{c}\stackrel{{\scriptstyle\stackrel{{\scriptstyle\textstyle
H_{1,0}}}{{\textstyle>}}}}{{\stackrel{{\scriptstyle\textstyle<}}{{\textstyle
H_{0}}}}}\end{array}$}\eta\quad\mbox{and}$ (45)
$\displaystyle\sum_{l=1}^{L}\frac{|{\mbox{\boldmath$v$}}_{t}^{\dagger}\widehat{{\mbox{\boldmath$M$}}}^{-1}{\mbox{\boldmath$z$}}_{l}|^{2}}{{\mbox{\boldmath$v$}}_{t}^{\dagger}\widehat{{\mbox{\boldmath$M$}}}^{-1}{\mbox{\boldmath$v$}}_{t}\sum_{m=1}^{L}{\mbox{\boldmath$z$}}_{m}^{\dagger}\widehat{{\mbox{\boldmath$M$}}}^{-1}{\mbox{\boldmath$z$}}_{m}}\mbox{$\begin{array}[]{c}\stackrel{{\scriptstyle\stackrel{{\scriptstyle\textstyle
H_{1,0}}}{{\textstyle>}}}}{{\stackrel{{\scriptstyle\textstyle<}}{{\textstyle
H_{0}}}}}\end{array}$}\eta,$ (47)

respectively.

Before presenting the detection and classification results for each of the
aforementioned cases, it is important to assess the convergence rate of the
estimation procedure under each hypothesis as well as the sensitivity of the
$P_{fa}$ with respect to $\rho_{c}$ and CNR. To this end, in Figure 2, we plot
the root mean square values for $\Delta\mbox{$\mathcal{L}$}_{i}(n)$,
$i=2,3,4$, versus $n$ under $H_{0}$, $H_{1,0}$, $H_{1,1}$, and $H_{1,2}$; as
for $H_{1,1}$ and $H_{1,2}$, we set $\theta_{1}=10^{\circ}$ and
$\theta_{2}=18^{\circ}$. Inspection of the figure highlights that a number of
iterations $n=5$ is enough to guarantee $\epsilon\leq 10^{-3}$ under each
hypothesis. As for the $P_{fa}$ behavior, in Figure 3, we estimate it for
different values of $\rho_{c}$ given CNR (subplot (a)) and for different
values of CNR given $\rho_{c}$ (subplot (b)) when the thresholds are computed
assuming the nominal values for these parameters (namely, $\rho_{c}=0.9$ and
$\mbox{CNR}=20$ dB) and $P_{fa}=10^{-3}$. It turns out that all the considered
architectures can guarantee $P_{fa}$ values contained within the interval
$[0.0007,\ 0.0011]$ providing a rather robust behavior to the considered
parameter variations.

Figure 2: Convergence curves under $H_{0}$, $H_{1,0}$, $H_{1,1}$, and
$H_{1,2}$ assuming $N=16$, $L=32$, $K=32$, $\theta_{1}=10^{\circ}$, and
$\theta_{2}=18^{\circ}$. Figure 3: $P_{fa}$ versus $\rho_{c}$ (subplot (a))
and CNR (subplot (b)) for the GAMF, GASD, AIC-D, BIC-D, and GIC-D with
different values of $\rho$ assuming $N=16$, $L=32$, and $K=32$.

In the next three figures, we investigate the detection and classification
performance when the conventional alternative hypothesis $H_{1,0}$, which
contemplates the presence of the signal of interest only (first scenario), is
in force. Specifically, in Figure 4, we show the $P_{d,0}$ curves as functions
of the SINR assuming $N=16$, $L=32$, $K=32$, and different values for $\rho$.
This preliminary analysis allows us to quantify the sensitivity of GIC-D with
respect to its tuning parameter in comparison with the other considered
architectures. It turns out that increasing $\rho$ leads to improved detection
performances for GIC-D, whose loss with respect to both the GAMF and GASD,
which overcome the other architectures, ranges from about $1.9$ dB for
$\rho=20$ to about $0.5$ dB for $\rho=80$ at $P_{d,0}=0.9$. On the other hand,
AIC-D and BIC-D experience a loss of about $2.2$ dB with respect to GAMF and
GASD at $P_{d,0}=0.9$. In Figure 5, we analyze the effects of $K$ on the
performance by doubling the value used in Figure 4. Moreover, in order to
quantify the loss due to the estimation of $M$ we also report the curves of
the clairvoyant detectors which assume that $M$ is known. The most evident
change with respect to the previous figure is that all the detection curves
clearly move towards the left part of the plot, namely a significant
improvement in performance occurs. In addition, GIC-D with $\rho=80$, GAMF,
and GASD almost share the same performance while AIC-D and BIC-D continue to
exhibit a loss of about $2$ dB at $P_{d,0}=0.9$ with respect to the former.
The figure also highlights that the loss associated to the estimation of $M$
is of about $3.5$ dB (at $P_{d,0}=0.9$). In the last figure of this first
study case (Figure 6), we analyze the classification capabilities of the
proposed architectures through the probabilities of classification ($P_{c}$)
and the $P_{cc}$ estimated at two different SINR values. The figure clearly
highlights the inclination of AIC-D to overestimate the hypothesis model,
whereas the penalty term of BIC-D allows to mitigate this effect even though
the $P_{cc}$ is less than $0.4$. On the other hand, GIC-D can be suitably
tuned in order to attain satisfactory classification performance.
Specifically, when $\rho=80$, GIC-D correctly classifies the environment with
a probability very close to $1$.

Figure 4: $P_{d,0}$ versus SINR for the GAMF, GASD, AIC-D, BIC-D, and GIC-D
with different values of $\rho$ assuming $N=16$, $L=32$, and $K=32$. Figure 5:
$P_{d,0}$ versus SINR for the GAMF, GASD, AIC-D, BIC-D, GIC-D with $\rho=80$,
AIC-D with known $M$, BIC-D with known $M$, and GIC-D with known $M$ and
$\rho=80$ assuming $N=16$, $L=32$, and $K=64$. Figure 6: Classification
probabilities for the AIC-D, BIC-D, GIC-D with $\rho=20$, and GIC-D with
$\rho=80$ under $H_{1,0}$ assuming $N=16$, $L=32$, and $K=32$. Figure 7:
$P_{d,1}$ versus SINR for the GAMF, GASD, AIC-D, BIC-D, and GIC-D with
different values of $\rho$ assuming $N=16$, $L=32$, $K=32$, and
$\theta_{1}=10^{\circ}$. Figure 8: $P_{d,1}$ versus SINR for the GAMF, GASD,
AIC-D, BIC-D, and GIC-D assuming different values of $\theta_{1}$, $N=16$,
$L=32$, and $K=32$. Figure 9: $P_{d,1}$ versus SINR for the GAMF, GASD, AIC-D,
AIC-D with known parameters, AIC-D with known $M$ only, AIC-D with known AoA
only, BIC-D, BIC-D with known parameters, BIC-D with known $M$ only, BIC-D
with known AoA only, GIC-D with $\rho=80$, GIC-D with $\rho=80$ and known
parameters, GIC-D with $\rho=80$ and known $M$ only, and GIC-D with $\rho=80$
and known AoA only, assuming $N=16$, $L=32$, $K=64$, and
$\theta_{1}=10^{\circ}$. Figure 10: Classification probabilities for the
AIC-D, BIC-D, and GIC-D with $\rho=20,80$ under $H_{1,1}$ assuming $N=16$,
$L=32$, $K=32$, and $\theta_{1}=10^{\circ}$. Figure 11: RMSE in angle versus
SINR for the AIC-D, BIC-D, and GIC-D with $\rho=20,80$ under $H_{1,1}$
assuming $N=16$, $L=32$, $K=32$, and $\theta_{1}=10^{\circ}$.

Now, we address the intermediate scenario, where besides the signal of
interest, a coherent signal enters the antenna from an angular direction
$\theta_{1}$. The hypothesis corresponding to this scenario is $H_{1,1}$. In
Figure 7, we plot $P_{d,1}$ versus SINR for $N=16$, $L=32$, $K=32$, and
$\theta_{1}=10^{\circ}$; moreover, several values of $\rho$ are considered.
Notice that in the presence of the additional coherent signal, GAMF and GASD
are no longer at the top of the performance ranking with the GASD not capable
of providing a $P_{d,1}$ above $0.1$ due to its selective behavior [12], at
least for the considered parameter values. On the other hand, the remaining
detectors share about the same performance with a gain of about $10$ dB with
respect to the GAMF; observe also that GIC-D slightly improves its performance
as $\rho$ grows. In the next figure, we assess the behavior of the considered
decision schemes when $\theta_{1}$ takes on different values. Specifically, we
set $\theta_{1}=15^{\circ},20^{\circ}$, whereas the remaining parameters are
the same as in the previous figure except for $\rho=80$. The figure shows that
the performance of the proposed architectures exhibits a gain of about $2$ dB
(at $P_{d,1}=0.9$) when the angular separation between the coherent signals
increases with the GIC-D slightly outperforming the other schemes, whereas the
performance of the GAMF and GASD degrades due to the fact that the composition
of the coherent signals results in a direction that moves away from the
nominal steering angle. In Figure 9, we account for a different value of $K$
leaving unaltered the parameters $N,L$ and also quantify the loss due to the
estimation of the AoA and/or $M$. As expected, increasing $K$ improves the
performance of all detectors. Moreover, it turns out that estimating the AoA
is not as crucial as the estimation of the covariance matrix since the loss
associated with the estimation of AoA is of about $1$ dB at $P_{d,1}=0.9$ and
with respect to the architectures where all the parameters are known, by
contrast that associated with the estimation of $M$ is approximately $3$ dB.
Again, GIC-D with $\rho=80$ slightly overcomes the other proposed schemes. In
the last two figures related to this intermediate scenario, we show the
classification and estimation performance. Specifically, Figure 10 contains
the classification histograms, whereas the curves of RMSE in angle versus the
SINR are presented in Figure 11. The former figure confirms the behavior
observed in Figure 6 and, hence, the superior performance of GIC-D with
respect to AIC-D and BIC-D that are inclined to overestimate the number of
coherent signals. As for the RMSE, all the proposed architectures share almost
the same performance for SINR$\geq-5$ dB. The main differences occur for low
SINR values where, based upon (43), the overestimation of the number of
coherent signals for the AIC-D and BIC-D leads to lower RMSE values than
GIC-D. As a matter of fact, given (43) and for low SINR values, it is more
likely to draw an estimate from the set provided by AIC-D/BIC-D that is in a
narrower neighborhood of a true angular position with respect to all the
estimates provided by GIC-D. As a consequence, the resulting error for
AIC-D/BIC-D takes on lower values than the error for GIC-D.

Finally, the last scenario is the most challenging since it encompasses the
presence of two coherent signals with nominal positions
$\theta_{1}=10^{\circ}$ and $\theta_{2}=18^{\circ}$ in addition to the signal
of interest. Besides, we consider two situations that differ in the actual
positions of the coherent signals. In the first situation, they are exactly
located at $\theta_{1}$ and $\theta_{2}$, whereas in the second case, the
positions of the two coherent signals are uniformly generated in the intervals
$[\theta_{1}-\Delta\theta,\ \theta_{1}+\Delta\theta]$ and
$[\theta_{2}-\Delta\theta,\ \theta_{2}+\Delta\theta]$, where
$\Delta\theta\in\\{0.3^{\circ},0.5^{\circ}\\}$. The detection performance for
matched signals is shown in Figure 12, where we also plot the architectures
that assume partial/full knowledge of the parameter values. Moreover, we
compare the results obtained through the exhaustive search for the AoA
estimation (left subplot) with those provided by the proposed suboptimum
procedure (right subplot). The figure highlights that the search procedures
return detection curves that are very close to each other. In addition, the
same remarks for Figure 9 also hold in this case. The classification
performance and the RMSE curves for both the exhaustive and suboptimum search
procedures are shown in Figure 13-15. Inspection of these figures confirms the
excellent classification capabilities of GIC-D, whereas AIC-D and BIC-D can
provide better AoA estimates than GIC-D for low SINR values ($\leq-10$ dB). In
addition, it is important to underline that there does not exist a valuable
difference in performance between the exhaustive search and the suboptimum
search algorithm. Finally, the last four figures assess the behavior of the
considered architectures (coupled with the suboptimum search procedure) when
the AoAs of the coherent signals are uniformly generated at each Monte Carlo
trial in an interval of length $2\Delta\theta$ and centered around the nominal
positions. Figures 16 and 17 assume $\Delta\theta=0.3$, whereas in Figures 18
and 19, we set $\Delta\theta=0.5$. These numerical examples highlight that,
from the detection point of view, the mismatch between the search grid points
and the actual positions of the coherent signals leads to a slight performance
deterioration, that is more noticeable for $\Delta\theta=0.5$, while leaving
the previously observed hierarchy unaltered. Note that we do not assume any
mismatch related to the signal of interest since it requires a different
performance analysis that accounts for the mismatch degree and is out of the
scope of the present work. As for the classification performance, AIC-D and
BIC-D continue to exhibit a clear inclination to overestimate the number of
signals, while GIC-D with $\rho=20$ returns a probability of correct
classification lower than that for matched signals. Finally, the
classification performance of GIC-D with $\rho=80$ is not degraded by the
coherent signal mismatches.

Summarizing, the above analysis has singled out GIC-D with $\rho=80$ as the
recommended detection architecture capable of providing a satisfactory
detection and classification performance in the presence of coherent signals
at least for the considered scenarios.

##  V Conclusions

In this paper, we focused on the adaptive radar detection in the presence of
fully correlated signals besides that of interest. Such additional signals may
be due to multipath propagation effects or to the action of malicious
platforms (smart jammers). In order to account for different operating
scenarios, at the design stage, we have considered a multiple-hypothesis test
that also includes the classical radar signal-plus-interference hypothesis and
devised likelihood-ratio-based decision schemes whose statistics under a
specific hypothesis depend on a suitable penalty factor tuned according to the
number of unknown parameters under that hypothesis (leveraging the approach of
the MOS rules). As a result, such architectures are provided with
classification capabilities returning, as a byproduct, an estimate of the
number of coherent signals impinging on the radar system. The performance
analysis has been carried out resorting to simulated data considering three
different scenarios with an increasing number of coherent signals. Moreover,
for comparison purposes, the curves for the GAMF and GASD have been also
reported. The analysis has singled out the GIC-D with $\rho=80$ as the
recommended detection architecture since it overcomes the remaining proposed
decision schemes in terms of both detection and classification performance.

Future research tracks may include the design of (possibly space-time)
processing architectures that account for coherent signals spread along the
range dimension or aimed at operating in scenarios where multiple coherent
and/or uncorrelated signals are present.

Figure 12: $P_{d,2}$ versus SINR for the GAMF, GASD, AIC-D, AIC-D with known
parameters, AIC-D with known $M$ only, AIC-D with known AoA only, BIC-D, BIC-D
with known parameters, BIC-D with known $M$ only, BIC-D with known AoA only,
GIC-D with $\rho=20,80$, GIC-D with $\rho=20,80$ and known parameters, GIC-D
with $\rho=20,80$ and known $M$ only, and GIC-D with $\rho=20,80$ and known
AoA only, exhaustive grid search (left subplot) and suboptimum search (right
subplot) assuming $N=16$, $L=32$, $K=32$, $\theta_{1}=10^{\circ}$, and
$\theta_{2}=18^{\circ}$. Figure 13: Classification probabilities for the
AIC-D, BIC-D, and GIC-D with $\rho=20,80$ under $H_{1,2}$ assuming $N=16$,
$L=32$, $K=32$, $\theta_{1}=10^{\circ}$, $\theta_{2}=18^{\circ}$, and the
exhaustive grid search. Figure 14: Classification probabilities for the AIC-D,
BIC-D, and GIC-D with $\rho=20,80$ under $H_{1,2}$ assuming $N=16$, $L=32$,
$K=32$, $\theta_{1}=10^{\circ}$, $\theta_{2}=18^{\circ}$, and the suboptimum
search. Figure 15: RMSE in angle versus SINR for the AIC-D, BIC-D, and GIC-D
with $\rho=20,80$ under $H_{1,2}$ assuming $N=16$, $L=32$, $K=32$,
$\theta_{1}=10^{\circ}$, $\theta_{2}=18^{\circ}$, exhaustive search grid (left
subplot), and suboptimum search (right subplot). Figure 16: $P_{d,1}$ versus
SINR (left subplot) and $P_{d,2}$ versus SINR (right subplot) for the GAMF,
GASD, AIC-D, BIC-D, and GIC-D with $\rho=20,80$ when the actual AoAs of the
coherent signals are uniformly generated in between $\theta_{i}-0.3^{\circ}$
and $\theta_{i}+0.3^{\circ}$, $i=1,2$, assuming $N=16$, $L=32$, $K=32$,
$\theta_{1}=10^{\circ}$, $\theta_{2}=18^{\circ}$, and the suboptimum search.
Figure 17: Classification probabilities for AIC-D, BIC-D, GIC-D with
$\rho=20,80$ under $H_{1,2}$ when the actual AoAs of the coherent signals are
uniformly generated in between $\theta_{i}-0.3^{\circ}$ and
$\theta_{i}+0.3^{\circ}$, $i=1,2$, assuming $N=16$, $L=32$, $K=32$,
$\theta_{1}=10^{\circ}$, $\theta_{2}=18^{\circ}$, and the suboptimum search.
Figure 18: $P_{d,1}$ versus SINR (left subplot) and $P_{d,2}$ versus SINR
(right subplot) for the GAMF, GASD, AIC-D, BIC-D, and GIC-D with $\rho=20,80$
when the actual AoAs of the coherent signals are uniformly generated in
between $\theta_{i}-0.5^{\circ}$ and $\theta_{i}+0.5^{\circ}$, $i=1,2$,
assuming $N=16$, $L=32$, $K=32$, $\theta_{1}=10^{\circ}$,
$\theta_{2}=18^{\circ}$, and the suboptimum search. Figure 19: Classification
probabilities for AIC-D, BIC-D, GIC-D with $\rho=20,80$ under $H_{1,2}$ when
the actual AoAs of the coherent signals are uniformly generated in between
$\theta_{i}-0.5^{\circ}$ and $\theta_{i}+0.5^{\circ}$, $i=1,2$, assuming
$N=16$, $L=32$, $K=32$, $\theta_{1}=10^{\circ}$, $\theta_{2}=18^{\circ}$, and
the suboptimum search.

## References

  * [1] M. A. Richards, J. A. Scheer, and W. A. Holm, _Principles of Modern Radar: Basic Principles_. Raleigh, NC: Scitech Publishing, 2010.
  * [2] W. L. Melvin and J. A. Scheer, _Principles of Modern Radar: Advanced Techniques_ , S. Publishing, Ed., Edison, NJ, 2013.
  * [3] M. A. Richards, W. L. Melvin, J. A. Scheer, and W. A. Holm, _Principles of Modern Radar: Radar Applications, Volume 3_ , ser. Electromagnetics and Radar. Institution of Engineering and Technology, 2013.
  * [4] E. J. Kelly, “An adaptive detection algorithm,” _IEEE Transactions on Aerospace and Electronic Systems_ , vol. 22, no. 2, pp. 115–127, 1986.
  * [5] E. J. Kelly and K. Forsythe, “Adaptive Detection and Parameter Estimation for Multidimensional Signal Models,” Lincoln Lab, MIT, Lexington, US, Technical Report 848, 1989.
  * [6] W. L. Melvin, “Space-time Adaptive Radar Performance in Heterogeneous Clutter,” _IEEE Transactions on Aerospace and Electronic Systems_ , vol. 36, no. 2, pp. 621–633, 2000.
  * [7] F. C. Robey, D. R. Fuhrmann, E. J. Kelly, and R. Nitzberg, “A CFAR adaptive matched filter detector,” _IEEE Transactions on Aerospace and Electronic Systems_ , vol. 28, no. 1, pp. 208–216, 1992.
  * [8] F. Gini and A. Farina, “Vector Subspace Detection in Compound-Gaussian Clutter Part I: Survey and New Results,” _IEEE Transactions on Aerospace and Electronic Systems_ , vol. 38, no. 4, pp. 1295–1311, 2002.
  * [9] D. Orlando and G. Ricci, “A Rao Test With Enhanced Selectivity Properties in Homogeneous Scenarios,” _IEEE Transactions on Signal Processing_ , vol. 58, no. 10, pp. 5385–5390, 2010.
  * [10] W. Liu, W. Xie, and Y. Wang, “Rao and Wald Tests for Distributed Targets Detection With Unknown Signal Steering,” _IEEE Signal Processing Letters_ , vol. 20, no. 11, pp. 1086–1089, 2013.
  * [11] Y. I. Abramovich and B. A. Johnson, “GLRT-based detection-estimation for undersampled training conditions,” _IEEE Transactions on Signal Processing_ , vol. 56, no. 8, pp. 3600–3612, 2008.
  * [12] F. Bandiera, D. Orlando, and G. Ricci, _Advanced Radar Detection Schemes Under Mismatched Signal Models_. San Rafael, US: Synthesis Lectures on Signal Processing No. 8, Morgan & Claypool Publishers, 2009.
  * [13] W. Liu, W. Xie, J. Liu, and Y. Wang, “Adaptive Double Subspace Signal Detection in Gaussian Background—Part I: Homogeneous Environments,” _IEEE Transactions on Signal Processing_ , vol. 62, no. 9, pp. 2345–2357, 2014.
  * [14] W. Liu, J. Liu, Y. Gao, G. Wang, and Y. L. Wang, “Multichannel signal detection in interference and noise when signal mismatch happens,” _Signal Processing_ , vol. 166, p. 107268, 2020.
  * [15] J. Liu, S. Sun, and W. Liu, “One-step persymmetric GLRT for subspace signals,” _IEEE Transaction on Signal Processing_ , vol. 14, no. 67, pp. 3639–3648, July 15 2019.
  * [16] C. Hao, S. Gazor, G. Foglia, B. Liu, and C. Hou, “Persymmetric adaptive detection and range estimation of a small target,” _IEEE Transactions on Aerospace and Electronic Systems_ , vol. 51, no. 4, pp. 2590–2604, 2015.
  * [17] G. Foglia, C. Hao, A. Farina, G. Giunta, D. Orlando, and C. Hou, “Adaptive Detection of Point-Like Targets in Partially Homogeneous Clutter With Symmetric Spectrum,” _IEEE Transactions on Aerospace and Electronic Systems_ , vol. 53, no. 4, pp. 2110–2119, 2017.
  * [18] C. Hao, D. Orlando, G. Foglia, and G. Giunta, “Knowledge-Based Adaptive Detection: Joint Exploitation of Clutter and System Symmetry Properties,” _IEEE Signal Processing Letters_ , vol. 23, no. 10, pp. 1489–1493, October 2016.
  * [19] G. Foglia, C. Hao, G. Giunta, and D. Orlando, “Knowledge-aided adaptive detection in partially homogeneous clutter: Joint exploitation of persymmetry and symmetric spectrum,” _Digital Signal Processing_ , vol. 67, no. Supplement C, pp. 131 – 138, 2017.
  * [20] P. Wang, Z. Sahinoglu, M. Pun, and H. Li, “Persymmetric Parametric Adaptive Matched Filter for Multichannel Adaptive Signal Detection,” _IEEE Transactions on Signal Processing_ , vol. 60, no. 6, pp. 3322–3328, 2012.
  * [21] D. K. Barton, _Radar Equations for Modern Radar_ , ser. Artech House radar library. Artech House, 2013.
  * [22] D. J. Bachmann, R. J. Evans, and B. Moran, “Game Theoretic Analysis of Adaptive Radar Jamming,” _IEEE Transactions on Aerospace and Electronic Systems_ , vol. 47, no. 2, pp. 1081–1100, 2011.
  * [23] H. Gao, J. Wang, C. Jiang, and X. Zhang, “Equilibrium between a statistical MIMO radar and a jammer,” in _2015 IEEE Radar Conference (RadarCon)_ , 2015, pp. 0461–0466.
  * [24] D. C. Schleher, _Electronic Warfare in the Information Age_ , ser. Artech House radar library. Artech House, 1999\.
  * [25] F. Neri, _Introduction to Electronic Defense Systems_. Norwood, MA: Artech House, 2018.
  * [26] N. Yuen and B. Friedlander, “DOA estimation in multipath: an approach using fourth-order cumulants,” _IEEE Transactions on Signal Processing_ , vol. 45, no. 5, pp. 1253–1263, 1997.
  * [27] A. J. Weiss and B. Friedlander, “Performance analysis of spatial smoothing with interpolated arrays,” _IEEE Transactions on Signal Processing_ , vol. 41, no. 5, pp. 1881–1892, 1993.
  * [28] Tie-Jun Shan and T. Kailath, “Adaptive beamforming for coherent signals and interference,” _IEEE Transactions on Acoustics, Speech, and Signal Processing_ , vol. 33, no. 3, pp. 527–536, 1985.
  * [29] H. L. Van Trees, _Optimum Array Processing (Detection, Estimation, and Modulation Theory, Part IV)_. John Wiley & Sons, 2002.
  * [30] Tie-Jun Shan, M. Wax, and T. Kailath, “On spatial smoothing for direction-of-arrival estimation of coherent signals,” _IEEE Transactions on Acoustics, Speech, and Signal Processing_ , vol. 33, no. 4, pp. 806–811, 1985.
  * [31] P. Stoica and R. Moses, _Spectral Analysis of Signals_. Pearson Prentice Hall, 2005.
  * [32] F. Ansari, M. Taban, and S. Gazor, “A novel sequential algorithm for clutter and direct signal cancellation in passive bistatic radars,” _EURASIP Journal on Advances in Signal Processing_ , no. 134, 2016.
  * [33] B. Feng, T. Wang, C. Liu, C. Chen, and W. Chen, “An effective CLEAN algorithm for interference cancellation and weak target detection in passive radar,” in _Conference Proceedings of 2013 Asia-Pacific Conference on Synthetic Aperture Radar (APSAR)_ , 2013, pp. 160–163.
  * [34] K. Kulpa, “The CLEAN type algorithms for radar signal processing,” in _2008 Microwaves, Radar and Remote Sensing Symposium_ , 2008, pp. 152–157.
  * [35] M. Viberg, B. Ottersten, and T. Kailath, “Detection and estimation in sensor arrays using weighted subspace fitting,” _IEEE Transactions on Signal Processing_ , vol. 39, no. 11, pp. 2436–2449, 1991.
  * [36] I. Ziskind and M. Wax, “Maximum likelihood localization of multiple sources by alternating projection,” _IEEE Transactions on Acoustics, Speech, and Signal Processing_ , vol. 36, no. 10, pp. 1553–1560, 1988.
  * [37] X. Tan, W. Roberts, J. Li, and P. Stoica, “Sparse Learning via Iterative Minimization With Application to MIMO Radar Imaging,” _IEEE Transactions on Signal Processing_ , vol. 59, no. 3, pp. 1088–1101, March 2011\.
  * [38] L. Yan, P. Addabbo, C. Hao, D. Orlando, and A. Farina, “New ECCM Techniques Against Noiselike and/or Coherent Interferers,” _IEEE Transactions on Aerospace and Electronic Systems_ , vol. 56, no. 2, pp. 1172–1188, 2020.
  * [39] P. Stoica and Y. Selen, “Model-order selection: A review of information criterion rules,” _IEEE Signal Processing Magazine_ , vol. 21, no. 4, pp. 36–47, 2004.
  * [40] P. Stoica and P. Babu, “On the Proper Forms of BIC for Model Order Selection,” _IEEE Transactions on Signal Processing_ , vol. 60, no. 9, pp. 4956–4961, September 2012.
  * [41] E. Conte, A. De Maio, and G. Ricci, “GLRT-based adaptive detection algorithms for range-spread targets,” _IEEE Transactions on Signal Processing_ , vol. 49, no. 7, pp. 1336–1348, July 2001.
  * [42] P. Addabbo, D. Orlando, and G. Ricci, “Adaptive Radar Detection of Dim Moving Targets in Presence of Range Migration,” _IEEE Signal Processing Letters_ , vol. 26, no. 10, pp. 1461–1465, Oct 2019.
  * [43] R. A. Horn and C. R. Johnson, _Matrix Analysis_ , C. U. Press, Ed., 1985.
  * [44] P. Stoica and Y. Selen, “Cyclic minimizers, majorization techniques, and the expectation-maximization algorithm: a refresher,” _IEEE Signal Processing Magazine_ , vol. 21, no. 1, pp. 112–114, 2004.