# CorefQA: Coreference Resolution as Query-based Span Prediction

Wei Wu♣, Fei Wang♣, Arianna Yuan◆♣, Fei Wu♠ and Jiwei Li♠♣
♠ Department of Computer Science and Technology, Zhejiang University
◆ Computer Science Department, Stanford University
♣ ShannonAI
<EMAIL_ADDRESS><EMAIL_ADDRESS>
{wei_wu<EMAIL_ADDRESS>
###### Abstract

In this paper, we present CorefQA, an accurate and extensible approach for the
coreference resolution task. We formulate the problem as a span prediction
task, like in question answering: A query is generated for each candidate
mention using its surrounding context, and a span prediction module is
employed to extract the text spans of the coreferences within the document
using the generated query. This formulation comes with the following key
advantages: (1) The span prediction strategy provides the flexibility of
retrieving mentions left out at the mention proposal stage; (2) In the
question answering framework, encoding the mention and its context explicitly
in a query makes it possible to have a deep and thorough examination of cues
embedded in the context of coreferent mentions; and (3) A plethora of existing
question answering datasets can be used for data augmentation to improve the
model’s generalization capability. Experiments demonstrate significant
performance boost over previous models, with 83.1 (+3.5) F1 score on the
CoNLL-2012 benchmark and 87.5 (+2.5) F1 score on the GAP benchmark.
111https://github.com/ShannonAI/CorefQA

##  1 Introduction

Recent coreference resolution systems (Lee et al., 2017, 2018; Zhang et al.,
2018a; Kantor and Globerson, 2019) consider all text spans in a document as
potential mentions and learn to find an antecedent for each possible mention.
There are two key issues with this paradigm, in terms of task formalization
and the algorithm.

At the task formalization level, mentions left out at the mention proposal
stage can never be recovered since the downstream module only operates on the
proposed mentions. Existing models often suffer from mention proposal (Zhang
et al., 2018a). The coreference datasets can only provide a weak signal for
spans that correspond to entity mentions because singleton mentions are not
explicitly labeled. Due to the inferiority of the mention proposal model, it
would be favorable if a coreference framework had a mechanism to retrieve
left-out mentions.

Original Passage In addition , many people were poisoned when toxic gas was
released. They were poisoned and did not know how to protect themselves
against the poison. Converted Questions Q1: Who were poisoned when toxic gas
was released? A1: [They, themselves] Q2: What was released when many people
were poisoned? A2: [the poison] Q3: Who were poisoned and did not know how to
protect themselves against the poison? A3: [many people, themselves] Q4: Whom
did they not know how to protect against the poison? A4: [many people, They]
Q5: They were poisoned and did not know how to protect themselves against
what? A5: [toxic gas]

Figure 1: An illustration of the paradigm shift from coreference resolution to
query-based span prediction. Spans with the same color represent coreferent
mentions. Note that we use a more direct strategy to generate the questions
based on the mentions.

At the algorithm level, existing end-to-end methods (Lee et al., 2017, 2018;
Zhang et al., 2018a) score each pair of mentions only based on mention
representations from the output layer of a contextualization model. This means
that the model lacks the connection between mentions and their contexts.
Semantic matching operations between two mentions (and their contexts) are
performed only at the output layer and are relatively superficial. Therefore
it is hard for their models to capture all the lexical, semantic and syntactic
cues in the context.

To alleviate these issues, we propose CorefQA, a new approach that formulates
the coreference resolution problem as a span prediction task, akin to the
question answering setting. A query is generated for each candidate mention
using its surrounding context, and a span prediction module is further
employed to extract the text spans of the coreferences within the document
using the generated query. Some concrete examples are shown in Figure 1.
222This is an illustration of the question formulation. The actual operation
is described in Section 3.4.

This formulation provides benefits at both the task formulation level and the
algorithm level. At the task formulation level, since left-out mentions can
still be retrieved at the span prediction stage, the negative effect of
undetected mentions is significantly alleviated. At the algorithm level, by
generating a query for each candidate mention using its surrounding context,
the CorefQA model explicitly considers the surrounding context of the target
mentions, the influence of which will later be propagated to each input word
using the self-attention mechanism. Additionally, unlike existing end-to-end
methods (Lee et al., 2017, 2018; Zhang et al., 2018a), where the interactions
between two mentions are only superficially modeled at the output layer of
contextualization, span prediction requires a more thorough and deeper
examination of the lexical, semantic and syntactic cues within the context,
which will potentially lead to better performance.

Moreover, the proposed question answering formulation allows us to take
advantage of existing question answering datasets. Coreference annotation is
expensive, cumbersome and often requires linguistic expertise from annotators.
Under the proposed formulation, the coreference resolution has the same format
as the existing question answering datasets (Rajpurkar et al., 2016a, 2018;
Dasigi et al., 2019a). Those datasets can thus readily be used for data
augmentation. We show that pre-training on existing question answering
datasets improves the model’s generalization and transferability, leading to
additional performance boost.

Experiments show that the proposed framework significantly outperforms
previous models on two widely-used datasets. Specifically, we achieve new
state-of-the-art scores of 83.1 (+3.5) on the CoNLL-2012 benchmark and 87.5
(+2.5) on the GAP benchmark.

##  2 Related Work

###  2.1 Coreference Resolution

Coreference resolution is a fundamental problem in natural language processing
and is considered as a good test of machine intelligence (Morgenstern et al.,
2016). Neural network models have shown promising results over the years.
Earlier neural-based models (Wiseman et al., 2016; Clark and Manning, 2015,
2016) rely on parsers and hand-engineered mention proposal algorithms. Recent
work (Lee et al., 2017, 2018; Kantor and Globerson, 2019) tackled the problem
in an end-to-end fashion by jointly detecting mentions and predicting
coreferences. Based on how entity-level information is incorporated, they can
be further categorized as (1) entity-level models Björkelund and Kuhn (2014);
Clark and Manning (2015, 2016); Wiseman et al. (2016) that directly model the
representation of real-world entities and (2) mention-ranking models (Durrett
and Klein, 2013; Wiseman et al., 2015; Lee et al., 2017) that learn to select
the antecedent of each anaphoric mention. Our CorefQA model is essentially a
mention-ranking model, but we identify coreference using question answering.

Figure 2: The overall architecture of our CorefQA model. The input passage is
first fed into the Mention Proposal Module 3.3 to obtain candidate mentions.
Then the Mention Linking Module 3.4 is used to extract coreferent mentions
from the passage for each proposed mention. The coreference clusters are
obtained using the scores produced in the above two stages.

###  2.2 Formalizing NLP Tasks as question answering

Machine reading comprehension is a general and extensible task form. Many
tasks in natural language processing can be framed as reading comprehension
while abstracting away the task-specific modeling constraints.

McCann et al. (2018) introduced the decaNLP challenge, which converts a set of
10 core tasks in NLP to reading comprehension. He et al. (2015) showed that
semantic role labeling annotations could be solicited by using question-answer
pairs to represent the predicate-argument structure. Levy et al. (2017)
reduced relation extraction to answering simple reading comprehension
questions, yielding models that generalize better in the zero-shot setting. Li
et al. (2019a, b) cast the tasks of named entity extraction and relation
extraction as a reading comprehension problem. In parallel to our work,
Aralikatte et al. (2019) converted coreference and ellipsis resolution in a
question answering format, and showed the benefits of training joint models
for these tasks. Their models are built under the assumption that gold
mentions are provided at inference time, whereas our model does not need that
assumption – it jointly trains the mention proposal model and the coreference
resolution model in an end-to-end manner. Chada (2019) proposed an extractive
QA model for the resolution of ambiguous pronouns, and showed better results
on the GAP dataset by only fine-tuning the pre-trained BERT model.

###  2.3 Data Augmentation

Data augmentation is a strategy that enables practitioners to significantly
increase the diversity of data available for training models. Data
augmentation techniques have been explored in various fields such as question
answering (Talmor and Berant, 2019), text classification (Kobayashi, 2018) and
dialogue language understanding (Hou et al., 2018). In coreference resolution,
Zhao et al. (2018); Emami et al. (2019); Zhao et al. (2019) focused on
debiasing the gender bias problem; Aralikatte et al. (2019) explored the
effectiveness of joint modeling of ellipsis and coreference resolution. To the
best of our knowledge, we are the first to use existing question answering
datasets as data augmentation for coreference resolution.

##  3 Model

In this section, we describe our CorefQA model in detail. The overall
architecture is illustrated in Figure 2.

###  3.1 Notations

Given a sequence of input tokens $X=\\{x_{1},x_{2},...,x_{n}\\}$ in a
document, where $n$ denotes the length of the document. $N=n*(n+1)/2$ denotes
the number of all possible text spans in $X$. Let $e_{i}$ denotes the $i$-th
span representation $1\leq i\leq N$, with the start index first(i) and the end
index last(i).
$e_{i}=\\{x_{\textsc{first}(i)},x_{\textsc{first}(i)+1},...,x_{\textsc{last}(i)-1},x_{\textsc{last}(i)}\\}$.
The task of coreference resolution is to determine the antecedents for all
possible spans. If a candidate span $e_{i}$ does not represent an entity
mention or is not coreferent with any other mentions, a dummy token $\epsilon$
is assigned as its antecedent. The linking between all possible spans $e$
defines the final clustering.

###  3.2 Input Representations

We use the SpanBERT model 333https://github.com/facebookresearch/SpanBERT to
obtain input representations following Joshi et al. (2019a). Each token
$x_{i}$ is associated with a SpanBERT representation $\bm{x}_{i}$. Since the
speaker information is indispensable for coreference resolution, previous
methods Wiseman et al. (2016); Lee et al. (2017); Joshi et al. (2019a) usually
convert the speaker information into binary features indicating whether two
mentions are from the same speaker. However, we use a straightforward strategy
that directly concatenates the speaker’s name with the corresponding
utterance. This strategy is inspired by recent research in personalized
dialogue modeling that use persona information to represent speakers (Li et
al., 2016; Zhang et al., 2018b; Mazaré et al., 2018). In subsection 5.2, we
will empirically demonstrate its superiority over the feature-based method in
Lee et al. (2017).

To fit long documents into SpanBERT, we use a sliding-window approach that
creates a $T$-sized segment after every $T$/2 tokens. Segments are then passed
to the SpanBERT encoder independently. The final token representations are
derived by taking the token representations with maximum context.

###  3.3 Mention Proposal

Similar to Lee et al. (2017), our model considers all spans up to a maximum
length $L$ as potential mentions. To improve computational efficiency, we
further prune the candidate spans greedily during both training and
evaluation. To do so, the mention score of each candidate span consists of
three parts: (1) $\bm{x}_{\textsc{first}(i)}$ is the start of a span; (2)
$\bm{x}_{\textsc{last}(i)}$ is the end of a span; and (3)
$\bm{x}_{\textsc{first}(i)}$ and $\bm{x}_{\textsc{last}(i)}$ form a valid
span. The third part (i.e., (3)) is necessary because each sentence can
contain multiple spans. The first part is computed by feeding
$\bm{x}_{\textsc{first}(i)}$ into a feed-forward layer:

$s_{\text{m}}(\bm{x}_{\textsc{first}(i)})=\textsc{ffnn}_{\text{}}([\bm{x}_{\textsc{first}(i)}])$
(1)

Similarly, the first part is computed by feeding $\bm{x}_{\textsc{last}(i)}$
into a feed-forward layer:

$s_{\text{m}}(\bm{x}_{\textsc{last}(i)})=\textsc{ffnn}_{\text{}}([\bm{x}_{\textsc{last}(i)}])$
(2)

The third part computed by feeding the concatenation of
$\bm{x}_{\textsc{first}(i)}$ and $\bm{x}_{\textsc{last}(i)}$ into into a feed-
forward layer:

$s_{\text{m}}(\bm{x}_{\textsc{first}(i)},\bm{x}_{\textsc{last}(i)})=\textsc{ffnn}_{\text{m}}([\bm{x}_{\textsc{first}(i)},\bm{x}_{\textsc{last}(i)}])$
(3)

$\textsc{ffnn}_{\text{}}($) denotes the feed-forward neural network that
computes a nonlinear mapping from the input vector to the mention score. The
three involved $\textsc{ffnn}_{\text{}}($) use separate sets of parameters.
The overall score for span $i$ being a mention is the average of the three
parts:

$\displaystyle s_{m}(i)=$
$\displaystyle[s_{\text{m}}(\bm{x}_{\textsc{first}(i)})+s_{\text{m}}(\bm{x}_{\textsc{end}(i)})$
(4) $\displaystyle+$ $\displaystyle
s_{\text{m}}(\bm{x}_{\textsc{first}(i)},\bm{x}_{\textsc{last}(i)})]/3$

We only keep up to $\lambda n$ (where $n$ is the document length) spans with
the highest mention scores.

#### Mention Proposal Pretraining

It is crucial that the mention proposal model is pretrained. Otherwise, most
of the proposed mentions that are fed to the linking stage are invalid
mentions. The mention proposal model is pretrained by jointly training three
binary classification models: (1) whether $\bm{x}_{\textsc{first}(i)}$ is the
start of a span; (2) whether $\bm{x}_{\textsc{last}(i)}$ is the end of a span;
and (3) whether $\bm{x}_{\textsc{first}(i)}$ and $\bm{x}_{\textsc{last}(i)}$
should be combined. This leads to the objective of the mention proposal model
as follows:

$\displaystyle\text{Loss(m)}=$
$\displaystyle\text{sigmoid}(s_{\text{m}}(\bm{x}_{\textsc{first}(i)}))$ (5)
$\displaystyle+$
$\displaystyle\text{sigmoid}(s_{\text{m}}(\bm{x}_{\textsc{end}(i)}))$
$\displaystyle+$
$\displaystyle\text{sigmoid}(s_{\text{m}}(\bm{x}_{\textsc{first}(i)},\bm{x}_{\textsc{last}(i)}))$

###  3.4 Mention Linking as Span Prediction

Given a mention $e_{i}$ proposed by the mention proposal network, the role of
the mention linking network is to give a score $s_{a}(i,j)$ for any text span
$e_{j}$, indicating whether $e_{i}$ and $e_{j}$ are coreferent. We propose to
use the question answering framework as the backbone to compute $s_{a}(i,j)$.
It operates on the triplet {context (X), query (q), answers (a)}. The context
$X$ is the input document. The query $q(e_{i})$ is constructed as follows:
given $e_{i}$, we use the sentence that $e_{i}$ resides in as the query, with
the minor modification that we encapsulates $e_{i}$ with special tokens
$<mention></mention>$ . The answers $a$ are the coreferent mentions of
$e_{i}$. A query $i$ is considered unanswerable in the following scenarios:
(1) the candidate span $e_{i}$ does not represent an entity mention or (2) the
candidate span $e_{i}$ represents an entity mention but is not coreferent with
any other mentions in $X$.

Following Devlin et al. (2019), we represent the input query and the context
as a single packed sequence. The for any span
$j=[{\textsc{first}(j)},...,{\textsc{last}(j)}]$, we first compute the score
of $i$ being the answer for query $q(e_{i})$, denoted by $s_{a}(j|i)$. Let
$\bm{x}_{\textsc{first}(j)}|i$ and $\bm{x}_{\textsc{last}(j)}|i$ respectively
denote the representations for first(j) and last(j) from BERT, where
$q(e_{i})$ is used as query concatenated to the context. $s_{a}(j|i)$ is
computed by feeding the first and the last of its constituent token
representations (i.e., $\bm{x}_{\textsc{first}(j)}|i$ and
$\bm{x}_{\textsc{last}(j)}|i$ ) into a feed-forward layer:

$s_{a}(j|i)=\textsc{ffnn}_{j|i}{[\bm{x}_{\textsc{first}(j)|i},\bm{x}_{\textsc{last}(j)|i}]}$
(6)

$\textsc{ffnn}_{j|i}$ denotes the feed-forward neural network that computes a
nonlinear mapping from the input vector to the mention score. Comparing Eq.8
with Eq.3, we can observe their relatedness and difference: both of the
equations compute scores for a span. But for Eq.8, the query $q(e_{i})$ is
additionally used to check whether span $j$ is the answer for $q(e_{i})$.

A closer look at Eq.8 reveals that it only models the uni-directional
coreference relation from $e_{i}$ to $e_{j}$, i.e., $e_{j}$ is the answer for
query $q(e_{i})$. This is suboptimal since if $e_{i}$ is a coreference mention
of $e_{j}$, then $e_{j}$ should also be the coreference mention $e_{i}$. We
thus need to optimize the bi-directional relation between $e_{i}$ and
$e_{j}$.444This bidirectional relationship is actually referred to as mutual
dependency and has shown to benefit a wide range of NLP tasks such as machine
translation Hassan et al. (2018) or dialogue generation Li et al. (2015). The
final score $s_{a}(i,j)$ is thus given as follows:

$s_{a}(i,j)=\frac{1}{2}(s_{a}(j|i)+s_{a}(i|j))$ (7)

$s_{a}(i|j)$ can be computed in the same way as $s_{a}(j|i)$, in which
$q(e_{i})$ is used as the query:

$s_{a}(i|j)=\textsc{ffnn}_{i|j}{[\bm{x}_{\textsc{first}(i)|j},\bm{x}_{\textsc{last}(i)|j}]}$
(8)

where $\bm{x}_{\textsc{first}(i)}|j$ and $\bm{x}_{\textsc{last}(i)}|j$
respectively denote the representations for first(i) and last(i) from BERT,
where $q(e_{j})$ is used as query concatenated to the context.

For a pair of text span $e_{i}$ and $e_{j}$, the premises for them being
coreferent mentions are (1) they are mentions and (2) they are coreferent.
This makes the overall score $s(i,j)$ for $e_{i}$ and $e_{j}$ the combination
of Eq.3 and Eq.7:

$s(i,j)=\lambda[s_{\text{m}}(i)+s_{\text{m}}(j)]+(1-\lambda)s_{a}(i,j)$ (9)

$\lambda$ is the hyperparameter to control the tradeoff between mention
proposal and mention linking.

###  3.5 Antecedent Pruning

Given a document $X$ with length $n$ and the number of spans $O(n^{2})$, the
computation of Eq.9 for all mention pairs is intractable with the complexity
of $O(n^{4})$. Given an extracted mention $e_{i}$, the computation of Eq.9 for
$(e_{i},e_{j})$ regarding all $e_{j}$ is still extremely intensive since the
computation of the backward span prediction score $s_{a}(i|j)$ requires
running question answering models on all query $q(e_{j})$. A further pruning
procedure is thus needed: For each query $q(e_{i})$, we collect $C$ span
candidates only based on the $s_{a}(j|i)$ scores, and then use Eq. 9 to
compute the overall scores.

###  3.6 Training

For each mention $e_{i}$ proposed by the mention proposal network, it is
associated with $C$ potential spans proposed by the mention linking network
based on $s(j|i)$, we aim to optimize the marginal log-likelihood of all
correct antecedents implied by the gold clustering. Following Lee et al.
(2017), we append a dummy token $\epsilon$ to the $C$ candidates. The model
will output it if none of the $C$ span candidates is coreferent with $e_{i}$.
For each mention $e_{i}$, the model learns a distribution $P(\cdot)$ over all
possible antecedent spans $e_{j}$ based on the global score $s(i,j)$ from Eq.
9:

$P(e_{j})=\frac{e^{s(i,j)}}{\sum_{j^{\prime}\in C}e^{s(i,j^{\prime})}}$ (10)

The mention proposal module and the mention linking module are jointly trained
in an end-to-end fashion using training signals from Eq.10, with the SpanBERT
parameters shared.

###  3.7 Inference

Given an input document, we can obtain an undirected graph using the overall
score, each node of which represents a candidate mention from either the
mention proposal module or the mention linking module. We prune the graph by
keeping the edge whose weight is the largest for each node based on Eq.10.
Nodes whose closest neighbor is the dummy token $\epsilon$ are abandoned.
Therefore, the mention clusters can be decoded from the graph.

###  3.8 Data Augmentation using Question Answering Datasets

We hypothesize that the reasoning (such as synonymy, world knowledge,
syntactic variation, and multiple sentence reasoning) required to answer the
questions are also indispensable for coreference resolution. Annotated
question answering datasets are usually significantly larger than the
coreference datasets due to the high linguistic expertise required for the
latter. Under the proposed QA formulation, coreference resolution has the same
format as the existing question answering datasets (Rajpurkar et al., 2016a,
2018; Dasigi et al., 2019a). In this way, they can readily be used for data
augmentation. We thus propose to pre-train the mention linking network on the
Quoref dataset Dasigi et al. (2019b), and the SQuAD dataset Rajpurkar et al.
(2016b).

###  3.9 Summary and Discussion

Comparing with existing models Lee et al. (2017, 2018); Joshi et al. (2019b),
the proposed question answering formalization has the flexibility of
retrieving mentions left out at the mention proposal stage. However, since we
still have the mention proposal model, we need to know in which situation
missed mentions could be retrieved and in which situation they cannot. We use
the example in Figure 1 as an illustration, in which {many people, They,
themselves} are coreferent mentions: If partial mentions are missed by the
mention proposal model, e.g., many people and They, they can still be
retrieved in the mention linking stage when the not-missed mention (i.e.,
themselves) is used as query. But, if all the mentions within the cluster are
missed, none of them can be used for query construction, which means they all
will be irreversibly left out. Given the fact that the proposal mention
network proposes a significant number of mentions, the chance that mentions
within a mention cluster are all missed is relatively low (which exponentially
decreases as the number of entities increases). This explains the superiority
(though far from perfect) of the proposed model. However, how to completely
remove the mention proposal network remains a problem in the field of
coreference resolution.

##  4 Experiments

###  4.1 Implementation Details

The special tokens used to denote the speaker’s name ($<speaker></speaker>$)
and the special tokens used to denote the queried mentions
($<mention></mention>$) are initialized by randomly taking the unused tokens
from the SpanBERT vocabulary. The sliding window size $T$ = 512, and the
mention keep ratio $\lambda$ = 0.2. The maximum length $L$ for mention
proposal = 10 and the maximum number of antecedents kept for each mention $C$
= 50. The SpanBERT parameters are updated by the Adam optimizer (Kingma and
Ba, 2015) with initial learning rate $1\times 10^{-5}$ and the task parameters
are updated by the Range optimizer 555https://github.com/lessw2020/Ranger-
Deep-Learning-Optimizer with initial learning rate $2\times 10^{-4}$.

###  4.2 Baselines

We compare the CorefQA model with previous neural models that are trained end-
to-end:

  * •

e2e-coref (Lee et al., 2017) is the first end-to-end coreference system that
learns which spans are entity mentions and how to best cluster them jointly.
Their token representations are built upon the GLoVe (Pennington et al., 2014)
and Turian (Turian et al., 2010) embeddings.

  * •

c2f-coref + ELMo (Lee et al., 2018) extends Lee et al. (2017) by combining a
coarse-to-fine pruning with a higher-order inference mechanism. Their
representations are built upon ELMo embeddings (Peters et al., 2018).

  * •

c2f-coref + BERT-large(Joshi et al., 2019b) builds the c2f-coref system on top
of BERT (Devlin et al., 2019) token representations.

  * •

EE + BERT-large (Kantor and Globerson, 2019) represents each mention in a
cluster via an approximation of the sum of all mentions in the cluster.

  * •

c2f-coref + SpanBERT-large (Joshi et al., 2019a) focuses on pre-training span
representations to better represent and predict spans of text.

| MUC | $\text{B}^{3}$ | $\text{CEAF}_{\phi_{4}}$ |
---|---|---|---|---
| P | R | F1 | P | R | F1 | P | R | F1 | Avg. F1
e2e-coref(Lee et al., 2017) | 78.4 | 73.4 | 75.8 | 68.6 | 61.8 | 65.0 | 62.7 | 59.0 | 60.8 | 67.2
c2f-coref + ELMo (Lee et al., 2018) | 81.4 | 79.5 | 80.4 | 72.2 | 69.5 | 70.8 | 68.2 | 67.1 | 67.6 | 73.0
EE + BERT-large (Kantor and Globerson, 2019) | 82.6 | 84.1 | 83.4 | 73.3 | 76.2 | 74.7 | 72.4 | 71.1 | 71.8 | 76.6
c2f-coref + BERT-large (Joshi et al., 2019b) | 84.7 | 82.4 | 83.5 | 76.5 | 74.0 | 75.3 | 74.1 | 69.8 | 71.9 | 76.9
c2f-coref + SpanBERT-large (Joshi et al., 2019a) | 85.8 | 84.8 | 85.3 | 78.3 | 77.9 | 78.1 | 76.4 | 74.2 | 75.3 | 79.6
CorefQA + SpanBERT-base | 85.2 | 87.4 | 86.3 | 78.7 | 76.5 | 77.6 | 76.0 | 75.6 | 75.8 | 79.9 (+0.3)
CorefQA + SpanBERT-large | 88.6 | 87.4 | 88.0 | 82.4 | 82.0 | 82.2 | 79.9 | 78.3 | 79.1 |  83.1 (+3.5)
Table 1: Evaluation results on the English CoNLL-2012 shared task. The average F1 of MUC, $\text{B}^{3}$, and $\text{CEAF}_{\phi_{4}}$ is the main evaluation metric. Ensemble models are not included in the table for a fair comparison. Model | M | F | B | O
---|---|---|---|---
e2e-coref | 67.2 | 62.2 | 0.92 | 64.7
c2f-coref + ELMo | 75.8 | 71.1 | 0.94 | 73.5
c2f-coref + BERT-large | 86.9 | 83.0 | 0.95 | 85.0
c2f-coref + SpanBERT-large | 88.8 | 84.9 | 0.96 | 86.8
CorefQA + SpanBERT-large | 88.9 | 86.1 | 0.97 | 87.5
Table 2: CorefQA achieves the state-of-the-art performance on all metrics
including F1 scores on Masculine and Feminine examples, a Bias factor (F / M)
and the Overall F1 score.

###  4.3 Results on CoNLL-2012 Shared Task

The English data of CoNLL-2012 shared task (Pradhan et al., 2012) contains
2,802/343/348 train/development/test documents in 7 different genres. The main
evaluation is the average of three metrics – MUC (Vilain et al., 1995),
$\text{B}^{3}$ (Bagga and Baldwin, 1998), and $\text{CEAF}_{\phi_{4}}$ (Luo,
2005) on the test set according to the official CoNLL-2012 evaluation scripts
666http://conll.cemantix.org/2012/software.html.

We compare the CorefQA model with several baseline models in Table 1. Our
CorefQA system achieves a huge performance boost over existing systems: With
SpanBERT-base, it achieves an F1 score of 79.9, which already outperforms the
previous SOTA model using SpanBERT-large by 0.3. With SpanBERT-large, it
achieves an F1 score of 83.1, with a 3.5 performance boost over the previous
SOTA system.

###  4.4 Results on GAP

The GAP dataset (Webster et al., 2018) is a gender-balanced dataset that
targets the challenges of resolving naturally occurring ambiguous pronouns. It
comprises 8,908 coreference-labeled pairs of (ambiguous pronoun, antecedent
name) sampled from Wikipedia.

We follow the protocols in Webster et al. (2018); Joshi et al. (2019b) and use
the off-the-shelf resolver trained on the CoNLL-2012 dataset to get the
performance of the GAP dataset. Table 2 presents the results. We can see that
the proposed CorefQA model achieves state-of-the-art performance on all
metrics on the GAP dataset.

##  5 Ablation Study and Analysis

|  Avg. F1 |  $\Delta$
---|---|---
CorefQA |  83.4 |
$-\text{--}$ SpanBERT |  79.6 |  -3.8
$-\text{--}$ Mention Proposal Pre-training |  75.9 |  -7.5
$-\text{--}$ Question Answering |  75.0 |  -8.4
$-\text{--}$ Quoref Pre-training |  82.7 |  -0.7
$-\text{--}$ Squad Pre-training |  83.1 |  -0.3
Table 3: Ablation studies on the CoNLL-2012 development set. SpanBERT token
representations, the mention-proposal pre-training, and the question answering
pre-training all contribute significantly to the good performance of the full
model.

We perform comprehensive ablation studies and analyses on the CoNLL-2012
development dataset. Results are shown in Table 3.

###  5.1 Effects of Different Modules in the Proposed Framework

#### Effect of SpanBERT

Replacing SpanBERT with vanilla BERT leads to a 3.5 F1 degradation. This
verifies the importance of span-level pre-training for coreference resolution
and is consistent with previous findings (Joshi et al., 2019a).

#### Effect of Pre-training Mention Proposal Network

Skipping the pre-training of the mention proposal network using golden
mentions results in a 7.2 F1 degradation, which is in line with our
expectation. A randomly initialized mention proposal model implies that
mentions are randomly selected. Randomly selected mentions will mostly be
transformed to unanswerable queries. This makes it hard for the question
answering model to learn at the initial training stage, leading to inferior
performance.

#### Effect of QA pre-training on the augmented datasets

One of the most valuable strengths of converting anaphora resolution to
question answering is that existing QA datasets can be readily used for data
augmentation purposes. We see a contribution of 0.7 F1 from pre-training on
the Quoref dataset (Dasigi et al., 2019a) and a contribution of 0.3 F1 from
pre-training on the SQuAD dataset (Rajpurkar et al., 2016a).

#### Effect of Question Answering

We aim to study the pure performance gain of the paradigm shift from mention-
pair scoring to query-based span prediction. For this purpose, we replace the
mention linking module with the mention-pair scoring module described in Lee
et al. (2018), while others remain unchanged. We observe an 8.1 F1 degradation
in performance, demonstrating the significant superiority of the proposed
question answering framework over the mention-pair scoring framework.

###  5.2 Analyses on speaker modeling strategies

1234567+$10$$20$$30$$40$$50$$60$$70$$80$$90$$100$Number of speakers per
document%F1(Speaker as feature)F1(Speaker as input)Frequency Figure 3:
Performance on the development set of the CoNLL-2012 dataset with various
number of speakers. F1(Speaker as feature): F1 score for the strategy that
treats speaker information as a mention-pair feature. F1(Speaker as input): F1
score for our strategy that treats speaker names as token input. Frequency:
percentage of documents with specific number of speakers.

We compare our speaker modeling strategy (denoted by Speaker as input), which
directly concatenates the speaker’s name with the corresponding utterance,
with the strategy in Wiseman et al. (2016); Lee et al. (2017); Joshi et al.
(2019a) (denoted by Speaker as feature), which converts speaker information
into binary features indicating whether two mentions are from the same
speaker. We show the average F1 scores breakdown by documents according to the
number of their constituent speakers in Figure 3.

Results show that the proposed strategy performs significantly better on
documents with a larger number of speakers. Compared with the coarse modeling
of whether two utterances are from the same speaker, a speaker’s name can be
thought of as speaker ID in persona dialogue learning Li et al. (2016); Zhang
et al. (2018b); Mazaré et al. (2018). Representations learned for names have
the potential to better generalize the global information of the speakers in
the multi-party dialogue situation, leading to better context modeling and
thus better results.

###  5.3 Analysis on the Overall Mention Recall

$10$$20$$30$$40$$50$$30$$40$$50$$60$$70$$80$$90$$100$the number of spans
$\lambda$ kept per wordMention Recall (%)Joshi et al. (2019a) (various
$\lambda$)Joshi et al. (2019a) (actual $\lambda$)Our model (various
$\lambda$)Our model (actual $\lambda$) Figure 4: Change of mention recalls as
we increase the number of spans $\lambda$ kept per word.

Since the proposed framework has the potential to retrieve mentions missed at
the mention proposal stage, we expect it to have higher overall mention recall
rate than previous models (Lee et al., 2017, 2018; Zhang et al., 2018a; Kantor
and Globerson, 2019).

We examine the proportion of gold mentions covered in the development set as
we increase the hyperparameter $\lambda$ (the number of spans kept per word)
in Figure 4. Our model consistently outperforms the baseline model with
various values of $\lambda$. Notably, our model is less sensitive to smaller
values of $\lambda$. This is because missed mentions can still be retrieved at
the mention linking stage.

###  5.4 Qualitative Analysis

1 |  [Freddie Mac] is giving golden parachutes to two of its ousted executives. …Yesterday Federal Prosecutions announced a criminal probe into [the company].
---|---
2 |  [A traveling reporter] now on leave and joins us to tell [her] story. Thank [you] for coming in to share this with us.
3 |  Paula Zahn: [Thelma Gutierrez] went inside the forensic laboratory where scientists are trying to solve this mystery. Thelma Gutierrez: In this laboratory alone [I] ’m surrounded by the remains of at least twenty different service members who are in the process of being identified so that they too can go home.
Table 4: Example mention clusters that were correctly predicted by our model,
but wrongly predicted by c2f-coref + SpanBERT-large. Bold spans in brackets
represent coreferent mentions. Italic spans represent the speaker’s name of
the utterance.

We provide qualitative analyses to highlight the strengths of our model in
Table 4.

Shown in Example 1, by explicitly formulating the anaphora identification of
the company as a query, our model uses more information from a local context,
and successfully identifies Freddie Mac as the answer from a longer distance.

The model can also efficiently harness the speaker information in a
conversational setting. In Example 3, it would be difficult to identify that
[Thelma Gutierrez] is the correct antecedent of mention [I] without knowing
that Thelma Gutierrez is the speaker of the second utterance. However, our
model successfully identifies it by directly feeding the speaker’s name at the
input level.

##  6 Conclusion

In this paper, we present CorefQA, a coreference resolution model that casts
anaphora identification as the task of query-based span prediction in question
answering. We showed that the proposed formalization can successfully retrieve
mentions left out at the mention proposal stage. It also makes data
augmentation using a plethora of existing question answering datasets
possible. Furthermore, a new speaker modeling strategy can also boost the
performance in dialogue settings. Empirical results on two widely-used
coreference datasets demonstrate the effectiveness of our model. In future
work, we will explore novel approaches to generate the questions based on each
mention, and evaluate the influence of different question generation methods
on the coreference resolution task.

## Acknowledgement

We thank all anonymous reviewers for their comments and suggestions. The work
is supported by the National Natural Science Foundation of China (NSFC No.
61625107 and 61751209).

## References

  * Aralikatte et al. (2019) Rahul Aralikatte, Matthew Lamm, Daniel Hardt, and Anders Søgaard. 2019.  Ellipsis and coreference resolution as question answering.  _CoRR_ , abs/1908.11141.
  * Bagga and Baldwin (1998) Amit Bagga and Breck Baldwin. 1998.  Algorithms for scoring coreference chains.  In _In The First International Conference on Language Resources and Evaluation Workshop on Linguistics Coreference_ , pages 563–566.
  * Björkelund and Kuhn (2014) Anders Björkelund and Jonas Kuhn. 2014.  Learning structured perceptrons for coreference resolution with latent antecedents and non-local features.  In _Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014, June 22-27, 2014, Baltimore, MD, USA, Volume 1: Long Papers_ , pages 47–57.
  * Chada (2019) Rakesh Chada. 2019.  Gendered pronoun resolution using bert and an extractive question answering formulation.  _arXiv preprint arXiv:1906.03695_.
  * Clark and Manning (2015) Kevin Clark and Christopher D. Manning. 2015.  Entity-centric coreference resolution with model stacking.  In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26-31, 2015, Beijing, China, Volume 1: Long Papers_ , pages 1405–1415.
  * Clark and Manning (2016) Kevin Clark and Christopher D. Manning. 2016.  Improving coreference resolution by learning entity-level distributed representations.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers_.
  * Dasigi et al. (2019a) Pradeep Dasigi, Nelson F. Liu, Ana Marasovic, Noah A. Smith, and Matt Gardner. 2019a.  Quoref: A reading comprehension dataset with questions requiring coreferential reasoning.  _CoRR_ , abs/1908.05803.
  * Dasigi et al. (2019b) Pradeep Dasigi, Nelson F Liu, Ana Marasovic, Noah A Smith, and Matt Gardner. 2019b.  Quoref: A reading comprehension dataset with questions requiring coreferential reasoning.  _arXiv preprint arXiv:1908.05803_.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers)_ , pages 4171–4186.
  * Durrett and Klein (2013) Greg Durrett and Dan Klein. 2013.  Easy victories and uphill battles in coreference resolution.  In _Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, EMNLP 2013, 18-21 October 2013, Grand Hyatt Seattle, Seattle, Washington, USA, A meeting of SIGDAT, a Special Interest Group of the ACL_ , pages 1971–1982.
  * Emami et al. (2019) Ali Emami, Paul Trichelair, Adam Trischler, Kaheer Suleman, Hannes Schulz, and Jackie Chi Kit Cheung. 2019.  The knowref coreference corpus: Removing gender and number cues for difficult pronominal anaphora resolution.  In _Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers_ , pages 3952–3961.
  * Hassan et al. (2018) Hany Hassan, Anthony Aue, Chang Chen, Vishal Chowdhary, Jonathan Clark, Christian Federmann, Xuedong Huang, Marcin Junczys-Dowmunt, William Lewis, Mu Li, et al. 2018.  Achieving human parity on automatic chinese to english news translation.  _arXiv preprint arXiv:1803.05567_.
  * He et al. (2015) Luheng He, Mike Lewis, and Luke Zettlemoyer. 2015.  Question-answer driven semantic role labeling: Using natural language to annotate natural language.  In _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, September 17-21, 2015_ , pages 643–653.
  * Hou et al. (2018) Yutai Hou, Yijia Liu, Wanxiang Che, and Ting Liu. 2018.  Sequence-to-sequence data augmentation for dialogue language understanding.  In _Proceedings of the 27th International Conference on Computational Linguistics, COLING 2018, Santa Fe, New Mexico, USA, August 20-26, 2018_ , pages 1234–1245.
  * Joshi et al. (2019a) Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, and Omer Levy. 2019a.  Spanbert: Improving pre-training by representing and predicting spans.  _CoRR_ , abs/1907.10529.
  * Joshi et al. (2019b) Mandar Joshi, Omer Levy, Daniel S. Weld, and Luke Zettlemoyer. 2019b.  BERT for coreference resolution: Baselines and analysis.  _CoRR_ , abs/1908.09091.
  * Kantor and Globerson (2019) Ben Kantor and Amir Globerson. 2019.  Coreference resolution with entity equalization.  In _Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers_ , pages 673–677.
  * Kingma and Ba (2015) Diederik P. Kingma and Jimmy Ba. 2015.  Adam: A method for stochastic optimization.  In _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_.
  * Kobayashi (2018) Sosuke Kobayashi. 2018.  Contextual augmentation: Data augmentation by words with paradigmatic relations.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 2 (Short Papers)_ , pages 452–457.
  * Lee et al. (2017) Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017.  End-to-end neural coreference resolution.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017_ , pages 188–197.
  * Lee et al. (2018) Kenton Lee, Luheng He, and Luke Zettlemoyer. 2018.  Higher-order coreference resolution with coarse-to-fine inference.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 2 (Short Papers)_ , pages 687–692.
  * Levy et al. (2017) Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. 2017.  Zero-shot relation extraction via reading comprehension.  In _Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), Vancouver, Canada, August 3-4, 2017_ , pages 333–342.
  * Li et al. (2015) Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2015.  A diversity-promoting objective function for neural conversation models.  _arXiv preprint arXiv:1510.03055_.
  * Li et al. (2016) Jiwei Li, Michel Galley, Chris Brockett, Georgios P Spithourakis, Jianfeng Gao, and Bill Dolan. 2016.  A persona-based neural conversation model.  _arXiv preprint arXiv:1603.06155_.
  * Li et al. (2019a) Xiaoya Li, Jingrong Feng, Yuxian Meng, Qinghong Han, Fei Wu, and Jiwei Li. 2019a.  A unified mrc framework for named entity recognition.  _arXiv preprint arXiv:1910.11476_.
  * Li et al. (2019b) Xiaoya Li, Fan Yin, Zijun Sun, Xiayu Li, Arianna Yuan, Duo Chai, Mingxin Zhou, and Jiwei Li. 2019b.  Entity-relation extraction as multi-turn question answering.  In _Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers_ , pages 1340–1350.
  * Luo (2005) Xiaoqiang Luo. 2005.  On coreference resolution performance metrics.  In _HLT/EMNLP 2005, Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference, 6-8 October 2005, Vancouver, British Columbia, Canada_ , pages 25–32.
  * Mazaré et al. (2018) Pierre-Emmanuel Mazaré, Samuel Humeau, Martin Raison, and Antoine Bordes. 2018\.  Training millions of personalized dialogue agents.  _arXiv preprint arXiv:1809.01984_.
  * McCann et al. (2018) Bryan McCann, Nitish Shirish Keskar, Caiming Xiong, and Richard Socher. 2018.  The natural language decathlon: Multitask learning as question answering.  _CoRR_ , abs/1806.08730.
  * Morgenstern et al. (2016) Leora Morgenstern, Ernest Davis, and Charles L. Ortiz Jr. 2016.  Planning, executing, and evaluating the winograd schema challenge.  _AI Magazine_ , 37(1):50–54.
  * Pennington et al. (2014) Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014.  Glove: Global vectors for word representation.  In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL_ , pages 1532–1543.
  * Peters et al. (2018) Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018.  Deep contextualized word representations.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 1 (Long Papers)_ , pages 2227–2237.
  * Pradhan et al. (2012) Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012.  Conll-2012 shared task: Modeling multilingual unrestricted coreference in ontonotes.  In _Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning - Proceedings of the Shared Task: Modeling Multilingual Unrestricted Coreference in OntoNotes, EMNLP-CoNLL 2012, July 13, 2012, Jeju Island, Korea_ , pages 1–40.
  * Rajpurkar et al. (2018) Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018.  Know what you don’t know: Unanswerable questions for squad.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 2: Short Papers_ , pages 784–789.
  * Rajpurkar et al. (2016a) Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016a.  Squad: 100, 000+ questions for machine comprehension of text.  In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016_ , pages 2383–2392.
  * Rajpurkar et al. (2016b) Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016b.  Squad: 100,000+ questions for machine comprehension of text.  _arXiv preprint arXiv:1606.05250_.
  * Talmor and Berant (2019) Alon Talmor and Jonathan Berant. 2019.  Multiqa: An empirical investigation of generalization and transfer in reading comprehension.  In _Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers_ , pages 4911–4921.
  * Turian et al. (2010) Joseph P. Turian, Lev-Arie Ratinov, and Yoshua Bengio. 2010.  Word representations: A simple and general method for semi-supervised learning.  In _ACL 2010, Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, July 11-16, 2010, Uppsala, Sweden_ , pages 384–394.
  * Vilain et al. (1995) Marc B. Vilain, John D. Burger, John S. Aberdeen, Dennis Connolly, and Lynette Hirschman. 1995.  A model-theoretic coreference scoring scheme.  In _Proceedings of the 6th Conference on Message Understanding, MUC 1995, Columbia, Maryland, USA, November 6-8, 1995_ , pages 45–52.
  * Webster et al. (2018) Kellie Webster, Marta Recasens, Vera Axelrod, and Jason Baldridge. 2018.  Mind the GAP: A balanced corpus of gendered ambiguous pronouns.  _TACL_ , 6:605–617.
  * Wiseman et al. (2016) Sam Wiseman, Alexander M. Rush, and Stuart M. Shieber. 2016.  Learning global features for coreference resolution.  In _NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego California, USA, June 12-17, 2016_ , pages 994–1004.
  * Wiseman et al. (2015) Sam Wiseman, Alexander M. Rush, Stuart M. Shieber, and Jason Weston. 2015.  Learning anaphoricity and antecedent ranking features for coreference resolution.  In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26-31, 2015, Beijing, China, Volume 1: Long Papers_ , pages 1416–1426.
  * Zhang et al. (2018a) Rui Zhang, Cícero Nogueira dos Santos, Michihiro Yasunaga, Bing Xiang, and Dragomir R. Radev. 2018a.  Neural coreference resolution with deep biaffine attention by joint mention detection and mention clustering.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 2: Short Papers_ , pages 102–107.
  * Zhang et al. (2018b) Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018b.  Personalizing dialogue agents: I have a dog, do you have pets too?  _arXiv preprint arXiv:1801.07243_.
  * Zhao et al. (2019) Jieyu Zhao, Tianlu Wang, Mark Yatskar, Ryan Cotterell, Vicente Ordonez, and Kai-Wei Chang. 2019.  Gender bias in contextualized word embeddings.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers)_ , pages 629–634.
  * Zhao et al. (2018) Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018\.  Gender bias in coreference resolution: Evaluation and debiasing methods.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 2 (Short Papers)_ , pages 15–20.


# Secrecy Capacity Maximization for a Hybrid Relay-RIS Scheme in mmWave MIMO
Networks

Edson Nobuyuki Egashira∗, Diana Pamela Moya Osorio∗, Nhan Thanh Nguyen∗, and
Markku Juntti∗ *Centre for Wireless Communications, University of Oulu,
P.O.Box 4500, FI-90014, Finland
Emails: {edson.egashira, diana.moyaosorio, nhan.nguyen<EMAIL_ADDRESS>
###### Abstract

The hybrid relay-reflecting intelligent surface (HR-RIS) has been recently
introduced as an efficient solution to overcome the double path loss and
limited beamforming diversity of the conventional fully passive reflecting
surface. This motivates us to investigate the application of the HR-RIS in
improving the secrecy capacity of millimeter wave multiple-input-multiple-
output (MIMO) systems with the presence of multi-antenna eavesdropper. The
joint optimization of the transmit beamformer and the relay-reflecting
coefficients at RIS is tackled via alternating optimization. In the proposed
solution, a closed-form expression for the optimal transmit beamformer at the
transmitter is derived, and a metaheuristic solution based on particle swarm
optimization is proposed to optimize the active and passive elements at the
HR-RIS. The simulation results verify that under various scenarios, the HR-RIS
provides significant improvement in the secrecy capacity with respect to the
conventional passive reflecting surface.

###### Index Terms:

Hybrid relay-reflecting intelligent surface (HR-RIS), RIS, mmWave, MIMO,
physical layer security, PSO.

##  I Introduction

For the sixth-generation (6G) of wireless networks, it is expected that the
radio environment becomes controllable and intelligent by leveraging the
emergence of new enabling technologies that allows the control of the wireless
propagation in order to improve reliability and coverage. Particularly,
reflecting intelligent surfaces (RIS) have shown a great potential to tackle
the challenges of security, energy and spectral efficiency in 6G. RIS is a
software-controlled surface composed by a planar array of reflecting elements,
which are capable to dynamically adjust their reflective coefficients, thus
controlling the amplitude and/or phase shift of reflected signals to enhance
the wireless propagation performance [1].

By smartly controlling the phase shifts of RIS, the reflected signals can
either be added coherently at the intended receiver to improve the received
signal power, or be added destructively at the non-desired receiver, thus
providing information-theoretic security guarantees and being an important
candidate for the design of physical layer security (PLS) schemes [2, 3]. In
this regard, RIS-based PLS schemes have recently attracted increasing
attention [4, 5, 6]. For instance, in [4, 5], the secrecy capacity is
evaluated for a multiple-input-single-output (MISO) RIS-aided secure system,
under the presence of an eavesdropper. In [4], Cui et al. have addressed the
impact of the transmit power at the access point (AP) on the secrecy capacity
by using a joint alternating optimization (AO)- based and semidefinite
relaxation (SDR) methods. In [5], a fractional programming (FP) and Manifold
Optimization (MO)- based algorithm is proposed to maximize the secrecy
capacity. In [6], the average system sum secrecy rate and secrecy outage
probability were analyzed for a system with multiple users, eavesdroppers and
RIS. By taking into account the imperfect CSI, the results confirm the
robustness and the ability to exploit the multiuser diversity.

Particularly, in the context of millimeter-wave (mmWave) multiple-input-
multiple-output (MIMO) systems that are recognized for providing ultra-high
data-rates, RIS can offer an important solution to the high propagation loss
nature of these systems in order to improve reliability and security [7, 8].
Thus, the authors in [7] investigate the secrecy rate of a RIS-assisted mmWave
MISO system in the presence of a single-antenna eavesdropper. Therein, an
Alternating Optimization (AO) approach is proposed to jointly optimize the
transmit beamformer at the base station and the passive reflecting elements at
the RIS. The proposed method has shown to boost the secrecy performance,
regardless of the eavesdropper’s positions. In [8], an AO-based algorithm is
proposed to maximize the secrecy rate for a mmWave MISO system by assuming
multiple RISs and a single-antenna passive eavesdropper. The spatially
distributed multiple RISs deployment showed a significant secrecy improvement
compared with a single RIS.

Despite the conventional RIS has shown to be a promising technology to enable
secure and intelligent radio environments, it has been proved that a very
large number of reflecting elements of a passive RIS is needed to outperform
the performance of decode-and-forward relaying [9]. Therefore, hybrid relay-
RIS (HR-RIS) scheme was introduced in [10] to leverage the benefits from both
passive RIS and relay. Specifically, Nguyen et al. proposed employing a single
or few active elements in the RIS to provide the signal amplification gain for
a significant improvement in the system spectral efficiency.

Inspired by the promising simultaneous reflecting-relaying capabilities of the
HR-RIS, this paper advances on the state-of-the art by investigating and
optimizing the secrecy performance of the HR-RIS scheme in a mmWave MIMO
system with the presence of a multiple-antenna eavesdropper. For this purpose,
the joint optimization of the transmit beamformer and the relay-reflecting
coefficients at RIS is attained by resorting to AO. Specifically, the closed-
form solution to the optimal transmit beamformer is first derived with given
HR-RIS coefficients. Then, we employ the particle swarm optimization (PSO)
approach to tackle the challenges in the HR-RIS coefficients design. Our
simulations for various scenarios show that the HR-RIS-aided mmWave MIMO
system significantly outperforms its conventional counterparts, including the
systems without RIS and with the passive RIS.

##  II System Model

Figure 1: An HR-RIS assisted mmWave MIMO system, with muiti-antenna Alice,
multi-antenna Bob, HR-RIS, and a multi-antenna Eve.

We consider a mmWave MIMO system consisting of a transmitter
($\mathrm{Alice}$), a receiver ($\mathrm{Bob}$), and an eavesdropper (Eve).
The eavesdropper is an active user, and the communications system is assisted
by an HR-RIS, as shown in Fig. 1. Let $N_{\mathrm{A}},N_{\mathrm{B}}$ and
$N_{\mathrm{E}}$ be the numbers of antennas at $\mathrm{Alice}$,
$\mathrm{Bob}$ and Eve, respectively, and the HR-RIS is equipped with $N$
elements, comprising $K$ active relay elements and $M$ passive reflecting
elements, such that $K+M=N$.

###  II-A Channel Model

Let us denote the channel matrices between $\mathrm{Alice}$ and HR-RIS,
between HR-RIS and $\mathrm{Bob}$, between HR-RIS and $\mathrm{Eve}$, between
$\mathrm{Alice}$ and $\mathrm{Bob}$, and between $\mathrm{Alice}$ and
$\mathrm{Eve}$ as
${{\boldsymbol{\mathrm{H}}}}_{\mathrm{AR}}\in\mathbb{C}^{N\times
N_{\mathrm{A}}}$,
${{\boldsymbol{\mathrm{H}}}}_{\mathrm{RB}}\in\mathbb{C}^{N_{\mathrm{B}}\times
N}$,
${{\boldsymbol{\mathrm{H}}}}_{\mathrm{RE}}\in\mathbb{C}^{N_{\mathrm{E}}\times
N}$,
${{\boldsymbol{\mathrm{H}}}}_{\mathrm{AB}}\in\mathbb{C}^{N_{\mathrm{B}}\times
N_{\mathrm{A}}}$, and
${{\boldsymbol{\mathrm{H}}}}_{\mathrm{AE}}\in\mathbb{C}^{N_{\mathrm{E}}\times
N_{\mathrm{A}}}$, respectively. We assume that $\mathrm{Alice}$ is able to
acquire the CSI of the $\mathrm{Bob}$ perfectly. However, the eavesdropper
node may try to hide its existence from $\mathrm{Alice}$ and despite the
signal leakage from $\mathrm{Eve}$ can still be utilized for channel
estimation at $\mathrm{Alice}$, the acquired CSI is expected to be outdated
[6]. Hence, we adopt the imperfect CSI model, where the channel coefficients
between HR-RIS and $\mathrm{Eve}$ and between $\mathrm{Alice}$ and
$\mathrm{Eve}$ are modeled by
${{\boldsymbol{\mathrm{H}}}}_{\mathrm{X}}=\bar{{{\boldsymbol{\mathrm{H}}}}}_{\mathrm{X}}+\boldsymbol{\Delta\mathrm{H}}_{\mathrm{X}}$,
where $\mathrm{X}=\\{\mathrm{RE},\mathrm{AE}\\}$,
$\bar{{{\boldsymbol{\mathrm{H}}}}}_{\mathrm{X}}$ is the outdated estimate of
the eavesdropper’s channel, and $\boldsymbol{\Delta\mathrm{H}}_{\mathrm{X}}$
is the CSI estimation error, modeled with the Gaussian CSI error model [11].
Thus, the CSI estimation error $\boldsymbol{\mathrm{\Delta H}}_{\mathrm{RE}}$
is given by

$\displaystyle\boldsymbol{\mathrm{\Delta H}}_{\mathrm{RE}}$
$\displaystyle=\sqrt{\frac{NN_{\mathrm{E}}}{L_{\mathrm{RE}}}}\boldsymbol{\delta}_{\mathrm{RE}}\Big{(}\beta_{\mathrm{RE_{1}}}\boldsymbol{\mathrm{a}}(\phi_{\mathrm{RE}}^{r})\boldsymbol{\mathrm{a}}^{\mathrm{H}}(\gamma_{\mathrm{RE}}^{t},\eta_{\mathrm{RE}}^{t})$
$\displaystyle\quad+\sum_{i=2}^{L_{\mathrm{RE}}}\beta_{\mathrm{RE_{i}}}\boldsymbol{\mathrm{a}}(\phi_{\mathrm{RE}}^{r})\boldsymbol{\mathrm{a}}^{\mathrm{H}}(\gamma_{\mathrm{RE}}^{t},\eta_{\mathrm{RE}}^{t})\Big{)},$
(1)

where
$\boldsymbol{\delta}_{\mathrm{RE}}\sim\mathcal{CN}(0,{{\boldsymbol{\mathrm{I}}}}_{N_{\mathrm{E}}}\sigma_{\Delta}^{2})$,
with $\mathcal{CN}(a,\,b)$ denoting as a complex circularly-symmetric Gaussian
distribution with mean $a$ and variance $b$, ${{\boldsymbol{\mathrm{I}}}}_{N}$
denotes an identity matrix of size $N\times N$, and $\sigma_{\Delta}^{2}$ is
the normalized variance of the error component of the channel.
$L_{\mathrm{RE}}$ is the number of resolvable paths,
$\beta_{\mathrm{{RE}_{1}}}$ and $\beta_{\mathrm{{RE}_{i}}}$ are the complex
gain, $\boldsymbol{\mathrm{a}}(\phi_{\mathrm{RE}}^{r})$ and
$\boldsymbol{\mathrm{a}}(\gamma_{\mathrm{RE}}^{t},\eta_{\mathrm{RE}}^{t})$ are
the normalized array response vectors. For the channel
$\boldsymbol{\mathrm{\Delta H}}_{\mathrm{AE}}$, a similar procedure is applied
as in (1).

In addition, all the channels are assumed to follow the Saleh-Valenzuela
channel model [12], which is widely assumed for mmWave systems [7]. As a
result, by considering uniform linear arrays (ULA) at the Alice, Bob, and
Eve’s nodes, and a uniform planar array (UPA) for the HR-RIS, the
Alice$\rightarrow$ HR-RIS channel is given by

$\displaystyle{{\boldsymbol{\mathrm{H}}}}_{\mathrm{AR}}$
$\displaystyle=\sqrt{\frac{N_{\mathrm{A}}N}{L_{\mathrm{AR}}}}\Big{(}\mathrm{\beta}_{\mathrm{AR1}}\boldsymbol{\mathrm{a}}(\gamma_{\mathrm{AR}}^{r},\eta_{\mathrm{AR}}^{r})\boldsymbol{\mathrm{a}}^{\mathrm{H}}(\phi_{\mathrm{AR}}^{t})$
$\displaystyle+\sum_{i=2}^{L_{\mathrm{AR}}}\beta_{\mathrm{AR}i}\boldsymbol{\mathrm{a}}(\gamma_{\mathrm{AR}}^{r},\eta_{\mathrm{AR}}^{r})\boldsymbol{\mathrm{a}}^{\mathrm{H}}(\phi_{\mathrm{AR}}^{t})\Big{)},$
(2)

where $L_{\mathrm{AR}}$ is the number of resolvable paths,
$\beta_{\mathrm{{AR}1}}\sim\mathcal{CN}(0,10^{-0.1\kappa_{\mathrm{{AR}1}}})$
is the complex gain of the line-of-sight (LOS) path,
$\beta_{\mathrm{AR}i}\sim\mathcal{CN}(0,10^{-0.1\kappa_{\mathrm{AR}i}})$ is
the complex gain of the $i$-th non-LOS (NLOS) path. The path losses
$\kappa_{\mathrm{AR}1}$ and $\kappa_{\mathrm{AR}i}$ are given by

$\displaystyle\kappa_{\mathrm{AR}1}$
$\displaystyle=a_{\mathrm{LOS}}+10b_{\mathrm{LOS}}\log_{10}(d_{\mathrm{AR}})+\mu_{\mathrm{LOS}},$
(3) $\displaystyle\kappa_{\mathrm{AR}i}$
$\displaystyle=a_{\mathrm{NLOS}}+10b_{\mathrm{NLOS}}\log_{10}(d_{\mathrm{AR}})+\mu_{\mathrm{NLOS}},$
(4)

where $d_{\mathrm{AR}}$ is the distance between Alice and HR-RIS, and
$\\{\mu_{\mathrm{LOS}},~{}\mu_{\mathrm{NLOS}}\\}\sim\\{\mathcal{N}(0,\sigma^{2}_{\mu_{\mathrm{LOS}}}),\mathcal{N}(0,~{}\sigma^{2}_{\mu_{\mathrm{NLOS}}})\\}$,
where $\mathcal{N}(a,\,b)$ denotes a Gaussian distribution with mean $a$ and
variance $b$. From (2),
$\boldsymbol{\mathrm{a}}(\gamma_{\mathrm{AR}}^{r},\eta_{\mathrm{AR}}^{r})$ and
$\boldsymbol{\mathrm{a}}(\phi_{\mathrm{AR}}^{t})$ are the normalized array
response vectors associated with the receiver and transmitter node,
respectively, $\gamma_{\mathrm{AR}}^{r}$ and $\eta_{\mathrm{AR}}^{r}$ are
their azimuth and elevation angles of arrival (AoA) associated with the HR-
RIS, and $\phi_{\mathrm{AR}}^{t}$ is the angle of departure (AoD) associated
with the Alice node. Note that $\boldsymbol{\mathrm{a}}^{\mathrm{H}}$ denotes
the Hermitian transpose of a vector $\boldsymbol{\mathrm{a}}$. By assuming an
antenna spacing of $\lambda/2$, the normalized array response vectors can be
expressed as

$\displaystyle[\boldsymbol{\mathrm{a}}(\gamma_{\mathrm{AR}}^{r},\eta_{\mathrm{AR}}^{r})]_{n}$
$\displaystyle=\frac{1}{\sqrt{N}}[e^{j\pi(m-1)\cos(\eta_{\mathrm{AR}}^{r})\sin(\gamma_{\mathrm{AR}}^{r})}$
$\displaystyle\quad\times
e^{j\pi(n-1)\sin(\eta_{\mathrm{AR}}^{r})}]^{\mathrm{T}},m=1,\cdots,N_{x}$
$\displaystyle\quad n=1,\cdots,N_{y},$ (5)
$\displaystyle[\boldsymbol{\mathrm{a}}(\phi_{\mathrm{AR}}^{t})]_{k}$
$\displaystyle=e^{j\pi(k-1)\sin(\theta_{Xi})},k=1,\cdots,N_{\mathrm{A}},$ (6)

where $N_{x}$ and $N_{y}$ are the number of rows and columns of coefficients
for a given HR-RIS, such that $N_{x}\times N_{y}=N$. Additionally, the
remaining channels
${{\boldsymbol{\mathrm{H}}}}_{\mathrm{RB}},{{\boldsymbol{\mathrm{H}}}}_{\mathrm{AB}},\bar{{{\boldsymbol{\mathrm{H}}}}}_{\mathrm{RE}}$,
and $\bar{{{\boldsymbol{\mathrm{H}}}}}_{\mathrm{AE}}$ can be modeled similarly
as in (2)–(6), and more details can be found in [13].

###  II-B Signal Model

For the optimal design of reflecting/amplifying coefficients, the phases and
amplitudes of the incident signals are adjusted so that the signals are added
constructively at $\mathrm{Bob}$. Let

$\alpha_{n}=\begin{cases}|\alpha_{n}|e^{j\theta_{n}},&\text{if}~{}n\in\mathbb{A}\\\
e^{j\theta_{n}},&\text{otherwise}\end{cases}$ (7)

denote the reflecting/amplifying coefficient associated with the $n$th element
of the HR-RIS, where $\theta_{n}\in(0,2\pi]$ is the phase shift and
$\mathbb{A}$ is the set of the positions of the active elements. Accordingly,
let $\boldsymbol{\Upsilon}=$
diag$\\{\alpha_{1},...,\alpha_{N}\\}\in\mathbb{C}^{N\times N}$ be the diagonal
matrix of the HR-RIS’s coefficients, and
$\boldsymbol{\Psi}\in\mathbb{C}^{N\times N}$ be the diagonal matrix including
only the active elements of HR-RIS. Therefore, the received signal at
$\mathrm{Bob}$, denoted by
$\boldsymbol{y}_{\mathrm{B}}\in\mathbb{C}^{N_{\mathrm{B}}\times 1}$, can be
expressed as

$\displaystyle\boldsymbol{y}_{\mathrm{B}}$
$\displaystyle=({{\boldsymbol{\mathrm{H}}}}_{\mathrm{RB}}\boldsymbol{\Upsilon}\mathrm{{{\boldsymbol{\mathrm{H}}}}_{\mathrm{AR}}}+{{\boldsymbol{\mathrm{H}}}}_{\mathrm{AB}})\boldsymbol{s}+{{\boldsymbol{\mathrm{H}}}}_{\mathrm{RB}}\boldsymbol{\Psi}\boldsymbol{n}_{\mathrm{R}}+\boldsymbol{n}_{\mathrm{B}}$
$\displaystyle=({{\boldsymbol{\mathrm{H}}}}_{\mathrm{RB}}\boldsymbol{\Upsilon}\mathrm{{{\boldsymbol{\mathrm{H}}}}_{\mathrm{AR}}}+{{\boldsymbol{\mathrm{H}}}}_{\mathrm{AB}})\boldsymbol{s}+\boldsymbol{n}_{\mathrm{T}},$
(8)

where $\boldsymbol{n}_{\mathrm{R}}\in\mathbb{C}^{N\times 1}$ and
$\boldsymbol{n}_{\mathrm{B}}\in\mathbb{C}^{N_{\mathrm{B}}\times 1}$ are the
complex additive Gaussian noise (AWGN) modeled with zero-mean circularly
symmetric complex Gaussian, with covariance matrices
$\sigma_{\mathrm{H}}^{2}{{\boldsymbol{\mathrm{I}}}}_{N}$ and
$\sigma_{\mathrm{B}}^{2}{{\boldsymbol{\mathrm{I}}}}_{N_{\mathrm{B}}}$,
respectively. Note that
$\boldsymbol{n}_{\mathrm{T}}={{\boldsymbol{\mathrm{H}}}}_{\mathrm{RB}}\boldsymbol{\Psi}\boldsymbol{n}_{\mathrm{R}}+\boldsymbol{n}_{\mathrm{B}}$
is the total effective noise at $\mathrm{Bob}$. Similarly, the received signal
at Eve, i.e.,
$\boldsymbol{\mathrm{y}}_{\mathrm{E}}\in\mathbb{C}^{N_{\mathrm{E}}\times 1}$
is given by

$\displaystyle\boldsymbol{\mathrm{y}}_{\mathrm{E}}$
$\displaystyle=({{\boldsymbol{\mathrm{H}}}}_{\mathrm{RE}}\boldsymbol{\Upsilon}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{AR}}+{{\boldsymbol{\mathrm{H}}}}_{\mathrm{AE}})\boldsymbol{s}+{{\boldsymbol{\mathrm{H}}}}_{\mathrm{RE}}\boldsymbol{\Psi}\boldsymbol{n}_{\mathrm{R}}+\boldsymbol{n}_{\mathrm{E}}$
$\displaystyle=({{\boldsymbol{\mathrm{H}}}}_{\mathrm{RE}}\boldsymbol{\Upsilon}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{AR}}+{{\boldsymbol{\mathrm{H}}}}_{\mathrm{AE}})\boldsymbol{s}+\boldsymbol{n}_{\mathrm{TE}},$
(9)

where $\boldsymbol{n}_{\mathrm{E}}\in\mathbb{C}^{N_{\mathrm{E}}\times 1}$ is
the AWGN with covariance matrix
$\sigma_{\mathrm{E}}^{2}{{\boldsymbol{\mathrm{I}}}}_{N_{\mathrm{E}}}$, and
$\boldsymbol{n}_{\mathrm{TE}}={{\boldsymbol{\mathrm{H}}}}_{\mathrm{RB}}\boldsymbol{\Psi}\boldsymbol{n}_{\mathrm{R}}+\boldsymbol{n}_{\mathrm{E}}$.
For ease of exposition, we assume that
$\sigma_{\mathrm{H}}^{2}=\sigma_{\mathrm{B}}^{2}=\sigma_{\mathrm{E}}^{2}=\sigma^{2}$,
$\boldsymbol{n}_{\mathrm{T}}\sim\mathcal{CN}(0,\,\sigma^{2}({{\boldsymbol{\mathrm{I}}}}_{N_{\mathrm{B}}}+{{\boldsymbol{\mathrm{H}}}}_{\mathrm{RB}}\boldsymbol{\Psi}\boldsymbol{\Psi}^{\mathrm{H}}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{RB}}^{\mathrm{H}}))$
and
$\boldsymbol{n}_{\mathrm{TE}}\sim\mathcal{CN}(0,\,\sigma^{2}({{\boldsymbol{\mathrm{I}}}}_{N_{\mathrm{E}}}+{{\boldsymbol{\mathrm{H}}}}_{\mathrm{RE}}\boldsymbol{\Psi}\boldsymbol{\Psi}^{\mathrm{H}}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{RE}}^{\mathrm{H}}))$.
For simplicity, let us consider
${{\boldsymbol{\mathrm{H}}}}_{\mathrm{B}}={{\boldsymbol{\mathrm{H}}}}_{\mathrm{RB}}\boldsymbol{\Upsilon}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{AR}}+{{\boldsymbol{\mathrm{H}}}}_{\mathrm{AB}}$
and
${{\boldsymbol{\mathrm{H}}}}_{\mathrm{E}}={{\boldsymbol{\mathrm{H}}}}_{\mathrm{RE}}\boldsymbol{\Upsilon}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{AR}}+{{\boldsymbol{\mathrm{H}}}}_{\mathrm{AE}}$.
Hence, the channel capacities between Alice and Bob and between Alice and Eve
can be respectively given by

$\displaystyle C_{l}$
$\displaystyle=\log_{2}\bigg{(}\det\bigg{(}{{\boldsymbol{\mathrm{I}}}}_{N_{\mathrm{B}}}+\frac{1}{\sigma^{2}}{{\boldsymbol{\mathrm{Q}}}}_{\mathrm{B}}^{-1}{\boldsymbol{\mathrm{H}}}_{\mathrm{B}}{{\boldsymbol{\mathrm{Q}}}}_{\mathrm{A}}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{B}}^{\mathrm{H}}\bigg{)}\bigg{)}$
(10) $\displaystyle C_{e}$
$\displaystyle=\log_{2}\bigg{(}\det\bigg{(}{{\boldsymbol{\mathrm{I}}}}_{N_{\mathrm{E}}}+\frac{1}{\sigma^{2}}{{\boldsymbol{\mathrm{Q}}}}_{\mathrm{E}}^{-1}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{E}}{{\boldsymbol{\mathrm{Q}}}}_{\mathrm{A}}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{E}}^{\mathrm{H}}\bigg{)}\bigg{)},$
(11)

respectively, where
${{\boldsymbol{\mathrm{Q}}}}_{\mathrm{A}}\in\mathbb{C}^{N_{\mathrm{A}}\times
N_{\mathrm{A}}}=\mathbb{E}\\{\boldsymbol{s}\boldsymbol{s}^{\mathrm{H}}\\}$ is
the transmit covariance matrix, satisfying
$\mathrm{tr}({{\boldsymbol{\mathrm{Q}}}}_{\mathrm{A}})\leq P_{T}$, in which
$P_{T}$ is the transmit power at Alice, and $\mathrm{tr}(\cdot)$ denotes the
trace of a squared matrix;
${{\boldsymbol{\mathrm{Q}}}}_{\mathrm{B}}\in\mathbb{C}^{N_{\mathrm{B}}\times
N_{\mathrm{B}}}={{\boldsymbol{\mathrm{I}}}}_{N_{\mathrm{B}}}+{{\boldsymbol{\mathrm{H}}}}_{\mathrm{RB}}\boldsymbol{\Psi}\boldsymbol{\Psi}^{\mathrm{H}}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{RB}}^{\mathrm{H}}$
and
${{\boldsymbol{\mathrm{Q}}}}_{\mathrm{E}}\in\mathbb{C}^{N_{\mathrm{E}}\times
N_{\mathrm{E}}}={{\boldsymbol{\mathrm{I}}}}_{N_{\mathrm{E}}}+{{\boldsymbol{\mathrm{H}}}}_{\mathrm{RE}}\boldsymbol{\Psi}\boldsymbol{\Psi}^{\mathrm{H}}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{RE}}^{\mathrm{H}}$
are the aggregate noise covariance matrices. By considering a single-stream
beamforming scheme [14], the optimal solution to
${{\boldsymbol{\mathrm{Q}}}}_{\mathrm{A}}$ has rank-one and can be expressed
as
${{\boldsymbol{\mathrm{Q}}}}_{\mathrm{A}}=\boldsymbol{\mathrm{w}}\boldsymbol{\mathrm{w}}^{\mathrm{H}}$,
where $\boldsymbol{\mathrm{w}}\in\mathbb{C}^{N_{\mathrm{A}}\times 1}$ is the
optimal transmit beamforming vector. Thus, by replacing the optimal
${{\boldsymbol{\mathrm{Q}}}}_{\mathrm{A}}$ into (10) and (11), the channel
capacities can be expressed as

$\displaystyle C_{l}$
$\displaystyle=\log_{2}\bigg{(}\det\bigg{(}{{\boldsymbol{\mathrm{I}}}}_{N_{\mathrm{B}}}+\frac{1}{\sigma^{2}}{{\boldsymbol{\mathrm{Q}}}}_{\mathrm{B}}^{-1}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{B}}{{\boldsymbol{\mathrm{w}}}}{{\boldsymbol{\mathrm{w}}}}^{\mathrm{H}}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{B}}^{\mathrm{H}}\bigg{)}\bigg{)}$
(12) $\displaystyle C_{e}$
$\displaystyle=\log_{2}\bigg{(}\det\bigg{(}{{\boldsymbol{\mathrm{I}}}}_{N_{\mathrm{E}}}+\frac{1}{\sigma^{2}}{{\boldsymbol{\mathrm{Q}}}}_{\mathrm{E}}^{-1}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{E}}{{\boldsymbol{\mathrm{w}}}}{{\boldsymbol{\mathrm{w}}}}^{\mathrm{H}}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{E}}^{\mathrm{H}}\bigg{)}\bigg{)},$
(13)

From (12) and (13), the secrecy capacity can be obtained as

$\displaystyle C_{S}$
$\displaystyle=[C_{l}-C_{e}]^{+}\triangleq\mathrm{max}[C_{l}-C_{e},0].$ (14)

##  III Secrecy Capacity Maximization

In this section, the solution to the secrecy capacity maximization problem is
obtained via an alternating optimization (AO), which alternatively optimize
the transmit beamforming at Alice, and the HR-RIS coefficients. More
specifically, we first find the solution to ${{\boldsymbol{\mathrm{w}}}}$ for
a given $\alpha_{n}$, and then, the optimal solution to $\alpha_{n}$ are
obtained for a given ${{\boldsymbol{\mathrm{w}}}}$.

###  III-A Transmit Beamforming Optimization

First, with fixed HR-RIS coefficients, i.e., fixed $\alpha_{n}$, we aim at
finding the optimal ${{\boldsymbol{\mathrm{w}}}}$ by solving the following
problem:

$\displaystyle\max_{{{\boldsymbol{\mathrm{w}}}}}$ $\displaystyle C_{l}-C_{e}$
(15) s.t. $\displaystyle||{{\boldsymbol{\mathrm{w}}}}||^{2}\leq P_{T},$

Note that we have neglected the operator $[\cdot]^{+}$ from (14) due to the
fact that the optimal values must be non-negative [4].

According to the Sylvester’s determinant identity, we have that
$\det({{\boldsymbol{\mathrm{I}}}}_{P}+\boldsymbol{\mathrm{AB}})=\det({{\boldsymbol{\mathrm{I}}}}_{Q}+\boldsymbol{\mathrm{BA}})$
for arbitrary matrices $\boldsymbol{\mathrm{A}}\in\mathbb{C}^{P\times Q}$ and
$\boldsymbol{\mathrm{B}}\in\mathbb{C}^{Q\times P}$. Therefore, $C_{l}$ and
$C_{e}$ in (12) and (13) can be rewritten as

$\displaystyle C_{l}$
$\displaystyle=\log_{2}\big{(}\det\big{(}{{\boldsymbol{\mathrm{I}}}}_{N_{\mathrm{B}}}+\frac{1}{\sigma^{2}}{{\boldsymbol{\mathrm{Q}}}}_{\mathrm{B}}^{-1}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{B}}{{\boldsymbol{\mathrm{w}}}}{{\boldsymbol{\mathrm{w}}}}^{\mathrm{H}}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{B}}^{\mathrm{H}}\big{)}\big{)}$
$\displaystyle\quad=\log_{2}\big{(}1+\frac{1}{\sigma^{2}}{{\boldsymbol{\mathrm{w}}}}^{\mathrm{H}}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{B}}^{\mathrm{H}}{{\boldsymbol{\mathrm{Q}}}}_{\mathrm{B}}^{-1}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{B}}{{\boldsymbol{\mathrm{w}}}}\big{)},$
(16) $\displaystyle C_{e}$
$\displaystyle=\log_{2}\big{(}\det\big{(}{{\boldsymbol{\mathrm{I}}}}_{N_{\mathrm{E}}}+\frac{1}{\sigma^{2}}{{\boldsymbol{\mathrm{Q}}}}_{\mathrm{E}}^{-1}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{E}}{{\boldsymbol{\mathrm{w}}}}{{\boldsymbol{\mathrm{w}}}}^{\mathrm{H}}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{E}}^{\mathrm{H}}\big{)}\big{)}$
$\displaystyle\quad=\log_{2}\big{(}1+\frac{1}{\sigma^{2}}{{\boldsymbol{\mathrm{w}}}}^{\mathrm{H}}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{E}}^{\mathrm{H}}{{\boldsymbol{\mathrm{Q}}}}_{\mathrm{E}}^{-1}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{E}}{{\boldsymbol{\mathrm{w}}}}\big{)}.$
(17)

Thus, problem (15) can be re-expressed as

$\displaystyle\max_{{{\boldsymbol{\mathrm{w}}}}}$
$\displaystyle\log_{2}\Bigg{(}\frac{1+\frac{1}{\sigma^{2}}{{\boldsymbol{\mathrm{w}}}}^{\mathrm{H}}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{B}}^{\mathrm{H}}{{\boldsymbol{\mathrm{Q}}}}_{\mathrm{B}}^{-1}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{B}}{{\boldsymbol{\mathrm{w}}}}}{1+\frac{1}{\sigma^{2}}{{\boldsymbol{\mathrm{w}}}}^{\mathrm{H}}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{E}}^{\mathrm{H}}{{\boldsymbol{\mathrm{Q}}}}_{\mathrm{E}}^{-1}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{E}}{{\boldsymbol{\mathrm{w}}}}}\Bigg{)}$
(18) s.t. $\displaystyle||{{\boldsymbol{\mathrm{w}}}}||^{2}\leq P_{T}.$

For ease of exposition, let us denote
$\boldsymbol{\mathrm{L}}=\frac{1}{\sigma^{2}}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{B}}^{\mathrm{H}}{{\boldsymbol{\mathrm{Q}}}}_{\mathrm{B}}^{-1}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{B}}$,
and
$\boldsymbol{\mathrm{E}}=\frac{1}{\sigma^{2}}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{E}}^{\mathrm{H}}{{\boldsymbol{\mathrm{Q}}}}_{\mathrm{E}}^{-1}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{E}}$,
which are constant with respect to variable ${{\boldsymbol{\mathrm{w}}}}$.
Furthermore, due to the monotonic increasing property of $\log_{2}(\cdot)$, we
can rewrite (18) as

$\displaystyle\max_{{{\boldsymbol{\mathrm{w}}}}}$
$\displaystyle\frac{1+{{\boldsymbol{\mathrm{w}}}}^{\mathrm{H}}\boldsymbol{\mathrm{L}}{{\boldsymbol{\mathrm{w}}}}}{1+{{\boldsymbol{\mathrm{w}}}}^{\mathrm{H}}\boldsymbol{\mathrm{E}}{{\boldsymbol{\mathrm{w}}}}}$
(19) s.t. $\displaystyle||{{\boldsymbol{\mathrm{w}}}}||^{2}\leq P_{T},$

Finally, the optimal solution to $\boldsymbol{\mathrm{w}}$ can be found as
[15]

$\boldsymbol{\mathrm{w}^{*}}=\sqrt{P_{T}}{{\boldsymbol{\mathrm{v}}}}_{\mathrm{m}}\bigg{[}\bigg{(}\boldsymbol{\mathrm{E}}+\frac{1}{P_{T}}{{\boldsymbol{\mathrm{I}}}}_{N_{\mathrm{A}}}\bigg{)}^{-1}\bigg{(}\boldsymbol{\mathrm{L}}+\frac{1}{P_{T}}{{\boldsymbol{\mathrm{I}}}}_{N_{\mathrm{A}}}\bigg{)}\bigg{]},$
(20)

where ${{\boldsymbol{\mathrm{v}}}}_{\mathrm{m}}[\cdot]$ represents the
normalized eigenvector corresponding to the largest eigenvalue of its
argument.

###  III-B Optimization of the Reflecting Coefficients of the HR-RIS

With given transmit beamforming vector $\boldsymbol{\mathrm{w}}$, the optimal
solution to the HR-RIS coefficients can be found by solving the following
problem:

$\displaystyle\max_{|\alpha_{n}|,\theta_{n}}$
$\displaystyle\log_{2}\Bigg{(}\frac{1+{{\boldsymbol{\mathrm{w}}}}^{\mathrm{H}}\boldsymbol{\mathrm{L}}{{\boldsymbol{\mathrm{w}}}}}{1+{{\boldsymbol{\mathrm{w}}}}^{\mathrm{H}}\boldsymbol{\mathrm{E}}{{\boldsymbol{\mathrm{w}}}}}\Bigg{)}$
(21) s.t. $\displaystyle P_{a}(\alpha_{n})\leq P_{\mathrm{max}}$
$\displaystyle|\alpha_{n}|=1,~{}\textrm{for}~{}n\notin\mathbb{A},$

where $P_{\mathrm{max}}$ is the power budget of the active elements, and
$P_{a}(\alpha_{n})$ is the transmit power of active elements at HR-RIS, given
by

$\displaystyle P_{a}(\alpha_{n})=$
$\displaystyle\text{tr}(\boldsymbol{\Psi}({{\boldsymbol{\mathrm{H}}}}_{\mathrm{AR}}{{\boldsymbol{\mathrm{Q}}}}_{\mathrm{A}}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{AR}}^{H}+\sigma^{2}{{\boldsymbol{\mathrm{I}}}}_{N})\boldsymbol{\Psi}^{\mathrm{H}})$
$\displaystyle=$
$\displaystyle\text{tr}(\boldsymbol{\Psi}\tilde{{\boldsymbol{\mathrm{H}}}}_{\mathrm{AR}}\tilde{{\boldsymbol{\mathrm{H}}}}_{\mathrm{AR}}^{H}\boldsymbol{\Psi})+\sigma^{2}\text{tr}(\boldsymbol{\Psi}\boldsymbol{\Psi}^{H})$
$\displaystyle=$
$\displaystyle\sum_{n\in\mathbb{A}}(|\alpha_{n}|^{2}||\tilde{{\boldsymbol{\mathrm{t}}}_{n}}||^{2})+\sigma^{2}\sum_{n\in\mathbb{A}}|\alpha_{n}|^{2}$
$\displaystyle=$ $\displaystyle\sum_{n\in\mathbb{A}}|\alpha_{n}|^{2}\xi_{n},$
(22)

where $\xi_{n}\triangleq|\tilde{t}_{n}|^{2}+\sigma^{2}$, and $\tilde{t}_{n}$
is the ${n}$th element of
$\tilde{{\boldsymbol{\mathrm{h}}}}_{\mathrm{AR}}\triangleq{{\boldsymbol{\mathrm{H}}}}_{\mathrm{AR}}{{\boldsymbol{\mathrm{w}}}}\in\mathbb{C}^{N\times
1}$. On the other hand, let
${\boldsymbol{\mathrm{r}}}_{n}\in\mathbb{C}^{N_{\mathrm{B}}\times 1}$ be the
$n$th column of ${{\boldsymbol{\mathrm{H}}}}_{\mathrm{RB}}$,
${\boldsymbol{\mathrm{t}}}_{n}^{\mathrm{H}}\in\mathbb{C}^{1\times
N_{\mathrm{A}}}$ be the $n$th row of
${{\boldsymbol{\mathrm{H}}}}_{\mathrm{AR}}$, and
${\boldsymbol{\mathrm{e}}}_{n}\in\mathbb{C}^{N_{\mathrm{E}}\times 1}$ be the
$n$th column of ${{\boldsymbol{\mathrm{H}}}}_{\mathrm{RE}}$ such that
${{\boldsymbol{\mathrm{H}}}}_{\mathrm{AR}}=\left[{\boldsymbol{\mathrm{t}}}_{1},...,{\boldsymbol{\mathrm{t}}}_{N}\right]^{\mathrm{H}}$,
${{\boldsymbol{\mathrm{H}}}}_{\mathrm{RB}}=\left[{\boldsymbol{\mathrm{r}}}_{1},...,{\boldsymbol{\mathrm{r}}}_{N}\right]$
and
${{\boldsymbol{\mathrm{H}}}}_{\mathrm{RE}}=\left[{\boldsymbol{\mathrm{e}}}_{1},...,{\boldsymbol{\mathrm{e}}}_{N}\right]$,
respectively. By noticing that $\boldsymbol{\Upsilon}$ and $\boldsymbol{\Psi}$
are diagonal matrices, we can express the effective channels as

$\displaystyle{{\boldsymbol{\mathrm{H}}}}_{\mathrm{B}}=\sum_{n=1}^{N}\alpha_{n}\boldsymbol{r}_{n}\boldsymbol{t}_{n}^{\mathrm{H}}+{{\boldsymbol{\mathrm{H}}}}_{\mathrm{AB}}$
(23)
$\displaystyle{{\boldsymbol{\mathrm{H}}}}_{\mathrm{E}}=\sum_{n=1}^{N}\alpha_{n}\boldsymbol{e}_{n}\boldsymbol{t}_{n}^{\mathrm{H}}+{{\boldsymbol{\mathrm{H}}}}_{\mathrm{AE}},$
(24)

and similarly, the covariance matrices can be reformulated as

$\displaystyle{{\boldsymbol{\mathrm{Q}}}}_{\mathrm{B}}={{\boldsymbol{\mathrm{I}}}}_{N_{\mathrm{B}}}{+}\Bigg{(}\sum_{n\in\mathbb{A}}\alpha_{n}\boldsymbol{r}_{n}\Bigg{)}\Bigg{(}\sum_{n\in\mathbb{A}}\alpha_{n}\boldsymbol{r}_{n}\Bigg{)}^{\mathrm{H}}$
(25)
$\displaystyle{{\boldsymbol{\mathrm{Q}}}}_{\mathrm{E}}={{\boldsymbol{\mathrm{I}}}}_{N_{\mathrm{E}}}+\Bigg{(}\sum_{n\in\mathbb{A}}\alpha_{n}\boldsymbol{e}_{n}\Bigg{)}\Bigg{(}\sum_{n\in\mathbb{A}}\alpha_{n}\boldsymbol{e}_{n}\Bigg{)}^{\mathrm{H}}.$
(26)

Thus, from (22)–(26) we can reformulate (21) as in (27), shown on the top of
the next page. Note we can also express the optimization problem in terms of
${|\alpha_{n}|}$ and $\theta_{n}$ by replacing (7) into (27).

$\displaystyle\max_{\alpha_{n}}$
$\displaystyle\log_{2}\Bigg{(}\frac{1{+}\frac{1}{\sigma^{2}}{{\boldsymbol{\mathrm{w}}}}^{\mathrm{H}}(\sum_{n=1}^{N}\alpha_{n}\boldsymbol{r}_{n}\boldsymbol{t}_{n}^{\mathrm{H}}{+}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{AB}})^{\mathrm{H}}\\{[{{\boldsymbol{\mathrm{I}}}}_{N_{\mathrm{B}}}+(\sum_{n\in\mathbb{A}}\alpha_{n}\boldsymbol{r}_{n})(\sum_{n\in\mathbb{A}}\alpha_{n}\boldsymbol{r}_{n})]^{\mathrm{H}}\\}^{-1}(\sum_{n=1}^{N}\alpha_{n}\boldsymbol{r}_{n}\boldsymbol{t}_{n}^{\mathrm{H}}{+}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{AB}}){{\boldsymbol{\mathrm{w}}}}}{1{+}\frac{1}{\sigma^{2}}{{\boldsymbol{\mathrm{w}}}}^{\mathrm{H}}(\sum_{n=1}^{N}\alpha_{n}\boldsymbol{e}_{n}\boldsymbol{t}_{n}^{\mathrm{H}}{+}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{AE}})^{\mathrm{H}}\\{[{{\boldsymbol{\mathrm{I}}}}_{N_{\mathrm{E}}}+(\sum_{n\in\mathbb{A}}\alpha_{n}\boldsymbol{e}_{n})(\sum_{n\in\mathbb{A}}\alpha_{n}\boldsymbol{e}_{n})]^{\mathrm{H}}\\}^{-1}(\sum_{n=1}^{N}\alpha_{n}\boldsymbol{e}_{n}\boldsymbol{t}_{n}^{\mathrm{H}}{+}{{\boldsymbol{\mathrm{H}}}}_{\mathrm{AE}}){{\boldsymbol{\mathrm{w}}}}}\Bigg{)}$
(27) s.t. $\displaystyle\sum_{n\in\mathbb{A}}|\alpha_{n}|^{2}\xi_{n}\leq
P_{\mathrm{max}},$
$\displaystyle|\alpha_{n}|=1,~{}\textrm{for}~{}n\notin\mathbb{A},$

Due to the non-convexity of the optimization problem in (27) the optimal
values for the active and passive elements of the HR-RIS cannot be obtained in
a closed form. However, we can provide local optimum results by applying
Particle Swarm Optimization (PSO) method as in [16]. The proposed PSO method
is presented in Algorithm 1.

Algorithm 1 PSO-based HR-RIS coefficients optimization

1:Set $m_{\mathrm{I}}$, $n_{\mathrm{P}}$,
${{\boldsymbol{\mathrm{x}}}}_{\mathrm{min}}$,
${{\boldsymbol{\mathrm{x}}}}_{\mathrm{max}}$, $w$, $c_{1}$, $c_{2}$, $CF$;

2:Randomly initialize the particles ;

3:for $t$ = 1 : $m_{\mathrm{I}}$ do

4: for $i$ = 1 : $n_{\mathrm{P}}$ do

5: Update the velocity:
${{\boldsymbol{\mathrm{v}}}}_{i}(t+1)=w*{{\boldsymbol{\mathrm{v}}}}_{i}(t)+r_{1}*c_{1}*({{\boldsymbol{\mathrm{p}}}}_{i}(t)-{{\boldsymbol{\mathrm{x}}}}_{i}(t))+r_{2}*c_{2}*({{\boldsymbol{\mathrm{p}}}}_{\mathrm{g}}(t)-{{\boldsymbol{\mathrm{x}}}}_{i}(t))$;

6: Update the position:
${{\boldsymbol{\mathrm{x}}}}_{i}(t+1)={{\boldsymbol{\mathrm{x}}}}_{i}(t)+{{\boldsymbol{\mathrm{v}}}}_{i}(t+1)$;

7: Evaluate the $CF$$({{\boldsymbol{\mathrm{x}}}}_{i}(t+1))$;

8: Update the personal and global best positions:

9: if $CF$(${{\boldsymbol{\mathrm{x}}}}_{i}(t+1)$ <
$CF$(${{\boldsymbol{\mathrm{p}}}}_{i}(t))$ then

10:
${{\boldsymbol{\mathrm{p}}}}_{i}(t+1)\leftarrow{{\boldsymbol{\mathrm{x}}}}_{i}(t+1)$

11: if $CF$(${{\boldsymbol{\mathrm{p}}}}_{i}(t+1)$) <
$CF$(${{\boldsymbol{\mathrm{p}}}}_{\mathrm{g}}(t))$ then

12:
${{\boldsymbol{\mathrm{p}}}}_{\mathrm{g}}(t+1)\leftarrow{{\boldsymbol{\mathrm{x}}}}_{i}(t+1)$

13: end if

14: end if

15: end for

16:end for

For the initial parameters, the maximum number of iterations, the number of
particles for the population, the inertia coefficient, the acceleration
coefficient for the cognitive component, the acceleration for the global
component, and the cost function are denoted by $m_{\mathrm{I}}$,
$n_{\mathrm{P}}$, $w$, $c_{1}$, $c_{2}$, $CF$, respectively. The vectors
${{\boldsymbol{\mathrm{x}}}}_{\mathrm{min}},{{\boldsymbol{\mathrm{x}}}}_{\mathrm{max}}$
represent the range of all reflecting coefficients, where
${{\boldsymbol{\mathrm{x}}}}_{\mathrm{min}}\in\mathbb{C}^{1\times 2N}$ =
$[1,...,1,0,...,0]$, and
${{\boldsymbol{\mathrm{x}}}}_{\mathrm{max}}\in\mathbb{C}^{1\times
2N}=[\sqrt{P_{\mathrm{max}}/\min\\{\xi_{1,...,K}\\}},...,\sqrt{P_{\mathrm{max}}/\min\\{\xi_{1,...,K}\\}},...,1,...,\\\
1,...,2\pi,...,2\pi]$. To ensure convergence, the stability and convergence
criteria are adopted based on constriction coefficient [16], given by

$\displaystyle\chi=\frac{2}{|2-\kappa-\sqrt{\kappa^{2}-4\kappa}|},$ (28)

where $\kappa=\kappa_{1}+\kappa_{2}$. For this approach, we set ${w}=\chi$,
$c_{1}=\chi\kappa_{1}$, $c_{2}=\chi\kappa_{2}$, and, in order to guarantee a
faster response, we adopt $\kappa_{1}=\kappa_{2}=2.05$ [16]. For the velocity
update expression, we consider $\\{r_{1},r_{2}\\}\sim\mathcal{U}(0,1)$, where
$\mathcal{U}(a,b)$ denotes a continuous uniform distribution with interval
$[a,b]$. We express the particle’s positions as vectors,
${{\boldsymbol{\mathrm{x}}}}_{i}(t)\in\mathbb{C}^{1\times
2N}=[|\alpha_{1}|,...,|\alpha_{K}|,...,|\alpha_{N}|,\theta_{1},...,\theta_{N}]$.
The first subset of each particle $\\{|\alpha_{1}|,...,|\alpha_{K}|\\}$
represents the amplitudes of the active coefficients of HR-RIS, the second
subset $\\{|\alpha_{K+1}|,...,|\alpha_{N}|\\}$ = 1 represents the remaining
amplitudes for the passive elements of HR-RIS, and for the last subset,
$\\{|\theta_{1}|,...,|\theta_{N}|\\}$ represents the phase shifts.

By noticing that an equivalent form of the optimization problem in (27), $CF$
can be expressed as

$\displaystyle CF({{\boldsymbol{\mathrm{x}}}})=$
$\displaystyle-\frac{1+{{\boldsymbol{\mathrm{w}}}}^{\mathrm{H}}\boldsymbol{\mathrm{L}}{{\boldsymbol{\mathrm{w}}}}}{1+{{\boldsymbol{\mathrm{w}}}}^{\mathrm{H}}\boldsymbol{\mathrm{E}}{{\boldsymbol{\mathrm{w}}}}}+P_{c}\left(\sum_{n=1}^{K}|{{\boldsymbol{\mathrm{x}}}}_{n}|^{2}\xi_{n}-P_{\mathrm{max}}\right),$
(29)

where $P_{c}$ is the punishment constraint. The Algorithm 1 starts by randomly
generating $n_{\mathrm{P}}$ particles, where each particle is limited to
${{\boldsymbol{\mathrm{x}}}}_{\mathrm{min}}$ and
${{\boldsymbol{\mathrm{x}}}}_{\mathrm{max}}$. We set the initial personal best
position to
${{\boldsymbol{\mathrm{p}}}}_{1}(1)={{\boldsymbol{\mathrm{x}}}}_{1}(1)$ and
the global best position to
${{\boldsymbol{\mathrm{p}}}}_{\mathrm{g}}(1)={{\boldsymbol{\mathrm{x}}}}_{n}$,
such that
$CF({{\boldsymbol{\mathrm{x}}}}_{n})<CF({{\boldsymbol{\mathrm{x}}}}_{i}),\forall
i=1,\cdots,n_{\mathrm{P}},i\neq n$. Then, steps 3–16 are executed in order to
find the optimal local solution,
${{\boldsymbol{\mathrm{p}}}}_{\mathrm{g}}(m_{\mathrm{I}})$. Notice that it
guarantees convergence due to the fact that for each iteration, all particles
are moving towards the personal and global optimum solution.

##  IV Numerical Results and Discussions

In this section, we evaluate the secrecy performance of the proposed system
through simulations of illustrative cases. According to real-world LOS channel
measurements [13], $a_{\mathrm{LOS}},b_{\mathrm{LOS}}$ and
$\mu_{\mathrm{LOS}}$ are, respectively, given by 61.4, 2, and 5.8 dB, and for
NLOS channel measurements, $a_{\mathrm{NLOS}}=72,b_{\mathrm{NLOS}}=2.92$, and
$\mu_{\mathrm{NLOS}}=8.7$ dB. For this system, we set the bandwidth to
${B}=251.2$ MHz, and the noise figure to ${NF}=10$ dB, and the noise power is
set to $\sigma^{2}=-174+10\log({B})+{NF}\approx-80$ dBm. Unless otherwise
specified, the transmit power at Alice is set to $P_{T}=20$ dBm, the number of
reflecting elements in the HR-RIS to $N$ = 40, the number of active elements
at HR-RIS to $K=2$, $N_{\mathrm{A}}=4,N_{\mathrm{B}}=2$ and
$N_{\mathrm{E}}=2$, $~{}\epsilon=0.1$, and the total number of resolvable
paths for each Saleh-Valenzuela channel are
$L_{\mathrm{AR}}=L_{\mathrm{RB}}=L_{\mathrm{AB}}=L_{\mathrm{RE}}=L_{\mathrm{AE}}=3$.
For Algorithm 1 parameters, we consider a population size of
$\textit{nPop}=20$, the maximum number of iterations to $\textit{mIt}=30$, and
the punishment constraint to $P_{c}=10^{3}$. We consider a two-dimensional
topology network, where Alice, HR-RIS, Bob and Eve are respectively located at
(0,0), (80,2), (90,0) and (100,0). Furthermore, the simulation results are
averaged over 1000 independent channel realizations. For comparison, we
evaluate the secrecy capacity of the following benchmark schemes: (i) Case
without RIS: optimization of transmit beamforming only; (ii) Case with optimal
beamforming via (20), and optimal phases considering a passive RIS: an AO-
based approach is considered, where the amplitudes of RIS are set to 1, and
the phases are optimized based on Algorithm 1; (iii) Case with optimal
beamforming as in (20) and optimal HR-RIS: a joint optimization between the
transmit beamforming $\boldsymbol{\mathrm{w}}$, amplitudes and phase
components of the HR-RIS is proposed based on (20) and Algorithm 1.

Figure 2: Secrecy capacity versus transmit power, considering different number
of antenna configurations for $N_{\mathrm{A}},N_{\mathrm{B}}$, and
$~{}N_{\mathrm{E}}$. The corresponding parameters are:
$\sigma_{\Delta}=0.1,N=40$. For the HR-RIS parameters, we consider
$K=2,P_{\mathrm{max}}$ = 10 dBm.

Fig. 2 shows the secrecy capacity versus the transmit power at Alice, $P_{T}$,
assuming different configurations of antenna elements at Alice, Bob and Eve.
In addition, we set $\epsilon=0.1$ and the reflecting coefficients at both
passive RIS and HR-RIS to $N=40$. For the HR-RIS parameters, we fix the number
of active elements to $K=2$, and the maximum power budget for the active
elements to $P_{\mathrm{max}}$ = 10 dBm. The secrecy capacity for the HR-RIS
scheme outperforms their counterparts, passive RIS and without RIS.
Furthermore, in the HR-RIS scheme, the secrecy capacity performance improves
as more antennas are employed at Bob node, considering a low-to-medium
transmit power regime. Otherwise, at high transmit power regime, a larger
number of antennas at the Alice leads to the best performance in terms of
secrecy capacity. We can also observe that, even for the HR-RIS scheme in the
worst case scenario (i.e., for $(N_{\mathrm{A}},N_{\mathrm{B}}$,
$N_{\mathrm{E}})=(2,2,4)$), it is shown that HR-RIS allows higher secrecy
rates than a passive RIS case with equal number of antennas at each node
(i.e., for $(N_{\mathrm{A}},N_{\mathrm{B}}$, $N_{\mathrm{E}})=(2,2,2)$).

Figure 3: Secrecy capacity versus distance between Alice and Eve, considering
$N=40,\sigma_{\Delta}=0.1$, and $(N_{\mathrm{A}},N_{\mathrm{B}}$,
$N_{\mathrm{E}})=(4,2,2)$. For the HR-RIS parameters, we consider
$K=\\{2,4\\}$ and $P_{\mathrm{max}}=\\{5,10\\}$ dBm.

Fig. 3 shows the secrecy capacity as a function of the distance between Alice
and Eve, $d_{\mathrm{AE}}$, considering different configurations of $K$ and
$P_{\mathrm{max}}$. Note that the system secrecy capacity performance can be
improved significantly with the use of the HR-RIS scheme, even when the
eavesdropper is closer to Alice. This can be explained by the fact that, from
the closed-form solution in (20), the transmit beamforming is designed to add
the transmitted signal constructively at Bob while adding destructively at
Eve’s node. For the HR-RIS scheme, note that the secrecy capacity decreases
when the eavesdropper node approaches the HR-RIS. This behavior can be caused
because, in this case, the channel condition for the HR-RIS$\rightarrow$Eve is
dramatically better than HR-RIS$\rightarrow$Bob counterpart. Also, even with
the optimized solution for the active and passive coefficients, a limited and
fixed active elements are not sufficient to overcome this issue. For a higher
$P_{\mathrm{max}}$, note that a higher secrecy capacity performance is
attained. However, when eavesdropper’s positions are closer to the HR-RIS, the
secrecy capacity is highly compromised. All in all, the HR-RIS system
outperforms the other benchmark schemes in terms of the secrecy capacity.

Figure 4: Secrecy capacity versus transmit power, for $N=\\{16,40\\}$ and
$\sigma_{\Delta}=\\{0.1,0.5\\}$, considering $(N_{\mathrm{A}},N_{\mathrm{B}}$,
$N_{\mathrm{E}})=(4,2,2)$. For the HR-RIS parameters, we consider
$K=2,P_{\mathrm{max}}$ = 10 dBm.

Fig. 4 illustrates the secrecy capacity versus the transmit power at Alice,
$P_{T}$, for different numbers of reflecting coefficients and different
normalized variance of the error component, $\epsilon$, considering the
passive RIS and HR-RIS schemes. For a high transmit power regime, by assuming
the same CSI’s eavesdropper accuracy, the proposed HR-RIS scheme with $N=14$
passive reflecting coefficients and $K=2$ active relay coefficients
outperforms the conventional RIS with $N=40$ reflecting coefficients’
counterpart. It is also observed that a larger normalized error variance leads
to an inaccurate joint transmit beamforming design at Alice and reflecting
coefficients at RIS, resulting in a lower system secrecy capacity.

##  V Conclusion

In this work, we investigated the secrecy capacity maximization of an HR-RIS-
assisted mmWave MIMO system, in the presence of a passive multi-antenna
eavesdropper, with imperfect CSI from eavesdropping links. We proposed an AO-
based algorithm by using the closed-form solution for the transmit beamformer
and PSO. For comparison purposes, we considered two benchmark schemes referred
to as $\mathrm{Op_{RIS}}$ and $\mathrm{No~{}RIS}$ schemes, which consider an
optimized passive RIS scheme and a system with no RIS, respectively. From our
results, the secrecy capacity for the proposed HR-RIS scheme showed to
outperform both $\mathrm{Op_{RIS}}$ and $\mathrm{No~{}RIS}$ counterparts.

## Acknowledgements

This work has been supported in part by Academy of Finland, 6G Flagship
program (Grant 346208), FAITH Project (Grant 334280), as well as Infotech of
Oulu Graduate School.

## References

  * [1] X. Guan, Q. Wu, and R. Zhang, “Intelligent reflecting surface assisted secrecy communication: Is artificial noise helpful or not?” _IEEE Wireless Commun. Lett._ , vol. 9, no. 6, pp. 778–782, 2020.
  * [2] D. P. Moya Osorio, J. D. Vega Sanchez, and H. Alves, _Physical Layer Security for 5G and Beyond in 5G REF: The Essential 5G Reference Online_. John Wiley & Sons, 2019.
  * [3] Q. Wu and R. Zhang, “Intelligent reflecting surface enhanced wireless network via joint active and passive beamforming,” _IEEE Trans. Wireless Commun._ , vol. 18, no. 11, pp. 5394–5409, 2019.
  * [4] M. Cui, G. Zhang, and R. Zhang, “Secure wireless communication via intelligent reflecting surface,” _IEEE Wireless Commun. Lett._ , vol. 8, no. 5, pp. 1410–1414, 2019.
  * [5] K. Feng, X. Li, Y. Han, S. Jin, and Y. Chen, “Physical layer security enhancement exploiting intelligent reflecting surface,” _IEEE Commun. Lett._ , vol. 25, no. 3, pp. 734–738, 2021.
  * [6] X. Yu, D. Xu, Y. Sun, D. W. K. Ng, and R. Schober, “Robust and secure wireless communications via intelligent reflecting surfaces,” _IEEE J. Sel. Areas Commun_ , vol. 38, no. 11, pp. 2637–2652, 2020.
  * [7] J. Qiao and M.-S. Alouini, “Secure transmission for intelligent reflecting surface-assisted mmWave and terahertz systems,” _IEEE Wireless Commun. Lett._ , vol. 9, no. 10, pp. 1743–1747, 2020.
  * [8] Y. Xiu, J. Zhao, C. Yuen, Z. Zhang, and G. Gui, “Secure beamforming for multiple intelligent reflecting surfaces aided mmWave systems,” _IEEE Commun. Lett._ , vol. 25, no. 2, pp. 417–421, 2021.
  * [9] E. Björnson, O. Özdogan, and E. G. Larsson, “Intelligent reflecting surface vs. decode-and-forward: How large surfaces are needed to beat relaying?” _IEEE Wireless Commun. Lett._ , vol. 9, no. 2, pp. 244–248, 2019\.
  * [10] N. T. Nguyen, D. Vu, K. Lee, and M. Juntti, “Hybrid relay-reflecting intelligent surface-assisted wireless communications,” _IEEE Trans. Veh. Technol._ , pp. 1–1, 2022.
  * [11] C. Wang, Z. Li, T.-X. Zheng, H. Chen, and X.-G. Xia, “Robust hybrid precoding design for securing millimeter wave IoT networks under secrecy outage constraint,” _IEEE Internet Things J._ , pp. 1–1, 2021.
  * [12] A. Saleh and R. Valenzuela, “A statistical model for indoor multipath propagation,” _IEEE J. Sel. Areas Commun_ , vol. 5, no. 2, pp. 128–137, 1987\.
  * [13] P. Wang, J. Fang, L. Dai, and H. Li, “Joint transceiver and large intelligent surface design for massive MIMO mmWave systems,” _IEEE Trans. Wireless Commun._ , vol. 20, no. 2, pp. 1052–1064, 2021.
  * [14] J. Wang, H. Wang, Y. Han, S. Jin, and X. Li, “Joint transmit beamforming and phase shift design for reconfigurable intelligent surface assisted MIMO systems,” _IEEE Trans. Cogn. Commun._ , vol. 7, no. 2, pp. 354–368, 2021\.
  * [15] K. Feng, X. Li, Y. Han, S. Jin, and Y. Chen, “Physical layer security enhancement exploiting intelligent reflecting surface,” _IEEE Commun. Lett._ , vol. 25, no. 3, pp. 734–738, 2021.
  * [16] M. Clerc and J. Kennedy, “The particle swarm-explosion, stability, and convergence in a multidimensional complex space,” _IEEE Trans. Evol. Comput._ , vol. 6, no. 1, pp. 58–73, 2002.