# Paying More Attention to Self-attention: Improving Pre-trained Language
Models via Attention Guiding

Shanshan Wang  Shandong UniversityQingdaoChina<EMAIL_ADDRESS>,
Zhumin Chen ∗ Shandong UniversityQingdaoChina<EMAIL_ADDRESS>,
Zhaochun Ren  Shandong UniversityQingdaoChina<EMAIL_ADDRESS>,
Huasheng Liang  WeChat, TencentGuangzhouChina<EMAIL_ADDRESS>,  Qiang
Yan  WeChat, TencentGuangzhouChina<EMAIL_ADDRESS>and  Pengjie Ren
Shandong UniversityQingdaoChina<EMAIL_ADDRESS>
(2022)

###### Abstract.

PLM have demonstrated their effectiveness for a broad range of information
retrieval and natural language processing tasks. As the core part of pre-
trained language models, multi-head self-attention is appealing for its
ability to jointly attend to information from different positions. However,
researchers have found that PLMs always exhibit fixed attention patterns
regardless of the input (e.g., excessively paying attention to ‘[CLS]’ or
‘[SEP]’), which we argue might neglect important information in the other
positions. In this work, we propose a simple yet effective attention guiding
mechanism to improve the performance of PLMs through encouraging the attention
towards the established goals. Specifically, we propose two kinds of attention
guiding methods, i.e., the attention map discrimination guiding (MDG) and the
attention pattern decorrelation guiding (PDG). The former definitely
encourages the diversity among multiple self-attention heads to jointly attend
to information from different representation subspaces, while the latter
encourages self-attention to attend to as many different positions of the
input as possible. We conduct experiments with multiple general pre-trained
models (i.e., BERT, ALBERT, and Roberta) and domain-specific pre-trained
models (i.e., BioBERT, ClinicalBERT, BlueBert, and SciBERT) on three benchmark
datasets (i.e., MultiNLI, MedNLI, and Cross-genre-IR). Extensive experimental
results demonstrate that our proposed MDG and PDG bring stable performance
improvements on all datasets with high efficiency and low cost.

Pre-trained models, multi-head self-attention, attention guiding, attention
map discrimination, attention pattern decorrelation

††copyright: rightsretained††journalyear: 2022††doi:
10.1145/1122445.1122456††price: 15.00††isbn: 978-1-4503-XXXX-X/18/06††ccs:
Information systems Clustering and classification††ccs: Information systems
Content analysis and feature selection††ccs: Computing methodologies
Contrastive learning††submissionid: 756

##  1\. Introduction

PLM have led to tremendous performance increase in a wide range of downstream
tasks, including machine translation (Ott et al., 2018), text classification
(Sun et al., 2019), document ranking (Qiao et al., 2019), etc. The core
component of pre-trained language models (PLMs) is the self-attention
mechanism, which allows the model to capture long-range dependency
information.

Recently, many studies focus on analyzing the self-attention mechanism, i.e.,
the weights and connections of attention, to interpret the network or
revealing the characteristic of PLMs (Clark et al., 2019; Kovaleva et al.,
2019; Vig and Belinkov, 2019). These exploration works have found a common
phenomenon: despite the success of the self-attention mechanism, these
language models exhibit simple attention patterns (Raganato et al., 2018;
Voita et al., 2019). For example, Devlin et al. (2019) and Kovaleva et al.
(2019) report the phenomenon that 40% of heads in a pre-trained BERT model
simply pay attention to the delimiters, such as ‘[CLS]’ and/or ‘[SEP].’
Moreover, Michel et al. (2019) demonstrate that multi-headed attentions in WMT
(Vaswani et al., 2017) and BERT (Devlin et al., 2019) are not necessary to
obtain competitive performance. Likewise, Raganato et al. (2020) also confirm
that most attentive connections in the encoder do not need to be learned at
all, because most self-attention patterns learned by the transformer
architecture merely reflect the positional encoding of contextual information.

The effectiveness of self-attention can be improved by introducing a variety
of information. For example, Li et al. (2018) demonstrate that the downstream
task can be improved by increasing the diversity of attention heads. Besides,
many researches focus on modifying self-attention through external information
such as syntactic supervision (Xia et al., 2021; Qi et al., 2020; Li et al.,
2020) to improve the input representation. Their results suggest that adding
additional information does help PLMs improve the effectiveness of downstream
tasks. However, since these methods modify the computational process of self-
attention, they must re-train the PLMs from scratch. As we all know, training
a PLM with a large amount of data from scratch will take a lot of computing
resources and time. In addition, extracting additional information, such as
syntactic structure, will further increase the computational burden.

Therefore, we seek to investigate the following research question in this
paper: is it possible to guide self-attention without extra information in the
fine-tuning phrase to improve the performance of downstream tasks? As shown in
Figure 1, the learned attention heads from PLMs without guiding always present
similar patterns, e.g., different heads attend to similar positions. On the
contrast, we seek to design an attention guiding mechanism so that
comprehensive and diverse information can be taken into account. We expect the
attention guiding mechanism acts as auxiliary objective to regularize the
fine-tuning of downstream tasks. A similar work is done in (Deshpande and
Narasimhan, 2020). They use several pre-defined attention patterns to guide
the training of PLMs. Specifically, they add an auxiliary loss to guide the
self-attention heads towards a set of pre-defined patterns (i.e., ‘[Next]’,
‘[Prev]’, ‘[First]’, ‘[Delim]’, and ‘[Period]’). Since these pre-defined
patterns only cover a few fixed patterns and cannot introduce more
information, the proposed method has limited ability in improving the
diversity of attention heads. Therefore, in this work, we propose to explore
the self-attention guiding methods without pre-defining attention patterns or
extra knowledge about the input to encourage the diversity among multiple
attention heads.

Figure 1. Illustration of attention guiding mechanism. The learned attention
heads always exhibits fixed and similar patterns, which might neglect
important information. Attention guiding aims to guide the learned attention
heads to attend to different parts of the inputs so that more important
information can be taken into account.

We propose an attention guiding mechanism to improve the performance of PLMs
by regularizing its self-attention by explicitly encourage the diversity among
multiple attention heads. Specifically, we propose two kinds of attention
guiding methods, i.e., the MDG and the PDG. The former is used to encourage
self-attention to attend to the information from different aspects of the
inputs by diverse attention maps. An attention map is distinctive in its own
right, and each could differ significantly from other attention maps
(Malisiewicz et al., 2011) so that it can capture the information of input
differently with others. The latter is used to improve the diversity of
attention patterns by encouraging self-attention to pay attention to more
diverse positions by reducing the correlations with different attention
patterns. We validate the effectiveness of the attention guiding mechanism on
multiple general and domain-specific PLMs by conducting experiments on three
benchmark datasets. Especially, we found that the proposed attention guiding
mechanism is still effective on small-scale datasets, demonstrating its
significance for low-resource settings.

Our main contributions are as follows:

  * •

We propose two self-attention guiding terms, i.e., MDG and PDG, for guiding
self-attention heads which enable PLMs to learn comprehensive and diverse
attention patterns.

  * •

We demonstrate the effectiveness of the attention guiding mechanism on seven
general and domain-specific PLMs across three different datasets and tasks.

##  2\. Methodology

###  2.1. Tasks

In this work, we take the following two different tasks as applications.

Task 1: Natural Language Inference. The goal of this task is to predict
whether a given hypothesis can be inferred from a given promise. This task is
formulated as a multi-class classification task. In the implementation, we add
a classification layer on top of the ‘[CLS]’ representation derived from the
output of the last layer of PLMs, like most methods (Reimers and Gurevych,
2019; Moons et al., 2020; Yates et al., 2021). The PLMs are fine-tuned via
minimizing the multi-class cross-entropy loss, which is defined as follows:

(1) $Loss_{t}=-\frac{1}{|D|}\sum_{i}^{|D|}log(p({y_{i}}|x_{i};\theta)),$

where $\theta$ denotes all trainable parameters in the PLM and the
classification layer, $|D|$ is the number of training samples, and ${y_{i}}$
is the ground truth for the $i$-th sample $x_{i}$.

Task 2: Across Medical Genres Querying. The objective of this task is to find
the research publication that supports the primary claim made in a health-
related news article. This task is formulated as a binary classification task.
Similarly, we add a classification layer on top of the ‘[CLS]’ representation
derived from the output of the last layer of PLMs. The PLMs are fine-tuned via
minimizing the binary cross-entropy loss as follows:

(2)
$Loss_{t}=-\frac{1}{|D|}\sum_{i=1}^{|D|}[-y_{i}log(\hat{y}_{i})-(1-y_{i})log(1-\hat{y}_{i})],$

where ${y_{i}}$ is the ground truth for the $i$-th sample $x_{i}$ and
$\hat{y_{i}}=p({y_{i}}|x_{i};\theta)$ is the probability that $i$-th sample
belongs to $y_{i}$.

###  2.2. Multi-head self-attention

A pre-trained language model (PLM) is normally a large-scale and powerful
neural network trained with huge amounts of data samples and computing
resources (Lai et al., 2021; Choudhury and Deshpande, 2021). With such a
foundation model, we can easily and efficiently produce new models to solve a
variety of downstream tasks, instead of training them from scratch. PLMs rely
on multi-head self-attention to capture dependencies between tokens (Devlin et
al., 2019). Given a hidden state $\bm{Z}$, multi-head self-attention first
projects it linearly into queries $\bm{Q}_{h}$, keys $\bm{K}_{h}$, and values
$\bm{V}_{h}$ using parameter matrices
$\bm{W}_{h}^{Q},\bm{W}_{h}^{K},\bm{W}_{h}^{V}$, respectively. The formulation
is as follows:

(3)
$\bm{Q}_{h},\bm{K}_{h},\bm{V}_{h}=\bm{Z}\bm{W}_{h}^{Q},\bm{Z}\bm{W}_{h}^{K},\bm{Z}\bm{W}_{h}^{V}.\\\
$

Then, the self-attention distribution $\bm{A}_{h}$ is computed via scaled dot-
product of query $\bm{Q}_{h}$ and key $\bm{K}_{h}$. These weights are assigned
to the corresponding value vectors $\bm{V}_{h}$ to obtain output states
$\bm{O}_{h}$:

(4)
$\bm{O}_{h}=\bm{A}_{h}\bm{V}_{h}~{}~{}with~{}~{}\bm{A}_{h}=softmax(\frac{\bm{Q}_{h}\bm{K}_{h}^{\top}}{\sqrt{d_{k}}}).\\\
$

Here $\bm{A}_{h}$ is the attention distribution produced by the $h$-th
attention head. $d_{k}$ is the hidden size. Finally, the output states
$\bm{O}_{h}$ of all heads are concatenated to produce the final states
$\bm{Z}$.

Figure 2.  The pipeline of the proposed PLM with attention guiding approach.
The attention guiding mechanism contains attention map discrimination guiding
(MDG) and attention pattern decorrelation guiding (PDG), respectively. For a
batch sample, we use the pre-trained model to encode each sample as a feature
vector $\bm{x_{i}}$, and as well as we obtain the attention matrix $\bm{M}$
for each sample. The optimal feature embedding is learned via task objective
plus the losses produced by two attention guiding methods, i.e., MDG and PDG.
Both MDG and PDG try to maximally scatter the attention heads of training
samples over the sequence length unit sphere.

###  2.3. Attention guiding

Figure 2 shows the workflow of our methodology. Firstly, we use the pre-
trained model to encode each sample $x_{i}$ and obtain its corresponding
attention matrix $\bm{M}$. Then the task-objective loss is calculated based on
sample representation $\bm{x}_{i}$. Meanwhile, the attention matrix $\bm{M}$
will be guided by two terms, i.e., attention map discrimination guiding (MDG)
and attention pattern decorrelation guiding (PDG), which aim to push the
diversity of self-attention and further improve the performance of the
downstream task. More specifically, for both of downstream tasks, besides the
objectives, we also introduce two auxiliary objectives to guide the self-
attention. Formally, the training objective of each task is revised as:

(5) $Loss_{total}=Loss_{t}+\alpha Loss_{mdg}+\beta Loss_{pdg},$

where $L_{total}$ is the ultimate training goal, and it contains three parts
of training loss. $Loss_{t}$ represents the task object, which varies with the
task. $Loss_{mdg}$ denotes the attention map discrimination guiding (MDG)
term, and $loss_{pdg}$ denotes the attention pattern decorrelation guiding
(PDG) term. These two terms can be either used individually or in combination
and they are regulated using $\alpha$ and $\beta$ respectively. Note that the
introduced regularization terms work like $L_{1}$ and $L_{2}$ terms which
don’t introduce any new parameters and only influence the fine-tuning of the
standard model parameters.

####  2.3.1. Attention Map Discrimination

For a given sample $x_{i}$, firstly, we need to get the representation of this
sample and its corresponding attention maps from the output of the pre-trained
models, such as BERT. The formula can be described as:

(6)
$\bm{h}_{i},\\{\bm{A}_{1},\bm{A}_{2},\bm{A}_{i},...,\bm{A}_{N_{l}*N_{h}}\\}=PLM(xi|\theta),$

where $\bm{h}_{i}$ denotes the hidden state of ‘[CLS]’ token and we regard it
as the representation of sample $x_{i}$.
$\\{\bm{A}_{1},\bm{A}_{2},\bm{A}_{i},...,\bm{A}_{N_{l}*N_{h}}\\}$ is the set
of multi-layer multi-head attention maps. $\bm{A}_{i}$ is the $i$-th attention
map, and there are $N_{l}\ast N_{h}$ attention maps produced by the pre-
trained model. $N_{l}$ and $N_{h}$ denote the layer number, and head number in
each layer, respectively.

Then, we add a classification layer on top of the ‘[CLS]’ representation
$\bm{h}_{i}$. Formally, the formula of the classification layer is implemented
by:

(7) $\bm{x}_{i}=\bm{W}Relu(\bm{h}_{i}),$

where $\bm{W}$ is the weight matrix, $Relu(\cdot)$ is the activation function,
and $\bm{x}_{i}$ is the final representation of the given sample. By Eq.1 or
Eq.2 which is determined by the downstream task, we can calculate the task-
objective loss $L_{t}$ with the supervision from the ground-truth label of
sample $x_{i}$.

To simplify the calculation, each attention map
$\bm{A}_{i}\in\mathbb{R}^{L\times L}$ is processed as one vector
$\bm{m}_{i}\in\mathbb{R}^{L}$ by summing up the attention values that all
tokens received. The corresponding formula of transforming the attention map
$\bm{A}_{i}$ to the attention vector $\bm{m}_{i}$ is:

(8) $\bm{m}_{i}=\sum_{j}\bm{A}_{i,j},$

where $i$ represents the $i$-th attention map and $j$ is the column index of
the attention map $\bm{A}_{i}$.

Since the self-attention mechanism in PLM is multi-layer multi-head
architecture, there are multiple attention vectors are produced. we organize
all the attention vectors, into a matrix $\bm{M}\in\mathbb{R}^{(N_{l}\ast
N_{h})\times L}$. Specifically, we concatenate all the attention vector
$m_{i}$ to construct the attention matrix $\bm{M}$. Formally, the
corresponding formula is as follows:

(9)
$\bm{M}=\bm{m}_{1}\oplus\bm{m}_{2}\oplus\bm{m}_{i},...,\oplus\bm{m}_{N_{l}\ast
N_{h}},$

where $\oplus$ denotes the concatenate operation and
$\bm{M}\in\mathbb{R}^{(N_{l}\ast N_{h})\times L}$ represents the attention
matrix.

Inspired by (Wu et al., 2018; Tao et al., 2021), we apply the instance
discrimination method to push the diversity of attention maps so that the rich
information of the input can be captured. The objective function is formulated
based on the softmax criterion. Each attention map is assumed to represent a
distinct class. That is, attention map $\bm{m}_{i}\in\mathbb{R}^{L}$, i.e. the
$i$-row of the attention matrix $\bm{M}$, is classified into the $i$-th class.
Accordingly, the weight vector for the $i$-th class can be approximated by a
vector $\bm{m}_{i}$. The probability of one attention map $\bm{m}$ being
assigned into the $i$-th class is:

(10)
$P(i|\bm{m})=\frac{exp(\bm{m}_{i}^{\top}\bm{m}/\tau)}{{\textstyle\sum_{j=1}^{N_{l}\ast
N_{h}}exp(\bm{m}_{j}^{\top}\bm{m}/\tau)}},$

where $\bm{m}_{j}^{\top}\bm{m}$ measures how well $\bm{m}$ matches the $j$-th
class because $\bm{m}_{j}$ is regarded as the weight of $j$-th class. $\tau$
is a temperature parameter that controls the concentration of the distribution
(Hinton et al., 2015), and $m$ is normalized to $\left\|m\right\|=1$. The
objective maximizes the joint probability ${\textstyle\prod_{i=1}^{N_{l}\ast
N_{h}}}P_{\theta}(i|f_{\theta}(\bm{m}_{i}))$ as

(11) $\begin{split}Loss_{mdg}&=-\sum_{i=1}^{N_{l}\ast
N_{h}}logP(i|f_{\theta}(\bm{m}_{i})),\\\ &=-\sum_{i}^{N_{l}\ast
N_{h}}log(\frac{exp(\bm{m}_{i}^{\top}\bm{m}_{i}/\tau)}{{\sum_{j=1}^{N_{l}\ast
N_{h}}exp(\bm{m}_{j}^{\top}\bm{m}_{i}/\tau)}}).\end{split}$

####  2.3.2. Attention Pattern Decorrelation

We have analyzed that the multi-head attention heads are likely to suffer from
the redundancy problem where each attention vector focuses on a very similar
region. To encourage each attention head to capture the information from
different positions of the inputs, at the same time, we propose another
attention guiding term, i.e., the attention pattern decorrelation guiding
(PDG).

Inspired by (Li et al., 2021a), we regard the $l$-th column of $\bm{M}$, i.e.,
$\bm{M}^{\top}_{l}\in\mathbb{R}^{N_{l}\ast N_{h}}$ as the soft representation
of the $l$-th attention pattern. Conventionally, attention patterns should be
independent to ensure that redundant information is reduced. The objective
function is applied to push the diversity of attention patterns and reduce
pattern redundancy, which tries to make the cross-correlation matrix computed
from the attention matrix as close to the identity matrix as possible.

The formula of PDG term aims to construct independent attention patterns and
is as follows:

(12) $Loss_{pdg}=\left\|\bm{M}^{\top}\bm{M}-\bm{I}\right\|_{F}^{2},$

where $\bm{M}\in\mathbb{R}^{(N_{l}\ast N_{h})\times L}$ is the attention
matrix, $\bm{I}\in\mathbb{R}^{L\times L}$ is the identity matrix and
$||.||^{2}_{F}$ denotes the squared Frobenius Norm (Yuan et al., 2020).
$\bm{M}^{\top}\bm{M}\in\mathbb{R}^{L\times L}$ can be seen as the cross-
correlation matrix of different attention patterns. Minimizing the difference
between the cross-correlation matrix and the identity matrix $\bm{I}$ is
equivalent to making the attention patterns diverse so that they focus on
different tokens (Park et al., 2020; Zbontar et al., 2021).

##  3\. Experimental Setup

To evaluate the effectiveness of our proposed attention guiding mechanism, we
conduct extensive experiments of a variety of pre-trained models on different
downstream tasks. We demonstrate that the attention guiding mechanism can
promote the diversity of attention heads and further improve the performance
on different downstream tasks.

###  3.1. Datasets and evaluation

Table 1. Performance comparison (%) of PLMs with or without attention guiding. Bold face indicates the improved performance with AG in terms of the corresponding metrics. ∗ means $p<0.05$ and ∗∗ means $p<0.01$ in t-test, respectively. SOTA represents the best performing methods by Nov. 2021, to the best of our knowledge. The matched test set of MultiNLI for the experiments.  Methods | MultiNLI | MedNLI | Cross-genre-IR
---|---|---|---
ACC | Precision | Recall | F1 | ACC | Precision | Recall | F1 | MRR | R@1 | R@3 | R@5 | R@20
SOTA | 87.90 | – | – | – | 84.00 | – | – | – | 69.50 | 62.50 | 74.30 | 78.30 | 82.80
BERT | 83.24 | 83.19 | 83.17 | 83.17 | 76.02 | 76.25 | 76.01 | 76.11 | 73.18 | 62.84 | 80.38 | 85.07 | 94.58
BERT+AG | 83.89 | 83.91 | 83.82 | 83.83 | 77.22∗ | 77.35∗ | 77.24∗ | 77.26∗ | 73.38 | 63.69 | 80.74 | 84.59 | 94.09
ALBERT | 83.09 | 83.05 | 83.04 | 83.02 | 77.76 | 78.04 | 77.78 | 77.84 | 71.00 | 62.06 | 76.07 | 80.02 | 93.21
ALBERT+AG | 84.41∗∗ | 84.44∗∗ | 84.37∗∗ | 84.37∗∗ | 78.76∗∗ | 78.72∗∗ | 78.80∗∗ | 78.74∗∗ | 71.86∗ | 62.06 | 78.08∗ | 82.95∗ | 93.42∗
Roberta | 85.95 | 85.91 | 85.94 | 85.91 | 80.17 | 80.12 | 80.12 | 80.14 | 78.67 | 71.04 | 83.58 | 86.85 | 94.86
Roberta+AG | 86.77∗ | 86.67∗ | 86.72∗ | 86.67∗ | 80.31 | 80.36 | 80.31 | 80.32 | 79.88∗ | 72.05∗ | 84.47∗ | 87.70∗ | 95.68∗
BioBERT | 81.37 | 81.35 | 81.33 | 81.28 | 81.86 | 81.81 | 81.84 | 81.83 | 83.72 | 76.48 | 88.83 | 92.52 | 97.21
BioBERT+AG | 81.54∗∗ | 81.46∗∗ | 81.47∗∗ | 81.45∗∗ | 82.63 | 82.68 | 82.65 | 82.64 | 83.76 | 75.87 | 90.17 | 92.88 | 96.84
ClinicalBERT | 79.68 | 79.81 | 79.69 | 79.66 | 80.73 | 80.67 | 80.71 | 80.66 | 71.83 | 61.99 | 78.30 | 83.58 | 92.03
ClinicalBERT+AG | 80.07∗∗ | 80.13∗∗ | 80.07∗∗ | 80.09∗∗ | 82.63∗ | 82.68∗ | 82.67∗ | 82.64∗ | 72.02∗ | 62.24∗ | 78.18 | 83.70∗ | 91.91
BlueBERT | 79.13 | 79.10 | 79.10 | 79.06 | 83.90 | 83.94 | 83.91 | 83.92 | 69.68 | 59.65 | 75.67 | 82.08 | 91.75
BlueBERT+AG | 79.21∗ | 79.17∗ | 79.17∗ | 79.13∗ | 84.32∗∗ | 84.46∗∗ | 84.33∗∗ | 84.34∗∗ | 71.02∗ | 61.47∗ | 77.61∗ | 81.53 | 91.38
SciBERT | 83.61 | 83.68 | 83.56 | 83.58 | 79.47 | 79.42 | 79.47 | 79.43 | 80.72 | 72.80 | 86.65 | 90.57 | 96.48
SciBERT+AG | 83.74∗ | 83.79∗ | 83.70∗ | 83.71∗ | 80.03∗ | 80.03∗ | 80.11∗ | 83.01∗ | 81.67∗ | 72.84∗ | 88.07∗ | 91.50∗ | 97.41∗

We conduct experiments on the following datasets.

  * •

MultiNLI111https://cims.nyu.edu/~sbowman/multinli/ (Williams et al., 2018) is
a crowd-sourced collection of 433k sentence pairs annotated with textual
entailment information, i.e., entailment, contradiction, and neutral. This
dataset is for the natural language inference task, which is also popular for
evaluating various PLMs (Deshpande and Narasimhan, 2020; Wang et al., 2021a).
Accuracy (ACC for short) is standard metric on this task. At the same time, we
also report other metrics commonly used in the classification tasks, such as
Precision, Recall and F1.

  * •

MedNLI222https://physionet.org/content/mednli/1.0.0/ (Romanov and Shivade,
2018) is for natural language inference in clinical domain, which has the same
data structure as MultiNLI. Accuracy is also the standard metric on this
dataset. Like on the MultiNLI, we also report the Precision, Recall and F1.

  * •

Cross-genre-IR333https://github.com/chzuo/emnlp2020-cross-genre-IR (Zuo et
al., 2020) is for the across medical genres querying task, where each claim
(i.e., he news headline) is associated with at least one peer-reviewed
research publication supporting it. For each claim, it needs to re-rank the
candidate publications to obtain the correct ones. Following the original
authors, we report the Mean Reciprocal Rank (i.e., MRR) and Recall@K (i.e.,
R@K = 1, 3, 5, 20) metrics.

###  3.2. PLMs for comparison

We consider seven transformer-based PLMs: three are pre-trained over general
language corpora (BERT, ALBERT, and Roberta) and four are pre-trained over
biomedical corpora (BioBERT, ClinicalBERT, BlueBert, and SciBERT).

  * •

BERT444https://huggingface.co/bert-base-uncased (Devlin et al., 2019) is a
multi-layer bidirectional Transformer encoder. Since the following versions of
the PLMs are often based on the BERT-base-uncased version (12 layers and 768
hidden size with 108M parameters), we use the BERT-base-uncased here for a
fair comparison.

  * •

Roberta555https://huggingface.co/roberta-base (Liu et al., 2019) has the same
architecture as BERT, but with a lot of changes on the training mechanism,
such as a more random mask mechanism. We use the Roberta-base here for
comparison.

  * •

ALBERT666https://huggingface.co/albert-base-v2 (Lan et al., 2020) compresses
the architecture of BERT by factorized embedding parameterization and cross-
layer parameter sharing. We use the ALBERT-base-v2 version.

  * •

BioBERT777https://huggingface.co/dmis-lab/biobert-v1.1 (Lee et al., 2020) is
the first BERT pre-trained on biomedical corpora. It is initialized with
BERT’s pre-trained parameters and then further pre-trained over PubMed
abstracts and full-text articles. We use the best version BioBERT V1.1.

  * •

ClinicalBERT888https://huggingface.co/emilyalsentzer/Bio˙ClinicalBERT
(Alsentzer et al., 2019) is initialized from BioBert v1.0 and further pre-
trained over approximately 2 million notes in the MIMIC-III v1.4 database.

  * •

BlueBERT999https://huggingface.co/ttumyche/bluebert (Peng et al., 2019) is
firstly initialized from BERT and further pre-trained over biomedical corpus
of PubMed and clinical notes.

  * •

SciBERT101010https://huggingface.co/nfliu/scibert˙basevocab˙uncased (Beltagy
et al., 2019) is a BERT-base model pre-trained on 1.4M papers from the
semantic scholar, with 18% of papers from the computer science and 82% from
the biomedical domain.

  * •

SOTA. We also compare with state-of-the-art methods on each dataset, which are
based on Roberta, BlueBERT and BERT, to the best of our knowledge (Tavan et
al., 2021; Peng et al., 2019; Zuo et al., 2020).

###  3.3. Implementation details

The proposed attention guiding mechanism acts on all attention heads out from
PLMs. We fine-tune all PLM models for 5 epochs, 20 epochs, and 5 epochs on the
MultiNLI, MedNLI, and Cross-genre-IR datasets, respectively. The hidden size
$d_{k}$ is 768 and sequence length $L$ is set to 256 of each PLM. We use the
Adam optimizer (learning rate 1e-05) for all models and the batch size is set
as the maximum according to the memory of a GeForce RTX 3090 GPU.
Specifically, the batch size of ALBERT is set to 56 and other PLMs is 64 on
different datasets. The $\alpha$ and $\beta$ in Eq. 5 are selected from the
set {0.1, 0.01, 0.001, 0.0001} according to grid search. The temperature
parameter in Eq. 11 is set to 1.0.

###  3.4. Results on different pre-trained models

The results of all PLMs on different tasks are listed in Table 1. From the
results, we have several observations.

First, the proposed AG can improve the performance of the PLMs on all tasks
generally. For example, on the MultiNLI and MedNLI datasets, all the selected
pre-trained models with attention guiding can promote the performance of
downstream tasks on all PLMs in terms of all metrics. Similarly, on the Cross-
genre-IR dataset, most of the metrics of the task can be promoted by our
attention guiding method. Moreover, the encouraging findings are that simply
adding AG to BlueBERT (i.e., BlueBERT+AG) outperforms SOTA on the MedNLI
dataset, and BioBERT+AG is far better than SOTA on the Cross-genre-IR dataset.
These figures show the effectiveness of the proposed attention guiding. The
reason why AG works is that the self-attention after guiding has a better
ability to attend to broader and more diverse tokens, which benefit for
downstream tasks.

Second, AG plays different roles on different models and datasets. For
example, the proposed attention mechanism always improves the performances on
MultiNLI and MedNLI datasets, while on the Cross-genre-IR dataset, some
metrics drop slightly, e.g., R@20 drops from 94.58% to 94.09% of BERT after
attention guiding. Moreover, we also observe that the performances of
different PLMs are always improved in terms of MRR, and more PLMs can be
improved when K is small in terms of R@K metrics. This suggests that the
attention guiding method may be influenced by some charismatics of the
evaluation metrics. For instance, the R@20 is difficult to be promoted by
attention guiding. However, although there is a slight decrease in some
metrics, in most cases, our attention guiding method can improve the pre-
trained model effectively.

Third, according to our results, AG plays a bigger role on small datasets. For
example, the biggest improvements of AG reach 3.58% in terms of F1 and 2.95%
in terms of R@5 on the MedNLI (11k sentence pairs) and Cross-genre-IR (48k
sentence pairs) datasets, respectively, which are greater than these on the
MultiNLI (443k sentence pairs) dataset, i.e., 0.98%. To further explore this
phenomenon, we vary the training size of MedNLI dataset to evaluate AG’s role,
and the details refer to §4.3.

###  3.5. Comparison with different attention guiding methods

Table 2. Performance comparison (%) of PLMs with different guiding methods. Significant improvements over the best baseline results are marked with ∗ (t-test, $p<0.05$). Methods | MultiNLI | MedNLI | Cross-genre-IR
---|---|---|---
ACC | Precision | Recall | F1 | ACC | Precision | Recall | F1 | MRR | R@1 | R@3 | R@5 | R@20
BERT | 83.24 | 83.19 | 83.17 | 83.17 | 76.02 | 76.25 | 76.01 | 76.11 | 73.18 | 62.84 | 80.38 | 85.07 | 94.58
BERT+AG | 83.89∗ | 83.91∗ | 83.82∗ | 83.83∗ | 77.22∗ | 77.35 | 77.24∗ | 77.26∗ | 73.38 | 63.69 | 80.74 | 84.59 | 94.09
+[First] | 83.65 | 83.63 | 83.58 | 83.59 | 76.02 | 76.38 | 76.01 | 76.13 | 72.81 | 62.24 | 80.24 | 85.44 | 94.70
+[Next] | 83.71 | 83.74 | 83.64 | 83.66 | 76.93 | 77.23 | 76.93 | 77.04 | 72.84 | 63.79 | 77.69 | 84.47 | 93.61
+[Prev] | 83.65 | 83.67 | 8.3.57 | 83.59 | 77.07 | 77.52 | 77.06 | 77.15 | 70.98 | 60.60 | 77.53 | 83.39 | 93.37
+[Period] | 83.70 | 83.71 | 83.64 | 83.65 | 76.02 | 76.38 | 76.01 | 76.13 | 73.85 | 64.40 | 79.87 | 85.56 | 93.73
+[Delim] | 83.65 | 83.67 | 83.59 | 83.60 | 75.67 | 75.71 | 75.66 | 75.69 | 70.97 | 61.69 | 76.31 | 82.16 | 90.86
+[PMI] | 83.68 | 83.69 | 83.61 | 83.62 | 77.07 | 77.60 | 77.05 | 77.16 | 72.41 | 62.61 | 79.11 | 84.29 | 93.12
+[Dependency] | 83.59 | 83.61 | 83.53 | 83.55 | 76.37 | 76.61 | 76.37 | 76.46 | 73.10 | 64.37 | 78.38 | 82.76 | 92.57
+[WordSim] | 83.73 | 83.74 | 83.67 | 83.68 | 76.72 | 77.03 | 76.72 | 76.81 | 74.15 | 64.48 | 80.91 | 85.80 | 94.22

We also study how the attention guiding mechanism compared with other related
works. Since all the related works need to re-train a new model, rather then
our work acts in the fine-tuning phase of the pre-trained model. Therefore, it
is difficult to compare them directly. For comparison, we implement different
methods to guide self-attention during the fine-tuning phase of pre-trained
models. Specifically, we implemented five kinds of attention guiding patterns
(i.e., ‘[Next]’, ‘[Prev]’, ‘[First]’, ‘[Delim]’, and ‘[Period]’) proposed by
Deshpande and Narasimhan (2020). In addition, we also implemented three kinds
of attention guiding methods based on the knowledge extracted from the input.
Specifically, the guiding methods based on knowledge are guiding attention
heads with co-occurrence relationships, syntactic dependencies, and similar
relationships between words/tokens, namely ‘[PMI]’, ‘[Dependency]’ and
‘[WordSim]’ respectively. These prior pieces of knowledge about self-attention
we used are proposed by Qi et al. (2020) and Xia et al. (2021). The results of
these different guiding methods are reported in Table 2, and Table 2 shows
that:

First, no matter which kind of guidance method can improve the effectiveness
of the pre-trained model, i.e., BERT, in varying degrees. For example, the ACC
of BERT on the MultiNLI dataset increases from 83.24% to 83.73% after the
guidance of word similarity knowledge (i.e., ‘[WordSim]’). The results also
show that self-attention heads need to be guided, as long as the guiding
method is reasonable, such as using some fixed common attention patterns or
using the knowledge derived from the input, the performance of mainstream
tasks can also be improved.

Second, although three types of attention guiding methods, proposed by ours,
Deshpande and Narasimhan (2020), Qi et al. (2020) and Xia et al. (2021) can
improve the performances of PLMs in mainstream tasks, different guiding
methods play different roles in different datasets. Our AG is superior to
other methods on almost all metrics, such as the ACC on MedNLI and MutiNLI
datasets. But on the Cross-genre-IR datasets, ‘[WordSim]’ method is better
than ours in terms of most metrics. This suggests that the effect of different
attention guiding methods may be affected by the dataset, and trying one or
more attention guiding methods on a specific dataset may maximize the
effectiveness of the pre-trained models. How to choose appropriate prior
knowledge or guidance methods for self-attention may be a problem that needs
further exploration.

##  4\. Analysis

Table 3. Performance comparison (%) of different methods. Bold face indicates the best results in terms of the corresponding metrics. The matched test set of MultiNLI for the experiments. Significant improvements over the best baseline results are marked with ∗ (t-test, $p<0.05$). Variants | MultiNLI | MedNLI | Cross-genre-IR
---|---|---|---
Roberta | BlueBERT | BioBERT
ACC | Precision | Recall | F1 | ACC | Precision | Recall | F1 | MRR | R@1 | R@3 | R@5 | R@20
Default | 86.77∗ | 86.67∗ | 86.72∗ | 86.67∗ | 84.32∗ | 84.46∗ | 84.33∗ | 84.34∗ | 83.76 | 75.87 | 90.17 | 92.88 | 96.84
w/o-AG | 85.95 | 85.91 | 85.94 | 85.91 | 83.90 | 83.94 | 83.91 | 83.92 | 83.72 | 76.48 | 88.83 | 92.52 | 97.21
w/o-MDG | 86.52 | 86.44 | 86.49 | 86.43 | 84.18 | 84.21 | 84.18 | 84.15 | 83.48 | 75.99 | 88.63 | 92.52 | 96.84
w/o-PDG | 86.57 | 86.48 | 86.52 | 86.48 | 82.98 | 83.15 | 82.98 | 83.02 | 83.28 | 75.63 | 88.83 | 92.76 | 96.72
Figure 3. Performances of each BERT layer with the proposed AG mechanism.
Figure (a), (b) and (c) demonstrate the results of using MDG, PDG, and both of
MDG and PDG to guide each layer of BERT, respectively.  Figure 4. Attention
heatmap for a random sample. Figure (a) represents attention without AG, and
Figure (b)-(d) represent attention with MDG, with PDG, and with both of MDG
and PDG, respectively. Figure 5. Attention principal component analysis for a
random sample. Figure (a) denotes the spatial distribution of multi-layer
multi-head attentions, and Figure (b)-(d) represent the attentions with MDG,
with PDG, and with both of MDG and PDG respectively.

###  4.1. Ablation study

To analyze where the improvements of AG come from, we conduct an ablation
study of the best model on each dataset. Obviously, it is much easier to
comprehend that models pre-trained by medical corpus are better at handling
medical-related tasks (i.e., MedNLI and Cross-genre-IR). Therefore, the best
pre-trained models on MultiNLI, MedNLI, and Cross-genre-IR datasets are
Roberta, BlueBERT, and BioBERT respectively. The detailed results are shown in
Table 3. We consider the following three settings: (1) w/o-AG denotes PLMs
without AG, i.e., baseline PLMs. (2) w/o-MDG denotes PLMs without the MDG term
but reserving the PDG term. (3) w/o-PDG denotes removing the PDG term but
reserving the MDG term.

The results in Table 3 show that MDG and PDG are helpful for PLMs as removing
either of them leads to a decrease in performance in almost all the metrics.
Besides, on the MedNLI and Cross-genre-IR datasets, the most obvious declines
are the variants removing PDG, i.e., w/o-PDG. This illustrates that the
attention pattern decorrelation guiding can bring more valuable attention
information for PLM because PDG can push each head to attend to different
positions of the inputs to capture diversity information.

We also note that the R@1 on the cross-genre-IR dataset declines slightly,
e.g., the R@1 drops from 76.48% to 75.87% with AG. Nevertheless, AG is still
effective as the other metrics (i.e., MRR, R@3, R@5, and R@20) still get
improvements after the AG mechanism.

###  4.2. Effect of AG on different layers

As the proposed AG (i.e., MDG and PDG) can be applied to any layers of PLMs,
we design experiments to see their effect on different layers of BERT. The
results of BERT with MDG, BERT with PDG, and BERT with both of MDG and PDG on
different layers are summarized in Figure 3. The blue dashed line indicates
BERT without AG.

From Figure 3, we can see that most layers can benefit from MDG and PDG
obviously, such as the ACC increases from 76.02% to 77.36% at layer 7 after
being guided by MDG. And similarly, the ACC is improved from 76.02% to 77.43%
at layer 9 after being guided by PDG. Moreover, lower and middle layers can
always benefit from PDG, while for top layers, there are some declines
occasionally. For example, at layer 11, the ACC drops from 76.02% to 75.81%
after being guided by PDG. On the contrary, some declines happen at the lower
and middle layers of BERT with MDG, e.g., the ACC of BERT is down 0.63% at
layer 5. That is understandable as the functions of MDG and PDG are different.
The MDG focuses on distinct attention heads, while PDG pushes each attention
head to attend to different tokens/positions of the inputs. So combining MDG
and PDG generally leads to an improvement of BERT on almost all layers. For
example, the performances increase by 0.21% and 1.51%, after being guided by
the MDG and PDG separately at layer 6, while the improvement reaches 1.76%
after combining MDG and PDG. Moreover, lower layers were found to perform
broad attention across all pairs of tokens (Clark et al., 2019). Therefore,
lower layers call for the guidance of PDG to promote the diversity of tokens
rather than MDG, compared to the middle and top layers.

###  4.3. Effect of AG on different training sizes

In Table 1, we found that the proposed AG brings more improvements on MedNLI
and Cross-Genre-IR datasets than on the MultiNLI dataset. To explore whether
our AG mechanism is affected by the size of the training datasets, we randomly
select 20% to 100% data from the training set of MedNLI for fine-tuning. The
detailed results are illustrated in Figure 6. We have the following
observations in this experiment.

The PDG and MDG can improve BERT at different training sizes generally, even
though the data size is small. Specifically, when only 20% of the training set
is used, the MDG increases the Accuracy by 0.84% (i.e., from 68.78% to 69.62%)
while PDG also improves the Accuracy by 0.70% (from 68.78% to 69.48%). The
reasons of MDG and PDG are effective on small datasets are that when there is
not enough training data, it is difficult for PLMs to be adapted to a
different task. In other words, the self-attention is not well fine-tuned.
Thus, the guidance of self-attention becomes particularly important. Moreover,
it is easy for PLMs to over-fit on small datasets. PDG and PDG could help in
some ways to alleviate such over-fitting issues.

Figure 6. Performance of BERT with AG on different amounts of training data.

###  4.4. Case study with attention heatmap

We visualize the changes of attention heads with/without the guidance of MDG
and PDG to see whether there is any diverse attention pattern after guidance.
We randomly select one sample from the test set of the MedNLI dataset and
visualize the average attention map from all heads at all layers of BERT in
Figure 4. To make it easy for observation, Figure 4 only shows the first 15
tokens in the sequence, and the selected token sequence is “[ ‘[CLS]’, ‘the’,
‘patient’, ‘denied’, ‘any’, ‘headache ’, ‘,’, ‘visual’, ‘changes’, ‘,’,
‘chest’, ‘pain’, ‘,’, ‘pl’, ‘##eur’] ".

As shown in Figure 4(d), the attention map with both MDG and PDG could pay
attention to more positions compared with Figure 4(a) which is not guided by
AG. For example, more attention is paid to the token in the last column (i.e.,
‘##eur’), which is overlooked by the attention map without AG in Figure 4(a).
In fact, the token ‘##eur’ and the previous token ‘pl’ constitute an important
medical concept ‘pleur’, which should be paid more attention. AG can make such
kinds of tokens get more attention, which is why PLMs can be improved by AG.

###  4.5. Case study with attention principal component analysis

In order to explore whether our AG mechanism promotes the richness and
diversity of self-attention, we randomly select some samples from the test set
of MedNLI dataset and perform dimensionality reduction through Principal
Component Analysis (PCA) (Jolliffe and Cadima, 2016) on BERT’s all attention
heads from all layers. Figure 5 shows the spatial distribution of each
attention head with/without the AG mechanism. From Figure 5, we can see that
the attention distributions with MDG, PDG and both of them ((b)-(d) of Figure
5) are more dispersed than the distribution without AG (see Figure 5(a)). This
suggests that the proposed AG mechanism (MDG and PDG included) is effective,
and AG does encourage self-attention to pay attention to wider positions of
the sequence.

Moreover, the distribution of multi-headed attention in Figure 5(b) (i.e.,
being guided by MDG) is more scattered than in Figure 5(c) (i.e., being guided
by PDG). Obviously, the reason is that MDG is designed to push the diversity
of different attention maps which will lead to scattered attention heads.

###  4.6. Time cost analysis

Most of the previous studies directly modify the computation process of self-
attention, e.g. (Qi et al., 2020; Xia et al., 2021), which means that they
need to re-train the PLMs. On the contrast, our method works in the fine-
tuning phrase, and does not need to re-train the PLMs. Thus, our AG also has
merits in terms of time cost.

Nevertheless, the calculation process of AG will take more time than directly
fine-tuning the pre-trained models on specific datasets. Table 4 shows the
per-epoch training time of different PLMs with or without our AG on three
datasets.

As can be seen from Table 4, the increased time cost is minor by adding AG.
Specifically, the extra time cost by AG per-epoch training is about 130
seconds, 7 seconds and 19 seconds on MultiNLI, MedNLI and Cross-genre-IR
datasets, respectively. We consider the time cost acceptable, since AG can
improve different pre-trained models significantly.

Table 4. Per-epoch training time (in seconds) on the three datasets using different PLMs with or without the proposed AG.  | MultiNLI | MedNLI | Cross-genre-IR
---|---|---|---
BERT | 2190 | 95 | 339
BERT+AG | 2334 | 112 | 361
ALBERT | 2639 | 96 | 401
ALBERT+AG | 2770 | 101 | 423
Roberta | 2190 | 82 | 340
Roberta+AG | 2344 | 87 | 354
BioBERT | 2225 | 82 | 341
BioBERT+AG | 2318 | 87 | 361
ClinicalBERT | 2228 | 82 | 339
ClinicalBERT+AG | 2372 | 85 | 355
BlueBERT | 2210 | 100 | 340
BlueBERT+AG | 2324 | 102 | 359
SciBERT | 2215 | 81 | 338
SciBERT+AG | 2345 | 86 | 360

##  5\. Related Work

Existing studies on self-attention can be roughly classified into three
groups: self-attention probing, self-attention revising and self-attention
guiding.

###  5.1. Self-attention probing

This line of research focuses on the pattern probing of self-attention, i.e.,
the analysis of the interpretability of the weights and connections in
particular. For example, Vig and Belinkov (2019) visualize attentions and
analyze the interaction between attention and syntax over a large corpus. They
find that different attentions target different parts of speech at different
layers of the model, and that the attentions align with the dependency
relations closely, especially in the middle layers. Similarly, Clark et al.
(2019) demonstrate through visual analysis and statistical analysis that the
substantial syntactic information is captured in BERT’s attentions. Kovaleva
et al. (2019) summarize 5 kinds of frequent attention patterns, called
vertical, diagonal, vertical+diagonal, block, and heterogeneous, respectively.
Voita et al. (2019) identify the most important heads in each encoder layer
using layer-wise relevance propagation, and then attempt to characterize the
roles they perform. Wang et al. (2021b) present DODRIO, an open-source
interactive visualization tool to help researchers and practitioners analyze
attention mechanisms to meet the needs of self-attention visualization.

###  5.2. Self-attention revising

This line of research modifies the attention formula to bias the attention
weights towards local areas (Xu et al., 2019; Fonollosa et al., 2019; Yang et
al., 2020). For example, Wu et al. (2019) and Yang et al. (2019) use
convolutional modules to replace self-attentions in some parts, making the
networks computationally more efficient. Raganato et al. (2020) design seven
predefined patterns, each of which takes the place of an attention head to
train Neural Machine Translation models without the need of learning them. The
advantage of this method is that it can reduce the parameter footprint without
loss of translation quality. Qi et al. (2020) consider direction mask, word
distance mask, and dependency distance mask simultaneously, and add them into
the attention calculation to obtain the structural priors. Similarly, Li et
al. (2021b) map each token into a tree node, and calculate the distance of any
two nodes, after which the distance is added to the attention calculation. Xia
et al. (2021) inject word similarity knowledge into the attention calculation
to make the BERT model aware of the word pair similarity.

###  5.3. Self-attention guiding

Different from the above two research lines, our work belongs to self-
attention guiding, which guides the learning of self-attention without
introducing any new parameters or modifying the attention calculation
formulas. Deshpande and Narasimhan (2020)’s work belongs to this category. In
their work, five fixed patterns are predefined based on the analyses of
attentions, based on which a regularization term is added to force the
attentions to approach the predefined attention patterns in the training phase
of PLMs. There are at least two differences compared with our work. First, we
do not need to predefine attention patterns. Instead, the attention is guided
adaptively through the MDG and PDG parts. Second, we do not need to train PLMs
from scratch. Our attention guiding method works in the fine-tuning phase of
the PLMs. We compared with Deshpande and Narasimhan (2020)’s work in Table 2,
demonstrating that our method achieves comparable or better performance
without introducing new knowledge or predefining attention patterns
beforehand.

##  6\. Conclusion and Future Work

In this work, we have proposed two kinds of attention guiding methods, i.e.,
the attention map discrimination guiding (MDG) and the attention pattern
decorrelation guiding (PDG), to improve the performance of PLMs by encouraging
the learned attentions to derive more information from the inputs and to be
more diverse. Experimental results of seven PLMs on three datasets have
validated the effectiveness of our proposed methods. Especially, we have found
that the proposed attention guiding mechanism works on small datasets and
large datasets, which is attractive as building large labeled dataset is time
consuming and labor intensive.

As to future work, we plan to explore how to incorporate more domain-specific
knowledge to guide self-attention learning in low-resource domains, e.g., the
relations of diseases, drugs, and symptoms in medical domains.

## Reproducibility

This work uses publicly available data. To facilitate the reproducibility of
the reported results, we release the code at
https://anonymous.4open.science/r/attentionGuiding-F6C0.

## References

  * (1)
  * Alsentzer et al. (2019) Emily Alsentzer, John R. Murphy, Willie Boag, Wei-Hung Weng, Di Jin, Tristan Naumann, and Matthew B. A. McDermott. 2019.  Publicly Available Clinical BERT Embeddings.  _CoRR_ abs/1904.03323 (2019).
  * Beltagy et al. (2019) Iz Beltagy, Kyle Lo, and Arman Cohan. 2019.  SciBERT: A Pretrained Language Model for Scientific Text. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP_. 3613–3618.
  * Choudhury and Deshpande (2021) Monojit Choudhury and Amit Deshpande. 2021.  How Linguistically Fair Are Multilingual Pre-Trained Language Models?. In _Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI_. 12710–12718.
  * Clark et al. (2019) Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D. Manning. 2019.  What Does BERT Look at? An Analysis of BERT’s Attention. In _Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, BlackboxNLP@ACL_. 276–286.
  * Deshpande and Narasimhan (2020) Ameet Deshpande and Karthik Narasimhan. 2020.  Guiding Attention for Self-Supervised Learning with Transformers. In _Findings of the Association for Computational Linguistics, EMNLP_. 4676–4686.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT_. 4171–4186.
  * Fonollosa et al. (2019) José A. R. Fonollosa, Noe Casas, and Marta R. Costa-jussà. 2019.  Joint Source-Target Self Attention with Locality Constraints.  _CoRR_ abs/1905.06596 (2019).
  * Hinton et al. (2015) Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. 2015\.  Distilling the Knowledge in a Neural Network.  _CoRR_ abs/1503.02531 (2015).
  * Jolliffe and Cadima (2016) Ian T Jolliffe and Jorge Cadima. 2016.  Principal component analysis: A review and recent developments.  _Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences_ 374, 2065 (2016), 20150202\.
  * Kovaleva et al. (2019) Olga Kovaleva, Alexey Romanov, Anna Rogers, and Anna Rumshisky. 2019.  Revealing the Dark Secrets of BERT. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP_. 4364–4373.
  * Lai et al. (2021) Huiyuan Lai, Antonio Toral, and Malvina Nissim. 2021\.  Thank you BART! Rewarding Pre-Trained Models Improves Formality Style Transfer. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP_. 484–494.
  * Lan et al. (2020) Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020\.  ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. In _8th International Conference on Learning Representations, ICLR_.
  * Lee et al. (2020) Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. 2020.  BioBERT: a pre-trained biomedical language representation model for biomedical text mining.  _Bioinform._ 36, 4 (2020), 1234–1240.
  * Li et al. (2018) Jian Li, Zhaopeng Tu, Baosong Yang, Michael R. Lyu, and Tong Zhang. 2018.  Multi-Head Attention with Disagreement Regularization. In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP_. 2897–2903.
  * Li et al. (2021a) Yunfan Li, Peng Hu, Jerry Zitao Liu, Dezhong Peng, Joey Tianyi Zhou, and Xi Peng. 2021a.  Contrastive Clustering. In _Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI_. 8547–8555.
  * Li et al. (2020) Zhongli Li, Qingyu Zhou, Chao Li, Ke Xu, and Yunbo Cao. 2020.  Improving BERT with syntax-aware local attention.  _arXiv preprint arXiv:2012.15150_ (2020).
  * Li et al. (2021b) Zhongli Li, Qingyu Zhou, Chao Li, Ke Xu, and Yunbo Cao. 2021b.  Improving BERT with Syntax-aware Local Attention. In _Findings of the Association for Computational Linguistics, ACL/IJCNLP_. 645–653.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  RoBERTa: A Robustly Optimized BERT Pretraining Approach.  _CoRR_ abs/1907.11692 (2019).
  * Malisiewicz et al. (2011) Tomasz Malisiewicz, Abhinav Gupta, and Alexei A. Efros. 2011\.  Ensemble of exemplar-SVMs for object detection and beyond. In _IEEE International Conference on Computer Vision, ICCV_. IEEE Computer Society, 89–96.
  * Michel et al. (2019) Paul Michel, Omer Levy, and Graham Neubig. 2019.  Are Sixteen Heads Really Better than One?. In _Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems_. 14014–14024.
  * Moons et al. (2020) Elias Moons, Aditya Khanna, Abbas Akkasi, and Marie-Francine Moens. 2020.  A comparison of deep learning methods for ICD coding of clinical records.  _Applied Sciences_ 10, 15 (2020), 5262.
  * Ott et al. (2018) Myle Ott, Sergey Edunov, David Grangier, and Michael Auli. 2018\.  Scaling Neural Machine Translation. In _Proceedings of the Third Conference on Machine Translation: Research Papers, WMT_. 1–9.
  * Park et al. (2020) Geondo Park, Chihye Han, Daeshik Kim, and Wonjun Yoon. 2020\.  MHSAN: Multi-Head Self-Attention Network for Visual Semantic Embedding. In _IEEE Winter Conference on Applications of Computer Vision, WACV_. IEEE, 1507–1515.
  * Peng et al. (2019) Yifan Peng, Shankai Yan, and Zhiyong Lu. 2019.  Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets. In _Proceedings of the 18th BioNLP Workshop and Shared Task, BioNLP@ACL_. 58–65.
  * Qi et al. (2020) Le Qi, Yu Zhang, Qingyu Yin, and Ting Liu. 2020\.  Multiple Structural Priors Guided Self Attention Network for Language Understanding.  _CoRR_ abs/2012.14642 (2020).
  * Qiao et al. (2019) Yifan Qiao, Chenyan Xiong, Zhenghao Liu, and Zhiyuan Liu. 2019\.  Understanding the Behaviors of BERT in Ranking.  _CoRR_ abs/1904.07531 (2019).
  * Raganato et al. (2020) Alessandro Raganato, Yves Scherrer, and Jörg Tiedemann. 2020\.  Fixed Encoder Self-Attention Patterns in Transformer-Based Machine Translation. In _Findings of the Association for Computational Linguistics, EMNLP_. 556–568.
  * Raganato et al. (2018) Alessandro Raganato, Jörg Tiedemann, et al. 2018\.  An analysis of encoder representations in transformer-based machine translation. In _Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, BlackboxNLP@EMNLP_.
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019.  Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP_. 3980–3990.
  * Romanov and Shivade (2018) Alexey Romanov and Chaitanya Shivade. 2018.  Lessons from natural language inference in the clinical domain. In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP_. 1586–1596.
  * Sun et al. (2019) Chi Sun, Xipeng Qiu, Yige Xu, and Xuanjing Huang. 2019\.  How to Fine-Tune BERT for Text Classification?. In _Chinese Computational Linguistics - 18th China National Conference, CCL_ , Vol. 11856. 194–206.
  * Tao et al. (2021) Yaling Tao, Kentaro Takagi, and Kouta Nakata. 2021\.  Clustering-friendly Representation Learning via Instance Discrimination and Feature Decorrelation. In _9th International Conference on Learning Representations, ICLR 2021_.
  * Tavan et al. (2021) Ehsan Tavan, Ali Rahmati, Maryam Najafi, Saeed Bibak, and Zahed Rahmati. 2021.  BERT-DRE: BERT with Deep Recursive Encoder for Natural Language Sentence Matching.  _CoRR_ abs/2111.02188 (2021).
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017\.  Attention is All you Need. In _Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems_. 5998–6008.
  * Vig and Belinkov (2019) Jesse Vig and Yonatan Belinkov. 2019.  Analyzing the structure of attention in a transformer language model. In _Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, BlackboxNLP@ACL_. 63–76.
  * Voita et al. (2019) Elena Voita, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov. 2019.  Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned. In _Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL_. 5797–5808.
  * Wang et al. (2021a) Wenhui Wang, Hangbo Bao, Shaohan Huang, Li Dong, and Furu Wei. 2021a.  MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers. In _Findings of the Association for Computational Linguistics, ACL/IJCNLP_ _(Findings of ACL, Vol. ACL/IJCNLP 2021)_. 2140–2151.
  * Wang et al. (2021b) Zijie J. Wang, Robert Turko, and Duen Horng Chau. 2021b.  Dodrio: Exploring Transformer Models with Interactive Visualization.  _CoRR_ abs/2103.14625 (2021).
  * Williams et al. (2018) Adina Williams, Nikita Nangia, and Samuel R. Bowman. 2018\.  A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT_. 1112–1122.
  * Wu et al. (2019) Felix Wu, Angela Fan, Alexei Baevski, Yann N. Dauphin, and Michael Auli. 2019.  Pay Less Attention with Lightweight and Dynamic Convolutions. In _7th International Conference on Learning Representations, ICLR_.
  * Wu et al. (2018) Zhirong Wu, Yuanjun Xiong, Stella X. Yu, and Dahua Lin. 2018\.  Unsupervised Feature Learning via Non-Parametric Instance Discrimination. In _2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018_. 3733–3742.
  * Xia et al. (2021) Tingyu Xia, Yue Wang, Yuan Tian, and Yi Chang. 2021\.  Using Prior Knowledge to Guide BERT’s Attention in Semantic Textual Matching Tasks. In _WWW ’21: The Web Conference 2021_. 2466–2475.
  * Xu et al. (2019) Mingzhou Xu, Derek F. Wong, Baosong Yang, Yue Zhang, and Lidia S. Chao. 2019.  Leveraging Local and Global Patterns for Self-Attention Networks. In _Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL_. 3069–3075.
  * Yang et al. (2019) Baosong Yang, Longyue Wang, Derek F. Wong, Lidia S. Chao, and Zhaopeng Tu. 2019.  Convolutional Self-Attention Networks. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,NAACL-HLT_. 4040–4045.
  * Yang et al. (2020) Shan Yang, Heng Lu, Shiyin Kang, Liumeng Xue, Jinba Xiao, Dan Su, Lei Xie, and Dong Yu. 2020.  On the localness modeling for the self-attention based end-to-end speech synthesis.  _Neural Networks_ 125 (2020), 121–130.
  * Yates et al. (2021) Andrew Yates, Rodrigo Nogueira, and Jimmy Lin. 2021\.  Pretrained Transformers for Text Ranking: BERT and Beyond. In _SIGIR ’21: The 44th International ACM Conference on Research and Development in Information Retrieval, SIGIR_. 2666–2668.
  * Yuan et al. (2020) Shi-Fang Yuan, Yi-Bin Yu, Ming-Zhao Li, and Hua Jiang. 2020\.  A direct method to Frobenius norm-based matrix regression.  _Int. J. Comput. Math._ 97, 9 (2020), 1767–1780.
  * Zbontar et al. (2021) Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. 2021.  Barlow Twins: Self-Supervised Learning via Redundancy Reduction. In _Proceedings of the 38th International Conference on Machine Learning, ICML_ , Vol. 139. 12310–12320.
  * Zuo et al. (2020) Chaoyuan Zuo, Narayan Acharya, and Ritwik Banerjee. 2020\.  Querying across genres for medical claims in news. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP_. 1783–1789.

  *[PLMs]: pre-trained language model
  *[MDG]: attention map discrimination guiding
  *[PDG]: attention pattern decorrelation guiding
  *[PLM]: pre-trained language model
  *[AG]: attention guiding
  *[PCA]: Principal Component Analysis


# Enhanced Strong Coupling between Spin Ensemble and non-Hermitian Topological
Edge States

Jie Qian  Interdisciplinary Center of Quantum Information, Zhejiang Province
Key Laboratory of Quantum Technology and Device, Department of Physics,
Zhejiang University, Hangzhou 310027, China  Jie Li  Interdisciplinary Center
of Quantum Information, Zhejiang Province Key Laboratory of Quantum Technology
and Device, Department of Physics, Zhejiang University, Hangzhou 310027, China
Shi-Yao Zhu  Interdisciplinary Center of Quantum Information, Zhejiang
Province Key Laboratory of Quantum Technology and Device, Department of
Physics, Zhejiang University, Hangzhou 310027, China  Hefei National
Laboratory, Hefei 230088, China  J. Q. You<EMAIL_ADDRESS>Interdisciplinary
Center of Quantum Information, Zhejiang Province Key Laboratory of Quantum
Technology and Device, Department of Physics, Zhejiang University, Hangzhou
310027, China  Yi-Pu Wang<EMAIL_ADDRESS>Interdisciplinary Center of
Quantum Information, Zhejiang Province Key Laboratory of Quantum Technology
and Device, Department of Physics, Zhejiang University, Hangzhou 310027, China

###### Abstract

Light-matter interaction is crucial to both understanding fundamental
phenomena and developing versatile applications. Strong coupling, robustness,
and controllability are the three most important aspects in realizing light-
matter interactions. Topological and non-Hermitian photonics, have provided
frameworks for robustness and extensive control freedom, respectively. How to
engineer the properties of the edge state such as photonic density of state,
scattering parameters by using non-Hermitian engineering while ensuring
topological protection has not been fully studied. Here we construct a parity-
time-symmetric dimerized photonic lattice and generate complex-valued edge
states via spontaneous PT-symmetry breaking. The enhanced strong coupling
between the topological photonic edge mode and magnon mode in a ferromagnetic
spin ensemble is demonstrated. Our research reveals the subtle non-Hermitian
topological edge states and provides strategies for realizing and engineering
topological light-matter interactions.

Introduction.—Topology has evolved as a powerful governing principle for
predicting and harnessing the robust propagation of currents in various
systems, including condensed matter system Burkov-16 ; Hasan-10 , acoustics
Zhaoju-15 ; Ma-19 ; Yihao-22 , mechanics Huber-16  and photonics Haldane-08 ;
Wang-09 ; Lu-14 ; Ozawa-19 . In topological photonics, a topological invariant
ensures robust localization or propagation of electromagnetic waves Blanco-
Redondo-18 ; Yang-18 ; Klembt-18 . On the other hand, non-Hermitian photonics
Feng-17 ; EI-Ganainy-18 ; Longhi-18  has also flourished in recent years, not
only due to the ubiquitous non-Hermiticity in nature Bender-07 , but also
because the non-Hermiticity provides additional degrees of freedom to
manipulate the wave behaviors. In pursuit of the simultaneous robustness and
greater control flexibility, as well as the interest in fundamental research,
non-Hermitian topological physics Ashida-20 ; Coulais-21 ; Bergholtz-21  has
received considerable attention and substantial development. Scientists
investigate new paradigms Yao-18 ; Yokomizo-19 ; CHL-20 ; Helbig-20 ; Xue-20
and explore potential applications in this interdisciplinary territory Zhao-19
; St-Jean-17 ; Parto-18 ; Hu-21 .

Figure 1: (a)(b) Schematic diagram of the Hermitian and non-Hermitian SSH
chains. (c) Eigenmodes of the Hermitian SSH chain are plotted in the complex
energy plane. The zero-energy modes exist in the band gap. (d)(f) Transmission
spectra of the Hermitian and non-Hermitian SSH chains. (e) Eigenmodes of the
non-Hermitian SSH chain are plotted in the complex energy plane. The
alternated on-site losses result in spontaneous PT-symmetry breaking of the
edge modes.  Figure 2: Photograph of the Hermitian SSH chain. The chain
contains six unit cells and twelve SRRs. The SRRs are labled by a site index
$s$. The YIG sphere is placed on the top of the device. (b)(e) The mappings of
the transmission spectra are plotted versus the electromagnet current and
probe frequency when a YIG sphere is placed at site-1 and site-12,
respectively. Strong coupling between the edge (bulk) mode and the magnon mode
is indicated by the large (small) level repulsion. (c) The squares of the
coupling strengths $g_{m,s}^{2}$ ($m$=8) (blue dots) are extracted when the
YIG sphere is positioned at the $s$-th site, where $m$ is the eigenmode index.
The gray bars represent the intensity distributions of the bulk state wave
function $|\varphi_{m,s}|^{2}$ ($m$=8). (d) The squares of the coupling
strengths $g_{m,s}^{2}$ ($m$=6,7) (red dots) are plotted versus site index
$s$. The intensity distributions of the edge state wave functions
$|\varphi_{m,s}|^{2}$ (m=6,7) are depicted by the gray bar.

A coupled system can have two forms of non-Hermiticity. One kind is generated
when there is asymmetric interaction between the sites, which leads to the
non-Hermitian skin effect Yao-18 ; Alvarez-18 . The other type, which is
caused by on-site loss, can lead to intriguing phenomena associated with the
parity-time (PT) symmetry. The PT-symmetric systems have received special
attention, because they were proved to have real spectra Bender-98 . A
sequence of studies have studied the topologically protected bound (defect)
states in PT-symmetric topological systems Schomerus-13 ; Malzard-15 ;
Weimann-17 ; Stegmaier-21 , where the defect states are real in the PT-
symmetry unbroken phase. Moreover, a number of studies have investigated
whether topological edge states exist in the PT-symmetric systems Esaki-11 ;
Hu-11 ; Xue-17 ; Cheng-22 , concluding that since the edge state is not an
eigenstate of the PT operator, an imaginary eigenvalue is obtained along with
the spontaneous PT-symmetry breaking. In this case, a non-Hermitian edge state
is obtained. We find that these imaginary edge states in the PT-symmetric
system are actually topologically protected by the particle-hole symmetry SM .
In the one-dimensional (1D) non-Hermitian PT-symmetric Su-Schrieffer-Heeger
(SSH) model Su-79 , the chiral symmetry of the system is broken, losing its
topological $\mathbb{Z}$ invariant, but the particle-hole symmetry of the
system is preserved and the system owns a topological $\mathbb{Z}_{2}$
invariant. In the presence of perturbations that do not violate the particle-
hole symmetry, the real parts of the eigenvalues of the edge modes remain 0,
reflecting the topologically protected characteristics. Under this situation,
the topological photonic mode with robust properties can be further
manipulated by non-Hermiticity, which is highly desirable for investigating
light-matter interactions Gutzler-21 ; Ruggenthaler-18 ; Kockum-19 .

To investigate the interaction between topological photonic modes and matters
Kim-21 , we employ the photon-magnon coupling system Huebl-PRL-2013 ; Tabuchi-
PRL-2013 ; Zhang-PRL-2014 ; Tobar-PRApp-2014 ; You-npj-2015 ; Wang-2019 ;
Wang-2020 ; Rameshti-22 ; Yuan-22 , which has benefits including the flexible
tunability and experimental demonstration at room temperature. In this Letter,
we use a set of lossy microwave resonators to build 1D non-Hermitian SSH
photonic lattices. By coupling a ferromagnetic spin ensemble (FSE) to
Hermitian and non-Hermitian SSH chains and monitoring the strength of the
coupling between the photonic modes and the magnon mode in the FSE, we verify
the topological edge states and bulk states. Non-Hermiticity introduced by the
on-site alternating losses breaks the passive PT-symmetry of zero-energy modes
and results in two complex-valued edge states, which localize exponentially at
the opposite ends of the chain [Fig. 1(b)]. Further, the photonic density of
state (PDOS) at boundaries is larger than that in the Hermitian case [Fig.
1(a)], which strengthens the coupling between the topological photonic mode
and the magnon mode. Our experiment demonstrates the potential of manipulating
the interaction between topological photonic states and matter by exploiting
non-Hermiticity.

System and model.—The SSH chain consists of six unit cells [Figs. 1(a) and
1(b)], in which each unit contains two split-ring-resonators (SRRs) fabricated
on the F4B substrate [Fig. 2(a)]. In the experiment, the SRR exhibits a
resonance at $\omega_{0}/2\pi$=5.62 GHz with an intrinsic loss of
$\gamma_{0}/2\pi$=24.42 MHz, and the topological property is unaltered by the
uniform losses along the chain Feng-17 . Therefore, SRRs with the same loss
can be used to build the Hermitian SSH model. Two neighboring SRRs are
separated by staggered spacings to realize the intracell and intercell
coupling rates, $v$ and $w$. Edge states appear in the finite chain when the
bulk winding number of the Hermitian Hamiltonian is $\mathcal{W}_{\rm{h}}$=1
Weimann-17 . The effective Hermitian SSH chain is designed in the topological
non-trivial phase ($v/2\pi$=216.5 MHz, $w/2\pi$=341 MHz) and the Hamiltonian
is written as SM :

$\mathcal{H}_{\rm{h}}/{\hbar}=\sum_{s=1}^{2N}(\omega_{0}-i\gamma_{0})\hat{a}_{s}^{\dagger}\hat{a}_{s}+\sum_{s=1}^{2N-2}(v\hat{a}_{s}\hat{a}_{s+1}^{\dagger}+w\hat{a}_{s+1}\hat{a}_{s+2}^{\dagger}),$
(1)

where ${\hat{a}}_{s}^{\dagger}~{}({\hat{a}}_{s})$ is the photon creation
(annihilation) operator of the $s$-th SRR. The uniform losses of the units
only yield all eigenvalues of the chain to have the same imaginary component
$i\gamma_{0}$. The eigenvalues of the coupled SRRs are plotted in the complex
plane, as shown in Fig. 1(c). A pair of zero-energy modes
(Re(${\widetilde{\omega}_{m=6,7}})-\omega_{0}$=0, green dots) appear in the
band gap (gray area), which are the edge modes. The measured transmission
spectrum of the chain is shown in Fig. 1(d), where the peaks correspond to the
resonances of the eigenmodes. By simulating the field distribution at the edge
mode frequency of $\omega_{0}/2\pi$=5.62 GHz, we find that the electromagnetic
field tends to localize at both edges of the chain, as predicted by wave
function distribution SM . In the low-frequency region, the measured spectrum
[Fig. 1(d), solid line] displays an amplitude deviation from that in the high-
frequency region. This is due to the residual dissipative coupling between
SRRs Weimann-17 ; SM .

Figure 3: Photograph of the non-Hermitian SSH chain. The integrated resistors
induce on-site losses. (b)(e) When the YIG sphere is placed at site-1 and
site-12, respectively, the mappings of the transmission spectra are plotted
versus the electromagnet current and probe frequency. The resonances with
frequencies of 5.48 GHz and 5.68 GHz correspond to the non-Hermitian edge mode
($m$=6,7) and bulk mode ($m$=8), respectively. (c) The squares of the coupling
strengths $g_{m,s}^{2}$ ($m$=8) (blue dots) are plotted versus site index $s$.
Gray bars represent the intensity distributions of the bulk state wave
function $|\varphi_{m,s}|^{2}$ ($m$=8). (d) The squares of the coupling
strengths $g_{6,s}^{2}$ (blue dots) and $g_{7,s}^{2}$ (red dots) are plotted
versus site index $s$. The intensity distributions of edge state wave
functions $|\varphi_{m,s}|^{2}$ are denoted by gray ($m$=6, low loss edge
state) and red ($m$=7, high loss edge state) bar diagrams, respectively.

Then, on-site non-Hermiticity is added to the SSH chain. As depicted in Fig.
3(a), resistors $R_{\rm{A}}=0.1~{}\Omega$ and $R_{\rm{B}}=2.7~{}\Omega$ are
integrated into odd and even sites of the chain, respectively, which induce
alternated losses of $\gamma_{\rm{A}}/2\pi$=36 MHz and
$\gamma_{\rm{B}}/2\pi$=73 MHz. The Hamiltonian becomes SM :

$\begin{split}\mathcal{H}_{\rm{nh}}/{\hbar}=&\sum_{s\in
X}(\omega_{0}-i\gamma_{\rm{A}})\hat{a}_{s}^{\dagger}\hat{a}_{s}+\sum_{s\in
Y}(\omega_{0}-i\gamma_{\rm{B}})\hat{a}_{s}^{\dagger}\hat{a}_{s}\\\
&+\sum_{s=1}^{2N-2}(v\hat{a}_{s}\hat{a}_{s+1}^{\dagger}+w\hat{a}_{s+1}\hat{a}_{s+2}^{\dagger}),\end{split}$
(2)

where $X$={1, 3, 5, …, 2$N$-1}, $Y$={2, 4, 6, …, 2$N$}, and $N$=6. The
integrated resistors shift $\omega_{0}/2\pi$ to 5.48 GHz, and the hopping
rates shift to $v/2\pi$=208.5 MHz, and $w/2\pi$=335.5 MHz. The alternated
losses make the system a passive PT-symmetric one. The spontaneous PT-symmetry
breaking occurs in zero-energy modes, resulting in a splitting of the
imaginary parts of zero-energy modes, as shown in Fig. 1(e). One with a low
loss Im(${\widetilde{\omega}}_{m=6})/2\pi$=40.42 MHz ($\rm{Edge}_{1}$, blue
dot) localizes at the left boundary of the chain, and the other with a high
loss Im(${\widetilde{\omega}}_{m=7})/2\pi$=68.58 MHz ($\rm{Edge}_{2}$, red
dot) localizes at the right, as schematically shown in Fig. 1(b). The bulk
Hamiltonian still preserves the PT-symmetry when
$\delta\gamma/2<\left|w-v\right|$, and
$\delta\gamma=\gamma_{\rm{B}}-\gamma_{\rm{A}}$. In this regime, the
topological property is still determined by the generalized integer winding
number $\mathcal{W}_{\rm{nh}}$ SM . $\mathcal{W}_{\rm{nh}}=1$ guarantees the
existence of two non-Hermitian topological edge modes.

Experiment results.—To investigate the edge modes engineered by the non-
Hermiticity, we measure the PDOS and linewidths of the edge and bulk modes in
both Hermitian and non-Hermitian cases. Notably, conventional detection of the
PDOS relies on the near-field radiation Bellec-13 , but in the non-Hermitian
situation, the local gain and loss will diminish its reliability. Using the
spin ensemble as a probe, we can directly detect the PDOS. In addition, it
allows us to study the strong coherent interaction between the topological
photonic modes and magnons.

In the experiment, the spin ensemble employed to couple with the chain is a
1-mm diameter yttrium iron garnet (YIG) sphere. The magnon mode in the sphere
interacts with the local photonic modes, with a coupling strength $g$
proportional to $\eta\chi\sqrt{nS\hbar\omega_{r}/2V}$ Tabuchi-PRL-2013 ;
Zhang-PRL-2014 , where $\eta\leq 1$ describes the spatial overlap and
polarization matching between the photonic mode and the magnon mode, $\chi$ is
the gyromagnetic ratio, $n$ is the total number of spins, $S$=5/2 is the spin
number of the ground state ${\rm Fe}^{3+}$ ion in YIG, $\omega_{r}$ is the
resonance frequency, and $V$ is the photonic mode volume. Consequently, the
square of the coupling strength $g^{2}$ directly reflects the PDOS at the
coupling location. Firstly, we move the YIG sphere to each site (labeled as
$s$, $s$=1,2,3,…,12) of the Hermitian chain, and obtain the PDOS distribution
of the $m$-th eigenmode by analyzing the transmission spectra. The bias
magnetic field is perpendicular to the device plane, and mappings of
transmission spectra are measured versus electromagnet current and probe
frequency. Figures 2(b) and 2(e), for instance, show the mappings when the YIG
sphere is placed at site-1 and site-12, respectively. The coupling strength
between $m$-th eigenmode of the chain and the magnon mode at the $s$-th site
is defined as $g_{m,s}$, which can be obtained by fitting the level repulsion
with:

${\widetilde{\omega}}_{m,s}^{\pm}=\frac{1}{2}\left[{\widetilde{\omega}}_{n}+{\widetilde{\omega}}_{m}\pm\sqrt{{{(\widetilde{\omega}}_{n}-\widetilde{\omega}}_{m})+4g_{m,s}^{2}}\right],$
(3)

where ${\widetilde{\omega}}_{n}=\omega_{n}-i\gamma_{n}$ and
${\widetilde{\omega}}_{m}=\omega_{m}-i(\gamma_{m}+\kappa_{m})$ are the
eigenvalues of the uncoupled magnon mode and the $m$-th eigenmode of the
chain, respectively. $\gamma_{n}$ is the total loss rate of the magnon mode,
$\gamma_{m}$ is the intrinsic loss rate of the $m$-th eigenmode, and
$\kappa_{m}$ is the extrinsic loss rate of the $m$-th eigenmode to the
input/output ports SM . Coupling strengths between the magnon mode and edge
modes ($m$=6,7) at site-1 and site-12 are obtained by fitting the level
repulsion depicted in Figs. 2(b) and 2(e), which are
$g_{\rm{edge},1}/2\pi=g_{\rm{edge},12}/2\pi=80$ MHz. Similarly, coupling
strengths between the magnon mode and bulk mode ($m$=8) at site-1 and site-12
are obtained as $g_{\rm{bulk},1}/2\pi=g_{\rm{bulk},12}/2\pi=37$ MHz.
$g_{m,s}^{2}$ as a function of the site index $s$ are illustrated in Figs.
2(c) and 2(d), denoted by blue ($m$=8) and red dots ($m$=6,7), respectively.
The observed $g_{m,s}^{2}$ are in good agreement with the intensity
distributions for the wave function $|\varphi_{m,s}|^{2}$ (gray bar diagram).

Figure 4: (a)(b) The measured absorptivity spectra for both Hermitian and non-
Hermitian SSH chains, where $\delta\gamma/2(w+v)=0$ and 0.034, respectively.
A1 (A2) is measured when loading signal to port 1 (2), as shown by the blue
(red) line. (c)(d) The real and imaginary parts of normalized eigenvalues,
which are plotted versus the non-Hermiticity $\delta\gamma/2(w+v)$. The left
inset figure between (a) and (b) shows the PT-symmetry spontaneous breaking
point (EP1) of the edge states.

Then, we couple the spin ensemble to the non-Hermitian SSH chain, as shown in
Fig. 3(a). Figures 3(b) and 3(e) display the mappingswhen the YIG sphere is
placed at site-1 and site-12, respectively. The mappings show similar amount
of level repulsion, but reflects very different linewidths of the edge modes.
Using Eq. (3), the loss of the edge mode at site-1 is fitted to be
$\gamma_{\rm{edge},1}/2\pi=41.1$ MHz, which is contributed by the addition of
the two edge modes ($m$=6,7). The relation is
$\gamma_{{\rm{edge}},s}=[{\rm{Im}}({\widetilde{\omega}}_{m=6})\cdot|\varphi_{6,s}|^{2}+{\rm{Im}}({\widetilde{\omega}}_{m=7})\cdot|\varphi_{7,s}|^{2}]/(|\varphi_{6,s}|^{2}+|\varphi_{7,s}|^{2})$,
and the wave functions of the edge modes $|\varphi_{m,s}|^{2}$ are displayed
as the bar diagram in Fig. 3(d). Similarly, we get
$\gamma_{edge,12}/2\pi$=67.9 MHz. More interestingly, the coupling strengths
between the magnon mode and edge modes at site-1 and site-12 are observed to
be $g_{\rm{edge},1}/2\pi=g_{\rm{edge},12}/2\pi$=112 MHz, which is larger than
that in the Hermitian case (80 MHz). We plot $g_{m,s}^{2}$ versus site index
$s$ for $m$=8 and $m$=6, 7 in Figs. 3(c) and 3(d), respectively. It can be
found that the bulk mode maintains expanded, similar to the Hermitian bulk
mode. But, as shown in Fig. 3(d), the low-loss edge state (Edge1) accumulates
at the left boundary, while high-loss edge state (Edge2) accumulates at the
right edge. The introduction of on-site loss does contribute to the increase
of PDOS at the boundaries. The mechanism can be interpreted as follows: When
the PT-symmetry of the edge states is broken, the energy flow between adjacent
resonators is partly blocked Peng-14 . The low-loss (high-loss) edge state
becomes more localized at the low-loss (high-loss) site, as shown in Figs.
1(b) and 3(a), it corresponds the left (right) boundary of the chain.

It is also intriguing to detect the properties of the non-Hermitian
topological edge states from spectroscopic measurements. In the PT-symmetry
unbroken phase, two topological edge states cannot be distinguished via
spectroscopic measurement, as shown in Fig. 4(a). The absorptivity spectra
$A_{1}$ measured when loading microwave to port 1 is totally coincident with
$A_{2}$ measured when loading microwave to port 2. In the symmetry broken
phase, two topological edge states can be distinguished in spectra, as shown
in Fig. 4(b). The spectra $A_{1}$ exhibits the low-loss state with a
relatively narrow bandwidth, while the spectra $A_{2}$ reveals the high-loss
state.

Finally, we anticipate to discuss about some additional characteristics of the
exceptional point (EP) in the non-Hermitian chain. The dimensionless
eigenvalues are defined as $\beta_{\rm{real}}+i\beta_{\rm{imag}}$, where
$\beta_{\rm{real}}=\left[\rm{Re}(\widetilde{\omega})-\omega_{0}\right]/(v+w)$,
$\beta_{\rm{imag}}=\left[|\rm{Im}(\widetilde{\omega})|-\bar{\gamma}\right]/(v+w)$,
and $\bar{\gamma}=(\gamma_{\rm{A}}+\gamma_{\rm{B}})/2$. In a finite SSH chain,
when increasing the non-Hermitian parameter $\delta\gamma/2(v+w)$, a series of
exceptional points are gradually reached [Figs. 4(c) and 4(d)]. It can be
found that the EP of the edge modes is distinctly away from the EPs of the
bulk modes. The edge modes experience spontaneous PT-symmetry breaking (SPTB)
at EP1, where $\delta\gamma/2(v+w)$ is only about 0.02. With the increase of
chain length, the non-Hermiticity needed for SPTB in edge modes decreases
exponentially. In the case of $N\gg 1$, any finite $\delta\gamma$ will lead to
the SPTB in edge modes SM . However, the minimum requirement of SPTB in bulk
mode needs $\delta\gamma/2\textgreater\left|w-v\right|$, which is much larger
than 0.02. Additional analysis is provided in the supplementary materials.

Conclusion.—We have implemented the PT-symmetric non-Hermitian topological SSH
model with microwave resonators and achieved the control of topological edge
states using the on-site non-Hermiticity. Through spontaneous PT-symmetry
breaking, we obtain the non-Hermitian edge modes, where the photonic mode
densities are enhanced at both ends of the chain. We realize the strong
coupling between the edge modes and the magnon mode in both Hermitian and non-
Hermitian cases. We experimentally verify that the coupling strength between
the non-Hermitian edge states and the spin ensemble is stronger than that in
the Hermitian situation. Our research illustrates non-Hermiticity engineered
topological edge states and paves a way for studying strong coherent
interaction between topological photonic modes and matter.

###### Acknowledgements.

This work is supported by the National Key Research and Development Program of
China (No. 2022YFA1405200), National Natural Science Foundation of China (No.
$92265202$, No. $11934010$, No. ${\rm U}1801661$, and No. $12174329$), and the
Fundamental Research Funds for the Central Universities (No. $2021{\rm
FZZX}001$-$02$).

## References

  * (1) A. A. Burkov, Topological semimetals, Nature Materials 15, 1145 (2016).
  * (2) M. Z. Hasan and C. L. Kane, Colloquium: Topological insulators, Rev. Mod. Phys. 82, 3045 (2010).
  * (3) Z. Yang, F. Gao, X. Shi, X. Lin, Z. Gao, Y. Chong, and B. Zhang, Topological Acoustics, Phys. Rev. Lett. 114, 114301 (2015).
  * (4) G. Ma, M. Xiao and C. T. Chan, Topological phases in acoustic and mechanical systems, Nat. Rev. Phys. 1, 281 (2019).
  * (5) H. Xue, Y. Yang, B. Zhang, Topological acoustics, Nature Reviews Materials 7, 974 (2022).
  * (6) S. D. Huber, Topological mechanics, Nat. Phys. 12, 621 (2016).
  * (7) F. D. M. Haldane and S. Raghu, Possible realization of directional optical waveguides in photonic crystals with broken time-reversal symmetry, Phys. Rev. Lett. 100, 013904 (2008).
  * (8) Z. Wang, Y. Chong, J. D. Joannopoulos, and M. Soljačić, Observation of unidirectional backscattering-immune topological electromagnetic states, Nature 461, 772 (2009).
  * (9) L. Lu, J. D. Joannopoulos, and M. Soljačić, Topological photonics, Nat. Photon. 8, 821 (2014).
  * (10) T. Ozawa et al., Topological photonics, Rev. Mod. Phys. 91, 015006 (2019).
  * (11) A. Blanco-Redondo, B. Bell, D. Oren, B. J. Eggleton and M. Segev, Topological protection of biphoton states, Science 362, 568 (2018).
  * (12) B. Yang et al., Ideal Weyl points and helicoid surface states in artificial photonic crystal structures, Science 359, 1013 (2018).
  * (13) S. Klembt et al., Exciton-polariton topological insulator, Nature, 562, 552 (2018).
  * (14) L. Feng, R. EI-Ganainy, and L. Ge, Non-Hermitian photonics based on parity–time symmetry, Nat. Photon. 11, 752 (2017).
  * (15) R. EI-Ganainy et al., Non-Hermitian physics and PT symmetry, Nat. Phys. 14, 11 (2018).
  * (16) Stefano Longhi, Parity-time symmetry meets photonics: A new twist in non-hermitian optics, Europhysics Letters 120, 64001 (2018).
  * (17) C. M. Bender, Making sense of non-hermitian hamiltonians, Reports on Progress in Physics 70, 947 (2007).
  * (18) Y. Ashida, Z. P. Gong, and M. Ueda, Non-Hermitian physics, Adv. Phys. 69, 249 (2020).
  * (19) C. Coulais, R. Fleury, and J. Van Wezel, Topology and broken Hermiticity, Nat. Phys. 17, 9 (2021).
  * (20) E. J. Bergholtz, J. C. Budich, and F. K. Kunst, Exceptional topology of non-Hermitian systems, Rev. Mod. Phys. 93, 015005 (2021).
  * (21) S. Yao and Z. Wang, Edge States and Topological Invariants of Non-Hermitian Systems, Phys. Rev. Lett. 121, 086803 (2018).
  * (22) K. Yokomizo and S. Murakami, Non-Bloch band theory of non-Hermitian systems, Phys. Rev. Lett. 123, 066404 (2019).
  * (23) C. H. Lee, L. Li, R. Thomale, and J. Gong, Unraveling non-Hermitian pumping: Emergent spectral singularities and anomalous responses, Phys. Rev. B 102, 085151 (2020).
  * (24) T. Helbig et al., Generalized bulk–boundary correspondence in non-Hermitian topolectrical circuits. Nat. Phys. 16, 747 (2020).
  * (25) L. Xiao, T. Deng, K. Wang, G. Zhu, Z. Wang, W. Yi, and P. Xue, Non-Hermitian bulk–boundary correspondence in quantum dynamics, Nat. Phys. 16, 761 (2020).
  * (26) H. Zhao et al., Non-Hermitian topological light steering, Science 365, 1163 (2019).
  * (27) P. St-Jean et al., Lasing in topological edge states of a one-dimensional lattice, Nat. Photon. 11, 651 (2017).
  * (28) M. Parto et al., Edge-Mode Lasing in 1D Topological Active Arrays, Phys. Rev. Lett. 120, 113901 (2018).
  * (29) B. Hu et al., Non-Hermitian topological whispering gallery, Nature 597, 655 (2021).
  * (30) V. M. Martinez Alvarez, J. E. Barrios Vargas, and L. E. F. Foa Torres, Non-Hermitian robust edge states in one dimension: Anomalous localization and eigenspace condensation at exceptional points, Phys. Rev. B 97, 121401(R) (2018).
  * (31) N. Okuma, K. Kawabata, K. Shiozaki, and M. Sato, Topological Origin of Non-Hermitian Skin Effects, Phys. Rev. Lett. 124, 086801 (2020).
  * (32) C. M. Bender and S. Boettcher, Real Spectra in Non-Hermitian Hamiltonians Having PT Symmetry, Phys. Rev. Lett. 80, 5243 (1998).
  * (33) H. Schomerus, Topologically protected midgap states in complex photonic lattices, Opt. Lett. 38, 1912 (2013)
  * (34) S. Malzard, C. Poli, and H. Schomerus, Topologically Protected Defect States in Open Photonic Systems with Non-Hermitian Charge-Conjugation and Parity-Time Symmetry, Phys. Rev. Lett. 115, 200402 (2015).
  * (35) S. Weimann et al., Topologically protected bound states in photonic parity-time-symmetric crystals, Nat. Mater. 16, 433-438 (2017).
  * (36) A. Stegmaier et al., Topological Defect Engineering and PT Symmetry in Non-Hermitian Electrical Circuits, Phys. Rev. Lett. 126, 215302 (2021).
  * (37) K. Esaki, M. Sato, K. Hasebe, and M. Kohmoto, Edge states and topological phases in non-Hermitian systems, Phys. Rev. B 84, 205128 (2011).
  * (38) Y. C. Hu and T. L. Hughes, Absence of topological insulator phases in non-Hermitian PT-symmetric Hamiltonians, Phys. Rev. B 84, 153101 (2011).
  * (39) L. Xiao, X. Zhan, Z. H. Bian, K. K. Wang, X. Zhang, X. P. Wang, J. Li, K. Mochizuki, D. Kim, N. Kawakami, W. Yi, H. Obuse, B. C. Sanders, P. Xue, Observation of topological edge states in parity–time-symmetric quantum walks, Nature Physics 13, 1117 (2017).
  * (40) D. Cheng et al., Truncation-dependent PT phase transition for the edge states of a two-dimensional non-Hermitian system, Phys. Rev. B 105, L201105 (2022).
  * (41) See Supplementary Materials at … for device details, Hamiltonian and topological invariant analysis, additional transmission mappings, and the experimental measurement details, which includes Refs. Esaki-11 ; Weimann-17 .
  * (42) W. P. Su, J. R. Schrieffer and A. J. Heeger, Solitons in Polyacetylene, Phys. Rev. Lett. 42, 1698 (1979).
  * (43) R. Gutzler, M. Garg, C. R. Ast, K. Kuhnke, and Kern, K. Light–matter interaction at atomic scales, Nat. Rev. Phys. 3, 441 (2021).
  * (44) M. Ruggenthaler, N. Tancogne-Dejean, J. Flick, H. Appel, and A. Rubio, From a quantum-electrodynamical light–matter description to novel spectroscopies, Nat. Rev. Chem. 2, 0118 (2018).
  * (45) A. F. Kockum, A. Miranowicz, S. De Liberato, S. Savasta, and F. Nori, Ultrastrong coupling between light and matter, Nat. Rev. Phys. 1, 19 (2019).
  * (46) E. Kim et al., Quantum Electrodynamics in a Topological Waveguide, Phys. Rev. X 11, 011015 (2021).
  * (47) H. Huebl, C. W. Zollitsch, J. Lotze, F. Hocke, M. Greifenstein, A. Marx, R. Gross, and S. T. B. Goennenwein, High Cooperativity in Coupled Microwave Resonator Ferrimagnetic Insulator Hybrids, Phys. Rev. Lett. 111, 127003 (2013).
  * (48) Y. Tabuchi, S. Ishino, T. Ishikawa, R. Yamazaki, K. Usami, and Y. Nakamura, Hybridizing Ferromagnetic Magnons and Microwave Photons in the Quantum Limit, Phys. Rev. Lett. 113, 083603 (2014).
  * (49) X. Zhang, C.-L. Zou, L. Jiang, and H. X. Tang, Strongly Coupled Magnons and Cavity Microwave Photons, Phys. Rev. Lett. 113, 156401 (2014).
  * (50) M. Goryachev, W. G. Farr, D. L. Creedon, Y. Fan, M. Kostylev, and M. E. Tobar, High-Cooperativity Cavity QED with Magnons at Microwave Frequencies, Phys. Rev. Applied 2, 054002 (2014).
  * (51) D. Zhang, X.-M. Wang, T.-F. Li, X.-Q. Luo, W. Wu, F. Nori, J. Q. You, Cavity quantum electrodynamics with ferromagnetic magnons in a small yttrium-iron-garnet sphere, npj Quantum Information 1, 15014 (2015).
  * (52) Y.-P. Wang, J. W. Rao, Y. Yang, P.-C. Xu, Y. S. Gui, B. M. Yao, J. Q. You, and C.-M. Hu, Nonreciprocity and Unidirectional Invisibility in Cavity Magnonics, Phys. Rev. Lett. 123, 127202 (2019).
  * (53) Y.-P. Wang and C.-M. Hu, Dissipative couplings in cavity magnonics, Journal of Applied Physics 127, 130901 (2020).
  * (54) B. Z. Rameshti, S. V. Kusminskiy, J. A. Haigh, K. Usami, D. Lachance-Quirion, Y. Nakamura, C. Hu, H. X. Tang, G. E. W. Bauer and Y. M. Blanter, Cavity Magnonics, Physics Reports 979, 1-60 (2022).
  * (55) H. Y. Yuan, Y. Cao, A. Kamra, P. Yan, and R. A. Duine, Quantum magnonics: when magnon spintronics meets quantum information science, Physics Reports 965, 1 (2022).
  * (56) M. Bellec, U. Kuhl, G. Montambaux, and F. Mortessagne, Tight-binding couplings in microwave artificial graphene, Phys. Rev. B 88, 115437 (2013).
  * (57) B. Peng, Ş. K. Özdemir, F. Lei, F. Monifi, M. Gianfreda, G. Long, S. Fan, F. Nori, C. M. Bender and L. Yang, Parity-time-symmetric whispering-gallery microcavities, Nat. Phys. 10, 394 (2014).