11institutetext:  Data and Web Science Group, University of Mannheim, Germany
11email<EMAIL_ADDRESS>
# NASTyLinker: NIL-Aware Scalable Transformer-based Entity Linker

Nicolas Heist  0000-0002-4354-9138 Heiko Paulheim  0000-0003-4386-8195

###### Abstract

Entity Linking (EL) is the task of detecting mentions of entities in text and
disambiguating them to a reference knowledge base. Most prevalent EL
approaches assume that the reference knowledge base is complete. In practice,
however, it is necessary to deal with the case of linking to an entity that is
not contained in the knowledge base (NIL entity). Recent works have shown
that, instead of focusing only on affinities between mentions and entities,
considering inter-mention affinities can be used to represent NIL entities by
producing clusters of mentions. At the same time, inter-mention affinities can
help to substantially improve linking performance for known entities. With
NASTyLinker, we introduce an EL approach that is aware of NIL entities and
produces corresponding mention clusters while maintaining high linking
performance for known entities. The approach clusters mentions and entities
based on dense representations from Transformers and resolves conflicts (if
more than one entity is assigned to a cluster) by computing transitive
mention-entity affinities. We show the effectiveness and scalability of
NASTyLinker on NILK, a dataset that is explicitly constructed to evaluate EL
with respect to NIL entities. Further, we apply the presented approach to an
actual EL task, namely to knowledge graph population by linking entities in
Wikipedia listings, and provide an analysis of the outcome.

###### Keywords:

NIL-Aware Entity Linking Entity Discovery Knowledge Graph Population NILK
Wikipedia Listings CaLiGraph.

##  1 Introduction

###  1.1 Motivation and Problem

Entity Linking (EL), i.e., the task of detecting mentions of entities in text
and disambiguating them to a reference knowledge base (KB), is crucial for
many downstream tasks like question answering [9, 40], or KB population and
completion [16, 19, 31]. One main challenge of EL is the inherent ambiguity of
mentioned entities in the text. Figure 1 shows four homonymous mentions of
distinct entities with the name James Lake (a lake in Canada, a lake in the
US, a musician, and a fictional character). Correctly linking the mentions in
1a and 1b is especially challenging as both point to lakes that are
geographically close.

In a typical EL setting, we assume that the training data contains mentions of
all entities to be linked against. This assumption is dropped in Zero-Shot EL
[26], where a linking decision is made on the basis of entity information in
the reference KB (e.g. textual descriptions, types, relations). In this
setting, a seminal approach has been introduced with BLINK [41]. Its core idea
is to create dense representations of mentions and entities with a Transformer
model [10] in a bi-encoder setting, retrieve mention-entity candidates through
Nearest Neighbor Search, and rerank candidates with a cross-encoder.

In a practical setting, we additionally encounter the problem of mentions
without a corresponding entity in the reference KB (which we refer to as NIL
mentions and NIL entities, respectively). In fact, the mention in Figure 1a is
the only one with a counterpart in the reference KB (i.e., Wikipedia). For the
other mentions, a correct prediction based on Wikipedia entities is
impossible. Instead, NIL-aware approaches could either (1) create an
(intermediate) entity representation for the NIL entity to link, or (2)
produce clusters of NIL mentions with all mentions in a cluster referring to
the same entity.

While this problem has been largely ignored by EL approaches for quite some
time, recent works demonstrate that reasonable predictions for NIL mentions
can be made by clustering mentions on the basis of inter-mention affinities
[1, 24]. Both compute inter-mention and mention-entity affinities using a bi-
encoder architecture on the basis of BLINK [41]. EDIN [24] is an approach of
category (1) that uses a dedicated adaptation dataset to create
representations for NIL entities in an unsupervised fashion. Hence, the
approach can only link to a NIL entity if there is at least one mention of it
in the adaptation dataset. For some EL tasks, especially as a prerequisite for
KB population, creating an adaptation dataset with good coverage is not
trivial because an optimal adaptation dataset has to contain mentions of all
NIL entities. Agarwal et al. [1] present an approach of category (2) that
creates clusters of mentions and entities in a bottom-up fashion by
iteratively merging the two most similar clusters, always under the constraint
that a cluster must contain at most one entity.

(a) A lake in Ontario, CA.
Lakes of Temagami

(b) A lake in Montana, US.
List of lakes of Powell County, Montana

(c) A musician in the band Vinyl Williams.
Vinyl Williams

(d) A character in a soap opera.
List of The Young and Restless characters

Figure 1: Listings in Wikipedia containing the mention James Lake. All of the
mentions refer to distinct entities. A dedicated Wikipedia page exists only
for the entity of the mention in (a).

###  1.2 Approach and Contributions

With NASTyLinker, we present an EL approach that is NIL-aware in the sense of
category (2) and hence avoiding the need for an adaptation dataset. Similar to
Agarwal et al. [1], it produces clusters of mentions and entities on the basis
of inter-mention and mention-entity affinities from a bi-encoder. NASTyLinker
relies on a top-down clustering approach that – in case of a conflict –
assigns mentions to the entity with the highest transitive affinity. Contrary
to Agarwal et al., who discard cross-encoders completely due to the quadratic
growth in complexity when evaluating inter-mention affinities, our experiments
show that applying a cross-encoder only for the refinement of mention-entity
affinities can result in a considerable increase of linking performance at a
reasonable computational cost. Our evaluation on the NILK dataset [21], a
dataset especially suited for the evaluation of NIL-aware approaches, shows
that NASTyLinker manages to make competitive predictions for NIL entities
while even slightly improving prediction performance for known entities. The
approach is designed in a modular way to make existing EL models NIL-aware by
post-processing the computed inter-mention and mention-entity affinities. By
applying NASTyLinker to a knowledge graph population task, we demonstrate its
ability to reliably link to known entities (up to 87% accuracy) and identify
NIL entities (up to 90% accuracy).

To summarize, the contributions of this paper are as follows:

  * •

We introduce the NASTyLinker approach, serving as an extension to existing EL
approaches by using a top-down clustering mechanism to consistently link
mentions to known entities and produce clusters for NIL mentions (Section 4).

  * •

In our experiments, we demonstrate the competitive linking performance and
scalability of the presented approach through an evaluation on the NILK
dataset (Section 5.4).

  * •

We use NASTyLinker for KB population by linking entities in Wikipedia
listings. We report on the linking statistics and provide a qualitative
analysis of the results (Section 5.5).

The produced code is part of the CaLiGraph extraction framework and publicly
available on GitHub.111https://github.com/nheist/CaLiGraph

##  2 Related Work

Entity Linking  Entity Linking has been studied extensively in the last two
decades [33, 37]. Initially, approaches relied on word and entity frequencies,
alias tables, or neural networks for their linking decisions [7, 13, 28]. The
introduction of pre-trained transformer models [10] made it possible to create
representations of mentions and entities from text without relying on other
intermediate representations. Gillick et al. [14] show how to learn dense
representations for mentions and entities, Logeswaran et al. [26] extend this
by introducing the zero-shot EL task and demonstrating that reasonable entity
embeddings can be derived solely from entity descriptions. Wu et al. [41]
introduce BLINK, the prevalent bi-encoder and cross-encoder paradigm for zero-
shot EL. Various improvements for zero-shot EL have been proposed based on
this paradigm. KG-ZESHEL [35] adds auxiliary entity information from knowledge
graph embeddings into the linking process; Partalidou et al. [30] propose
alternative pooling functions for the bi-encoder to increase the accuracy of
the candidate generation step.

Cross-Document Coreference Resolution  NIL-Aware EL is closely related to
Cross-Document Coreference Resolution (CDC), the task of identifying
coreferent entity mentions in documents without explicitly linking them to
entities in a KB [5]. Dutta and Weikum [11] explicitly tackle CDC in
combination with EL by applying clustering to bag-of-words representations of
entity mentions. More recently, Logan IV et al. [25] evaluate greedy nearest-
neighbour and hierarchical clustering strategies for CDC, however, without
explicitly evaluating them with respect to EL.

Entity Discovery and NIL-Aware EL  The majority of EL approaches may identify
NIL mentions (for instance, through a binary classifier or a ranking that
explicitly includes NIL), but does not process them in any way [37, 38]. In
2011, the TAC-KBP challenge [22] introduced a task that includes NIL
clustering; in the NEEL challenge [36] that is based on microposts, NIL
clustering was part of the task as well. Approaches that tackled these tasks
typically applied clustering based on similarity measures over the entity
mentions in the text [6, 12, 15, 29, 32]. More recently, Angell et al. [3]
train two separate bi-encoders and cross-encoders to compute inter-mention and
mention-entity affinities. Subsequently, they apply a bottom-up clustering for
refined linking predictions within single biomedical documents. Agarwal et al.
[1] extend the approach to cross-document linking through a clustering based
on minimum spanning trees over all mentions in the corpus. Clusters are formed
by successively adding edges to a graph as long as the constraint that a
cluster can contain at most one entity is not violated. They omit the cross-
encoder and employ a custom training procedure for the bi-encoder instead.
They explicitly evaluate their approach w.r.t. NIL entity discovery by
removing a part of the entities in the training set from zero-shot EL
benchmark datasets. In our approach, we employ a similar method for computing
affinities but employ a top-down clustering approach that aims to better
identify clusters of NIL mentions. The EDIN pipeline [24] also applies
clustering w.r.t. inter-mention and mention-entity affinities, but only to
identify NIL mention clusters on a dedicated adaptation dataset. Subsequently,
the entity index is enhanced with pooled representations of these clusters to
make a prediction of NIL entities possible. In their clustering phase, they
first produce groups of mentions and then identify NIL mention clusters by
checking whether less than 70% of the mentions are referring to the same
entity. As we aim to apply NIL-aware EL for KB population, relying on an
adaptation dataset is not possible. Still, we include the clustering method of
the EDIN pipeline in our experiments to compare how well the approaches detect
NIL mention clusters.

##  3 Task Formulation

A document corpus $\mathcal{D}$ contains a set of textual entity mentions
$\mathcal{M}$. Each of the mentions $m\in\mathcal{M}$ refers to an entity $e$
in the set of all entities $\mathcal{E}$. Given a knowledge base K with known
entities $\mathcal{E}^{k}$, the task in standard EL is to assign an entity
$\hat{e}\in\mathcal{E}^{k}$ to every mention in $\mathcal{M}$. In this
setting, we assume that $\mathcal{E}=\mathcal{E}^{k}$, i.e., all entities are
contained in K.

In NIL-aware EL, we drop the assumption that every mention links to an entity
contained in K. Instead there is a set of NIL entities $\mathcal{E}^{n}$ with
$\mathcal{E}^{k}\cup\mathcal{E}^{n}=\mathcal{E}$ and
$\mathcal{E}^{k}\cap\mathcal{E}^{n}=\emptyset$. For mentions $\mathcal{M}^{k}$
that refer to entities in K, the task is still to predict an entity
$\hat{e}\in\mathcal{E}^{k}$. For mentions $\mathcal{M}^{n}$ that refer to
entities not contained in K, the task is to predict a cluster identifier
$c\in\mathcal{C}$ so that the clustering $\mathcal{C}$ resembles the
distribution of mentions in $\mathcal{M}^{n}$ to entities in $\mathcal{E}^{n}$
as closely as possible. We assume that we are additionally operating in a
zero-shot setting, i.e., the training portion $\mathcal{D}_{train}$ of the
document corpus may not contain mentions for all entities in $\mathcal{E}$.

Note that, similar to related works [1, 26], we assume that the textual entity
mentions are already given. Further, we only investigate the relevant steps
for KB population, i.e., detection and disambiguation of NIL entities. While
we discard the indexing aspect, an EL model which includes the entities in
$\mathcal{E}^{n}$ can still be created in a subsequent step by training a new
model on the enhanced KB.

##  4 NASTyLinker: An Approach for NIL-Aware and Scalable Entity Linking

In this section, we describe our proposed approach for making NIL-aware EL
predictions. Figure 2 depicts the three main phases of the NASTyLinker
approach. In the Linking Phase, we first retrieve inter-mention and mention-
entity affinities from an underlying EL model for the subsequent clustering.
We define constraints for such a model and describe the one used in our
experiments in Section 4.1. During the Clustering Phase, clusters of mentions
and entity candidates are created using greedy nearest-neighbour clustering
(Section 4.2). Finally, we retrieve entity candidates for every cluster. In
the Conflict Resolution Phase, clusters are split based on transitive mention-
entity affinities to ensure that a cluster contains at most one known entity
(Section 4.3).

Figure 2: Main phases of the NASTyLinker approach. Dotted lines show top-k
affinity scores, solid lines indicate the highest transitive affinity scores.

###  4.1 Entity Linking Model

In the Linking Phase we compute the k most similar mentions and entities for
every mention in $\mathcal{M}$ (dotted lines in Figure 2). The underlying EL
model has to provide a function $\phi$ with $\phi(m,e)\in[0;1]$ for the
similarity between mention $m$ and entity $e$ as well as
$\phi(m,m^{\prime})\in[0;1]$ for the similarity between mentions $m$ and
$m^{\prime}$. In addition to that, it must be possible to retrieve the top k
mention and entity candidates for a given mention in an efficient manner.

For our experiments with NASTyLinker, we choose the BLINK architecture [41] as
the underlying EL model as it provides the foundation for many state-of-the-
art EL models. Furthermore, as the bi-encoder creates embeddings for mentions
and entities alike, methods for an approximate nearest neighbour search like
FAISS [23] can be used to retrieve linking candidates efficiently. As the
application of the cross-encoder is the most time-consuming part of this
model, we explore in our experiments the trade-off between linking performance
and runtime when reranking only inter-mention affinities, only mention-entity
affinities, or both.

Partalidou et al. [30] propose several layouts for structuring the input
sequence of mentions and entities for the Transformer model. We achieved the
best results with the mention layout

        [CLS] [<type>] <mention label> [CTX] <mention context> [SEP]

and the entity layout

        [CLS] [<type>] <entity title> [CTX] <entity description> [SEP]

where [CTX] is a special delimiter token and [<type>] is a placeholder for a
special token of the mention type (POS-tag) or entity type (top-level type in
the KB). For optimization, we stick to Wu et al. [41] and use in-batch (hard)
negatives for the bi-encoder, and bi-encoder-generated negatives for the
cross-encoder.

###  4.2 Cluster Initialization

To produce an initial mention clustering, we follow Logan IV et al. [25] and
use a greedy nearest-neighbour clustering. Given the mention affinity
threshold $\tau_{m}$, the mentions $\mathcal{M}$ are grouped into clusters
$\mathcal{C}$ so that two mentions $m,m^{\prime}\in\mathcal{M}$ belong to the
same cluster if $\phi(m,m^{\prime})>\tau_{m}$.

Further, we assign entity candidates to the clusters using a threshold for
entity affinity $\tau_{e}$. For a cluster $C\in\mathcal{C}$ with mentions
$M_{c}$, we select the known entities with the highest affinity to each
cluster mention:

$E^{k}_{c}=\bigcup_{m\in
M_{c}}\\{\mathop{argmax}\limits_{e\in\mathcal{E}^{k}}\phi(m,e):\phi(m,e)>\tau_{e}\\}.$
(1)

In Figure 2, the dotted lines represent affinities greater than the thresholds
$\tau_{m}$ and $\tau_{e}$, respectively. Cluster $C_{1}$ contains four loosely
connected mentions with $m_{1}$ and $m_{2}$ directly connected to the entity
candidate $e_{a}$. Either all four mentions refer to $e_{a}$ as they are
transitively connected, or some mentions refer to an entity in
$\mathcal{E}^{n}$ (e.g., a situation like in Figure 1a and 1b). Cluster
$C_{2}$ contains several mentions with two known entity candidates $e_{b}$ and
$e_{c}$, making a trivial assignment of mentions to entities impossible.
Finally, cluster $C_{3}$ contains three connected mentions without any
assigned entity candidates, most likely representing a NIL entity. Conflicts
like the ones occurring in the former two clusters are resolved in the
subsequent resolution phase.

###  4.3 Cluster Conflict Resolution

The objectives of the Conflict Resolution Phase are twofold: For every cluster
$C\in\mathcal{C}$ we (1) find sub-clusters with $|E^{k}_{c}|=1$ (c.f.
$C_{1B}$, $C_{2A}$, and $C_{2B}$ in Figure 2), and (2) identify mentions in
$M_{c}$ that do not refer to any entity in $E^{k}_{c}$. For these, we create
one or more sub-clusters representing the NIL entities $E^{n}_{c}$ of $C$
(c.f. $C_{1A}$ and $C_{3}$ in Figure 2).

For conflict resolution, we view a cluster $C\in\mathcal{C}$ as a graph
$G_{c}$ with $M_{c}\cup E^{k}_{c}$ as nodes, and affinities above threshold as
edges. To ensure objective (1), we assign every mention in a cluster to the
candidate entity with the highest transitive, defined as follows:

$\phi^{*}(m,e)=\mathop{max}\limits_{m\sim e\in G_{c}}\prod_{u,v}^{m\sim
e}\phi(u,v)$ (2)

with $m\sim e$ denoting a path from a mention $m$ to an entity $e$ in $G_{c}$
and $(u,v)$ a single edge. The rationale for this metric is to favour strong
contextual similarity between mentions over the mediocre similarity between a
mention and an entity. As the entity context is coming from a different data
corpus (i.e., information from a KB) than the mention context, it is more
likely to happen that the contexts for a mention and its linked entity are
dissimilar than the contexts of two mentions linking to the same entity.

Example 1 With affinities $\phi(m_{6},e_{b})=0.9$, $\phi(m_{6},m_{7})=0.9$,
$\phi(m_{7},e_{c})=0.8$, and paths $m_{7}-m_{6}-e_{b}$, $m_{7}-e_{c}$ from
Figure 2, we find that $\phi^{*}(m_{7},e_{b})=0.81>\phi^{*}(m_{7},e_{c})=0.8$,
resulting in the assignment of $m_{7}$ to the cluster of $e_{b}$ in spite of
$e_{c}$ being the most likely entity for $m_{7}$ w.r.t. $\phi$.

To ensure objective (2), we introduce a threshold $\tau_{a}$ as a lower limit
for the transitive affinity between a mention and an entity. We label mentions
as NIL mentions if they do not have a transitive affinity higher than the
threshold to any entity in $E^{k}_{c}$:

$M^{n}_{c}=\\{m\in M_{c}|\nexists e\in E^{k}_{c}:\phi^{*}(m,e)>\tau_{a}\\}$
(3)

From $M^{n}_{c}$ we produce one or more mention clusters similar to the
initialization step in Section 4.2.

Example 2 With $\tau_{a}=0.75$, affinities $\phi(m_{1},e_{a})=0.9$,
$\phi(m_{1},m_{3})=0.8$, $\phi(m_{3},m_{4})=0.9$, and path
$m_{4}-m_{3}-m_{1}-e_{a}$ from Figure 2, we find that
$\phi^{*}(m_{3},e_{a})=0.72<\tau_{a}$ and
$\phi^{*}(m_{4},e_{a})=0.648<\tau_{a}$. $m_{3}$ and $m_{4}$ are labelled as
NIL mentions and form - due to their direct connection - the single cluster
$C_{1B}$.

The function $\phi^{*}$ can be computed efficiently on a graph using
Dijkstra’s algorithm with $-log\phi$ as a function for edge weights. Edges are
only inserted in the graph for $\phi>\tau_{a}$, avoiding undefined edge
weights in the case of $\phi=0$.

##  5 Experiments

We first describe the datasets and experimental setup used for the evaluation
of NASTyLinker. Then, we compare the performance of our approach with related
NIL-aware clustering approaches on the NILK dataset [21] and analyze its
potential to scale. Finally, we report on the application of NASTyLinker for
KB population by linking entities in Wikipedia listings.

###  5.1 Datasets

Table 1: Mention and entity occurrences in the partitions of the datasets. NIL mention counts for $\mathcal{D}^{L}$ are estimated w.r.t. partial completeness assumption. Furthermore, the number of NIL entities $\mathcal{E}^{n}$ in the listings dataset is not known. For $\mathcal{D}^{L}_{pred}$ a single mention count is displayed as we cannot know whether a mention in $\mathcal{M}$ links to an entity in $\mathcal{E}^{k}$ or $\mathcal{E}^{n}$. Dataset |  $|\mathcal{M}^{k}|$ |  $|\mathcal{M}^{n}|$ |  $|\mathcal{E}^{k}|$ |  $|\mathcal{E}^{n}|$
---|---|---|---|---
NILK |  Training ($\mathcal{D}^{N}_{train}$) |  85,052,764 |  11,327,039 |  3,382,497 |  282,210
|  Validation ($\mathcal{D}^{N}_{val}$) |  10,525,107 |  11,162,948 |  1,422,812 |  135,276
|  Test ($\mathcal{D}^{N}_{test}$) |  10,451,126 |  11,162,497 |  1,422,815 |  135,279
LISTING |  Training ($\mathcal{D}^{L}_{train}$) |  11,690,019 |  16,760,273 |  3,073,238 |  ?
|  Validation ($\mathcal{D}^{L}_{val}$) |  13,882,641 |  12,272,941 |  1,695,156 |  ?
|  Test ($\mathcal{D}^{L}_{test}$) |  13,884,066 |  12,259,072 |  1,701,015 |  ?
|  Prediction ($\mathcal{D}^{L}_{pred}$) | 18,658,271 |  ? |  ?

####  5.1.1 NILK

NILK is a dataset that is explicitly created to evaluate EL both for known and
NIL entities. It uses Wikipedia as a text corpus and Wikidata [39] as
reference KB. All entities contained in Wikidata up to 2017 are labelled as
known entities and entities added to Wikidata between 2017 and 2021 are
labelled as NIL entities. Mention and entity counts of NILK are displayed in
Table 1. About 1% of mentions in NILK are NIL mentions, and about 6% of
entities are NIL entities. NIL entities are probably slightly biased towards
more popular entities, as the fact that they are present in Wikidata hints at
a certain popularity, which may be higher than the popularity of an average
NIL entity. Hence, the average number of mentions per NIL entity is quite high
in this dataset: half of the entities are mentioned more than once, and more
than 15% are even mentioned more than 5 times. Mention boundaries are already
given and the authors define partitions for training, validation, and test,
which are split in a zero-shot manner w.r.t. NIL entities. As mention context,
the authors provide 500 characters before and after the actual mention
occurrence in a Wikipedia page. As entity descriptions, we use Wikipedia
abstracts.222While there are entities in Wikidata which do not have a
Wikipedia page, this case does not occur in NILK by construction.

####  5.1.2 Wikipedia Listings

The LISTING dataset was extracted in prior work [20] and consists of entity
mentions in enumerations and tables of Wikipedia. Instead of all possible
mentions, the focus is only on subject entities, which we define as all
entities in a listing appearing as instances to a common concept [19]. So
every item in a listing is assumed to have one main entity the item is about.
For example, in Fig. 1d, the soap opera characters are considered entity
mentions, while the actors are not.

As reference KB we use CaLiGraph [17, 18]. Mention and entity statistics are
given in Table 1. We partition the data into train, validation, and test while
making sure that listings on a page are all in the same split. Contrary to
NILK, the LISTING dataset does not contain explicit labels for NIL entities.
Instead, we define NIL entities using the partial completeness assumption
(PCA). Given a listing with multiple mentions, we only incorporate them into
training or test data if at least one mention is linked to a known entity.
Then, by PCA, we assume that all mentions that can be linked are actually
linked. All other mentions are assigned a new unique entity identifier. The
prediction partition $\mathcal{D}^{L}_{pred}$, however, contains all mentions
without a linked entity (i.e., they may link to a known or to a NIL entity).
We use the text of the listing item as mention context for the dataset, and we
use Wikipedia abstracts as entity descriptions.

We have considered further datasets that were used for evaluation of NIL-aware
approaches for evaluation (e.g. from challenges like TAC-KBP or Microposts
[8]), but discarded them due to their small size or not being free to use.

###  5.2 Metrics

####  5.2.1 Classification Metrics

We compute precision, recall, and F1-score as well as aggregations of the
metrics on the instance level (micro average). As the evaluated approaches are
not aware of the true NIL entities, they assign cluster identifiers to (what
they assume to be) NIL mentions. To compute the classification metrics, it is
necessary to map the cluster identifiers to actual NIL entities. Kassner et
al. [24] allow the assignment of multiple cluster identifiers to the same NIL
entity. This assumption would yield overly optimistic results. Instead, we
only allow one-to-one mappings between cluster identifiers and NIL entities.
Finding an optimal assignment for this scenario is equivalent to solving the
linear sum assignment problem [2], for which efficient algorithms exist.

####  5.2.2 Clustering Metrics

Following related approaches [1, 24], we additionally provide normalized
mutual information (NMI) and adjusted rand index (ARI) as clustering metrics
for the comparison of the approaches to settings where no gold labels of NIL
entities may be available.333We implement further clustering metrics
(B-Cubed+, CEAF, MUC) but do not list them as they are similar to or
adaptations of the classification metrics. For known entities, however, the
classification metrics will most likely be more expressive than the clustering
metrics as the latter treat multiple clusters with the same known entity as
their label still as separate clusters.

###  5.3 Evaluated Approaches

####  5.3.1 EL Model

We compute inter-mention and mention-entity affinities with a bi-encoder
similar to BLINK [41]. As the reranking of bi-encoder results with a cross-
encoder is costly, we evaluate different scenarios where the cross-encoder is
omitted (No Reranking), applied to inter-mention affinities only (Mention
Reranking), applied to mention-entity affinities only (Entity Reranking), or
applied to both (Full Reranking). We use the Sentence-BERT implementation of
the bi-encoder and cross-encoder [34] with all-MiniLM-L12-v2 and distilbert-
base-cased as respective base models. The base models are fine-tuned for at
most one million steps on the training partitions of the datasets. Longer
fine-tuning did not yield substantial improvements. We use a batch size of 256
for the bi-encoder and 128 for the cross-encoder. For efficient retrieval of
candidates from the bi-encoder, we apply approximate nearest neighbour search
with hnswlib [27].

We use the plain bi-encoder and cross-encoder predictions of the EL model as
baselines. Additionally, we evaluate a trivial Exact Match approach, where we
link a mention to an entity if their textual representations match
exactly.444We apply simple preprocessing like lower-casing and removal of
special characters. In case of multiple matches, the more popular entity
(w.r.t. ingoing and outgoing links in the KB) is selected. Naturally, this
approach cannot handle NIL entities.

####  5.3.2 Clustering Approaches

Apart from the NASTyLinker clustering as described in Section 4, we apply the
clustering approaches of Kassner et al. [24] and Agarwal et al. [1] for
comparison.555We tried to compare with the full approach of Agarwal et al. but
they do not provide any code and our efforts to re-implement it did not yield
improved results. The clustering approach of Kassner et al., which we call
Majority Clustering, applies a greedy clustering and assigns a known entity
$e$ to a cluster if at least 70% of mentions in the cluster have the highest
affinity to $e$. Similarly to NASTyLinker, they use hyperparameters as
thresholds for minimum inter-mention and mention-entity affinities.

The clustering approach of Agarwal et al., which we call Bottom-Up Clustering,
starts with an empty graph and iteratively adds the edge with the highest
affinity, as long as it does not violate the constraint of a cluster having at
most one entity. They use a single hyperparameter as a threshold for the
minimum affinity of an edge, be it inter-mention or mention-entity.

####  5.3.3 Hyperparameter Tuning

We select the hyperparameters of the EL model
($k,learning\\_rate,warmup\\_steps$) and the thresholds of all three
clustering approaches w.r.t. micro F1-score on the validation partition of the
datasets. For a fair comparison, we also test multiple values for the
threshold for entity assignment of Majority Clustering, which in the original
paper was fixed at 0.7.

Our experiments are run on a single machine having 96 CPUs, 1TB of RAM, and an
NVIDIA RTX A6000 GPU with 48GB of RAM.

###  5.4 Entity Linking Performance

We tune hyperparameters by evaluating on $\mathcal{D}^{N}_{val}$. For the EL
model, we use a k of 4, a learning rate of 2e-5, and no warmup steps. For
$\tau_{m}$, a value between 0.8 and 0.9 works best for all approaches. For
$\tau_{e}$, the best values revolve around 0.9 for NASTyLinker and Bottom-Up
Clustering, and around 0.8 for Majority Clustering. We use an affinity
threshold $\tau_{a}$ of 0.75 for NASTyLinker and find that the 0.7 threshold
of Majority Clustering produces the best results.

Table 2: Results for the test partition $\mathcal{D}^{N}_{test}$ of the NILK dataset. Approach | Known | NIL | Micro
---|---|---|---
|  F1 |  NMI |  ARI |  F1 |  NMI |  ARI |  F1 |  NMI |  ARI
No Clustering |  Exact Match |  79.5 |  — |  — |  00.0 |  — |  — |  78.1 |  — |  —
|  Bi-Encoder |  80.8 |  — |  — |  00.0 |  — |  — |  79.1 |  — |  —
|  Cross-Encoder |  89.0 |  — |  — |  00.0 |  — |  — |  87.1 |  — |  —
Clustering & No Reranking |  Bottom-Up |  64.6 |  99.0 |  97.5 |  41.6 |  94.8 |  81.8 |  64.1 |  96.8 |  93.5
|  Majority |  59.4 |  99.3 |  98.0 |  49.8 |  92.7 |  82.7 |  59.2 |  96.6 |  94.6
|  NASTyLinker |  76.8 |  98.6 |  95.3 |  40.8 |  95.2 |  76.8 |  76.0 |  97.3 |  90.3
Clustering & Mention Reranking |  Bottom-Up |  65.7 |  97.1 |  98.9 |  41.5 |  94.6 |  10.0 |  65.1 |  96.0 |  66.0
|  Majority |  66.6 |  92.4 |  74.8 |  44.0 |  94.4 |  73.2 |  66.1 |  92.4 |  70.4
|  NASTyLinker |  74.2 |  99.0 |  96.6 |  39.2 |  85.6 |  16.5 |  73.5 |  95.5 |  81.6
Clustering & Entity Reranking |  Bottom-Up |  89.0 |  99.3 |  96.2 |  41.6 |  94.1 |  58.0 |  87.9 |  98.2 |  92.6
|  Majority |  74.2 |  99.1 |  99.3 |  54.1 |  89.3 |  92.5 |  73.7 |  96.6 |  94.6
|  NASTyLinker |  90.4 |  99.3 |  95.5 |  43.7 |  94.6 |  85.3 |  89.4 |  98.5 |  84.1
Clustering & Full Reranking |  Bottom-Up |  84.2 |  99.6 |  98.9 |  41.8 |  84.6 |  3.2 |  83.3 |  96.2 |  65.5
|  Majority |  80.3 |  95.1 |  95.9 |  51.7 |  90.0 |  39.2 |  79.6 |  93.9 |  70.4
|  NASTyLinker |  87.9 |  99.5 |  99.2 |  42.5 |  87.6 |  33.6 |  86.9 |  97.4 |  71.7

####  5.4.1 NILK Results

As shown in Table 2, we evaluate all clustering approaches on
$\mathcal{D}^{N}_{test}$ in different reranking scenarios. We find Exact Match
already to be a strong baseline for known entities with an F1 of 79.5%, which
the Cross-Encoder outperforms by approximately 10%. Even without reranking,
the three clustering approaches are able to achieve an F1-score between 40%
and 50% for NIL entities. Overall, Majority Clustering is best suited to
identify NIL entities. It is the only one to substantially benefit from
reranking, increasing the F1-score by 10% when applying entity reranking.
Especially for linking known entities, applying only entity reranking is the
most favourable scenario, leading even to slight improvements over the
baseline approaches that focus only on known entities.

As the reranking of mentions tends to lead to a decrease in results while
considerably increasing runtime, we omit mention reranking (and hence, full
reranking) in experiments with Wikipedia listings. In the remaining scenarios,
NASTyLinker finds the best balance between the linking of known entities and
the identification of NIL entities w.r.t. F1-score and NMI.

####  5.4.2 Runtime and Scalability

The fine-tuning of the bi-encoder and cross-encoder models took 2 hours and 12
hours, respectively. For prediction with a k of 4 on $\mathcal{D}^{N}_{test}$,
the bi-encoder needed 6 hours. Reranking entity affinities with the cross-
encoder took 38 hours. Clustering the results with any of the three approaches
took an additional 8 to 12 minutes.

Figure 3: Runtime of NASTyLinker components for predictions on samples of
$\mathcal{D}^{N}_{test}$.

In Figure 3 we give an overview of the runtime of NASTyLinker components,
compared over various sample sizes of $\mathcal{D}^{N}_{test}$. Overall, we
can see that the total runtime scales linearly. With a smaller sample size,
the computation of embeddings and nearest neighbour search with the bi-encoder
is responsible for a larger fraction of the total runtime. We find that this
is due to the relatively large overhead of creating the index for the
approximate nearest neighbour search. With increasing sample size, this factor
is less important for the overall runtime. In general, entity reranking is
responsible for most of the total runtime.

The runtime of the clustering itself is responsible for approximately 1% of
total runtime and we do not expect it to increase substantially, as Dijkstra’s
algorithm has log-linear complexity and the size of mention clusters can be
controlled by the threshold $\tau_{m}$. Hence, the runtime of NASTyLinker is
expected to grow proportionally to the runtime of BLINK [41] for increasing
sizes of datasets. If runtime is an important factor, one might consider
skipping entity reranking as NASTyLinker still produces reasonable results
when relying on bi-encoder affinities only.

###  5.5 Linking Entities in Wikipedia Listings

As the average mention context length in the LISTING dataset is lower than the
one in NILK, fine-tuning the EL models took only a total of 8 hours. We find
that most of the hyperparameters chosen for NILK are a reasonable choice for
this dataset as well. For entity reranking, however, the approaches produce
better results when the thresholds $\tau_{m}$ and $\tau_{a}$ are slightly
increased to 0.9 and 0.85.

####  5.5.1 Results on Test Partition

Linking results for $\mathcal{D}^{L}_{test}$ are provided in Table 3. As we
rely on PCA for the labelling of NIL mentions, we only know whether a mention
is a NIL mention without knowing which NIL mentions refer to the same entity.
Hence, we can only compute results for known entities and for overall
predictions. For the latter, we simply assume that any prediction made for a
NIL mention is incorrect. With this assumption, we are obviously not able to
produce realistic performance estimates, but we are able to see the impact of
being NIL-aware (and hence, make no prediction for NIL mentions) on the
overall linking performance.

Due to their majority mechanism, Majority Clustering identifies known entities
with very high precision, but at the cost of a reduced recall. The scores of
Bottom-Up Clustering and NASTyLinker are comparable when considering known
entities, but diverge w.r.t. the micro average. In the entity reranking
scenario, NASTyLinker achieves the overall best micro F1-score with 86.7%.
This, however, has to be taken with a grain of salt as we do not know how many
of the heuristically labelled NIL mentions are actually referring to NIL
entities and how many refer to known entities.

Table 3: Results for the test partition $\mathcal{D}^{L}_{test}$ of the LISTING dataset. No results for NIL are given because the real NIL entities $\mathcal{E}^{n}$ are not available for this dataset. For the micro average, we label every prediction made for a mention linked to an entity in $\mathcal{E}^{n}$ as incorrect. Approach | Known | Micro
---|---|---
|  P |  R |  F1 |  P |  R |  F1
No Clustering |  Exact Match |  91.4 |  73.5 |  81.5 |  81.1 |  73.5 |  77.1
|  Bi-Encoder |  88.6 |  88.6 |  88.6 |  62.6 |  88.6 |  73.4
|  Cross-Encoder |  93.7 |  93.8 |  93.8 |  66.2 |  93.8 |  77.6
Clustering & No Reranking |  Bottom-Up |  89.7 |  84.9 |  87.2 |  63.9 |  84.9 |  72.9
|  Majority |  95.2 |  67.9 |  79.2 |  78.1 |  67.9 |  72.6
|  NASTyLinker |  90.6 |  78.5 |  84.1 |  70.7 |  78.5 |  74.4
Clustering & Entity Reranking |  Bottom-Up |  94.2 |  90.8 |  92.5 |  75.3 |  90.8 |  82.3
|  Majority |  98.8 |  76.2 |  86.0 |  93.4 |  76.2 |  83.9
|  NASTyLinker |  97.0 |  87.0 |  91.8 |  88.5 |  87.0 |  87.7

####  5.5.2 Knowledge Graph Population Statistics

The partition $\mathcal{D}^{L}_{pred}$ of the LISTING dataset contains only
mentions for which we don’t know whether they link to a known or to a NIL
entity. To make predictions for these mentions, we run the NASTyLinker
approach on the whole LISTING corpus, i.e. on a total of 38 million mentions,
as we need representations of all known entities for the clustering step.
These mentions were extracted from 2.9 million listings on 1.4 million
Wikipedia pages. As reference KB, we use the knowledge graph CaLiGraph which
is based on Wikipedia and hence contains entities for all 5.8 million
Wikipedia articles.

The total runtime was 62 hours, with 14 hours for the bi-encoder, 47 hours for
the cross-encoder, and 45 minutes for the clustering. We find 13.4 million
mentions (i.e., 70%) to be NIL mentions which refer to 7.6 million NIL
entities. The remaining 5.2 million mentions refer to 1.4 million entities
that exist in CaLiGraph already. By integrating the discovered NIL entities
into CaLiGraph, we would increase its entity count by 130%. Further, the
discovered mentions for known entities can be used to enrich the
representations of the entities in the knowledge graph through various
knowledge graph completion methods [19].

####  5.5.3 Qualitative Analysis

To evaluate the actual linking performance on the set of unlabeled mentions
$\mathcal{D}^{L}_{pred}$, we conducted a manual inspection of the results. We
randomly picked 100 mentions and 100 clusters666The sampling of clusters was
stratified w.r.t. cluster size. and identified, if incorrect, the type of
error.777We evaluated the linking and clustering decision w.r.t. the top-4
mention and entity candidates produced by the bi-encoder. Although recall@4
for the bi-encoder is 97%, some relevant candidates might have been missed.
The results of this evaluation are given in Table 4. Overall, we find the
outcome to agree with the results of NASTyLinker on $\mathcal{D}^{L}_{test}$.
Hence, the approach produces highly accurate results, which we observed even
for difficult cases. For example, the approach correctly created NIL entity
clusters for the mention North Course referring to a racing horse (in pages
Appleton Stakes and Oceanport Stakes), a golf course in Ontario, CA (in page
Tournament Players Club), and a golf course in Florida, US (in page Pete Dye).

Table 4: Results of the manual evaluation of 100 clusters and 100 mentions. Columns group the results by actual entity type (known, NIL, overall), rows group by prediction outcome. Accuracy values may deviate by $\pm$9.6% for mentions and by $\pm$7.0% for clusters (95% confidence). Prediction | Mentions | Clusters
---|---|---
|  $\mathcal{E}^{k}$ |  $\mathcal{E}^{n}$ |  $\mathcal{E}$ |  $\mathcal{E}^{k}$ |  $\mathcal{E}^{n}$ |  $\mathcal{E}$
Correct |  20 |  64 |  84 |  8 |  71 |  79
Incorrectly linked to NIL entity |  3 |  — |  3 |  1 |  — |  1
Incorrectly linked to known entity |  — |  7 |  7 |  — |  3 |  3
Not all mentions of entity in cluster |  — |  — |  — |  8 |  0 |  8
Mentions from multiple entities in cluster |  — |  — |  — |  1 |  4 |  5
Ignored (mention extracted incorrectly) |  — |  — |  6 |  — |  — |  4
Total Count |  23 |  71 |  94 |  18 |  78 |  96
Accuracy (%) |  87.0 |  90.1 |  89.4 |  44.4 |  91.0 |  82.3

While the linking performance is quite consistent for mentions, the
correctness of clusters for known entities is significantly lower than for NIL
entities.888For the evaluation to be significant, we treat all clusters
referring to the same known entity as a single cluster. This drop in
performance is not due to NASTyLinker being incapable of linking to known
entities correctly (as the accuracy of 87% on mention-level shows). Instead,
it can rather be attributed to the fact that clusters of known entities
contain 3.8 mentions on average, while clusters of NIL entities contain 1.7
mentions on average. Hence, the likelihood of missing at least one mention is
a lot higher, which is also the main error for known clusters.

Compared to the results on NILK, the linking accuracy for NIL mentions is much
higher. We explain this with the different kinds of NIL entities contained in
the two datasets. While an average NIL entity is mentioned 4.6 times in NILK,
our results indicate that this number is approximately 1.7 for the LISTING
dataset. The latter dataset may hence contain a lot of easy-to-link mentions
by assigning them their own cluster.

##  6 Conclusion and Outlook

With NASTyLinker, we introduce a NIL-aware EL approach that is capable of
making high-quality predictions for known and for NIL entities. In the
practical setting of EL in Wikipedia listings, we show that our approach can
be used to populate a knowledge graph with a large number of additional
entities as well as to enrich representations of existing entities.

Although the results look promising at a first glance, there is still a lot to
improve as even small errors can multiply in downstream applications. For
future work, we plan to concentrate on establishing a full end-to-end pipeline
that includes the detection of mention, as recent works demonstrate how this
can substantially reduce runtime without a decrease in performance [4, 24].
This will also open the path to a training procedure that considers NIL
entities already during the creation of embeddings. Additionally, we will
explore how the dependencies between items in listings can be exploited to
further improve predictions.

## References

  * [1] Agarwal, D., Angell, R., Monath, N., McCallum, A.: Entity linking and discovery via arborescence-based supervised clustering. arXiv preprint arXiv:2109.01242 (2021)
  * [2] Alfaro, C.A., Perez, S.L., Valencia, C.E., Vargas, M.C.: The assignment problem revisited. Optimization Letters 16(5), 1531–1548 (2022)
  * [3] Angell, R., Monath, N., Mohan, S., Yadav, N., McCallum, A.: Clustering-based inference for biomedical entity linking. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. pp. 2598–2608 (2021)
  * [4] Ayoola, T., Tyagi, S., Fisher, J., Christodoulopoulos, C., Pierleoni, A.: ReFinED: An efficient zero-shot-capable approach to end-to-end entity linking. arXiv preprint arXiv:2207.04108 (2022)
  * [5] Bagga, A., Baldwin, B.: Entity-based cross-document coreferencing using the vector space model. In: COLING 1998 Volume 1: The 17th International Conference on Computational Linguistics (1998)
  * [6] Blissett, K., Ji, H.: Cross-lingual NIL entity clustering for low-resource languages. In: Proceedings of the Second Workshop on Computational Models of Reference, Anaphora and Coreference. pp. 20–25 (2019)
  * [7] Cucerzan, S.: Large-scale named entity disambiguation based on Wikipedia data. In: Proceedings of the 2007 joint conference on empirical methods in natural language processing and computational natural language learning (EMNLP-CoNLL). pp. 708–716 (2007)
  * [8] Dadzie, A., Preotiuc-Pietro, D., Radovanovic, D., Basave, A.E.C., Weller, K. (eds.): Proceedings of the 6th Workshop on ’Making Sense of Microposts’ co-located with the 25th International World Wide Web Conference (WWW 2016), Montréal, Canada, April 11, 2016, CEUR Workshop Proceedings, vol. 1691. CEUR-WS.org (2016), http://ceur-ws.org/Vol-1691
  * [9] Das, R., Godbole, A., Kavarthapu, D., Gong, Z., Singhal, A., Yu, M., Guo, X., Gao, T., Zamani, H., Zaheer, M., et al.: Multi-step entity-centric information retrieval for multi-hop question answering. In: Proceedings of the 2nd Workshop on Machine Reading for Question Answering. pp. 113–118 (2019)
  * [10] Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
  * [11] Dutta, S., Weikum, G.: C3el: A joint model for cross-document co-reference resolution and entity linking. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. pp. 846–856 (2015)
  * [12] Fahrni, A., Heinzerling, B., Göckel, T., Strube, M.: Hits’monolingual and cross-lingual entity linking system at tac 2013. In: TAC. Citeseer (2013)
  * [13] Ganea, O.E., Hofmann, T.: Deep joint entity disambiguation with local neural attention. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. pp. 2619–2629 (2017)
  * [14] Gillick, D., Kulkarni, S., Lansing, L., Presta, A., Baldridge, J., Ie, E., Garcia-Olano, D.: Learning dense representations for entity retrieval. In: Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL). pp. 528–537 (2019)
  * [15] Greenfield, K., Caceres, R.S., Coury, M., Geyer, K., Gwon, Y., Matterer, J., Mensch, A.C., Sahin, C.S., Simek, O.: A reverse approach to named entity extraction and linking in microposts. In: # Microposts. pp. 67–69 (2016)
  * [16] Heist, N., Hertling, S., Ringler, D., Paulheim, H.: Knowledge graphs on the web-an overview. Knowledge Graphs for eXplainable Artificial Intelligence pp. 3–22 (2020)
  * [17] Heist, N., Paulheim, H.: Uncovering the semantics of wikipedia categories. In: The Semantic Web–ISWC 2019: 18th International Semantic Web Conference, Auckland, New Zealand, October 26–30, 2019, Proceedings, Part I 18. pp. 219–236. Springer (2019)
  * [18] Heist, N., Paulheim, H.: Entity extraction from Wikipedia list pages. In: European Semantic Web Conference. pp. 327–342. Springer (2020)
  * [19] Heist, N., Paulheim, H.: Information extraction from co-occurring similar entities. In: Proceedings of the Web Conference 2021. pp. 3999–4009 (2021)
  * [20] Heist, N., Paulheim, H.: Transformer-based subject entity detection in Wikipedia listings. arXiv preprint arXiv:2210.01482 (2022)
  * [21] Iurshina, A., Pan, J., Boutalbi, R., Staab, S.: Nilk: Entity linking dataset targeting NIL-linking cases. In: Proceedings of the 31st ACM International Conference on Information & Knowledge Management. pp. 4069–4073 (2022)
  * [22] Ji, H., Grishman, R., Dang, H.T., Griffitt, K., Ellis, J.: Overview of the TAC 2010 knowledge base population track. In: Third text analysis conference (TAC 2010). vol. 3, pp. 3–3 (2010)
  * [23] Johnson, J., Douze, M., Jégou, H.: Billion-scale similarity search with GPUs. IEEE Transactions on Big Data 7(3), 535–547 (2019)
  * [24] Kassner, N., Petroni, F., Plekhanov, M., Riedel, S., Cancedda, N.: Edin: An end-to-end benchmark and pipeline for unknown entity discovery and indexing. arXiv preprint arXiv:2205.12570 (2022)
  * [25] Logan IV, R.L., McCallum, A., Singh, S., Bikel, D.: Benchmarking scalable methods for streaming cross document entity coreference. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). pp. 4717–4731 (2021)
  * [26] Logeswaran, L., Chang, M.W., Lee, K., Toutanova, K., Devlin, J., Lee, H.: Zero-shot entity linking by reading entity descriptions. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. pp. 3449–3460 (2019)
  * [27] Malkov, Y.A., Yashunin, D.A.: Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. IEEE transactions on pattern analysis and machine intelligence 42(4), 824–836 (2018)
  * [28] Milne, D., Witten, I.H.: Learning to link with Wikipedia. In: Proceedings of the 17th ACM conference on Information and knowledge management. pp. 509–518 (2008)
  * [29] Monahan, S., Lehmann, J., Nyberg, T., Plymale, J., Jung, A.: Cross-lingual cross-document coreference with entity linking. In: TAC (2011)
  * [30] Partalidou, E., Christou, D., Tsoumakas, G.: Improving zero-shot entity retrieval through effective dense representations. In: Proceedings of the 12th Hellenic Conference on Artificial Intelligence. pp. 1–5 (2022)
  * [31] Paulheim, H.: Knowledge graph refinement: A survey of approaches and evaluation methods. Semantic web 8(3), 489–508 (2017)
  * [32] Radford, W., Hachey, B., Honnibal, M., Nothman, J., Curran, J.R.: Naıve but effective nil clustering baselines–cmcrc at tac 2011. In: Proceedings of Text Analysis Conference (TAC 2011). Citeseer (2011)
  * [33] Rao, D., McNamee, P., Dredze, M.: Entity linking: Finding extracted entities in a knowledge base. In: Multi-source, multilingual information extraction and summarization, pp. 93–115. Springer (2013)
  * [34] Reimers, N., Gurevych, I.: Sentence-bert: Sentence embeddings using siamese BERT-networks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). pp. 3982–3992 (2019)
  * [35] Ristoski, P., Lin, Z., Zhou, Q.: KG-ZESHEL: Knowledge graph-enhanced zero-shot entity linking. In: Proceedings of the 11th on Knowledge Capture Conference. pp. 49–56 (2021)
  * [36] Rizzo, G., Pereira, B., Varga, A., Van Erp, M., Cano Basave, A.E.: Lessons learnt from the Named Entity rEcognition and linking (NEEL) challenge series. Semantic Web 8(5), 667–700 (2017)
  * [37] Sevgili, Ö., Shelmanov, A., Arkhipov, M., Panchenko, A., Biemann, C.: Neural entity linking: A survey of models based on deep learning. Semantic Web (Preprint), 1–44 (2022)
  * [38] Shen, W., Wang, J., Han, J.: Entity linking with a knowledge base: Issues, techniques, and solutions. IEEE Transactions on Knowledge and Data Engineering 27(2), 443–460 (2014)
  * [39] Vrandečić, D., Krötzsch, M.: Wikidata: a free collaborative knowledgebase. Communications of the ACM 57(10), 78–85 (2014)
  * [40] Wang, Z., Ng, P., Nallapati, R., Xiang, B.: Retrieval, re-ranking and multi-task learning for knowledge-base question answering. In: Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. pp. 347–357 (2021)
  * [41] Wu, L., Petroni, F., Josifoski, M., Riedel, S., Zettlemoyer, L.: Scalable zero-shot entity linking with dense entity retrieval. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). pp. 6397–6407 (2020)


# Study of afterpulsing in optical image intensifiers

Ryan Mahon  Dmitry Orlov  Rene Glazenborg  Andrei Nomerotski  Brookhaven
National Laboratory, Upton NY 11973, USA  Photonis Netherlands BV, 9301 ZR
Roden, The Netherlands

###### Abstract

We will describe the characteristics of the afterpulsing effect seen in the
optical intensifiers. It can be caused by either secondary electrons produced
by primary photoelectrons hitting the micro-channel plate surface or by
electron emission from the photocathode induced by the ion feedback. The
result of this effect are additional pulses delayed with respect to the
primary parent pulses. Using a fast data-driven camera, Tpx3Cam, we were able
to clearly show afterpulsing present in the data at short time differences and
small distances from the primary pulse, as well as to show the evolution of
the afterpulsing effect with increasing time difference. We also studied the
afterpulsing spatial distribution and observed an azimuthal asymmetry, which
we attribute to the afterpulsing ion component.

##  1 Introduction

Optical intensifiers discussed here are vacuum devices with fast
amplification, which allow for optical cameras to reach higher sensitivity.
The photons are converted to photoelectrons on a photocathode and undergo
avalanche amplififcation in the micro-channel plate (MCP) before they are
directed to a scintillator to produce a fast flash of light. As a result, a
single photon is converted to the flash, which is registered by a camera. A
variety of camera types are used with intensifiers, primarily cameras based on
charge-coupled devices, CCD, and on complimentary metal-oxide-semiconductor or
CMOS sensors [1, 2, 3, 4, 5]. Here we will be focusing on single photon
sensitive cameras, in particular, on the performance of intensified Tpx3Cam, a
data driven camera with nanosecond scale timing resolution and fast readout
[6]. In our measurements we used the Tpx3Cam as a tool to investigate
properties of intensifiers and, therefore, our results are applicable to all
types of intensified cameras.

Below we will describe the characteristics of afterpulsing, which are the
production of additional pulses delayed with respect to the primary parent
pulse. Afterpulsing can mimic and distort various effects in quantum optics
and quantum information science experiments, which involve registration and
analysis of single photons, especially when studying the temporal and spatial
correlations of multiple photons. An important example is the Hanbury Brown -
Twiss (HBT) effect, which manifests itself as correlations of thermal photon
pairs [7, 8, 9]. Other examples are coincidences of single photons in the
Hong-Ou-Mandel (HOM) interference [4, 10, 11] and false detection of bright
states in ion crystals in Paul traps [12].

In the following sections we describe the intensified Tpx3Cam camera used for
the measurements and provide details of the measurements to characterize the
afterpulsing phenomenon.

##  2 Tpx3Cam fast camera

Imaging of single photons throughout these experiments was done with a
Tpx3Cam, a data-driven time-stamping camera [13, 14, 6, 15, 16]. This camera
has high quantum efficiency (QE) [17] and a silicon optical sensor. This
sensor is then bump-bonded to Timepix3 [18], which is an application specific
integrated circuit (ASIC) with $256\times 256$ pixels of $55\times 55$
$\mu$m2. The time of arrival (ToA) is measured using the electronics present
in each pixel, which processes the incoming signals for hits that cross a
predefined threshold of around 600 electrons, with a precision of 1.56 ns.
Another important information, measured simultaneously with ToA for each hit,
is the time-over-threshold (ToT), which is related to the deposited energy in
each pixel. The ToT values are stored together with the ToA values as time
codes. The readout of Timepix3 is data driven, with a pixel deadtime of 475 ns
+ ToT, which allows for multi-hit functionality of every pixel, each
independent from others and very fast at a 80 Mpix/sec bandwidth [19].

###  2.1 Intensified Camera

For the single photon sensitive operation, the signal is amplified with the
addition of CricketTM [20], which contains integrated image intensifier, power
supply, and relay optics to project the light flashes from the intensifier
output window directly on to the optical sensor in the camera. The image
intensifier is a vacuum device comprised of a photocathode followed with a
micro-channel plate (MCP) and fast scintillator P47 with risetime of about 7
ns and maximum emission at 430 nm [21]. The image intensifier operates at a
high gain such that a single detected photon can creates a signal well above
the threshold of the Timepix3 camera. The quantum efficiency of the camera is
determined primarily by the intensifier photocathode with a variety of
photocathodes available, covering the spectral range from 120–900 nm[20, 6].
As an example, the hi-QE-green photocathode in the intensifier has QE of about
30% in the range of 400 - 480 nm and hi-QE-red photocathode has QE of about
20% in the range of 550 - 850 nm [22]. The MCP in the intensifier can have an
improved detection efficiency close to 100% [23].

Similar configurations of the intensified Tpx3Cam were used before for
characterization of quantum networks [10, 24], quantum target detection [25,
26], quantum imaging [11, 27, 28], studies of micromotion in ion traps [12,
29, 30, 31], neutron detection [32, 33, 34, 35], optical readout of time-
projection chamber (TPC) [36, 37] and lifetime imaging [38, 39, 40] studies.

###  2.2 Data post-processing

The left side of Figure 1 shows an example of raw TOA information for a pair
of single photons registered in Tpx3Cam. Upon observation, it can be seen that
photons appear as small groups of hit pixels. Subsequently, these pixels are
organized into clusters within a predefined time window of 300 ns, using a
recursive algorithm [14]. Since all hit pixels measure ToA and ToT
independently and provide the position information, it can be used for
centroiding to determine the coordinates of single photons. The ToT
information is used for the weighted average, giving an estimate of the x, y
coordinates for the incoming single photon with improved spatial resolution
[38]. The timing of the photon is estimated by using ToA of the pixel with the
largest ToT in the cluster. The above ToA is then adjusted for the so-called
time-walk, an effect caused by the variable pixel electronics time response,
which depends on the amplitude of the input signal [14, 41]. With this
correction, a 2 ns time resolution (rms) can be achieved for single photons
[24].

##  3 Afterpulsing

Figure 1: Left: An example of raw TOA information for a pair of single photons
registered in Tpx3Cam. Two clusters are seen with one delayed from the other
by about 250 ns, likely caused by afterpulsing. Right: Two mechanisms of
afterpulsing due to the ion feedback and secondary electrons.

Alterpulsing is an important feature of the intensifier. It can be caused by
secondary electrons or ions, which start another multiplication process in the
vicinity of the primary pulse with a short delay [42]. The right part of
Figure 1 illustrates these two mechanisms of afterpulsing generation. The
first one is induced by primary photoelectrons hitting the web of the input
MCP surface and producing secondary electrons. The number of secondary
electrons is typically about 5 – 10. Most of them have low energy of few eV
and are quickly collected by the nearest MCP channels, thus contributing to
the primary pulse. However, some of the secondary electrons have higher energy
and are able to fly away from the MCP and create their own avalanches. This
happens within sub-ns time and with space distribution extended to double
distance between photocathode and MCP, so to about 0.2 – 0.4 mm.

Another source of afterpulsing is a back flow of positively charged ions,
which are generated by electron-ion stimulated desorption with the electron
avalanche propagating through the MCP channels. These ions drift back to the
photocathode, hitting its surface and generating emission of photoelectrons.
Due to the ion higher mass the afterpulse is delayed with time distribution of
the ion-induced afterpulses extending to 100’s of ns. The left part of Figure
1 shows two clusters of hits resulting from two photons in a pair hitting the
Tpx3Cam. There is a clear delay between these two clusters of about 300-400ns
showing a clear example of the afterpulsing effect.

Measurements described here were performed with the hi-QE-red photocathode
[20]. We used the dark counts as a source of single photons with total rate of
about 80 kHz distributed uniformly across the photocathode. This occupancy is
entirely due to the thermal photoelectrons from the photocathode. After going
through the registration chain (MCP-P47-Tpx3Cam), these signals look exactly
the same and have same properties as response from the single photons.

To find the afterpulses in the data, we look at the differences in time and
position between successive hits. Figure 2 shows two dimensional distributions
of spatial differences in both the x and y directions for increasingly large
time differences between the hits. Each of six graphs in the figure represents
a 5 ns wide slice of data with different time difference between the pulses.
The empty $3\times 3$ pixel area in the middle is due to the requirement for
the two hits to be separated by at least one pixel. There is an obvious excess
of pulses spatially close to each other due to the afterpulsing. It also can
be seen from this figure that the afterpulsing effect quickly dissipates
compared to its abundance in the 0-5 ns range and becomes very small (but
still visible) by the time difference of about 200 ns between consecutive
photon hits.

Figure 2: An evolution of the spatial distribution of afterpulses with
increasing time between hits. Each graph represents a 5 ns wide slice of data
with varying time difference between the pulses. The empty $3\times 3$ pixel
square in the middle is due to the requirement of the two hits to be separated
by at least one pixel.

All of these graphs, except the one corresponding to the first 5 ns bin, also
have an obvious asymmetry of the azimuthal distribution, which indicates a
preferred direction. This is likely related to the orientation of channels in
the MCP glass with respect to the sensor coordinate system. We verified this
by rotating the intensifier by 90∘ with respect to the sensor, observing that
the asymmetry followed the rotation.

An interesting issue to explore was to look at the ratio of the sum of all ToT
values found within the asymmetrical region compared to the full ”donut”
shaped region as shown in the left part of Figure 3. The right part of the
figure shows this ratio over increasing time differences from 0 to 1000 ns
with binning of 10 ns. It can be seen that the ratio is about 0.3 at small
time differences corresponding to the ratio of corresponding geometrical areas
and indicating azimuthal uniformity. The ratio reaches a peak of about 0.5
after 100 ns when the shape becomes asymmetric, then begins to drop off and
falls back to the original 0.3 value, which is again consistent with a uniform
dark count rate occupancy. This behavior suggests that the afterpulsing is
only present until about 300 ns.

Figure 3:  Left: Diagram of spatial differences between the primary and
secondary hits showing a selection of the asymmetrical region with approximate
area of 30% with respect to the total area in the circle. Right: Ratio of the
sum of ToT values within asymmetrical region to the total ToT in the full area
within the circle.

From the above measurements we see that there are two phases of afterpulsing:
a bright and brief initial phase, which lasts about 10 ns or less and is
symmetric, and an asymmetric delayed phase with lower intensity and duration
of about 300 ns. We hypothesise that the first phase is due to the secondary
electrons which are produced at the MCP surface and, therefore, are symmetric
and the second phase is explained by generation of ions within the MCP
channels during the multiplication process. The ions are driven by electric
field towards the photocathode and acquire a direction as they move along the
channels. The channels have a small angle, about 7 degrees, with respect to
the orthogonal direction and, we believe, it is this that creates the
asymmetry in the distribution. They also have a delayed response due to their
slow movement and large distance to the photocathode.

Figure 4: Left: Distribution of distances between two successive hits. Right:
Distributions of time difference between successive hits for all distances and
distances less than 10 pixels.

To characterize further the afterpulsing, we plot distributions of distance
between successive hits and their separation in time, see Figure 4. There is
an obvious excess for the small values in both distributions, which we
estimate as deviations from the expected smooth behavior and interpret as
afterpulses. We found that the fraction of afterpulsing events closer than 10
pixels is 1.6%. The right side of Figure 4 shows distributions of time
differences between successive hits for two separate cases. The first case
considers all distance values. The flat background in this case corresponds to
the random coincidences between the dark count rate (DCR) hits. The second
distribution considers time differences only for cases where the corresponding
distance between successive hits is less than 10 pixels. It can be seen here
that the distribution drops to very small values after about 100 ns with a
fraction of events closer than 100 ns to be 1.4%. This is about the same
number as obtained earlier for the spatial distribution.

Figure 5: Distribution of distances for various time difference bins.

Figure 5 shows three different distributions of the distances between
successive hits selected for various time difference bins. In all cases the
bin width is 5 ns to maintain the same normalization at large distances, where
the afterpulsing is not important. The peaks at small distance values are a
result of afterpulsing and it can be seen that the peaks decrease as the time
difference increases. We also note that the earliest hits in the 0 to 5 ns bin
have a distribution, which is more peaked at 0 meaning smaller spatial
separations between the parent hit and afterpulse in the electron phase of
afterpulsing. For the last bin at 400 ns the afterpulsing is not visible
confirming that the ion phase of afterpulsing ends at about 300 ns.

Figure 6: Left: Distribution of number of clusters, including zero, in the
1000 ns time window after the parent pulse. Right: Distribution of number of
clusters, excluding zero, in the 1000 ns time window after the parent pulse.
The distributions are normalized to the first bin.

At last, we studied the multiplicity of afterpulses. The left plot in Figure 6
shows a distribution of number of clusters found after the primary pulse,
normalized to the number of counts at zero, which corresponds to the case of
no afterpulses. These distributions represent the number of clusters found
within a 1000 ns time period after an initial cluster and within a predefined
distance window, from 0 to 20 pixels. Again, it illustrates the fact that
afterpulses are dominated by the hits close in time and space to the original
parent hit. The right plot shows the same information but it starts at one so
excluding zero hits and requiring presence of at least one afterpulse. These
distributions show that there is no big difference in production of multiple
afterpulses between different mechanisms, that would be distinguishable in the
Tpx3Cam camera.

##  4 Conclusions

We studied the afterpulsing effect seen in the optical intensifiers using a
fast data-driven camera, Tpx3Cam. We were able to clearly show afterpulsing
present in the data at short time differences and distances from the primary
pulse, as well as show the evolution of the afterpulsing effect with
increasing time differences. We also studied the afterpulsing spatial
distribution, which shows a considerable azimuthal asymmetry. We demonstrated
different properties of afterpulsing for two different phases of this effect
attributing them to either secondary electron or ion mechanism. Performance of
the fast Tpx3Cam camera was the key element for the measurement, providing
necessary temporal and spatial information about the afterpulses, which was
necessary to distinguish different mechanism contributions.

The results of these studies provide useful insights for challenging
measurements of photon coincidences with intensified cameras, which may also
have spatial correlations. As an example we refer to the measurement of
bunched photons due the HOM effect reported in [10] and note that the authors
had to introduce a 28% correction to account for the afterpulsing, while the
probability for a single photon to have an afterpulse, which would pass the
analysis selections was 0.19%. We note that in this and all similar cases the
afterpulsing effect can be evaluated by collecting independent datasets and by
subtracting the afterpulsing contribution to the expected signal.

## References

  * [1] Giorgio Brida, Lucia Caspani, Alessandra Gatti, Marco Genovese, Alice Meda, and Ivano Ruo Berchera.  Measurement of sub-shot-noise spatial correlations without background subtraction.  Phys. Rev. Lett., 102:213602, May 2009.
  * [2] Matthew Reichert, Xiaohang Sun, and Jason W. Fleischer.  Quality of spatial entanglement propagation.  Phys. Rev. A, 95:063836, Jun 2017.
  * [3] Bradley M. Jost, Alexander V. Sergienko, Ayman F. Abouraddy, Bahaa E. A. Saleh, and Malvin C. Teich.  Spatial correlations of spontaneously down-converted photon pairs detected with a single-photon-sensitive ccd camera.  Opt. Express, 3(2):81–88, Jul 1998.
  * [4] Michał Jachura and Radosław Chrapkiewicz.  Shot-by-shot imaging of hong–ou–mandel interference with an intensified scmos camera.  Opt. Lett., 40(7):1540–1543, Apr 2015.
  * [5] Robert Fickler, Mario Krenn, Radek Lapkiewicz, Sven Ramelow, and Anton Zeilinger.  Real-time imaging of quantum entanglement.  Scientific Reports, 3:1914, May 2013.
  * [6] Andrei Nomerotski.  Imaging and time stamping of photons with nanosecond resolution in timepix based optical cameras.  Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 937:26–30, September 2019.
  * [7] R. Hanbury Brown and R. Q. Twiss.  A test of a new type of stellar interferometer on sirius.  Nature, 178(4541):1046–1048, November 1956.
  * [8] Richard Hanbury Brown.  The Intensity Interferometer.  Taylor & Francis, London, 1974.
  * [9] C. Foellmi.  Intensity interferometry and the second-order correlation function $g^$\\{$(2)$\\}$$ in astrophysics.  Astronomy & Astrophysics, 507(3):1719–1727, September 2009.
  * [10] Andrei Nomerotski, Dimitrios Katramatos, Paul Stankus, Peter Svihra, Guodong Cui, Sonali Gera, Mael Flament, and Eden Figueroa.  Spatial and temporal characterization of polarization entanglement.  International Journal of Quantum Information, 18(01):1941027, Feb 2020.
  * [11] Yingwen Zhang, Duncan England, Andrei Nomerotski, and Benjamin Sussman.  High speed imaging of spectral-temporal correlations in hong-ou-mandel interference.  Optics Express, 29(18):28217, August 2021.
  * [12] Liudmila A. Zhukas, Maverick J. Millican, Peter Svihra, Andrei Nomerotski, and Boris B. Blinov.  Direct observation of ion micromotion in a linear paul trap.  Physical Review A, 103(2), February 2021.
  * [13] M. Fisher-Levine and A. Nomerotski.  Timepixcam: a fast optical imager with time-stamping.  Journal of Instrumentation, 11(03):C03016, 2016.
  * [14] A. Zhao et al.  Coincidence velocity map imaging using tpx3cam, a time stamping optical camera with 1.5 ns timing resolution.  Review of Scientific Instruments, 88(9):113104, November 2017.
  * [15] A. Nomerotski, M. Chekhlov, D. Dolzhenko, R. Glazenborg, B. Farella, M. Keach, R. Mahon, D. Orlov, and P. Svihra.  Intensified tpx3cam, a fast data-driven optical camera with nanosecond timing resolution for single photon detection in quantum applications.  Journal of Instrumentation, 18(01):C01023, January 2023.
  * [16] www.amscins.com.
  * [17] A. Nomerotski, I. Chakaberia, M. Fisher-Levine, Z. Janoska, P. Takacs, and T. Tsang.  Characterization of timepixcam, a fast imager for the time-stamping of optical photons.  Journal of Instrumentation, 12(01):C01017, 2017.
  * [18] T. Poikela et al.  Timepix3: a 65k channel hybrid pixel readout chip with simultaneous toa/tot and sparse readout.  Journal of instrumentation, 9(05):C05013, 2014.
  * [19] B. van der Heijden, J. Visser, M. van Beuzekom, H. Boterenbrood, S. Kulis, B. Munneke, and F. Schreuder.  Spidr, a general-purpose readout system for pixel asics.  Journal of instrumentation, 12(02):C02040, 2017.
  * [20] https://www.photonis.com/product/hi-qe photocathodes.
  * [21] B. Winter, S. J. King, M. Brouard, and C. Vallance.  A fast microchannel plate-scintillator detector for velocity map imaging and imaging mass spectrometry.  Review of Scientific Instruments, 85(2):023306, February 2014.
  * [22] D.A. Orlov, J. DeFazio, S. Duarte Pinto, R. Glazenborg, and E. Kernen.  High quantum efficiency s-20 photocathodes in photon counting detectors.  Journal of Instrumentation, 11(04):C04015–C04015, April 2016.
  * [23] D.A. Orlov, T. Ruardij, S. Duarte Pinto, R. Glazenborg, and E. Kernen.  High collection efficiency MCPs for photon counting detectors.  Journal of Instrumentation, 13(01):C01047–C01047, jan 2018.
  * [24] Christopher Ianzano, Peter Svihra, Mael Flament, Andrew Hardy, Guodong Cui, Andrei Nomerotski, and Eden Figueroa.  Fast camera spatial characterization of photonic polarization entanglement.  Scientific Reports, 10(1), Apr 2020.
  * [25] Yingwen Zhang, Duncan England, Andrei Nomerotski, Peter Svihra, Steven Ferrante, Paul Hockett, and Benjamin Sussman.  Multidimensional quantum-enhanced target detection via spectrotemporal-correlation measurements.  Phys. Rev. A, 101:053808, May 2020.
  * [26] P. Svihra, Y. Zhang, P. Hockett, S. Ferrante, B. Sussman, D. England, and A. Nomerotski.  Multivariate discrimination in quantum target detection.  arXiv:2005.00612, May 2020.
  * [27] A. Nomerotski, M. Keach, P. Stankus, P. Svihra, and S. Vintskevich.  Counting of hong-ou-mandel bunched optical photons using a fast pixel camera.  arXiv:2005.07982, May 2020.
  * [28] Xiaoqin Gao, Yingwen Zhang, Alessio D’Errico, Khabat Heshami, and Ebrahim Karimi.  High-speed imaging of spatiotemporal correlations in hong-ou-mandel interference.  Optics Express, 30(11):19456, May 2022.
  * [29] Liudmila A. Zhukas, Peter Svihra, Andrei Nomerotski, and Boris B. Blinov.  High-fidelity simultaneous detection of a trapped-ion qubit register.  Physical Review A, 103(6), June 2021.
  * [30] Alexander Kato, Apurva Goel, Raymond Lee, Zeyu Ye, Samip Karki, Jian Jun Liu, Andrei Nomerotski, and Boris B. Blinov.  Two-tone doppler cooling of radial two-dimensional crystals in a radio-frequency ion trap.  Physical Review A, 105(2), February 2022.
  * [31] Alexander Kato, Andrei Nomerotski, and Boris B. Blinov.  Micromotion-synchronized pulsed doppler cooling of trapped ions.  Physical Review A, 107(2), February 2023.
  * [32] G. D'Amen, M. Keach, A. Nomerotski, P. Svihra, and A. Tricoli.  Novel imaging technique for $\alpha$-particles using a fast optical camera.  Journal of Instrumentation, 16(02):P02006–P02006, February 2021\.
  * [33] A. S. Losko, Y. Han, B. Schillinger, A. Tartaglione, M. Morgano, M. Strobl, J. Long, A. S. Tremsin, and M. Schulz.  New perspectives for neutron imaging through advanced event-mode data acquisition.  Scientific Reports, 11(1), 2021.
  * [34] Jianqing Yang, Jianrong Zhou, Xingfen Jiang, Jinhao Tan, Lianjun Zhang, Jianjin Zhou, Xiaojuan Zhou, Wenqin Yang, Yuanguang Xia, Jie Chen, XinLi Sun, Quanhu Zhang, Jiang Li, Zhijia Sun, and Yuanbo Chen.  A novel energy resolved neutron imaging detector based on a time stamping optical camera for the CSNS.  Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 1000:165222, June 2021.
  * [35] Tianqi Gao, Gabriele D’Amen, Sergey Burdin, Mohammad Alsulimane, Paul Campbell, Cinzia Da Via, Andrei Nomerotski, Adam Roberts, Peter Svihra, Konstantinos Mavrokoridis, Jon Taylor, and Alessandro Tricoli.  Novel imaging technique for thermal neutrons using a fast optical camera.  Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 1046:167604, January 2023.
  * [36] A. Roberts, P. Svihra, A. Al-Refaie, H. Graafsma, J. Küpper, K. Majumdar, K. Mavrokoridis, A. Nomerotski, D. Pennicard, B. Philippou, S. Trippel, C. Touramanis, and J. Vann.  First demonstration of 3d optical readout of a TPC using a single photon sensitive timepix3 based camera.  Journal of Instrumentation, 14(06):P06001–P06001, June 2019.
  * [37] Adam Lowe, Krishanu Majumdar, Konstantinos Mavrokoridis, Barney Philippou, Adam Roberts, Christos Touramanis, and Jared Vann.  Optical readout of the ARIADNE LArTPC using a timepix3-based camera.  Instruments, 4(4):35, November 2020.
  * [38] Liisa M. Hirvonen, Merlin Fisher-Levine, Klaus Suhling, and Andrei Nomerotski.  Photon counting phosphorescence lifetime imaging with TimepixCam.  Review of Scientific Instruments, 88(1):013104, January 2017.
  * [39] Rajannya Sen, Liisa M. Hirvonen, Alexander Zhdanov, Peter Svihra, Stefan Andersson-Engels, Andrei Nomerotski, and Dmitri Papkovsky.  New luminescence lifetime macro-imager based on a tpx3cam optical camera.  Biomed. Opt. Express, 11(1):77–88, Jan 2020.
  * [40] Rajannya Sen, Alexander V. Zhdanov, Thomaz F. S. Bastiaanssen, Liisa M. Hirvonen, Peter Svihra, Patrick Fitzgerald, John F. Cryan, Stefan Andersson-Engels, Andrei Nomerotski, and Dmitri B. Papkovsky.  Mapping o2 concentration in ex-vivo tissue samples on a fast PLIM macro-imager.  Scientific Reports, 10(1), November 2020.
  * [41] D. Turecek, J. Jakubek, and P. Soukup.  USB 3.0 readout and time-walk correction method for timepix3 detector.  Journal of Instrumentation, 11(12):C12065–C12065, Dec 2016.
  * [42] D. A. Orlov, R. Glazenborg, R. Ortega, and E. Kernen.  Uv/visible high-sensitivity mcp-pmt single-photon ghz counting detector for long-range lidar instrumentations.  CEAS Space Journal, 11(4):405–411, Dec 2019.


# Nonvolatile Magneto-Thermal Switching in MgB2

Hiroto<EMAIL_ADDRESS>Yoshikazu Mizuguchi1 1Department of Physics,
Tokyo Metropolitan University, Hachioji, Tokyo, 192-0397, Japan

###### Abstract

Ongoing research explores thermal switching materials to control heat flow.
Specifically, there has been interest in magneto-thermal switching (MTS)
materials based on superconductors, which only exhibited switching behavior
when a magnetic field was applied. However, a recent report highlighted
nonvolatile MTS in commercial Sn-Pb solders, attributed to magnetic flux
trapping. In this study, we focused on flux trapping in a type-II
superconductor MgB2. Magnetization and thermal conductivity measurements under
magnetic fields were conducted on polycrystalline MgB2. We confirmed that
magnetic flux was indeed trapped in MgB2 even after demagnetization.
Additionally, we observed nonvolatile MTS in MgB2 as well as Sn-Pb solders.
These results suggest that the nonvolatile MTS may be a widespread
characteristic of superconducting materials with flux trapping.

The recent advancements in electronic device technology have spurred research
into thermal switching materials, which enable control of heat flow through
external parametersNianbei2012 ; Wehmeyer2017 . Recent progress has been made
in the development of thermal switching materials, where the control of
thermal conductivity ($\kappa$) is achieved through the application of
electricIhlefeld2015  and magnetic fieldsKimling2013 ; Hirata2023 . Among
these materials, superconductors have received particular attention in
magneto-thermal switching (MTS) research Yoshida_2023_Nb ; yoshida_2023_Pb .
Here, we introduce an index to assess the effectiveness of MTS known as the
MTS ratio (MTSR). The MTSR is calculated as the ratio of the change in
$\kappa$ between the presence and absence of a magnetic field. The MTSR is
expressed as [$\kappa(H)$ \- $\kappa$(0 Oe)] / $\kappa$(0 Oe). It is widely
recognized that, in the normal state, heat is carried by charge carriers,
whereas in the superconducting state, heat transport by Cooper pairs is
negligible. Consequently, the phase transition from the superconducting state
to the normal state results in an increase in $\kappa$. Recent studies
reported MTSR of 650 % for NbYoshida_2023_Nb  and over 1000 % for high purity
5N-Pbyoshida_2023_Pb . However, previously reported MTS using superconductors
had a limitation, $\kappa(H)$ returned to its initial value $\kappa$(0 Oe)
when the magnetic field was reduced to zero, indicating that MTS was effective
only in the presence of a magnetic field. In the most recent discovery
reported in arXiv: 2307.05957 (preprint)Arima_2023_dolder , a nonvolatile MTS,
which retains the altered $\kappa$($H$) even when the magnetic field is
completely removed, has been identified. Surprisingly, this nonvolatile MTS
material was discovered in commercially available Sn-Pb solders. The
nonvolatile MTSR is defined as [$\kappa$ (0 Oe, demagnetized) - $\kappa$(0 Oe,
initial)]/$\kappa$ (0 Oe, initial), and it has been determined that the
nonvolatile MTSR of flux-core-free Sn45-Pb55 solder was 150 %. The origin of
nonvolatile MTS in Sn-Pb solders is attributed to the presence of magnetic
flux trapped in the solder even after the applied magnetic field is removed,
resulting in a partial loss of superconducting bulkiness at $H=0$ Oe. While
magnetic flux trapping in Sn-Pb solders is relatively rare due to both Sn and
Pb being type-I superconductors, the magnetic flux trap after demagnetization
is commonly observed in type-II superconductor samples.

In this study, our primary focus is on exploring the occurrence of nonvolatile
MTS in type-II superconductors, with particular emphasis on MgB2, which has
been studied for its flux trapping propertiesMikheenko_2007 ; YAMAMOTO2006806
. MgB2 was discovered in 2001 and stands out among intermetallic
superconductors for having the highest superconducting transition temperature
$T_{\rm SC}\sim 39$ K under ambient pressure Nagamatsu2001 . This compound
exhibits a unique characteristic as a multi-gap superconductor, with multiple
conduction bands and independent superconducting gaps present on the Fermi
surfaceUchiyama2002 ; Tsuda2003 . Shortly after its discovery, it was observed
that grain boundaries in MgB2 could serve as effective pinning centers,
contributing to high critical current density ($J_{\rm c}$) in superconducting
materialsLarbalestier2001 ; Yamamoto_2005 ; KATSURA2007572 ; takano2001 .
Consequently, extensive research has been conducted to investigate the
relationship between magnetic flux trapping at grain boundaries and $J_{\rm
c}$.

Until now, the association between magnetic flux trapping and nonvolatile MTS
has solely been reported in Sn-Pb solders. To gain a deeper understanding of
this phenomenon, it is essential to explore other materials. MgB2 presents an
appealing platform for investigating nonvolatile MTS due to the existing body
of research on flux trapping effects at grain boundariesMikheenko_2007 . While
previous studies have conducted thermal conductivity measurements under
magnetic field on MgB2Wu_2004 ; Mucha_2003 , there has been no specific focus
on nonvolatile MTS. In this study, magnetization measurements and thermal
conductivity measurements under magnetic fields were conducted for commercial
MgB2. Notably, nonvolatile MTS was also observed in MgB2.

Polycrystalline MgB2 used in this experiment was a commercially available
powder sample (99%, KOJUNDO). Before the measurements, the powder sample
underwent a high-pressure sintering process. In this experiment, high-pressure
sintering was performed at relatively low temperatures to suppress grain
growth. The specific conditions for this high-pressure sintering entailed a
pressure of 3 GPa and a temperature of 400 ∘C, sustained around 30 minutes.
The crystal structure was examined through powder X-ray diffraction employing
the Cu-K$\alpha$ radiation using the $\theta$-2$\theta$ method (Miniflex-600
RIGAKU). The Rietveld refinement of the XRD data was performed using the
RIETAN-FP packageizumi2007 . The scanning electron microscope (SEM, TM3030,
Hitachi High-Tech) was used for microstructure observation. The thermal
conductivity was measured using a Physical Property Measurement System (PPMS,
Quantum Design) equipped with a thermal transport option (TTO). The
measurement employed a four-probe steady-state method, incorporating a heater,
two thermometers, and a base-temperature terminal. For the thermal
conductivity measurements of MgB2, a cylindrical sample with a diameter of
4.61 mm and a height of 4.10 mm was employed. The magnetization measurements
were carried out using a superconducting quantum interference device (SQUID)
magnetometry technique, employing the Magnetic Property Measurement System
(MPMS3, Quantum Design) in a VSM (vibrating sample magnetometry) mode. In this
experiment, thermal conductivity measurements were conducted on a high-
pressure sintered MgB2 sample within a week. Subsequently, the sample was
crushed, and further analyses including XRD and magnetization measurements,
and SEM imaging were performed. All the experiments were carried out using the
same batch of sample.

Figure 1 illustrates the XRD patterns obtained from the high-pressure sintered
MgB2 sample. In the high-pressure sintered sample, the presence of MgB4 and
MgO were detected as an impurity, alongside the main MgB2 peaks. The
reliability factor, denoted as $R_{\rm wp}$, was determined to be $R_{\rm wp}$
= 3.7 %, and the goodness-of-fit indicator, represented by $S$, was calculated
as $S$ = 1.8. The results of Rietveld refinement indicated that the sample
composition consisted of approximately 90 % MgB2, 5 % MgB4, and 5% MgO. The
as-purchased MgB2 powder contained a similar amount of MgB4 and MgO. The
discrepancy with the nominal purity of 99% MgB2 is likely a result of certain
compounds not being accounted for in the chemical analysis. Furthermore, the
XRD profile exhibited broadening, implying lattice strain induced by the high-
pressure sintering process.

Figure 2 shows the SEM image of the high-pressure sintered MgB2. Numerous
granular grains were observed in the structure of the high-pressure sintered
MgB2, with the majority of the grain sizes measuring less than approximately 5
$\mu$m.

Figure 3 (a) illustrates the temperature dependence of the magnetization 4$\pi
M$ measured at 10 Oe under both zero-field-cooling (ZFC) and field-cooling
(FC) conditions. The magnetization measurement under ZFC demonstrates a large
shielding signal below $T_{\rm SC}\sim$ 39 K. The difference between ZFC and
FC measurements is a characteristic behavior commonly observed in type-II
superconductors. The temperature dependence of 4$\pi M$ exhibited broadening,
which has also been reported in previous studies on high-pressure sintered
MgB2takano2001 . The exact cause of this broadening is not yet clear, but the
inhomogeneity of the crystals likely plays a role, as suggested by the broad
profile observed in the XRD measurement. Figure 3 (b) depicts the temperature
dependence of 4$\pi M$ measured at 10 Oe after FC at three different fields :
1000 Oe, 10000 Oe, and 70000 Oe. In all cases, 4$\pi M$ exhibited
ferromagnetic-like behavior below $T_{\rm SC}$, similar to the findings of
previously reported hydrogen-rich superconductorsMinkov2023  and Sn-Pb
soldersArima_2023_dolder , implying the presence of trapped magnetic flux at
grain boundaries of MgB2. The value of magnetization at 1.8 K increased as the
field increased from 1000 Oe to 10000 Oe, but it did not change further with
the application of a higher magnetic field. This suggests that the amount of
trapped magnetic flux increases with the applied magnetic field, but there is
a threshold where the trapped magnetic flux saturates. To further discuss, we
show the 4$\pi M$-$H$ curves at 2.5 K and 4.0 K in Figs. 3(c) and 3(e),
respectively. These curves display the distinct shape commonly observed in
type-II superconductors, which signifies the presence of flux trapping in the
material. As depicted in Figures 3(d) and 3(f), the inner magnetic flux
density ($B$) given by $B=H+4\pi M$ near 0 Oe is displayed at 2.5 K and 4.0 K.
The results at 2.5 K and 4.0 K showed similarities: immediately after the
zero-field-cooling, the initial magnetic flux density of MgB2 was $B$ = 0.
However, upon applying a magnetic field to MgB2, $B$ did not return to its
initial value when the applied field reached $H$ = 0, due to the magnet flux
trapping. The magnetic flux density trapped at $H$ = 0 Oe was 500 G for both
temperatures.

Figure 4 (a) depicts the temperature dependence of $\kappa$ in both a zero
magnetic field and a magnetic field of 10000 Oe. In the absence of a magnetic
field, $\kappa$ decreased as the temperature decreased. The observed variation
in the slope of $\kappa$ at approximately 10 K was consistent with previous
measurements on polycrystalline MgB2SCHNEIDER20016 . Furthermore, $\kappa$ at
50 K in this experiment was approximately 3.5 W/Km, which aligns with the
order of magnitude reported in previous studies, where values ranged from 5
W/KmPODDAR2003191  to 9 W/KmSCHNEIDER20016 . It is noted that thermal
conductivity is a sensitive indicator of grain boundaries, and therefore, the
discrepancy with previous studies is attributed to the sample dependence. When
a magnetic field of 10000 Oe was applied, a similar trend in $\kappa$ was
observed, but the decrease in $\kappa$ was suppressed. This can be attributed
to the suppression of the superconducting state in MgB2 under the magnetic
field. Figures 4(b) and 4(c) illustrate the magnetic field dependence of
$\kappa$ at 2.5 K and 4 K, respectively. When the MgB2 was zero-field-cooled
to 2.5 K, the initial $\kappa$ in the absence of magnetic field was 6.9 mW/Km.
When a magnetic field was applied, $\kappa$ increased and reached a value of
14.0 mW/Km at 10000 Oe. As the magnetic field gradually decreased from 10000
Oe, $\kappa$ showed a decrease. However, the value at 0 Oe deviated from the
initial value, indicating nonvolatile MTS. Upon further reduction of the
magnetic field, a minimum value of $\kappa$ was observed, followed by an
increase in $\kappa$. Similar trends were observed when the magnetic field was
increased from -10000 Oe. As mentioned earlier, the presence of approximately
500 G of trapped magnetic flux in MgB2 after demagnetization partially
suppresses the superconducting state and prevented $\kappa$ from returning to
its initial value. The nonvolatile MTSR observed in MgB2 at 2.5 K in this
experiment was 18 %, which is smaller than to that of flux-core-free Sn45-Pb55
solderArima_2023_dolder . Furthermore, nonvolatile MTS was also observed at
4.0 K, although the nonvolatile MTSR decreased to that at 2.5 K, reaching 15
%.

The primary discovery of this study is the confirmation of nonvolatile MTS
occurring in the magnetic flux trapped at the grain boundaries of the type-II
superconductor MgB2. This finding diverges from prior research, which
predominantly focused on composites such as Sn-Pb solders. Notably, the
phenomenon of flux trapping at grain boundaries has been observed not only in
MgB2 but also in other type-II superconductors, including cuprate
superconductors and iron-based superconductors Hilgenkamp2002 . This suggests
that the trapping of flux at grain boundaries is a widespread occurrence in
various types of type-II superconducting materials. In this study, the maximum
value of the nonvolatile MTSR achieved for MgB2 remained relatively small at
18 % at 2.5 K. To further enhance the nonvolatile MTSR, potential methods
include controlling the grain boundary size to increase the trapped magnetic
flux and regulating the thermal conductivity in the normal conducting region.
However, further systematic investigations are required in this regard. Recent
advancements in machine learning have contributed to the elucidation of heat
conduction mechanisms in grain boundaries and nanopolycrystals Fujii2020 .
Given that nonvolatile MTS is a relatively new phenomenon, it is crucial to
not only investigate the thermal conductivity under magnetic field in various
materials but also consider theoretical approaches that utilize machine
learning to gain a deeper understanding of nonvolatile MTS.

The motivation for this study was derived from the discovery of nonvolatile
MTS induced by magnetic flux trapping in Sn-Pb solders. Drawing inspiration
from this phenomenon, our research focused on investigating the magnetic field
dependence of thermal conductivity in type-II superconductor MgB2, a material
renowned for its ability to trap magnetic flux at grain boundaries. Through
our experiments, we successfully observed nonvolatile MTS in MgB2 and
identified magnetic flux trapping as the underlying mechanism. Moving forward,
it is imperative to extend this research to encompass other type-II
superconductors with effective pinning centers. Such endeavors will contribute
to a deeper understanding of nonvolatile MTS at a fundamental level and
facilitate improvements in both the nonvolatile MTSR and the operational
temperature range, thereby paving the way for potential engineering
applications.

## acknowledgment

We thank O. Miura and K. Uchida for supports in experiments and fruitful
discussion on the results. This work was partly supported by JST-ERATO
(JPMJER2201), TMU Research Project for Emergent Future Society, and Tokyo
Government-Advanced Research (H31-1).

## References

  * (1) N. Li, J. Ren, L. Wang, G. Zhang, P. Hänggi, and B. Li: Rev. Mod. Phys. 84 (2012) 1045.
  * (2) G. Wehmeyer, T. Yabuki, C. Monachon, J. Wu, and C. Dames: Appl. Phys. Rev. 4 (2017) 041304.
  * (3) J. F. Ihlefeld, B. M. Foley, D. A. Scrymgeour, J. R. Michael, B. B. McKenzie, D. L. Medlin, M. Wallace, S. Trolier-McKinstry, and P. E. Hopkins: Nano Lett 15 (2015) 1791.
  * (4) J. Kimling, J. Gooth, and K. Nielsch: Phys. Rev. B 87 (2013) 094409\.
  * (5) K. Hirata, K. Kuga, M. Matsunami, M. Zhu, J. P. Heremans, and T. Takeuchi: AIP Advances 13 (2023) 015016.
  * (6) M. Yoshida, M. R. Kasem, A. Yamashita, K. Uchida, and Y. Mizuguchi: Appl. Phys. Express 16 (2023) 033002.
  * (7) M. Yoshida, H. Arima, A. Yamashita, K. Uchida, and Y. Mizuguchi: arXiv:2305.08332 (2023).
  * (8) H. Arima, M. R. Kasem, H. Sepehri-Amin, F. Ando, K. Uchida, and Y. Mizuguchi: arXiv:2307.05957 (2023).
  * (9) P. Mikheenko, E. Martínez, A. Bevan, J. S. Abell, and J. L. MacManus-Driscoll: Supercond Sci Technol 20 (2007) S264.
  * (10) A. Yamamoto, J. Shimoyama, S. Ueda, Y. Katsura, I. Iwayama, S. Horii, and K. Kishio: Physica C: Superconductivity and its Applications 445-448 (2006) 806.
  * (11) J. Nagamatsu, N. Nakagawa, T. Muranaka, Y. Zenitani, and J. Akimitsu: Nature 410 (2001) 63.
  * (12) H. Uchiyama, K. M. Shen, S. Lee, A. Damascelli, D. H. Lu, D. L. Feng, Z.-X. Shen, and S. Tajima: Phys. Rev. Lett. 88 (2002) 157002.
  * (13) S. Tsuda, T. Yokoya, Y. Takano, H. Kito, A. Matsushita, F. Yin, J. Itoh, H. Harima, and S. Shin: Phys. Rev. Lett. 91 (2003) 127001.
  * (14) D. C. Larbalestier, L. D. Cooley, M. O. Rikel, A. A. Polyanskii, J. Jiang, S. Patnaik, X. Y. Cai, D. M. Feldmann, A. Gurevich, A. A. Squitieri, M. T. Naus, C. B. Eom, E. E. Hellstrom, R. J. Cava, K. A. Regan, N. Rogado, M. A. Hayward, T. He, J. S. Slusky, P. Khalifah, K. Inumaru, and M. Haas: Nature 410 (2001) 186.
  * (15) A. Yamamoto, J. Shimoyama, S. Ueda, Y. Katsura, S. Horii, and K. Kishio: Supercond Sci Technol 18 (2004) 116.
  * (16) Y. Katsura, A. Yamamoto, S. Ueda, I. Iwayama, S. Horii, J. ichi Shimoyama, and K. Kishio: Phys. C: Supercond. 460-462 (2007) 572.
  * (17) Y. Takano, H. Takeya, H. Fujii, H. Kumakura, T. Hatano, K. Togano, H. Kito, and H. Ihara: Appl. Phys. Lett. 78 (2001) 2914.
  * (18) B.-M. Wu, D.-S. Yang, W.-H. Zeng, S.-Y. Li, B. Li, R. Fan, X.-H. Chen, L.-Z. Cao, and M. Ausloos: Supercond Sci Technol 17 (2004) 1458.
  * (19) J. Mucha, M. Pekala, J. Szydłowska, W. Gadomski, J. Akimitsu, J.-F. Fagnard, P. Vanderbemden, R. Cloots, and M. Ausloos: Supercond Sci Technol 16 (2003) 1167.
  * (20) F. Izumi and K. Momma: Solid State Phenomena 130 (2007) 15.
  * (21) V. S. Minkov, V. Ksenofontov, S. L. Bud’ko, E. F. Talantsev, and M. I. Eremets: Nat. Phys. (2023).
  * (22) M. Schneider, D. Lipp, A. Gladun, P. Zahn, A. Handstein, G. Fuchs, S.-L. Drechsler, M. Richter, K.-H. Müller, and H. Rosner: Phys. C: Supercond. 363 (2001) 6.
  * (23) A. Poddar, B. Bandyopadhyay, P. Mandal, D. Bhattacharya, P. Choudhury, U. Sinha, and B. Ghosh: Phys. C: Supercond. 390 (2003) 191.
  * (24) H. Hilgenkamp and J. Mannhart: Rev. Mod. Phys. 74 (2002) 485.
  * (25) S. Fujii, T. Yokoi, C. A. J. Fisher, H. Moriwake, and M. Yoshiya: Nat. Commun. 11 (2020) 1854.

Figure 1: The observed patterns ($+$ symbols), calculated patterns (green
solid line), and the difference between them (blue solid line) for the high-
pressure sintered MgB2 at a temperature of $T$=293 K. These patterns were
obtained through Rietveld refinement analysis of the XRD data. The red, blue,
and green tricks represent the positions of possible Bragg reflections of
MgB2, MgB4, and MgO, respectively. The red, blue and, green numbers indicate
the Miller indices of MgB2, MgB4, and MgO, respectively. Figure 2: SEM
micrographs of high-pressure sintered MgB2 Figure 3: (a) Temperature
dependence of magnetization 4$\pi M$ for MgB2 under conditions of zero field
cooling (ZFC) and feild cooling (FC) at 10 Oe. (b) Temperature dependence of
4$\pi M$ for MgB2 measured at 10 Oe after field cooling under 1000 Oe (red),
10000 Oe (green) and 70000 Oe (blue). (c-f) 4$\pi M$-$H$ and $B$-$H$ curves
measured at 2.5 K and 4.0 K  Figure 4: (a) Temperature dependence of $\kappa$
measured at $H$ = 0 Oe and 10000 Oe. (b,c) $\kappa$-$H$ curves measured at $T$
= 2.5 K and 4.0 K.