# Entity Summarization: State of the Art and Future Challenges

Qingxia Liu<EMAIL_ADDRESS>Gong Cheng<EMAIL_ADDRESS>Kalpa
Gunaratna111Part of this work has been done while he was at Kno.e.sis Center,
Wright State University<EMAIL_ADDRESS>Yuzhong Qu<EMAIL_ADDRESS>State Key Laboratory for Novel Software Technology, Nanjing University, China
Samsung Research America, Mountain View CA, USA

###### Abstract

The increasing availability of semantic data has substantially enhanced Web
applications. Semantic data such as RDF data is commonly represented as
entity-property-value triples. The magnitude of semantic data, in particular
the large number of triples describing an entity, could overload users with
excessive amounts of information. This has motivated fruitful research on
automated generation of summaries for entity descriptions to satisfy users’
information needs efficiently and effectively. We focus on this prominent
topic of entity summarization, and our research objective is to present the
first comprehensive survey of entity summarization research. Rather than
separately reviewing each method, our contributions include (1) identifying
and classifying technical features of existing methods to form a high-level
overview, (2) identifying and classifying frameworks for combining multiple
technical features adopted by existing methods, (3) collecting known
benchmarks for intrinsic evaluation and efforts for extrinsic evaluation, and
(4) suggesting research directions for future work. By investigating the
literature, we synthesized two hierarchies of techniques. The first hierarchy
categories generic technical features into several perspectives: frequency and
centrality, informativeness, and diversity and coverage. In the second
hierarchy we present domain-specific and task-specific technical features,
including the use of domain knowledge, context awareness, and personalization.
Our review demonstrated that existing methods are mainly unsupervised and they
combine multiple technical features using various frameworks: random surfer
models, similarity-based grouping, MMR-like re-ranking, or combinatorial
optimization. We also found a few deep learning based methods in recent
research. Current evaluation results and our case study showed that the
problem of entity summarization is still far from being solved. Based on the
limitations of existing methods revealed in the review, we identified several
future directions: the use of semantics, human factors, machine and deep
learning, non-extractive methods, and interactive methods.

###### keywords:

Entity Summarization , Triple Ranking , Semantic Data

##  1 Introduction

Semantic data has been used to broadly refer to structured or semi-structured
data that allows machines to easily understand and manipulate the conveyed
information. It facilitates data integration and enables applications to
derive value from each other. Today, Web applications make information
publicly accessible not only as human-readable web pages but also as machine-
readable semantic data. For example, semantic data has been either embedded in
HTML web pages using markup formats like RDFa [1] or Microdata [2], or served
directly as dump files or Linked RDF Data [3]. Semantic data in the form of a
knowledge graph helps enterprises to drive products and make them more
intelligent [4].

Figure 1: A running example of semantic data.

Many of these different formats of semantic data essentially adopt a common
generalized data model. They describe entities with property-value pairs,
which collectively form _entity-property-value triples_ , or _triples_ for
short. As a running example for this article, Figure 1 shows a set of triples
about people in a research group. An entity has one or more types such as
Person, Professor, and Engineer. Properties are divided into three categories:
TYPE, attributes, and relations, according to their binding values.

  * 1.

TYPE has entity types (called classes) as values.

  * 2.

Attributes have primitive data values (called literals) as values, e.g., title
whose value is a string.

  * 3.

Relations have entities as values, e.g., worksAt whose value is an Institute
entity.

A set of triples can be represented as a directed graph where vertices
represent entities annotated with types and attributes, interconnected by arcs
representing relations. For example, Figure 2 depicts the triples in Figure 1.

Figure 2: A graph representation of our running example of semantic data.

The Web has witnessed an explosive growth of semantic data over the past few
years. As of September 2020, more than 86 billion triples had been embedded in
50% of HTML web pages and found in 15 million
domains222http://webdatacommons.org/structureddata/. As of May 2020, more than
one thousand datasets are available as Linked RDF Data, covering government,
life sciences, publications, social networking, and other
domains333https://lod-cloud.net/. Besides, knowledge graphs comprising tens of
billions of triples have been built in some of the largest technology
companies [4].

###  1.1 Motivation and Application for Entity Summarization

Semantic data, though primarily aiming to be consumed by machines, is
sometimes exposed to human users in a fairly plain format. For example, to
show an entity mentioned in a search query, Google Search retrieves from its
Knowledge Graph a set of triples describing the entity. These triples are
shown on the right-hand side of Google’s search results pages [5]. However,
the description of an entity may comprise dozens or hundreds of triples,
exceeding the capacity of a typical user interface—particularly on mobile
devices. A user, if being served with all of those triples, would suffer
information overload and find it difficult to quickly identify the small
subset of facts that are truly needed.

To address this problem, one promising solution is to not show the entire
entity description but _provide a short summary for the entity_. A carefully
generated summary, though possibly providing less and incomplete information,
has the potential to cost-effectively satisfy a user’s information need. For
example, instead of showing all the triples describing an entity in the
Knowledge Graph, Google provides “the best summary” for the entity by choosing
and presenting a subset of triples that the user is likely to be searching for
that particular entity [5]. This problem of automatically summarizing entity
descriptions has been termed _entity summarization_ by the research community.

In addition to search engines, entity summarization supports a multiplicity of
other applications [6, 7]. For example, entity summarization facilitates
document browsing. Some applications adopt entity linking techniques [8] to
enrich the content of a document (e.g., a news article) by linking entity
mentions in text with their corresponding entities in semantic data. Triples
describing these entities are then extracted from semantic data, to provide
additional information to the document and allow exploratory browsing options
that are relevant to the document content. Entity summarization helps to
control the number of triples presented to users at a manageable level and
provide the most useful triples [9, 10, 11].

Entity summarization has also assisted many research activities. For example,
in crowdsourced entity linking [12], human participants manually link entity
mentions in text to entities in semantic data. After seeing an entity mention,
a participant retrieves a set of candidate entities from semantic data. Each
of them is described by possibly a large set of triples. Entity summarization
retains the most distinctive triples, and helps the participant quickly
identify the correct entity [13]. Analogously, by identifying the most similar
triples, entity summarization has facilitated human intervention in entity
resolution to quickly determine whether two entity descriptions refer to the
same real-world object [14, 15, 16]. In these activities, human participants
complete their tasks more efficiently when they use summaries of entity
descriptions.

###  1.2 Problem Statement

We present a generalized definition of entity summarization that is compatible
with many existing research efforts. Our definition is independent of the
concrete processed data format and targeted application scenario.

##### Semantic Data

Let $\mathbb{E},\mathbb{P},\mathbb{C},\mathbb{L}$ be the sets of all entities,
properties, classes, and literals, respectively. Properties are divided into
$\texttt{TYPE}\in\mathbb{P}$, attributes $\mathbb{A}\subseteq\mathbb{P}$, and
relations $\mathbb{R}\subseteq\mathbb{P}$. Semantic data is a set of entity-
property-value triples denoted by
$T\subseteq\mathbb{E}\times\mathbb{P}\times(\mathbb{C}\cup\mathbb{L}\cup\mathbb{E})$,
or more precisely:

$T\subseteq(\mathbb{E}\times\\{\texttt{TYPE}\\}\times\mathbb{C})\cup(\mathbb{E}\times\mathbb{A}\times\mathbb{L})\cup(\mathbb{E}\times\mathbb{R}\times\mathbb{E})\,.$
(1)

For readers who are familiar with Semantic Web standards, an entity-property-
value triple can be represented as a subject-predicate-object triple in RDF
[17], which is a framework for representing information in the Web and has a
graph-based data model. TYPE, attributes, and relations correspond to
rdf:type, data properties, and object properties, respectively. However, we do
not use Semantic Web terms in this article because entity summarization is not
specific to semantic data that complies with Semantic Web standards.

Note that in RDF, property values can also be blank nodes which often
represent anonymous entities/classes or n-ary relations. A blank node is not a
self-contained value but relies on other triples describing it. Triples
involving blank nodes are difficult to handle and hence are usually ignored by
current research on entity summarization.

##### Entity Description

For convenience, let $\mathtt{ent}(t),\mathtt{prp}(t),\mathtt{val}(t)$ return
the entity, property, and value in a triple $t\in T$, respectively. In
semantic data $T$, the description of an entity $e\in\mathbb{E}$ consists of
the subset of triples where $e$ is described as an entity or as a property
value:

$\mathtt{Desc}(e)=\\{t\in T:\mathtt{ent}(t)=e\text{ or
}\mathtt{val}(t)=e\\}\,.$ (2)

For example, in Figure 1, the description of entity Qu comprises eight
triples, including $\langle$Liu, supervisor, Qu$\rangle$ where Qu is the
property value. For convenience, when the description of Qu is to be
summarized, we re-write the above triple as $\langle$Qu, supervisor-,
Liu$\rangle$ where supervisor- represents the inverse of supervisor, and Liu
becomes the property value. In this way, in $\mathtt{Desc}(e)$ which is to be
summarized, every triple has $e$ at the entity position. Summarization is then
focused on the property-value pair represented by each triple.

##### Entity Summary

Existing efforts to summarize entity descriptions are mainly extractive
solutions. They define a summary $S$ of entity $e$ as a size-constrained
subset of triples selected from the description of $e$. A size constraint is
usually an upper bound $k$ on the number of selected triples, i.e., a summary
$S\subseteq\mathtt{Desc}(e)$ satisfies $|S|\leq k$. For example, given $k=5$,
any 1–5 triples selected from the description of Qu in Figure 1 form a summary
of Qu.

##### Entity Summarization

The problem of entity summarization is formulated as finding an optimal
summary:

$\text{find
}\operatorname*{arg\,max}_{S\subseteq\mathtt{Desc}(e)}{\mathtt{score}(S|T)},\quad\text{
subject to }|S|\leq k\,,$ (3)

where $\mathtt{score}(S|T)$ is the quality score of summary $S$ given $T$.
Note that the score of $S$ is conditioned on $T$. Although we focus on
selecting triples from $\mathtt{Desc}(e)$, all the triples in $T$—including
those outside $\mathtt{Desc}(e)$—are considered as input because they are
often useful when assessing triples in $\mathtt{Desc}(e)$.

An algorithm or system for solving the problem of entity summarization is
called an entity summarizer. Different entity summarizers define and compute
$\mathtt{score}(S|T)$ in different ways. Some reduce Eq. (3) to a ranking
problem, e.g., [18, 19, 20]. They assume

$\mathtt{score}(S|T)=\sum_{t\in S}{\mathtt{score}(t|T)}\,,$ (4)

where $\mathtt{score}(t|T)$ is the quality score of triple $t\in S$. So the
problem turns into ranking the triples in $\mathtt{Desc}(e)$ and then
generating a summary by choosing $k$ top-ranked triples. Ranking is also a
major step in many other entity summarizers, e.g., [21, 22, 23, 24]. As entity
summarization is closely related to triple ranking, we will not particularly
make a distinction between summary-based methods and ranking-based methods.

###  1.3 Scope of Survey

The identification of entity summarizers for our survey was done according to
the following strategy.

First, we scanned the proceedings of a set of relevant conference series and
the volumes of some relevant journals starting from 2007, covering (Semantic)
Web, information retrieval, database and knowledge management:

  * 1.

conferences: WWW, WISE, APWeb, ISWC, ESWC, JIST, SIGIR, WSDM, ECIR, CIKM,
SIGMOD, PODS, PVLDB, ICDE, EDBT, and

  * 2.

journals: ACM Trans. Web, World Wide Web J., J. Web Semant., Semant. Web J.,
Int. J. Semant. Web Inf. Syst., ACM Trans. Inf. Syst., Inf. Process. Manag.,
Inf. Retr. J., Inf. Syst., IEEE Trans. Knowl. Data Eng., Data Knowl. Eng.,
Knowl. Inf. Syst., ACM Trans. Database Syst., VLDB J.

Second, we used the following query to search for papers in Google Scholar:

> “entity summarization” OR “entity summarisation” .

We chose relevant papers from the collected papers based on whether their
studied problem is compatible with our problem statement presented in Section
1.2. Furthermore, we followed citations and references to consider additional
papers that cite or are cited by the above relevant papers. With the defined
search strategy and selection criterion, we are confident that the risk of
introducing a researcher bias into the survey is low.

###  1.4 Related Problems and Surveys

There have been a few survey papers for related summarization problems
including document summarization, graph summarization, and ontology
summarization.

##### Document Summarization

Document summarization has been studied for decades [25, 26]. It is
fundamentally different from entity summarization. Triples in an entity
description are structured, whereas sentences in a document are unstructured
text. However, some techniques for document summarization have inspired the
development of entity summarizers. For example, RELIN [18] which is an early
entity summarizer uses a graph-based model which originates from the LexRank
method [27] for document summarization.

##### Graph Summarization

Semantic data can be represented as a graph. Graph summarization aims to
reveal patterns in the data. A graph summary is generally an abstract
representation of the original graph. For example, it can be a single super-
graph where super-vertices represent collections of vertices in the original
graph; or it can be a set of frequent subgraph patterns. In the graph
representation of semantic data, the description of an entity is represented
as the neighborhood of the entity. Interesting patterns can hardly be mined
from such a small star-shaped subgraph, and hence methods for graph
summarization [28, 29] are not suitable for entity summarization. Instead,
entity summarizers usually adopt extractive solutions.

##### Ontology Summarization

An ontology provides an explicit specification of a vocabulary for a shared
domain. It is often used as a schema of semantic data. Methods for ontology
summarization mainly represent an ontology as a graph, and then extract a
subset of top-ranked terms and/or axioms based on graph centrality [30, 31].
These methods are not suitable for entity summarization because their graph
representations are specifically designed for schema-level ontologies rather
than instance-level entity descriptions. However, graph centrality measures
are universal and may apply to the graph representation of semantic data.

###  1.5 Contribution of the Article

The far-reaching application of entity summarization has led to fruitful
research outcomes in recent years. However, there is a lack of comprehensive
literature survey on this research topic. To the best of our knowledge, we
provide the first technical review of existing entity summarizers. Instead of
separately describing each summarizer, we identify their technical features
and hierarchically organize the features to categorize the broad spectrum of
research on entity summarization. Then we investigate various ways of
combining multiple features to assemble a full entity summarizer. Our survey
also addresses recent efforts on using deep neural networks for entity
summarization, and covers known benchmarks and efforts for evaluation as well
as a case study. Based on the review, we suggest directions for future work to
overcome the limitations of existing research.

###  1.6 Structure of the Article

The rest of this article is organized as follows. Section 2 reviews technical
features for entity summarization. Section 3 reviews frameworks for feature
combination and describes a set of representative entity summarizers. Section
4 reviews recent deep learning based entity summarizers. Section 5 reviews
evaluation efforts for entity summarization. Finally, we suggest future
directions in Section 6.

##  2 Technical Features for Entity Summarization

We broadly divide the technical features used in existing entity summarizers
into generic features and specific features, and we organize them into
hierarchies that are presented in Figure 3 and Figure 4, respectively. Many of
these features can find their counterparts in document summarization [25, 26],
but they have been successfully adapted for semantic data. Table 1 lists
entity summarizers and the technical features they use.

Figure 3: A hierarchy of generic features for entity summarization. Figure 4:
A hierarchy of specific features for entity summarization. Table 1: Entity
Summarizers (Sorted by Publication Date) and Their Technical Features (Blanks:
Not Used)

| Generic Features |  Specific Features
---|---|---
|  Frequency and Centrality | Informativeness |  Diversity and Coverage (Similarity) |  Domains, Contexts, and Personalization
Falcons [32] |  |  |  bag-of-words |  query relevance
XRed [33] |  |  |  |  entity interdependence
Zhang et al. [34] |  weighted PageRank |  |  |  click-through data
RELIN [18] |  weighted PageRank | statistical |  |
Thalhammer et al. [19] |  | statistical |  |  movie domain
Yovisto [35] |  $\mathtt{tf}$ |  |  |  academic domain, Wikipedia links
MMR-QSFS [36] |  |  |  |  query relevance
DIVERSUM [37] |  $\mathtt{lf}$ |  |  discrete |
SUMMARUM [38] |  PageRank |  |  discrete |
FACES [21] |  $\mathtt{vf}_{\mathbb{T}}$ | statistical |  discrete, bag-of-words |
COMB [13] |  $\mathtt{lf}$ | statistical |  edit-distance-like, numerical, logical |  document relevance, entity interdependence
TimeMachine [39] |  |  |  |  timeline domain
C3D+P [15] |  $\mathtt{lf}$ | statistical |  edit-distance-like, numerical, logical |  entity interdependence
TRank++ [9] |  $\mathtt{vf}_{\mathbb{E}}$ | ontological |  |  document relevance
FACES-E [22] |  $\mathtt{vf}_{\mathbb{T}}$ | statistical |  discrete, bag-of-words |
CD [40] |  | statistical |  edit-distance-like, numerical, logical |
Li et al. [23] |  $\mathtt{gf}_{\mathbb{E}}$, $\mathtt{lf}$ |  |  |  movie domain
CES [20] |  weighted PageRank | statistical |  |  session relevance
LinkSUM [24] |  $\mathtt{gf}_{\mathbb{T}}$, $\mathtt{lf}$, PageRank |  |  discrete |  Wikipedia links
Aemoo [41] |  $\mathtt{tf}$ |  |  |
DynES [42] |  $\mathtt{gf}_{\mathbb{E}}$, $\mathtt{gf}_{\mathbb{T}}$, $\mathtt{ntf}$, $\mathtt{vf}_{\mathbb{E}}$, $\mathtt{vf}_{\mathbb{T}}$ |  |  |  query relevance
REMES [11] |  $\mathtt{vf}_{\mathbb{E}}$ | statistical |  bag-of-words, structural |  entity interdependence
Multi-EGS [43] |  $\mathtt{tf}$, $\mathtt{ntf}$, $\mathtt{vf}_{\mathbb{T}}$ |  |  edit-distance-like |  Wikipedia categories
ES-LDA [44] |  LDA |  |  |  Wikipedia categories
ES-LDAext [45] |  LDA |  |  |
CTab [16] |  $\mathtt{gf}_{\mathbb{E}}$ |  |  bag-of-words, numerical |  entity interdependence
BAFREC [46] |  $\mathtt{gf}_{\mathbb{T}}$, $\mathtt{vf}_{\mathbb{T}}$ | ontological |  bag-of-words |
KAFCA [47] |  FCA |  |  |
MPSUM [48] |  LDA |  |  discrete |  Wikipedia categories
Gottschalk et al. [49] |  |  |  |  timeline domain, Wikipedia links
VISION-KG [50] |  $\mathtt{gf}_{\mathbb{E}}$, $\mathtt{gf}_{\mathbb{T}}$, $\mathtt{ntf}$, $\mathtt{vf}_{\mathbb{E}}$, $\mathtt{vf}_{\mathbb{T}}$ |  |  structural |  query relevance

###  2.1 Generic Features

Generic features apply to a wide range of domains, tasks, and users. We
identify generic features from existing entity summarizers and group them into
three categories: frequency/centrality, informativeness, and
diversity/coverage.

####  2.1.1 Frequency and Centrality

To generate a summary, an entity summarizer usually needs to measure the
salience of each single triple for ranking. In the following we will first
show measures based on frequency and centrality. Frequency is a common measure
that has been widely used in document summarization [25, 26]. Words that are
frequently seen in a document are often topical words repeated by the author,
and thus are usually (though not always) believed to be important. Graph
centrality computes extended frequency and may be more effective and robust.

Frequency of Property. To score an entity-property-value triple for entity
summarization, [13, 15, 16, 23, 24, 35, 37, 41, 42, 43, 46, 50] calculate the
frequency of occurrence of the property over different scopes of data and
interpret frequency in different ways.

In [16, 23, 42, 50], the _global frequency_ of a property $p$ is calculated
over all the entity descriptions in a dataset:

$\mathtt{gf}_{\mathbb{E}}(p)=|\\{e^{\prime}\in\mathbb{E}:\exists
t\in\mathtt{Desc}(e^{\prime}),~{}\mathtt{prp}(t)=p\\}|\,.$ (5)

###### Example 1

In Figure 1, worksAt describes three entities (Qu, Cheng, and Hu) and hence
$\mathtt{gf}_{\mathbb{E}}(\texttt{worksAt})=3$; directs describes only one
entity (Qu) and hence $\mathtt{gf}_{\mathbb{E}}(\texttt{directs})=1$.

Another way of calculating global frequency is done at the triple level [24,
42, 46, 50]:

$\mathtt{gf}_{\mathbb{T}}(p)=|\\{t\in\mathbb{T}:\mathtt{prp}(t)=p\\}|\,.$ (6)

Its slight difference from $\mathtt{gf}_{\mathbb{E}}$ is that
$\mathtt{gf}_{\mathbb{T}}$ magnifies the frequency of multi-valued properties.

###### Example 2

In Figure 1, while
$\mathtt{gf}_{\mathbb{E}}(\texttt{knows})=\mathtt{gf}_{\mathbb{E}}(\texttt{worksAt})=3$,
knows is a multi-valued property and has a higher triple-level frequency
$\mathtt{gf}_{\mathbb{T}}(\texttt{knows})=5>\mathtt{gf}_{\mathbb{T}}(\texttt{worksAt})=3$.

In [35, 41, 43], frequency is calculated over only those entity descriptions
of the same type. Priority is given to _type-specific frequent properties_
rather than generically frequent properties describing various types of
entities. Formally, let $c\in\mathbb{C}$ be the type of the entity to be
summarized, and let $\mathtt{Inst}(c)\subseteq\mathbb{E}$ be the instances of
$c$, i.e., the set of entities having $c$ as their type. The type-specific
frequency of a property $p$ is

$\mathtt{tf}(p)=|\\{e^{\prime}\in\mathtt{Inst}(c):\exists
t\in\mathtt{Desc}(e^{\prime}),~{}\mathtt{prp}(t)=p\\}|\,.$ (7)

###### Example 3

In Figure 1, worksAt describes only one entity (Qu) of type Professor and
hence $\mathtt{tf}(\texttt{worksAt})=1$ w.r.t.
$\mathtt{Inst}(\texttt{Professor})=\\{\texttt{Qu}\\}$.

A related method is to calculate the number of types of entities that a
property $p$ describes [42, 43, 50]:

$\mathtt{ntf}(p)=|\\{c^{\prime}\in\mathbb{C}:\exists
e^{\prime}\in\mathtt{Inst}(c^{\prime}),~{}t\in\mathtt{Desc}(e^{\prime}),~{}\mathtt{prp}(t)=p\\}|\,.$
(8)

It calculates frequency _at the class level_.

###### Example 4

In Figure 1, worksAt describes two types of entities (Person and Professor)
and hence $\mathtt{ntf}(\texttt{worksAt})=2$.

In [13, 15, 23, 24, 37], the scope of calculation is narrowed to
$\mathtt{Desc}(e)$—the entity description to be summarized, to assess the
_local frequency_ of a property $p$:

$\mathtt{lf}(p)=|\\{t\in\mathtt{Desc}(e):\mathtt{prp}(t)=p\\}|\,.$ (9)

Again, (locally) multi-valued properties have higher frequency.

###### Example 5

Inside the description of Qu in Figure 1, knows describes Qu in two triples
and hence $\mathtt{lf}(\texttt{knows})=2$; worksAt appears in only one triple
and hence $\mathtt{lf}(\texttt{worksAt})=1$.

Whereas most of the above methods favor frequent properties, some prefer
_infrequent properties_ , leading to seemingly contradictory conclusions. For
example, in [23], both the most frequent and the most infrequent properties
are prioritized. Similarly, in [24], globally frequent but locally infrequent
properties are preferred because they show (global) popularity and (local)
exclusivity. Locally infrequent properties are also believed to be
discriminating in [13, 15], and hence are useful for comparing entities. This
contradicts the preference for locally frequent properties in [37]. In [43],
properties that have higher type-specific frequency (i.e., $\mathtt{tf}$) but
describe fewer types of entities (i.e., $\mathtt{ntf}$) are ranked higher,
which is conceptually similar to the TF-IDF scheme (short for term
frequency—inverse document frequency) used in document summarization [25, 26].
In Section 2.1.2 we will discuss infrequency from the angle of
informativeness.

Frequency of Property Value. In [42, 50], the score of a triple depends on the
frequency of occurrence of the property value $v$ over all the entity
descriptions in a dataset:

$\mathtt{vf}_{\mathbb{E}}(v)=|\\{e^{\prime}\in E:\exists
t\in\mathtt{Desc}(e^{\prime}),\mathtt{val}(t)=v\\}|\,.$ (10)

###### Example 6

In Figure 1, Jia appears as a property value describing three entities (Qu,
Cheng, and Hu) and hence $\mathtt{vf}_{\mathbb{E}}(\texttt{Jia})=3$; Zhang
describes two entities (Qu and Cheng) and hence
$\mathtt{vf}_{\mathbb{E}}(\texttt{Zhang})=2$.

A slightly different way of calculating frequency is done at the triple level
[21, 22, 42, 11, 43, 46, 50]:

$\mathtt{vf}_{\mathbb{T}}(v)=|\\{t\in\mathbb{T}:\mathtt{val}(t)=v\\}|\,.$ (11)

###### Example 7

In Figure 1, while
$\mathtt{vf}_{\mathbb{E}}(\texttt{Websoft})=\mathtt{vf}_{\mathbb{E}}(\texttt{Jia})=3$,
Websoft has a higher triple-level frequency
$\mathtt{vf}_{\mathbb{T}}(\texttt{Websoft})=4>\mathtt{vf}_{\mathbb{T}}(\texttt{Jia})=3$.

In particular, if the property is restricted to TYPE and the value is a class,
the frequency of this class amounts to the number of its instances. In [9],
this number is calculated over different scopes of entities, such as all the
entities in a dataset or all the similar/related entities.

Centrality of Property Value. In the graph representation of semantic data,
the frequency of a property value which is a vertex in the graph is called its
degree. Degree is one of the simplest _graph centrality_ measures, using only
the neighborhood structure of a vertex. More generally, graph centrality
assesses the extent to which a vertex is central in a graph. Among others, the
well-known PageRank algorithm [51] uses a model of a random surfer who
continuously walks in the graph, either moving from a vertex to a random
neighbor or jumping to a random vertex. The probability that the surfer is
located at a vertex after a sufficiently large number of steps is defined as
the PageRank centrality of the vertex. Compared with degree, PageRank is more
powerful as it exploits the whole structure of a graph. PageRank has been used
to score property values in entity summarization [24, 38].

In the original implementation of PageRank, the outgoing arcs of a vertex are
uniformly weighted, and the random surfer is equally likely to move to any
neighbor. However, arcs in semantic data can heterogeneously represent
different types of relations having different semantics.

###### Example 8

In Figure 2, the outgoing arcs of Qu represent three types of relations
(knows, worksAt, and directs).

The random surfer may be more likely to move along certain arcs. Taking this
into account, [34] allows the random surfer to choose different arcs with
different probabilities, which are defined in a domain-specific way.

Centrality of Triple. The weighted PageRank model is also adopted by [18],
which presents a new graph representation for entity descriptions inspired by
LexRank [27] for document summarization. The idea is to represent all the
triples in an entity description as vertices of a complete graph. Every pair
of vertices is connected by an arc weighted by the relatedness (i.e.,
similarity) of the two corresponding triples.

###### Example 9

In Figure 1, the description of Qu is represented by a complete graph
containing eight vertices, each representing a triple in the description.

A triple is directly scored as a single unit by running PageRank on this
graph, to assess its topical centrality. This weighting scheme is extended in
[20] to also consider topic coherence in a query context. Compared with the
separation of property scoring and value scoring, such a joint model seems
more suitable for the entity summarization task.

LDA. Going beyond frequency, recent research has started to adapt more
powerful statistical methods in information retrieval for entity
summarization. Among others, Latent Dirichlet Allocation (LDA) [52] is a
generative statistical model, assuming that each document is a mixture of a
small number of topics and each topic uses a small number of words frequently.
In [44, 45, 48] where LDA is used for entity summarization, properties are
treated as topics, and each property is a distribution over all the property
values. With an LDA model learned from data, properties and/or their values
are scored by their probabilities.

FCA. In [47], the Formal Concept Analysis (FCA) is performed to aggregate
properties and values into a hierarchy. A triple is scored based on the depth
of its elements in the hierarchy. The method implicitly gives preference to
triples with infrequent properties and frequent values. Here, frequency is
computed at the word level.

####  2.1.2 Informativeness

Another group of features for measuring the salience of a triple is
informativeness. This concept has been widely implemented in document
summarization [25, 26], where a word appearing in fewer documents carries more
information, and thus is more important. In the following we show two kinds of
informativeness measures used in existing entity summarizers.

Statistical Informativeness. A property-value pair $\langle p,v\rangle$
describing fewer entities in a dataset is considered more important [11, 13,
15, 18, 19, 20, 21, 22, 40]. This statistical method has _statistical and
information theoretic explanations_. We can treat the occurrence of $\langle
e,p,v\rangle$ in $\mathtt{Desc}(e)$ as a probabilistic event. The
informativeness of $\langle p,v\rangle$, i.e., the information content
associated with the event of its occurrence, is measured by self-information,
namely the negative logarithm of the probability of this event, where the
probability is estimated using relative frequency observed in a dataset:

$\mathtt{si}(\langle
p,v\rangle)=-\log\frac{|\\{e^{\prime}\in\mathbb{E}:\langle
e^{\prime},p,v\rangle\in\mathtt{Desc}(e^{\prime})\\}|}{|\mathbb{E}|}\,.$ (12)

A property-value pair that occurs less frequently will, once observed, carry a
larger amount of information. Therefore, the self-information of a property-
value pair shows its power of characterizing an entity.

###### Example 10

In Figure 1, $\langle\texttt{worksAt},~{}\texttt{Websoft}\rangle$ occurs in
three out of the eight entity descriptions in the data and hence
$\mathtt{si}(\langle\texttt{worksAt},~{}\texttt{Websoft}\rangle)=-\log\frac{3}{8}$;
$\langle\texttt{directs},~{}\texttt{Websoft}\rangle$ occurs in only one entity
description and hence
$\mathtt{si}(\langle\texttt{directs},~{}\texttt{Websoft}\rangle)=-\log\frac{1}{8}$
representing higher informativeness.

Note that frequency, infrequency, and informativeness not necessarily
conflict. They may be observed in different aspects of a property-value pair.
For example, it is possible that an informative property-value pair has a
locally infrequent property and has a globally frequent entity as the value.

###### Example 11

In Figure 1, $\langle\texttt{directs},~{}\texttt{Websoft}\rangle$ has high
statistical informativeness. The local frequency of the property
$\mathtt{lf}(\texttt{directs})=1$ is very low while the global frequency of
the value $\mathtt{vf}_{\mathbb{T}}(\texttt{Websoft})=4$ is relatively high.

Ontological Informativeness. If the property is restricted to TYPE and the
value is a class, there is another way of measuring informativeness based on
_ontological semantics_ in the schema of data, aka an ontology. Classes in an
ontology typically form a subsumption hierarchy.

###### Example 12

In Figure 1, Professor can be a sub-class of Person in the schema.

A class deeper in the hierarchy has a more specific meaning than an upper-
level one, and its occurrence carries more information [9, 46]. Not
surprisingly, such a class also has larger self-information because it has
fewer instances. However, in practice, statistical informativeness and
ontological informativeness are not equivalent due to the unbalance of the
class hierarchy (hurting the accuracy of ontological informativeness) and/or
the incompleteness of the data (hurting the accuracy of statistical
informativeness).

####  2.1.3 Diversity and Coverage

Simply choosing a subset of the most salient triples may not generate a good
summary because the information they provide may cover limited aspects of an
entity, and on the other hand overlap with each other. Improving the diversity
of a summary to more comprehensively cover the original information has been
considered in document summarization [25, 26] and also in entity
summarization. The core task here is to measure the similarity between
triples, and then avoid selecting very similar triples into a summary. In the
following, we show various measures of similarity between two triples.

Discrete Similarity. One simple method is to choose triples that have
different properties [37, 21, 22, 48] or different values [24, 38] for a
summary. This method essentially uses discrete similarity. For two properties
(or two values) $i$ and $j$, their similarity is binary:

$\mathtt{ds}(i,j)=\begin{cases}1&\text{if $i=j$,}\\\ 0&\text{if $i\neq
j$.}\end{cases}$ (13)

That is, different properties and different values are individually separate.
Implicit relationships between them are not explored.

Textual Similarity. Properties and values are not just symbols but associated
with natural language labels, from which textual similarity can be computed.

In [13, 15, 40, 43], _edit-distance-like string metrics_ are used to measure
the similarity between the names of two properties and the similarity between
the string forms of two property values.

###### Example 13

In Figure 1, Websoft and “Director of Websoft” are similar property values.

In [11, 16, 21, 22, 32, 46], a string (e.g., “Director of Websoft”) is
represented as _a bag of words_ (e.g., “director”, “of”, “websoft”), or a word
vector. The similarity between two triples can be the cosine similarity
between the two corresponding vectors [32]. To handle the sparseness of
vectors and glean sense-level or higher-level abstract meanings of words, [11,
16, 21, 22] use WordNet [53] to expand words by including their synonyms or
hypernyms.

Semantic Similarity. Going beyond the superficial discrete and textual
methods, semantic similarity aims to more deeply understand the meaning of
data.

In [13, 15, 16, 40], when comparing the string forms of two property values,
if both of them represent numbers, their string similarity will be substituted
by a special _numerical similarity_ which treats them as numbers instead of
ordinary strings. In [13, 15, 40], the similarity between two numbers $n_{i}$
and $n_{j}$ is computed as follows, ranging from -1 (dissimilar) to 1
(similar):

$\begin{split}\mathtt{ns}(n_{i},n_{j})=\begin{cases}1&\text{if
$n_{i}=n_{j}$,}\\\ -1&\text{if $n_{i}\neq n_{j}$ and $n_{i}n_{j}\leq 0$,}\\\
\frac{\min\\{|n_{i}|,|n_{j}|\\}}{\max\\{|n_{i}|,|n_{j}|\\}}&\text{if
$n_{i}\neq n_{j}$ and $n_{i}n_{j}>0$.}\end{cases}\end{split}$ (14)

###### Example 14

“99” and “100” comprise different characters, but their numerical similarity
$\mathtt{ns}(99,100)=\frac{99}{100}$ is reasonably large.

In [13, 15, 40], they exploit ontological semantics to identify redundant
triples via logical reasoning.

###### Example 15

In the description of Qu in Figure 1,
$\langle\texttt{Qu},~{}\texttt{TYPE},~{}\texttt{Person}\rangle$ and
$\langle\texttt{Qu},~{}\texttt{TYPE},~{}\texttt{Professor}\rangle$ are
redundant and will not be selected together into an entity summary because
Professor is a sub-class of Person.

Structural Similarity. By representing semantic data as a graph, similarity
can be computed based on the graph structure. Graph embedding techniques [54]
convert vertices into vectors in a low dimensional space where graph
structural information and graph features are maximally preserved.

In [11], the similarity between two property values is measured by the cosine
angle between their embedding vectors generated by the RDF2Vec model [55].

In [50], embedding vectors for property values are generated by the SSP model
[56]. For the set of property values in an entity summary, their embedding
vectors are projected by locality-sensitive hashing into segmented real values
where similar vectors will collide with high probability. Therefore, the
number of projected segments represents diversity.

###  2.2 Specific Features

All the aforementioned features assess the salience of triples only by
analyzing semantic data, thereby being universally usable. By contrast, the
following features exploit external resources or consider external factors.
They exhibit effectiveness for specific domains or specific tasks. These
specific features use domain knowledge, be informed of contexts, or realize
personalization. While some of these specific features are conceptually
generalizable, e.g., for many domains we can exploit certain domain-specific
knowledge, it would be labour-intensive to adapt existing concrete
implementation to other domains and tasks, e.g., identifying and collecting
useful knowledge for a given domain.

####  2.2.1 Domain Knowledge

To summarize entities in a specific domain, it is possible and sometimes
essential to obtain prior domain knowledge about the importance of certain
triples. For example, [35] summarizes the descriptions of academic videos, and
gives priority to triples involving Place or Event entities because these
classes are known to be important in the domain. In [23], to summarize the
description of a film, the triple describing its most important actor is
always selected into the summary. The importance of an actor is proportional
to the number of films the actor stars in. In [19], user ratings are leveraged
to identify similar films by collaborative filtering. To summarize the
description of a film, triples that also describe its most similar films are
considered important. In [39, 49], a timeline of events is generated for an
entity to show the most important milestones and relationships. During event
selection, date relevance and temporal diversity are considered to avoid
visual crowding along the time axis for easy user interaction.

In the description of an entity that has a Wikipedia page, a property value
whose Wikipedia page is backlinked to the above page is considered important
[24, 35, 49]. Wikipedia categories can be used to enrich semantic data for
LDA-based methods [44, 48]. It is also possible to group entities by their
Wikipedia categories from which an effective ranking of properties can be
derived [43].

####  2.2.2 Context Awareness

For the same set of triples, some entity summarizers can produce different
summaries depending on the context in which the generated summary is to be
used.

In entity search engines, a _keyword query_ forms the context, and the
relevance of a triple to the query is calculated for ranking and snippet
generation [32, 36, 42, 50].

###### Example 16

In Figure 1,
$\langle\texttt{Qu},~{}\texttt{directs},~{}\texttt{Websoft}\rangle$ is
relevant to the query “researchers at Websoft”.

This kind of context is extended from one single query to a _query session_ in
[20].

In summary-assisted Web browsing, the _content of a document_ surrounding the
mention of an entity forms the context. In [9, 13], preference is given to
property values whose types are the same as or similar to the types of the
other entities mentioned in the context.

In collective entity summarization [11, 13, 15, 16, 33], summaries of multiple
entity descriptions are jointly generated to help users compare or connect
these entities. When summarizing each of these entity descriptions, their
_interdependence_ forms the context. Their different values of the same
property and/or their similar property-value pairs are prioritized.

###### Example 17

In Figure 1, when jointly summarizing the descriptions of Qu and Cheng,
$\langle\texttt{Qu},~{}\texttt{worksAt},~{}\texttt{Websoft}\rangle$ and
$\langle\texttt{Cheng},~{}\texttt{worksAt},~{}\texttt{Websoft}\rangle$ are
likely to be selected because they provide a connection between the two
entities.

####  2.2.3 Personalization

It is useful to generate personalized entity summaries to meet users’
individual needs. For example, [34] infers a user’s preference from his or her
_clicks_ when interacting with an application. More clicks on triples about a
particular property indicate that the property is more important.

##  3 Frameworks for Feature Combination and Representative Entity Summarizers

A practical entity summarizer often relies on multiple features. However,
different features may have conflicting objectives. In this section, we sketch
out representative entity summarizers and show how they choose and combine
multiple features.

###  3.1 Simple Frameworks

Li et al. To integrate two or more features, one intuitive strategy is to
separately rank triples based on each individual feature, and then take the
union of their top-ranked results. For example, Li et al. [23] compute the
degree of specificness and the degree of generality for each property based on
its local frequency ($\mathtt{lf}$) and global frequency
($\mathtt{gf}_{\mathbb{E}}$), respectively. A generated entity summary
consists of $z$ most specific properties and $k-z$ most general properties,
where $0\leq z\leq k$ is a parameter to tune.

###### Example 18

In the description of Qu in Figure 1, each triple is separately ranked by the
specificness of its property which is proportional to $\mathtt{lf}$, and by
the generality of its property which is proportional to
$\mathtt{gf}_{\mathbb{E}}$. Triples having the most specific properties such
as directs and those having the most general properties such as TYPE will be
selected into the summary.

LinkSUM. Alternatively, multiple ranking criteria can be combined using an
aggregate function, e.g., summation [24, 50], multiplication [19, 24], or
their mixture [43]. For example, in LinkSUM [24], properties and property
values are scored separately. The score of a property $p$ is given by the
product of its three features:

$\mathtt{FRQ}(p)*\mathtt{EXC}(p)*\mathtt{DSC}(p)\,,$ (15)

where $\mathtt{FRQ}$ (short for frequency) represents global frequency
($\mathtt{gf}_{\mathbb{T}}$), $\mathtt{EXC}$ (short for exclusivity)
represents the reciprocal of local frequency ($\mathtt{lf}$), and
$\mathtt{DSC}$ (short for description) represents the number of annotations of
$p$, including the labels, domains, and ranges of $p$. The score of a property
value $v$ is a linear combination of its normalized PageRank score and
backlink ($\mathtt{BL}$) score:

$\alpha\cdot\frac{\mathtt{PageRank}(v)}{\max_{v^{\prime}\in\mathbb{E}}{\mathtt{PageRank}(v^{\prime})}}+(1-\alpha)\cdot\mathtt{BL}(v)\,,$
(16)

where $\alpha\in[0,1]$ is a parameter to tune, and $\mathtt{BL}(v)=1$ if $v$
is backlinked to the entity to be summarized, otherwise $\mathtt{BL}(v)=0$.
LinkSUM selects property values having the highest scores. When a property
value appears in more than one triple, the property having the highest score
is selected.

###### Example 19

In the description of Qu in Figure 1, for each triple its property and its
value are scored by Eq. (15) and Eq. (16), respectively. Triples are ranked by
the scores of their property values. Ties are broken by the scores of the
properties; for each unique value (e.g., Websoft), only one triple having the
top-ranked property (e.g., worksAt) is preserved.

Remarks. Despite simplicity, the above frameworks are not suitable for
incorporating diversity and coverage features discussed in Section 2.1.3.

###  3.2 Random Surfer Model

RELIN. In RELIN [18], a weighted PageRank model is used to rank triples by
computing their PageRank centrality in a graph where vertices represent
triples to be ranked and are pairwise adjacent to each other. In PageRank, a
random surfer at each step either moves from a vertex (i.e., a triple) to a
random neighbor or jumps to a random vertex. The two types of actions can have
different probabilities, which are exploited in RELIN to represent
informativeness and relatedness. Specifically, the probability that a random
surfer jumps to a random triple $t$, denoted by $p_{J}(t)$, is proportional to
the statistical informativeness ($\mathtt{si}$) of $t$:

$p_{J}(t)=\mathtt{si}(\langle\mathtt{prp}(t),\mathtt{val}(t)\rangle)\,.$ (17)

The probability that a random surfer moves from a triple $t^{\prime}$ to a
random neighbor $t$, denoted by $p_{M}(t^{\prime},t)$, is proportional to the
topical relatedness between $t^{\prime}$ and $t$. Topical relatedness is
computed using pointwise mutual information (PMI), a measure of association
used in information theory:

$p_{M}(t^{\prime},t)=\sqrt{\mathtt{PMI}(\mathtt{prp}(t^{\prime}),\mathtt{prp}(t))\cdot\mathtt{PMI}(\mathtt{val}(t^{\prime}),\mathtt{val}(t))}\,.$
(18)

To compute $\mathtt{PMI}$, for two properties their names are used, and for
two property values their string forms are used, denoted by $s^{\prime}$ and
$s$. RELIN employs a Web search engine to estimate probabilities in PMI
computation:

$\mathtt{PMI}(s^{\prime},s)=\frac{\frac{\mathtt{hits}(s^{\prime};s)}{N}}{\frac{\mathtt{hits}(s^{\prime})}{N}\cdot\frac{\mathtt{hits}(s)}{N}}\,,$
(19)

where $\mathtt{hits}$ represents the number of results returned by the Web
search engine for a query, and $N$ is a large integer for normalization.
Finally, the PageRank centrality of triple $t$, i.e., the probability that a
random surfer arrives at $t$, is iteratively defined as

$\mathtt{pr}(t)=(1-d)\cdot
p_{J}(t)+d\cdot\sum_{t^{\prime}\in\mathtt{Nbr}(t)}{\mathtt{pr}(t^{\prime})\cdot
p_{M}(t^{\prime},t)}\,,$ (20)

where $d\in[0,1]$ is a damping factor, and $\mathtt{Nbr}(t)$ denotes the
neighbors of $t$.

###### Example 20

In the description of Qu in Figure 1, the random surfer is more likely to jump
to triples such as
$\langle\texttt{Qu},~{}\texttt{directs},~{}\texttt{Websoft}\rangle$ which has
high statistical informativeness, and is also more likely to move to this
triple from some other triples such as
$\langle\texttt{Qu},~{}\texttt{worksAt},~{}\texttt{Websoft}\rangle$ due to the
high PMI between their (identical) property values.

CES. The above weighted PageRank model can integrate more than two ranking
features. The probability of a move or a jump can be defined as a linear
combination of multiple ranking functions. For example, CES [20] extends RELIN
by adding session relevance to the model. Session relevance is the relevance
of a triple to a query session. In CES, probability is a linear combination of
statistical informativeness, topical relatedness, and session relevance.

Remarks. PageRank and other random surfer models are suitable for integrating
centrality-based and importance-style features. However, one of their
shortcomings is that diversity and coverage features cannot be naturally
modeled.

###  3.3 Similarity-based Grouping

DIVERSUM. To explicitly support diversity and coverage, DIVERSUM [37]
disallows selected triples to have the same property. In other words, triples
are grouped by their properties. Only one triple in each group can be
selected. To rank triples in different groups, the local frequency
($\mathtt{lf}$) of their properties is compared.

###### Example 21

In the description of Qu in Figure 1, the triples about TYPE and knows are
prioritized due to their relatively high $\mathtt{lf}$. However, for each of
these properties, only one triple having this property will be selected into
the summary.

FACES(-E). A similar framework is adopted by FACES [21] and its extension
FACES-E [22]. They represent a triple as a bag of words expanded with
hypernyms in WordNet, and group triples by an incremental and hierarchical
text clustering algorithm. The incremental nature of the approach enables it
to work even in a streaming data environment where the total number of data
points (i.e., triples) are not known a priori. Within each group, a triple $t$
is scored by the product of its statistical informativeness ($\mathtt{si}$)
and the frequency of its property value ($\mathtt{vf}_{\mathbb{T}}$):

$\mathtt{si}(\langle\mathtt{prp}(t),\mathtt{val}(t)\rangle)\cdot\mathtt{vf}_{\mathbb{T}}(\mathtt{val}(t))\,.$
(21)

If the number of groups is larger than the number of triples to select (i.e.,
$k$), at most one top-ranked triple will be selected from each group.
Otherwise, at least one top-ranked triple will be selected from each group. A
similar strategy is adopted by MPSUM [48].

###### Example 22

In the description of Qu in Figure 1, the three triples where “Websoft” is
mentioned may form a group where the top-ranked triple is
$\langle\texttt{Qu},~{}\texttt{directs},~{}\texttt{Websoft}\rangle$ because it
has the highest statistical informativeness and also the highest frequency of
property value.

Remarks. Although grouping-based frameworks support diversity and coverage
features, they assume that the similarity between triples is an equivalence
relation and induces a partition of triples. However, similarity is not
necessarily a binary function but is generally a numerical function. A strict
partitioning of triples may be inflexible and inaccurate.

###  3.4 MMR-like Re-Ranking

MMR-QSFS. MMR [57] (short for Maximal Marginal Relevance) is an information
retrieval framework that improves the diversity of the results by selecting
items iteratively. In each iteration, the item to select is the candidate that
maximizes its quality score and minimizes its similarity with the items
already selected in previous iterations. Candidates are re-ranked in each
iteration. MMR-QSFS [36] adopts this framework to generate diversified query-
relevant entity summaries. Specifically, let $S$ be the current entity summary
consisting of the triples selected in previous iterations. Initially $S$ is
empty. In each iteration, the MMR score of each candidate triple $t$ is given
by

$\mathtt{mmr}(t)=\lambda\cdot\mathtt{qr}(t)-(1-\lambda)\cdot\max_{t^{\prime}\in
S}{\mathtt{sim}(t,t^{\prime})}\,,$ (22)

where $\mathtt{qr}(t)$ is the relevance of $t$ to the query,
$\mathtt{sim}(t,t^{\prime})$ is the similarity between two triples $t$ and
$t^{\prime}$, and $\lambda\in[0,1]$ is a parameter to tune.

###### Example 23

In the description of Qu in Figure 1, assume
$\langle\texttt{Qu},~{}\texttt{worksAt},~{}\texttt{Websoft}\rangle$ is the
triple selected in the first iteration. Then for the next iteration, the MMR
score of the candidate triple
$\langle\texttt{Qu},~{}\texttt{directs},~{}\texttt{Websoft}\rangle$ will be
reduced because it is very similar to a selected triple.

Remarks. This framework achieves a trade-off between importance and diversity
when selecting triples. Compared with grouping, MMR is more flexible and fully
exploits the numerical values of similarity. More generally, this trade-off
can be reformulated as a linear combination of importance and diversity over
all the triples in a summary, as we will see in Section 3.5. From this point
of view—treating the linear combination as an objective function to optimize,
MMR is actually a greedy algorithm which may not produce an optimum solution.

###  3.5 Combinatorial Optimization

CD. To overcome the sub-optimality of MMR, it would be straightforward to
directly model entity summarization as a combinatorial optimization problem.
In CD [40], for each triple $t_{i}\in\mathtt{Desc}(e)$, let $x_{i}$ be a
binary variable representing whether $t_{i}$ is selected into the summary
($x_{i}=1$) or not ($x_{i}=0$). The problem is to

$\begin{split}\text{maximize
}&\sum_{i=1}^{|\mathtt{Desc}(e)|}{\sum_{j=i}^{|\mathtt{Desc}(e)|}{p_{ij}x_{i}x_{j}}}\\\
\text{subject to }&\sum_{i=1}^{|\mathtt{Desc}(e)|}{x_{i}}\leq k\,,\\\
&x_{i}\in\\{0,1\\}\text{ for }i=1,\ldots,|\mathtt{Desc}(e)|\,,\\\ \text{where
}&p_{ij}=\begin{cases}\gamma\cdot\mathtt{si}(\langle\mathtt{prp}(t_{i}),\mathtt{val}(t_{i})\rangle)&\text{if
$i=j$}\,,\\\ \delta\cdot(-\mathtt{sim}(t_{i},t_{j}))&\text{if $i\neq
j$}\,,\end{cases}\end{split}$ (23)

where $\mathtt{si}(\langle\mathtt{prp}(t_{i}),\mathtt{val}(t_{i})\rangle)$ is
the statistical informativeness of $t_{i}$, $\mathtt{sim}(t_{i},t_{j})$ is the
similarity between $t_{i}$ and $t_{j}$ which integrates string similarity,
numerical similarity, and logical reasoning, and $\gamma,\delta>0$ are
parameters to tune. The objective function is to maximize the statistical
informativeness of selected triples while minimizing their similarity to
improve diversity. This problem is an instance of the quadratic knapsack
problem, which has effective heuristic algorithms [58].

###### Example 24

In Figure 1, when summarizing the description of Qu, once
$\langle\texttt{Qu},~{}\texttt{directs},~{}\texttt{Websoft}\rangle$ has been
greedily selected into a summary due to its high statistical informativeness,
it will be unlikely to select
$\langle\texttt{Qu},~{}\texttt{worksAt},~{}\texttt{Websoft}\rangle$ into the
same summary because it is very similar to a selected triple.

C3D+P. The quadratic knapsack model is also adopted in C3D+P [15], where the
goal is to generate a summary for comparing a pair of entities. The objective
function considers not only statistical informativeness and intra-entity
triple similarity as in Eq. (23), but also inter-entity triple similarity and
dissimilarity for comparing entities.

###### Example 25

In Figure 1, when jointly summarizing the descriptions of Qu and Cheng for
comparing them, their common property-value
$\langle\texttt{knows},~{}\texttt{Zhang}\rangle$ will be selected into their
summaries because it informatively characterizes a commonality between them.

REMES. An extended quadratic multidimensional knapsack model is used in REMES
[11], where the goal is to generate a summary to characterize the relatedness
among a set of entities $E$. For each entity $e_{i}\in E$ and each triple
$t_{a}\in\mathtt{Desc}(e_{i})$, let $x_{i,a}$ be a binary variable
representing whether $t_{a}$ is selected into the summary of $e_{i}$
($x_{i,a}=1$) or not ($x_{i,a}=0$). The problem is to

$\begin{split}\text{maximize
}&\sum_{i=1}^{|E|}{\sum_{j=i}^{|E|}{\sum_{a=1}^{|\mathtt{Desc}(e_{i})|}{\sum_{b=1}^{|\mathtt{Desc}(e_{j})|}{p_{i,a,j,b}\cdot
x_{i,a}\cdot x_{j,b}}}}}\\\ \text{subject to
}&\sum_{a=1}^{|\mathtt{Desc}(e_{i})|}{x_{i,a}}\leq k\text{ for
}i=1,\ldots,|E|\,,\\\ &x_{i,a}\in\\{0,1\\}\text{ for }i=1,\ldots,|E|\text{ and
}a=1,\ldots,|\mathtt{Desc}(e_{i})|\,,\\\ \text{where
}&p_{i,a,j,b}=\begin{cases}\alpha\cdot\mathtt{si}(\langle\mathtt{prp}(t_{a}),\mathtt{val}(t_{a})\rangle)\cdot\mathtt{vf}_{\mathbb{T}}(\mathtt{val}(t_{a}))&\text{if
$i=j$ and $a=b$}\,,\\\ \beta\cdot(-\mathtt{rel}(t_{a},t_{b}))&\text{if $i=j$
and $a\neq b$}\,,\\\ \gamma\cdot\mathtt{rel}(t_{a},t_{b})&\text{if $i\neq
j$}\,,\end{cases}\end{split}$ (24)

where $\mathtt{si}(\langle\mathtt{prp}(t_{a}),\mathtt{val}(t_{a})\rangle)$ is
the statistical informativeness of $t_{a}$,
$\mathtt{vf}_{\mathbb{T}}(\mathtt{val}(t_{a}))$ is the frequency of the
property value of $t_{a}$, $\mathtt{rel}(t_{a},t_{b})$ is the relatedness
between $t_{a}$ and $t_{b}$, and $\alpha,\beta,\gamma>0$ are parameters to
tune. The objective function is to maximize the statistical informativeness
and property value frequency of selected triples, minimize the relatedness
between selected intra-entity triples to improve diversity, and maximize the
relatedness between selected inter-entity triples to characterize relatedness
between entities. Relatedness is computed using both the overlap between two
bags of words expanded with WordNet and the cosine angle between two graph
embedding vectors.

###### Example 26

In Figure 1, when jointly summarizing the descriptions of Qu, Cheng, and Hu
for showing their relatedness, their common property-value
$\langle\texttt{worksAt},~{}\texttt{Websoft}\rangle$ will be selected into
their summaries.

COMB. The quadratic multidimensional knapsack model is also adopted in COMB
[13], where the goal is to generate a summary for distinguishing between a set
of candidate entities that may be mentioned in a document. The objective
function considers not only statistical informativeness and triple
dissimilarity but also the relevance of these entities to the document
content. Entities and the document content are all represented as vectors in a
space where each dimension represents a class. Weights in vectors are computed
in a way similar to the well-known term frequency—inverse document frequency
in information retrieval. The weight of a class is proportional to its number
of instances in the document but inversely proportional to its total number of
instances.

Remarks. In addition to knapsack-like modeling, [16, 33, 39] develop
formulations of task-specific combinatorial optimization problems. Compared
with MMR, combinatorial optimization provides a more principled way of
formulating the problem of entity summarization. However, their difference is
mainly theoretical rather than practical, because the formulated combinatorial
optimization problems are often NP-hard and are solved by heuristic or greedy
algorithms, which are conceptually similar to MMR.

###  3.6 Learning to Rank

From the view of machine learning, all the above frameworks are unsupervised.
When labeled data is available for training, it is possible to consider a
supervised learning framework. For example, for each triple we can define a
feature vector where each feature is given by one of the above-mentioned
technical features. Entity summarization is then modeled as a learning-to-rank
problem, which can be solved by decision tree and linear regression [9],
support vector machine [49], gradient tree boosting [36, 42], or more
sophisticated learners.

Supervised learning has so far not been widely used for entity summarization
due to the shortage of labeled data. Existing attempts [9, 36, 42, 49] mainly
focus on specific applications where labeled data is available, such as user
clicks for the generation of query-relevant entity summaries. In Section 4, we
will detail two recent entity summarizers using deep neural networks.

##  4 Deep Neural Networks for Entity Summarization

The success of deep learning in tasks like speech recognition, image
classification, and natural language processing suggests a promising direction
for entity summarization research. Recently, a few efforts [59, 60] have been
made to apply deep neural networks to the entity summarization task. The idea
of these entity summarizers is to supervisedly learn a scoring function for
triple ranking, where the scoring function is modeled as a parameterized deep
neural network.

ESA. The first entity summarizer using deep neural networks is ESA [59]. In
the first step, each triple in $\mathtt{Desc}(e)$ is represented by a vector
concatenating the embedding vectors of its property and value. The embedding
vector of the property is learned using a word embedding technique [61]. The
embedding vector of the value is derived from a pre-trained TransE model [62]
which encodes structural features. In the second step, the vector
representations of all the triples in $\mathtt{Desc}(e)$ are fed as a sequence
into a BiLSTM network, which outputs a vector representation $\mathbf{h}$ for
$\mathtt{Desc}(e)$. Based on this representation, in the last step, a
supervised attention mechanism is used to score each triple in
$\mathtt{Desc}(e)$ by the dot product of its vector representation and
$\mathbf{h}$. These dot products generate the final attention vector after
using softmax function. The learning task here is to minimize the cross-
entropy loss between the generated attention vector and the attention vector
derived from labeled entity summaries.

DeepLENS. One shortcoming of ESA is, due to the use of BiLSTM for encoding a
set as a sequence, the output is sensitive to the chosen order. DeepLENS [60]
overcomes this problem with a different network. In the first step, both the
property and the value in each triple are represented by their pre-trained
word embedding vectors, which in turn are concatenated to represent the
triple. Different from ESA, graph structure is completely ignored in DeepLENS.
In the second step, each candidate triple $t\in\mathtt{Desc}(e)$ is encoded by
a multi-layer perceptron (MLP) which outputs a vector $\mathbf{h}$. All the
triples in $\mathtt{Desc}(e)$ are encoded by another MLP and then are summed
and weighted using attention from $\mathbf{h}$ to generate the vector
representation of $\mathtt{Desc}(e)$ denoted by $\mathbf{d}$. This
representation is permutation invariant to the order of the triples in
$\mathtt{Desc}(e)$. In the last step, $\mathbf{h}$ and $\mathbf{d}$ are
concatenated and fed into a MLP to generate the score of $t$. The learning
task here is to minimize the mean squared error loss between the generated
scores and the scores derived from labeled entity summaries.

Remarks. Compared with traditional entity summarizers reviewed in Section 3
which choose and combine multiple features, deep learning based entity
summarizers avoid manual feature engineering. However, they rely on a large
number of labeled entity summaries for training. Manually labeling entity
summaries is expensive.

##  5 Evaluation of Entity Summarization

Methods for evaluating entity summarization are broadly divided into intrinsic
methods and extrinsic methods.

Intrinsic evaluation directly measures the quality of a machine-generated
summary by comparing it with a human-made ground-truth summary. Intrinsic
evaluation is popular because it is relatively easy to perform and the results
are reproducible. In this section, we present metrics used in intrinsic
evaluation, describe known benchmarks offering ground-truth summaries created
for evaluation, and illustrate some evaluation results.

Extrinsic evaluation indirectly measures the quality of a machine-generated
summary by applying it in a downstream task and measuring users’ effectiveness
and efficiency in completing the task based on the summary. Extrinsic methods
are usually adopted to evaluate task-specific entity summarizers. In this
section we will review extrinsic evaluation reported in the literature.
However, extrinsic evaluation is difficult to replicate as human users are
involved.

###  5.1 Intrinsic Evaluation

####  5.1.1 Evaluation Metrics

Let $S_{m}$ be a machine-generated entity summary. Let $S_{h}$ be a human-made
ground-truth summary. Intrinsic evaluation compares $S_{m}$ with $S_{h}$ and
scores $S_{m}$ based on the extent to which $S_{m}$ is similar to $S_{h}$.
When there are multiple ground-truth summaries made by different human experts
for an entity, the mean score of $S_{m}$ is calculated.

Two similarity metrics have been used in the literature: quality and F1.

Quality Score. Entity summarizers take a size constraint $k$ as input, to
bound the number of triples in a summary. In intrinsic evaluation, $k$ is
commonly set to $|S_{h}|$, and hence we usually have $|S_{m}|=|S_{h}|$. In
[18, 20, 21, 22, 24], the quality score of $S_{m}$ is computed based on its
overlap with $S_{h}$:

$\text{Quality}=|S_{m}\cap S_{h}|\,.$ (25)

One shortcoming of this simple metric is the lack of normalization. Therefore,
the results in different settings of $k$ are not comparable. Besides, this
metric may not be fair when $|S_{m}|\neq|S_{h}|$.

F1 Score. To overcome the above shortcomings, recent evaluation efforts use
standard information retrieval metrics:

$\text{Precision}=\frac{|S_{m}\cap
S_{h}|}{|S_{m}|}\,,\quad\text{Recall}=\frac{|S_{m}\cap
S_{h}|}{|S_{h}|}\,,\quad\text{F1}=\frac{2\cdot\text{Precision}\cdot\text{Recall}}{\text{Precision}+\text{Recall}}\,.$
(26)

The results of Precision, Recall, and F1 score are in the range of $[0,1]$.

Note that we will trivially have Precision$=$Recall$=$F1 if $|S_{m}|=|S_{h}|$.
However, even if we set $k=|S_{h}|$, some entity summarizers may output less
than $k$ triples, i.e., $|S_{m}|<|S_{h}|$. For example, DIVERSUM [37]
disallows selected triples to have the same property. It is possible that an
entity description contains less than $k$ distinct properties and hence
DIVERSUM has to output less than $k$ triples. In this case,
Precision$\neq$Recall, and one should rely on F1 score.

Ranking-based Metrics. As mentioned in Section 1.2, some methods formulate and
solve entity summarization as a triple ranking problem. Given an input entity
description $\mathtt{Desc}(e)$, they output a ranking of the triples in
$\mathtt{Desc}(e)$. To evaluate this ranking, we can treat the $k$ top-ranked
triples as a machine-generated summary and evaluate it using the above-
mentioned set-based metrics such as Precision. That amounts to calculating
Precision at $k$, which is a popular evaluation metric in information
retrieval. We can also use other information retrieval metrics such as Mean
Average Precision (MAP) and Normalized Discounted Cumulative Gain (NDCG) to
directly evaluate the entire ranking. We will not detail these metrics as they
have not been widely used in entity summarization research.

Table 2: Benchmarks for Intrinsic Evaluation of Entity Summarization

|  Entities |  Number of Entities |  Entity Types
---|---|---|---
Benchmark for Evaluating RELIN [18] |  DBpedia |  149 |  not specified
WhoKnows?Movies! [63]111http://yovisto.com/labs/iswc2012 |  Freebase |  60 |  movie
Benchmark for Evaluating DIVERSUM [37] |  IMDb |  20 |  actor
Langer et al. [64] |  DBpedia |  14 |  theoretical physicist, Grammy winner, male “best actor”, Academy Award winner, super heavyweight boxer, U.S. president, etc.
Benchmark for Evaluating FACES [21]222http://wiki.knoesis.org/index.php/FACES |  DBpedia |  50 |  politician, actor, scientist, song, film, country, city, river, company, game, etc.
FRanCo [65] |  DBpedia |  265 |  from 189 classes
Benchmark for Evaluating FACES-E [22]222http://wiki.knoesis.org/index.php/FACES |  DBpedia |  80 |  not specified
ESBM v1.2 [66]333https://w3id.org/esbm |  DBpedia, LinkedMDB |  175 |  agent, event, location, species, work, film, person

####  5.1.2 Benchmarks

Table 2 presents known efforts in creating benchmarks offering human-made
ground-truth entity summaries or triple scores for intrinsic evaluation. All
these ground truths are generic, i.e., they are created not for any particular
task or any individual user, but for general purposes. At the time of writing
this article, only four of these benchmarks for intrinsic evaluation of entity
summarization are online accessible. Their hyperlinks are included in Table 2
as footnotes. Most of these efforts use entities in DBpedia which contains RDF
data extracted from Wikipedia. Some use LinkedMDB or IMDb, both of which
contain semantic data for the movie domain.

The authors of RELIN [18] invited 24 human experts to independently create
ground-truth summaries for 149 entities randomly selected from DBpedia. Each
entity description contained 20–40 triples and was summarized by an average of
4.43 participants. A participant created two summaries: one containing 5
selected triples and the other containing 10 triples.

The authors of FACES [21] and FACES-E [22] followed a similar procedure. To
evaluate FACES, 15 human experts were invited to independently create ground-
truth summaries for 50 entities in DBpedia. Each entity was described by at
least 17 distinct properties and an average of 44 triples, and was summarized
by at least 7 participants. To evaluate FACES-E, 17 human experts were invited
to summarize 80 entities. Each entity was summarized by at least 4
participants. These benchmarks are available online.

Langer et al. [64] and FRanCo [65] also used entities in DBpedia. Instead of
entity summaries, they collected graded importance score of each triple. In
[64], 10 human experts were invited to independently label the triples in 14
entity descriptions as highly relevant, relevant, less relevant, or
irrelevant. In [65], 265 entities were processed by at least 5 participants.
An entity description contained 10–150 triples with a mean value of 48. Each
participant selected the importance of each fact in an entity description on
the Likert scale, scoring it from 1 to 5.

Whereas all the above efforts used DBpedia, the authors of DIVERSUM [37]
selected 20 entities from IMDb. They did not collect ground-truth summaries
from human experts but extracted from the Wikipedia info-box concerning each
entity, which was edited by a large population of editors and could be viewed
as a high-quality entity summary.

The WhoKnows?Movies! game [63] was designed to collect ground-truth importance
of triples. This online quiz game attracted 217 players who answered 8,308
questions about 2,829 triples in the descriptions of 60 movies taken from
Freebase. A question was generated out of a triple asking for movies having a
given property-value pair, e.g., an actor. The ratio of correctly answered
questions based on a triple could be used as an indicator of the importance of
the triple. This benchmark is available online.

The Entity Summarization Benchmark (ESBM) v1.2 [66] used entities from DBpedia
and LinkedMDB. Ground-truth summaries for 175 entities were collected from 30
independent human experts. Each entity description contained at least 20
triples, and was summarized by 6 participants. A participant created two
summaries: one containing 5 selected triples and the other containing 10
triples. At the time of writing this article, ESBM v1.2 is the largest
available benchmark for intrinsic evaluation of entity summarization. It has
been used to evaluate a relatively large number of existing entity
summarizers.

####  5.1.3 Evaluation Results

Most benchmarks described in Section 5.1.2 have been used to evaluate none or
only a few entity summarizers. ESBM v1.2 is an exception, which was published
together with the evaluation results of nine unsupervised entity summarizers:
RELIN [18], DIVERSUM [37], FACES [21], FACES-E [22], CD [40], LinkSUM [24],
BAFREC [46], KAFCA [47], and MPSUM [48]. In [60], the evaluation results of
two deep learning based entity summarizers on ESBM v1.2 were reported: ESA
[59] and DeepLENS [60]. Here we will not repeat these detailed evaluation
results but we highlight the main conclusion: DeepLENS outperforms all the
competitors on ESBM v1.2 but still, there is a large gap between DeepLENS and
ORACLE which represents the best possible performance that can be achieved.
For example, when $k=5$, for DBpedia entities the mean F1 score of DeepLENS is
0.404 while ORACLE reaches 0.595. Note that the mean F1 score of ORACLE is
below 1 due to the existence of multiple different ground-truth summaries
which cannot simultaneously match a machine-generated summary. The large gaps
suggest much room for improving entity summarization techniques in future
research.

Figure 5: Ground-truth summaries under $k=5$ for entity dbr:Cindy_Mackey in
ESBM v1.2. Figure 6: Summaries generated by different entity summarizers under
$k=5$ for entity dbr:Cindy_Mackey in ESBM v1.2.

Moreover, we perform a case study with the entity dbr:Cindy_Mackey selected
from ESBM v1.2 to illustrate and analyze the outputs of the evaluated entity
summarizers. Figure 5 shows its six human-made ground-truth summaries under
$k=5$. Figure 6 shows its nine machine-generated summaries with F1 scores.
RELIN achieved the lowest F1 score. Its random surfer model selected redundant
triples (t11/t12/t13/t14/t15) due to their high topical relatedness to each
other. CD reduced redundancy by penalizing similar triples in its objective
function and achieved a higher F1 score, but it overemphasized informativeness
(t54/t55). FACES-E showed a better trade-off between informativeness and
popularity (t41/t42) and obtained a higher F1 score. DIVERSUM selected diverse
and locally frequent properties but paid no attention to property values and
selected some less informative values (t21/t22). MPSUM selected more favorable
property values (t91/t92) but still suffered redundancy (t93/t94). LinkSUM and
FACES could not process type- and literal-valued triples. LinkSUM tended to
select diverse and globally frequent property values but introduced many
redundant properties (t61/t62/t64/t65). FACES strictly avoided selecting
redundant properties from the same cluster of triples, leading to a summary
comprising only two triples for this entity (t31/t32). BAFREC selected several
favorable triples based on ontological informativeness (t71) and the frequency
of property value (t73). KAFCA achieved the highest F1 score. It successfully
selected a triple (t81) having a locally infrequent property and a value
containing a locally frequent word "1961". This triple appeared in most
ground-truth summaries but was rarely selected by other entity summarizers.

Table 3: Extrinsic Evaluation of Entity Summarization

| Task | Entities | Evaluation Metrics
---|---|---|---
RELIN [18] | entity matching | DBpedia, Freebase | accuracy, time
Yovisto [35] | entity search | DBpedia | success rate, questionnaire, time, recall
MMR-QSFS [36] | entity search | IMDb | precision, recall
COMB [13] | entity linking | DBpedia | accuracy, time, questionnaire
C3D+P [15] | entity matching | DBpedia, GeoNames, LinkedMDB | accuracy, time
REMES [11] | document understanding | DBpedia | questionnaire
CTab [16] | entity matching | DBpedia, Freebase, Wikidata, YAGO | accuracy, time, questionnaire

###  5.2 Extrinsic Evaluation

Table 3 presents known efforts in extrinsic evaluation of entity
summarization. They mainly use entities in RDF data representing encyclopedic
knowledge such as DBpedia, Freebase, Wikidata, and YAGO. Some use IMDb and
LinkedMDB for the movie domain, or GeoNames for the geography domain. Entity
summaries have been used to support a variety of downstream tasks for
extrinsic evaluation. Most evaluated entity summarizers were specifically
developed for the chosen task, while RELIN [18] was designed for general
purposes.

Entity Matching. RELIN [18], C3D+P [15], and CTab [16] were applied to support
the task of entity matching. Two or more entities were selected from different
datasets as candidate matches referring to the same real-world object. Human
users were invited to judge the correctness of each candidate match of
entities, by comparing their summaries. An entity summarizer would be
considered effective if the entity summaries it generated could help users
accurately and quickly complete judgments. In [16], human users were also
asked to answer a post-task questionnaire about the usefulness of the
presented entity summaries.

Entity Linking. COMB [13] was applied to support the task of entity linking. A
set of entities (e.g., the Apple fruit, the Apple Inc.) were selected from
DBpedia as candidates that a text span in a document (e.g., “apple”) might
refer to. Human users were invited to select the right entity, based on the
context of the text span and the summaries of the candidate entities. An
entity summarizer would be considered effective if the entity summaries it
generated could help users accurately and quickly complete selection. Human
users were also asked to rate the usefulness of each presented summary.

Document Understanding. REMES [11] was applied to support the task of document
understanding. To help users understand a document such as a news article, a
set of entities from DBpedia mentioned in the document were identified. Human
users were invited to understand the document assisted with the summaries of
these entities as background knowledge, and were asked to answer a post-task
questionnaire about the usefulness of the presented entity summaries.

Entity Search. Yovisto [35] and MMR-QSFS [36] were applied to support the task
of entity search. Human users were invited to identify entities relevant to a
query. Entity summaries were shown as search result snippets to support
relevance judgment [36], or were provided as navigation options for
exploratory search from the current entity to its related entities [35]. An
entity summarizer would be considered effective if the entity summaries it
generated could help users accurately and quickly identify relevant entities.
Accuracy was measured by information retrieval metrics such as precision and
recall, and by the proportion of successfully completed tasks. In [35], human
users were also asked to answer a post-task questionnaire about the usefulness
of the presented entity summaries.

##  6 Conclusion and Future Directions

We have presented a comprehensive technical review of existing entity
summarizers. To identify the most salient triples of an entity, existing
methods compute frequency and centrality, measure informativeness, assess
diversity and coverage, and consider domains, contexts, and personalization.
Multiple features are combined in different ways: from grouping and re-ranking
frameworks to random surfer, combinatorial optimization, and learning to rank
models. Methods using deep neural networks have also emerged. Although the
research on entity summarization is progressing rapidly, the problem is still
far from being solved, as suggested by the current evaluation results. Based
on our review of the state of the art, we identify the following research
directions for future work to overcome the limitations of existing research.
Encouragingly, preliminary attempts along some directions have made promising
progress.

##### Use of Semantics

As shown in Table 1, existing entity summarizers mainly rely on statistical
features such as frequency, centrality, and statistical informativeness.
Indeed, they are useful when the volume of semantic data is large and rising
quickly. However, it is not the magnitude but the semantics that distinguishes
semantic data from other types of structured data. Entity summarizers that
process semantic data are expected to exploit semantics in the data. Among
others, classes and properties are associated with meaning characterized by
axioms. Such ontological semantics is rich, but unfortunately, it has not been
fully considered in existing research. We have only witnessed some shallow use
of class hierarchies for calculating informativeness [9, 46] as reviewed in
Section 2.1.2 and for identifying similar triples [13, 15, 40] as reviewed in
Section 2.1.3. More semantics and more powerful reasoning capabilities can be
useful for entity summarization. For example, to generate summaries for
comparing two or more entities, one potentially useful method for identifying
implicit inter-entity dissimilarity is to detect conflicting triples using
reasoning and consistency checking techniques [67, 68].

##### Human Factors

As shown in Table 1, most technical features in use are data-centric. They
analyze various aspects of semantic data such as frequency and similarity.
However, if entity summaries are generated to be presented to human users,
more research attention needs to be given to human factors since the ultimate
goal of entity summarization is to find triples that are truly useful for
human users. For example, when summarizing the description of a book, its ISBN
is statistically informative as it uniquely identifies a book. This triple is
likely to be selected into an entity summary by many existing summarizers, and
indeed it is useful for applications like entity resolution. However, it may
not be interesting to many human users because the information it provides is
not directly useful. Here, a desired feature would be to assess the
meaningfulness or human friendliness of a triple to the lay audience. One
potentially effective method for identifying human-friendly triples is to
measure the readability of their textual forms [69]. The measurement of text
readability has been studied for decades [70], providing a lot of tools for
reuse. Another research direction about human factors is to learn users’
preference from their behavior and generate personalized entity summaries. We
have seen an early attempt [34] in Section 2.2.3 but further studies are in
demand.

##### Machine and Deep Learning

As reviewed in Section 3.6, only a few entity summarizers use supervised
methods [9, 36, 42, 49]. These methods focus on specific applications where
labeled data is available for training. For generic entity summarization, it
is expensive to manually label a large set of entity summaries as training
data. This has hindered the development of supervised entity summarization in
a general setting. Recent benchmarks for intrinsic evaluation of entity
summarization have made available some ground-truth entity summaries. Based on
that, new supervised entity summarizers have emerged, including a few deep
learning based methods [59, 60] as reviewed in Section 4. However, compared
with the fruitful research on supervised document summarization [25, 26],
there is still plenty of room for applying machine and deep learning
techniques to entity summarization. To obtain more labeled entity summaries
for training, one possible solution is to programmatically generate labels.
For example, in [71], entity descriptions in DBpedia are automatically aligned
with abstracts of DBpedia articles to identify key triples for each entity and
form a summary. A large number of labeled entity summaries can be generated in
this way, though containing noise and hence the supervision would be weak
[72].

##### Non-Extractive Methods

All the entity summarizers reviewed in this article use extractive methods.
They generate a summary for an entity description by extracting a subset of
triples. In document summarization [25, 26], a paradigm complementary to
extractive methods is non-extractive summarization, which generates an
abstractive summary consisting of ideas or concepts that are taken from the
original description but are re-interpreted in different and better forms. For
document summarization, non-extractive methods are complex as they need
extensive natural language processing. For entity summarization, the
exploration of non-extractive methods is in the initial phase. It is still
open to discuss the concrete form of an abstractive entity summary. One
promising direction is to generate a textual summary, as text may be more
readable than triples. A textual summary can either be generated from a
computed triple-structured summary using natural language generation
techniques [73], or be directly generated from an entity description using
sequence-to-sequence models [74, 75, 76] which combine triple selection and
text generation into a hybrid process.

##### Interactive Methods

Entity summarizers are not perfect at all times. Indeed, as reviewed in
Section 5.1.3, they are still far from perfect. When an one-shot summary fails
to fulfill a user’s information needs, the user may expect to see an improved
summary after interacting with the summarizer. This kind of interactive entity
summarization has not received much research attention. For document
summarization, interactive methods solicit feedback from the user to capture
opinions and interests [77, 78]. Entity summarization can follow this line of
research, and user feedback can be positive or negative opinions about
triples. However, considering the structured and semantic nature of entity
descriptions, directly adapting text-based methods for interactive document
summarization is unlikely to be effective. Research is needed for finding
suitable models for characterizing a user’s interaction with an entity
summarizer. In a pioneer work [79], reinforcement learning is employed to
model interactive entity summarization where a user can provide negative
feedback on presented triples and then be served with an improved summary.
Apart from that, more effective methods may be developed by incorporating
state-of-the-art interactive information retrieval approaches such as online
learning to rank [80].

##  7 Acknowledgements

This work was supported by the NSFC [grant number 62072224 and 61772264].

## References

  * [1] B. Adida, M. Birbeck, S. McCarron, I. Herman, RDFa core 1.1 - third edition (2015).
URL http://www.w3.org/TR/rdfa-core/

  * [2] I. Hickson, HTML microdata (Oct. 2013).
URL http://www.w3.org/TR/microdata/

  * [3] C. Bizer, T. Heath, T. Berners-Lee, Linked data - the story so far, Int. J. Semantic Web Inf. Syst. 5 (3) (2009) 1–22.  doi:10.4018/jswis.2009081901.
  * [4] N. F. Noy, Y. Gao, A. Jain, A. Narayanan, A. Patterson, J. Taylor, Industry-scale knowledge graphs: lessons and challenges, Commun. ACM 62 (8) (2019) 36–43.  doi:10.1145/3331166.
  * [5] Google, Introducing the knowledge graph: Things, not strings (May 2012).
URL http://googleblog.blogspot.com/2012/05/introducing-knowledge-graph-things-
not.html

  * [6] K. Gunaratna, Semantics-based summarization of entities in knowledge graphs, Ph.D. thesis, Wright State University (2017).
  * [7] A. Thalhammer, Linked data entity summarization, Ph.D. thesis, Karlsruher Institut für Technologie (2017).
  * [8] W. Shen, J. Wang, J. Han, Entity linking with a knowledge base: Issues, techniques, and solutions, IEEE Trans. Knowl. Data Eng. 27 (2) (2015) 443–460.  doi:10.1109/TKDE.2014.2327028.
  * [9] A. Tonon, M. Catasta, R. Prokofyev, G. Demartini, K. Aberer, P. Cudré-Mauroux, Contextualized ranking of entity types based on knowledge graphs, J. Web Sem. 37-38 (2016) 170–183.  doi:10.1016/j.websem.2015.12.005.
  * [10] A. Thalhammer, A. Rettinger, ELES: combining entity linking and entity summarization, in: Proc. 16th International Conference on Web Engineering (ICWE), 2016, pp. 547–550.  doi:10.1007/978-3-319-38791-8\\_45.
  * [11] K. Gunaratna, A. H. Yazdavar, K. Thirunarayan, A. P. Sheth, G. Cheng, Relatedness-based multi-entity summarization, in: Proc. 26th International Joint Conference on Artificial Intelligence (IJCAI), 2017, pp. 1060–1066.  doi:10.24963/ijcai.2017/147.
  * [12] G. Demartini, D. E. Difallah, P. Cudré-Mauroux, Zencrowd: leveraging probabilistic reasoning and crowdsourcing techniques for large-scale entity linking, in: Proc. 21st World Wide Web Conference (WWW), 2012, pp. 469–478.  doi:10.1145/2187836.2187900.
  * [13] G. Cheng, D. Xu, Y. Qu, Summarizing entity descriptions for effective and efficient human-centered entity linking, in: Proc. 24th International Conference on World Wide Web (WWW), 2015, pp. 184–194.  doi:10.1145/2736277.2741094.
  * [14] D. Xu, G. Cheng, Y. Qu, Facilitating human intervention in coreference resolution with comparative entity summaries, in: Proc. 11th Extended Semantic Web Conference (ESWC), 2014, pp. 535–549.  doi:10.1007/978-3-319-07443-6\\_36.
  * [15] G. Cheng, D. Xu, Y. Qu, C3D+P: A summarization method for interactive entity resolution, J. Web Sem. 35 (2015) 203–213.  doi:10.1016/j.websem.2015.05.004.
  * [16] J. Huang, W. Hu, H. Li, Y. Qu, Automated comparative table generation for facilitating human intervention in multi-entity resolution, in: Proc. 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), 2018, pp. 585–594.  doi:10.1145/3209978.3210021.
  * [17] R. Cyganiak, D. Wood, M. Lanthaler, RDF 1.1 concepts and abstract syntax (2014).
URL https://www.w3.org/TR/rdf11-concepts/

  * [18] G. Cheng, T. Tran, Y. Qu, RELIN: relatedness and informativeness-based centrality for entity summarization, in: Proc. 10th International Semantic Web Conference (ISWC), Part I, 2011, pp. 114–129.  doi:10.1007/978-3-642-25073-6\\_8.
  * [19] A. Thalhammer, I. Toma, A. J. Roa-Valverde, D. Fensel, Leveraging usage data for linked data movie entity summarization, in: Proc. 2nd International Workshop on Usage Analysis and the Web of Data (USEWOD), 2012.
  * [20] J. Yan, Y. Wang, M. Gao, A. Zhou, Context-aware entity summarization, in: Proc. 17th International Conference on Web-Age Information Management (WAIM), Part I, 2016, pp. 517–529.  doi:10.1007/978-3-319-39937-9\\_40.
  * [21] K. Gunaratna, K. Thirunarayan, A. P. Sheth, FACES: diversity-aware entity summarization using incremental hierarchical conceptual clustering, in: Proc. 29th AAAI Conference on Artificial Intelligence (AAAI), 2015, pp. 116–122.
  * [22] K. Gunaratna, K. Thirunarayan, A. P. Sheth, G. Cheng, Gleaning types for literals in RDF triples with application to entity summarization, in: Proc. 13th Extended Semantic Web Conference (ESWC), 2016, pp. 85–100.  doi:10.1007/978-3-319-34129-3\\_6.
  * [23] Y. Li, L. Zhao, A common property and special property entity summarization approach based on statistical distribution, in: Proc. 2nd International Workshop on Summarizing and Presenting Entities and Ontologies (SumPre), 2016\.
  * [24] A. Thalhammer, N. Lasierra, A. Rettinger, Linksum: Using link analysis to summarize entity data, in: Proc. 16th International Conference on Web Engineering (ICWE), 2016, pp. 244–261.  doi:10.1007/978-3-319-38791-8\\_14.
  * [25] Z. Nasar, S. W. Jaffry, M. K. Malik, Textual keyword extraction and summarization: State-of-the-art, Inf. Process. Manage. 56 (6).  doi:10.1016/j.ipm.2019.102088.
  * [26] M. Gambhir, V. Gupta, Recent automatic text summarization techniques: a survey, Artif. Intell. Rev. 47 (1) (2017) 1–66.  doi:10.1007/s10462-016-9475-9.
  * [27] G. Erkan, D. R. Radev, Lexrank: Graph-based lexical centrality as salience in text summarization, J. Artif. Intell. Res. 22 (2004) 457–479.  doi:10.1613/jair.1523.
  * [28] S. Cebiric, F. Goasdoué, H. Kondylakis, D. Kotzinos, I. Manolescu, G. Troullinou, M. Zneika, Summarizing semantic graphs: a survey, VLDB J. 28 (3) (2019) 295–327.  doi:10.1007/s00778-018-0528-3.
  * [29] Y. Liu, T. Safavi, A. Dighe, D. Koutra, Graph summarization methods and applications: A survey, ACM Comput. Surv. 51 (3) (2018) 62:1–62:34.  doi:10.1145/3186727.
  * [30] S. Pouriyeh, M. Allahyari, Q. Liu, G. Cheng, H. R. Arabnia, M. Atzori, K. Kochut, Graph-based methods for ontology summarization: A survey, in: Proc. 1st IEEE International Conference on Artificial Intelligence and Knowledge Engineering (AIKE), 2018, pp. 85–92.  doi:10.1109/AIKE.2018.00020.
  * [31] S. Pouriyeh, M. Allahyari, Q. Liu, G. Cheng, H. R. Arabnia, M. Atzori, F. G. Mohammadi, K. Kochut, Ontology summarization: Graph-based methods and beyond, Int. J. Semantic Computing 13 (2) (2019) 259–283.  doi:10.1142/S1793351X19300012.
  * [32] G. Cheng, Y. Qu, Searching linked objects with falcons: Approach, implementation and evaluation, Int. J. Semantic Web Inf. Syst. 5 (3) (2009) 49–70.  doi:10.4018/jswis.2009081903.
  * [33] Z. Liu, P. Sun, Y. Chen, Structured search result differentiation, PVLDB 2 (1) (2009) 313–324.
  * [34] J. Zhang, C. Ma, C. Zhao, J. Zhang, L. Yi, X. Mao, A novel ranking framework for linked data from relational databases, Tsinghua Science & Technology 15 (6) (2010) 642–649.
  * [35] J. Waitelonis, H. Sack, Towards exploratory video search using linked data, Multimedia Tools Appl. 59 (2) (2012) 645–672.  doi:10.1007/s11042-011-0733-1.
  * [36] L. Zhang, Y. Zhang, Y. Chen, Summarizing highly structured documents for effective search interaction, in: Proc. 35th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), 2012, pp. 145–154.  doi:10.1145/2348283.2348306.
  * [37] M. Sydow, M. Pikula, R. Schenkel, The notion of diversity in graphical entity summarisation on semantic knowledge graphs, J. Intell. Inf. Syst. 41 (2) (2013) 109–149.  doi:10.1007/s10844-013-0239-6.
  * [38] A. Thalhammer, A. Rettinger, Browsing dbpedia entities with summaries, in: Proc. ESWC 2014 Satellite Events, 2014, pp. 511–515.  doi:10.1007/978-3-319-11955-7\\_76.
  * [39] T. Althoff, X. L. Dong, K. Murphy, S. Alai, V. Dang, W. Zhang, Timemachine: Timeline generation for knowledge-base entities, in: Proc. 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), 2015, pp. 19–28.  doi:10.1145/2783258.2783325.
  * [40] D. Xu, L. Zheng, Y. Qu, CD at ENSEC 2016: Generating characteristic and diverse entity summaries, in: Proc. 2nd International Workshop on Summarizing and Presenting Entities and Ontologies (SumPre), 2016.
  * [41] A. G. Nuzzolese, V. Presutti, A. Gangemi, S. Peroni, P. Ciancarini, Aemoo: Linked data exploration based on knowledge patterns, Semantic Web 8 (1) (2017) 87–112.  doi:10.3233/SW-160222.
  * [42] F. Hasibi, K. Balog, S. E. Bratsberg, Dynamic factual summaries for entity cards, in: Proc. 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), 2017, pp. 773–782.  doi:10.1145/3077136.3080810.
  * [43] E. Kim, K. Choi, Entity summarization based on entity grouping in multilingual projected entity space, IEICE Transactions 100-D (9) (2017) 2138–2146.
  * [44] S. A. Pouriyeh, M. Allahyari, K. Kochut, G. Cheng, H. R. Arabnia, ES-LDA: entity summarization using knowledge-based topic modeling, in: Proc. 8th International Joint Conference on Natural Language Processing (IJCNLP), Volume 1, 2017, pp. 316–325.
  * [45] S. A. Pouriyeh, M. Allahyari, K. Kochut, G. Cheng, H. R. Arabnia, Combining word embedding and knowledge-based topic modeling for entity summarization, in: Proc. 12th IEEE International Conference on Semantic Computing (ICSC), 2018, pp. 252–255.  doi:10.1109/ICSC.2018.00044.
  * [46] H. Kroll, D. Nagel, W.-T. Balke, Bafrec: Balancing frequency and rarity for entity characterization in linked open data, in: Proc. 1st International Workshop on Entity REtrieval (EYRE), 2018.
  * [47] E.-K. Kim, K.-S. Choi, Entity summarization based on formal concept analysis, in: Proc. 1st International Workshop on Entity REtrieval (EYRE), 2018.
  * [48] D. Wei, S. Gao, Y. Liu, Z. Liu, L. Huang, Mpsum: Entity summarization with predicate-based matching, in: Proc. 1st International Workshop on Entity REtrieval (EYRE), 2018.
  * [49] S. Gottschalk, E. Demidova, Eventkg - the hub of event knowledge on the web - and biographical timeline generation, Semantic Web (2019) 1–32doi:10.3233/SW-190355.
  * [50] J. Wei, S. Han, L. Zou, VISION-KG: topic-centric visualization system for summarizing knowledge graph, in: Proc. 13th ACM International Conference on Web Search and Data Mining (WSDM), ACM, 2020, pp. 857–860.  doi:10.1145/3336191.3371863.
  * [51] L. Page, S. Brin, R. Motwani, T. Winograd, The pagerank citation ranking: Bringing order to the web, Tech. rep., Stanford InfoLab (Nov. 1999).
  * [52] D. M. Blei, A. Y. Ng, M. I. Jordan, Latent dirichlet allocation, Journal of Machine Learning Research 3 (2003) 993–1022.
  * [53] G. A. Miller, Wordnet: A lexical database for english, Commun. ACM 38 (11) (1995) 39–41.  doi:10.1145/219717.219748.
  * [54] H. Cai, V. W. Zheng, K. C. Chang, A comprehensive survey of graph embedding: Problems, techniques, and applications, IEEE Trans. Knowl. Data Eng. 30 (9) (2018) 1616–1637.  doi:10.1109/TKDE.2018.2807452.
  * [55] P. Ristoski, H. Paulheim, Rdf2vec: RDF graph embeddings for data mining, in: Proc. 15th International Semantic Web Conference (ISWC), Part I, 2016, pp. 498–514.  doi:10.1007/978-3-319-46523-4\\_30.
  * [56] H. Xiao, M. Huang, L. Meng, X. Zhu, SSP: semantic space projection for knowledge graph embedding with text descriptions, in: S. P. Singh, S. Markovitch (Eds.), Proc. 31st AAAI Conference on Artificial Intelligence (AAAI), 2017, pp. 3104–3110.
  * [57] J. G. Carbonell, J. Goldstein, The use of mmr, diversity-based reranking for reordering documents and producing summaries, in: Proc. 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), 1998, pp. 335–336.  doi:10.1145/290941.291025.
  * [58] Z. Yang, G. Wang, F. Chu, An effective GRASP and tabu search for the 0-1 quadratic knapsack problem, Comput. Oper. Res. 40 (5) (2013) 1176–1185.  doi:10.1016/j.cor.2012.11.023.
  * [59] D. Wei, Y. Liu, ESA: entity summarization with attention, in: Proc. 2nd International Workshop on Entity REtrieval (EYRE), 2019.
  * [60] Q. Liu, G. Cheng, Y. Qu, DeepLENS: deep learning for entity summarization, in: Proc. of the Workshop on Deep Learning for Knowledge Graphs (DL4KG), 2020\.
  * [61] Y. Bengio, R. Ducharme, P. Vincent, C. Janvin, A neural probabilistic language model, J. Mach. Learn. Res. 3 (2003) 1137–1155.
  * [62] A. Bordes, N. Usunier, A. García-Durán, J. Weston, O. Yakhnenko, Translating embeddings for modeling multi-relational data, in: Proc. 27th Annual Conference on Neural Information Processing Systems (NIPS), 2013, pp. 2787–2795.
  * [63] A. Thalhammer, M. Knuth, H. Sack, Evaluating entity summarization using a game-based ground truth, in: Proc. 11th International Semantic Web Conference (ISWC), Part II, 2012, pp. 350–361.  doi:10.1007/978-3-642-35173-0\\_24.
  * [64] P. Langer, P. Schulze, S. George, M. Kohnen, T. Metzke, Z. Abedjan, G. Kasneci, Assigning global relevance scores to dbpedia facts, in: Proc. 30th International Conference on Data Engineering (ICDE) Workshops, 2014, pp. 248–253.  doi:10.1109/ICDEW.2014.6818334.
  * [65] T. Bobic, J. Waitelonis, H. Sack, Franco - A ground truth corpus for fact ranking evaluation, in: Joint Proc. 1st International Workshop on Summarizing and Presenting Entities and Ontologies (SumPre) and 3rd International Workshop on Human Semantic Web Interfaces (HSWI), 2015.
  * [66] Q. Liu, G. Cheng, K. Gunaratna, Y. Qu, ESBM: an entity summarization benchmark, in: Proc. 17th Extended Semantic Web Conference (ESWC), 2020, pp. 548–564.  doi:10.1007/978-3-030-49461-2\\_32.
  * [67] H. Paulheim, H. Stuckenschmidt, Fast approximate a-box consistency checking using machine learning, in: Proc. 13th Extended Semantic Web Conference (ESWC), 2016, pp. 135–150.  doi:10.1007/978-3-319-34129-3\\_9.
  * [68] Y. Gu, J. Z. Pan, G. Cheng, H. Paulheim, G. Stoilos, Local abox consistency prediction with transparent tboxes using gated graph neural networks, in: Proc. 14th International Workshop on Neural-Symbolic Learning and Reasoning (NeSy), 2019.
  * [69] Q. Liu, G. Cheng, Y. Qu, ESSTER at the EYRE 2020 entity summarization task, in: Proc. 3rd International Workshop on EntitY Retrieval and lEarning (EYRE), 2020\.
  * [70] K. Collins-Thompson, Computational assessment of text readability: A survey of current and future research, ITL Int. J. Appl. Linguist. 165 (2) (2014) 97–135.  doi:10.1075/itl.165.2.01col.
  * [71] D. Xu, G. Cheng, Y. Qu, Preferences in wikipedia abstracts: Empirical findings and implications for automatic entity summarization, Inf. Process. Manage. 50 (2) (2014) 284–296.  doi:10.1016/j.ipm.2013.12.001.
  * [72] J. Li, G. Cheng, Q. Liu, W. Zhang, E. Kharlamov, K. Gunaratna, H. Chen, Neural entity summarization with joint encoding and weak supervision, in: Proc. 29th International Joint Conference on Artificial Intelligence (IJCAI), 2020, pp. 1644–1650.  doi:10.24963/ijcai.2020/228.
  * [73] A. Gatt, E. Krahmer, Survey of the state of the art in natural language generation: Core tasks, applications and evaluation, J. Artif. Intell. Res. 61 (2018) 65–170.  doi:10.1613/jair.5477.
  * [74] R. Lebret, D. Grangier, M. Auli, Neural text generation from structured data with application to the biography domain, in: Proc. 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2016, pp. 1203–1213.
  * [75] B. Hachey, W. Radford, A. Chisholm, Learning to generate one-sentence biographies from wikidata, in: Proc. 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL), Volume 1, 2017, pp. 633–642.
  * [76] P. Vougiouklis, H. ElSahar, L. Kaffee, C. Gravier, F. Laforest, J. S. Hare, E. Simperl, Neural wikipedian: Generating textual summaries from knowledge base triples, J. Web Semant. 52-53 (2018) 1–15.  doi:10.1016/j.websem.2018.07.002.
  * [77] R. Yan, J. Nie, X. Li, Summarize what you are interested in: An optimization framework for interactive personalized summarization, in: Proc. 2011 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2011, pp. 1342–1351.
  * [78] P. V. S. Avinesh, C. M. Meyer, Joint optimization of user-desired content in multi-document summaries by learning from user feedback, in: Proc. 55th Annual Meeting of the Association for Computational Linguistics (ACL), Volume 1, 2017, pp. 1353–1363.  doi:10.18653/v1/P17-1124.
  * [79] Q. Liu, Y. Chen, G. Cheng, E. Kharlamov, J. Li, Y. Qu, Entity summarization with user feedback, in: Proc. 17th Extended Semantic Web Conference (ESWC), 2020, pp. 376–392.  doi:10.1007/978-3-030-49461-2\\_22.
  * [80] R. Jagerman, H. Oosterhuis, M. de Rijke, To model or to intervene: A comparison of counterfactual and online learning to rank from user interactions, in: Proc. 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), 2019, pp. 15–24.  doi:10.1145/3331184.3331269.