11institutetext: School of Data Science, Fudan University, Shanghai, China
11email:
<EMAIL_ADDRESS>School of Computer Science, Fudan University, Shanghai, China 22email:
<EMAIL_ADDRESS>22email<EMAIL_ADDRESS>33institutetext:
Fudan-Aishu Cognitive Intelligence Joint Research Center, Shanghai, China
44institutetext: Meituan, Beijing, China 44email<EMAIL_ADDRESS>
# Large-scale Multi-granular Concept Extraction Based on Machine Reading
Comprehension††thanks: This work is supported by Science and Technology on
Information Systems Engineering Laboratory at the 28th Research Institute of
China Electronics Technology Group Corporation, Nanjing Jiangsu, China (No.
05202002), National Key Research and Development Project (No. 2020AAA0109302),
Shanghai Science and Technology Innovation Action Plan (No. 19511120400) and
Shanghai Municipal Science and Technology Major Project (No. 2021SHZDZX0103).

Siyu Yuan 11 0000-0001-8161-6429 Deqing Yang🖂 11 0000-0002-1390-3861
Jiaqing Liang 22 0000-0003-0670-5602 Jilun Sun 22 0000-0003-4611-5367
Jingyue Huang 11 0000-0002-3880-6785 Kaiyan Cao 11 0000-0001-5763-3050 Yanghua
Xiao🖂 2233 0000-0001-8403-9591 Rui Xie 44 0000-0002-1116-7418

###### Abstract

The concepts in knowledge graphs (KGs) enable machines to understand natural
language, and thus play an indispensable role in many applications. However,
existing KGs have the poor coverage of concepts, especially fine-grained
concepts. In order to supply existing KGs with more fine-grained and new
concepts, we propose a novel concept extraction framework, namely _MRC-CE_ ,
to extract large-scale multi-granular concepts from the descriptive texts of
entities. Specifically, MRC-CE is built with a machine reading comprehension
model based on BERT, which can extract more fine-grained concepts with a
pointer network. Furthermore, a random forest and rule-based pruning are also
adopted to enhance MRC-CE’s precision and recall simultaneously. Our
experiments evaluated upon multilingual KGs, i.e., English _Probase_ and
Chinese _CN-DBpedia_ , justify MRC-CE’s superiority over the state-of-the-art
extraction models in KG completion. Particularly, after running MRC-CE for
each entity in CN-DBpedia, more than 7,053,900 new concepts (instanceOf
relations) are supplied into the KG. The code and datasets have been released
at https://github.com/fcihraeipnusnacwh/MRC-CE.

###### Keywords:

Concept Extraction Knowledge Graph Machine Reading Comprehension Multi-
granular Concept Concept Overlap

##  1 Introduction

The concepts in knowledge graphs (KGs) [36, 3, 41] enable machines to better
understand natural languages and thus play an increasingly significant role in
many applications, such as question answering [7], personalized recommendation
[32], commonsense reasoning [25], etc. Particularly, fine-grained concepts
greatly promote the downstream applications. For example, if entity _Google_
has ‘technology company’ and ‘search engine company’ as its concepts, a job
recommender system would recommend Google rather than Wal-Mart to a graduate
from computer science department based on such fine-grained concepts.

Although there have been a great number of efforts on constructing KGs in
recent years, the concepts in existing KGs are still far from being complete,
especially for fine-grained concepts. In the widely used Chinese KG _CN-
DBpedia_ [41], there are nearly 17 million entities but only 0.27 million
concepts in total, and more than 20% entities even have no concepts. In
general, fine-grained concepts contain more than one modifier, and thus have
longer characters (words). However, the average character length of CN-DBpedia
concepts is only 3.62, implying that most of them are coarse-grained.

The poor coverage of concepts, especially fine-grained concepts, in the
existing KGs is due to their approaches’ drawbacks. Most of the existing
concept acquisition approaches are based on _generation_ or _extraction_ from
texts. Generation methods often generate coarse-grained concepts from free
texts since they are inclined to generate high-frequency words [2, 34].
Extraction methods mainly have three types of models. Pattern-matching models
[36, 24, 3, 45] focus on extracting concepts from texts based on handcrafted
patterns or rules, but the recall of concept extraction (CE) is low due to
their limited generalization ability. Classification models [15, 40, 18, 9]
identify concepts through classifying a given entity into a predefined concept
set based on text information, but can not find new concepts. Sequence
labeling models [38, 22] treat the CE problem as a sequence labeling task, but
can not handle the problem of _concept overlap_. The concept overlap refers to
the phenomenon that a concept term is the subsequence of another concept term.
For example, in Fig. 1, once a sequence labeling model labels ‘company’ as
Concept 1, it would not label ‘multinational technology company’ as a fine-
grained concept, since it can not mark a token with different labels.

Figure 1: The abstract text of entity _Google_. The problem of concept overlap
in the text is challenging for traditional extraction models to extract
multiple concepts from the same span.

Machine reading comprehension (MRC) model can extract the answer from the
contextual texts for a proposed question. Inspired by MRC model’s excellent
extraction capability, we handle CE problem as an MRC task in this paper, and
propose a novel CE framework named _MRC-CE_ , to implement large-scale multi-
granular CE from the descriptive texts of given entities. MRC-CE can extract
more multi-granular concepts than previous concept extraction models due to
the following reasons. Firstly, MRC-CE is built with an MRC model based on
BERT [8] (BERT-MRC), which can find abundant new concepts and handle the
problem of _concept overlap_ well with a pointer network [37]. Secondly, a
random forest [30] and rule-based pruning are adopted to filter the candidate
concepts output by BERT-MRC, which enhances MRC-CE’s precision and recall
simultaneously. Furthermore, MRC-CE has been proven capable of acquiring
large-scale multi-granular concepts through our extensive experiments.

The major contributions of this paper can be summarized as follows:

1\. To the best of our knowledge, this is the first work to employ MRC model
in text-based CE for large-scale KG completion.

2\. Through our experiments, more than 7,053,900 new concepts (instanceOf
relations) were extracted by our MRC-CE for completing the large-scale Chinese
KG _CN-DBpedia_. Furthermore, the online service of CE based on MRC-CE is
released on http://kw.fudan.edu.cn.

The rest of this paper is organized as follows. Section 2 is the review of
related works and Section 3 is the detailed introduction of our framework,
respectively. We display our experiment results in Section 4 to justify the
effectiveness and rationality of MRC-CE. We also display the CE results of
applying our framework on CN-DBpedia and Meituan111https://www.meituan.com in
Section 5, and at last conclude our work in Section 6.

##  2 Related Work

In this section, we review some works related to concept acquisition.

###  2.1 Ontology Extension

Ontology [5] extension focuses more on identifying hypernym-hyponym
relationships [47, 23, 46]. Unlike it, our goal is to acquire more new
concepts, especially fine-grained concepts, to complete the existing KGs.
Thus, we did not compare MRC-CE with these methods in our experiments.

###  2.2 IsA relation and Entity typing

IsA relation extraction [29, 12] is extracting the subsumption (subClassOf)
relation between two classes. Entity type aims to classify an entity into a
predefined set of types (concepts), such as person, location and organization
without new concept recognition [42, 17, 40]. However, in our setting in this
paper, the entity is given at first and its candidate concepts are not pre-
defined. Thus, our work is different to isA relation extraction and entity
typing, and thus the methods of these two tasks were not compared in our
experiments.

###  2.3 Text-based Concept Extraction

In this paper, we only focus on extracting the concepts already existing in
the texts rather than concept generation. Hence we pay attention to extraction
models rather than generative models [2, 34]. The existing CE methods can be
divided into three categories.

####  2.3.1 Pattern-matching Method

Pattern-matching methods [36, 24, 3, 39, 45, 16, 41, 4] try to extract
concepts from free texts based on handcrafted patterns or rules. They can
acquire accurate concepts, but only have low recall due to the poor
generalization ability. Comparatively, our MRC-CE achieves CE task based on
MRC model beyond the limitation of handcrafted patterns, and thus acquires
more concepts.

####  2.3.2 Learning-based Method

Classification models transform CE into classification [17, 40, 18, 9] to
determine which concept in a predefined set meets hypernym-hyponym relation
with the given entity, but they can not acquire new concepts. Sequence
labeling models have been proven effective on extraction tasks [21, 38, 1,
22]. Given the extraction feature, sequence labeling models can also extract
concepts from the texts describing entities as our MRC-CE. Recently, pre-
trained contextual embeddings have been widely used in sequence labeling
models [6, 44, 43, 35] to gain good performance, but can not handle the
problem of _concept overlap_.

####  2.3.3 Knowledge-based Method

These methods [27, 4, 26] mainly use the external information from KGs to
achieve extraction tasks, resulting in good CE. However, these models have
poor generalization ability and only focus on a specific field.

###  2.4 Machine Reading Comprehension

MRC [19] is a task to seek the answer from contextual texts for a proposed
question, which can be categorized into four classes according to answer
format, i.e., cloze test, multiple choice, span extraction and free answering.
The span extraction [31, 33] is most related and similar to our task. The
state-of-the-art pre-trained language models [8, 20] are often applied in MRC
tasks. More recently, many researchers have employed MRC model in
accomplishing other NLP tasks, including nested NER [13] and RE [14].
Comparatively, our MRC-CE is the first attempt of employing MRC model in text-
based CE.

##  3 Methodology

In this section, we introduce our CE framework in detail.

###  3.1 Task Definition

Our CE task can be formulated as follows. Given an entity $e$ and its relevant
descriptive text, denoted as a sequence of words
$X=\\{x_{1},x_{2},...,x_{n}\\}$ where $x_{i}(1\leq i\leq n)$ is a word token,
the CE task aims to find one or multiple spans from $X$ as $e$’s concept(s).

###  3.2 Data Construction

The descriptive text of high quality plays an important role in text-based CE.
Since our task scenario is span extraction, the input text should contain the
concept(s) of the given entity. We consider the given entity’s abstract in
encyclopedia since it is well structured and explicitly mentions the
concept(s) of the given entity. In the following introduction, the input text
is always referred to the abstract of a given entity. The construction details
of our English and Chinese datasets will be introduced in Subsec. 4.1.

###  3.3 Summary

To employ MRC model for CE, we need to construct appropriate questions towards
which the spans in the text are extracted. Since our target is to extract
concepts from the abstract of a given entity, we set the question $Q$ as ‘What
is the concept for [entity]?’. The pipeline of our MRC-CE is displayed in Fig.
2, which can be divided into three modules, i.e., the concept extractor,
concept selector and concept pruner. The first module is BERT-MRC built with a
pointer network [37], which receives the input text and extracts candidate
multi-granular concepts from the text. The second module adopts a random
forest [30] to select the desirable concepts from the candidates output by
BERT-MRC. The third module prunes away the unsatisfactory concepts from the
second module’s outputs based on some pruning rules.

Figure 2: The pipeline of our MRC-CE framework.

###  3.4 MRC-based Concept Extractor

The BERT-MRC in our framework is built with a BERT [8] encoder and a pointer
network to generate candidate spans along with their corresponding confidence
scores of being concepts. We took each entity’s concepts from the KG which
also exist in the entity’s abstract text as the real labels of one training
sample.

####  3.4.1 BERT Encoding

In this module, BERT is used as an encoder layer to generate an embedding for
each input token, based on which the subsequent pointer network predicts the
confidence score for each candidate extracted span. At first, the tokens of
input $X$ and the tokens of input $X$ and the question $Q$ are concatenated
together as follows, to constitute the input of BERT encoder.

$\\{[CLS]q_{1},q_{2},...,q_{m}[SEP]x_{1},x_{2},...,x_{n}\\}$

where [CLS] and [SEP] are special tokens, and $q_{i}(1\leq i\leq m)$ is a
token of $Q$. Then, BERT encoder outputs the text token embedding matrix
$\mathbf{E}\in\mathbb{R}^{n\times d}$, where $d$ is the embedding dimension.

####  3.4.2 Generating Candidate Concepts by Pointer Network

With the token embeddings in $\mathbf{E}$, a pointer network is built to
predict the probability of a token being the start position and end position
of the answer, through a fully connected layer after the BERT encoder. Then,
the confidence score of each span can be calculated as the sum of the
probability of its start token and end token. One span can be output
repeatedly as the same subsequence of multiple extracted concepts through an
appropriate selection threshold. This strategy enables extracting multi-
granular concepts. For example, in Fig. 3, ‘company’ is extracted for multiple
times corresponding to three concepts of different granularity, if the
confidence score threshold is set to 0.85.

Figure 3: The structure of BERT-MRC.

Specifically, we use $\mathbf{p}^{start},\mathbf{p}^{end}\in\mathbb{R}^{n}$ to
denote the vectors storing each token’s probability of being the start
position and end position, respectively. They are calculated based on
$\mathbf{E}$ as

$[\mathbf{p}^{start};\mathbf{p}^{end}]=softmax(\mathbf{E}\mathbf{W}+\mathbf{B})$
(1)

where $\mathbf{W},\mathbf{B}\in\mathbb{R}^{n\times 2}$ are both trainable
parameters. Given a span with $x_{i}$ and $x_{j}$ as the start token and the
end token, respectively, its confidence score ${cs}_{ij}\in\mathbb{R}$ can be
calculated as

$cs_{ij}=p^{start}_{i}+p^{end}_{j}$ (2)

Then, BERT-MRC generates a ranking list of candidate concepts (spans) along
with confidence scores, and outputs the concepts with the confidence scores
larger than the selection threshold.

####  3.4.3 BERT-MRC Loss

We adopt cross entropy function $CrossEntropy(\cdot)$ as the loss function of
BERT-MRC. Specifically, suppose the set $Y_{start}\in\mathbb{R}^{n}$ (or
$Y_{end}\in\mathbb{R}^{n}$) contains the real label of an input token
$x_{i}(1\leq i\leq n)$ being the start (or end) position of a concept. And
$Y_{span}\in\mathbb{R}^{C^{2}_{n}}$ contains the real label of a span
$x_{i}x_{j}$ where $x_{i}$ and $x_{j}$ are the start position and end position
of a concept, respectively. Then, we have the following three losses for the
predictions of the three situations:

$\begin{split}\mathcal{L}_{start}=CrossEntropy(\mathbf{p}^{start},Y_{start})\\\
\mathcal{L}_{end}=CrossEntropy(\mathbf{p}^{end},Y_{end})\\\
\mathcal{L}_{span}=CrossEntropy(\mathbf{cs},Y_{span})\end{split}$ (3)

Then, the comprehensive loss of training BERT-MRC is

$\mathcal{L}=\alpha\mathcal{L}_{start}+\beta\mathcal{L}_{end}+(1-\alpha-\beta)\mathcal{L}_{span}$
(4)

where $\alpha,\beta\in(0,1)$ are control parameters. We use Adam [10] to
optimize the loss.

###  3.5 Selecting Concepts by Random Forest

The second module (concept selector) is built to select desirable concepts
from the candidate spans extracted by BERT-MRC. We argue that it is
unsatisfactory to output the concepts by choosing a specific threshold to
directly truncate BERT-MRC’s output ranking list. If we adopt a relatively big
threshold, we can get more accurate concepts but may lose some correct
concepts. If recall is preferred, precision might be hurt. To achieve a better
trade-off between precision and recall, we adopt a classifer to predict
whether a candidate extracted by BERT-MRC deserves being reserved. Such
concept selector filters out the desirable concepts in terms of the
significant features rather than the span confidence score, and thus improves
the performance of concept filtering.

Specifically, we feed the classifer with the following features. At first, we
adopt the span confidence score (feature A), start position probability
(feature B) and end position probability (feature C) of each candidate span,
which are obtained from BERT-MRC. Furthermore, we use another two features.
Feature D is whether the current candidate span is an existing concept in the
KG. In addition, the most important objective of MRC-CE is to handle the
problem of concept overlap for fine-grained CE. Therefore, we consider another
feature E whether the current span contains other candidate span(s).

Since there are only five simple features, we do not need to choose deep
models. For the traditional classification model, random forest[30] has better
accuracy than many other models, since it adopts ensemble learning. Therefore,
we adopt random forest as the classifier.

We did not directly use the concepts existing in the KG as the training
samples’ real labels of random forest given that many correct concepts not
existing in the KG can be extracted by BERT-MRC. Thus, we invited some
volunteers to label the training samples manually. Specifically, we randomly
sampled 1,000 results from BERT-MRC’s outputs and requested the volunteers to
label whether the result is a correct concept of the corresponding entity.
Then, we saved the features and the labels of the results to constitute the
training samples. The significance values of the five features obtained
through training the random forest towards the Chinese corpus (CN-DBpedia) are
listed in Table 1. From the table, we find that except for feature A, feature
D is the most significant feature to select desirable concepts. It implies
that referring to the existing concepts in KGs is very crucial to extract new
concepts.

Table 1: The significance values of the five features in MRC-CE’s random forest for Chinese KG _CN-DBpedia_. Feature | A | B | C | D | E
---|---|---|---|---|---
Significance | 0.34 | 0.20 | 0.16 | 0.25 | 0.05

###  3.6 Rule-based Concept Pruning

There are still some wrong concepts being reserved after above concept
selection. The errors can be roughly categorized into three classes as below.

1\. The extracted concepts are semantically exclusive with each other. For
example, both ‘president’ and ‘vice president’ could be extracted by BERT-MRC,
but they are mutually exclusive in terms of conceptualization. Obviously, a
president cannot be a vice president simultaneously.

2\. The modifier of a concept is mistakenly extracted as another concept. For
example, ‘railway’ and ‘station’ are both extracted as concepts from the span
‘railway station’.

3\. A correct concept is mistakenly mixed with a functional word. For example,
both ‘is ancient costume drama’ and ‘in high school’ are wrong concepts, where
‘is’ and ‘in’ are the redundant tokens.

In this step, some pruning strategies are executed based on explicitly rules,
since above errors can be easily recognized according to some pre-defined
patterns. For example, for the class that the extracted concepts are mutually
exclusive in semantics, we set our pruning rule case by case rather than
setting a general rule.

##  4 Experiments

In this section, in order to justify our framework’s CE capability for KG
completion, we evaluate its performance upon a Chinese KG _CN-DBpedia_ and an
English KG _Probase_.

###  4.1 Datasets

##### Chinese Dataset

We obtained the latest data of CN-DBpedia from its open
websites222http://kw.fudan.edu.cn/cndbpedia and randomly selected 100,000
entities along with their concepts and abstract texts, as the sample pool.
Then, we randomly selected 500 samples from the sample pool as the test set,
and the rest samples were divided as the training and validation set according
to 9:1, to learn the models. For BERT-MRC, we took each entity’s concepts from
the KG which also exist in the entity’s abstract text as the real labels of
one training sample. For training the random forest, we invited some
volunteers to label whether the candidate concepts from BERT-MRC are the given
entity’s concepts since many correct concepts not existing in the KG can be
extracted by BERT-MRC. When evaluating the models with the test samples, we
invited some volunteers to assess whether the concepts extracted by the models
are correct, given that many new concepts not existing in the KG may be
extracted.

##### English Dataset

Probase333https://www.microsoft.com/en-us/research/project/probase/ has the
hypernym-hyponym relations between concepts and entities, but no abstract
texts of entities. Hence, we first fetched the entities along with their
concepts from Probase, and then crawled entities’ abstract texts from
Wikipedia444https://www.wikipedia.org/. Particularly, we totally sampled
50,000 entities with their concepts and abstract texts. Then, we constructed
the training, validation and test set the same as the Chinese dataset.

###  4.2 Baselines

We compared our MRC-CE with the following five state-of-the-art models to
justify MRC-CE’s performance. Please note that, the models we selected are NER
and Open IE models since NER and Open IE are extraction tasks which are mostly
meet our task scenario. Besides, We also compare the pattern matching
approaches.

We did not compare the methods for ontology extension and the generation
models, since both of them do not meet our task scenario, i.e., text-based CE.
The entity typing models and classification models were also not included
because they can not meet the objective to complete existing KGs with new
concepts. The knowledge-based methods were also excluded since most of them
are only applicable to specific fields.

##### Hearst Patterns [28]

Hearst patterns are the basic and popular rules for extracting concepts from
free texts. We totally designed 5 Hearst patterns to achieve concept
extraction, which are listed in Table 2 555We translate Chinese patterns for
CN-DBpedia into English. We allowed leading and following noun phrases to
match limited modifiers to extract fine-grained concepts.

Table 2: The full set of Hearst patterns for _CN-DBpedia_ and _Probase_. CN-DBpedia | X is Y
---|---
X is a type/a of Y…
X is one of Y
X belongs to Y
Y is located/founded/ in…
Probase | X is a Y that/which/who
X is one of Y
X is a member/part/form… of Y
X refers to Y
As Y, X is …

##### XLNet-NER [44]

With the capability of modeling bi-directional contexts, XLNet demonstrates an
excellent sequence labeling model in many NLP tasks, such as NER and Open IE,
which is also applicable to our task.

##### FLAIR [1]

FLAIR is a novel NLP framework combining different word and document
embeddings to achieve excellent results. FLAIR can be employed in our CE task
since it can extract spans from the text.

##### KVMN [22]

As a sequence labeling model, KVMN was proposed to handle NER by leveraging
different types of syntactic information through attentive ensemble.

##### XLM-R [6]

It is a transformer-based multilingual masked language model incorporating XLM
[11] and RoBERTa [20]. It can achieve CE task since it is also a sequential
labeling model.

##### BERT-BiLSTM-CRF [43, 35]

It is an advanced version of BERT built with BiLSTM and CRF. This language
model is used to obtain optimal token embeddings, based on which the
downstream tasks such as NER and Open IE, CE are well achieved.

###  4.3 Experiment Settings

In MRC-CE, we adopted a BERT-base with 12 layers, 12 self-attention heads and
H = 768 as BERT-MRC’s encoder. The training settings are: batch size = 4,
learning rate = 3e-5, dropout rate = 0.1 and training epoch = 2. In addition,
we set $\alpha=0.3,\beta=0.25$ in Eq. 4 based on parameter tuning. Besides, we
chose 50 as the tree number of the random forest.

###  4.4 Experiment Results

####  4.4.1 CE Performance Comparison

We refer to an extracted concept of a given entity that already exists in the
KG as an _EC_ (existing concept), and refer to an extracted correct concept of
a given entity that does not exist in the KG as a _NC_ (new concept). NCs are
more significant than ECs given that the primary objective of our work is KG
completion, i.e., supplying new instanceOf relations. Please note that a NC in
fact corresponds to a new instanceOf relation to a given entity rather than
being a unique new concept, because the concept linked by this new instanceOf
relation may already exist in the KG as another entity’s concept.

All models’ CE results of the Chinese dataset and the English dataset are
listed in Table 3, where EC # and NC # are the number of existing concepts and
new concepts (instanceOf relations) of the entities, respectively. From the
table we find that, compared with other models, MRC-CE extracts more NCs. As
we claimed before, our framework is capable of extracting fine-grained or
long-tail concepts from the texts. To justify it, we counted the average
number of characters (for Chinese) or words (for English) constituting ECs (EC
length) and NCs (NC length). The larger NC length is, the more possible the NC
is to be fine-grained/long-tail. Although Hearst Patterns’s NC length is
larger than MRC-CE’s in the Chinese dataset, it ignores some coarse-grained
concepts and thus it can not achieve multi-granular concept extraction. For
example, as shown in Table 6, Hearst Patterns only regards ‘the railway
station of JR East Japan’ as a whole concept, whereas MRC-CE can extract
‘station’, ‘railway station’ and ‘the railway station of JR East Japan’
simultaneously from the same span.

Table 3: CE performance comparisons of 500 test samples.

Dataset | Model | EC # | NC # | EC | NC | OC | Prec. | R-Recall | R-F1
---|---|---|---|---|---|---|---|---|---
length | Length | ratio
| Hearst | 158 | 222 | 2.63 | 7.1 | NA | 95.24% | 27.24% | 42.36%
| Patterns
| XLNet-NER | 391 | 46 | 2.61 | NA | 89.92% | 5.64% | 10.62%
| FLAIR | 405 | 63 | 3.11 | NA | 95.51% | 7.73% | 14.30%
CN- | KVMN | 247 | 253 | 4.03 | NA | 64.27% | 31.04% | 41.86%
DBpedia | XLM-R | 89 | 250 | 5.35 | NA | 76.35% | 30.67% | 43.77%
| BERT- | 411 | 25 |  | 4.32 | NA | 88.26% | 3.07% | 5.93%
| BiLSTM-CRF |
| MRC-CE | 519 | 323 |  | 4.91 | 29.35% | 92.22% | 39.63% | 55.44%
| Hearst | 308 | 402 | 1.25 | 2.04 | NA | 95.56% | 20.18% | 33.32%
| Patterns
| XLNet-NER | 534 | 398 | 1.42 | NA | 92.00% | 19.98% | 32.83%
| FLAIR | 307 | 141 | 1.67 | NA | 84.69% | 7.08% | 13.06%
Probase | KVMN | 186 | 404 | 1.96 | NA | 47.50% | 20.28% | 28.43%
| XLM-R | 672 | 322 | 1.48 | NA | 81.74% | 16.16% | 26.99%
| BERT- | 191 | 154 |  | 1.68 | NA | 81.18% | 7.73% | 14.12%
| BiLSTM-CRF |
| MRC-CE | 636 | 626 |  | 2.31 | 36.82% | 90.08% | 31.38% | 46.54%

To prove MRC-CE’s capability of extracting overlapped concepts, we further
recorded the ratio of overlapped concepts (one is another one’s subsequence)
to all extracted concepts, denoted as OC ratio in Table 3. The precision
(Prec.) is the ratio of the correct concepts assessed by the volunteers to all
concepts extracted by the model. FLAIR obtains the highest Prec. mainly due to
its high precision of ECs, but it is howbeit meaningless to KG completion. The
denominator of recall is the number of all new (correct) concepts in the input
text. Since it is difficult to know all new concepts in the input text except
for the costly human assessment, we report the relative recall (R-Recall) to
measure the new concept extraction ability of the models. Specifically, the
new concepts extracted by all models are regarded as the overall NCs. Then,
the relative recall is calculated as NC # divided by the number of overall
NCs. Accordingly, the relative F1 (R-F1) can also be calculated with Prec. and
R-Recall. The results of Table 3 show that MRC-CE can gain satisfactory
precision and recall simultaneously.

####  4.4.2 Ablation Study

We further display the results of ablation study on the Chinese dataset, to
investigate the effectiveness of each module in MRC-CE. At first, we took the
BERT-MRC with fixed threshold truncation (FTT) as one ablated variant of our
framework[13], denoted as BERT-MRC+FTT. In this variant, we simply chose the
spans output by BERT-MRC that have the confidence score higher than 0.8, as
the extracted concepts. Furthermore, we respectively appended the rest two
modules to BERT-MRC, to propose another two ablated variants, denoted as BERT-
MRC+RF and BERT-MRC+RULE.

Table 4: CE results of ablation study.

Dataset | Model | EC # | NC # | Prec. | R-Recall | R-F1
---|---|---|---|---|---|---
CN-DBpedia | BERT-MRC+FTT | 522 | 311 | 87.96% | 83.83% | 85.84%
BERT-MRC+RF | 519 | 323 | 90.83% | 87.06% | 88.91%
BERT-MRC+RULE | 522 | 311 | 91.84% | 83.83% | 87.65%
BERT-MRC+RF+RULE | 519 | 323 | 92.22% | 87.06% | 89.57%
Probase | BERT-MRC+FTT | 646 | 502 | 79.56% | 71.21% | 75.15%
BERT-MRC+RF | 636 | 626 | 89.44% | 88.79% | 89.12%
BERT-MRC+RULE | 646 | 502 | 84.72% | 71.21% | 77.38%
BERT-MRC+RF+RULE | 636 | 626 | 90.08% | 88.79% | 89.43%

The comparison results of all variants and MRC-CE (BERT-MRC+RF+RULE) are
listed in Table 4. The results show that Prec. drops 1.39% and 0.64% without
RULE (BERT-MRC+RF vs. BERT-MRC+RF+RULE), as well as R-Recall drops 3.23% and
17.58% without RF (BERT-MRC+RULE vs. BERT-MRC+RF+RULE) in CN-DBpedia and
Probase, respectively. It proves that MRC-CE can obtain better performance
with RF and RULE since they are complementary to each other.

Meanwhile, we investigate different BERT-based encoders’ influence on CE
performance. To this end, we replaced BERT with RoBERTa [20] in Concept
Extractor module. Then, RF and RULE were also adopted to filter out the
candidate spans output by RoBERTa-MRC. The comparison results are shown in
Table 5, showing that although the variants with RoBERTa gain higher Prec.,
they are inferior to the indicators with BERT on other metrics. It is possibly
due to that, RoBERTa tends to extract the spans the same as the concepts
existing in the KG from the texts, and thus it is easy to ignore the new
concepts.

Table 5: Comparisons between RoBERTa and BERT in CN-DBpedia.

Encoder | Model | EC# | NC# | EC | NC | OC | Prec. | R-Recall. | R-F1.
---|---|---|---|---|---|---|---|---|---
length | Length | ratio
RoBERTa | +FTT | 403 | 105 | 2.63 | 4.97 | 4.75% | 92.87% | 25.80% | 40.38%
+RF | 404 | 119 | 4.92 | 6.69% | 89.71% | 29.24% | 44.10%
+RULE | 403 | 105 | 4.91 | 4.73% | 95.31% | 25.80% | 40.61%
+RF+RULE | 404 | 119 | 5.01 | 6.93% | 94.92% | 29.24% | 44.71%
BERT | +FTT | 522 | 311 | 2.63 | 4.72 | 27.74% | 87.96% | 76.41% | 81.78%
+RF | 519 | 323 | 4.75 | 27.26% | 90.83% | 79.36% | 84.71%
+RULE | 522 | 311 | 4.74 | 27.83% | 91.84% | 76.41% | 83.42%
+RF+RULE | 519 | 323 | 4.91 | 29.35% | 92.22% | 79.36% | 85.31%

####  4.4.3 Case Study

Table 6: The CE results of two specific entities along with their existing
concepts in the KG.

Dataset | Entity | Abstract text | Model | Correct extracted concept
---|---|---|---|---
|  |  | Existing concepts | location, station
|  |  | XLNet-NER | station
|  | Prince station is the | FLAIR | station
CN- | Prince | railway station of | KVMN | railway station
DBpedia | Station | JR East Japan | XLM-R | railway station
|  | and Tokyo Metro. | BERT-BiLSTM-CRF | station
|  |  | Hearst Patterns | the railway station of JR East Japan
|  |  | MRC-CE | station, railway station,
|  |  | the railway station of JR East Japan
|  | Franklin Delano Roosevelt | Existing concepts | figure, president, leader, politician
|  | was an American politician | XLNet-NER | president,leader
|  | who served as the 32nd | figure, politician
| Franklin | president. He became a | FLAIR | politician
| Delano | central figure in world | KVMN | politician
Probase | Roosevelt | events during the first half | XLM-R | figure, president, leader, politician
|  | of the 20th century.As a | BERT-BiLSTM-CRF | politician
|  | dominant leader of his party. | Hearst Patterns | American politician
|  | He built New Deal Coalition. | MRC-CE | figure, politician, leader,
|  |  | president, American politician

We further delve into the concepts extracted by MRC-CE and baselines through
some case studies. Table 6 lists the correct concepts extracted by the models
along with the existing concepts of two entities666 _Prince Station_ ’s
abstract text and CE results were translated from Chinese.. It shows that MRC-
CE extracts more fine-grained and overlapped concepts from the texts than the
baselines.

##  5 Applications

We further demonstrate MRC-CE’s advantages on the KG completion of some real
applications.

###  5.1 KG Completion

Table 7: The statistics of extracted concepts for all entities in CN-DBpedia.

Data | instanceOf | Unique | NC # | Avg. NC # | Avg. concept # | Avg. character #
---|---|---|---|---|---|---
type | relation # | concept # |  | per entity | per entity | per concept
Original | 11,494,627 | 270,025 | - | - | 2.04 | 3.62
Extracted | 9,021,805 | 894,689 | 7,053,900 | 3.16 | 5.20 | 4.92

After training MRC-CE, we ran it for all entities in CN-DBpedia to supply
substantial new instanceOf relations including unique new concepts. Please
note that the instanceOf relations only focus on the relations between the
entity and concept. The related statistics of CE results are listed in Table
7. Please note that the extracted concepts counted in the table include some
wrong concepts. According to the table, MRC-CE extracts more than 7,053,900
new instanceOf relations (NC #). In addition, the extracted concepts have more
characters than the original concepts existing in CN-DBpeida. The results
justify MRC-CE’s capability of extracting large-scale multi-granular concepts
for KG completion.

###  5.2 Domain Concept Acquisition

Our MRC-CE have achieved excellent results concept acquisition on the abstract
texts of given entities. In order to verify MRC-CE’s capability of acquiring
concepts for a certain domain, we also employed MRC-CE in the domain of
Meituan which is a famous Chinese e-business platform. Specifically, we
collected 117,489 Food & Delight entities in Meituan along with their
descriptive texts from CN-DBpedia. After running MRC-CE, we got 458,497 new
instanceOf relations from the texts, and the CE precision is 78.0% based on
human assessments on 100 samples, justifying that MRC-CE can be successfully
applied to domain concept acquisition.

##  6 Conclusion

In this paper, we propose a concept extraction framework MRC-CE to achieve
large-scale multi-granular concept completion for existing KGs. MRC-CE is
capable of extracting massive multi-granular concepts from entities’
descriptive texts. In our framework, a BERT-based MRC model with a pointer
network is built to handle the problems of concept overlap. Meanwhile, a
random forest and rule-based pruning are also employed to obtain satisfactory
concept extraction (CE) precision and recall simultaneously. Our extensive
experiments have justified that our MRC-CE not only has excellent CE
performance, but also is competent to acquire large-scale concepts for
multilingual KGs. Furthermore, MRC-CE makes a great contribution to supply
sufficient concepts for the Chinese KG _CN-Dbpedia_.

## References

  * [1] Akbik, A., Bergmann, T., Blythe, D., Rasul, K., Schweter, S., Vollgraf, R.: Flair: An easy-to-use framework for state-of-the-art nlp. In: Proc. of NAACL. pp. 54–59 (2019)
  * [2] Alomari, S., Abdullah, S.: Improving an ai-based algorithm to automatically generate concept maps. Computer and Information Science 12(4), 72 (2019)
  * [3] Auer, S., Bizer, C., Kobilarov, G., Lehmann, J., Cyganiak, R., Ives, Z.G.: Dbpedia: A nucleus for a web of open data. In: Proc. of ISWC (2007)
  * [4] Bai, H., Xing, F.Z., Cambria, E., Huang, W.B.: Business taxonomy construction using concept-level hierarchical clustering. arXiv preprint arXiv:1906.09694 (2019)
  * [5] Budin, G.: Ontology-driven translation management. Knowledge systems and translation (2005)
  * [6] Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., Grave, E., Ott, M., Zettlemoyer, L., Stoyanov, V.: Unsupervised cross-lingual representation learning at scale. arXiv preprint arXiv:1911.02116 (2019)
  * [7] Cui, W., Xiao, Y., Wang, W.: KBQA: an online template based question answering system over freebase. In: Proc. of IJCAI (2016)
  * [8] Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
  * [9] Ji, J., Chen, B., Jiang, H.: Fully-connected lstm–crf on medical concept extraction. International Journal of Machine Learning and Cybernetics pp. 1–9 (2020)
  * [10] Kingma, J., Ba, J.: Adam: A method for stochastic optimization. In: Proc. of ICLR (2015)
  * [11] Lample, G., Conneau, A.: Crosslingual language model pretraining. In: Proc. of NeurIPS (2019)
  * [12] Li, N., Tian, M., Lv, S.: Extracting hierarchical relations between the back-of-the-book index terms. In: Workshop on Chinese Lexical Semantics. pp. 433–443. Springer (2019)
  * [13] Li, X., Feng, J., Meng, Y., Han, Q., Wu, F., Li, J.: A unified mrc framework for named entity recognition. In: Proc. of ACL (2020)
  * [14] Li, X., Yin, F., Sun, Z., Li, X., Yuan, A., Chai, D., Zhou, M., Li, J.: Entity-relation extraction as multi-turn question answering. arXiv preprint arXiv:1905.05529 (2019)
  * [15] Liang, J., Xiao, Y., Wang, H., Zhang, Y., Wang, W.: Probase+: Inferring missing links in conceptual taxonomies. IEEE Trans. Knowl. Data Eng. 29(6), 1281–1295 (2017)
  * [16] Liang, J., Xiao, Y., Wang, H., Zhang, Y., Wang, W.: Probase+: Inferring missing links in conceptual taxonomies. IEEE TKDE 29(6), 1281–1295 (2017)
  * [17] Liang, J., Zhang, Y., Xiao, Y., Wang, H., Wang, W., Zhu, P.: On the transitivity of hypernym-hyponym relations in data-driven lexical taxonomies. In: Proc. of AAAI. vol. 31 (2017)
  * [18] Liao, J., Sun, F., Gu, J.: Combining concept graph with improved neural networks for chinese short text classification. In: Proc. of JIST. pp. 205–212. Springer (2019)
  * [19] Liu, S., Zhang, X., Zhang, S., Wang, H., Zhang, W.: Neural machine reading comprehension: Methods and trends. Applied Sciences 9(18), 3698 (2019)
  * [20] Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., Stoyanov, V.: Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019)
  * [21] Nguyen, A.D., Nguyen, K.H., Ngo, V.V.: Neural sequence labeling for vietnamese pos tagging and ner. In: Proc. of IEEE-RIVF. pp. 1–5. IEEE (2019)
  * [22] Nie, Y., Tian, Y., Song, Y., Ao, X., Wan, X.: Improving named entity recognition with attentive ensemble of syntactic information. arXiv preprint arXiv:2010.15466 (2020)
  * [23] Petrucci, G., Rospocher, M., Ghidini, C.: Expressive ontology learning as neural machine translation. JWS 52, 66–82 (2018)
  * [24] Ponzetto, S.P., Strube, M.: Wikitaxonomy: A large scale knowledge resource. In: Proc. of ECAI. vol. 178, pp. 751–752. Citeseer (2008)
  * [25] Poria, S., Hussain, A., Cambria, E.: Emosenticspace: Dense concept-based affective features with common-sense knowledge. In: Multimodal Sentiment Analysis, pp. 85–116. Springer (2018)
  * [26] Preum, S.M., Shu, S., Alemzadeh, H., Stankovic, J.A.: Emscontext: Ems protocol-driven concept extraction for cognitive assistance in emergency response. In: Proc. of AAAI. pp. 13350–13355 (2020)
  * [27] Qiu, J., Chai, Y., Tian, Z., Du, X., Guizani, M.: Automatic concept extraction based on semantic graphs from big data in smart city. IEEE Transactions on Computational Social Systems 7(1), 225–233 (2019)
  * [28] Roller, S., Kiela, D., Nickel, M.: Hearst patterns revisited: Automatic hypernym detection from large text corpora. In: Proc. of ACL (2018)
  * [29] Ruan, D.r., He, X.y., Li, D.y., Gao, K.: Modeling and extracting hyponymy relationships on chinese electric power field content. In: 2016 8th International Conference on Modelling, Identification and Control (ICMIC). pp. 439–443. IEEE (2016)
  * [30] Sammut, C., Webb, G.I. (eds.): Encyclopedia of Machine Learning and Data Mining. Springer (2017). https://doi.org/10.1007/978-1-4899-7687-1
  * [31] Seo, M., Kembhavi, A., Farhadi, A., Hajishirzi, H.: Bidirectional attention flow for machine comprehension. arXiv preprint arXiv:1611.01603 (2016)
  * [32] Sharma, R., Gopalani, D., Meena, Y.: Concept-based approach for research paper recommendation. In: Proc. of PRIS. pp. 687–692. Springer (2017)
  * [33] Shen, Y., Huang, P.S., Gao, J., Chen, W.: Reasonet: Learning to stop reading in machine comprehension. In: Proc. of ACM SIGKDD. pp. 1047–1055 (2017)
  * [34] Shvets, A., Wanner, L.: Concept extraction using pointer-generator networks. arXiv preprint arXiv:2008.11295 (2020)
  * [35] Song, Y., Tian, S., Yu, L.: A method for identifying local drug names in xinjiang based on bert-bilstm-crf. Automatic Control and Computer Sciences 54(3), 179–190 (2020)
  * [36] Suchanek, F.M., Kasneci, G., Weikum, G.: Yago: a core of semantic knowledge. In: Proc. of WWW (2007)
  * [37] Vinyals, O., Fortunato, M., Jaitly, N.: Pointer networks. In: Proc. of NIPS. pp. 2692–2700 (2015)
  * [38] Wei, Z., Su, J., Wang, Y., Tian, Y., Chang, Y.: A novel hierarchical binary tagging framework for joint extraction of entities and relations. arXiv preprint arXiv:1909.03227 (2019)
  * [39] Wu, W., Li, H., Wang, H., Zhu, K.Q.: Probase: A probabilistic taxonomy for text understanding. In: Proc. of ACM SIGMOD. pp. 481–492 (2012)
  * [40] Xu, B., Luo, Z., Huang, L., Liang, B., Xiao, Y., Yang, D., Wang, W.: Metic: Multi-instance entity typing from corpus. In: Proc. of CIKM. pp. 903–912 (2018)
  * [41] Xu, B., Xu, Y., Liang, J., Xie, C., Liang, B., Cui, W., Xiao, Y.: Cn-dbpedia: A never-ending chinese knowledge extraction system. In: Proc. of IEA-AIE. pp. 428–438. Springer (2017)
  * [42] Xu, B., Zhang, Y., Liang, J., Xiao, Y., Hwang, S.w., Wang, W.: Cross-lingual type inference. In: Proc. of DASFAA. pp. 447–462. Springer (2016)
  * [43] Yang, Y., Shen, X., Wang, Y.: Bert-bilstm-crf for chinese sensitive vocabulary recognition. In: Proc. of ISICA. pp. 257–268. Springer (2019)
  * [44] Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R.R., Le, Q.V.: Xlnet: Generalized autoregressive pretraining for language understanding. In: Proc. of NIPS. pp. 5753–5763 (2019)
  * [45] Yao, J., Cui, B., Cong, G., Huang, Y.: Evolutionary taxonomy construction from dynamic tag space. WWW 15(5-6), 581–602 (2012)
  * [46] Yilahun, H., Abdurahman, K., Imam, S., Hamdulla, A.: Automatic extraction of uyghur domain concepts based on multi-feature for ontology extension. IET Networks 9(4), 200–205 (2020)
  * [47] Zhao, G., Zhang, X.: Domain-specific ontology concept extraction and hierarchy extension. In: Proc. of NLPIR. pp. 60–64 (2018)


# PHENIX photons and dileptons

Takao Sakaguchi, for the PHENIX Collaboration  Brookhaven National Laboratory,
Physics Department

###### Abstract

Electro-magnetic probes such as dileptons and photons are strong probes to
investigate the thermodynamical state of the early stages of collisions since
they leave the system unscathed. The PHENIX experiment has measured both
photons and dileptons in p+p, d+Au and Au+Au collisions. An excess of dilepton
yield over the expected hadronic contribution is seen in 0.2-0.8 GeV/$c^{2}$
in Au+Au collisions, which is prominent in lower $p_{T}$ and most central.
Direct photons are measured through their internal conversion to electron
pairs. We saw a large enhancement in Au+Au collisions over p+p yield scaled by
the number of binary collisions. It turned out from the latest results on d+Au
collisions that this enhancement is not explainable by a nuclear effect.

###### keywords:

##  1 Introduction

Many intriguing phenomena have been observed at RHIC where a hot and dense
matter is expected to be created. The large suppression of the yield of the
single hadrons at high transverse momentum ($p_{T}$) suggested that the matter
is so dense to stem fast partons with large $Q^{2}$ emerged from the very
early stage in the medium [1]. The large elliptic flow of particles and its
scaling in terms of kinetic energy of the particles suggests that the system
is locally in equilibrium at as early as 0.3 fm/c. Although the hadronic
probes already exhibited many interesting phenomena, since they are suffered
from strong interactions in later stages to some extent, observation of more
direct and penetrating probes have been desired.

Electro-magnetic probes such as photons or dileptons are ideal in this sense,
since they interact with medium or other particles only electro-magnetically,
once produced [2]. Therefore, these probes are of interest already from the
beginning of the history of the heavy ion collisions. At the leading order,
the production processes of photons are annihilation
($q\bar{q}\rightarrow\gamma g$) and Compton scattering ($qg\rightarrow\gamma
q$). Their yields are proportional to $\alpha\alpha_{s}$, which are $\sim$40
times lower than the ones from strong interaction. The dilepton production is
mainly from annihilation of quark and anti-quark
($q\bar{q}\rightarrow\gamma^{*}$), and the yield is proportional to
$\alpha^{2}$. Except for some difference in production processes, photons and
dileptons can be treated as same observables. A theoretical calculation tells
that photons and dileptons share similar emission sources [3]. Fig 1 shows a
mapping of photons and dileptons in one coordinate.

Figure 1: Photons and dileptons in one coordinate with three degrees of
freedom.

PHENIX has measured dileptons and photons since the RHIC has started its
running in 2000. In this paper, we show the most recent results on dileptons
and photons in p+p, d+Au and Au+Au collisions at $\sqrt{s_{NN}}$=200 GeV.

##  2 Dilepton analysis

The PHENIX detector has a capability of measuring momentum of charged
particles with high accuracy and of identifying electrons with high efficiency
and strong rejection power of hadrons [4]. In measuring dileptons, we have a
huge combinatorial background $e^{+}e^{-}$ arising from random combination of
electrons from photons converted at beam pipe, Dalitz decays of $\pi^{0}$ or
$\eta$. We constructed combinatorial background by combining electrons or
positrons from different events, and subtracted them from foreground mass
distributions.

The subtracted distributions still include unphysical correlated pairs from
back-to-back jets. We estimated the contribution using PYTHIA event generator
followed by a detector simulation, and subtracted it off from the invariant
mass distribution. The signal to background ratio is $\sim 10^{-2}$ in total.
The resulting distribution is corrected for efficiencies of single electrons,
and compared to various hadronic contributions.

##  3 Low mass and low $p_{T}$ dileptons in Au+Au collisions

Fig 2(a) shows the invariant mass distribution of electron pairs measured in
Au+Au collisions at $\sqrt{s_{NN}}$=200GeV [5]. The data is compared with
electron pairs from various hardonic sources calculated using a Monte Carlo
and filtered by the PHENIX acceptance. A large excess is seen in 0.2-0.8
GeV/$c^{2}$ in Minimum bias Au+Au collisions. In order to study this excess in
detail, we divided the data set into various $p_{T}$, mass and centrality
bins. Fig. 2(b) and (c) show the mass spectra in various $p_{T}$ and
centrality bins, respectively.

Figure 2: (a, left) Invariant mass distributions in Minimum bias Au+Au events
with hadronic cocktail calculation, and the ones in (b, middle) various
$p_{T}$ regions and in (c, right) centrality classes.

The excess dies out at higher $p_{T}$ and lower centralities, which suggests
that the excess is contributed from a thermal source. We made summary plots
for the detail studies as shown in Fig. 3(a) and (b).

Figure 3: (a, left) Yield and (b, right) slope parameters of dileptons in low
mass and low $p_{T}$ region. For slope parameters, solid points are calculated
from truncated means of the mass distributions, while open points are from
exponential fits to the distributions.

As we already saw in the invariant mass distributions, the yield of excess
region increases as collisions become more central. The slope parameters are
low for low $m_{T}-m_{0}$ and high for high $m_{T}-m_{0}$, implying that the
local slopes are roughly proportional to the energy of photons emitted. This
is another evidence of that they are from thermal sources.

##  4 Low mass and high $p_{T}$ dileptons in Au+Au and p+p

Turning the region of interest to higher $p_{T}$, where $p_{T}>>M$, the yield
of dileptons are considered to be dominated by internal conversion of real
photons as depicted in Fig 4(a).

Figure 4: (a, left ) Diagram of real photons and internal conversion of
photons. (b, right) Fit to data with direct photon and hadronic cocktail
distributions.

Taking this advantage, we performed direct photon measurement through dilepton
measurement [6]. The invariant mass distribution of Dalitz decay of $\pi^{0}$,
$\eta$ and direct photons can be calculated using Kroll-Wada formula as shown
below.

$\frac{1}{N_{\gamma}}\frac{dN_{ee}}{dm_{ee}}=\frac{2\alpha}{3\pi}\sqrt{1-\frac{4m_{e}^{2}}{m_{ee}^{2}}}\left(1+\frac{2m_{e}^{2}}{m_{ee}^{2}}\right)\frac{1}{m_{ee}}|F(m_{ee}^{2})|^{2}\left(1-\frac{m_{ee}^{2}}{M^{2}}\right)^{3}$

Then, the real invariant mass distribution is fitted with a function of:

$F=(1-r)f_{c}+rf_{d}$

where $f_{c}$ is the cocktail calculation, $f_{d}$ is the mass distribution
for direct photons, and $r$ is the free parameter in the fit. Using Kroll-Wada
formula, $r$ is converted to direct photon to inclusive photon ratio as
follows:

$r=\frac{\gamma^{*}_{dir}(m_{ee}>0.15)}{\gamma^{*}_{inc}(m_{ee}>0.15)}\propto\
\frac{\gamma^{*}_{dir}(m_{ee}\approx 0)}{\gamma^{*}_{inc}(m_{ee}\approx 0)}\
=\frac{\gamma_{dir}}{\gamma_{inc}}\equiv r_{\gamma}$

Once $r_{\gamma}$ is obtained, the invariant yield of direct photons are
calculated as $\gamma_{inc}\times r_{\gamma}$. The procedure demonstrated in
Fig 4(b) is for 1.0$<p_{T}<$1.5 GeV/c. The dotted lines show the contributions
from various hadrons, the solid blue is the sum of these contributions, and
the solid red line show the distribution by direct photons converted
internally. The $r$ value is determined by the fit to the data. The error of
the fit corresponds to the statistical error. We applied the procedure for
various $p_{T}$s and centralities in p+p and Au+Au collisions, and obtained
the $p_{T}$ spectra as shown in Fig 5(a).

Figure 5: (a, left) Direct photon yields in p+p and Au+Au collisions. (b,
right) Comparison of the yield in 0-20 % Au+Au collisions with various
theoretical models.

The result is obtained from 810 million Minimum bias Au+Au events collected in
RHIC Year-4 run, and 1.5 billion minimum bias events and 270 million high
$p_{T}$ single electron trigger events collected in RHIC Year-5 p+p run. The
distributions are for 0-20 %, 20-40 % centrality and Minimum Bias collisions
in case of Au+Au collisions. The distributions were then fitted with the p+p
fit plus exponential function. We obtain slopes and dN/dy for three
centralities as shown in Table 1.

Table 1: dN/dy and slopes of direct photon $p_{T}$ distributions obtained from the fit to the data with the p+p fit plus exponential function. Centrality | dN/dy ($p_{T}>$1 GeV/c) | Slope (MeV) |  $\chi^{2}$/DOF
---|---|---|---
0-20 % | 1.50$\pm$0.23$\pm$0.35 | 221$\pm$19$\pm$19 | 4.7/4
20-40 % | 0.65$\pm$0.08$\pm$0.15 | 217$\pm$18$\pm$16 | 5.0/4
MinBias | 0.49$\pm$0.05$\pm$0.11 | 233$\pm$14$\pm$19 | 3.2/4

There are many models in the market that have a wide range of initial
temperatures and thermalization times. Fig 5(b) shows the comparison of the
data with the models. Since the initial temperature and thermalization time
are highly correlated, the comparison of data and models does not help
constraining two parameters to definite numbers.

##  5 Low mass and high $p_{T}$ dileptons in d+Au

It has been a question whether or not the excess is purely from the source
that only exists in Au+Au collisions. A possible effect that increases the the
yield in nucleus collisions is $k_{T}$ broadening, or so-called Cronin effect.
In order to quantify the effect, we analyzed d+Au data from RHIC Year-8 run
with the same procedure. Fig. 6(a) shows the invariant mass distribution in
d+Au collisions for the lowest $p_{T}$ where the direct photon yield is
significant. After repeating over $p_{T}$ bins, we obtained the $r_{\gamma}$
as a function of $p_{T}$. Fig 6(b) shows the $r_{\gamma}$ in various collision
systems. The lines show the ratios calculated using the direct photon
contributions estimated by NLO pQCD prediction scaled by the number of binary
collisions ($N_{coll}$). Three lines show the predictions with three mass
scales ($\mu=p_{T}/2,p_{T},2p_{T}$). The p+p ratio is very close to the NLO
pQCD prediction, while Au+Au data largely deviates in low $p_{T}$ from the
prediction. The d+Au data shows a moderate excess, but the excess is not as
much as the one in Au+Au collisions.

Figure 6: (a, left) Invariant mass distribution together with the calculation
for hadronic cocktail and direct photons. The procedure to obtain
$r^{{}^{\prime}}$ is described in analysis section. (b, right) Ratios of
$\gamma_{dir}$ to $\gamma_{inc}$ for p+p, d+Au and Au+Au collisions.

From the ratios obtained, we calculated the direct photon spectra, again
following the procedure explained previously. Fig 7(a) shows the direct photon
spectra for d+Au Minimum bias events, together with the real photon result
from RHIC Year-3 run.

Figure 7: (a, left) Direct photon yield in d+Au collisions from this analysis
together with Run3 real photon result and p+p yield scaled by $N_{coll}$. (b,
right) The same data with NLO pQCD calculation, instead of p+p yield.

A comparison is made with direct photon spectra from p+p collisions scaled by
$N_{coll}$. Fig. 7(b) shows a comparison of data with NLO pQCD calculation
scaled by $N_{coll}$. In both cases, the yield in d+Au collisions are higher
than the one expected from p+p yield, suggesting that the nuclear effect is
seen in d+Au collisions.

The ultimate interest in the series of the direct photon measurements is
whether or not the excess seen in Au+Au collisions can be explained by cold
nuclear effect (Cronin effect). In order to evaluate this, we compared the
Au+Au yield with the d+Au yield scaled by the difference of $N_{coll}$ as
shown in Fig 8(a).

Figure 8: (a, left) Direct photon yields in Au+Au and d+Au collisions scaled
by the difference of $N_{coll}$ in both collision systems. (b, right) Ratio of
Au+Au yield to d+Au yield scaled by $N_{coll}$.

As seen in the plot, the Au+Au yield is higher than the one in d+Au in
$p_{T}<$2.5 GeV/c, which is close to what is expected in a literature [3].
When we divide Au+Au by $N_{coll}$-scaled d+Au data, we clearly see the
existence of an additional effect in Au+Au collisions (Fig. 8(b)).

Since both statistical and systematic uncertainties are still very large in
d+Au measurement, it is hard to exhibit a concrete quantitative conclusion.
However, in order to visually improve the result, we tried to parameterize the
d+Au points by using a fit function. Fig 9(a) shows a fit to the d+Au points
and Fig 9(b) shows the fit scaled by $N_{coll}$ and Au+Au points.

Figure 9: (a, left) Fit to the d+Au invariant yield, and (b, right) comparison
of the fit and Au+Au yield.

The excess in Au+Au for $p_{T}<$2.5GeV/c still looks to be definite compared
to the yield in d+Au, implying that the excess is caused by a source created
in hot dense medium.

##  6 Toward better measurement of dileptons

The statistical uncertainty and a part of systematic uncertainty in dilepton
measurement are governed by the a background arising from random combinations
of electrons from photons converted at beam pipe and Dalitz decay of $\pi^{0}$
and $\eta$. If we could tag and remove such electrons from foreground electron
pair measurement, the errors would be reduced significantly.

We developed a hadron blind detector (HBD) to realize this tagging. The
detector is windowless Čerenkov type detector with CF4 gas in its radiator. We
operate the detector in magnetic field free region in order to look for an
opening angle of electron pairs. The principle of operation is following; if
we see a cluster that has charge corresponding to the number of photo-
electrons for single electrons, we keep the track associated with the cluster.
If we see a cluster with double of the number of the photon-electrons, we
reject the tract associated with the cluster, assuming that this is
contributed by two electrons, which are likely by photon conversion or Dalitz
decay. The detail of the detector description and its performance can be found
in [7]. Fig. 10 shows the invariant mass distributions with and without using
HBD information from Year-9 p+p run at $\sqrt{s}$=500GeV.

Figure 10: Dilepton invariant mass spectra (a, left) without HBD matching and
(b, right) with HBD matching and a charge cut.

The signal to background ratios in $\phi$, $\omega$ and $\rho$ meson region
are significantly improved after applying a charge cut in HBD. We are planning
to also apply the charge cut in HBD in Au+Au collisions in RHIC Year-10 run
data. It is essential in Au+Au collisions because the particle multiplicity is
much higher than that in d+Au or p+p.

##  7 Summary

Electro-magnetic probes such as dileptons and photons are strong probes to
investigate the thermodynamical state of the early stages of collisions since
they leave the system unscathed. The PHENIX experiment has measured both
photons and dileptons in p+p, d+Au and Au+Au collisions. An excess of dilepton
yield over the expected hadronic contribution is seen in 0.2-0.8 GeV/$c^{2}$
in Au+Au collisions, which is prominent in lower $p_{T}$ and most central. The
invariant slopes are low for low $m_{T}-m_{0}$ and high for high
$m_{T}-m_{0}$, implying that the local slopes are roughly proportional to the
energy of photons emitted. Direct photons are measured through their internal
conversion to electron pairs. We saw a large enhancement in Au+Au collisions
over p+p yield scaled by the number of binary collisions. It turned out from
the latest results on d+Au collisions that this enhancement is not explainable
by a nuclear effect.

## References

  * [1] K. Adcox et al. [PHENIX Collaboration], Nucl. Phys. A 757, 184 (2005) [arXiv:nucl-ex/0410003].
  * [2] P. Stankus, Ann. Rev. Nucl. Part. Sci. 55, 517 (2005).
  * [3] S. Turbide, R. Rapp and C. Gale, Phys. Rev. C 69, 014903 (2004) [arXiv:hep-ph/0308085].
  * [4] K. Adcox et al. [PHENIX Collaboration], Nucl. Instrum. Meth. A 499, 469 (2003).
  * [5] A. Adare et al. [PHENIX Collaboration], Phys. Rev. C 81, 034911 (2010) [arXiv:0912.0244 [nucl-ex]].
  * [6] A. Adare et al. [PHENIX Collaboration], Phys. Rev. Lett. 104, 132301 (2010) [arXiv:0804.4168 [nucl-ex]].
  * [7] M. Makek (PHENIX coll.), this proceedings.