# UVIT Observations of UV-Bright Stars in four Galactic Globular Clusters

Ranjan Kumar1 Ananta C. Pradhan1 M. Parthasarathy2 Devendra K. Ojha3 Abhisek
Mohapatra1 Jayant Murthy2 1Dept. of Physics and Astronomy, National Institute
of Technology, Rourkela - 769 008, India
email<EMAIL_ADDRESS>
2Indian Institute of Astrophysics, Bangalore - 560 034, India
3Dept. of Astronomy and Astrophysics, Tata Institute of Fundamental Research
(TIFR), Mumbai - 400 005, India

(2019)

###### Abstract

We have performed photometric analysis of four Galactic globular clusters
(GGCs): NGC 4147, NGC 4590, NGC 5053 and NGC 7492 using far-UV and near-UV
filters of the Ultraviolet Imaging Telescope (UVIT) on-board AstroSat. With
the help of color-magnitude diagrams (CMDs), we have identified $\sim$ 150
blue horizontal branch stars (BHBs), and $\sim$ 40 blue straggler stars (BSS)
in the four GGCs. We study the temperature and radial distribution of BHBs and
BSS for the four GGCs.

###### keywords:

globular clusters: individual (NGC 4147, NGC 4590, NGC 5053 and NGC 7492),
stars: horizontal-branch, techniques: photometric

††volume: 351††journal: Star Clusters: From the Milky Way to the Early
Universe††editors: A. Bragaglia, M.B. Davies, A. Sills & E. Vesperini, eds.

##  1 Introduction

The ultraviolet (UV) light in old stellar populations of Galactic globular
clusters (GGCs) is dominated by hot stars, such as blue horizontal branch
stars (BHBs), blue hook stars (BHk), extreme blue horizontal branch stars
(eBHBs), blue straggler stars (BSS), post asymptotic giant branch stars (pAGB)
and hot sub-dwarfs (sdB, sdO) (Rood et al., 1998). UV emission is strongly
dependent on the source temperature, therefore hot BHBs, among others are the
strongest contributors to the far-UV flux of GGCs. Far-UV colors are well co-
related with horizontal branch morphology in GGCs and have been studied vastly
using Hubble Space Telescope (HST), Galaxy Evolution Explorer (GALEX) and
Ultraviolet Imaging Telescope (UVIT) observations (see Schiavon et al., 2012;
Dalessandro et al., 2012; Lagioia et al., 2015; Piotto et al., 2015; Milone et
al., 2015; Subramaniam et al., 2017; Sahu et al., 2019). GGCs in halo region
of Milky Way are less crowded in UV and hence, their observation in UV regime
allows one to probe and separate out the bright stars effectively. Schiavon et
al. (2012) have presented UV CMDs of 44 GGCs using GALEX observations and
Piotto et al. (2015) have done the same for 58 GGCs from HST observations. The
relative contributions of the various types of stars and the factors that
might lead to larger or smaller populations of UV-Bright stars have remained
an open question (Greggio & Renzini, 1990; Dorman et al., 1995; Lee et al.,
2002; Ambika et al., 2004; Jasniewicz et al., 2004; Sohn et al., 2006;
Dalessandro et al., 2012). We present UV photometric study of four GGCs: NGC
4147, NGC 4590, NGC 5053 and NGC 7492 with the observations taken from UVIT
on-board AstroSat (Tandon et al., 2017).

##  2 Observation and Data Reduction

The UVIT on-board AstroSat is performing observations in FUV (130-180 nm) and
NUV (200-300 nm) bands, each having five filters with resolutions better than
$1.8^{\prime\prime}$ since its launch in 2015 (Tandon et al., 2017). We have
observed four GGCs, NGC 4147, NGC 4590, NGC 5053 and NGC 7492, using both the
FUV and NUV filters of UVIT. The observational details are given in Table 1.
We reduced the data and produced good quality images of the clusters using a
customized software package CCDLAB (Postma, & Leahy, 2017). We performed the
crowded field photometry on the images using DAOPHOT package (Stetson, 1987)
available in IRAF. We have done the interstellar extinction correction on
observed sources using Cardelli et al. (1989) extinction law considering
E(B-V) value for each cluster from Schlegel et al. (1998) extinction map (see
Table 2). We considered sources with AB magnitude limit up to 22.5 and 23.0 in
FUV and NUV, respectively. We cross-matched the UVIT observed sources with
GAIA DR2 catalogue (Gaia Collaboration et al., 2018) to get proper motion
(PMRA, PMDEC) of observed sources. We separated out the cluster members from
the field stars by selecting $1\sigma$ Gaussian distribution from mean PMRA
and PMDEC as cluster members. The number of sources we obtained in FUV (NUV)
as cluster members for GGCs, NGC 4590, NGC 5053, and NGC 7492 are 71(1875),
30(539) and 29(178), respectively. Since NGC 4147 has observation only with
FUV filters, we could extract 45 cluster members in the FUV band.

Table 1: UVIT observation log of four GGCs

Telescope FUV NUV Filter Name BaF2 Sapphire Silica NUVB15 NUVB13 NUVB4
$\lambda_{eff.}$ of filters (in $\mathrm{\SIUnitSymbolAngstrom}$) 1541 1608
1717 2196 2447 2632 Cluster Name ($gl,gb$) in degrees Exposure Time in seconds
NGC 4147 ($252.85^{\circ},77.19^{\circ}$) 1536 1648 1209 - - - NGC 4590
($299.62^{\circ},36.05^{\circ}$) 860 - 1406 1221 1739 1607 NGC 5053
($335.70^{\circ},78.95^{\circ}$) - 1423 217 - 1503 817 NGC 7492
($53.39^{\circ},-63.48^{\circ}$) - - 850 - 3276 3014

##  3 Color Magnitude Diagram

Table 2: Cluster parameters of GGCs and extracted UV-Bright sources from CMDs.

Cluster parameters of GGCs Number of UV-Bright sources GGCs Gal. lat.
Metallicity Distance E(B-V) age BHBs BSS eBHB in degree [Fe/H] (kpc) (mag)
(Gyr) FUV NUV FUV NUV NGC 7492 $-63.48^{\circ}$ -1.80 26.3 0.0310 12.0 28 28 5
5 - NGC 5053 $78.95^{\circ}$ -2.27 17.4 0.0149 12.3 25 35 2 29 1 NGC 4590
$36.05^{\circ}$ -2.23 10.3 0.0523 12.5 57 97 3 7 - NGC 4147 $77.19^{\circ}$
-1.80 19.3 0.0221 13.5 34 - 4 - -

(a) (b)

Figure 1:  Fig. 1(a) shows the FUV($BaF_{2}$) - NUV(B4) vs FUV($BaF_{2}$) and
Fig. 1(b) shows the NUV(B4) - Gaia(G) vs NUV(B4) CMDs of NGC 4590. Various
stellar populations present in the cluster are denoted with different symbols:
filled circles show the BHBs, filled squares show BSS, lower triangles show
AGBs, upper triangles show RGBs, and MS+SGBs are shown with open circles. Gray
horizontal line in Fig. 1(b) shows the MS-turn off point in NUV(B4).

After getting the catalog of sources for each cluster we have classified
various stellar populations by plotting CMDs using various FUV, NUV and
optical filters. We have shown the CMD, FUV($BaF_{2}$) - NUV(B4) vs
FUV($BaF_{2}$) (Fig. 1(a)), and CMD, NUV(B4) - Gaia (G) vs NUV(B4) (Fig.
1(b)), for one of the clusters NGC 4590. The stellar populations present in
the cluster were separated by identifying their various regions in the UV CMDs
previously suggested by Schiavon et al. (2012); Subramaniam et al. (2017);
Sahu et al. (2019). We can see that in FUV only BHBs and BSS are visible (Fig.
1(a)), whereas in NUV BHBs, BSS, RGBs, SGBs and MSs are clearly visible (Fig.
1(b)). We see the main sequence turn-off at 20.3 ABmag in NUV(B4) filter (Fig.
1(b)). We are able to see the MS branch in NUV band which is not visible in
the FUV band. We have seen a similar trend of distribution of sources in the
CMDs of other clusters as well. A list of UV-Bright sources extracted for the
four GGCs are given in Table 2.

##  4 Temperature and Radial Distribution of BHBs and BSS

Figure 2: Temperature distribution of BHBs and BSS observed in the four GGCs.

There have been several studies of temperature distribution of BHBs and BSS of
GGCs in UV using GALEX (Schiavon et al., 2012), HST (Lagioia et al., 2015) and
UVIT (Sahu et al., 2019) observations. Schiavon et al. (2012); Lagioia et al.
(2015) have used zero age horizontal branch (ZAHB) models to extract effective
temperature ($T_{eff}$) from the UV CMDs, whereas Sahu et al. (2019) have used
the color-temperature relation from the Kurucz stellar atmosphere model
(Castelli, & Kurucz, 2003) and the SED fitting on the BHBs and BSS sources. In
order to extract out $T_{eff}$ of all the observed BHBs and BSS in our
observed four GGCs, we have used the color-temperature relation from Kurucz
stellar atmosphere model (Castelli, & Kurucz, 2003). Based on the color
combinations used in FUV and NUV CMDs for all the clusters, we generated
theoretical colors for a range of temperature (4,000K to 30,000K) using
cluster parameters of observed GGCs given in Table 2 and a surface gravity,
log(g)=4.0 for BHBs and BSS (suggested by Lagioia et al., 2015; Sahu et al.,
2019, for BHBs). Then observed colors and theoretical colors were matched,
within $\sigma_{color}\leq 0.01$ which have max $\Delta T=100K$, to extract
the corresponding $T_{eff}$ for each BHB star and BSS. The $T_{eff}$
distribution of 152 BHBs and 42 BSS observed with FUV and NUV filters of UVIT
are plotted in Fig. 2. The $T_{eff}$ of BHBs ranges from 8,500K to 17,000K,
whereas the $T_{eff}$ of BSS ranges from 9,500K to 12,000K. The temperature
distribution shows that there are two groups of BHBs present in the observed
GGCs. We also see a gap between 11,500K and 12,000K, which suggests the
Grundahl-jump in temperature distribution of BHBs (Grundahl et al., 1999).

We took the core radius, half light radius and the tidal radius from the
updated catalogue of GGCs (Harris, 2010) and then studied the radial
distribution of cluster members observed in FUV and NUV bands. We found that
the FUV emissions arise only from the BHBs and BSS sources which are
concentrated within half light radius of the clusters. The NUV emissions arise
from BHBs, BSS, RGBs and SGBs and their radial distribution goes up to the
tidal radius of clusters. A detailed analysis on radial distribution and
density of cluster members for all clusters will be done in forthcoming
papers.

Acknowledgements: This research is supported by ISRO RESPOND PROJECT No.
ISRO/RES/2/409/17-18. This publication uses data from the AstroSat mission of
the Indian Space Research Organization (ISRO), archived at the Indian Space
Science Data Centre (ISSDC).

## References

  * Ambika et al. (2004) Ambika, S., Parthasarathy, M., Aoki, W., et al. 2004, A&A, 417, 293
  * Cardelli et al. (1989) Cardelli, J. A., Clayton, G. C., & Mathis, J. S. 1989, ApJ, 345, 245
  * Castelli, & Kurucz (2003) Castelli, F., & Kurucz, R. L. 2003,Modelling of Stellar Atmospheres, A20
  * Dalessandro et al. (2012) Dalessandro, E., Schiavon, R. P., Rood, R. T., et al. 2012, AJ, 144, 126
  * Dorman et al. (1995) Dorman, B., O’Connell, R. W., & Rood, R. T. 1995, ApJ, 442, 105
  * Gaia Collaboration et al. (2018) Gaia Collaboration, Brown, A. G. A., Vallenari, A., et al. 2018, A& A, 616, A1
  * Greggio & Renzini (1990) Greggio, L., & Renzini, A. 1990, ApJ, 364, 35
  * Grundahl et al. (1999) Grundahl, F., Catelan, M., Landsman, W. B., Stetson, P. B., & Andersen, M. I. 1999, ApJ, 524, 242
  * Harris (2010) Harris, W. E. 2010, arXiv e-prints, arXiv:1012.3224
  * Jasniewicz et al. (2004) Jasniewicz, G., de Laverny, P., Parthasarathy, M., et al. 2004, A&A, 423, 353
  * Lagioia et al. (2015) Lagioia, E. P., Dalessandro, E., Ferraro, F. R., et al. 2015, ApJ, 800, 52
  * Lee et al. (2002) Lee, H.-c., Lee, Y.-W., & Gibson, B. K. 2002, AJ, 124, 2664
  * Milone et al. (2015) Milone, A. P., Marino, A. F., Piotto, G., et al. 2015, ApJ, 808, 51
  * Piotto et al. (2015) Piotto, G., Milone, A. P., Bedin, L. R., et al. 2015, AJ, 149, 91
  * Postma, & Leahy (2017) Postma, J. E., & Leahy, D. 2017, PASP, 129, 115002
  * Rood et al. (1998) Rood, R. T., Dorman, B., Ferraro, F. R., Paltrinieri, B., & Fusi Pecci, F. 1998, Ultraviolet Astrophysics Beyond the IUE Final Archive, 413, 515
  * Sahu et al. (2019) Sahu, S., Subramaniam, A., Côté, P., Rao, N. K., & Stetson, P. B. 2019, MNRAS, 482, 1080
  * Schiavon et al. (2012) Schiavon, R. P., Dalessandro, E., Sohn, S. T., et al. 2012, AJ, 143, 121
  * Schlegel et al. (1998) Schlegel, D. J., Finkbeiner, D. P., & Davis, M. 1998, ApJ, 500, 525
  * Sohn et al. (2006) Sohn, S. T., O’Connell, R. W., Kundu, A., et al. 2006, AJ, 131, 866
  * Stetson (1987) Stetson, P. B. 1987, PASP, 99, 191
  * Subramaniam et al. (2017) Subramaniam, A., Sahu, S., Postma, J. E., et al. 2017, AJ, 154, 233
  * Tandon et al. (2017) Tandon, S. N., Subramaniam, A., Girish, V., et al. 2017, AJ, 154, 128


# OneRel: Joint Entity and Relation Extraction with One Module in One Step

Yu-Ming Shang1, Heyan Huang1,2, Xian-Ling Mao1
Corresponding author.

###### Abstract

Joint entity and relation extraction is an essential task in natural language
processing and knowledge graph construction. Existing approaches usually
decompose the joint extraction task into several basic modules or processing
steps to make it easy to conduct. However, such a paradigm ignores the fact
that the three elements of a triple are interdependent and indivisible.
Therefore, previous joint methods suffer from the problems of cascading errors
and redundant information. To address these issues, in this paper, we propose
a novel joint entity and relation extraction model, named OneRel, which casts
joint extraction as a fine-grained triple classification problem.
Specifically, our model consists of a scoring-based classifier and a relation-
specific horns tagging strategy. The former evaluates whether a token pair and
a relation belong to a factual triple. The latter ensures a simple but
effective decoding process. Extensive experimental results on two widely used
datasets demonstrate that the proposed method performs better than the state-
of-the-art baselines, and delivers consistent performance gain on complex
scenarios of various overlapping patterns and multiple triples.

## Introduction

Extracting pairs of entities and their relations in the form of (head,
relation, tail) or $(h,r,t)$ from unstructured text is an important task in
natural language processing and knowledge graph construction. Traditional
pipeline approaches (Zelenko, Aone, and Richardella 2003; Zhou et al. 2005;
Chan and Roth 2011) treat entity recognition and relation classification as
two separate sub-tasks. Although flexible, pipeline methods ignore the
interactions between the two sub-tasks and are susceptible for the problem of
error propagation (Li and Ji 2014). Therefore, recent studies focus on
building joint models to obtain entities together with their relations through
a unified architecture.

Figure 1:  The extraction processes of existing approaches and our method. The
dotted arrow indicates the dependencies between triple elements. Note that
there are various extraction paradigms in multi-module multi-step approaches,
e.g., $(h,t)\rightarrow r$, $h\rightarrow r\rightarrow t$,
$r\rightarrow(h,t)$. We just use $h\rightarrow r\rightarrow t$ to illustrate
the shortcomings of this kind of methods.

To make the complex task easy to conduct, existing studies usually decompose
the joint extraction into several basic modules or processing steps (Yu et al.
2020; Zhao et al. 2021b). As shown in Figure 1, according to the extraction
procedure of triple elements, these approaches fall into two categories:
multi-module multi-step and multi-module one-step. The first category utilizes
different modules in the framework of cascading classification (Fu, Li, and Ma
2019; Yuan et al. 2020; Wei et al. 2020; Zheng et al. 2021; Zhao et al. 2021a,
b) or text generation (Zeng et al. 2018; Zeng, Zhang, and Liu 2020; Ye et al.
2021) to obtain entities and relations step-by-step. Although promising, this
kind of models suffers from the problem of cascading errors since mistakes in
early steps may affect the prediction results of later steps. The second
category attempts to identify entities and relations separately, and then
combine them into triples based on their latent correlations (Wang et al.
2020; Sui et al. 2020; Wang et al. 2021). However, due to insufficient mutual
constraints between entities and relations in the separate recognition
process, such methods tend to produce redundant information, leading to errors
when assembling triples (Zheng et al. 2017).

In fact, the fundamental reason for the above problems is that the
decomposition-based paradigm ignores an important property of a triple — its
head entity, relation and tail entity are interdependent and indivisible. In
other words, it is unreliable to extract one element without fully perceiving
the information of the other two elements. To fill this gap, we try to
accomplish the joint extraction task from the perspective of triple
classification. For example, as shown in Figure 1, “Bionico” and “Mexico” are
two words in the sentence and Country is a pre-defined relation, all of them
are visible in training data. Intuitively, the triple (Bionico, Country,
Mexico) can be directly identified by judging its correctness. The idea brings
three advantages as follows. First, head entity, relation and tail entity are
simultaneously fed into one classification module, making it possible to fully
capture the dependencies between triple elements, thereby reducing redundant
information. Second, only one-step classification is used, which is able to
effectively avoid the cascading errors. Third, the simple architecture of one-
module one-step empowers the network straightforward and easy to train.

Inspired by the above idea, in this paper, we propose a novel joint entity and
relation extraction model, named OneRel, which is capable of extracting all
triples from unstructured text with one module in one step. Considering that
an entity may consist of multiple tokens, we design a scoring-based classifier
and cast the joint extraction task into a fine-grained triple classification
problem. Specifically, for a token pair $(w_{i},w_{j})$ and a pre-defined
relation $r_{k}$, the scoring-based classifier measures the correctness of the
combination $(w_{i},r_{k},w_{j})$, which will be assigned with a meaningful
tag if it is valid and “-” otherwise. To this end, for an input sentence, the
output of OneRel is a three-dimensional matrix with each entry corresponding
to the classification result of $(w_{i},r_{k},w_{j})$. In order to decode
entities and relations from the output matrix accurately and efficiently, we
introduce a novel relation-specific horns tagging (Rel-Spec Horns Tagging for
short) strategy to determine the boundary tokens of head entities and tail
entities. Experimental results on two widely used benchmark datasets prove
that the proposed method outperforms previous approaches and achieves the
state-of-the-art performance.

In summary, our contributions are as follows:

  * •

We provide a novel perspective to transform joint extraction into fine-grained
triple classification, making it possible to capture the information of head
entities, relations and tail entities at the same time.

  * •

Following our perspective, we introduce a novel scoring-based classifier and a
Rel-Spec Horns Tagging strategy. The former is responsible for parallel
tagging, and the latter ensures efficient decoding.

  * •

We evaluate our model on two public datasets, and the results indicate that
our method performs better than state-of-the-art baselines, especially for
complex scenarios of overlapping triples.

## Related Work

Existing joint methods can be roughly divided into two classes according to
their extraction procedure of triple elements:

The first class is multi-module multi-step, which uses different modules and
interrelated processing steps to extract entities and relations serially. For
example, a line of works first identify all entities in a sentence, and then
perform relation classification between every entity pairs (Katiyar and Cardie
2017; Tan et al. 2019; Fu, Li, and Ma 2019; Liu et al. 2020). The second line
of works first detect the relations expressed by a sentence rather than
preserve all redundant relations; then head entities and tail entities are
predicted (Zeng et al. 2018; Yuan et al. 2020; Zheng et al. 2021; Ma, Ren, and
Zhang 2021). The third line of works first distinguish all head entities, and
then inference corresponding relations and tail entities via sequence labeling
or question answering (Wei et al. 2020; Yu et al. 2020; Zhao et al. 2021a, b;
Ye et al. 2021). Despite their success, the multi-module multi-step methods
suffer from the problem of cascading errors, as the mistakes in early steps
cannot be corrected in later steps.

The second class is multi-module one-step, which extracts entities and
relations in parallel, and then combines them into triples. For example, Miwa
and Bansal (2016); Zhang, Zhang, and Fu (2017); Wang et al. (2020, 2021) treat
entity recognition and relation classification as a table-filling problem,
where each entry represents the interaction between two individual words. Sui
et al. (2020) formulate the joint extraction task as a set prediction problem,
avoiding considering the prediction order of multiple triples. However, due to
insufficient mutual constraints between entities and relations in the separate
recognition process, such multi-module one-step approaches cannot fully
capture the dependencies between predicted entities and relations, resulting
in redundant information during triple construction.

Different from existing methods, in this paper, we propose to treat the joint
extraction task as a fine-grained triple classification problem, which is able
to extract triples from sentences in a one-module one-step manner. Therefore,
the aforementioned cascading errors and redundant information can be greatly
addressed. Besides, the classical model Novel-Tagging (Zhang, Zhang, and Fu
2017) designs a complex tagging strategy to establish connections between
entities and relations, and can also identify triples from sentences in one-
step. Nevertheless, this technique cannot handle overlapping cases because it
assumes every entity pair holds at most one relation.

Figure 2:  Examples of the Rel-Spec Horns Tagging. For the convenience to
explanation, we illustrate the sub-matrix with a given relation, e.g., Located
in. So, the matrix $\bm{M}$ degenerates into two dimensions, where the rows
represent head entities and the columns represent tail entities.

## Method

In this section, we first give the task definition and notations. Then, we
introduce our Rel-Spec Horns Tagging strategy and its decoding algorithm.
Finally, we provide a detailed formalization of the scoring-based classifier.

### Task Definition

Given a sentence $\mathcal{S}=\\{w_{1},w_{2},...,w_{L}\\}$ with $L$ tokens and
$K$ predefined relations $\mathcal{R}=\\{r_{1},r_{2},...,r_{K}\\}$. The
purpose of joint entity and relation extraction is to identify all possible
triples $\mathcal{T}=\\{(h_{i},r_{i},t_{i})\\}_{i=1}^{N}$ in $\mathcal{S}$,
where $N$ is the number of triples, $h_{i},t_{i}$ are the head entity and tail
entity composed of several consecutive tokens, i.e., entity.span $=w_{p:q}$,
where $w_{p:q}$ refers to the concatenation of $w_{p}$ to $w_{q}$.

Note that different triples may share overlapping entities, which poses a big
challenge for joint extraction task (Zeng et al. 2018).

### Relation Specific Horns Tagging

For a sentence, we design a classifier to assign tags for all possible
$(w_{i},r_{k},w_{j})$ combinations, where
$w_{i},w_{j}\in\mathcal{S},r_{k}\in\mathcal{R}$. We maintain a three-
dimensional matrix $\bm{M}^{L\times K\times L}$ to store the classification
results (Tagging). Therefore, in the test phase, our task is to decode
entities and relations from the matrix $\bm{M}$ (Decoding).

#### Tagging

We employ the “BIE” (Begin, Inside, End) signs to indicate the position
information of a token in entities. For example, “HB” means the beginning
token of the head entity, “TE” means the end token of the tail entity. As
shown in Figure 2 (a), for a sentence that expresses the triple (New York
City, Located in, New York State), there are nine special tags (blue tags) in
the relation-specific sub-matrix $\bm{M}_{r=Located\;in}$.

According to the insight that an entity can be determined by detecting its
boundary tokens (Wei et al. 2020), four types of tags are used in our tagging
strategy: (1) HB-TB. This tag refers to that both positions are respectively
the beginning tokens of a paired head and tail conditioned on a specific
relation. For example, there is a relation Located in between the two entities
“New York City” and “New York State”, therefore, the classification tag of the
combination (“New”, Located in, “New”) is assigned with the tag “HB-TB”. (2)
HB-TE. This tag means that the token corresponding to the row is the beginning
of a head entity, meanwhile the token corresponding to the column is the end
of a tail entity. For instance, “New” is the start token of “New York City”
and “State” is the end token of “New York State”, so the combination of
(“New”, Located in, “State”) is assigned with the tag “HB-TE”. (3) HE-TE. This
tag shares a similar logic with “HB-TB”, which means two positions are
respectively the end tokens of a paired head entity and tail entity
conditioned on a specific relation. For example, the combination of (“City”,
Located in, “State”) is assigned with “HE-TE”. (4) “-”. All cells other than
the above three cases will be marked as “-”. As we can see from Figure 2 (b)
and (c), because only the three corners of the rectangle need to be labeled,
we vividly name this method as Rel-Spec Horns Tagging.

Obviously, the tagged matrix $\bm{M}$ is sparse, which has the following
advantages: First, using three instead of nine special tags can effectively
narrow down the potential searching space when conducting classification.
Second, a sparse $\bm{M}$ means that there are sufficient negative samples
during the training process. Third, the sparsity of $\bm{M}$ ensures the
simplicity and efficiency of triple elements decoding.

Furthermore, our Rel-Spec Horns Tagging can naturally address the complex
scenarios with overlapping patterns. Specifically, for EntityPairOverlap (EPO)
case, entity pairs will be labeled in different sub-matrices according to
their relations. For example in Figure 2 (a) and (b), (New York City, Located
in, New York State) and (New York State, Contains, New York City) are two EPO
triples, thus, the two entity pairs are marked in $\bm{M}_{r=Located\;in}$ and
$\bm{M}_{r=Contains}$, respectively. For SingleEntityOverlap (SEO) scenario,
if two triples contain the same relation, the two entity pairs will be marked
in different parts of $\bm{M}_{r=i}$, otherwise they will be labeled in
different sub-matrices according to their relations. For the most complicated
HeadTailOverlap (HTO111HTO is also called SOO when a triple is represented by
(subject, relation, object).) pattern, e.g. the triple (New York City, City
Name, New York) in Figure 2 (c), entity pairs (red tags) are located near the
diagonal of $\bm{M}_{r=City\;name}$ and can still be easily decoded.

#### Decoding

The tagged matrix $\bm{M}^{L\times K\times L}$ marks the boundary tokens of
paired head entities and tail entities, as well as the relations between them.
Therefore, decoding triples from $\bm{M}$ becomes easy and straightforward.
That is, for each relation, the spans of head entities are spliced from “HB-
TE” to “HE-TE”; the spans of tail entities are spliced from “HB-TB” to “HB-
TE”; and two paired entities share the same “HB-TE”.

### Scoring-based Classifier

For an input sentence, we employ a pre-trained BERT (Devlin et al. 2019) as
sentence encoder to capture the $d$-dimensional token embedding $\bm{e}_{i}$
for each token:

$\\{\bm{e}_{1},\bm{e}_{2},...,\bm{e}_{L}\\}=BERT(\\{\bm{x}_{1},\bm{x}_{2},...,\bm{x}_{L}\\}),$
(1)

where $\bm{x}_{i}$ is the input representation of each token. It is the
summation over the corresponding token embedding and positional embedding.

Then, we enumerate all possible $(\bm{e}_{i},\bm{r}_{k},\bm{e}_{j})$
combinations and design a classifier to assign high-confidence tags, where
$\bm{r}_{k}$ is the randomly initialized relation representation. Intuitively,
we can employ a simple classification network whose input is
$(\bm{e}_{i},\bm{r}_{k},\bm{e}_{j})$ to achieve this goal. However, this
intuition has two flaws: On the one hand, a simple classifier is not only
unable to fully explore the interactions between entities and relations, but
also difficult to model the inherent structural information of triples. On the
other hand, using $(\bm{e}_{i},\bm{r}_{k},\bm{e}_{j})$ as input means the
model needs performing at least $L\times K\times L$ calculations to classify
all combinations, which is unacceptable in terms of time.

Inspired by knowledge graph embedding techniques, we borrow the idea from HolE
(Nickel, Rosasco, and Poggio 2016), whose score function is defined as:

$f_{r}(h,t)=\bm{r}^{T}(\bm{h}\star\bm{t}),$ (2)

where $\bm{h}$, $\bm{t}$ are head and tail representations, respectively.
$\star$ means circular correlation, which is used to mine the latent
dependencies between two entities. Here, we redefine the $\star$ operator as a
non-linear concatenation projection:

$\bm{h}\star\bm{t}=\phi(\bm{W}[\bm{h};\bm{t}]^{T}+\bm{b}),$ (3)

where $\bm{W}\in\mathbb{R}^{d_{e}\times 2d}$, $\bm{b}$ are trainable weight
and bias, $d_{e}$ denotes the dimension of entity pair representations. $[;]$
is the concatenation operation and $\phi(\cdot)$ is the ReLU activation
function. The new definition brings the following benefits: First, the score
function of our classifier can be seamlessly connected with the output of
sentence encoder. Second, the mapping function from entity features to entity
pair representations can be learned adaptively via the matrix $\bm{W}$. Third,
the concatenation between two entities is not commutative, i.e.,
$[\bm{h};\bm{t}]\neq[\bm{t};\bm{h}]$, which is indispensable for modeling
asymmetric relations.

Next, we use all relation representations $\bm{R}\in\mathbb{R}^{d_{e}\times
4K}$ to simultaneously compute the salience of $(w_{i},r_{k},w_{j})_{k=1}^{K}$
for a token-pair $(w_{i},w_{j})$ at once, where 4 is the number of
classification tags. Therefore, the final score function of our method is
defined as:

$\bm{v}_{(w_{i},r_{k},w_{j})_{k=1}^{K}}=\bm{R}^{T}\phi(\texttt{drop}(\bm{W}[\bm{e}_{i};\bm{e}_{j}]^{T}+\bm{b})),$
(4)

where $\bm{v}$ is the score vector, $\texttt{drop}(\cdot)$ denotes the dropout
strategy (Srivastava et al. 2014) that is used to prevent over-fitting. As a
result, we achieve the parallel scoring with only two layers of fully
connected networks (the matrix $\bm{R}$ can also be regarded as a trainable
weight), and reduce the processing steps of actual implementation to $L\times
1\times L$, even better than TPLinker (Wang et al. 2020). Moreover, the score
function conforms the idea of HolE and is capable of capturing the correlation
and mutual exclusion between relations, which will be verified in the
experiments.

Finally, we feed the score vector of $(w_{i},r_{k},w_{j})$ into a softmax
function to predict corresponding tags:

$P(\mathrm{y}_{(w_{i},r_{k},w_{j})}|\mathcal{S})=Softmax(\bm{v}_{(w_{i},r_{k},w_{j})})$
(5)

The objective function of OneRel is defined as:

$\displaystyle\mathcal{L}_{triple}=$ $\displaystyle-\frac{1}{L\times K\times
L}\times$ (6) $\displaystyle\sum_{i=1}^{L}\sum_{k=1}^{K}\sum_{j=1}^{L}\log
P(\mathrm{y}_{(w_{i},r_{k},w_{j})}=g_{(w_{i},r_{k},w_{j})}|\mathcal{S}),$

where $g_{(w_{i},r_{k},w_{j})}$ denotes the gold tag obtained from
annotations.

## Experiments

In this section, extensive experiments are conducted to validate the
effectiveness of the proposed OneRel and analyze its properties.

Category | Dataset | Details of Test Set
---|---|---
Train | Valid | Test | Relations | Normal | SEO | EPO | HTO | N=1 | N=2 | N=3 | N=4 | N>5 | Triples
NYT∗ | 56,195 | 4,999 | 5,000 | 24 | 3,266 | 1,297 | 978 | 45 | 3,244 | 1,045 | 312 | 291 | 108 | 8,110
WebNLG∗ | 5,019 | 500 | 703 | 171 | 245 | 457 | 26 | 84 | 266 | 171 | 131 | 90 | 45 | 1,591
NYT | 56,195 | 5,000 | 5,000 | 24 | 3,222 | 1,273 | 969 | 117 | 3,240 | 1,047 | 314 | 290 | 109 | 8,120
WebNLG | 5,019 | 500 | 703 | 216 | 239 | 448 | 6 | 85 | 256 | 175 | 138 | 93 | 41 | 1,607
Table 1: Statistics of datasets. $N$ is the number of triples in a sentence.

### Experimental Settings

#### Datasets and Evaluation Metrics

Following previous works (Wei et al. 2020; Wang et al. 2020; Zheng et al.
2021), we evaluate our model and all baselines on two widely used datasets:
NYT (Riedel, Yao, and McCallum 2010) and WebNLG (Gardent et al. 2017). The
former is generated for distant supervised relation extraction. The latter is
originally created for natural language generation (NLG). Both NYT and WebNLG
have two versions: one version only annotates the last word of entities, and
the other version annotates the whole span of entities. We denote the first
version datasets as NYT∗ and WebNLG∗, and the second version as NYT and
WebNLG. To further study the capability of the proposed OneRel in handling
complex scenarios, we split the test set by overlapping patterns and triple
number. Detailed statistics of the two datasets are described in Table 1.

Three standard evaluation metrics are used in our experiments, i.e., micro
Precision (Prec.), Recall (Rec.) and F1-score (F1). During evaluation, we
adopt Partial Match for NYT∗ and WebNLG∗: an extracted triple (head, relation,
tail) is considered to be correct only if the relation and the last word of
head and tail are all correct; and use Exact Match for NYT and WebNLG: a
predicted triple is regarded to be correct only if the whole span of two
entities and relation are all exactly matched.

#### Implementation Details

In our experiments, all training process is completed on a work station with
an AMD 7742 2.25G CPU, 256G memory, a single RTX 3090 GPU, and Ubuntu 20.04.
For the pre-trained BERT, we reuse the base cased English model released by
Huggingface222https://huggingface.co/bert-base-cased, which contains 12
Transformer blocks and the hidden size $d$ is 768. We tune our model on the
valid set and use grid search to adjust important hyper-parameters.
Specifically, the batch size is set to 8/6 on NYT/WebNLG, and all parameters
are optimized by Adam algorithm (Kingma and Ba 2015) with a learning rate of
1e-5. The dimension of the hidden layer $d_{e}$ is set to $3\times d$, the
dropout probability in equation (4) is 0.1, the max sequence length is set to
100.

#### Baselines

We compare our model with ten state-of-the-art baselines: GraphRel (Fu, Li,
and Ma 2019), RSAN (Yuan et al. 2020), MHSA (Liu et al. 2020), CasRel (Wei et
al. 2020), TPLinker (Wang et al. 2020), SPN (Sui et al. 2020), CGT (Ye et al.
2021), CasDE (Ma, Ren, and Zhang 2021), RIFRE (Zhao et al. 2021a), PRGC (Zheng
et al. 2021).

Note that the sentence encoders used in GraphRel, RSAN and MHSA are LSTM
networks, while other baselines employ a pre-trained BERT to obtain feature
representations. For fair comparison, the reported results for all baselines
are directly from the original literature. We also conduct ablation test:
OneRel- is the model that replaces the classifier of OneRel with
$f(w_{i},r_{k},w_{j})=\bm{W}[\bm{e_{i}};\bm{r_{k}};\bm{e_{j}}]+\bm{b}$.

### Results and Analysis

#### Main Results

Table 2 shows the comparison results of our model against ten baselines on NYT
and WebNLG in terms of Partial Match and Exact Match. It can be observed that
our method, OneRel, outperforms all the ten baselines and achieves the state-
of-the-art F1-score on all datasets. Especially on WebNLG∗ and WebNLG, OneRel
obtains the best performance in terms of all three evaluation metrics, and
improves the three indicators on WebNLG to above 90% for the first time. We
attribute the outstanding performance of OneRel to its two advantages: First,
OneRel solves the joint extraction task from the perspective of fine-grained
triple classification. Thus, the information of entities and relations can be
combined at the same time during extraction, and the redundant information can
also be reduced. Second, the combination of the scoring-based classifier and
the Rel-Spec Horns Tagging accomplishes entity and relation extraction in a
straightforward way, effectively avoiding the problem of cascading errors.

Compared with the representative multi-module multi-step method PRGC, OneRel
achieves 1.3 and 2.5 absolute gain in F1-score on WebNLG∗ and WebNLG,
respectively. This demonstrates that extracting entities and relations
simultaneously in one step can effectively address the cascading errors
problem. Besides, for another multi-module one-step model TPLinker which
employs four independent tags to detect entities and relations, our OneRel
outperforms it by 0.9, 2.4, 0.9 and 4.3 absolute gains on the four datasets,
respectively. Such results confirm that using one module to extract all triple
elements in one step is effective for exploring the interactions between
entities and relations. These all indicates that One-module One-step is
expected to become a new paradigm for completing joint extraction task.

We can also observe that among BERT-based models, OneRel- achieves a
competitive performance with CasRel, TPLinker and CasDE; OneRel obtains the
best performance on all datasets. In addition to BERT, OneRel- and OneRel only
use one and two layers of fully connected network, respectively. Their
architectures are much simpler than most baselines and easier to train.
Besides, the performance of OneRel is much better than that of OneRel- on all
datasets, which reveals that it is crucial to capture the dependencies between
entities and relations when designing a classifier. This conclusion points the
way for us to design stronger models in the future.

Model | Partial Match | Exact Match
---|---|---
NYT∗ | WebNLG∗ | NYT | WebNLG
Prec. | Rec. | F1 | Prec. | Rec. | F1 | Prec. | Rec. | F1 | Prec. | Rec. | F1
GraphRel (Fu, Li, and Ma 2019) | 63.9 | 60.0 | 61.9 | 44.7 | 41.1 | 42.9 | - | - | - | - | - | -
RSAN (Yuan et al. 2020) | - | - | - | - | - | - | 85.7 | 83.6 | 84.6 | 80.5 | 83.8 | 82.1
MHSA (Liu et al. 2020) | 88.1 | 78.5 | 83.0 | 89.5 | 86.0 | 87.7 | - | - | - | - | - | -
CasRel (Wei et al. 2020) | 89.7 | 89.5 | 89.6 | 93.4 | 90.1 | 91.8 | - | - | - | - | - | -
TPLinker (Wang et al. 2020) | 91.3 | 92.5 | 91.9 | 91.8 | 92.0 | 91.9 | 91.4 | 92.6 | 92.0 | 88.9 | 84.5 | 86.7
SPN (Sui et al. 2020) | 93.3 | 91.7 | 92.5 | 93.1 | 93.6 | 93.4 | 92.5 | 92.2 | 92.3 | - | - | -
CGT (Ye et al. 2021) | 94.7 | 84.2 | 89.1 | 92.9 | 75.6 | 83.4 | - | - | - | - | - | -
CasDE (Ma, Ren, and Zhang 2021) | 90.2 | 90.9 | 90.5 | 90.3 | 91.5 | 90.9 | 89.9 | 91.4 | 90.6 | 88.0 | 88.9 | 88.4
RIFRE (Zhao et al. 2021a) | 93.6 | 90.5 | 92.0 | 93.3 | 92.0 | 92.6 | - | - | - | - | - | -
PRGC (Zheng et al. 2021) | 93.3 | 91.9 | 92.6 | 94.0 | 92.1 | 93.0 | 93.5 | 91.9 | 92.7 | 89.9 | 87.2 | 88.5
OneRel- | 91.3 | 90.5 | 90.9 | 93.8 | 91.4 | 92.6 | 91.1 | 90.4 | 90.8 | 90.5 | 88.2 | 89.4
OneRel | 92.8 | 92.9 | 92.8 | 94.1 | 94.4 | 94.3 | 93.2 | 92.6 | 92.9 | 91.8 | 90.3 | 91.0

Table 2: Precision(%), Recall (%) and F1-score (%) of our proposed OneRel and
baselines.

#### Detailed Results on Complex Scenarios

Model | NYT∗ | WebNLG∗
---|---|---
Normal | EPO | SEO | HTO | N=1 | N=2 | N=3 | N=4 | N$\geq$5 | Normal | EPO | SEO | HTO | N=1 | N=2 | N=3 | N=4 | N$\geq$5
CasRel | 87.3 | 92.0 | 91.4 | 77.0§ | 88.2 | 90.3 | 91.9 | 94.2 | 83.7 | 89.4 | 94.7 | 92.2 | 90.4§ | 89.3 | 90.8 | 94.2 | 92.4 | 90.9
TPLinker | 90.1 | 94.0 | 93.4 | 90.1§ | 90.0 | 92.8 | 93.1 | 96.1 | 90.0 | 87.9 | 95.3 | 92.5 | 86.0§ | 88.0 | 90.1 | 94.6 | 93.3 | 91.6
SPN | 90.8 | 94.1 | 94.0 | - | 90.9 | 93.4 | 94.2 | 95.5 | 90.6 | - | - | - | - | 89.5 | 91.3 | 96.4 | 94.7 | 93.8
PRGC | 91.0 | 94.5 | 94.0 | 81.8 | 91.1 | 93.0 | 93.5 | 95.5 | 93.0 | 90.4 | 95.9 | 93.6 | 94.6 | 89.9 | 91.6 | 95.0 | 94.8 | 92.8
OneRel | 90.6 | 95.1 | 94.8 | 90.8 | 90.5 | 93.4 | 93.9 | 96.5 | 94.2 | 91.9 | 95.4 | 94.7 | 94.9 | 91.4 | 93.0 | 95.9 | 95.7 | 94.5

Table 3: F1-score (%) on sentences with different overlapping patterns and
different triple numbers. $\S$ marks the results reported by (Zheng et al.
2021).

To verify the ability of our method in handling overlapping patterns and
multiple triples, we conduct two extended experiments on different subsets of
NYT∗ and WebNLG∗. We select four powerful models as baselines and the detailed
results are shown in Table 3.

It can be observed that our OneRel achieves the best F1-score on 13 of the 18
subsets, especially for the most complex cases HTO and $N\geq 5$. The HTO
pattern contains two situations: one is nested entities that most previous
approaches cannot accurately identify, e.g., the triple (Bruce Lee, Family
name, Lee). The other is that the head entity and tail entity share the same
words, for example, the sentence “Native Americans in the United States are
one of the ethnic groups of the country.” in WebNLG∗ expresses the triple
(States, ethnicGroup, States). Besides, a $N\geq 5$ sentence may contain SEO,
EPO and HTO patterns at the same time, which brings a big challenge to
existing approaches. Nevertheless, our OneRel achieves the best performance on
both HTO and $N\geq 5$ of NYT∗ and WebNLG∗, which adequately proves that our
Rel-Spec Horns Tagging is able to address the overlapping triples problem from
design, and more robust than baselines when dealing with the complicated
scenarios.

#### Results on Different Sub-tasks

Model | Element | NYT∗ | WebNLG∗
---|---|---|---
Prec. | Rec. | F1 | Prec. | Rec. | F1
CasRel | $(h,t)$ | 89.2 | 90.1 | 89.7 | 95.3 | 91.7 | 93.5
$r$ | 96.0 | 93.8 | 94.9 | 96.6 | 91.5 | 94.0
$(h,r,t)$ | 89.7 | 89.5 | 89.6 | 93.4 | 90.1 | 91.8
SPN | $(h,t)$ | 93.2 | 92.7 | 92.9 | 95.0 | 95.4 | 95.2
$r$ | 96.3 | 95.7 | 96.0 | 95.2 | 95.7 | 95.4
$(h,r,t)$ | 93.3 | 91.7 | 92.5 | 93.1 | 93.6 | 93.4
PRGC | $(h,t)$ | 94.0 | 92.3 | 93.1 | 96.0 | 93.4 | 94.7
$r$ | 95.3 | 96.3 | 95.8 | 92.8 | 96.2 | 94.5
$(h,r,t)$ | 93.3 | 91.9 | 92.6 | 94.0 | 92.1 | 93.0
OneRel | $(h,t)$ | 93.3 | 93.4 | 93.3 | 96.2 | 96.5 | 96.3
$r$ | 96.7 | 96.9 | 96.8 | 96.7 | 97.0 | 96.8
$(h,r,t)$ | 92.8 | 92.9 | 92.8 | 94.1 | 94.4 | 94.3
Table 4: Results on triple elements. $(h,t)$ denotes the entity pair and $r$
means the relation.

We further explore the performance of OneRel on different sub-tasks, i.e.,
entity pair recognition and relation classification. From Table 4, it can be
found that our OneRel outperforms all the baselines on most test instances of
NYT∗ and all the indicators of WebNLG∗. These encouraging results once again
verifies our motivation.

Interestingly, the four models show the same trend on the two datasets: for
NYT∗, there is an obvious gap between the F1-score on $(h,t)$ and $r$; for
WebNLG∗, the performance on $(h,t)$ and $r$ are much higher than that of
$(h,r,t)$. Based on this phenomenon, previous researchers have analyzed that
entity pair recognition and triple formation are two bottlenecks of the joint
extraction task (Sui et al. 2020). We believe that in addition to the above
reasons, the characteristic of the datasets is also an important factor.
Concretely, NYT∗ contains lots of EPO triples, which means that the impact of
wrongly recognized entity pairs is much greater than the influence of
mistakenly classified relations. Suppose a sentence that expresses three
triples (Obama, President of, United States), (Obama, Live in, United States),
(Obama, Place of birth, United States). If one relation is wrongly classified,
the recall on $r$ may be 0.67. While if one entity pair is incorrectly
recognized, the recall on $(h,t)$ may drop to 0. In contrast, the proportion
of EPO patterns in WebNLG∗ is much smaller than that of NYT∗ (3.6% vs 19.6%),
thus, the performance of models on entity pair recognition is consistent with
that on relation classification. Besides, WebNLG∗ contains more relations than
NYT∗ (171 vs 24) and some of them are confusing, e.g., Leader and LeaderName.
So, the triple formation on WebNLG∗ is more difficult than NYT∗.

#### Model Efficiency

Dataset | Model | Training Time | Inference Time | F1
---|---|---|---|---
NYT∗ | TPLinker | 1592 | 46.2 | 91.9
OneRel | 1195 | 41.5 | 92.9
WebNLG∗ | TPLinker | 599 | 40.1 | 91.9
OneRel | 88 | 45 | 94.3
Table 5: Comparison of the model efficiency. Training Time (s) means the time
required to train one epoch, Inference Time (ms) is the time to predict
triples of one sentence.

We evaluate the model efficiency with respect to Training Time and Inference
Time of the most similar baseline TPLinker in two datasets NYT∗ and WebNLG∗,
and the results are shown in Table 5. In this experiment, the batch size of
the two models during training and testing are set to 6 and 1, respectively,
and the max length of input sentence is 100. Although the theoretical
complexity of the two models is $\mathcal{O}({KL^{2}})$, OneRel is better than
TPLinker in terms of parallel processing, i.e., OneRel processes $K$ relations
at a time, while TPLinker handles one relation at each step. Therefore, when
the size of relations increases from 24 (NYT∗) to 171 (WebNLG∗), the training
time of OneRel increases from 1.3$\times$ faster than TPLinker to an
astonishing 6.8$\times$. As opposed to what we observed in Training Time, the
F1-score of OneRel is much better than that of TPLinker. This confirms the
efficiency and the learning ability of our proposed classifier. Besides, the
inference time of OneRel is similar to that of TPLinker on NYT∗ and WebNLG∗,
which illustrates the effectiveness and rationality of our proposed Rel-Spec
Horns Tagging.

#### Topology Structure of Relations

Our scoring-based classifier borrows the idea of HolE (Nickel, Rosasco, and
Poggio 2016), and theoretically, it should also be able to learn the
correlation and mutual exclusion between relations. To verify the learning
ability of our classifier, we visualize the relation representations of NYT by
the $t-$SNE (Maaten and Hinton 2008), which is a nonlinear dimensionality
reduction algorithm. We omit 6 long-tail relations that appear less than 50
times in whole training set, and the results are shown in Figure 3. It can be
observed that the topology structure of relations reflects their inherent
connections. For instance, the relations related to people are on the left
region of the figure, while the relations related to location are on the
right. Especially, the spatial positions of place_lived, place_of_birth and
place_of_death are very close, which is in consistent with our common sense.
This property is critical for predicting EPO triples. That is, if the model
predicts the relation place_of_birth for an entity pair, we can infer that the
entity pair may hold another relation place_lived with a high-probability.
These all suggest that our method has learned not only the features of a
specific dataset, but also the general knowledge that conforms to the real
world. Thus, our OneRel may have generalization capabilities.

Figure 3:  (Best viewed in color and zoom in.) Visualization of relations on
NYT dataset.

## Conclusion

In this paper, we provide a novel perspective to transform the joint
extraction task into a fine-grained triple classification problem, and propose
a novel joint model with a scoring-based classifier and Rel-Spec Horns Tagging
strategy to obtain triples with one module in one step, which greatly
alleviates the problems of cascading errors and redundant information.
Experiments on public datasets show that our model performs better than the
state-of-the-art approaches on different scenarios.

In the future, we would like to explore the following directions:

  * •

To improve the efficiency of the model, we design a simplified version of HolE
as the score function. We will next try to design a more efficient and
powerful score function to further strengthen its ability of capturing the
connections between entities and relations.

  * •

We would like to explore the idea of triple classification in other
information extraction problems, such as event extraction.

## Acknowledgments

The work is supported by National Key R&D Plan (No. 2018YFB1005100), National
Natural Science Foundation of China (No. 61751201, 61772076, and 61732005),
Natural Science Fund of Beijing (No. Z181100008918002), Central Leading Local
Project(No. 2020L3024), BDAS National Engineering Laboratory Open Project (No.
CASNDST202006) and Fujian Provincial DSTLST Project (No. 2019H0026).

## References

  * Chan and Roth (2011) Chan, Y. S.; and Roth, D. 2011.  Exploiting Syntactico-Semantic Structures for Relation Extraction.  In _Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies_ , 551–560. Portland, Oregon, USA: Association for Computational Linguistics.
  * Devlin et al. (2019) Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2019.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , 4171–4186. Association for Computational Linguistics.
  * Fu, Li, and Ma (2019) Fu, T.-J.; Li, P.-H.; and Ma, W.-Y. 2019.  GraphRel: Modeling Text as Relational Graphs for Joint Entity and Relation Extraction.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , 1409–1418. Florence, Italy: Association for Computational Linguistics.
  * Gardent et al. (2017) Gardent, C.; Shimorina, A.; Narayan, S.; and Perez-Beltrachini, L. 2017.  Creating Training Corpora for NLG Micro-Planners.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , 179–188. Association for Computational Linguistics.
  * Katiyar and Cardie (2017) Katiyar, A.; and Cardie, C. 2017.  Going out on a limb: Joint Extraction of Entity Mentions and Relations without Dependency Trees.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , 917–928. Vancouver, Canada: Association for Computational Linguistics.
  * Kingma and Ba (2015) Kingma, D. P.; and Ba, J. 2015.  Adam: A Method for Stochastic Optimization.  In _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_.
  * Li and Ji (2014) Li, Q.; and Ji, H. 2014.  Incremental Joint Extraction of Entity Mentions and Relations.  In _Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , 402–412. Baltimore, Maryland: Association for Computational Linguistics.
  * Liu et al. (2020) Liu, J.; Chen, S.; Wang, B.; Zhang, J.; Li, N.; and Xu, T. 2020.  Attention as Relation: Learning Supervised Multi-head Self-Attention for Relation Extraction.  In _Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI 2020_ , 3787–3793.
  * Ma, Ren, and Zhang (2021) Ma, L.; Ren, H.; and Zhang, X. 2021.  Effective Cascade Dual-Decoder Model for Joint Entity and Relation Extraction.  _CoRR_ , abs/2106.14163.
  * Maaten and Hinton (2008) Maaten, L. v. d.; and Hinton, G. 2008.  Visualizing data using t-SNE.  _Journal of machine learning research_ , 9(Nov): 2579–2605.
  * Miwa and Bansal (2016) Miwa, M.; and Bansal, M. 2016.  End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , 1105–1116. Association for Computational Linguistics.
  * Nickel, Rosasco, and Poggio (2016) Nickel, M.; Rosasco, L.; and Poggio, T. 2016.  Holographic embeddings of knowledge graphs.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 30.
  * Riedel, Yao, and McCallum (2010) Riedel, S.; Yao, L.; and McCallum, A. 2010.  Modeling Relations and Their Mentions without Labeled Text.  In _Machine Learning and Knowledge Discovery in Databases, European Conference, ECML PKDD 2010, Barcelona, Spain, September 20-24, 2010, Proceedings, Part III_ , volume 6323, 148–163. Springer.
  * Srivastava et al. (2014) Srivastava, N.; Hinton, G. E.; Krizhevsky, A.; Sutskever, I.; and Salakhutdinov, R. 2014.  Dropout: a simple way to prevent neural networks from overfitting.  _J. Mach. Learn. Res._ , 15(1): 1929–1958.
  * Sui et al. (2020) Sui, D.; Chen, Y.; Liu, K.; Zhao, J.; Zeng, X.; and Liu, S. 2020.  Joint Entity and Relation Extraction with Set Prediction Networks.  _arXiv preprint arXiv:2011.01675_.
  * Tan et al. (2019) Tan, Z.; Zhao, X.; Wang, W.; and Xiao, W. 2019.  Jointly Extracting Multiple Triplets with Multilayer Translation Constraints.  In _The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019_ , 7080–7087. AAAI Press.
  * Wang et al. (2021) Wang, Y.; Sun, C.; Wu, Y.; Zhou, H.; Li, L.; and Yan, J. 2021.  UniRE: A Unified Label Space for Entity Relation Extraction.  In Zong, C.; Xia, F.; Li, W.; and Navigli, R., eds., _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021_ , 220–231.
  * Wang et al. (2020) Wang, Y.; Yu, B.; Zhang, Y.; Liu, T.; Zhu, H.; and Sun, L. 2020.  TPLinker: Single-stage Joint Extraction of Entities and Relations Through Token Pair Linking.  In _Proceedings of the 28th International Conference on Computational Linguistics_ , 1572–1582. International Committee on Computational Linguistics.
  * Wei et al. (2020) Wei, Z.; Su, J.; Wang, Y.; Tian, Y.; and Chang, Y. 2020.  A Novel Cascade Binary Tagging Framework for Relational Triple Extraction.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , 1476–1488. Association for Computational Linguistics.
  * Ye et al. (2021) Ye, H.; Zhang, N.; Deng, S.; Chen, M.; Tan, C.; Huang, F.; and Chen, H. 2021.  Contrastive Triple Extraction with Generative Transformer.  In _Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Virtual Event, February 2-9, 2021_ , 14257–14265. AAAI Press.
  * Yu et al. (2020) Yu, B.; Zhang, Z.; Shu, X.; Liu, T.; Wang, Y.; Wang, B.; and Li, S. 2020.  Joint Extraction of Entities and Relations Based on a Novel Decomposition Strategy.  In _ECAI 2020 - 24th European Conference on Artificial Intelligence_ , volume 325 of _Frontiers in Artificial Intelligence and Applications_ , 2282–2289. IOS Press.
  * Yuan et al. (2020) Yuan, Y.; Zhou, X.; Pan, S.; Zhu, Q.; Song, Z.; and Guo, L. 2020.  A Relation-Specific Attention Network for Joint Entity and Relation Extraction.  In _Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI 2020_ , 4054–4060.
  * Zelenko, Aone, and Richardella (2003) Zelenko, D.; Aone, C.; and Richardella, A. 2003.  Kernel Methods for Relation Extraction.  _J. Mach. Learn. Res._ , 3: 1083–1106.
  * Zeng, Zhang, and Liu (2020) Zeng, D.; Zhang, H.; and Liu, Q. 2020.  CopyMTL: Copy Mechanism for Joint Extraction of Entities and Relations with Multi-Task Learning.  In _The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, New York, NY, USA, February 7-12, 2020_ , 9507–9514. AAAI Press.
  * Zeng et al. (2018) Zeng, X.; Zeng, D.; He, S.; Liu, K.; and Zhao, J. 2018.  Extracting Relational Facts by an End-to-End Neural Model with Copy Mechanism.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , 506–514. Association for Computational Linguistics.
  * Zhang, Zhang, and Fu (2017) Zhang, M.; Zhang, Y.; and Fu, G. 2017.  End-to-End Neural Relation Extraction with Global Optimization.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017_ , 1730–1740. Association for Computational Linguistics.
  * Zhao et al. (2021a) Zhao, K.; Xu, H.; Cheng, Y.; Li, X.; and Gao, K. 2021a.  Representation iterative fusion based on heterogeneous graph neural network for joint entity and relation extraction.  _Knowledge-Based Systems_ , 106888.
  * Zhao et al. (2021b) Zhao, T.; Yan, Z.; Cao, Y.; and Li, Z. 2021b.  A Unified Multi-Task Learning Framework for Joint Extraction of Entities and Relations.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 35(16): 14524–14531.
  * Zheng et al. (2021) Zheng, H.; Wen, R.; Chen, X.; Yang, Y.; Zhang, Y.; Zhang, Z.; Zhang, N.; Qin, B.; Ming, X.; and Zheng, Y. 2021.  PRGC: Potential Relation and Global Correspondence Based Joint Relational Triple Extraction.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing_ , 6225–6235. Association for Computational Linguistics.
  * Zheng et al. (2017) Zheng, S.; Wang, F.; Bao, H.; Hao, Y.; Zhou, P.; and Xu, B. 2017.  Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , 1227–1236. Vancouver, Canada: Association for Computational Linguistics.
  * Zhou et al. (2005) Zhou, G.; Su, J.; Zhang, J.; and Zhang, M. 2005.  Exploring Various Knowledge in Relation Extraction.  In _ACL 2005, 43rd Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, 25-30 June 2005, University of Michigan, USA_ , 427–434.