# Modeling Label Correlations for Ultra-Fine Entity Typing with
Neural Pairwise Conditional Random Field

Chengyue Jiang , Yong Jiang∗, Weiqi Wu, Pengjun Xie, Kewei Tu
DAMO Academy, Alibaba Group, China
<EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS>This work was done during
Chengyue Jiang’s internship at DAMO Academy, Alibaba Group. Yong Jiang and
Kewei Tu are corresponding authors.

###### Abstract

Ultra-fine entity typing (UFET) aims to predict a wide range of type phrases
that correctly describe the categories of a given entity mention in a
sentence. Most recent works infer each entity type independently, ignoring the
correlations between types, e.g., when an entity is inferred as a president,
it should also be a politician and a leader. To this end, we use an undirected
graphical model called pairwise conditional random field (PCRF) to formulate
the UFET problem, in which the type variables are not only unarily influenced
by the input but also pairwisely relate to all the other type variables. We
use various modern backbones for entity typing to compute unary potentials,
and derive pairwise potentials from type phrase representations that both
capture prior semantic information and facilitate accelerated inference. We
use mean-field variational inference for efficient type inference on very
large type sets and unfold it as a neural network module to enable end-to-end
training. Experiments on UFET show that the Neural-PCRF consistently
outperforms its backbones with little cost and results in a competitive
performance against cross-encoder based SOTA while being _thousands of times_
faster. We also find Neural-PCRF effective on a widely used fine-grained
entity typing dataset with a smaller type set. We pack Neural-PCRF as a
network module that can be plugged onto multi-label type classifiers with ease
and release it in
http://github.com/modelscope/adaseq/tree/master/examples/NPCRF.

##  1 Introduction

Entity typing assigns semantic types to entities mentioned in the text. The
extracted type information has a wide range of applications. It acts as a
primitive for information extraction Yang and Zhou (2010) and spoken language
understanding Coucke et al. (2018), and assists in more complicated tasks such
as machine reading comprehension Joshi et al. (2017) and semantic parsing
Yavuz et al. (2016). During its long history of development, the granularity
of the type set for entity typing and recognition changes from coarse (sized
less than 20) Tjong Kim Sang and De Meulder (2003); Hovy et al. (2006), to
fine (sized around 100) Weischedel and Brunstein (2005); Ling and Weld (2012);
Gillick et al. (2014); Ding et al. (2021b), to ultra-fine and free-form (sized
10k) Choi et al. (2018). The expansion of the type set reveals the diversity
of real-world entity categories and the importance of a finer-grained
understanding of entities in applications Choi et al. (2018).

The increasing number of types results in difficulties in predicting correct
types, so a better understanding of the entity types and their correlations is
needed. Most previous works solve an $N$-type entity typing problem as $N$
independent binary classifications Ling and Weld (2012); Gillick et al.
(2014); Choi et al. (2018); Onoe and Durrett (2019); Ding et al. (2021a); Pan
et al. (2022); Li et al. (2022). However, types are highly correlated and
hence they should be predicted in a joint manner. As an example, when ‘Joe
Biden’ is inferred as a president, it should also be inferred as a politician,
but not a science fiction. Type correlation is partially specified in fine-
grained entity typing (FET) datasets by a two or three-level type hierarchy,
and is commonly utilized by a hierarchy-aware objective function Ren et al.
(2016); Jin et al. (2019); Xu and Barbosa (2018). However, type hierarchies
cannot encode type relationships beyond strict containment, such as similarity
and mutual exclusion Onoe et al. (2021), and could be noisily defined Wu et
al. (2019) or even unavailable (in ultra-fine entity typing (UFET)). Many
recent works handle these problems by embedding types and mentions into
special spaces such as the hyperbolic space López and Strube (2020) or box
space Onoe et al. (2021) that can be trained to latently encode type
correlations without a hierarchy. Although these methods are expressive for
modeling type correlations, they are constrained by these special spaces and
thus incapable of being combined with modern entity typing backbones such as
prompt learning Ding et al. (2021a) and cross-encoder Li et al. (2022), and
cannot integrate prior type semantics.

In this paper, we present an efficient method that expressively models type
correlations while being backbone-agnostic. We formulate the UFET and FET
problems under a classical undirected graphical model (UGM) Koller and
Friedman (2009) called pairwise conditional random field (PCRF) Ghamrawi and
McCallum (2005). In PCRF, types are binary variables that are not only unarily
influenced by the input but also pairwisely relate to all the other type
variables. We formulate the unary potentials using the type logits provided by
any modern backbone such as prompt learning. To compose the pairwise
potentials sized $O(4N^{2})$ ($N$ is the number of types, 10k for UFET), we
use matrix decomposition, which is efficient and able to utilize prior type
semantic knowledge from word embeddings Pennington et al. (2014). Exact
inference on such a large and dense UGM is intractable, and therefore we apply
mean-field variational inference (MFVI) for approximate inference. Inspired by
Zheng et al. (2015), we unfold the MFVI as a recurrent neural network module
and connect it with the unary potential backbone to enable end-to-end training
and inference. We call our method Neural-PCRF (NPCRF). Experiments on UFET
show that NPCRF consistently outperforms the backbones with negligible
additional cost, and results in a strong performance against cross-encoder
based SOTA models while being _thousands of times_ faster. We also found NPCRF
effective on a widely used fine-grained entity typing dataset with a smaller
type set. We pack NPCRF as a network module that can be plugged onto multi-
label type classifiers with ease and release it in
http://github.com/modelscope/adaseq/tree/master/examples/NPCRF.

Figure 1: (a) Our pairwise CRF (PCRF) of ultra-fine entity typing. $X$ is the
entity mention and its context. (b) The neural network architecture
corresponding to the Neural-PCRF.

##  2 Related Work

###  2.1 Fine-grained and Ultra-Fine Entity Typing

Fine-grained Gillick et al. (2014); Ling and Weld (2012); Weischedel and
Brunstein (2005) (FET) and ultra-fine Choi et al. (2018) entity typing (UFET)
share common challenges. Researchers put lots of effort into better utilizing
the distantly and manually labeled training data, including rule-based
denoising Gillick et al. (2014), multi-round automatic denoising Onoe and
Durrett (2019); Pan et al. (2022); Dai et al. (2021), and partial label
embedding Ren et al. (2016). Some recent works introduce better entity typing
backbones compared with commonly used multi-label classifiers Gillick et al.
(2014); Choi et al. (2018); Onoe and Durrett (2019), such as prompt learning
Ding et al. (2021a), cross-encoders with indirect supervision from natural
language inference Li et al. (2022) and using a graph neural network to
enhance mention representations by sibling mentions for FET Chen et al.
(2022). Another line of work concentrates on modeling the given or latent type
correlations in entity typing, which is more relevant to our work. Ren et al.
(2016); Jin et al. (2019); Xu and Barbosa (2018) leverage a given type
hierarchy through a hierarchy-aware loss for FET. However, in UFET, the type
hierarchy is not given. Therefore various special type embeddings are designed
to learn the latent type correlation during training. Xiong et al. (2019)
utilize graph embedding of a label co-occurrence graph counted from training
data, López and Strube (2020) embeds entity mentions, contexts and labels into
the hyperbolic space that can latently encodes tree structures, and Onoe et
al. (2021) embeds those into the box space that encodes label dependency
through the topology of hyperrectangles. Unlike them, we treat types as random
variables under an undirected probabilistic graphical model, and only require
the type logits, therefore our neural PCRF is backbone-agnostic. Liu et al.
(2021) propose a label reasoning network (LRN) that generates types in an
auto-regressive manner, while our method decodes types in parallel and can be
combined with prompt-based backbones which LRN cannot do.

###  2.2 Pairwise Conditional Random Field

Pairwise conditional random field Ghamrawi and McCallum (2005) is a classical
undirected graphical model proposed for modeling label correlations. In the
deep learning era, PCRF was first found effective when combined with a
convolutional neural network (CNN) Fukushima and Miyake (1982) for semantic
segmentation in computer vision Zheng et al. (2015); Chandra and Kokkinos
(2016); Arnab et al. (2016); Shen et al. (2017); Lê-Huu and Alahari (2021a),
in which it was used to encourage adjacent pixels being segmented together. In
contrast to its popularity in computer vision, PCRF is much less explored in
natural language processing. Ghamrawi and McCallum (2005) and Wang et al.
(2017) apply PCRF on an n-gram feature based non-neural sentence classifier
with up to 203 classes. Besides the difference in tasks, numbers of classes
and backbones, our method is different from theirs in two main aspects: (1) We
unfold mean-field approximate inference of PCRF as a recurrent neural network
module for efficient end-to-end training and inference, while they use the
‘supported inference method’ Ghamrawi and McCallum (2005) which is intractable
for large type sets and incompatible with neural backbones (2) We design the
parameterization of the pairwise potential function based on the technique of
matrix decomposition to accelerate training and inference, and encode type
semantics which is important for entity typing Li et al. (2022) and sentence
classification Mueller et al. (2022). Our parameterization also conforms to
intrinsic properties of pairwise potentials (explained in Sec. 3.4). Hu et al.
(2020) investigates different potential function variants for linear-chain
CRFs for sequence labeling, while we design pairwise potential functions for
PCRF.

##  3 Method

###  3.1 Problem Definition

Entity typing datasets consist of $M$ entity mentions $m_{i}$ with their
corresponding context sentences $c_{i}$:
$\mathcal{D}=\\{(m_{1},c_{1}),\cdots,(m_{M},c_{M})\\}$, and a type set
$\mathcal{Y}$ of size $N$. The task of entity typing is to predict the types
$\bm{y}_{i}^{p}$ of the entity mention $m_{i}$ in the given context $c_{i}$,
where $\bm{y}_{i}^{p}$ is a subset of the type set. The number of gold types
$|\bm{y}_{i}^{g}|$ could be larger than one in most entity typing datasets.
The average number of gold types per instance $\text{avg}({|\bm{y}_{i}^{g}|})$
and the size of the type set $|\mathcal{Y}|$ vary in different datasets.

###  3.2 PCRF for Entity Typing

We first describe our pairwise conditional random field for the entity typing
Ghamrawi and McCallum (2005), as shown in Fig. 1(a). $x$ denotes a data point
$(m,c)\in\mathcal{D}$. $Y_{j}\in\\{0,1\\}$ denotes the binary random variable
for the $j$-th type in type set $\mathcal{Y}$. The type variables $Y_{1:N}$
are unarily connected to the input, model how likely a type is given the
input, and pairwisely connect a type variable with all other type variables to
model type correlations. Let $\bm{y}\in Y_{1}\times Y_{2}\times\cdots\times
Y_{N}$ be an assignment of all the type variables. The probability of $\bm{y}$
given a data point $x_{i}$ under a conditional random field can be factorized
as:

$p(\bm{y}|x)=\frac{1}{Z}\exp{\Big{(}\sum_{j}\theta_{u}(y_{j};x)+\sum_{j<k}\theta_{p}(y_{j},y_{k})\Big{)}}$

where $\theta_{u},\theta_{p}$ are scoring functions for unary and pairwise
edges, and $Z$ is the partition function for normalization.

###  3.3 Unary Potential Function

As illustrated in Figure 1(b), we parameterize the log unary potential
function $\theta_{u}(y_{j};x)$ using modern entity typing backbones based on
pretrained language models (PLM)111We use PLMs from https://huggingface.co/.We
introduce two of the backbones below.

#### Multi-label Classifier (MLC)

Given a data point $(m,c)$, we formulate the input as:

$[\text{CLS}]\ c\ [\text{SEP}]\ m\ [\text{SEP}]$

and feed it into RoBERTa-large Liu et al. (2019) to obtain the $[\text{CLS}]$
embedding $\bm{h}$. Then we obtain the unary type scores as follows.

$\displaystyle\theta_{u}(y_{j}=1;x)$ $\displaystyle=\text{Linear}(\bm{h})[j]$
$\displaystyle\theta_{u}(y_{j}=0;x)$ $\displaystyle=0$

#### Prompt Learning (PL)

We adopt prompt learning for entity typing Ding et al. (2021a); Pan et al.
(2022) as our second backbone. We feed:

$[\text{CLS}]\ c\ [\text{P}_{1}]\ m\ [\text{P}_{2}]\ [\text{P}_{3}]\
[\text{MASK}]\ [\text{SEP}]$

into Bert-cased Devlin et al. (2018), where
$[\text{P}_{1}],[\text{P}_{2}],[\text{P}_{3}]$ are trainable soft prompt
tokens. We use a verbalizer $\mathcal{V}$ to map types to subwords,
$\mathcal{V}_{y}$ denotes the subwords corresponding to type
$y\in\mathcal{Y}$, e.g.,
$\mathcal{V}_{\text{\emph{living\\_thing}}}=\\{\text{\emph{`living',
`thing'}}\\}$. We obtain the unary score of choosing type $y_{j}$ using the
averaged masked language model logits of $\mathcal{V}_{y_{j}}$:

$\theta_{u}(y_{j}=1;x)=\sum_{w\in\mathcal{V}_{y_{j}}}s([\text{MASK}]=w)/|\mathcal{V}_{y_{j}}|$

Similarly, we set $\theta(y_{j}=0;x)=0$.

###  3.4 Pairwise Potential Function

potential | encode | property
---|---|---
$\Theta_{11}(j,k)=\theta_{p}(y_{j}=1,y_{k}=1)$ | co-occurrence | $\Theta_{11}=\Theta_{11}^{T}$
$\Theta_{00}(j,k)=\theta_{p}(y_{j}=0,y_{k}=0)$ | co-absence | $\Theta_{00}=\Theta_{00}^{T}$
$\Theta_{10}(j,k)=\theta_{p}(y_{j}=1,y_{k}=0)$ | co-exclusion | $\Theta_{10}=\Theta_{01}^{T}$
$\Theta_{01}(j,k)=\theta_{p}(y_{j}=0,y_{k}=1)$ | co-exclusion | $\Theta_{01}=\Theta_{10}^{T}$

Table 1: Properties of the four log potential matrices.

Pairwise log potential $\theta_{p}(y_{j},y_{k})$ encoding type correlations
should naturally satisfy the properties shown in Table 1. Specifically,
$\Theta_{11}$ and $\Theta_{00}$ are symmetric matrices and encode co-
occurrence and co-absence respectively, while $\Theta_{10}=\Theta_{01}^{T}$
and are asymmetric ( e.g., “not a person but a president” $\not=$ “not a
president but a person”). Directly parameterizing these 4 potential matrices
Ghamrawi and McCallum (2005); Wang et al. (2017) ignores these intrinsic
properties and results in an unbearable number of model parameters for
datasets with a large type set (e.g., 400M parameters for 10331-type UFET,
which is more than Bert-large).

To tackle these problems, we parameterize the pairwise potential based on
matrix rank decomposition, i.e., we represent each $N\times N$ matrix as the
product of two $N\times R$ matrices, where $R$ is the number of ranks.
Crucially, we use pretrained word embedding to derive the two $N\times R$
matrices, so that they can encode type semantics. Specifically, we obtain a
type embedding matrix $\bm{E}\in\mathbb{R}^{N\times 300}$ based on
300-dimension GloVe embedding Pennington et al. (2014). For a type phrase
consisting of multiple words (e.g., _“living_thing”_), we take the average of
the word embeddings. We then transform $\bm{E}$ into two embedding spaces
encoding _“occurrence”_ and _“absence”_ respectively using two feed-forward
networks.

$\displaystyle\text{FFN}(\bm{E})$
$\displaystyle=\bm{W_{2}}(\tanh(\text{dropout}(\bm{W_{1}}\bm{E}))$ (1)
$\displaystyle\bm{E}_{0}$
$\displaystyle=\text{FFN}_{1}(\bm{E})\in\mathbb{R}^{N\times R}$
$\displaystyle\bm{E}_{1}$
$\displaystyle=\text{FFN}_{2}(\bm{E})\in\mathbb{R}^{N\times R}$

To enforce the intrinsic properties of the four matrices, we parameterize them
as follows:

$\displaystyle\Theta_{00}$ $\displaystyle=\bm{E}_{0}\bm{E}_{0}^{T}$
$\displaystyle\Theta_{11}$ $\displaystyle=\bm{E}_{1}\bm{E}_{1}^{T}$ (2)
$\displaystyle\Theta_{01}$ $\displaystyle=-\bm{E}_{0}\bm{E}_{1}^{T}$
$\displaystyle\Theta_{10}$ $\displaystyle=-\bm{E}_{1}\bm{E}_{0}^{T}$

The negative signs in defining $\Theta_{01},\Theta_{10}$ ensure that they
encode co-exclusion, not similarity. As we will show in the next subsection,
we do not need to actually recover these large matrices during inference,
leading to lower computational complexity.

###  3.5 Mean-field Variational Inference

We aim to infer the best type set given the input.

$\bm{y}^{p}=\operatornamewithlimits{argmax}_{\bm{y}\in Y_{1}\times\cdots\times
Y_{N}}p(\bm{y}|x)$

The MAP (maximum-a-posteriori) inference over this large and dense graph is
NP-hard. We instead use mean-field variational inference (MFVI) to
approximately infer the best assignment $\bm{y}^{p}$. MFVI approximates the
true posterior distribution $p(y_{j}|x)$ by a variational distribution
$q(y_{j})$ that can be factorized into independent marginals Wainwright et al.
(2008). It iteratively minimizes the KL-divergence between $p$ and $q$. We
initialize the $q$ distribution by unary scores produced by backbones:

$q^{0}(y_{j})=\text{softmax}(\theta_{u}(y_{j};x))$

and derive the MFVI iteration as follows:

$\displaystyle\begin{aligned}
&q^{t+1}(y_{j}=1)\propto\exp\Big{(}\theta_{u}(y_{j}=1;x)+\\\
&\sum_{k\not=j}(\Theta_{11}^{(jk)}\cdot
q^{t}(y_{k}=1))+\Theta_{10}^{(jk)}\cdot q^{t}(y_{k}=0)\Big{)}\\\
&q^{t+1}(y_{j}=0)\propto\exp\Big{(}\theta_{u}(y_{j}=0;x)+\\\
&\sum_{k\not=j}(\Theta_{00}^{(jk)}\cdot
q^{t}(y_{k}=0))+\Theta_{01}^{(jk)}\cdot q^{t}(y_{k}=1)\Big{)}\\\
\end{aligned}$

We rewrite the formulas in the vector form with our parametrization.
$\bm{\theta}_{1}^{u}\in\mathbb{R}^{N}$ denotes the unary score vector of
$\theta_{u}(y_{j}=1;x)$ for all $j$, and we define $\bm{\theta}_{0}^{u}$
similarly. $\bm{q}_{0}^{t},\bm{q}_{1}^{t}$ are vectors of the $q$
distributions at the $t$-th iteration.

$\displaystyle(\bm{q}_{0}^{0}[j],\bm{q}_{1}^{0}[j])=\text{softmax}(\bm{\theta}_{0}^{u}[j],\bm{\theta}_{1}^{u}[j])$
$\displaystyle\bm{q}_{1}^{t+1}\propto\exp\Big{(}\bm{\theta}_{1}^{u}+\bm{E}_{1}\big{(}\bm{E}_{1}^{T}\bm{q}^{t}_{1}\big{)}-\bm{E}_{1}\big{(}\bm{E}_{0}^{T}\bm{q}^{t}_{0}\big{)}\Big{)}$
$\displaystyle\bm{q}_{0}^{t+1}\propto\exp\Big{(}\bm{\theta}_{0}^{u}+\bm{E}_{0}\big{(}\bm{E}_{0}^{T}\bm{q}^{t}_{0}\big{)}-\bm{E}_{0}\big{(}\bm{E}_{1}^{T}\bm{q}^{t}_{1}\big{)}\Big{)}$

Note that by following these update formulas, we do not need to recover the
$N\times N$ matrices, and hence the time complexity of each iteration is $NR$.
Since the iteration number $\mathcal{T}$ is typically a small constant
($\mathcal{T}<7$), the computational complexity of the whole MFVI is $O(NR)$.
The predicted type set of $x_{i}$ is obtained by:

$\bm{y}^{p}=\\{y_{j}\ |\
\bm{q}_{1}^{\mathcal{T}}[j]>0.5,y_{j}\in\mathcal{Y}\\}$

We follow the treatment of MFVI as entropy-regularized Frank-Wolfe Lê-Huu and
Alahari (2021b) and introduce another hyper-parameter $\lambda$ to control the
step size of each update. Let $\bm{q}^{t}=[\bm{q}^{t}_{0};\bm{q}^{t}_{1}]$,

$\bm{q}^{t+1}=\bm{q}^{t}+\lambda(\bm{q}^{t+1}-\bm{q}^{t})$

###  3.6 Unfolding Mean-field Variational Inference as a Recurrent Neural
Network

We follow Zheng et al. (2015) and treat the mean-field variational inference
procedure with a fixed number of iterations as a recurrent neural network
(RNN) parameterized by the pairwise potential functions. As shown in the top
part of Figure 1(b), the initial hidden state of the RNN is the type
distribution $\bm{q}^{0}$ produced by the unary logits, and the final hidden
states $\bm{q}^{\mathcal{T}}$ after $\mathcal{T}$ iterations are used for end-
to-end training and inference.

###  3.7 Training Objective and Optimization

We use the Binary Cross Entropy loss for training:

$\displaystyle\begin{aligned}
\mathcal{L}=-\frac{1}{N}\sum_{j=1}^{N}\Big{(}\alpha\cdot
y_{j}^{g}\log\bm{q}^{\mathcal{T}}_{1}[j]+\\\
(1-y_{j}^{g})\log(1-\bm{q}^{\mathcal{T}}_{1}[j])\Big{)}\end{aligned}$

$\mathcal{L}$ is the loss of each instance $x$, $y_{j}^{g}\in\\{0,1\\}$ is the
gold annotation of $x$ for type $y_{j}$. We follow previous works Choi et al.
(2018); Dai et al. (2021) and use $\alpha$ as a weight for the loss of
positive types. We train the pretrained language model, label embedding matrix
$\bm{E}$ and FFNs. We use AdamW Loshchilov and Hutter (2018) for optimization.

##  4 Experiment

###  4.1 Datasets and Experimental Settings

We mainly evaluate our NPCRF method on the ultra-fine entity typing dataset:
UFET Choi et al. (2018). We also conduct experiments on the augmented version
Choi et al. (2018) of OntoNotes Gillick et al. (2014) to examine if our method
also works for datasets with smaller type sets and fewer gold types per
instance. We show the dataset statistics in Table 2. Note that UFET also
provides 25M distantly labeled training data extracted by linking to KB and
parsing. We follow recent works Pan et al. (2022); Liu et al. (2021) and only
use the manually annotated 2k data for training. We use standard metrics for
evaluation: for UFET, we report macro-averaged precision (P), recall (R), and
F1; for OntoNotes, we report macro-averaged and micro-averaged F1. We run
experiments three times and report the average results.

dataset | $|\mathcal{Y}|$ | $\text{avg}(|\bm{y}_{i}^{g}|)$ | train/dev/test | hierarchy
---|---|---|---|---
UFET | 10331 | 5.4 | 2k/2k/2k | No
OntoNotes | 89 | 1.5 | 0.8M/2k/9k | 3-layer

Table 2: $|\mathcal{Y}|$ denotes the size of the type set,
$\text{avg}(|\bm{y}_{i}^{g}|)$ denotes the average number of gold types per
instance.

###  4.2 Baseline Methods

Model | P | R | F1
---|---|---|---
_non-PLM model for reference_
UFET-BiLSTM Choi et al. (2018) † | 48.1 | 23.2 | 31.3
LabelGCN Xiong et al. (2019) †◇ | 50.3 | 29.2 | 36.9
_PLM-based models_
LDET Onoe and Durrett (2019) † | 51.5 | 33.0 | 40.1
Box4Types Onoe et al. (2021) †◇ | 52.8 | 38.8 | 44.8
LRN Liu et al. (2021) ◇ | 54.5 | 38.9 | 45.4
MLMET Dai et al. (2021) † | 53.6 | 45.3 | 49.1
LITE+L Li et al. (2022) | 48.7 | 45.8 | 47.2
DFET Pan et al. (2022) | 55.6 | 44.7 | 49.5
LITE+NLI+L Li et al. (2022) † | 52.4 | 48.9 | 50.6
_PLM-based models w/ or w/o NPCRF_
MLC-RoBerta | 47.8 | 40.4 | 43.8
MLC-RoBerta w/ NPCRF | 48.7 | 45.5 | 47.0
|  |  |
LabelGCN-RoBerta◇ | 51.2 | 41.0 | 45.5
LabelGCN-RoBerta w/ NPCRF | 48.7 | 45.9 | 47.3
|  |  |
PL-Bert-base Ding et al. (2021a) | 57.8 | 40.7 | 47.7
PL-Bert-base w/ NPCRF | 52.1 | 47.5 | 49.7
|  |  |
PL-Bert-large | 59.3 | 42.6 | 49.6
PL-Bert-large w/ NPCRF | 55.3 | 46.7 | 50.6

Table 3: Macro-averaged UFET result. LITE+L is LITE without NLI pretraining,
LITE+L+NLI is the full LITE model, LabelGCN-RoBerta denotes our implementation
of LabelGCN with RoBerta-large as mention encoder. Methods marked by $\dagger$
use either distantly labeled training data or additional pretraining tasks and
$\Diamond$ marker denotes method focusing on modeling label correlations.

MLC-RoBerta and PL-Bert introduced in Sec. 3.3 are natural baselines. We
compare their performances with and without NPCRF.
UFET Choi et al. (2018) A multi-label linear classifier (MLC) with a backbone
using BiLSTM, GloVe, and CharCNN to encode the mention.
LDET Onoe and Durrett (2019) An MLC with Bert-base-uncased and ELMo Peters et
al. (2018) and trained by 727k examples automatically denoised from the
distantly labeled UFET.
LabelGCN Xiong et al. (2019) An MLC with BiLSTM and multi-head self-attention
to encode the mention and context, and a GCN running on a fixed co-occurrence
type graph to obtain better type embedding. Type scores are dot-product of
mention and type embedddings. For fair comparison, we replace their mention
encoder with RoBerta-large. To our knowledge, LabelGCN cannot be directly
combined with prompt learning.
Box4Type Onoe et al. (2021) use Bert-large as backbone and project mentions
and types to the box space for training and inference; trained on the same
727k data Onoe and Durrett (2019).
LRN Liu et al. (2021) generate types using Bert-base and an LSTM decoder in a
seq2seq manner, and use 2k manually labeled data for training.
MLMET Dai et al. (2021) A multi-label linear classifier using Bert-base, first
pretrained by the distantly-labeled data augmented by masked word prediction,
then finetuned and self-trained on the 2k human-annotated data.
DFET Pan et al. (2022) A 3-round automatic denoising method for 2k mannually
labeled data, using PL-Bert as backbone.
LITE Li et al. (2022) Previous SOTA system that formulates entity typing as
natural language inference, treating $x_{i}$ as premise and types as
hypothesis, concatenating them and feeding them into RoBerta-large to score
types. LITE models the correlation better between the input and types, and has
great zero-shot performance but needs to concatenate $x_{i}$ with all the $N$
types, resulting in very slow inference. LITE is pretrained on MNLI Williams
et al. (2018) and trained on 2k annotated data, Its authors find that the
performance drops when using distantly labeled data.

###  4.3 UFET Result

As shown in Table 3, our NPCRF can be integrated with various entity typing
backbones and enhance their performance to SOTA on UFET.

MLC-RoBerta w/ NPCRF improves the basic MLC-RoBerta backbone by $+3.2$ F1
score and reaches the best performance among models using the MLC architecture
except for MLMET which uses millions of distantly labeled data. LabelGCN Xiong
et al. (2019) utilizing fixed type co-occurrence information to obtain type
embedding is still effective with the replaced RoBerta encoder and improves F1
by $+1.7$ over MLC-RoBerta while our method produce a further $+1.8$ F1
improvement, because NPCRF models not only co-occurrence (in $\Theta_{11}$),
but also semantic co-absence and co-exclusion in our pairwise potentials, and
it also explicitly updates these potentials during training rather than using
fixed type co-occurrence information.

Prompt-learning based backbones Ding et al. (2021a); Pan et al. (2022) such as
PL-BERT are already strong in UFET, and our method further improves PL-BERT-
Base by $+1.9$ F1 and reaches $49.7$ F1 which is slightly better than MLMET
and DFET and is the SOTA performance for Bert-base models, on par with
performance of Bert-large models. Also worth noting is that our method is
trained in a simpler single-round end-to-end manner compared with other
competitors requiring multi-round training Dai et al. (2021) and denoising Pan
et al. (2022) procedure. For prompt-learning powered by Bert-large models,
NPCRF boosts the performance of PL-Bert-large by $1.0$ F1, and results in
performance on par with the previous SOTA system LITE+NLI+L with much faster
inference speed (discussed in Sec. 5.5). Note that the improvement is smaller
on large models because large models are stronger in inferring types without
pairwise potentials from limited and noisy contexts. We also observe that
models with NPCRF tend to predict more types and therefore have higher recalls
and sometimes lower precisions. It is possibly because NPCRF can use type
dependencies to infer additional types that are not directly supported by the
input. We will discuss it in detail in Sec. 5.4.

###  4.4 FET Result

We present the performance of NPCRF on the augmented OntoNotes dataset in
Table 4. The results show that MLC-RoBerta w/ NPCRF outperforms MLC-RoBerta by
$+1.5$ in macro-F1 and $+1.7$ in micro-F1, and reaches competitive
performances against a recent SOTA system DFET focusing on denoising data. In
general, we find our method still effective, but the improvement is less
significant compared with UFET, especially for PL-Bert. One possible reason is
that the average number of types per instance in OntoNotes is 1.5, and
therefore the type-type relationship is less important. Another possible
reason is that some type semantics are already covered in the prompt-based
method through the verbalizer, so our method fails to boost the performance of
PL-Bert.

Model | Ma-F1 | Mi-F1 |
---|---|---|---
_PLM-based models_
LDET Onoe and Durrett (2019) † | 84.5 | 79.2 |
Box4Types Onoe et al. (2021) †◇ | 77.3 | 70.9 |
LRN Liu et al. (2021) ◇ | 84.8 | 80.1 |
MLMET Dai et al. (2021) † | 85.4 | 80.4 |
LITE Li et al. (2022) | 86.6 | 81.4 |
DFET Pan et al. (2022) | 87.1 | 81.5 |
_PLM-based models w/ or w/o NPCRF_
MLC-RoBerta | 85.4 | 80.2 |
MLC-RoBerta w/ NPCRF | 86.9 | 81.9 |
|  |  |
LabelGCN-RoBerta ◇ | 85.5 | 80.2 |
LabelGCN-RoBerta w/ NPCRF | 86.2 | 81.3 |
|  |  |
PL-Bert-Base Ding et al. (2021a) | 84.8 | 79.6 |
PL-Bert-Base w/ NPCRF | 85.2 | 80.0 |

Table 4: Results on the augmented OntoNotes datasets. Ma and Mi are
abbreviations of macro and micro. Methods marked by $\dagger$ use either
distantly labeled training data or additional pretraining tasks and the
$\Diamond$ marker denotes methods modeling label correlations.

##  5 Analysis

###  5.1 Ablation Study

We show the ablation study in Table 5. It can be seen that: (1) Performance
drops when we randomly initialize label embedding $\bm{E}$ (denoted by \- w/o
GloVe), which indicates that GloVe embedding contains useful type semantics
and helps build pairwise type correlations. (2) Performance drops when we
remove the hidden layer and the $\tanh$ nonlinearity in FFNs (Eq. 1), showing
the benefit of expressive parameterization. (3) When we remove the entire FFN
layers and use $\bm{E}$ to parameterize the four matrices in (Eq. 2) model
training fails to converge, showing that a reasonable parameterization is
important.

Model | Ma-P | Ma-R | Ma-F1
---|---|---|---
PL-Bert w/ NPCRF | 52.1 | 47.5 | 49.7
\- w/o GloVe | 48.5 | 49.1 | 48.8
\- w/o FFN hidden | 51.1 | 47.0 | 49.0
\- w/o FFN | - | - | -
\- w/o NPCRF | 57.8 | 40.7 | 47.7

Table 5: Ablation study on UFET test set, “-” means training fails to
converge.

###  5.2 Performance on Coarser Granularity

We evaluate the performance of NPCRF on type sets with coarser granularity.
UFET Choi et al. (2018) splits the types into coarse-grained, fine-grained and
ultra-fine-grained types. We discard ultra-fine-grained types to create the
fine-grained setting (130 types), and further discard the fine-grained types
to create the coarse-grained setting (9 types). As shown in Table 6, NPCRF
still has positive effect in these two settings, while LabelGCN does not.

Model | Mi-P | Mi-R | Mi-F1
---|---|---|---
_coarse-grained, 9-class, $\text{avg}(|\bm{y}_{i}^{g}|)=0.95$_
MLC-RoBERTa | 76.5 | 75.8 | 76.2
LabelGCN-RoBERTa | 77.5 | 74.7 | 76.1
MLC-RoBERTa w/ NPCRF | 78.0 | 76.0 | 77.0
_fine-grained, 130-class, $\text{avg}(|\bm{y}_{i}^{g}|)=1.61$_
MLC-RoBERTa | 62.8 | 67.3 | 64.9
LabelGCN-RoBERTa | 59.7 | 69.6 | 64.3
MLC-RoBERTa w/ NPCRF | 62.7 | 68.8 | 65.6

Table 6: Performances on coarser granularity.

(a)

(b)

(c)

Figure 2: Two MFVI cases. We show, from top to bottom, how the type
probabilities $\bm{q}_{1}^{t}$ change with iteration given the mention and its
context. The reddish grids (probability $>0.5$) indicates the chosen types,
and gold types are colored red on the x-axis.

###  5.3 Model Performance at Each Iteration

We evaluate the per-iteration performance of PL-Bert w/ NPCRF on the test set
and show the results in Table 7. We obtain the prediction $\bm{y}_{t}^{p}$ of
each iteration $t$ by binarizing $\bm{q}_{t}$ for each instance:
$\bm{y}^{p}_{t}=\\{y_{j}\ |\ \bm{q}_{1}^{t}[j]>0.5,y_{j}\in\mathcal{Y}\\}$.
The results show that model prediction at iteration 0 (i.e., based solely on
unary scores) has a high recall and a very low precision, while NPCRF keeps
correcting wrong candidates during consecutive iterations to reach a higher F1
score. We show some concrete cases in Sec. 5.4.

Iteration | Ma-P | Ma-R | Ma-F1 | $\text{avg}(|\bm{y}_{i,t}^{p}|)$
---|---|---|---|---
$\bm{q}^{0}$ | 15.3 | 74.5 | 25.4 | 27.92
$\bm{q}^{1}$ | 43.2 | 53.6 | 47.8 | 6.52
$\bm{q}^{2}$ | 50.0 | 48.2 | 49.1 | 4.95
$\bm{q}^{3}$ | 52.1 | 47.4 | 49.7 | 4.65

Table 7: Model performance at each iteration, $\text{avg}(|\bm{y}_{i}^{p}|)$
denotes the average number of predicted types.

###  5.4 Case Study and Visualization

#### MFVI Iterations

We show the per-iteration predictions of PL-Bert w/ NPCRF for two inputs in
Figure 2. As can be seen, NPCRF tends to delete wrong types through
iterations, such as _“history”, “nation”_ in Fig. 2(a) and _“pitcher”,
“soccer_player”_ in Fig. 2(b). This results in higher precision (as shown in
Table 7). However, we find NPCRF is also capable of increasing the
probabilities of some types, e.g., _“ballplayer”_ in the second case. NPCRF
may also erroneously deprecate gold types and results in lower recall. As been
shown in 2(c), NPCRF wrongly deletes gold types such as _“adult”, “man”_ while
it correctly predicts _“president”_ which is not annotated as gold label, and
increases the score of _“compaigner”_.

#### Pairwise Potentials

We show the four learned pairwise potentials in Appendix A.1.

###  5.5 Efficiency

We compare the training and inference efficiency of different methods on the
UFET dataset in Table 8. We run all these methods on one Tesla V100 GPU three
times, and report the average speed (number of sentences per second) during
training and inference. Except for LRN which is based on Bert-base-uncased,
all the other methods are based on RoBERTa-large. Results show that NPCRF (4
iterations, using the best hyper-parameters) is the fastest to model type
correlations (compared with LabelGCN and LRN), it slows down training by 15.7%
and inference by only 13.8%. NPCRF is much faster than LITE which is based on
the cross-encoder architecture in inference222Prompt-learning based method has
similar time complexity as MLC based method theoretically. We do not compare
them because of differences in the PLMs and implementations..

Model (sents/sec) | Train | Inference
---|---|---
MLC-RoBERTa | 37.04 | 142.86
MLC-RoBERTa w/ NPCRF | 31.25 | 123.46
|  |
LabelGCN-RoBERTa | 27.78 | 90.91
LRN | 19.59 | 20.88
|  |
LITE | 20.41 | 0.03

Table 8: Comparison of training and inference speed of different methods in
Table 3.

##  6 Conclusion and Future Work

We propose NPCRF, a method that efficiently models type correlation for ultra-
fine and fine-grained entity typing, and is applicable to various entity
typing backbones. In NPCRF, the unary potential is formulated as the type
logits of modern UFET backbones, the pairwise potentials are derived from type
phrase representations that both capture prior semantic information and
facilitate accelerated inference. We unfold mean-field variational inference
of NPCRF as a neural network for end-to-end training and inference. We find
our method consistently outperforms its backbone, and reach competitive
performance against very recent baselines on UFET and FET. NPCRF is efficient
and require low additional computation costs. For future work, modeling
higher-order label correlations, injecting prior knowledge into pairwise
potentials, and extending NPCRF to other tasks are worth exploring.

## Limitations

As shown in the experiments, the main limitation of NPCRF is that it has less
positive effect on tasks that do not require understanding type correlation
(e.g., tasks with small label sets and a low number of gold labels per
instance), and on models that already model label semantics quite well (e.g.,
prompt-based methods). Another limitation of NPCRF is that, although it can be
combined with many backbones, there still exist some backbones that cannot
directly use NPCRF, such as models that generate types one by one in an auto-
regressive way (e.g., LRN) and models that cannot efficiently compute label
logits (e.g., LITE).

##  7 Acknowledgement

This work was supported by the National Natural Science Foundation of China
(61976139) and by Alibaba Group through Alibaba Innovative Research Program.

## References

  * Arnab et al. (2016) Anurag Arnab, Sadeep Jayasumana, Shuai Zheng, and Philip HS Torr. 2016.  Higher order conditional random fields in deep neural networks.  In _European conference on computer vision_ , pages 524–540. Springer.
  * Chandra and Kokkinos (2016) Siddhartha Chandra and Iasonas Kokkinos. 2016.  Fast, exact and multi-scale inference for semantic image segmentation with deep gaussian crfs.  In _European conference on computer vision_ , pages 402–418. Springer.
  * Chen et al. (2022) Yi Chen, Jiayang Cheng, Haiyun Jiang, Lemao Liu, Haisong Zhang, Shuming Shi, and Ruifeng Xu. 2022.  Learning from sibling mentions with scalable graph inference in fine-grained entity typing.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 2076–2087, Dublin, Ireland. Association for Computational Linguistics.
  * Choi et al. (2018) Eunsol Choi, Omer Levy, Yejin Choi, and Zettlemoyer. 2018.  Ultra-fine entity typing.  In _Proceedings of the ACL_. Association for Computational Linguistics.
  * Coucke et al. (2018) Alice Coucke, Alaa Saade, Adrien Ball, Théodore Bluche, Alexandre Caulier, David Leroy, Clément Doumouro, Thibault Gisselbrecht, Francesco Caltagirone, Thibaut Lavril, et al. 2018.  Snips voice platform: an embedded spoken language understanding system for private-by-design voice interfaces.  _arXiv preprint arXiv:1805.10190_.
  * Dai et al. (2021) Hongliang Dai, Yangqiu Song, and Haixun Wang. 2021.  Ultra-fine entity typing with weak supervision from a masked language model.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 1790–1799, Online. Association for Computational Linguistics.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_.
  * Ding et al. (2021a) Ning Ding, Yulin Chen, Xu Han, Guangwei Xu, Pengjun Xie, Hai-Tao Zheng, Zhiyuan Liu, Juanzi Li, and Hong-Gee Kim. 2021a.  Prompt-learning for fine-grained entity typing.  _arXiv preprint arXiv:2108.10604_.
  * Ding et al. (2021b) Ning Ding, Guangwei Xu, Yulin Chen, Xiaobin Wang, Xu Han, Pengjun Xie, Haitao Zheng, and Zhiyuan Liu. 2021b.  Few-NERD: A few-shot named entity recognition dataset.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 3198–3213, Online. Association for Computational Linguistics.
  * Fukushima and Miyake (1982) Kunihiko Fukushima and Sei Miyake. 1982.  Neocognitron: A self-organizing neural network model for a mechanism of visual pattern recognition.  In _Competition and cooperation in neural nets_ , pages 267–285. Springer.
  * Ghamrawi and McCallum (2005) Nadia Ghamrawi and Andrew McCallum. 2005.  Collective multi-label classification.  In _Proceedings of the 14th ACM International Conference on Information and Knowledge Management_ , CIKM ’05, page 195–200, New York, NY, USA. Association for Computing Machinery.
  * Gillick et al. (2014) Dan Gillick, Nevena Lazic, Kuzman Ganchev, Jesse Kirchner, and David Huynh. 2014\.  Context-dependent fine-grained entity type tagging.  _arXiv preprint arXiv:1412.1820_.
  * Hovy et al. (2006) Eduard Hovy, Mitchell Marcus, Martha Palmer, Lance Ramshaw, and Ralph Weischedel. 2006.  OntoNotes: The 90% solution.  In _Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers_ , pages 57–60, New York City, USA. Association for Computational Linguistics.
  * Hu et al. (2020) Zechuan Hu, Yong Jiang, Nguyen Bach, Tao Wang, Zhongqiang Huang, Fei Huang, and Kewei Tu. 2020.  An investigation of potential function designs for neural CRF.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 2600–2609, Online. Association for Computational Linguistics.
  * Jin et al. (2019) Hailong Jin, Lei Hou, Juanzi Li, and Tiansi Dong. 2019.  Fine-grained entity typing via hierarchical multi graph convolutional networks.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 4969–4978, Hong Kong, China. Association for Computational Linguistics.
  * Joshi et al. (2017) Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. 2017.  Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension.  _arXiv preprint arXiv:1705.03551_.
  * Koller and Friedman (2009) Daphne Koller and Nir Friedman. 2009.  _Probabilistic graphical models: principles and techniques_.  MIT press.
  * Lê-Huu and Alahari (2021a) Đ Khuê Lê-Huu and Karteek Alahari. 2021a.  Regularized frank-wolfe for dense crfs: Generalizing mean field and beyond.  _Advances in Neural Information Processing Systems_ , 34.
  * Lê-Huu and Alahari (2021b) Đ Khuê Lê-Huu and Karteek Alahari. 2021b.  Regularized frank-wolfe for dense crfs: Generalizing mean field and beyond.  _Advances in Neural Information Processing Systems_ , 34.
  * Li et al. (2022) Bangzheng Li, Wenpeng Yin, and Muhao Chen. 2022.  Ultra-fine entity typing with indirect supervision from natural language inference.  _arXiv preprint arXiv:2202.06167_.
  * Ling and Weld (2012) Xiao Ling and Daniel S Weld. 2012.  Fine-grained entity recognition.  In _Twenty-Sixth AAAI Conference on Artificial Intelligence_.
  * Liu et al. (2021) Qing Liu, Hongyu Lin, Xinyan Xiao, Xianpei Han, Le Sun, and Hua Wu. 2021.  Fine-grained entity typing via label reasoning.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 4611–4622, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_.
  * López and Strube (2020) Federico López and Michael Strube. 2020.  A fully hyperbolic neural model for hierarchical multi-class classification.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 460–475, Online. Association for Computational Linguistics.
  * Loshchilov and Hutter (2018) Ilya Loshchilov and Frank Hutter. 2018.  Fixing weight decay regularization in adam.
  * Mueller et al. (2022) Aaron Mueller, Jason Krone, Salvatore Romeo, Saab Mansour, Elman Mansimov, Yi Zhang, and Dan Roth. 2022.  Label semantic aware pre-training for few-shot text classification.  _arXiv preprint arXiv:2204.07128_.
  * Onoe et al. (2021) Yasumasa Onoe, Michael Boratko, Andrew McCallum, and Greg Durrett. 2021.  Modeling fine-grained entity types with box embeddings.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 2051–2064, Online. Association for Computational Linguistics.
  * Onoe and Durrett (2019) Yasumasa Onoe and Greg Durrett. 2019.  Learning to denoise distantly-labeled data for entity typing.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 2407–2417, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Pan et al. (2022) Weiran Pan, Wei Wei, and Feida Zhu. 2022.  Automatic noisy label correction for fine-grained entity typing.  _arXiv preprint arXiv:2205.03011_.
  * Pennington et al. (2014) Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014.  GloVe: Global vectors for word representation.  In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1532–1543, Doha, Qatar. Association for Computational Linguistics.
  * Peters et al. (2018) Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018.  Deep contextualized word representations.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 2227–2237, New Orleans, Louisiana. Association for Computational Linguistics.
  * Ren et al. (2016) Xiang Ren, Wenqi He, Meng Qu, Clare R Voss, Heng Ji, and Jiawei Han. 2016.  Label noise reduction in entity typing by heterogeneous partial-label embedding.  In _Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining_ , pages 1825–1834.
  * Shen et al. (2017) Falong Shen, Rui Gan, Shuicheng Yan, and Gang Zeng. 2017.  Semantic segmentation via structured patch prediction, context crf and guidance crf.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 1953–1961.
  * Tjong Kim Sang and De Meulder (2003) Erik F. Tjong Kim Sang and Fien De Meulder. 2003.  Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition.  In _Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003_ , pages 142–147.
  * Wainwright et al. (2008) Martin J Wainwright, Michael I Jordan, et al. 2008.  Graphical models, exponential families, and variational inference.  _Foundations and Trends® in Machine Learning_ , 1(1–2):1–305.
  * Wang et al. (2017) Bingyu Wang, Cheng Li, Virgil Pavlu, and Javed Aslam. 2017.  Regularizing model complexity and label structure for multi-label text classification.  _arXiv preprint arXiv:1705.00740_.
  * Weischedel and Brunstein (2005) Ralph Weischedel and Ada Brunstein. 2005.  Bbn pronoun coreference and entity type corpus.  Technical Report LDC2005T33, inguistic Data Consortium.
  * Williams et al. (2018) Adina Williams, Nikita Nangia, and Samuel Bowman. 2018.  A broad-coverage challenge corpus for sentence understanding through inference.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 1112–1122, New Orleans, Louisiana. Association for Computational Linguistics.
  * Wu et al. (2019) Junshuang Wu, Richong Zhang, Yongyi Mao, Hongyu Guo, and Jinpeng Huai. 2019.  Modeling noisy hierarchical types in fine-grained entity typing: A content-based weighting approach.  In _IJCAI_ , pages 5264–5270.
  * Xiong et al. (2019) Wenhan Xiong, Jiawei Wu, Deren Lei, Mo Yu, Shiyu Chang, Xiaoxiao Guo, and William Yang Wang. 2019.  Imposing label-relational inductive bias for extremely fine-grained entity typing.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 773–784, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Xu and Barbosa (2018) Peng Xu and Denilson Barbosa. 2018.  Neural fine-grained entity type classification with hierarchy-aware loss.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 16–25, New Orleans, Louisiana. Association for Computational Linguistics.
  * Yang and Zhou (2010) Li Yang and Yanhong Zhou. 2010.  Two-phase biomedical named entity recognition based on semi-crfs.  In _2010 IEEE Fifth International Conference on Bio-Inspired Computing: Theories and Applications (BIC-TA)_ , pages 1061–1065.
  * Yavuz et al. (2016) Semih Yavuz, Izzeddin Gur, Yu Su, Mudhakar Srivatsa, and Xifeng Yan. 2016.  Improving semantic parsing via answer type inference.  In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_ , pages 149–159, Austin, Texas. Association for Computational Linguistics.
  * Zheng et al. (2015) Shuai Zheng, Sadeep Jayasumana, Bernardino Romera-Paredes, Vibhav Vineet, Zhizhong Su, Dalong Du, Chang Huang, and Philip HS Torr. 2015.  Conditional random fields as recurrent neural networks.  In _Proceedings of the IEEE international conference on computer vision_ , pages 1529–1537.

##  Appendix A Example Appendix

###  A.1 Pairwise Potentials

We recover the learned log potentials
$\Theta_{00},\Theta_{11},\Theta_{01},\Theta_{10}$ using Eq. 2. For
visualization, we pick three sets of types, types in the same set are
relevant, and types between different set are irrelevant, and draw the heatmap
of log potentials of them in Fig. 3. The figure shows that: (1) the learned
potentials obey the intrinsic properties, $\Theta_{00},\Theta_{11}$ are
symmetric and different from each other, $Theta_{01}=\Theta_{10}^{T}$ and they
are asymmetric. (2) $\Theta_{00},\Theta_{11}$ do encode different levels of
similarity, the scores representing the co-absence and co-occurrence of types
in the same set are higher than types in different set. (3) The $\Theta_{01}$
and $\Theta_{01}$ obey the intrinsic properties and cells with high scores
usually represent the co-exclusion of types from different sets. In general,
$\Theta_{00}$ and $\Theta_{11}$ are less interpretable compared to
$\Theta_{00}$ and $\Theta_{11}$.

Figure 3: Visualization of pairwise potentials. The horizontal axis is for
$y_{k}$, and the vertical axis is for $y_{j}$, for example,
$\theta_{p}(\text{location}=0,\text{downtown}=1)=\Theta_{01}[6,9]$.


# Local spiral structure traced by Red Clump stars

Zehao lin  Ye Xu  Purple Mountain Observatory, Chinese Academy of Sciences,
Nanjing 210008, People’s Republic of China  University of Science and
Technology of China, 96 Jinzhai Road, Hefei 230026, People’s Republic of China
Ligang Hou  National Astronomical Observatories, Chinese Academy of Sciences,
20A Datun Road, Chaoyang District Beijing 100101, PR China  Dejian Liu  Purple
Mountain Observatory, Chinese Academy of Sciences, Nanjing 210008, People’s
Republic of China  University of Science and Technology of China, 96 Jinzhai
Road, Hefei 230026, People’s Republic of China  Yingjie Li  Purple Mountain
Observatory, Chinese Academy of Sciences, Nanjing 210008, People’s Republic of
China  Chaojie Hao  Purple Mountain Observatory, Chinese Academy of Sciences,
Nanjing 210008, People’s Republic of China  University of Science and
Technology of China, 96 Jinzhai Road, Hefei 230026, People’s Republic of China
Jingjing Li  Purple Mountain Observatory, Chinese Academy of Sciences, Nanjing
210008, People’s Republic of China  Shuaibo Bian  Purple Mountain Observatory,
Chinese Academy of Sciences, Nanjing 210008, People’s Republic of China
University of Science and Technology of China, 96 Jinzhai Road, Hefei 230026,
People’s Republic of China

###### Abstract

Using the cross-matched data of Gaia EDR3 and the 2MASS Point Source Catalog,
a sample of RC stars with parallax accuracies better than 20% is identified
and used to reveal the nearby spiral pattern traced by old stars. As shown in
the overdensity distribution of RC stars, there is an arc-like feature
extended from $l\sim$ 90∘ to $\sim$ 243∘, which passed close to the Sun. This
feature is probably an arm segment traced by old stars, indicating the Galaxy
potential in the vicinity of the Sun. By comparing to the spiral arms depicted
by young objects, we found that there are considerable offsets between the two
different components of Galactic spiral arms. The spiral arm traced by RC
stars tends to have a larger pitch angle, hence a more loose wound pattern.

Galaxy structure(622); Galaxy stellar content(621); Solar neighbourhood(1509)

##  1 Introduction

As observers, we are deeply embedded in the Galactic disc, making it a
difficult issue in astronomy to accurately reveal the spiral pattern of the
Milky Way and understand its formation and evolution. In comparison to the
Milky Way Galaxy, external spiral galaxies which are face on to can be
intuitively mapped, studies on which have promoted our understanding of the
Galactic spiral structure. It is suggested that there may be two different
components of spiral arms for a spiral galaxy (Dobbs & Baba, 2014, and
references therein), one component consists of the star formation arms (SF-
arms) traced by diffuse or dense interstellar gas, and also young objects
(e.g., massive OB stars, Hii regions, young open clusters), the other is
composed of a spiral pattern indicated by the potential of old stars
(hereafter P-arms throughout this work). Early views suggested that the
optical (indicating the SF-arms) and near-infrared (hereafter NIR, indicating
the P-arms) morphologies of spiral galaxies were decoupled (e.g. Block &
Puerari, 1999). Later on, the Ohio State University (OSU) Bright Spiral Galaxy
Survey found a good correlation between the optical and NIR band morphologies
for most of their observed spiral galaxies, however, dramatic differences were
also noticed in some cases (Eskridge et al., 2002). A simple picture may not
be applied to all different types of spiral galaxies. With some understanding
of the spiral galaxies, we look back on our own Milky Way. Is there any
similarities and/or differences between two different spiral-arm components of
the Milky Way through different tracers?

Since the 1950s, much efforts have been devoted to depict the spiral structure
of the Milky Way through the distributions or kinematics of OB stars (Morgan
et al., 1952, 1953), neutral hydrogen gas (Christiansen & Hindman, 1952; van
de Hulst et al., 1954; Oort et al., 1958; Bok, 1959; Levine et al., 2006), and
some other kinds of spiral tracers (e.g., Hii regions, CO gas, young open
clusters and Cepheid variable stars Skowron et al. (2019); Poggio et al.
(2021)), most of which were based on kinematic methods. Until the Bar and
Spiral Structure Legacy (BeSSeL) Survey (Reid et al., 2019), which has
obtained precise trigonometric parallaxes (Xu et al., 2006) for about 200
high-mass star-forming regions (HMSFRs), the four segments of SF-arms together
with the Local Arm and several arm spurs have been depicted. This picture of
SF-arms based on HMSFRs was also confirmed or extended by using other young
objects with precise parallax measurements, e.g. massive OB stars (Xu et al.,
2018, 2021) and young open clusters ($<$ 20 Myr, Hao et al., 2021).

In the meantime, a different morphology for the P-arms of the Milky Way was
proposed. The statistics of NIR and far-infrared (FIR) emissions of the
Galactic disc indicate a two-armed spiral pattern, i.e. the Scutum-Centaurus
Arm and the Perseus Arm (Drimmel, 2000; Drimmel & Spergel, 2001). The other
two arms (Sagittarius-Carina and Norma) could perhaps be presented only in gas
and young objects, as they were traced only by the FIR emission. Then, the
Spitzer/GLIMPSE mid-infrared survey also supports that the Milky Way is a
barred, two-armed spiral galaxy (Benjamin et al., 2003, 2005; Churchwell et
al., 2009), because a large number of red clump stars detected by GLIMPSE
present obvious convergence near the tangential directions of the Scutum-
Centaurus Arm, in contrast, the tangential directions corresponding to the
Sagittarius-Carina Arm and Norma Arm are indecipherable. Since the past few
years, the morphological and kinematic properties of spiral arms traced by
stars in the vicinity of the Sun can be explored with the aid of the ESA’s
Gaia mission. By using the Gaia second data release (DR2), Miyachi et al.
(2019) studied the surface density distribution of a number of turn-off stars.
These stars with an age of $\sim$1 Gyr exhibit a marginal arm-like overdensity
in the Galactic longitude of 90${}^{\circ}~{}<~{}l~{}<$ 190∘, which is close
to the Local Arm traced by gas and young objects, but with a slightly larger
pitch angle. Khoperskov et al. (2020) identified six stellar density
structures in a guiding coordinate space based on the Gaia DR2 sources having
6D phase-space coordinates. These structures were interpreted as features
related to spiral arms and resonances. Meanwhile, Hunt et al. (2020) presented
a different viewpoint. They suggested that the stellar density structures
found by Khoperskov et al. (2020) are known kinematic moving groups, rather
than spiral arms. Poggio et al. (2021) derived the density distribution of a
large number of upper main sequence stars (O, B, and A-type, Poggio et al.,
2018), which cover a wide age range. The resulting maps exhibit extended arm-
like structures indicating the arm segments in the Solar neighborhood. In
addition, Hao et al. (2021) confirmed that the distribution of old open
clusters (ages $>$ 1 Gyr) in the vicinity of the Sun does not present any
obvious arm-like features.

The difference between the SF-arms and P-arms is a fundamental characteristic
of the Milky Way, closely related to the formation mechanism of Galactic
spiral structure, but still controversial in observation. To resolve this
issue, high precision distance measurements for a large number of old stars
are necessary. By taking advantage of the latest dataset of Gaia Early Data
Release 3 (hereafter Gaia EDR3, Gaia Collaboration et al., 2021), it is now
possible to identify a large number of red clump stars with high-precision
parallaxes, and depict the nearby P-arms traced by older stars (ages $\sim$2
Gyr). In this work, we present the results and compare them with the SF-arms
indicated by gas and young objects, aim to better understand the properties of
spiral structure in the Milky Way.

This work is organized as follows. The sample of red clump stars is identified
and described in Sect. 2. Their distribution on the Galactic disc and the
properties of P-arms are given in Sect. 3. Discussions and Conclusions are in
Sect. 4.

##  2 Sample of red clump stars

In the late evolutionary stage of low mass ($<$ 2.2 M⊙) stars, they will
undergo core-helium and shell-hydrogen burning, and evolve to the phase of red
clump stars, which have a narrow range of intrinsic luminosities and colors
(Girardi, 2016). Red clump (RC) stars account for a large proportion of the
cataloged GLIMPSE stars (Churchwell et al., 2009). As discussed in Sect. 1,
the observational evidence for the two-armed picture of Galactic P-arms is
primarily from the statistics of the surface density of GLIMPSE RC stars as a
function of the Galactic longitude (Churchwell et al., 2009). The RC K-giants
observed in K-band are always dominated by objects with ages of around 2 Gyr
(Salaris, 2012). With a median absolute magnitude $M_{\mathrm{K_{S}}}$ =
$-$1.61 mag (Alves, 2000; Cabrera-Lavers et al., 2007) and $M_{\mathrm{G}}$ =
0.5 mag (Gaia Collaboration et al., 2018; Andrae et al., 2018), RC stars
within a few kpc of the Sun can be detected by Gaia, even in the presence of
interstellar dust extinction, making them excellent tracers for the properties
of P-arms in the Solar vicinity.

Figure 1: Color-magnitude diagrams: (Left) $M_{K_{S}}$ vs. $(J-K_{S})_{0}$;
(Right) $M_{G}$ vs. $({G_{bp}-G_{rp}})_{0}$, for the selected Gaia
EDR3$-$2MASS stars with $|Z|$ $<$ 0.3 kpc. To plot the diagrams, the cell size
is set to be 0.1 mag. Only the cells containing more than 100 stars are
presented. The black dashed boxes indicate the aggregation area of RC stars
adopted in this work.

###  2.1 Sample selection

The Two Micron All Sky Survey (2MASS, Skrutskie et al., 2006) observed the
entire celestial sphere in three NIR bands, i.e. J (1.25 $\mu$m), H (1.65
$\mu$m), and KS (2.16 $\mu$m). The Gaia EDR3 has obtained astrometric
parameters for about 1.468 billion stars (Gaia Collaboration et al., 2016,
2021), with a typical parallax uncertainty of 0.02$-$0.03 mas for G $<$ 15,
and 0.07 mas at G$=$17\. By combining the astrometric/photometric data of Gaia
EDR3 and the NIR photometric data of 2MASS, a large number of RC stars can be
identified.

As the first step, we extract the sources by using the official Gaia
EDR3$-$2MASS cross-match best neighbor table (Marrese et al., in
prep.,111https://archives.esac.esa.int/gaia Marrese et al., 2019). For the
2MASS sources in the Point Source Catalog (Skrutskie et al., 2006), only those
stars with $J$ and $K_{S}$ band magnitudes brighter than 14 mag are retained
in consideration of the sensitivity. Considering the completeness of the
cross-matched sample of Gaia EDR3$-$2MASS (Bennett & Bovy, 2019), the Gaia
sources with G band magnitude brighter than 17 mag are selected. It is also
required that the sources should have relative parallax uncertainties
$\varpi$/$\sigma_{\varpi}>$ 5, where $\varpi$ and $\sigma_{\varpi}$ are the
parallax and its uncertainty without systematic error, respectively. Then, the
distance $d$ is simply calculated through $d~{}=~{}1/(\varpi-\varpi_{0})$, and
$\varpi_{0}$ represents the systematic zero point offset of the Gaia EDR3
parallaxes (Lindegren et al., 2021). As we focus on the properties of spiral
arms in the Galactic disc, the stars with $|Z|$ $>$ 0.3 kpc are excluded.
Here, $Z$ is the vertical height.

To derive the color-magnitude diagrams for classifying RC stars, extinction
parameters for these sources need to be corrected. In this work, we adopt the
extinction parameters corrected with the
MWDUST222https://github.com/jobovy/mwdust (Bovy et al., 2016) algorithm, which
combines the three dimensional (3D) dust maps of Marshall et al. (2006), Green
et al. (2019), and Drimmel et al. (2003). This algorithm can determine the
extinction value at a given Galactic longitude, latitude and distance.
Besides, the extinction curve with $R_{V}$ = 3.1 (Cardelli et al., 1989;
O’Donnell, 1994) is adopted. Together with the mean wavelengths of 2MASS $J$
and $K_{S}$ bands (Cohen et al., 2003), Gaia $G$, $G_{bp}$, and $G_{rp}$ bands
(Weiler, 2018), we can derive the extinction coefficient relations, they are
$A_{J}$ = 2.5 $A_{K_{S}}$, $E({J-K_{S}})$ = 1.5 $A_{K_{S}}$, $A_{G}$ = 7.2
$A_{K_{S}}$, and $E({G_{bp}-G_{rp}})$ = 0.5 $A_{G}$. The calculated relation
of $E({G_{bp}-G_{rp}})$ = 0.5 $A_{G}$ is well consistent with that of Andrae
et al. (2018).

Figure 2: KS-band extinction of the selected RC stars as a function of the
distance to the Sun for the inner (Left) and the outer Galaxy regions (Right).
The contours indicated by dotted-, dashed-, and solid-lines enclose the 99%,
95%, and 80% of the RC stars in our sample, respectively.

The absolute color and magnitude for a matched Gaia EDR3$-$2MASS source can be
calculated via:

$\displaystyle
M_{K_{S}}~{}=~{}m_{K_{S}}~{}+~{}5~{}\times~{}\log{(\varpi-\varpi_{0})}~{}-~{}10~{}-~{}A_{K_{s}},$
(1) $\displaystyle({J-K_{S}})_{0}~{}=~{}({J-K_{S}})_{obs}~{}-~{}E({J-K_{S}}),$
(2) $\displaystyle
M_{G}~{}=~{}m_{G}~{}+~{}5~{}\times~{}\log{(\varpi-\varpi_{0})}~{}-~{}10~{}-~{}A_{G},$
(3)
$\displaystyle({G_{bp}-G_{rp}})_{0}~{}=~{}({G_{bp}-G_{rp}})_{obs}~{}-~{}E({G_{bp}-G_{rp}}).$
(4)

The absolute color-magnitude diagrams for the selected sample is presented in
Fig. 1. In the diagrams, the RC stars are relatively isolated from other types
of stars, hence can be easily identified (Girardi, 2016). It should be
mentioned that, the actual extinction of a specific star depends on its
spectral type and also the extinction itself (e.g., Jordi et al., 2010), which
may deviate from the statistical extinction distribution of MWDUST. This
effect results in the stretch of aggregation area of RC stars in the color-
magnitude diagrams. Taking this into account, we select the RC stars by using
their expected colour dependency and also the extinction uncertainty. The red
clump stars typically have a median absolute magnitude $M_{K_{S},RC}$ =
$-$1.61 mag, with a Gaussian width of $\sigma_{K_{S}}$ = 0.22 mag (Alves,
2000; Cabrera-Lavers et al., 2007), and an intrinsic colour of
$(J-K_{S})_{0,RC}$ = 0.65 mag. Considering a 2$\sigma$ confidence level, the
stars with $M_{K_{S}}$ falling within [$-$2.05,$-$1.17] mag are selected as
candidates of red clump stars. Then, the sample selection box is obtained in
consideration of the properties of RC stars along the extinction direction as
shown in Fig. 1, where the black solid line corresponds to the offset of
median magnitude due to the extinction uncertainty. In this work, the RC stars
are selected by requiring that 0.3 $<$ $(J-K_{S})_{0}$ $<$ 1.2, $-$3.725 $<$
1.5 $\times$ $M_{K_{S}}$ $-$ $(J-K_{S})_{0}$ $<$ $-$2.405, 0.8 $<$
$(G_{bp}-G_{rp})_{0}$ $<$ 2.3, and $-$2.4 $<$ $M_{G}$ $-$ 2 $\times$
$(G_{bp}-G_{rp})_{0}$ $<$ $-$1.4, as indicated by the black dashed boxes in
Fig. 1. In total, 2,685,075 candidates of RC star are selected by these
criteria. To evaluate the influence of the adopted 3D extinction model, we
also reproduce the sample of RC stars with a different 3D extinction algorithm
of Amores et al. (2021), and found that the derived structure features in the
overdensity maps of RC stars are consistent.

To inspect the reliability of the sample, we cross-matched the selected
candidates of RC stars with the catalogue of Sanders & Das (2018). Sanders &
Das (2018) obtained the parameters of distances, masses and ages for about 3
million stars from Gaia DR2, based on the spectroscopic parameters derived
from several large spectroscopic surveys. We found that there are 92,525
matched sources, 87,623 ($\sim$ 94.7%) of them are giant stars with a typical
age of $\sim$3 Gyr. It supports that the overwhelming majority of our selected
sources are indeed old stars.

Figure 3: Left: number density distribution of RC stars projected onto the
Galactic plane. middle: overdensity distribution of the RC stars with
$\varpi$/$\sigma_{\varpi}$ $>$ 5 and $|Z|$ $<$ 0.3 kpc, calculated with a
local density scale length of 0.3 kpc and a mean density scale length of 2
kpc. Only the points with $\Sigma(X,Y)$ $>$ 0.003 are presented in
consideration of statistics. Right: simlar with middle panel, while presents
the overdensity distribution of the RC stars with $\varpi$/$\sigma_{\varpi}$
$>$ 5 and $|Z|$ $>$ 0.5 kpc. The Galactic centre is at (0,0) kpc, and the Sun
(Sun symbol) is located at (0, 8.15) kpc. The black solid line presents the
fitted Local P-arm (see detail in Sect. 3.2) and the dashed line presents the
Local Arm fitted by Miyachi et al. (2019). The orange dash line contour the
region that RC stars are completed.

###  2.2 Sample completeness

An important issue in uncovering the spiral arms with optically or NIR
selected stars is the sample completeness. It depends on the dust extinction,
which is related with the spiral structure to some degree. In Fig. 2, we
present the extinction $A_{K_{S}}$ of the selected RC stars in this work.
Considering that the dust extinction in the inner Galaxy region
($-90^{\circ}~{}<~{}l~{}<90^{\circ}$) is more severe than the outer Galaxy
($90^{\circ}~{}<~{}l~{}<270^{\circ}$), we simply divide the RC sample into two
different parts. For the vast majority of the selected RC stars ($>$ 99%), the
extinction in the direction of the inner Galaxy is less than 0.54 mag, and
less than 0.50 mag for the outer Galaxy direction. For more distant stellar
objects, extinction values are bound to be greater than the maximum of the
current sample.The completeness of samples drops at the farther distance.
Considering that Gaia dataset of stars is complete down to at least $G=$ 17
mag, as shown in Fig. 2, the RC stars selected in this work is believed to be
complete at $d\lesssim$ 2.2 kpc for the inner Galaxy direction, and at
$d\lesssim$ 2.7 kpc for the outer Galaxy regions. At $G=$ 17 mag, the median
error of the Gaia EDR3 parallax is around 0.07. At 2.2$/$2.7 kpc the parallax
uncertainties (6.5$/$5.3) is large than 5. Fig. 2 also shows the situation
that the sample completeness is insufficient. The high extinction data is
missing beyond the distance far from the sample completeness (the red dashed
lines in Fig. 2). Therefore, at present, we only discuss the distribution of
old stars within the limits as indicated by sample completeness limit. Hence,
the dataset of RC stars is expected to depict the properties of the nearby
arm-segments of the Local arm. At larger distances (e.g., $d\gtrsim 3$ kpc),
the sample is not complete, but a large number of RC stars are identified.

##  3 Distribution in the Galactic plane

The distribution of RC stars projected onto the Galactic plane is shown in
left panel of Fig. 3. In the studied area of the Galactic disc, the number
density of RC stars decreases as the increase of Galactocentric distance. At a
Galactocentric distance of $R_{\mathrm{GC}}\sim$ 6 kpc, where presents an
aggregation of RCs, the number density of RCs is about 1500 per 0.01 kpc2, and
drops to less than 500 per 0.01 kpc2 at $R_{\mathrm{GC}}\sim$ 10 kpc. From the
number density distribution, no obvious arm-like features are discernible.
This is not unexpected, as the features shown in the left panel of Fig. 3 is
completely dominated by the global density profile of stars in the Galactic
disc, which is roughly exponential. The P-arms indicated by the overdensities
of old stars would be less significant and concealed beneath the global
density profile. In addition, in comparison to the gas component or young
objects, the velocity dispersion of old stars is larger, the overdensities of
old stars (e.g., indicating the P-arms) causing by any perturbations in the
Galactic disc would be less sharp and more difficult to identify from
observations. In order to reveal the possible underlying P-arms, a bivariate
kernel density estimator method (Feigelson & Babu, 2012; Poggio et al., 2021)
is adopted. This method may be also helpful to reveal the underlying
overdensity structures outside the completeness limits of the RC star sample,
because the P-arms are expected to leave some marks on the distribution of the
detected distant stars.

Following Poggio et al. (2021), the stellar overdensity, $\Delta_{\Sigma}$, is
calculated by:

$\displaystyle\Delta_{\Sigma}(X,Y)=\frac{\Sigma(X,Y)}{\langle\Sigma(X,Y)\rangle}-1;$
(5)
$\displaystyle\Sigma(X,Y)=\frac{1}{N~{}h^{2}}\sum^{N}_{i=1}\left[K\left(\frac{X-x_{i}}{h}\right)~{}K\left(\frac{Y-y_{i}}{h}\right)\right];$
(6) $\displaystyle
K\left(\frac{X-x_{i}}{h}\right)=\frac{3}{4}\left(1-\left(\frac{X-x_{i}}{h}\right)^{2}\right).$
(7)

Here, $\Sigma(X,Y)$ is the local density at the position ($X,Y$) in the
Galactic disc, $K$ is the kernel function, $h$ is the kernel bandwidth which
is adopted as 0.3 kpc in this work, ($x_{i},y_{i}$) is the coordinate of the
$i$-th RC star, and $N$ is the total number of RC stars involved in the
calculation. The mean density $\langle\Sigma(X,Y)\rangle$ is calculated in a
similar way as $\Sigma(X,Y)$ but with $h=$ 2 kpc. The result is given in the
middle panel of Fig. 3.

As shown in the overdensity maps in middle panel of Fig. 3, there is a weak
arm-like feature extended from $l\sim$ 90∘ to $l\sim$ 243∘, passed close to
the Sun. Most part of this feature is within the completeness limit of the RC
sample ($\sim$ 2.2$-$2.7 kpc, see Sect. 2.2), hence is very likely a real
structure (as the black solid line shown in the middle panel in Fig. 3). This
feature is close to the Local Arm depicted by gas and young objects. Besides,
for comparision, we aslo show the overdensity maps of RC stars away from the
Galactic plane ($|Z|>$ 0.5 kpc) in right panel of Fig. 3. The arm-like
structure is only present near the Galactic plane. This may imply that the
disk stars show a spiral response under the action of spiral potential for a
stellar disc (Lin & Shu, 1964; Kalnajs, 1965; Dobbs & Baba, 2014). When the
stars are far away from the Galactic plane, the influence of spiral potential
weakens, and the overdensity distribution shows a relatively uniform
distribution.

In addition, there are obvious overdensity of RC stars in the regions far away
from the Sun. There may be two different explanations about these overdensity
features. One is the influence of the sample completeness, as many of these
overdensity regions are close to the completeness limit of the RC stars.
Another explanation is the spiral structure traced by old stars. We look
forward to verifying these structures with more complete samples in the
future.

###  3.1 Comparison with previous works

With a sample of turn-off stars of ages $\sim$ 1 Gyr, Miyachi et al. (2019)
identified a marginal arm-like overdensity in the Galactic longitude range of
90${}^{\circ}~{}<~{}l~{}<$ 190∘. This feature is confirmed with the
overdensity map of RC stars of ages $\sim$ 2 Gyr. As shown in the middle panel
of Fig. 3, the arm-like overdensity proposed by Miyachi et al. (2019) is part
of the intermediate arm-like features identified with RC stars in the same
longitude range.

Recently, Poggio et al. (2021) derived the density distribution of a large
number of upper main sequence stars (UMS stars). Their upper main sequence
stars consist of O-, B- and A-type stars as discussed in Poggio et al. (2018).
We overplotted their results on the overdensity map of RC stars in Fig. 4. The
morphology of the overdensity regions given by Poggio et al. (2021) is
generally similar to that of RC stars. Three arm-segmemts were proposed by
Poggio et al. (2021) based on the distribution of UMS stars, which is similar
to our interpretation about the overdensity map of RC stars.

Figure 4: Overdensity distribution of RC stars as the middle panel of Fig. 3,
overplotted with the overdensity map of UMS stars (Poggio et al., 2021). The
black contours represent the overdensity levels of 0 (dotted), 0.2 (dashed),
and 0.4 (solid) of UMS stars, respectively. The orange dash line contour the
region that RC stars are completed.

With UMS stars, Poggio et al. (2021) identified the Sagittarius-Carina Arm and
also the Scutum-Centaurus Arm (also see their Fig. 2). RC stars also presents
over-density in the Scutum-Centaurus Arm and Perseus Arm identified by Poggio
et al. (2021), while it is outside the complete region. We only show the
coincidence of this structure here.

Due to the limitation of sample completeness at present, we only compare RC
stars and UMS stars in area of high completeness (orange contour region in
Fig. 4). In the overdensity map of UMS stars, the arm-like feature near the
Sun is more obvious. We speculate that, the main cause is that the sample used
by Poggio et al. (2021) is a combination of stars covering a wide range of
ages, e.g., the typical lifetime for a B-type star is $\sim$ 0.1 Gyr, and
$\sim$ 1 Gyr for a A-type star, comparable to that of RC stars ($\sim$ 2 Gyr).
As discussed in Poggio et al. (2018), approximately 55% of their UMS stars are
OB type, and 40% are A-type stars. Therefore, the overdensity map of UMS stars
seems to present combined features of young objects and old stars. The older
OCs (e.g., ages $>$ 0.2 Gyr) show a different distribution than the populous
OCs with ages of tens of million years (Hao et al., 2021), this may indicate
the dynamic evolution or state of star move out of spiral SF-arms. For the
purpose of revealing the possible differences between SF-arms and P-arms in
the Milky Way, the RC stars adopted in this work would be a suitable dataset
for illustrating the properties of P-arms.

Figure 5: Same as the middle panel of Fig. 3, overplotted with the fitted
Local P-arm (black solid line) and the dashed lines present the SF-arm fitted
by HMSFR masers (Reid et al., 2019). The SF-arm of the Outer Arm (red), the
Perseus Arm (black), the Local Arm (dark cyan), the Sagittarius-Carina Arm
(magenta), and the Scutum-Centaurus Arm (blue) are indicated with different
colors. The green dots present the local maxima of the Local P-arm in
different $\beta$. The orange dash line contour the region that RC stars are
completed.

Figure 6: Panel a: The overdensity map of RC stars, but overlaid with the
distributions of young OCs, massive O-type stars and HMSFR masers having
accurate parallax distances. Profiles of the overdensity of RC stars as a
function of the Galactocentric distance ($R_{\mathrm{GC}}$) along different
slices ($-15^{\circ}$, $-10^{\circ}$, $-5^{\circ}$, $0^{\circ}$, $+5^{\circ}$,
$+10^{\circ}$, and $+15^{\circ}$) are presented. Each slice corresponds to a
constant Galactocentric azimuth angle $\beta$. Error bars show the
uncertainties calculated by 100 times of resamples. Additionally, a scaled
overdensity profile of young objects as a function of the Galactocentric
distance (blue solid line) is also shown for each $\beta$ slice, in order to
make a comparison with that of the RC stars (red solid line, the overdensity
values and errors of RC stars in this figure are multiplied by a factor of 2).
Between the two orange vertical dashed lines is the complete interval of the
sample of RC stars.

###  3.2 Local P-arm revealed by RC stars

A discernible feature passes close to the Sun, with a over-density in the
azimuth angle range of $-15^{\circ}\lesssim\beta\lesssim 15^{\circ}$. The
overdensity regions from (X,Y)$\sim$(2,8) kpc to ($-$2,9) kpc are close to the
Local Arm, and also possibly related with an arm-segment. As this feature is
also shown by Miyachi et al. (2019) and Poggio et al. (2021) with different
stellar samples, it is very likely a real arm-like structure. This arm-like
feature is probably corresponding to the segments of the Local P-arm traced by
older stars of $\sim$2 Gyr. The Galactocentric distance of local maxima
corresponding to the Local P-arm along different azimuth angle (i.e., $\beta$
= $-15^{\circ}$, $-10^{\circ}$, $-5^{\circ}$, $+5^{\circ}$, $+10^{\circ}$, and
$+15^{\circ}$) are 9.75, 9.35, 9.3, 8.7, 8.55, 8.40 kpc, respectively. While
at $\beta$ = $0^{\circ}$, there seems a spur to connect the Local P-arm and
outer over-density arc-like feature. Thus, we discarded the local maxima in
this azimuth angle. Under the assumption that they are the segments of nearby
P-arms, we can use logarithmic spiral-arm model,
$\ln(R/R_{ref})=-(\beta-\beta_{ref})\tan{\psi}$, to fit this structures (as
green dots scattered in Fig. 5). The fitted pitch angle 15$\fdg$9 $\pm$
1$\fdg$1 for the Local P-arm. This fitted pitch angle is significantly larger
than that of the Local ($\psi\sim 11^{\circ}$) (Reid et al., 2019), as shown
in Fig. 5. By researching turn-off stars in the Solar vicinity, Miyachi et al.
(2019) found that the pitch angle of the P-arm is slightly larger than the SF-
arm. The results for RC stars also consist with this result. The Milky Way’s
old stars may exhibit a more loose spiral structure.

###  3.3 Comparison with SF-arms

As discussed above, the two different components of spiral arms (SF-arms and
P-arms) in a galaxy may be decoupled in their positions. In order to reveal
the possible differences between the identified P-arms and the SF-arms of the
Milky Way, we compare the distribution of young objects with the overdensity
map of RC stars.

The segments of SF-arms traced by young objects (i.e. with ages less than
$\sim$ 20 Myr) have been updated and well studied in the past few years, by
using HMSFR masers, OB-type stars and young open clusters with accurate
parallax distances. For these young objects, we adopted the data given by Reid
et al. (2019), Xu et al. (2021) and Hao et al. (2021). The samples of young
objects are also suffered from the completeness issues. The available dataset
of young objects have been used to trace the SF-arms in the Galaxy disc as far
as about 5 kpc from the Sun. The results are shown in Fig. 6. The prominent
feature is that the segments of P-arms traced by RC stars (ages $\sim$ 2 Gyr)
are obviously deviated from that of SF-arms, especially for the intermediate
arm-like feature, which is not suffered from the sample completeness issue
significantly. To indicate it clearly, a bivariate kernel density estimator
method is also applied to the young objects, then used to calculate the
overdensity profile of young objects as a function of the Galactocentric
distance along the slices with azimuth angle of $-15^{\circ}$, $-10^{\circ}$,
$-5^{\circ}$, $0^{\circ}$, $+5^{\circ}$, $+10^{\circ}$, and $+15^{\circ}$.
Take the result of $\beta$ = $+10^{\circ}$ as an example (see Fig. 6).

The young objects gather in the areas with $R_{\mathrm{GC}}$ = 6.2, 8.0 and
9.9 kpc, correspond to the SF-arms of Sagittarius-Carina, Local and Perseus,
respectively. In comparison, the overdensity of RC stars appear near
$R_{\mathrm{GC}}$ = 8.5 kpc within the area of high completeness. As mentioned
in Sect. 3.2, this area may correspond to the Local P-arm. There is an obvious
outward drift between the Local P-arm and SF-arm. In previous works, the
knowledge of P-arms of the Milky Way is primarily from analyzing the profiles
of integrated NIR or FIR emission/sources as a function of Galactic longitude
(Drimmel, 2000; Drimmel & Spergel, 2001; Benjamin et al., 2005; Churchwell et
al., 2009). The P-arms in the Galactic disc indicate a two-major armed spiral
pattern, i.e. the Scutum-Centaurus Arm and the Perseus Arm (Drimmel, 2000;
Drimmel & Spergel, 2001). However, there is no obvious over-density of RC
stars in the area of $R_{\mathrm{GC}}$ = 9.9 kpc, which is correspond to the
SF-arms of Perseus Arm. There may be the similar outward shift of the Perseus
P-arm. We look forward to a larger and more complete sample to obtain the
distribution of Perseus P-arm. Similar properties are also noticed for the
other $\beta$ slices, implying that the offsets between SF-arms and P-arms in
the vicinity of the Sun are general phenomena.

As shown by studies on some external galaxies (e.g., Yu & Ho, 2018), the
possible offsets between SF-arms and P-arms are generally small, especially
for the regions close to the galaxy co-rotation radius. It is believed that
the co-rotation radius of the Milky Way is close to the Solar circle (e.g.,
Dias et al., 2019). Under the circumstances, the P-arm of the Local arm traced
by RC stars have larger pitch angles than SF-arms, and tend to shift outward
toward the anti-Galactic centre direction. The offset between SF-arm and P-arm
are increased as the arm segments spiral outward.

##  4 Discussions and Conclusions

In this work, a large number of RC stars with ages $\sim$2 Gyr is identified
by using the survey data of 2MASS and Gaia EDR3. In the Solar vicinity, we
found that there are obvious overdensity regions of RC stars. A picture
composed of three features could be used to interpret these overdensity
features. Two of the three structures are located outside the completeness
limit of the data sample. And one of the three structures probably correspond
to the segments of P-arms in the vicinity of the Sun, which have not been well
traced in previous works. This arm-like features possibly correspond to the
Local P-arm.

In comparison to the P-arms, the properties of SF-arms in the Milky Way have
been better illustrated. By comparing the overdensity distributions of RC
stars with that of young objects, we confirm that there are obvious offsets
between the segments of P-arms and SF-arms in the vicinity of the Sun even
though the completeness issue of the sample is considered to some degree. The
segment of the Local Arm traced by RC stars has larger pitch angles than that
of SF-arms, tend to spiral outward toward the anti-Galactic centre direction.
From the studies on external spiral galaxies, it is found that the SF-arms of
galaxies statistically have smaller pitch angles than that of P-arms (e.g., Yu
& Ho, 2018), hence with a more tightly wound pattern. However, the relation
between P-arm and SF-arm for the Local seems contradict to the measured arm
tangential directions and corotation radius of the Milky Way. Based on the
density wave theory, the azimuthal offsets between P-arms and SF-arms will be
opposite for the Galaxy regions inside and outside the corotation radius
(Roberts, 1969; Yuan & Grosbol, 1981; Martínez-García et al., 2009). The
corotation radius for the Milky Way is around the Solar circle. In the inner
Galaxy, the P-arms traced by old stars have also been found to be outside the
corresponding SF-arms by analyzing the arm tangential directions (e.g., Hou &
Han, 2015; Vallée, 2014, 2016, 2018), which is similar to the properties for
the Local P-arm noticed in this work. However, it is not clear whether the
systematic spatial offsets or age pattern exist or not (Vallée, 2018; He et
al., 2021) beyond the outer Galaxy. More tests based on observations are
needed and such contradictions are expected to be solved by more complete
samples of RC stars and young objects with accurately measured distances.

We would like to thank the anonymous referee for the helpful comments and
suggestions that helped to improve the paper. This work was funded by the NSFC
Grands 11933011, 11873019 and 11988101, the Natural Science Foundation of
Jiangsu Province (Grants No. BK20210999) and the Key Laboratory for Radio
Astronomy. L.G.H. thanks the support from the Youth Innovation Promotion
Association CAS. L.Y.J. thanks the support of the Entrepreneurship and
Innovation Program of Jiangsu Province. This work has made use of data from
the European Space Agency(ESA) mission Gaia (https://www.cosmos.esa.int/gaia),
processed by the Gaia Data Processing and Analysis Consortium (DPAC,
https://www.cosmos.esa.int/web/gaia/dpac/consortium). Funding for the DPAC has
been provided by national institutions, in particular the institutions
participating in the Gaia Multilateral Agreement. This publication makes use
of data products from the Two Micron All Sky Survey, which is a joint project
of the University of Massachusetts and the Infrared Processing and Analysis
Center/California Institute of Technology, funded by the National Aeronautics
and Space Administration and the National Science Foundation.

## References

  * Alves (2000) Alves, D. R. 2000, ApJ, 539, 732. doi:10.1086/309278
  * Amores et al. (2021) Amores, E. B., Jesus, R. M., Moitinho, A., et al. 2021, MNRAS, 508, 1788. doi:10.1093/mnras/stab2248
  * Andrae et al. (2018) Andrae, R., Fouesneau, M., Creevey, O., et al. 2018, A&A, 616, A8. doi:10.1051/0004-6361/201732516
  * Benjamin et al. (2003) Benjamin, R. A., Churchwell, E., Babler, B. L., et al. 2003, PASP, 115, 953. doi:10.1086/376696
  * Benjamin et al. (2005) Benjamin, R. A., Churchwell, E., Babler, B. L., et al. 2005, ApJ, 630, L149. doi:10.1086/491785
  * Bennett & Bovy (2019) Bennett, M. & Bovy, J. 2019, MNRAS, 482, 1417. doi:10.1093/mnras/sty2813
  * Block & Puerari (1999) Block, D. L. & Puerari, I. 1999, A&A, 342, 627
  * Bok (1959) Bok, B. J. 1959, The Observatory, 79, 58
  * Bovy et al. (2016) Bovy, J., Rix, H.-W., Green, G. M., et al. 2016, ApJ, 818, 130. doi:10.3847/0004-637X/818/2/130
  * Cabrera-Lavers et al. (2007) Cabrera-Lavers, A., Hammersley, P. L., González-Fernández, C., et al. 2007, A&A, 465, 825. doi:10.1051/0004-6361:20066185
  * Cardelli et al. (1989) Cardelli, J. A., Clayton, G. C., & Mathis, J. S. 1989, ApJ, 345, 245. doi:10.1086/167900
  * Christiansen & Hindman (1952) Christiansen, W. N. & Hindman, J. V. 1952, Australian Journal of Scientific Research A Physical Sciences, 5, 437. doi:10.1071/PH520437
  * Churchwell et al. (2009) Churchwell, E., Babler, B. L., Meade, M. R., et al. 2009, PASP, 121, 213. doi:10.1086/597811
  * Cohen et al. (2003) Cohen, M., Wheaton, W. A., & Megeath, S. T. 2003, AJ, 126, 1090. doi:10.1086/376474
  * Dias et al. (2019) Dias, W. S., Monteiro, H., Lépine, J. R. D., et al. 2019, MNRAS, 486, 5726. doi:10.1093/mnras/stz1196
  * Dobbs & Baba (2014) Dobbs, C. & Baba, J. 2014, PASA, 31, e035. doi:10.1017/pasa.2014.31
  * Drimmel (2000) Drimmel, R. 2000, A&A, 358, L13
  * Drimmel & Spergel (2001) Drimmel, R. & Spergel, D. N. 2001, ApJ, 556, 181. doi:10.1086/321556
  * Drimmel et al. (2003) Drimmel, R., Cabrera-Lavers, A., & López-Corredoira, M. 2003, A&A, 409, 205. doi:10.1051/0004-6361:20031070
  * Eskridge et al. (2002) Eskridge, P. B., Frogel, J. A., Pogge, R. W., et al. 2002, ApJS, 143, 73. doi:10.1086/342340
  * Feigelson & Babu (2012) Feigelson, E. D. & Babu, G. J. 2012, Modern Statistical Methods for Astronomy, by Eric D. Feigelson , G. Jogesh Babu, Cambridge, UK: Cambridge University Press, 2012
  * Gaia Collaboration et al. (2016) Gaia Collaboration, Prusti, T., de Bruijne, J. H. J., et al. 2016, A&A, 595, A1. doi:10.1051/0004-6361/201629272
  * Gaia Collaboration et al. (2018) Gaia Collaboration, Babusiaux, C., van Leeuwen, F., et al. 2018, A&A, 616, A10. doi:10.1051/0004-6361/201832843
  * Gaia Collaboration et al. (2021) Gaia Collaboration, Brown, A. G. A., Vallenari, A., et al. 2021, A&A, 649, A1. doi:10.1051/0004-6361/202039657
  * Girardi (2016) Girardi, L. 2016, ARA&A, 54, 95. doi:10.1146/annurev-astro-081915-023354
  * Green et al. (2019) Green, G. M., Schlafly, E., Zucker, C., et al. 2019, ApJ, 887, 93. doi:10.3847/1538-4357/ab5362
  * Hao et al. (2021) Hao, C. J., Xu, Y., Hou, L. G., et al. 2021, A&A, 652, A102. doi:10.1051/0004-6361/202140608
  * He et al. (2021) He, Z.-H., Xu, Y., & Hou, L.-G. 2021, Research in Astronomy and Astrophysics, 21, 009. doi:10.1088/1674-4527/21/1/9
  * Hou & Han (2015) Hou, L. G. & Han, J. L. 2015, MNRAS, 454, 626. doi:10.1093/mnras/stv1904
  * Hunt et al. (2020) Hunt, J. A. S., Johnston, K. V., Pettitt, A. R., et al. 2020, MNRAS, 497, 818. doi:10.1093/mnras/staa1987
  * Jordi et al. (2010) Jordi, C., Gebran, M., Carrasco, J. M., et al. 2010, A&A, 523, A48. doi:10.1051/0004-6361/201015441
  * Kalnajs (1965) Kalnajs, A. J. 1965, Ph.D. Thesis
  * Khoperskov et al. (2020) Khoperskov, S., Gerhard, O., Di Matteo, P., et al. 2020, A&A, 634, L8. doi:10.1051/0004-6361/201936645
  * Levine et al. (2006) Levine, E. S., Blitz, L., & Heiles, C. 2006, Science, 312, 1773. doi:10.1126/science.1128455
  * Lin & Shu (1964) Lin, C. C. & Shu, F. H. 1964, ApJ, 140, 646. doi:10.1086/147955
  * Lindegren et al. (2021) Lindegren, L., Bastian, U., Biermann, M., et al. 2021, A&A, 649, A4. doi:10.1051/0004-6361/202039653
  * Marrese et al. (2019) Marrese, P. M., Marinoni, S., Fabrizio, M., et al. 2019, A&A, 621, A144. doi:10.1051/0004-6361/201834142
  * Marshall et al. (2006) Marshall, D. J., Robin, A. C., Reylé, C., et al. 2006, A&A, 453, 635. doi:10.1051/0004-6361:20053842
  * Martínez-García et al. (2009) Martínez-García, E. E., González-Lópezlira, R. A., & Bruzual-A, G. 2009, ApJ, 694, 512. doi:10.1088/0004-637X/694/1/512
  * Miyachi et al. (2019) Miyachi, Y., Sakai, N., Kawata, D., et al. 2019, ApJ, 882, 48. doi:10.3847/1538-4357/ab2f86
  * Morgan et al. (1952) Morgan, W. W., Sharpless, S., & Osterbrock, D. 1952, AJ, 57, 3. doi:10.1086/106673
  * Morgan et al. (1953) Morgan, W. W., Whitford, A. E., & Code, A. D. 1953, ApJ, 118, 318. doi:10.1086/145754
  * O’Donnell (1994) O’Donnell, J. E. 1994, ApJ, 422, 158. doi:10.1086/173713
  * Oort et al. (1958) Oort, J. H., Kerr, F. J., & Westerhout, G. 1958, MNRAS, 118, 379. doi:10.1093/mnras/118.4.379
  * Poggio et al. (2018) Poggio, E., Drimmel, R., Lattanzi, M. G., et al. 2018, MNRAS, 481, L21. doi:10.1093/mnrasl/sly148
  * Poggio et al. (2021) Poggio, E., Drimmel, R., Cantat-Gaudin, T., et al. 2021, A&A, 651, A104. doi:10.1051/0004-6361/202140687
  * Reid et al. (2019) Reid, M. J., Menten, K. M., Brunthaler, A., et al. 2019, ApJ, 885, 131. doi:10.3847/1538-4357/ab4a1
  * Roberts (1969) Roberts, W. W. 1969, ApJ, 158, 123. doi:10.1086/1501771
  * Salaris (2012) Salaris, M. 2012, Ap&SS, 341, 65. doi:10.1007/s10509-011-0927-z
  * Sanders & Das (2018) Sanders, J. L. & Das, P. 2018, MNRAS, 481, 4093. doi:10.1093/mnras/sty2490
  * Skowron et al. (2019) Skowron, D. M., Skowron, J., Mróz, P., et al. 2019, Science, 365, 478. doi:10.1126/science.aau3181
  * Skrutskie et al. (2006) Skrutskie, M. F., Cutri, R. M., Stiening, R., et al. 2006, AJ, 131, 1163. doi:10.1086/498708
  * Vallée (2014) Vallée, J. P. 2014, AJ, 148, 5. doi:10.1088/0004-6256/148/1/5
  * Vallée (2016) Vallée, J. P. 2016, ApJ, 821, 53. doi:10.3847/0004-637X/821/1/53
  * Vallée (2018) Vallée, J. P. 2018, ApJ, 863, 52. doi:10.3847/1538-4357/aad054
  * van de Hulst et al. (1954) van de Hulst, H. C., Muller, C. A., & Oort, J. H. 1954, Bull. Astron. Inst. Netherlands, 12, 117
  * Weiler (2018) Weiler, M. 2018, A&A, 617, A138. doi:10.1051/0004-6361/201833462
  * Xu et al. (2006) Xu, Y., Reid, M. J., Zheng, X. W., et al. 2006, Science, 311, 54. doi:10.1126/science.1120914
  * Xu et al. (2018) Xu, Y., Bian, S. B., Reid, M. J., et al. 2018, A&A, 616, L15. doi:10.1051/0004-6361/201833407
  * Xu et al. (2021) Xu, Y., Hou, L. G., Bian, S. B., et al. 2021, A&A, 645, L8. doi:10.1051/0004-6361/202040103
  * Yu & Ho (2018) Yu, S.-Y. & Ho, L. C. 2018, ApJ, 869, 29. doi:10.3847/1538-4357/aaeacd
  * Yuan & Grosbol (1981) Yuan, C. & Grosbol, P. 1981, ApJ, 243, 432. doi:10.1086/158610